Reliable XML Web Service (1)

Web|xml Reliable XML Web Service
Eric Schmidt
Microsoft corporation,xml Core Services Group, project manager
December 11, 2001

Download the sample code for this column.

Note: To download the code associated with this article, you need to:
Visual Studio. NET Release Candidate (English)
SQL Server 2000 (English)
On the PDC, I talked about the topic of reliable XML Web service (Web services), which stems from multiple exchanges over the past year. Reliability is one of the five most important issues that developers face in implementing decentralized Web services in the many common questions about building XML Web service. If it's not too hard to separate, I'm going to talk this month about the tricky issue of building a reliable XML Web Service.

Overview
The most striking aspect of Global XML Web Services Architecture (GXA [English]) is that you can extend the architecture using a synthetic processing protocol. These protocols are implemented primarily through SOAP headers, providing a wide range of services including security, encryption, routing, and reliability. When you start building an GXA application, you'll find that GXA is essentially a messaging architecture that provides collaborative capabilities between systems and services through standards-based coding Technology (SOAP). So far, most implementations have focused on SOAP 1.1 and WSDL-compliant services, so WEB service implementations can work in concert with multiple languages and operating systems.

This is a terrific concept. Communication can be made between any two systems as long as they can parse the XML and understand the rules of the SOAP specification. However, simple message exchange does not meet the needs of complex business applications. Real applications, regardless of their internal domain architecture, require standardized services, such as security, authorization, and reliability at the WEB service message processing layer. There is a huge incentive behind the creation and implementation of Global XML Web Services architecture, specifically soap, SOAP modules, and infrastructure protocols. With the release of four new specifications (Ws-routing, ws-referral, ws-licensing and ws-security) This October, we have embarked on the next generation of XML Web Service implementations. Despite the release of so many new specifications, there are still two areas where there is no public specification, transaction processing and reliable message processing, mainly because these infrastructure protocols depend on the underlying SOAP module.

This column discusses the implications of reliability and reliable message processing from the perspective of GXA environments. And I'm going to spend some time exploring what needs to be done to develop a reliability protocol by extending existing WEB service classes in the. NET framework. This column has two main purposes:

Let the reader understand the concept of reliability, to prepare for the implementation of various specifications in the future. Note that this article is not a specification, but just an article designed to trigger the reader to think about the issues discussed below.
Describes the powerful, standards-based capabilities of WEB services and SOAP classes in the. NET framework.
Reliability of XML Web Service
Let's separate the questions. As we said earlier, the GXA service implementation is a message processing service that needs to send and receive standards-based encoding messages in a decentralized environment. The primary transport protocol for sending SOAP messages in a WEB service implementation is HTTP, which is easy to implement and manage, but is inherently unreliable. There is no need to delve into the exact cause of HTTP unreliability, but it is sufficient to know that HTTP does not have standards-based services to ensure that endpoints or servers can receive requests. Although built-in network-layer devices can generate errors in the event of a general catastrophic failure, such as not finding a resource, there is no mechanism to ensure that clients receive requests or responses in a reliable manner.

HTTP failures are typically handled through simple resend operations, but in a business-processing environment This is neither conducive to efficiency nor effectiveness. It leads to unnecessary traffic and increases the risk of duplicate transactions.

There are a variety of messaging technologies available on the market to address this problem more effectively, including transport protocols (such as HTTPR), enterprise infrastructure (such as MSMQ and MQ Series), and business processing protocols such as EbXML. Although each technology has advantages for a particular implementation, it cannot solve the reliability problem in a scalable way that can be applied across domains on all transport protocols, and the functional level of message exchange and processing varies.

With all these questions in mind, I decided to summarize a list of requirements to see what it would take to implement a reliability prototype in a WEB service environment.

Here's a list of the major requirements for the reliability layer I created myself:

Standards-based and applied to the message protocol layer
Confirm Send
Ordered send
Symmetric dialogue
Speed up asynchronous processing
Based on standard
The agreement must be made up of existing standards-based technologies. Furthermore, the Protocol should extend the SOAP 1.1 specification and apply it to the message encoding layer instead of the transport layer. This allows the message to be transmitted over all available mechanisms (from HTTP to some dedicated socket implementations).

Confirm Send
In order to be rule-based, the protocol must adopt some kind of sending acknowledgement mechanism, that is, messages sent using the protocol should receive one and only one acknowledgement of the status of the message from the processor.

Ordered send
Sequential delivery introduces the concept of dialogue, where clients and servers can exchange messages and confirmations (confirmations are also part of a conversation that can be uniquely identified). When a message is received, the message is checked for sorting, ensuring that the processor receives an ordered set of messages.

Symmetric dialogue
Protocols built on the dialogue mechanism must also ensure the symmetry of messages and confirmations. You must ensure that each message is processed only once, and only one acknowledgment is generated.

Speed up asynchronous processing
This is the most important point in the requirements list, so stay in the final note. HTTP is based on the synchronous request response model and is suitable for simple applications that handle small tasks or short running times. The WEB service implementation has a little secret that the user does not need to understand from the processing point of view how the service is implemented on the backend. That is, a WEB service implementation can take up to three seconds or three hours to process a request. This causes the message processing architecture to be inefficient and not extendable. What we need is a processing model that speeds up the asynchronous message processing architecture. Note, however, that an asynchronous model implementation is much more complex than a tightly coupled request-response implementation scenario because it requires more infrastructure.

Let me explain. In a synchronous message processing model using standard HTTP, the client remains stagnant until the request is sent to the server and the response is received from the server. During this time, a number of catastrophic events can occur:

The connection may be disconnected for external reasons, thereby losing the request or response.
The server may timeout due to offline or overload.
The server process may depend on the downlink service, and the response time for this service is beyond control.
Synchronization model


Figure 1: Synchronization model

Whether the problem is network-related or application-related, ensuring reliability requires some kind of additional protocol-driven infrastructure. In this discussion, I want to focus on how messages are handled on the transport layer. The transport layer is a critical part of the application that can be used to implement the reliability layer and separate the final process of the message. More specifically, each message (regardless of which application is targeted) needs to be read from the network layer and allocated to the appropriate application resources. We can add an additional protocol to send reliability confirmation and execute persistent storage here so that the application resources can choose when and how to process the message. Also, this new protocol can help us detach or speed up the asynchronous processing model. The following explains how to complete this procedure.

In the following asynchronous model, a request is sent to the SOAP server. The server reads the message stream from the network layer and immediately returns an HTTP 202 response to the client. This process is synchronized only when sending messages to the server, which can reduce connection-related problems. When the server arrives, the message is routed to the reliability layer, where it checks to verify that the message is expired, duplicated, and ordered. The message is then persisted (in the relational database) and a confirmation of its status is sent to the client. Finally, the message is assigned to the correct application functionality.

Asynchronous model


Figure 2: Asynchronous model

In an HTTP environment, you can control when a response is sent to a client. By controlling the time that a response is sent to the client, you can minimize the risk of downlink processing affecting communication reliability. In SOAP, this is done through a one-way messaging delivery. It instructs the underlying SOAP processor to immediately send an HTTP 202 response to the client notifying the client that it has received the message and has successfully assigned the message to the correct resource for processing. The processor then sends a response to the client about the status of the message. The benefits of this model are described in detail later in this article.

Build the reliability layer
Having identified the above requirements, we discuss how to use the. NET framework to establish a reliability protocol for WEB service implementation scenarios. Based on the above requirements, I have created a small API to provide the available implementation scenarios.

Agreement: ERICRP
The first problem is defining how to decompose the Reliability Protocol (ERICRP). The following are the key points of the agreement:

The protocol is used as a prototype for teaching.
This protocol is performed primarily through the extended SOAP message processing layer on the SOAP processing layer.
The SOAP header is used to encode the information required for the processing layer.
This protocol requires that the implementation scheme have some form of persistent storage to record messages. This implementation uses Microsoft SQL Server 2000.
Note: Regardless of the manner in which the reliability layer is implemented in a SOAP environment, additional infrastructure is required in addition to the SOAP parser.
This protocol supports the concept of dialogue, which means that multiple messages can be sorted to ensure orderly delivery.
All implementations of the protocol are limited by the ERICRP namespace.
ERICRP is based on a two-party dialogue scenario, where two services can be negotiated through the XML Web services Architecture (HTTP, SOAP, and WSDL).
The client is responsible for all corrections to the message. (This article is discussed in detail later)
The server is only responsible for sending confirmations based on specific criteria.
The server does not log expired messages that are received.
The server does not log incoming unordered messages.
The server does not log incoming duplicate messages.
Processing APIs
In this prototype, I built six major classes and a small database. I call the class the processing API. These classes are used by WEB service clients and servers to monitor and correct messages using the ERICRP reliability protocol. All classes belong to the ERICRP namespace:

Client.conversationmanager: Used by clients to create a conversation environment for WEB service message associations and message monitoring.
Client.rpclienttrace: Used by Web service clients, these client methods perform ERICRP reliability protocols on outbound messages.
Server.conversationmanager: Used by the WEB service server to log and process inbound messages.
Server.rpservertrace: Used by Web service servers that perform ERICRP reliability protocols on inbound messages.
Reliabilityinfo: it has double function. It can be used by Client.conversationmanager to provide reliability information for records, or it can be used by Web service client proxies to create the necessary SOAP header information for outbound messages.
Acknowledgment: is used by Server.conversationmanager to send confirmation to the client.
The working principle of ERICRP
Before I look at the code, I want to explain how the protocol works from a user's perspective. For example, I have a simple Web service proxy class that lets you send order messages to a Web service. Clients who intend to use the API need to do the following:

First, create an instance of the Client.conversationmanager class and start a new conversation. For example:

private void Begin ()
{
Rpclient = new EricRP.Client.ConversationManager ();

Rpclient.messagesent = _
New EricRP.Client.ConversationManager.MessageSentEventHandler (process);

Rpclient.conversationstarted + = New _
EricRP.Client.ConversationManager.ConversationStartedHandler (constarted);

Rpclient.beginconversation ();
}

The rpclient variable is valid at the class level and will be used later. I also set up some event handlers.

Next, use the order agent and cooperate with the Reliabilityinfo class to send a piece of reliable information. Create an instance of Purchaseorderproxy first, just as you would normally do for a WEB service client. Then create an instance of the Reliabiltiyinfo class, pass the Conversationmanager to the constructor, and set the reliability properties. The attributes that require special attention are maxretry, expiredate, and Ackurl. Maxretry and expiredate are used to restrict the activity of the message, prevent it from being sent indefinitely, and the WEB service will use Ackurl when sending a receive acknowledgement to the client. After you set these properties, you can set the agent's Reliableheader property and invoke the method that you want.

private void SendMessage ()
{
Clientproxies.purchaseorderproxy po = new Clientproxies.purchaseorderproxy ();

Ericrp.reliabilityinfo rinfo = new Ericrp.reliabilityinfo (rpclient);
Rinfo.status = ReliabilityInfo.MessageStatus.New;
Rinfo.senddate = System.DateTime.Now;
Rinfo.expiredate = System.DateTime.Now.AddHours (4);
Rinfo.maxretry = 5;
Rinfo.ackurl = "Http://localhost:8082/ericRPAck/POAck.asmx";

Po. Reliableheader = Rinfo;
Po. Submitmessage ("very much want them to get this order!") ");
}

This is a screenshot of a client test program written to illustrate this feature. Note that we have sent five messages altogether. The third message expires before it arrives at the destination, and in accordance with the ERICRP protocol, the message is discarded and the server does not process it. The Forth message is an unordered message because the server does not receive a valid third message. Any subsequent messages are unordered until the third message is sent again. If you requery Client.conversationmanager, you will find that the five message is also unordered.



Figure 3: Client test program