Introduction to. Net open-source shuttle service bus (ESB)

The shuttle service bus is a free. Net open-source software project. It provides a new method for developing a message-oriented event-driven architecture (EDA) system. Although it is still in its infancy, it has been applied to production systems.

The related points are as follows:

• Developed Using C # (Based on. NET 3.5)
• Core functions do not depend on any third-party products or projects
• Supports both command messages and event messages (pub/Sub)
• Integrated message Distribution Function
• Contains a command line management program to easily meet various operation requirements
• Widely used interfaces for replacement or expansion
• Fault tolerance capability provided by automatic retry

Why use the service bus?

Although some thought patterns need to be changed when using the service bus, it is of great benefit to do so. By designing a service bus in your system, you can implement very specific functions on a specific endpoint, in this way, you can focus on designing the software that runs in an isolated environment. You can perform independent version control and maintenance on such endpoints, provided that the decoupling work you have done on these endpoints has reached the desired level.

Basically, you will eventually send messages between different components andAsynchronousProcessing. This can lead to the "fire-and-forget" situation, so you need to carefully consider the way the system works. The resulting user experience may be completely different from the traditional implementation, that is, the way to immediately process all the operations requested by the user.

For example, simply send an email. You can set an endpoint to process the relevant Command Message type ). Then, when you need to send an email, you can use the following code:

Bus.Send(new SendEMailCommand              {                    From = "someone@from.com",                    To = "someone@to.com",                    Subject = "testing e-mail",                    Body = "Hello"              });

You may want to know what is the difference between sending this email directly and yourself:

• This message is sent and called instantly, instead of waiting for the call to be executed
• Because requests are queued, the bottleneck is avoided.
• If the email fails to be sent, the email will be resending automatically.
• As long as the obtained data is correct, it can be ensured that the mail will eventually be sent; or, at least in the case of a problem, it can be handled manually.

Therefore, the client code does not need to care about how the end point sends data. The endpoint may send data using the simple mail Transmission Protocol (SMTP), or even using a custom web service (Web-service.

How does shuttle exert its magic power?

The shuttle service bus depends on two things:

• Message, and
• Queue)

The queue infrastructure can be any implementation. Generally, you want to useTrueFor example, Microsoft Message Queue (MSMQ ). Currently, shuttle directly supports the following Queues: Microsoft Message Queue (MSMQ) and SQL Server table-based queues ). If you want to use any other implementation, the problem to be solved is to implement the relevant interface, so you will do well. Shuttle uses a Uniform Resource Identifier (URI) structure to represent a queue, for example:

• MSMQ: // {MACHINE}/{queue}
• SQL: // {connection-name}/{table}

To implement your own queue, you only need to simply select a solution and parse this structure in your queue implementation.

To organize your endpoints, shuttle uses a general service host to simplify deployment. To make a new endpoint take effect is to solve the configuration problem of the queue used by the endpoint, and then start your service, as shown in the following configuration file and code snippet:

<?xml version="1.0"?><configuration>  <configSections>    <section      name="serviceBus"      type="Shuttle.ESB.Core.ServiceBusSection, Shuttle.ESB.Core"/>  </configSections>  <serviceBus>    <inbox      workQueueUri="msmq://./inbox-work"      journalQueueUri="msmq://./inbox-journal"      errorQueueUri="msmq://./shuttle-error"      />  </serviceBus></configuration>    public class ServiceBusHost : IHost, IDisposable    {        private IServiceBus bus;public void Dispose(){    bus.Dispose();}public void Start(){    bus = ServiceBus             .Default()             .Start();}    }

It should be noted that the general-purpose host can run both in the form of console applications and be installed as a service. This makes the debugging end point very easy, because when you debug in Visual Studio, you can specify a universal host as the startup application.

Once a common host gets a specific message from the inbox queue, the common host will try to find the handler and send the message to the handler for processing. For messages that cannot find the processing program, are moved to the error queue, or are discarded, the processing method depends on the information specified in the configuration file.

Command Message and event message

Because a command is a clear request that requires specific functions to be executed, the command is sent to a separate endpoint. This means that in order to send a message, you need to know that an endpoint has implemented a specific behavior. Therefore, commands lead to a higher degree of behavior coupling. When a message is sent, the endpoint of the received message can be configured, And the endpoint can be changed at any time.

For example, you may have the following command:

• Sendemailcommand
• Convertdocumentcommand
• Deletefilecommand
• Cancelordercommand

On the contrary, events may have no or multiple subscribers ). Generally, events should be clearly defined, because this is required from the business perspective. Therefore, unless it is defined based on future requirements, the event will have at least one subscriber. Once an event is published, each subscriber receives a copy of the event message.

This is different from message distribution, because a distributed message is only sent to the inbox queue of a worker.

For example, you may have the following events:

• Emailsentevent
• Documentconvertedevent
• Filedeletedevent
• Ordercancelledevent

To add any necessary ad-hoc data to a message, the shuttle service bus allows you to specify a message header for the message. In addition, you can use the correlation ID to group related messages.

Scalability)

Shuttle gives you a lot of scalability. Because messages are queued, there will be no urgent bottlenecks. Even if some endpoints may be configured with multiple threads, this does not mean that a specific Endpoint will not cause performance degradation due to receiving excessive messages. Each endpoint must have a built-in capability to distribute messages to other endpoints. Once any other endpoints are notifiedDistributor (distributor)It has Idle threads that can execute work, and can be used to send messages. To achieve this, you only need to configure it.

Module)

Shuttle uses an observed pipeline structure. Events are registered in the pipeline in a specific order, and the observer (observer) can also be registered in the pipeline to respond to their respective events.

To maintain extensibility, you can register your module implementation into shuttle. By adding an observer to respond to the events of the respective pipelines, these modules are usually connected to the specific pipelines. In addition, you can even add custom events to the pipeline as needed.

How does one support dependency injection and logging?

Since shuttle uses interfaces so widely for decoupling, You can freely access any dependency injection (DI) or logging implementation based on your preferences. By default, the implementation does not depend on any third-party components. However, there are also Castle Windsor implemented by dependency injection and log4net implemented by logging.

Shuttle case in production environment

Shuttle's implementation in a large short-term insurance company in South Africa has been greatly successful, replacing the outdated document index system.

The customer sends a claim-related document via email, and the email will be received by the Lotus Domino Email system. The filenet email Manager application then extracts these emails and places them in the IBM filenet Content Engine. The Content Engine is configured to respond to any emails classified as submitted new documents. The Content Engine is designed to process these delivered mails and write an XML file containing the relevant data.

From there, the shuttle Content Engine endpoint will pick up these XML files and then release events to show that new content has been placed in the Content Engine. Because the structure of this endpoint is not specific to the index process, it is possible to reuse this endpoint to publish any new content. In addition, the content engine is responsible for writing relevant XML documents.

Shuttle indexing (INDEX) endpoints subscribe to those new content messages. Once a message is delivered, the endpoints start to track all the documents delivered together with a specific email, this is because each email has a unique identifier that has been attached to the Content Engine metadata of each document. Once all documents related to an email are collected, the command message is sent to the shuttle document conversion (document conversion) endpoint, it is used to convert the body of an HTML-formatted email and all JPG files to a tiff-formatted document.

Document conversion endpoints do not know anything about the indexing process, but only perform various document conversions. Once the conversion of a document is complete, an event is published to notify the document that the conversion has been successful or failed. Any system that requires document conversion subscribes to these event messages. In order to establish messages for a specific system or messages not for a specific system, the request conversion system can use the correlation ID in the conversion Request command ), and/or add a name/value pair to the outgoing conversion Request command message. These header information will always be appended to any related message sent by shuttle.

Once all required document conversions are completed, a command message is sent to the shuttle ovaflo endpoint to create an indexed workflow instance in the IBM filenet Process Engine (process engine. Ovaflo is a meta-Workflow Framework product developed by ovations group.

To execute the actual index, you can access the network-based index application. This application obtains the next available indexing workflow instance from the ovaflo endpoint and displays the classification of relevant documents. Each document is linked to a specific claim and any other index data entered. In addition, it can also be used to process requests initiated by system users to convert certain documents to the TIFF format. This feature is especially useful when you place different types of documents in one file. Once you are satisfied with the data, you can submit the task. This step is processed asynchronously after a command message is sent, so that you can continue to process the next index task immediately.

In some cases, you may find that the work from the Web Front-end is queued in the work of the background system. For example, this is the case for document conversion. Before a request is initiated from the front end, all the conversions required for mail delivery are being processed. We can simply install a single Shuttle document conversion (document conversion) Endpoint using the same compilation assembly on a general host, but we need to modify the configuration of this endpoint, to use its own queue. The front-end then sends the conversion request to this high-priority endpoint, and then those transformations will be processed in a timely manner. All background conversions will still be sent to the original endpoint.

In this example, shuttle is used to bond different systems. Because the system has the required built-in Fault Tolerance capabilities, it is much more stable than the previous system. In addition, the system performance is very good because there is no backlog of work to be created.

Conclusion

Shuttle provides you with another free choice when implementing the Enterprise Service Bus (ESB. This project is hosted on codeplex:

• Project Website
• Wikipedia
  

Advantages:

• New project
• Highly scalable
• Free Open Source Software
• Command Line Management Program

Disadvantages:

• New project
• Process status data must be processed manually

Over time and with the support of the community, there are various implementations available for expansion, including more queues, dependency injection (DI), and other options.

 

Original English version