Introduction to Message middleware and WEBSPHEREMQ

Transferred from: http://www.ibm.com/developerworks/cn/websphere/library/techarticles/loulijun/MQnewer/MQnewer.html

Message Middleware Overview

Message Queuing technology is a technique for exchanging information among distributed applications. Message Queuing can reside in memory or on disk, and queues store messages until they are read by the application. With Message Queuing, applications can execute independently-they do not need to know each other's location, or wait for the receiving program to receive this message before proceeding.

In distributed computing environments, in order to integrate distributed applications, developers need to provide effective means of communication for distributed applications in heterogeneous network environments. In order to manage the information that needs to be shared, it is important to provide the public information exchange mechanism for the application.

The methods of distributed application design include: Remote Procedure Call (PRC)--one of the basic standard components of Distributed computing Environment (DCE); object transaction Monitoring (OTM)--The combination of object-oriented industry standard and transaction processing (TP) monitoring technology based on CORBA; Message Queuing ( MessageQueue)--loosely coupled method for constructing distributed applications.

(a) Distributed computing environment/Remote Procedure Call (DCE/RPC)

RPC is the component of DCE and is an application-integrated software standard released by the Open Software Foundation (OSF). RPC mimics a program that uses a function reference to refer to another program's traditional programming method, which is the form of a procedure call, and once invoked, the program's control turns to the called program.

When RPC is implemented, the called procedure can reside in another system, either locally or in a remote place, and is executed. When the called program finishes processing the input data, the result is placed in the return variable of the procedure call back to the calling program. After the RPC is complete, program control returns to the calling program immediately. So RPC mimics the call/return structure of the subroutine, which provides only the synchronous data exchange between the client (the calling program) and the server (called procedure).

(b) Object Transaction monitoring (OTM)

The combination of object-oriented industry standard and transaction processing (TP) monitoring technology based on CORBA defines: Architecture using object-oriented techniques and methods, public client/server programming interface, and guidelines for transferring and translating data between multiple platforms , developing the language of Distributed Application Interface (IDL), and providing a wide and consistent pattern for client/server application of structural distribution.

(c) Message Queuing (Messages queue)

The messaging team is listed as a loosely coupled method for constructing distributed applications that are implemented synchronously or asynchronously. Message Queuing API calls are embedded in new or existing applications that provide information exchange by sending messages to memory or disk-based queues or reading them out. Message Queuing can be used in an app to perform a variety of functions, such as requiring services, exchanging information, or asynchronous processing.

Middleware is a kind of independent system software or service program, and distributed Application system uses this software to share resources among different technologies, manage computing resources and network communication. It is a key software in the computer system, it can realize the interconnection and interoperability of the application, and can ensure the safe, reliable and efficient operation of the system. The middleware is located between the user application and the operating system and the network software, it provides the common communication means for the application, and is independent of the network and the operating system. Middleware provides developers with an application interface for all environments, and when an application embeds its function calls, it can take advantage of the capabilities of the specific operating system and network environment it runs to perform communication functions for the application.

If there is no message middleware to complete the exchange of information, application developers in order to transfer data, it is necessary to learn how to use the network and operating system software functions, write the appropriate application to send and receive information, and exchange information there is no standard method, each application must be specific programming and multi-platform, Communication of one or more applications in different environments. For example, in order to achieve communication between different host systems on the network, it will require knowledge of how information is exchanged on the network (such as socket programming with TCP/IP), and in order to achieve communication between different processes within the same host, knowledge of the operating system's Message Queuing or named pipes (Pipes) will be required.

There are many types of middleware, such as transaction management middleware (such as IBM's CICS), Web application Server middleware for Java applications (such as IBM's WebSphere Application Server), and message transfer middleware (MOM). It simplifies the transmission of data between applications, shields the underlying heterogeneous operating systems and network platforms, provides consistent communication standards and application development, and ensures reliable, cross-platform information transfer and data exchange in a distributed computing network environment. It is based on the storage-forwarding mechanism of message queue, and provides the unique asynchronous transmission mechanism, which can realize application integration and data exchange based on message transmission and asynchronous transaction processing.

IBM message Middleware MQ is the most market-share messaging middleware product in the industry with its unique security mechanism, simple and fast programming style, excellent stability, scalability and cross-platform, as well as strong transaction processing and messaging capabilities.

MQ has a strong cross-platform, and it supports up to 35 platforms. It supports a variety of mainstream UNIX operating system platforms such as: HP-UX, AIX, SUN Solaris, Digital Unix, Open VMX, SUNOS, NCR Unix, support for various host platforms such as os/390, Mvs/esa, Vse/esa Windows NT Server is also supported. Support for Windows9x/windows nt/windows 2000 and Unix (UnixWare, Solaris) and major Linux versions (Redhat, Turbolinux, etc.) on PC platforms. In addition, MQ supports a variety of other operating system platforms such as: OS/2, as/400, Sequent Dynix, SCO OpenServer, SCO UnixWare, tandem, etc.

Back to top of page

Basic concepts of MQ:

1) Queue Manager

The queue Manager is the topmost concept in the MQ system, which provides us with a queue-based messaging service.

2) message

In MQ, we define our application as a message in the data that MQ transmits, we can define the content of the message and understand the message in a broad sense, such as: The user's various types of data files, an application to other applications issued processing requests, etc. can be used as messages. The message is made up of two parts:

A Message descriptor (MSG discription or message Header) that describes the characteristics of the message, such as the priority of the message, the life cycle, the message ID, and so on;

The message body, which is the User data section. In MQ, messages are classified into two types, non-persistent (non-persistent) messages and persistent (persistent) messages, non-persistent messages stored in memory, designed to improve performance, and unrecoverable when the system is out of power or the MQ Queue Manager restarts. When the user's reliability requirements for the message are not high, but focus on the performance of the system, you can use this type of message, such as: When the stock information is published, because the stock information is constantly updated, we may be published every few seconds, new messages will continue to overwrite the old message. The permanent message is stored on the hard disk, and records the data log, it is high reliability, in the network and system failure, etc. can ensure that the message is not lost, not heavy.

In addition, in MQ, there is the concept of logical messages and physical messages. With logical messages and physical messages, we can segment large messages, or we can classify several of their own complete messages into a group of application logic.

3) Queue

A queue is a secure place to store messages, and the queue stores messages until it is processed by the application.

Message Queuing works in the following ways:

A) program a forms a call to the Message Queuing system, which informs the Message Queuing system that the message is ready to go to program B;

b) The Message Queuing system sends this message to the system where program B resides and places it in the queue of program B;

c) After the appropriate time, program B reads this message from its queue and processes this information.

Because of the Advanced program design idea and the internal working mechanism, MQ can guarantee the reliable transmission of the message under various network conditions, and can overcome the status of poor or unstable network line quality, in the transmission process, if the communication line fails or the remote host fails, the local application will not be affected. You can continue to send data without waiting for network failure recovery or the remote host to run again.

In MQ, queues are divided into many types, including: local queues, remote queues, template queues, dynamic queues, alias queues, and so on.

The local queue is also divided into ordinary local queues and transmission queues, the normal local queue is the application through the API read and write operations of the queue, the transmission queue can be understood as a storage-forwarding queue, such as: We sent a message to the MQ system to send to the remote host, while the network has failed, MQ will place the message in the transmit queue and then send it to the remote destination when the network resumes.

A remote queue is a local definition of a destination queue, which resembles an address pointer to a destination queue on a remote host, which is only a definition and does not really occupy disk storage space.

Template queues and dynamic queues are a feature of MQ, a typical use of which is to be used as a system extensibility consideration. We can create a template queue, and when new queues are needed, each time a template queue is opened, MQ automatically generates a dynamic queue, we can also specify that the dynamic queue is a temporary queue or a permanent queue, and if it is a temporary queue we can delete it while shutting it down, instead, if it is a permanent queue, We can keep it forever and use it for me.

4) Channel

A channel is a conduit for passing messages between queue managers in an MQ system, a logical concept built on top of a physical network connection, and the essence of MQ products.

In MQ, there are three main classes of channel types, the message channel, the MQI channel, and the cluster channel. Message channels are used to transfer messages between MQ servers and servers, it is emphasized that the channel is one-way, it has send (sender), receive (receive), requestor (requestor), service (server) and other different types, For use by users in different situations. The MQI channel is used to communicate and transmit messages between MQ client and MQ server, unlike message channels, where the transmission is bidirectional. The cluster (Cluster) channel is used for communication between queue managers that are located inside the same MQ cluster.

Back to top of page

How MQ Works

As shown in the figure:

First of all, in the case of local communication, application A and application B run on the same system a, and they can communicate with each other through Message Queuing technology: Application A sends a message to queue 1, and the information is available when application B needs it.

Next is the case of remote communication, if the target of the information transfer to the application C on system B, this change will not affect application A, application A to queue 2 send a message, system A's MQ discovery Q2 point to the destination queue is actually located in System B, It places the information in a special queue on the local-transmission queue (transmission). We set up a message channel from System A to system B, and the Message channel Agent reads the message from the transmission queue, passes this message to System B, and waits for confirmation. Only after MQ receives confirmation that system B has successfully received the information does it actually delete the information from the transmission queue. If the communication line is not available, or system B is not running, the information is left in the transmission queue until it is successfully delivered to the destination. This is the most basic and important technology of MQ-ensuring that information is transmitted and is delivered once and only once (once-and-only-once).

MQ provides loosely coupled connection methods for application integration, because applications that share information do not need to know each other's physical location (network address), do not need to know how to communicate with each other, do not need to be running at the same time, and do not need to operate in the same operating system or network environment.

Back to top of page

Basic configuration examples for MQ

In the figure above, to achieve communication on two hosts on the network, if we use point-to-point communication, we should at least establish the following MQ objects:

In sender A:

1) Set up the queue Manager qma:crtmqm-q QMA

2) define local transmission queue: Define qlocal (QMB) usage (XMITQ) defpsist (yes)

3) Create a remote queue: Define Qremote (QR. TOB) Rname (LQB) rqmname (QMB) XMITQ (QMB)

4) define the Send channel: Define Channel (a.to. b) Chltype (SDR) conname (' IP of B ') XMITQ (QMB) + trptype (TCP)

On receiver B:

1) Set up the queue Manager qmb:crtmqm-q QMB

2) define local queue Qlb:define qlocal (LQB)

3) Create the Receive channel: Define Channel (a.to. B) Chltype (RCVR) trptype (TCP)

With the above configuration, we can implement one-way communication from host A to B, to achieve two-way communication between them, you can refer to this example to create the desired MQ object.

Back to top of page

MQ's communication mode

1) Point-to-point communication: Point-to-point mode is the most traditional and common means of communication, it supports a pair of one or one-to-many, many-to-many, many-to-one multiple configuration, support tree-like, mesh and many other topologies.

2) Multipoint broadcast: MQ is suitable for different types of applications. One of the important, and growing, "multicast" applications is the ability to send messages to multiple target sites (Destination List). You can use an MQ directive to send a single message to multiple target sites and ensure that information is reliably provided for each site. Not only does MQ provide multicast functionality, but it also has intelligent message distribution, where MQ sends a copy of a message to multiple users on the same system and the list of recipients on that system to the target MQ system. The target MQ system replicates these messages locally and sends them to the queue on the list to minimize the amount of network traffic.

3) Publish/subscribe (publish/subscribe) mode: The Publish/Subscribe feature enables the distribution of messages to break through the boundaries of the destination queue's geographic direction, so that messages are distributed according to specific topics or even content, and users or applications can receive the required messages based on the subject or content. The Publish/Subscribe feature makes the coupling between sender and receiver looser, the sender does not have to care about the destination address of the receiver, and the receiver does not have to care about the sending address of the message, but simply sends and receives the message based on the subject of the message. In the MQ family product, MQ Event Broker is a product dedicated to data communication using publish/subscribe technology that supports both queue-based and directly TCP/IP two-based publishing and subscriptions.

4) Cluster (Cluster): To simplify system configuration in point-to-point communication mode, MQ provides a Cluster (cluster) solution. A cluster is similar to a domain, where communication between queue managers within a cluster does not require the creation of a message channel between 22, but instead uses the cluster (Cluster) channel to communicate with other members, greatly simplifying system configuration. In addition, the queue managers in the cluster can automatically load balance, and when a queue manager fails, the other queue manager can take over its work, thereby greatly improving the system's high reliability.

Back to top of page

MQ Server and MQ Client

MQ products are divided into server and client two versions, in the environment of MQ server, there are Queue Manager, queue, message channel and other objects, it provides a comprehensive message service, MQ client provides us with an MQ application development and running environment, it is MQ The client implementation of the API. In the client environment, there is no Queue Manager, queue and other objects, it through the MQI channel and the server to establish communication, and send messages from the client to the server side of the queue, or from the server side of the queue to obtain messages, it is more suitable for network conditions or real-time communication situation. It is also noted that the use of MQ client does not result in loss or completeness of data. The MQ client provides the following benefits: Work mode for synchronous processing, reduced system burden, reduced system administration overhead, reduced disk space requirements, and so on.

Back to top of page

MQ's API

MQ supports a variety of programming languages, including: C, C + +, Java, VisualBasic, COBOL, PL/1, RPG, etc., while also supporting a variety of popular development tools, such as: WebSphere Studio application Developer, Powerbuiler, Microsoft Visual C + +, visual Basic, Delphi, and more. Furthermore, MQ provides a unified programming interface on different platforms, and the porting of programs between different platforms can be done only by recompiling. MQ's API interface is easy to learn, and users can write complex applications as quickly as possible using only 13 of MQ's most common and powerful function calls. Users can focus on the implementation of the application business logic rather than the underlying communication, exception handling, and so on.

In C, for example, the development process for an MQ application is as follows:

MQCONN ()/ * Establish a connection to the Queue Manager */ MQOPEN ()//Open the queue for read/write operations */ MQPUT ()/* Put the message in the queue */ MQGET ()/* Read the message from the queue * / MQINQ ()/* Properties of the query queue */ Mqset ()//Set the properties of the queue */ Mqclose ()/* After the read and write operations are complete, close the queue/ Mqdisc ()//Disconnect the Queue Manager and release the associated resources */