Introduction to Message middleware and WEBSPHEREMQ

Turn from: http://www.ibm.com/developerworks/cn/websphere/library/techarticles/loulijun/MQnewer/MQnewer.html

Message Middleware Overview

Message Queuing technology is a technique for exchanging information between distributed applications. Message queues can reside on memory or on disk, and queues store messages until they are read by the application. With Message Queuing, applications can be executed independently-they do not need to know each other's location, or they do not need to wait for the receiving program to receive this message before proceeding with execution.

In distributed computing environment, in order to integrate distributed applications, developers need to provide effective means of communication for distributed applications in heterogeneous network environments. In order to manage the information that needs to be shared, it is important to provide a public information exchange mechanism for the application.

The main methods of designing distributed application are: Remote Procedure call (PRC), one of the basic standard components of Distributed computing Environment (DCE); object transaction Monitoring (OTM)--The combination of object-oriented industry standard and transaction processing (TP) monitoring technology based on CORBA; Message Queuing ( MessageQueue)--a loosely coupled method for constructing distributed applications.

(a) Distributed computing environment/Remote Procedure Call (DCE/RPC)

RPC is a component of DCE, an application-integrated software standard released by the Open Software Foundation (OSF). RPC mimics the traditional programming method used by a program to refer to another program using a function reference, which is the form of a procedure call, and once invoked, the control of the program shifts to the invoked program.

In the case of an RPC implementation, the invoked procedure can reside and execute in another system, either locally or remotely. When the invoked program completes processing the input data, the result is returned to the calling program in the return variable of the procedure call. When RPC completes, the program is immediately returned to the calling program. Thus RPC mimics the call/return structure of a subroutine, which provides only synchronous data exchange between the client (the caller) and the server (the invoked procedure).

(b) Object Transaction monitoring (OTM)

Based on CORBA's combination of object-oriented industry standard and transaction processing (TP) monitoring technology, the CORBA specification defines the architecture of object-oriented technology and method, the common Client/server programming interface, and the guidelines for transmitting and translating data between multiple platforms. The development of the Distributed Application Interface language (IDL) and so on, and provides a broad and consistent pattern for the Client/server application of the structure distribution.

(c) Message Queuing (Messages queue)

The message team lists loosely coupled methods for constructing distributed applications that are implemented synchronously or asynchronously. The API call for Message Queuing is embedded in a new or existing application that provides an exchange of information via a message sent to memory or a disk-based queue or read from it. Message Queuing can be used in applications to perform a variety of functions, such as requesting services, exchanging information, or asynchronous processing.

Middleware is a kind of independent system software or service program, distributed Application system uses this kind of software to share resources among different technologies, manage computing resources and network communication. It is a key software in the computer system, it can realize the interconnection and interoperability of the application, and can ensure the safe, reliable and efficient operation of the system. Middleware is located between user application and operating system and network software, it provides common communication means for application, and is independent of network and operating system. Middleware provides developers with an application interface that is common to all environments, and when embedded in the application, it can use the functions of the specific operating system and network environment in which it runs to perform communication functions for the application.

If there is no message middleware to complete the exchange of information, application developers in order to transfer data, must learn how to use the network and operating system software functions, to write appropriate applications to send and receive information, and exchange of information there is no standard method, each application must be specific programming, and multiple platforms, One or more application communications in different environments. For example, in order to achieve communication between different host systems on the network, it will require knowledge of how to exchange information on the network (for example, by using TCP/IP sockets); In order to achieve communication between different processes within the same host, knowledge such as the operating system message queue or named pipe (pipes) is required

The message transfer middleware (MOM) is one of the many types of middleware currently available, such as transaction management middleware (IBM's CICS), Web application Server middleware for Java applications (such as IBM's WebSphere Application server), and so on. It simplifies the transmission of data between applications, masks low-level heterogeneous operating systems and network platforms, provides consistent communication standards and application development, and ensures reliable, cross-platform information transmission and data exchange in distributed computing networks. It is based on the storage-forwarding mechanism of message queues and provides a unique asynchronous transmission mechanism, which can realize application integration and data exchange based on message transmission and asynchronous transaction processing.

With its unique security mechanism, simple and fast programming style, exceptional stability, scalability and Cross-platform, and powerful transaction processing and messaging capabilities, the IBM messaging middleware MQ becomes the industry's highest-market-share messaging middleware product.

MQ has a powerful cross-platform, it supports up to 35 of the number of platforms. It supports a variety of mainstream UNIX operating system platforms, such as HP-UX, AIX, SUN Solaris, Digital Unix, Open VMX, SUNOS, NCR Unix, and support for a variety of host platforms, such as: os/390, Mvs/esa, Vse/esa ; Windows NT servers are also supported. Supports Windows9x/windows nt/windows 2000 and Unix (UnixWare, Solaris) and major Linux versions (Redhat, Turbolinux, etc.) on the PC platform. In addition, MQ also supports a variety of other operating system platforms, such as: OS/2, as/400, Sequent Dynix, SCO OpenServer, SCO UnixWare, tandem, etc.

Back to the top of the page

The basic concept of MQ:

1) Queue Manager

The queue Manager is the topmost concept in the MQ system, and it provides a queue based messaging service for us.

2) message

In MQ, we define the data that the application sends to MQ as a message, and we can define the content of the message and understand the message in a broad sense, such as the user's various types of data files, the processing requests that an application makes to other applications, and so on. The message consists of two parts:

The message descriptor (the messages discription or the message Header) that describes the characteristics of the message, such as the priority of the message, the lifecycle, the information ID, and so on;

The body of the message, the User data section. In MQ, messages are classified into two types, non-persistent (non-persistent) messages and persistent (persistent) messages, and non-persistent messages are stored in memory and are designed to improve performance and are not recoverable when the system drops power or the MQ Queue Manager restarts. When the user's reliability requirements for the message are not high, while focusing on the performance of the system, you can use this type of message, such as: When the stock information is released, because the stock information is constantly updated, we may be published every few seconds, new messages will continue to overwrite the old message. Permanent messages are stored on the hard disk, and record the data log, it has high reliability, in the network and system failure, etc. can ensure that the message is not lost, not heavy.

In addition, there is the concept of logical and physical messages in MQ. With logical and physical messages, we can segment large messages, or we can classify several of our own complete messages into one group for processing.

3) Queue

Queues are safe repositories of messages, and queues store messages until they are processed by the application.

Message Queuing works in the following ways:

(a) program a forms a call to the Message Queuing system, which informs the Message Queuing system that the message is ready for program B;

b The Message Queuing system sends this message to the system where program B resides and places it in the queue of program B;

c After the appropriate time, program B reads the message from its queue and processes this information.

Due to the introduction of advanced program design ideas and internal working mechanism, MQ can guarantee the reliable transmission of the message under various network conditions, can overcome the poor or unstable status of the network line quality, in the transmission process, if the communication line fails or the remote host fails, the local application will not be affected, You can continue to send data without waiting for network failure recovery or remote host to run again.

In MQ, queues are divided into a number of types, including local queues, remote queues, template queues, dynamic queues, alias queues, and so on.

The local queue is divided into normal local queues and transport queues, which are queues that the application reads and writes through the API; transport queues can be understood as storage-forwarding queues, such as: We send a message to the MQ system to the remote host, where the network fails, MQ will put the message in the transport queue and, when the network resumes, send it to the remote destination.

A remote queue is a local definition of the destination queue, which resembles an address pointer that points to a destination queue on a remote host that is simply a definition and does not really occupy disk storage space.

Template queues and dynamic queues are a feature of MQ, a typical use of which is to be used as a system scalability consideration. We can create a template queue, when new queues are needed in the future, MQ automatically generates a dynamic queue each time a template queue is opened, and we can specify that the dynamic queue is either a temporary queue or a permanent queue, and if it is a temporary queue we can delete it while closing it, instead, if it is a permanent queue, We can keep it forever, for my own use.

4) Channel

A channel is a conduit for sending messages between queue managers in an MQ system, a logical concept built on a physical network connection, and the essence of MQ products.

In MQ, there are three main types of channel, namely Message Channel, MQI Channel and cluster channel. The message channel is used to transmit messages between MQ servers and servers, and it should be emphasized that the channel is one-way, it has sent (sender), received (receive), requester (requestor), server (s), and so on, different types, For use by users in different situations. The MQI channel is used to communicate and transmit messages between MQ client and MQ server, and its transport is bidirectional, unlike the message channel. The cluster (Cluster) channel is used for communication between queue managers within the same MQ cluster.

Back to the top of the page

How MQ Works

As shown in the figure:

First, with the local traffic, application A and application B run on the same system a, and they can communicate with each other using Message Queuing technology: Application A sends a message to queue 1, which is available when application B needs it.

Next is the case of remote communication, if the destination of the information transfer is changed to Application C on system B, this change will not affect application A, application A sends a message to queue 2, the MQ Discovery Q2 of System A is actually in system B, It places the information in a special local queue-transmission queue (transmission queue). We build a message channel from System A to system B, the message channel agent reads the message from the transmission queue, passes this message to System B, and waits for confirmation. This information is actually removed from the transmission queue only if the MQ receives confirmation that system B successfully received the information. If the communication line is not working, or system B is not running, the information will remain in the transmission queue until it is successfully routed to the destination. This is the most basic and important technology for MQ--ensuring information transmission, and is once and only once (once-and-only-once) delivery.

MQ provides an integrated, loosely coupled approach to connecting, because the application of shared information does not need to know each other's physical location (the network address), does not need to know how to establish communication between each other, does not need to be running simultaneously, does not need to run in the same operating system or network environment.

Back to the top of the page

An example of the basic configuration of MQ

In the above illustration, to implement communication on two hosts on the network, if point-to-point communication is used, we should at least establish the following MQ objects:

At sender A:

1) Establish the Queue Manager Qma:crtmqm-q QMA

2 Define a local transport queue: Define Qlocal (QMB) usage (XMITQ) defpsist (yes)

3 Create a remote queue: Define Qremote (QR). TOB) Rname (LQB) rqmname (QMB) XMITQ (QMB)

4 Define Send channel: Define channel (A.to). b) Chltype (SDR) conname (' IP of B ') XMITQ (QMB) + trptype (TCP)

In receiver B:

1) Establish the Queue Manager Qmb:crtmqm-q QMB

2) define the local queue Qlb:define qlocal (LQB)

3 Create the Receive channel: Define Channel (A.to). B) Chltype (RCVR) trptype (TCP)

With the above configuration, we can implement one-way communication from host A to B, to achieve two-way communication between the two, you can refer to this example to create the desired MQ object.

Back to the top of the page

MQ Mode of communication

1 Point-to-Point communication: Point-to-Point mode is the most traditional and common way of communication, it supports a pair of more than one or one pairs, many pairs of many, many multiple configuration, support tree-like, mesh and other topologies.

2 Multicast: MQ applies to different types of applications. One of the important, but also developing, "multicast" applications is the ability to send messages to multiple target sites (destination List). You can use an MQ directive to send a single message to multiple target sites and ensure that information is reliably provided for each site. MQ not only provides multicast functionality, but also has intelligent message distribution capabilities that, when sending a message to multiple users on the same system, MQ sends a replicated version of the message and a list of recipients on the system to the target MQ system. The target MQ system replicates these messages locally and sends them to the queue on the list to minimize the amount of network traffic.

3 Publish/Subscribe (publish/subscribe) mode: The Publish/Subscribe feature enables the distribution of messages to break the geographic direction of the destination queue, so that messages are distributed according to specific topics or even content, and users or applications can receive the messages they need based on the subject or content. The Publish/Subscribe feature makes the coupling between the sender and receiver looser, the sender does not have to care about the destination address of the receiver, and the receiver does not care about the sending address of the message, but only sends and receives messages based on the subject of the message. In the MQ family product, MQ Event Broker is a product dedicated to data communication using publish/subscribe technology, and it supports publications and subscriptions based on queues and direct TCP/IP in two ways.

4) Cluster (Cluster): In order to simplify the system configuration in Point-to-Point communication mode, MQ provides Cluster (cluster) solution. A cluster is similar to a domain, in which the queue managers within a cluster do not need to establish a message channel between 22, but instead use a cluster (Cluster) channel to communicate with other Members, thus greatly simplifying the system configuration. In addition, there is automatic load balancing between queue managers in a cluster, and when a queue manager fails, other queue managers can take over its work, greatly improving the system's high reliability.

Back to the top of the page

MQ Server and MQ Client

MQ products are divided into server and client two versions, in the operating environment of the MQ server, there are queue managers, queues, message channels and other objects, it provides comprehensive messaging services; MQ client provides us with an MQ application development and running environment, which is MQ The client implementation of the API. In the client environment, there are no queue managers, queues, and other objects that establish communication between the MQI channel and the server, send messages from the client to the server-side queue, or get messages from the server-side queues, which are more appropriate for better network conditions or real-time communication. It is also noted that the use of MQ client does not result in data loss or incompleteness. MQ Client provides the following benefits: Working mode for synchronization, reducing system burden, reducing system management overhead, and reducing disk space requirements.

Back to the top of the page

MQ's API

MQ supports a variety of programming languages, including: C, C + +, Java, VisualBasic, COBOL, PL/1, RPG, and many other popular development tools, such as WebSphere Studio application Developer, Powerbuiler, Microsoft Visual C + +, visual Basic, Delphi, and so on. Furthermore, MQ provides a unified programming interface on different platforms, and the porting of programs between different platforms can be accomplished simply by recompiling. MQ API interface is very easy to learn, users only need to take advantage of MQ 13 of commonly used and powerful function calls, you can at the fastest speed, write a variety of complex applications. Users can focus on the implementation of the application business logic instead of the underlying communication, exception handling, and so on.

Take the C language as an example, the development process of an MQ application is as follows:

Mqconn ()/ * Set up a connection to the Queue Manager/* Mqopen ()/* Open the queue for read/write Operations/ mqput ()//* to put messages into queues/ mqget ()/* read messages from queues * / MQINQ ()/* Query queue Properties/ Mqset ()/* Set the properties of the queue/ mqclose ()/* After the read and write operations are completed, the queue is closed/ mqdisc ()/* Disconnect from the queue manager, releasing related resources */