Introduction to message middleware and WebSphere MQ

Message

 

Message Queue is a technology used to exchange information between distributed applications. Message queues can reside in the memory or disk, and the queues store messages until they are read by applications. Through message queues, applications can execute independently-they do not need to know the location of each other, or do not need to wait for the receiving program to receive the message before continuing the execution.

To integrate distributed applications in a distributed computing environment, developers need to provide effective communication means for distributed applications in a heterogeneous network environment. To manage information that needs to be shared, it is important to provide a public information exchange mechanism for applications.

The main methods for designing distributed applications are: Remote Process calling (PRC), one of the basic standards of the distributed computing environment (DCE), and Object Transaction monitoring (OTM) -- a combination of CORBA-based object-oriented industrial standards and transaction processing (TP) monitoring technology; messagequeue-a loosely coupled method for constructing distributed applications.

(A) distributed computing environment/remote process call (DCE/RPC)

RPC is a component of DCE and a software standard for Application Integration released by the Open Software Foundation (OSF. RPC imitates a program using function reference to reference the traditional programming method of another program. This reference is a process call. Once called, the program control is switched to the called program.

When RPC is implemented, the called process can be performed locally or remotely in another system. . When the called program completes processing Input Data In the returned variable of the process call, the result is returned to the calling program. After RPC is complete, the program control is immediately returned to the calling program. Therefore, RPC imitates the call/return structure of subprograms. It only provides synchronous data exchange between the client (the calling program) and the server (the called process.

(B) Object Transaction monitoring (OTM)

The combination of the object-oriented industrial standard and transaction processing (TP) Monitoring Technology Based on CORBA defines the architecture using the object-oriented technology and method in the CORBA specification; public Client/Server programming interfaces; guidelines for data transmission and Translation between multiple platforms; languages for developing distributed application interfaces (IDL, it also provides a wide and consistent mode for constructing distributed client/server applications.

(C) Message Queue)

Message Queue provides a loosely coupled method for constructing distributed applications that are implemented in synchronous or asynchronous mode. API calls of message queues are embedded into new or existing applications, and information exchange is provided by sending messages to memory or disk-based queues or reading from them. Message queues can be used in applications to perform a variety of functions, such as requiring services, exchanging information, or asynchronous processing.

Middleware is an independent system software or service program. distributed application systems share resources between different technologies to manage computing resources and network communication. It is a key software in computer systems. It can achieve application interconnection and interoperability, and ensure the security, reliability, and efficiency of the system. Middleware is located between user applications and operating systems and network software. It provides public communication means for applications and is independent of networks and operating systems. Middleware provides developers with application interfaces that are common to all environments. When an application is embedded with its function calls, it can use the features of its specific operating system and network environment, execute the communication function for the application.

If no message-oriented middleware completes information exchange, application developers must learn how to use the functions of the network and operating system software to write corresponding applications to send and receive information in order to transmit data, there is no standard method for information exchange. Each application must be programmed to communicate with one or more applications on multiple platforms and in different environments. For example Host Communication between systems requires knowledge about how to exchange information on the network (for example, socket programming with TCP/IP). To achieve communication between different processes on the same host, knowledge of operating system message queues or named pipelines will be required.

Currently, there are many types of middleware, such as transaction management middleware (such as IBM CICs) and Web Application Server middleware for Java applications (such as IBM WebSphere Application Server, message transmission middleware (MOM) is one of them. It simplifies data transmission between applications, shields underlying heterogeneous operating systems and network platforms, and provides consistent communication standards and application development, ensures reliable cross-platform information transmission and data exchange in a distributed computing network environment. It is based on the storage-forwarding mechanism of message queues and provides a unique asynchronous transmission mechanism. It can implement Application Integration and data exchange based on message transmission and asynchronous transaction processing.

With its unique security mechanism, simple and fast programming style, outstanding stability, scalability and cross-platform, and powerful service logic and messaging capabilities, IBM message middleware MQ, it has become the most popular message-oriented middleware product in the industry.

MQ is powerful across platforms. It supports up to 35 platforms. It supports various mainstream UNIX operating system platforms, such as HP-UX, Aix, Sun Solaris, Digital UNIX, open vmx, SunOS, and NCR Unix. It supports various host platforms, such: OS/390, MVS/ESA, VSE/ESA, and Windows NT servers are also supported. Windows 9x, Windows NT, Windows 2000, Unix (unixware, Solaris), and major Linux versions (RedHat and turbolinux) are supported on the PC platform ). In addition, MQ supports other operating system platforms, such as OS/2, AS/400, sequent Dynix, SCO openserver, SCO unixware, and tandem.

Basic concepts of MQ:

1) Queue Manager

The queue manager is a top-level concept in the MQ system. It provides queue-based message services for us.

2) messages

In MQ, we define the data transmitted by the application to MQ as a message. We can define the message content and give a broad understanding of the message, for example: various types of data files of the user. The processing requests sent by an application to other applications can be used as messages. A Message consists of two parts:

Message Descriptor (Message discription or message header) describes the features of a message, such as the priority, lifecycle, and ID of a message;

The message body, that is, the user data. In MQ, messages are divided into two types: non-permanent (non-persistent) messages and permanent (persistent) messages. Non-permanent messages are stored in the memory, it is designed to improve performance. It cannot be restored when the system powers down or the MQ Queue Manager restarts. This type of message can be used when users have low requirements on message reliability and focus on system performance. For example, when publishing stock information, because the stock information is constantly updated, we may release it every several seconds, and new messages will continuously overwrite old messages. Permanent messages are stored on hard disks and recorded data logs. They have high reliability and ensure that messages are not lost or duplicated in the case of network or system faults.

In addition, there are also concepts of logical messages and physical messages in MQ. Using Logical messages and physical messages, we can segment large messages or combine several complete messages into a group of application logic for processing.

3) queue

A queue is a secure place for storing messages. A queue stores messages until they are processed by applications.

Message Queue: Work :

A) program a forms a call to the Message Queue System. This call informs the message queue system that the message is ready and sent to program B;

B) The message queue system sends the message to the system where program B resides and puts it in the queue of program B;

C) after the appropriate time, program B reads the message from its queue and processes the message.

With advanced programming ideas and internal working mechanisms, MQ can ensure reliable message transmission in various network conditions and overcome the status quo of poor or unstable network line quality, during transmission, if the communication line fails or the remote host fails, local applications will not be affected. You can continue to send data, you do not have to wait for the Network to recover or the remote host to run normally.

In MQ, queues are divided into many types, including local queues, remote queues, template queues, dynamic queues, and alias queues.

Local queues are divided into common local queues and transmission queues. Common Local queues are queues for applications to perform read and write operations on them through APIS. Transmission queues can be understood as storage-forwarding queues, for example: we send a message to the MQ system and send it to the remote host. When the network fails, MQ stores the message in the transmission queue for temporary storage. When the network recovers, the message is sent to the remote destination.

A Remote Queue is a local definition of a destination queue. It is similar to an address pointer pointing to a destination queue on a remote host. It is just a definition and does not really occupy disk storage space.

Template queue and dynamic queue are a special feature of MQ. A typical purpose of them is to consider the scalability of the system. We can create a template queue. When a new queue needs to be added in the future, MQ automatically generates a dynamic queue every time a template queue is opened, we can also specify this dynamic queue as a temporary queue or a permanent queue. If it is a temporary queue, we can delete it while disabling it. On the contrary, if it is a permanent queue, we can keep it permanently for my use.

4) Channels

A channel is a pipeline for passing messages between queue managers in the MQ system. It is a logical concept built on a physical network connection and is also the essence of MQ products.

In MQ, there are three main channel types: Message channel, mqi channel, and cluster channel. The message channel is used to transmit messages between the MQ server and the server. It must be emphasized that the channel is one-way, and it also has a sender, receive (receive ), different types such as requestor and server are available to users under different circumstances. The mqi channel is used for communication and message transmission between MQ client and MQ server. Different from the message channel, the mqi Channel transmits messages in two directions. The cluster channel is used for communication between queue managers in the same MQ cluster.

How MQ works

:

First, let's look at the local communication situation. Application A and application B run on the same system A, and they can communicate with each other through Message Queue Technology: application A sends a message to queue 1, which is available when application B needs it.

The second is remote communication. If the information transmission target is changed to application C on system B, this change will not affect application, application A sends a message to queue 2. System A's MQ finds that the target queue pointed to by Q2 is actually located in system B, it stores information in a local special queue-transmission Queue (transmission Queue ). We create a message channel from system A to system B. The message channel proxy reads the message from the transmission queue and transmits the message to system B. Then, wait for confirmation. MQ deletes the message from the transmission queue only after it is successfully received by system B. If the communication line is disconnected or system B is not running, the information will remain in the transmission queue until it is successfully transmitted to the destination. This is the most basic and important technology of MQ-ensuring information transmission and only one (once-and-only-once) transfer.

MQ provides a loosely coupled connection method for application integration, because applications that share information do not need to know the physical location (network address) of each other, and do not need to know how to establish communication between each other; you do not need to be in the running status at the same time; you do not need to be running in the same operating system or network environment.

Basic MQ configuration example

To achieve communication between the two hosts on the network, if point-to-point communication is adopted, we should at least establish the following MQ objects:

In sender:

1) create a queue manager qma: crtmqm-Q qma

2) define a local transmission queue: Define qlocal (qmb) usage (xmitq) defpsist (yes)

3) create a Remote Queue: Define qremote (QR. TOB) rname (lqb) rqmname (qmb) xmitq (qmb)

4) define the sending channel: Define channel (A. to. B) chltype (SDR) conname ('IP of B ') xmitq (qmb) + trptype (TCP)

In receiver B:

1) create a queue manager qmb: crtmqm-Q qmb

2) define the local queue qlb: Define qlocal (lqb)

3) create a receiving channel: Define channel (A. to. B) chltype (rcvr) trptype (TCP)

After the above configuration, we can achieve one-way communication from host a to host B. To achieve two-way communication between the two, refer to this example to create the required MQ object.

MQ Communication Mode

1) point-to-point communication: the point-to-point mode is the most traditional and common communication mode. It supports one-to-one, one-to-many, multiple-to-many, multiple-to-one, and other configuration methods, supports tree, mesh, and other topology structures.

2) Multi-Point broadcast: MQ is applicable to different types of applications. The important and developing application is the "Multi-Point broadcast" application, which can send messages to multiple destination sites (destination list ). You can use an MQ command to send a single message to multiple target sites and ensure reliable information for each site. MQ not only provides the multi-point broadcast function, but also provides the smart message distribution function. When a message is sent to multiple users on the same system, MQ sends a copy version of the message and the receiver list on the system to the target MQ system. The target MQ system replicates these messages locally and sends them to the queue on the list to minimize the amount of network transmission.

3) publishing/subscription (publish/subscribe) mode: The Publish/subscribe function allows message delivery to break through the geographical limitations of the target queue, so that messages can be distributed according to specific topics and even content, the user or application can receive the required message based on the topic or content. The Publish/subscribe function makes the coupling relationship between the sender and receiver more loose. The sender does not have to care about the recipient's destination address, and the receiver does not have to care about the message sending Address, instead, send and receive messages based on the message topic. In the MQ family, MQ event broker is a product dedicated to data communication using the publish/subscribe technology, it supports publishing and subscription Based on queues and directly based on TCP/IP.

4) Cluster: MQ provides a cluster solution to simplify the system configuration in point-to-point communication mode. A cluster is similar to a domain. A message channel does not need to be established between the queue managers in the cluster. Instead, the cluster channel is used to communicate with other members, this greatly simplifies the system configuration. In addition, the queue managers in the cluster can automatically perform load balancing. When a queue manager fails, other queue managers can take over its work, thus greatly improving the high reliability of the system.

MQ server and MQ Client

MQ is divided into two versions: server and client. In the running environment of the MQ server, there are queue managers, queues, message channels, and other objects. It provides comprehensive message services; MQ client provides a development and running environment for MQ applications. It is the client implementation of MQ APIs. In the client environment, there are no queue manager, queue, and other objects. It establishes communication with the server through the mqi channel and sends messages from the client to the server Queue, or get messages from the server Queue, which is more suitable for good network conditions or real-time communication. At the same time, it should be pointed out that using MQ client will not cause data loss or integrity. MQ client provides the following benefits: Suitable for synchronous processing, reducing System Burden, reducing system management overhead, and reducing disk space requirements.

MQ APIs

MQ supports multiple programming languages, including C, C ++, Java, VisualBasic, COBOL, PL/1, and RPG. It also supports a variety of popular development tools, such: webSphere Studio Application Developer, powerbuiler, Microsoft Visual C ++, Visual Basic, and Delphi. In addition, MQ provides unified programming interfaces on different platforms. You only need to re-compile the interfaces to port programs on different platforms. MQ APIs are easy to learn. You only need to use 13 common and powerful function calls of MQ to write various complex applications at the fastest speed. Users can focus on the implementation of application business logic, rather than the underlying communication and exception handling.

Taking the C language as an example, the development process of an MQ application is as follows:

Mqconn ()/* establish a connection with the queue manager */mqopen ()/* Open the queue for read/write operations */mqput () /* put messages into the queue */mqget ()/* read messages from the queue */mqinq ()/* query queue attributes */mqset () /* set the queue attributes */mqclose ()/* after reading and writing, close the queue */mqdisc ()/* and disconnect the queue manager, release related resources */

Middleware Overview