Architecture Design: Inter-system Communication (22)-Improved ACTIVEMQ performance (UP)

The above "Architecture design: Inter-system Communication (--ACTIVEMQ) installation and use"

3, ACTIVEMQ performance optimization ideas

The two sections in the previous article mainly introduce the installation and basic use of the message middleware ACTIVEMQ. From the installation configuration and sample code given in the previous article, we have neither modified any configuration of the ACTIVIEMQ service node nor implemented any cluster scenarios. This situation is only suitable for you to familiarize yourself with the working principle and basic operation of the ACTIVEMQ, but if you want to apply ACTIVIEMQ in a production environment, the operation described above is far from digging out its potential performance.

Based on the central idea presented in this series of articles, the system's performance levels include: code-level performance, rule performance, storage performance, network performance, and multi-node collaboration methods (cluster scenarios), so our central approach to optimizing ACTIVEMQ is to optimize the performance of ACTIVEMQ single nodes first, Then configure the ACTIVEMQ cluster. Below we follow this idea, step by step introduce and ACTIVEMQ performance related to those things.

By default, ACTIVEMQ network information is delivered based on the bio-mode in the network IO model. So in order to improve the performance of ACTIVEMQ single node, we should first configure a more efficient network IO model for each individual MQ service node (for a detailed explanation of the network IO Model, refer to the other articles in this series: Architecture Design: Inter-system Communication (3)-- IO communication model and Java practice,
Architecture Design: Inter-system Communication (4)--io communication model and Java practice Medium, Architecture Design: Inter-system Communication (5)--io communication model and Java practice in the next chapter, this idea is one of the core ideas of the whole series of articles.

In addition, we need to consider a storage scheme for ACTIVEMQ, which allows it to efficiently complete storage operations for "persisted" messages (also including temporary storage of "non-persistent" messages) * *. While the knowledge and examples of the storage section are covered in one of my other series of feature articles-system storage (which will soon be produced), we will talk about the choice of storage options in order to understand the performance optimization of the message middleware software.

In addition, we also know that the use of cluster scheme can increase the performance and stability of the entire software, so after the completion of single-node optimization, we also need to provide some kind of cluster scheme to combine multiple activemq, so that they work together (must be working together, It is impossible to improve performance and stability simply by installing multiple ACTIVEMQ nodes without synergy.

Apache Activemq is an open source software that you can download to its source code on ACTIVEMQ's website and perform code-level performance modifications, but obviously this is not a way to get an efficient fee. Therefore, I do not recommend that you and your team change the ACTIVEMQ code implementation (learn about other people's code implementation or can be).

4. Network io4-1, basic connection configuration in ACTIVEMQ

In the above, we mentioned that ACTIVEMQ supports a variety of message protocols, including the AMQP protocol, the MQTT protocol, the Openwire protocol, the STOMP protocol, and so on. On the official website of ACTIVEMQ, all the message protocols currently supported in ACTIVEMQ are listed: AMQP, MQTT, Openwire, REST, Stomp, XMPP;

different protocols require different network listening ports , which are set in the./conf/conf/activemq.xml master configuration file of the ACTIVEMQ installation directory. The master configuration file is described in XML format, where the "transportconnectors" tag describes the network listening ports for various protocols, as shown in the following example:

<transportconnectors>    <transportconnector name="Openwire" uri="tcp://0.0.0.0:61616?" maximumconnections=1000&amp;wireformat.maxframesize=104857600 "/>    <transportconnector name="AMQP" uri="amqp://0.0.0.0:5672?" maximumconnections=1000&amp;wireformat.maxframesize=104857600 "/>    <transportconnector name="Stomp" uri= "stomp:// 0.0.0.0:61613 "/>    <transportconnector name="Mqtt" uri="mqtt://0.0.0.0:1883?" maximumconnections=1000&amp;wireformat.maxframesize=104857600 "/>    <transportconnector name="ws" uri="ws://0.0.0.0:61614?" maximumconnections=1000&amp;wireformat.maxframesize=104857600 "/></transportconnectors>

The access port number configured with the Openwire protocol is 61616 of all IP devices on this computer (0.0.0.0 represents all IP devices on this computer); The access port number configured for the AMQP protocol is 5672 of the native IP devices, and the access port number configured for the STOMP protocol is 61613 of the native IP devices, and so on. Here are a few facts to note:

• The name attribute and the URI attribute of each "transportconnector" tag must be filled in, and the value of the name attribute can be filled out arbitrarily, and it will be displayed as a connector element in the Connections column of the ACTIVEMQ management interface;

• Each "transportconnector" tagged URI element has a fixed notation: The URI header is the specified protocol name, such as AMQP, Mqtt, stomp, and so on. Then is the HOST/IP domain name, specifies the route information that the port listens to, do not use the loopback address such as localhost or 127.0.0.1, otherwise cannot connect to the ACTIVEMQ through the network, next is the port information, the specified port cannot repeat, otherwise it will create a conflict.

• URI parameter section, each protocol has some specific parameters, readers can refer to the ACTIVEMQ official website, about the "agreement" section of the introduction: http://activemq.apache.org/protocols.html. However, some parameters are common to various protocols, such as the "Maximumconnections" attribute used in the above example, representing the maximum number of connections supported on this port; "Wireformat.maxframesize" Zodiac represents "one full message" for the support agreement The maximum amount of data in bytes, which you can find in the parameter description for wire formats in the ACTIVEMQ official website: http://activemq.apache.org/configuring-wire-formats.html.

Any transport which involves marshalling messages onto some kind of the network transport like TCP or UDP would typically use T He openwire format. This was configurable to customize what things appear on the wire.

• URI parameter section, the parameters that can be shared by each protocol also include parameters related to basic connection properties, which can be found in the official website: http://activemq.apache.org/connection-configuration-uri.html Detailed instructions in the. Alternatively, if you are using the TCP protocol, you can also include a TCP-related attribute description in the URI parameter section, as described in the official website http://activemq.apache.org/tcp-transport-reference.html If you are using the UDP protocol (which is of course not recommended), you can also include a UDP-related attribute description in the URI parameter section, as detailed in the official website http://activemq.apache.org/udp-transport-reference.html.

• In the "transportconnector" tag, there are optional properties in addition to the "name" attribute and the "uri" attribute that must be filled in, for example: Enablestatusmonitor, updateclusterclients. A detailed description of the properties can be found in the section "Server Side Options" in the Official document Http://activemq.apache.org/configuring-transports.html. In the following articles, we will continue to use some of these settings

4-2. Special instructions

• In the configuration information given above, you can find that when we describe various message protocols, the header of the URI description information is the protocol name: for example, when a listening port describes the AMQP protocol, the URI description format is "amqp://..."; Describes the listening port of the STOMP protocol. The URI description format used is "stomp://...". Only when the Openwire protocol is described, the URI header uses the "tcp://...". This is because the default message protocol in ACTIVEMQ is Openwire:

Openwire is binary protocol designed for working with Message oriented middleware. It is the native wire format of ActiveMQ.

Openwire is we cross language wire Protocol to allow native access to ActiveMQ from a number of different languages and P Latforms. The Java openwire transport is the default transport in ActiveMQ 4.x or later.

• We also mentioned above that ACTIVEMQ fully supports the AMQP protocol. But readers will find that there is no such structure as Exchange in ACTIVEMQ. What's going on here? In fact, in the AMQP Version 1.0 specification document developed by the International Standards Organization (ISO) and IEC, the OASIS Advanced Message queueing Protocol
  (AMQP) Version 1.0), does not say that the AMQP message must pass the exchange rules to reach the queue, nor does it stipulate that exchange must implement some sort of routing for the rule. Therefore, when supporting the AMQP protocol, it is necessary to have routing rules such as Exchange, depending on the messaging middleware software vendor of AMQP's own decision . The following code is an example of sending an AMQP message using the JMS API to connect to the AMQP port of Activemq:

Package MQ. Test. AMQP;Import Javax. JMS. Connection;Import Javax. JMS. Destination;Import Javax. JMS. MessageProducer;Import Javax. JMS. Session;Import Javax. JMS. TextMessage;import org. Apache. Qpid. AMQP_1_0. JMS. Impl. Connectionfactoryimpl;public class JMSProducer {public static void main (string[] args) throws Throwable {//Note that the JMS-AMQP is using Apache QP The implementation of the ID. If you need to run this code, please import the QPID client/ * * <dependency> * <groupId>org.apache.qpid</groupId> * <artifactid>         ;qpid-amqp-1-0-client-jms</artifactid> * <version>0.32</version> * </dependency> * */Connectionfactoryimpl factory = Connectionfactoryimpl. Createfromurl("amqp://192.168.61.138:5672");Connection Connection = Factory. Createqueueconnection();Connection. Start();Set up a session to connect to a queue called/test session session = Connection. CreateSession(False, Session. AUTO_acknowledge);Destination Queue = Session. Createqueue("/test");MessageProducer MessageProducer = Session. Createproducer(queue);Start Sending message TextMessage Outmessage = Session. Createtextmessage();Outmessage. SetText("23456656457567456");MessageProducer. Send(Outmessage);Close MessageProducer. Close();Connection. Close();}}

4-3, network Configuration optimization

So, readers, do you think the connection port configuration in the previous section is too lengthy to manage? This is true, and we will only use several fixed protocols in the actual work . So activemq in version 5.13.0+, Openwire, STOMP, AMQP, MQTT, the port monitoring of the four major protocols was merged and represented using the Auto keyword . In other words, ACTIVEMQ will listen to the message status of this port and automatically match the appropriate protocol format. The configuration is as follows:

<transportConnectors>    <transportConnector name="auto" uri="auto://0.0.0.0:61617?maximumConnections=1000" /></transportConnectors>

In the URI configuration information above, all common connection configuration, wire Formats configuring, Server side options, and TCP Transport can be used Configuration items. But this optimization simply makes ACTIVEMQ's connection management simple and does not improve the processing performance of individual nodes.

If you do not specifically specify a network listening port for ACTIVEMQ, these ports will use the Bio Network IO Model . So in order to improve the performance of the single-node network throughput first, we need to explicitly specify the active network IO model as follows:

<transportConnectors>      <transportConnector name="nio" uri="nio://0.0.0.0:61618?maximumConnections=1000"/>  </transportConnectors>

Note that the URI format header begins with "NiO", indicating that the port uses the NIO network IO model based on the TCP protocol. However, such a setup can only enable this port to support the Openwire protocol. So how do we let this port support the NIO network IO model and allow it to support multiple protocols? The server-side settings of the ACTIVEMQ allow developers to use the "+" symbol to set a number of features for the port, as follows:

<transportConnector name="stomp+nio" uri="stomp+nio://0.0.0.0:61613?transport.transformer=jms"/>// 表示这个端口使用NIO模型支持Stomp协议<transportConnector name="amqp+ssl" uri="amqp+ssl://localhost:5671"/>// 表示这个端口支持amqp和ssl密文传输

So if we need a port that supports the NIO network IO model and it supports multiple protocols, you can configure the following:

<transportConnector name="auto+nio" uri="auto+nio://0.0.0.0:61608?maximumConnections=1000" />

In addition, if you are configuring for a production environment, you should at least configure the maximum number of connections supported by this port, set the maximum transfer value for each message, set the maximum number of worker threads used by NIO for the thread pool (of course, you already know where the documents for these settings are located, So you can set the property to increase or decrease according to your own situation):

<transportConnector name="auto+nio" uri="auto+nio://0.0.0.0:61608?maximumConnections=1000&amp;wireFormat.maxFrameSize=104857600&amp;org.apache.activemq.transport.nio.SelectorManager.corePoolSize=20&amp;org.apache.activemq.transport.nio.SelectorManager.maximumPoolSize=50" />

The following drawings are the contents of the Connections page displayed in the ACTIVEMQ Admin console after changing the network connection settings. Note that the WS-Protocol port is an extra-reserved configuration-because the protocol in Auto mode does not support WS:

5. Several basic concepts in the JMS specification

Since ACTIVEMQ is the complete implementation of the JMS specification, in order to clarify how ACTIVEMQ is stored and dispatched, it is necessary to first explain several concepts related to JMS and storage and dispatch. They are: Messaging Mode (subscription-publish and load-balanced mode), message store mode (persistent and non-persistent messages), and subscription models (persistent and non-persistent subscriptions).

5-1. Subscription release mode and load Balancing mode

In the previous article we have detailed the subscription-release mode and load-balancing mode (HTTP://BLOG.CSDN.NET/YINWENJIE/ARTICLE/DETAILS/50916518#T5). In the official documents of ACTIVEMQ, their English names are topics and queue respectively. Both of these message "send-accept" patterns are standard patterns in the JMS specification. In order to make readers in subsequent readings not to be confused with the other concepts in JMS, I will use the canonical name in the Post article.

• Subscription-Release mode:

In subscription-release mode, messages are duplicated in multiple copies and sent to all subscribers individually. In fact, as you'll see in the later description, the process of copying is actually not as simple as you might think.

• Load Balancing Mode:

In load Balancing mode, a message is sent to a message consumer, and if the current queue does not have a message consumer, the message is stored. You will also find that the process is not simple, as described in the following article.

5-2. Persistent messages and non-persistent messages

The names of non-persisted and non-persisted messages in JMS are: Non_persistent message and persistent meaage. They refer to whether the message is persisted in any one of the send-accept modes (subscription-release mode and load-balanced mode).

The non_persistent message is stored only in the memory area of the JMS service node and is not stored on some persistent media (we will describe the persistence media that ACITVEMQ can support in the following: Kahabd, AMQ, and relational data). In the extreme case, the memory area of the JMS service node is not used enough, and only some kind of auxiliary scheme will be used for the dump (for example, ACTIVEMQ will use a "temporary storage area" on the disk for staging). Once the JMS service node is down, these non_persistent message will be lost .

The definition of persistent meaage in JMS is that these messages are not affected by the JMS service-side exception state, and the JMS server stores these messages using some persistent storage scheme until the JMS server considers these persistent Meaage was successfully processed by the consumer side . For example, the persistent storage scenarios that can be selected in Activemq include: Kahadb, AMQ, and relational databases.

In the JMS standard API, use Setdeliverymode to flag whether the sender of the message is the persistent meaage sent or the non_persistent message. Examples are as follows:

 ... for  (int  index  = 0  ; index  < 10 ;    index  + +) {TextMessage outmessage = Session.createtextmessage ();    Outmessage.settext ( "This is the message content sent:"  + index ); if  (index % 2  = = 0    ) {Sender.setdeliverymode (deliverymode.non_persistent);    } else  {sender.setdeliverymode (deliverymode.persistent); } sender. send  (outmessage);} ......

then when the JMS service node restarts (note that it is not a producer restart), only 5 of the 10 messages sent in the above code can be saved .

5-3. Continuous subscription and non-persistent subscription

Continuous and non-persistent subscriptions are subdivision processing strategies for the subscription-release model , and the standard salutation in the JMS specification is: Durable-subscribers and non-durable subscribers.

Durable-subscribers means that in "subscription-release" mode, even subscribers marked as Durable-subscribers are offline (perhaps because the subscriber is down or because the Subscriber is intentionally offline), "subscribe-publish" The topic queue of the pattern also holds these messages (depending on the persistence policy effect of the message, the save mechanism is different) until the next subscriber labeled Durable-subscribers is back online and the message is processed correctly. In other words, whether a subscriber marked as durable-subscribers can get a message, and whether it has been offline has no relationship .

Non-durable subscribers means that in subscription-release mode, the topic queue in subscription-release mode does not retain messages for subscribers who are already offline. When the latter sends the message to the currently online subscribers according to the established broadcast rules, the message can be marked as "processing complete."

================== Next

Architecture Design: Inter-system Communication (22)-Improved ACTIVEMQ performance (UP)