In-depth understanding of the RABBITMQ service of message middleware technology

What do you mean Message Queuing?

A message is the data that is transferred between apps. Messages can be very simple, such as containing only text strings, or they can be more complex and may contain embedded objects.
Message Queuing is a way of communicating between applications, which can be returned immediately after a message is sent, and the message system ensures reliable delivery of the message. The message publisher simply publishes the message into MQ without having to take care of who gets it, and the message user takes the message out of MQ regardless of who posted it. So the publisher and the user do not have to know the other side of the existence.

Why use Message Queuing?

Message Queuing is an asynchronous collaboration mechanism between applications, when do you need to use MQ?
Like the user orders, generate order, settlement, timed to the system registered users push activity messages, some common process class business will use the Message Queuing service.

I. Introduction of RABBITMQ

RABBITMQ is a message agent for receiving and sending messages, you can think, he is a post office, when you send mail sent to the mailbox, you can be sure that the postman will send the mail to the person who needs to receive the mail, will not send the wrong. In this analogy, RABBITMQ is a mailbox, and it can be understood as the post office and the postman, who are responsible for sending messages out and for receiving information.
The main difference between the RABBITMQ and the post office is that it does not process paper mail, but instead receives, stores, and sends binary data blocks, which is what we typically call messages.

Ii. Basic concepts of RABBITMQ

is the internal structure of the RABBITMQ service

• 1) Message
  Message, which consists of two parts: the message header and the message body. The message body is opaque, but the message header consists of a number of attributes, including: Routing-key (route key), priority (precedence), Delivery-mode (persistent storage).
• 2) Publisher
  Producer, also the producer of the message, which is the application that publishes messages to the exchanger
• 3) Exchange
  A switch that receives messages sent by the producer and then routes them to a queue in the server
• 4) Binding
  Bindings for communication between Message Queuing and the exchanger. is also the rule for message routing, which is equivalent to a routing table.
• 5) Queue
  Message Queuing, which is used to hold messages until they are sent to consumers. A message can go into one or more queues, except that the consumer takes away the message, otherwise it is in the message queue.
• 6) Connection
  Network connections, such as: A TCP connection
• 7) Channel
  Channel, a separate two-way data transmission channel in a multiplexed connection. Whether it's a publishing message, subscribing to a queue, or receiving a message is done through a channel. Multiplexing channels are used to reduce the consumption of system resources.
• 8) Consumer
  Consumer, which is the client application that receives the message from the producer.
• 9) Virtual Host
  A virtual host, a switch, and a message queue-related object. A vhost can actually be seen as a rabbitmp server with its own queues, switches, bindings, and permissions mechanisms. RABBITMQ default Vhost is/.

Iii. Characteristics of RabbitMQ

RabbitMQ is an open source implementation of AMQP developed by the Erlang language. Amqp:advanced Message queue, advanced Messaging Queuing protocol. It is an open standard of Application layer protocol, which is designed for message-oriented middleware, and the client and message middleware based on this protocol can deliver messages, which is not limited by the conditions of product and development language.

Originally originated from the financial system, RabbitMQ is used to store and forward messages in distributed systems, which perform well in terms of ease of use, scalability, and high availability. Specific features include:

• 1) Reliability (reliability)
  RabbitMQ uses a number of mechanisms to ensure reliability, such as persistence, transmission acknowledgement, and release acknowledgement.

• 2) Flexible Routing (Flexible Routing)
  The message is routed through Exchange before the message enters the queue. For a typical routing feature, RabbitMQ has provided some built-in Exchange to implement. For more complex routing capabilities, you can bind multiple exchange together and implement your own exchange through a plug-in mechanism.

• 3) message cluster (clustering)
  Multiple RabbitMQ servers can form a single cluster, forming a logical Broker.

• 4) High Availability (highly Available Queues)
  Queues can be mirrored on machines in a cluster so that queues are still available in the event of a partial node problem.

• 5) multiple protocols (multi-protocol)
  RabbitMQ supports a variety of Message Queuing protocols, such as STOMP, MQTT, and so on.

• 6) Multi-lingual client (many clients)
  RabbitMQ supports almost all common languages, such as Java,. NET, Ruby, and so on.

• 7) Management interface (Management UI)
  RabbitMQ provides an easy-to-use user interface that enables users to monitor and manage many aspects of message Broker.

• 8) tracking mechanism (tracing)
  If the message is abnormal, RabbitMQ provides a message tracking mechanism that the consumer can use to find out what happened.

• 9) Plug-in mechanism (Plugin system)
  RabbitMQ provides a number of plugins to extend from a variety of ways, or to write your own plugins.

Reference article http://www.rabbitmq.com/

Iv. working process of RABBITMQ

• 1) The client connects to the Message Queuing server and opens a channel
• 2) client declares an exchange, queue, and configures related properties
• 3) client uses routing key to establish a good binding relationship between Exchange and queue
• 4) The client passes the message to the exchanger
• 5) After the switch receives the message, the message is routed to the message queue according to the predetermined key and binding relationship
  

V. RABBITMQ Service installation and deployment

1) First need to install the Erlang environment

https://bintray.com/rabbitmq/rpm/erlang[[email protected] ~]# rpm -ivh erlang-20.3.8-1.el7.centos.x86_64.rpm警告：erlang-20.3.8-1.el7.centos.x86_64.rpm: 头V4 RSA/SHA1 Signature, 密钥 ID 6026dfca: NOKEY准备中...                         ################################# [100%]正在升级/安装...   1:erlang-20.3.8-1.el7.centos   ################################# [100%][[email protected] ~]# erlErlang/OTP 20 [erts-9.3.3] [source] [64-bit] [smp:1:1] [ds:1:1:10] [async-threads:10] [hipe] [kernel-poll:false]Eshell V9.3.3  

2) Download and install RABBITMQ

https://github.com/rabbitmq/rabbitmq-server/releases/download/v3.7.6/rabbitmq-server-3.7.6-1.el7.noarch.rpm[[ Email protected] ~]# ll rabbitmq-server-3.7.6-1.el7.noarch.rpm-rw-r--r--1 root root 9511623 June rabbitmq-se Rver-3.7.6-1.el7.noarch.rpm[[email protected] ~]# rpm--import https://dl.bintray.com/rabbitmq/Keys/ Rabbitmq-release-signing-key.asc[[email protected] ~]# Yum install rabbitmq-server-3.7.6-1.el7.noarch.rpm[[ Email protected] ~]# chkconfig rabbitmq-server on Note: The request is being forwarded to SYSTEMCTL enable Rabbitmq-server.service. Created symlink From/etc/systemd/system/multi-user.target.wants/rabbitmq-server.service To/usr/lib/systemd/system /rabbitmq-server.service. [[email protected] ~]#/sbin/service rabbitmq-server startredirecting to/bin/systemctl start        Rabbitmq-server.service[[email protected] ~]# ps-ef|grep rabbitmqrabbitmq 5609 1 23 14:22? 00:00:02/usr/lib64/erlang/erts-9.3.3/bin/beam.smp-w w-a 64-mbas Ageffcbf-mhas AGEFFCBF-MBLmbcs 512-mhlmbcs 512-mmmcs 30-p 1048576-t 5000000-stbt db-zdbbl 1280000-k true---Root/usr/lib64/erlang-prognam E erl---home/var/lib/rabbitmq---pa/usr/lib/rabbitmq/lib/rabbitmq_server-3.7.6/ebin-noshell-noinput-s rabbit Boot -sname [email protected]-boot start_sasl-kernel inet_default_connect_options [{nodelay,true}]-SASL Errlog_type ERROR-SASL sasl_error_logger false-rabbit lager_log_root "/VAR/LOG/RABBITMQ"-rabbit lager_default_file "/var/log/ Rabbitmq/[email protected] "-rabbit lager_upgrade_file"/var/log/rabbitmq/[email protected]_upgrade.log "-rabbit enabled_plugins_file"/etc/rabbitmq/enabled_plugins "-rabbit plugins_dir"/usr/lib/rabbitmq/plugins:/usr/ Lib/rabbitmq/lib/rabbitmq_server-3.7.6/plugins "-rabbit Plugins_expand_dir"/var/lib/rabbitmq/mnesia/[email  Protected] "-os_mon start_cpu_sup false-os_mon start_disksup false-os_mon start_memsup false-mnesia dir"/var/lib/rabbi Tmq/mnesia/[email protected] "-kernel inet_dist_listeN_min 25672-kernel Inet_dist_listen_max 25672rabbitmq 5752 1 0 14:22?        00:00:00/USR/LIB64/ERLANG/ERTS-9.3.3/BIN/EPMD-DAEMONRABBITMQ 5921 5609 0 14:22?        00:00:00 erl_child_setup 1024rabbitmq 5937 5921 0 14:22?        00:00:00 inet_gethost 4rabbitmq 5938 5937 0 14:22? 00:00:00 inet_gethost 4root 5951 5300 0 14:23 pts/0 00:00:00 grep--color=auto rabbitmq[[email protected] ~ ]# netstat-lntup |grep 5672-bash:netstat: Command not found [[email protected] ~]# netstat-lntup |grep 5672tcp 0 0                 0.0.0.0:25672 0.0.0.0:* LISTEN 5609/beam.smp tcp6 0 0::: 5672       :::* LISTEN 5609/BEAM.SMP

3) View status information

[[email protected] ~]# rabbitmqctl statusstatus of node [email protected] ... [{pid,5609}, {running_applications, [{rabbit, ' RabbitMQ ', ' 3.7.6 '}, {Mnesia, ' Mnesia CXC 138 ', ' 4.15.3 '}, {R Abbit_common, "Modules shared by Rabbitmq-server and Rabbitmq-erlang-client", "3.7.6"}, {Ranch_prox      Y_protocol, "Ranch Proxy protocol Transport", "1.5.0"}, {Ranch, "Socket acceptor pool for TCP protocols.", "1.5.0"},  {SSL, "ERLANG/OTP SSL Application", "8.2.6"}, {Public_key, "Public Key Infrastructure", "1.5.2"}, {ASN1, "the Erlang ASN1 compiler version 5.0.5 "," 5.0.5 "}, {inets," inets CXC 138 "," 6.5.2 "}, {jsx," a streaming, evented json par Sing toolkit "," 2.8.2 "}, {Os_mon," CPO CXC 138 "," 2.4.4 "}, {Xmerl," XML parser "," 1.3.16 "}, {crypto," crypto "      , "4.2.2"}, {Recon, "Diagnostic Tools for production use", "2.3.2"}, {lager, "Erlang Logging Framework", "3.5.1"},     {goldrush, "Erlang event Stream Processor", "0.1.9"}, {compiler, "ERTS CXC 138", "7.1.5"}, {syntax_tools, "syntax Tools", "2.1.4"}, {syslog, "an RFC 3164 and RFC 5424       Compliant logging framework. "," 3.4.2 "}, {SASL," SASL CXC 138 One "," 3.1.2 "}, {stdlib," ERTS CXC 138 10 "," 3.4.5 "}, {kernel, "ERTS CXC 138", "5.4.3"}]}, {Os,{unix,linux}}, {erlang_version, "ERLANG/OTP [erts-9.3.3] [source] [64-bit] [Smp:1:1] [Ds:1:1:10] [Async-threads:64] [HiPe]      [kernel-poll:true]\n "}, {memory, [{connection_readers,0}, {connection_writers,0}, {connection_channels,0}, {connection_other,0}, {queue_procs,0}, {queue_slave_procs,0}, {plugins,5936}, {other_proc,20754544 }, {metrics,184432}, {mgmt_db,0}, {mnesia,72912}, {other_ets,1873688}, {binary,55376}, {Msg_      index,28720}, {code,25082003}, {atom,1041593}, {other_system,9173572}, {allocated_unused,8315896}, {reserved_unallocated,3670016}, {Strategy,rss}, {Total,[{erlang,58272776},{rss,70258688},{allocated,66588672}]}]}, {alarms,[]}, {listeners,[{clustering,25672, "::"},{amqp,5672, "::"}]}, {vm_ Memory_calculation_strategy,rss}, {vm_memory_high_watermark,0.4}, {vm_memory_limit,3281294131}, {disk_free_limit, 50000000}, {disk_free,122394349568}, {file_descriptors, [{total_limit,924},{total_used,2},{sockets_limit,829},{ sockets_used,0}]}, {processes,[{limit,1048576},{used,204}]}, {run_queue,0}, {uptime,111}, {kernel,{net_ticktime,60 }}]

Vi. introduction of the RABBITMQ CLI

Comes with multiple command-line tools
1) Rabbitmqctl
Manage and Manipulate commands

• Stop node
• Access node status, active configuration, health check
• Virtual Host Management
• User and Rights Management
• Policy Management
• list queues, connections, channels, communications, consumer
• Cluster member management
  For more information, please refer to the following: http://www.rabbitmq.com/rabbitmqctl.8.html

[[email protected] ~]# rabbitmqctl list_bindings    #查看绑定信息Listing bindings for vhost /...[[email protected] ~]# rabbitmqctl list_exchanges   #查看交换器Listing exchanges for vhost / ...amq.rabbitmq.trace  topicamq.match   headers    directamq.headers headersamq.direct  directamq.rabbitmq.event  topicamq.topic   topicamq.fanout  fanout[[email protected] ~]# rabbitmqctl list_queues   #查看队列Timeout: 60.0 seconds ...Listing queues for vhost / ...

2) Rabbitmq-plugins
is a tool for managing plug-ins: lists, enabling and disabling them

[[email protected] ~]# rabbitmq-plugins--helpusage:rabbitmq-plugins [-n <node>] [-l] [-Q] <command> [ <command options>]general Options:-N node-q quiet-l longnamesdefault node is "[Email protected]", W Here's the ' server ' is the local hostname. On a hostnamed "server.example.com", the node name of the RabbitMQ Erlang node willusually be [email protected] (Unle SS Rabbitmq_nodename have been set to Somenon-default value at broker startup time). The output of hostname-s is usuallythe correct suffix for use after the ' @ ' sign. See Rabbitmq-server (1) Fordetails of configuring the RABBITMQ broker. Quiet output mode is selected with the "-Q" flag. Informational messages Aresuppressed When Quiet mode was in effect. If RabbitMQ Broker uses long node names for Erlang distribution, the "longnames" option should be specified. Some commands accept a optional virtual host parameter for Whichto display results. The default value is "/". Commands:disable <plugin>|--all [--offline] [--online] enable <plugin>|--all [--offline] [--online] help <command> list [patter n] [--verbose] [--minimal] [--enabled] [--implicitly-enabled] set [<plugin>] [--offline] [--ONLINE]&LT;TIMEOUT&G T -Operation timeout in seconds. Default is "infinity". Reference article: http://www.rabbitmq.com/rabbitmq-plugins.8.html

3) Rabbitmqadmin
It can perform some of the same operations as the Web interface, Rabbitmqadmin is just a dedicated HTTP client.
After installing the management plug-in, http://{host name}:15672/cli/rabbitmqadmin for download

[[email protected] ~]# wget https://raw.githubusercontent.com/rabbitmq/rabbitmq-management/v3.7.6/bin/rabbitmqadmin[[email protected] ~]# cp rabbitmqadmin /usr/local/bin/[[email protected] ~]# chmod +x /usr/local/bin/rabbitmqadmin

Vii. RABBITMQ Cluster

RabbitMQ is developed with Erlang, which is very handy because Erlang is inherently a distributed language, but it does not natively support load balancing.

The cluster nodes of the RabbitMQ include:

• 1) Memory node
  The memory node is simply putting all the data in memory, saving only state to memory (exception: Persistent queue content will be saved to disk)
• 2) disk node.
  The disk node puts the data on the disk, saves the state to memory and disk, the memory node does not write to the disk, but it performs better than the disk node, in the cluster, only one disk node to save the state is sufficient, if only the memory nodes in the cluster, then cannot stop them, otherwise all States, messages, etc. will be lost.
  

However, as mentioned earlier, if the message is persisted when the message is posted, even the memory node, the data is securely placed on the disk. A RabbitMQ cluster can share user, Vhost, queue, Exchange, and so on, all the data and state must be replicated on all nodes, with the exception of those currently belonging only to the node that created it, although they are visible and can be read by all nodes. The RabbitMQ node can be dynamically added to the cluster, a node that can be added to the cluster, or a basic load balancer from the cluster ring cluster.

The Rabbit model is roughly divided into the following three types:

• 1) Single mode
• 2) Normal mode
• 3) Mirroring mode

Single mode: The simplest case, non-clustered mode.

Normal mode: the default cluster mode.
For a queue, the message entity exists only in one of the nodes, and a, B two nodes have only the same metadata, that is, the queue structure. When the message enters the A-node Queue, consumer pulls from the B-node, RabbitMQ temporarily transmits the message between A and B, takes the message entity in a and sends it to the consumer by B. So consumer should try to connect to each node and fetch messages from it. That is, for the same logical queue, a physical queue is established on multiple nodes. Otherwise, whether consumer a or B, the export is always a, it will create a bottleneck. One problem with this pattern is that when a node fails, the B node cannot fetch the message entities that are not yet consumed in the a node. If you do a message persistence, you have to wait for the A node to recover before it can be consumed.

Mirroring mode:
The desired queue is made into a mirror queue, which exists in multiple nodes and belongs to the HA scheme of RabbitMQ.
This mode solves the above problem, the essence of which differs from the normal pattern in that the message entity is actively synchronizing between the mirror nodes rather than being temporarily pulled when the consumer is fetching data. The side effects of this mode is also obvious, in addition to reducing the system performance, if the number of mirror queue, coupled with a large number of messages, the network bandwidth within the cluster will be greatly consumed by this synchronous communication, in the case of high reliability requirements of the application.

Copyright belongs to the author: from the author of the original works of migrant workers, if required to reprint, please indicate the source, or will be held accountable for legal responsibility

In-depth understanding of the RABBITMQ service of message middleware technology