RABBITMQ Basic Concept Introduction

"Introduction"

Have you ever encountered two (multiple) systems that need to synchronize certain data through a timed task? Are you struggling with the problem of calling and communicating between different processes of heterogeneous systems? If so, congratulations, the messaging service makes it easy for you to solve these problems.
The Messaging service specializes in solving data exchange (message notification/communication) issues between multiple systems and heterogeneous systems, and you can also use it for inter-system service invocation (RPC). The RABBITMQ that this article will introduce is one of the most mainstream message middleware.

"RABBITMQ Introduction"

AMQP, Advanced message Queuing Protocol, is an open standard for application-layer protocols designed for message-oriented middleware. Message middleware is mainly used for decoupling between components, the sender of the message does not need to know the existence of the message consumer, and vice versa.
The main features of AMQP are message-oriented, queue, routing (including point-to-point and publish/subscribe), reliability, and security.
RABBITMQ is an open source AMQP implementation that is written in Erlang and supported by a variety of clients such as Python, Ruby,. NET, Java, JMS, C, PHP, ActionScript, XMPP, stomp, etc. Support Ajax. It is used to store and forward messages in distributed system, which is very good in ease of use, extensibility, high availability and so on.
The following will focus on some of the basic concepts in RABBITMQ, and understand these concepts as the basis for using good RABBITMQ.

connectionfactory, Connection, Channel

ConnectionFactory, Connection, and channel are the most basic objects in the API RABBITMQ provided externally. Connection is the socket link for RABBITMQ, which encapsulates some of the logic associated with the socket protocol. ConnectionFactory is a manufacturing facility for connection.
Channel is one of the most important interfaces we have with RABBITMQ, and most of our business operations are done in the channel interface, including defining queue, defining exchange, binding queue and exchange, publishing messages, and so on.

Queue

a queue is an internal object of RABBITMQ that is used to store messages, with representations.

messages in the RABBITMQ can only be stored in a queue, the producer (p in) produces the message and is eventually delivered to the queue, and the consumer (c) can get the message from the queue and consume it.

multiple consumers can subscribe to the same queue, and messages in the queue are evenly distributed to multiple consumers, rather than every consumer receiving all the messages and processing.

Message Acknowledgment

in real-world applications, consumers may receive messages in the queue, but they do not complete the performance outage (or other surprises), which can cause messages to be lost. To prevent this from happening, we can ask the consumer to send a receipt after consuming the message to RABBITMQ,RABBITMQ to receive the message receipt (msg acknowledgment) before removing the message from the queue If RABBITMQ does not receive a receipt and detects that the consumer's RABBITMQ connection is broken, RABBITMQ will send the message to other consumers (if multiple consumers exist) for processing. There is no concept of timeout here, and a consumer processing the message longer will not cause the message to be sent to other consumers unless its RABBITMQ connection is broken.
There is another problem here, if our developers forget to send receipts to RABBITMQ after they have processed the business logic, this will result in more and more messages piling up in serious bug--queue, and consumers will repeat consuming these messages and repeating the business logic after they restart ...

In addition the pub message is no ACK.

Message Durability

if we want to not lose the message even when the RABBITMQ service is restarted, we can set the queue and message to be persisted (durable), which ensures that our RABBITMQ messages will not be lost in most cases. But still can not solve the small probability of the loss of events (such as the RABBITMQ server has received the message of the producer, but not enough time to persist the message when the RABBITMQ server is powered off), if we need to manage this small probability event, then we have to use the transaction. Since this is a simple introduction to RABBITMQ, RABBITMQ related transactions will not be explained here.

Prefetch count

we said earlier that if more than one consumer subscribes to a message in the same queue, the message in the queue is split to multiple consumers. At this point, if the processing time of each message is different, it may cause some consumers to be busy, while others quickly finish the work at hand and always idle. We can set Prefetchcount to limit the number of messages each queue sends to each consumer, such as when we set up prefetchcount=1, and the queue sends a message to each consumer every time The queue will send a message to the consumer after the consumer finishes processing the message.

Exchange

in the previous section we saw that the producer delivered the message to the queue, which in fact never happened in RABBITMQ. The real situation is that the producer sends the message to Exchange (the X in the exchanger), and the Exchange routes the message to one or more queue (or discards).

What logic does exchange follow to route messages to a queue? This will be described in the binding section.
There are four types of exchange in RABBITMQ, and different types have different routing policies, which are described in the Exchange Types section.

Routing Key

when a producer sends a message to exchange, a routing key is typically specified to specify the routing rules for the message, and the routing key needs to be used in conjunction with Exchange type and binding key to finally take effect.
in the case where Exchange type is fixed with the binding key (which is usually fixed in normal use), our producer can determine where the message flows by specifying routing key when sending a message to exchange.
the length limit set for routing key is limited to 255 bytes rabbitmq.

Binding

Exchange is associated with the queue through binding in RABBITMQ so that RABBITMQ knows how to properly route messages to the specified queue.

Binding Key

at the same time as binding (binding) Exchange and queue, a binding key is typically specified; When a consumer sends a message to exchange, a routing key is typically specified; When the binding key and the routing When the key matches, the message is routed to the corresponding queue. This will be illustrated in the Exchange Types section, which will list the actual examples.
These bindings allow the same binding key to be used when binding multiple queues to the same exchange.
The binding key does not take effect in all cases, it relies on exchange type, for example, exchange of type fanout ignores the binding key, but instead routes the message to all the queue bound to that exchange.

Exchange Types

RABBITMQ Common Exchange type has fanout, direct, topic, headers Four (the AMQP specification also mentions two types of exchange, respectively, system and custom, not described here), The following are described separately.

fanout

the Fanout type of Exchange routing rule is simple, and it routes all messages sent to that Exchange to all the queues it binds to.

, all messages sent to Exchange (X) by the producer (P) are routed to the two queue in the graph and ultimately consumed by two consumers (C1 and C2).

Direct

the direct type of Exchange routing rule is also simple, which routes messages to a queue whose binding key exactly matches the routing key.

As an example of configuration, we send a message to exchange with routingkey= "error", the message is routed to Queue1 (amqp.gen-s9b ..., This is the queue name automatically generated by RABBITMQ) and Queue2 (Amqp.gen-agl ... If we send a message with routingkey= "info" or routingkey= "warning", the message will only be routed to Queue2. If we send a message with another routingkey, the message is not routed to the two queue.

Topic

The preceding Exchange routing rules for the direct type are exactly the same as the binding key and routing key, but this strict matching method does not meet the actual business requirements in many cases. Topic type of exchange has been extended on matching rules, similar to the exchage of the direct type, and to the queue where the message is routed to the binding key that matches the routing key, but there are some differences in the matching rules, which contract:

• routing key is a period number ". "Delimited string (we will be the period number".) "Each separated section of a separate string is called a word", such as "Stock.usd.nyse", "NYSE.VMW", "Quick.orange.rabbit"
• the binding key, like routing key, is also the period number ". "Delimited string
• there can be two special characters "*" and "#" in the binding key to make a fuzzy match, where "*" is used to match a word, "#" is used to match multiple words (can be 0)

As an example of configuration in, routingkey= "Quick.orange.rabbit" messages are routed to Q1 and q2,routingkey= "Lazy.orange.fox" messages are routed to q1,routingkey= " Lazy.brown.fox "messages that are routed to q2,routingkey=" Lazy.pink.rabbit "will be routed to Q2 (only sent to Q2 once, although this routingkey matches the Q2 of two Bindingkey) , routingkey= "Quick.brown.fox", routingkey= "Orange", routingkey= "quick.orange.male.rabbit" messages will be discarded, Because they don't match any bindingkey.

Headers

The headers type of exchange does not rely on the matching rules of routing key and the binding key to route messages, but rather matches the headers attributes in the message content sent.
specifies a set of key-value pairs when a queue is bound to exchange, and when a message is sent to Exchange, RABBITMQ takes the headers of the message (which is also a key-value pair). Compares whether the key-value pair exactly matches the key-value pair specified by the queue with Exchange bindings, or if the message is routed to the queue in an exact match, otherwise it is not routed to the queue.
This type of exchange is not used (but should also be useful), so don't introduce it.

RPC

MQ itself is asynchronous-based message processing, and all the Producers (P) in the previous example send the message to RABBITMQ without knowing that the consumer (C) is processing success or failure (even if there is no consumer to handle the message).
However, in the actual application scenario, we will probably need some synchronous processing and need to wait for the server to finish processing my message before proceeding to the next step. This is equivalent to the RPC (remote Procedure call, which is called remotely). RPC is also supported in RABBITMQ.

The mechanism for implementing RPC in RABBITMQ is:

• when a client sends a request (message), the properties of the message (messageproperties, which are defined in the AMQP protocol in the 14 properties, which are sent along with the message) set two values ReplyTo (a queue name, Used to tell the server to send a message to the queue after processing is complete, and correlationid (the identification number of this request, the server will need to return this property after processing is completed, the client is based on this ID to know which request was executed successfully or failed to execute)
• The server receives the message and processes
• after the server finishes processing the message, a reply message is generated to the queue specified by ReplyTo with the Correlationid property
• after the client has subscribed to the queue specified by ReplyTo, and receives a reply message from the server, it analyzes which request was executed based on the Correlationid property, and performs subsequent business processing based on the result of the execution.

Summary

This article introduces the most important concepts in RABBITMQ, and makes full use of the capabilities provided by RABBITMQ to handle most of our asynchronous operations.

RABBITMQ Basic Concept Introduction