Relationship of exchange, route, queue in RABBITMQ

As can be seen from the AMQP protocol, MessageQueue, Exchange, and binding form the core of the AMQP protocol, and we'll take a holistic look at how these three major components can be leveraged from the perspective of application use Rabbit MQ constructs message queues and considerations for using them.

• 1. Declaration MessageQueue

In Rabbit MQ, whether a producer sends a message or a consumer accepts a message, it first needs to declare a MessageQueue. Is there a question of whether the producer statement or the consumer statement? To solve this problem, we need to make clear:

A) The consumer is unable to subscribe or obtain information in the MessageQueue that does not exist.

b) After the message is accepted by Exchange, if there is no matching queue, it is discarded.

After understanding the above two points, it is easy to understand that if a consumer declares a queue, there is a risk that the message that the producer has sent is discarded before the queue is declared. If your app allows messages to be lost through a mechanism that sends a message back, there's no problem with using this scenario. However, if the scheme is not acceptable, it is necessary for both producers and consumers to try to establish a message queue before sending or receiving the message. Here's one thing to be clear, if the client tries to establish a message queue that already exists, Rabbit MQ will not do anything and return the client to build successfully.

If a consumer is listening to a queue's message in one channel, Rabbit MQ does not allow the consumer to declare the other queue in the same channels. Rabbit MQ, a queue can be declared with the Queue.declare command, and the following properties of the queue can be set:

A) Exclusive: Exclusive queue, if a queue is declared as an exclusive queue, the queue is only visible to the connection that first declares it, and is automatically deleted when the connection is broken. There are three points to note here: First, the exclusive queue is visible based on the connection, and the different channels of the same connection can simultaneously access the exclusive queue created by the same connection. Second, for the first time, if a connection has declared an exclusive queue, the other connection is not allowed to establish an exclusive queue of the same name, which is different from the normal queue. Third, even if the queue is persisted, the exclusive queue will be automatically deleted once the connection is closed or the client exits. This queue applies to scenarios where only one client sends a read message.

b) Auto-delete: Automatic deletion, if the queue does not have any subscriptions to the consumer, the queue will be automatically deleted. This queue applies to temporary queues.

c) Durable: persistence, this will be discussed later as a special chapter.

d) Other options, such as if the user simply wants to query whether a queue already exists, if it does not exist, and does not want to establish the queue, You can still call Queue.declare, except that you need to set the parameter passive to true, pass it to Queue.declare, return true if the queue already exists, and return error if it does not exist, but not create a new queue.

• 2. Producers send messages

In the AMQP model, Exchange is a critical component that accepts producer messages and routes messages to Message Queuing. ExchangeType and binding determine the routing rules for messages. So the producer wants to send a message, it must first declare an exchange and the corresponding binding for that exchange. Can be done through Exchangedeclare and Bindingdeclare. In Rabbit MQ, declaring an exchange requires three parameters: Exchangename,exchangetype and durable. Exchangename is the name of the exchange that needs to be specified when creating the binding and the producer pushes the message through publish. ExchangeType, which refers to the type of exchange, in RABBITMQ, there are three types of exchange:direct, fanout, and topic, and different exchange shows different routing behavior. Durable is the persistent property of this exchange, which is discussed in the Message Persistence section. Declaring a binding requires providing a queuename,exchangename and a bindingkey. Let's examine the different routing rules that different ExchangeType show.

When a producer sends a message, it needs to specify a Routingkey and Exchange,exchange after receiving the Routingkey, the ExchangeType will be judged:

A) in the case of the direct type, the Routingkey in the message is compared to the bindingkey in all the binding associated with the exchange, and if it is equal, it is sent to the queue corresponding to the binding.

b) In the case of a fanout type, a message is sent to all Queues that have a Binding defined by the Exchange, which is actually a broadcast behavior.

c) If it is a topic type, the Routingkey and Bindingkey are matched according to the regular expression, and if the match succeeds, it is sent to the corresponding queue.

• 3. Consumer subscription messages

In RABBITMQ, there are 2 ways consumers get messages in a queue:

A) one is to subscribe to a message in a queue via the Basic.consume command, and the channel automatically receives the next message after the last message is processed. (The same channel message processing is serial). The client will receive messages from the queue until the channel is closed or the subscription is canceled.

b) Another way is to actively obtain the messages in the queue through the Basic.get command, but it is absolutely not possible to call Basic.get in lieu of basic.consume by looping, because Basic.get rabbitmq is actually executing, is to first consume a queue, then retrieve the first message, and then cancel the subscription. If you are a high-throughput consumer, it is advisable to use Basic.consume.

If more than one consumer subscribes to the same queue at the same time, RABBITMQ distributes the message in a circular fashion, and each message can be received by only one subscriber. For example, there are queue queues, where Clienta and ClientB both consume the queue, Messagea arrives in the queue, is dispatched to the Clienta,clienta server to receive the response, and the server deletes Messagea Another message Messageb arrives in the queue, the server sends the message to CLIENTB according to the "cyclic push" principle, and then receives the CLIENTB confirmation, deletes the MESSAGEB, and waits for the next message, the server sends the message to Clienta again.

Here we can see that after the consumer receives the message, all need to send a confirmation command to the server, This is the call Basic.ack implementation that can be displayed in Handledelivery, or the Autoack property is true when consume a queue. This ACK is simply a notification that the server can safely delete the message instead of notifying the producer, unlike RPC. If the consumer disconnects the connection before it has time to return an ACK after receiving the message, the message server will retransmit the message to the next subscriber, and the message will be stored without the subscriber.

Since RABBITMQ provides a command to ACK a message, it also provides a command to reject a message. When a client error occurs, the call to the Basic.reject command rejects a message, you can set a Requeue property, and if true, the message server will retransmit the message to the next subscriber, or, if False, the message will be deleted directly. Of course, it is also possible to have the message server delete the message directly and not retransmit it by means of an ACK.

• 4. Persistence:

Rabbit MQ defaults to non-durable queues, Exchange, binding, and messages in queues, which means that once the message server restarts, all declared queues, exchange,binding, and messages in the queue are lost. By setting the durable property of exchange and MessageQueue to true, you can make the queue and exchange persistent, but this does not make the messages in the queue persist, requiring the producer to set delivery mode to 2 when sending messages , only these 3 settings are complete to ensure that the server restart does not affect existing queues. It is important to note that only durable is true for exchange and durable for ture queues to bind, otherwise at bind time, RABBITMQ will be thrown wrong. Persistence can have a significant impact on the performance of RABBITMQ, which can be reduced by more than 10 times times.

• 5. Transaction:

Support for transactions is an important feature of the AMQP protocol. Suppose that when a producer sends a persistent message to the server, because the consume command itself does not have any response returned, the producer cannot know that the message has been lost even if the server crashes without persisting the message. If a transaction is used at this time, that is, a transaction is opened through Txselect (), then a message is sent to the server, and then the transaction is committed through Txcommit (), it is guaranteed that if Txcommit () commits, the message will be persisted if Txcommit () If the server crashes without committing, the message will not be received by the server. Of course Rabbit MQ also provides the Txrollback () command for rolling back a transaction.

• 6. Confirm mechanism:

Using transactions is a guarantee that only committed transactions will be executed by the server. But it also synchronizes the client with the messaging server, which deviates from the nature of Message Queuing decoupling. Rabbit MQ provides a more lightweight mechanism to ensure that a producer can perceive whether a server message has been routed to the correct queue--confirm. If the channel is set to the confirm state, the message sent through the channel is assigned a unique ID, and once the message is correctly routed to the matching queue, the server returns to the producer a confirm that confirm contains the ID of the message. The producer will know that the message has been properly distributed. For persistent messages, confirm is returned only if the message is persisted. The biggest advantage of the confirm mechanism is that it is asynchronous, and the producer can continue to perform other tasks after sending the message. When the server returns confirm, the producer's callback function is triggered, and the producer processes the confirm information in the callback function. If an exception occurs on the message server, which causes the message to be lost, it is returned to the producer with a nack indicating that the message has been lost so that the producer can ensure that the message is not lost by re-sending the message. The confirm mechanism is much better than the transaction in performance. However, the confirm mechanism, can not be rolled back, that is, once the server crashes, the producer can not get confirm information, the producer actually does not know whether the message has been persisted, only to continue to re-send to ensure that the message is not lost, but if the original persisted message, and will not be rolled back, In this way, there will be two identical messages in the queue, and the system needs to support the de-weight.

• Other:

Broker: The Message Queuing server entity is simply the case.
Exchange: A message switch that specifies what rules the message is routed to and to which queue.
Queue: A message queue carrier in which each message is put into one or more queues.
Binding: Bind, which is the role of binding exchange and queue according to routing rules.
Routing key: The routing keyword, exchange messages are delivered based on this keyword.
Vhost: Virtual host, a broker can open multiple vhost, as a separate user permissions.
Producer: The message producer is the program that delivers the message.
Consumer: The message consumer is the program that receives the message.
Channel: The message channels, in each connection of the client, multiple channels can be established, each channel represents a session task.

The use of Message Queuing is probably as follows:

(1) The client connects to the Message Queuing server and opens a channel.
(2) The client declares an exchange and sets the related properties.
(3) The client declares a queue and sets the related properties.
(4) The client uses routing key to establish a good binding relationship between Exchange and queue.
(5) Clients post messages to exchange.

exchanges, queues, and bindingsExchanges, queues, and bindings are three basic concepts, and their role is: Exchangesis where producers publish their messages, Queuesis where the messages end up and is received by consumers, and Bindingsis how the messages get routed from the exchange to particular queues. Here we use a simple mind map to organize the above concepts: The above also mentions a concept of vhost, Vhost is to organize the concept of exchanges, queues, and bindings, from which we begin to speak: VHostVhosts is also a basic concept of AMQP, connected to RABBITMQ by default there is a vhost called "/" is available, local debugging can be directly used when the default vhost. This "/" Access can use the guest username (password guest) Access. You can use the Rabbitmqctl tool to modify the permissions and passwords for this account, which must be a concern in a production environment.     For security and portability reasons, exchange within a vhost cannot bind to other vhost. The vhost can be planned according to the business function group, where the Vhost is created in a clustered environment as long as the vhost is created on a node within the entire cluster. Vhost and permissions are not created through the AMQP protocol, and are created and managed using Rabbitmqctl in RABBITMQ. How to create a vhostVhost and permission (permission) information is not created by AMQP but is added and managed through the Rabbitmqctl tool. after Vhost, let's take a look at the heavy news: message   MessageThe message consists of two parts: payload and label. "Payload" is the actual data to be transferred, as for the format of the data RABBITMQ does not care, "label" describes payload, including Exchange name and optional topic Tag. Once the message is in the consumer there is only payload part, and the label part is not brought here.     RABBITMQ doesn't tell you who sent the message. It's like you got a letter but the envelope is blank. Of course, you want to know who sent or have a way, in the message content contains the sender's information can be. The concept of consumer and producer for messages is that sending and receiving do not correspond to client and server. By channel we can create many parallel transport TCP links that are no longer a bottleneck, We can think of RABBITMQ as an application-level router. How consumer messages are receivedConsumer there are two ways to receive messages: Basic.consumeThe subscription queue. The channel will go into receive mode until you unsubscribe. Subscription mode consumer when the previous message processing is complete (ACK or deny), the new message is actively received. If the message arrives at the queue, it wants to be processed as soon as possible. You should also use the Basic.consume command. In another case, we don't have to keep the subscription, as long as we use the Basic.get command to proactively get the message. After the current message processing is complete, continue to get the message to take the initiative basic.get do not " Using Basic.ge "t as another form of basic.consume in a loop, as this approach has an additional cost compared to Basic.consume: Basic.get essentially is to subscribe to the queue to retrieve a message after the unsubscribe. Basic.consume is usually used when the consumer throughput is large. What if there's no consumer?If the message is not consumer, it will stay in the queue honestly. multiple consumer subscribe to the same queueAs long as consumer subscribes to the queue, the message is sent to the consumer. Our question is, how is the message in the queue distributed in this case? If a rabbit queue has multiple consumer, a message specific to the queue will only be sent to one of the consumer. Message ConfirmationAll received messages require a response message (ACK) to be sent. There are two ways in which the consumer uses Basic.ack to explicitly send an ACK, and one is to specify Auto_ack as true when subscribing to a queue.   As soon as the news arrived consumer there RABBITMQ would think the message had been ACK. Note that there is no relationship between the response and the sender of the message, the ACK is just consumer to RABBITMQ to confirm that the message has been correctly received, RABBITMQ can safely remove the message, that's all. What if I don't respond correctly?If consumer has received a message and has not sent an ACK to RABBITMQ, RABBITMQ will assume that the message is not delivered successfully and will be re-posted to another consumer. If your application is broken, you can set up a fallback program to continue processing the message. If the consumer itself has a problem with the logic and does not send ACK processing, RABBITMQ no longer sends a message to the consumer. RABBITMQ will think that this consumer has not finished processing the last message, No ability to continue receiving new messages. We can make good use of this mechanism, and if the processing process is quite complex, the application can delay sending an ACK until the processing is complete. This effectively controls the load on this side of the application without being hit by a lot of messages. Reject Message  Because the ACK response message has not been issued because the message is to be rejected, there are two options for rejecting the message here: 1.Consumer to disconnect RABBITMQ so RABBITMQ will re-queue the message,   Disposed of by other consumer. This method is supported in all versions of RABBITMQ. The disadvantage of this is that the disconnection increases the additional burden on the RABBITMQ, especially if the consumer exception occurs when every message fails to process properly.   2. RabbitMQ 2.0.0 can use the Basic.reject command and receive the command RabbitMQ will be re-posted to the other consumer. If you set Requeue to FALSE,RABBITMQ, the message is removed from the queue directly. In fact, there is a choice is to ignore this message directly and send an ACK, when you are clear until the message is abnormal, there will be no consumer can handle, you can discard the exception data. Why send a Basic.reject message instead of an ACK? Later versions of RABBITMQ may introduce the "Dead letter" queue, use Basic.reject and set Requeue to False if you want to use dead letter to do some articles. Message PersistenceThe persistence of the message requires that the delivery mode value be set to 2 when the message is delivered. Because the message is actually stored in the queue, "with Mao" logically, Message persistence also requires that Exchange and queue are also persisted. This is the three conditions that message persistence must meet. The cost of persistence is performance loss, disk IO is much slower than RAM (using SSDs can significantly improve the performance of message persistence), Persistence greatly reduces the messages that RABBITMQ can process per second. The performance gap between the two may be more than 10 times times. Message RecoveryConsumer the ACK message is returned after retrieving a message from the durable queue, RABBITMQ marks it for easy subsequent garbage collection. If a persistent message is not taken away by consumer, Exchange and queue (and bingding relationships) are automatically rebuilt after RABBITMQ restarts, and messages are re-entered into the corresponding queues,exchanges through persistent log rebuilds. with, Mao? And then we'll see where the message actually resides: Queue QueueQueues is the place where massage is located and waiting to be received, and messages are placed in a queue unless they are thrown into a black hole. Queue is well suited for load balancing, RABBITMQ can be rotated in several consumer (round-robin) . How to create a queueBoth consumer and producer can create a queue, and if consumer is created to avoid consumer subscribing to a nonexistent queue, there is a risk that the message has been delivered but consumer has not yet created the queue. Then the news will be thrown into the black hole, in other words the message is lost; a good way to avoid this is to try to create a queue producer and consumer.   If consumer cannot complete the creation of a new queue if it has subscribed to another queue, the previous subscription must be canceled to set the channel to transfer mode ("transmit") in order to create a new channel.   When creating a queue, it is usually necessary to specify a name, which is convenient for consumer subscriptions. Even if you do not specify rabbit it assigns a random name to it, which is useful when using a temporary anonymous queue to complete a RPC-OVER-AMQP call. There are also two very useful options when creating a queue: Exclusive-when set to True, your queue becomes private and can is only consumed by your app. This is useful if you need to limit a queue to only one consumer. Auto-delete-the queue is automatically deleted when the last consumer Unsubscribes.   If you want to create a temporary queue that uses only one consumer, you can use Auto-delete and Exclusive.consumer once disconnected to automatically delete the queue. What happens when you create a queue repeatedly? If the queue creation option is exactly the same, RABBITMQ directly returns success, and if the name is the same but the creation option is inconsistent, the creation failure is returned. If you want to check if the queue exists, You can set the passive option for the Queue.declare command to true: If the queue exists, it will return success if the queue does not exist and the creation logic will not be executed. How is a message routed from a dynamic to a different queue? That's what it looks like.   Bindings and Exchanges how messages are sent to the queue  How is the message sent to the queue? It's about AMQP. Bindings and Exchanges. The delivery of messages to the queue is done through exchange, and as in the case of live mail delivery, there are rules that bind the queue to exchange via routing key in RABBITMQ. This binding relationship is the BIND. Message sent to RABBITMQ will carry a routing key (even if it is an empty key), RABBITMQ will match routing key according to bindings, if the match success message is forwarded to the specified queue, If there is no match to the queue message it will be thrown into the black hole. How to send to multiple queuesis the message distributed to multiple queues? The AMQP protocol defines several different types of exchange:direct, Fanout, topic, and headers. Each of these implements a routing algorithm. The routed message of the header does not rely on routing key but rather matches the header portion of the AMQP message, which is the same as direct exchange mentioned below, but with much less performance and is rarely used in real-world scenarios. Direct ExchangeRouting key exact match before forwarding
fanout ExchangeIgnoring routing key, the message is broadcast directly to all bound queue
Topic ExchangeMatch to Routing key pattern Exchange PersistenceQueue creation and exchange are not persisted by default, and queue and exchange disappear after a node restart, specifically specifying the durable property for queue and exchange. consumer is directly creating a TCP link to rabbitmq? Here's the answer: ChannelWhether you want to publish a message or get a message, your application needs to connect to RABBITMQ over TCP. The channel is created after the application connects and passes permission authentication to execute the AMQP command. Channel is built on the actual TCP connection above the communication pipeline, The idea of introducing the channel concept instead of directly sending the AMQP command directly over a TCP link is a two-factor consideration: the creation of hundreds or thousands of TCP links, on the one hand, and the wasted TCP link On the one hand, the system bottleneck will soon be touched. The communication between multiple processes and RABBITMQ can be done on a TCP link after the channel has been introduced. We can make the TCP analogy an optical cable, then the channel is like a fiber in the fiber cable. References[1] rabbits and warrens http://blogs.digitar.com/jjww/2009/01/rabbits-and-warrens/[2] rabbit and rabbit nest http// blog.ftofficer.com/2010/03/translation-rabbitmq-python-rabbits-and-warrens/

Relationship of exchange, route, queue in RABBITMQ