Go RABBITMQ Message Queuing (ix): Publisher's message acknowledgement mechanism

The message acknowledgement mechanism between queue and consumer is mentioned in the previous article: by setting an ACK. So publisher can not know that his post message has reached the queue, or even a step closer, whether a certain consumer processing it? After all, for some very important data, Publisher may need to confirm that a message has been handled correctly.

In our system, we did not implement this acknowledgement, that is, Publisher does not care if the message is consume. He just publish his state to the upper layer, which is handled by the logic of the upper layer. If the message is not handled correctly, some state loss may result. However, the impact of this exception is negligible because it provides other mechanisms for forcing the full state to be refreshed.

For some asynchronous operations, such as a client needs to create a filesystem, this may take a long time, or even a few seconds. By this time RPC can solve this problem. Therefore, there is no publisher-side confirmation mechanism.

So is there a mechanism to ensure that Publisher can perceive that its message is handled? The answer is yes. Here to thank the classmate: he in my "RABBITMQ Message Queue (iii): task distribution mechanism" after the message discussed the problem, but also found some information. Here I tidied up his reprint and an article and an original article. The cohesion has been attached.

1. Transaction mechanism VS Publisher Confirm

If the standard AMQP protocol is used, the only way to ensure that the message is not lost is to use the transaction mechanism-to make the channel in transactional mode, to publish the message to it, and to perform a commit action. In this way, the transaction mechanism can incur a significant amount of overhead and result in a 250% decrease in throughput. To remedy the problems caused by the transaction, the confirmation mechanism (Publisher Confirm) was introduced.

In order to enable the confirm mechanism, the client first sends the Confirm.select method frame. Depending on whether the No-wait property is set, the broker will determine whether to answer with Confirm.select-ok. Once the Confirm.select method is used on the channel, the channel will be in confirm mode. The channel in transactional mode can no longer be set to confirm mode, and vice versa.
Once the channel is in confirm mode, both the broker and the client start the message count (based on Confirm.select, counting from 1). After processing the message, the broker confirm it on the current channel by sending a basic.ack. The value of the Delivery-tag field identifies the sequence number of the message being confirm. Broker can also set the multiple field in Basic.ack to indicate that all messages to the specified serial number have been handled correctly by the broker.

In exceptional cases, the broker will not be able to successfully process the corresponding message, at which point the broker will send Basic.nack instead of Basic.ack. In this case, the meaning of each field value in the Basic.nack is the same as the corresponding field in Basic.ack, and the value of the Requeue field should be ignored. by Nack one or more messages, the broker indicates that it cannot finish processing the corresponding message and refuses to be responsible for the processing of those messages. In this case, the client can choose to re-publish the message.

After the channel is set to confirm mode, all subsequent messages that are publish will be confirm (i.e. ACK) or nack once. But there is no guarantee of the speed of the message being confirm, and the same message will not be confirm and nack.

2. When is the message confirmed?

The broker will confirm the message in the following scenario:

• Broker discovers that the current message cannot be routed to the specified queues (if the mandatory attribute is set, the broker sends Basic.return first)
• Messages of non-persistent properties arrive in all of the queue (and mirror queue) that they should have reached
• The persistent message arrives in all the queue (and mirror queue) it should reach, and is persisted to disk (Fsync)
• Persistent messages are consume from all the queue in which they reside (acknowledge if necessary)

The broker loses the persisted message if the broker is in an exception before writing the above message to disk. Under certain conditions, this situation can cause the broker to run in a strange way. For example, consider the following scenario:

1. A client publish persistent messages into a persistent queue
2. Another client consume the message from the queue (note: The message has a persistent property and the queue is persisted) when it has not been ACK
3. Broker Exception Restart
4. Client re-connects and starts consume message

In the above scenario, the client has reason to believe that the message needs to be re-deliver by (broker). But this is not true: rebooting (possibly) will cause the broker to lose the message. To ensure persistence, the client should use the confirm mechanism. If Publisher uses a channel that is set to confirm mode, Publisher will not receive an ACK for the lost message (this is because consumer did not ack the message and the message was not written to disk).

3. Programming implementation

The first thing to do is to differentiate the AMQP protocol mandatory and immediate flag bits.

Mandatory and immediate are the two flags in the Basic.pulish method in the AMQP protocol that have the ability to return messages to the producer when they are unreachable in the message delivery process. The specific difference is:
1. Mandatory flag bit
When the mandatory flag bit is set to true, if Exchange cannot find a qualifying queue based on its type and message Routekey, The Basic.return method is called to return the message to the producer, and when mandatory is set to false, the broker throws the message directly.
2. Immediate flag bit
When the immediate flag bit is set to true, this message is not placed in the queue if Exchange finds that there are no consumers on the corresponding queue when the message is route to queue (s). When all the queues (one or more) associated with the message Routekey do not have a consumer, the message is returned to the producer through the Basic.return method.

Refer to reference 1 for specific code references.

Resources:

1. http://blog.csdn.net/jiao_fuyou/article/details/21594205

2. http://blog.csdn.net/jiao_fuyou/article/details/21594947

3. http://my.oschina.net/moooofly/blog/142095

Go RABBITMQ Message Queuing (ix): Publisher's message acknowledgement mechanism