RABBITMQ duplicate ACK causes message loss

RABBITMQ Duplicate ACK causes message lossToo important so reproduced, as a private, I hope the original author do not blame Me 

RABBITMQ Duplicate acknowledgement causes message loss

background

RABBITMQ in the application scenario, most of the work Queue Work-queue mode is used.

In a common work queue pattern, consumer workers will continuously poll to pull the latest messages from the queue and allow multiple workers to be added when the load pressure on the queue increases.
However, executing a task can take a considerable amount of time, which is determined by the business characteristics, and if the worker performs an exception or even goes down during a task, the message is lost, which is a difficult problem for simple Message Queuing.

RABBITMQ uses a message acknowledgement mechanism to prevent such problems, in which the worker needs to return an ACK response to the MQ Server to indicate that the message has been processed;
RABBITMQ will re-post the message in the following cases:
1 Client does not respond to ACK and actively closes Channel;
2 client does not respond to ACK, network abnormal disconnection;

The message's re-sending mechanism has no timeout limit, so long as the client does not respond to the ACK, it will be delivered;
If the message persistence mechanism is enabled, the message will be further protected.

  problem description and Analysis

1 Clients for simplified answer processing, you can set auto-answer options such as:

  Boolean autoack = false;
  Channel.basicconsume (Task_queue_name, autoack, consumer);

2 If you do not enable auto-answer, you need to apply code to answer it manually:

    try {
          doWork (message);
        } finally {
          logger.info ("xxx work done");
          Channel.basicack (Envelope.getdeliverytag (), false);
        }

3 when both scenarios exist

Because of the client's coding error, the auto-answer option was enabled, and the answer code was executed in the application code:

     Enable Autoack
     Boolean autoack = true;
     Consumerchannel.basicconsume (QueueName, Autoack, this);

     //...

     Snipper from Consumer.handledelivery method
     //Send ACK to server  
     try {
            Consumerchannel.basicack ( Deliverytag, True);
        } catch (Exception e) {
     }

Once more confirmed, the application code seems to be all as usual. However, when the message is frequently tested, it is found that there is a case of random loss processing.
Check the RABBITMQ server log for the following exception:

  {amqp_error,precondition_failed, "Unknown delivery tag 1", ' Basic.ack '}
  ...
  {amqp_error,precondition_failed, "Unknown delivery tag 1", ' Basic.ack '}
  ...
  {amqp_error,precondition_failed, "Unknown delivery tag 1", ' Basic.ack '}
  ...

Tip Unknown delivery tag=1, which is a token used by MQ server for message acknowledgement, and the service side is printing incorrectly because it is unrecognized.
Another phenomenon is the continuous sending and receiving of messages 5 times, the loss of message processing 1 times, and the RABBITMQ server error log 4 times.

After analysis, we find the cause of the problem:
RABBITMQ for each channel maintained a delivery tag counter, where the positive self-increment, the new message delivery when the self-increment, when the message response to the self-reduction;
In a continuous sending and receiving scenario, because the message is sent in a shorter interval, some of the messages are discarded as consumer because of the repeated acknowledgment of the RABBITMQ as handled.

Solutions

Cancel the auto-answer mechanism of consumer, keep only the handle of manual reply, solve the problem. 

References

About the RABBITMQ message acknowledgement mechanism:
Http://www.rabbitmq.com/confirms.html#when