Using MQ to ensure the eventual consistency of distributed transactions

Objective

Before we discussed how to split an order with a single service (https://www.cnblogs.com/linkstar/p/9610268.html)
From monomer to micro-service split, at that time we just to the original whole service to do a simple split, but in practice certainly will encounter a lot of problems, so we here to solve one of the most easily and most likely in the actual problems encountered, business.

In the monolithic architecture, it is easy to maintain a transaction, we want to roll back a transaction operation is also very easy, and after separating into microservices, we want to maintain a transaction on multiple services is more difficult. Here we no longer discuss the implementation of step-by-turn transactions, but instead discuss a final consistency that we often hear.

To put it simply, many of the applications in practice, in most cases, want to ensure that as long as the final result is the right, the middle of the process may be many setbacks are not related.

We continue to take the example of single, want to ensure that as long as the last money buckle, order production, is right. Suppose one of the services hang up, it may be through constant retry, and finally, after the normal service, the result is right, also satisfied. In this case, we call it "final consistency". (It's not too much to explain here)

Reasons for using MQ

So why use MQ for eventual consistency? I think a lot of people also know that there are many people on the web saying that MQ achieves eventual consistency, but never think carefully about why MQ is used. I have listed these reasons for your reference:
1. MQ can solve a series of problems caused by HTTP request exception
2, MQ can have multiple consumers, fully fit micro-services
3, MQ essence is the queue, under high concurrency, you can ensure that your data is normal
...

Logical flowchart

?

The logic of the producer
1. Order Warehousing
2, the message record storage
3. Send message (using confirmation mode)
4, MQ received a message to the production end of a confirmation message
5, the production side monitoring this confirmation message
6. Manipulate the status of the message table according to the listening results
7, timed task timed to operate the message status is 1 unsent records, that is, those who did not hear the results of the record to resend

The logic of the consumer
1. Incoming messages will be received
2, processing message failed message record status is not processed
3. Processing message successfully modified message record status for processing success
4. Messages that receive the same message ID are discarded directly
5, scheduled tasks to operate those that have not been processed, and has been over a period of time the message
6, for those who have been dealing with the failure, and for a long time have no way to deal with the success of the news by manual or other means to deal with

Issues to be aware of

1, 100% delivery of the message
First of all we need to ensure that the message must be delivered, if we can not guarantee the message delivery success, then the logic will not go down.

2, we need to ensure the idempotent (no matter how many operations, the result of the final results and only one result is the same. ）
The message will certainly appear the problem of repeated delivery, which should be guaranteed by the consumer, repeated messages can not be consumed

The way to ensure these two problems is hidden in the logical flowchart above, please carefully try to figure out the intention.

Code implementation

！！！ Note is in the Distributed-transaction branch!!!
Https://github.com/LinkinStars/MicroServiceExample

Reference Blog

1190000012762869

Specific other details here is very clear, here no longer repeat, for distributed transactions, has always been a problem, the current solution has many, adapt to different application scenarios, so need to be based on the actual project use scene to select.

Using MQ to ensure the eventual consistency of distributed transactions