The solution of distributed transaction based on database transaction message

Reprint Please specify source: http://www.cnblogs.com/lizo/p/8516502.html

Overview

When libraries are no longer supporting the current business, we tend to consider sub-libraries (split horizontally or vertically). However, there is an unavoidable problem in the library, that is, a matter of affairs. There are many distributed transaction solutions on the Internet, such as XA,TCC, but the most common, but also the lowest cost of transformation is the use of final consistency to ensure distributed transactions.
It is common to use message middleware (RABBITMQ,ROCKETMQ) to resolve eventual consistency through transactional messages. Refer to Https://zhuanlan.zhihu.com/p/25933039?utm_source=tuicool&utm_medium=referral.

This article will use the database to achieve the final consistency of the implementation scenario.

noun explanation

• Main Library-database for business access before splitting
• Sub-Library-after splitting, part of the business data is put into the library

Note: Some of the following things should be considered when using transactional messages, whether database-based or Message Queuing.

Database-based transaction message transaction messages

The so-called database-based transaction message, in fact, is very well understood, is to create a message queue-like table in the database, to hold transaction messages. Before a split, there are multiple primary library data operations in a transaction. Such as

However, after splitting the database, a business is split into the sub-Library, so that the original single-Library transaction is broken, but by splitting the business using a transaction message instead (the Transaction message table is also in the main library, so this is a library transaction), The subsequent execution of the business logic of the transaction message by other means, so that the final consistency can be achieved, such as

Transaction Message Executor

As mentioned earlier, a transaction message requires a processor to perform the business logic corresponding to the transaction message. The transaction handler should be read and executed sequentially.

Imagine a scenario where a message processing failure occurs, and if the executor waits for the execution of the current message to continue executing (even if the message never succeeds), then the execution of the subsequent message is affected, causing a problem for the whole system.

Therefore, the message processor is to ensure that message processing as fast as possible, but also to ensure that the end of the message can be successfully executed. There are 2 tasks that must be set up in a running water executor:

• The first task, the Message Processing task, has the fastest execution of the message, and if the message processing fails, the message is skipped to continue execution of the subsequent message.
• The second task, the Message Validation task, is to check the message sequentially to ensure that all the running water is successful, and if it fails, retries, and sends an alarm after multiple retries fail to allow manual intervention. Such as

Characteristics of message execution

• Latency processing. Messages are not processed in real time, but are executed asynchronously using a message executor. Therefore, in the original logic, it is important to pay special attention to whether the subsequent process has a real-time dependency on the result of the message processing (for example, the message processing results are used in subsequent business logic to do some calculations, etc.).
• Deal with disorder. Since the messages are not necessarily sequential, all guarantees that even the generated streams will be executed before the problem can occur.
• Ultimate success. For each inserted message, ensure that the message must be executed successfully

How to confirm that the message was executed successfully

Imagine, if the sub-Library business execution succeeds (update the library), and then go to update the message state (main library), so, another thing is a depot transaction, so, to think of other ways to avoid, the simplest way is to build a message table in the library, to save the processing of successful messages. This way, by comparing the main library and the library's message table, you know which transaction messages are not successfully executed.

Message Processor Basic framework

As described earlier, the core functions of the message processor are:

• Get the message and send the message to the business put processing
• Ensure success of message execution

In order to complete the above functions, a message Processing task and a message validation task are required to trigger these 2 tasks by a scheduled task (for example, 5s triggers once)

Message processing tasks

The message Processing task is to notify the business system to execute by scanning the messages to be processed.

Again, the message Processing task does not control whether the message executes successfully. are executed in the order of message queue tables.

Message Validation Task

The verification task is to compare the message records in the main library and the library (all the streams recorded in the main library, the successful execution of the records in the library), retry the unsuccessful messages, and if the multiple retries fail, an alarm is required and manual intervention is needed.

Compare the same points with transactional messages based on message middleware

• is to ensure eventual consistency with asynchrony:
• You can control the rate at which messages are executed asynchronously and can take advantage of the load balancing of RPC calls
• Message processing must support retry and idempotent
• Transaction message asynchronous execution failed, no way to roll back the transaction that generated the transaction message

Submission of different point message transactions

Using message middleware, it is common to write code that submits the middleware transaction messages in code, similar to the following

 Public Boolean transaction (String text) {    try  {        send transaction message        Execute local transaction        COMMIT TRANSACTION message         return  true;     Catch (tmcexception e) {         returnfalse;    }}

However, in a real-world project, the issue of the Propagation of transactions (spring's transaction annotations are to support transactional propagation) requires modification of the business code. But using database-based Message Queuing does not have this problem

@Transactional  Public void Publishas (String text) {       performs local transaction logic    INSERT Transaction Message}

So in the existing code transformation (especially in complex systems), the use of database transaction messages can reduce code changes

No callback check required

We know that when using the message middleware, we need to implement a callback interface, when the transaction message is not a commit for a long time, it will call the interface to confirm whether a commit is required (for example, sending a message succeeds, but the network is not available at commit). and the data Bureau-based transactional message queue doesn't have this problem

More Database access resources

Database-based transaction messages also have a significant disadvantage:

• Consume more database space and database access resources
• Need to write extra DAO layer code

Summary

The basic idea of database-based and message-queuing-based transactional messages is the same, using eventual consistency to avoid the additional system complexity and code overhead associated with distributed transactions. Database-based transaction messages in the existing business transformation, the code changes less, and does not require the introduction of additional message middleware, but the problem is to more access to the database. The problem of message-based middleware is how to avoid how to deal with the problem of communication with message middleware. Of course, the above is just my personal understanding, if the system has any unreasonable design or improvement of the place, welcome to discuss.

The solution of distributed transaction based on database transaction message