Talking about distributed transaction

Preface Application Scenario

Transactions must meet the characteristics of traditional transactions, namely atomicity, consistency, separation, and persistence. But in the distributed transaction process,

Some venues such as in the e-commerce system, when a user orders, in addition to inserting a record in the order form, the corresponding commodity table of this product quantity must be reduced by 1, how to guarantee?

In the search ad system, when a user clicks on an ad, in addition to adding a record in the Click event Table,
You have to go to the Merchant account list to find this merchant and deduct advertising fees, how to guarantee?

A local transaction
Take user A to transfer User B as an example, assuming there is

User A Account form: a (Id,userid,amount)

User B Account table: B (Id,userid,amount)

User's userid=1;

The action of transferring 10,000 dollars from User A to User B is divided into two steps:

1) User A table deduction 10,000: Update a set amount=amount-10000 where userid=1;

2) User B table added 10,000: Update b set amount=amount+10000 where userid=1;

How do you make sure that user a User B balances? Some people say this is very simple, can be resolved with a transaction.

Begin transactionupdate A set amount=amount-10000 where userid=1; update B set amount=amount+10000 where Userid=1; End Transactioncommit;

Very correct! If you use spring, an annotation will take care of the above transaction function.

@Transactional (rollbackfor=exception.class) public void Update () {updateatable ();//Update a table updatebtable ();//update B table}

If the system is small and the data tables are on a DB instance, the local transaction method described above can work well, but if the system is large in size,
For example, the User a account table and User B account tables are obviously not on the same db instance, they tend to be distributed across different physical nodes, where local transactions are lost.

Since local transactions fail, distributed transactions naturally mount on the stage.
Two XA

XA is a specification of distributed transactions presented by the X/open organization. The XA specification primarily defines the interface between the (global) transaction manager (Transaction Manager) and (local) resource Manager (Resource Manager).
The XA interface is a two-way system interface that forms a communication bridge between the transaction manager (Transaction Manager) and one or more resource managers (Resource Manager).
The reason why XA needs to introduce a transaction manager is that in a distributed system, theoretically (referring to a paper such as Fischer), two machines are theoretically unable to achieve a consistent state, requiring a single point of coordination.
The transaction manager controls the global transaction, manages the transaction life cycle, and coordinates the resources. The resource manager is responsible for controlling and managing actual resources, such as databases or JMS queues.
Describes the relationship between the transaction manager, the resource manager, and the application:

32 Phase Commit Agreement

Distributed transactions must satisfy the characteristics of traditional transactions, that is, atomicity, consistency, separation and persistence. However, some sites (servers) may fail during distributed transaction processing,
Or you cannot access certain sites because of a network failure. In order to prevent the partial failure of the distributed system, the inconsistency of data is generated.
A two-phase commit protocol (two-phase commit Protocol) is used in the control of distributed transactions. The commit of a transaction is divided into two phases:

Pre-submission phase (Pre-commit Phase)
Post-decision stage (Post-decision Phase)

Two-phase commits are used to coordinate the activities of multiple servers participating in an update to prevent the inconsistency of the data when a partial failure of the distributed system occurs. For example, if an update operation requires that a record at three different nodes be changed, and if one node fails, the other two nodes must detect the failure and cancel the changes they have made.

To support two-phase commits, the servers involved in a distributed update transaction must be able to communicate with each other. In general, a server is designated as a "control" or "commit" server and monitors information from other servers.

During a distributed update, each server first flags the part of the distributed transaction that they have completed (but not committed) assigned to them, and is ready to commit (so that their updated parts become permanent). This is the first phase of a two-phase commit. If one node fails to respond, the control server instructs the other nodes to undo the effects of the various parts of the distributed transaction. If all nodes are answered ready for submission, the control server instructs them to commit and wait for their response. The wait for confirmation phase is the second stage.
After receiving the instructions that can be submitted, each server submits its own portion of the distributed transaction and sends back the submission completion information to the control server.

In a distributed transaction, there must be a site server as the Coordinator (Coordinator), which can make requests to servers in other venues and respond to their responses, which control the submission or revocation of a distributed transaction. The server for other sites involved in the distributed transaction is called a contributor (Participant).

The process for transaction two phase commit is as follows:
Two-phase commits are initiated when the application issues a commit command to the coordinator. The submission enters the first stage, i.e. the pre-submission phase. In this phase:
(1) The facilitator prepares a local (that is, locally) to submit and write a "pre-commit" log entry in the log, and contains the names of all participants with the transaction.
(2) The facilitator asks the participant if the transaction can be submitted. A participant may not be able to submit for several reasons. For example, the server provides a constraint (Constraints) when the delay check does not meet the constraints, cannot be committed, the participant's own server process or hardware fails, cannot be committed, or the coordinator does not access a participant (network failure), when the coordinator is considered to have received a A negative answer.
(3) If the participant is able to submit, write the "Ready to submit" log entry in its own log, which is immediately written to the hard disk and then sent back to the coordinator, ready to submit the answer.
(4) The coordinator waits for all participants to answer, and if a participant sends back a negative answer, the coordinator revokes the transaction and sends a "undo" message to all participants, ending the distributed transaction and undoing all the effects of the transaction.

If all participants return a message that is ready to commit, the commit of the transaction goes to the second stage, the post-decision submission phase. In this phase:
(1) The coordinator writes the "commit" log entry in the log and writes it to the hard disk immediately.
(2) The facilitator issues a "submit this transaction" order to the participant. After each participant receives the command, it writes the "commit" log entry in its own log and writes it to the hard disk immediately. It then sends back the "committed" message, releasing the resource that the transaction occupies.
(3) When all participants return the "committed" message, the coordinator writes the "transaction commit complete" log entry in the log to release the resource that the coordinator occupies. This completes the commit of the distributed transaction.

It's not that difficult to implement distributed transactions based on two-phase commits today, and if you use Java, you can use open source software Atomikos to quickly implement it.

Disadvantages

However, anyone who has used these two-stage submissions can find that the performance is too poor to be suitable for high-concurrency systems at all. Why?

1) Two-phase commit involves multiple nodes of network communication, communication time is too long!

2) transaction time is longer than the time of the lock, and the resource waiting time is also increased.

Four use Message Queuing to avoid distributed transactions
If we look closely at life, many scenes of life have given us hints.
For example, in Beijing is famous Yao Kee soy point soy and pay the money, they will not direct your point of soy to you, often is to give you a small ticket, and then let you take the small ticket to the shipping area line to take.
Why do they have to separate the two movements of the payment and the pickup? For many reasons, one of the important reasons is to enhance their reception capacity (concurrency is higher).

Return to our question, as long as this small ticket in, you finally can get soy. The same is true of the transfer service, when user a account is deducted 10,000,
We just generate a voucher (message), the voucher (message) that says "Let User B account increase 10,000", as long as this voucher (message) can be reliably saved,
We can finally take this voucher (message) to increase the User B account by 10,000, that is, we can rely on this credential (message) to complete the final consistency.

4.1 How to reliably save credentials (messages)

There are two ways of doing this:

4.1.1 the way the business is coupled with the message

User A while completing the deduction, simultaneously records the message data, this message data and the business data is kept in the same database instance (the Message Record table table name is the MSG);

Begin transactionupdate A set amount=amount-10000 where userid=1; INSERT into message (USERID, amount,status ) VALUES (1, 10000, 1); End Transactioncommit;

The above-mentioned transaction can guarantee that the message will be saved as long as the money is deducted from User A's account.

When the above transaction is successfully submitted, we notify User B through the real-time message service that User B has successfully sent a reply success message after the successful processing, and user A receives the reply and deletes the message data.

4.1.2 Service and Message decoupling method

The way in which messages are saved makes the message data and business data tightly coupled, architecturally elegant, and prone to other problems. To understand decoupling, you can use the following methods.

1) User A before the deduction transaction submission, to the real-time message service request to send a message, the real-time message service only records the message data, not really sent, only the message sent after the transaction is not committed;

2) When user A's debit transaction is successfully submitted, it is sent to the real-time message service. The real-time message service sends the message only after it has been confirmed to send the command.

3) Cancel the send to the real-time messaging service when the user a debit transaction commits a failed rollback. The message will not be sent after the cancellation command is received;

4) For those unacknowledged messages or canceled messages, a message status confirmation is required that the system periodically queries the user a system for the status of the message and updates it. Why this step is needed
For example, suppose that after the 2nd step, when the user a debit transaction was successfully submitted, the system hangs and the message status is not updated to "confirm send", causing the message to not be sent.

Advantages: The independent storage of message data reduces the coupling between the business system and the message system;

Cons: Two requests are required for one message to be sent, and the business processing service needs to implement the message state callback interface.

4.2 How to resolve a message repeat delivery problem

Another serious problem is the repeated delivery of the message, with our User a transfer to User B, for example, if the same message is repeated two times, then our User B account will be increased by 20,000 instead of 10,000.

Why is the same message being repeated? For example, when User B finishes processing the message MSG, sends a successful message to user A, under normal circumstances, user A should delete msg, but if User A is hung up by this tragedy,
After restarting the message MSG will continue to send Msg.

The workaround is simple, add the Message Application state table (message_apply) to User B, which is a ledger for recording the consumption of messages, one message at a time.
Before actually executing, go to the Message Application state table to query again, if the description is found to duplicate the message, discard it, if not found to execute, and insert into the Message Application state table (the same transaction).

For each msg in Queuebegin transactionselect count (*) as CNT from message_apply where msg_id=msg.msg_id;if cnt==0 THENUPDA Te B set amount=amount+10000 where Userid=1;insert into message_apply (msg_id) values (msg.msg_id); End Transactioncommit;

　　

Talking about distributed transaction