Implementation of Mycat Distributed transaction

Introduction: Mycat has become a powerful open source distributed database middleware product. In the face of enterprise application of massive data transaction processing, is currently the best open source solution. However, if you want to keep the data in multiple machines consistent, the general solution is to introduce "coordinator" to uniformly schedule execution of all nodes.
This article is selected from the Distributed Database architecture and enterprise practice-based on mycat middleware.

As the amount of concurrency, the volume of data is increasing and the business has been refined to no longer be broken down by business, we have to use distributed database to improve the performance of the system. In a distributed system, each node is physically relatively independent, and data operations on each node can satisfy the ACID. However, there is no way to know the execution of other node transactions between the independent nodes, if you want to keep the data in multiple machines consistent, you must ensure that the data operations on all nodes are either successful or not, the more general solution is to introduce "coordinator" to uniformly schedule execution of all nodes.

XA specification

The X/open organization (now the Open group) defines a distributed transaction processing model. The X/open DTP Model (1994) includes the application (AP), transaction manager (TM), Resource Manager (RM), Communications Explorer (CRM) four section. The transaction manager (TM) is the transaction middleware, the Resource Manager (RM) is the database, and the Communication Resource Manager (CRM) is the message middleware. Typically, a transaction within a database is treated as a local transaction, while the distributed transaction is a global transaction. Global transactions mean that in a distributed transaction processing environment, multiple databases may need to work together to accomplish a task, which is a global transaction. It is possible to update several different databases in a transaction, when a database commits to its own internal operations, not only to the success of its own operations, but also to the successful operation of other databases related to the global transaction. If either operation of any of the databases fails, all operations done by all the databases participating in the transaction must be rolled back. XA is the interface specification (i.e. interface function) between the transaction middleware defined by the X/open DTP and the database, which is used by the transaction middleware to inform the start, end, commit and rollback of the database transaction, and the XA interface function is provided by the database vendor, which derives the two-phase submission agreement and the three-phase commit agreement.

Two-phase commit

preparation phase refers to the transaction Coordinator (the transaction manager) sends a prepared message to each contributor (Resource Manager), each of which either returns a failed message directly (such as a permission validation failure) or executes the transaction locally, writes the local redo and undo logs but does not commit, The preparation phase can be further divided into the following three steps.   the
(1) Coordinator node asks all participant nodes if they can perform the commit operation (vote) and begins to wait for the response from each participant node.  
(2) The contributor node performs all transaction operations up to the start of the query and writes the undo and redo information to the log.  
(3) Each participant node responds to a query initiated by the coordinator node. If the transaction operation of the contributor node is actually successful, it returns a "consent" message and returns an "abort" message if the transaction operation of the contributor node fails to actually execute.   The
commit phase refers to sending a rollback (Rollback) message directly to each participant if the coordinator receives a failure message from the participant or a timeout, or a commit message is sent, and the participant executes the commit or rollback operation according to the Coordinator's instructions. Releases the lock resources that are used by all transactions during processing.   The disadvantages of the
two phase commit are as follows.  
(1) A synchronous blocking problem in which all participating nodes are transaction-blocking, and when participants occupy public resources, other third-party nodes have to be blocked when they access public resources.  
(2) Single point of failure, because of the importance of the Coordinator, once the coordinator fails, the participants will continue to block.  
(3) inconsistent data, in the 2nd phase of the two-phase commit, when a local network exception occurs after the coordinator sends a commit request to the participant or the coordinator fails during the sending of a commit request, only a subset of the participants receive the commit Request, while the participant commits a commit after receiving the commit request, the other part of the machine that does not receive the commit request cannot perform the transaction commit, and the entire distributed system is inconsistent with the data.  
due to defects such as synchronous blocking, single point problem, inconsistent data, and downtime caused by two-phase commits, the researchers improved on the basis of phase two submissions and presented three-phase submissions.

Three-phase commit

A three-phase commit (Three-phase commit,3pc), also known as a three-phase commit protocol (Three-phase Commitprotocol), is an improved version of two-phase commit (2PC). The three-phase submission divides the preparation phase of the two-phase submission into two, so that three-phase submissions have Cancommit, Precommit, and docommit three stages.
(1) Cancommit stage: The Cancommit phase of the three-phase commit is very much like the preparation phase of the two-phase commit, where the facilitator sends a commit request to the participant, and the participant returns a Yes response if it can commit, otherwise returns NO response.
(2) Precommit stage: The coordinator decides whether the precommit operation of the transaction can be recorded according to the reaction of the participant. According to the response, there are two possible.

• If the coordinator obtains feedback from all participants is yes response, then the transaction is executed.

• If any participant sends no response to the coordinator or waits for a timeout after the coordinator has not received a response from the participant, the transaction is interrupted.

(3) Docommit phase: This phase of the real transaction commit, can also be divided into the execution of commits, the interruption of the transaction two execution.

The procedure to execute the submission is as follows.

• After the coordinator receives the ACK response sent by the participant, it enters the submission state from the pre-commit state and sends a DOCOMMIT request to all participants.

• After the transaction submission participant receives the DOCOMMIT request, it performs a formal transaction commit and releases all the transaction resources after the transaction commits are completed.

• After the transaction commits, an ACK response is sent to the coordinator.

• After the coordinator receives an ACK response from all participants, the transaction is completed. The process of interrupting a transaction is as follows.

• The coordinator sends an abort request to all participants.

• After the participant receives the abort request, it takes advantage of the undo information recorded in the 2nd stage to perform a rollback of the transaction and releases all transaction resources after the rollback is complete.

• After the contributor completes the transaction rollback, an ACK message is sent to the coordinator.

• After the coordinator receives an ACK message from the participant's feedback, it performs a transaction interruption.

Implementation of distributed transaction in Mycat

Mycat has fully supported XA distributed strong transaction types since version 1.6, with a simple example to understand the use of XA in Mycat.
The user application side (AP) uses the following process:
(1) Set autocommit=0
A transaction cannot be automatically committed at the application level;
(2) Set Xa=on
Set XA to open in SQL;
(3) Execute SQL
Insert into Travelrecord (id,name) VALUES (1, ' N '), (6000000, ' A '), (321, ' D '), (13400000, ' C '), (n, ' E ');
(4) Commit or rollback
Commit the transaction (commit succeeded or rollback exception).
A complete flowchart.
650) this.width=650; "src=" http://download.broadview.com.cn/Original/1701ac4c16f450169022 "title=" "style=" border : 0px;vertical-align:middle;margin:auto; "/>
The implementation process for the internal implementation side of the Mycat is as follows:
(1) Set autocommit=0
Set the autocommit in Mysqlconnection to false;
(2) Set Xa=on
Open the XA transaction manager in Mycat, generate XID with the Mycatserver.getinstance (). Genxatxid (), start the XA transaction with the XA start XID command, and continue to assemble the SQL service (mycat RT data is fragmented onto different nodes), assemble the Xa end xid,xa PREPARE XID to commit the last 1pc and log the logs to Tm.log, and if there is an exception in the 1pc phase, roll back the transaction XA ROLLBACK XID directly.
(3) in the Multi-node MySQL all 2pc commit (XA commit), after the successful submission, the transaction end, if there is an exception, the transaction is resubmitted or rolled back.
The exception handling process for XA distributed transactions in Mycat is as follows:
(1) One-stage commit exception: If 1pc commits any MySQL node that cannot commit or an exception, the transaction of all nodes is rolled back, throwing an exception to the application side transaction rollback.
(2) Mycat Crash Recovery
After the mycat crashes, restart recovery based on the Tm.log transaction log, Mycat execute the transaction log to find the XA transactions that have been prepared in each node, commit or rollback.

1. Related class description

Send set xa = On via user application side; SQL turns on the functionality of the MYCAT internal XA transaction manager, and the transaction manager implements XA transaction management for the MySQL database, and the implementation code for the specific transaction management functions is as follows:

• Mysqlconnection: Database connection.

• Nonblockingsession: User connection Session.

• Multinodecoordinator: Facilitator.

• Commitnodehandler: Shard commit processing.

• Rollbacknodehandler: Shard rollback processing.

2. Code parsing

The source code for XA transaction initiation is as follows:

public class mysqlconnection extends backendaioconnection {    // Set the open transaction     private void getautocommitcommand (Stringbuilder sb, boolean  autocommit)  {        if  (autocommit)  {             sb.append ("SET autocommit=1;");         } else {             sb.append ("set autocommit=0;");         }    }    public  Void execute (Routeresultsetnode rrn, serverconnection sc,boolean autocommit)   Throws unsupportedencodingexception {        if (! Modifiedsqlexecuted && rrn.ismodifysql ())  {            modifiedSQLExecuted = true;         }        //getting the current transaction  ID         string xatxid = sc.getsession2 (). GetXaTXID ();         synanddoexecute (Xatxid, rrn, sc.getcharsetindex (),  Sc.gettxisolation (), autocommit)     }............//omit the code here, it is recommended that readers refer to  GitHub  warehouse   mycat-server  Project  mysqlconnection.java Source}

After the user application side setting is manually submitted, Mycat will join in the current connection

SET autocommit=0;

Add the statement to StringBuffer and wait for it to be committed to the database.
User connection Session Source code is as follows:

public class Nonblockingsession implements Session {...//omit the code here, it is recommended that readers refer to the Mycat-server project of the GitHub repository nonblockingsession . java source}set XA = on; statement parsing

The user application side sends the statement to Mycat, which is parsed by the SQL statement parser and referred to sethandle for processing c.getsession2 (). Setxatxenabled (True);
Call the Setxatxenable D method in nonblocksession to set up the XA switch to start and generate XID with the following code:

public void Setxatxenabled (Boolean xatxenabled) {logger.info ("XA Transaction enabled, con" + this.getsource ());    if (xatxenabled && this.xatxid = = null) {Xatxid = Genxatxid (); }}

In addition, Nonblocksession receives a commit from the user's application side, invoking the commit method to handle the logic of the transaction commit.
In the commit () method, the first is the number of check nodes, a node and multiple nodes are divided into different processing procedures, here only the next number of nodes processing method Checkdistritransaxandexecute ();
This method commits the transactions of multiple nodes.
The source code of the Facilitator is as follows:

public class Multinodecoordinator implements ResponseHandler {...//omit this code and suggest that readers refer to the multinode of the GitHub warehouse Mycat-server project Coordinator.java Source}

In Nonblocksession's Checkdistritransaxandexecute () method, the Nonblocksession session class invokes the Multinodecoordinator class dedicated to multi-node collaboration for specific processing , in the Multinodecoordinator class, the Executebatchnodecmd method joins the processing of XA 1PC commits, and the code snippet is as follows:

for  (Routeresultsetnode rrn : session.gettargetkeys ())  {         if  (Mysqlcon.getxastatus ()  == txstate.tx_started_state) {         //recovery Log         participantlogentry[started] = new         Participantlogentry (Xatxid,conn.gethost (), 0,conn.getschema (), ((mysqlconnection)  conn). GetXaStatus ());         string[] cmds = new string[]{"XA END   " + xatxid," xa prepare  " + xaTxId};         if  (logger.isdebugenabled ())  {             logger.debug ("start execute the batch cmd : " + cmds[0]  +  ";"  +cmds[1]+ "," + "Current connectIon: "+conn.gethost () +": "+conn.getport ());        }     mysqlcon.execbatchcmd (CMDS);     } ...}

mysqlconnection mysqlcon =  (mysqlconnection)  conn;switch  (Mysqlcon.getxastatus ()) {     case TxState.TX_STARTED_STATE:    if  ( Mysqlcon.batchcmdfinished ()) {        string xatxid =  Session.getxatxid ();        string cmd =  "XA  commit  " + xaTxId;        if  ( Logger.isdebugenabled ())  {             Logger.debug ("START&NBSP;EXECUTE&NBSP;THE&NBSP;CMD&NBSP;:" +cmd+ ", Current host:" +mysqlCon.getHost () + ":" + Mysqlcon.getport ());        }         //recovery log        CoordinatorLogEntry  Coordinatorlogentry =inmemoryrepository.get (XATXID);    &nbSp;    for (int i=0; i<coordinatorlogentry.participants.length;i++) {             logger.debug ("[In  Memorycoordinatorlogentry] "+coordinatorlogentry.participants[i]);             if (Coordinatorlogentry.participants[i].resourcename.equals (Conn.getSchema ())) {                  coordinatorlogentry.participants[i].txstate =txstate.tx_prepared_state;             }        }         inmemoryrepository.put (Session.getxatxid (), coordinatorlogentry);         filerepository.writecheckpoint ( Inmemoryrepository.getallcoordinatorlogentries ());         //send&Nbsp;commit        mysqlcon.setxastatus (TxState.TX_PREPARED_STATE);         mysqlcon.execcmd (CMD);    }     return, ...}

public class commitnodehandler implements responsehandler {    / /End  xa    public void commit (backendconnection conn)  {         ............//omit the code here, it is recommended that readers refer to  GitHub  warehouse  MyCAT-Server  project  commitnodehandler.java Source     }    //Submit  XA      @Override     public void okresponse (byte[] ok, backendconnection  conn)  {        ............//omit the code here, it is recommended that readers reference  GitHub  warehouse  MyCAT-Server  Project  CommitNodeHandler.java  Source} 

In Mycat also supports the single node MySQL database XA transaction processing, in the Commitnodehandler class is the single node XA two stage processing, the processing mode and Multinodecoordinator similar, through the Commit method 1p C commits, and the 2pc phase of the transaction is committed through the Okresponse method.
The source code for the rollback processing of the Shard transaction is as follows:

public class Rollbacknodehandler extends Multinodehandler {....//Omit the code here, it is recommended that readers refer to the Mycat-server project of the GitHub repository Rollbacknod Ehandler.java Source}

The rollback processing of XA transactions is added to the rollback method of Rollbacknodehandler, and the rollback initiated by the user application side is processed in this method.

for  (Final routeresultsetnode node : session.gettargetkeys ())  {     ......    //support the XA rollback     mysqlconnection mysqlcon =  (mysqlconnection)  conn;    if ( Session.getxatxid ()!=null)  {        string xatxid =  session.getxatxid ();         mysqlcon.execcmd ("XA END "  + xaTxId +  ";");         mysqlcon.execcmd ("xa rollback "  + xaTxId  +  ";");     }else {    conn.rollback ();     } ...} 

Similarly, this method initiates an XA rollback instruction for all MySQL database nodes.
This article is selected from the Distributed Database architecture and enterprise practice-based on MYCAT middleware, this link can be viewed at the blog point of view website.
650) this.width=650; "Src=" http://img.blog.csdn.net/20170111135317066?watermark/2/text/ ahr0cdovl2jsb2cuy3nkbi5uzxqvynjvywr2awv3mjawng==/font/5a6l5l2t/fontsize/400/fill/i0jbqkfcma==/dissolve/70/ Gravity/southeast "alt=" Picture description "title=" "style=" Border:0px;vertical-align:middle;margin:auto; "/>
Want to get more good articles in time, you can search "blog point of View" or scan the QR code below and follow.
650) this.width=650; "src=" http://img.blog.csdn.net/20161128135240324 "alt=" Picture description "title=" "style=" border:0px; Vertical-align:middle;margin:auto; "/>

This article is from the blog of "Blog View blog", make sure to keep this source http://bvbroadview.blog.51cto.com/3227029/1891061

Implementation of Mycat Distributed transaction