Mysql semi-sync source code implementation, mysqlsemi-sync

Mysql semi-sync source code implementation, mysqlsemi-sync

Mysql replication briefly introduces the causes of mysql semi-sync, and explains how semi-sync ensures that no data is lost. This article focuses on the implementation of semi-sync, and presents the implementation process of semi-sync with the source code. For the latest semi-sync source code, refer to the official version 5.7 implementation, https://github.com/mysql/mysql-server.

Correct posture for enabling semi-sync
By default, mysql replication is asynchronous. mysql uses parameters to control the semi-sync switch. Specifically, the master database is controlled by the rpl_semi_sync_master_enabled parameter, and the slave database is controlled by the rpl_semi_sync_slave_enabled parameter. After these two parameters are enabled, the mysql semi-sync feature is enabled. Note: To ensure that the semi-sync takes effect immediately for the slave database, restart the slave IO thread. In addition, another important parameter is rpl_semi_sync_master_timeout, which is used to control the time when the master waits for semi-slave ack packets, in milliseconds. The default value is 10000. If the master instance waits for timeout, it is switched to normal asynchronous replication.

master:set global rpl_semi_sync_master_enabled=1;set global rpl_semi_sync_master_timeout=xxx;slave:stop slave io_thread;set global rpl_semi_sync_slave_enabled=1;start slave io_thread;

Note that the preceding two parameters only indicate that the master-slave has the basic conditions for enabling semi-sync, but whether the replication runs in semi-sync Mode, you must also determine the status value of Rpl_semi_sync_master_status. For example, because slave has a great delay (beyond rpl_semi_sync_master_timeout) compared with the master, the replication is switched to normal replication. For children's shoes that require code debugging, rpl_semi_sync_master_trace_level and rpl_semi_sync_slave_trace_level are very important. By setting the level value, you can print the detailed process of log recording semi-sync to facilitate problem locating.

Semi-sync implementation
In the end, semi-sync is also a type of replication. It is only implemented by adding steps on the basis of normal replication. Therefore, semi-sync does not change the basic replication framework. Our discussion also covers the master and slave aspects.
1. master (master database)

The master database consists of three parts: (1 ). binlog dump thread that is responsible for interfacing with the slave-io thread and sends the binlog to slave (2 ). the working thread for writing transactions on the master database, (3 ). the ack_cycler thread that receives the semi-slave message.

(1) binlog dump Process
The main execution logic is in the mysql_binlog_send function.

1. Determine whether the slave is semi_slave. Call add_slave to add semi-slave to the listener queue of the ack_receiver thread. The logic is whether the rpl_semi_sync_slave parameter is set for the slave session.

2. Read the binlog from the specified position point based on the slave request offset and binlog File
3. Collect data in the binlog File Based on the file and Position
4. Call updateSyncHeader to set the semi-sync mark of the data packet header
Determines whether to set based on the real-time semi-sync running status (this status is set by the ack_receiver thread based on whether the slave-ack message is received in time)
5. Call my_net_write to send the binlog
6. Call net_flush to confirm that the Network Package is sent out.
If all generated binlogs have been processed, call wait_for_update_bin_log to wait for binlog updates.

(2). semi-synchronous transaction commit Process
Mysql5.6 and later are submitted in group mode. there is a queue in each stage. After semi-sync is added, another semi-sync stage is added.
1. flush stage:
The queue ensures that the order of writing binlogs is the same as that of innodb-commit. Through the queue, you can ensure that the binlog-cache of each transaction is written sequentially, and then only one write operation (flush_cache_to_file) is performed ). After the flush phase, if sync_binlog is not 1, a new binlog is generated to the master. If sync_binlog is 1, a new binlog is generated to the dump thread after the sync phase is completed. Here I understand that if it is not set to 1, there may be no subsequent sync phase, and the operating system cache also has binlog data, so we can notify you after the flush phase; when sync_binlog is 1, The binlog of the master database can be implemented first, which is always more than the slave database. However, if sync_binlog is not set to 1, for example, 1000, the slave database may have more binlogs than the master database in case of host exceptions. Check whether to skip the sync phase based on the sync_binlog settings.
2. sync phase:
Sync_binlog_file
3. semi_sync stage:
Call_after_sync: Wait for the standby database to respond. Call the cond_timewait wait condition variable & COND_binlog_send _
4. commit stage:
Innodb-commit
WaitAfterCommit: Wait for the standby database to respond. Call the cond_timewait wait condition variable & COND_binlog_send _. Finally, based on the settings of semi-sync replication mode (AFTER_COMMIT, AFTER_SYNC), we can determine whether it is step (3) or step (4) waiting.

(3) process for receiving slave-ack packets
The work of this process is mainly in the ack_receiver thread. The main function of this thread is to listen to the ack package of semi-slave and check whether the master-slave link is working in the semi-synchronous state, switch between the common replication and semi-synchronous replication based on the actual running status. After the rpl_semi_sync_master_enabled parameter of the master database is enabled, the thread starts and closes the parameter, and the thread disappears.
The process is as follows:
1. traverse the semi-slave Array
2. Use the select function to monitor whether each slave has a network package.
3. Call my_net_read to read package data
4. Call reportReplyPacket to process the semi-sync replication status
If the standby database has obtained the latest binlog point, it will wake up the waiting working thread.
5. Call reportReplyBinlog to wake up the waiting thread, mysql_cond_broadcast (& COND_binlog_send _);

2. slave (slave database)

Mainly implemented in (handle_slave_io)
1. After I/O-thread is started, call safe_connect to establish a connection with the master.
2. Call the request_dump function to process the binlog logic of the request.
(1) execute the command SET @ rpl_semi_sync_slave = 1 and SET a local variable. Use this parameter to mark slave as semi-slave.
(2). Send the command COM_BINLOG_DUMP request log
Collect logs cyclically from the master to process logs.
{
1. Call read_event to collect logs from the master (if there is no network package, it will block waiting)
2. Call slaveReadSyncHeader to check whether the network header has the semi-sync mark.
3. Call queue_event to write logs to relay-log. In this process, logs of server-id are filtered out.
4. If there is a semi-sync mark, call the slaveReply function to send ack packets.
}