Redis Master/Slave Synchronization

Slave status change # defineREDIS_REPL_NONE0 * Noactivereplication * # defineREDIS_REPL_CONNECT1 * Mustconnecttomaster * # defineREDIS_REPL_CONNECTING2 * Connectingtomaster * # defineREDIS_REPL_RECEIVE_PONG3 * WaitforPINGreply

Slave status change # define REDIS_REPL_NONE 0/* No active replication */# define REDIS_REPL_CONNECT 1/* Must connect to master */# define REDIS_REPL_CONNECTING 2/* Connecting to master */# define slave 3/* Wait for PING reply

Slave status change

#define REDIS_REPL_NONE 0 /* No active replication */#define REDIS_REPL_CONNECT 1 /* Must connect to master */#define REDIS_REPL_CONNECTING 2 /* Connecting to master */#define REDIS_REPL_RECEIVE_PONG 3 /* Wait for PING reply */#define REDIS_REPL_TRANSFER 4 /* Receiving .rdb from master */#define REDIS_REPL_CONNECTED 5 /* Connected to master */

Server. repl_state stores the current replication status of the slave.

* Redis is initialized as REDIS_REPL_NONE at startup.

* After the slaveof command is received, the redis is converted to slave and the status changes to REDIS_REPL_CONNECT.

The periodically executed replicationCron checks the current status of the slave to perform different operations.

• If the slave is in REDIS_REPL_CONNECT, call connectWithMaster to establish a connection to the master, and the status is changed to REDIS_REPL_CONNECTING.
• After the connection is established, send PING to the master and the status is converted to REDIS_REPL_RECEIVE_PONG.
• After PONG is received, if the master is configured to require authentication, slave sends the AUTH message. After AUTH is passed, slave sends the REPLCONF listening-port message to the master to notify the listening port; finally, the SYNC command is sent to the master to request the rdb file to be synchronized, And the status is converted to REDIS_REPL_TRANSFER.
• After the master sends rdb to slave, slave starts to read the rdb file (the master will first inform the total length of the rdb file before sending rdb). After slave accepts the rdb file, emptyDb clears the database and calls rdbLoad to load the rdb file received from the master. After the load is complete, the status changes to REDIS_REPL_CONNECTED. Then, slave will continue to accept new operations synchronized from the master, to ensure the consistency of the master and slave databases.

Master Status Change

#define REDIS_REPL_WAIT_BGSAVE_START 6 /* We need to produce a new RDB file. */#define REDIS_REPL_WAIT_BGSAVE_END 7 /* Waiting RDB file creation to finish. */#define REDIS_REPL_SEND_BULK 8 /* Sending RDB file to slave. */#define REDIS_REPL_ONLINE 9 /* RDB file transmitted, sending just updates. */

All slave lists are maintained in server. slave. replstate records the current synchronization status of each slave (from the master's perspective ).

• When the master receives the sync/psync command from the slave, the status of the slave is converted to REDIS_REPL_WAIT_BGSAVE_START
• The master starts rdbSaveBackground for the slave background and converts the slave status to REDIS_REPL_WAIT_BGSAVE_END (if the master has received sync, the master is dumping the rdb file for a slave, then the new slave can directly enter the REDIS_REPL_WAIT_BGSAVE_END state)
• After rdbSaveBackground is complete, the slave state is converted to REDIS_REPL_SEND_BULK, and the master starts to send the rdb file to slave. before sending the file, the master sends the length information of the rdb file.
• After the rdb file is sent, the slave status is changed to REDIS_REPL_ONLINE.

After the master starts rdbSaveBackground for the slave, updates on the master will accumulate in the slave connection buffer. After the slave accepts the rdb file, it will synchronize the accumulated updates in the buffer to the slave.

Master-slave synchronization Sequence

1. The Administrator sends the slaveof master-ip master-port to the server and changes the server to the slave of the master.
2. Slave establishes a network connection to the master
3. Slave sends PING to the master to check the network status. The master replies + PONG
4. Slave sends AUTH authentication to the master (optional), master Recovery + OK
5. Slave sends REPLCONF listening-port to master, master Recovery + OK
6. Slave sends sync/psync to the master, and the master starts to dump rdb files in the background.
7. After the master dump rdb is complete, send the rdb file to slave. slave accepts the rdb file and clears the database. load the rdb file received from the master.
8. The master node synchronizes the update operations accumulated during rdb dump and transmission to the slave.
9. The master and slave data are consistent, and all the update operations received by the master will be synchronized to the slave.

Master-slave synchronization status maintenance

The master periodically ping the slave (10 s by default). When the ping fails and the slave exceeds 60 s (default), the Server Load balancer instance is considered to have crashed and the Server Load balancer instance is disconnected; similarly, if the server Load balancer fails to receive the PING from the master after a certain period of time, the Server Load balancer considers the master to have crashed and disconnects the master.

When the master and slave connections are disconnected, slave needs to re-request to the master to synchronize the rdb file. redis 2.8 supports incremental synchronization to a certain extent. When the master node synchronizes the synchronized data to slave, it will be stored in a circular buffer (1 MB by default), which will always store the latest synchronization data. In addition, the master and slave will record the offset of the current synchronization.

When slave is disconnected from the master, slave will first try PSYNC incremental synchronization. If the master has not been restarted during the disconnection, And the slave offset is in the master's annular buffer, you can perform incremental synchronization directly, for example:

Assume that the circular buffer length is 1000, and the synchronization offset of the master is 10000. The synchronization data between 9500 and is recorded in the buffer. When the slave is synchronized to, the master is disconnected; after the slave re-establishes a connection with the master, it will carry the offset 9500, requesting incremental synchronization. The master finds that the slave offset is between-and can be incrementally synchronized, send the synchronized data from 9501-10000 to slave to ensure the consistency between the master and backup.

The post Redis Master/Slave synchronization appeared first on Yun Notes.