Principle of redis master-slave Replication

1. Preface

Similar to the reason for MySQL master-slave replication, although redis reads and writes fast, it also produces a particularly high read pressure. In order to share the read pressure, redis supports master-slave replication. redis master-slave replication can be divided into full synchronization and incremental synchronization based on whether the data is full or not.

2. Implementation of the old version copy Function

Redis replication supports synchronization and command propagation:

(1) The synchronization operation updates the status of the slave database to be consistent with that of the master database.

(2) The command propagation operation is used to make the slave server return to the same status when the master server status is changed.

2.1 Synchronization

Synchronization is completed by Sync:

(1) The slave server sends the sync command to the master server.

(2) The master server that receives the sync command executes the bgsave command to generate the RDB file and uses a buffer cache to generate all the write commands during the RDB.

(3) When RDB is generated, RDB is sent to the slave server and loaded from the server to RDB. The status is the same as that of the master server.

(4) The master server then sends all the write commands in the buffer zone to the slave server, and the status is consistent.

2.2 command Propagation

After the synchronization operation is complete, the master and slave statuses are the same. However, when the client executes a command to the master server, the master server changes and the slave server status is different, the master server sends a write command to the slave server so that the slave server returns to the same status as the master server.

2.3 disadvantages of the old version replication function

Before redis2.8, master-slave Server replication can be divided into two types:

(1) initial replication: The slave server has never copied other master servers before, or the slave server has changed the master server.

(2) Post-disconnection replication: If the Master/Slave server is in the Command propagation stage, the Master/Slave server is disconnected and re-connected, the slave server continues to replicate the master server.

Why is efficiency low? In the old version of replication, the replication after disconnection and reconnection is implemented through the sync command. Previously, the sync command is implemented by the master server to regenerate the RDB file, sending data to the slave server is inefficient at generating RDB again, and the master server is blocked when generating RDB. Therefore, old version replication is inefficient.

3. Implement the New copy Function

To solve the inefficiency of the old version of replication, the new version of replication uses the psync command to replace the sync command. The psync command has full and partial re-synchronization:

(1) completely Re-Sync: Same as sync above

(2) partial reconnection: This function is used to handle reconnection after disconnection. After reconnection, only write commands During disconnection are executed instead of full synchronization.

Part 1 re-synchronization implementation

Partial synchronization consists of the following three parts:

(1) The replication offset of the master server and the replication offset of the slave server

(2) Master Server replication backlog Buffer

(3) server running ID

3.1.1 copy offset

Both the master and slave servers maintain a replication offset:

• Each time the master server sends a n-byte command to the slave server, it adds n At its own copy offset.
• Each time the server receives n bytes of data, it adds n At its own copy offset.

For example, if the current offset of the master-slave server is 100 and the master-slave server sends a 33-byte command to the slave server, the replication offset of the master-slave server is 133 at this time, so you can determine whether the current status of the master-slave server is consistent, you can determine by the copy offset:If the replication offset is the same, the status is consistent. Otherwise, the status is inconsistent.. If the slave server fails to receive the command while the master server keeps sending the command, the offset must be different, so the status is inconsistent, so if we re-transmit the commands missed in the middle to the slave server, this is related to the backlog buffer.

3.1.2 copy the backlog Buffer

The replication backlog buffer is a fixed-length first-in-first-out queue maintained by the master server. When the master server executes command propagation, it not only transmits the command to the slave server, but also writes the command to the buffer zone:

 

As shown in, this is the status quo of a backlog of buffers. After the slave server is reconnected, A psycn command is sent to the master server, and the current replication offset (offset) of the slave server is taken over, the master server determines whether full synchronization or partial synchronization is performed based on the offset of the slave server:

(1) If the offset from the slave server is 10086, the offset to be copied should start from 10087. If the buffer exists, all data starting from 10087 will be sent to the slave server, therefore, partial re-synchronization is performed.

(2) Conversely, if it does not exist, full synchronization is required!

For example, if you perform partial re-synchronization, send a continue reply to the slave server.

3.1.3 server running ID

In addition to the replication offset and the replication backlog buffer, you also need to use the server running ID (run ID) to implement partial re-synchronization ):

• Every redis server,Whether the master server or slave server has its own running ID;

When the slave server replicates the master server for the first time, the master server will send its own running ID to the slave server, and the slave server will save the running ID (Note: YesThe slave server saves the ID of the master server.).

When the slave server is disconnected and re-connected to the master server, the slave server sends the previously saved running ID to the master server currently connected:

• If the running ID saved on the slave server is the same as the running ID of the currently connected master server, it indicates that the master server that is currently connected is copied before the slave server is disconnected, the master server can continue to perform some re-synchronization operations;
• Conversely, if the running ID saved by the slave server is different from the running ID of the currently connected master server, it indicates that the master server copied before the slave server is disconnected is not the master server currently connected, and the master server will perform the full re-synchronization operation on the slave server.

4. Heartbeat Mechanism

During command propagation, the slave server sends the command "replconf ack <replication_offset>" to the master server every second, each time this command is sent, the slave server will report its own copy offset to the master server. If it exceeds one second, it indicates a problem with the master-slave connection. In this case, the set key value sent by the master server to the slave server is lost. It doesn't matter. The master server will know immediately, but it should be noted that the offset at this time is inconsistent, not caused by disconnection, but because the commands sent from the master server to the slave server are lost, the connection is normal, which is called the loss of commands.

 

 

Principle of redis master-slave Replication