Redis master-slave Replication

1. In master-slave replication, databases are divided into two types: master database and slave database for data synchronization ). The master database can perform read/write operations. When data changes due to write operations, the data is automatically synchronized to the slave database. The slave database is generally read-only (which can be written in specific cases, specified by the slave-read-only parameter) and accepts data from the master database. A master database can have multiple slave databases, A slave database can only have one master database. Ii. configuration method 2.1 Client commands

Run the slave masterip masterport command on the slave server;

2.2 configuration file

Configure slaveof <masterip> <masterport> in the configuration file. In addition to IP and port information, you can configure other Master/Slave information;

Iii. Implementation of the old version replication function (earlier than redis2.8)

After the slave sends the slaveof command to the master, the slave first performs the synchronization operation and then transmits the command;

Redis's replication function is mainly divided into two operations: Sync and command propagate;

First, the status of the master database and the slave database is consistent through synchronization, and then the master changes are synchronized to the slave in real time through command propagation;

Synchronization: update the slave status to the same state as the master;

Command propagation: when the master is modified, changes are synchronized to the slave in real time;

3.1 Synchronization

1. Slave sends the sync command to the master;
2. After receiving the sync command, the master executes the bgsave command, generates an RDB file in the background, and uses a buffer to record the write commands executed from now on;
3. The master node sends the generated RDB file to the slave;
4. Send the buffer write command to slave;

3.2 command Propagation

After the synchronization operation is completed, the master sends the write command that it executes to slave and slave to execute the command. The master and slave are consistent;

3.3 defects of the old version replication function

Two replication scenarios:

Initial replication: slave has not copied any master server, or the copied master server is different from the previous one;

Replication after disconnection: the master and slave in the Command propagation phase are disconnected, reconnected, and re-copied (synchronous first, then Command Propagation), which is less efficient;

3.4 The sync command consumes a lot of resources.

• The maser executes bgsave to fatalize a large number of CPU, memory, and Io resources;
• RDB file transmission consumes network resources;
• A blocking occurs when slave loads RDB files;

4. Implementation of the new copy function (beginning with redis2.8)

To solve the inefficient replication problem after disconnection, redis2.8 started to use the psync command instead of the sync command;

4.1 psync two modes

• Full re-synchronization: used for initial replication. Like the sync command, the master creates an RDB file, writes the command after the buffer is saved, and then sends it to slave;
• Partial re-synchronization: used for re-replication after disconnection. After the master and slave are disconnected, re-establish the connection. If conditions permit, the master can only send write commands during the disconnection to slave, which can also complete synchronization, and the efficiency is high.

Part 1 Implementation of re-synchronization

Partial synchronization consists of the following three parts:

• The replication offset between the master and slave;
• Master replication backlog buffer;
• The running ID of the server;

After the reconnection, slave will send its offset to the master, and the master will know what data the slave needs to synchronize. To achieve full or partial re-synchronization, select based on the slave replication offset and the relationship between the backlog buffer;

4.2.1 copy offset

Both master and slave maintain a replication offset. Determine whether the master and slave are consistent based on the offset.

Each time the master wants to spread n Bytes through slave, it will add its own copy offset + N;

Each time slave receives n Bytes, it will replace its own copy offset with N;

4.2.2 copy the backlog Buffer

The replication backlog buffer is a fixed-length first-in-first-out queue maintained by the master. The default value is 1 MB. It is used to save a certain number of latest write commands.

When the master transmits the command to the slave, it also writes the command into the replication backlog buffer;

The replication offset of the slave received by the master after the reconnection is in the replication backlog buffer, indicating that all the data to be synchronized can be obtained from the replication backlog buffer, and partial re-synchronization is performed; otherwise, perform full re-synchronization.

Reasonable Setting of the size of the replication backlog buffer can effectively use part of the re-synchronization mode;

Size formula: buffer size = disconnection time in seconds * write command per second;

4.2.3 server running ID

Each redis server has its own running ID, which is generated at server startup and consists of 40 random hexadecimal characters.

During initial replication, the master sends its running ID to slave and saves it;

After the master and slave are disconnected and re-connected, slave sends the saved Master operation ID to the master server that is currently connected.

If the sent ID is the same as the current master server ID, partial re-synchronization is attempted; otherwise, full re-synchronization is performed;

4.3 psync command implementation

• Slave has not copied any master or executed slaveof no one: slave sends psync? -1 command to request full re-synchronization;
• Slave has copied the master: slave sends psync <runid> <OFFSET>, and the master determines which synchronization to perform;
• The master returns the + fullresync <runid> <OFFSET> reply, and completes the re-synchronization. The slave saves the runid and uses the offset as its own initialization offset;
• If the master returns the + continue reply, partial re-synchronization is performed, and the slave waits for the data;
• The master returns-err, indicating that the master version is earlier than 2.8. The slave sends the sync command for full re-synchronization;

5. Replication implementation

 

　　

Redis master-slave Replication