Redis design and implementation Learning notes-master-slave replication

To provide the full cache availability, you can add an alternate server to the primary server, that is, from the server. The current server is set up as a slave server for a server through the slaveof command.

Old Version Synchronization process

Older versions (before 2.8) The master-slave server data synchronization process is as follows:

1. Sends the sync command from the server to the primary server.
2. When the master server receives the Sync command, it calls the Bgsave command to generate an RDB file, in which the client modifies the master server's command in a buffer.
3. The primary server sends the generated RDB file to the slave server and updates the database state from the server by loading the Rdb file.
4. The primary server synchronizes the commands in the buffer to the slave server and executes these commands from the server to update the database.

The old version in sync command synchronous processing has a very large performance flaw, after the master-slave server disconnection after the synchronization action, will also generate a full RDB file and sent to the server load, but the master and slave server database state is basically consistent before the disconnection, Inconsistent parts only the main server executes that part of the command to modify the database, so this time the sync command is very wasteful, because the Rdb file when generating a very CPU, memory and IO resources, and send the Rdb file to the slave server also consumes a lot of network bandwidth resources. It is not necessary and unreasonable to perform the sync command in most cases, since the server is also blocking the process of loading the Rdb file without any corresponding commands.
In order to solve the performance problem of the version Sync command prior to 2.8, version 2.8 designed a new command Psync,psync command divided into full resynchronization and partial resynchronization, the full resynchronization process is used for initial replication from the server initialization and Sync command basically consistent, Psync is used to re-copy after disconnection, in the condition allows case, it does not generate an RDB file, but instead gives a reply from the server to a +continue that performs a partial resynchronization, and sends commands from the server to modify the database executed by the server to execute these commands from the server to synchronize the database.
The partial resynchronization feature is comprised of the following sections:

• Replication offsets from the primary server and replication offsets from the server: when the primary server synchronizes commands from the server, both the primary and slave servers record a copy offset, and the two replication offsets are the same when the master-slave server's database state is consistent. If these two offsets are inconsistent, the state of the current master-slave server does not match.
• Replication backlog buffer for the primary server: The replication backlog is a fixed-size FIFO queue that pops up the oldest inserted data when the queue is full, and puts the command into the buffer when the command propagates on the primary server, which contains two parts of data, offsets, and bytes. At the time of replication, the offset is escalated to the primary server from the server, and the primary service checks whether the current offset exists in the buffer, if there is a partial resynchronization, if there is no full resynchronization. Because this backlog is a fixed-size queue, when disconnected from the server for a long time, the replication offset from the server is probably no longer in the buffer, which can only be full resynchronization.
• Running ID of the server: the primary server will send the ID to the slave server, save the primary server ID from the server, and when the disconnection is connected, the primary server ID that was previously saved is escalated to the primary server, and the primary server checks whether the primary server ID copied from the server is the same as its own ID, if the same Perform a partial resynchronization if different instructions are logged from the server before the state is not the current primary server, then a full resynchronization is required.

The Psync command implements the initial replication or the previous slaveof no one command, performing a full resynchronization: sending the Psync? -1 commands to the master server.
If a primary server has been replicated from the server, and the Psync <runid> <offset> command is sent to the primary server when a new replication is started, Runid is the last replicated primary server id,offset is the replication offset from the server. The master server determines which synchronization to make based on the two parameters, determines if the server ID is the same as the native, if the replication offset is in the buffer, and the master server has three replies:

• Reply +fullresync <runid> <offset> Perform a full resynchronization, from the server offset as the initial replication offsets
• Reply to +continue, which means performing a partial resynchronization, waiting for the primary server to send the missing data from the server
• Reply to-err, which indicates that the primary server version is less than 2.8 and does not support the Psync command

New version Synchronization procedure 2.8 and later version replication process:

1. Set the primary server address and port by calling the saveof <master_ip> <master_port> command.
2. Establish a socket connection.
3. Send a ping command to check whether the master-slave server can handle the command properly.
4. Authentication, the Masterauth is set from the server and the primary server is set up Requirepass is required for authentication. Both of these options are either set or not set, and an error occurs if you set only one command from the server to the primary server.
5. Send port information by executing the command replconf Listening-port <port-number>, and sending the listener port number from the server to the primary server.
6. Synchronize to send the Psync command from the server to the primary server.
7. Command propagation, after the completion of synchronization, the master server will be executed after the write command to the slave server to ensure that the status of the same.

Heartbeat detection during the command propagation phase, the command is sent to the primary server from the server's default frequency per second: replconf ACK <replication_offset>,replication_offset is the copy offset from the server, The command has three functions:

• Detects the network connection status from the server, detects if the master-slave server connection is normal, and if the primary server does not receive the replconf ACK command from the server for a certain amount of time, then there may be a problem with their connection.
• Auxiliary implementation of the Min-slaves option, Min-slaves-to-write and min-slaves-max-lag two options to prevent the primary server from performing write commands in an unsafe situation, min-slaves-to-write 3 Min-slaves-max-lag 10 indicates that the primary server rejects the write command if it is less than 3 from the server, or if 3 latencies from the server are greater than 10 seconds.
• When the detection command is lost, the primary server receives the REPLCONF ACK command from the server and checks whether the offset from the server is consistent with the primary server, and if the inconsistency sends a command from the server offset from the backlog buffer to the slave server.

Redis design and implementation Learning notes-master-slave replication