Redis Replication (Replication)

Overview

Redis supports a simple and easy-to-use master-slave replication (Master-slave replication) feature that allows the slave server (slave server) to be an exact replica of the master server.

Here are a few important aspects of the Redis replication feature:

• Redis uses asynchronous replication. Starting with Redis 2.8, the processing progress of the replication stream (replication stream) is reported to the primary server at the frequency of every second from the server.

• A primary server can have multiple slave servers.

• Not only the master server can have from the server, from the server can also have its own from the server, multiple from the server can form a graph structure.

• The replication feature does not block the primary server: even if one or more of the primary synchronizations are in progress from the server, the master server can continue to process command requests.

• Replication does not block from the server: As long as the corresponding settings are made in the redis.conf file, the server can also use the old version of the dataset to process the command query, even if the server is in the initial synchronization.

  However, the connection request is blocked during the time that the old version dataset was deleted from the server and loaded into the new version dataset.

  You can also configure the slave server to send an error to the client when the connection to the primary server disconnects.

• Replication can be used solely for data redundancy, or it can improve extensibility by having multiple requests for read-only commands from the server (scalability): For example, a heavy SORT command can be handed to a subordinate node to run.

• The primary server can be protected from persistent operations by replicating: Simply turn off the persistence of the primary server and then perform the persistence operation from the server.

Configuration

Configuring a Slave server is as simple as adding the following line to the configuration file:

Slaveof 192.168.1.1 6379

Of course, you need to replace the 192.168.1.1 and 6379 in your code with the IP and port number of your home server.

Another way is to call the slaveof command, enter the IP and port of the primary server, and then sync will start:

127.0.0.1:6379> slaveof 192.168.1.1 10086OK

If the master server has a password set through the Requirepass option, we must also make the appropriate authentication settings for the slave server in order for the synchronization operation to proceed smoothly.

For a running server, you can use the client to enter the following command:

Config set Masterauth <password>

To permanently set this password, you can add it to the configuration file:

Masterauth <password>

Data security for replication when you turn off primary server persistence

When configuring the Redis replication feature, it is highly recommended to turn on the persistence feature of the primary server. Otherwise, due to delays and other issues, the deployed service should avoid automatic pull-up.

To help understand the risk of automatic pull-out when the primary server shuts down, refer to the following example that will cause the master-slave server to lose all data:

1. Assume that node A is the primary server and that persistence is turned off. and Node B and node C replicate data from Node A

2. Node A crashes, and then the auto pull service restarts Node A. Since the persistence of Node A is turned off, there is no data after the reboot

3. Node B and node C will copy the data from Node A, but the data for a is empty, so the copy of the data it saves is deleted.

Even using Sentinel to achieve high availability of redis is dangerous when you turn off persistence on the primary server and turn on the auto-pull process at the same time. Because the primary server may pull up so fast that Sentinel does not detect that the primary server has been rebooted during the configured heartbeat interval, and then performs the above data loss process.

Data security is extremely important at any time, so you should prevent the primary server from shutting down and automatically pulling up while persisting.

How the Replication function works

Whether it is a first-time connection or a reconnection, a SYNC command is sent from the server to the primary server when a slave server is established.

The master server that receives the SYNC command will begin executing BGSAVE and will save all newly executed write commands to a buffer during the save operation execution.

When the BGSAVE finishes executing, the master server sends the. rdb file that the save operation sends to the slave server, receives the. rdb file from the server, and loads the data in the file into memory.

The master server then sends all the content accumulated in the Write command buffer to the slave server in the format of the Redis command protocol.

You can verify the synchronization process yourself by using the telnet command: first connect to the Redis server that is processing the command request, and then send it the Sync command, and after a while you will see that the Telnet session (session) received a large segment of data (. rdb file) from the server , you will then see that all the write commands that were executed by the server are resent to the Telnet session.

Even if multiple slave servers are sending sync to the primary server, the primary server can handle all of these synchronization requests from the server by simply executing the BGSAVE command once.

From the server can be automatically re-connected when the connection between the master and slave server is disconnected, before the Redis 2.8 version, the reconnection from the server will always perform a full resynchronization operation, but starting from the Redis 2.8 version, from the server can Select whether to perform a full resynchronization or partial resynchronization (partial resynchronization) based on the primary server.

Read-only slave server

Starting with Redis 2.6, read-only mode is supported from the server, and this mode is the default mode from the server.

Read-only mode is controlled by the slave-read-only option in the redis.conf file, or it can be turned on or off via the CONFIG SET command.

Read-only from the server will refuse to execute any write commands, so there will be no case of accidentally writing data to the slave server because of an operation error.

Managed commands such as DEBUG and CONFIG are still available even if the server is read-only, so we should not expose the server to the Internet or any non-trusted network. However, using the command renaming option in redis.conf, we can increase the security of read-only slave servers by prohibiting the execution of certain commands.

You might be wondering, since the write data from the server will be overwritten by the resynchronization data, or it may be lost when restarting from the server, so why make one from the server writable?

The reason is that some of the temporary data that is not important can still be saved on top of the server. For example, the client can save the accessibility (reachability) information of the primary server from the server to implement a failover (failover) policy.

The primary server performs a write operation only if there are at least N slave servers

Starting with Redis 2.8, to ensure data security, you can configure the primary server to execute the write command only if there are at least N currently connected from the server.

However, because Redis uses asynchronous replication, the write data sent by the primary server is not necessarily received from the server, so the likelihood of data loss is still there.

Here's how this feature works:

• Pings the primary server once per second from the server and reports on the processing of the replication stream.

• The primary server logs the last time that each PING was sent to it from the server.

• The user can configure, specify the maximum network latency min-slaves-max-lag , and the minimum number of slave servers required to perform the write operation min-slaves-to-write .

If there is at least min-slaves-to-write one slave server and these servers have a latency value of less than a min-slaves-max-lag second, the primary server performs a write operation to the client request.

You can consider this feature as a conditional relaxed version of C in the CAP theory: Although the persistence of the write operation is not guaranteed, at least the window that loses the data is strictly limited to the specified number of seconds.

On the other hand, if the condition does not reach min-slaves-to-write and the min-slaves-max-lag specified condition, then the write operation will not be executed, and the primary server will return an error to the client requesting the write operation.

Here are two options for this feature and the parameters they require:

• min-slaves-to-write <number of slaves>

• min-slaves-max-lag <number of seconds>

For more information, refer to the sample files that are included with Redis source code redis.conf .

Redis Replication (Replication)