Redis Replication (Replication) _redis

Redis Replication (Replication)

This document is translated from: Http://redis.io/topics/replication.

Redis supports the simple and Easy-to-use master-slave Replication (Master-slave replication) feature, which makes it an exact replica of the primary server (master server) from the server (slave server).

Here are a few important aspects of the Redis replication feature:

Redis uses asynchronous replication. Starting with Redis 2.8, the processing progress of the replication stream (replication stream) is reported to the primary server from the server at the first frequency per second.

A master server can have more than one from the server.

Not only the primary server can have from the server, from the server can also have its own from the server, multiple from the server can form a graph structure.

Replication does not block the primary server: even if one or more of the initial synchronizations are in progress from the server, the primary server can continue processing the command request.

Replication also does not block from the server: As long as the appropriate settings are made in the redis.conf file, the server can use the old version of the dataset to process the command query even though the initial synchronization from the server is in progress.

However, connection requests are blocked during the time that the old version dataset is deleted from the server and the new version dataset is loaded.

You can also configure the server to send an error to the client when the connection to the primary server is disconnected.

Replication can be used simply for data redundancy (redundancy), or to promote extensibility (scalability) by having multiple requests for read-only commands from the server: for example, heavy SORT commands can be handed over to a subordinate node to run.

You can make the primary server exempt from persistent operations by replicating functionality: Simply turn off the persistence of the primary server and then perform the persistence operation from the server. Data security for replication when you turn off primary server persistence

When configuring the Redis replication feature, it is strongly recommended that you turn on the persistence of the primary server. Otherwise, the deployed services should avoid automatic pull up due to issues such as latency.

To help understand the dangers of automatic pull when the master server shuts down for persistence, refer to the following examples that will result in the total loss of master-slave server data:

1. Suppose node A is the primary server and the persistence is turned off. and Node B and node C copy data from Node A

2. Node A crashes, and then the node A is restarted by the automatic pull service. Since the persistence of Node A is turned off, there is no data after the reboot

3. Node B and node C will copy the data from Node A, but the data for a is empty, so the copy of the data you saved is deleted.

In the case of turning off the persistence on the primary server and opening the automatic pull process at the same time, even using Sentinel to achieve high availability of redis is very dangerous. Because the primary server may pull up so fast that Sentinel does not detect that the primary server has been restarted during the configured heartbeat interval and then executes the data loss process above.

Data security is extremely important at all times, so you should prevent the primary server from shutting down and automatically pulling when it is persisted. How the Copy function works

Whether it is initial or reconnection, a SYNC command will be sent from the server to the primary server when a server is established.

The primary server receiving the SYNC command starts executing the Bgsave and saves all newly executed write commands to a buffer during the execution of the save operation.

When Bgsave executes, the master server sends the. rdb file from the server to receive the. rdb file from the server, and loads the data from the file into memory.

The master server then sends all the content accumulated in the Write command buffer to the server in the format of the Redis command protocol.

You can use the Telnet command to personally verify the synchronization process by first connecting the Redis server that is processing the command request, and then sending the sync command to it, and for a while you will see that the Telnet session receives a large segment of data from the server (. rdb file) , and then you will see that all write commands that have been executed at the server are sent back to the Telnet session.

Even if there are multiple simultaneous send sync from the server to the primary server, the primary server can handle all of the synchronization requests from the server by simply executing the bgsave command once.

Automatic reconnection can be made from the server when the connection between the master and slave servers is disconnected, before the Redis version 2.8, the reconnection from the server will always perform a full resynchronization operation, but starting from the Redis 2.8 version, the server can To choose whether to perform a full or partial resynchronization (partial resynchronization) based on the situation of the primary server. Partial resynchronization

Starting with Redis 2.8, the master-slave server can try to continue the original replication process (process) without having to perform a full resynchronization, after the transient network connectivity failure.

This feature requires the primary server to create a memory buffer (in-memory backlog) for the replicated stream being sent, and a copy offset (replication offset) and a primary server ID are logged between the primary server and all from the server (master run ID), reconnect from the server when a network connection is disconnected, and request the primary server to continue with the original replication process: If the primary server ID from the server record is the same as the ID of the primary server that is currently being connected, And the data specified from the offset of the server record is still stored in the copy stream buffer of the primary server, then the primary server sends the part of the data that is missing from the server, and the replication work can continue. Otherwise, a full resynchronization operation will be performed from the server.

This partial resync feature of Redis 2.8 uses a new Psync internal command, while the previous version of Redis 2.8 has only the SYNC command, but as long as the server is Redis 2.8 or more, it will decide whether to use the version of the primary server Psync or Sync: If the primary server is Redis 2.8 or older, use the Psync command from the server to synchronize. If the primary server is a version prior to Redis 2.8, sync is performed from the server using the Sync command. Configuration

Configuring one from the server is very simple, just add the following line to the configuration file:

Slaveof 192.168.1.1 6379

Of course, you need to replace the 192.168.1.1 and 6379 in your code with the IP and port number of your home server.

Another approach is to invoke the Slaveof command, enter the IP and port of the primary server, and then the synchronization begins:

127.0.0.1:6379> slaveof 192.168.1.1 10086
OK

Read-only from the server

Starting with Redis 2.6, read-only mode is supported from the server, and the mode is the default mode from the server.

Read-only mode is controlled by the slave-read-only option in the redis.conf file, or it can be turned on or off by the CONFIG SET command.

Read-only from the server will refuse to execute any write commands, so it does not appear that the data was accidentally written to the server because of an error in the operation.

Although management commands such as DEBUG and CONFIG are still available, even if they are read-only from the server, we should not expose the server to the Internet or any other unreliable network. However, using the command renaming option in redis.conf, we can improve the security of read-only from the server by preventing certain commands from being executed.

You may be curious to know that since write data from the server will be overwritten by the resynchronization data and may be lost when restarting from the server, then why make one from the server writable.

The reason is that some of the less important temporary data can still be saved on the server. For example, a client can implement a failover (failover) policy by saving the master server's accessibility (reachability) information from the server. Configuration from server-related

If the primary server has a password set through the Requirepass option, then in order for the synchronization operation from the server to proceed smoothly, we must also make the appropriate authentication settings from the server.

For a running server, you can use the client to enter the following command:

Config set Masterauth <password>

To permanently set this password, you can add it to the configuration file:

Masterauth <password>

There are also several options that are related to the replication stream buffers used by the primary server when performing a partial resynchronization, with detailed information referenced in the REDIS.CONF sample file included with the Redis source code. The primary server performs a write operation only if there are at least N from the server

Starting with Redis 2.8, to ensure data security, you can configure the primary server to execute write commands only if there are at least N currently connected from the server.

However, because Redis uses asynchronous replication, the write data sent by the primary server is not necessarily received from the server, so the likelihood of data loss is still present.

Here's how this feature works: PING The primary server at one time per second from the server and report on the processing of the replication stream. The master server records the last time a PING was sent to it from the server. The user can configure to specify the maximum min-slaves-max-lag of network latency and at least the number of min-slaves-to-write from the server required to perform the write operation.

If there are at least min-slaves-to-write from the server, and these servers have a delay value of less than min-slaves-max-lag seconds, the primary server performs the write operation requested by the client.

You can consider this feature as a conditional relaxed version of C in CAP theory: Although it does not guarantee the persistence of write operations, at least the window that loses data is strictly limited to the specified number of seconds.

On the other hand, if the condition does not reach the conditions specified by Min-slaves-to-write and Min-slaves-max-lag, the write operation is not performed, and the primary server returns an error to the client requesting the write operation.

Here are two options for this feature and the parameters they need: Min-slaves-to-write <number of slaves> min-slaves-max-lag <number of seconds>

Detailed information can refer to the Redis.conf sample file included with the Redis source code.

From:http://redisdoc.com/topic/replication.html