About the replication in Redis

First, Introduction

The replication mechanism of Redis allows slave to transfer copies over the network from master to full data backup. Has the following characteristics:

• Asynchronous replication

• You can configure a master multi-slave

• Can be configured from server can cascade from server, both M->s->s

• M replication is non-blocking (during replication, M is still able to process client requests)

• s replication is also non-blocking (can also accept requests from the client, but it uses the previous old data) can be configured to determine whether s in the replication with the old data in response to the client's request, if configured to No, Then slave will return an error message to the client. However, when the new data is received completely, the new data must be replaced with the old data, that is, delete the old data, in the time window of the replacement data, slave will reject the client's request and connection

• The ability to persist the entire data set to the hard disk by replication to prevent Master from persisting every time. Simply configure Master to not perform the save operation, and then connect to the previous slave, which is configured to perform the save operation occasionally. However, it is important to note that in this use case, you must ensure that master does not start automatically

Ii. Security of replication when the master persistence function is turned off

When there is a need to use the replication mechanism, it is generally strongly recommended to open the persistent switch of master. even if the persistent switch is not turned on in order to avoid the lingering effects of persistence, you should also configure Master to not start automatically

To better understand the danger of a master without persistence if it allows automatic start-up. You can look at the following failure scenarios:

Let's say we have a Redis node A, set to master, and turn off persistence, and another two nodes B and C are its slave and copy the data from a.

• If the a node crashes causing all data to be lost, it will restart the system to restart the process. However, because the persistence feature is turned off, its data set is empty even if it is restarted. While B and C still copy data from A through the replication mechanism, B and C are copied from a to an empty dataset, and the empty dataset is used to replace its own non-empty data set. So it's the equivalent of losing all the data.

• This can happen even if you use some HA tools, such as Sentinel, to monitor the master-slaves cluster, because master may crash and recover quickly. Too fast to cause Sentinel to notice a failure happening.

• The security of the data is important, the persistent switch is closed and replication occurs, you should prohibit the instance from starting

Three, M/s replication working principle

• When Master and slave start, regardless of whether the slave is the first connection to master, it sends a SYNC command to master to request the data to be replicated

• When master receives the SYNC command, the data is persisted in the background, and during persistence, Master continues to receive requests from the client, which caches the requests that might modify the dataset in memory. When the persistence is complete, Master sends the data set to Slave,slave to persist the received data and then load it into memory. The master then sends the command that was previously cached in memory to slave

• When the connection between master and slave is disconnected for some reason, slave can automatically reconnect master, and if master receives multiple slave concurrent connection requests, it will only be persisted once. This persistent data is then sent to multiple concurrent connections slave

• When master and slave are disconnected, the entire data is typically copied. However, starting with the redis2.8 version, partial replication is supported

Data partial replication

Starting with version 2.8, slave and master are able to replicate only part of the data after the network connection has been disconnected.

Master creates a wait queue for a replication stream in its memory, and master and all of its slave maintain the process ID of the copied data subscript and master, so when the network connection disconnects, slave requests master to continue with the outstanding replication. Start with the recorded data subscript. If the process ID changes, or the data subscript is not available, then all data will be copied at once. The commands that support partial data replication arePSYNC

Replication that do not require hard drive participation

In general, one replication needs to write the memory data to the hard disk, then read the data from the hard disk into memory and send it to slave. For slow drives, this operation can result in a performance penalty for master. The Redis2.8 version starts with an experimental addition to the non-hard copy feature. This function can send data directly from memory to slave without having to go through the storage of the hard disk. However, this feature is currently in the experimental phase and has not yet been formally released.

Iv. Configuration M/s

Slaveof <masterip> <masterport>slave instances need to be configured for this item, point to Master (IP, port) Masterauth <master-password> If the master instance has password protection enabled, the configuration entry requires a master startup password, and if Master does not enable the password, the configuration item needs to be commented out slave-serve-stale-data specifies the action when the slave and master connection is interrupted. The default is yes, which indicates that slave will continue to answer requests from the client, but the data may be out of date (because the connection is interrupted and cannot be synchronized from master) if configured as No, slave in addition to the normal answer "INFO" and "slaveof" commands, The remaining request commands from the client are answered by "SYNC with Master in progress" until the slave connection to master is rebuilt successfully or the slave is promoted to master. SLAVE-READ-ONLY Specifies whether the slave is read-only and the default is yes. If configured as no, this means that slave is writable, but the written content will be deleted after the master-slave synchronization is completed Repl-ping-slave-periodredis deployed to replication mode, Slave will ping the package to master with a predetermined period (default 10s), which can change the default period repl-timeout 2 cases of timeouts are specified by the configuration: 1) Bulk transfer I/O timeout; 2) master data or ping response timeout need to pay special attention to: If the default value is modified, the user must enter a value greater than the Repl-ping-slave-period configuration value, otherwise, when the master-slave link delay is high, Frequent timeoutrepl-disable-tcp-nodelay Specifies whether to disable the No_delay option for the socket when synchronizing data to slave. If configured to Yes, disable No_delay, then the TCP stack will merge packet unified send, which can reduce the number of packets between master and slave nodes and save bandwidth, but will increase the time of data synchronization to slave if configured to No, indicating that No_delay is enabled, The TCP stack does not delay the timing of packets, so the delay of data synchronization will be reduced, but the need for greater bandwidth is usually configured to No to reduce synchronization delay, but in the case of high Network load between master and slave nodes, can be configured as Yes Note: socket No_delThe AY option involves a congestion control algorithm for the TCP stack-nagle ' s algorithmslave-priority specifies the priority of the slave. In a deployment environment where not more than 1 slave exist, when Master is down, Redis Sentinel promotes the slave with the lowest priority value to master. It is important to note that if the configuration item is 0, the corresponding slave will never be Sentinel automatically promoted to master

Replication related configuration is relatively simple, only need to add the following line to the slave configuration file

Slaveof 192.168.1.1 6379

You can also send SLAVEOF commands to slave via the client. The no-drive replication feature can be configured by using repl-diskless-sync another configuration item to repl-diskless-sync-delay configure the time between waiting for multiple slave to request together when the first request is received

1. Start two Redis servers at the same time, consider starting two Redis servers on the same machine, listening to different ports, such as 6379 and 6380

2, the configuration file is 6379.conf 6380.conf

3. Start and configure

./src/redis-server  6380.conf./src/redis-server  

4. Testing

[Email protected] redis]#/src/redis-cli-  p 6379127.0.0.1:6379> flushdbok127.0.0.1:6379> set AA 1ok127.0.0.1:6379> set BB 2ok127.0.0.1:6379> set CC 3ok127.0.0.1:6379> exit[[email protected] redis]#./src/  REDIS-CLI-  P 6380127.0.0.1:6380> keys *) "BB" 2) "AA" 3) "CC" 127.0.0.1:6380> get AA "1" 127.0.0.1:6380> get BB "2"

V. Only read slave

Starting with the redis2.6 version, slave supports read-only mode and is the default. Configuration items can be configured and slave-read-only support clients using CONFIG SET commands to dynamically modify the configuration

Read-only slave will reject all write requests, and read-only slave are not intended to protect against untrusted clients, after all, some administrative commands, such as DEBUG and CONFIG in read-only mode, can be used. If you do want to ensure security, you can rename some commands in the configuration file. You may be wondering why you can restore a read-only slave to writable, even if it is writable, but as long as you slave the data in sync with master, you lose the data written in slave. However, there are some legitimate scenarios where it is necessary to store instantaneous data. However, you may consider abolishing this feature later. Setting a slave to authenticate to a master

If master requirepass sets a password through a configuration item, slave requires a password to be verified each time the synchronization operation is made, by adding the following configuration entry in the slave configuration file:

Masterauth <password>

You can also send the following commands at run time through the client:

Config set Masterauth <password>

Six, at least n slave is allowed to write data to master

Starting with the redis2.8 version, master can be configured to accept requests to write data only if master currently has at least n slave connected; Because Redis is asynchronously replicated, it does not guarantee that slave will receive a write request, so there is always a time window for data loss to exist

This mechanism works as follows:

• Slave send ping heartbeats per second to master, asking how much data is currently replicated

• Master will record the last time it received a ping heartbeat from a Slave.

• A user can configure a time to specify a time-out period when the ping heartbeat should not be sent over

If master has at least N slave, and the ping heartbeat timeout does not exceed m seconds, then it receives a write request. You might think this is a weaker version of C (consistency) in Cap theory, because writing requests doesn't guarantee data consistency, but in doing so, at least data loss is limited to a limited time, or M seconds

If neither n nor m conditions are met, Master will reply to an error message. Write requests will not be processed

There are two configuration items to configure the N and M mentioned above:

Min-slaves-to-write <number of Slaves>min-slaves-max-lag <number of seconds>

About the replication in Redis