About Replication in redis

About Replication in redis

I. Introduction

Redis's replication mechanism allows slave to transmit copies from the master over the network to complete data backup. It has the following features:

• Asynchronous replication
• One master, multiple slaves can be configured
• The slave server can be configured to cascade slave servers, that is, M-> S
• M replication is non-blocking (during replication, M can still process client requests)
• S replication is also non-blocking (it can also accept requests from the client, but it uses old data) the configuration can be used to determine whether S uses old data to respond to client requests during replication. If the configuration is set to no, slave will return an error message to the client. However, when the new data is fully received, you must replace the new data with the old data, that is, delete the old data. In this time window, slave will reject client requests and connections.
• Replication can be used to prevent the master from persisting the entire dataset to the hard disk every time it persists. You only need to configure the master to do not save the operation, and then connect to a server Load balancer. This Server Load balancer is configured to save the operation from time to time. However, in this case, make sure that the master will not start automatically.

Ii. Replication security when the Master persistence function is disabled

When you need to use the replication mechanism, it is generally strongly recommended to enable the persistence switch of the master. Even if you do not enable the persistence switch to avoid the delay caused by persistence, you should also configure the master as not automatically started.

In order to better understand the danger of a non-persistent master enabling Automatic startup. Let's look at the following failure situation:

Suppose we have A redis node A, which is set to master and the persistence function is disabled. The other two nodes B and C are their slave and copy data from.

• If A node crashes and all data is lost, it will restart the system to restart the process. However, since the persistence function is disabled, even if it is restarted, its dataset is empty. B and C will still copy data from A through the replication mechanism, so B and C will copy data from A to an empty dataset, use this empty dataset to replace its own non-empty dataset. So it is equivalent to losing all the data.
• Even if some HA tools, such as sentinel, are used to monitor the master-slaves cluster, the above situation occurs because the master may crash and recover quickly. The speed is too fast, so that sentinel cannot detect a failure.
• Data security is very important. When the persistence switch is disabled and replication occurs, you should disable auto-start of the instance.

Iii. Working Principle of M/S replication

• When the master and slave are started, whether or not the slave is connected to the Master for the first time, it will send a SYNC command to the master to request data replication.
• After receiving the SYNC command, the master will perform data persistence in the background. During the persistence period, the master will continue to receive client requests, and it will cache the requests that may modify the dataset in the memory. After the persistence is complete, the master sends the data set to slave, and the slave persists the received data and then loads it into the memory. Then, the master sends the commands cached in the memory to the slave.
• When the connection between the master and slave is closed for some reason, slave can automatically re-connect to the Master. If the master receives multiple concurrent connection requests from slave, it will only perform one persistence, then, the persistent data is sent to the slave of multiple concurrent connections.
• When the master and slave are disconnected and reconnected, the whole data is usually copied. However, redis2.8 and later versions support partial replication.

Partial data replication

Starting from version 2.8, slave and master can only perform partial data replication after the network connection is disconnected and reconnected.

The master creates a replication stream waiting queue in its memory. Both the master and all its slave maintain the copied data subscript and the master process id. Therefore, when the network connection is disconnected, slave will request the master to continue the unfinished replication, starting from the recorded data subscript. If the process id changes or the data subscript is unavailable, all data will be copied once. The command that supports partial data replication is PSYNC.

Replication without hard disk involvement

Generally, data in the memory needs to be written to the hard disk for one copy, and then read from the hard disk into the memory, and then sent to the slave. For hard disks with slow speed, this operation will cause performance loss to the master. Redis2.8 and later versions, with the feature of no hard disk copy. This function can directly send data from the memory to the slave without passing through the hard disk storage. However, this function is currently in the experimental stage and has not yet been officially released.

4. Configure M/S

Slaveof <masterip> <masterport>

The slave instance needs to be configured to point to the master (ip, port)

Masterauth <master-password>

If password protection is enabled for the master instance, you must enter the master Startup Password for this configuration item. If the master does not enable the password, comment out this configuration item.

Slave-serve-stale-data

Specifies the action when the server Load balancer instance is disconnected from the master instance. The default value is yes, indicating that slave will continue to respond to requests from the client, but the data may have expired (due to connection interruption, the data cannot be synchronized from the master)

If it is set to no, except for the "INFO" and "SLAVEOF" commands normally answered by slave, other request commands from the client will receive a "SYNC with master in progress" response,

Until the connection between the slave and the master is successfully rebuilt or the slave is promoted to the master.

 

Slave-read-only

Whether the slave is read-only. The default value is yes. If it is set to no, this indicates that slave is writable, but the written content will be deleted after the master-slave synchronization is complete.

Repl-ping-slave-period

After Redis is deployed in Replication mode, slave sends a PING packet to the master at a predetermined period (10 s by default). This configuration can change this default period.

Repl-timeout

In either case, timeout is specified by this configuration: 1) Bulk transfer I/O timeout; 2) master data or ping response timeout

Note: If the default value is modified, the value entered by the user must be greater than the configured value of repl-ping-slave-period. Otherwise, frequent timeout occurs when the latency of the master-slave link is high.

Repl-disable-tcp-nodelay

Specifies whether to disable socket NO_DELAY when synchronizing data to slave.

If yes is configured, NO_DELAY is disabled, the TCP protocol stack combines packets for unified sending, which can reduce the number of packets between the master and slave nodes and save bandwidth, but it will increase the time for data synchronization to slave.

If no is configured, it indicates that NO_DELAY is enabled, the TCP protocol stack will not delay the packet sending time, so that the latency of Data Synchronization will be reduced, but greater bandwidth is required.

In general, it should be set to no to reduce synchronization latency, but when the network load between the master and slave nodes is already high, it can be set to yes

Note: The socket NO_DELAY option involves the TCP protocol stack congestion control Algorithm-Nagle's Algorithm

Slave-priority

Specify the slave priority. In a deployment environment where more than one slave exists, when the master node goes down, Redis Sentinel will upgrade the slave with the smallest priority value to the master node.

Note that if this configuration item is set to 0, the corresponding slave will never be automatically upgraded to master by Redis Sentinel.

Replication-related configuration is relatively simple. You only need to add the following line to the slave configuration file.

Slaveof 192.168.1.1 6379

You can also send the SLAVEOF command to slave through the client. You can use repl-diskless-sync to configure the diskless replication function. Another configuration item, repl-diskless-sync-delay, is used to configure the first request to be received, interval between requests waiting for multiple slave instances

1. Start two Redis servers at the same time. You can consider starting two Redis servers on the same machine to listen to different ports, such as 6379 and 6380.

2. The configuration file is 6379. conf 6380. conf.

3. Start and configure

./Src/redis-server 6380. conf

./Src/redis-server 6379. conf

4. Test

[Root @ localhost redis] #./src/redis-cli-p 6379

127.0.0.1: 6379> flushdb

OK

127.0.0.1: 6379> set aa 1

OK

127.0.0.1: 6379> set bb 2

OK

127.0.0.1: 6379> set cc 3

OK

127.0.0.1: 6379> exit

[Root @ localhost redis] #./src/redis-cli-p 6380

127.0.0.1: 6380> keys *

1) "bb"

2) "aa"

3) "cc"

127.0.0.1: 6380> get aa

"1"

127.0.0.1: 6380> get bb

"2"

5. only read slave

From redis2.6, slave supports read-only mode and is default. You can configure slave-read-only through the configuration item, and the client can use the config set command to dynamically modify the configuration.

Read-Only slave rejects all write requests. Read-Only slave does not prevent untrusted clients. After all, some management commands such as DEBUG and CONFIG can still be used in read-only mode. To ensure security, you can rename some commands in the configuration file. Maybe you may wonder why a read-only slave can be restored to writable. Even though it can be written, as long as slave synchronizes the data of the master, the data written in slave will be lost. However, there are still some legal application scenarios that require storing instantaneous data to use this feature. However, we may consider abolishing this feature later. Setting a slave to authenticate to a master

If the master has set a password through the requirepass configuration item, slave needs to verify the password for each synchronization operation. You can add the following configuration items to the slave configuration file:

Masterauth <password>

You can also run the following command on the client:

Config set masterauth <password>

6. At least N slave data can be written to the master.

From redis2.8, the master can be configured to accept data writing requests only when the master has at least N slave connections. Because redis is asynchronously replicated, therefore, it cannot ensure that slave will receive a write request, so there is always a time window for data loss.

The working principle of this mechanism is as follows:

• Slave sends ping heartbeat to the master every second and asks how much data is being copied.
• The master will record the last time it received the ping heartbeat from a slave.
• You can configure a time to specify a timeout value that cannot exceed the sent ping heartbeat.

If the master has at least N slave instances and the ping heartbeat timeout cannot exceed M seconds, it will receive write requests. You may think this is like the consistency of the C version in CAP theory, because write requests do not guarantee data consistency, but in this case, at least data loss is limited to a limited period of time, that is, M seconds.

If the N and M conditions cannot be met, the master will return an error message. Write requests will not be processed either.

There are two configuration items used to configure N and M mentioned above:

Min-slaves-to-write <number of slaves>

Min-slaves-max-lag <number of seconds>

You may also like the following articles about Redis. For details, refer:

Install and test Redis in Ubuntu 14.04

Basic configuration of Redis master-slave Replication

Redis cluster details

Install Redis in Ubuntu 12.10 (graphic explanation) + Jedis to connect to Redis

Redis series-installation, deployment, and maintenance

Install Redis in CentOS 6.3

Learning notes on Redis installation and deployment

Redis. conf

Redis details: click here
Redis: click here

This article permanently updates the link address: