[Don't forget beginner's mind]
Earlier, we briefly translated the contents of the Redis cluster, in the process of building a cluster, we often use the function is master-slave replication, redundant backup. In this article we will introduce this part of the content. All right, let's get to the text section right away.
--------------------------------------------------------------------------------------------------------------- ------------------------
The Redis replication feature is very easy to use and is configured primarily to allow Redis's slave server to replicate the server that is fully consistent with master from the replication feature. Here are a few important aspects of the Redis replication feature:
- Redis uses asynchronous replication. Starting with version 2.8, the slave server will report the processing progress of the replication stream to the master server at a frequency of 1s.
- A master server can have multiple slaves servers.
- Slave servers can also be linked to other slave servers. In addition to linking to the master server, the slave server can also form a graph structure with multiple slave.
- The Redis replication feature does not block the master server. In other words, the master server can continue to accept service requests when initializing replication to one or more slave.
- The replication feature also does not block the slave server. When slave is initializing a copy, slave is also able to handle the older version of the data service, provided that you have configured the redis.conf file accordingly. Alternatively, you can configure the slave to return an error when it disconnects from master. However, after the asynchronous initialization, the old data must be deleted and the new data must be loaded into the slave. At this point, the slave will be blocked until the copy is complete.
- Replication can also be used to promote extensibility, allowing multiple slave to provide read-only services (for example, heavy Sort command operations can be handed over to the retelling node). Or simply use the copy function for data redundancy (Datar redundancy).
- Use the replication feature to prevent the master node from performing persistent operations. You only need to turn off the persistence of the primary server and then perform the persistence operation from the server. Special: Under this configuration, make sure that the master server does not restart automatically.
Security of replication when the master node shuts down the persistence featurewhen Redis replication is turned on in settings, it is strongly recommended to turn on persistence in the master node, or it is absolutely impossible, because of latency issues, the Redis instance is configured to avoid an automatic restart. to better understand why it is dangerous to configure the master node of the shutdown persistence feature to restart automatically, let's look at the following types of failures that occur when data is removed from the master node and all of its corresponding slave nodes:
- Suppose the A node is the master node, and the B,c node is the corresponding copy.
- A When a crash occurs (configured to restart automatically), the process restarts. However, since the persistence feature is turned off, the data inside the node is empty after the reboot.
- The node b,c will be copied from a, when the data set in a is empty, so b,c will completely delete the data that they previously backed up.
for high availability, Redis Sentinel (Sentinel) is introduced, and persistence on the master node is turned off, and both are configured to restart automatically, which is also very risky. For example, the master node may have completed a reboot very quickly, so that SENTIENL did not perceive the failure, and then the result would be the same as the example above. at any time, the security of the data is very important, therefore, when using the replication function, if the master node does not have the persistence function, the automatic restart function must be forbidden. how the replication function worksWhen you create a slave node, the slave node sends a sync command to the master node. Whether it's the first link or the reconnection. When the maste node accepts the Sync command, the master node starts the background save feature and begins to cache all incoming new commands that will modify the dataset. When the background save task finishes, the master node transmits the data file to the slave node, and the slave node saves the data file to the local disk and loads it into memory. The master node then passes all the commands that were previously saved to the slave node. This feature will be done in accordance with the instruction flow and is in the same format as the Redis protocol itself. You can verify the synchronization process yourself by using the Telnet command. First, connect to the Redis server that is processing the command request, then send the Sync command, and then you will see a large amount of data transfer and each command received on the master node will be run again in the Telnet session. When the master node disconnects from the slave node, the Slave node can automatically override the link to the master node. If the master node receives multiple simultaneous replication requests, the master node only needs to perform a sync command once, and it can process the synchronization requests to all the slave servers. when the master node is disconnected from the slave node, a full sync command will be executed. Then, after the 2.8 release, the slave node chooses whether to have a full or partial synchronization based on the situation of the primary server. partial resynchronizationstarting with version 2.8, the master node and the slave node typically have the ability to replicate the continuation of a breakpoint in the case of a wire break, without the need for a full copy process. This feature requires the master server to create a replication stream buffer in memory. Also, both master and slave need to record the next replication offset and a master node ID, and when the network connection disconnects, the slave node will reconnect and request the master server to continue with the original replication process. if the master node ID is consistent with the ID of the slave node record, and the data specified by the copy offset of the slave record still exists in the master node, the replication stream will continue to be transferred from the breakpoint. if only one of the above conditions is not satisfied, start a complete replication process. The partial resynchronization feature of the Redis2.8 version uses a new Psync command, and the previous version only has the Sync command. However, as long as the server version is above 2.8, it will automatically choose to use Psync or sync. No disk copyTypically, a completed resynchronization request requires an RDB file to be created, and the Rdb file is loaded from disk into the slave node. If this is done on a very slow disk, this is a very stressful operation for the master node. Starting with version 2.8.18, experimental support for disk-free replication. Under this configuration, a child process can send an RDB directly over a network connection to slave, instead of a disk as an intermediate store. ConfigurationConfiguring a Slave server is very simple, just add this one in the configuration file:
slaveof 127.0.0.1 6379
of course, you need to replace the IP and port in your code with the IP and port of your own master server. Another way is to use the slaveof command, enter the IP and port of the master server, and then the master node will start synchronizing.
127.0.0.1:6379> slaveof 127.0.0.1 6379OK
There are also a number of parameters that are specifically tuned in memory for replication storage during partial resynchronization of the master node. The no disk copy feature can be turned on using the Repl-diskless-sync parameter. In order to wait for more slave nodes to arrive after the first slave node arrives, delay the transfer action, which can be controlled by the Repl-diskless-sync-delay parameter. For more information, see the distributed configuration of the redis.conf file. read-only slave serverstarting with version 2.6, the slave node is read-only and is also the default. This mode can also be controlled by the slave-read-only in the redis.conf file, or it can be set by the Config set command. The slave node in read-only mode rejects all write commands, and therefore does not occur because the error operation is possible to write data in slave. However, this does not mean that we can expose a slave node to the Internet or to untrusted network clients, because the commands of the management class, such as the Debug,config command, are still available. However, we can disable the Rename-command command in the redis.conf file to enhance the security of the read-only slave node. You may wonder why it may appear that re-overwriting the read-only configuration and the slave node becomes the target of the write operation. Although these writes are discarded when the slave node synchronizes with the master node or slave restarts, there is still a legitimate use of some of the temporary data stored in the writable slave node. However, in the future, this may be removed. authorization configuration for slave nodesIf the master node has access passwords set through Requirepass, the corresponding password must also be configured in the configuration slave node in order to perform all the synchronization operations. for a running slave, you can use the following command:
Config set Masterauth <password>
to permanently set this password, you can add it to the configuration file:
Masterauth <password>
The master server performs a write operation only if the N-slave server is in a condition starting with Redis 2.8, to ensure data security, you can configure the primary server to execute the write command only if there are at least N currently connected from the server.
However, because Redis uses asynchronous replication, the write data sent by the primary server is not necessarily received from the server, so the likelihood of data loss is still there.
Here's how this feature works:
- The slave server pings the master server once per second and reports on the processing of the replication stream.
- The master server will record the last time the ping was sent by each slave server.
- Users can configure the maximum time Min-slaves-max-lag (M) for network latency and the minimum number of slave servers required to perform write operations Min-slaves-to-write (N).
If there are at least n slave servers and the delay is less than m seconds, then the write operation will be accepted. You can consider this feature as a loose condition of C in the CAP theory, although the persistence of the write operation is not guaranteed, but at least the window that loses the data is strictly limited to the specified description. If these conditions are not met, the master node will return an error and the write operation will be rejected. here are two options and required parameters for this feature:
- Min-slaves-to-write <number of Slaves>
- Min-slaves-max-lag <number of Seconds>
For detailed information, refer to the Redis.conf sample files that are included with the Redis source codereplication strategy with extended time keyThe Redis timeout configuration allows the key to have a limited time in effect. Since this feature has been able to calculate the effective time for Redis instances, however, Redis's slave nodes accurately replicate these keys with time-outs, even when these keys are changed by the Lua script. to achieve such a feature, Redis cannot rely on the ability to synchronize time between Master and slave, because this synchronization problem is unresolved and will result in the creation of competitive conditions and inconsistent data sets, so Redis uses the following three policy implementations to allow key with expiration time to be used:
- slave does not expire the key, Instead, wait for the master node to expire it. When the master node expires the time of a key, use the Del command to notify all slave nodes to delete the key.
- However, Due to the master driver expiration time, at some point it is still possible for slave to retain some key that has actually expired because the master node does not provide the DEL command in a timely manner. In order to handle the slave node using its correct time to report a nonexistent key,
- in Lua script execution, The timeout setting for key will not be executed. During the execution of the Lua script, it is conceptually stated that the time on master is stopped, so that a given key will only exist or not exist for the full time of a given script execution period. This prevents key from timing out during script execution and is necessary to send the same script to slave, which guarantees the same effect on the dataset.
as expected, once a failover occurs, slave becomes master, which begins to independently determine the expiration time of the key, rather than asking the previous master. ----------------------------------------------------------------------------------------------------------- --------------------------at this point, NoSQL redis---Master-slave replication
References:
Official Document: Http://redis.io/topics/replication
Other information: http://doc.redisfans.com/topic/cluster-tutorial.html
NoSQL Redis---master-slave replication
