Redis replication and scalable cluster setup

Redis's master-slave copy strategy is realized through its persistent RDB file, the process is to dump out Rdb file, Rdb file to Slave, and then synchronize the operation of the dump to slave in real time. The following is an article on the principle of Redis reproduction, the author of the article for Sina Weibo Tianqi classmate (@ Rocking Bach).

This article discusses the replication capabilities of the Redis and the advantages and disadvantages of the Redis replication mechanism itself, as well as the problem of cluster setup. 

redis replication Process Overview

Redis's replication capabilities are based on a memory snapshot-based persistence strategy that we discussed earlier. In other words, no matter what your persistence strategy chooses, as long as you use the Redis copy function, there will be a memory snapshot occurs, then first of all pay attention to your system memory capacity planning, The reason can refer to the Redis disk IO issue mentioned in my previous article.

The Redis replication process is a set of state machine flows at the slave and master terminals, and the status information involved is:

Slave End:

Redis_repl_none
redis_repl_connect
redis_repl_connected

Master End:

Redis_repl_wait_bgsave_start
redis_repl_wait_bgsave_end
redis_repl_send_bulk
redis_repl_online

The entire state machine process process is as follows: The 1.Slave end adds the slave of directive to the configuration file, and the slave reads the configuration file at startup and the initial state is redis_repl_connect. The 2.Slave is connected to master in the timer task Servercron (REDIS), sends the Sync command, and then blocks waiting for master to send back its memory snapshot file (the latest version of Redis does not need to block slave). 3.Master to receive sync command simple to determine whether there is an ongoing memory snapshot child process, no immediate start memory snapshot, and then wait for its end, when the snapshot is completed will send the file to the slave end. 4.Slave to receive the memory snapshot files from master, save to the local, after receiving completion, empty the memory table, re-read the memory snapshot files from master, rebuild the entire memory table data structure, and finally state the status of Redis_repl_connected State, Slave state machine Flow complete. 5.Master in the process of sending a snapshot file, any commands that will change the dataset are temporarily saved to the Send Cache team column (list data structure) of the Slave network connection, which is sent to slave after the snapshot is completed, followed by the same command, and the status is Redis_repl_online.

The entire replication process is complete, as shown in the following illustration:

defects of Redis replication mechanism

As can be seen from the above process, slave from the library when connecting to the master library, Master will take a snapshot of the memory, and then the entire snapshot file to slave, that is, there is no copy as MySQL as the concept of replication, that is, no incremental replication, which will bring a lot of problems to the entire

For example, a running master library on the line is configured with a simple read-write separation from the library, and when slave is disconnected from master for the network or other reason, the memory snapshot of the entire master needs to be retrieved when the slave is reconnected. Slave all the data is cleared, and then the entire memory table is restored, on the one hand slave recovery time will be very slow, on the other hand will also bring pressure on the main library.

So based on the above reasons, if your Redis cluster needs master-slave replication, then it is best to configure all of the libraries from the library beforehand, avoiding further additions from the library. the cache or the storage

After we analyzed the Redis replication and persistence function, we are not difficult to draw a conclusion, in fact Redis currently released version is still a stand-alone version of the idea, the main problem is focused on, the persistence of the way is not mature, the replication mechanism there are relatively large defects, Then we began to rethink the positioning of Redis: Cache or storage.

If, as a cache, it seems that in addition to some very special business scenarios, you have to use some kind of REDIS data structure, we use memcached may be more appropriate, after all memcached regardless of the client package and the server itself more time-tested.

If it's a storage storage, the biggest problem we face is that there is no way to solve the redis problem, whether it is persistent or duplicated, that is, a redis is hung up, there is no good way to quickly recover, usually dozens of g of persistent data, Redis Restart loading takes several hours, and replication is flawed and how to solve it. 

Redis Scalable Cluster Setup 1. Active replication avoids redis replication defects.

Since Redis's replication capabilities are flawed, we may as well abandon the replication capabilities provided by Redis itself, and we can use the proactive replication approach to build our cluster environment.

The so-called active replication refers to the data stored by the business end or through the agent middleware to Redis or multiple write, through the data storage to achieve the same purpose as replication, active replication is not limited to the use of Redis cluster, At present, many companies use the technology of active replication to solve the problem of replication between MySQL master and slave, such as Twitter has also developed a special middleware for replication and partition gizzard (Https://github.com/twitter/gizzard).

Although active replication solves the problem of delay in passive replication, it also brings new problems, that is, the consistency of data, the data is written 2 or more times, how to ensure the consistency of multiple data. If your application does not require a high level of data consistency, allowing for final consistency, the simple solution is to allow the client to simultaneously fetch multiple data and verify the data in the form of a timestamp or vector clock, if your application requires very high consistency Then we need to introduce some complex consistency algorithms such as Paxos to ensure the consistency of the data, but the write performance will decrease a lot.

Through active replication, data storage we are no longer worried about Redis single point of failure, if a group of Redis cluster hangs, we can let the business quickly switch to another group of Redis, reduce business risk. 

2. Redis online expansion through presharding.

By proactively replicating, we have solved the problem of Redis single point of failure, then there is an important problem to be solved: capacity planning and online expansion.

We previously analyzed the Redis scenario is that all data is stored in memory, and memory capacity is limited, then first need to base on the volume of business data for initial capacity planning, such as your business data needs 100G storage space, assuming that the server memory is 48G, So based on the last issue we discussed about Redis disk IO, we need a 3~4 server to store it. This is actually a capacity plan for the existing business situation, if the business growth quickly, will soon find that the current capacity is not enough, redis stored in the data will soon exceed the size of physical memory, then how to Redis online expansion.

Redis's author proposes a scheme called presharding to solve the problem of dynamic expansion and data partitioning, in fact, the way to deploy multiple instances of Redis on the same machine, when the capacity is not enough to split multiple instances onto different machines, so as to achieve the effect of expansion.

The split process is as follows: 1. Start the Redis instance of the corresponding port on the new machine. 2. Configure the new port to be migrated from the library. 3. To be replicated, after synchronizing with the main library, switch all client configurations to the new port from the library. 4. Configure the new master library from the library. 5. Remove the old Port instance. 6. Repeat the above process to migrate all ports to the specified server.

The split process above is a smooth migration process proposed by the Redis author, but the split method relies heavily on the Redis itself, and if the primary database snapshot data file is too large, the replication process will be long and will bring pressure to the main library. So the process of doing this split is best chosen for business access to low peak period. 

the improvement idea of Redis copy

Our online system uses our own improved version of the Redis, mainly to solve the Redis without incremental replication of the defects, can do similar to MySQL Binlog can be done from the library request log location for incremental replication.

Our persistence scheme is to write the Redis aof file first, and to automatically split the aof file by file size, turn off the Redis rewrite command, and then do the memory snapshot storage at the low peak time of the business and write the current aof file location to the snapshot file. This allows us to keep the snapshot files consistent with the location of the aof file. So we get a snapshot of the memory at some point in the system, and we know the location of the corresponding aof file at the same time, so when we send the Sync command from the library, we first send the snapshot file to the library, The aof file location stored in the snapshot file is then removed from the library, and the location is sent to the main library, which then sends all commands after that location, and the subsequent replication is the incremental information after that location.

the combination of Redis and MySQL

Now most Internet companies use MySQL as the main data persistence storage, so how to make Redis and MySQL very good combination. We mainly use a MySQL as the main library, Redis as a high-speed data query from the library of heterogeneous read and write separation scheme.

To this end we specialize in the development of our own MySQL replication tools, can be convenient real-time synchronization of MySQL data to the Redis.

(Mysql-redis heterogeneous read-write separation) 

Summary: 

1.Redis replication Features no incremental replication, each time the connection will be the main library of the entire memory snapshot to the library, so you need to avoid to the online service more pressure on the main library from the library. 2.Redis of replication because of the use of snapshot persistence, so if your redis persistence option is to choose Log Append (AOF), then the system may at the same time do both the AOF log file synchronous brush disk, and do snapshot disk operation, This time Redis's responsiveness will be affected. So if you choose aof Persistence, you need to be more cautious about adding from the library. 3. Active replication and Presharding methods can be used for Redis cluster construction and online expansion.

This article, together with the previous 2 articles, basically analyzes and discusses the most common functions and usage scenarios and optimization of Redis. The actual Redis also has many other auxiliary function, the Redis author also is trying the new Idea unceasingly, here does not enumerate, the interesting friend may study under, Also welcome to discuss with me, my microblog (Http://weibo.com/bachmozart) @ Rocking Bach.

Source: www.infoq.com