The solution of Redis master-slave replication problem and expansion problem

Reproduced in: http://www.itxuexiwang.com/a/shujukujishu/redis/2016/0216/106.html?1455867541

First, to solve the master-slave replication problem

When using Redis as a storage engine, and using Redis? Read and write separation, from the machine as a read, from the machine down or disconnected from the host needs to reconnect the host, reconnect the host will trigger the full amount of master-slave replication, the main opportunity to generate memory snapshots, the host can still provide services to external However, as read from the machine, will not be able to provide external services, if the volume of data is large, the recovery time will be quite long. To solve the Redis master-slave copy problem, there are two solutions:

Active replication
The so-called active replication, is the business layer to write multiple redis, avoid Redis's own master-slave replication. However, the synchronization of their own, there will be a consistency problem, in order to ensure that the master-slave consistency, need to add a series of verification mechanisms. And this approach reduces system performance.
Modify source code to support incremental synchronization
Redis Write AoF file, turn off the Redis rewrite aof file function, in order to avoid the file too large, you can implement the file segmentation function.
During low-peak business hours, the memory snapshot is generated and the point at which the snapshot is AoF is recorded.
When the slave is re-connected, the slave sends the synchronization command to the host, the host receives the command, sends the latest snapshot file to the slave, recovers from the snapshot file, and obtains the corresponding aof point of the snapshot, the slave will aof dots to the host, and the host synchronizes all the data operations after that point in the aof file to the slave. The effect of achieving an incremental synchronization.

Second, solve the expansion problem

The Redis author's idea is: Redis presharding (http://oldblog.antirez.com/post/redis-presharding.html)

Budget set Redis instances number, assuming number of instances N,n = number of machines * Number of Redis instances for single machine
A later extension requires only a subset of the Redis instances on the old machine to be migrated to the new machine for smooth expansion.
The migration steps are as follows:

The instances are created on the new machine, and each instance is set to the slave of the migrated instance.
After the master-slave replication is complete, the Setup program makes the new instance primary.
Stop the Old instance
After the above steps, the memory of the old machine becomes larger and the last memory is a Redis instance per machine.

According to the author of the article, a machine to launch multiple instances, in fact, does not consume too much resources, because Redis is light enough, and the other instances of one after another to rewrite the aof file or to generate memory snapshots, can reduce the memory footprint, without affecting external services

The solution of Redis master-slave replication problem and expansion problem