Redis the problem of master-slave reproduction and the solution of expansion problem _redis

First, to solve the problem of master and slave duplication

When using Redis as the storage engine, and the use of Redis read and write separation, from the machine as read, from the machine down or disconnected from the host need to reconnect the host, reconnect the host will trigger a full master-slave replication, this time the main opportunity to generate memory snapshots, the host can still provide services to the outside, But as a read from the machine, can not provide external services, if the amount of data, recovery time will be quite long. In order to solve the Redis master copy problem, there are two solutions:

Active replication
The so-called active replication, is the business layer double write multiple redis, avoid redis from the master-slave copy. But their own dry synchronization, will produce consistency problems, in order to ensure that the master-slave consistency, the need to join a series of validation mechanisms. And such practices can degrade system performance.
Modify source code to support incremental sync
Redis Write AoF file, turn off Redis rewrite aof file function, in order to avoid file too large, you can achieve the file segmentation function.
During the business low peak period, a memory snapshot is generated and the point at which the snapshot is AoF is recorded.
When you connect from the machine, send the sync command from the machine to the host, after receiving the command, the host sends the latest snapshot file to the machine, recovers from the snapshot file from the machine, and obtains the corresponding aof point of the snapshot, sends the AOF moan to the host from the machine, and the host synchronizes all the data operations from the AOF file to the machine. The effect of incremental synchronization is achieved.

Second, to solve the problem of expansion

Redis Author's idea is: Redis presharding (http://oldblog.antirez.com/post/redis-presharding.html)

Budget set Redis instances quantity, suppose instance quantity n,n = machine Number * Single Machine Redis instance number
Later extensions only need to migrate some Redis instances on the old machine to the new machine to achieve smooth expansion.
The migration steps are as follows:

Creates an instance on the new machine, and each instance is set to the from machine of the instance being migrated.
After the master-slave copy completes, the Setup program will take the new instance as the main.
Stop an old instance
After the previous steps, the memory of the old machine becomes larger, and the last memory is a Redis instance of each machine.

According to the author of the article, a machine to start multiple instances, in fact, does not consume too much resources, because the redis is light enough, another number of instances one by one to rewrite the aof file or generate memory snapshots, can reduce the memory footprint, without affecting the external services.