Detailed Redis master-slave synchronization

Master-Slave synchronization is the literal meaning of who-based, who as the deputy, synchronous, to form a synchronous effect. So how much do you know about Redis's master-slave synchronization knowledge? This paper mainly introduces the master-slave synchronous analysis of Redis, for reference only, hope to help everyone.

First, the principle of Redis master-Slave synchronization

1.1 Redis Master-Slave synchronization process

After you have configured master for the slave server connection, slave establishes a connection to master and then sends the Sync command. Whether the connection was first synchronized or after a disconnected connection, master initiates a background process to save the database snapshot to a file. At the same time, the master master process starts collecting new write commands and caches them. When the background process finishes writing the file, master sends the snapshot file to Slave,slave to save the file to disk, and then loads it into memory to restore the database snapshot to slave. Slave after the recovery of the snapshot file is complete, master forwards the cached commands to the Slave,slave Update memory database. Subsequent master-received write commands are sent to slave by starting the connection. Commands that synchronize data from master to slave and commands sent from client to master use the same protocol format. Slave can automatically reestablish the connection when master and slave's connection is broken. If master receives multiple slave simultaneous connection commands at the same time, it will only use the START process to write database mirroring and then send it to all slave.

1.2 Redis master-Slave synchronization features

Master-Slave synchronization has obvious distributed cache characteristics, mainly including these aspects:

1) A master can have multiple slave, a slave can also have multiple slave;
2) The slave can not only connect to the Master,slave but also connect other slave to form a tree-like structure;
3) Master-Slave synchronization does not block the primary, but it blocks slave. This means that when one or more slave and master synchronize data for the first time, master can continue to process requests from the client. Conversely, the slave will block requests that cannot process the client when the data is first synchronized;
4) Master-slave synchronization can be used to improve the scalability of the system, we can use a number of slave dedicated to client read requests, can also be used to do simple data redundancy or only on slave to persist in order to improve the overall performance of the cluster.

1.3 Redis Active Synchronization Setup method

There are two ways to do synchronization settings for a master-slave Redis server. Needs to be done on the slave server, specifying the REDIS server (either master or slave) that the slave needs to connect to.

1.3.1 Settings in the configuration file

Set in the configuration file (redis.conf) of the Redis server as slave.

Conf code

Slaveof 10.1.1.102 6379 #指定master的ip和端口

Obviously, this is a very simple setup, but the configuration file needs to be modified, and the configuration file is loaded when the server starts. So the server does not start can not be modified, the operation is not flexible.

This configuration is appropriate for the initial configuration at deployment time.

1.3.2 settings in the Redis client

This is illustrated by the Jedis, the official Redis recommendation, which is also based on Jedis. Here the Jedis object instance belongs to slave, and the parameter is the address and port of the server.

Java code

Slavejdedis.slaveof ("10.1.1.102", 6379); #指定master的ip和端口 Slavejdedis.slaveofnoone (); #取消指定master, I became a master.

The master and slave server's master-slave relationship can be easily modified by client-specified mode. So this is a good way to adjust the master and slave servers online as needed.

1.3.3 Current Master-Slave synchronization problems

Because master and slave servers are not automatically elected by Redis, Manual participation is required, so the master-slave switchover cannot be completed automatically. There is a problem, when and by whom to trigger the switchover. I looked under the client is not this ability, must be the words need to increase their own.

Jedis currently randomly chooses which Redis server to read, so for automatic distributed reads we need to do two packages for Jedis.

1) need to develop a mechanism to detect the working status of Master and slave as soon as possible;
2) need to define a master and slave automatic switching strategy;
3) You need to define a mechanism that can randomly read any redis server;

These features can be implemented on the client side, but the effect is not very good. If the server itself can support the more perfect, but from the introduction of the Redis official website, it seems that there is no mention of such a demand, there is no such planning.

Second, the Redis mainstream client introduction

On the official Redis website, the 5 redis Java client software is listed. The Jedis is the official Redis recommended Java client, which has been maintained and updated. Currently the latest stable version of the server is Redis2.4.17, the latest beta version of Redis 2.6.0 RC7.

2.1 Jedis

Jedis is the official version of the Java client recommended by Redis. Currently the latest version is Jedis 2.1.0-5, fully compatible with Redis 2.0.0. This client has been maintained and updated.

2.2 Jredis

Jredis has not been updated for a long time and is fully compatible with the Redis 2.0.0 version. This update is compatible with the latest Redis2.6.0 beta version, which was updated May ago.

2.3 Jdbc-redis

Jdbc-redis is the JDBC driver for Redis, a NoSQL database. can only be downloaded to the March 2009 release of the Jdbc-redis_0.1_beta version, is no longer maintained.

2.4 RJC

RJC provides an Apache DBCP-style connection pool. The update was discontinued 1 years ago and can be fully compatible with Redis 2.0.0.

2.5 Redis-protocol

This update is the fastest and most frequent and can be compatible with the latest Redis 2.6.0 version. However, it is positioned to fully support the REDIS protocol for more efficient data interaction with Redis servers. Therefore, the function of Redis server is not fully played.

2.6 Individual Java Client overall evaluation

On the whole, each client basically realizes the basic functions of the Redis Protocol protocol definition. The Redis-protocol update has the most complete support for the Redis protocol recently, and Jedis provides more configuration operations for Redis servers, which is most convenient to use. Other clients are rarely maintained, and the functionality is general.

If you want to extend the functionality of the client in small amounts, it is quickest to develop based on Jedis.

Based on Redis-protocol is the best choice if you want to limit the functionality of compatible and extended clients to the maximum.

Third, Redis master-Slave synchronization of the use of recommendations

Redis Master-Slave synchronization is not supported in all current Java clients. The main reason should be due to the implementation mechanism limitations of the Redis server itself. It is possible to do so, but the effect may be compromised.

3.1 By encapsulating the Jdedis to achieve

1) Add a management class, responsible for maintaining the Redis server Cluster server topology relationship;
2) added a monitoring class, responsible for monitoring and maintaining the server running status in Redis server cluster;
3) Add a master selection policy class that determines the switch timing for master and slave and chooses the most appropriate Redis server to act as master.
4) Add a proxy class to take over the current Jedis client's read and write operations to the Redis server. The application layer uses the Jedis client through the proxy class. The proxy class needs to ensure that the Redis server cluster is transparent to the application layer.