Redis cluster solution and implementation

Previously, I made a redis cluster solution, which lasted less than half a year and was stable online.
I can share my experiences with you. I wrote an article about data online service exploration experience, which can be used as a background.

Our redis cluster mainly undertakes the following services:
1. Real-time recommendation
2. User Profiles
3. Integrity score Service

Cluster status: the cluster peak QPS is about 999, And the RW response time is about lines in 1 ms.
Entire cluster:
1. redis node: 8 physical machines; GB memory each; 8 instances on each machine
2. sentienl: 3 VMS

Cluster Solution
Redis node is composed of a group of redis instances. A group of redis instatnce can have one master instance and multiple slave instances.

The official redis cluster is still in beta version. For details, refer to redis cluster tutorial.
During the investigation, I paid special attention to keepalived + VIP and twemproxy.
However, we finally decided to implement a set based on redis Sentinel. The entire project was about one person/one month and a half.

Overall Design 1. Data hash is distributed on different redis instatnce
1. Sentinel is used for switching M/s.
2. Write: only write the master instance to get the current master instane from sentinel.
3. Read: select a redis instance from the redis node based on the weight to read, and poll other instances if the instance fails or times out.
4. access through the RPC service. The RPC server encapsulates the redis client, which is developed based on Jedis.
5. Batch write/delete: transactions are not guaranteed

Rediskey

Public class rediskey implements serializable {Private Static final long serialversionuid = 1l; // familyprivate string family of different businesses; private string key ;...... // The Key physically stored on redis is the value private string makeredishashkey () {return string after murmurhash. valueof (murmurhash. hash64 (makerediskeystring ();} // reidskey by family. private string makerediskeystring () {return family + ":" + key;} // returns the user's rediskeypublic string getrediskey () after hash () {return makeredishashkey ();}.....}

To avoid conflicts between multiple business keys, define independent faimily for each business.
For performance considerations, refer to the redis storage design. The key actually stored on redis is the value after hash.

Currently, the following interfaces are supported:

Public interface redisuseinterface {/*** get value through rediskey ** @ Param rediskey * Key * @ return * In redis returns value, if the query fails, null */Public String get (final rediskey) throws exception is returned./*** insert <K, v> data to redis ** @ Param rediskey * The redis key * @ Param value * The redis value * @ return * success return "OK ", if insertion fails, null */Public String set (final rediskey, final string value) throws exception is returned; /*** write data in batches to redis ** @ Param rediskeys * The redis key list * @ Param values * The redis value list * @ return * returns "OK ", if insertion fails, null */Public String mset (final arraylist <rediskey> rediskeys, final arraylist <string> values) is returned. Throws exception; /*** delete a piece of data from redis ** @ Param rediskey * The redis key * @ return * an integer greater than 0 if one or more keys were removed 0 if none of specified key existed */public long del (rediskey) throws exception; /*** batch delete data from redis ** @ Param rediskey * The redis key * @ return * returns the number of successfully deleted data records */public long del (arraylist <rediskey> rediskeys) throws exception;/*** insert <K, v> data to redis ** @ Param rediskey * The redis key * @ Param value * The redis value * @ return * success return "OK ", if insertion fails, null */Public String setbyte (final rediskey, final byte [] value) throws exception is returned./*** insert <K, v> data to redis ** @ Param rediskey * The redis key * @ Param value * The redis value * @ return * success return "OK ", if insertion fails, null */Public String setbyte (final string rediskey, final byte [] value) throws exception is returned; /*** get value through rediskey *** @ Param rediskey * Key * @ return * In redis returns value, if the query fails, null */Public byte [] getbyte (final rediskey) throws exception is returned; /*** set the timeout time on the specified key ** @ Param rediskey * The redis key * @ Param seconds * The expire seconds * @ return * 1: Success, 0: failed */public long expire (rediskey, int seconds) throws exception ;}

Write redis process 1. Calculate the redis key Hash Value
2. Get the redis node number based on the hash value
3. Obtain the redis node master from sentinel
4. Write Data to redis

// Obtain the redis nodeint slot = getslot (keyhash); redisdatanode redisnode = rdlist. get (slot); // write masterjedissentinelpool JP = redisnode. getsentinelpool (); Jedis je = NULL; Boolean success = true; try {je = JP. getresource (); Return je. set (key, value);} catch (exception e) {log. error ("Maybe master is down", e); E. printstacktrace (); success = false; If (Je! = NULL) JP. returnbrokenresource (JE); throw E;} finally {If (success & je! = NULL) {JP. returnresource (JE );}}

Read process 1. Calculate the redis key Hash Value
2. Get the redis node number based on the hash value
3. Select a redis instatnce based on the weight
4. Round-Robin reading

// Obtain which redis nodeint slot to read = getslot (keyhash); redisdatanode redisnode = rdlist. get (slot); // select a work instatnceint Rn = redisnode according to the weight. getworkinstance (); // round-robin int cursor = rn; do {try {jedispool JP = redisnode. getinstance (cursor ). getjp (); Return getimpl (JP, key);} catch (exception e) {log. error ("maybe a redis instance is down, slot: [" + slot + "]" + E); E. printstacktrace (); cursor = (cursor + 1) % redisnode. g Etinstancecount (); If (cursor = rn) {Throw e ;}} while (cursor! = Rn );

During Weight Calculation initialization, each redis instatnce will be assigned a weight value weight
Obtain the redis instance code based on the weight:

Public int getworkinstance () {// If weight is not defined, a redis instanceif (maxweight = 0) {return (INT) (math. random () * random_size % redisinstancelist. size ();} // obtain the random number int Rand = (INT) (math. random () * random_size % maxweight); int sum = 0; // select redis instancefor (INT I = 0; I <redisinstancelist. size (); I ++) {sum + = redisinstancelist. get (I ). getweight (); If (RAND <sum) {return I ;}} return 0 ;}

Redis cluster solution and implementation