Redis cluster scenarios and implementations-YFK's column-Blog channel-csdn.net

"Announcements" blog system optimization upgradeUnity3d Learning, a step closer to VR developmentBole Recruiting started.virtual reality, a probe into what Redis cluster scheme and implementation 2014-08-30 17:2043035 People readComments (All)CollectionReportThis article has been included in: Classification: Redis (a) author similar article X

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.

Before doing a Redis cluster scheme, ran for six months, the line runs very stable
Almost can share with you the experience, the previous written an article of data online services, some of the exploration experience, can be used as a background reading

Application of our Redis cluster is mainly undertaken by the following services:
1. Real-time recommendation
2. User Portrait
3. Integrity Score Service

Cluster status cluster peak QPS around 1W, rw response time 999 line around 1ms
Entire cluster:
1. Redis node: 8 physical machines, each 128G RAM, 8 instance per machine
2. Sentienl:3 Virtual Machine

Cluster scenarios
Redis node consists of a set of Redis Instance, a set of Redis instatnce can have a master Instance, multiple slave Instance

The official Redis cluster is also in the beta version, see Redis cluster tutorial
During the research, I was particularly concerned about KEEPALIVED+VIP and twemproxy.
But finally decided to implement a set based on Redis Sentinel, the entire project is about 1 people/1.5 months

Overall design 1. Data hash distributed on different Redis instatnce
2. m/s switch with Sentinel
3. Write: Only master Instance is written, and the current master Instane is obtained from Sentinel
4. READ: Select a Redis instance read from the Redis node based on the weights, the failure/timeout polling other instance
5. With RPC service access, the RPC server side encapsulates the Redis client and the client is based on Jedis development
6. Batch Write/delete: Transaction not Guaranteed

Rediskey

public class Rediskey implements serializable{private static final long serialversionuid = 1l;//different familyprivate stri per business ng Family;private string key;......//The key that is physically stored on the Redis is the value after MurmurHash private String Makeredishashkey () {return String.valueof (Murmurhash.hash64 (makerediskeystring ()));} Reidskey is composed of Family.key, private String makerediskeystring () {return family + ":" + Key;} Returns the user's hash after rediskeypublic String Getrediskey () {return Makeredishashkey ();} .....}

Family exists to avoid multiple business key conflicts, define its own independent faimily for each business
For performance reasons, refer to the Redis storage design, where the key that is actually stored on the Redis is the value after the hash

Interfaces currently supported by the interface include:

public interface redisuseinterface{/** *  get value *  *  through Rediskey @param The key *  in  rediskey *           redis @return   *            successfully returned value, the query does not return null  */public string get (Final rediskey rediskey)  throws exception;/** *   Insert <k,v> data to redis *  *  @param  redisKey *            the redis key *  @param  value *            the redis value *  @return    *            successfully returned "OK", insert failed to return Null */public  string set (Final rediskey rediskey, final string value)  throws  exception;/** *  BatchWrite data to redis *  *  @param  redisKeys *            the redis key list *  @param  values *            the redis value list *  @return   *            successfully returned "OK", insert failed to return null */ Public string mset (final arraylist<rediskey> rediskeys, final arraylist <string> values)  throws Exception;/** *  Delete a data from Redis  *  *  @param  rediskey *           the redis  key *  @return   *           an  integer greater than 0 if one or more keys were  Removed 0 if none&nBsp;of the specified key existed */public long del (RedisKey redisKey)  throws Exception;/** *  Delete data in bulk from Redis  *  *  @param  rediskey *            the redis key *  @return   *            returns the number of successfully deleted data bars  */public  Long del (Arraylist<rediskey> rediskeys)  throws Exception;/** *  inserting <k,v > Data to redis *  *  @param  redisKey *            the redis key *  @param  value *            the redis value *  @return   *             successfully returned "OK", insert failed to return null */public string  setbyte (final redIskey rediskey, final byte[] value)  throws Exception;/** *  inserting <k,v > Data to redis *  *  @param  redisKey *            the redis key *  @param  value *            the redis value *  @return   *             successfully returned "OK", insert failed to return null */public string  setbyte (Final string rediskey, final byte[] value)  throws Exception;/ ** *  get value *  *  @param by Rediskey  redisKey *     The key *  @return in        redis   *             successful return of value, no query to return Null */public byte[] getbyte (final  rediskey rediskey) &Nbsp;throws exception;/** *  Setting the timeout time on the specified key  *  *  @param  rediskey *            the redis key *  @param  seconds *  the expire seconds *  @return   *            1:success, 0:failed */public Long  Expire (Rediskey rediskey, int seconds)  throws exception;}

Write Redis Process 1. Calculate Redis Key Hash value
2. Get Redis node number based on hash value
3. Obtain the Master of Redis node from Sentinel
4. Write data to Redis

Gets the write which Redis nodeint slot = Getslot (Keyhash); Redisdatanode Redisnode = Rdlist.get (slot);//write Masterjedissentinelpool JP = Redisnode.getsentinelpool (); Jedis JE = null;boolean Success = True;try {JE = Jp.getresource (); return Je.set (key, value);} catch (Exception e) {Log.err Or ("Maybe master is down", e); E.printstacktrace (); success = False;if (JE! = null) Jp.returnbrokenresource (JE); throw e;} Finally {if (Success && JE! = null) {Jp.returnresource (JE);}}

Read Process 1. Calculate Redis Key Hash value
2. Get Redis node number based on hash value
3. Select a Redis instatnce by weight
4. Polling Read

Gets the read which Redis nodeint slot = Getslot (Keyhash); Redisdatanode Redisnode = Rdlist.get (slot);//Select a job based on weight instatnceint rn = Redisnode.getworkinstance ();//poll int cursor = Rn;do {try {jedispool JP = redisnode.getinstance (cursor). GETJP (); return Getimpl (JP, key);} catch (Exception e) {Log.error ("Maybe a redis instance is down, slot: [" + slot + "]" + e); E.printstacktrace (); cursor = (cursor + 1)% Redisnode.getins Tancecount (); if (cursor = = RN) {throw e;}}} while (cursor! = RN);

When the weight calculation is initialized, a weight value is assigned to each Redis instatnce weight
Get the code for Redis instance by weight:

public int getworkinstance () {//Weight not defined, a redis instanceif (maxweight = = 0) {return (int) (Math.random () * is completely randomly selected) Random_size% redisinstancelist.size ());} Gets the random number int rand = (int) (Math.random () * random_size% maxweight); int sum = 0;//Select Redis instancefor (int i = 0; i < red Isinstancelist.size (); i++) {sum + = Redisinstancelist.get (i). Getweight (); if (Rand < sum) {return i;}} return 0;}

• Top

• 14

• Step

• 0

• Previous Article HBase atomicity Assurance

• Next Article Vector Clock Understanding

My Kind of article Redis (+) Http://blog.csdn.net

Reference Knowledge Base

For
More information, please refer to:

• Guess you're looking for

View Comments* The above user statements represent only their personal views and do not represent the views or positions of the CSDN website.

Redis cluster scenarios and implementations-YFK's column-blog channel-csdn.net