How to use Shardedjedis (distributed) under Jedis (ii)

Described in the previous articleShardedjedisand demonstrates a simple example, in this article we introduce theShardedjedisThe principle.

1. Getting the data process

First , let's take a look at an inheritance relationship of Shardedjedis.

After reading the diagram, then we step by step with our code call to see, to our simplest Shardedjedis. Get ()Method for example:

This is called a getshard method, the parameter is our incoming key, and then returns a normal Jedis object, then this getshard is used to do what, as you may have guessed, this method will be based on our incoming key to do a consistent hash judgment, and then return to the key on the Redis instance of a Redis connection, the Redis connections returned by different keys may be different.

Entering the Getshard method, you will find that this implementation is implemented in the sharded class (see the class diagram above to find the top-level sharded Class), the code is as follows:

The above method is the layer call relationship, not in detail here, we mainly look at the red box that method implementation (the above nodes variable is a TreeMap type, algo is the Hashing type , that is, the hash algorithm used by the key Shard, which is briefly mentioned in the previous article), then the meaning of this code is probably guessed.

1. is to remove a partial view after a treemap greater than or equal to key Sortmap
2. The value of the first key-value pair is obtained in Sortmap, and then an S object is returned,
3. Then according to this S object, go to resources (resources = new Linkedhashmap<shardinfo<r>, r> ()) in Get one R Object

So what does this Sand R object represent? Look at the code below.

It can be concluded that S = jedisshardinfo, R = Jedis object, that is, the TreeMap stores the information of the server partition virtual node, The physical connection of the server is stored in the Linkedhashmap.

Jedisshardinfo Specific information is as follows: It contains some information about the Jedis server, and most importantly, there is a weight field in its parent class as the weight of this Jedis server.

OK, so we know that it is actually based on the information of the Jedis server to get a Jedis connection, back to the upper call.

We can comb this logic:

1. When we use Shardedjedis to check a key, first it will take this key to a hash algorithm
2. Based on this hash value and then go to TreeMap, find out which instance the key falls in and return the specific information for the Redis instance
3. Based on the Redis instance information, find this Jedis connection in a Linkedhashmap that holds the Jedis link and the instance information correspondence
4. Finally, the Jedis connection is returned, and the command operation of the object is executed (as in the case of a stand-alone operation)

So when is the physical connection of our nodes Server virtual node and the resources Server initialized, then continue to see

2.nodes and Resources initialization

We see how the sharded is constructed.

Here's a initialize method to see its implementation

Specific details will not say, according to the above, we know is in this side of the initialization, each server node using the hash algorithm divided into 160 virtual nodes (can be configured to divide the weight), stored in the TreeMap,

The information and physical connection of each server node are then saved in Linkedhashmap with key value pairs.

Then, through a series of searches, we find that Sharded's construction method is actually done at the time of our Jedispool.getresource () .

Nani? Initialize every jedispool.getresource () ? That will not cause very slow, actually do not worry, its bottom is the use of Commons.pool to connect the pool of some operations, according to our configured connection pool parameters to generate the corresponding connection and save, the details of which is very complex, Bloggers did not continue to delve into the implementation of the database connection pool is actually consistent.

3. Summary

shardedjedis Distributed concrete implementation ideas:

• Redis server node Partitioning: divide each server node into 160 virtual nodes using hash algorithm (can be configured to divide weights)

• Dividing the virtual nodes into TreeMap storage

• Physical connection to each Redis server in Linkedhashmap storage

• The same hash algorithm is used for key or Keytag, then the hash is obtained from TreeMap, and the nearest node is stored; When the hash value of key is greater than the maximum value of the virtual node hash, the first virtual node is saved.

• Sharded uses the hash algorithm: MD5 and MurmurHash two kinds, the default 64-bit MurmurHash algorithm;

Well, the approximate implementation as mentioned above, is the picture may be a bit messy, if you have any problems or found something wrong, please tell me!

How to use Shardedjedis (distributed) under Jedis (ii)