Java implementation of consistent hashing algorithm

The consistent hashing algorithm is a common algorithm in distributed system. For example, a distributed storage system, to store the data on a specific node, if the common hash method is used to map the data to specific nodes, such as Key%n,key is the key,n number of machines, if a machine joins or exits the cluster, then all data mappings are invalid , the data migration is done if it is a persistent store, and if it is a distributed cache, the other caches fail.

Therefore, a consistent hash algorithm is introduced:

The data is mapped to a large space using a hash function (such as MD5), as shown in the figure. When data is stored, a hash value is first obtained, corresponding to each position in the loop, such as K1 corresponds to the position shown in the figure, then a machine node B is located clockwise and the K1 is stored in the B node.

If the B node is down, the data on B falls on the C node, as shown in the following illustration:

In this way, only the C node will be affected, and the data a,d by the other nodes will not be affected. However, this will create an "avalanche" situation, that is, c node due to the commitment of the B-node data, so C node load will be higher, C node is very easy to also go down, so in turn, so that the whole cluster is hung up.

To this end, the concept of "virtual node" is introduced: that is, there are many "virtual nodes" in this ring, the data storage is to find a virtual node in the clockwise direction of the loop, each virtual node will be associated to a real node, the following figure is used: