Consistent hash algorithm (consistent hash)

Distributed Cache issues

Suppose we have a website, recently found that as traffic increases, the server pressure is increasing, the way to read and write the database directly is not very good, so we want to introduce memcached as a caching mechanism. Now we have three machines available as memcached servers, as shown in.

Obviously, the simplest strategy is to randomly send each memcached request to a memcached server, but this strategy can pose two problems: one is that the same data may be present on different machines, resulting in data redundancy, and second, it is possible that some data has been cached but access is not hit , because all access to the same key is not guaranteed to be sent to the same server. Therefore, the stochastic strategy is very bad both in terms of time efficiency and space efficiency.

To solve these problems, just do the following: Ensure that access to the same key is sent to the same server. There are many ways to do this, and the most common method is to compute the hash. For example, for each access, the hash value can be computed as follows:

h = Hash(key) % 3

Where hash is a hash mapping function from string to positive integer. Thus, if we have memcached server numbered 0, 1, 2, then we can calculate the server number H according to the above and key, then go to access.

Although this approach solves the two problems mentioned above, there are some other problems. If the above method is abstracted, it can be considered by:

h = Hash(key) % N

This calculation calculates which server the request for each key should be sent to, where N is the number of servers, and the server is numbered by 0– (N-1).

The problem with this algorithm is that fault tolerance and extensibility are not good. Fault tolerance refers to whether the whole system can run correctly and efficiently when one or several servers in the system become unavailable, and extensibility refers to whether the whole system can run correctly and efficiently when a new server is added.

Now assume that a server is down, so in order to fill the vacancy, to remove the outage server from the numbered list, the following server in order to move forward one and the number of the value minus one, each key will be h = Hash(key) % (N-1) recalculated; Similarly, if a new server, although the original server number does not change, However, you will press the h = Hash(key) % (N+1) hash value to recalculate. Therefore, once there is a server change in the system, a large number of keys will be relocated to different servers resulting in a large number of cache misses. This is a very bad situation in a distributed system.

A well-designed distributed hashing scheme should have good monotonicity, that is, the increase or decrease of service nodes will not cause a lot of hash relocation. A consistent hashing algorithm is one such hash scheme.

Consistent hashing algorithmAlgorithm Brief

In simple terms, a consistent hash organizes the entire hash value space into a virtual ring, such as assuming that the value space of a hash function h is 0-(2^32)-1 (that is, the hash value is a 32-bit unsigned shape), and the entire hash space loop is as follows:

The entire space is organized in a clockwise direction. 0 and (2^32)-1 coincide in the direction of 0 points.

The next step is to use H to make a hash of each server, select the server's IP or hostname as the keyword hash, so that each machine can determine its location on the Hashi, this assumes that the above three servers using the IP address hash after the location of the ring space is as follows:

Next, use the following algorithm to locate the data access to the appropriate server: The data key using the same function H to calculate the hash value H, by the H to determine the position of this data on the ring, from this position along the ring clockwise "walk", the first server encountered is the server it should be located.

For example, we have a, B, C, D four data objects, after hashing, the position on the ring space is as follows:

Based on the consistent hashing algorithm, data A is set to server 1, D is set to server 3, and B and C are set to server 2, respectively.

Fault Tolerance and Scalability analysis

The following is an analysis of the fault tolerance and extensibility of the consistent hashing algorithm. Now assume that server 3 is down:

You can see that a, C, B are not affected at this point, only the D node is relocated to server 2. In general, in a consistent hashing algorithm, if a server is unavailable, the affected data is only data between the server and the previous server in its ring space (that is, the first server encountered in the counterclockwise direction), and the others are unaffected.

Consider the other case if we add a server to the system memcached Server 4:

At this point A, D, C are unaffected and only B needs to be relocated to the new server 4. In general, in a consistent hashing algorithm, if you add a server, the affected data is only the data between the new server and the previous server in its ring space (that is, the first server encountered in the counterclockwise direction), and the others are unaffected.

To sum up, the consistency hashing algorithm can only reposition a small subset of data in the ring space for the increment and decrease of the nodes, which has good fault tolerance and expansibility.

Virtual node

The consistency hashing algorithm is too young for the service node, and is prone to data skew due to uneven node division. For example, there are two servers in our system, and their rings are distributed as follows:

This inevitably results in a large amount of data being concentrated on server 1, and only a very small number will be located on server 2. In order to solve this data skew problem, the consistent hashing algorithm introduces the virtual node mechanism, that is, to compute multiple hashes for each service node, and to place a service node, called a virtual node, for each computed result location. This can be done by adding numbers to the server IP or host name. For example, we decided to compute three virtual nodes for each server, so we can calculate "Memcached server 1#1", "Memcached server 1#2", "Memcached server 1#3", "Memcached Server 2#1 "," Memcached server 2#2 "," Memcached server 2#3 "hash values, resulting in six virtual nodes:

Consistent hash algorithm (consistent hash)