Consistent Hash Algorithm and Its Application in distributed systems are carried by codinglabs)

Summary

This article will introduce the consistent hash algorithm (Consistent hashing) and Its Application in distributed systems from the actual application scenarios. First, this article will describe a problem scenario that is often encountered in daily development, so as to introduce the consistent hash algorithm and how this algorithm solves this problem. Next, we will describe this algorithm in detail, we also discuss topics related to this algorithm application, such as virtual nodes.

Distributed cache Problems

Suppose we have a website. Recently we found that as the traffic increases, the server pressure is getting higher and higher, and the method of Directly Reading and Writing databases is not very powerful, so we want to introduce memcached as a cache mechanism. Now we have three servers available as memcached servers, as shown in.

Obviously, the simplest policy is to randomly send each memcached request to a memcached server. However, this policy may cause two problems: one is that the same data may be stored on different machines, resulting in data redundancy. The other is that some data may have been cached but not accessed, because it is not guaranteed that all accesses to the same key will be sent to the same server. Therefore, random policies are inefficient at both time efficiency and space efficiency.

To solve the above problem, you only need to do the following: ensure that access to the same key will be sent to the same server. Many methods can achieve this. The most common method is to calculate the hash. For example, you can use the following algorithm to calculate the hash value for each access:

H = hash (key) % 3

Hash is a hash ing function from a string to a positive integer. In this way, if the memcached server numbers are 0, 1, and 2 respectively, then the server number H can be calculated based on the formula and key and then accessed.

This method solves the two problems mentioned above, but there are some other problems. If the preceding method is abstracted, it can be considered as follows:

H = hash (key) % N

This formula calculates the server to which each key request should be sent, where N is the number of servers and the server is numbered by 0-(N-1.

The problem with this algorithm is fault tolerance and poor scalability. Fault Tolerance means whether the entire system can run correctly and efficiently when one or more servers in the system become unavailable. extensibility means that when a new server is added, whether the entire system can run correctly and efficiently.

Assume that one server is down. To fill in the vacancy, remove the server from the number list, the following server moves one digit in order and the number value is reduced by one, then each key is recalculated by H = hash (key) % (N-1); similarly, if a new server is added, the hash value is re-calculated based on H = hash (key) % (n + 1), although the number of the original server does not need to be changed. Therefore, if a server changes, a large number of keys will be relocated to different servers, resulting in a large number of cache hits. This situation is very bad in distributed systems.

A well-designed distributed Hash scheme should have good monotonicity, that is, the increase or decrease of service nodes will not cause a large number of hash relocation. Consistent hash algorithm is such a hash scheme.

Brief description of consistent Hash Algorithm

Consistent hashing was first proposed in the paper consistent hashing and random trees: distributed caching protocols for relieving hot spots on the World Wide Web. In short, consistent hash organizes the entire hash value space into a virtual ring. For example, assume that the value space of a hash function H is 0-2.^{32-1 (that is, the hash value is a 32-bit unsigned integer). The entire hash space ring is as follows:}

The entire space is organized clockwise. 0 and 2^{32-1 in the middle of the zero point.}

Next, use h to hash each server. You can select the IP address or Host Name of the server as the key word to hash the server, so that each machine can determine its position in the hash ring, here, we assume that the IP addresses of the above three servers are hashed and the location in the ring space is as follows:

Next, use the following algorithm to locate data access to the corresponding server: Calculate the hash value h using the same function h for the data key, and determine the location of the data on the Ring Based on H, the first server is the server to be located.

For example, we have four data objects, A, B, C, and D. After hash calculation, the location in the ring space is as follows:

According to the consistent hash algorithm, data a is set to Server 1, data D is set to Server 3, and data B and C are set to server 2 respectively.

Fault Tolerance and Scalability Analysis

The following describes the fault tolerance and scalability of the consistent hash algorithm. Assume that server 3 is down:

We can see that a, c, and B are not affected at this time. Only node D is relocated to server 2. Generally, in a consistent hash algorithm, if a server is unavailable, then, the affected data is only the data between the server and the first server in the ring space (that is, the first server encountered walking in the counterclockwise direction). Other data will not be affected.

Next we will consider another situation, if we add a server memcached Server 4 to the system:

At this time, a, d, and C are not affected. Only B needs to be relocated to the new server 4. Generally, in a consistent hash algorithm, if a server is added, the affected data is only sent from the new server to the first server in the ring space (that is, the first server encountered walking in a counterclockwise direction) data, other data is not affected.

To sum up, the consistent hash algorithm only needs to relocate a small part of data in the ring space for node increase and decrease, which has good fault tolerance and scalability.

Virtual node

When the number of service nodes is too small, the consistent hash algorithm is prone to data skew due to uneven node segments. For example, there are two servers in our system, and the ring distribution is as follows:

At this time, a large amount of data will inevitably be concentrated on Server 1, and only a very small amount will be located on Server 2. To solve this data skew problem, the consistent hash algorithm introduces a virtual node mechanism, that is, multiple hashing is calculated for each service node, and one service node is placed for each computing result location, it is called a virtual node. You can add a number after the Server IP address or host name. For example, in the above situation, we decided to calculate three virtual nodes for each server, therefore, memcached Server 1 #1, memcached Server 1 #2, memcached Server 1 #3, memcached Server 2 #1, and memcached Server 2 #2 "," memcached Server 2 #3 "hash value, as a result, six virtual nodes are formed:

At the same time, the data locating algorithm remains unchanged, but there is only one step more ing between virtual nodes and actual nodes, for example, data from three virtual nodes, memcached Server 1 #1, memcached Server 1 #2, and memcached Server 1 #3, are all located on Server 1. This solves the problem of data skew when there are few service nodes. In practical applications, the number of virtual nodes is usually set to 32 or greater, so even a few service nodes can achieve relatively even data distribution.

Summary

Currently, consistent hashing is basically the standard configuration of distributed system components. For example, various clients of memcached provide built-in consistent hashing support. This article briefly introduces this algorithm. For more information, see consistent hashing and random trees: distributed caching protocols for relieving hot spots on the World Wide Web. a c language version is provided for your reference.