Consistent ha system algorithm

Usage Scenarios

Now we assume that there are 100 Redis data servers, a data 101 comes in, hash formula hash (i) &100, calculate the server stored, assuming hash (i) = i, then the data is hashed to the server labeled 1, Then this time the server has a new one, and then hash formula hash (i)%101, this time when the request to access Data 101, was assigned to server No. 0, but in fact, this time the data is in the server number 1th.

So this time a lot of data fails (not accessible).

So this time, we assume that the new server, if it is persistent storage, we can let the server cluster to re-hash the data, data migration, and then restore, but this time means that each time the server is increased, the cluster needs a lot of communication, data migration, This overhead is very large. If it's just a cache, then the cache will fail. So what happens at this time?

As we can see, the key problem is that when the number of servers changes, we can ensure that the old data can be calculated according to the old algorithm, and the new data can calculate the server of the data according to the new hashing algorithm.

For example, we have ABCD four servers, these four servers are assigned to a ring in 0~232, such as 0~230 storage on a server, 230 +1~231 storage on the B server ..... CDs are evenly divided in this way. Our hash space is also zoned 0~232, and then the data comes in to 232 modulo, get a value K1, we according to K1 in the position of the ring, get assigned to the server, K1 is assigned to the B server. At this time, we have a server B failed.

We can see that if the B fails, then if there is persistent storage, data recovery is required, the data of B is migrated to C, and there is no need to make any changes to the data that was hashed in a and d. Similarly, if we are adding a new server, it is only necessary to migrate one part of the data to the new server.

The consistent hash algorithm reduces the changes in the data mapping relationship and does not bring about global changes like hash (i)%n

And there is a benefit, assuming that we use the UID as the hash range (that is, 232 above), then assume that some UID access is very frequent, and this part of the UID is concentrated on the B server, then the load of B is much higher than the other servers. That's the problem with hot data. At this point we can add servers to the UID space where B is located, reducing the pressure on B.

In fact, there is a better solution: virtual node.

The above-mentioned situation is that the real server is hashed on 232 as a node. We assume that only 4 servers (such as), and then a above the hot data, the result of a hang up, and then do data recovery, a data migration to B, then B need to withstand the a+b data, also can not bear, also hung .... Then continue the CD all hung up. This has resulted in

Avalanche effect.

An analysis of the causes of the avalanche effect above:
If there is no hotspot data, the pressure on each machine is M/2 (assuming the maximum load capacity of each machine is m), it would not have been a problem, but, at this time a server because of the hotspot data is hung, and then a's data migration to B, Cause B to be subjected to the pressure to M (also regardless of hot data access pressure), so this failure B is required, and then c at least to withstand the pressure of 1.5M .... Then we hang together ...
So we can see through the above, the reason why we hang together, because if a machine is hung, then its pressure is all assigned to a machine, resulting in an avalanche.

If we hang up, the data is evenly distributed to the BCD, and each machine is subjected to M/6 pressure, and then everyone will not hang up (regardless of hot data).

The virtual node is introduced here,

The space on the ring is divided into 8 parts, then a storage A1 and A2 ...
At this point, if the a server is hung up, the access pressure is assigned to C2 and D1, which is the C and D servers, rather than the front, all assigned to B.

The introduction of virtual nodes is mainly due to the ability to drain pressure to different servers if one server is hung up. Summary: The consistent hash Algorithm (DHT) solves the problem of data hashing caused by increasing or decreasing the server by reducing the scope of the impact, thus solving the load balance problem in distributed environment, if there is hot data, then by adding nodes, divide the hot zone and distribute the pressure to other servers. The state of the load balancer is re-reached.

Tair load Balancing is the consistent hash algorithm used ~ ~ ~
Consistent hash algorithm is widely used in distributed environment, as long as it involves the load balancing problem of distributed storage, the consistent hash is a good solution.

Consistent ha system algorithm