Summary of consistency hash and crush algorithm

The same: All addresses how data is stored and routed in the data cache system.

Differences: The difference between the virtual node and the physical node mapping method is different

Because the generic hash function returns an int (32bit) type of hashcode. Therefore, you can represent the hashcode that the hash function can return as a range of 0---(2^32)-1 rings

Data and nodes use the same hash function to ensure that the data and nodes are mapped to the same hash space. So, in a clockwise direction, the data is stored in the clockwise direction of the machine in which it is located. This is how the consistent hashing algorithm allocates data!

Physical node: Use IP or a unique machine to identify the key to the ring, hash algorithm CRC32, md5,sha-1 and other cryptographic algorithms or more efficient murmurhash non-cryptographic algorithms.

Virtual node: Using ip+ "#" +num ("192.168.0.1#1"), different virtual nodes (I different) have different hash values, but all correspond to the same actual machine node

Data: Using the object name (full path name) is called key, the algorithm is also MD5

Equalization: Introduce a virtual node (equivalent to the PG in CEPH) to ensure that the number of replicas * Actual number of nodes, replicas must remain in different physical nodes

Virtual node is the actual node (machine) in the hash space of the replica (replica), a real node (machine) corresponding to a number of "virtual node", the corresponding number has become "Replication Number", "Virtual node" in the hash space in the hash value.

The hash calculation of "virtual node" can be based on the IP address of the corresponding node plus the number suffix. For example, assume that the IP address of the NODE1 is 192.168.1.100. Before introducing "Virtual node", calculate the hash value of cache A: hash ("192.168.1.100"), after introducing "virtual node", calculate the hash value of "virtual section" point Node1-1 and Node1-2: hash ("192.168.1.100#1") ; Node1-1hash ("192.168.1.100#2"); Node1-2

Monotonicity: The method of mapping into the hash ring is guaranteed.

Consistent hashing takes the following approach: Introduce the concept of a ring, such as the first diagram above. The machine is mapped to this ring first, and then the data is mapped to the ring through the same hash function, and the data is stored on the machine where it moves clockwise. The decoupling between the data and the number of machines is realized by the ring as the intermediary. Thus, when the number of machines changes, it will only affect the data storage of the neighboring machines of the ring where the machine is added or removed, while the data on other machines is not affected.

The PG value setting in Ceph Reference: 2 of the second side.

The Learning Ceph book also has the same computational formula:

/max_replication_count)/Pool_count

The result of the settlement is taken up to the value of the n-th square of the 2. For example, the total number of OSD is 160, the number of copies of the number of 3,pool is also 3, then the result is calculated according to the above formula is 1777.7. Take 2 of the n-th is 2048, then the number of PG allocated per pool is 2048.

Virtual node settings in a consistent hash algorithm:

Number of current nodes * copies

Routing process/algorithm for data: Ceph called crush

Object-->virtual Node-->machine

For a real machine node, the corresponding Numberofreplicas virtual node,

Different virtual nodes (I different) have different hash values, but all correspond to the same actual machine node,

The virtual node is generally distributed evenly over the ring, and the data is stored in a clockwise virtual node,

Gets a nearest clockwise node, taking a hash based on the given key,

Then get the actual node corresponding to the nearest virtual node in the clockwise direction,

View the hashcode values generated by the MD5 algorithm---represent the individual virtual node locations throughout the hash ring

Code reference: the first link below.

There are two key problems in the implementation of the consistent hash algorithm, one is the choice of data structure for node storage and lookup, and the other is the choice of node hash algorithm.

First, we talk about the data structure used for storing nodes in the consistency hash algorithm. By understanding the principle of consistent hash, we know that nodes can be imagined to be stored in a circular data structure (such as), nodes A, B, C, d by hash value in the ring distribution is ordered, that is, the node can be stored in a hash value in an orderly queue. As shown, when a hash value of -2^20 's request point P finds a routing node, the consistent hash algorithm routes to the first node in the clockwise direction of the hash value (B), which is equivalent to the order structure of the storage node, by the key value of the query to find the smallest node larger than the key value. Therefore, we should choose a data structure, it should be efficient to support nodes to delete frequently, also must have the ideal query efficiency. Then, the red and black trees can meet these requirements. A red-black tree is a binary lookup tree that is approximately balanced, because operations such as inserting, deleting, and finding the worst-case time for a value are required to be proportional to the height of the tree, a theoretical upper limit on the height that allows the red and black trees to be efficient in the worst case, and different from the normal two-fork lookup tree. Therefore, we choose to use the red and black tree as the node storage structure, in addition to the need to implement basic red and black tree insertion, deletion, find the basic function, we should also add another query lookup function, to find the smallest node larger than the key.

C++map internal implementation is the red-black tree, that is, the direct use of stl::map can be.

Code reference:

Consistency hashing algorithm and C + + implementation

Reference:

Consistency hashing algorithm Learning and Java Code Implementation analysis

A little progress every day--five minutes understanding the consistent hashing algorithm (consistent hashing)

Consistent hash algorithm

Summary of consistency hash and crush algorithm