Application of consistent Hash algorithm in Memcached, hashmemcached

Application of consistent Hash algorithm in Memcached, hashmemcached
Preface

We should all know that Memcached can only be implemented on the client to implement distributed computing. Currently, consistent hash algorithms are widely used. the conventional method is to calculate the remainder of the hash value of the server and the total number of servers, that is, hash % N. The disadvantage of this method is that when the server is increased or decreased, there will be a large number of caches that need to be re-allocated and will result in uneven cache allocation (there may be many allocated by one server, but few others ).

Today, we share a consistent hash algorithm called ketama, which effectively suppresses uneven cache distribution through the concept of virtual nodes and different cache allocation rules, and minimize the redistribution of cache when servers increase or decrease.

Implementation

Suppose we now have N Memcached servers. If we use unified rules to perform the Set and Get operations on memcached. objects with different keys are evenly distributed and stored on these servers. The Get operation also extracts objects from the corresponding Server according to the same rules, in this way, isn't each Server a whole?

What is a rule?

As shown in, we now have five Memcached servers (A, B, C, D, E). we concatenate them to form A ring, each Server represents a point on the ring, and each point has a unique Hash value. There are 2 ^ 32 points on the ring.

How can we determine the specific distribution of each Server? Here, we use the "Ketama" Hash algorithm to calculate the Hash value of each Server. After obtaining the Hash value, we can correspond to the point on the ring. (the IP address of the Server can be used as the Key of the Hash algorithm .)

The advantage is that when I add Server F, then, I only need to re-allocate the hash value of the object between C and F from the original D to F. The cache on other servers does not need to be re-allocated, in addition, the new Server can also help buffer the pressure on other servers in a timely manner.

So far, we have solved the problem that a large number of caches need to be re-allocated when adding or removing servers. How can we solve the problem of uneven cache allocation? Because our server only occupies six points on the ring, and there are a total of 2 ^ 32 points on the ring, which can easily lead to many hot spots on a server, there are few hot spots on a certain platform.

The concept of "virtual node" solves the problem of load imbalance. by dividing each physical Server into N virtual Server nodes (N is usually determined based on the number of physical servers. Here, a better threshold is 250 ). in this way, each physical Server actually corresponds to N virtual nodes. when there are more storage points, the load of each Server needs to be balanced. just like the subway station exit, the more exits, the fewer congested each exit.

Code implementation:

// Save information about all virtual nodes. key: hash key and value of the virtual node: The Real server private Dictionary corresponding to the virtual node <uint, string> hostDictionary = new Dictionary <uint, string> (); // Save the hash key of all virtual nodes. private uint [] ketamaHashKeys = new uint [] {} has been sorted in ascending order. // Save the Real server Host address private string [] realHostArr = new string [] {}; // The number of virtual nodes corresponding to each Real server private int VirtualNodeNum = 250; public KetamaVirtualNodeInit (string [] hostArr) {this. realHostArr = hostArr; this. initVirtualNodes () ;}/// <summary> /// initialize the virtual node /// </summary> private void InitVirtualNodes () {hostDictionary = new Dictionary <uint, string> (); List <uint> hostKeys = new List <uint> (); if (realHostArr = null | realHostArr. length = 0) {throw new Exception ("cannot input an empty Server set") ;}for (int I = 0; I <realHostArr. length; I ++) {for (int j = 0; j <VirtualNodeNum; j ++) {byte [] nameBytes = Encoding. UTF8.GetBytes (string. format ("{0}-node {1}", realHostArr [I], j); // call the Ketama hash algorithm to obtain the hash key uint hashKey = BitConverter. toUInt32 (new KetamaHash (). computeHash (nameBytes), 0); hostKeys. add (hashKey); if (hostDictionary. containsKey (hashKey) {throw new Exception ("the same hash key is found when a virtual node is created. Check whether the same Server is passed in.");} hostDictionary. add (hashKey, realHostArr [I]) ;}} hostKeys. sort (); ketamaHashKeys = hostKeys. toArray ();}

 

Allocation Rules of consistent hash Algorithms

Now we know the Hash values of all virtual nodes. Now let's take a look at how to get an object to the Server or get the object Key.

When you Set an object, use the Key of the object as the Key of the "Ketama" algorithm. After calculating the Hash value, we need to perform the following steps.

1: Check whether the virtual node has the same Hash value as the current object. If yes, directly store the object to the node with the same Hash value. The subsequent steps will not proceed.

2: if not, find the first node that is larger than the Hash value of the current object (the Hash value of the node is sorted in ascending order, and the circle is arranged in clockwise order ), the node closest to the object, and then stores the object to the node.

3: If the Server with a larger Hash value than the object is not found, it proves that the Hash value of the object is between the last node and the first node, that is, between E and A on the ring. in this case, the object is directly stored in the first node, that is,.

Code implementation:

/// <Summary> /// obtain the actual Server Based on the hash key /// </summary> /// <param name = "hash"> </param>/ // <returns> </returns> public string GetHostByHashKey (string key) {byte [] bytes = Encoding. UTF8.GetBytes (key); uint hash = BitConverter. toUInt32 (new KetamaHash (). computeHash (bytes), 0); // find the Server with the same hash value as the current one. int I = Array. binarySearch (ketamaHashKeys, hash); // if I is less than zero, it indicates that no virtual node with the same hash value if (I <0) {// continue to perform bitwise complementing for I, Obtain the first virtual node in the array that is greater than the current hash value ~ I; // If I after bitwise completion is greater than or equal to the size of the array, it indicates that no virtual node in the array is greater than the current hash value. // at this time, the first server if (I> = ketamaHashKeys is taken directly. length) {I = 0 ;}}// return the actual server host address return hostDictionary [ketamaHashKeys [I] According to the hash key of the virtual node;}

 

Get an object. It also uses the "Ketama" algorithm to calculate the Hash value, then searches for nodes like the Set process, and then retrieves the object directly.

So what does Ketama look like? Let's look at the code implementation.

/// <Summary> // Ketama hash encryption algorithm // For details about HashAlgorithm, see the link to MSDN /// http://msdn.microsoft.com/zh-cn/library/system.security.cryptography.hashalgorithm%28v=vs.110%29.aspx /// </summary> public class KetamaHash: hashAlgorithm {private static readonly uint FNV_prime = 16777619; private static readonly uint offset_basis = 2166136261; protected uint hash; public KetamaHash () {HashSizeValue = 32;} public override void Initialize () {hash = offset_basis;} protected override void HashCore (byte [] array, int ibStart, int cbSize) {int length = ibStart + cbSize; for (int I = ibStart; I <length; I ++) {hash = (hash * FNV_prime) ^ array [I] ;}} protected override byte [] HashFinal () {hash + = hash <13; hash ^ = hash> 7; hash + = hash <3; hash ^ = hash> 17; hash + = hash <5; return BitConverter. getBytes (hash );}}

Test performance

Finally, I will refer to the algorithms written by BeitMemcached and the old generation (Discuz! For more information, see SPYMemcached.

The source code is downloaded later.

Result: The time for searching for 100 keys is more than times faster than that of the old version, but the load balancing is worse.

Test data:

1: The real servers are all five

2: randomly generate string keys (the generation method takes the old generation directly)

3: 250 virtual nodes

My version:

Old Generation version: