Consistent hash data

I recently studied consistent hash. This section describes its paper (consistent
Hashing and random trees: distributed caching protocols for relieving hot spots on the World Wide Web bydavid
Karger et al) appeared 10 years ago, but it was not until recently that more and more services began to use consistent hash. These services include Amazon's dynamo and memcached (to last. FM salute ). So what is consistent?
What about hash? Why do you need to pay attention to it?

The requirement of consistent hash comes from some restrictions encountered when running a cache cluster (such as Web Cache. If you have a cluster composed of N cache machines, the most common load balance mode is to place the incoming object o on the server numbered Hash (o) mod n. You will find this solution beautiful, until one day, you have to add or remove some cache machines for various reasons. At this time, the number of machines in the cluster has changed, each object is hashed to a new machine. This will be a disaster, because the server that actually stores the content will be dragged down by requests from the cache cluster. At this time, the entire system looks like there is no cache. This is why everyone cares about consistent.
Hash, because you need to use it to avoid system corruption.

If so, when a cache machine is added to the cluster, the machine only reads the expected objects from other cache machines, when a cache machine is removed from the cluster, it is best to allocate the objects it caches to other cache machines (without moving more data ). This ideal situation is what consistent hash pursues and implements: if possible, always allocate the same group of objects to the same machine.

The basic idea behind the consistent hash algorithm is to use the same hash function for the object and cache machine. The advantage of this operation is that the cache machine can be mapped to an interval, which contains the hash value of a certain number of objects. If a cache machine is removed, the interval mapped to it is managed by a cache machine adjacent to it.

Description

Let's take a deeper look at consistent hash. Hash maps the object and cache to a value range. Java programmers should be familiar with hash-The hashcode method of each object will return^{31,
231-1. We map the beginning and end of the value range to a ring. Describes a group of objects (1,
2, 3, 4) and a group of cache (A, B, C) are mapped to the hash ring respectively. (Image Source: Web
Caching with consistent hashing by David kargerEt
Al)}

Figure 1

To determine the cache where an object will be cached, we start clockwise from this object and know that we encounter a cache point. In this case, we can see that object 1 and 4 are cache A, object 2 is cache B, and cache C caches object 3. When cache C is removed, what will happen? In this case, object 3 is cached by cache A, and no other objects need to be moved. If 2 and D are added to the cache cluster, d caches object 3 and 4 and leaves object 1 to.

Figure 2

Everything is fine, except that the spacing assigned to each cache is too random, so that the object allocation is extremely uneven. To solve this problem, we introduce the concept of "virtual nodes": Each cache has multiple copies in the hash ring, that is, every time we add a cache, multiple points will be added for the cache on the ring.

The following code creates a simulation experiment to store 10,000 objects to 10 caches. You will see the impact of virtual nodes in the plot diagram below. On the X axis is the number of copies of each cache (logarithm scale ). When the value of X is small, we can see that the distribution of objects in caches is unbalanced (the Y axis represents the standard deviation of objects distribution in caches as a percentage ). As the cache replica increases, the distribution of objects tends to be more balanced. This experiment shows that the replica of each cache is about 5%-10%, which can balance the distribution of objects (standard deviation is between and)

Experiment result

Implementation

The following is a simple implementation of Java. To make the consistent hash effect obvious, it is very important to use a mix hash function. Most implementations of the hashcode method of objects in Java do not provide good mix performance. Therefore, we provide a hashfunction interface to facilitate custom hash functions. We recommend MD5.

import java.util.Collection;import java.util.SortedMap;import java.util.TreeMap;public class ConsistentHash<T> { private final HashFunction hashFunction; private final int numberOfReplicas; private final SortedMap<Integer, T> circle = new TreeMap<Integer, T>(); public ConsistentHash(HashFunction hashFunction, int numberOfReplicas,     Collection<T> nodes) {   this.hashFunction = hashFunction;   this.numberOfReplicas = numberOfReplicas;   for (T node : nodes) {     add(node);   } } public void add(T node) {   for (int i = 0; i < numberOfReplicas; i++) {     circle.put(hashFunction.hash(node.toString() + i), node);   } } public void remove(T node) {   for (int i = 0; i < numberOfReplicas; i++) {     circle.remove(hashFunction.hash(node.toString() + i));   } } public T get(Object key) {   if (circle.isEmpty()) {     return null;   }   int hash = hashFunction.hash(key);   if (!circle.containsKey(hash)) {     SortedMap<Integer, T> tailMap = circle.tailMap(hash);     hash = tailMap.isEmpty() ? circle.firstKey() : tailMap.firstKey();   }   return circle.get(hash); }}

 
The above Code uses an integer sorted map to represent hash circle. WhenConsistentHashWhen a node is created, it is added to the Circle map (numberOfReplicasControl ). The position of each replica is determined by the node name plus the hash value corresponding to a digital suffix. Find the node (getMethod), we put the object's hash value into the map for search. In most cases, a node does not exactly overlap with this object (even if each node has a certain number of replica, the hash value space is much larger than the number of nodes ).tailMapFind the next key in the map. If the tail map is empty, we turn around and find the first key in the circle.
Use
So how should you use the consistent hash? Generally, you can use some libraries instead of writing code yourself. For example, the memcached-distributed memory cache system mentioned above already has a client that supports consisitent hash. Richard from last. fm
Ketama implemented by Jones is the first, and now Dustin
Java implementation contributed by sallings. It is interesting that only the client needs to implement consisitent.
Hash algorithm. The server code does not need to be changed. Other systems that use consisitent hash include Chord, a distributed Hash Table Implementation, Amazon dynamo, and a key-value storage system. (No open source)
Libketama-a consistent hashing algo for memcache clients
Posted by muesli indevelopment
Wednesday, 2017l 11. 2007
We wrote ketama to replace how our memcached clients mapped keys to servers. Previously, clients mapped keys-> servers like this:
Server = serverlist [Hash (key) % serverlist. Length];
This meant that whenever we added or removed servers from the pool, everything hashed to different servers, which divide tively wiped the entire cache. we add (and sometimes remove) servers from the memcached pool often enough to warrant writing this-if your
Memcached pool never changes, you can probably stop reading now
Ketama is an implementation of a consistent hashing algorithm, meaning you can add or remove servers from the memcached pool without causing a complete remap of all keys.
Here's how it works:
-Take your list of servers (eg: 1.2.3.4: 11211, 5.6.7.8: 11211, 9.8.7.6: 11211)
-Hash each server string to several (100-200) unsigned ints
-Conceptually, these numbers are placed on a circle called the Continuum. (imagine a clock face that goes from 0 to 2 ^ 32)
-Each number links to the server it was hashed from, so servers appear at several points on the continuum, by each of the numbers they hashed.
-To map a key-> server, hash your key to a single unsigned int, and find the next biggest number on the continuum. the server linked to that number is the correct server for that key.
-If you hash your key to a value near 2 ^ 32 and there are no points on the continuum greater than your hash, return the first server in the Continuum.
If you then add or remove a server from the list, only a small proportion of keys end up mapping to different servers.
The majority of the Code is a C library (libketama) and a PhP4 extension that wraps it. I 've also included a class from our Java client. (Java collections makes it rather easy ). we use a single-server memcache client wrapped with a native PHP class to make
It multi-server capable, so we just replaced the hashing method with a ketama_find_server call. (shocould be easy enough to plug this into libmemcache if need be)
Http://static.last.fm/ketama/ketama-0.1.1.tar.bz2
We 've been using this in production for all our PHP installand Java services at last. FM for Round 10 days now. we deployed it just in time to smooth over moving loads of webservers between datacenters.
For further information, please refer to the README inside the tarball or these threads on the memcached mailing list:
Http://lists.danga.com/pipermail/memcached/2007-April/003853.html
Http://lists.danga.com/pipermail/memcached/2007-April/003834.html