Python consistent hash distributed

Source: Internet
Author: User

Hash_ring

#-*-coding:utf-8-*-"""    Hash_ring ~~~~~~~~~~~~~~ Implements Consistent hashing that can is used when the number of server nodes can I    Ncrease or decrease (like in memcached). Consistent hashing is a scheme, a hash table functionality in a, a, that, the adding or removing of one slo    T does significantly change the mapping of keys to slots.            More information about consistent hashing can is read in these articles: "Web Caching with consistent hashing": Http://www8.org/w8-papers/2a-webserver/caching/paper2.html "Consistent hashing and random Trees:dis tributed caching protocols for relieving hot spots on the World Wide Web (1997) ": http://citeseerx.ist.psu.edu/                            legacymapper?did=38148 Example of Usage:: memcache_servers = [' 192.168.0.246:11212 ', ' 192.168.0.247:11212 ', ' 192.168.0.249:11212 '] ring = hashring (memcache_servers) s   erver = Ring.get_node (' My_key ') : copyright:2008 by Amir Salihefendic. : License:bsd"""ImportMathImportSYS fromBisectImportbisectifSys.version_info >= (2, 5):    Importhashlib Md5_constructor=HASHLIB.MD5Else:    ImportMD5 Md5_constructor=md5.newclasshashring (object):def __init__(Self, Nodes=none, weights=None):"""' nodes ' is a list of objects that has a proper __str__ representation.  ' Weights ' is dictionary that sets weights to the nodes.        The default weight is, all nodes be equal. """self.ring=dict () Self._sorted_keys=[] Self.nodes=nodesif  notweights:weights={} self.weights=weights self._generate_circle ()def_generate_circle (self):"""generates the circle. """Total_weight=0 forNodeinchSelf.nodes:total_weight+ = Self.weights.get (node, 1)         forNodeinchSelf.nodes:weight= 1ifNodeinchSelf.weights:weight=self.weights.get (node) factor= Math.floor ((40*len (self.nodes) *weight)/total_weight);  forJinchRange (0, int (factor)): B_key= Self._hash_digest ('%s-%s'%(node, j)) forIinchRange (0, 3): Key= Self._hash_val (B_key,LambdaX:x+i*4) Self.ring[key]=node Self._sorted_keys.append (key) Self._sorted_keys.sort ()defGet_node (Self, string_key):"""Given A string key a corresponding node in the hash ring is returned.        If the hash ring is empty, ' None ' is returned. """POS=Self.get_node_pos (String_key)ifPos isNone:returnNonereturnself.ring[Self._sorted_keys[pos]]defGet_node_pos (Self, string_key):"""Given A string key a corresponding node in the hash ring was returned along with it's position in the ring.        If the hash ring is empty, (' none ', ' none ') is returned. """        if  notself.ring:returnNone Key=Self.gen_key (string_key) nodes=Self._sorted_keys POS=bisect (nodes, key)ifpos = =len (nodes):return0Else:            returnPOSdefIterate_nodes (self, string_key, distinct=True):"""Given A string key it returns the nodes as a generator that can hold the key.        The generator iterates one time through the ring starting at the correct position.        If ' distinct ' is set and then the nodes returned would be unique, i.e. no virtual copies would be returned. """        if  notself.ring:yieldNone, none returned_values=set ()defDistinct_filter (value):ifSTR (value) not inchReturned_values:returned_values.add (str (value))returnValue POS=Self.get_node_pos (String_key) forKeyinchSelf._sorted_keys[pos:]: Val=Distinct_filter (Self.ring[key])ifVal:yieldVal forI, keyinchEnumerate (Self._sorted_keys):ifI <Pos:val=Distinct_filter (Self.ring[key])ifVal:yieldValdefGen_key (self, key):"""Given A string key it returns a Long value, this long value represents a place on the hash ring.        MD5 is currently used because it mixes well. """B_key=self._hash_digest (Key)returnSelf._hash_val (B_key,Lambdax:x)def_hash_val (self, B_key, ENTRY_FN):return((B_KEY[ENTRY_FN (3)] << 24)                | (B_KEY[ENTRY_FN (2)] << 16)                | (B_KEY[ENTRY_FN (1)] << 8)                |b_key[entry_fn (0)])def_hash_digest (self, key): M=md5_constructor () m.update (bytes (key,encoding='Utf-8'))        #return Map (Ord, M.digest ())        returnlist (M.digest ())" "memcache_servers = [' 192.168.0.246:11212 ', ' 192.168.0.247:11212 ', ' 192.168.0.249 : 11212 ']ring = hashring (memcache_servers) server = Ring.get_node (' My_key ')" "#Increase Weightmemcache_servers= ['192.168.0.246:11212',                    '192.168.0.247:11212',                    '192.168.0.249:11212']weights= {    '192.168.0.246:11212': 1,    '192.168.0.247:11212': 2,    '192.168.0.249:11212': 1}ring=hashring (memcache_servers, weights) server= Ring.get_node ('My_key')Print(server)

Increased removal of the machine may not be able to find the data

Python consistent hash distributed

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.