Php implementation method for memcache consistent hash, memcachehash

Php implementation method for memcache consistent hash, memcachehash
I recently read some distributed articles, so I used php to implement consistent hash to train my hands. In the past, I used the most primitive hash Modulo for distribution, when a memcache instance is added or deleted during the production process, all data becomes invalid. Consistent hash is used to solve this problem and minimize the amount of invalid data. For more information, see google!

Php implementation efficiency is lacking. If it is necessary to be efficient, it is better to write extension.

Tests show that each of the five memcache instances generates 100 virtual nodes, and the set and get1000 times are added, which is five times slower than a single memcache instance set. Therefore, the efficiency is average and needs to be optimized!

Before reading this article, you 'd better know the binary search method.


Implementation process:


Memcache configures ip + port + virtual node serial number for hash and uses crc32 to form a closed loop.

Perform crc32 on the key to be operated

Find the nearest virtual node in the virtual node loop using the binary method.

Extract the real memcache ip address and port from the virtual node for single-instance connection

<? Php/*** php implementation method of memcache consistent hash * @ author http://www.lai18.com * @ date 2015-04-19 */class memcacheHashMap {private $ _ node = array (); private $ _ nodeData = array (); private $ _ keyNode = 0; private $ _ memcache = null; // Number of virtual nodes generated by each physical server [Note: the more nodes, the better the uniformity of cache distribution, and the more resource consumption during the set get operation. 10 physical servers use 200 more reasonable] private $ _ virtualNodeNum = 200; private function _ construct () {$ config = array (// five memcache servers '2017. 0.0. O'clock 11', '192. 0.0.1: 100', '192. 0.0.1: 100', '192. 0.0.1: 100', '192. 0.0.1: 100'); if (! $ Config) throw new Exception ('cache config null'); foreach ($ config as $ key => $ value) {for ($ I = 0; $ I <$ this-> _ virtualNodeNum; $ I ++) {$ this-> _ node [sprintf ("% u", crc32 ($ value. '_'. $ I)] = $ value. '_'. $ I; // create 200 virtual nodes for each memcache server in a loop} ksort ($ this-> _ node ); // The 1000 created virtual nodes are sorted by key names in ascending order} // instantiate the static public function getInstance () {static $ memcacheObj = null; if (! Is_object ($ memcacheObj) {$ memcacheObj = new self ();} return $ memcacheObj ;} // find the location of the corresponding virtual node based on the sent key: private function _ connectMemcache ($ key) {$ this-> _ nodeData = array_keys ($ this-> _ node ); // an array of keys for all virtual nodes $ this-> _ keyNode = sprintf ("% u", crc32 ($ key )); // calculate the hash value of the key $ nodeKey = $ this-> _ findServerNode (); // find the corresponding virtual node // If the loop is exceeded, use the binary method to find the nearest node, then, judge the beginning and end of the ring, and obtain the nearest node if ($ this-> _ keyNode> end ($ this-> _ nodeData) {$ this-> _ KeyNode-= end ($ this-> _ nodeData); $ nodeKey2 = $ this-> _ findServerNode (); if (abs ($ nodeKey2-$ this-> _ keyNode) <abs ($ nodeKey-$ this-> _ keyNode) $ nodeKey = $ nodeKey2;} var_dump ($ this-> _ node [$ nodeKey]); list ($ config, $ num) = explode ('_', $ this-> _ node [$ nodeKey]); if (! $ Config) throw new Exception ('cache config error'); if (! Isset ($ this-> _ memcache [$ config]) {$ this-> _ memcache [$ config] = new Memcache; list ($ host, $ port) = explode (':', $ config); $ this-> _ memcache [$ config]-> connect ($ host, $ port );} return $ this-> _ memcache [$ config];} // The binary method is used to locate the nearest virtual node location: private function _ findServerNode ($ m = 0, $ B = 0) {$ total = count ($ this-> _ nodeData); if ($ total! = 0 & $ B = 0) $ B = $ total-1; if ($ m <$ B) {$ avg = intval ($ m + $ B) /2); if ($ this-> _ nodeData [$ avg] ==$ this-> _ keyNode) return $ this-> _ nodeData [$ avg]; elseif ($ this-> _ keyNode <$ this-> _ nodeData [$ avg] & ($ avg-1> = 0) return $ this-> _ findServerNode ($ m, $ avg-1); else return $ this-> _ findServerNode ($ avg + 1, $ B );} if (abs ($ this-> _ nodeData [$ B]-$ this-> _ keyNode) <abs ($ this-> _ nodeData [$ m]-$ this-> _ key Node) return $ this-> _ nodeData [$ B]; else return $ this-> _ nodeData [$ m];} public function set ($ key, $ value, $ expire = 0) {return $ this-> _ connectMemcache ($ key)-> set ($ key, json_encode ($ value), 0, $ expire );} public function add ($ key, $ value, $ expire = 0) {return $ this-> _ connectMemcache ($ key)-> add ($ key, json_encode ($ value ), 0, $ expire);} public function get ($ key) {return json_decode ($ this-> _ connect Memcache ($ key)-> get ($ key), true);} public function delete ($ key) {return $ this-> _ connectMemcache ($ key) -> delete ($ key) ;}}$ runData ['in in _ time'] = microtime (true); // Add getfor ($ I = 0 for 10 thousand sets tests; $ I <10000; $ I ++) {$ key = md5 (mt_rand (); $ B = memcacheHashMap: getInstance ()-> set ($ key, time (), 10);} var_dump (number_format (microtime (true)-$ runData ['in in _ time'], 6 )); $ runData ['in in _ time'] = microtime (true); $ m = n Ew Memcache; $ m-> connect ('2017. 0.0.1 ', 11211); for ($ I = 0; $ I <10000; $ I ++) {$ key = md5 (mt_rand ()); $ B = $ m-> set ($ key, time (), 0, 10);} var_dump (number_format (microtime (true) -$ runData ['in in _ time'], 6);?>