Php implementation of memcache consistent hash

This article mainly introduces the php implementation method of memcache consistent hash. The example analyzes the implementation principle and related skills of memcache's hash consistency, for more information about how to implement memcache consistent hash, see the following example. Share it with you for your reference. The details are as follows:

I recently read some distributed articles, so I used php to implement consistent hash to train my hands. in the past, I used the most primitive hash modulo for distribution, when a memcache instance is added or deleted during the production process, all data becomes invalid. Consistent hash is used to solve this problem and minimize the amount of invalid data. For more information, see google!

Php implementation efficiency is lacking. if it is necessary to be efficient, it is better to write extension.

Tests show that each of the five memcache instances generates 100 virtual nodes, and the set and get1000 times are added, which is five times slower than a single memcache instance set. Therefore, the efficiency is average and needs to be optimized!

Before reading this article, you 'd better know the binary search method.

Implementation process:

Memcache configures ip + Port + virtual node serial number for hash and uses crc32 to form a closed loop.
Perform crc32 on the key to be operated
Find the nearest virtual node in the virtual node loop using the binary method.
Extract the real memcache ip address and port from the virtual node for single-instance connection

The code is as follows:

<? Php
Class memcacheHashMap {
Private $ _ node = array ();
Private $ _ nodeData = array ();
Private $ _ keyNode = 0;
Private $ _ memcache = null;
// The number of virtual nodes generated by each physical server [Note: The more nodes, the better the uniformity of cache distribution. at the same time, the set get operation consumes more resources and 10 physical servers, use 200 is more reasonable]
Private $ _ virtualNodeNum = 200;
Private function _ construct (){
$ Config = array (// five memcache servers
'127. 0.0.1: 100 ',
'127. 0.0.1: 100 ',
'127. 0.0.1: 100 ',
'127. 0.0.1: 100 ',
'127. 0.0.1: 8080'
);
If (! $ Config) throw new Exception ('cache config Null ');
Foreach ($ config as $ key => $ value ){
For ($ I = 0; $ I <$ this-> _ virtualNodeNum; $ I ++ ){
$ This-> _ node [sprintf ("% u", crc32 ($ value. '_'. $ I)] = $ value. '_'. $ I; // create 200 virtual nodes for each memcache server in a loop
}
}
Ksort ($ this-> _ node); // The 1000 virtual nodes created are sorted by key names from small to large.
}
// Instantiate this class
Static public function getInstance (){
Static $ memcacheObj = null;
If (! Is_object ($ memcacheObj )){
$ MemcacheObj = new self ();
}
Return $ memcacheObj;
}
// Locate the corresponding virtual node based on the sent key
Private function _ connectMemcache ($ key ){
$ This-> _ nodeData = array_keys ($ this-> _ node); // an array of keys for all virtual nodes
$ This-> _ keyNode = sprintf ("% u", crc32 ($ key); // calculate the hash value of the key
$ NodeKey = $ this-> _ findServerNode (); // find the corresponding virtual node
// If the loop is exceeded, use the binary method to search for the nearest node from the beginning and end of the loop to make a judgment and obtain the closest node.
If ($ this-> _ keyNode> end ($ this-> _ nodeData )){
$ This-> _ keyNode-= end ($ this-> _ nodeData );
$ NodeKey2 = $ this-> _ findServerNode ();
If (abs ($ nodeKey2-$ this-> _ keyNode) <abs ($ nodeKey-$ this-> _ keyNode) $ nodeKey = $ nodeKey2;
}
Var_dump ($ this-> _ node [$ nodeKey]);
List ($ config, $ num) = explode ('_', $ this-> _ node [$ nodeKey]);
If (! $ Config) throw new Exception ('cache config error ');
If (! Isset ($ this-> _ memcache [$ config]) {
$ This-> _ memcache [$ config] = new Memcache;
List ($ host, $ port) = explode (':', $ config );
$ This-> _ memcache [$ config]-> connect ($ host, $ port );
}
Return $ this-> _ memcache [$ config];
}
// The binary method is used to locate the nearest virtual node location based on the given value.
Private function _ findServerNode ($ m = 0, $ B = 0 ){
$ Total = count ($ this-> _ nodeData );
If ($ total! = 0 & $ B = 0) $ B = $ total-1;
If ($ m <$ B ){
$ Avg = intval ($ m + $ B)/2 );
If ($ this-> _ nodeData [$ avg] =$ this-> _ keyNode) return $ this-> _ nodeData [$ avg];
Elseif ($ this-> _ keyNode <$ this-> _ nodeData [$ avg] & ($ AVG-1> = 0) return $ this-> _ findServerNode ($ m, $ AVG-1 );
Else return $ this-> _ findServerNode ($ avg + 1, $ B );
}
If (abs ($ this-> _ nodeData [$ B]-$ this-> _ keyNode) <abs ($ this-> _ nodeData [$ m]-$ this-> _ keyNode) return $ this-> _ nodeData [$ B];
Else return $ this-> _ nodeData [$ m];
}
Public function set ($ key, $ value, $ expire = 0 ){
Return $ this-> _ connectMemcache ($ key)-> set ($ key, json_encode ($ value), 0, $ expire );
}
Public function add ($ key, $ value, $ expire = 0 ){
Return $ this-> _ connectMemcache ($ key)-> add ($ key, json_encode ($ value), 0, $ expire );
}
Public function get ($ key ){
Return json_decode ($ this-> _ connectMemcache ($ key)-> get ($ key), true );
}
Public function delete ($ key ){
Return $ this-> _ connectMemcache ($ key)-> delete ($ key );
}
}
$ RunData ['In in _ time'] = microtime (true );
// Test set and get for 10 thousand times
For ($ I = 0; I I <10000; $ I ++ ){
$ Key = md5 (mt_rand ());
$ B = memcacheHashMap: getInstance ()-> set ($ key, time (), 10 );
}
Var_dump (number_format (microtime (true)-$ runData ['In in _ time'], 6 ));
$ RunData ['In in _ time'] = microtime (true); $ m = new Memcache;
$ M-> connect ('2017. 0.0.1 ', 127 );
For ($ I = 0; I I <10000; $ I ++ ){
$ Key = md5 (mt_rand ());
$ B = $ m-> set ($ key, time (), 0, 10 );
}
Var_dump (number_format (microtime (true)-$ runData ['In in _ time'], 6 ));
?>

I hope this article will help you with php programming.