Memcache Distributed Cache

Reprint Address: http://www.cnblogs.com/phpstudy2015-6/p/6713164.html

That leaves with the wind

1, memcached Distributed introduction

Memcached, although called a "distributed" cache server, has no "distributed" functionality on the server side. Memcache cluster hosts can not communicate with each other, and its "distributed" is further implemented based on the client-side program logic algorithm.

Take a look at the following diagram:

Based on our brief analysis of the process of set and get of distributed memcached

Set procedure:

1, first through the application set (' Key ', ' value ')

2, enter the program, using key through the logical algorithm to derive the node location that the key needs to be stored

3. Connect the corresponding memcached server according to the node location and send the SET command

Get procedure:

1, first through the application get (' key ')

2, then use the key through the logical algorithm to obtain the storage node of the key

3. Connect the corresponding memcached server according to the node and send the Get command

There are many ways to implement memcached, the most common of which is the distribution of uniform hash ideas (for short, uniform hash distribution). Good things, of course, need a second liepin to foil its merits, so in addition to explain the uniform hash distributed, will also talk about modulo distribution. To further analyze their pros and cons.

This example will be implemented in PHP code, of course, the most important is the idea and Method! After all, these two things are interlinked in any language.

2, Modulus algorithm method

What is the method of modulo algorithm distributed? is to convert the key to a 32-bit number and divide it with the total number of memcached servers to obtain the remainder. And this remainder is the node of the memcached server. With this node we can identify the memcached server and send a command to memcached to execute.

Diagram Analysis:

The entire process is shown.

1), PHP code implementation

GetModMemcache.class.php

 1 <?php 2 #分布式memcache (modulo calculation) 3 class Getmodmemcache 4 {5 Private $total = ';   #存储memcache服务器的总数 6 Private $servers =array (); #存储memcache服务器的具体信息 7/** 8 * @desc Constructor 9 *10 * @param $serversArr Array | Memcache Server specific Information one */12 public function __construct ($SERVERSARR) ($this->total=count ($serversAr R), $this->servers= $serversArr,}17/**19 * @desc calculate the storage location of the $key (that is, which server) * @para M string | Key string *23 * @return int returns the first server */25 protected function position ($key) #使用crc32 ()      , convert the string to 32 for the number sprintf ('%u ', CRC32 ($key))% $this->total; #取余29}30/**32 * @desc get memcached Object *34 * @param $position int | Location information for key *36 * @return Object returns the instantiated Memcached object */38 protected function getmemcached ($position) 39 {    $host = $this->servers[$position] [' Host ']; #服务器池中某台服务器host41 $port =$this->servers[$position [' Port ']; #服务器池中某台服务器port42 $m = new memcached (); $m->addserver ($host, $port); }46/**48 * @desc Set key-value value *50 * @param string | Key string Mixed * @param | The value can be any valid non-resource PHP type *53 * @return return result */55 public function Setkey ($key, $value), $num      = $this->position ($key); Echo $num;   #调试用59 $m = $this->getmemcached ($num);         #获取memcached对象60 return $m->set ($key, $value),}62 GetKey ($key) 64 {65 $num = $this->position ($key), $m = $this->getmemcached ($num); $m->get ($key); 68}69 70 }72 $arr =array (' host ' = ' 192.168.95.11 ', ' port ' = ' 11210 '), Array (' host ' = ' 192.168.95.1 ') 1 ', ' port ' = ' 11211 '), an array (' host ' = ' 192.168.95.11 ', ' port ' = ' 11212 '), and $mod =new Getmodmemcache ($ ARR);/*82 #存储数据83 $A= $mod->setkey (' Key3 ', ' key33333 '), echo "<pre>", Print_r ($a), echo "</pre>";d ie;87 */88/*89 # Get Data $b = $mod->getkey (' key1 '); echo "<pre>"; Print_r ($b); "</pre>";d ie;94 */95?>

2), the corresponding test

1, continuous insertion of three data

#set (' Key1 ', ' value11111 '); #node =1

#set (' Key2 ', ' value22222 '); #node =1

#set (' Key3 ', ' value33333 ';) #node =0

2, respectively Telnet connection 192.168.95.11: (11210, 11211, 11212)

11210 contains Key3 data

11211 contains Key1, Key2 data

11212 data not included

3. Using the program get data

Results are able to take the data out

3) Advantages and disadvantages

Advantages:

1, simple and practical easy to understand

2. Uniform Data Distribution

Disadvantages:

1, a memcached server can not automatically adjust the group to process data, so that part of the data can not use the cache, has been continuously from the database to obtain data.

2, when the need to expand the time, to increase the number of memcached servers, so the original cached data is not able to be hit, that is, the data is useless.

3, consistent hashing algorithm mode

What is the uniform hashing algorithm distributed?

Imagine that all 32-bit numbers are spread clockwise from small to large on a ring;

Next, assign each storage node a name and convert it to a 32-bit number through the CRC32 function, which is the storage node for the memcached server

Next, the key is also converted through the CRC32 function to a 32-bit number, where it is located in the clockwise direction of the first encountered storage node corresponding to the memcached server is the key of the final storage server.

1), image analysis

Assuming that the Node1 node server is hung, the data that was originally stored on the Node1 node can also be stored in the NODE3 node, according to the principle of clockwise.

Assuming there is a need for expansion, the increase of two memcached servers, what will happen? Please look at the analysis

The results show that only a small amount of data is affected, and that the impact is within an acceptable range relative to the overall data.

From the above diagram we can easily find that there is a disadvantage, that is, using the CRC32 function we can not control the location of the memcached storage node, and the total number of nodes relative to 2 of the 32 is how small. If even these storage nodes are very close to each other, then there is bound to be a memcached server to withstand the overwhelming majority of the data cache.

Please look at the analysis:

Workaround:

Map a real storage node to multiple virtual storage nodes, that is, real node + suffix is processed by CRC32 (for example: Node1_1, Node1_2, node1_3 、.....、 node1_n)

See Node Distribution:

Three real nodes become 30 storage nodes on the ring, which avoids the problem that the storage nodes are too close and the data cache is unevenly distributed, and the storage mechanism has no change.

2), PHP code implementation

ConsistentHashMemcache.class.php

  1 <?php 2 #分布式memcache consistent hashing algorithm (with ring data structure) 3 class Consistenthashmemcache 4 {5 Private $virtualNode = ';    #用于存储虚拟节点个数 6 Private $realNode =array ();      #用于存储真实节点 7 Private $servers =array ();   #用于存储memcache服务器信息 8 #private $totalNode =array (); #节点总数 9/** * @desc Constructor One * * @param $servers Array | Memcache Server information * @param $virtualNode int | Number of virtual nodes, default 64 */Public function __construct ($servers, $virtualNode =64) $this->servers = $servers; $this->realnode=array_keys ($servers); $this->virtualnode= $virtualNode; /** * @return Int returns 32 digits of the number * */private Function hash ($STR)   Urn sprintf ('%u ', CRC32 ($STR)); #将字符串转换为32位的数字)/** * @desc Processing node * * * @param $realNode Array | Real node * @param $virturalNode int | Number of Virtual nodes * * @return Array returns all node information 37 */=array Private Function Dealnode ($realNode, $virtualNode) ($totalNode) ($r Ealnode as $v) ($i =0; $i < $virtualNode $i + +) Ode= $this->hash ($v. '-'. $i); $totalNode [$hashNode]= $v;     Ksort ($totalNode); #按照索引进行排序, ascending return $totalNode; /** * @desc Get Key's true storage node * * * @param $key string | Key String * * * * @return String returns real node GetNode ($key)    Ode= $this->dealnode ($this->realnode, $this->virtualnode); #获取所有虚拟节点 +/-#查看虚拟节点总数 echo "<pre>"; Print_r ($totalNode); echo "</pre>";d ie;            * * * $hashNode = $this->hash ($key); #key的哈希节点. foreach ($totalNode as $k = = $v) #循环总结点环查找 70 {71 if ($k >= $hashNode) #查找第一个大于key哈希节点的值 $v;               #返回真实节点 the "* *}}" return reset ($totalNode); #假若总节点环的值都比key哈希节点小, the value of the first total Hashi is returned in the sum of the values of/**. * @desc return memcached Object Bayi * @param $key S Tring | Key value of * * * @return Object * * * getmemcached ($key) $node = $this             ->getnode ($key); #获取真实节点 echo $key. ' Real node: '. $node. ' <br/> ';    #测试使用, check the true node of key $host = $this->servers[$node [' Host '];    #服务器池中某台服务器host $port = $this->servers[$node [' Port '];                    #服务器池中某台服务器port $m = new memcached ();            #实例化 $m->addserver ($host, $port);                              #添加memcache服务器 94 return $m; #返回memcached对象/** 98 * @desc set Key-value value */100 public functIon Setkey ($key, $value) 101 {102 $m = $this->getmemcached ($key); 103 return $m->set ($key, $value); 1 }105 106/**107 * @desc get the value108 */109 public Function GetKey ($key) in key, 111 $m =$ This->getmemcached ($key);->get return $m ($key), 113}114,}117 118?>

3), test

1. View all virtual nodes

A total of 64*3=132 virtual nodes (virtual node settings or small, usually set in 100~200)

2. Set Test

1 include './consistenthashmemcache.class.php '; 2 header ("CONTENT-TYPE:TEXT/HTML;CHARSET=UTF8;"); 3 $arr =array (4     ' Node1 ' =>array (' host ' = ' 192.168.95.11 ', ' port ' = ' 11210 '), 5     ' Node2 ' =>array (' Host ' = ' 192.168.95.11 ', ' port ' = ' 11211 '), 6     ' Node3 ' =>array (' host ' = ' 192.168.95.11 ', ' port ' = ' = ') 11212 '), 7     ); 8  9 $c =new Consistenthashmemcache ($arr), #测试set12 $c->setkey (' aaa ', ' 11111 '), $c->setkey (' BBB ', ' 22222 '); $c->setkey (' CCC ', ' 33333 ');

Telnet connection respectively 192.168.95.11: (11210, 11211, 11212)

Get (' AAA '), get (' BBB ') can fetch value in node Node1

Get (' CCC ') can fetch value in node Node3

3. Get test

1 include './consistenthashmemcache.class.php '; 2 header ("CONTENT-TYPE:TEXT/HTML;CHARSET=UTF8;"); 3 $arr =array (4     ' Node1 ' =>array (' host ' = ' 192.168.95.11 ', ' port ' = ' 11210 '), 5     ' Node2 ' =>array (' Host ' = ' 192.168.95.11 ', ' port ' = ' 11211 '), 6     ' Node3 ' =>array (' host ' = ' 192.168.95.11 ', ' port ' = ' = ') 11212 '), 7     ); 8  9 $c =new Consistenthashmemcache ($arr), #测试get11 echo $c->getkey (' aaa '). ' <br/> '; Echo $c->getkey (' BBB '). ' <br/> '; Echo $c->getkey (' CCC '). ' <br/> ';

4. Advantages and Disadvantages

The distributed code complexity of uniform hashing is a bit higher than that of modulo mode, but it is also within acceptable limits and does not pose any obstacle problems. On the contrary, its advantages are very significant, through the implementation of virtual nodes, can make the non-controllable storage node can be distributed evenly on the ring, so that the data is evenly cached in each host. Second, adding and removing virtual nodes has a very small impact on the overall data that was previously cached.

Memcache Distributed Cache