Magent, Memcached load uniformity and fault tolerance

A Background

Memcached in the implementation of distributed cluster deployment, There is no communication between the Memcached server, the server is pseudo-distributed, the implementation of distributed is by the client, The client implements the distributed algorithm to save the data to different Memcached server . We have introduced in the previous article, when our client to save also key for Keyvalue,value for DataValue, memcached server We did a test to verify that:

Two Problem analysis

memcached does not handle fault tolerance, that is, when the memcached server hangs or memcached Restart the Resurrection, because Memcached Restart after no data, high concurrency in large-scale systems, there is no way to accept, we in the case of the cluster, to do load balancing and fault tolerance , in this case, the server side hangs, the load is not so balanced, the cache data will be lost , re-operation of the database, increased system performance, data loss fault tolerance is not doing so well .

Three Solve

magent is an open source Memcached proxy Server software that can be used to build Memcached Services for highly available clustered applications, backed up Memcached data, although Memcached The service hangs, the front end can also obtain the data, the client first connects to Magent Proxy Server , and then magent Proxy Server can connect more than one Memcached Server, then can make data saving and backup data. This data is not lost and data integrity is preserved.

Four. Implement

First step : Download and install magent and install

(1)http://memagent.googlecode.com/files/magent-0.5.tar.gz

(2) Install the Memcached service First, this has been installed in the previous section

(3) install magent on Linux

#tar zxvf magent-0.5.tar.gz

#/sbin/ldconfig

#sed-i "s#libs =-levent#libs =-levent-lm#g" Makefile

#make

#cp magent/usr/bin/magent

in the implementation of the issue of make, Magen installation problems we have introduced how to solve

(4) Verify that there is no correct installation

#magent can display the Help information correctly, it means that the installation is correct:

Step two: Set the Magent agent's Memcached service and back up the Memcached service

(1) start three Memcached processes, respectively:

/opt/bin/memcached-d-M 5-u root-l 192.168.74.129 -P 12000-c 256-p/tmp/memcached.pid

/opt/bin/memcached-d-M 5-u root-l 192.168.74.130 -P 13000-c 256-p/tmp/memcached.pid

/opt/bin/memcached-d-M 5-u root-l 192.168.74.130 -P 14000-c 256-p/tmp2/memcached.pid

(2) Start a magent process:

Magent-u root-n 51200 -L 192.168.74.129-p 11000-s 192.168.74.130:13000-s 192.168.74.130:14000-b 192.168.74 .129:12000

- s for memcached to be written , - b for backup memcached, -N Number, set Max connections, default is 4096

Step three: Test magent

by connecting to the cache proxy server through the client, the cache proxy server connects the cache server, and the cache proxy server can connect multiple Memcached servers to each Memcached Server for data synchronization, we test Memcached One of them hangs off, the same client key can also get data from the cache service, and test the data saved to the proxy server, the corresponding Memcached Server also has data.

(1) Our client connects to the proxy server and then saves the data,and theMemcached Server has saved to the corresponding data.

The first step: We connect our proxy server 192.168.74.129:11000, then save the data, here we directly in the Linux On-Service simulations:

We set the key to keyValue3 value to dataValue3

Step Two: We check that the write memcached Service has no corresponding add

We view is saved in 192.168.74.130:13000 or 192.168.74.130:14000,:

The Memcached Service written at this time is on the 192.168.74.130:13000 service.

Step three: At this point , the backup Memcached server should have corresponding data:

We have just 192.168.74.129:12000 for the backup memcached service, at this time have saved the corresponding data, when the written emcached server hangs off can recover data from here.

(2) When our client connects to the proxy server and then modifies the value correspondingto the original key,Memcached The server has the corresponding data should also be the corresponding modification

First step:replace Modify,:

The second step: we look at the proxy server has no corresponding changes

Our client connects to the proxy server:

Our original key is keyValue3 value changed to dataValueNew3

Step Three: we check that the write memcached Service has no corresponding modifications

：

The memcached service that was written before is the key to the 192.168.74.130:13000 service. KeyValue3 value changes to dataValueNew3

The third step: At this time the backup Memcached server should have the corresponding data modification, :

(3) when we hang up the corresponding Memcached Service, the data will not get the corresponding value through the original key .

Before we view The value of the keyValue3 on the 192.168.74.130:13000 server, We turn off the service, so that assuming the service has been hung out, verify that The ability to get data is to handle fault tolerance.

：

And then we shut down the 192.168.74.130:13000 server process .

in the proxy service 192.168.74.129:11000 get keyValue3 can also get data

   ( 4 ) reboot Memcached service is in 192.168.74.130:13000 service, verify that you can still get the data?

We previously looked at the value of keyValue3 on the 192.168.74.130:13000 server, We have turned off the service, we restarted 192.168.74.130:13000 Service, in 192.168.74.129:11000 Agent service can get the data, let's restart it first. 192.168.74.130:13000 server,:

First step: View 192.168.74.130:13000 's keyValue3 First

：

At this point 192.168.74.130:13000 has not saved keyValue3 corresponding value. because memcached doesn't do persistent data here.

Second step: Backup first view 192.168.74.129:12000 's keyValue3

：

in this case, there are keyValue3 corresponding values in the backup, so the data can be recovered from the backup memcached service, which plays a role of fault tolerance .

We get keyValue3 corresponding value by 192.168.74.129:11000 Agent service

：

At this point the 192.168.74.130:13000 service restarts, through 192.168.74.129:11000 Proxy service cannot get the value corresponding to the KEYVALUE3.

Magent, Memcached load uniformity and fault tolerance