Use magent to build a memcached Cluster

Use magent to build a memcached Cluster

Although memcached can use distributed caching to prevent the loss of all cached data when memcached goes down, it is difficult to escape part of the data.

We can use the magent agent memcached to implement master-slave backup to ensure that the cache data is intact, and the magent can continue to be used, but the general working principle is as follows:

1. The magent writes data to the master memcached and memcached each time, and writes the same algorithm to the master memcached;

2. When the primary memcached goes down, magent reads data from memcached;

3. after the master memcached is restored, magent reads data from the master memcached again. Because the master memcached has just been restored and no data exists, some data cannot be read, this is also a major drawback of magent.

There are several ideas about the disadvantages of magent:

1. in the production environment, the primary memcached may be down, and most of the time it is working. Instead, memcached is used only after the primary memcached is down, therefore, the space allocated from memcached cannot be the same as that of the primary memcached, which is undoubtedly a waste of valuable memory space.

2. since memcached allocates a small amount of space, and as more data is stored, the cached data is constantly evicted out of memory due to expiration. Therefore, after the primary memcached goes down, it can only temporarily relieve the pressure on the database.

3. When the primary memcached goes down, it is not recommended to start it directly, or restart it when the database is under a relatively low pressure. It should be cache push.

Conclusion: In addition to the master and slave aspects, I also introduced magent-magent to implement Load Balancing FOR THE memcached entry. That is to say, read/write requests are allocated to two magent entries according to certain algorithms, it can achieve high availability and load balancing.

I. Installation

1. Install memcached-1.2.2

Cd libevent-2.0.22-stable. /configure -- prefix =/usr/local/libeventmakemake installcd memcached-1.2.2. /configure -- prefix =/usr/local/memcached -- with-libevent =/usr/local/libeventmakemake install ### installation error ### memcached. c: In the 'Add _ iov' function: memcached. c: 696: 30: Error: 'iov _ MAX 'is not declared (used for the first time in this function) memcached. c: 696: 30: Note: Each undeclared identifier only reports make [2]: *** [memcached-memcached.o] Error 1 needs to modify memcached once in the function it appears. c file:/* FreeBSD 4.x doesn't have IOV_MAX exposed. */# ifndef IOV_MAX # if defined (_ FreeBSD _) | defined (_ APPLE _) # define IOV_MAX 1024 # endif changed: /* FreeBSD 4.x doesn't have IOV_MAX exposed. */# ifndef IOV_MAX # define IOV_MAX 1024 # endif ### startup error ### [root @ test memcached-1.2.2] #/usr/local/memcached/bin/memcached/usr/local /memcached/bin/memcached: error while loading shared libraries: libevent-2.0.so.5: cannot open shared object file: No such file or directory need to be modified: ln-s/usr/local/libevent/lib/libevent-2.0.so.5/lib64/libevent-2.0.so.5

2. Install magent-0.5

 

 

Cd magent-0.5/sbin/ldconfigsed-I "s # LIBS =-levent-lm # g" Makefilemake reports the following error: gcc-Wall-O2-g-c-o magent. o magent. cmagent. c: 65: 19: Error: event. h: there is no such file or directory magent. c: 124: error: the type of the field 'ev 'is incomplete. c: 154: error: the type of the field 'ev 'is incomplete. c: In the 'server _ free' function: magent. c: 468: Warning: Implicit declaration function 'event _ del 'magent. c: In the 'put _ server_0000_pool 'function: magent. c: 522: Error: 'SIZE' cannot be used for an incomplete type 'struct event' magent. c: In the 'out _ string' function: m Agent. c: 696: Error: 'ev _ write' is not declared (used for the first time in this function) magent. c: 696: Error: (even if it appears multiple times in a function, each undeclared identifier is in its magent. c: 696: error: the function is only reported once .) Magent. c: 699: Warning: Implicit declaration function 'event _ set' magent. c: 699: Error: 'ev _ PERSIST 'is not declared (used for the first time in this function) magent. c: 700: Warning: Implicit declaration function 'event _ add' magent. c: In the 'do _ transcation' function: magent. c: 816: Error: 'SIZE' cannot be used for an incomplete type 'struct event' magent. c: 818: Error: 'ev _ PERSIST 'is not declared (used for the first time in this function) magent. c: 818: Error: 'ev _ write' is not declared (used for the first time in this function) magent. c: In the 'start _ backup_transcation 'function: magent. c: 902: Error: 'SIZE' cannot be used for an incomplete type 'struct event' magent. c: 904: Error: 'ev _ PERSIST 'is not declared (Used for the first time in this function) magent. c: 904: Error: 'ev _ write' is not declared (used for the first time in this function) magent. c: In the 'try _ backup_server 'function: magent. c: 994: Error: 'SIZE' cannot be used for an incomplete type 'struct event' magent. c: 996: Error: 'ev _ PERSIST 'is not declared (used for the first time in this function) magent. c: 996: Error: 'ev _ write' is not declared (used for the first time in this function) magent. c: In the 'drive _ memcached_server 'function: magent. c: 1014: Error: 'ev _ write' is not declared (used for the first time in this function) magent. c: 1060: Error: 'ev _ read' is not declared (used for the first time in this function) magent. c: 1062: Error: 'ev _ PERSIST 'is not declared (used for the first time in this function) magent. c: In the 'process _ get_response 'function: magent. c: In the 'process _ get_response 'function: magent. c: 1236: Error: 'ev _ write' is not declared (used for the first time in this function) magent. c: 1238: Error: 'ev _ PERSIST 'is not declared (used for the first time in this function) magent. c: In the 'process _ update_response' function: magent. c: 1283: Error: 'ev _ write' is not declared (used for the first time in this function) magent. c: 1285: Error: 'ev _ PERSIST 'is not declared (used for the first time in this function) magent. c: In the 'drive _ backup_server 'function: magent. c: 1305: Error: 'ev _ write' is not declared (used for the first time in this function) magent. c: 1343: Error: 'ev _ PERSIST 'is not declared (in this function One-time use) magent. c: 1343: Error: 'ev _ read' is not declared (used for the first time in this function) magent. c: In the 'drive _ client' function: magent. c: 1618: Error: 'ev _ read' is not declared (used for the first time in this function) magent. c: 1669: Error: 'ev _ write' is not declared (used for the first time in this function) magent. c: 1682: Error: 'ev _ PERSIST 'is not declared (used for the first time in this function) magent. c: In the 'server _ accept 'function: magent. c: 1734: Error: 'SIZE' cannot be used for an incomplete type 'struct event' magent. c: 1735: Error: 'ev _ read' is not declared (used for the first time in this function) magent. c: 1735: Error: 'ev _ PERSIST 'is not declared (used for the first time in this function) magent. c: In the 'main' function: magent. C: 2106: Warning: Implicit declaration function 'event _ init 'magent. c: 2111: Error: 'ev _ read' is not declared (used for the first time in this function) magent. c: 2111: Error: 'ev _ PERSIST 'is not declared (used for the first time in this function) magent. c: 2121: warning: the implicit declaration function 'event _ loop 'make: *** [magent. o] Error 1 modify MakefileLIBS =-levent-lm-L/usr/local/libevent/libINCLUDE =-I/usr/local/libevent/include and re-make, the following gcc-Wall-O2-g-I/usr/local/libevent/include-c-o magent is displayed. o magent. cmagent. c: In the 'writev _ list' function: magent. c: 623: Error: 'ssi ZE_MAX 'is not declared (used for the first time in this function) magent. c: 623: Error: (even if it appears multiple times in a function, each undeclared identifier is in its magent. c: 623: error: the function is only reported once .) Make: *** [magent. o] Error 1. You need to modify vim ketama. h add # ifndef SSIZE_MAX in the first line and # define SSIZE_MAX 32676 in the fourth line. Then add # endif and make again, if the following message appears, [root @ test magent] # makegcc-Wall-O2-g-I/usr/local/libevent/include-c-o magent is successfully created. o magent. cgcc-Wall-O2-g-I/usr/local/libevent/include-c-o ketama. o ketama. cgcc-Wall-O2-g-o magent. o ketama. if an error is reported after o-levent-lm-L/usr/local/libevent/lib executes the magent, run cp magent/usr/bin/magentbash: magent: command not found indicates that the execution of the magent is successful [root @ test magent] # magentplease provide-s "ip: port" argumentmemcached agent v0.4 Build-Date: Apr 23 2015 21: 24: 28. Usage:-h this message-u uid-g gid-p port, default is 11211. (0 to disable tcp support)-s ip: port, set memcached server ip and port-B ip: port, set backup memcached server ip and port-l ip, local bind ip address, default is 0.0.0.0-n number, set max connections, default is 4096-D don't go to background-k use ketama key allocation algorithm-f file, unix socket path to listen on. default is off-I number, set max keep alive connections for one memcached server, default is 20-v verbose

II. Deployment

 

As shown in:

1. magent1 and magent2 serve as the total entry point of memcached. We use algorithms to achieve load balancing and allocate read/write requests. No matter which entry is used, the same is true for allocating memcached to the backend, because they are allocated to memcached, they all use the same algorithm consistent-hash.

2. backend mecached1, mecached2, mecached3, and mecached4 multi-bit 4 Master memcached

3. magent3 serves as both the slave and the memcached entry. Its backend also has two memcached5 and mecached6. magent3, mecached5, and mecached6 are composed

Workflow:

1. magent1, magent2 accept the write request, write the key into the mecached1-mecached4, and also write the key from memcached, that is, magent3, magent3 and then write them into mecached5, mecached6;Both master and slave use the same allocation algorithm.

2. magent1 and magent2 will read from the master memcached rather than from memcached;

3. Once a memcached goes down in the mecached1-mecached4, magent1 and magent2 will read data from memcached, that is, magent3, so that the cached data will not be lost;

4. When the memcache in the master node is restored, it will be added to the master memcached again. At this time, magent1 and magent2 will not read data from memcached, but the write is still normal;

Start as follows:

 

memcached1-memcached6/usr/local/memcached-1.4.22/bin/memcached -u root -d -p 11211 -m 1024 -c 102400/usr/local/memcached-1.4.22/bin/memcached -u root -d -p 11212 -m 1024 -c 102400/usr/local/memcached-1.4.22/bin/memcached -u root -d -p 11213 -m 1024 -c 102400/usr/local/memcached-1.4.22/bin/memcached -u root -d -p 11214 -m 1024 -c 102400/usr/local/memcached-1.4.22/bin/memcached -u root -d -p 11215 -m 1024 -c 102400/usr/local/memcached-1.4.22/bin/memcached -u root -d -p 11216 -m 1024 -c 102400magent3/usr/bin/magent -u root -n 102400 -l 192.168.3.127 -p 12002 -s 192.168.3.127:11215 -s 192.168.3.127:11216magent1:usr/bin/magent -u root -n 102400 -l 192.168.3.127 -p 12000 -s 192.168.3.127:11211 -s 192.168.3.127:11212 -s 192.168.3.128:11213 -s 192.168.3.127:11214 -b 192.168.3.127:12002magent2:usr/bin/magent -u root -n 102400 -l 192.168.3.127 -p 12001 -s 192.168.3.127:11211 -s 192.168.3.127:11212 -s 192.168.3.128:11213 -s 192.168.3.127:11214 -b 192.168.3.127:12002

Magent1, magent2 algorithm:

 

1. based on the total number of magent entries (2 here), divide by 2 to obtain the remainder. If 0 is allocated to magent1, if 1 is allocated to magent2, in this way, requests are evenly distributed to magent1 and magent2.

Note:

1. if memcached reads and writes a large amount of data, it must turn off the firewall or set the firewall to not use the trace chain table. Otherwise, the/var/log/message error "kernel: nf_conntrack: table full, dropping packet ". For details, refer to the blog post http://blog.csdn.net/yanggd1987/article/details/45886913.

2. Disable using service iptables stop and cannot be viewed using iptables-L-vnx. Therefore, this command will enable the firewall. Although the rules are cleared, the connection trace table will still be recorded.