Build memcached cluster with Magent

Although Memcached is able to pass the distributed cache, the memcached will not lose all the cached data, but some of the data is still difficult to escape.

We can use the Magent proxy memcached to implement the master-slave backup to ensure that the cache data intact, and magent can also be used as a continuation, but the general principle of operation is as follows:

1.magent each write data will be written to the main memcached and from the memcached, and to the master-slave memcached write the same algorithm;

2. When the main memcached is down, Magent will read the data from the memcached;

3. When the primary memcached is restored, Magent will re-read the data to the main memcached, which is a major drawback of magent because the primary memcached has just recovered and there is no data, which can cause some data to be unreadable.

There are several ideas for the disadvantages of magent:

1. In the production environment, the likelihood of the primary memcached being down is very small, most of the time is working, and the memcached is only used after the main memcached is down, so the space allocated from memcached is unlikely to be the same as the main memcached. This is undoubtedly a waste of valuable memory space.

2. Since the allocation of space from the memcached is small, and as the data deposited will be more and more, will cause the cached data is constantly expired out of memory, so after the main memcached outage, only temporarily to alleviate the role of database pressure.

3. When the main memcached is down, it is not advisable to start it directly, or to start it when the database is less stressful, just preheat the cache.

Summary: I introduced magent in addition to the master-slave aspect, but also take into account the magent---magent to achieve the load balance of memcached, that is, read and write requests according to a certain algorithm allocated to two magent entrance, can achieve high availability, but also load balance.

One. Installation section

1. Installing memcached-1.2.2

CD libevent-2.0.22-stable./configure--prefix=/usr/local/libeventmakemake INSTALLCD memcached-1.2.2./configure-- prefix=/usr/local/memcached--with-libevent=/usr/local/libeventmakemake install## #安装错误 # # #memcached. C: in function ' add_ Iov ' in: memcached.c:696:30: Error: ' Iov_max ' is not declared (first used within this function) Memcached.c:696:30: NOTE: Each undeclared identifier is reported only once in the function it appears make[2]: * * * [ MEMCACHED-MEMCACHED.O] Error 1 need to modify MEMCACHED.C file:/* FreeBSD 4.x doesn ' t has iov_max exposed. */#ifndef iov_max#if defined (__freebsd__) | | Defined (__apple__) # define Iov_max 1024#ENDIF#ENDIF changed to:/* FreeBSD 4.x doesn ' t have iov_max exposed. */#ifndef iov_max# define Iov_max 1024#endif## #启动错误 ###[[email protected] memcached-1.2.2]#/usr/local/memcached/bin/ Memcached/usr/local/memcached/bin/memcached:error while loading shared libraries:libevent-2.0.so.5:cannot open Shared object File:no such file or directory needs to be modified: Ln-s/usr/local/libevent/lib/libevent-2.0.so.5/lib64/ Libevent-2.0.so.5

2. Installing magent-0.5

CD magent-0.5/sbin/ldconfigsed-i "s#libs =-levent#libs =-levent-lm#g" Makefilemake error is as follows: Gcc-wall-o2-g-c-o magent. O magent.cmagent.c:65:19: Error: Event.h: There is no file or directory magent.c:124: Error: The type of field ' EV ' is not fully magent.c:154: error: The type of field ' EV ' is not completely magent.c: In the function ' Server_free ': magent.c:468: Warning: Implicitly declaring the function ' Event_del ' magent.c: in function ' Put_server_into_pool ': magent.c:522: Error: ' sizeof ' cannot be used for incomplete type ' struct event ' magent.c: in function ' out_string ': magent.c:696: Error: ' Ev_write ' is not declared (first used within this function) magent.c:696: Error: (Even if multiple occurrences occur within a function, each undeclared identifier is reported only once within its magent.c:696: Error: function. magent.c:699: Warning: Implicitly declaring function ' Event_set ' magent.c:699: Error: ' Ev_persist ' is not declared (first used within this function) magent.c:700: Warning: Implicitly declaring function ' Event_ Add ' magent.c: in function ' do_transcation ': magent.c:816: Error: ' sizeof ' cannot be used for incomplete type ' struct event ' magent.c:818: Error: ' Ev_ PERSIST ' not declared (first used within this function) magent.c:818: Error: ' Ev_write ' is not declared (first used within this function) Magent.c: In function ' start_backup_transcation ': magent.c:902: Error: ' sizeof ' cannot be used for incomplete type ' struct event ' magent.c:904: Error: ' Ev_persist ' is not declared (first used within this function) magent.c:904: Error: ' Ev_ WRITE ' not declared (first used within this function) Magent.c: in function ' Try_backup_serveR ' in: magent.c:994: Error: ' sizeof ' cannot be used for incomplete type ' struct event ' magent.c:996: Error: ' Ev_persist ' is not declared (first used within this function) magent.c:996: Error: ' Ev_write ' is not declared (first used within this function) Magent.c: In function ' Drive_memcached_server ': magent.c:1014: Error: ' Ev_write ' is not declared (first use within this function) magent.c:1060: Error: ' Ev_read ' is not declared (first used within this function) magent.c:1062: Error: ' Ev_persist ' is not declared (first used within this function) Magent.c: in function ' process_get_ Response ' in: magent.c: in function ' Process_get_response ': magent.c:1236: Error: ' Ev_write ' is not declared (first used within this function) magent.c:1238: Error: ' Ev_ PERSIST ' not declared (first used within this function) Magent.c: In function ' Process_update_response ': magent.c:1283: Error: ' Ev_write ' is not declared (first use within this function) magent.c:1285: Error: ' Ev_persist ' is not declared (first used within this function) Magent.c: In function ' Drive_backup_server ': magent.c:1305: Error: ' Ev_write ' is not declared ( First use within this function) magent.c:1343: Error: ' Ev_persist ' is not declared (first used within this function) magent.c:1343: Error: ' Ev_read ' is not declared (first used within this function) Magent.c: in function ' Drive_client ' in: magent.c:1618: Error: ' Ev_read ' is not declared (first used within this function) magent.c:1669: Error: ' Ev_write ' is not declared (first used within this function) magent.c : 1682: Error: ' Ev_persist ' is not declared (first used within this function) Magent.c: In function ' server_accept ': magent.c:1734: Error: ' sizeof ' cannot be used for incomplete type ' struct Eve 'NT ' magent.c:1735: Error: ' Ev_read ' is not declared (first used within this function) magent.c:1735: Error: ' Ev_persist ' is not declared (first used within this function) Magent.c: In function ' main ': MAGENT.C:2106: Warning: Implicitly declaring function ' Event_init ' magent.c:2111: Error: ' Ev_read ' is not declared (first used within this function) magent.c:2111: Error: ' Ev_persist ' is not declared ( First use within this function) magent.c:2121: Warning: Implicitly declaring a function ' event_loop ' Make: * * * [MAGENT.O] Error 1 Modify makefilelibs =-levent-lm-l/usr/local/ Libevent/libinclude=-i/usr/local/libevent/include again make, appears the following gcc-wall-o2-g-i/usr/local/libevent/include-c-o MAGENT.O magent.cmagent.c: In function ' writev_list ': magent.c:623: Error: ' Ssize_max ' is not declared (first used within this function) magent.c:623: Error: ( Even if multiple occurrences occur within a function, each undeclared identifier is reported only once within its magent.c:623: Error: function. Make: * * [MAGENT.O] Error 1 need to modify VIM ketama.h in the first line plus #ifndef Ssize_max in line fourth plus # define Ssize_max 32676 at the end add #endif and then re-make, The following success occurs [[email protected] magent]# makegcc-wall-o2-g-i/usr/local/libevent/include-c-o magent.o magent.cgcc-wa Ll-o2-g-i/usr/local/libevent/include-c-o ketama.o ketama.cgcc-wall-o2-g-o magent magent.o ketama.o-levent-lm-l /usr/local/libevent/lib after executing magent andError, you need to execute CP magent/usr/bin/magentbash:magent:command not found execution magent The following indicates success [[email protected] magent]#  Magentplease provide-s "Ip:port" argumentmemcached agent v0.4 BUILD-DATE:APR, 21:24:28usage:-H this message-u Uid-g gid-p Port, default is 11211. (0 to disable TCP support)-S Ip:port, set memcached server IP and port-b ip:port, set backup memcached server IP and P Ort-l IP, local bind IP address, default is 0.0.0.0-n number, set Max connections, default is 4096-d don ' ts go to BAC Kground-k use Ketama key allocation algorithm-f file, UNIX sockets path to listen on. Default is off-i number, set Max keep alive connections for one memcached server, default is 20-V verbose

Two. Deployment

As shown in the following:

1.magent1, Magent2 as the total entrance of the memcached, we use the algorithm to achieve load balancing, allocating read and write requests, regardless of which portal is allocated to the back end of the memcached is the same, Because they allocate memcached use the same algorithm Consistent-hash

2. Back-end MECACHED1,MECACHED2,MECACHED3,MECACHED4 multiple 4 primary memcached

3.magent3 as from, at the same time also from the entrance of the memcached, the back end there are two memcached5,mecached6, composed of Magent3,mecached5,mecached6 together from the memcached

Work Flow:

1.magent1,magent2 accepts the write request, writes the key to the MECACHED1-MECACHED4 separately, and writes the key to the memcached, which is Magent3, MAGENT3, and then writes MECACHED5 separately. The same allocation algorithm used in MECACHED6;

2.magent1,magent2 accept the read request, will be read to the main memcached respectively, and do not want to read from the memcached;

3. Once there is a memcached outage in Mecached1-mecached4, Magent1 and Magent2 will read the data from memcached, that is magent3, to achieve the effect of the cache data is not lost;

4. When the primary memcache is restored, it will be added to the main memcached again, and Magent1 and Magent2 will not read the data from the memcached, but the write is still normal;

Start as follows:

Memcached1-memcached6/usr/local/memcached-1.4.22/bin/memcached-u root-d-P 11211-m 1024-c 102400/usr/local/ Memcached-1.4.22/bin/memcached-u root-d-P 11212-m 1024-c 102400/usr/local/memcached-1.4.22/bin/memcached-u root-d- P 11213-m 1024-c 102400/usr/local/memcached-1.4.22/bin/memcached-u root-d-P 11214-m 1024-c 102400/usr/local/memcach  Ed-1.4.22/bin/memcached-u root-d-P 11215-m 1024-c 102400/usr/local/memcached-1.4.22/bin/memcached-u root-d-P 11216 -M 1024-c 102400magent3/usr/bin/magent-u root-n 102400-l 192.168.3.127-p 12002-s 192.168.3.127:11215-s 192.168.3.1 27:11216magent1:usr/bin/magent-u root-n 102400-l 192.168.3.127-p 12000-s 192.168.3.127:11211-s 192.168.3.127:11212- s 192.168.3.128:11213-s 192.168.3.127:11214-b 192.168.3.127:12002magent2:usr/bin/magent-u root-n 102400-l 192.168.3. 127-p 12001-s 192.168.3.127:11211-s 192.168.3.127:11212-s 192.168.3.128:11213-s 192.168.3.127:11214-b 192.168.3.127: 1250W

Magent1,magent2 algorithm:

1. Depending on the number of total entrances to the magent (here is 2), the method of dividing by 2 is used, if 0 is allocated to Magent1, if 1 is assigned to Magent2, the request is evenly distributed to magent1 and Magent2

Note:

1. If memcached read and write more data, it will be turned off the firewall or set the firewall to not use the Tracking link table, otherwise it will cause/var/log/message error "kernel:nf_conntrack:table full, dropping Packet", please refer to the blog post http://blog.csdn.net/yanggd1987/article/details/45886913 for details.

2. Use service iptables stop to turn off and cannot be viewed with iptables-l-vnx, so using this command will turn the firewall on, although the rule is cleared, the connection trace table is still logged.

Build memcached cluster with Magent