Deployment of simple Memcached proxy Twemproxy

The deployment of simple Memcached proxy Twemproxy is a lightweight Redis and Memcached proxy, which is mainly used to reduce the number of connections to the backend cache server. The cache server cluster management tool open-source by Twitter is mainly used to make up for the shortcomings pointed out by Redis and Memcached in cluster management.

The deployment of simple Memcached proxy Twemproxy is a lightweight Redis and Memcached proxy, which is mainly used to reduce the number of connections to the backend cache server. The cache server cluster management tool open-source by Twitter is mainly used to make up for the shortcomings pointed out by Redis and Memcached in cluster management.

Simple MemcachedProxy TwemproxyDeployment

Twemproxy (also known as nutcracker) is a lightweight Redis and Memcached proxy, mainly used to reduce the number of connections to the backend cache server. The cache server cluster management tool open-source by Twitter is mainly used to make up for the shortcomings pointed out by Redis and Memcached in cluster management.

Twemproxy is a fast single-thread proxy program that supports Memcached ASCII protocol and the updated Redis protocol.

The most amazing thing about Twemproxy is that it can detach a node when it fails, and then try again (immediately) after a while, alternatively, you can strictly connect to the server according to the key written in the configuration file.

Installation and deployment

Existing Test Machine: 192.168.11.51/52/68

Install libevent and memcached on the 51 and 52 test machines to start two memcached instances respectively;

Then install twemproxy on 68, configure parameters, and start the twemproxy instance.

Install and start memcachedInstance

For detailed steps, see the previous blog "install and configure Memcached 1.4.22" to start the following instances:

/usr/local/bin/memcached -d -m 128 -u memcached -l 192.168.11.51 -p 11211 -c 1024 -P /var/run/memcached/memcached1.pid/usr/local/bin/memcached -d -m 128 -u memcached -l 192.168.11.51 -p 11212 -c 1024 -P /var/run/memcached/memcached2.pid/usr/local/bin/memcached -d -m 128 -u memcached -l 192.168.11.52 -p 11211 -c 1024 -P /var/run/memcached/memcached1.pid/usr/local/bin/memcached -d -m 128 -u memcached -l 192.168.11.52 -p 11212 -c 1024 -P /var/run/memcached/memcached2.pid

Install and start twemproxyInstance

1Install autoconf

cd /tmpwget http://ftp.gnu.org/gnu/autoconf/autoconf-2.69.tar.gztar zxvf autoconf-2.69.tar.gzcd autoconf-2.69./configure --prefix=/usr/make && make install

2Install twemproxy

cd /tmpwget https://github.com/twitter/twemproxy/archive/master.zipunzip master.zip -d /usr/local/cd /usr/localmv twemproxy-master twemproxycd twemproxyCFLAGS="-ggdb3 -O0" autoreconf -fvi./configure --prefix=/usr/local/twemproxy --enable-debug=logmake && make install

3, View help

[root@test01 twemproxy]# ./sbin/nutcracker -h

This is nutcracker-0.4.0Usage: nutcracker [-?hVdDt] [-v verbosity level] [-o output file][-c conf file] [-s stats port] [-a stats addr][-i stats interval] [-p pid file] [-m mbuf size]Options:-h, --help : this help-V, --version : show version and exit-t, --test-conf : test configuration for syntax errors and exit-d, --daemonize : run as a daemon-D, --describe-stats : print stats description and exit-v, --verbose=N : set logging level (default: 5, min: 0, max: 11)-o, --output=S : set logging file (default: stderr)-c, --conf-file=S : set configuration file (default: conf/nutcracker.yml)-s, --stats-port=N : set stats monitoring port (default: 22222)-a, --stats-addr=S : set stats monitoring ip (default: 0.0.0.0)-i, --stats-interval=N : set stats aggregation interval in msec (default: 30000 msec)-p, --pid-file=S : set pid file (default: off)-m, --mbuf-size=N : set size of mbuf chunk in bytes (default: 16384 bytes)

4, Modify the configuration file

mkdir /etc/nutcrackercp ./conf/nutcracker.yml /etc/nutcracker/vi /etc/nutcracker/nutcracker.yml

memcached:listen: 192.168.11.55:22121hash: fnvla_64distribution: ketamatimeout: 400backlog: 1024preconnect: trueauto_eject_hosts: trueserver_retry_timeout: 30000server_failure_limit: 3servers:- 192.168.11.51:11211:1- 192.168.11.51:11212:1- 192.168.11.52:11211:1- 192.168.11.52:11212:1

Parameter parsing:

Listen: IP address and port used to start the twemproxy service

Hash: specifies a specific hash function.

Distribution: specifies a specific hash algorithm.

Preconnect: a Boolean value if the control's nutcracker frontend is connected to all server processes in the pool to start. The default value is false.

Auto_eject_hosts: whether to temporarily remove the node when the node cannot respond

Server_retry_timeout: retry time (MS)

Server_failure_limit: the number of node failures is removed.

Servers: all memcached nodes (IP: port number: weight)

5, Configure to start the service

cp ./scripts/nutcracker.init /etc/init.d/nutcrackerchmod 755 /etc/init.d/nutcrackervi /etc/init.d/nutcracker

1. added the definition daemo.

daemon="/usr/local/twemproxy/sbin/nutcracker"

2. replace all

daemon --user ${USER} ${prog} $OPTIONS

Is

${daemo} $OPTIONS

chkconfig --add nutcrackerchkconfig --level 35 nutcracker onchkconfig --list nutcracker

vi /etc/profile

Add:

export PATH="$PATH:/usr/local/twemproxy/sbin"

. /etc/profileecho $PATH

6, Test the configuration and start the service

nutcracker -t -c /etc/nutcracker/nutcracker.ymlservice nutcracker start

Data writing test

[root@test01 init.d]# telnet 192.168.11.55 11211

Trying 192.168.11.55...Connected to 192.168.11.55.Escape character is '^]'.set key1 0 0 11STOREDset key2 0 0 222STOREDset key3 0 0 3333STOREDset key4 0 0 44444STOREDset key5 0 0 555555STOREDquitConnection closed by foreign host.

[root@test01 ~]# telnet 192.168.11.51 11211

Trying 192.168.11.51...Connected to 192.168.11.51.Escape character is '^]'.get key1ENDget key2ENDget key3ENDget key4ENDget key5ENDquitConnection closed by foreign host.

[root@test01 ~]# telnet 192.168.11.51 11212

Trying 192.168.11.51...Connected to 192.168.11.51.Escape character is '^]'.get key1ENDget key2ENDget key3ENDget key4ENDget key5ENDquitConnection closed by foreign host.

[root@test02 ~]# telnet 192.168.11.52 11211

Trying 192.168.11.52...Connected to 192.168.11.52.Escape character is '^]'.get key1ENDget key2ENDget key3ENDget key4ENDget key5ENDquitConnection closed by foreign host.

[root@test02 ~]# telnet 192.168.11.52 11212

Trying 192.168.11.52...Connected to 192.168.11.52.Escape character is '^]'.get key1VALUE key1 0 11ENDget key2VALUE key2 0 222ENDget key3VALUE key3 0 3333ENDget key4VALUE key4 0 44444ENDget key5VALUE key5 0 555555ENDquitConnection closed by foreign host.

We can see that all the data is obtained from Port 11212 of port 52.

Port 11212 of port 52 is stopped.

Write data to the proxy.

[root@test03 init.d]# telnet 192.168.11.55 11211

Trying 192.168.11.55...Connected to 192.168.11.55.Escape character is '^]'.set username 0 0 6ryanxuSTOREDset aa 0 0 2aaSTOREDquitConnection closed by foreign host.

[root@test03 init.d]# telnet 192.168.11.55 11211

Trying 192.168.11.55...Connected to 192.168.11.55.Escape character is '^]'.get key1SERVER_ERROR Connection refusedConnection closed by foreign host.

[root@test03 init.d]# telnet 192.168.11.55 11211

Trying 192.168.11.55...Connected to 192.168.11.55.Escape character is '^]'.get key1SERVER_ERROR Connection refusedConnection closed by foreign host.

[root@test03 init.d]# telnet 192.168.11.55 11211

Trying 192.168.11.55...Connected to 192.168.11.55.Escape character is '^]'.get key1ENDget key2ENDget key3ENDquitConnection closed by foreign host.

After a memcached instance fails, twemproxy can be automatically removed. After the Restoration, twemproxy can automatically identify, recover, and re-join the memcached group for reuse.

Problem summary

1) in the yml configuration file, each parameter value pair delimiter ":" requires a space.

2) the parameters at different levels need to be indented and distinguished. it is best to use the tab key to indented. Otherwise, the nutcracker process cannot be started.

3) when auto_eject_hosts: true, after a memcached instance is closed, the system prompts "(error) ERR Connection refused" when writing data ". This is related to the small value of the server_retry_timeout parameter. 30000 is a good choice.