Memcached proxy server software: Note used by magent

[Author: Zhang banquet this article version: V1.0 last modified: 2009.01.19 reprint please indicate the original link: http://blog.s135.com/post/393/

]

Magent is an open-source memcached proxy server software. Its project URL is:

Http://code.google.com/p/memagent/

1. installation steps:

1. Compile and install libevent:

Wget http://monkey.org /~ Provos/libevent-1.4.9-stable.tar.gz

Tar zxvf libevent-1.4.9-stable.tar.gz

CD libevent-1.4.9-stable/

./Configure -- prefix =/usr

Make & make install

CD ../

2. Compile and install memcached:

Wget http://danga.com/memcached/dist/memcached-1.2.6.tar.gz

Tar zxvf memcached-1.2.6.tar.gz

CD memcached-1.2.6/

./Configure -- With-libevent =/usr

Make & make install

CD ../

3. Compile and install magent:

Mkdir magent

CD magent/

Wget http://memagent.googlecode.com/files/magent-0.5.tar.gz

Tar zxvf magent-0.5.tar.gz

/Sbin/ldconfig

Sed-I "s # libs =-Levent-LM # G" makefile

Make

CP magent/usr/bin/magent

CD ../

 

Ii. instances used:

Memcached-M 1-u root-D-l 127.0.0.1-P 11211

Memcached-M 1-u root-D-l 127.0.0.1-P 11212

Memcached-M 1-u root-D-l 127.0.0.1-P 11213

Magent-u root-N 51200-l 127.0.0.1-P 12000-s 127.0.0.1: 11211-s 127.0.0.1: 11212-B 127.0.0.1: 11213

1. Start three memcached processes on ports 11211, 11212, and 11213 respectively, and enable magent on ports 12000;

2. Ports 11211 and 11212 are primarily memcached, and ports 11213 are backup memcached;

3. Connect to magent 12000, set key1 and set key2. According to the hash algorithm, key1 is written to memcached of ports 11212 and 11213, and key2 is written to memcached of ports 11212 and 11213;

4. When memcached on ports 11211 and 11212 die, connect to the magent on port 12000 to retrieve data from memcached on port 11213;

　
5. When memcached on ports 11211 and 11212 is restarted and connected to port 12000, The magent will restart from Port 11211 or 11212
Memcached retrieves data. Since there is no data after the two memcached instances restart, magent gets a null value, even though there are still several memcached instances on port 11213.
(This issue has yet to be improved ).

Iii. Entire test process:

[Root @ centos52 ~] # Telnet 127.0.0.1 12000

Trying 127.0.0.1...

Connected to 127.0.0.1.

Escape Character is '^]'.
Stats

Memcached agent v0.4

Matrix 1-> FIG: 11211, pool size 0

Matrix 2-> FIG: 11212, pool size 0

End
Set key1 0 0 8

Zhangyan

Stored
Set key2 0 0 8

Zhangyan

Stored
Quit

Connection closed by foreign host.

[Root @ centos52 ~] # Telnet 127.0.0.1 11211

Trying 127.0.0.1...

Connected to 127.0.0.1.

Escape Character is '^]'.
Get key1

End
Get key2

Value key2 0 8

Zhangyan

End
Quit

Connection closed by foreign host.

[Root @ centos52 ~] # Telnet 127.0.0.1 11212

Trying 127.0.0.1...

Connected to 127.0.0.1.

Escape Character is '^]'.
Get key1

Value key1 0 8

Zhangyan

End
Get key2

End
Quit

Connection closed by foreign host.

[Root @ centos52 ~] # Telnet 127.0.0.1 11213

Trying 127.0.0.1...

Connected to 127.0.0.1.

Escape Character is '^]'.
Get key1

Value key1 0 8

Zhangyan

End
Get key2

Value key2 0 8

Zhangyan

End
Quit

Connection closed by foreign host.

Simulate memcached death on ports 11211 and 11212

[Root @ centos52 ~] # Ps-Ef | grep memcached

Root 6589 1 0? 00:00:00 memcached-M 1-u root-D-l 127.0.0.1-P 11211

Root 6591 1 0? 00:00:00 memcached-M 1-u root-D-l 127.0.0.1-P 11212

Root 6593 1 0? 00:00:00 memcached-M 1-u root-D-l 127.0.0.1-P 11213

Root 6609 6509 0 000:00:00 pts/0 grep memcached

[Root @ centos52 ~] # Kill-9 6589

[Root @ centos52 ~] # Kill-9 6591

[Root @ centos52 ~] # Telnet 127.0.0.1 12000

Trying 127.0.0.1...

Connected to 127.0.0.1.

Escape Character is '^]'.
Get key1

Value key1 0 8

Zhangyan

End
Get key2

Value key2 0 8

Zhangyan

End
Quit

Connection closed by foreign host.

Simulate memcached Restart for ports 11211 and 11212

[Root @ centos52 ~] # Memcached-M 1-u root-D-l 127.0.0.1-P 11211

[Root @ centos52 ~] # Memcached-M 1-u root-D-l 127.0.0.1-P 11212

[Root @ centos52 ~] # Telnet 127.0.0.1 12000

Trying 127.0.0.1...

Connected to 127.0.0.1.

Escape Character is '^]'.
Get key1

End
Get key2

End
Quit

Connection closed by foreign host.

 

Appendix usage
:

Magent
Is a simple but useful proxy program for memcached
Servers.

It features:

• Keeps connections to memcached servers
• Supports following memcached commands
  1. Get gets
  2. Delete
  3. Incr decr
  4. Add set replace prepend append
  5. CAS
• Event-driven by using libevent
  Library
• Supports ketama Algorithm
• Backup servers farm
• UNIX domain socket

Usage:

  -h this message
  -u uid
  -g gid
  -p port, default is 11211. (0 to disable tcp support)
  -s ip:port, set memcached server ip and port
  -b ip:port, set backup memcached server ip and port
  -l ip, local bind ip address, default is 0.0.0.0
  -n number, set max connections, default is 4096
  -D do not go to background
  -k use ketama key allocation algorithm
  -f file, unix socket path to listen on. default is off
  -i number, max keep alive connections for one memcached server, default is 20
  -v verbose

Changelog:

2010/4/14: memcached agent 0.6

1. Add connection keepalive Handler
2. Bug fix, more robust, more debug messages

Examples:

  magent -s 10.1.2.1 -s 10.1.2.2:11211 -b 10.1.2.3:14000 -v

Notes:

1. Current magent's speed is about 80% of memcached's. Finding way to speed it up (not good)
2. Adding Backup Server does no harm to magent's performance. (good)
3. Magent's CPU usage is less than memcached's and has very little memory usage. (good)