Redis/Memcache代理服務Twemproxy簡介

標籤：des   style   http   io   ar   color   os   sp   for   

zhanhailiang 日期：2014-12-14

簡介

twemproxy，也叫nutcracker，是twtter開源的Redis和MemcacheProxy 伺服器。

功能

• Fast.
• Lightweight.
• Maintains persistent server connections.
• Keeps connection count on the backend caching servers low.
• Enables pipelining of requests and responses.
• Supports proxying to multiple servers.
• Supports multiple server pools simultaneously.
• Shard data automatically across multiple servers.
• Implements the complete memcached ascii and redis protocol.
• Easy configuration of server pools through a YAML file.
• Supports multiple hashing modes including consistent hashing and distribution.
• Can be configured to disable nodes on failures.
• Observability through stats exposed on stats monitoring port.
• Works with Linux, *BSD, OS X and Solaris (SmartOS)

安裝

從官網下載安裝包編譯並安裝：distribution tarball:

[[email protected]~/software/nutcracker-0.3.0]# ./configure --prefix=/usr/local/nutcracker-0.3.0[[email protected]~/software/nutcracker-0.3.0]# make[[email protected]~/software/nutcracker-0.3.0]# make install[[email protected]~/software/nutcracker-0.3.0]# cp -R conf /usr/local/nutcracker-0.3.0[[email protected]~/software/nutcracker-0.3.0]# ln -s /usr/local/nutcracker-0.3.0 /usr/local/nutcracker[[email protected]~/software/nutcracker-0.3.0]# ln -s /usr/local/nutcracker/sbin/nutcracker /usr/local/bin/nutcracker

配置

twemproxy支援通過YAML文法配置，其支援的指令集如下：

• listen: The listening address and port (name:port or ip:port) for this server pool.
• hash: The name of the hash function. Possible values are:
  • one_at_a_time
  • md5
  • crc16
  • crc32 (crc32 implementation compatible with libmemcached)
  • crc32a (correct crc32 implementation as per the spec)
  • fnv1_64
  • fnv1a_64
  • fnv1_32
  • fnv1a_32
  • hsieh
  • murmur
  • jenkins
• hash_tag: A two character string that specifies the part of the key used for hashing. Eg "{}" or "$$".Hash tag enable mapping different keys to the same server as long as the part of the key within the tag is the same.
• distribution: The key distribution mode. Possible values are:
  • ketama
  • modula
  • random
• timeout: The timeout value in msec that we wait for to establish a connection to the server or receive a response from a server. By default, we wait indefinitely.
• backlog: The TCP backlog argument. Defaults to 512.
• preconnect: A boolean value that controls if nutcracker should preconnect to all the servers in this pool on process start. Defaults to false.
• redis: A boolean value that controls if a server pool speaks redis or memcached protocol. Defaults to false.
• server_connections: The maximum number of connections that can be opened to each server. By default, we open at most 1 server connection.
• auto_eject_hosts: A boolean value that controls if server should be ejected temporarily when it fails consecutively server_failure_limit times. See liveness recommendations for information. Defaults to false.
• server_retry_timeout: The timeout value in msec to wait for before retrying on a temporarily ejected server, when auto_eject_host is set to true. Defaults to 30000 msec.
• server_failure_limit: The number of conseutive failures on a server that would leads to it being temporarily ejected when auto_eject_host is set to true. Defaults to 2.
• servers: A list of server address, port and weight (name:port:weight or ip:port:weight) for this server pool.

以本文為例，配置一個Redis代理和一個Memcache代理服務如下：

[[email protected]~/software/nutcracker-0.3.0]# cat /usr/local/nutcracker/conf/nutcracker.ymlalpha:  listen: 127.0.0.1:22121  hash: fnv1a_64  hash_tag: "{}"  distribution: ketama  auto_eject_hosts: false  timeout: 400  redis: true  servers:   - 127.0.0.1:6379:1beta:  listen: 127.0.0.1:22122  hash: fnv1a_64  distribution: ketama  timeout: 400  backlog: 1024  preconnect: true  auto_eject_hosts: true  server_retry_timeout: 2000  server_failure_limit: 3  servers:   - 127.0.0.1:11211:1   - 127.0.0.1:11212:1

其中，Redis執行個體 127.0.0.1:6379 和 Memcache執行個體 127.0.0.1:11211 127.0.0.1:11212 請預先啟動好。

/etc/init.d/redis_6379 start/usr/local/memcached-1.4.20/bin/memcached -d -m 64 -l 127.0.0.1 -p 11211 -u root/usr/local/memcached-1.4.20/bin/memcached -d -m 64 -l 127.0.0.1 -p 11212 -u root

測試

https://github.com/billfeller/billfeller.github.io/blob/master/code/twenproxyTest.php

啟動服務：

[[email protected]/usr/local/nutcracker/conf]# nutcracker -c nutcracker.yml[Sun Dec 14 20:59:04 2014] nc.c:187 nutcracker-0.3.0 built for Linux 2.6.32-431.23.3.el6.x86_64 x86_64 started on pid 14359[Sun Dec 14 20:59:04 2014] nc.c:192 run, rabbit run / dig that hole, forget the sun / and when at last the work is done / don‘t sit down / it‘s time to dig another one

執行測試指令碼：

[[email protected]~/wade/git/billfeller.github.io/code]# /usr/local/php/bin/php twenproxyTest.php bool(true)string(1) "1"int(1)bool(false)bool(true)int(2)bool(true)int(3)bool(true)bool(false)

Redis/Memcache代理服務Twemproxy簡介