CODIS Cluster Deployment Combat

I. Summary

1, toss Codis cluster has been fast two months, thanks has been Codis author Liu Qi and Huangdongxu patient support, here to give you a praise, now I have a business run in the Codis cluster, currently only cut the entire business of 10%, is expected next week will be a full amount of cut to Codis , this time you must be particularly interested to know codis stability? There is no bug ah, you want to know is also I want to know, get up, use to know, anyway, we do not find any problem, some small problems have been timely contact the author to change, OK, not nonsense, The purpose of this article is to help you understand the rapid deployment of CODIS beginners (the official deployment document is a little bit the should be shipped.) There is also the need to make a backup for the post-cluster deployment.

2, Codis is a distributed Redis solution, for the upper-level applications, connect to Codis Proxy and connect to the native Redis Server no obvious difference (some commands are not supported), the upper application can be used as a single-use Redis, Cod Is the bottom will handle the forwarding of requests, non-stop data migration, and so on, all the things behind, for the front of the client is transparent, you can simply think behind the connection is an infinite memory of the Redis service, of course, In the previous time, the official Redis 3.0 out of the stable version, 3.0 support cluster function, CODIS implementation principle and 3.0 of the cluster function almost, I know now the American Regiment, Ali has used 3.0 of the cluster function, our side of the business is mainly php,3.0 Cluster SDK currently does not seem to support the PHP language, everyone who's PHP application on the 3 group, please contact me, I go to learn, about Redis Common cluster technology, please go to @ Shida Teacher Xiao's Infoq column Redis clustering technology and CODIS practice

Second, the structure

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M00/6B/E1/wKiom1U4wNbil8RQAAIeNjYHmcE992.jpg "title=" 1111111111111.png "alt=" Wkiom1u4wnbil8rqaaienjyhmce992.jpg "/>

Third, the role of the batch

Zookeeper cluster: 10.10.0.4710.10.0.4810.10.1.76codis-config, codis-ha:10.10.32.10:18087codis-proxy:10.10.32.10 : 1900010.10.32.49:19000codis-server:10.10.32.42:6379, 10.10.32.43:6380 (master, slave) 10.10.32.43:6379, 10.10.32.44:6380 (primary, From) 10.10.32.44:6379, 10.10.32.42:6380 (master, slave)

Iv. deployment

1, Installation Zookeeper

Yum-y Install zookeeper JDK # #安装服务

Vim/etc/hosts # #添加host10.10.0.47 zookeeper-node110.10.0.48 zookeeper-node210.10.1.76 Zookeeper-node3

Vim/etc/zookeeper/conf/zoo.cfg # #撰写zk的配置文件maxClientCnxns =50ticktime=2000initlimit=10synclimit=5datadir=/data/ zookeeper/clientport=2181server.1=zookeeper-node1:2888:3888server.2=zookeeper-node2:2888:3888server.3= zookeeper-node3:2888:3888

mkdir/data/zookeeper/# #创建zk的datadir目录echo "2" >/data/zookeeper/myid # #生成ID, it's important to note that myID corresponding zoo.cfg server.id, such as zookeeper-node2 corresponding myID should be 2/usr/lib/zookeeper/bin/zkserver.sh start # # service start

2, go Install (Codis is written in the go language, so those machines need to install you know)

wget HTTPS://STORAGE.GOOGLEAPIS.COM/GOLANG/GO1.4.1.LINUX-AMD64.TAR.GZTAR-ZXVF GO1.4.1.LINUX-AMD64.TAR.GZMV go/usr /local/cd/usr/local/go/src/bash all.bashcat >> ~/.BASHRC << _bashrc_exportexport goroot=/usr/local/ Goexport path=\ $PATH: \ $GOROOT/binexport goarch=amd64export goos=linux_bashrc_exportsource ~/.BASHRC

3, download and compile Codis (Codis-config, Codis-proxy, Codis-server machine)

Mkdir/data/goexport Gopath=/data/go/usr/local/go/bin/go Get github.com/wandoulabs/codiscd/data/go/src/github.com/ Wandoulabs/codis/./bootstrap.shmake Gotest

V. Service start-up and initialization of the cluster

1. Start Dashboard (operation on Codis-config)

Cat/etc/codis/config_10.ini # #撰写配置文件zk =10.10.0.47:2181,10.10.0.48:2181,10.10.1.76:2181product=zh_newsproxy_id= codis-proxy_10net_timeout=5000proto=tcp4dashboard_addr=10.10.32.10:18087

cd/data/go/src/github.com/wandoulabs/codis/&&/bin/codis-config-c/etc/codis/config_10.ini Dashboard &

2. Initialize slots (operation on Codis-config)

cd/data/go/src/github.com/wandoulabs/codis/&&/bin/codis-config-c/etc/codis/config_10.ini slot Init

3. Start Codis Redis, same as the official Redis server parameters (Codis-server operation)

cd/data/go/src/github.com/wandoulabs/codis/&&/bin/codis-server/etc/redis_6379.conf &

4, add Redis server group, each server group as a Redis server groups exist, only one master, can have multiple slave, the Group ID only supports integers greater than or equal to 1 (codis-confi Operation on G)

cd /data/go/src/github.com/wandoulabs/codis/./bin/codis-config  -c /etc/codis/config_10.ini server add 1 10.10.32.42:6379 master./bin/ codis-config -c /etc/codis/config_10.ini server add 1 10.10.32.43:6380  slave./bin/codis-config -c /etc/codis/config_10.ini server add 2  10.10.32.43:6379 master./bin/codis-config -c /etc/codis/config_10.ini server add  2 10.10.32.44:6380 slave./bin/codis-config -c /etc/codis/config_10.ini server  add 3 10.10.32.44:6379 master./bin/codis-config -c /etc/codis/config_10.ini  server add 3 10.10.32.42:6380 slave 

5, set the slot range of the server group service Codis using pre-sharding technology to achieve the data shard, by default divided into 1024 slots (0-1023), for each key, by the following formula to determine the owning slot I D:slotid = CRC32 (key)% 1024 each slot will have a specific server group ID to indicate which server group the data for this slot is provided by. (Operation on Codis-config)

Cd/data/go/src/github.com/wandoulabs/codis/./bin/codis-config-c/etc/codis/config_10.ini Slot Range-set 0 300 1 Online./bin/codis-config-c/etc/codis/config_10.ini slot range-set 301 2 online./bin/codis-config-c/etc/codis/ Config_10.ini Slot Range-set 701 1023 3 Online

6. Start Codis-proxy (operation on Codis-proxy)

Cat/etc/codis/config_10.ini # #撰写配置文件zk =10.10.0.47:2181,10.10.0.48:2181,10.10.1.76:2181product=zh_newsproxy_id= CODIS-PROXY_10 # #10.10.32.49 to codis-proxy_49, multiple proxy,proxy_id need unique net_timeout=5000proto=tcp4dashboard_addr= 10.10.32.10:18087

cd/data/go/src/github.com/wandoulabs/codis/&&/bin/codis-proxy-c/etc/codis/config_10.ini-l/data/log/ Codis-proxy_10.log--cpu=4--addr=0.0.0.0:19000--http-addr=0.0.0.0:11000 &cd/data/go/src/github.com/ wandoulabs/codis/&&/bin/codis-proxy-c/etc/codis/config_49.ini-l/data/log/codis-proxy_49.log--cpu=4--a ddr=0.0.0.0:19000--http-addr=0.0.0.0:11000 &

OK, the entire cluster has been built. Show Show

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M01/6B/E2/wKiom1U4woPj8_kiAAIq1SRxwxo029.jpg "title=" 222222222222.png "alt=" Wkiom1u4wopj8_kiaaiq1srxwxo029.jpg "/>

Vi. Codis-server ha

Codis-ha implementation of Codis-server master-slave switch, codis-server the main library hangs will promote a library from the main library, hanging from the library will set this from the library from the cluster offline

1. Installation

Export Gopath=/data/go/usr/local/go/bin/go Get github.com/ngaut/codis-hacd/data/go/src/github.com/ngaut/ Codis-hago BUILDCP codis-ha/data/go/src/github.com/wandoulabs/codis/bin/How to use: codis-ha--codis-config=dashboard Address : 18087--productname= Cluster project name

2. Using Supervisord to manage codis-ha process

Yum-y Install Supervisord

Add the following in/etc/supervisord.conf: [Program:codis-ha]autorestart = Truestopwaitsecs = 10startsecs = 1stopsignal = Quitcommand =/data/go/src/github.com/wandoulabs/codis/bin/codis-ha--codis-config=10.10.32.17:18087--productName =dh_tianqiuser = Rootstartretries = 3autostart = Trueexitcodes = 0,2

3. Start Supervisord Service

/etc/init.d/supervisord Startchkconfig Supervisord on

At this time, Ps-ef |grep codis-ha you back to find Codis-ha process has started, this time you go to stop a codis-server master, see if Slave will be promoted to master it

Vii. about monitoring

About the whole Codis cluster monitoring, we use the Zabbix, the monitoring of the indicators is relatively simple, so this piece of people have any good suggestions for me to mention Kazakhstan

Zookeeper: Monitoring the port connectivity of each node (thinking about monitoring the process later)

Codis-proxy: Monitoring the port connectivity, this monitoring is not enough

Codis-server: Monitor memory usage, number of connections, connectivity

Codis-ha: Monitoring Process

Dashboard: Monitoring Port connectivity

Viii. problems encountered in the use of the process

1, codis-proxy log cutting, codis-proxy the default log level is info, the log volume is very large, we have more than 50 g log per day, the current codis-proxy also does not support hot restart, want to modify the startup parameters or more troublesome, Recommended Logrotate for log cutting

2, Codis-proxy the monitoring address of the default no specific IPv4, that is, codis-proxy boot after no 0.0.0.0:19000 such a monitoring, this will cause the problem is the front-end LVS no way load balancing codis-proxy, can not forward the request , this issue has been contacted by the author, and in the Codis-proxy boot configuration file with PROTO=TCP4 This parameter supports monitoring IPv4

3, the addition of Redis Server Group, non-codis-server (native Redis) unexpectedly can also join into the Codis cluster, in the Redis and codis-server there is a physical machine on the clear, it is easy to add the wrong, hoping to have a verification, Non-codis-server cannot join the CODIS cluster

4, CODIS cluster internal communication is through the hostname, if the hostname does not do the domain name resolution that dashboard is accessed through the host name Proxy http-addr address, which will lead to the Web interface can not see the op/s data, As to whether there are any other problems, I have not found on this side, it is recommended that internal communications directly with the intranet IP

PS: I have established a management Codis QQ group, interested in Codis can join the Exchange, Group No.: 183613045

This article is from the "Dick Silk ops Man" blog, please be sure to keep this source http://navyaijm.blog.51cto.com/4647068/1637688

Codis Cluster Deployment combat