Redis High-availability architecture

I. BACKGROUND

The company's business in a large number of use of redis, access to a large business we have in use Codis cluster, Redis 3 Group, speaking of Redis 3 cluster, we have been on the line for more than half a year, the cluster itself has not had a task problem, but because we this business is overseas, Cluster built on the AWS EC2, due to EC2 network jitter or EC2 itself, leading to the master-slave switch, the current AWS technology is following up, the cluster of the current QPS 50w+, the cluster itself has achieved high availability and scale-out, but the actual situation some small business does not need to cluster, A single instance can meet the business needs, then we have to find out how to guarantee the high availability of a single instance, recently also looking at the relevant documents, do some testing, we have in the use of Redis master-slave +lvs Bleaching VIP program, there is the use of Redis master-slave + Sentinel Drift VIP program, There are even in the code logic do failover and so on, a variety of scenarios have, below I introduce the Redis master-slave + Sentinel drift VIP scheme, we intend to use this program on a large scale online.

Second, the environment

#redis100.10.32.54:6400 Main Library 100.10.32.55:6400 100.10.32.250 vip#sentinel100.10.32.54:26400 Sentinel from Library Local Node 100.10.32.54:26400 Sentinel local node 100.10.32.57:26400 Sentinel quorum node

III. deployment

1. Installation

Yum-y Install Redis

2. Compose Redis configuration files (100.10.32.54 and 100.10.32.55)

vim /etc/redis_6400.confdaemonize yespidfile  "/var/run/redis_6400.pid" Port 6400tcp-backlog  65535bind 0.0.0.0timeout 0tcp-keepalive 0loglevel noticelogfile  "/var/log/redis /redis_6400.log "Maxmemory 8gbmaxmemory-policy allkeys-lrudatabases 16save 900 1save  300 10save 60 10000stop-writes-on-bgsave-error yesrdbcompression yesrdbchecksum  yesdbfilename  "Dump.rdb" dir  "/data/redis/6400" slave-serve-stale-data yesslave-read-only  yesrepl-disable-tcp-nodelay noslave-priority 100appendonly noappendfilename  " Appendonly.aof "appendfsync everysecno-appendfsync-on-rewrite noauto-aof-rewrite-percentage  100auto-aof-rewrite-min-size 64mblua-time-limit 5000slowlog-log-slower-than  10000slowlog-max-len 128notify-keyspace-events  "" hash-max-ziplist-entries  512hash-max-ziplist-value 64list-max-ziplist-entries 512list-max-ziplist-value 64set-max-intset-entries 512zset-max-ziplist-entries 128 

3. Compose Sentinel profiles (100.10.32.54, 100.10.32.55, and 100.10.32.57)

Vim/etc/redis-sentinel6400.confdaemonize yesport 26400dir "/data/redis/redis_sentinels" Pidfile "/var/run/redis/ Sentinel6400.pid "logfile"/data/redis/redis_sentinels/sentinel6400.log "Sentinel Monitor master6400 100.10.32.54 6400 2sentinel down-after-milliseconds master6400 6000sentinel failover-timeout master6400 18000sentinel Client-reconfig-script master6400/opt/notify_master6400.sh # #仲裁节点无需添加这行配置, The Client-reconfig-script parameter is called during Sentinel's failover process to call the script drift VIP to the new master

Ps:

For some of the operating principles and parameter descriptions of Sentinel, see: http://redisdoc.com/topic/sentinel.html

4, write the floating VIP script (100.10.32.54, 100.10.32.55)

vim/opt/notify_master6400.sh#!/bin/bashmaster_ip=$6local_ip= ' 100.10.32.54 ' #从库修改为100.10.32.55vip= ' 100.10.32.250 ' netmask= ' interface= ' eth0 ' if [${master_ip} = ${local_ip}];        THEN/SBIN/IP addr Add ${vip}/${netmask} Dev ${interface}/sbin/arping-q-C 3-a ${VIP}-I ${interface} Exit 0else/sbin/ip addr del ${vip}/${netmask} dev ${interface} exit 0fiexit 1

chmod +x/opt/notify_master6400.sh #赋予可执行权限

Ps:

Here's how this script works, Sentinel will have 6 parameters in the process of doing failover, namely <master-name>, <role>, <state>, <from-ip >, <from-port>, <to-ip>, <to-port>, where the 6th parameter from-ip is the IP of the new master, corresponding to the master_ip in the script, The following if judge everyone should be very clear, if master_ip=local_ip, then bind VIP, conversely delete VIP.

5. Start Redis Service (100.10.32.54, 100.10.32.55)

Redis-server/etc/redis_6400.conf

6. Initialize master/slave (100.10.32.55)

Redis-cli-p 6400 slaveof 10.10.32.54 6400

7. Bind VIP to Main library (100.10.32.54)

/SBIN/IP addr Add 100.10.32.250/24 dev eth0

8. Launch Sentinel Service (100.10.32.54, 100.10.32.55, 100.10.32.57)

Redis-server/etc/redis-sentinel6400.conf--sentinel

At this point, the entire high-availability solution has been built.

[Email protected] tmp]# redis-cli-h 100.10.32.54-p 6400 Info replication# Replicationrole:masterconnected_slaves:1sla Ve0:ip=100.10.32.55,port=6400,state=online,offset=72669,lag=1master_repl_offset:72669repl_backlog_active:1repl _backlog_size:1048576repl_backlog_first_byte_offset:2repl_backlog_histlen:72668

[Email protected] tmp]# redis-cli-h 100.10.32.54-p 26400 info sentinel# sentinelsentinel_masters:1sentinel_tilt:0sent inel_running_scripts:0sentinel_scripts_queue_length:0master0:name=master6400,status=ok,address= 100.10.32.54:6400,slaves=1,sentinels=3

[[Email protected] tmp]# IP a |grep eth02:eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> MTU Qdisc MQ State up Qlen 1 inet 100.10.32.54/24 BRD 100.10.32.255 scope global eth0 inet 100.10.32.250/24 scope Global secondary eth0

Iv. Testing

1. Shut down the main library

Redis-cli-h 100.10.32.54-p 6400 shutdown

2, see if the library is promoted to the main library

[Email protected] tmp]# redis-cli-h 100.10.32.55-p 6400 Info replication# replicationrole:masterconnected_slaves:0 Master_repl_offset:0repl_backlog_active:0repl_backlog_size:1048576repl_backlog_first_byte_offset:0repl_backlog _histlen:0

3, see if VIP drift to 100.10.32.55 on

[[Email protected] tmp]# IP a |grep eth02:eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> MTU Qdisc MQ State up Qlen 1 inet 100.10.32.55/24 BRD 100.10.32.255 scope global eth0 inet 100.10.32.250/24 scope Global secondary eth0

4. Watch Sentinel's monitoring status

[Email protected] tmp]# redis-cli-p 26400 info sentinel# sentinelsentinel_masters:1sentinel_tilt:0sentinel_running_ Scripts:0sentinel_scripts_queue_length:0master0:name=master6400,status=ok,address=100.10.32.55:6400,slaves=1, Sentinels=3

This article is from the "Dick Silk ops Man" blog, please be sure to keep this source http://navyaijm.blog.51cto.com/4647068/1745569

Redis High-availability architecture