Redis dual-machine Hot Standby Solution

References:

Http://patrick-tang.blogspot.com/2012/06/redis-keepalived-failover-system.html

Http://deidara.blog.51cto.com/400447/302402

Http://my.oschina.net/guol/blog/182491

Http://shiguanghui.iteye.com/blog/2001499

Background

Currently, the official Redis cluster solution is still under development and testing and is not integrated into the stable version. And currently the official development of Redis Cluster provides the function is not perfect can refer to the official website or http://www.redisdoc.com/en/latest/topic/cluster-spec.html), in the production environment is not recommended to use. The survey found that a single IP address should be used for access on the market, and keepalived is mostly used for hot standby of redis as a transitional solution.

Environment deployment

Environment Introduction:
Master: 192.168.1.218 redis, keepalived
Slave: 192.168.1.219 redis, keepalived
Virtural IP Address (VIP): 192.168.1.220

 

Design Concept:

Two redis server master-slave backups. Provides high availability of the redis service, master-slave backup of two keepalived services, and high availability of the VIP service.

1) Each redis server has a master node, starting from two configuration files (redis_master.conf, redis_slave.conf) and starting the service through the startup script. the startup script will detect the roles of other servers in the redis cluster, if a master service exists, it will be started as the slave role; otherwise, it will be started as the master server;

2) keepalived monitoring script, scheduled frequency: once per second) checks whether the current server obtains the cluster VIP. If the server obtains the cluster VIP, it sets the redis server on the server as the master. At the same time, set other remote redis servers to slave; ensure that the role of the redis server that obtains the cluster VIP is master, and the other is set to slave.

3) The keepalived monitoring script also automatically checks whether the current redis server is normal. If two consecutive detection exceptions occur, stop the keepalived service of the local machine and release the cluster VIP, let it drift to other servers that can provide redis services;

4) When both the Master and Slave work normally, the Master is responsible for the service and the Slave is responsible for data synchronization. When the Master fails and the Slave is normal, the Slave takes over the service and disables the Master-Slave replication function; when the Master node returns to normal, the Server Load balancer synchronizes data from the Server Load balancer instance. After the data is synchronized, the Master node replication function is disabled to restore the Master node identity. At the same time, the Server Load balancer instance restores the Server Load balancer instance identity after the Master node. And then cyclically.

Implementation steps:

---- Create a dedicated user

Useradd-g develop redisadmin
Echo Hisun @ 1125 | passwd -- stdin redisadmin

Note: The following deployment processes are performed under the root (or account with sudo permissions) account.

---- Install and configure redis

1. Download The redis source code

Cd

Wget http://download.redis.io/releases/redis-2.8.3.tar.gz

2. Install redis

Tar-zxvf redis-2.8.3.tar.gz

Cd redis-2.8.3

# You do not need to execute configure for reds installation.

Make

# Test

Make test

#### The following errors may occur when you execute make test on a slow machine, without any impact

# *** [Err]: Test replication partial resync: no backlog in tests/integration/replication-psync.tcl

3. Configure redis

# Create a redis home directory

Mkdir-p/usr/local/redis-2.8.3/{bin, conf, logs}

Cp-a-R-p src/redis-server/usr/local/redis-2.8.3/bin/

Cp-a-R-p src/redis-cli/usr/local/redis-2.8.3/bin/

Cp-a-R-p src/redis-benchmark/usr/local/redis-2.8.3/bin/

Cp-a-R-p src/redis-sentinel/usr/local/redis-2.8.3/bin/

Cp-a-R-p src/redis-check-dump/usr/local/redis-2.8.3/bin/

Cp-a-R-p src/redis-check-aof/usr/local/redis-2.8.3/bin/

# Create a redis Startup Script

Vi/usr/local/redis-2.8.3/redis-start.sh

#### The following is the configuration on the master. You only need to modify the corresponding LOCALIP and REMOTEIP for the configuration on the slave.

#!/bin/bashREDISPATH=/usr/local/redis-2.8.3REDISCLI=$REDISPATH/bin/redis-cliLOGFILE=$REDISPATH/logs/redis-state.logLOCALIP=192.168.1.218REMOTEIP=192.168.1.219REMOTEREDISROLE=`$REDISCLI -h $REMOTEIP info | grep "role"`if grep "role:master" <<< $REMOTEREDISROLE ; then        #start as slave        $REDISPATH/bin/redis-server $REDISPATH/conf/redis_slave.conf        if [ "$?" == "0" ];then                echo "[INFO]`date +%F/%H:%M:%S` :$LOCALIP start as slave successful." >> $LOGFILE        else                echo "[ERROR]`date +%F/%H:%M:%S` :$LOCALIP start as slave error." >> $LOGFILE        fielse        #start as master        $REDISPATH/bin/redis-server $REDISPATH/conf/redis_master.conf        if [ "$?" == "0" ];then                echo "[INFO]`date +%F/%H:%M:%S` :$LOCALIP start as master successful." >> $LOGFILE        else                echo "[ERROR]`date +%F/%H:%M:%S` :$LOCALIP start as master error." >> $LOGFILE        fifi

# Create redis close script vi/usr/local/redis-2.8.3/redis-stop.sh #### The following is the configuration on the master, the configuration on the slave is the same.

#!/bin/bashREDISPATH=/usr/local/redis-2.8.3LOGFILE=$REDISPATH/logs/redis-state.logkill -9 `ps -ef|grep '/bin/redis-server'|grep -v grep|awk  '{print $2}'`if [ "$?" == "0" ];then        echo "[INFO]`date +%F/%H:%M:%S` :redis shutdown completed!" >> $LOGFILEelse        echo "[ERROR]`date +%F/%H:%M:%S` :redis is not started." >> $LOGFILEfi

# Create a redis configuration file

Cp-a-R-p redis. conf/usr/local/redis-2.8.3/conf/redis_master.confcp-a-R-p redis. conf/usr/local/redis-2.8.3/conf/redis_slave.conf # modify the redis_master.conf configuration item:

#### Configuration items ####### daemonize nodaemonize yes # bind 127.0.0.1bind 192.168.1.218logfile "/usr/local/redis-2.8.3/logs/redis. log "# modify other configurations according to the actual production environment ############################## ############################# configuration items of the slave server redis_master.conf #### ### daemonize nodaemonize yes # bind 127.0.0.1bind 192.168.1.219logfile "/usr/local/redis-2.8.3/logs/redis. log "# modify other configurations according to the actual production environment ############################## ##########################

# Modify the configuration items of redis_slave.conf:

#### Configuration items of the master server redis_slave.conf ######## daemonize nodaemonize yes # bind 127.0.0.1bind 192.168.1.218logfile "/usr/local/redis-2.8.3/logs/redis. log "# slaveof <masterip> <masterport> slaveof 192.168.1.219 6379 # modify other configurations according to the actual production environment ################### ######################################## #192.168.1.219 slave server redis_slave.conf configuration ######## daemonize nodaemonize yes # bind 127.0.0.1bind 192.168.1.219logfile "/usr/local/redis-2.8.3/logs/redis. log "# slaveof <masterip> <masterport> slaveof 192.168.1.218 6379 # modify other configurations according to the actual production environment ################### #####################################

# Modify the owner and permissions of redis

Chmod-R 750/usr/local/redis-2.8.3/

Chown-R redisadmin: develop/usr/local/redis-2.8.3/

---- Install and configure keepalived

1. Download The keepalived source code Release 1.2.9

Note: An error occurred while testing the latest version 1.2.10.
Wget http://www.keepalived.org/software/keepalived-1.2.9.tar.gz
2. Install keepalived

Install the following dependency package: make gcc libpopt-dev libnl-dev libcurl4-openssl-dev popt openssl
Cd

Tar-zxvf keepalived-1.2.9.tar.gz

Cd keepalived-1.2.9

./Configure -- prefix =/usr/local/keepalived

Make & make install

3. Configure keepalived

# The following configuration files can be created on the Master and Slave according to the actual situation ):

Mv/usr/local/keepalived/etc/keepalived. conf/usr/local/keepalived/etc/keepalived. conf-bak
Vim/usr/local/keepalived/etc/keepalived. conf

Vrrp_script chk_redis {script "/usr/local/keepalived/etc/keepalived/scripts/chk_redis.sh" ### monitoring script interval 2 ### monitoring time} vrrp_instance VI_1 {state MASTER ## # Set as MASTER interface eth3 ### monitor the network adapter, set virtual_router_id 51 priority 101 ### Weight Value advert_int 1 authentication {auth_type PASS ### encrypt auth_pass redis ### password} track_script {chk_redis ### execute the above definition chk_redis} virtual_ipaddress {192.168.1.220 ### VIP }}

Create scripts for monitoring Redis on Master and Slave
Mkdir/usr/local/keepalived/etc/keepalived/scripts
Vi/usr/local/keepalived/etc/keepalived/scripts/chk_redis.sh

#### The following is the configuration on the master. You only need to modify the corresponding LOCALIP and REMOTEIP for the configuration on the slave.

#!/bin/bashREDISPATH=/usr/local/redis-2.8.3REDISCLI=$REDISPATH/bin/redis-cliLOGFILE=$REDISPATH/logs/redis-state.logLOCALIP=192.168.1.218REMOTEIP=192.168.1.219VIP=192.168.1.220VIPALIVE=`ip a | grep "$VIP"`if [ "$VIPALIVE" == "" ]; thenecho "[info]:"`date`" keepalived server is pengding or stop" >> $LOGFILEelseecho "bbb" >> $LOGFILE#check local service is runningif [ "`$REDISCLI –h $LOCALIP –p 6379 PING`" == "PONG" ]; then# check local redis server role.REDISROLE=`$REDISCLI info | grep "role"`if grep "role:slave" <<< $REDISROLE ; then#change local redis server as master echo "[info1]:"`date`" Run SLAVEOF NO ONE cmd ..." >> $LOGFILE$REDISCLI SLAVEOF NO ONE >> $LOGFILE 2>&1#change remoting redis server as slaveREMOTEREDISROLE=`$REDISCLI -h $REMOTEIP info | grep "role"`if grep "role:master" <<< $REMOTEREDISROLE ; thenecho "[info2]:"`date`" Run remote server SLAVEOF cmd ..." >> $LOGFILE$REDISCLI -h $REMOTEIP SLAVEOF $LOCALIP 6379 >> $LOGFILE  2>&1fielseREMOTEREDISROLE=`$REDISCLI -h $REMOTEIP info | grep "role"`if grep "role:master" <<< $REMOTEREDISROLE ; thenecho "[info3]:"`date`" Run remote server SLAVEOF cmd ..." >> $LOGFILE$REDISCLI -h $REMOTEIP SLAVEOF $LOCALIP 6379 >> $LOGFILE  2>&1fifielseecho "[warn]:"`date`"  redis server($LOCALIP) is not health..." >> $LOGFILEsleep 1if [ "`$REDISCLI –h $LOCALIP –p 6379 PING`" != "PONG" ]; thenecho "[error]:"`date`"  redis server($LOCALIP) will be stop..." >> $LOGFILEservice keepalived stopfififi

Important:Place the corresponding configuration file in the corresponding place.

# First, set the keepalived Startup File on two servers:

Cp-a-R-p/usr/local/keepalived/etc/rc. d/init. d/keepalived/etc/rc. d/init. d/keepalived

Chmod 750/etc/rc. d/init. d/keepalived

Chown root/etc/rc. d/init. d/keepalived
# Create a link to the configuration file on two servers:

Mkdir/etc/keepalived/

Ln-s/usr/local/keepalived/etc/keepalived. conf/etc/keepalived. conf

Ln-s/usr/local/keepalived/etc/sysconfig/keepalived

 

System Test

Note: you must first start redis and then start keealived. Otherwise, redis_check.sh will automatically disable keepalived.

After the script is created, perform the test as follows:
1. Start Redis on the Master
/Usr/local/redis-2.8.3/redis-start.sh

# Directly kill the process or execute the following script when the process is disabled

/Usr/local/redis-2.8.3/redis-stop.sh

2. Start Redis on Slave
/Usr/local/redis-2.8.3/redis-start.sh

# Directly kill the process or execute the following script when the process is disabled

#/Usr/local/redis-2.8.3/redis-stop.sh

3. Start Keepalived on the Master.
/Etc/rc. d/init. d/keepalived start

# Closing Method

#/Etc/rc. d/init. d/keepalived stop

4. Start Keepalived on Slave.
/Etc/rc. d/init. d/keepalived start

# Closing Method

#/Etc/rc. d/init. d/keepalived stop

This article from the "stick" blog, please be sure to keep this source http://luyx30.blog.51cto.com/1029851/1350832