Redis雙機熱備方案

參考資料：

 

http://patrick-tang.blogspot.com/2012/06/redis-keepalived-failover-system.html

http://deidara.blog.51cto.com/400447/302402

http://my.oschina.net/guol/blog/182491

http://shiguanghui.iteye.com/blog/2001499

 

背景

 

目前，Redis叢集的官方方案還處在開發測試中，未整合到穩定版中。且目前官方開發中的Redis Cluster提供的功能尚不完善可參考官方網站或http://www.redisdoc.com/en/latest/topic/cluster-spec.html），在生產環境中不推薦使用。通過調研發現市面上要實現採用單一的IP來訪問，大多採用keepalived實現redis的雙機熱備作為過渡方案。

 

環境部署

 

環境介紹：   
Master: 192.168.1.218     redis，keepalived    
Slave: 192.168.1.219        redis，keepalived    
Virtural IP Address (VIP):  192.168.1.220

 

設計思路：

兩個redis server主從備份。提供redis 服務高可用；兩個keepalived 服務主從備份，提供VIP 服務的高可用。

1)每台redis server分別有主，從兩個設定檔(redis_master.conf, redis_slave.conf)，通過啟動指令碼啟動服務，啟動指令碼會檢測這個redis叢集中的其他伺服器的角色，如果有master 服務存在，則以slave角色啟動，否則將自己以master伺服器啟動；

2) keepalived 監控指令碼，定時頻率：每秒一次）檢測當前伺服器是否擷取叢集VIP，如果擷取叢集VIP，則將本伺服器上的redis伺服器設定為master。同時將遠端的其他redis伺服器設定為slave；保證擷取叢集VIP的redis 伺服器角色為master，其他的設定為slave。

3) keepalived 監控指令碼，還會自動檢測當前redis伺服器是否正常，如果連續兩次檢測異常，則停掉本原生keepalived服務，釋放叢集VIP，讓去漂移到其他可以提供redis 服務的伺服器上；

4)當 Master 與 Slave 均運作正常時, Master負責服務，Slave負責同步資料；當 Master 掛掉，Slave 正常時, Slave接管服務，同時關閉主從複製功能；當 Master 恢複正常，則從Slave同步資料，同步資料之後關閉主從複製功能，恢複Master身份，於此同時Slave等待Master同步資料完成之後，恢複Slave身份。然後依次迴圈。

 

實施步驟：

----建立專用使用者

useradd -g develop redisadmin   
echo Hisun@1125|passwd --stdin redisadmin

說明：以下部署過程都是在root(或具備sudo許可權的帳號)賬戶下進行。

----安裝配置redis

1.下載redis源碼

cd

wget http://download.redis.io/releases/redis-2.8.3.tar.gz

2.安裝redis

tar -zxvf redis-2.8.3.tar.gz

cd redis-2.8.3

#reds的安裝可以不用執行configure

make

#測試

make test

####在速度較慢的機器上執行make test可能出現下列錯誤，無影響

#*** [err]: Test replication partial resync: no backlog in tests/integration/replication-psync.tcl

3.配置redis

#建立redis主目錄

mkdir -p /usr/local/redis-2.8.3/{bin,conf,logs}

cp -a -R -p src/redis-server /usr/local/redis-2.8.3/bin/

cp -a -R -p src/redis-cli /usr/local/redis-2.8.3/bin/

cp -a -R -p src/redis-benchmark /usr/local/redis-2.8.3/bin/

cp -a -R -p src/redis-sentinel /usr/local/redis-2.8.3/bin/

cp -a -R -p src/redis-check-dump /usr/local/redis-2.8.3/bin/

cp -a -R -p src/redis-check-aof /usr/local/redis-2.8.3/bin/

#建立redis啟動指令碼

vi /usr/local/redis-2.8.3/redis-start.sh

####以下為master上的配置，slave上的配置只需要修改對應的LOCALIP和REMOTEIP即可。

#!/bin/bashREDISPATH=/usr/local/redis-2.8.3REDISCLI=$REDISPATH/bin/redis-cliLOGFILE=$REDISPATH/logs/redis-state.logLOCALIP=192.168.1.218REMOTEIP=192.168.1.219REMOTEREDISROLE=`$REDISCLI -h $REMOTEIP info | grep "role"`if grep "role:master" <<< $REMOTEREDISROLE ; then        #start as slave        $REDISPATH/bin/redis-server $REDISPATH/conf/redis_slave.conf        if [ "$?" == "0" ];then                echo "[INFO]`date +%F/%H:%M:%S` :$LOCALIP start as slave successful." >> $LOGFILE        else                echo "[ERROR]`date +%F/%H:%M:%S` :$LOCALIP start as slave error." >> $LOGFILE        fielse        #start as master        $REDISPATH/bin/redis-server $REDISPATH/conf/redis_master.conf        if [ "$?" == "0" ];then                echo "[INFO]`date +%F/%H:%M:%S` :$LOCALIP start as master successful." >> $LOGFILE        else                echo "[ERROR]`date +%F/%H:%M:%S` :$LOCALIP start as master error." >> $LOGFILE        fifi

#建立redis關閉指令碼vi /usr/local/redis-2.8.3/redis-stop.sh####以下為master上的配置，slave上的配置相同。

#!/bin/bashREDISPATH=/usr/local/redis-2.8.3LOGFILE=$REDISPATH/logs/redis-state.logkill -9 `ps -ef|grep '/bin/redis-server'|grep -v grep|awk  '{print $2}'`if [ "$?" == "0" ];then        echo "[INFO]`date +%F/%H:%M:%S` :redis shutdown completed!" >> $LOGFILEelse        echo "[ERROR]`date +%F/%H:%M:%S` :redis is not started." >> $LOGFILEfi

#建立redis設定檔

cp -a -R -p redis.conf /usr/local/redis-2.8.3/conf/redis_master.confcp -a -R -p redis.conf /usr/local/redis-2.8.3/conf/redis_slave.conf#修改redis_master.conf對應配置項：

####192.168.1.218主伺服器redis_master.conf對應配置項#######daemonize nodaemonize yes#bind 127.0.0.1bind 192.168.1.218logfile "/usr/local/redis-2.8.3/logs/redis.log"#其他配置依據實際生產環境修改############################################################192.168.1.219從伺服器redis_master.conf對應配置項#######daemonize nodaemonize yes#bind 127.0.0.1bind 192.168.1.219logfile "/usr/local/redis-2.8.3/logs/redis.log"#其他配置依據實際生產環境修改########################################################

#修改redis_slave.conf對應配置項：

####192.168.1.218主伺服器redis_slave.conf對應配置項########daemonize nodaemonize yes#bind 127.0.0.1bind 192.168.1.218logfile "/usr/local/redis-2.8.3/logs/redis.log"# slaveof <masterip> <masterport>slaveof 192.168.1.219 6379#其他配置依據實際生產環境修改############################################################192.168.1.219從伺服器redis_slave.conf對應配置項########daemonize nodaemonize yes#bind 127.0.0.1bind 192.168.1.219logfile "/usr/local/redis-2.8.3/logs/redis.log"# slaveof <masterip> <masterport>slaveof 192.168.1.218 6379#其他配置依據實際生產環境修改########################################################

#修改redis的屬主和許可權

chmod –R 750 /usr/local/redis-2.8.3/

chown –R redisadmin:develop /usr/local/redis-2.8.3/

----安裝配置keepalived  

1.下載keepalived源碼Release 1.2.9

注意：最新版為1.2.10測試過程中出錯.  
wget http://www.keepalived.org/software/keepalived-1.2.9.tar.gz
2.安裝keepalived

需要安裝以下依賴包： make gcc libpopt-dev libnl-dev libcurl4-openssl-dev popt openssl  
cd

tar –zxvf keepalived-1.2.9.tar.gz

cd keepalived-1.2.9

./configure --prefix=/usr/local/keepalived

make && make install

3.配置keepalived

#在Master和Slave上建立如下設定檔可根據實際情況調整）：

mv  /usr/local/keepalived/etc/keepalived/keepalived.conf /usr/local/keepalived/etc/keepalived/keepalived.conf-bak
vim /usr/local/keepalived/etc/keepalived/keepalived.conf

vrrp_script chk_redis {        script "/usr/local/keepalived/etc/keepalived/scripts/chk_redis.sh"   ###監控指令碼        interval 2                                        ###監控時間}vrrp_instance VI_1 {        state MASTER                            ###設定為MASTER        interface eth3                          ###監控網卡,依據實際情況來定          virtual_router_id 51        priority 101                            ###權重值        advert_int 1         authentication {                     auth_type PASS             ###加密                     auth_pass redis            ###密碼        }        track_script {                chk_redis                       ###執行上面定義的chk_redis        }        virtual_ipaddress {             192.168.1.220                         ###VIP        }}

在Master和Slave上建立監控Redis的指令碼  
mkdir /usr/local/keepalived/etc/keepalived/scripts  
vi /usr/local/keepalived/etc/keepalived/scripts/chk_redis.sh

####以下為master上的配置，slave上的配置只需要修改對應的LOCALIP和REMOTEIP即可。

#!/bin/bashREDISPATH=/usr/local/redis-2.8.3REDISCLI=$REDISPATH/bin/redis-cliLOGFILE=$REDISPATH/logs/redis-state.logLOCALIP=192.168.1.218REMOTEIP=192.168.1.219VIP=192.168.1.220VIPALIVE=`ip a | grep "$VIP"`if [ "$VIPALIVE" == "" ]; thenecho "[info]:"`date`" keepalived server is pengding or stop" >> $LOGFILEelseecho "bbb" >> $LOGFILE#check local service is runningif [ "`$REDISCLI –h $LOCALIP –p 6379 PING`" == "PONG" ]; then# check local redis server role.REDISROLE=`$REDISCLI info | grep "role"`if grep "role:slave" <<< $REDISROLE ; then#change local redis server as master echo "[info1]:"`date`" Run SLAVEOF NO ONE cmd ..." >> $LOGFILE$REDISCLI SLAVEOF NO ONE >> $LOGFILE 2>&1#change remoting redis server as slaveREMOTEREDISROLE=`$REDISCLI -h $REMOTEIP info | grep "role"`if grep "role:master" <<< $REMOTEREDISROLE ; thenecho "[info2]:"`date`" Run remote server SLAVEOF cmd ..." >> $LOGFILE$REDISCLI -h $REMOTEIP SLAVEOF $LOCALIP 6379 >> $LOGFILE  2>&1fielseREMOTEREDISROLE=`$REDISCLI -h $REMOTEIP info | grep "role"`if grep "role:master" <<< $REMOTEREDISROLE ; thenecho "[info3]:"`date`" Run remote server SLAVEOF cmd ..." >> $LOGFILE$REDISCLI -h $REMOTEIP SLAVEOF $LOCALIP 6379 >> $LOGFILE  2>&1fifielseecho "[warn]:"`date`"  redis server($LOCALIP) is not health..." >> $LOGFILEsleep 1if [ "`$REDISCLI –h $LOCALIP –p 6379 PING`" != "PONG" ]; thenecho "[error]:"`date`"  redis server($LOCALIP) will be stop..." >> $LOGFILEservice keepalived stopfififi

重要：將相應的設定檔放到相應的地方.

#首先在2台伺服器上設定keepalived的開機檔案：

cp -a -R -p /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/keepalived

chmod 750 /etc/rc.d/init.d/keepalived

chown root /etc/rc.d/init.d/keepalived
#然後在2台伺服器上建立設定檔的連結：

mkdir /etc/keepalived/

ln -s /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf

ln -s /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/keepalived

 

系統測試

注意:一定要先啟動redis，再啟動keealived，否則redis_check.sh會將keepalived自動關閉。

指令碼建立完成以後，我們開始按照如下流程進行測試：
1.啟動Master上的Redis
/usr/local/redis-2.8.3/redis-start.sh

#關閉時，直接殺死進程或執行以下指令碼

/usr/local/redis-2.8.3/redis-stop.sh

2.啟動Slave上的Redis
/usr/local/redis-2.8.3/redis-start.sh

#關閉時，直接殺死進程或執行以下指令碼

#/usr/local/redis-2.8.3/redis-stop.sh

3.啟動Master上的Keepalived
/etc/rc.d/init.d/keepalived start

#關閉方法

#/etc/rc.d/init.d/keepalived stop

4.啟動Slave上的Keepalived
/etc/rc.d/init.d/keepalived start

#關閉方法

#/etc/rc.d/init.d/keepalived stop

本文出自 “堅持” 部落格，請務必保留此出處http://luyx30.blog.51cto.com/1029851/1350832