Currently, Redis does not have an official HA scenario similar to MySQL proxy or Oracle RAC.
The Redis author has a plan called Redis Sentinel, which is said to have three major functions: monitoring, alerting and automatic failover.
But unfortunately it is not possible to complete the development in the short term.
Therefore, how to automatically transfer in the event of a failure is a problem that needs to be solved.
Through the search for some information on the Internet, it is recommended to use Haproxy or keepalived to achieve, in fact, if it is to do failover rather than load balancing, keepalived efficiency is certainly more than Haproxy, So I decided to adopt the keepalived scheme.
Environment Introduction:
master:10.6.1.143
slave:10.6.1.144
Virtural IP Address (VIP): 10.6.1.200
Design ideas:
When Master and Slave are working normally, Master is responsible for service, Slave is responsible for standby;
When Master hangs up, Slave is normal, Slave takes over the service and shuts down the master-slave copy function;
When Master returns to normal, the data is synchronized from slave, the master-slave replication function is turned off after the data is synchronized, and the master identity is restored, while slave waits for the master synchronization data to complete before resuming slave identity.
Then loop in turn.
It is important to note that this requires a localization strategy on both master and slave, otherwise, in the automatic switching process, the other party's data will be emptied by the non-open side, resulting in a complete loss of data.
Below, is the specific implementation steps:
Installing keepalived on Master and slave
$ sudo apt-get install keepalived
Modify the/etc/hosts file for master and slave
$ sudo vim/etc/hosts
127.0.0.1localhost
10.6.1.143redis
10.6.1.144redis-slave
There is no configuration file after the default installation is complete keepalived, so we need to create it manually:
First, create the following configuration file on master:
$ sudo vim/etc/keepalived/keepalived.conf
Vrrp_script Chk_redis {
Script "/etc/keepalived/scripts/redis_check.sh" # # #监控脚本
Interval 2 # # #监控时间
}
Vrrp_instance Vi_1 {
State MASTER # # #设置为MASTER
Interface Eth0 # # #监控网卡
VIRTUAL_ROUTER_ID 51
Priority 101 # # #权重值
Authentication {
Auth_type PASS # # #加密
Auth_pass Redis # # #密码
}
Track_script {
Chk_redis # # #执行上面定义的chk_redis
}
virtual_ipaddress {
10.6.1.200 # # #VIP
}
notify_master/etc/keepalived/scripts/redis_master.sh
notify_backup/etc/keepalived/scripts/redis_backup.sh
notify_fault/etc/keepalived/scripts/redis_fault.sh
notify_stop/etc/keepalived/scripts/redis_stop.sh
}
Then, create the following configuration file on slave:
$ sudo vim/etc/keepalived/keepalived.conf
Vrrp_script Chk_redis {
Script "/etc/keepalived/scripts/redis_check.sh" # # #监控脚本
Interval 2 # # #监控时间
}
Vrrp_instance Vi_1 {
State BACKUP # # #设置为BACKUP
Interface Eth0 # # #监控网卡
VIRTUAL_ROUTER_ID 51
Priority # # # #比MASTRE权重值低
Authentication {
Auth_type PASS
Auth_pass Redis # # #密码与MASTRE相同
}
Track_script {
Chk_redis # # #执行上面定义的chk_redis
}
virtual_ipaddress {
10.6.1.200 # # #VIP
}
notify_master/etc/keepalived/scripts/redis_master.sh
notify_backup/etc/keepalived/scripts/redis_backup.sh
notify_fault/etc/keepalived/scripts/redis_fault.sh
notify_stop/etc/keepalived/scripts/redis_stop.sh
}
Creating a script to monitor Redis on master and slave
$ sudo mkdir/etc/keepalived/scripts
$ sudo vim/etc/keepalived/scripts/redis_check.sh
#!/bin/bash
Alive= '/opt/redis/bin/redis-cli PING '
If ["$ALIVE" = = "PONG"]; Then
Echo $ALIVE
Exit 0
Else
Echo $ALIVE
Exit 1
Fi
Write the following key scripts that are responsible for the operation:
notify_master/etc/keepalived/scripts/redis_master.sh
notify_backup/etc/keepalived/scripts/redis_backup.sh
notify_fault/etc/keepalived/scripts/redis_fault.sh
notify_stop/etc/keepalived/scripts/redis_stop.sh
Because the keepalived is called according to the state when the state is converted:
Notify_master is called when the master State is entered
Notify_backup is called when the backup status is entered
Enter fault status call Notify_fault when abnormal conditions are found
Call Notify_stop when the keepalived program terminates
First, create the Notity_master and Notify_backup scripts on the Redis master:
$ sudo vim/etc/keepalived/scripts/redis_master.sh
#!/bin/bash
Rediscli= "/opt/redis/bin/redis-cli"
Logfile= "/var/log/keepalived-redis-state.log"
echo "[Master]" >> $LOGFILE
Date >> $LOGFILE
echo "Being master ..." >> $LOGFILE 2>&1
echo "Run slaveof cmd ..." >> $LOGFILE
$REDISCLI slaveof 10.6.1.144 6379 >> $LOGFILE 2>&1
Sleep #延迟10秒以后待数据同步完成后再取消同步状态
echo "Run slaveof NO one cmd ..." >> $LOGFILE
$REDISCLI slaveof NO one >> $LOGFILE 2>&1
$ sudo vim/etc/keepalived/scripts/redis_backup.sh
#!/bin/bash
Rediscli= "/opt/redis/bin/redis-cli"
Logfile= "/var/log/keepalived-redis-state.log"
echo "[Backup]" >> $LOGFILE
Date >> $LOGFILE
echo "Being slave ..." >> $LOGFILE 2>&1
Sleep #延迟15秒待数据被对方同步完成之后再切换主从角色
echo "Run slaveof cmd ..." >> $LOGFILE
$REDISCLI slaveof 10.6.1.144 6379 >> $LOGFILE 2>&1
Next, create the Notity_master and Notify_backup scripts on the Redis slave:
$ sudo vim/etc/keepalived/scripts/redis_master.sh
#!/bin/bash
Rediscli= "/opt/redis/bin/redis-cli"
Logfile= "/var/log/keepalived-redis-state.log"
echo "[Master]" >> $LOGFILE
Date >> $LOGFILE
echo "Being master ..." >> $LOGFILE 2>&1
echo "Run slaveof cmd ..." >> $LOGFILE
$REDISCLI slaveof 10.6.1.143 6379 >> $LOGFILE 2>&1
Sleep #延迟10秒以后待数据同步完成后再取消同步状态
echo "Run slaveof NO one cmd ..." >> $LOGFILE
$REDISCLI slaveof NO one >> $LOGFILE 2>&1
$ sudo vim/etc/keepalived/scripts/redis_backup.sh
#!/bin/bash
Rediscli= "/opt/redis/bin/redis-cli"
Logfile= "/var/log/keepalived-redis-state.log"
echo "[Backup]" >> $LOGFILE
Date >> $LOGFILE
echo "Being slave ..." >> $LOGFILE 2>&1
Sleep #延迟15秒待数据被对方同步完成之后再切换主从角色
echo "Run slaveof cmd ..." >> $LOGFILE
$REDISCLI slaveof 10.6.1.143 6379 >> $LOGFILE 2>&1
Then create the same script as the following in master and slave:
$ sudo vim/etc/keepalived/scripts/redis_fault.sh
#!/bin/bash
Logfile=/var/log/keepalived-redis-state.log
echo "[Fault]" >> $LOGFILE
Date >> $LOGFILE
$ sudo vim/etc/keepalived/scripts/redis_stop.sh
#!/bin/bash
Logfile=/var/log/keepalived-redis-state.log
echo "[Stop]" >> $LOGFILE
Date >> $LOGFILE
Add executable permissions to the script:
$ sudo chmod +x/etc/keepalived/scripts/*.sh
After the script has been created, we begin to test it in the following process:
1. Start Redis on Master
$ Sudo/etc/init.d/redis Start
2. Start Redis on the slave
$ Sudo/etc/init.d/redis Start
3. Start the keepalived on master
$ sudo/etc/init.d/keepalived Start
4. Start the keepalived on the slave
$ sudo/etc/init.d/keepalived Start
5. Try to connect Redis via VIP:
$ redis-cli-h 10.6.1.200 INFO
The connection was successful and slave was connected.
Role:master
Slave0:10.6.1.144,6379,online
6. Try inserting some data:
$ redis-cli-h 10.6.1.200 SET Hello Redis
Ok
Read data from VIP
$ redis-cli-h 10.6.1.200 GET Hello
"Redis"
reading data from Master
$ redis-cli-h 10.6.1.143 GET Hello
"Redis"
reading data from slave
$ redis-cli-h 10.6.1.144 GET Hello
"Redis"
Below, a simulated failure occurs:
Kill the Redis process on master:
$ sudo killall-9 redis-server
View the keepalived log on Master
$ tailf/var/log/keepalived-redis-state.log
[Fault]
Thu Sep 08:29:01 CST 2012
At the same time the log on slave shows:
$ tailf/var/log/keepalived-redis-state.log
[Master]
Fri Sep 14:14:09 CST 2012
Being master ....
Run slaveof cmd ...
Ok
Run slaveof NO One cmd ...
Ok
Then we can see that slave has taken over the service and assumed the role of master.
$ redis-cli-h 10.6.1.200 INFO
$ redis-cli-h 10.6.1.144 INFO
Role:master
Then we restore the master Redis process
$ Sudo/etc/init.d/redis Start
View the keepalived log on Master
$ tailf/var/log/keepalived-redis-state.log
[Master]
Thu Sep 08:31:33 CST 2012
Being master ....
Run slaveof cmd ...
Ok
Run slaveof NO One cmd ...
Ok
At the same time the log on slave shows:
$ tailf/var/log/keepalived-redis-state.log
[Backup]
Fri Sep 14:16:37 CST 2012
Being slave ....
Run slaveof cmd ...
Ok
You can see that the current master has resumed the master role again, and failover and automatic recovery have been successful.
Reproduced
Why many people use keepalived to achieve Redis failover