Highly available Redis cluster based on keepalived and Redis Sentinel

Hardware

Machine name	Ip	Role
Master	192.168.0.2	The master server for Redis
Slave1	192.168.0.3	Redis's Slave server
Slave2	192.168.0.4	Redis's Slave server
Route1	192.168.0.5 "Virtual ip:192.168.0.7"	Keepalived and Redis Sentinel Server, which hosts the VIP "virtual IP" of the write Redis, writes the master master designation for dual-machine hot standby
Route2	192.168.0.6 "Virtual ip:192.168.0.8"	Keepalived and Redis Sentinel server, host read Redis VIP, do read load balancer and write the master backup route specified for dual machine hot standby

Installation and configuration see this article installation and configuration

Detailed keepalived configuration, route1

! Configuration File for keepalived

global_defs {
   notification_email {
     mailbox
   }
   Notification_email_ From mailbox
   smtp_server Mailbox server address
   smtp_connect_timeout
   router_id lvs_devel
}
vrrp_instance vi_1 { State
    MASTER
    interface eth0 #eth0是要绑定的网卡
    virtual_router_id #同一个vrrp_instance中的值必须一样  
    Priority   #master的值要高于backup的
    Advert_int 1  
    authentication {
        Auth_type PASS
        auth_pass 1111
    }    
    virtual_ipaddress {
        192.168.0.7  #用于双机热备的虚拟ip
    }
   
}
virtual_server 192.168.0.7 6379 {
    delay_loop 3
    lb_algo wrr
    lb_kind DR
    persistence_timeout Protocol
    TCP
    real_server 192.168.0.2 6379 {
        weight 8
        notify_down The path/script name of the script to execute after the Redis service failed  #服务失败后要执行的脚本
        Tcp_check {
        connect_timeout 1
        nb_get_retry 3
        delay_before_retry 3
        connect_port 6379
        }
    }
}

Configuration file for Route2

! Configuration File for keepalived global_defs {notification_email {mailbox} Notification_email_from mailbox SMTP _server Mailbox server address Smtp_connect_timeout router_id Lvs_devel} vrrp_instance vi_1 {State MASTER interface E Th0 #eth0是要绑定的网卡 virtual_router_id #同一个vrrp_instance中的值必须一样 priority #master的值要高于backup的 advert_in T 1 authentication {auth_type PASS auth_pass 1111} virtual_ipaddress {192.168 .0.7 #用于双机热备的虚拟ip} notify_master "/etc/script/redis.sh"} vrrp_instance vi_2 {state Master #将此slave作为读数据的ma Ster interface Eth0 virtual_router_id 101 priority 151 Advert_int 1 authentication {Auth_type PAS S Auth_pass 1111} virtual_ipaddress {192.168.0.8 #用于读取数据的负载均衡的虚拟ip}} virtual_server 192.168.0.7 6 379 {Delay_loop 3 Lb_algo WRR lb_kind DR persistence_timeout protocol TCP real_server 192.
       168.0.3 6379 { Weight 8 Tcp_check {connect_timeout nb_get_retry 3 delay_before_retry 3 Conne Ct_port 6379}}} virtual_server 192.168.0.8 6379 {delay_loop 3 lb_algo wrr lb_kind DR Persi Stence_timeout protocol TCP real_server 192.168.0.3 6379 {weight 5 tcp_check {Connect _timeout nb_get_retry 3 delay_before_retry 3 Connect_port 6379}} real_server 192.168.0.4 6379 {Weight 5 tcp_check {connect_timeout Nb_get_retry 3 delay_be
 Fore_retry 3 Connect_port 6379}}

Redis Maintenance Script redis.sh

#!/usr/bin/env bash
/usr/local/bin/redis-cli-h 192.168.0.2-p 6379 shutdown
/usr/local/bin/redis-cli-h 192.168.0.3-p 6379 Config set appendonly no

This script is used in the Route2 notify_master, which is executed when Route2 enters master

Set up a master-slave relationship for Redis

Setting up a Redis Sentinel, configuration file

# Example Sentinel.conf # port <sentinel-port> # The Port of this sentinel instance would run on Port 26379 # sen Tinel Monitor <master-name> <ip> <redis-port> <quorum> # # tells Sentinel to monitor this slave, a
nd to consider it in O_down # (objectively down) state only if at least <quorum> Sentinels agree.
# # Note:master name should not include special characters or spaces.
# The valid charset is a-Z 0-9 and the three characters ".-_". Sentinel Monitor MyMaster 192.168.0.2 6379 2 #此处的意思是需要两个哨兵来确认服务是否挂掉 # Sentinel Auth-pass <master-name> <password
> # # Set the password to authenticate with the master and slaves.
# Useful if there is a password set in the Redis instances to monitor. # # Note that the master password are also used for slaves, so it's not # possible to set a different password in Masters
and Slaves instances # If you want to is able to monitor these instances with Sentinel. # # However you can have Redis InstancES without the authentication enabled # mixed with Redis instances requiring the authentication (as long as the # password Set is the same for all the instances requiring the password) as # The AUTH command'll has no effect in Redis instance
s with authentication # switched off. # # Example: # Sentinel Auth-pass mymaster mysuper--secret-0123passw0rd # Sentinel Down-after-milliseconds <master-n ame> <milliseconds> # # milliseconds the master (or any attached slave or Sentinel) should # is Unreach Able (as in, not acceptable reply-to-PING, continuously, for the # specified-period) in order-consider it in S_down STA
TE (subjectively # down).
# # Default is seconds. Sentinel Down-after-milliseconds MyMaster # Sentinel Can-failover <master-name> <yes|no> # Specify if
This Sentinel can start the failover for this master.  Sentinel Can-failover MyMaster Yes # Sentinel Parallel-syncs <master-name> <numslaves> # How many slaves weCan reconfigure to the new slave simultaneously # during the failover. 
Use a low number if you use the slaves to serve query # to avoid, all the slaves would be unreachable at about the same
# time while performing the synchronization with the master.  Sentinel Parallel-syncs MyMaster 1 # Sentinel Failover-timeout <master-name> <milliseconds> # Specifies the Failover timeout in milliseconds. When this time had elapsed # without any progress in the failover process, it's considered concluded by # The Sentinel Ev En if not all the attached slaves were correctly configured # to replicate with the new master (however a "best effort" SL
aveof Command # is sent to all the slaves before). # # Also When 25% of this time have elapsed without any advancement, and there # is a leader switch (the Sentinel does not s
Tarted the failover but was now # elected as leader), the Sentinel would continue the failover doing a # "takeover".
# # Default is minutes. Sentinel Failover-tiMeout MyMaster 90000 # SCRIPTS Execution # # Sentinel Notification-script and Sentinel Reconfig-script is used in order # To configure scripts that is called to notify the system administrator # or to reconfigure clients after a failover. The scripts is executed # with the following rules for error handling: # # If script exists with ' 1 ' The execution is RET
ried later (up to a maximum # number of times currently set to 10).
# # If script exists with "2" (or a higher value) the script execution is # not retried.
# # If script terminates because it receives a signal the behavior is the same # as exit code 1. # # A script has A maximum running time of seconds.

After this limit is # reached the script was terminated with a SIGKILL and the execution retried. # NOTIFICATION SCRIPT # # Sentinel Notification-script <master-name> <script-path> # # Call the specified ification script for any sentienl event this is # generated at the WARNING level (for Instance-sdown,-odOwn, and so forth). # This script should notify the system administrator via email, SMS, or any # other messaging system, which there is someth
ing wrong with the monitored # Redis systems.
# # The script is called with just, Arguments:the first is the event type # and the second the event description.
# # The script must exist and be executable on order for Sentinel to start if # This option is provided.  # # Example: # # Sentinel Notification-script mymaster/var/redis/notify.sh # clients Reconfiguration Script # # Sentinel Client-reconfig-script <master-name> <script-path> # When the failover starts, ends, or is aborted, a scrip T can is called in # Order to perform application-specific tasks to notify the clients that the # configuration have change
D and the master are at a different address. # # The script is called in the following cases: # # Failover started (a Slave is already promoted) # Failover finished ( All the additional slaves already reconfigured) # FailoveR aborted (in this case the script is previously called when the # failover started, and now gets Calle
D again with swapped # addresses). # # The following arguments is passed to the script: # # <master-name> <role> <state> <from-ip> & lt;from-port> <to-ip> <to-port> # <state> is ' start ', ' End ' or ' abort ' # <role> is either "le  Ader "or" Observer "# # Arguments From-ip, From-port, To-ip, To-port is used to communicate # The old address of the
Master and the new address of the elected slave # (now A-master) in the case state is "start" or "End". # for Abort instead the ' from ' is the address of the promoted slave and # ' to ' is the address of the original master add
Ress, since the failover # was aborted.
# # This script should is resistant to multiple invocations.

 # # Example: # # Sentinel Client-reconfig-script mymaster/var/redis/reconfig.sh

A configuration that requires special attention, without which keepalived does not forward, and if Redis and KeepAlive are the same machine, they will be forwarded, but if they are independent, Realserver will not receive the transfer packet

Configure Master
Vim/etc/sysctl.conf, add the following

Net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
Net.ipv4.conf.all.arp_announce = 2

Execute command

Sysctl-p
IP addr add 192.168.0.7/32 dev lo
ip Add list

The specified IP appears on the LO Ring

Configure SLAVE1

Vim/etc/sysctl.conf, add the following

Net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
Net.ipv4.conf.all.arp_announce = 2

Execute command

Sysctl-p
IP addr add 192.168.0.7/32 dev Lo

IP addr Add 192.168.0.8/32 dev Lo

IP Add list

Configure Slave2

Vim/etc/sysctl.conf, add the following

Net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
Net.ipv4.conf.all.arp_announce = 2

Execute command

Sysctl-p
IP addr add 192.168.0.8/32 dev lo
ip Add list

Setting up Sentinel

The Redis Sentinel is set to 2 in the configuration file, so the route1 and route2 two machines need to be configured

Launch a Redis Sentinel

/usr/local/bin/redis-server/etc/redis/sentinel.conf--sentinel