Linux High Availability KeepAlive

VRRP: Virtual Routing Redundancy Protocol (Vsan Router redundancy Protocol), redundant routing single point of failure, the implementation of an address bound to n servers (only one), through the multicast signal, when a point of failure, the election of another point when the route.
KeepAlive Workflow: Monitoring process watchdog monitoring VRRP and checkers, monitoring the process of triggering system calls with the mail service, to adjust the floating IP, built-in Ipvs rules decorate with the start Ipvs script.

Installation

NIC multicast must be turned on

ens34: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500ip link set multicast on  dev ens33ip link set multicast off dev ens33

Installation

[[email protected] ~]# yum -y install keepalived

VRRP use

Master node Configuration

  [[email protected] keepalived]# vim/etc/keepalived/keepalived.confglobal_defs {notification_email {[email protected]} notification_email_from [email protected] smtp_server 127.0.0.1 Smtp_connec T_timeout router_id node1 #单个id vrrp_mcast_group4 224.1.101.33 #组播地址}vrrp_instance vi_1 {s    Tate master #当前节点在此虚拟路由器上的初始状态; only one is master and the rest should be #通告进行所需 interface for backup interface ens34                  virtual_router_id #虚拟路由器标识: Vrid (0-255), uniquely identifies the virtual router priority #优先级 Advert_int 1        #vrrp通告的时间间隔, default 1s authentication {auth_type PASS auth_pass 1111} virtual_ipaddress { 192.168.1.199} #状态改变时候出发钩子函数 notify_master "/etc/keepalived/notify.sh master" #当前节点成为主节点时触发的脚本 Notify_ Backup "/etc/keepalived/notify.sh backup" #当前节点转为备节点时触发的脚本 notify_fault "/etc/keepalived/notify.sh fault" #当前节点转为 "failed "state-triggered script}  

From node (differentiate master node with 3 places)

global_defs {   notification_email {        [email protected]   }   notification_email_from [email protected]   smtp_server 127.0.0.1   smtp_connect_timeout 30   router_id node2                         #1.标识   vrrp_mcast_group4 224.1.101.33}vrrp_instance VI_1 {    state BACKUP                           #2.备用    interface ens34    virtual_router_id 51    priority 96                            #3.优先级    advert_int 1    authentication {        auth_type PASS        auth_pass 1111    }    virtual_ipaddress {        192.168.1.199     }    notify_master "/etc/keepalived/notify.sh master"    notify_backup "/etc/keepalived/notify.sh backup"    notify_fault "/etc/keepalived/notify.sh fault"}

Message Notification script (if it is a double-master, the script is followed by parameter differentiation)

[[email protected] keepalived]# vim notify.sh#!/root/.virtualenvs/shellenv/bin/pythonfrom Raven Import Clientimport subprocessimport sysif len (SYS.ARGV) < 2:exit () notify = Sys.argv[1]interface = ' ens34 ' res = subprocess . getstatusoutput ("IP addr show%s|egrep-o ' inet [0-9]{1,3}\.[ 0-9]{1,3}\. [0-9] {1,3}\. [0-9] {1,3} ' |egrep-o ' [^a-z].* ' "% (interface)") IP = Res[1]ip = ip.replace (' \ n ', ') res = subprocess.getstatusoutput ("hostname" ) hostname = Res[1]res = Subprocess.getstatusoutput ("uptime") CPU = Res[1]res = Subprocess.getstatusoutput ("Free-h") Memory = Res[1]res = Subprocess.getstatusoutput ("df-h") disk = Res[1]notify_message = {' master ': ' Change to Master ', ' Backu P ': ' Change-to-backup ', ' fault ': ' Server error '}message = Notify_message.get (notify, ' {%s} no this notify '% (notify)) info = ' '--------info of%s-----hostname:%sinterface:%sip:%scpu:%smemory:%sdisk:%smessage:%s ' '% (Hostname, Hostname, Interface, IP, CPU, MEMORY,DISK, message) client = Client (' Https://[email protecTed]/1219450 ') client.capturemessage (info) 

Start

[[email protected] keepalived]# systemctl  start keepalived

Master Node notification

[[email protected] keepalived]# tcpdump -i ens34 -nn host 224.1.101.33tcpdump: verbose output suppressed, use -v or -vv for full protocol decodelistening on ens34, link-type EN10MB (Ethernet), capture size 262144 bytes13:04:17.240395 IP 192.168.1.200 > 224.1.101.33: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20

Standby node Receipt notification

[[email protected] keepalived]# tcpdump -i ens34 -nn host 224.1.101.33tcpdump: verbose output suppressed, use -v or -vv for full protocol decodelistening on ens34, link-type EN10MB (Ethernet), capture size 262144 bytes13:05:36.437749 IP 192.168.1.200 > 224.1.101.33: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20

Master node View

[[email protected] keepalived]# ip a3: ens34: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000    link/ether 00:0c:29:46:a6:40 brd ff:ff:ff:ff:ff:ff    inet 192.168.1.200/24 brd 192.168.1.255 scope global ens34       valid_lft forever preferred_lft forever    inet 192.168.1.198/32 scope global ens34:0       valid_lft forever preferred_lft forever    inet6 fe80::20c:29ff:fe46:a640/64 scope link       valid_lft forever preferred_lft forever

From the node

此时从节点没有 ens34:0

Primary node failure

#ip到从节点上[[email protected] keepalived]# ifconfigens34:0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500        inet 192.168.1.199  netmask 255.255.255.255  broadcast 0.0.0.0        ether 00:0c:29:34:f5:84  txqueuelen 1000  (Ethernet)

Primary node recovery because the priority is higher than from the node, so the IP goes to the master node

Mailbox Read
Main Service Open:

The primary node stops the service

Log configuration

[[email protected] keepalived]# vim /etc/sysconfig/keepalivedKEEPALIVED_OPTIONS="-D -S 3"[[email protected] keepalived]# vim /etc/rsyslog.conflocal3.*                                                /var/log/keepalive.log[[email protected] keepalived]# systemctl restart rsyslog[[email protected] keepalived]# systemctl restart keepalived

LVS Dual Master Model

Dual master configuration Single node configuration, the other node only need to change the master-slave, Routerid, priority can

global_defs {notification_email {[email protected]} Notification_email_from [email protected]    Smtp_server 127.0.0.1 smtp_connect_timeout router_id node1 vrrp_mcast_group4 224.1.101.33}vrrp_instance VI_1 { State MASTER Interface Ens34 virtual_router_id advert_int 1 Authentication {auth_t    ype PASS auth_pass 1111} virtual_ipaddress {192.168.1.198 dev ens34 label ens34:0} Notify_master "/etc/keepalived/notify.sh master" notify_backup "/etc/keepalived/notify.sh Backup" Notify_fault "/etc /keepalived/notify.sh fault "}vrrp_instance vi_2 {State BACKUP interface ens34 virtual_router_id 9 8 Advert_int 1 Authentication {auth_type PASS auth_pass 2222} virtual_ipaddress {192.1 68.1.197} notify_master "/etc/keepalived/notify.sh master" notify_backup "/etc/keepalived/notify.sh Backup" n Otify_fault "/etc/keepalived/notify.sh fault "}virtual_server 192.168.1.198 {delay_loop 1 lb_algo wrr lb _kind DR Protocol TCP sorry_server 192.168.1.200 real_server 192.168                        .1.202 80{weight 1 http_get {url {path/index.html Status_code Nb_get_retry 3 Delay_before_retry 2 C Onnect_timeout 3}} real_server 192.168.1.203 80{weight 1 Tcp_check {Nb_get_r Etry 3 Delay_before_retry 2 connect_timeout 3}}}virtual_server 192.168.1.197 80 {Delay_loop 1 Lb_algo wrr lb_kind DR Protocol TCP sorry_server 192.168.1.200-Real_server 192.168.1.2                        80{weight 1 http_get {url {path/index.html      Status_code 200          } nb_get_retry 3 Delay_before_retry 2 connect_timeout 3} } real_server 192.168.1.203 80{weight 1 Tcp_check {nb_get_retry 3 Delay_ Before_retry 2 connect_timeout 3}}

  [[email protected] keepalived]# ipvsadm-lnip Virtual Server version 1.2.1 (size=4096) Prot localaddress :P ort Scheduler Flags--remoteaddress:port Forward Weight activeconn inactconntcp 192.168.1.197:80 WRR-& Gt 192.168.1.202:80 Route 1 0 0-192.168.1.203:80 Route 1 0 0TC             P 192.168.1.198:80 WRR-192.168.1.202:80 Route 1 0 0-192.168.1.203:80 Route 1 0 0  

[[email protected] keepalived]# ipvsadm -lnIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags  -> RemoteAddress:Port           Forward Weight ActiveConn InActConnTCP  192.168.1.197:80 wrr  -> 192.168.1.202:80             Route   1      0          0  -> 192.168.1.203:80             Route   1      0          0TCP  192.168.1.198:80 wrr  -> 192.168.1.202:80             Route   1      0          0  -> 192.168.1.203:80             Route   1      0          0

Thus: Ipvs rule two machine configuration is the same, just VIP is fluttering, keepalive for LVS also realized the health state monitoring

marvindeMacBook-Pro:~ marvin$ curl http://192.168.1.197/node4marvindeMacBook-Pro:~ marvin$ curl http://192.168.1.197/node3#停掉node3节点marvindeMacBook-Pro:~ marvin$ curl http://192.168.1.197/node4marvindeMacBook-Pro:~ marvin$ curl http://192.168.1.197/node4

Custom Monitoring Model-nginx

Configure manual Downline file scripts

[[email protected] keepalived]# vim down.sh [[ -f /etc/keepalived/down ]] && exit 1 || exit 0

Configuration file

[[email protected] keepalived]# vim keepalived.conf global_defs {notification_email {[email protected    ]} Notification_email_from [email protected] smtp_server 127.0.0.1 smtp_connect_timeout router_id node1        Vrrp_mcast_group4 224.1.101.33}vrrp_script chk_down {script "/etc/keepalived/down.sh" Interval 1 WEIGHT-20 Fall 1 Rise 1}vrrp_script Chk_nginx {script "pidof nginx && exit 0 | |    Exit 1 "Interval 1 weight-3 Fall 2 rise 2 #有可能重启后又down 2 times more reasonable}vrrp_instance vi_1 {State MASTER Interface ens34 virtual_router_id Advert_int 1 Authentication {Auth_type PASS a    Uth_pass 1111} track_script {Chk_down Chk_nginx} virtual_ipaddress {192.168.1.198/24 } notify_master "/etc/keepalived/notify.sh master" notify_backup "/etc/keepalived/notify.sh Backup" Notify_fau LT "/etc/keepalived/notify.sh Fault"}

/etc/keepalived/notify.sh is mainly added to the inside.

res = subprocess.getstatusoutput("systemctl start mynginx")res = subprocess.getstatusoutput("pidof nginx")nginxpid = res[1]

Test

[[email protected] keepalived]# tcpdump-i ens34-nn host 224.1.101.33[[email protected] keepalived]# touch down15:29:27.073406 IP 192.168.1.200 > 224.1.101.33:vrrpv2, advertisement, Vrid Wuyi, Prio, AuthType simple, INTVL 1 s, length 2015:29:28.074393 IP 192.168.1.200 > 224.1.101.33:vrrpv2, advertisement, Vrid Wuyi, Prio, AuthType simple, INTVL 1s, length 2015:29:28.074710 IP 192.168.1.201 > 224.1.101.33:vrrpv2, advertisement, Vrid Wuyi, Prio, AuthType s imple, intvl 1s, length 20[[email protected] keepalived]# rm-f down15:30:17.179289 IP 192.168.1.201 > 224.1.101. 33:vrrpv2, advertisement, Vrid Wuyi, Prio, authtype simple, intvl 1s, length 2015:30:17.180088 IP 192.168.1.200 > 224 .1.101.33:vrrpv2, advertisement, Vrid Wuyi, Prio, authtype simple, intvl 1s, length 20[[email protected] keepalive d]# systemctl Stop mynginx15:31:01.251469 IP 192.168.1.200 > 224.1.101.33:vrrpv2, advertisement, Vrid Wuyi, Prio, au Thtype simple, intvl 1s, lenGth 2015:31:02.253183 IP 192.168.1.200 > 224.1.101.33:vrrpv2, advertisement, Vrid Wuyi, Prio, AuthType simple, INTVL 1s, length 2015:31:03.254806 IP 192.168.1.200 > 224.1.101.33:vrrpv2, advertisement, Vrid Wuyi, Prio, AuthType simple  , intvl 1s, length 2015:31:03.255137 IP 192.168.1.201 > 224.1.101.33:vrrpv2, advertisement, Vrid, Prio, AuthType simple, intvl 1s, length 2015:30:17.180088 IP 192.168.1.200 > 224.1.101.33:vrrpv2, advertisement, Vrid, Prio 100, authtype simple, intvl 1s, length 2015:30:18.181628 IP 192.168.1.200 > 224.1.101.33:vrrpv2, advertisement, Vrid, PR Io, authtype simple, intvl 1s, then automatically started because notify_backup "/etc/keepalived/notify.sh backup" This script wrote Nginx boot

Mail View

Linux High Availability KeepAlive