1. Introduction to keepalived
Keepalived is an open-source software that can implement high-availability resources. Its main components include core, check, vrrp, libipfwc, and libipvs. The functions of each component are described here.
Core: The core Component of keepalived. It is responsible for starting and maintaining the main process and loading the parsing configuration file.
Check: Responsible for healthchecker, various health check methods, corresponding configuration parsing, and LVS configuration parsing.
Vrrp: A sub-process of vrrpd.
Libipfwc: used in combination with iptables's ipchains library.
Libipvs: used in combination with LVS.
After keepalived is started, three processes are generated: master process, VRRP sub-process, and healthchecker sub-process.
VRRP protocol is a basis for High Availability of keepalived. The following describes the implementation principle of VRRP:
VRRP (VRRP router) is an "election" protocol that dynamically assigns the responsibility of a vro to other vrouters in the same VRRP group, advantages of VRRP:
Redundancy: Multiple router devices can be used as the default gateway of the LAN client, greatly reducing the possibility that the default gateway becomes a single point of failure;
Load Sharing: allow traffic from LAN clients to be shared by multiple router devices;
Multiple VRRP groups: up to 255 VRRP groups can be configured on one vro physical interface;
Multi-IP Address: You can configure multiple IP addresses on the same physical interface based on the Interface alias to allow access to multiple subnets on the same physical interface;
Preemption: when the master node fails, higher priority backup is allowed to become the master node;
Announcement Protocol: Use the multicast address 224.0.0.18 specified by IANA to advertise VRRP;
VRRP tracing: changes the VRRP priority based on the interface status to determine the optimal VRRP router as the master;
Ii. experiment environment:
192.168.30.116 OS: CentOS 6.4 x86_64 master.luojianlong.com
192.168.30.117 OS: Centos 6.4 x86_64 backup.luojianlong.com
Keepalived version: keepalived-1.2.7
First, install keepalived and haproxy on two servers respectively. Because the system version is Centos 6.4, the two software has been integrated in the internal environment, so use yum for installation.
[Root @ master ~] # Yum-y install keepalived haproxy
[Root @ backup ~] # Yum-y install keepalived haproxy
Next, configure the priority of the master and backup servers, virtual_router_id (the two servers of the same instance must be the same), and edit the configuration file:
[Root @ master ~] # Cp/etc/keepalived. conf/etc/keepalived. conf. bak
[Root @ backup ~] # Cp/etc/keepalived. conf/etc/keepalived. conf. bak
[Root @ master ~] # Vi/etc/keepalived. conf
! Configuration File for keepalived
Global_defs {
Notification_email {
Acassen@firewall.loc
Failover@firewall.loc
Sysadmin@firewall.loc
}
Notification_email_from Alexandre.Cassen@firewall.loc
Smtp_server 192.168.200.1
Smtp_connect_timeout 30
Router_id LVS_DEVEL
}
Vrrp_instance VI_1 {
State MASTER
Interface eth0
Virtual_router_id 51
Priority100
Advert_int 1
Authentication {
Auth_type PASS
Auth_pass 1111
}
Virtual_ipaddress {
192.168.30.230
}
}
[Root @ backup ~] # Vi/etc/keepalived. conf
! Configuration File for keepalived
Global_defs {
Notification_email {
Acassen@firewall.loc
Failover@firewall.loc
Sysadmin@firewall.loc
}
Notification_email_from Alexandre.Cassen@firewall.loc
Smtp_server 192.168.200.1
Smtp_connect_timeout 30
Router_id LVS_DEVEL
}
Vrrp_instance VI_1 {
State BACKUP
Interface eth0
Virtual_router_id 51
Priority 99
Advert_int 1
Authentication {
Auth_type PASS
Auth_pass 1111
}
Virtual_ipaddress {
192.168.30.230
}
}
Global_defs: The global configuration identifier is broken;
Icationication_email: the email address sent when an alarm is triggered;
Notification_email_from: indicates the source address of the sent mail;
Smtp_server: smtp server address for sending emails;
Router_id: Machine ID;
Vrrp_instance: defines a vrrp instance;
State: state specifies the Initial state of the instance (Initial), that is, after the configuration, the Initial state of this server is specified here, but not here, you still have to determine the priority through the election. If the priority is set to master here, but if the priority is lower than that of the other server, this server will send its own priority when sending the announcement, the other one finds that the priority is not as high as that of his own, so he will seize the master again;
Interface: The network card bound to the instance, because the virtual IP address must be added to the existing network card;
Virtual router id: VRID is set here, which is very important. If the same VRID is a group, it determines the multicast MAC address;
Priority 100: set the priority of the current node. The priority is higher than that of the master node;
Advert int: Check Interval. The default value is 1 second;
Virtual ipaddress: the VIP address, that is, the virtual IP address, is added and deleted as the state changes. It is added when the state is master and deleted when the state is backup, priority is mainly determined here, and it has little to do with the value set by state. Multiple IP addresses can be set here;
Authentication: Set authentication here;
Auth type: authentication method, which can be PASS or AH;
Auth pass: the authentication password;
Start keepalived of the two servers
[Root @ master ~] # Service keepalived start
Starting keepalived: [OK]
[Root @ backup ~] # Service keepalived start
Starting keepalived: [OK]
[Root @ master ~] # Tail-f/var/log/messages
Jan 10 11:40:56 localhost Keepalived_healthcheckers [19368]: Using LinkWatch kernel netlink reflector...
Jan 10 11:40:56 localhost Keepalived_vrrp [19369]: Opening file '/etc/keepalived. conf '.
Jan 10 11:40:56 localhost Keepalived_vrrp [19369]: Configuration is using: 63019 Bytes
Jan 10 11:40:56 localhost Keepalived_vrrp [19369]: Using LinkWatch kernel netlink reflector...
Jan 10 11:40:56 localhost Keepalived_vrrp [19369]: VRRP sockpool: [ifindex (2), proto (112), fd ()]
Jan 10 11:40:57 localhost Keepalived_vrrp [19369]: VRRP_Instance (VI_1) Transition to MASTER STATE
Jan 10 11:40:58 localhost Keepalived_vrrp [19369]: VRRP_Instance (VI_1) Entering MASTER STATE
Jan 10 11:40:58 localhost Keepalived_vrrp [19369]: VRRP_Instance (VI_1) setting protocol VIPs.
Jan 10 11:40:58 localhost Keepalived_healthcheckers [19368]: Netlink reflector reports IP 192.168.30.230 added
Jan 10 11:40:58 localhost Keepalived_vrrp [19369]: VRRP_Instance (VI_1) Sending gratuitous ARPs on eth0 for 192.168.30.230
Jan 10 11:41:03 localhost Keepalived_vrrp [19369]: VRRP_Instance (VI_1) Sending gratuitous ARPs on eth0 for 192.168.30.230
Jan 10 11:41:14 localhost Keepalived_vrrp [19369]: VRRP_Instance (VI_1) attached ed lower prio advert, forcing new election
Jan 10 11:41:14 localhost Keepalived_vrrp [19369]: VRRP_Instance (VI_1) Sending gratuitous ARPs on eth0 for 192.168.30.230
[Root @ master ~] # Ip addr
1: lo: <LOOPBACK, UP, LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
Link/loopback 00: 00: 00: 00: 00: 00 brd 00: 00: 00: 00: 00: 00
Inet 127.0.0.1/8 scope host lo
Inet6: 1/128 scope host
Valid_lft forever preferred_lft forever
2: eth0: <BROADCAST, MULTICAST, UP, LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
Link/ether 00: 0c: 29: f3: fc: ba brd ff: ff
Inet 192.168.30.116/24 brd 192.168.30.255 scope global eth0
Inet 192.168.30.230/32 scope global eth0
Inet6 fe80: 20c: 29ff: fef3: fcba/64 scope link
Valid_lft forever preferred_lft forever
It is found that the defined virtual ipaddress is on the master server, because it has a higher priority.
Stop the keepalived server of the master server and check whether the IP address will be transferred to backup.
[Root @ master ~] # Service keepalived stop
Stopping keepalived: [OK]
[Root @ backup ~] # Tail-f/var/log/messages
Jan 10 12:12:46 localhost Keepalived_vrrp [18581]: Opening file '/etc/keepalived. conf '.
Jan 10 12:12:46 localhost Keepalived_vrrp [18581]: Configuration is using: 63017 Bytes
Jan 10 12:12:46 localhost Keepalived_vrrp [18581]: Using LinkWatch kernel netlink reflector...
Jan 10 12:12:46 localhost Keepalived_healthcheckers [18580]: Opening file '/etc/keepalived. conf '.
Jan 10 12:12:46 localhost Keepalived_healthcheckers [18580]: Configuration is using: 7324 Bytes
Jan 10 12:12:46 localhost Keepalived_healthcheckers [18580]: Using LinkWatch kernel netlink reflector...
Jan 10 12:12:46 localhost Keepalived_vrrp [18581]: VRRP sockpool: [ifindex (2), proto (112), fd ()]
Jan 10 12:12:47 localhost Keepalived_vrrp [18581]: VRRP_Instance (VI_1) Transition to MASTER STATE
Jan 10 12:12:47 localhost Keepalived_vrrp [18581]: VRRP_Instance (VI_1) stored ed higher prio advert
Jan 10 12:12:47 localhost Keepalived_vrrp [18581]: VRRP_Instance (VI_1) Entering BACKUP STATE
Jan 10 12:16:27 localhost Keepalived_vrrp [18581]: VRRP_Instance (VI_1) Transition to MASTER STATE
Jan 10 12:16:28 localhost Keepalived_vrrp [18581]: VRRP_Instance (VI_1) Entering MASTER STATE
Jan 10 12:16:28 localhost Keepalived_vrrp [18581]: VRRP_Instance (VI_1) setting protocol VIPs.
Jan 10 12:16:28 localhost Keepalived_healthcheckers [18580]: Netlink reflector reports IP 192.168.30.230 added
Jan 10 12:16:28 localhost Keepalived_vrrp [18581]: VRRP_Instance (VI_1) Sending gratuitous ARPs on eth0 for 192.168.30.230
Jan 10 12:16:33 localhost Keepalived_vrrp [18581]: VRRP_Instance (VI_1) Sending gratuitous ARPs on eth0 for 192.168.30.230
[Root @ backup ~] # Ip addr
1: lo: <LOOPBACK, UP, LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
Link/loopback 00: 00: 00: 00: 00: 00 brd 00: 00: 00: 00: 00: 00
Inet 127.0.0.1/8 scope host lo
Inet6: 1/128 scope host
Valid_lft forever preferred_lft forever
2: eth0: <BROADCAST, MULTICAST, UP, LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
Link/ether 00: 0c: 29: 5b: 50: f9 brd ff: ff
Inet 192.168.30.117/24 brd 192.168.30.255 scope global eth0
Inet 192.168.30.230/32 scope global eth0
Inet6 fe80: 20c: 29ff: fe5b: 50f9/64 scope link
Valid_lft forever preferred_lft forever
The IP address has been transferred to the backup server.
Restart keepalived of the master
[Root @ master ~] # Service keepalived start
Starting keepalived: [OK]
[Root @ backup ~] # Tail-f/var/log/messages
Jan 10 12:12:46 localhost Keepalived_vrrp [18581]: VRRP sockpool: [ifindex (2), proto (112), fd ()]
Jan 10 12:12:47 localhost Keepalived_vrrp [18581]: VRRP_Instance (VI_1) Transition to MASTER STATE
Jan 10 12:12:47 localhost Keepalived_vrrp [18581]: VRRP_Instance (VI_1) stored ed higher prio advert
Jan 10 12:12:47 localhost Keepalived_vrrp [18581]: VRRP_Instance (VI_1) Entering BACKUP STATE
Jan 10 12:16:27 localhost Keepalived_vrrp [18581]: VRRP_Instance (VI_1) Transition to MASTER STATE
Jan 10 12:16:28 localhost Keepalived_vrrp [18581]: VRRP_Instance (VI_1) Entering MASTER STATE
Jan 10 12:16:28 localhost Keepalived_vrrp [18581]: VRRP_Instance (VI_1) setting protocol VIPs.
Jan 10 12:16:28 localhost Keepalived_healthcheckers [18580]: Netlink reflector reports IP 192.168.30.230 added
Jan 10 12:16:28 localhost Keepalived_vrrp [18581]: VRRP_Instance (VI_1) Sending gratuitous ARPs on eth0 for 192.168.30.230
Jan 10 12:16:33 localhost Keepalived_vrrp [18581]: VRRP_Instance (VI_1) Sending gratuitous ARPs on eth0 for 192.168.30.230
Jan 10 12:18:20 localhost Keepalived_vrrp [18581]: VRRP_Instance (VI_1) stored ed higher prio advert
Jan 10 12:18:20 localhost Keepalived_vrrp [18581]: VRRP_Instance (VI_1) Entering BACKUP STATE
Jan 10 12:18:20 localhost Keepalived_vrrp [18581]: VRRP_Instance (VI_1) removing protocol VIPs.
Jan 10 12:18:20 localhost Keepalived_healthcheckers [18580]: Netlink reflector reports IP 192.168.30.230 removed
[Root @ master ~] # Ip addr
1: lo: <LOOPBACK, UP, LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
Link/loopback 00: 00: 00: 00: 00: 00 brd 00: 00: 00: 00: 00: 00
Inet 127.0.0.1/8 scope host lo
Inet6: 1/128 scope host
Valid_lft forever preferred_lft forever
2: eth0: <BROADCAST, MULTICAST, UP, LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
Link/ether 00: 0c: 29: f3: fc: ba brd ff: ff
Inet 192.168.30.116/24 brd 192.168.30.255 scope global eth0
Inet 192.168.30.230/32 scope global eth0
Inet6 fe80: 20c: 29ff: fef3: fcba/64 scope link
Valid_lft forever preferred_lft forever
The IP address has been transferred to the master server again.
Now we have compiled a haproxy status detection script to implement haproxy health check:
[Root @ master ~] # Cat haproxy_pid.sh
#! /Bin/bash
While:
Do
Haproxypid = 'ps-C haproxy -- no-header | wc-l'
If [$ haproxypid-eq 0]; then
Service haproxy start
Sleep 5
Haproxypid = 'ps-C haproxy -- no-header | wc-l'
Echo $ haproxypid
If [$ haproxypid-eq 0]; then
/Etc/init. d/keepalived stop
Fi
Fi
Sleep 5
Done
# Start the backup haproxy
[Root @ backup ~] # Service haproxy start
Starting haproxy: [OK]
Simulate a fault. First enable the httpd process and modify the haproxy listening port to 80, so that the haproxy process cannot be started. Check whether the resources will be transferred to the backup server.
[Root @ master ~] # Vi/etc/haproxy. cfg
Change frontend main *: 5000 to frontend main *: 80
# Start the httpd process
[Root @ master ~] # Scp-pr/etc/haproxy. cfg root@192.168.30.117:/etc/haproxy/
[Root @ master ~] # Service httpd start
Starting httpd: apr_sockaddr_info_get () failed for master.luojianlong.com
Httpd: cocould not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName
[OK]
[Root @ master ~] # Netstat-antpl | grep: 80
Tcp 0 0: 80: * LISTEN 19965/httpd
# Running status detection script
[Root @ master ~] # Nohup/root/haproxy_pid.sh &
[Root @ master ~] # Scp-pr haproxy_pid.sh root@192.168.30.117/
[Root @ bakcup ~] # Nohup/root/haproxy_pid.sh &
[Root @ master ~] # Tail-f/var/log/messages
Jan 10 12:02:29 localhost Keepalived_vrrp [19849]: VRRP sockpool: [ifindex (2), proto (112), fd ()]
Jan 10 12:02:29 localhost Keepalived_vrrp [19849]: VRRP_Instance (VI_1) Transition to MASTER STATE
Jan 10 12:02:30 localhost Keepalived_vrrp [19849]: VRRP_Instance (VI_1) Entering MASTER STATE
Jan 10 12:02:30 localhost Keepalived_vrrp [19849]: VRRP_Instance (VI_1) setting protocol VIPs.
Jan 10 12:02:30 localhost Keepalived_vrrp [19849]: VRRP_Instance (VI_1) Sending gratuitous ARPs on eth0 for 192.168.30.230
Jan 10 12:02:30 localhost Keepalived_healthcheckers [19848]: Netlink reflector reports IP 192.168.30.230 added
Jan 10 12:02:35 localhost Keepalived_vrrp [19849]: VRRP_Instance (VI_1) Sending gratuitous ARPs on eth0 for 192.168.30.230
Jan 10 12:14:49 localhost Keepalived [19847]: Stopping Keepalived v1.2.7 (02/21, 2013)
Jan 10 12:14:49 localhost Keepalived_vrrp [19849]: VRRP_Instance (VI_1) sending 0 priority
Jan 10 12:14:49 localhost Keepalived_vrrp [19849]: VRRP_Instance (VI_1) removing protocol VIPs.
[Root @ master ~] # Ip addr
1: lo: <LOOPBACK, UP, LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
Link/loopback 00: 00: 00: 00: 00: 00 brd 00: 00: 00: 00: 00: 00
Inet 127.0.0.1/8 scope host lo
Inet6: 1/128 scope host
Valid_lft forever preferred_lft forever
2: eth0: <BROADCAST, MULTICAST, UP, LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
Link/ether 00: 0c: 29: f3: fc: ba brd ff: ff
Inet 192.168.30.116/24 brd 192.168.30.255 scope global eth0
Inet6 fe80: 20c: 29ff: fef3: fcba/64 scope link
Valid_lft forever preferred_lft forever
[Root @ backup ~] # Ip addr
1: lo: <LOOPBACK, UP, LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
Link/loopback 00: 00: 00: 00: 00: 00 brd 00: 00: 00: 00: 00: 00
Inet 127.0.0.1/8 scope host lo
Inet6: 1/128 scope host
Valid_lft forever preferred_lft forever
2: eth0: <BROADCAST, MULTICAST, UP, LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
Link/ether 00: 0c: 29: 5b: 50: f9 brd ff: ff
Inet 192.168.30.117/24 brd 192.168.30.255 scope global eth0
Inet 192.168.30.230/32 scope global eth0
Inet6 fe80: 20c: 29ff: fe5b: 50f9/64 scope link
Valid_lft forever preferred_lft forever
[Root @ backup ~] # Ps aux | grep haproxy
Haproxy 19054 0.0 0.0 18688 1280? Ss/usr/sbin/haproxy-D-f/etc/haproxy. cfg-p/var/run/haproxy. pid
Root 19097 0.0 0.0 103248 828 pts/0 S + grep haproxy
It is found that the IP address has been transferred to the backup server, achieving high availability of keepalived for haproxy faults.
Common Startup errors of keepalived:
5913 May 16 15:26:04 localhost Keepalived_vrrp: ip address associated with VRID not present in specified ed packet: 192.168.57.75
5914 May 16 15:26:04 localhost Keepalived_vrrp: one or more VIP associated with VRID mismatch actual MASTER advert
5915 May 16 15:26:04 localhost Keepalived_vrrp: bogus VRRP packet already ed on eth0 !!!
5916 May 16 15:26:04 localhost Keepalived_vrrp: VRRP_Instance (VI_1) ignoring expired ed advertisment...
5917 May 16 15:26:05 localhost Keepalived_vrrp: ip address associated with VRID not present in specified ed packet: 192.168.57.75
5918 May 16 15:26:05 localhost Keepalived_vrrp: one or more VIP associated with VRID mismatch actual MASTER advert
5919 May 16 15:26:05 localhost Keepalived_vrrp: bogus VRRP packet already ed on eth0 !!!
5920 May 16 15:26:05 localhost Keepalived_vrrp: VRRP_Instance (VI_1) ignoring specified ed advertisment.
Solution:
The virtual_router_id values in the same network segment cannot be the same. If the values are the same, the VRRP error packet will be received in messages. Therefore, you need to change virual_router_id.
For details about HAproxy, click here
HAproxy: click here
Recommended reading:
Haproxy + Keepalived build Weblogic high-availability server Load balancer Cluster
Keepalived + HAProxy configure high-availability Load Balancing
Haproxy + Keepalived + Apache configuration notes in CentOS 6.3
Haproxy + KeepAlived WEB Cluster on CentOS 6
Haproxy + Keepalived build high-availability Load Balancing