LVS + Keepalived enables high availability, load balancing Web clusters, and lvskeepalived

LVS + Keepalived enables high availability, load balancing Web clusters, and lvskeepalived

Introduction:

LVS is short for Linux Virtual Server, which means a Virtual Server cluster system. This project was established by Dr. Zhang Wenyu in May 1998, it is one of the first free software projects in China.

Advantages of LVS Server Load balancer cluster system:

1. improve throughput

To achieve higher throughput, you only need to add Real-server in LVS. The overhead only increases linearly. If you choose to replace a higher-performance server to achieve a considerable throughput, the overhead will be much higher.

2. Redundancy

If a Real-server in LVS cannot provide external services due to upgrade or other reasons, the termination and recovery operations will not cause service interruption to users.

3. Adaptability

Whether the throughput needs to change gradually or rapidly, the increase or decrease of servers is transparent to customers.

Three forwarding mechanisms of the lvs Server Load balancer cluster system:

1. Virtual Server via NAT (VS/NAT)

In VS/NAT network address translation mode, the server can run any operating system that supports the TCP/IP protocol. It only needs a public IP address configured on The LVS host, private addresses can be used for real backend servers.
Its disadvantage is its Limited scalability. When the number of real backend servers reaches a certain level, the LVS local machine may become a cluster bottleneck, because the client request and response packets in the entire cluster must pass through the LVS Server Load balancer host.

2. Virtual Server via IP Tunneling (VS/TUN)

In VS/tun ip tunneling mode, the Server Load balancer LVS host only distributes requests to different real servers, and the real server directly returns the results to the client. In this way, LVS can process massive requests without becoming a bottleneck in the cluster system.
In addition, the IP tunneling mode does not limit the actual server location, as long as the system supports the IP tunneling protocol and the network is reachable. An IP Tunnel overhead is added, and not all systems support this protocol.

3. Virtual Server via Direct Routing (VS/DR)

Direct VS/DR routing mode. Similarly, the LVS host only processes client-to-server connections, and the response information is directly returned to the client by the Real Server.
In DR mode, all real servers must have at least one network card in the same physical network segment as The LVS host, and the real server network device or device alias does not respond to ARP.

Keepalived was initially designed by LVS to monitor the status of each service node in the cluster system. Later, it added VRRP (Virtual routing redundancy protocol) to solve the single point of failure (spof) Problem in Static Routing,
The VRRP protocol enables uninterrupted and stable network operation. Therefore, Keepalived not only provides server status detection and fault isolation functions, but also HA Cluster functions.

Keepalived is an extension project of LVS. Therefore. Keepalived can be seamlessly integrated with LVS to easily build a high-performance Server Load balancer cluster system.

Environment Description: (CentOS 6.6 _ x86_64 minimal)

Lvs vip 192.168.214.50

LVS Master 192.168.214.20
LVS Backup 192.168.214.30

Real-Server-1 192.168.214.10
Real-Server-1 192.168.214.40

Software Package:

Ipvsadm http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.26.tar.gz
Keepalived http://www.keepalived.org/software/keepalived-1.2.10.tar.gz

1. Install the environment dependency package

shell > yum install -y gcc gcc-c++ makepcre pcre-devel kernel-devel openssl-devel libnl-devel popt-devel popt-static

Ii. Download and install ipvsadm and keepalived

shell > wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.26.tar.gzshell > tar zxf ipvsadm-1.26.tar.gzshell > cd ipvsadm-1.26shell > make ; make installshell > ipvsadm -vipvsadm v1.26 2008/5/15 (compiled with popt and IPVS v1.2.1)

# Installation is simple

Shell> wget http://www.keepalived.org/software/keepalived-1.2.10.tar.gzshell> tar zxf keepalived-1.2.10.tar.gzshell> cd keepalived-1.2.10shell>. /configure -- sysconf =/etc -- with-kernel-dir =/usr/src/kernels/2.6.32-504.23.4.el6.x86 _ 64 # -- sysconf specifies keepalived. conf storage location -- with-kernel-dir specifies to use the header file in the kernel source code, that is, the include directory shell> make; make installshell> ln-s/usr/local/sbin/keepalived/sbin/shell> keepalived-vKeepalived v1.2.10 (07/01, 2015)

# Installation is quite simple

3. Configure keepalived. conf

Shell> vim/etc/keepalived. conf! Configuration File for keepalivedglobal_defs {router_id MASTER # keepalived server identifier, which can be set at Will (seemingly globally unique )} vrrp_instance VI_1 {# define a VRRP instance named VI_1 state MASTER # Keepalived server role, MASTER-based and BACKUP-based interfaces: eth0 # specify the HA Monitoring Network interface virtual_router_id 51 # virtual route ID. The same VRRP instance uses a unique identifier, the MASTER and BACKUP must have the same priority 100 # Node priority. The MASTER priority of the same VRRP instance must be greater than the BACKUP advert_int 1 # synchronization check interval between MASTER and BACKUP, in seconds auth Entication {# communication authentication type and password between nodes, in the same VRRP instance, MASTER/BACKUP must use the same password to communicate with auth_type PASS auth_pass 888} virtual_ipaddress {## virtual IP address, also known as the drifting IP address. You can use ip add to check whether the host is bound to 192.168.214.50} virtual_server 192.168.214.50 80 {## define virtual server delay_loop 6 ## define the Health Check Interval, in seconds, lb_algo rr # Server Load balancer scheduling algorithm supports rr, wrr, lc, wlc, lblc, sh, dh, and other lb_kind DR # LVS Server Load balancer mechanisms, NAT, TUN, and DR nat_mask bandwidth 255.255.0 ## subnet mask persistence_timeout 120 # session persistence time, in seconds. The dynamic page session persistence function is provided. The same IP address is continuously allocated to the same node server for a period of time. protocol TCP # forwarding protocol type, support TCP and UDP real_server 192.168.214.10 80 {## define the node server weight 1 # node weight value. The larger the number, the higher the weight, the more connections allocated to it. It is mainly used for the health check of the script TCP_CHECK {## executed after the node server is DOWN. The performance of the backend node servers is not uniform: yy_down/etc/keepalived/real_down.sh, support HTTP_GET, SSL_GET, TCP_CHECK, SMTP_CHECK, and MISC_CHECK connect_port 80 # Check port. If this parameter is not specified, connect_timeout 3 is specified by real_server by default. # No response timeout, unit: seconds nb_get_retry 3 # Number of Retries delay_before_retry 3 # Retry Interval, in seconds} real_server 192.168.214.40 80 {## weight 1 notify_down/etc/keepalived/real_down.sh TCP_CHECK {connect_port 80 connect_timeout 3 nb_get_retry 3 limit 3 }}}

shell > vim /etc/keepalived/real_down.sh#!/bin/bashMail='admin@mail.com'echo "Real Server State is DOWN" | mail -s "Keepalived Mail" $Mail

# For the script executed when the real server is DOWN, LVS must start Sendmail, Postfix, or other mail services

4. Start Keepalived and add it to startup and configure firewall rules

shell > /etc/init.d/keepalived startshell > chkconfig --add keepalivedshell > chkconfig --level 35 keepalived onshell > iptables -I INPUT 4 -p tcp --dport 80 -j ACCEPTshell > service iptables save

# Now we can see the bound VIP through ip add, and the detection results of VIP binding and real server are also found in the/var/log/messages log file.

5. All operations are performed on the MASTER host. Configure the BACKUP host below.

# Steps 1, 2, and 4 are exactly the same. Modify router_id, state, and priority in keepalived. conf in step 3 to BACKUP, BACKUP, and 80.

# You can also use ip add to view the VIP information. However, it is in the BACKUP status and is normal if no VIP is bound. You can see that it is in the BACKUP status through the/var/log/messages log file.

6. configure Real-server and backend node server

Real-Server-1

shell > yum -y install httpdshell > echo "welcome to Real-Server-1" > /var/www/html/index.htmlshell > /etc/init.d/httpd startshell > iptables -I INPUT -p tcp --dport 80 -j ACCEPT

Real-Server-2

shell > yum -y install httpdshell > echo "welcome to Real-Server-2" > /var/www/html/index.htmlshell > /etc/init.d/httpd startshell > iptables -I INPUT -p tcp --dport 80 -j ACCEPT

Shell> vim/etc/init. d/realserver. sh # This script must be created for Real-Server-1/Real-Server-2 and set to start at startup #! /Bin/bashVIP = '2017. 168.214.50 '. /etc/init. d/functionscase "$1" in start)/sbin/ifconfig lo: 0 $ VIP broadcast $ VIP netmask 255.255.255.255 up echo "1">/proc/sys/net/ipv4/conf/lo/arp_ignore echo "2">/proc/sys/net/ ipv4/conf/lo/arp_announce echo "1">/proc/sys/net/ipv4/conf/all/arp_ignore echo "2">/proc/sys/net/ipv4/ conf/all/arp_announce echo "LVS Real-Server Start Success ";; stop)/sbin/ifconfig lo: 0 down echo "0">/proc/sys/net/ipv4/conf/lo/arp_ignore echo "0">/proc/sys/net/ipv4/conf/lo/arp_announce echo "0">/proc/sys/net/ipv4/conf/all/arp_ignore echo "0">/proc/sys/net/ipv4/conf/all/arp_announce echo" LVS Real-Server Stop Success ";; *) echo "Usage: $0 (start | stop)" exit 1 esacshell> chmod a + x/etc/init. d/realserver. shshell>/etc/init. d/realserver. sh startshell> echo "/etc/init. d/realserver. sh start ">/etc/rc. local ## add to startup

# This script is used to bind the VIP to the node server and suppress ARP requests in response to the VIP.
# The purpose of this operation is to prevent the node server from responding to the ARP broadcast of the VIP address (because the node server is bound to the VIP address, if they are not set, the node server will respond, and it will be messy)

VII. Test

1. First, test whether the node server can access

# Access 192.168.214.10 192.168.214.40 separately. If the page shows Welcome to Real-Server-1 and Welcome to Real-Server-2, the node Server is normal.

2. check whether all node servers are bound with VIP addresses.

# Use ifconfig lo: 0 on the node server. If lo: 0 is bound to VIP 192.168.214.50, the node server is normal.

3. Check whether the MASTER server is bound to a VIP or whether the status of the MASTER server is successful.

# Use ip add to check whether the server is bound to a VIP, tail/var/log/messages to check whether the server status is MASTER, and whether the node server has been connected successfully:

Jul 2 17:38:44 localhost Keepalived_healthcheckers[1015]: TCP connection to [192.168.214.10]:80 success.Jul 2 17:38:44 localhost Keepalived_healthcheckers[1015]: Adding service [192.168.214.10]:80 to VS [192.168.214.50]:80Jul 2 17:44:01 localhost Keepalived_healthcheckers[1015]: TCP connection to [192.168.214.40]:80 success.Jul 2 17:44:01 localhost Keepalived_healthcheckers[1015]: Adding service [192.168.214.40]:80 to VS [192.168.214.50]:80

# Server status information:

Jul 2 13:07:48 localhost Keepalived_vrrp[1016]: VRRP_Instance(VI_1) Transition to MASTER STATEJul 2 13:07:49 localhost Keepalived_vrrp[1016]: VRRP_Instance(VI_1) Entering MASTER STATEJul 2 13:07:49 localhost Keepalived_vrrp[1016]: VRRP_Instance(VI_1) setting protocol VIPs.Jul 2 13:07:49 localhost Keepalived_vrrp[1016]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.214.50Jul 2 13:07:49 localhost Keepalived_healthcheckers[1015]: Netlink reflector reports IP 192.168.214.50 added

4. Check whether the BACKUP server is bound to a VIP and whether the status is BACKUP. Check whether the backend node server is successful.

# If no VIP address is bound and the status of BACKUP is normal, the backend node server can also be detected as normal. The related information is as follows:

Jul 2 17:27:06 localhost Keepalived_vrrp[1017]: VRRP_Instance(VI_1) Entering BACKUP STATEJul 2 17:27:06 localhost Keepalived_vrrp[1017]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(11,12)]Jul 2 17:27:06 localhost Keepalived_healthcheckers[1016]: Using LinkWatch kernel netlink reflector...Jul 2 17:38:40 localhost Keepalived_healthcheckers[1016]: TCP connection to [192.168.214.10]:80 success.Jul 2 17:38:40 localhost Keepalived_healthcheckers[1016]: Adding service [192.168.214.10]:80 to VS [192.168.214.50]:80Jul 2 17:44:01 localhost Keepalived_healthcheckers[1016]: TCP connection to [192.168.214.40]:80 success.Jul 2 17:44:01 localhost Keepalived_healthcheckers[1016]: Adding service [192.168.214.40]:80 to VS [192.168.214.50]:80

5. Client Access VIP test Server Load balancer

# Access 192.168.214.50 directly. If you can access the page, the access is successful.
# After multiple accesses, the page does not change, that is, there is no Server Load balancer.
# The following information is displayed by running the ipvsadm-Ln command on the MASTER node:

shell > ipvsadm -LnIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags  -> RemoteAddress:Port           Forward Weight ActiveConn InActConnTCP  192.168.214.50:80 rr persistent 120  -> 192.168.214.10:80            Route   1      0          0  -> 192.168.214.40:80            Route   1      0          87

# Is it normal that 87 accesses are allocated to the node server 192.168.214.40?
# Don't worry. This is normal! Do you still remember Xia Yuhe on the Lake of Daming? Oh no, do you still remember the persistence_timeout 120 parameter?
# Because session persistence is set, continuous access is allocated to a node server.
# Comment out this parameter and test it once (remember to restart keepalived ).

shell > ipvsadm -LnIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags  -> RemoteAddress:Port           Forward Weight ActiveConn InActConnTCP  192.168.214.50:80 rr  -> 192.168.214.10:80            Route   1      0          26  -> 192.168.214.40:80            Route   1      0          26

# You can see that the request is evenly distributed to the current node server (rr round robin scheduling algorithm), and the page is also switched on the two node servers.

# What happens when the node server goes down?

shell > tail /var/log/messagesJul 2 18:38:36 localhost Keepalived_healthcheckers[1636]: TCP connection to [192.168.214.40]:80 failed !!!Jul 2 18:38:36 localhost Keepalived_healthcheckers[1636]: Removing service [192.168.214.40]:80 from VS [192.168.214.50]:80

# Check failed. The Node server 192.168.214.40 is removed from IPVS

shell > ipvsadm -LnIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags  -> RemoteAddress:Port           Forward Weight ActiveConn InActConnTCP  192.168.214.50:80 rr  -> 192.168.214.10:80            Route   1      0          17

# The down node server is removed

Jul 2 18:45:48 localhost Keepalived_healthcheckers[1705]: TCP connection to [192.168.214.40]:80 success.Jul 2 18:45:48 localhost Keepalived_healthcheckers[1705]: Adding service [192.168.214.40]:80 to VS [192.168.214.50]:80

# After the node server is repaired, the node server is automatically added to the cluster.

6. Client Access VIP test high availability

# When the MASTER server cannot provide services, the VIP will be automatically removed from the MASTER server, and the BACKUP server will be upgraded to the MASTER state, bound to the VIP and take over services.
# After the MASTER repairs and joins the network, it automatically grabs the VIP and becomes the MASTER identity. This is not ideal in a busy network environment.
# You can use the nopreempt parameter to set the do not preemptible function. When setting this parameter, you must set the status of the previously fixed MASTER machine to BACKUP, with the highest priority.