Keepalived + nginx for high availability and load balancing

A few days ago, we used Heartbeat as a solution for the high-availability service architecture. Today we have tested a brand new solution that uses Keepalived to implement this function. Keepalived is a high-performance high-availability or hot standby solution for servers. Keepalived can be used to prevent server spof.

A few days ago, we used Heartbeat as a solution for the high-availability service architecture. Today we have tested a brand new solution that uses Keepalived to implement this function. Keepalived is a high-performance high-availability or hot standby solution for servers. Keepalived can be used to prevent server spof.

A few days ago, we used Heartbeat as a solution for the high-availability service architecture. Today we have tested a brand new solution that uses Keepalived to implement this function.

Keepalived is a high-performance server high-availability or hot standby solution, keepalived can be used to prevent single point of failure (single point of failure means that once a single point of failure occurs, the entire system architecture will become unavailable, with Nginx, you can achieve high availability of web Front-end services.

The implementation of Keepalived is based on the VRRP protocol. Keepalived uses the VRRP Protocol to implement HA.

VRRP (Virtual Router Redundancy Protocol) Protocol is used to implement Router Redundancy. VRRP virtualizes two or more Router devices into one device, A vroip IP (one or more) is provided externally. In a vro group, if the vro that actually owns this IP address works normally, it is a MASTER or is elected by an algorithm, the MASTER implements various network functions for the virtual router IP address, such as ARP requests, ICMP, and data forwarding. Other devices do not own the IP address and the status is BACKUP, in addition to receiving the VRRP status notification information of the MASTER, no external network function is executed. When the host fails, BACKUP takes over the network function of the original MASTER.

VRRP uses multicast data to transmit VRRP data. VRRP uses a special virtual source MAC address to send data instead of the MAC address of the network adapter, when VRRP is running, only the MASTER router regularly sends VRRP notification information, indicating that the MASTER is working normally and the vro IP (Group). BACKUP only receives VRRP data and does not send data, if you do not receive the notice from the MASTER within a certain period of time, each BACKUP node will declare itself as the MASTER node, send the notice information, and re-execute the MASTER node election status.

1. Install the Keeplived dependency

Install some dependent libraries before installing keepalived.

Install openssl

Yum install openssl *

Install popt

Yum install popt *

Install ipvsadm

Yum isntall ipvsadm

Install libnl-dev

Yum install libnl-dev *

2. Install Keepalived

Keepalived installation package address:

Http://www.keepalived.org/software/keepalived-1.2.7.tar.gz

Download, decompress, and compile the configuration

./Configure -- prefix =/usr/local/keepalived

To compile the configuration, make sure that the following items are in the "Yes" status:

Use IPVS Framework: Yes IPVS sync daemon support: Yes IPVS use libnl: Yes Use VRRP Framework: Yes

Then you can compile and install it:

Make & make install

Because keepalived is not installed in the default path (/usr/local by default), you need to do some work after the installation is complete.

Cp/usr/local/keepalived/sbin/keepalived/usr/sbin/# copy the keepalived Startup file to the default path, you can also set the path of the environment variable to implement cp/usr/local/keepalived/etc/rc. d/init. d/keepalived/etc/init. d/# copy the service startup script, this allows you to control the keepalived service cp/usr/local/keepalived/etc/sysconfig/# copy the keepalived service script to the default address, and modify the init. the configuration in the d/keepalived file implements mkdir-p/etc/keepalived/cp/usr/local/keepalived/etc/keepalived. conf/etc/keepalived/vi/etc/keepalived. conf # copy the default configuration file to the default path. You can also copy it in/etc/init. d/keepalived: set the path chkconfig keepalived on # Start the service at startup

3. Configure Keepalived

The next step is the configuration. It is very simple. Go to the configuration file directly.

First, the master server:

Global_defs {icationication_email # notification email, according to the actual situation configure {admin@example.com} icationication_email_from admin@example.comsmtp _ server 127.0.0.1stmp _ connect_timeout 30router_id node1 # node name ID, used for notification} vrrp_instance VI_NODE {state MASTER # configure the MASTER server interface eth0 # communication network adapter virtual_router_id 100 # routing ID priority 200 # priority, 0-254 advert_int 5 # notification interval, in actual deployment, you can set a smaller value to reduce the latency of authentication {auth_type PASS auth_pass 123456 # verification password for inter-host verification} virtual_ipaddress {192.168.1.206 # virtual ip address, multiple ip addresses can be defined }}

Next, configure the slave server:

Global_defs {icationication_email {admin@example.com} routing admin@example.com smtp_server 127.0.0.1 route 30 router_id node2} vrrp_instance VI_NODE {state BACKUP # interface eth0 # slave server's communication Nic virtual_router_id 100 # route ID, same as the master server priority 100 # priority, less than the master server can advert_int 5 # Here is the notification receiving interval, and the master server needs to set the same authentication {auth_type PASS auth_pass 123456 # verification password, same as the master server} virtual_ipaddress {192.168.1.206 # virtual IP address, must also be the same as the master server }}

The above settings are the most basic settings. The function is to switch the virtual IP address to the slave server if the master server's Keepalived service stops (normally the server goes down, after the master server recovers, It switches back to the master server.

However, in many cases, we are faced with nginx failure, and Keepalived cannot play a role at this time. At this time, we need to improve Keepalived. Add a custom script to Keepalived to monitor the running status of negvel. If the nginx process ends, kill the Keepalived process to switch between the master and slave servers.

We are modifying the configuration file of the master server configured above, and adding script implementation in the middle

Global_defs {icationication_email {admin@example.com} icationication_email_from admin@example.com smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id nginx_master} vrrp_script chk_http_port {script "/usr/local/keepalived/nginx. sh "# Here add the script link interval 3 # script execution interval weight 2 # priority change caused by script results} vrrp_instance VI_NODE {state MASTER interface eth0 virtual_router_id 100 priority 200 advert_int 5 authentication {auth_type PASS auth_pass 123456} track_script {chk_http_port # Add script execution} virtual_ipaddress {192.168.1.206 }}

For more information about the configuration, see Keepalived.

If we use LVS + Keepalived integration, keepalived can be used instead of ipvsadm to configure LVS, which can be easily done through configuration. This is a detailed description of Keepalived + LVS configuration in another article.

After modifying the configuration file, we will write the nginx. sh configured above. Of course, we assume that Nginx has been installed successfully.

#!/bin/bashA=`ps -C nginx --no-header |wc -l`if [ $A -eq 0 ];then   killall keepalivedfi

The above script simply checks whether the nginx process exists and kill the keepalived process if it does not exist.

Next let's modify the above script. When the script detects that nginx is not running, it will try to start nginx. If it fails, the keepalived process will be stopped.

#! /Bin/bashA = 'ps-C nginx-no-header | wc-l' if [$ A-eq 0]; then/usr/local/nginx/sbin/nginx # nginx command path sleep 3 if ['ps-C nginx -- no-header | wc-l'-eq 0]; then killall keepalived fifi

Save the script to the specific path configured above (I am here/usr/local/keepalived), and then the important step is to modify the execution permission of the script.

Chmod + x nginx. sh

4. Run Keepalived.

After the configuration is complete, you can run it to check the effect. Start nginx and keepalived on the master and slave servers respectively.

Service keepalived start

After the master server is started, run the ip address a command to view the network information of the master server. The virtual ip address 192.168.1.206 is generated under the eth0 Nic and can be accessed to nginx through this ip address.

Then we close the nginx process (if you have configured an attempt to restart it, you should note that). Then we can use ps-e to check whether the keepalived process is disabled. Normally, we can check the network information, you can see that the virtual ip address under the eth0 Nic has been removed, and then you can see in the network information of the server that the eth0 Nic is bound to the virtual ip address, this ip address is used to access nginx from the slave server. This is the restart of nginx and keepalieved on the master server. We can find that the virtual ip address is bound to the master server.

In this way, the master-slave Hot Standby function is enabled.

Pay attention to the firewall problem, which has plagued me for a long time. I found some information to solve the problem.

Because Keepalived uses multicast to notify the other party of the survival status and the sending priority, and runs multicast to elect the MASTER, 224.0.0.18 is a common multicast address, enable the firewall to allow the multicast address to communicate:

1. If the default firewall is used, you only need to add:

Iptables-I RH-Firewall-1-INPUT-d 224.0.0.18-j ACCEPT

2. If you use a script to configure a firewall, add the following rules:

Iptables-a output-o eth0-d 224.0.0.18-j ACCEPT iptables-a output-o eth0-s ACCEPT-j ACCEPT iptables-a input-I eth0-d 224.0.0.18-j ACCEPT iptables -a input-I eth0-s 224.0.0.18-j ACCEPT

5. Summary

• Keepalived implements dual-machine Hot Standby through virtual routing, which is superior to other solutions.
• Because it is a fixed master-slave Hot Standby, this solution is more suitable for the performance of the two mutual standby servers.
• Keepalived can also implement dual-master-slave mutual backup. It can be achieved through configuring mutual master-slave, and then through DNS load balancing to different VIPs.

Original article address: Keepalived + nginx achieves high availability and load balancing. Thank you for sharing it with me.