Keepalived + nginx for high availability and load balancing

In a previous article on heartbeat as a solution for a highly available service architecture, today there is a new solution that uses keepalived to implement this feature.

Keepalived is a high-performance server-ready or hot-standby solution, keepalived can be used to prevent a single point of failure of a server (a single point of failure is the failure of a certain point of the entire system architecture will be unavailable), The high availability of Web front-end services can be achieved with nginx.

The keepalived implementation is based on the VRRP protocol, keepalived is the use of VRRP protocol to achieve high availability (HA).

VRRP (Virtual Router Redundancy Protocol) protocol is used to implement router redundancy protocol, the VRRP protocol to virtual two or more router devices into a device, external to provide virtual router IP (one or more), and within the router group, If the router that actually owns this external IP is master if it works, or if it is elected by the algorithm, master implements various network functions for the virtual router IP, such as ARP request, ICMP, and data forwarding, etc. other devices do not own the IP, Status is backup, which does not perform external network functions except receiving the VRRP status notification information of master. When the host fails, backup takes over the network functionality of the original master.

The VRRP protocol uses multicast data to transmit VRRP data, VRRP data sends data using a special virtual source MAC address instead of the MAC address of its own network card, and the VRRP runtime only has the master router periodically sending VRRP notification messages. Indicates that master is working properly and the virtual router IP (group), backup receives only VRRP data, does not send data, and if no notification of master is received within a certain period of time, each backup will declare itself master, send notification information, Re-conduct the master election status.

1. Installing keeplived dependencies

Before installing keepalived, you should also install some dependent libraries

Installing OpenSSL

Yum Install openssl*

Installing popt

Yum Install popt*

Installing Ipvsadm

Yum Isntall Ipvsadm

Installing Libnl-dev

Yum Install libnl-dev*

2. Installing keepalived

Keepalived installation package Address:

Http://www.keepalived.org/software/keepalived-1.2.7.tar.gz

Download the extracted post-compilation configuration

./configure--prefix=/usr/local/keepalived

The compile configuration needs to ensure that several items are in the Yes State:

Use IPVS framework:yes IPVS sync daemon support:yes IPVS use Libnl:yes use VRRP framework:yes

You can then compile and install the following:

Make && make install

Because there is no default path installation using Keepalived (default is/usr/local), some work is required after the installation is complete

cp /usr/local/keepalived/sbin/keepalived /usr/sbin/#复制keepalived启动文件到默认路径，也可以通过设置环境变量的path实现cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/#复制服务启动脚本到，以便可以通过service控制keepalived服务cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/#复制keepalived服务脚本到默认的地址，也通过修改init.d/keepalived文件中的相应配置实现mkdir -p /etc/etc/keepalived/cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/vi /etc/keepalived/keepalived.conf#复制默认配置文件到默认路径，其实也可以在/etc/init.d/keepalived中设置路径chkconfig keepalived on#开机启动服务

3. Configure keepalived

Next is the configuration, very simple, directly on the configuration file

First the primary server:

Global_defs{notification_email  #通知email, configure {[email protected]< according to the actual situation Span class= "o" >}notification_email_from [email protected]smtp_server 127.0.0.1stmp_connect_timeout 30router_  ID node1  #节点名标识, mainly used in notifications }vrrp_instance vi_node {State MASTER  #配置为主服务器 interface eth0  #通讯网卡 virtual_router_id 100  #路由标识 Priority 200  #优先级, 0-254 advert_int 5  #通知间隔, Actual deployment can be set to a smaller point, reduce latency authentication {auth_type PASS auth_pass 123456 # Authentication password for communication between host authentication } virtual_ipaddress {192.168.1.206  #虚拟ip, you can define multiple Span class= "o" >}}            

Next is from the server settings:

global_defs {notification_email {[ Email protected]  Notification_email_from [email protected] smtp_server 127.0.0.1 stmp_ Connect_timeout router_id node2}vrrp_instance vi_node {State BACKUP  #与主服务器对应 interface eth0  #从服务器的通信网卡 virtual_router_id 100  #路由标识, and primary server same priority 100  #优先级, less than the primary server can Advert_int 5  #这里是接受通知间隔, with the primary server to set the same authentication {auth_type PASS Auth_pass Span class= "M" >123456  #验证密码, same as primary server { 192.168.1.206  #虚拟IP, same as the primary server }      

The above setting is the most basic setting, and the function is to switch the virtual IP to the primary server after the primary server is restored, if the primary keepalived stop service (in general, the server is down).

But in many cases we are facing the situation is nginx hanging off, and this time keepalived can not play a role, then we need to improve the next keepalived. By adding a custom script to keepalived to monitor the running state of the Neginx, if the nginx process ends, the kill keepalived process is used to achieve the switching function of the master-slave server.

We are modifying the configuration file of the primary server configured above and adding the script implementation in the middle

Global_defs{Notification_email{[email protected] } Notification_email_from [email protected] smtp_s erver 127.0.0.1 smtp_connect_timeout router_id nginx_master}vrrp_script chk_http_port {script "/usr/local/keepalived/nginx.sh " #在这里添加脚本链接 interval 3  #脚本执行间隔 weight 2  #脚本结果导致的优先级变更 }vrrp_instance vi_ NODE {State MASTER interface eth0 virtual_router_id the priority Advert_int 5 authentication {auth_type PASS auth_pass 123456  track_script {chk_http_port  #添加脚本执行 } virtual_ipaddress {192.168.1.206 }              

The specific configuration can refer to another article keepalived configuration detailed

If we use the lvs+keepalived integration, then keepalived can replace the Ipvsadm to configure the LVS, can be easily configured to handle, this in another article Keepalived+lvs configuration explained

After modifying the configuration file we write our above configured nginx.sh, of course we assume that Nginx has been installed to complete

#!/bin/bashA=`ps -C nginx --no-header |wc -l`if [ $A -eq 0 ];then killall keepalivedfi

The script above simply checks to see if the Nginx process exists and does not exist on the kill keepalived process.

Next we modify the above-mentioned script, when the script detects that Nginx is not running, will try to start the Nginx, if the failure to stop the keepalived process

#!/bin/bashA=`ps -C nginx –no-header |wc -l`if [ $A -eq 0 ];then /usr/local/nginx/sbin/nginx #nginx命令的路径 sleep 3 if [ `ps -C nginx --no-header |wc -l` -eq 0 ];then killall keepalived fifi

Save the script to the specific path configured above (I'm/usr/local/keepalived here), and then an important step is to modify the execution permissions of the script

chmod +x nginx.sh

4. Running keepalived

After the configuration is complete, you can run to see the effect, respectively, on the master-slave server to start Nginx and keepalived

Service keepalived Start

After starting the • IP A. Command to view the network information of the master server, you can see the ETH0 network card generated under the 192.168.1.206 this virtual IP, and through this IP access to Nginx

Then we close the Nginx process (if you have configured an attempt to restart that should be noted), and then we can see ps -e whether the keepalived process is turned off, under normal circumstances to view the network information, you can be seen eth0 network card under the virtual IP has been lifted, Then in the network information from the server can be seen from the server Eth0 network card binding virtual IP, through this IP access to the server from the Nginx, this is our restart the main server Nginx and keepalieved, we can find that the virtual IP is tied back to the primary server.

This realizes the basic double-click Master-Slave hot standby function.

The problem with the firewall here is that this problem has plagued me for a long time. Find some information to solve the problem

Because the keepalived is through the multicast to inform the other party whether the survival, and send priority, and through the multicast to elect master, and 224.0.0.18 is a common multicast address, firewall open allow this multicast address communication can be:

1. If you are using the default firewall, you only need to add:

Iptables-i rh-firewall-1-input-d 224.0.0.18-j ACCEPT

2. If you are using your own script to set up a firewall, you need to add the following rules

Iptables-a output-o eth0-d 224.0.0.18-j accept iptables-a output-o eth0-s 224.0.0.18-j Accept Iptables-a input-i eth0-d 224.0.0.18-j Accept Iptables-a input-i eth0-s 224.0.0.18-j Accept

5. Summary

• Keepalived the dual-machine hot standby via virtual routing, which has some advantages over other schemes.
• Because it is a fixed master-slave hot standby, this scheme is more suitable for two interoperability server performance differences
• Keepalived can also achieve dual-master interoperability, by setting the main standby, and then through the DNS load balancing to different VIP can be achieved

Keepalived + nginx for high availability and load balancing