Nginx+keepalived Dual Master High Availability load Balancer

Lab environment and software version:
CentOS version: 6.6 (2.6.32.-504.el6.x86_64)
Nginx Version: nginx-1.6.3
Keepalived version: keepalived-1.2.7

Main lb1:lb-110-05

Main lb2:lb-111-06

First, installation preparation and dependence (with SECURECRT interactive window at the same time to two LB operation, only paste out the LB1 operation process here)

[Email protected] ~]# mkdir Tools

[Email protected] ~]# mkdir/application

[Email protected] ~]# yum-y install pcre pcre-devel OpenSSL openssl-devel gcc gcc-c++ make Automake popt-devel

[Email protected] ~]# CD Tools

[Email protected] tools]# Tar XF nginx-1.6.3.tar.gz

Second, nginx+keepalived installation combat (with SECURECRT interactive window at the same time to two LB operation, only posted LB1 operation process here)

1. Installing Nginx

[Email protected] tools]# CD nginx-1.6.3

[Email protected] nginx-1.6.3]# useradd nginx-s/sbin/nologin-m

[Email protected] nginx-1.6.3]#/configure--prefix=/application/nginx-1.6.3--with-http_stub_status_module-- With-http_ssl_module--user=nginx--group=nginx

[[email protected] nginx-1.6.3]# echo $? #检查上一步操作是否正确, the correct return result is 0, and vice versa is 1
0

[[email protected] nginx-1.6.3]# make && make install

[Email protected] nginx-1.6.3]# ln-s/application/nginx-1.6.3/application/nginx

[Email protected] nginx-1.6.3]#/application/nginx/sbin/nginx-t #检查nginx的语法是否正确和测试是否成功
Nginx:the configuration file/application/nginx-1.6.3/conf/nginx.conf syntax is OK
Nginx:configuration file/application/nginx-1.6.3/conf/nginx.conf Test is successful

[Email protected] nginx-1.6.3]#/application/nginx/sbin/nginx #启动nginx服务

[Email protected] nginx-1.6.3]# Netstat-tunlp|grep #检查nginx服务是否启动成功

TCP 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 4329/nginx

2. Installing keepalived

[Email protected] nginx-1.6.3]# CD.

[Email protected] tools]# Tar XF keepalived-1.2.7.tar.gz

[Email protected] tools]# CD keepalived-1.2.7

[Email protected] keepalived-1.2.7]#./configure

[[email protected] keepalived-1.2.7]# make && make install

[Email protected] keepalived-1.2.7]# cp/usr/local/etc/rc.d/init.d/keepalived/etc/rc.d/init.d/

[Email protected] keepalived-1.2.7]# cp/usr/local/etc/sysconfig/keepalived/etc/sysconfig/

[Email protected] keepalived-1.2.7]# mkdir/etc/keepalived

[Email protected] keepalived-1.2.7]# cp/usr/local/etc/keepalived/keepalived.conf/etc/keepalived/

[Email protected] keepalived-1.2.7]# cp/usr/local/sbin/keepalived/usr/sbin/

3. Join the boot start

[email protected] ~]# cat >>/etc/rc.local<<eof
>/usr/local/nginx/sbin/nginx
>/etc/init.d/keepalived Start
> EOF

[Email protected] ~]# cat/etc/rc.local
#!/bin/sh
#
# This script is executed *after* all and the other init scripts.
# can put your own initialization stuff in here if you don ' t
# want to does the full Sys V style init stuff.

Touch/var/lock/subsys/local
/application/nginx/sbin/nginx
/etc/init.d/keepalived start

Third, configuration nginx+keepalived

1. Configure Nginx

[Email protected] ~]# cd/application/nginx/conf

[email protected] conf]# CP nginx.conf Nginx.conf.bak

[Email protected] conf]# VI nginx.conf

User Nginx Nginx;
Worker_processes 1;
Events {
Worker_connections 1024;
}

HTTP {
Include Mime.types;
Default_type Application/octet-stream;
Sendfile on;
Keepalive_timeout 65;

Upstream myserver{
Ip_hash; #用ip哈希算法保持会话
Server 10.0.0.7:80 max_fails=3 fail_timeout=20s;
Server 10.0.0.8:80 max_fails=3 fail_timeout=20s;
}
server {
Listen 80;
server_name 192.168.0.110;
Location/{
Index index.php index.htm index.html;
Proxy_redirect off;
Proxy_set_header Host $host;
Proxy_set_header X-real-ip $remote _addr;
Proxy_next_upstream http_500 http_502 http_503 error timeout invalid_header;
Proxy_set_header x-forwarded-for $remote _addr;
Proxy_pass http://MyServer;
}
}
}

Save exit, the LB1 nginx.conf configuration file with SCP push to LB2/application/nginx/conf directory.

[Email protected] conf]#/application/nginx/sbin/nginx-t
Nginx:the configuration file/application/nginx-1.6.3/conf/nginx.conf syntax is OK
Nginx:configuration file/application/nginx-1.6.3/conf/nginx.conf Test is successful

[[email protected] conf]#/application/nginx/sbin/nginx-s reload #平滑重启nginx, no impact on service usage, improve user experience

[Email protected] conf]# ps-ef|grep nginx|grep-v grep #检查nginx服务是否启动成功
Root 4329 1 0 17:08? 00:00:00 Nginx:master Process/application/nginx/sbin/nginx
Nginx 6330 4329 0 18:01? 00:00:00 Nginx:worker Process

2. Configure keepalived

2.1 LB1 's keepalived configuration

[Email protected] ~]# Cp/etc/keepalived/keepalived.conf/etc/keepalived/keepalived.conf.bak

[Email protected] ~]# vi/etc/keepalived/keepalived.conf
! Configuration File for Keepalived

Global_defs {
Notification_email {
[Email protected]
[Email protected]
[Email protected]
}
Notification_email_from [email protected]
Smtp_server 192.168.200.1
Smtp_connect_timeout 30
router_id Nginx_devel
}

Vrrp_instance NGINX_HA1 {
State MASTER
Interface eth1
VIRTUAL_ROUTER_ID 51
Priority 100
Advert_int 1
Authentication {
Auth_type PASS
Auth_pass 1111
}
virtual_ipaddress {
192.168.0.110/24 Dev eth1
}
}
Vrrp_instance NGINX_HA2 {
State BACKUP
Interface eth1
VIRTUAL_ROUTER_ID 52
Priority 99
Advert_int 1
Authentication {
Auth_type PASS
Auth_pass 1111
}
virtual_ipaddress {
192.168.0.111/24 Dev eth1
}
}

2.2 LB2 's keepalived configuration

[Email protected] ~]# vi/etc/keepalived/keepalived.conf
! Configuration File for Keepalived

Global_defs {
Notification_email {
[Email protected]
[Email protected]
[Email protected]
}
Notification_email_from [email protected]
Smtp_server 192.168.200.1
Smtp_connect_timeout 30
router_id Nginx_devel
}

Vrrp_instance NGINX_HA1 {
State BACKUP
Interface Eth2
VIRTUAL_ROUTER_ID 51
Priority 99
Advert_int 1
Authentication {
Auth_type PASS
Auth_pass 1111
}
virtual_ipaddress {
192.168.0.110/24 Dev eth2
}
}
Vrrp_instance NGINX_HA2 {
State MASTER
Interface Eth2
VIRTUAL_ROUTER_ID 52
Priority 100
Advert_int 1
Authentication {
Auth_type PASS
Auth_pass 1111
}
virtual_ipaddress {
192.168.0.111/24 Dev eth2
}
}

3. Start keepalived

[Email protected]~]#/etc/init.d/keepalived start #先启动主LB1

[Email protected] ~]#/etc/init.d/keepalived start #随后再启动LB2

Iv. viewing results After service startup is successful

[[email protected] ~]# IP a
1:lo: <LOOPBACK,UP,LOWER_UP> MTU 65536 qdisc noqueue State UNKNOWN
Link /loopback 00:00:00:00:00:00 BRD 00:00:00:00:00:00
inet 127.0.0.1/8 Scope host lo
Inet6:: 1/128 scope host
Val Id_lft Forever Preferred_lft Forever
2:eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> MTU Qdisc Pfifo_fast State Up Qlen
Link/ether 00:0c:29:5c:2d:57 brd ff:ff:ff:ff:ff:ff
inet 192.168.0.110/24 BRD 192.168.0.255 Scope glob Al eth0
Inet6 fe80::20c:29ff:fe5c:2d57/64 scope link
Valid_lft forever preferred_lft Forever
3:eth1: <broa Dcast,multicast,up,lower_up> MTU Qdisc Pfifo_fast State up Qlen +
Link/ether 00:0c:29:5c:2d:61 BRD ff:ff:ff : Ff:ff:ff
inet 10.0.0.5/24 BRD 10.0.0.255 scope global eth1
inet 192.168.0.110/24 scope global eth1
Inet6 FE80 :: 20c:29ff:fe5c:2d61/64 scope link
Valid_lft forever preferred_lft Forever

[[Email protected] ~]# IP A
1:lo: <LOOPBACK,UP,LOWER_UP> MTU 65536 qdisc noqueue State UNKNOWN
Link/loopback 00:00:00:00:00:00 BRD 00:00:00:00:00:00
inet 127.0.0.1/8 Scope host Lo
INET6:: 1/128 Scope Host
Valid_lft Forever Preferred_lft Forever
2:eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> MTU Qdisc pfifo_fast State up Qlen 1000
Link/ether 00:0c:29:50:8e:3a BRD FF:FF:FF:FF:FF:FF
inet 192.168.0.111/24 BRD 192.168.0.255 Scope Global eth1
Inet6 FE80::20C:29FF:FE50:8E3A/64 Scope link
Valid_lft Forever Preferred_lft Forever
3:eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> MTU Qdisc pfifo_fast State up Qlen 1000
Link/ether 00:0c:29:50:8e:44 BRD FF:FF:FF:FF:FF:FF
inet 10.0.0.6/24 BRD 10.0.0.255 Scope Global eth2
inet 192.168.0.111/24 Scope Global eth2
Inet6 FE80::20C:29FF:FE50:8E44/64 Scope link
Valid_lft Forever Preferred_lft Forever

V. Test High Availability

[[email protected] ~]#/etc/init.d/keepalived stop
stopping keepalived: [OK]
[[email protected] ~ ]# IP a
1:lo: <LOOPBACK,UP,LOWER_UP> MTU 65536 qdisc noqueue State UNKNOWN
Link/loopback 00:00:00:00:00:00 BRD 00:00:00:00:00:00
inet 127.0.0.1/8 Scope host lo
Inet6:: 1/128 scope host
Valid_lft forever Preferred_lft Forever
2:eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> MTU Qdisc pfifo_fast State up Qlen
Link/ether 0 0:0C:29:50:8E:3A BRD ff:ff:ff:ff:ff:ff
inet 192.168.0.111/24 BRD 192.168.0.255 scope global eth1
Inet6 fe80::20c: 29FF:FE50:8E3A/64 scope link
Valid_lft forever preferred_lft Forever
3:eth2: <broadcast,multicast,up,lower_ Up> MTU Qdisc Pfifo_fast State up Qlen
link/ether 00:0c:29:50:8e:44 brd ff:ff:ff:ff:ff:ff
inet 10.0.0 .6/24 BRD 10.0.0.255 Scope global eth2
Inet6 fe80::20c:29ff:fe50:8e44/64 scope link
Valid_lft forever preferred_l FT Forever

[[email protected] ~]# IP a
1:lo: <LOOPBACK,UP,LOWER_UP> MTU 65536 qdisc noqueue State UNKNOWN
Link /loopback 00:00:00:00:00:00 BRD 00:00:00:00:00:00
inet 127.0.0.1/8 Scope host lo
Inet6:: 1/128 scope host
Val Id_lft Forever Preferred_lft Forever
2:eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> MTU Qdisc Pfifo_fast State Up Qlen
Link/ether 00:0c:29:5c:2d:57 brd ff:ff:ff:ff:ff:ff
inet 192.168.0.110/24 BRD 192.168.0.255 Scope glob Al eth0
Inet6 fe80::20c:29ff:fe5c:2d57/64 scope link
Valid_lft forever preferred_lft Forever
3:eth1: <broa Dcast,multicast,up,lower_up> MTU Qdisc Pfifo_fast State up Qlen +
Link/ether 00:0c:29:5c:2d:61 BRD ff:ff:ff : Ff:ff:ff
inet 10.0.0.5/24 BRD 10.0.0.255 scope global eth1
inet 192.168.0.110/24 scope global eth1
inet 192.1 68.0.111/24 Scope Global Secondary eth1
Inet6 fe80::20c:29ff:fe5c:2d61/64 scope link
Valid_lft forever preferred_ LfT Forever

Nginx+keepalived Dual Master High Availability load Balancer