Nginx High-availability cluster architecture (KEEPALIVED+NGINX+IPVSADM)

Case description

In order to solve Nginx single point of failure problem, using keepalived to realize the dual-machine hot standby, keepalived is a high-performance server or hot standby solution, keepalived can be used to prevent the server single point of failure, with Nginx can achieve the web High availability of front-end services. Load balancing is achieved using IPVSADM polling rules.

Case topology

Case Environment

Host	Operating System	IP Address	main software
Keepalivedmaster	Centos7	192.168.100.130	Keepalived, Ipvsadm
Keepalivedbackup	Centos7	192.168.100.140	Keepalived, Ipvsadm
Nginx1	Centos7	192.168.100.100	nginx-1.14.0
Nginx2	Centos7	192.168.100.110	nginx-1.14.0
Client	Win7	192.168.100.55	Web browser

Case actions

Install two nginx servers
1. Install the environment package

Yum install GCC GCC -c++ pcre pcre-devel zlib-devel-y

2. Download nginx package

CD/usr/local /
Wget HTTP: / / http://nginx.org/download/nginx-1.14.0.tar.gz

3. Decompression and compilation

The tar ZXF nginx - 1.14.0. Tar. Gz
CD nginx - 1.14.0
Useradd-m-s /sbin/nologin nginx
. / configure \
-- prefix = / usr/local/nginx \
- user = nginx \
- group = nginx \
- with - http_stub_status_module
Make && make install

4, to optimize

Ln -s/usr/local/nginx/sbin/nginx/usr/local/sbin /

5. Modify your web site

[[email protected] nginx]# CD /usr/local/nginx/ HTML /



6. Close the firewall and open the Nginx service

[[email protected] HTML]# systemctl stop firewalld.service
[[email protected] sbin]# setenforce 0
[[email protected] sbin] # nginx
[[email protected] sbin]# netstat -ntap | grep nginx # checks if nginx port is open
TCP 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 40686/nginx: master

7. Client test access

Ii. Configure ipvsamd scheduling server
1. Download ipvsadm and keepaliveed

[[email protected] ~]# yum install keepalived ipvsadm-y

2. Close the firewall and turn on the routing and forwarding function

[[email protected] ~]# systemctl stop firewalld.service
[[email protected] ~]# systemctl disable firewalld.service
[[email protected] ~]# setenforce 0
[[email protected] ~]# vim /etc/sysctl.conf # add the following four lines
Net. Ipv4. Ip_forward = 1
Net. Ipv4. Conf. All. Send_redirects = 0
Net. Ipv4. Conf. Default. Send_redirects = 0
Net. Ipv4. Conf. Ens33. Send_redirects =
Sysctl-p # refresh takes effect immediately

3. Copy ens33 network card and set up virtual IP

[[email protected] ~]# CD /etc/sysconfig/network-scripts/
[[email protected] network-scripts]# cp ifcfg-ens33
[[email protected] network-scripts]# vim ifcfg-ens33:0 # delete the original and add the following 4 lines
DEVICE = ens33:0
ONBOOT = yes
IPADDR = 192.168.100.10
NETMASK = 255.255.255.0
[[email protected] network-scripts]# ifup ens33:0 # start ens33:0

4. Configure the ipvsadm startup script
Vim/etc/init. D/Dr. Sh

#! / bin/bash
GW = 192.168.100.1
# gateway
VIP = 192.168.100.10
# virtual IP
RIP1 = 192.168.100.100
#nginx1 server IP
RIP2 = 192.168.100.110
#nginx2 server IP
Case "$1" in
Start)
/ sbin/ipvsadm - save > / etc/sysconfig/ipvsadm
Systemctl start ipvsadm
/sbin/ifconfig ens33:0 $VIP broadcast $VIP netmask 255.255.255.255 broadcast $VIP up
/sbin/route add-host $VIP dev ens33:0
/sbin/ ipvsadm-a-t $VIP: 80-s rr
/sbin/ ipvsadm-a-t $VIP: 80-r $RIP1: 80-g
/sbin/ ipvsadm-a-t $VIP: 80-r $rip2:80-g
Echo "ipvsadm starting -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- [ok]." "
;;
Stop)
/ sbin/ipvsadm - C
Systemctl stop ipvsadm
The ifconfig ens33:0 down
The route del $VIP
Echo "ipvsamd stoped -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- - [ok]"
;;
The status)
If [! -e /var/lock/subsys/ipvsadm]; then
Echo "ipvsadm stoped -- -- -- -- -- -- -- -- -- -- -- -- -- -- --"
The exit 1
The else
Echo "ipvsamd running -- -- -- -- -- -- -- -- -- [ok]." "
fi
;;
*)
Echo "Usage: $0 {start | stop | status}"
The exit 1
esac
The exit 0
5. Add script permissions and start ipvsadm

[[email protected] network-related scripts]# chmod +x /etc/init.d/ Dr. Sh
[[email protected] network-scripts]# service Dr. Sh start

Three, back to Nginx node server configuration virtual IP
1. Copy the ifcfg-lo network card and modify it

[[email protected] ~]# CD /etc/sysconfig/network-scripts/
[[email protected] network-scripts]# cp ifcfg-lo ifcfg-lo:0
[[email protected] network-scripts]# vim ifcfg-lo:0 # delete the original and add the following 4 lines
DEVICE = lo: 0
IPADDR = 192.168.100.10
NETMASK = 255.255.255.0
ONBOOT = yes

2. Set ifcfg-lo:0 to start the script

[[email protected] network-related scripts]# CD /etc/init.d/
[[email protected] init. D]# vim lo. Sh

#! / bin/bash
VIP = 192.168.100.10
# virtual IP
Case "$1" in
Start)
Ifconfig lo:0 $VIP netmask 255.255.255.255 broadcast $VIP
/sbin/route add-host $VIP dev lo:0
Echo "1" > / proc/sys/net/ipv4 / conf/lo/arp_ignore
Echo "2" > / proc/sys/net/ipv4 / conf/lo/arp_announce
Echo "1" > / proc/sys/net/ipv4 / conf/all/arp_ignore
Echo "2" > / proc/sys/net/ipv4 / conf/all/arp_announce
Sysctl -p > / dev/null 2 > &1
Echo "RealServer Start OK"
;;
Stop)
The ifconfig lo: 0 down
Route del $VIP /dev/null 2>&1
Echo "0" > / proc/sys/net/ipv4 / conf/lo/arp_ignore
Echo "0" > / proc/sys/net/ipv4 / conf/lo/arp_announce
Echo "0" > / proc/sys/net/ipv4 / conf/all/arp_ignore
Echo "0" > / proc/sys/net/ipv4 / conf/all/arp_announce
Echo "RealServer Stopd"
;;
*)
Echo "Usage: $0 {start | stop}"
The exit 1
esac
The exit 0
3. Add script permission and start lo:0 network card

[[email protected] init. D]# chmod +x lo.sh
[[email protected] init. D]# service lo. Sh start
[[email protected] init. D]# ifup lo:0

4. Local self-test site again

[[email protected] init. D]# firefox "http://127.0.0.1/" &



4. Configure Keepalived

Vim/etc/keepalived/keepalived. Conf (master-slave content server configuration are as follows)

! The Configuration File for keepalived

Global_defs {
Notification_email {
[email protected]
[email protected]
[email protected]
}
Protected notification_email_from [email]
Smtp_server 127.0.0.1 # points to the local address
Smtp_connect_timeout 30
Router_id LVS_01 # is distinguished from server LVS_02
vrrp_skip_check_adv_addr
vrrp_strict
Vrrp_garp_interval 0
Vrrp_gna_interval 0
}

Vrrp_instance VI_1 {
State MASTER # from server for BACKUP
Interface ens33 # binds the real nic to ens33
Virtual_router_id 51
The priority of 100
Advert_int 1
The authentication {
The auth_type PASS
Auth_pass 1111
}
Virtual_ipaddress {# virtual IP address
192.168.100.10
}
}

Virtual_server 192.168.100.10 80 {# virtual IP address, port number 80
Delay_loop 6
Lb_algo rr
Lb_kind NAT
Persistence_timeout 50
Protocol TCP

Real_server 192.168.100.100 80 {#nginx1ip address, port 80
The weight of 1
Change TCP_CHECK {# to TCP_CHECK and delete the original url8 line
Connect_port 80
Connect_timeout 3
Nb_get_retry 3
Delay_before_retry 3
}
}
Real_server 192.168.100.110 443 {#nginx2ip address, port 80
The weight of 1
Change TCP_CHECK {# to TCP_CHECK and delete the original url8 line
Connect_port 80
Connect_timeout 3
Nb_get_retry 3
Delay_before_retry 3
}
}
}
2. Open keepalived service

[[email protected] ~]# systemctl start keepalived. Service
[[email protected] ~]# systemctl status keepalived. Service #

Client testing
1. Test nginx website by visiting virtual IP

2. Shut down the virtual IP of Master's keepalived server and access it again
[[email protected] network-scripts]# systemctl stop keepalived. Service

Conclusion:
1. Due to the virtual machine simulation case used in this paper, all software packages were completed in NAT mode, and then modified to host only mode.
2. As for keepalived, please check the firewall and close it again.
3. The master's keepalved was down during the test, and it was found that the site could not be accessed, so we need to check the virtual IP and restart ipvsadm.

Nginx highly available cluster architecture (Keepalived+ Nginx +ipvsadm