Nginx master cluster

When talking with some friends about Nginx + Keepalived technology, although I have successfully implemented the Nginx + Keepaived project for many times, these are all work with a single-master Nginx, since Nginx is only in the backup state for a long time, we want to put both Nginx server load balancers in the working state...

When talking with some friends about Nginx + Keepalived technology, although I have successfully implemented the Nginx + Keepaived project for many times, these are all work with a single-master Nginx, since Nginx is only in the backup state for a long time, we want to put both Nginx load balancers in the working state. In fact, Nginx + Keepalived is also easy to implement. This method is applicable to small and medium-sized websites.
Generally, for the convenience of maintenance, the servers of the Enterprise website are all in their internal data centers, and only two ports 80 and 443 of the Keepalived VIP address are opened, which are mapped out through the Juniper SSG550 firewall, the Internet DNS corresponds to the mapped public IP address. The firewall and network security of this architecture are described as follows:
The system architecture only maps ports 80 and 443 of the intranet VIP to the Juniper SSG550 firewall on the Internet. All other ports are disabled, and all intranet machines are disabled; the Internet DNS points to the internet address mapped by Juniper SSG550.
Nginx server load balancer generally encounters the following faults: 1. network faults such as server network cable looseness; 2. crash due to server hardware failure; 3. nginx service process will die (this situation will be encountered theoretically, but in fact this situation has not occurred on Linux servers in the production environment, it is enough to prove the stability of Nginx as a load balancer/reverse proxy server, we can solve this problem through technical means ).
Test lab environment:
One of the main Nginx: 192.168.1.5
Master Nginx 2: 192.168.1.6
Web server 1: 192.168.1.17
Web server 2: 192.168.1.18
VIP address 1: 192.168.1.8
VIP address 2: 192.168.1.9
I. Nginx and Keepalived are relatively simple to install. I will not repeat them here. For details, refer to the articles in my topic series. refer to the following link:
User www;
Worker_processes 8;
Pid/usr/local/nginx/logs/nginx. pid;
Worker_rlimit_nofile 51200;
Events
{
Use epoll;
Worker_connections 51200;
}
Http {
Include mime. types;
Default_type application/octet-stream;
Server_names_hash_bucket_size 128;
Client_header_buffer_size 32 k;
Large_client_header_buffers 4 32 k;
Client_max_body_size 8 m;
Sendfile on;
Tcp_nopush on;
Keepalive_timeout 60;
Tcp_nodelay on;
Fastcgi_connect_timeout 300;
Fastcgi_send_timeout 300;
Fastcgi_read_timeout 300;
Fastcgi_buffer_size 64 k;
Fastcgi_buffers 4 64 k;
Fastcgi_busy_buffers_size 128 k;
Fastcgi_temp_file_write_size 128 k;
Gzip on;
Gzip_min_length 1 k;
Gzip_buffers 4 16 k;
Gzip_http_version 1.0;
Gzip_comp_level 2;
Gzip_types text/plain application/x-javascript text/css application/xml;
Gzip_vary on;

Upstream backend
{
Ip_hash;
Server 192.168.1.17: 80;
Server 192.168.1.18: 80;
}
Server {
Listen 80;
Server_name www.1paituan.com;
Location /{
Root/var/www/html;
Index. php index.htm index.html;
Proxy_redirect off;
Proxy_set_header Host $ host;
Proxy_set_header X-Real-IP $ remote_addr;
Proxy_set_header X-Forwarded-For $ proxy_add_x_forwarded_for;
Proxy_pass http: // backend;
}

Location/nginx {
Access_log off;
Auth_basic "NginxStatus ";
# Auth_basic_user_file/usr/local/nginx/htpasswd;
}

Log_format access' $ remote_addr-$ remote_user [$ time_local] "$ request "'
'$ Status $ body_bytes_sent "$ http_referer "'
'"$ Http_user_agent" $ http_x_forwarded_for ';
Access_log/data/logs/access. log access;
}
 
II. configure the Keepalived file. here I will briefly describe the principle. In fact, two instances are generated through Keepalived, and two Nginx instances are backed up to each other, that is, the first is the backup machine of the second machine, the second machine is also the first slave machine, and the generated two VIP addresses correspond to our website http://www.1paituan.com, so that you can access our website through DNS round robin on the public network, if any Nginx machine suffers hardware damage, Keepalived automatically switches its VIP address to another machine without affecting client access, this is the same principle as our previous LVS + Keepalived multi-instance. I believe everyone can understand it.
The Keepalived. conf configuration file of one of the main Nginx machines is as follows:
! Configuration File for keepalived
Global_defs {
Notification_email {
Yuhongchun027@163.com
}
Notification_email_from keepalived@chtopnet.com
Smtp_server 127.0.0.1
Smtp_connect_timeout 30
Router_id LVS_DEVEL
}
Vrrp_instance VI_1 {
State MASTER
Interface eth0
Virtual_router_id 51
Priority100
Advert_int 1
Authentication {
Auth_type PASS
Auth_pass 1paituan.com
}
Virtual_ipaddress {
192.168.1.8
}
}
Vrrp_instance VI_2 {
State BACKUP
Interface eth0
Virtual_router_id 52
Priority 99
Advert_int 1
Authentication {
Auth_type PASS
Auth_pass 1paituan.com
}
Virtual_ipaddress {
192.168.1.9
}
}
 
The keepalivd. conf configuration file for the master Nginx 2 is as follows:
! Configuration File for keepalived
Global_defs {
Notification_email {
Yuhongchun027@163.com
}
Notification_email_from keepalived@chtopnet.com
Smtp_server 127.0.0.1
Smtp_connect_timeout 30
Router_id LVS_DEVEL
}
Vrrp_instance VI_1 {
State BACKUP
Interface eth0
Virtual_router_id 51
Priority 99
Advert_int 1
Authentication {
Auth_type PASS
Auth_pass 1 paituan
}
Virtual_ipaddress {
192.168.1.8
}
}
Vrrp_instance VI_2 {
State MASTER
Interface eth0
Virtual_router_id 52
Priority100
Advert_int 1
Authentication {
Auth_type PASS
Auth_pass 1 paituan
}
Virtual_ipaddress {
192.168.1.9
}
}
 
The process script for monitoring Nginx on two machines. the script content is as follows:
#! /Bin/bash
While:
Do
Nginxpid = 'PS-C nginx -- no-header | wc-L'
If [$ nginxpid-eq 0]; then
/Usr/local/nginx/sbin/nginx
Sleep 5
Nginxpid = 'PS-C nginx -- no-header | wc-L'
Echo $ nginxpid
If [$ nginxpid-eq 0]; then
/Etc/init. d/keepalived stop
Fi
Fi
Sleep 5
Done
 
The command is as follows:
Nohup sh/root/nginxpid. sh &
 
I downloaded this script directly from the production server. do not doubt that it will cause endless loops and validity issues. I would like to explain it a bit. this is an infinite loop script, put it on the primary Nginx machine (because it is mainly used to provide services at present), execute it every 5 seconds, and use the ps-C command to collect whether the nginx PID value is 0, if it is 0 (that is, the Nginx process is dead), try to start the nginx process; if it continues to be 0, that is, the nginx startup is not changed, then disable the Keeplaived process of the local machine, the VIP address will be taken over by the slave machine. of course, the entire website will be provided by Nginx of the slave machine, so as to ensure the high availability of the Nginx process.
4. after The Nginx and Keealived programs of the two primary Nginx servers are started normally, the normal IP addresses of the two servers are displayed as follows:
This is the IP addr command of the machine whose ip address is 192.168.1.5. The result is as follows:
1: lo: Mtu 16436 qdisc noqueue
Link/loopback 00: 00: 00: 00: 00: 00 brd 00: 00: 00: 00: 00: 00
Inet 127.0.0.1/8 scope host lo
2: eth0: Mtu 1500 qdisc pfifo_fast qlen 1000
Link/ether 00: 0c: 29: 99: fb: 32 brd ff: ff
Inet 192.168.1.5/24 brd 192.168.1.255 scope global eth0
Inet 192.168.1.8/32 scope global eth0
 
This is the IP addr command of the machine whose ip address is 192.168.1.6. The result is as follows:
1: lo: Mtu 16436 qdisc noqueue
Link/loopback 00: 00: 00: 00: 00: 00 brd 00: 00: 00: 00: 00: 00
Inet 127.0.0.1/8 scope host lo
Inet6: 1/128 scope host
Valid_lft forever preferred_lft forever
2: eth0: Mtu 1500 qdisc pfifo_fast qlen 1000
Link/ether 00: 0c: 29: 7d: 58: 5e brd ff: ff
Inet 192.168.1.6/24 brd 192.168.1.255 scope global eth0
Inet 192.168.1.9/32 scope global eth0
Inet6 fe80: 20c: 29ff: fe7d: 585e/64 scope link
Valid_lft forever preferred_lft forever
3: sit0: Mtu 1480 qdisc noop
Link/sit 0.0.0.0 brd 0.0.0.0
 
5. the test process is as follows:
1. we need to use killall to kill the Nginx process on the two primary Nginx servers, and then access the IP addresses 192.168.1.8 and 192.168.1.9 (simulating DNS polling) on the client to check whether the two IP addresses can access the Web server normally.
2. restart the master Nginx load balancer of 192.168.1.5. the test process is as follows;
3. restart the master Nginx load balancer of 192.168.1.6. the test process is as follows;
4. try to shut down the machines 192.168.1.5 and 192.168.1.6 respectively. the test process is as follows. does it affect normal website access?
6. problems to be solved in production:
I. monitoring services such as Cacti and Nagios need to be redeployed because the client currently accesses two server load balancer instances separately;
2. log collection needs to be redeployed. Currently, access logs are distributed on two server load balancer instances;
3. google indexing should be considered;
4. certificate issues, required by both machines;
5. I have not thought of any other questions for the time being.
Author: "Fuqin liquor"