Nginx + keepalived for dual-machine hot standby and tomcat Load Balancing

Nginx + keepalived for dual-machine hot standby and tomcat Load Balancing

Environment Description:

nginx1:192.168.2.47nginx2:192.168.2.48tomcat1:192.168.2.49tomcat2:192.168.2.50vip:192.168.2.51

I. nginx Configuration
1. Install the PCRE Library required by nginx
Wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.10.tar.gz

tar -zxvf pcre-8.10.tar.gzcd pcre-8.10./configuremakemake install

2. Install nginx
Wget http://nginx.org/download/nginx-0.8.52.tar.gz

groupadd wwwuseradd -g www wwwtar zxvf nginx-0.8.52.tar.gzcd nginx-0.8.52/./configure --user=www --group=www --prefix=/usr/local/nginx --with-http_stub_status_module --with-http_ssl_modulemakemake install

Note: If the following error occurs

./Configure: Error: SSL modules require the OpenSSL library. centos to install OpenSSL-develubuntu, install: sudo apt-Get install libssl-Dev.

3. modify the configuration file to the following:

User WWW; worker_processes 2; PID logs/nginx. PID; worker_rlimit_nofile 51200; events {use epoll; worker_connections 51200;} HTTP {include mime. types; default_type application/octet-stream; keepalive_timeout 120; server_tokens off; send_timeout 60; tcp_nodelay on; upstream effecats {server 192.168.2.50: 8080; server 192.168.2.49: 8080; # ip_hash; # ip_hash can solve the session problem without sharing sessions} server {Listen 80; SERVER_NAME 192.168.2.48; Location/{proxy_pass http: // Tomcats; proxy_set_header host $ host; proxy_set_header X-real-IP $ remote_addr; proxy_set_header X-forwarded-for $ proxy_add_x_forwarded_for ;} log_format access_log '$ remote_addr-$ remote_user [$ time_local] $ request ''" $ status "$ response" $ http_referer "'' "$ http_user_agent" "$ http_x_forwarded_for "'; access_log/usr/local/nginx/logs/access. log access_log ;}}

4. Test the configuration file

/usr/local/nginx/sbin/nginx -t

If the following conditions occur:

/usr/local/nginx/sbin/nginx: error while loading shared libraries: libpcre.so.0: or directory

Solution:

sudo ln -s /usr/local/lib/libpcre.so.0 /usr/lib/libpcre.so.0

/Usr/local/nginx/sbin/nginx-T
The following information is displayed as correct

the configuration file /usr/local/nginx/conf/nginx.conf syntax is okconfiguration file /usr/local/nginx/conf/nginx.conf test is successful

5. Optimize Kernel Parameters
Vim/etc/sysctl. conf is added at the end.

net.ipv4.tcp_max_syn_backlog = 65536net.core.netdev_max_backlog = 32768net.core.somaxconn = 32768net.core.wmem_default = 8388608net.core.rmem_default = 8388608net.core.rmem_max = 16777216net.core.wmem_max = 16777216net.ipv4.tcp_timestamps = 0net.ipv4.tcp_synack_retries = 2net.ipv4.tcp_syn_retries = 2net.ipv4.tcp_tw_recycle = 1net.ipv4.tcp_tw_reuse = 1net.ipv4.tcp_mem = 94500000 915000000 927000000net.ipv4.tcp_max_orphans = 3276800net.ipv4.ip_local_port_range = 1024  65535

Save and exit.

sysctl -p

6. Cut the nginx log script

#!/bin/bashPATH_LOGS="/usr/local/nginx/logs"YEAR=`date -d "-1 days" +"%Y"`MONTH=`date -d "-1 days" +"%m"`mkdir -p $PATH_LOGS/$YEAR/$MONTHmv $PATH_LOGS/access.log $PATH_LOGS/$YEAR/$MONTH/access_$(date -d "-1 days" +"%Y%m%d").logkill -USR1 `cat $PATH_LOGS/nginx.pid`

Add the script to crontab and execute it at every day.
Note: The installation steps of nginx on the slave server are the same as those above.

Ii. install and configure keepalived
1. download the required software
Wget http://keepalived.org/software/keepalived-1.1.19.tar.gz
Wget http://rpm5.org/files/popt/popt-1.16.tar.gz
2. Install popt
Popt is required for compiling keepalived; otherwise, the following error is reported:

configure: error: Popt libraries is required

tar -zxvf popt-1.16.tar.gzcd popt-1.16./configuremakemake install

3. Install keepalived

tar -zxvf keepalived-1.1.19.tar.gzcd keepalived-1.1.19./configure --prefix=/usr/local/keepalivedmakemake install

4. modify the configuration file to the following:

Vim/usr/local/keepalived/etc/keepalived. conf

! Configuration file for keepalived global_defs {router_id lvs_devel} vrrp_script monitor_nginx {script "/root/scripts" # Place routing interval 2 weight 2} vrrp_instance vi_1 {state master Interface eth0 virtual_router_id 51 priority 100 advert_int 1 authentication {auth_type pass auth_pass 1234} track_script {monitor_nginx} virtual_ipaddress {192.168.2.51 }}

Note: monitor_nginx.sh is the script used to monitor nginx processes. The content is as follows:

#!/bin/bashif [ "$(ps -ef | grep "nginx: master process"| grep -v grep )" == "" ]then /usr/local/nginx/sbin/nginx sleep 5 if [ "$(ps -ef | grep "nginx: master process"| grep -v grep )" == "" ] then killall keepalived fifi

5. Start keepalived.

/usr/local/keepalived/sbin/keepalived -D -f /usr/local/keepalived/etc/keepalived/keepalived.conf

Note: The keepalived installation on the slave is the same as the above. You only need to change the configuration file to the following (change the master to backup)

! Configuration file for keepalived global_defs {router_id lvs_devel} vrrp_script failed {script "/root/scripts/messages" interval 2 weight 2} vrrp_instance vi_1 {state backup # change to backup interface eth0 virtual_router_id 51 priority # advert_int 1 authentication {auth_type pass auth_pass 1234} track_script {monitor_nginx} virtual_ipaddress {192.168.2.51} lower than the master Value }}

Iii. Test Procedure

1. Access the VIP to check whether the backend Tomcat can be accessed normally.
2. Stop one Tomcat and check whether the access can be forwarded to another server.
3. Stop any nginx process on the two nginx servers to check whether the monitoring process script will automatically start nginx.
4. Stop the keepalived process on any nginx instance to check whether the other instance takes over the VIP instance.
For example, stop keepalived on the master, for example, killall keepalived, and check whether the backup machine has taken over. If the backup machine logs are generated after the backup machine takes over
Tail/var/log/syslog

Keepalived_vrrp: VRRP_Instance(VI_1) Transition to MASTER STATEKeepalived_vrrp: VRRP_Instance(VI_1) Entering MASTER STATEKeepalived_vrrp: VRRP_Instance(VI_1) setting protocol VIPs.Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.2.51

Logs on the master machine are displayed.

Keepalived_vrrp: Terminating VRRP child process on signalKeepalived_vrrp: VRRP_Instance(VI_1) removing protocol VIPs.

Now, restart keepalived on the master and you will see that the master takes over the VIP and provides external services. The backup still returns to the backup state. If this is not the case, check the configuration file and steps.

The current backup log is as follows:

Keepalived_vrrp: VRRP_Instance(VI_1) Received higher prio advertKeepalived_vrrp: VRRP_Instance(VI_1) Entering BACKUP STATEKeepalived_vrrp: VRRP_Instance(VI_1) removing protocol VIPs.

The master log is as follows:

Keepalived_vrrp: VRRP_Script(Monitor_Nginx) succeededKeepalived_vrrp: VRRP_Instance(VI_1) Transition to MASTER STATEKeepalived_vrrp: VRRP_Instance(VI_1) Entering MASTER STATEKeepalived_vrrp: VRRP_Instance(VI_1) setting protocol VIPs.Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.2.51

Nginx + keepalived for dual-machine hot standby and tomcat Load Balancing