nginx+keepalived implementation of Web server load Balancing

Description

Operating system: CentOS 5.X 64-bit

Web server: 192.168.21.127, 192.168.21.128

Sites: Bbs.111cn.net and Sns.111cn.net deployed on two Web servers

To achieve the purpose:

Add two servers (main main mode) to achieve Web server load balancing through nginx+keepalived

Architecture Planning:

Nginx server: 192.168.21.129, 192.168.21.130

Virtual Server (VIP): 192.168.21.252, 192.168.21.253

After deployment is complete:

1, vip:192.168.21.253 Point 192.168.21.129;vip:192.168.21.252 point to 192.168.21.130;

2, when 192.168.21.129 downtime, vip:192.168.21.253 drift to 192.168.21.130;

3, when 192.168.21.130 downtime, vip:192.168.21.252 drift to 192.168.21.129;

The main advantage of this is that the two servers provide services while they are each other's backup servers.

Specific actions:

Part one: Operate on two NGINX servers separately

First, close the SELinux, configure the firewall

1, Vi/etc/selinux/config

#SELINUX =enforcing #注释掉

#SELINUXTYPE =targeted #注释掉

Selinux=disabled #增加

: wq! #保存退出

Setenforce 0 #使配置立即生效

2, Vi/etc/sysconfig/iptables #编辑

-A rh-firewall-1-input-d 224.0.0.18-j ACCEPT #允许组播地址通信

-A rh-firewall-1-input-p vrrp-j ACCEPT #允许VRRP (Virtual Router Redundancy association) communication

-A rh-firewall-1-input-m state--state new-m tcp-p TCP--dport 80-j ACCEPT #允许80端口通过防火墙

: wq! #保存退出

/etc/init.d/iptables Restart #重启防火墙使配置生效

Second, install Nginx

1, install the compilation toolkit (use CentOS Yum command to install, install the package more, convenient configuration LNMP environment later)

Yum install-y make apr* autoconf automake Curl curl-devel gcc gcc-c++ gtk+-devel zlib-devel OpenSSL openssl-devel Vel gd kernel keyutils patch perl kernel-headers compat* cpp glibc libgomp libstdc++-devel keyutils-libs-devel libsepol-d Evel Libselinux-devel krb5-devel libxpm* freetype freetype-devel freetype* fontconfig fontconfig-devel libjpeg* libpng* Php-common php-gd gettext gettext-devel ncurses* libtool* libxml2 libxml2-devel patch Policycoreutils Bison

2. Download package

(1) http://nginx.org/download/nginx-1.4.7.tar.gz #下载Nginx

(2) ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.34.tar.gz #下载pcre (support nginx pseudo static)

(3) http://labs.frickle.com/files/ngx_cache_purge-2.1.tar.gz #下载ngx_cache_purge (convenient to extend the configuration Nginx cache server later)

Upload the above packages to the/USR/LOCAL/SRC directory

3. Installation Pcre

Cd/usr/local/src

Mkdir/usr/local/pcre #创建安装目录

Tar zxvf pcre-8.34.tar.gz

CD pcre-8.34

./configure--prefix=/usr/local/pcre #配置

Make

Make install

4. Installation Nginx

Cd/usr/local/src

Groupadd www #添加www组

Useradd-g www www-s/bin/false #创建nginx运行账户www并加入到www组, www users are not allowed to log on directly to the system

CD/USR/LOCAL/SRC #进入安装目录

Tar zxvf ngx_cache_purge-2.1.tar.gz #解压

Tar zxvf nginx-1.4.7.tar.gz #解压

CD nginx-1.4.7

./configure--prefix=/usr/local/nginx--without-http_memcached_module--user=www--group=www--with-http_stub_ Status_module--with-openssl=/usr/--with-pcre=/usr/local/src/pcre-8.34--add-module=. /ngx_cache_purge-2.1 #配置

Note:--with-pcre=/usr/local/src/pcre-8.34 points to the source package decompression path, not the installation path, otherwise it will be an error

Make #编译

Make install #安装

/usr/local/nginx/sbin/nginx #启动nginx

Set Nginx to open boot

Vi/etc/rc.d/init.d/nginx #编辑启动文件添加下面内容

=======================================================

#!/bin/sh

#

# Nginx-this script starts and stops the Nginx daemon

#

# Chkconfig:-85 15

# Description:nginx is a HTTP (s) server, HTTP (s) reverse

# Proxy and IMAP/POP3 proxy server

# Processname:nginx

# config:/etc/nginx/nginx.conf

# config:/usr/local/nginx/conf/nginx.conf

# Pidfile:/usr/local/nginx/logs/nginx.pid

# Source function library.

. /etc/rc.d/init.d/functions

# Source Networking configuration.

. /etc/sysconfig/network

# Check that networking are up.

["$NETWORKING" = "no"] && exit 0

nginx= "/usr/local/nginx/sbin/nginx"

prog=$ (basename $nginx)

Nginx_conf_file= "/usr/local/nginx/conf/nginx.conf"

[-f/etc/sysconfig/nginx] &&. /etc/sysconfig/nginx

Lockfile=/var/lock/subsys/nginx

Make_dirs () {

# Make required Directories

User= ' $nginx-v 2>&1 | grep "Configure arguments:" | Sed ' s/[^*]*--user= ([^]*). *//g '-'

If [-Z "' grep $user/etc/passwd ']"; Then

Useradd-m-s/bin/nologin $user

Fi

Options= ' $nginx-v 2>&1 | grep ' Configure arguments: '

for opt in $options; Todo

If [' Echo $opt | grep '. *-temp-path ']; Then

Value= ' echo $opt | Cut-d "=" F 2 '

if [!-D "$value"]; Then

# echo "Creating" $value

Mkdir-p $value && chown-r $user $value

Fi

Fi

Done

}

Start () {

[x $nginx] | | Exit 5

[f $NGINX _conf_file] | | Exit 6

Make_dirs

Echo-n $ "Starting $prog:"

Daemon $nginx-C $NGINX _conf_file

Retval=$?

Echo

[$retval-eq 0] && Touch $lockfile

Return $retval

}

Stop () {

Echo-n $ "Stopping $prog:"

Killproc $prog-quit

Retval=$?

Echo

[$retval-eq 0] && rm-f $lockfile

Return $retval

}

Restart () {

#configtest | | Return $?

Stop

Sleep 1

Start

}

Reload () {

#configtest | | Return $?

Echo-n $ "Reloading $prog:"

Killproc $nginx-hup

Retval=$?

Echo

}

Force_reload () {

Restart

}

Configtest () {

$nginx-T-C $NGINX _conf_file

}

Rh_status () {

Status $prog

}

Rh_status_q () {

Rh_status >/dev/null 2>&1

}

Case "$" in

Start

Rh_status_q && Exit 0

$

;;

Stop

Rh_status_q | | Exit 0

$

;;

Restart|configtest)

$

;;

Reload

Rh_status_q | | Exit 7

$

;;

Force-reload)

Force_reload

;;

Status

Rh_status

;;

Condrestart|try-restart)

Rh_status_q | | Exit 0

;;

*)

echo $ "Usage: $ {start|stop|status|restart|condrestart|try-restart|reload|force-reload|configtest}"

Exit 2

Esac

=======================================================

: wq! #保存退出

chmod 775/etc/rc.d/init.d/nginx #赋予文件执行权限

Chkconfig Nginx on #设置开机启动

/etc/rc.d/init.d/nginx Restart #重新启动Nginx

Service Nginx Restart

=======================================================

Third, the configuration Nginx

Cp/usr/local/nginx/conf/nginx.conf/usr/local/nginx/conf/nginx.confbak #备份nginx配置文件

1. Set up Nginx operation account

vi/usr/local/nginx/conf/nginx.conf #编辑, modify

Find user nobody;

User www www. #在第一行

: wq! #保存退出

2, prohibit the Nginx empty host head

Vi/usr/local/nginx/conf/nginx.conf #编辑

Locate the server and add the following on the line above:

##############################

server {

Listen default;

server_name _;

Location/{

root HTML;

return 404;

}

Location ~/.ht {

Deny all;

}

}

##############################

: wq! #保存退出

/etc/rc.d/init.d/nginx Restart #重启nginx

After this setting, NULL host header access jumps directly to the nginx404 error page.

3, add Nginx virtual host contains files

cd/usr/local/nginx/conf/#进入nginx安装目录

mkdir Vhost #建立虚拟目录

Vi/usr/local/nginx/conf/nginx.conf #编辑

Find the code you added in the previous step and add the following at the end:

Include vhost/*.conf;

: wq! #保存退出

For example:

##############################

server {

Listen default;

server_name _;

Location/{

root HTML;

return 404;

}

Location ~/.ht {

Deny all;

}

}

Include vhost/*.conf;

##############################

4. Add Web server list file

cd/usr/local/nginx/conf/#进入目录

Touch mysvrhost.conf #建立文件

Vi/usr/local/nginx/conf/nginx.conf #编辑

Find the code you added in the previous step and add a row below

Include mysvrhost.conf;

: wq! #保存退出

5, set Nginx global parameters

Vi/usr/local/nginx/conf/nginx.conf #编辑

Worker_processes 2; # Number of worker processes, or twice times the core of the CPU

Events

{

Use Epoll; #增加

Worker_connections 65535; #修改为65535, maximum number of connections.

}

############ #以下代码在 http {partially added and modified ##############

Server_names_hash_bucket_size 128; #增加

Client_header_buffer_size 32k; #增加

Large_client_header_buffers 4 32k; #增加

Client_max_body_size 300m; #增加

Tcp_nopush on; #修改为on

Keepalive_timeout 60; #修改为60

Tcp_nodelay on; #增加

Server_tokens off; #增加, Nginx version information is not displayed

gzip on; #修改为on

Gzip_min_length 1k; #增加

Gzip_buffers 4 16k; #增加

Gzip_http_version 1.1; #增加

Gzip_comp_level 2; #增加

Gzip_types text/plain application/x-javascript text/css application/xml; #增加

Gzip_vary on; #增加

6. Set up a list of Web servers

cd/usr/local/nginx/conf/#进入目录

VI mysvrhost.conf #编辑, add the following code

Upstream Osyunweihost {

Server 192.168.21.127:80 weight=1 max_fails=2 fail_timeout=30s;

Server 192.168.21.128:80 weight=1 max_fails=2 fail_timeout=30s;

Ip_hash;

}

7. New Virtual Host configuration file

Cd/usr/local/nginx/conf/vhost #进入虚拟主机目录

Touch osyunwei.conf #建立虚拟主机配置文件

VI osyunwei.conf #编辑

Log_format access ' $remote _addr-$remote _user [$time _local] $request '

"$status" $body _bytes_sent "$http _referer"

' $http _user_agent ', ' $http _x_forwarded_for ';

Server

{

Listen 80;

server_name bbs.111cn.net sns.111cn.net;

Location/

{

Proxy_next_upstream http_502 http_504 error timeout invalid_header;

Proxy_pass Http://osyunweihost;

#proxy_redirect off;

Proxy_set_header Host $host;

Proxy_set_header x-forwarded-for $remote _addr;

}

Location/nginxstatus {

Stub_status on;

Access_log on;

Auth_basic "Nginxstatus";

#auth_basic_user_file pwd;

}

Access_log/usr/local/nginx/logs/access.log access;

}

: wq! #保存配置

Service Nginx Restart #重启nginx

Iv. installation of keepalived

System Yun-wei Www.111cn.net warm reminder: qihang01 original Content © Copyright, reproduced please specify the source and the original link

Download keeplived:http://www.keepalived.org/software/keepalived-1.2.12.tar.gz

Upload keepalived-1.2.12.tar.gz to/usr/local/src directory

Cd/usr/local/src

Tar zxvf keepalived-1.2.12.tar.gz

CD keepalived-1.2.12

./configure--prefix=/usr/local/keepalived #配置, you must see the following prompts to indicate that you are properly configured to continue Setup

Use Ipvs Framework:yes

Ipvs Sync Daemon Support:yes

Use VRRP Framework:yes

Make #编辑

Make install #安装

Cp/usr/local/keepalived/etc/sysconfig/keepalived/etc/sysconfig/keepalived

Mkdir/etc/keepalived

Ln-s/usr/local/keepalived/etc/keepalived/keepalived.conf/etc/keepalived/

cp/usr/local/keepalived/sbin/keepalived/usr/sbin/

cp/usr/local/keepalived/etc/rc.d/init.d/keepalived/etc/rc.d/init.d/

chmod +x/etc/rc.d/init.d/keepalived #添加执行权限

Chkconfig keepalived on #设置开机启动

Service keepalived Start #启动

Service keepalived Stop #关闭

Service keepalived Restart #重启

V. Configuration of keepalived

Cp/usr/local/keepalived/etc/keepalived/keepalived.conf/usr/local/keepalived/etc/keepalived/keepalived.conf-bak

Vi/usr/local/keepalived/etc/keepalived/keepalived.conf #编辑, modified to the following code

#########################################################

#以下为192.168.21.129 Server

! Configuration File for Keepalived

Global_defs {

Notification_email {

Acassen@firewall.loc

Failover@firewall.loc

Sysadmin@firewall.loc

}

Notification_email_from Alexandre.Cassen@firewall.loc

Smtp_server 192.168.200.1

Smtp_connect_timeout 30

router_id Lvs_devel

}

Vrrp_script Chk_nginx {

Script "/etc/keepalived/check_nginx.sh" #Nginx服务监控脚本

Interval 2

Weight 2

}

Vrrp_instance Vi_1 {

State MASTER

Interface eth0

VIRTUAL_ROUTER_ID 51

Priority 100

Advert_int 1

Authentication {

Auth_type Pass

Auth_pass 1111

}

Track_script {

Chk_nginx #监测nginx进程状态

}

virtual_ipaddress {

192.168.21.253

}

Notify_master the ARP record of the "/etc/keepalived/clean_arp.sh 192.168.21.253" #更新虚拟服务器 (VIP) address to the gateway

}

Vrrp_instance Vi_2 {

State BACKUP

Interface eth0

VIRTUAL_ROUTER_ID 52

Priority 99

Advert_int 1

Authentication {

Auth_type Pass

Auth_pass 1111

}

virtual_ipaddress {

192.168.21.252

}

Notify_master the ARP record of the "/etc/keepalived/clean_arp.sh 192.168.21.252" #更新虚拟服务器 (VIP) address to the gateway

}

#########################################################

: wq! #保存退出

#########################################################

#以下为192.168.21.130 Server

! Configuration File for Keepalived

Global_defs {

Notification_email {

Acassen@firewall.loc

Failover@firewall.loc

Sysadmin@firewall.loc

}

Notification_email_from Alexandre.Cassen@firewall.loc

Smtp_server 192.168.200.1

Smtp_connect_timeout 30

router_id Lvs_devel

}

Vrrp_script Chk_nginx {

Script "/etc/keepalived/check_nginx.sh" #Nginx服务监控脚本

Interval 2

Weight 2

}

Vrrp_instance Vi_1 {

State BACKUP

Interface eth0

VIRTUAL_ROUTER_ID 51

Priority 99

Advert_int 1

Authentication {

Auth_type Pass

Auth_pass 1111

}

Track_script {

Chk_nginx #监测nginx进程状态

}

virtual_ipaddress {

192.168.21.253

}

Notify_master the ARP record of the "/etc/keepalived/clean_arp.sh 192.168.21.253" #更新虚拟服务器 (VIP) address to the gateway

}

Vrrp_instance Vi_2 {

State MASTER

Interface eth0

VIRTUAL_ROUTER_ID 52

Priority 100

Advert_int 1

Authentication {

Auth_type Pass

Auth_pass 1111

}

virtual_ipaddress {

192.168.21.252

}

Notify_master the ARP record of the "/etc/keepalived/clean_arp.sh 192.168.21.252" #更新虚拟服务器 (VIP) address to the gateway

}

#########################################################

: wq! #保存退出

Vi. setting up Nginx service monitoring scripts

touch/usr/local/keepalived/check_nginx.sh

Ln-s/usr/local/keepalived/check_nginx.sh/etc/keepalived/check_nginx.sh

Vi/etc/keepalived/check_nginx.sh #编辑, add the following code

#########################################################

#!/bin/sh

If [$ (ps-c nginx--no-header | wc-l)-EQ 0]; Then

/etc/rc.d/init.d/nginx start

Fi

Sleep 2

If [$ (ps-c nginx--no-header | wc-l)-EQ 0]; Then

/etc/rc.d/init.d/keepalived stop

Fi

#########################################################

: wq! #保存退出

chmod +x/usr/local/keepalived/check_nginx.sh #添加执行权限

Vii. set ARP record to update virtual Server (VIP) address to Gateway script

touch/usr/local/keepalived/clean_arp.sh

Ln-s/usr/local/keepalived/clean_arp.sh/etc/keepalived/clean_arp.sh

Vi/etc/keepalived/clean_arp.sh #编辑, add the following code

#!/bin/sh

Vip=$1

gateway=192.168.21.2 #网关地址

/sbin/arping-i eth0-c 5-s $VIP $GATEWAY &>/dev/null

: wq! #保存退出

chmod +x/usr/local/keepalived/clean_arp.sh #添加脚本执行权限

Service Nginx Restart #重启nginx

Service keepalived Restart #重启keepalived

Part II: Test nginx+keepalived whether the normal operation

First, bbs.111cn.net parsing to 192.168.21.253;sns.111cn.net resolution to 192.168.21.252;

Executing commands on two Nginx servers: 192.168.21.129, 192.168.21.130: IP addr

As shown in the following illustration:

System Yun-wei Www.111cn.net warm reminder: qihang01 original Content © Copyright, reproduced please specify the source and the original link

It can be seen now that vip:192.168.21.253 points to 192.168.21.129;vip:192.168.21.252 point 192.168.21.130;

Open in Browser

http://bbs.111cn.net/

http://sns.111cn.net/

As shown in the following illustration:

At this time, BBS and SNS domain name are balanced to 192.168.21.127 above

Second, stop the 192.168.21.127 above Nginx service

Service Nginx Stop

Continue to open the two URLs above, as shown in the following illustration:

At this time, BBS and SNS domain name are balanced to 192.168.21.128 above (because 192.168.21.127 server Nginx service is shut down, realize failover)

Third, close the keepalived service above 192.168.21.129

Service keepalived Stop

At this point, execute the command on two keepalived servers: 192.168.21.129, 192.168.21.130: IP addr

As shown in the following illustration:

It can be seen that vip:192.168.21.253 and 192.168.21.252 all point to 192.168.21.130;

At this point, open http://bbs.111cn.net/as shown in the following illustration:

can be accessed normally

Iv. Restore the keepalived service above the 192.168.21.129, restore the Nginx service above the 192.168.21.127, and stop 192.168.21.130 above keepalived service

Service keepalived Stop

Executing commands on two keepalived servers: 192.168.21.129, 192.168.21.130: IP addr

As shown in the following illustration:

It can be seen that vip:192.168.21.253 and 192.168.21.252 all point to 192.168.21.129;

At this point, open http://sns.111cn.net/as shown in the following illustration:

can be accessed normally