Implement Web server load Balancing in Linux (haproxy+keepalived)

Source: Internet
Author: User
Tags chmod failover mkdir syslog iptables net domain haproxy firewall

Description
Operating system: CentOS 5.X 64-bit
Web server: 192.168.21.127, 192.168.21.128
Sites: Bbs.111cn.net and Sns.111cn.net deployed on two Web servers
To achieve the purpose:
Add two servers (main main mode) to achieve Web server load balancing through haproxy+keepalived
Architecture Planning:
Haproxy server: 192.168.21.129, 192.168.21.130
Virtual Server (VIP): 192.168.21.253, 192.168.21.254
After deployment is complete:
1, vip:192.168.21.253 Point 192.168.21.129;vip:192.168.21.254 point to 192.168.21.130;
2, when 192.168.21.129 downtime, vip:192.168.21.253 drift to 192.168.21.130;
3, when 192.168.21.130 downtime, vip:192.168.21.254 drift to 192.168.21.129;
The main advantage of this is that the two servers provide services while they are each other's backup servers.
Specific actions:
Part one: Operate on two HAPROXY servers separately
First, close the SELinux, configure the firewall
1, Vi/etc/selinux/config
#SELINUX =enforcing #注释掉
#SELINUXTYPE =targeted #注释掉
Selinux=disabled #增加
: wq! #保存退出
Setenforce 0 #使配置立即生效
2, Vi/etc/sysconfig/iptables #编辑
-A rh-firewall-1-input-d 224.0.0.18-j ACCEPT #允许组播地址通信
-A rh-firewall-1-input-p vrrp-j ACCEPT #允许VRRP (Virtual Router Redundancy association) communication
-A rh-firewall-1-input-m state--state new-m tcp-p TCP--dport 80-j ACCEPT #允许80端口通过防火墙
: wq! #保存退出
/etc/init.d/iptables Restart #重启防火墙使配置生效
Second, install Haproxy
1. Create Haproxy run accounts and groups
Groupadd Haproxy #添加haproxy组
Useradd-g haproxy haproxy-s/bin/false #创建nginx运行账户haproxy并加入到haproxy组, do not allow haproxy users to log on directly to the system
2, install the compilation tool
Yum install gcc gcc-c++ make Openssl-devel kernel-devel
3. Installation Haproxy
Haproxy Download Address: http://haproxy.1wt.eu/download/1.4/src/haproxy-1.4.24.tar.gz
Upload haproxy-1.4.24.tar.gz to/usr/local/src directory
CD/USR/LOCAL/SRC #进入软件包存放目录
Tar zxvf haproxy-1.4.24.tar.gz #解压
CD haproxy-1.4.24 #进入安装目录
Make target=linux26 cpu=x86_64 prefix=/usr/local/haprpxy #编译
Make install Prefix=/usr/local/haproxy #安装
Parameter description:
Target=linux26 #使用uname-R to view the kernel, such as: 2.6.18-371.el5, this argument is linux26
Cpu=x86_64 #使用uname-R to view system information, such as x86_64 x86_64 x86_64 Gnu/linux, at which point this parameter is x86_64
Prefix=/usr/local/haprpxy #/usr/local/haprpxy for HAPRPXY installation path
4, set Haproxy
Mkdir-p/usr/local/haproxy/conf #创建配置文件目录
Mkdir-p/etc/haproxy #创建配置文件目录
Cp/usr/local/src/haproxy-1.4.24/examples/haproxy.cfg/usr/local/haproxy/conf/haproxy.cfg #拷贝配置模板文件
Ln-s/usr/local/haproxy/conf/haproxy.cfg/etc/haproxy/haproxy.cfg #添加配置文件软连接
Cp-r/usr/local/src/haproxy-1.4.24/examples/errorfiles/usr/local/haproxy/errorfiles #拷贝错误页面
Ln-s/usr/local/haproxy/errorfiles/etc/haproxy/errorfiles #添加软连接
Mkdir-p/usr/local/haproxy/log #创建日志文件目录
Touch/usr/local/haproxy/log/haproxy.log #创建日志文件
Ln-s/usr/local/haproxy/log/haproxy.log/var/log/haproxy.log #添加软连接
Cp/usr/local/src/haproxy-1.4.24/examples/haproxy.init/etc/rc.d/init.d/haproxy #拷贝开机启动文件
chmod +x/etc/rc.d/init.d/haproxy #添加脚本执行权限
Chkconfig Haproxy on #设置开机启动
Ln-s/usr/local/haproxy/sbin/haproxy/usr/sbin #添加软连接
5. Configure HAPROXY.CFG Parameters
Cp/usr/local/haproxy/conf/haproxy.cfg/usr/local/haproxy/conf/haproxy.cfg-bak #备份
Vi/usr/local/haproxy/conf/haproxy.cfg #编辑, modify
#####################################################################
# This config needs haproxy-1.1.28 or haproxy-1.2.1
Global
Log 127.0.0.1 local0 #在本机记录日志
Log 127.0.0.1 Local1 Notice
#log Loghost local0 Info
Maxconn 65535 #每个进程可用的最大连接数
Chroot/usr/local/haproxy #haproxy安装目录
UID #运行haproxy的用户uid ( cat/etc/passwd view)
GID #运行haproxy的组uid ( cat/etc/group view)
Daemon #以后台守护进程运行
Pidfile/usr/local/haproxy/haproxy.pid #将所有进程写入pid文件
#debug #调试模式
#quiet #安装模式
Defaults
#log Global
Log 127.0.0.1 Local3 #日志文件设置
Mode HTTP #运行模式tcp, HTTP, health
Option Httplog
Option Httpclose #每次请求完毕后主动关闭http通道
Option Dontlognull #不记录健康检查的日志信息
Option Redispatch #如果后端有服务器宕机, forcing a switch to a normal server
Option Abortonclose #丢弃由于客户端等待时间过长而关闭连接但仍在haproxy等待队列中的请求
Option Forwardfor #获取客户端真实访问IP
Option Originalto #记录客户端访问的目的IP
Maxconn 65535 #每个进程可用的最大连接数
Balance Source #同一IP地址的所有请求都发送到同一服务器
Retries 3 #三次连接失败, the service is not available
Contimeout 5000 #连接超时
Clitimeout 50000 #客户端超时
Srvtimeout 50000 #服务器超时
Timeout Check 5s #检测超时
Stats Refresh 30s #统计页面自动刷新时间
Stats Uri/haproxy-status #统计页面URL路径
Stats Realm Haproxy-status #统计页面输入密码框提示信息
Stats Auth admin:123456 #统计页面用户名和密码
Stats Hide-version #隐藏统计页面上HAProxy版本信息
Frontend Web #自定义描述信息
Bind:80 #监听80端口
ACL bbs.111cn.net HDR (host)-I bbs.111cn.net #规则设置, I followed by the domain name to be accessed, if access to the bbs.111cn.net domain, load balanced to bbs.111cn.net scope
Use_backend bbs.111cn.net if Bbs.111cn.net #acl和if后面的名称必须相同这里为bbs. 111cn.net
ACL sns.111cn.net HDR (host)-I sns.111cn.net #规则设置, I followed by the domain name to be accessed, if access to the sns.111cn.net domain, load balanced to sns.111cn.net scope
Use_backend sns.111cn.net If sns.111cn.net
Backend Bbs.111cn.net
Mode http
Balance Source
#option httpchk/index.php #检测服务器此文件是否存在, if not, the server connection exception is considered, this parameter can not set
Server 192.168.21.127 192.168.21.127:80 check Inter rise 3 Fall 3 weight #inter 2000 heartbeat detection time; rise 33 connection succeeded, the server is normal, Fall 33 times Connection failed, indicates server exception, weight 100 weight setting
Server 192.168.21.128 192.168.21.128:80 check Inter rise 3 Fall 3 Weight 100
Backend Sns.111cn.net
Mode http
Balance source #设置负载均衡模式, source saves the session value, Roundrobin polling mode
#option httpchk/index.php #检测服务器此文件是否存在, if not, the server connection exception is considered, this parameter can not set
Server 192.168.21.127 192.168.21.127:80 check Inter rise 3 Fall 3 Weight 100
Server 192.168.21.128 192.168.21.128:80 check Inter rise 3 Fall 3 Weight 100
#errorloc 503 http://www.111cn.net/404.html
ErrorFile 403/etc/haproxy/errorfiles/403.http
ErrorFile 500/etc/haproxy/errorfiles/500.http
ErrorFile 502/etc/haproxy/errorfiles/502.http
ErrorFile 503/etc/haproxy/errorfiles/503.http
ErrorFile 504/etc/haproxy/errorfiles/504.http
#####################################################################
: wq! #保存退出
Service Haproxy Start #启动
Service Haproxy Stop #关闭
Service Haproxy Restart #重启
6. Set up Haproxy log
Vi/etc/syslog.conf #编辑, increase at the bottom
# Haproxy.log
local0.*/var/log/haproxy.log
local3.*/var/log/haproxy.log
: wq! #保存退出
Vi/etc/sysconfig/syslog #编辑修改
Syslogd_options= "-r-m 0" #接收远程服务器日志
: wq! #保存退出
Service syslog Restart #重启syslog
Iii. installation of keepalived
Download keeplived:http://www.keepalived.org/software/keepalived-1.2.12.tar.gz
Upload keepalived-1.2.12.tar.gz to/usr/local/src directory
Cd/usr/local/src
Tar zxvf keepalived-1.2.12.tar.gz
CD keepalived-1.2.12
./configure #配置, you must see the following prompts that the configuration is correct to continue the installation
Use Ipvs Framework:yes
Ipvs Sync Daemon Support:yes
Use VRRP Framework:yes
Make #编辑
Make install #安装
cp/usr/local/etc/sysconfig/keepalived/etc/sysconfig/
Mkdir/etc/keepalived
cp/usr/local/etc/keepalived/keepalived.conf/etc/keepalived/
cp/usr/local/sbin/keepalived/usr/sbin/
cp/usr/local/etc/rc.d/init.d/keepalived/etc/rc.d/init.d/
chmod +x/etc/rc.d/init.d/keepalived #添加执行权限
Chkconfig keepalived on #设置开机启动
Service keepalived Start #启动
Service keepalived Stop #关闭
Service keepalived Restart #重启
Iv. Configuration of Keepalived
Cp/etc/keepalived/keepalived.conf/etc/keepalived/keepalived.conf-bak
Vi/etc/keepalived/keepalived.conf #编辑, modified to the following code
#########################################################
#以下为192.168.21.129 server:
! Configuration File for Keepalived
Global_defs {
Notification_email {
Acassen@firewall.loc
Failover@firewall.loc
Sysadmin@firewall.loc
}
Notification_email_from Alexandre.Cassen@firewall.loc
Smtp_server 192.168.200.1
Smtp_connect_timeout 30
router_id Lvs_devel
}
Vrrp_script Chk_haproxy {
Script "/etc/keepalived/check_haproxy.sh" #HAproxy服务监控脚本
Interval 2
Weight 2
}
Vrrp_instance Vi_1 {
State MASTER
Interface eth0
VIRTUAL_ROUTER_ID 51
Priority 100
Advert_int 1
Authentication {
Auth_type Pass
Auth_pass 1111
}
Track_script {
Chk_haproxy #监测haproxy进程状态
}
virtual_ipaddress {
192.168.21.253
}
Notify_master the ARP record of the "/etc/keepalived/clean_arp.sh 192.168.21.253" #更新虚拟服务器 (VIP) address to the gateway
}
Vrrp_instance Vi_2 {
State BACKUP
Interface eth0
VIRTUAL_ROUTER_ID 52
Priority 99
Advert_int 1
Authentication {
Auth_type Pass
Auth_pass 1111
}
virtual_ipaddress {
192.168.21.254
}
Notify_master the ARP record of the "/etc/keepalived/clean_arp.sh 192.168.21.254" #更新虚拟服务器 (VIP) address to the gateway
}
#########################################################
: wq! #保存退出
#########################################################
#以下为192.168.21.130 server:
192.168.21.130
! Configuration File for Keepalived
Global_defs {
Notification_email {
Acassen@firewall.loc
Failover@firewall.loc
Sysadmin@firewall.loc
}
Notification_email_from Alexandre.Cassen@firewall.loc
Smtp_server 192.168.200.1
Smtp_connect_timeout 30
router_id Lvs_devel
}
Vrrp_script Chk_haproxy {
Script "/etc/keepalived/check_haproxy.sh" #HAproxy服务监控脚本
Interval 2
Weight 2
}
Vrrp_instance Vi_1 {
State BACKUP
Interface eth0
VIRTUAL_ROUTER_ID 51
Priority 99
Advert_int 1
Authentication {
Auth_type Pass
Auth_pass 1111
}
Track_script {
Chk_haproxy #监测haproxy进程状态
}
virtual_ipaddress {
192.168.21.253
}
Notify_master the ARP record of the "/etc/keepalived/clean_arp.sh 192.168.21.253" #更新虚拟服务器 (VIP) address to the gateway
}
Vrrp_instance Vi_2 {
State MASTER
Interface eth0
VIRTUAL_ROUTER_ID 52
Priority 100
Advert_int 1
Authentication {
Auth_type Pass
Auth_pass 1111
}
virtual_ipaddress {
192.168.21.254
}
Notify_master the ARP record of the "/etc/keepalived/clean_arp.sh 192.168.21.254" #更新虚拟服务器 (VIP) address to the gateway
}
#########################################################
System Yun-wei Www.111cn.net warm reminder: qihang01 original content copyright, reproduced please indicate the source and the original link
: wq! #保存退出
V. Set up Haproxy Service monitoring script
Vi/etc/keepalived/check_haproxy.sh #编辑, add the following code
#########################################################
#!/bin/sh
If [$ (ps-c haproxy--no-header | wc-l)-EQ 0]; Then
/etc/rc.d/init.d/haproxy start
Fi
Sleep 2
If [$ (ps-c haproxy--no-header | wc-l)-EQ 0]; Then
/etc/rc.d/init.d/keepalived stop
Fi
#########################################################
: wq! #保存退出
chmod +x/etc/keepalived/check_haproxy.sh #添加执行权限
Vi. setting ARP records to update virtual server (VIP) addresses to the gateway script
Vi/etc/keepalived/clean_arp.sh #编辑, add the following code
#!/bin/sh
Vip=$1
gateway=192.168.21.2 #网关地址
/sbin/arping-i eth0-c 5-s $VIP $GATEWAY &>/dev/null
: wq! #保存退出
chmod +x/etc/keepalived/clean_arp.sh #添加脚本执行权限
Seven, System kernel optimization
Execute the following command on the two HAPROXY servers separately
echo 1024 60999 >/proc/sys/net/ipv4/ip_local_port_range
echo >/proc/sys/net/ipv4/tcp_fin_timeout
echo 4096 >/proc/sys/net/ipv4/tcp_max_syn_backlog
echo 262144 >/proc/sys/net/ipv4/tcp_max_tw_buckets
echo 262144 >/proc/sys/net/ipv4/tcp_max_orphans
echo >/proc/sys/net/ipv4/tcp_keepalive_time
Echo 1 >/proc/sys/net/ipv4/tcp_tw_recycle
echo 0 >/proc/sys/net/ipv4/tcp_timestamps
echo 0 >/PROC/SYS/NET/IPV4/TCP_ECN
Echo 1 >/proc/sys/net/ipv4/tcp_sack
echo 0 >/proc/sys/net/ipv4/tcp_dsack
Part II: Test haproxy+keepalived whether the normal operation
First, open the Haproxy monitoring page
Http://bbs.111cn.net/haproxy-status
Enter user name, password
Admin
123456
After you sign in, the following figure shows
Second, bbs.111cn.net parsing to 192.168.21.253;sns.111cn.net resolution to 192.168.21.254;
Executing commands on two haproxy servers: 192.168.21.129, 192.168.21.130: IP addr
As shown in the following illustration:
It can be seen now that vip:192.168.21.253 points to 192.168.21.129;vip:192.168.21.254 point 192.168.21.130;
Open in Browser
http://bbs.111cn.net/
http://sns.111cn.net/
As shown in the following illustration:
At this time, BBS and SNS domain name are balanced to 192.168.21.127 above
Third, stop the 192.168.21.127 above Nginx service
Service Nginx Stop
Continue to open the two URLs above, as shown in the following illustration:
At this time, BBS and SNS domain name are balanced to 192.168.21.128 above (because 192.168.21.127 server Nginx service is shut down, realize failover)
Four, close the 192.168.21.129 haproxy Service
Service Haproxy Stop
At this point, execute the command on two Haproxy servers: 192.168.21.129, 192.168.21.130: IP addr
As shown in the following illustration:
System Yun-wei Www.111cn.net warm reminder: qihang01 original content copyright, reproduced please indicate the source and the original link
It can be seen that vip:192.168.21.253 and 192.168.21.254 all point to 192.168.21.130;
At this point, open http://bbs.111cn.net/as shown in the following illustration:
can be accessed normally
V. Restore the Haproxy service above the 192.168.21.129 and restore the Nginx service above 192.168.21.127
Stop the keepalived service 192.168.21.130 above
Service keepalived Stop
Executing commands on two haproxy servers: 192.168.21.129, 192.168.21.130: IP addr
As shown in the following illustration:
It can be seen that vip:192.168.21.253 and 192.168.21.254 all point to 192.168.21.129;
At this point, open http://sns.111cn.net/as shown in the following illustration:
can be accessed normally
Note:
To view the Haproxy log file:
Tail-f/var/log/haproxy.log

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.