Load balancing using haproxy + keepalived

Software load balancing is generally implemented in two ways: software load implementation based on the operating system and software load implementation based on third-party applications. LVS is a kind of soft load based on the Linux operating system. HAProxy is an open-source software load based on the third application. Nginx can also be used, but nginx only works on Layer 7 networks. Reference Software for details Server load balancerGenerally, two methods are used for implementation: software load implementation based on the operating system and software load implementation based on third-party applications. LVS is a kind of soft load based on the Linux operating system. HAProxy is an open-source software load based on the third application. You can also use NginxBut NginxIt only works on a 7-layer network. For more information, see "software-level Server load balancer"This article is very simple, detailed, and very good.

Compared with LVS, HAProxy is much easier to use and has rich functions. Currently, HAProxy supports two main proxy modes: "tcp", that is, layer-4 (mostly used for mail servers, internal protocol communication servers, etc.), and layer-7 (HTTP ). In layer-4 mode, HAProxy only forwards two-way traffic between the client and the server. In layer-7 mode, HAProxy analyzes the protocol and controls the protocol by allowing, rejecting, exchanging, adding, modifying, or deleting specified content in a request or response, this operation is based on specific rules.

I use HAProxy mainly because it has the following advantages:
I. free open-source, stability is also very good, this can be seen through some of my small projects, single Haproxy also runs well, stability can be comparable to LVS;
2. according to the official documents, HAProxy can run full 10Gbps-New benchmark of HAProxy at 10 Gbpsusing Myricom's 10GbE memory (Myri-10GPCI-Express), which is also amazing as a software-level load balancing;
3. HAProxy can be used as MySQL, mail, or other non-web load balancing. we often use HAProxy as MySQL (read) load balancing;
4. a page with powerful monitoring server status is provided. in the actual environment, we use Nagios for email or SMS alerts. this is one of the reasons why I like it very much;
5. HAProxy supports virtual hosts.

1. Experiment environment
System version: CentOS release 5.9 (Final) x86 32-bit
HAProxy version: 1.4.8
Keepalived version: 1.2.4
MASTER_IP: 192.168.207.130
BACKUP_IP: 192.168.207.131
VIP: 192.168.207.141
WEB_1: 192.168.207.129 port 80
WEB_2: 192.168.207.130 port 8080
WEB_3: 192.168.207.131 port 8080
System version: CentOS release 5.9 (Final) x86 32-bit
HAProxy version: 1.4.8
Keepalived version: 1.2.4
 
MASTER_IP: 192.168.207.130
BACKUP_IP: 192.168.207.131
VIP: 192.168.207.141
WEB_1: 192.168.207.129 port 80
WEB_2: 192.168.207.130 port 8080
WEB_3: 192.168.207.131 port 8080
IptablesAnd selinux are both disabled

2. install Haproxy
Cd/usr/local/src
Mkdir haproxy
Cd haproxy
Wgethttp: // haproxy.1wt. eu/download/1.4/src/haproxy-1.4.8.tar.gz
Tar xf haproxy-1.4.8.tar.gz
Cd haproxy-1.4.8
Make TARGET = linux26; make install
# Here you can also write make TARGET = linux26 -- prefix =/usr/local/haproxy to add a path for haproxy installation.
Cd/usr/local/src
Mkdir haproxy
Cd haproxy
Wget tarxf haproxy-1.4.8.tar.gz
Cd haproxy-1.4.8
Make TARGET = linux26; make install
# Here you can also write make TARGET = linux26 -- prefix =/usr/local/haproxy to add a path for haproxy installation.
Write the haproxy Configuration File. the master and slave files are the same. use this configuration file [plain] view plaincopyprint? Useraddhaproxy
Vi/etc/haproxy. cfg
Useradd haproxy
Vi/etc/haproxy. cfg

The content is as follows:
Global
Log 127.0.0.1 local0 # This is the log
Maxconn 65535
User haproxy
Group haproxy
Daemon # enable later processes
Nbproc8 # Number of haproxy processes
Pidfile/tmp/haproxy. pid
ULTS
Log 127.0.0.1 local3
Mode http
Option httplog
Option httpclose
Option dontlognull
Option forwardfor
Option redispatch
Retries 2
Maxconn 65535
Balanceroundrobin # in order to better observe the results, here we use roundrobin for simple round training and the source algorithm.
Stats uri/web-status # This is the managed url. After configuration, you only need to enter http: // ip or VIP/web-status to access it.
Contimeout 5000
Clitimeout 50000
Srvtimeout 50000
Listen web_vip0.0.0.0: 80 # do not bind the VIP address here. the keepalived master server is also available, but the backup server cannot, because only when keepalived is the master, the VIP will be set on this server. so, use 0.0.0.0.
Mode http
Option httplog
Log global
Option httpchk HEAD/index.html HTTP/1.0 then you have to create an index.html page on the three webservers. you can add different content to the index.html page for testing. the content should be set to 129: 80,130: 8080,131: 8080. if not, haproxy determines that the corresponding web server has crashed.
Server web1 192.168.207.129: 80 weight 5 check inter 2000 rise 2 fall 3
Server web2 192.168.207.130: 8080 weight 5 check inter2000 rise 2 fall 3
Server web3 192.168.207.131: 8080 weight 5 check inter2000 rise 2 fall 3 global
Log 127.0.0.1 local0 # This is the log
Maxconn 65535
User haproxy
Group haproxy
Daemon # enable later processes
Nbproc8 # Number of haproxy processes
Pidfile/tmp/haproxy. pid
ULTS
Log 127.0.0.1 local3
Mode http
Option httplog
Option httpclose
Option dontlognull
Option forwardfor
Option redispatch
Retries 2
Maxconn 65535
Balanceroundrobin # in order to better observe the results, here we use roundrobin for simple round training and the source algorithm.
Stats uri/web-status # This is the managed url. After configuration, you only need to enter http: // ip or VIP/web-status to access it.
Contimeout 5000
Clitimeout 50000
Srvtimeout 50000
Listen web_vip0.0.0.0: 80 # do not bind the VIP address here. the keepalived master server is also available, but the backup server cannot, because only when keepalived is the master, the VIP will be set on this server. so, use 0.0.0.0.
Mode http
Option httplog
Log global
Option httpchk HEAD/index.html HTTP/1.0 then you have to create an index.html page on the three webservers. you can add different content to the index.html page for testing. the content should be set to 129: 80,130: 8080,131: 8080. if not, haproxy determines that the corresponding web server has crashed.
Server web1 192.168.207.129: 80 weight 5 check inter 2000 rise 2 fall 3
Server web2 192.168.207.130: 8080 weight 5 check inter2000 rise 2 fall 3
Server web3 192.168.207.131: 8080 weight 5 check inter2000 rise 2 fall 3
If you do not understand the configuration parameters of haproxy, you can search for the keywords to explain them. it is very convenient, but it is in English. well, you can do it yourself.
Here, we will also talk about more and more HAProxy algorithms, including the following eight types:
① Roundrobin indicates simple round robin. this is basically what server load balancer has;
② Static-rr, indicating that according to the weight, we recommend that you pay attention to it;
③ Leastconn, which indicates that the minimum number of connectors is handled first. we recommend that you pay attention to it;
④ Source indicates the Request source IP address. This is similar to the IP_hash mechanism of Nginx. We use it as a solution to the session problem. we recommend that you pay attention to it;
⑤ Ri, indicating the URI based on the request;
⑥ Rl_param: indicates the request URl parameter 'balance url_param' requires an URL parametername;
7. hdr (name) indicates locking each HTTP request based on the HTTP request header;
Extends rdp-cookie (name) indicates that each TCP request is locked and hashed based on the cookie (name.
Run
Haproxy-f/etc/haproxy. cfg haproxy-f/etc/haproxy. cfg
This is a later load balancing mode with haproxy + 3 web
You can enter the http: // haproxy_ip address in the browser to see how haproxy runs.
Add/web-status to the url to access the management interface.

3. install keepalived
Wgethttp: // www.keepalived.org/software/keepalived-1.2.4.tar.gz
Tar-zxvf keepalived-1.2.4.tar.gz
Cd keepalived-1.2.4
./Configure -- prefix =/usr/local/keepalived
Make & make install
Cp/usr/local/keepalived/etc/rc. d/init. d/keepalived/etc/init. d/keepalived
Cp/usr/local/keepalived/sbin/keepalived/usr/sbin/
Cp/usr/local/keepalived/etc/sysconfig/
Mkdir-p/etc/keepalived/
Cp/usr/local/keepalived/etc/keepalived. conf/etc/keepalived. conf
Chmod + x/etc/init. d/keepalived
Wgethttp: // www.keepalived.org/software/keepalived-1.2.4.tar.gz
Tar-zxvf keepalived-1.2.4.tar.gz
Cd keepalived-1.2.4
./Configure -- prefix =/usr/local/keepalived
Make & make install
Cp/usr/local/keepalived/etc/rc. d/init. d/keepalived/etc/init. d/keepalived
Cp/usr/local/keepalived/sbin/keepalived/usr/sbin/
Cp/usr/local/keepalived/etc/sysconfig/
Mkdir-p/etc/keepalived/
Cp/usr/local/keepalived/etc/keepalived. conf/etc/keepalived. conf
Chmod + x/etc/init. d/keepalived

Write the configuration file of keepalived
I/etc/keepalived. conf vi/etc/keepalived. conf

The content is as follows:
# Configuration File for keepalived
Global_defs {
Router_id LVS_DEVEL
}
Vrrp_script chk_http_port {
Script "/etc/keepalived/checkHaproxy. sh"
Interval 2
Weight 2
}
Vrrp_instance VI_1 {
StateMASTER # Change from keepalived to BACKUP
Interface eth0
Virtual_router_id 51
Priority104 # Change from keepalived to 100, as long as it is smaller than the master, and the number is from 0 ~ 255. the greater the number, the greater the weight.
Advert_int 1
Authentication {
Auth_type PASS
Auth_pass 1111
}
Track_script {
Chk_http_port
}
Virtual_ipaddress {
192.168.207.141
}
} # Configuration File for keepalived
Global_defs {
Router_id LVS_DEVEL
}
Vrrp_script chk_http_port {
Script "/etc/keepalived/checkHaproxy. sh"
Interval 2
Weight 2
} Vrrp_instance VI_1 {
StateMASTER # Change from keepalived to BACKUP
Interface eth0
Virtual_router_id 51
Priority104 # Change from keepalived to 100, as long as it is smaller than the master, and the number is from 0 ~ 255. the greater the number, the greater the weight.
Advert_int 1
Authentication {
Auth_type PASS
Auth_pass 1111
}
Track_script {
Chk_http_port
}
Virtual_ipaddress {
192.168.207.141
}
}


Notes:
1. you must add executable permissions to/etc/keepalived/checkHaproxy. sh, such as chmod + x/etc/keepalived/checkHaproxy. sh.

2. write the configuration file in regular format, braces {}. leave a space with the previous keyword.
Write the/etc/keepalived/checkHaproxy. sh script as follows:
#! /Bin/bash
A = 'PS-C haproxy -- no-header | wc-L'
If [$ A-eq 0]; then
/Usr/local/sbin/haproxy-f/etc/haproxy. cfg
Echo "Haproxystart"
Sleep 3
If ['PS-C haproxy -- no-header | wc-L'-eq 0]; then
/Etc/init. d/keepalived stop
Echo "keepalived stop"
Fi
Fi
#! /Bin/bash
A = 'PS-C haproxy -- no-header | wc-L'
If [$ A-eq 0]; then
/Usr/local/sbin/haproxy-f/etc/haproxy. cfg
Echo "Haproxy start"
Sleep 3
If ['PS-C haproxy -- no-header | wc-L'-eq 0]; then
/Etc/init. d/keepalived stop
Echo "keepalived stop"
Fi
Fi
This means to first check whether haproxy is started. if the haproxy process is not started first, stop the keepalived process if it cannot be started. then, the main process will be programmed from keepalived to continue working.
This script can be written by yourself. it is quite simple.

4. open the service and verify that the three web services are opened
Enable the later two master-slave keepalived servers, and keepalived will automatically open the haproxy service because of/etc/keepalived/checkHaproxy. sh, haha

Therefore, after the service is enabled, it can run on the master keepalived.
Ip addr
We will see that eth0 has an VIP virtual address. to understand the role of keepalived, we should first turn off the keepalived of the master server.
/Etc/init. d/keepalived stop/etc/init. d/keepalived stop

View the/var/log/messages log of keepalived.
Apr 18 12:20:23 localhost Keepalived_vrrp: VRRP_Instance (VI_1) Transition to MASTER STATE
Apr 18 12:20:24 localhost Keepalived_vrrp: VRRP_Instance (VI_1) Entering MASTER STATE
Apr 18 12:20:24 localhost Keepalived_vrrp: VRRP_Instance (VI_1) setting protocol VIPs.
Apr 18 12:20:24 localhost Keepalived_vrrp: VRRP_Instance (VI_1) Sending gratuitous ARPs on eth0 for 192.168.207.141
Apr 18 12:20:24 localhost Keepalived_vrrp: Netlink reflectorreports IP 192.168.207.141 added
Apr 18 12:20:24 localhost Keepalived_healthcheckers: Netlinkreflector reports IP 192.168.207.141 added
Apr 18 12:20:24 localhost avahi-daemon [4208]: Registering newaddress record for 192.168.207.141 on eth0.
Apr 18 12:20:23 localhost Keepalived_vrrp: VRRP_Instance (VI_1) Transition to MASTER STATE
Apr 18 12:20:24 localhost Keepalived_vrrp: VRRP_Instance (VI_1) Entering MASTER STATE
Apr 18 12:20:24 localhost Keepalived_vrrp: VRRP_Instance (VI_1) setting protocol VIPs.
Apr 18 12:20:24 localhost Keepalived_vrrp: VRRP_Instance (VI_1) Sending gratuitous ARPs on eth0 for 192.168.207.141
Apr 18 12:20:24 localhost Keepalived_vrrp: Netlink reflectorreports IP 192.168.207.141 added
Apr 18 12:20:24 localhost Keepalived_healthcheckers: Netlinkreflector reports IP 192.168.207.141 added
Apr 18 12:20:24 localhost avahi-daemon [4208]: Registering newaddress record for 192.168.207.141 on eth0.

Open the master keepalived service and you will see
Apr 18 12:20:29 localhost Keepalived_vrrp: VRRP_Instance (VI_1) Sending gratuitous ARPs on eth0 for 192.168.207.141
Apr 18 12:21:07 localhost Keepalived_vrrp: VRRP_Instance (VI_1) stored Ed higher prio advert
Apr 18 12:21:07 localhost Keepalived_vrrp: VRRP_Instance (VI_1) Entering BACKUP STATE
Apr 18 12:21:07 localhost Keepalived_vrrp: VRRP_Instance (VI_1) removing protocol VIPs.
Apr 18 12:21:07 localhost Keepalived_vrrp: Netlink reflectorreports IP 192.168.207.141 removed
Apr 18 12:21:07 localhost Keepalived_healthcheckers: Netlinkreflector reports IP 192.168.207.141 removed
Apr 18 12:21:07 localhost avahi-daemon [4208]: Withdrawing addressrecord for 192.168.207.141 on eth0.