Introduction of TENS concurrent haproxy balanced load system

Introduction and orientation of Haproxy

Haproxy provides high availability , load Balancing , and proxies based on TCP and HTTP applications to support virtual hosts , a free, fast, and reliable solution. According to official data, its maximum limit supports 10G concurrency.

Haproxy is especially useful for Web sites that are heavily loaded, which typically require session maintenance or seven-tier processing. Haproxy is running on the current hardware and can support tens of thousands of concurrent connections altogether. and its operating mode makes it easy and secure to integrate into your current architecture while protecting your Web server from being exposed to the network.

It supports network switching from layer 4 to 7, covering all TCP protocols. That is, Haproxy even supports Mysql's balanced load.

If in the function, can use proxy reverse proxy way realizes the Web balanced load, such product has many. including Nginx,apacheproxy,lighttpd,cheroke and so on.

But to be clear, Haproxy is not an Http server. All of the above mentioned products with reverse agent balance load are exclusively WEB servers. Simply put, they can provide static (Html,jpg,gif ...). or dynamic (php,cgi ...) File transfer and processing. And Haproxy is only, and is specialized in a balanced load of the application agent. It itself does not provide HTTP services.

But its configuration is simple, has the very good server health Check function also has the special system condition monitoring page, when its proxy backend server fails, Haproxy will automatically remove the server, and then automatically join the server after the recovery. Since version 1.3, Frontend,backend,frontend has also been introduced to make rule matching based on the contents of any HTTP request headers and then directed the request to the relevant backend.

In addition, version 1.3 is a version of the active development phase that supports the following new features:

L Content Exchange: You can select a set of servers based on any part of the request (request), such as the requested URI, host header (header), Cookie, and everything else. Of course, there are more requirements for this feature for statically detached sites.

Transparent Agent: You can connect back-end servers with either the client IP address or any other address. This feature is only available when the Linux 2.4/2.6 kernel has cttproxy patches. This feature also makes it possible for a particular server to process some of the traffic without modifying the server's address.

L Faster Scheduler based on tree: 1.2.16 The above version requires that all timeouts be set to the same value to support tens of thousands of full speed connections. This feature has been ported to 1.2.17.

L Kernel TCP stitching: avoids the kernel to the user and then the user to the kernel side of the data copy, improve throughput while reducing the CPU utilization rate. Haproxy 1.3 supports Linux L7SW to meet the demand for several Gbps throughput on commercial hardware.

L Connection rejection: because the overhead of maintaining a connection is very low, sometimes we need to limit the attack worm (attack bots), which means restricting their connections to open and limiting their damage. This has been developed for a small DDoS attack site and has saved a lot of sites.

Small head processing: it makes it simpler to write rules based on headers, and can handle portions of a URI.

• Fast and reliable head handling: Full RFC2616 compliant integrity checks are used to parse and index a general request for just under 2ms of time.

Modular design: Allow more people to join this project, debugging is also very simple. Poller have been separated and have made their development a lot simpler. HTTP has been detached from TCP, which makes it very easy to add new seven-layer features. Other subsystems will soon be modular

L Speculative I/O processing: Attempts to read data from it before a socket is ready. Poller only guesses which one may be ready and which is not, try to guess, and if successful, some expensive system calls can be omitted. If it fails, these system calls are invoked. The known use of Linux Epoll () has been lifted at least 10% of the net.

L ACLs: Use any combination of arbitrary rules as an execution condition for an action.

L TCP protocol Check: The ACL is combined to check any part of the request before forwarding. This allows you to perform protocol validation instead of blindly forwarding. For example, allow SSL but deny SSH.

L More load balancing algorithms: Now, Dynamic weighted rotation (dynamical Round Robin), weighted source address hash (weighted source hash), weighted URL hash and weighted parameter hash (weighted Parameter hash) has been implemented. Other algorithms such as weighted measured Response time will also be implemented soon.

Installation and Configuration

The Haproxy configuration is fairly simple,

Download the latest version from the official website: http://www.haproxy.org.

# wget http://haproxy.1wt.eu/download/1.3/src/haproxy-1.3.20.tar.gz

# tar ZCVF haproxy-1.3.20.tar.gz

# CD haproxy-1.3.20

# Make Target=linux26 Prefix=/usr/local/haprpxy

# Make Install Prefix=/usr/local/haproxy

After installation, enter the installation directory to create the configuration file

# Cd/usr/local/haproxy

# VI Haproxy.cfg

configuration content is as follows:

Global
Log 127.0.0.1 Local0
#log 127.0.0.1 Local1 Notice
#log Loghost local0 Info
Maxconn 4096
Chroot/usr/local/haproxy
UID #所属运行的用户uid
GID #所属运行的用户组
Daemon
Nbproc 1
Pidfile/usr/local/haproxy/run/haproxy.pid
#debug
#quiet

Defaults
Log Global
Log 127.0.0.1 Local3 #日志文件的输出定向
Mode HTTP #所处理的类别
Option Httplog #日志类别
Option Httpclose
Option Dontlognull
Option Forwardfor
Option Redispatch
Retries 2 #设置多个haproxy并发进程提高性能
Maxconn 2000
Balance Roundrobin #负载均衡算法
Stats Uri/haproxy-stats #haproxy The access address of the monitor page
# accessible via Http://localhost:1080/haproxy-stats
Contimeout 5000
Clitimeout 50000
Srvtimeout 50000

Listen localhost 0.0.0.0:1080 #运行的端口及主机名
Mode http
Option Httpchk get/index.htm #健康检测
Server S1 127.0.0.1:3121 weight 3 check #后端的主机 IP & Tradeoff
Server s2 127.0.0.1:3122 weight 3 check #后端的主机 IP & Tradeoff

To start the service:
#/usr/local/haproxy/sbin/haproxy-f/usr/local/haproxy/haproxy.cfg

Restart Service:
#/usr/local/haproxy/sbin/haproxy-f/usr/local/haproxy/haproxy.cfg-st ' Cat/usr/local/haproxy/logs/haproxy.pid ' ( No Line wrap)

Stop service:
# Killall Haproxy

Of course, to facilitate the system to load at boot time, you can also create a startup script:
# Vim/etc/rc.d/init.d/haproxy contents are as follows:

#! /bin/sh
Set-e

Path=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/haproxy/sbin
Progdir=/usr/local/haproxy
Progname=haproxy
daemon= $PROGDIR/sbin/$PROGNAME
config= $PROGDIR/conf/$PROGNAME. conf
pidfile= $PROGDIR/run/$PROGNAME. pid
desc= "Haproxy Daemon"
Scriptname=/etc/init.d/$PROGNAME

# gracefully Exit if the package has been removed.
Test-x $DAEMON | | Exit 0

Start ()
{
Echo-n "Starting $DESC: $PROGNAME"
$DAEMON-F $CONFIG
echo "."
}

Stop ()
{
Echo-n "Stopping $DESC: $PROGNAME"
Haproxy_pid=cat $PIDFILE
Kill $haproxy _pid
echo "."
}

Restart ()
{
Echo-n "Restarting $DESC: $PROGNAME"
$DAEMON-F $CONFIG-P $PIDFILE-sf $ (cat $PIDFILE)
echo "."
}

Case "$" in
Start
Start
;;
Stop
Stop
;;
Restart)
Restart
;;
*)
echo "Usage: $SCRIPTNAME {start|stop|restart}" >&2
Exit 1
;;
Esac
Exit 0

Grant executable permission after saving
# chmod +x/etc/rc.d/init.d/haproxy

You can use the service Haproxy Start|stop|restart to control the startup stop and restart of services.
and load the boot service startup list with the following command
# chkconfig--add Haproxy

Configuration log:
# vim/etc/syslog.conf

Increase at the bottom
local3.*/var/log/haproxy.log
local0.*/var/log/haproxy.log

Reboot core Log service to make configuration effective
# Service Syslog Restart

And then you can view the logs.
# Tail–f/var/log/harpoxy.log

Aug 15:32:06 localhost haproxy[64136]: Proxy www started.
Aug 15:32:06 localhost haproxy[64136]: Proxy Cherokee started.
Aug 15:32:06 localhost haproxy[64136]: Proxy WAP started.
Aug 15:32:06 localhost haproxy[64136]: Proxy pic started.
Aug 15:32:06 localhost haproxy[64136]: Proxy img started.
Aug 15:32:06 localhost haproxy[64136]: Proxy public started.
Aug 15:32:06 localhost haproxy[64136]: Proxy public started.
Aug 15:32:59 localhost haproxy[64137]: 219.142.128.30:6416 [22/aug/2009:15:32:59.754] public stats/<stats> 0/- 1/-1/-1/0 17329--pr--0/0/0/0/0 0/0 "Get/?stats http/1.1"
Aug 15:32:59 localhost haproxy[64137]: 219.142.128.30:6416 [22/aug/2009:15:32:59.754] public stats/<stats> 0/- 1/-1/-1/0 17329--pr--0/0/0/0/0 0/0 "Get/?stats http/1.1"

Application Examples

WEB Balancing Load & virtual host

Re-open the profile haproxy.cfg and pay attention to the balanced host option in the bottom section
Listen localhost 0.0.0.0:1080 #运行的端口及主机名
Mode http
Option Httpchk get/index.htm #用于健康检测的后端页面
Server S1 127.0.0.1:3121 weight 3 check #后端的主机 IP & Tradeoff
Server s2 127.0.0.1:3122 weight 3 check #后端的主机 IP & Tradeoff

In the experiment, our back end was squid separated by 2 ports on the same server.
Take one of the examples:

Server S1 127.0.0.1:3121 weight 3 Check

S1 is a server alias that you can define yourself
127.0.0.1:3121 the IP address and port number of the server
Weight 3 can be allocated to the request of the high and low, the higher the number of requests allocated to the greater the number of
Check accepts a timed check of the haproxy to determine the health of the backend server.

If you need to configure a virtual host, quite simple, tight need to modify localhost for your virtual host domain name, add to the Haproxy configuration, and then assign the backend server parameters.

Cases:

Listen www.x1.com 0.0.0.0:1080 #运行的端口及主机名
Mode http
Option Httpchk get/index.htm #用于健康检测的后端页面
Server S1 127.0.0.1:3121 weight 3 check #后端的主机 IP & Tradeoff
Server s2 127.0.0.1:3122 weight 3 check #后端的主机 IP & Tradeoff

Listen www.x2.com 0.0.0.0:1080 #运行的端口及主机名
Mode http
Option Httpchk get/index.htm #用于健康检测的后端页面
Server S1 127.0.0.1:3121 weight 3 check #后端的主机 IP & Tradeoff
Server s2 127.0.0.1:3122 weight 3 check #后端的主机 IP & Tradeoff

After you save the configuration, reload it, you can take effect, refresh the Administration page and see the new virtual host. Performance Comparison

Here, we use the most fiery HTTP and front-end Web balanced load server Nginx and Haproxy to do a simple performance comparison.

Test environment:

cpu:xeon2.8g X2
ram:4g
Os:redhat As5.3 X64

Tools: Apache AB
Parameters: Ab-i-C 500-n 100000 (500 concurrent, 1W requests)
End Service: 2 squid need to achieve a balanced load

The results are as follows:

####### Nginx + haproxy: (Sent by the Nginx through the reverse proxy to haproxy, and balanced load)

Concurrency level:500
Time taken for tests:53.758 seconds
Complete requests:100000
Failed requests:0
Write errors:0
Total transferred:38600386 bytes
HTML transferred:0 bytes

Requests per second:1860.19 [#/sec] (mean)
Time per request:268.790 [MS] (mean)
Time/request:0.538 [MS] (mean, across all concurrent requests)
Transfer rate:701.21 [Kbytes/sec] Received
####### haproxy: (Balanced load by haproxy alone)
Concurrency level:500
Time taken for tests:32.562 seconds
Complete requests:100000
Failed requests:0
Write errors:0
Total transferred:36606588 bytes
HTML transferred:0 bytes
Requests per second:3071.02 [#/sec] (mean)
Time per request:162.812 [MS] (mean)
Time/request:0.326 [MS] (mean, across all concurrent requests)
Transfer rate:1097.85 [Kbytes/sec] Received
####### nginx: (Balanced load by nginx alone)
Concurrency level:500
Time taken for tests:36.539 seconds
Complete requests:100000
Failed requests:0
Write errors:0
Total transferred:38600000 bytes
HTML transferred:0 bytes
Requests per second:2736.82 [#/sec] (mean)
Time per request:182.694 [MS] (mean)
Time/request:0.365 [MS] (mean, across all concurrent requests)
Transfer rate:1031.65 [Kbytes/sec] Received

Repeated tests to produce the results:
Haproxy the performance of the balance load alone is the strongest, exceeding the nginx.
However, the Nginx + haproxy performance is the weakest, it should be with the adoption of a 2-layer reverse proxy.
Therefore, the use of Haproxy to replace the Nginx with the balanced load function will give performance discounts.
But although so haproxy to the balanced load function is far more mature than Nginx, such as session pasting, cookie guidance, etc. are not Nginx.
can choose to match according to the need.
Introduction to related startup parametersIntroduction to related startup Parameters

#./haproxy–help//haproxy Related Command parameter introduction.

Haproxy-f < configuration files >

[-N Maximum concurrent connections total] [Ñ The maximum number of concurrent per listener] [d] [d] [Q] [-V] [-c] [-P <pid file] [-S] [-l] [-DK]

[-ds] [-de] [-DP] [-db] [M < memory limit M>] [{-sf|-st} pidlist ...]

-D foreground, debug mode

-D daemon Mode boot

-Q Quiet mode, do not output information

-V Verbose mode

-C syntax checking of configuration files

-s display statistic data

-L Display Detailed statistics

-DK does not use Kqueue

-ds does not use speculative Epoll

-de does not use Epoll

-DP does not use poll

-DB Disables background mode, the program runs in the foreground

-SF <pidlist>

After the program is started, send the finish signal to the process in the pidlist, which is placed at the end of the command line

-st <pidlist>

When the program starts, it sends a terminate signal to the process in the pidlist, which is placed at the end of the command line resource (resources)

This article only as a primer, haproxy configuration with its function far more than these. More information can be obtained from the following websites.

· Haproxy Chinese http://cn.haproxy.org

· Haproxy English http://www.haproxy.org

· China Open Source Community http://www.oschina.net

Reprinted from: Open Source China Community (http://www.oschina.net)