Ubuntu + haproxy + keepalived master Load

Ubuntu + haproxy + keepalived master load 1. test environment: System: ubuntu server 12.04 haproxy version: 1.4.24 keepalived version: keepalived-1.2.7 haporxy01: eth0: appseth1: 192.168.100.36 haporxy02: eth0: 172.16.1.37 eth1: 192.168.100.37 vip1: 172.16.1.30 2: 172.16.1.31 nginx x1website: nginx1 nginx2 website: ningx2 nginx3 website: nginx3 nginx4 website: nginx4 2. network Structure user | (vip1) | (vip2) haproxy01 ------- keepalived ------- hapro Xy02/\ ---------------------------------------------------------- | nginx1 nginx2 nginx3 nginx4 | Release 3. installation 1. install keepalvied host haproxy01: wget http://www.keepalived.org/software/keepalived-1.2.7.tar.gz tar xf keepalived-1.2.7.tar.gz cd keepalived-1.2.7. /configure -- prefix =/usr/local/prompt: checking for openssl/ssl. h... no c Onfigure: error :!!! OpenSSL is not properly installed on your system .!!! !!! Can not include OpenSSL headers files .!!! Install libssl. dev apt-get install libssl. dev continue :. /configure -- prefix =/usr/local/prompt: checking for poptGetContext in-lpopt... no configure: error: Popt libraries is required install libpopt-dev apt-get install libpopt-dev continue :. /configure -- prefix =/usr/local/make install 2. edit keepalived. conf mkdir/etc/keepalived/vi/etc/keepalived. conf ######## keepalived. conf ######## global_defs {router_ I D LVS_DEVEL} vrrp_instance VI_1 {state MASTER # haproxy02: BACKUP interface eth0 virtual_router_id 51 priority 91 # advert_int 1 authentication {auth_type PASS auth_pass 123456789} virtual_ipaddress {172.16.1.30} vrrp_instance VI_2 {state BACKUP # haproxy02: MASTER interface eth0 virtual_router_id 52 priority 90 # smaller than haproxy02 advert_int 1 authentication {auth_type PASS auth_pass 1 23456789} virtual_ipaddress {172.16.1.31 }######## keepalived. conf ######## 3. start keepalived/usr/local/sbin/keepalvied-f/etc/keepalived. conf 4. view the vip ip addr: eth0: inet 172.16.1.30/32 scope global eth0 inet 172.16.1.31/32 scope global eth0 # (keepalived of haproxy02 is not started, vip2 is in haproxy01, after keepalived in haproxy02 is started, vip2 automatically jumps to eth0 in haproxy02.) The installation of the host haproxy02: keepalived is the same as that of keepalived. conf :##### ### Keepalived. conf ######## global_defs {router_id LVS_DEVEL} vrrp_instance VI_1 {state BACKUP # haproxy01: MASTER interface eth0 virtual_router_id 51 priority 90 # smaller than haproxy01 advert_int 1 authentication {auth_type PASS auth_pass 123456789} virtual_ipaddress {172.16.1.30} vrrp_instance VI_2 {state MASTER # haproxy01: BACKUP interface eth0 virtual_router_id 52 priority 91 # greater than haproxy01 advert _ Int 1 authentication {auth_type PASS auth_pass 123456789} virtual_ipaddress {172.16.1.31 }######## keepalived. conf ######## start keepalived/usr/local/sbin/keepalvied-f/etc/keepalived. conf view vip: ip addr display: eth0: inet 172.16.1.31/32 scope global eth0 #(vip2 disappears on eth0 in haproxy, only vip1) stop any keepalived service, the other one can automatically generate vip1 and vip2 to ensure the high availability of the two haproxy servers. install the haproxy wget http://haproxy.1wt.eu/download/1. 4/src/haproxy-1.4.24.tar.gz tar xf haproxy-1.4.24.tar.gz cd haproxy-1.4.24 view ubuntu version: uname-a display: linux ubuntu37 3.2.0-51-generic # 77-Ubuntu SMP Wed Jul 24 20:18:19 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux make TARGET = 37-ubuntu make PREFIX =/usr/local/haproxy install create haproxy user useradd haproxy 2. edit haproxy. cfg mkdir/etc/haproxy vi/etc/haproxy. cfg ######## haproxy. cfg ######## global log 12 7.0.0.1 local0 maxconn 51200 user haproxy group haproxy daemon defaults log limit local3 mode http option dontlognull balance limit retries 2 option redispatch option abortonclose maxconn 51200 contimeout 5000 clitimeout 50000 limit 50000 listen limit bind limit: 80 mode http option httpclose option forceclose option forwardfor option originalto option redispatch balance lea Stconn timeout check 5S stats uri/stats # haproxy load monitoring page example: http: // 172.16.1.30/stats refresh 15 s # monitoring page refresh time stats realm baison-test-Haproxy stats auth admin: admin # account password stats hide-version # hide haproxy version information cookie SESSION_COOKIE insert indirect nocache maxconn 40960 server nginx1 192.168.100.11: 80 weight 1 cookie nginx1 check inter 2000 rise 2 fall 3 server nginx2 192.168.100.12: 80 weight 1 Cookie nginx2 check inter 2000 rise 2 fall 3 listen reply bind failed: 80 mode http option reply option forwardfor option originalto option redispatch balance resume cookie SESSION_COOKIE insert indirect nocache maxconn 40960 server nginx3 192.168.100.13: 80 weight 1 cookie nginx3 check inter 2000 rise 2 fall 3 server nginx4 192.168.100.14: 80 weight 1 cookie nginx4 Check inter 2000 rise 2 fall 3 ####### haproxy. cfg ######## start haproxy:/usr/local/haproxy/sbin/haproxy-f/etc/haproxy. cfg host haproxy02: the installation is the same as above, haproxy. cfg starts haproxy as above:/usr/local/haproxy/sbin/haproxy-f/etc/haproxy. cfg 4: access through a browser: http: // 172.16.1.30 page: nginx1, nginx2 round robin switch http: // 172.16.1.31 page: for nginx3 and nginx4 round-robin switching, you can view the load on the monitoring page: http: // 172.16.1.30/stats to stop any haproxy service. The site is not affected. 5: haproxy day Zhi 1. in/etc/rsyslog. in conf: add the haproxy log Path: local0. */var/log/haproxy. log local3. */var/log/haproxy. log cancel Note: # $ ModLoad imudp = "$ ModLoad imudp # $ UDPServerRun 514 =" $ UDPServerRun 514 restart rsyslog service rsyslog restart 2. check that the services file grep 514/etc/services displays: syslog 514/udp is OK !!! If not, manually add: vi/etc/default/rsyslog RSYSLOGD_OPTIONS = "-r-c 5" to restart the syslog service. service rsyslog restart 6: optimize the haproxy Server 1. the kernel result is as follows: sysctl-p net. ipv4.ip _ forward = 1 net. ipv4.ip _ nonlocal_bind = 1 net. ipv4.tcp _ syncookies = 1 net. ipv4.tcp _ tw_reuse = 1 net. ipv4.ip _ local_port_range = 1024 65535 net. nf_conntrack_max = 1310720 net. ipv4.tcp _ tw_reuse = 1 net. ipv4.tcp _ fin_timeout = 15 net. core. netdev_max_backlog = 4096. net. core. rmem_max = 16777216. net. core. somaxconn = 1310720 net. core. wmem_max = 16777216. net. ipv4.tcp _ max_syn_backlog = 20480 net. ipv4.tcp _ max_tw_buckets = 400000. net. ipv4.tcp _ no_metrics_save = 1 net. ipv4.tcp _ rmem = 4096 87380 16777216 net. ipv4.tcp _ syn_retries = 2 net. ipv4.tcp _ synack_retries = 2 net. ipv4.tcp _ wmem = 4096 65536 16777216 vm. min_free_kbytes = 65536. net. ipv4.tcp _ sack = 1 net. ipv4.tcp _ timestamps = 1 net. ipv4.tcp _ tw_recycle = 1 copy to sysctl. vi/etc/sysctl in conf. conf 2. run the ulimit value permanently: ulimit-SHn display: 1024 permanent:. in/etc/pam. d/login, add: pam_limits.so (sometimes added by default) find file location: find/-name pam_limits.so display:/lib/x86_64-linux-gnu/security/pam_limits.sob. in/etc/security/limits. in conf, add: root soft nofile 10240 # actual value root hard nofile 10240 # actual value c. modify/etc/rc. add: echo 8061540>/proc/sys/fs/file-max execute: echo 8061540>/proc/sys/fs/file-max 3. kernel Parameter Optimization for nginx server sysctl-p net. ipv4.tcp _ fin_timeout = 30 net. ipv4.tcp _ keepalive_intvl = 2 net. ipv4.tcp _ keepalive_probes = 2 net. ipv4.tcp _ keepalive_time = 120. net. ipv4.tcp _ syn_retries = 10 net. ipv4.tcp _ sack = 1 net. ipv4.tcp _ timestamps = 1