ubuntu+haproxy+keepalived主主負載

ubuntu+haproxy+keepalived主主負載 一.測試環境：系統：ubuntu server 12.04   haproxy版本：1.4.24  keepalived版本：keepalived-1.2.7  haporxy01：eth0：172.16.1.36  eth1：192.168.100.36  haporxy02：eth0：172.16.1.37  eth1：192.168.100.37  vip1：172.16.1.30  vip2：172.16.1.31  nignx1 website顯示：nginx1  nginx2 website顯示：ningx2  nginx3 website顯示：nginx3  nginx4 website顯示：nginx4 二.網路結構                                    user                                                 |                                      |            (vip1)                  |                 (vip2)          haproxy01-------keepalived-------haproxy02             / \                                          / \                 /   \                                        /   \           /     \                                      /     \          /       \                                    /       \      --------------------------------------------------------      | nginx1   nginx2                  nginx3   nginx4 |      --------------------------------------------------------三.安裝1.安裝keepalvied  主機haproxy01：   wget http://www.keepalived.org/software/keepalived-1.2.7.tar.gz  tar xf keepalived-1.2.7.tar.gz  cd keepalived-1.2.7  ./configure --prefix=/usr/local/ 提示：  checking for openssl/ssl.h... no  configure: error:   !!! OpenSSL is not properly installed on your system. !!!  !!! Can not include OpenSSL headers files.            !!! 安裝libssl.dev  apt-get install libssl.dev 繼續：  ./configure --prefix=/usr/local/ 提示：  checking for poptGetContext in -lpopt... no  configure: error: Popt libraries is required 安裝libpopt-dev  apt-get install libpopt-dev 繼續：  ./configure --prefix=/usr/local/  make  make install 2.編輯keepalived.conf  mkdir /etc/keepalived/  vi /etc/keepalived/keepalived.conf ########  keepalived.conf  ########global_defs {        router_id LVS_DEVEL} vrrp_instance VI_1 {        state MASTER  # haproxy02:BACKUP        interface eth0        virtual_router_id 51        priority 91   # 比haproxy02大        advert_int 1        authentication {                auth_type PASS                auth_pass 123456789        }         virtual_ipaddress {                172.16.1.30        } vrrp_instance VI_2 {        state BACKUP  # haproxy02:MASTER        interface eth0        virtual_router_id 52        priority 90   # 比haproxy02小        advert_int 1        authentication {                auth_type PASS                auth_pass 123456789        }         virtual_ipaddress {                172.16.1.31        }########  keepalived.conf  ######## 3.啟動keepalived  /usr/local/sbin/keepalvied -f /etc/keepalived/keepalived.conf  4.查看vip  ip addr顯示：  eth0：      inet 172.16.1.30/32 scope global eth0      inet 172.16.1.31/32 scope global eth0 #（haproxy02的keepalived未啟動，vip2在haproxy01中，haproxy02中keepalived啟動後，vip2自動跳轉到haproxy02中eth0上） 主機haproxy02： keepalived安裝同上，keepalived.conf如下： ########  keepalived.conf  ########global_defs {        router_id LVS_DEVEL} vrrp_instance VI_1 {        state BACKUP  # haproxy01:MASTER        interface eth0        virtual_router_id 51        priority 90   # 比haproxy01小        advert_int 1          authentication {            auth_type PASS            auth_pass 123456789        }         virtual_ipaddress {                172.16.1.30        } vrrp_instance VI_2 {        state MASTER  # haproxy01:BACKUP        interface eth0        virtual_router_id 52        priority 91   # 比haproxy01大        advert_int 1        authentication {            auth_type PASS            auth_pass 123456789        }         virtual_ipaddress {                172.16.1.31        }########  keepalived.conf  ######## 啟動keepalived  /usr/local/sbin/keepalvied -f /etc/keepalived/keepalived.conf查看vip：  ip addr顯示：  eth0：      inet 172.16.1.31/32 scope global eth0 # (haproxy中eth0上vip2消失，只有vip1) 停止任意一台keepalived服務，另外一台均可自動產生vip1和vip2，確保2台haproxy高可用性 2.安裝haproxy  wget http://haproxy.1wt.eu/download/1.4/src/haproxy-1.4.24.tar.gz  tar xf haproxy-1.4.24.tar.gz   cd haproxy-1.4.24查看ubuntu版本資訊：  uname -a顯示：  Linux ubuntu37 3.2.0-51-generic #77-Ubuntu SMP Wed Jul 24 20:18:19 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux  make TARGET=37-ubuntu  make PREFIX=/usr/local/haproxy install建立haproxy使用者    useradd haproxy 2.編輯haproxy.cfg  mkdir /etc/haproxy  vi /etc/haproxy/haproxy.cfg ########  haproxy.cfg  ########global     log 127.0.0.1 local0     maxconn 51200     user haproxy     group haproxy     daemon defaults     log  127.0.0.1 local3     mode http     option dontlognull     balance roundrobin         retries 2     option redispatch     option abortonclose        maxconn 51200     contimeout 5000     clitimeout 50000     srvtimeout 50000 listen haproxy01     bind 172.16.1.30:80     mode http     option  httpclose      option forceclose     option  forwardfor     option originalto     option  redispatch     balance leastconn     timeout check 5s     stats uri /stats          # haproxy負載監控頁面 例：http://172.16.1.30/stats     stats refresh 15s       # 監控頁面重新整理時間     stats realm baison-test-Haproxy     stats auth admin:admin      # 監控頁面帳號密碼     stats hide-version              # 隱藏haproxy版本資訊     cookie  SESSION_COOKIE  insert indirect nocache  maxconn 40960     server nginx1 192.168.100.11:80 weight 1 cookie nginx1 check inter 2000 rise 2 fall 3      server nginx2 192.168.100.12:80 weight 1 cookie nginx2 check inter 2000 rise 2 fall 3 listen haproxy02     bind 172.16.1.31:80     mode http     option  httpclose      option forceclose     option  forwardfor     option originalto     option  redispatch     balance leastconn     cookie  SESSION_COOKIE  insert indirect nocache     maxconn 40960     server nginx3 192.168.100.13:80 weight 1 cookie nginx3 check inter 2000 rise 2 fall 3     server nginx4 192.168.100.14:80 weight 1 cookie nginx4 check inter 2000 rise 2 fall 3########  haproxy.cfg  ########啟動haproxy：  /usr/local/haproxy/sbin/haproxy -f /etc/haproxy/haproxy.cfg 主機haproxy02：安裝同上，haproxy.cfg同上啟動haproxy：  /usr/local/haproxy/sbin/haproxy -f /etc/haproxy/haproxy.cfg 四：測試通過瀏覽器訪問：  http://172.16.1.30頁面顯示：nginx1、nginx2輪詢切換  http://172.16.1.31頁面顯示：nginx3、nginx4輪詢切換可通過監控頁面查看負載情況：  http://172.16.1.30/stats 停止任意一台haproxy服務，網站均不受影響 五：haproxy日誌1.在/etc/rsyslog.conf中：添加haproxy日誌路徑:  local0.* /var/log/haproxy.log   local3.* /var/log/haproxy.log取消注釋:  #$ModLoad imudp  ==》$ModLoad imudp  #$UDPServerRun 514 ==》$UDPServerRun 514重啟rsyslog服務  service rsyslog restart 2.核實services檔案  grep 514 /etc/services顯示：  syslog 514/udp    為ok!!!如不存在，手動添加：  vi /etc/default/rsyslog  RSYSLOGD_OPTIONS="-r -c 5"重啟syslog服務，  service rsyslog restart 六：最佳化haproxy伺服器1.核心結果如下：  sysctl -p  net.ipv4.ip_forward = 1  net.ipv4.ip_nonlocal_bind = 1  net.ipv4.tcp_syncookies = 1  net.ipv4.tcp_tw_reuse = 1  net.ipv4.ip_local_port_range = 1024 65535  net.nf_conntrack_max = 1310720  net.ipv4.tcp_tw_reuse = 1  net.ipv4.tcp_fin_timeout = 15  net.core.netdev_max_backlog = 4096  net.core.rmem_max = 16777216  net.core.somaxconn = 1310720  net.core.wmem_max = 16777216  net.ipv4.tcp_max_syn_backlog = 20480  net.ipv4.tcp_max_tw_buckets = 400000  net.ipv4.tcp_no_metrics_save = 1  net.ipv4.tcp_rmem = 4096 87380 16777216  net.ipv4.tcp_syn_retries = 2  net.ipv4.tcp_synack_retries = 2  net.ipv4.tcp_wmem = 4096 65536 16777216  vm.min_free_kbytes = 65536  net.ipv4.tcp_sack = 1  net.ipv4.tcp_timestamps = 1  net.ipv4.tcp_tw_recycle = 1拷貝至sysctl.conf中  vi /etc/sysctl.conf 2.ulimit數值永久化執行：  ulimit -SHn顯示：  1024永久化：a.在/etc/pam.d/login中，添加：  pam_limits.so (有時候系統預設添加)尋找檔案位置：   find / -name pam_limits.so顯示：  /lib/x86_64-linux-gnu/security/pam_limits.sob.在/etc/security/limits.conf中，添加：  root    soft nofile 10240 #實際值  root    hard nofile 10240 #實際值c.修改 /etc/rc.local 添加：  echo 8061540 > /proc/sys/fs/file-max執行：  echo 8061540 > /proc/sys/fs/file-max 3.nginx伺服器核心參數最佳化sysctl -p  net.ipv4.tcp_fin_timeout = 30  net.ipv4.tcp_keepalive_intvl = 2  net.ipv4.tcp_keepalive_probes = 2  net.ipv4.tcp_keepalive_time = 120  net.ipv4.tcp_syn_retries = 10  net.ipv4.tcp_sack = 1  net.ipv4.tcp_timestamps = 1