用HAproxy+keepalived+mysql Replication 構建基於企業級負載平衡

   最近用HAproxy+keepalived+mysql複製測試高可用性Linux系統叢集。
   HAProxy是一款免費的提供高可用性、負載平衡以及基於TCP第四層）和HTTP第七層）應用的代理軟體，藉助HAProxy可以快速並且可靠的提供基於TCP和HTTP應用的代理解決方案。
   Keepalived主要作用是LoadBalance master和LoadBalance backup之間的健全狀態檢查，實現故障轉換。
   Mysql Replication主要作用是提高mysql並處理資料的能力以及實現容災備份的作用。

項目拓撲圖：

650) this.width=650;" src="http://www.bkjia.com/uploads/allimg/131227/1J91CH0-0.jpg" title="1.jpg" />

Haproxy伺服器及其IP地址規劃：

650) this.width=650;" src="http://www.bkjia.com/uploads/allimg/131227/1J91C212-1.jpg" title="0.jpg" />

項目實施：

1.Mysql主從複製配置。

1)在master伺服器上建立mysql使用者授權複製賬戶）。grant replication slave on *.* to'rep'@'192.168.1.244'identified by 'rep123';2)編輯master伺服器的mysql設定檔my.cnf。server-id = 1 //指定伺服器的IDlog-bin = mysql-bin //開啟二進位日誌binlog-ignore = mysql //忽略mysql和information_schema 資料庫binlog-ignore = information_schemabinlog-do-db = blog    //同步資料庫，預設同步所有資料庫3)查看master狀態。mysql> show master status;+---------------+----------+--------------+--------------------------+| File          | Position | Binlog_Do_DB | Binlog_Ignore_DB         |+---------------+----------+--------------+--------------------------+| binlog.022343 |   339244 | blog         | mysql,information_schema |+---------------+----------+--------------+--------------------------+1 row in set (0.00 sec)4)在slave端建立資料庫blog,匯出master端的blog庫，匯入到此庫，並修改mysql主設定檔my.cnf server-id = 2 重啟mysql資料庫。mysql> change master to    -> master_host='192.168.1.243',    -> master_port=3306,    -> master_user='rep',    -> master_password='rep123',    -> master_log_file='binlog.022343',    -> master_log_pos=339244;5)mysql主從同步測試，show slave status\G;能看到Slave_IO_Running和Slave_SQL_Running都為YES即可。mysql> show slave status\G;*************************** 1. row ***************************             Slave_IO_State: Waiting for master to send event                Master_Host: 192.168.1.243                Master_User: rep                Master_Port: 3306              Connect_Retry: 60            Master_Log_File: binlog.022343        Read_Master_Log_Pos: 339110             Relay_Log_File: relaylog.005481              Relay_Log_Pos: 339244      Relay_Master_Log_File: binlog.022343           Slave_IO_Running: Yes          Slave_SQL_Running: Yes            Replicate_Do_DB:        Replicate_Ignore_DB: mysql,test,information_schema         Replicate_Do_Table:     Replicate_Ignore_Table:    Replicate_Wild_Do_Table:Replicate_Wild_Ignore_Table:                 Last_Errno: 0                 Last_Error:               Skip_Counter: 0        Exec_Master_Log_Pos: 339110            Relay_Log_Space: 339244            Until_Condition: None             Until_Log_File:              Until_Log_Pos: 0         Master_SSL_Allowed: No         Master_SSL_CA_File:         Master_SSL_CA_Path:            Master_SSL_Cert:          Master_SSL_Cipher:             Master_SSL_Key:      Seconds_Behind_Master: 01 row in set (0.00 sec)

2.Haproxy安裝及其配置，master和backup安裝配置都是完全一樣的。

wget http://haproxy.1wt.eu/download/1.4/src/haproxy-1.4.24.tar.gztar -zxvf haproxy-1.4.24.tar.gzcd haproxy-1.4.24make TARGET=linux26 PREFIX=/usr/local/haproxymake install PREFIX=/usr/local/haproxycd /usr/local/haproxymkdir conf logs   //在此目錄下面建立conf，logs目錄分別存放HAproxy的設定檔，PID檔案和記錄檔。vim conf/haproxy.confglobal        maxconn 50000        chroot /usr/local/haproxy        uid 99        gid 99        daemon        quiet        nbproc  2        pidfile /usr/local/haproxy/logs/haproxy.pid        #debugdefaults        log     global        mode    http        option  httplog   #每次請求完畢後主動關閉http通道        option  dontlognull #不記錄健全狀態檢查的日誌資訊        option forwardfor        option redispatch        option abortonclose        retries 3        log 127.0.0.1 local3        maxconn 20000        contimeout      5000        clitimeout      50000        srvtimeout      50000listen 192.168.1.236       bind *:80       mode http       stats uri /admin                    #後端伺服器狀態查看地址       stats auth admin:admin             #狀態查看頁面登陸帳號密碼       balance source   #調度演算法，source是和nginx的ip_hash同理，解決session問題       option httpclose       option forwardforserver web1 192.168.1.248:80 weight 5 check inter 2000 rise 2 fall 5server web2 192.168.1.249:8080 weight 5 check inter 2000 rise 2 fall 5#啟動Haproxy服務/usr/local/haproxy/sbin/haproxy -f haproxy.conf#haproxy啟動指令碼#!/bin/bashBASE_DIR="/usr/local/haproxy"ARGV="$@"start(){echo "START HAPoxy SERVERS"$BASE_DIR/sbin/haproxy -f $BASE_DIR/conf/haproxy.conf}stop(){echo "STOP HAPoxy Listen"kill -TTOU $(cat $BASE_DIR/logs/haproxy.pid)echo "STOP HAPoxy process"kill -USR1 $(cat $BASE_DIR/logs/haproxy.pid)}case $ARGV instart)startERROR=$?;;stop)stopERROR=$?;;restart)stopstartERROR=$?;;*)echo "hactl.sh [start|restart|stop]"esacexit $ERROR

3.Haproxy開啟系統日誌支援。

vim /etc/syslog.conf#添加：local3.*        /var/log/haproxy.loglocal0.*        /var/log/haproxy.logvim /etc/sysconfig/syslog#修改：SYSLOGD_OPTIONS="-r -m 0"#重新啟動syslog服務/etc/init.d/syslog restart

4.Keepalived的安裝配置。

wget http://www.keepalived.org/software/keepalived-1.2.5.tar.gzln -s /usr/src/kernels/2.6.18-164.el5-x86_64/ /usr/src/linuxtar -zxvf keepalived-1.2.5.tar.gzcd keepalived-1.2.5./configure --prefix=/usr/local/keepalivedmake && make installcp keepalived/etc/init.d/keepalived.rh.init /etc/init.d/keepalivedchmod +x /etc/init.d/keepalivedcp keepalived/etc/init.d/keepalived.sysconfig /etc/sysconfig/keepalivedchkconfig --add keepalivedchkconfig --level 35 keepalived oncp /usr/local/keepalived/sbin/keepalived /bin/mkdir -p /etc/keepalivedcp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/#Haproxy master設定檔vim /etc/keepalived/keepalived.conf! Configuration File for keepalivedglobal_defs {    notification_email {    shifeng_zhang88@163.com    }    notification_email_from shifeng_zhang88@163.com    smtp_server smtp.163.com    smtp_connect_timeout 30    router_id LVS_Master}vrrp_script chk_http_port {    script  "/etc/keepalived/check_haproxy.sh"    interval        5       #指令碼執行間隔    weight         -5       #執行指令碼後優先順序變更：5表示優先順序+5；-5則表示優先順序-5}vrrp_instance VI_A {    state MASTER    interface eth0    virtual_router_id 50    priority 100    advert_int 1    authentication {        auth_type PASS        auth_pass sfzhang1109    }track_script {    chk_http_port    }    virtual_ipaddress {        192.168.1.236    #haproxy虛擬IP    }}#Haproxy backup設定檔只需改變state和priority的值state BACKUPpriority 50#啟動keepalived服務/etc/init.d/keepalived start

5.編輯check_haproxy.sh指令碼，需要將haproxy啟動指令碼放到/etc/init.d/裡面。

vim /etc/keepalived/check_haproxy.sh#!/bin/bashA=`ps -C haproxy --no-header |wc -l`if [ $A -eq 0 ];then/etc/init.d/haproxy restartecho "Start haproxy" &> /dev/nullsleep 3if [ `ps -C haproxy --no-header |wc -l` -eq 0 ];then/etc/init.d/keepalived stopecho "Stop keepalived" &> /dev/nullfifi#chomd +x /etc/keepalived/check_haproxy.sh

6.Haproxy+Keepalived高可用性測試。

1）Haproxy+Keepalived 高可用測試。

   測試方法：停止master的keepalived服務，查看備用的keepalived的日誌發現當主節點宕機時，備用節點角色會立即變為主節點，啟用VIPS協議，並把VIP地址立刻綁定到eth0網卡上面,當主節點恢複時則做相反的工作。

650) this.width=650;" src="http://www.bkjia.com/uploads/allimg/131227/1J91631C-2.jpg" title="3.jpg" />

2)Haproxy+Keepalived負載平衡測試

   測試方法：當用戶端訪問VIP的時候，haproxy根據設定的調度演算法和權重把訪問請求分發到後端的WEB伺服器上面，從而實現了負載平衡的功能。

3)Haproxy+Keepalived 容錯移轉測試

   測試方法：當後端的WEB伺服器down機時，haproxy會自動檢測到並把請求發送到正常的伺服器上面,通過haproxy監控頁面http://192.168.1.235/admin）可以查看。

650) this.width=650;" src="http://www.bkjia.com/uploads/allimg/131227/1J91BX7-3.jpg" title="00.jpg" />

    說明：希望大家提出寶貴的意見）

      1）拓撲圖只是項目的一個雛形，後期還要添加監控伺服器和郵件等伺服器。

      2）Mysql主庫負責使用者資料的寫入，slave負責使用者資料的讀取和資料的備份等，除了要監控mysql主從外還要監控Seconds_Behind_Master選項，以免產生主從資料不同步或者延時。

      3）當WEB的負載很大的時候，可以在haproxy添加多台物理機即可，並考慮把項目的圖片分離出來做成單獨的圖片伺服器。