apache+inotify-tools+keepalived+lvs-DR模式配置高可用負載平衡叢集

標籤：web伺服器   虛擬機器   作業系統   master   

環境：虛擬機器VMware workstation 9

      作業系統：Redhat 5.6 i386

一、keepalived+LVS-DR模式配置高可用負載平衡

拓撲如下：650) this.width=650;" src="http://s3.51cto.com/wyfs02/M01/30/9C/wKioL1OpF3qQUXSnAAFmxuu-O0I274.jpg" title="LVS+keepalived+Apache+DR.jpg" alt="wKioL1OpF3qQUXSnAAFmxuu-O0I274.jpg" />

二、伺服器IP配置資訊

四台伺服器均有VMware虛擬機器實現，兩台HA主機、兩台web伺服器。網路方式都設定為NAT模式

1、master：192.168.80.145

2、slaver：192.168.80.137

3、web1:192.168.80.144

4、web2:192.168.80.134

5、VIP：192.168.80.200

三、安裝配置keepalived

1、主輔伺服器部署keepalived

yum install gcc gcc-c++ autoconf automake wget openssl-devel kernel-devel  -ytar -zxf keepalived-1.1.20.tar.gz -C /usr/srccd /usr/src/keepalived-1.1.20./configure --prefix=/usr/local/keepalived --sysconf=/etc --with-kernel-dir=/usr/src/kernels/2.6.18-238.el5make && make installcp /usr/local/keepalived/sbin/keepalived /usr/sbincp /usr/local/keepalived/bin/genhash /usr/sbin//etc/init.d/keepalived startchkconfig keepalived on

注:主要痛點解決依賴關係

2、分別設定web伺服器1和2

yum install httpd -yecho "web1/web2 ok!!!">/var/www/html/index.htmlecho "check web1 ok!!! /check web2 ok !!!">/var/www/html/check_web1.html/etc/init.d/httpd start

3、主輔伺服器部署ipvsadm

tar -zxf ipvsadm-1.24.tar.gz -C /usr/srccd /usr/src/ipvsadm-1.24make && make install

註：安裝前檢查核心是否支援ipvsadm  

 cat /boot/config-2.6.18-238.el5 |grep -i ipvs# IPVS transport protocol load balancing support# IPVS scheduler# IPVS application helpermodprobe -l |grep ipvs/lib/modules/2.6.18-238.el5/kernel/net/ipv4/ipvs/ip_vs.ko/lib/modules/2.6.18-238.el5/kernel/net/ipv4/ipvs/ip_vs_dh.ko/lib/modules/2.6.18-238.el5/kernel/net/ipv4/ipvs/ip_vs_ftp.ko/lib/modules/2.6.18-238.el5/kernel/net/ipv4/ipvs/ip_vs_lblc.ko/lib/modules/2.6.18-238.el5/kernel/net/ipv4/ipvs/ip_vs_lblcr.ko/lib/modules/2.6.18-238.el5/kernel/net/ipv4/ipvs/ip_vs_lc.ko/lib/modules/2.6.18-238.el5/kernel/net/ipv4/ipvs/ip_vs_nq.ko/lib/modules/2.6.18-238.el5/kernel/net/ipv4/ipvs/ip_vs_rr.ko/lib/modules/2.6.18-238.el5/kernel/net/ipv4/ipvs/ip_vs_sed.ko/lib/modules/2.6.18-238.el5/kernel/net/ipv4/ipvs/ip_vs_sh.ko/lib/modules/2.6.18-238.el5/kernel/net/ipv4/ipvs/ip_vs_wlc.ko/lib/modules/2.6.18-238.el5/kernel/net/ipv4/ipvs/ip_vs_wrr.ko

4、配置主輔keepalived設定檔

/etc/keepalived/keepalived.conf! Configuration File for keepalivedglobal_defs {   notification_email {     [email protected]   }   notification_email_from [email protected]   smtp_server 127.0.0.1   smtp_connect_timeout 30   router_id httpd}vrrp_script check_http {        script "/root/bash/check_httpd.sh"        weight -5        interval 1    }vrrp_instance VI_1 {    state MASTER/BACKUP    interface eth0    virtual_router_id 240    priority 100/98    advert_int 1#    track_script {#        check_http#    }    authentication {        auth_type PASS        auth_pass 3333    }    virtual_ipaddress {       192.168.80.200/24 dev eth0    }}virtual_server 192.168.80.200 80 {    delay_loop 6    lb_algo rr    lb_kind DR    nat_mask 255.255.255.0    persistence_timeout 0    protocol TCP    real_server 192.168.80.134 80 {        weight 1        HTTP_GET {            url {              path /check_web1.html              digest 68f36ac34591233a3ca3b5def1bace34            }            connect_timeout 3            nb_get_retry 3            delay_before_retry 3        }    }    real_server 192.168.80.144 80 {        weight 1        HTTP_GET {            url {              path /check_web1.html              digest 68f36ac34591233a3ca3b5def1bace34            }            connect_timeout 3            nb_get_retry 3            delay_before_retry 3        }    }}

註：（1）、主輔設定優先權

    （2）、genhash值的計算、web1、web2的httpd服務開啟時才能

    （3）、使用的是HTTP_GET檢查web伺服器的健康狀態

    （4）、lvs模式設定

    （5）、persistence_timeout 設定為0 方便後面測試

5、檢查部署情況

master檢查ipvsadm 情況

ipvsadm -lnIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags  -> RemoteAddress:Port           Forward Weight ActiveConn InActConnTCP  192.168.80.200:80 rr  -> 192.168.80.144:80            Route   1      0          0           -> 192.168.80.134:80            Route   1      0          0

6、主伺服器重啟keepalived服務

tail -f /var/log/messages

May 17 00:37:48 master Keepalived: Terminating on signalMay 17 00:37:48 master Keepalived: Stopping Keepalived v1.1.20 (05/16,2014) May 17 00:37:48 master Keepalived_vrrp: Terminating VRRP child process on signalMay 17 00:37:49 master Keepalived_healthcheckers: Terminating Healthchecker child process on signalMay 17 00:37:49 master Keepalived_vrrp: VRRP_Instance(VI_1) removing protocol VIPs.May 17 00:37:49 master avahi-daemon[3543]: Withdrawing address record for 192.168.80.200 on eth0.May 17 00:37:49 master Keepalived: Starting Keepalived v1.1.20 (05/16,2014) May 17 00:37:49 master Keepalived_healthcheckers: Netlink reflector reports IP 192.168.80.145 addedMay 17 00:37:49 master Keepalived_healthcheckers: Registering Kernel netlink reflectorMay 17 00:37:49 master Keepalived_healthcheckers: Registering Kernel netlink command channelMay 17 00:37:49 master Keepalived: Starting Healthcheck child process, pid=12821May 17 00:37:49 master Keepalived: Starting VRRP child process, pid=12822May 17 00:37:49 master Keepalived_vrrp: Netlink reflector reports IP 192.168.80.145 addedMay 17 00:37:49 master Keepalived_vrrp: Registering Kernel netlink reflectorMay 17 00:37:49 master Keepalived_vrrp: Registering Kernel netlink command channelMay 17 00:37:49 master Keepalived_vrrp: Registering gratutious ARP shared channelMay 17 00:37:49 master Keepalived_vrrp: Opening file ‘/etc/keepalived/keepalived.conf‘. May 17 00:37:49 master Keepalived_vrrp: Configuration is using : 37747 BytesMay 17 00:37:49 master Keepalived_vrrp: Using LinkWatch kernel netlink reflector...May 17 00:37:49 master Keepalived_vrrp: VRRP sockpool: [ifindex(2), proto(112), fd(10,11)]May 17 00:37:49 master Keepalived_healthcheckers: Opening file ‘/etc/keepalived/keepalived.conf‘. May 17 00:37:49 master Keepalived_healthcheckers: Configuration is using : 14817 BytesMay 17 00:37:49 master Keepalived_healthcheckers: Using LinkWatch kernel netlink reflector...May 17 00:37:49 master Keepalived_healthcheckers: Activating healtchecker for service [192.168.80.134:80]May 17 00:37:49 master Keepalived_healthcheckers: Activating healtchecker for service [192.168.80.144:80]May 17 00:37:50 master Keepalived_vrrp: VRRP_Instance(VI_1) Transition to MASTER STATEMay 17 00:37:51 master Keepalived_vrrp: VRRP_Instance(VI_1) Entering MASTER STATEMay 17 00:37:51 master Keepalived_vrrp: VRRP_Instance(VI_1) setting protocol VIPs.May 17 00:37:51 master Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.80.200May 17 00:37:51 master Keepalived_vrrp: Netlink reflector reports IP 192.168.80.200 addedMay 17 00:37:51 master Keepalived_healthcheckers: Netlink reflector reports IP 192.168.80.200 addedMay 17 00:37:51 master avahi-daemon[3543]: Registering new address record for 192.168.80.200 on eth0.May 17 00:37:56 master Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.80.200May 17 00:38:05 master Keepalived: Terminating on signalMay 17 00:38:05 master Keepalived: Stopping Keepalived v1.1.20 (05/16,2014) May 17 00:38:05 master Keepalived_vrrp: Terminating VRRP child process on signalMay 17 00:38:05 master Keepalived_healthcheckers: Terminating Healthchecker child process on signalMay 17 00:38:05 master Keepalived_vrrp: VRRP_Instance(VI_1) removing protocol VIPs.May 17 00:38:05 master avahi-daemon[3543]: Withdrawing address record for 192.168.80.200 on eth0.May 17 00:38:52 master Keepalived: Starting Keepalived v1.1.20 (05/16,2014) May 17 00:38:52 master Keepalived_healthcheckers: Netlink reflector reports IP 192.168.80.145 addedMay 17 00:38:52 master Keepalived_healthcheckers: Registering Kernel netlink reflectorMay 17 00:38:52 master Keepalived_healthcheckers: Registering Kernel netlink command channelMay 17 00:38:52 master Keepalived: Starting Healthcheck child process, pid=12838May 17 00:38:52 master Keepalived: Starting VRRP child process, pid=12839May 17 00:38:52 master Keepalived_vrrp: Netlink reflector reports IP 192.168.80.145 addedMay 17 00:38:52 master Keepalived_vrrp: Registering Kernel netlink reflectorMay 17 00:38:52 master Keepalived_vrrp: Registering Kernel netlink command channelMay 17 00:38:52 master Keepalived_vrrp: Registering gratutious ARP shared channelMay 17 00:38:52 master Keepalived_healthcheckers: Opening file ‘/etc/keepalived/keepalived.conf‘. May 17 00:38:52 master Keepalived_healthcheckers: Configuration is using : 14817 BytesMay 17 00:38:52 master Keepalived_healthcheckers: Using LinkWatch kernel netlink reflector...May 17 00:38:52 master Keepalived_healthcheckers: Activating healtchecker for service [192.168.80.134:80]May 17 00:38:52 master Keepalived_vrrp: Opening file ‘/etc/keepalived/keepalived.conf‘. May 17 00:38:52 master Keepalived_healthcheckers: Activating healtchecker for service [192.168.80.144:80]May 17 00:38:52 master Keepalived_vrrp: Configuration is using : 37747 BytesMay 17 00:38:52 master Keepalived_vrrp: Using LinkWatch kernel netlink reflector...May 17 00:38:52 master Keepalived_vrrp: VRRP sockpool: [ifindex(2), proto(112), fd(10,11)]May 17 00:38:53 master Keepalived_vrrp: VRRP_Instance(VI_1) Transition to MASTER STATE

7、在web伺服器上分別執行以下指令碼

#!/bin/bash#description:start realserver#chkconfigVIP1=192.168.80.200case "$1" instart)echo " start LVS of REALServer"/sbin/ifconfig lo:0 $VIP1 broadcast $VIP1 netmask 255.255.255.255 upecho "1" >/proc/sys/net/ipv4/conf/lo/arp_ignoreecho "2" >/proc/sys/net/ipv4/conf/lo/arp_announceecho "1" >/proc/sys/net/ipv4/conf/all/arp_ignoreecho "2" >/proc/sys/net/ipv4/conf/all/arp_announce;;stop)/sbin/ifconfig lo:0 downecho "close LVS Directorserver"echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignoreecho "0" >/proc/sys/net/ipv4/conf/lo/arp_announceecho "0" >/proc/sys/net/ipv4/conf/all/arp_ignoreecho "0" >/proc/sys/net/ipv4/conf/all/arp_announce;;*)echo "Usage: $0 {start|stop}"exit 1esac

8、現在重啟主輔伺服器上的keepalived服務、web伺服器上的httpd服務，關閉master的keepalived服務做測試。

keepalived+LVS-DR模式就部署就完成了 

9、在web1上部署inotify-tools工具、web2上部署rsync服務，用於當web1網頁內容變化時，web2伺服器的網頁內容的一致性問題

web1上部署inotify-tools

tar -zxf inotify-tools-3.13.tar.gz -C /usr/srccd /usr/src/inotify-tools-3.13./configure --prefix=/usr/local/inotify_toolsmake && make install cp /usr/local/inotify_tools/bin/inotifywait /usr/sbin/cp /usr/local/inotify_tools/bin/inotifywatch /usr/sbin/chmod 755 inotify-rsync.shinotify-rsync.sh#!/bin/bashhost1=192.168.80.137src=/var/www/htmldst1=testuser1=root/usr/local/inotify_tools/bin/inotifywait -mrq --timefmt ‘%d/%m/%y %H:%M‘ --format ‘%T %w%f%e‘ -e modify,delete,create,attrib  $src | while read files        do        /usr/bin/rsync -vzrtopg --delete --progress --password-file=/etc/rsync.passwd [email protected]$host1::$dst1                echo "${files} was rsynced" >>/tmp/rsync.log 2>&1         done                  inotify-rsync.sh  &

web2上部署rsync服務

chmod 600 /etc/rsync.passwd/etc/rsync.passwdroot:strong/etc/rsyncd.confstrict modes = yesport = 873pid file = /var/run/rsyncd.pidlock file = /var/run/rsync.locklog file = /var/log/rsyncd.logmotd file = /etc/rsyncd.motduse chroot =timeout =[test]path = /var/www/comment = rsync filesmax connections = 5uid = rootgid = rootignore errorsread only = nowrite only = nolist = nohosts allow = *hosts deny = 10.1.1.1auth users = rootsecrets file = /etc/rsync.passwd啟動rsync服務chkconfig rsync on/etc/init.d/xinted restart

10、至此apache+rsync+inotify-tools+keepalived+lvs-DR模式HA負載平衡部署就此結束

本文出自 “IT碼農” 部落格，謝絕轉載！