雙節點(nginx+keepalived)為兩台apache伺服器提供負載平衡

標籤：

說明：本實驗為雙節點nginx為兩台apache伺服器提供負載平衡，本文不是做lvs，所以realserver不是配置在keepalived.conf而是在nginx的設定檔中upstream。
此架構需考慮的問題：
1）Master沒掛，則Master佔有vip且nginx運行在Master上
2）Master掛了，則backup搶佔vip且在backup上運行nginx服務
3）如果master伺服器上的nginx服務掛了，則vip資源轉移到backup伺服器上
4）檢測後端伺服器的健康狀態
Master和Backup兩邊都開啟nginx服務，無論Master還是Backup，當其中的一個keepalived服務停止後，vip都會漂移到keepalived服務還在的節點上，如果要想使nginx服務掛了，vip也漂移到另一個節點，則必須用指令碼或者在設定檔裡面用shell命令來控制。

配置步驟如下
1.初始化4台測試server，該關的關了

[[email protected] ~]# vim /etc/hosts192.168.1.200   ng-vip192.168.1.101   ng-master192.168.1.102   ng-slave192.168.1.161   web1192.168.1.162   web2[[email protected] ~]# yum clean all[[email protected] ~]# systemctl stop firewalld.service[[email protected] ~]# systemctl disable firewalld.service[[email protected] ~]# sed -i "s/SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config

2.配置web1,web2的apache服務，兩台一樣的方法

[[email protected] ~]# yum -y install httpd[[email protected] ~]# systemctl start httpd[[email protected] ~]# systemctl enable httpdln -s ‘/usr/lib/systemd/system/httpd.service‘ ‘/etc/systemd/system/multiuser.target.wants/httpd.service‘[[email protected] ~]# cat /var/www/html/index.htmlhello this lvs-web1[[email protected] ~]# yum -y install httpd[[email protected] ~]# systemctl start httpd[[email protected] ~]# systemctl enable httpdln -s ‘/usr/lib/systemd/system/httpd.service‘ ‘/etc/systemd/system/multiuser.target.wants/httpd.service‘[[email protected] ~]# cat /var/www/html/index.htmlhello this lvs-web2

3.通過yum安裝配置nginx節點，兩台一樣的方法

[[email protected] ~]# vim /etc/yum.repos.d/nginx.repo[nginx]name=nginx repobaseurl=http://nginx.org/packages/centos/7/x86_64/gpgcheck=0enabled=1[[email protected] ~]# yum clean all[[email protected] ~]# yum -y install nginx[[email protected] ~]# vim /usr/share/nginx/html/index.html<h1>Welcome to ng-master!</h1>[[email protected] ~]# cd /etc/nginx/conf.d/[[email protected] conf.d]# mv default.conf default.conf.1[[email protected] ~]# vim /etc/nginx/conf.d/web.conf    upstream myapp1 {        server web1;        server web2;    }    server {        listen 80;        location / {            proxy_pass http://myapp1;        }    }[[email protected] ~]# systemctl restart nginx.service[[email protected] ~]# vim /etc/yum.repos.d/nginx.repo[nginx]name=nginx repobaseurl=http://nginx.org/packages/centos/7/x86_64/gpgcheck=0enabled=1[[email protected] ~]# yum clean all[[email protected] ~]# yum -y install nginx[[email protected] ~]# vim /usr/share/nginx/html/index.html<h1>Welcome to ng-master!</h1>[[email protected] ~]# cd /etc/nginx/conf.d/[[email protected] conf.d]# mv default.conf default.conf.1[[email protected] ~]# vim /etc/nginx/conf.d/web.conf    upstream myapp1 {        server web1;        server web2;    }    server {        listen 80;        location / {            proxy_pass http://myapp1;        }    }[[email protected] ~]# systemctl restart nginx.service

4.在主nginx伺服器上安裝keepalived，並配置nginx服務健康檢測指令碼

[[email protected] conf.d]# yum -y install keepalived[[email protected] conf.d]# cd /etc/keepalived/[[email protected] keepalived]# cp keepalived.conf keepalived.conf.1[[email protected] keepalived]# vim keepalived.confglobal_defs {   notification_email {     [email protected]   }   notification_email_from [email protected]   smtp_server smtp.mail.com   smtp_connect_timeout 30   router_id HA_MASTER1  #表示運行keepalived伺服器的一個標識，發郵件時顯示在郵件主題中的資訊}vrrp_script chk_http_port {script "/usr/local/keepalived/nginx.sh" ####檢測nginx狀態的指令碼連結interval 2weight 2}vrrp_instance VI_2 {   #vrrp執行個體    state MASTER     #MASTER/BACKUP    interface eno16777736    ####HA 監測網路介面    virtual_router_id 51  #虛擬路由辨別碼，是一個數字，同一個VRRP執行個體使用唯一的標識,master和backup要一樣    priority 100          #用於主從模式，優先順序主高於100，從低於100    advert_int 1           #主備之間的通告間隔秒數    authentication {        #認證用於主從模式，mater和backup配置一樣        auth_type PASS          ###主備切換時的驗證        auth_pass 1111          #密碼    }track_script {chk_http_port ### 執行監控的服務}    virtual_ipaddress { 192.168.1.200/24 dev eno16777736 label eno16777736:1   ###########虛擬ip    }}[[email protected] keepalived]# mkdir -p /usr/local/keepalived[[email protected] keepalived]# vim /usr/local/keepalived/nginx.sh#!/bin/bashif [ `ps -C nginx --no-header |wc -l` -eq 0 ];thenkillall keepalivedfi[[email protected] keepalived]# chmod 755 /usr/local/keepalived/nginx.sh[[email protected] keepalived]# systemctl start keepalived[[email protected] keepalived]# ifconfig -aeno16777736: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500        inet 192.168.1.101  netmask 255.255.255.0  broadcast 192.168.1.255        inet6 fe80::20c:29ff:fefe:6f3  prefixlen 64  scopeid 0x20<link>        ether 00:0c:29:fe:06:f3  txqueuelen 1000  (Ethernet)eno16777736:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500        inet 192.168.1.200  netmask 255.255.255.0  broadcast 0.0.0.0        ether 00:0c:29:fe:06:f3  txqueuelen 1000  (Ethernet)lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536        inet 127.0.0.1  netmask 255.0.0.0        inet6 ::1  prefixlen 128  scopeid 0x10<host>        loop  txqueuelen 0  (Local Loopback)

5.在備nginx伺服器上安裝keepalived，並配置nginx服務健康檢測指令碼，與主略有不同

[[email protected] conf.d]# yum -y install keepalived[[email protected] conf.d]# cd /etc/keepalived/[[email protected] keepalived]# cp keepalived.conf keepalived.conf.1[[email protected] keepalived]# vim keepalived.confglobal_defs {   notification_email {     [email protected]   }   notification_email_from [email protected]   smtp_server smtp.mail.com   smtp_connect_timeout 30   router_id HA_MASTER1  #表示運行keepalived伺服器的一個標識，發郵件時顯示在郵件主題中的資訊}vrrp_script chk_http_port {script "/usr/local/keepalived/nginx.sh" ####檢測nginx狀態的指令碼連結interval 2weight 2}vrrp_instance VI_2 {   #vrrp執行個體    state BACKUP     #MASTER/BACKUP    interface eno16777736    ####HA 監測網路介面    virtual_router_id 51  #虛擬路由辨別碼，是一個數字，同一個VRRP執行個體使用唯一的標識,master和backup要一樣    priority 80          #用於主從模式，優先順序主高於100，從低於100    advert_int 1           #主備之間的通告間隔秒數    authentication {        #認證用於主從模式，mater和backup配置一樣        auth_type PASS          ###主備切換時的驗證        auth_pass 1111          #密碼    }track_script {chk_http_port ### 執行監控的服務}    virtual_ipaddress { 192.168.1.200/24 dev eno16777736 label eno16777736:1   ###########虛擬ip    }}[[email protected] keepalived]# mkdir -p /usr/local/keepalived[[email protected] keepalived]# vim /usr/local/keepalived/nginx.sh#!/bin/bashif [ `ps -C nginx --no-header |wc -l` -eq 0 ];thenkillall keepalivedfi[[email protected] keepalived]# chmod 755 /usr/local/keepalived/nginx.sh[[email protected] keepalived]# systemctl start keepalived[[email protected] keepalived]# ifconfig -aeno16777736: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500        inet 192.168.1.102  netmask 255.255.255.0  broadcast 192.168.1.255        inet6 fe80::20c:29ff:fe87:fd0e  prefixlen 64  scopeid 0x20<link>        ether 00:0c:29:87:fd:0e  txqueuelen 1000  (Ethernet)lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536        inet 127.0.0.1  netmask 255.0.0.0        inet6 ::1  prefixlen 128  scopeid 0x10<host>        loop  txqueuelen 0  (Local Loopback)

6.測試：通過瀏覽器訪問測試http://192.168.1.200/，可發現流量在web1和web2之間跳轉.

6.1測試關閉主nginx節點上的keepalived伺服器，發綁定的vip在主節點消失

[[email protected] keepalived]# systemctl stop keepalived.service [[email protected] keepalived]# ifconfig -aeno16777736: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500        inet 192.168.1.101  netmask 255.255.255.0  broadcast 192.168.1.255        inet6 fe80::20c:29ff:fefe:6f3  prefixlen 64  scopeid 0x20<link>        ether 00:0c:29:fe:06:f3  txqueuelen 1000  (Ethernet)lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536        inet 127.0.0.1  netmask 255.0.0.0        inet6 ::1  prefixlen 128  scopeid 0x10<host>        loop  txqueuelen 0  (Local Loopback)vip在卻在備節點上出現[[email protected] keepalived]# ifconfig -aeno16777736: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500        inet 192.168.1.102  netmask 255.255.255.0  broadcast 192.168.1.255        inet6 fe80::20c:29ff:fe87:fd0e  prefixlen 64  scopeid 0x20<link>        ether 00:0c:29:87:fd:0e  txqueuelen 1000  (Ethernet)eno16777736:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500        inet 192.168.1.200  netmask 255.255.255.0  broadcast 0.0.0.0        ether 00:0c:29:87:fd:0e  txqueuelen 1000  (Ethernet)lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536        inet 127.0.0.1  netmask 255.0.0.0        inet6 ::1  prefixlen 128  scopeid 0x10<host>        loop  txqueuelen 0  (Local Loopback)通過瀏覽器訪問測試http://192.168.1.200/，可發現流量依然在web1和web2之間跳轉。

6.2再次啟動主節點的keepalived服務，發現vip又重新漂移會主節點

[[email protected] keepalived]# systemctl start keepalived.service   [[email protected] keepalived]# ifconfig -aeno16777736: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500        inet 192.168.1.101  netmask 255.255.255.0  broadcast 192.168.1.255        inet6 fe80::20c:29ff:fefe:6f3  prefixlen 64  scopeid 0x20<link>        ether 00:0c:29:fe:06:f3  txqueuelen 1000  (Ethernet)eno16777736:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500        inet 192.168.1.200  netmask 255.255.255.0  broadcast 0.0.0.0        ether 00:0c:29:fe:06:f3  txqueuelen 1000  (Ethernet)通過瀏覽器訪問測試http://192.168.1.200/，可發現流量依然在web1和web2之間跳轉。

6.3關閉nginx主節點上的nginx服務，發現vip從主節點消失，keepalived服務關閉，vip在備節點上出現。

[[email protected] keepalived]# systemctl stop nginx.service [[email protected] keepalived]# ifconfig -aeno16777736: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500        inet 192.168.1.101  netmask 255.255.255.0  broadcast 192.168.1.255        inet6 fe80::20c:29ff:fefe:6f3  prefixlen 64  scopeid 0x20<link>        ether 00:0c:29:fe:06:f3  txqueuelen 1000  (Ethernet)lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536        inet 127.0.0.1  netmask 255.0.0.0        inet6 ::1  prefixlen 128  scopeid 0x10<host>        loop  txqueuelen 0  (Local Loopback)[[email protected] keepalived]# systemctl status keepalived keepalived.service - LVS and VRRP High Availability Monitor   Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled)   Active: inactive (dead)通過瀏覽器訪問測試http://192.168.1.200/，可發現流量依然在web1和web2之間跳轉。

 6.4再次啟動主節點的nginx和keepalived服務後，VIP又漂回主節點。

[[email protected] keepalived]# systemctl start nginx.service [[email protected] keepalived]# systemctl start keepalived通過瀏覽器訪問測試http://192.168.1.200/，可發現流量依然在web1和web2之間跳轉。

參考：

http://www.linuxdiyf.com/linux/12955.html
http://nginx.org/en/linux_packages.html
http://blog.csdn.net/e421083458/article/details/30086413
http://my.oschina.net/u/1458120/blog/208740

雙節點(nginx+keepalived)為兩台apache伺服器提供負載平衡