LVS-NAT && LVS-DR模型

標籤：lvs-dr   lvs-nat   模型   

Lvs三種模式

         NAT

         DR

         TUN

 

LVS-NAT：目標地址轉換

         注意事項：

         realserver的網關必須指向DIP，通常都是私人地址

         directory位於client和realserver之間，並負責處理進出的所有通訊

         directory支援連接埠映射，向外提供服務的連接埠 和realserver的連接埠可以不相同

         realserver可以使用任意作業系統

         較大規模應用情境中，directory易成為系統瓶頸

LVS-DR：直接路由

         各叢集節點，必須要跟directory在同一物理網路中

         RIP可以使用公網地址，實現遠端管理（也可以使用私人地址）

         directory僅負責處理入展請求，響應報文則有realserver 直接發往用戶端

         叢集節點不能將網關指向DIP

         directory不支援連接埠映射

LVS-TUN：隧道

         叢集節點可以跨越互連網internet

         RIP必須是公網地址

         directory僅處理入展請求，響應報文則由realserver直接發往用戶端

         realserver網關不能指向directory

         只有支援隧道功能的OS才能用於realserver

         不支援端映射

拓撲如下：

         LVS-NAT

650) this.width=650;" src="http://s3.51cto.com/wyfs02/M01/33/31/wKioL1Oqi6-D7cIJAADwu4uolyA602.jpg" title="1.png" alt="wKioL1Oqi6-D7cIJAADwu4uolyA602.jpg" />

第一：確保兩台web可以正常訪問

第二：網路連接正常（Directory 兩塊網卡，NAT串連真機，vmnet2 串連兩台WEB server）

第三：配置LVS Directory

         #grep -i ‘vs‘ /boot/config-2.6.32-431.el6.x86_64

         說明已經支援ipvs

         #yum list | grep ipvsadm

         #yum install ipvsadm #安裝ipvsadm工具

         定義叢集服務：

         #ipvsadm -A -t 172.16.100.100:80 -s rr

         添加叢集主機：

         ipvsadm-a -t 172.16.100.100:80 -r 10.0.0.2 –m

         ipvsadm-a -t 172.16.100.100:80 -r 10.0.0.3 –m

         開啟Directory的路由轉寄功能，所有的主機均關閉防火牆。

         個主機之間使用ntpdate進行實踐同步

# ipvsadm -L -n

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

 -> RemoteAddress:Port          Forward Weight ActiveConn InActConn

TCP 172.16.100.100:80 rr

 -> 10.0.0.2:80                 Masq    1      0         5        

 -> 10.0.0.3:80                 Masq    1      0         5        

[[email protected] ~]#

可以看到訪問統計資訊

# ipvsadm -L --stats

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port               Conns   InPkts OutPkts  InBytes OutBytes

 -> RemoteAddress:Port

TCP 172.16.100.100:http               15       81       81    7392     8045

 -> 10.0.0.2:http                      7       35       35    3406     3832

 -> 10.0.0.3:http                      8       46       46    3986     4213

[[email protected] ~]#

Ipvs規則的儲存：

ipvsadm -S > /etc/ipvs.conf

載入配置：

ipvsadm -R < /etc/ipvs.conf #下次啟動的話，加入到開機啟動即可

 

 

LVS-DR：直接路由

650) this.width=650;" src="http://s3.51cto.com/wyfs02/M01/33/36/wKioL1Oqi-nyMgVQAAJkdvIeEyw949.jpg" title="2.png" alt="wKioL1Oqi-nyMgVQAAJkdvIeEyw949.jpg" />

Directory配置VIP

# cat /etc/sysconfig/network-scripts/ifcfg-eth0:0

DEVICE=eth0:0

TYPE=Ethernet

ONBOOT=yes

NM_CONTROLLED=yes

BOOTPROTO=none

IPADDR=172.16.100.200

NETMASK=255.255.255.0

GATEWAY=172.16.100.2

IPV6INIT=no

USERCTL=no

[[email protected] ~]#

配置路由：

# route add -host 172.16.100.200 dev eth0:0

RS伺服器上千萬記得不能先配置VIP地址，必須等關掉arp廣播才可以配置

配置RS1伺服器，首先關閉ARP報文功能：

# sysctl -w net.ipv4.conf.eth0.arp_announce=2

# sysctl -w net.ipv4.conf.all.arp_announce=2

# sysctl -w net.ipv4.conf.eth0.arp_ignore=1

# sysctl -w net.ipv4.conf.all.arp_ignore=1

或者

echo 1 >/proc/sys/net/ipv4/conf/all/arp_ignore

這樣的形式都可以的

配置VIP地址：

#ifconfig lo:0 172.16.100.200 broadcast 172.16.100.200 netmask 255.255.255.255 up

# route add -host 172.16.100.200 dev lo:0

配置RS2伺服器：

# sysctl -w net.ipv4.conf.eth0.arp_announce=2

# sysctl -w net.ipv4.conf.all.arp_announce=2

# sysctl -w net.ipv4.conf.eth0.arp_ignore=1

# sysctl -w net.ipv4.conf.all.arp_ignore=1

#ifconfig lo:0 172.16.100.200 broadcast 172.16.100.200 netmask 255.255.255.255 up

# route add -host 172.16.100.200 dev lo:0

 

LVS-Directory配置：

# ipvsadm -A -t 172.16.100.200:80 -s wlc

# ipvsadm -a -t 172.16.100.200:80 -r172.16.100.101 -g -w 2

# ipvsadm -a -t 172.16.100.200:80 -r172.16.100.102 -g -w 1

# ipvsadm -L -n

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

 -> RemoteAddress:Port          Forward Weight ActiveConn InActConn

TCP 172.16.100.200:80 wlc

 -> 172.16.100.101:80           Route   2      0         0        

 -> 172.16.100.102:80           Route   1      0         0         

[[email protected] ~]#

通過瀏覽器訪問：

http://172.16.100.200就可以訪問到RS1 和RS2 兩台伺服器

這裡提供兩個指令碼：

Directory的指令碼：

#!/bin/bash

#

# LVS script for VS/DR

#

. /etc/rc.d/init.d/functions

#

VIP=172.16.100.200

RIP1=172.16.100.101

RIP2=172.16.100.102

PORT=80

 

#

case "$1" in

start)          

 

 /sbin/ifconfig eth0:1 $VIP broadcast $VIP netmask 255.255.255.255 up

 /sbin/route add -host $VIP dev eth0:1

 

# Since this is the Director we must beable to forward packets

 echo 1 > /proc/sys/net/ipv4/ip_forward

 

# Clear all iptables rules.

 /sbin/iptables -F

 

# Reset iptables counters.

 /sbin/iptables -Z

 

# Clear all ipvsadm rules/services.

 /sbin/ipvsadm -C

 

# Add an IP virtual service for VIP192.168.0.219 port 80

# In this recipe, we will use theround-robin scheduling method.

# In production, however, you should use aweighted, dynamic scheduling method.

 /sbin/ipvsadm -A -t $VIP:80 -s wlc

 

# Now direct packets for this VIP to

# the real server IP (RIP) inside thecluster

 /sbin/ipvsadm -a -t $VIP:80 -r $RIP1 -g -w 1

 /sbin/ipvsadm -a -t $VIP:80 -r $RIP2 -g -w 2

 

 /bin/touch /var/lock/subsys/ipvsadm &> /dev/null

;;

 

stop)

# Stop forwarding packets

 echo 0 > /proc/sys/net/ipv4/ip_forward

 

# Reset ipvsadm

 /sbin/ipvsadm -C

 

# Bring down the VIP interface

 /sbin/ifconfig eth0:1 down

 /sbin/route del $VIP

 

 /bin/rm -f /var/lock/subsys/ipvsadm

 

 echo "ipvs is stopped..."

;;

 

status)

  if[ ! -e /var/lock/subsys/ipvsadm ]; then

   echo "ipvsadm is stopped ..."

 else

    echo"ipvs is running ..."

   ipvsadm -L -n

  fi

;;

*)

 echo "Usage: $0 {start|stop|status}"

;;

esac

 

RS指令碼：

#!/bin/bash

#

# Script to start LVS DR real server.

# description: LVS DR real server

#

. /etc/rc.d/init.d/functions

 

VIP=172.16.100.200

host=`/bin/hostname`

 

case "$1" in

start)

      # Start LVS-DR real server on this machine.

       /sbin/ifconfig lo down

       /sbin/ifconfig lo up

       echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore

       echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce

       echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore

       echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce

 

       /sbin/ifconfig lo:0 $VIP broadcast $VIP netmask 255.255.255.255 up

       /sbin/route add -host $VIP dev lo:0

 

;;

stop)

 

       # Stop LVS-DR real server loopback device(s).

       /sbin/ifconfig lo:0 down

       echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore

       echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce

       echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore

        echo 0 >/proc/sys/net/ipv4/conf/all/arp_announce

 

;;

status)

 

       # Status of LVS-DR real server.

       islothere=`/sbin/ifconfig lo:0 | grep $VIP`

       isrothere=`netstat -rn | grep "lo:0" | grep $VIP`

       if [ ! "$islothere" -o ! "isrothere" ];then

           # Either the route or the lo:0 device

           # not found.

           echo "LVS-DR real server Stopped."

       else

           echo "LVS-DR real server Running."

       fi

;;

*)

           # Invalid entry.

           echo "$0: Usage: $0 {start|status|stop}"

           exit 1

;;

esac

 

 

 

使用以下指令碼在Directory上進行主機的健全狀態檢查： 

#!/bin/bash

#

VIP=172.16.100.200

CPORT=80

FAIL_BACK=127.0.0.1

RS=("172.16.100.101""172.16.100.102")

declare-a RSSTATUS

RW=("2""1")

RPORT=80

TYPE=g

CHKLOOP=3

LOG=/var/log/ipvsmonitor.log

 

addrs(){

  ipvsadm -a -t $VIP:$CPORT -r $1:$RPORT -$TYPE-w $2

  [ $? -eq 0 ] && return 0 || return 1

}

 

delrs(){

  ipvsadm -d -t $VIP:$CPORT -r $1:$RPORT

  [ $? -eq 0 ] && return 0 || return 1

}

 

checkrs(){

  local I=1

  while [ $I -le $CHKLOOP ]; do

    if curl --connect-timeout 1 http://$1&> /dev/null; then

      return 0

    fi

    let I++

  done

  return 1

}

 

initstatus(){

  local I

  local COUNT=0;

  for I in ${RS[*]}; do

    if ipvsadm -L -n | grep"$I:$RPORT" && > /dev/null ; then

      RSSTATUS[$COUNT]=1

    else

      RSSTATUS[$COUNT]=0

    fi

  let COUNT++

  done

}

 

initstatus

while:; do

  let COUNT=0

  for I in ${RS[*]}; do

    if checkrs $I; then

      if [ ${RSSTATUS[$COUNT]} -eq 0 ]; then

         addrs $I ${RW[$COUNT]}

         [ $? -eq 0 ] &&RSSTATUS[$COUNT]=1 && echo "`date +‘%F %H:%M:%S‘`, $I isback." >> $LOG

      fi

    else

      if [ ${RSSTATUS[$COUNT]} -eq 1 ]; then

         delrs $I

         [ $? -eq 0 ] &&RSSTATUS[$COUNT]=0 && echo "`date +‘%F %H:%M:%S‘`, $I isgone." >> $LOG

      fi

    fi

    let COUNT++

  done

  sleep 5

done

LVS的持久化串連：

在基於SSL，需要用到持久串連

PPC:將來自同於一個用戶端對同一個叢集的請求，始終定向值前端選定的RS（持久連接埠端串連）

PCC:將來自同一用戶端對所有連接埠的請求，始終定向至此前選定的RS（持久用戶端串連）

把所有連接埠統統定義為叢集服務，所有請求指向RS

PNMPP：持久防火牆標記串連。

PCC

# ipvsadm –C

# iptables –t mangle –A PREROUTING –d  172.16.100.200 –i eth0 –p tcp –dport 80 –j MARK–set-mark 8

# iptables –t mangle –A PREROUTING –d  172.16.100.200 –i eth0 –p tcp –dport 443 –j MARK–set-mark 8

# ipvsadm –A –f 8 –s rr –p 600

# ipvsadm –a –f 8 –r 172.16.100.101 –g –w 2

# ipvsadm –a –f 8 –r 172.16.100.102 –g –w 2

 

本文出自 “Sword Slave” 部落格，請務必保留此出處http://diudiu.blog.51cto.com/6371183/1430812