IPVS-DR precautions

This article mainly explains centos5.x LVs Dr Mode, environment construction, theoretical knowledge, please refer to LVS Chinese official website

Http://www.linuxvirtualserver.org/zh/index.html

This page may show garbled, if there is garbled, right-click on the status bar on IE, and then select the encoding in the page bar dropdown options, there (GB2312 and UTF-8), if UTF-8 display garbled, then select GB2312. If GB2312 display garbled, select UTF-8.

1. LVS Environment Group Network

Because the VM environment is used, the PC as the client (that is, the public address), the scheduler and the server are VMS in the VM, so the customer and all the servers are physically connected to the network, the logical topology diagram is as follows:

2, Ipvsadm pre-installation Preparation

External IP Address	Internal IP Address	Role
10.0.0.30/24	192.168.1.1/24	LVS Scheduler (virtual machine)
10.0.0.237/24	192.168.1.2/24	RS1 (virtual machine)
10.0.0.238/24	192.168.1.3/24	RS2 (virtual machine)
10.0.0.130/24	No	Client (own computer)

Note: The actual application of RS1 RS2 No external address, in the current environment, there is no physical dual network card environment, a network card is used to configure multiple IP addresses.

3, httpd and Ipvsadm download

1) Modify the host name on each server:

[Email protected] ~]# hostname LVS1

[Email protected] ~]# hostname RS1

[Email protected] ~]# hostname RS2

2) Install the HTTPD service on RS1 RS2

[email protected] ~]# Yum installhttpd–y

[[email protected] ~]#/etc/init.d/httpd start

[Email protected] ~# PS-EF|GREPHTTPD

[email protected] ~]# Yum installhttpd–y

[Email protected] ~]#/etc/init.d/httpdstart

[[email protected] ~# ps-ef|grephttpd<== See if httpd service is up

[[email protected] ~]# cat/etc/httpd/conf/httpd.conf|grep documentroot <== Find the System Site Directory path

# documentroot:the Directoryout of which you'll serve your

DocumentRoot "/var/www/html"

# This should is changed towhatever you set DocumentRoot to.

# documentroot/www/docs/dummy-host.example.com

[Email protected] ~]# echo "RS1" >/var/www/html/index.html

[Email protected] src]# echo "RS2" >/var/www/html/index.html

HTTPD Service test:

Test success!

3) Download Ipvsadm

wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.24.tar.gz

wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.26.tar.gz

I use the source package download directory/usr/local/src

My LVS Environment:

[Email protected] src]# cat/etc/redhat-release

CentOS Release 5.8 (Final)

[Email protected] src]# UNAME-RM

2.6.18-308.el5 x86_64

ipvsadm-1.26 requirements for the kernel, requires kernel version in 2.6.28 and later versions, if the system environment is met, but also to install the dependency package yum install libnl*popt*-y.

My kernel version is not satisfied, use ipvsadm-1.24.

[Email protected] src]# tar-zxfipvsadm-1.24.tar.gz

[[email protected] src]# ln-s/usr/src/kernels/2.6.18-348.12.1.el5-x86_64/usr/src/linux<== compiled with a path to/usr/src/ Linux, which can be viewed in makefile, fails to compile if no link is made

[Email protected] src]# ll/usr/src/

Total 16

Drwxr-xr-x 2 root root 4096 May11 Debug

Drwxr-xr-x 3 root root 4096Aug 5 03:53 kernels

lrwxrwxrwx1 root root 5 03:56 Linux->/usr/src/kernels/2.6.18-348.12.1.el5-x86_64<== to use ll/usr/src/view if the soft chain After a flash, the link failed.

[Email protected] src]# CD ipvsadm-1.24

[[email protected] ipvsadm-1.24]# Make&&make Install <== If the compilation fails, follow the steps above to debug the error.

[[email protected] ipvsadm-1.24]# lsmod |grepip_vs <== See if the kernel module has IPVSADM

[Email protected] ipvsadm-1.24]# Ipvsadm

IP Virtual Server version 1.2.1 (size=4096)

Prot Localaddress:portscheduler Flags

Remoteaddress:port Forward Weight activeconn inactconn

#还有一种加载ipvsadm方法, with Modprobe Ip_vs.

[[email protected] ipvsadm-1.24]# lsmod |grepip_vs <== View kernel module, there is a representative ipvsadm loaded into the kernel, when the LVS installation is complete

Ip_vs 122113 0

4, LVS load Balancing configuration

1) Configure VIP and Dip

IP Configuration Reference Step 2 installation Preparation

[Email protected] ~]# ifconfig eth010.0.0.30 netmask 255.255.255.0 #VIP

[Email protected] ~]# ifconfigeth0:1 192.168.1.1 netmask 255.255.255.0 #DIP

2) Add Real server

[[email protected] ~]# ipvsadm–c <== manually empty the original table contents

[[email protected] ~]# Ipvsadm--set30 5 <== Set Connection timeout

[[email protected] ~]# ipvsadm-a-t10.0.0.30:80-s wrr-p + <==-A add address,-T specify VIP TCP port,-s specify scheduling algorithm –p session hold Time

[[email protected] ~]# ipvsadm-l–n <== View table contents, added a set of VIP addresses and ports

IP Virtual Server version 1.2.1 (size=4096)

Prot Localaddress:portscheduler Flags

Remoteaddress:port Forward Weight activeconn inactconn

TCP 10.0.0.30:80 WRR Persistent 20

[[email protected] ~]# ipvsadm-a-t10.0.0.30:80-r 192.168.1.2:80-g-W 1 <==-A specify real server, on-t LVS vip,-r Real Server IP and port,-w weight Value,-G First select Dr Mode (-M for NAT mode)

[Email protected] ~]# ipvsadm-a-t10.0.0.30:80-r 192.168.1.3:80-g-W 1

[[email protected] ~]# ipvsadm-l–n <== View table contents, add two real servers with load forwarding

IP Virtual Server version 1.2.1 (size=4096)

Prot Localaddress:portscheduler Flags

Remoteaddress:port Forward Weight activeconn inactconn

TCP 10.0.0.30:80 WRR Persistent 20

-192.168.1.2:80 Route 1 0 0

-192.168.1.3:80 Route 1 0 0

If the add server address is wrong, you can use the following command to delete, that is, large a added to large d Delete, small a add to small d delete

Ipvsadm–d–t10.0.0.30:80–s WRR

Ipvsadm–d–t10.0.0.30:80–r 192.168.1.2:80

5. Real Web server configuration and ARP suppression

[[email protected] ~]# ifconfig eth0|grep "inet addr" <== This address is used to log on to your computer with the CRT software, but also used to send ARP request client Mac (because it is directly returned to the client page, And the current network is the same physical network segment, must be the same network segment is normal).

inet addr:10.0.0.237 bcast:10.0.0.255 mask:255.255.255.0

[[email protected] ~]# ifconfig eth0:1192.168.1.2 netmask 255.255.255.0 <== Configure internal IP address

[[email protected] ~]# ifconfig lo10.0.0.30 netmask 255.255.255.255 <== Each real server needs to configure VIP

Why do you want to bind a VIP? Because the DR Mode principle is the LVS scheduler to modify the MAC address for the real server address, the purpose of the message IP or VIP has not been modified, the network card will only be the destination Mac and destination IP are their own messages sent to the CPU, so the real server must be configured vip!

Why use the LO port? Prevent local LAN IP address collisions because the LO port is a local loopback port!

[Email protected] src]# ifconfigeth0|grep "inet addr" <== This address is used to log on to your computer with the CRT software, but also used to send ARP request client Mac (because it is directly returned to the client page, And the current network is the same physical network segment, must be the same network segment is normal).

inet addr:10.0.0.238 bcast:10.0.0.255 mask:255.255.255.0

[[email protected] src]# ifconfig eth0:1192.168.1.3 netmask 255.255.255.0 <== Configure internal IP address

[Email protected] src]# ifconfig lo10.0.0.30 netmask 255.255.255.255

At this point, the configuration is complete, just a problem when the test, once again landed in the CRT 10.0.0.30 address, landing to RS1 up, and then refreshed, landed on the RS2 up. How to solve it?

Through analysis, it is concluded that when the 10.0.0.30 is accessed, the ARP request 10.0.0.30 Mac is sent, and there are 3 10.0.0.30 addresses in this network, at this time, who gives the ARP response to the client, who is visiting. If RS1 's ARP responds first, it accesses the RS1.

Workaround:

Suppress ARP Response

This suppression is only configured on the real server

[Email protected] ~]# echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore

[Email protected] ~]# echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce

[Email protected] ~]# echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore

[Email protected] ~]# echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce

[Email protected] src]# echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore

[Email protected] src]# echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce

[Email protected] src]# echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore

[Email protected] src]# echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce

1 2 What are the content, can be viewed in the final appendix, you can also search the Web

6. LVS Load Balancing test

The value shown is RS1, at which point the link is assigned to 192.168.1.2

We are using the WRR scheduling algorithm, the algorithm, whose weight value is high, will be prioritized to whom

View the configuration and find the same weight value

[Email protected] ~]# ipvsadm-l-N

IP Virtual Server version 1.2.1 (size=4096)

Prot Localaddress:portscheduler Flags

Remoteaddress:port forwardweight Activeconn Inactconn

TCP 10.0.0.30:80 WRR Persistent 20

-192.168.1.2:80 Route 1 0 0

-192.168.1.3:80 Route 1 0 0

Due to the session hold function, the weight values are then modified

[[email protected] ~]# ipvsadm-d-t10.0.0.30:80-r 192.168.1.3:80 <== Delete original configuration

[[email protected] ~]# ipvsadm-a-t10.0.0.30:80-r 192.168.1.3:80-g-W 2 <== reconfigure weight value to 2

[Email protected] ~]# ipvsadm-l-N

IP Virtual Server version 1.2.1 (size=4096)

Prot Localaddress:portscheduler Flags

Remoteaddress:port forwardweight Activeconn Inactconn

TCP 10.0.0.30:80 WRR Persistent 20

-192.168.1.3:80 Route 2 0 0

-192.168.1.2:80 Route 1 0 0

To test again:

7, attached: ARP suppression parameters

ARP Response Limit

1) Arp_ignore:

Define a different answer mode for ARP that has a destination address of local IP 0

0-(default): Responds to ARP query requests for any local IP address on any network interface

1-Answer only the destination IP address is the ARP query request that visited the local address of the network interface

2-Answer only the destination IP address is the ARP query request that accesses the local address of the network interface, and the visiting IP must be within the subnet segment of the network interface

3-Do not respond to ARP requests from the network interface, but only to the unique and connected addresses of the settings

4-7-Leave unused

8-ARP queries that do not respond to all (local addresses)

2) Arp_announce:

On the network interface, the local IP address of the emitted, ARP response, to make the appropriate level of restriction: to determine the different degrees of limitation, announcing the interface from the local source IP address to make an ARP request

0-(default) Any local address on any network interface (ETH0,ETH1,LO)

1-try to avoid an ARP response that does not take the local address of the network interface subnet segment. It is useful when the source IP address of the originating ARP request is set to reach this network interface through a route. This checks whether the visiting IP is one of the IP within the subnet segment on all interfaces. If the IP is not in the subnet segment of each network interface, then the Level 2 is used for processing.

2-Use the most appropriate local address for the query target. In this mode, the source address of this IP packet is ignored and an attempt is to select a local address that can communicate with that address. First, select the local address in the subnet of all network interfaces that contains the destination IP address in the subnet. If no appropriate address is found, the current send network interface or other network interface that is likely to receive the ARP response will be selected for sending.

Attention:

IPVS-DR precautions