Detailed description of the dr mode of lvs

Detailed description of the dr mode of lvs

I. Environment Introduction

Three virtual machines are used for lvs Server Load balancer experiments. vm01 is a server Load balancer with only one Nic. vm02 and vm03 are real web servers.

vm01eth0:192.168.46.131vip:192.168.46.200vm02eth1:192.168.46.130vm03eth1:192.168.46.134

IIHow the lvs dr mode works

Official principles: ctor receives user requests, selects a realserver Based on the server Load balancer algorithm, forwards packets, and then the realserver directly replies to the user.

Instance scenario device list:

650) this. width = 650; "style =" margin: 0px; padding: 0px; "border =" 0 "alt =" instance scenario Device List "src =" http://www.bkjia.com/uploads/allimg/131228/012203FB-0.png "width =" 498 "/>

The client is a machine in the same network segment as the vip. If it is accessed by an external user, replace the client with the gateway, because the IP address header is unchanged and the source mac address is changed.

① The client sends a request to the target vip and the Director receives the request. The IP header and data frame header information are as follows:

650) this. width = 650; "style =" margin: 0px; padding: 0px; "border =" 0 "alt =" client sends a request to the target vip "src =" http://www.bkjia.com/uploads/allimg/131228/012203H36-1.png "width =" 498 "/>

② VS selects an active realserver (Suppose 192.168.57.122) based on the load balancing algorithm, takes the mac address of the NIC where the RIP is located as the target mac address, and sends it to the LAN. The IP header and data frame header information are as follows:

650) this. width = 650; "style =" margin: 0px; padding: 0px; "border =" 0 "alt =" 183324832.jpg" src = "http://images.51cto.com/files/uploadimg/20110823/183324832.jpg" width = "498"/>

③ The realserver (192.168.57.122) receives the frame in the LAN and finds that the destination IP address (VIP) matches the local IP address. Then, the packets are re-encapsulated and sent to the LAN. The IP header and data frame header information are as follows:

650) this. width = 650; "style =" margin: 0px; padding: 0px; "border =" 0 "alt =" 183345470.jpg" src = "http://www.bkjia.com/uploads/allimg/131228/0122033M8-3.jpg" width = "498"/>

④ If the client and VS are in the same network segment, the client (192.168.57.135) will receive this reply message. If the network segment is exceeded, the packets are returned to the user through the gateway/router over the Internet.

Iii. Server Load balancer Installation

IPVS is the basis of the entire server Load balancer. Without this foundation, failure isolation and Failover are meaningless.

[root@vm01 ~]# yum install -y ipvsadm[root@vm01 ~]# lsmod  |grep ip_vs[root@vm01 ~]# modprobe  ip_vs[root@vm01 ~]# lsmod  |grep ip_vsip_vs                 122241  0

Here, we cannot rely on processes to determine whether the ip_vs module is loaded, because it is a command and no process is generated.

4. Create an lvsdr Startup Script

For the command of ipvsadm, refer:

-A -- add-service adds A new virtual server record in the virtual server table of the kernel. Also
Is to add a new virtual server.
-E -- edit-service: edit a virtual server record in the kernel virtual server table.
-D -- delete-service: delete a virtual server record in the kernel virtual server table.
-C -- clear clears all records in the kernel virtual server table.
-R -- restore virtual server rules
-S -- save saves the Virtual Server rule and outputs the readable format of the-R Option
-A -- add-server: add a new real server to a record in the kernel virtual server table
Record. That is, add a new real server to a virtual server.
-E -- edit-server: edit a Real server record in a virtual server record
-D -- delete-server: delete a Real server record in a virtual server record.
-L |-l -- list displays the kernel virtual server table
-Z -- zero virtual service table counters are cleared to clear the current number of connections)
-- Settcptcpfinudp
-- Start-daemon: start the synchronization daemon. It can be followed by a master or backup.
It indicates that LVSRouter is a master or backup. You can also use the keepalived
VRRP function.
-- Stop-daemon: stop synchronization daemon
-H -- help: displays help information.
Other options:
-T -- tcp-serviceservice-address indicates that the virtual server provides tcp services.
[Vip: port] or [real-server-ip: port]
-U -- udp-serviceservice-address indicates that the virtual server provides udp services.
[Vip: port] or [real-server-ip: port]
-F -- fwmark-servicefwmark indicates the service type marked by iptables.
-S -- the scheduling algorithm used by schedulerscheduler, which has the following options:
Rr | wrr | lc | wlc | lblc | lblcr | dh | sh | sed | nq,
The default scheduling algorithm is: wlc.
-P -- persistent [timeout] persistent and stable service. This option means that it is from the same customer.
Multiple requests from the same real server will be processed. The default timeout value is 300 seconds.
-M -- netmasknetmaskpersistentgranularitymask
-R -- real-serverserver-address Real Server [real-Server: port]
-G -- gatewaying specifies that the LVS working mode is direct routing mode, which is also the default LVS Mode)
-I -- ipip: Specify the LVS working mode as tunnel mode.
-M -- masquerading: Specify the LVS working mode as NAT
-W -- weightweight: actual server weight
-- Mcast-interfaceinterface: Synchronous interface for Multicast
-C -- connection displays the current connection of LVS, for example, ipvsadm-L-c.
-- Timeout: displays the timeout value of tcptcpfinudp, for example, ipvsadm-L -- timeout.
-- Daemon: displays the synchronization daemon status.
-- Stats display statistics
-- Rate: Display rate information
-- Sort sorts and outputs virtual servers and real servers
-- Numeric-n numeric form of output IP address and port

[root@vm01 ~]# vim  /etc/init.d/lvsdr#!/bin/bashVIP=192.168.46.200RIP1=192.168.46.130RIP2=192.168.46.134case "$1" instart)echo "start LVS of DirectorServer DR"/sbin/iptables -F/sbin/ipvsadm -C/sbin/ifconfig eth2:0 $VIP  broadcast $VIP  netmask 255.255.255.0  up/sbin/ipvsadm  -A -t $VIP:80  -s rr/sbin/ipvsadm  -a -t $VIP:80  -r  $RIP1 -g/sbin/ipvsadm  -a -t $VIP:80  -r  $RIP2 -g/sbin/ipvsadm;;stop)echo "stop  LVS of DirectorServer DR"echo "0" >/proc/sys/net/ipv4/ip_forward/sbin/ipvsadm -C/sbin/ifconfig eth0:0 down;;*)echo "argements error";;esac[root@vm01 ~]#chown +x  /etc/init.d/lvsdr

V. configuration on vm02 and vm03

#vim /etc/init.d/lvsdrrip#!/bin/bash#DR serverVIP=192.168.46.200case "$1" instart)echo "start LVS of DR"/sbin/ifconfig lo:0 $VIP broadcast $VIP netmask 255.255.255.255 up/sbin/route add  -host  $VIP dev lo:0echo "1" > /proc/sys/net/ipv4/conf/lo/arp_ignoreecho "2" > /proc/sys/net/ipv4/conf/lo/arp_announceecho "1" > /proc/sys/net/ipv4/conf/all/arp_ignoreecho "2" > /proc/sys/net/ipv4/conf/all/arp_announce;;stop)echo "stop LVS of DR"/sbin/ifconfig lo:0 downecho "0" > /proc/sys/net/ipv4/conf/lo/arp_ignoreecho "0" > /proc/sys/net/ipv4/conf/lo/arp_announceecho "0" > /proc/sys/net/ipv4/conf/all/arp_ignoreecho "0" > /proc/sys/net/ipv4/conf/all/arp_announce;;*)echo "Usage:$0 {start|stop}"esacexit 1

Note: The VIP address in DR mode must be in the same network segment as the server.

Echo "1"; echo "2" is used to suppress arp broadcast.

# Chmodo + x/etc/init. d/lvsdr

# Servicelvsdrripstart

Vi. Test

[root@vm01~]#ipvsadm  -lncIPVS  connection     entriespro expire state source  virtual  destinationTCP01:56 FIN_WAIT  192.168.1.203:35952  192.168.1.30:80 192.168.1.202:80TCP01:44 FIN_WAIT  192.168.1.203:35951  192.168.1.30:80 192.168.1.201:80

We can see that these connections are evenly allocated to two hosts.

650) this. width = 650; "title =" c00001zdj87l86gii@gjecg1kk.jpg "alt =" 211914237.jpg" src = "http://www.bkjia.com/uploads/allimg/131228/01220321I-4.jpg" width = "650"/>

7. Questions about the dr mode of lvs

1. How does LVS/DR process request packets and modify the IP packet content?

1.1vs/dr itself does not care about the information above the IP layer, even if the port number is also the TCP/IP protocol stack to determine whether the correct, vs/dr itself mainly do the following:

1) receive client requests and select the ip address of a realserver Based on the server Load balancer algorithm you set;

2) use the mac address corresponding to the selected ip address as the target mac address, and then encapsulate the ip package into a frame and forward it to the RS;

3) record the connection information in hashtable.

Vs/dr does very few things and is also very simple, so it is very efficient, not much worse than the Hardware load balancing device.

The general flow of data packets and data frames is as follows: client --> VS --> RS --> client

1.2 The answer was answered earlier. vs/dr will not modify the content of the IP package.

2. Why does RealServer configure VIP on the lo interface? Can I configure the VIP address on the egress Nic?

2.1 To enable RS to process IP packets whose destination address is vip, RS must first receive the packet.

Configure the vip address on lo to receive the packet and return the result to the client.

2.2 The answer is that the VIP cannot be set on the egress Nic. Otherwise, the client's arprequest will be responded, resulting in disorder of client/gatewayarptable, and the entire loadbalance will not work properly.

3. Why does RealServer suppress arp frames?

What is arp?

We know that when we enter a URL in the browser, the DNS server automatically resolves it as an IP address. The browser actually looks for an IP address instead of a URL. So how does the IP address be converted to the second physical address, that is, the MAC address? In Lan, this is done through ARP. ARP is of great significance to network security. ARP spoofing is achieved by forging IP addresses and MAC addresses, which can generate a large amount of ARP traffic in the network to block the network. Therefore, network administrators should have an in-depth understanding of the ARP protocol.

I. What is ARP?

ARP is the abbreviation of AddressResolutionProtocol. In the LAN, the actual transmission is frame, and the frame contains the MAC address of the target host. In Ethernet, to directly communicate with another host, you must know the MAC address of the target host. But how can I obtain the target MAC address? It is obtained through the Address Resolution Protocol. The so-called "Address Resolution" refers to the process in which the host converts the target IP address to the target MAC address before sending the frame. The basic function of ARP is to query the MAC address of the target device through the IP address of the target device to ensure smooth communication.

Ii. Working Principles of ARP
Each computer with TCP/IP protocol installed has an ARP cache table. The IP addresses in the table correspond to the MAC addresses one by one, as shown in the appendix.

Appendix

Take host A192.168.1.5) as an example to send data to host B192.168.1.1. When sending data, host A searches for the target IP address in its ARP cache table. If you find the target MAC address, you can directly write the target MAC address into the frame and send it. If the corresponding IP address is not found in the ARP cache table, host A sends A broadcast on the network. The target MAC address is "FF. FF. FF. FF. FF. FF ", which means to send such a question to all hosts in the same network segment:" What is the MAC address of 192.168.1.1?" Other hosts on the network do not respond to ARP requests. Only when host B receives the frame, it responds to host A as follows: "the MAC address of 192.168.1.1 is 00-aa-00-62-c6-09 ". In this way, host A knows the MAC address of host B and can send information to host B. At the same time, it also updates its ARP cache table. The next time it sends a message to host B, it can directly find it from the ARP cache table. The ARP cache table adopts an aging mechanism. If a row in the table is not used for a period of time, it will be deleted. This can greatly reduce the length of the ARP cache table and speed up query.

Iii. View ARP cache tables

ARP cache tables can be viewed, added, and modified. At the command prompt, enter "arp-a" to view the content in the ARP cache table, as shown in the figure below.

The "arp-d" command can be used to delete a row in the ARP table. The "arp-s" command can be used to manually specify the corresponding IP address and MAC address in the ARP table.

This issue has been explained in the previous issue. Here we will further discuss it with the Implementation command. We will make the following adjustments during implementation and deployment:

echo"1">/proc/sys/net/ipv4/conf/lo/arp_ignoreecho"2">/proc/sys/net/ipv4/conf/lo/arp_announceecho"1">/proc/sys/net/ipv4/conf/all/arp_ignoreecho"2">/proc/sys/net/ipv4/conf/all/arp_announce

I believe that many people will not understand what their role is, but they must know that they must. I am not going to discuss it in detail here. I just want to make a few notes to add.

3.1

echo"1">/proc/sys/net/ipv4/conf/lo/arp_ignoreecho"2">/proc/sys/net/ipv4/conf/lo/arp_announce

You can skip these two steps because arp has no significance for logical interfaces.

3.2 If your RS's external network interface is eth0

echo"1">/proc/sys/net/ipv4/conf/all/arp_ignoreecho"2">/proc/sys/net/ipv4/conf/all/arp_announce

What is actually to be executed is:

echo"1">/proc/sys/net/ipv4/conf/eth0/arp_ignoreecho"2">/proc/sys/net/ipv4/conf/eth0/arp_announce

Therefore, I personally suggest adding the above two to your script, because in case the default values of the above two are not 0, there may be problems.

Arp_ignore:

Define ARP queries with the target IP address as the local IP address in different response modes 0

0-(default): responds to arp query requests from any network interface to any local IP address.

1-only answers ARP query requests whose target IP address is the local address of the Access Network Interface

2-only answers ARP query requests whose target IP address is the local address of the access network interface. The access IP address must be in the subnet segment of the network interface.

3-do not return arp requests in the network, but only respond to the set unique and connection address

4-7-reserved unused

8-does not respond to all local addresses) arp queries

Determines the IP address that sends the ARP request to the outside even if the VIP address

Arp_announce-INTEGER

When an ARP response is sent from a local IP address on a network interface, the following restrictions are imposed:

Determine the limits to different degrees, and announce the interface for sending Arp requests to local IP addresses

0-(default) any local address on any network interface eth0, eth1, lo)

1-avoid arp responses from local addresses that are not in the subnet segment of the network interface. it is useful when the source IP address that initiates an ARP request is set to reach this network interface through a route. check whether the access IP address is one of the ip addresses in the subnet segment of all interfaces. if the access IP address does not belong to the subnet segment of each network interface, level 2 is used for processing.

2-use the most appropriate local address for the query target. in this mode, the source address of the IP packet is ignored and the local address that can communicate with the IP packet is selected. first, select the local address of the destination IP address in the out-of-the-box access subnet of all network interfaces. if no suitable address is found, the current sending network interface or other network interfaces that may receive the ARP response will be selected for sending.

Limits the use of the local vip address as the preferred Network Interface

4. Why are LVS/DRloadbalancerdirector and RS in the same network segment?

From the first question, we should understand how vs/dr forwards requests to RS? It is implemented at the data link layer, so director must be in the same network segment as RS.

5. Why should I configure an ip address (DIP) in addition to the VIP address for the eth0 interface on ctor )?

5.1 if keepalived or other tools are used for HA or LoadBalance, DIP is required for health check.

5.2 HA or LoadBalance without health check mechanism has no practical significance.

6. Do I need to enable LVS/DRip_forward?

No. Because director and realserver are in the same network segment, you do not need to enable forwarding.

7. Must the director vip netmask be 255.255.255.255?

In lvs/dr, the director vip netmask does not need to be set to 255.255.255.255, and does not need to be set again.

routeadd-host$VIPdeveth0:0

Director's vip is intended to be advertised as a normal IP address. Do not make such a special announcement.

This article from "Good to live" blog, please be sure to keep this source http://wolfword.blog.51cto.com/4892126/1217585