This article describes how to deploy the DR mode of the experimental LVS in a KVM virtual environment. Contains the network structure diagram, how to configure. and using tcpdump to analyze IP packets.
Network structure diagram
KVM Node (client)192.168.7.2 (CIP)
|
br_e Network Bridge / \
(VIP)
eth7:192.168.7.33 eth14:192.168.7.99
LVS Virtual Host Server virtual host
(DIP) (RIP)
eth8:192.168.88.10 eth12:192.168.88.20
\ /
Br_lvs Network Bridge
Experimental Steps1. Create Bridge:br_e and Br_lvs, and set ip# KVM node separately $ ifconfig br_e 192.168.7.2 netmask 255.255.255.0
2.lvs Configuration
$ ipvsadm-a-T 192.168.7.33:8000-s RR
$ ipvsadm-a-T 192.168.7.33:8000-r 192.168.88.20-g
3.server configuration # Let the host handle DST for 192.168.7.33 IP packet # subnet mask must be 255.255.255.255, assuming 255.255.255.0 does not process IP packets
$ ifconfig lo:0 192.168.7.33 netmask 255.255.255.255 up
/etc/sysctl.conf configuration (doesn't seem to need to be set?) )Ip_forward=0 # does not need to be configured as 1rp_filter = 0 # Default configuration
# troubleshoot ARP issues
#Official documentation says LO does not need to set arp# log off no problem # Net.ipv4.conf.lo.arp_ignore = 0# Net.ipv4.conf.lo.arp_announce = 0# eth14 is the NIC on the Br_e # Only if the target IP of the ARP request and the ETH14 binding IP are the same, ETH14 will process # otherwise LVS VIP will and Server VIP conflict # in this case, br_ E Network will exist two 192.168.7.33net.ipv4.conf.eth14.arp_ignore = # for eth14 send ARP request when SRC IP? Net.ipv4.conf.eth14.arp_announce = Net.ipv4.conf.all.arp_ignore = # net.ipv4.conf.all.arp_announce = 2
4. Open Web Service on server virtual host # Default listener 8000port
$ python-m Simplehttpserver
5. On the KVM node (client)
$ Curl http://192.168.7.33:8000
tcpdump Analysis IP packet
[KVM Send:br_e]Emitting IP packets (SYN)
00:1a:a0:21:21:65 > de:ad:be:ef:24:46, 192.168.7.2.51091 > 192.168.7.33.8000:flags [S], seq 3773377896
[LVS RECV:BR_E]LVS receive IP packets
00:1a:a0:21:21:65 > de:ad:be:ef:24:46, 192.168.7.2.51091 > 192.168.7.33.8000:flags [S], seq 3773377896,
[LVs Send:br_lvs] LVS change IP packet mac, target Mac to server Mac
00:16:3e:5d:a6:b3 > 52:54:00:ba:3d:b9, 192.168.7.2.51091 > 192.168.7.33.8000:flags [S], seq 3773377896,
[Server RECV:BR_LVS] server receives IP packets (Mac same)
00:16:3e:5d:a6:b3 > 52:54:00:ba:3d:b9, 192.168.7.2.51091 > 192.168.7.33.8000:flags [S], seq 3773377896
[Server Send:br_e] Server issues IP packets (ACK). The IP packet src is the LVS IP. DST is the IP of the client
52:54:00:c1:a7:5e > 00:1a:a0:21:21:65, 192.168.7.33.8000 > 192.168.7.2.51091:flags [S.], seq 3629588945, Ack 3773377897
[KVM Recv:br_e]Client receives IP packet (ACK)
52:54:00:c1:a7:5e > 00:1a:a0:21:21:65, 192.168.7.33.8000 > 192.168.7.2.51091:flags [S.], seq 3629588945, Ack 37733 77897
Arp_ignore and Arp_announce
Arp_announce-integerDefine different restriction levels for announcing the Localsource IP address from IP packets in ARP requests sent Oninter face:0-(default) Use of any local address, configured on any interface1-try to avoid local addresses that is not in the Target ' ssubnet for this interface. This mode was useful when targethosts reachable via this interface require the source IPaddress in ARP requests to BES part Of their logical networkconfigured on the receiving interface. When we generate therequest we'll check all our subnets that include thetarget IP and would preserve the source address I F it is fromsuch subnet. If There is no such subnet we select SourceAddress according to the rules for level 2.2-always use the best local addres s for this target. In this mode we ignore the source address in the IP packetand try to select Local address so we prefer for talks withthe Target host. Such Local address is selected by Lookingfor primary IP addresses on all we subnets on the Outgoinginterface thaT include the target IP address. If No suitablelocal address is found we select the first local addresswe has on the outgoing interface or on all other in Terfaces,with the hope we'll receive reply for our request Andeven sometimes no matter the source IP address we announce .
The max value from Conf/{all,interface}/arp_announce is used.
Increasing the restriction level gives more chance forreceiving answer from the resolved target while decreasingthe level Announces more valid sender ' s information.
Arp_ignore-integerDefine different modes for sending replies on response Toreceived ARP requests that resolve local target IP addresses:0- (default): Reply for any local target IP address, configuredon any interface1-reply only if the target IP address is Loc Al addressconfigured on the incoming interface2-reply only if the target IP address was local addressconfigured on the Coming interface and both with Thesender ' s IP address is part from same subnet on the interface3-do not reply for Loca L addresses configured with scope Host,only resolutions for global and link addresses is replied4-7-reserved8-do not Reply for all local addresses
The max value from Conf/{all,interface}/arp_ignore are usedwhen ARP request is received on the {interface}
Lvs:dr mode (Direct Routing) Deployment experiment
