LVS load Balancing, Ipvs NAT mode, Dr Mode implementation

LVS is a clustering technology that uses IP load balancing technology and content-based request distribution technology. The Scheduler (Director) has a good throughput rate, transfers requests evenly to different servers, and the scheduler automatically screens out server failures, making a set of servers a high-performance, high-availability virtual server.

The command-line management tool for the LVS user space is Ipvsadm,ipvs is the hook function that works on the NetFilter input in the kernel, and the incoming messages are manipulated before entering the user space.

There are four types of LVS, namely Lvs-nat,lvs-dr,lvs-tun and Lvs-fullnat. The most commonly used are the first two (LVS-NAT,LVS-DR).

Lvs-nat: is one of the simplest ways that all realserver point their gateways to the director. The client requests the IP on the director, then the message to the director after the Dnat conversion, the request director's message is repackaged into a message, the director's IP as the source address, and then request RS (Realserver), The RS responds directly to the director and the Director sends the results to the client. Simply put: Multi-objective Dnat (iptables), which can be forwarded by modifying the destination IP address of the request message (and possibly modifying the target port) to the RIP address of the selected RS;

Structure diagram of the Lvs-nat type:

Features of Lvs-nat:

(1) RS should and dip should use the private network address, and RS gateway to point to dip;

(2) The request and response messages are forwarded through the Director; In highly loaded scenarios, the director may become a system bottleneck;

(3) Support port mapping;

(4) RS can use any operating system (OS);

(5) RS RIP and Director Dip must be on the same IP network;

Advantages: Easy to implement and easy to understand;

Disadvantage: The director will be called an optimization bottleneck, all the messages must go through the director, if the director is broken, the consequences are very serious

LVS-DR: is forwarded by modifying the destination MAC address in the request;

Features of LVS-DR:

(1) Ensure that the front-end router sends the target IP to the VIP request message to director, the back-end of the RS to be able to respond to the VIP address to the client;

Solution:

Static bindings

Arptables

Modifying the parameters of the RS host core

(2) The rip of Rs can use the private address, but also can use the public network address;

(3) RS and director must be in the same physical network;

(4) The request message is dispatched through the director, but the response message must not pass through the Director;

(5) port mapping is not supported;

(6) RS can be most OS;

(7) RS Gateway cannot point to dip;

In order for the RS to not respond to the VIP ARP response, need to configure arp_ignore=1 others to request, when not to enter from which interface, let the address of the interface to respond, if not it does not respond; arp_announce=2 only let the address of the physical network card be communicated to others In order for the RS to send the message to the client at the VIP source address, Rs rip on the physical network card, VIP on the LO (virtual device), the message out to define the lo this device, is added a route (Route add-host VIP Dev LO:0) This ensures that the source IP is the VIP when the RS responds to the client.

Lvs-tun: Is the IP tunnel, is to encapsulate an IP packet to the header of an IP packet, so that the target of one IP address of the data message can be encapsulated and forwarded to another IP address.

Lvs-fullnat: is to do Snat and Dnat when the message comes in.

The latter two are not very common.

Architecture diagram for LVS-DR

It's just a ballpark figure, dip,vip,rip the same network segment.

Dip,vip,rip the architecture diagram for different network segments is:

The following is an example of a lvs-nat:

With the physical machine as a test machine, as a client, 172.16.249.123 this machine for director, it has two network cards, a configuration 172.16.249.123 for the external network address, another piece configured to 192.168.36.1 do intranet address

192.168.36.2 and 192.168.36.3 are Web servers that are realserver, and the gateways to the two intranet hosts point to 192.168.36.1

The network type of intranet is selected as VMnet2,

After doing the test can not communicate between the intranet,

Then to open the httpd service for it, and then configure the test page, in order to demonstrate the effect we put two Web server Pages into different, (in fact, the exact same resources)

Open the HTTPD service, and then test the contents of the page as follows:

Install Ipvsadm for director, attach the local CD-ROM, and then write the points of the Yum source

Yum Install Ipvsadm

And then on the director test, whether the normal access

Ensure that the director's firewall is off

Director's core forwarding to open

and make Ipvs rules on the director.

First add the Cluster service ipvsadm-a-t 172.16.249.123:80-s rr-t 172.16.249.123:80 refers to this address (172.16.249.123) TCP protocol 80 port open Cluster service –s RR indicates the scheduler algorithm For the argument

Ipvsadm-a-T 172.16.249.123:80-s RR

Then add RS (realserver) in the Cluster service

So it's done, and then the test

And then you're under the refresh,

But your httpd service can not open long links, or you will be new several times a RS will replace one

Under Driector, you can also see that each host responds to a service

These rules use/ETC/SYSCONFIG/IPVSADM to save rules to/etc/sysconfig/ipvsadm

Ipvsadm–r </etc/sysconfig/ipvsadm overloading this rule

Examples of LVS-DR:

We use the VIP and Dip,rip in the same network segment to do, so simple

First, the planning Director's network card is eth0, using bridge mode, only one network card

The address is configured for the Dip:172.16.249.123 gateway to point 172.16.0.1;VIP onto the eth0:0 alias 172.16.249.199

RS1 network card with bridge, rip1:172.16.249.115 Gateway point to 172.16.0.1;lo:0 : VIP 172.16.249.199

RS2 network card with bridge, rip2:172.16.249.124 Gateway point to 172.16.0.1;lo:0 : VIP172.16.249.199

First, the director's other VMnet2 NIC is disconnected, then the dip is configured to 172.16.249.123,

The NIC for RS1 and RS2 is bridged, with the address 172.16.249.115 and 172.16.249.124,

And then test the ping172.16.0.1 ping172.16.249.123 from Rs. It's a proven network.

Then configure the VIP on the director to 172.16.249.199/32, the mask uses 32 bits, is to make it only for the load balancer scheduling

This time the RS host ping VIP 172.16.249.199 is through

Then you have to restrict which interface is going into the message, which interface to respond to

This configuration of the Diretor does not have to be, but the RS must be configured

Route Add-host 172.16.249.199dev eth0:0

Then set the RS

First, we need to add these two parameters

Arp_ignore=1

arp_announce=2

Then configure the VIP, and define the message from which interface to go out from which interface

Echo 1 >/proc/sys/net/ipv4/conf/all/arp_ignore

Echo 1 >/proc/sys/net/ipv4/conf/eth0/arp_ignore in order to disable thoroughly, put this one also on

Echo 2 >/proc/sys/net/ipv4/conf/all/arp_announce

Echo 2 >/proc/sys/net/ipv4/conf/eth0/arp_announce in order to disable thoroughly, put this one also on

Then configure the VIP address

Ifconfig lo:0 172.16.249.199/32 broadcast172.16.249.199 up

or ifconfigeth0:0. 172.16.249.199 netmask 255.255.255.255 broadcast 172.16.249.199 up

Route add-host 172.16.249.199 Dev lo:0

And then the other RS is also configured like this

Echo 1 >/proc/sys/net/ipv4/conf/all/arp_ignore

Echo 1 >/proc/sys/net/ipv4/conf/eth0/arp_ignore in order to disable thoroughly, put this one also on

Echo 2 >/proc/sys/net/ipv4/conf/all/arp_announce

Echo 2 >/proc/sys/net/ipv4/conf/eth0/arp_announce in order to disable thoroughly, put this one also on

Then configure the VIP address

Ifconfig lo:0 172.16.249.199/32 broadcast172.16.249.199 up

Route Add-host 172.16.249.199dev lo:0

So the RS is ready, just now the Web service has started, in order to verify that the 80 port is enabled

Then use the RIP address request on the Director to test

And then clear the previous rules on the director,

Then reconfigure the IPVSADM rule

Ipvsadm-a-T 172.16.249.199:80-s RR

Ipvsadm-a-T 172.16.249.199:80-r172.16.249.115-g

Ipvsadm-a-T 172.16.249.199:80-r172.16.249.124-g

Using the physical machine browser to access the problem can not be at any time the argument, a period of time to switch, should be a browser cache problem; Open a virtual machine test no problem

This is easy to do this problem, under the virtual machine Test no problem, to prove that the experiment is successful, is affected by the network or browser,

Below we use the VIP and Dip,rip in different network segment to do (between the physical machine and director to add a router)

First, the planning Director's network card is eth0, using bridge mode

The address is configured as dip:192.168.36.10, and the VIP is placed on the eth0:0 alias 10.1.1.2

RS1 's Nic vmnet2,rip:192.168.36.8;lo:0 : VIP 10.1.1.2

RS2 's Nic vmnet2,rip:192.168.36.9;lo:0 : VIP 10.1.1.2

The address of the physical machine is 172.16.249.100

Add a route

Route Add 10.1.1.0 Mask 255.255.255.010.1.1.1

First to have a router, open a virtual machine to open the core forwarding function, and then add two network card (also can be a network card), the type is VMnet2, the address is eth1:192.168.36.1/24, eth1:0 for 10.1.1.2/24 ; one for bridging as 172.16.249.117

The routing entry is

A virtual machine to do director, a network card, network card type is VMnet2, address eth0 is DIP192.168.36.10/24

eth0:0 to 10.1.1.2/8

To add a route to the director

Route add default GW 10.1.1.1

The IP address of RS is 192.168.36.8 and 192.168.36.9 default gateway is pointing to 192.168.36.1

So the physical network is created.

Under Ping test

The director is able to ping through the 172.16.249.110.

From the physical machine can also ping through the 10.1.1.2,

Can ping the physical machine from RS

Then start the RS 80 service and you can set up the Cluster service on the director

Test the following Web services from the Director

The Cluster service configuration is then done on the director, configured as

The two parameters are then configured in the RS

Arp_ignore=1

arp_announce=2

Then configure the VIP, and define the message from which interface to go out from which interface

Echo 1 >/proc/sys/net/ipv4/conf/all/arp_ignore

Echo 1 >/proc/sys/net/ipv4/conf/eth0/arp_ignore in order to disable thoroughly, put this one also on

Echo 2 >/proc/sys/net/ipv4/conf/all/arp_announce

Echo 2 >/proc/sys/net/ipv4/conf/eth0/arp_announce in order to disable thoroughly, put this one also on

Then configure the VIP address

Ifconfig lo:0 10.1.1.2/32 Broadcast 10.1.1.2 up

Route add-host 10.1.1.2 Dev lo:0

And then the other RS is also configured like this

Echo 1 >/proc/sys/net/ipv4/conf/all/arp_ignore

Echo 1 >/proc/sys/net/ipv4/conf/eth0/arp_ignore in order to disable thoroughly, put this one also on

Echo 2 >/proc/sys/net/ipv4/conf/all/arp_announce

Echo 2 >/proc/sys/net/ipv4/conf/eth0/arp_announce in order to disable thoroughly, put this one also on

Then configure the VIP address

Ifconfig lo:0 10.1.1.2/32 Broadcast 10.1.1.2 up

Route Add-host 10.1.1.2 devlo:0

It can be written in a script.

Then execute the script and verify

and then test it.

Look at the director and you can see that the number of connections is the same

From here can be seen, the above Dip,vip,rip in the same network segment, the physical machine browser measurement rotation, is due to the MAC address, which added a router, isolated some ARP broadcast messages, so the physical host is not affected.

So LVs Lvs-nat and LVS-DR experiment done, if have advice and suggestions can leave a message.

This article turns from: Programmer encouragement Teacher

LVS load Balancing, Ipvs NAT mode, Dr Mode implementation