A detailed analysis of the LVS principle in cluster load technology

About LVS
The rapid growth of the internet has led to a rapid increase in the number of access to multimedia Web servers and the ability of servers to provide a large number of concurrent access services, so the CPU, I/O processing capacity can quickly become a bottleneck for large-load servers. Because the performance of a single server is always limited, simply improving hardware performance does not really solve the problem. For this reason, multi-server and load-balancing techniques must be used to meet the needs of a large number of concurrent accesses. Linux Virtual Server (SERVERS,LVS) uses load balancing technology to make multiple servers a virtual server. It provides an easy-to-scale, affordable solution for adapting to fast-growing network access requirements

LVS structure and working principle
I. Structure of the LVS
LVS consists of the front-end load balancer (load balancer,lb) and the real-world server (real server,rs) of the backend. RS can be connected via a local area network or wide area network. This structure of LVS is transparent to the user, and the user sees only one virtual server as lb, and the RS group that provides the service is not visible. When a user's request is sent to the virtual server, LB forwards the user request to Rs according to the set packet forwarding policy and the load balancing scheduling algorithm. RS then returns the user request result to the user.

Two. LVS Kernel model

650) this.width=650; "src=" Http://s5.51cto.com/wyfs02/M00/83/69/wKioL1dzH2ehi8LfAACTrnkUSNA440.jpg "title=" 001. JPG "alt=" wkiol1dzh2ehi8lfaactrnkusna440.jpg "/>

1. When the client's request reaches the load balancer's kernel space, it first reaches the prerouting chain. 2. When the kernel discovers that the destination address of the request packet is local, send the packet to the input chain. 3.LVS consists of the ipvsadm of the user space and the Ipvs of the kernel space, Ipvsadm is used to define the rules, Ipvs uses the Ipvsadm defined rules work, IPVS works on the input chain, and when the packet reaches the input chain, it is first Ipvs checked. If the destination address and port in the packet are not inside the rule, then the packet will be released to the user space. 4. If the destination address and port inside the packet are within the rule, then this data message will be modified to the destination address as a pre-defined backend server and sent to the postrouting chain. 5. Finally, the backend server is sent through the postrouting chain.

Three. Package forwarding model for LVS
1.NAT Models:

650) this.width=650; "src=" Http://s4.51cto.com/wyfs02/M01/83/6A/wKioL1dzI7jzvOgZAABXr1zmtqQ618.jpg "title=" 002. JPG "alt=" wkiol1dzi7jzvogzaabxr1zmtqq618.jpg "/>

1. The client sends the request to the front-end load balancer, the request message Source address is CIP (client IP), the following is referred to as CIP), the destination address is VIP (load balancer front-end address, hereafter collectively referred to as VIP). 2. After the load balancer receives the message, it discovers that the request is the address that exists in the rule, then it changes the target address of the client request message to the RIP address of the back-end server and sends the message out according to the algorithm. 3. After the message is sent to the real server, the message will respond to the request and return the response message to the LVS because the destination address is itself. 4. The LVS then modifies the source address of this message to native and to the client

注意：在NAT模式中，Real Server的网关必须指向LVS，否则报文无法送达客户端

2.DR Models:

650) this.width=650; "src=" Http://s1.51cto.com/wyfs02/M01/83/6A/wKioL1dzJM6iD6AEAABfKXT-jdg121.jpg "title=" 003. JPG "alt=" wkiol1dzjm6id6aeaabfkxt-jdg121.jpg "/>

1. The client sends the request to the front-end load balancer, the request source address is CIP, the destination address is VIP. 2. After receiving the message, the load balancer discovers that the request is the address that exists in the rule, then it changes the source MAC address of the client request message to its own Dip MAC address, the target Mac changes to the MAC address of the RIP, and sends this packets to Rs. 3.RS Discovery Request message in the purpose of the MAC is itself, will receive the secondary message, after processing the request message, the response message through the LO interface sent to the ETH0 network card directly sent to the client.

注意:需要设置lo接口的VIP不能响应本地网络内的arp请求.




3.TUN Models:

650) this.width=650; "src=" Http://s1.51cto.com/wyfs02/M02/83/6C/wKiom1dzMG2SKzabAABi8Ue00Oc391.jpg "title=" 004. JPG "alt=" wkiom1dzmg2skzabaabi8ue00oc391.jpg "/>

1. The client sends the request to the front-end load balancer, the request source address is CIP, the destination address is VIP. 2. After receiving the message, the load balancer discovers that the request is the address that exists in the rule, then it will encapsulate a layer of IP packet at the header of the client request packet, change the source address to dip, change the target address to rip, and send this packets to Rs. 3.RS after receiving the request message, it will first open the first layer of encapsulation, and then found that there is a layer of IP header is the target address of its own LO interface VIP, so will process the request message, and send the response message through the LO interface sent to the ETH0 network card directly to the client.

Note: VIPs that need to set the Lo interface cannot appear on the network.

Four. LVS Scheduling algorithm
The scheduling algorithm of LVS is divided into two types: static and dynamic.
1. Static algorithm (4 kinds): Only according to the algorithm scheduling, regardless of the backend server actual connection situation and load situation

1.RR: The Round call dispatch (Round Robin) Scheduler uses the "round call" scheduling algorithm to sequentially allocate external requests to real servers in the cluster, which treats each server equally, regardless of the actual number of connections on the server and the system load 2. WRR: The weighted round call (Weight RR) Scheduler dispatches access requests based on the different processing capabilities of the real server through the "Weighted round call" scheduling algorithm. This ensures that the processing capacity of the server handles more access traffic. The scheduler can automatically inquire about the load of the real server and adjust its weights dynamically. 3.DH: Target Address hash schedule (Destination hash) based on the destination IP address of the request, as a hash key (HashKey) from the static distribution of the hash list to find the corresponding server, if the server is available and not overloaded, send the request to the server, otherwise return empty. 4.SH: Source Address hash (source hash) Origin address hash "scheduling algorithm according to the requested source IP address, as a hash key (HashKey) from the static distribution of the hash list to find the corresponding server, if the server is available and not overloaded, send the request to the server, otherwise return empty

2. Dynamic Algorithms (6): The front-end scheduler allocates requests based on the actual connection of the backend real servers

1.LC: The Least-Link (least connections) scheduler dynamically dispatches network requests to servers with the fewest number of links established through the "least-connection" scheduling algorithm. If the real server of the cluster system has similar system performance, the "Minimum connection" scheduling algorithm can be used to balance the load well. 2.WLC: Weighted minimum Connection (this is the default) (Weighted least connections) in the cluster system, the server performance difference is large, the scheduler uses "weighted least link" scheduling algorithm to optimize load balancing performance, Servers with higher weights will withstand a large percentage of the active connection load scheduler can automatically query the actual server load situation, and dynamically adjust its weight. 3.SED: Shortest delay scheduling (shortest expected delay ) improves,overhead =  (active+1) *256/weighting on WLC basis, No longer consider the inactive state, the current number of active + one implementation, the number of the smallest, accept the next request, +1 of the purpose is to consider the weight of the time, inactive connection too many defects: When the permissions are too large, the idle server is always in a non-connected state. 4.NQ never queue/minimum queue scheduling (NEVER QUEUE SCHEDULING NQ) No queues are required. If there is a  realserver number of connections = 0 directly assigned to the past, do not need to perform the SED operation, to ensure that there is no space for a host. On the basis of the SED no matter how many, the second must give the next, to ensure that there will not be a host is not very idle, regardless of the inactive connection, only with nq,sed to consider the active state connection, UDP for DNS does not need to consider the inactive connection, The HTTPD service needs to consider the stress of inactive connections to the server. 5.LBLC: Local Least-Link (locality-based least connections) local-based least-link "scheduling algorithm is load-balanced against the target IP address, It is mainly used in cache cluster system to find out the most recently used server of the target IP address according to the target IP address of the request, if the server is available and not overloaded, send the request to the server, if the server does not exist, or if the server is overloaded and the server is in half of the workload, use " Least link "principle select an available server to send the request to this server 6. LBLCR: Least locality-based connection with replication (locality-based least connections with replication) Local least-link-based scheduling algorithm with replication is also a load balancer for the destination IP address, At present, it is mainly used in the cache cluster system, which differs from the LBLC algorithm in that it maintains a mapping from a target IP address to a set of servers, while the LBLC algorithm maintains a mapping from a destination IP address to a server. The algorithm finds the corresponding server group for the destination IP address according to the destination IP address of the request. Select a server from the server group according to the "minimum connection" principle, if the server is not overloaded, send the request to the server, if the server is overloaded, select a server from the cluster by the "minimum Connection" principle, add the server to the server group, and send the request to the server. When the server group has not been modified for some time, the busiest server is removed from the server group to reduce the degree of replication.

This article from "Opensamlee" blog, declined reprint!

A detailed analysis of the LVS principle in cluster load technology