Lvs is software
load balancing. LVS is based on the netfilter framework and mainly works on the INPUT chain. Register the ip_vs_in HOOK function on INPUT to perform the main process of IPVS.
When a user visits a website, the user data passes through layers of the network, and finally enters the LVS server network card through the switch, and enters the kernel network layer. After entering PREROUTING, after routing search, it is determined that the destination VIP of the visit is the local IP address, so the data packet enters the INPUT chain. IPVS is working on the INPUT chain. It will judge whether the request is an IPVS service according to the visited VIP+port, if it is Then call the registered IPVS HOOK function to perform the IPVS related main process, forcibly modify the related data of the data packet, and send the data packet to the POTROUTING chain. After receiving the data packet on POSTROUTING, according to the target IP address (back-end server), through routing, the data packet is finally sent to the back-end server.
3.1 Basic introduction
LVS (Linux Virtual Server) is the Linux virtual server. It is an open source
load balancing project led by Dr. Zhang Wensong. Currently LVS has been integrated into the Linux kernel module. This project implements an IP-based data request load balancing scheduling scheme in the Linux kernel;
3.2 Working mode
The open source LVS version has 3 working modes. Each mode has a completely different working principle. It is said that each mode has its own advantages and disadvantages, which are suitable for different application scenarios, but the ultimate essential function is to achieve balanced traffic scheduling and good The scalability. Mainly include the following three modes
DR mode
NAT mode
Tunnel mode
Related term introduction
CIP: Client IP, which means the client IP address.
VIP: Virtual IP, which represents the IP address that the
load balancing provides external access. Generally, the load balancing IP will achieve high availability through Virtual IP.
RIP: RealServer IP, which represents the real server IP address of the load balancing backend.
DIP: Director IP, which represents the IP address used by the load balancer to communicate with the back-end server.
CMAC: The MAC address of the client, which should be exactly the MAC address of the router connected to the LVS.
VMAC: The MAC address corresponding to the VIP of the load balancing LVS.
DMAC: The MAC address corresponding to the DIP of the load balancing LVS.
RMAC: The MAC address corresponding to the RIP address of the back-end real server.
3.2.1 DR mode
In fact, DR is the most commonly used working mode because of its powerful performance. The following describes the specific principles of the DR mode with a process of requesting and responding to the data stream:
first step:
When the client requests the website homepage, after DNS resolution to IP, it sends the request data to the website server, the data packet reaches the website load balancing LVS server through the layered network, and the data packet when it reaches the LVS network card: the source IP is the client IP address CIP , The destination IP is Sina’s external server IP address, which is VIP; at this time, the source MAC address is CMAC, which is actually the MAC address of the router connected to LVS (for easy understanding, it is recorded as CMAC), and the destination MAC address is the MAC corresponding to VIP. Denoted as VMAC.
The second step:
After the data packet arrives at the network card, it passes through the link layer to the PREROUTING position (just entered the network layer). After searching the route and finding that the destination IP is the VIP of LVS, it will be delivered to the INPUT chain. At this time, the MAC, IP, and Port of the data packet are not modified. .
third step:
The data packet arrives in the INPUT chain, and INPUT is the main working position of LVS. At this time, LVS will confirm whether it is a service defined by LVS according to the destination IP and Port. If it is a defined VIP service, it will select one of the RealServers as the back-end server RS1 according to the configured Service information, and then RS1 as the target Look up the route in the Out direction, determine the next hop information and the network card through which the data packet is sent. Finally, the data packet is sent to the OUTPUT chain through INET_HOOK (the Out direction has just entered the network layer).
the fourth step:
After the data packet passes through the POSTROUTING chain, it is transferred from the network layer to the link layer, and the destination MAC address is modified to the RealServer server MAC address, marked as RMAC; and the source MAC address is modified to the MAC address corresponding to selfIP in the same network segment of LVS and RS. Recorded as DMAC. At this point, the data packet is forwarded to the RealServer server through the switch
the fifth step:
After the request packet reaches the RealServer server, the link layer checks that the destination MAC is its own network card address. At the network layer, search for the route, the destination IP is VIP (VIP is configured on lo), and it is determined that the data packet is the local host, and after passing through the protocol stack, it is copied to the application (for example, the nginx server here). After nginx responds to the request, it generates a response data pack. Use the destination VIP as the dst to find the Out route, determine the next hop information and send the network card device information, and send the data packet. At this time, the source and destination IP of the data packet are VIP and CIP respectively, and the source MAC address is the RMAC of RS1, and the destination MAC is the MAC address of the next hop (router), marked as CMAC (for easy understanding, marked as CMAC). Then the data packet is forwarded to the real client through the router connected to RS.
3.2.2 NAT mode
first step:
The user requests the data packet to pass through the layer-by-layer network and reach the lvs network card. At this time, the source IP of the data packet is CIP and the destination IP is VIP.
Enter the prerouting position of the network layer through the network card, find the route according to the destination IP, confirm that it is the local IP, and forward the data packet to INPUT. At this time, the source and destination IP are unchanged.
The second step:
After reaching lvs, check whether it is IPVS service through the destination IP and destination port. If it is an IPVS service, an RS will be selected as the back-end server, the destination IP of the data packet will be changed to RIP, and the destination IP will be RIP to find routing information, determine the next hop and egress information, and forward the data packet to the output.
third step:
After the modified data packet is processed by postrouting and link layer, it reaches the RS server. At this time, the source IP of the data packet is CIP and the destination IP is RIP.
the fourth step:
After the data packets arriving at the RS server are checked by the link layer and the network layer, they are sent to the user space nginx program. After the nginx program is processed, the response packet is sent. Since the default gateway on the RS is configured as the lvs device IP, the nginx server will forward the packet to the next hop, which is the lvs server. At this time, the source IP of the data packet is RIP, and the destination IP is CIP.
the fifth step:
After receiving the RS response data packet, the lvs server found that the destination IP was not the local IP based on the route search, and the lvs server opened the forwarding mode, so the data packet was forwarded to the forward chain, and the data packet was not modified at this time.
The sixth step:
After receiving the response packet, lvs looks up the service and connection table according to the destination IP and destination port, changes the source IP to VIP, determines the next hop and egress information through route lookup, sends the packet to the gateway, and passes through the complex network Reach the user client, and finally complete a request and response interaction.
3.2.3 Tunnel mode
first step:
The user requests the data packet to pass through the multi-layer network and arrive at the lvs network card. At this time, the source IP of the data packet is cip and the destination IP is vip.
The second step:
Enter the prerouting position of the network layer through the network card, find the route according to the destination ip, confirm that it is the local ip, and forward the data packet to the input chain to reach lvs. At this time, the source and destination ips have not changed.
third step:
After reaching lvs, check whether it is an IPVS service through the destination ip and destination port. If it is an IPVS service, an rs will be selected as the backend server, and rip will be used as the destination ip to find routing information, determine the next hop, dev and other information, and then an additional IP header is added in front of the IP header (with dip as the source, rip For the destination ip), the data packet is forwarded to the output.
the fourth step:
According to the routing information, the data packet is finally sent to the router gateway through the lvs network card, and then reaches the back-end server through the network.
the fifth step:
After the back-end server receives the data packet, the ipip module unloads the Tunnel header. The source IP is cip and the destination IP is VIP. As VIP is configured on tunl0, it is determined as the local IP after route search and sent to the application. program. After the application nginx normally responds to the data, it uses vip as the source ip and cip as the destination ip. The data packets are sent out of the network card and finally reach the client.
