How centos uses LVS + keepalive to build clusters

A load balancing cluster is short for a load balance cluster. Common open-source load balancing software include nginx, LVS, haproxy, commercial Hardware load balancing equipment F5, and netscale. Here we will learn LVS in Linux and record it in detail.

I. Basic Introduction to Server Load balancer LVS
The architecture and principle of the LB cluster is very simple, that is, when a user's request comes, it will be directly distributed to the Director server, and then it will make the user's request according to the configured scheduling algorithm, smart and balanced distribution to real backend servers. In order to avoid different data from user requests on different machines, shared storage is required to ensure that the data requested by all users is the same.

LVS is short for Linux virtual server, that is, Linux virtual server. This is an open-source project initiated by Dr. Zhang Wenyu, whose official website is http://www.linuxvirtualserver.org and now LVS is already part of the Linux kernel standard. The technical goal achieved by LVS is to achieve a high-performance and high-availability Linux server cluster through the Load Balancing Technology achieved by LVS and the Linux operating system, which has good reliability, scalability and operability. In this way, the optimal performance is achieved at a low cost. LVS is an open-source software project for implementing Server Load balancer clusters. The LVS architecture is logically divided into the scheduling layer, server cluster layer, and shared storage.


Ii. basic operating principles of LVS

1. When a user initiates a request to the server Load balancer scheduler (Director server), The scheduler sends the request to the kernel space.
2. The prerouting chain first receives a user request, determines whether the target IP address is a local IP address, and sends data packets to the input chain.
3. ipvs works on the input chain. When a user request reaches the input, ipvs compares the user request with the custom cluster service, if the user requests a defined Cluster service, ipvs will forcibly modify the destination IP address and port in the data packet and send the new data packet to the postrouting chain.
4. After receiving data packets, the postrouting chain finds that the target IP address is exactly the same as its own backend server. In this case, the packet is finally sent to the backend server through routing.


Iii. Composition of LVS
LVS consists of two programs, including ipvs and ipvsadm.

1. ipvs (IP Virtual Server): A piece of code that runs in the kernel space, called ipvs, is the code that truly takes effect for scheduling.
2. ipvsadm: The other part is working in the user space, called ipvsadm. It is responsible for compiling rules for the ipvs kernel framework and defining who is the cluster service and who is the real backend server)


Iv. LVS Related Terms
1. DS: Director server. It refers to the frontend Server Load balancer node.
2. RS: Real Server. Real backend working servers.
3. VIP: the IP address of the target user request, which is directly requested from the outside.
4. Dip: Director Server IP address, which is used to communicate with internal hosts.
5. Rip: The Real Server IP address and the IP address of the backend server.
6. CIP: Client IP address, which is the IP address used to access the client.

Below is a summary of the principles and features of the three working modes.


V. LVS/NAT principles and features
1. Understand the implementation principles of NAT and the changes of data packets.


(A) when a user request arrives at ctor server, the requested datagram is first sent to the prerouting chain of the kernel space. In this case, the source IP address of the message is CIP and the target IP address is VIP.
(B) The prerouting check finds that the destination IP address of the data packet is the local machine and sends the data packet to the input chain.
(C). ipvs checks whether the service requested by the data packet is a cluster service. If so, modify the target IP address of the data packet to the IP address of the backend server, and then send the data packet to the postrouting chain. In this case, the source IP address of the packet is CIP and the target IP address is Rip.
(D). The postrouting Chain sends data packets to the Real Server through routing.
(E) Compare the Real Server and find that the target is its own IP address, and start to build a response packet and send it back to Director server. In this case, the source IP address of the packet is Rip and the target IP address is CIP.
(F) before responding to the client, the Director server changes the source IP address to its own VIP address and then responds to the client. In this case, the source IP address of the packet is VIP and the target IP address is CIP.

2. Characteristics of LVS-NAT Model

The RS should use a private address. The RS gateway must point to the dip

Dip and rip must be in the same network segment

Request and response messages must go through the ctor server. In high-load scenarios, Director server is easy to become a performance bottleneck.

Support Port ing

RS can use any operating system

Defect: the pressure on Director server is high, and requests and responses must go through the Director Server


Vi. LVS/DR principles and features
1. Reset the target MAC address of the Request Message to the MAC address of the selected RS.


(A) when a user request arrives at Director server, the requested datagram is first sent to the prerouting chain of the kernel space. In this case, the source IP address of the message is CIP and the target IP address is VIP.
(B) prerouting checks and finds that the destination IP address of the data packet is the local machine and sends the data packet to the input chain.
(C) s checks whether the service requested by the data packet is a cluster service. If yes, modify the source MAC address in the request packet to the MAC address of the dip, and change the target MAC address to the MAC address of the rip, then, send the data packet to the postrouting chain. At this time, neither the Source IP nor the target IP address is modified. Only the MAC address with the source MAC address dip is modified, and the target MAC address is the MAC address of rip.
(D) Because DS and RS are in the same network, they are transmitted through Layer 2. The postrouting chain checks the MAC address of the target MAC address as rip, and then the packet is sent to the Real Server.
(E) When RS finds that the MAC address of the request packet is its own MAC address, it receives the packet. After the processing is complete, the response packet is sent to the eth0 Nic through the lo interface and then sent out. In this case, the source IP address is VIP and the target IP address is CIP.
(F) The response message is finally delivered to the client.

2. Characteristics of LVS-DR Model

Feature 1: ensure that the front-end router sends all the VIP packets from the target address to the ctor server instead of Rs.

RS can use a private address or a public address. If a public address is used, you can directly access rip through the Internet.

RS and Director server must be in the same physical network

All request packets are sent to ctor server, but the response packets must not be sent to Director server.

Address Translation and port ing are not supported.

RS can be a common operating system.

The RS gateway is never allowed to point to dip (because we do not allow it to pass through ctor)

Configure the IP address of the VIP through the lo interface on RS

Defect: Rs and DS must be in the same IDC

3. Features 1 solutions:

Bind static address routing on the front-end router and route the VIP address to the ctor server only.

Problem: The user may not have the routing operation permission, because it may be provided by the carrier, so this method may not be practical.

Arptables: Implements firewall rules at the ARP level during ARP resolution to filter Rs responses to ARP requests. This is provided by iptables.

Modify the kernel parameters (arp_ignore and arp_announce) on RS to configure the VIP on RS on the alias of the LO interface, and restrict it from responding to the VIP address resolution request.


VII. LVS/TUN principles and features
Encapsulation of multiple IP addresses outside the original IP packets, internal IP addresses (Source IP Address: CIP, destination IIP: VIP), outer IP addresses (Source IP Address: dip, destination IP address: Rip)


(A) when a user request arrives at Director server, the requested datagram is first sent to the prerouting chain of the kernel space. In this case, the source IP address is CIP and the target IP address is VIP.
(B) prerouting checks and finds that the destination IP address of the data packet is the local machine and sends the data packet to the input chain.
(C) ipvs checks whether the service requested by the data packet is a cluster service. If so, a layer of IP packet is encapsulated in the first part of the request message, the source IP address is dip, and the target IP address is Rip. Then send it to the postrouting chain. The source IP address is dip and the target IP address is Rip.
(D) The postrouting Chain sends data packets to Rs based on the latest encapsulated IP packets (because an IP header is added to the outer encapsulation, it can be understood that the packets are transmitted through tunnel at this time ). The source IP address is dip and the target IP address is Rip.
(E) After Rs receives the packet and finds it is its own IP address, it receives the packet and removes the outermost layer of the IP address, it will find that there is another layer of IP header in it, in addition, the target is its own lo interface VIP, then Rs starts to process this request. After processing is complete, the RS sends the request to the eth0 Nic through the lo interface and then transmits the request to the external Nic. In this case, the source IP address is VIP and the target IP address is CIP.
(F) The response message is finally delivered to the client.

LVS-tun model features

Rip, VIP, and dip are all public IP addresses.

The RS gateway does not and cannot point to dip

All request packets are sent to ctor server, but the response packets must not be sent to Director server.

Port ing is not supported.

The RS system must support tunnels.

In fact, Dr implementation is the most commonly used method in enterprises, while NAT configuration is relatively simple and convenient. Later, we will summarize the specific configuration process of DR and Nat.


8. LVS eight Scheduling Algorithms
1. Scheduling rr
This algorithm is the simplest, that is, scheduling requests to different servers in a sequential manner. The biggest feature of this algorithm is simplicity. The round-robin algorithm assumes that all servers have the same ability to process requests. The scheduler distributes all requests evenly to each Real Server regardless of backend Rs configuration and processing capabilities, distributed in a very balanced manner.

2. WRR
This algorithm has another weight concept than the RR algorithm. You can set a weight for Rs. The higher the weight, the more requests are distributed and the weight ranges from 0 to 100. It is mainly an optimization and supplement to the RR algorithm. LVS considers the performance of each server and adds the weight to each server, if the weight of server a is 1 and that of server B is 2, the requests scheduled to server B are twice that of server. Servers with higher weights process more requests.

3. Minimum link LC
This algorithm determines the number of backend Rs connections to whom the request is distributed. For example, if the number of RS1 connections is less than the number of rs2 connections, the request is sent to RS1 first.

4. weighted least Link (wlc)
This algorithm has an additional weight concept than LC.

5. Local-based least-connection scheduling algorithm lblc
This algorithm is a scheduling algorithm for the target IP address of the request packet. The algorithm first searches for all servers using the nearest Target IP address based on the request's target IP address, if this server is still available and capable of processing this request, the scheduler will try to select the same server; otherwise, it will continue to select other feasible servers.

6. Complicated local least locality-based connection algorithm lblcr
The record does not indicate the connection record between the target IP address and a server. It maintains the ing between the target IP address and a group of servers to prevent the server load from being too high.

7. Target address hash scheduling algorithm DH
This algorithm uses the hash function to map the target IP address to the server based on the target IP address. When the server is unavailable or the load is too high, requests sent to the target IP address are sent to the server.

8. Source Address hash scheduling algorithm sh
Similar to the target address hash scheduling algorithm, the algorithm statically allocates fixed server resources based on the source address hash algorithm.

 

Reprinted from: http://www.thinkphp.cn/topic/51067.html

How centos uses LVS + keepalive to build clusters