In-depth analysis of Server Load balancer policies

In practical applications, we may not just want to evenly allocate client service requests to internal servers, regardless of whether the servers are down or not. Instead, we want the Pentium III server to accept more service requests than the Pentium II server. One server that processes fewer service requests can allocate more service requests, the faulty server will no longer accept service requests until the fault recovers.

Select an appropriate Server Load balancer policy so that multiple devices can complete the task together, eliminating or avoiding the bottleneck of unbalanced network load distribution and long response time of data traffic congestion. For different Server Load balancer modes, Layer 2, Layer 3, Layer 4, and Layer 7 Server Load balancer of OSI reference models have corresponding Server Load balancer policies.

The advantages and disadvantages of Server Load balancer policies and the difficulty of implementation are two key factors: 1. Server Load balancer algorithms; 2. detection methods and capabilities for network system conditions.

Considering the different types of service requests, different processing capabilities of servers, and uneven load distribution caused by random selection, In order to rationally allocate the load to multiple internal servers, you need to apply a Server Load balancer algorithm that correctly reflects the processing capabilities and network status of each server:

Round robin: each request from the network is distributed to an internal server in turn, starting from 1 to n and then restarting. This balancing algorithm is suitable for the situation where all servers in the server group have the same hardware and software configurations and the average service requests are relatively balanced.

Weighted Round Robin (Weighted Round Robin): assigns different weights to each server based on the server's different processing capabilities, so that it can accept service requests with corresponding weights. For example, if the weight of server a is set to 1, the weight of server B is 3, and the weight of server C is 6, server A, server B, and server C will receive 10%, 30%, and 60% service requests respectively. This balancing algorithm ensures higher utilization of high-performance servers and avoids overload of Low-performance servers.

Random balancing: Randomly allocates requests from the network to multiple internal servers.

Weighted random balancing (weighted random): This balancing algorithm is similar to the weighted round robin algorithm, but it is a random selection process when processing request sharing.

Response time: the Server Load balancer device sends a detection request (such as PING) to each internal server ), then, the server determines which server to respond to the client's service request based on the shortest response time of the internal server to the probe request. This balancing algorithm can better reflect the current running status of the server, but the fastest response time only refers to the fastest response time between the Server Load balancer device and the server, instead of the fastest response time between the client and the server.

Least connections (least connection): the time for each request service of the client to stay on the server may vary greatly, if a simple round robin or random balancing algorithm is used, the connection processes on each server may be significantly different, and the load balancing is not achieved. The least connections balancer algorithm records the number of connections that are being processed by the server. When there is a new service connection request, the current request will be allocated to the server with the least connections, so that the Server Load balancer is more in line with the actual situation and the load is more balanced. This balanced algorithm is suitable for long-time request services, such as ftp.

Processing capacity balancing: This balancing algorithm will allocate service requests to the lightest server with the processing load (which is calculated based on the server CPU model, number of CPUs, memory size, and current number of connections, considering the processing capability of internal servers and the current network running status, this balancing algorithm is relatively more accurate, especially suitable for Layer 7 (Application Layer) load balancing.

DNS response balancing (Flash DNS): On the internet, whether it is HTTP, FTP or other service requests, the client generally finds the exact IP address of the server through domain name resolution. In this balancing algorithm, Server Load balancer devices in different geographic locations receive domain name resolution requests from the same client, resolve the domain name to the IP address of the corresponding server (that is, the IP address of the server in the same geographical location as the Server Load balancer device) at the same time and return it to the client, the client will continue to request the service with the domain name resolution IP address received first, and ignore the response from other IP addresses. When a Server Load balancer policy is applicable to global load balancing, it is meaningless for local load balancing.

Although multiple Server Load balancer algorithms can better distribute data traffic to servers for load balancing, if the server Load balancer policy does not detect the network system status, if a fault occurs between a server or a Server Load balancer device and the server network, the Server Load balancer device still directs some data traffic to the server, this will inevitably cause a large number of service requests to be lost and will not meet the uninterrupted availability requirements. Therefore, a good load balancing policy should be able to detect network faults, server system faults, and Application Service faults:

Ping Detection: The ping method is used to detect the server and network system conditions. This method is simple and fast, but can only roughly detect whether the network and the operating system on the server are normal, there is no way to Detect application services on the server.

TCP open Detection: each service opens a TCP connection to check whether a TCP port (for example, port 23 for telnet and port 80 for HTTP) on the server is opened to determine whether the service is normal.

Http url Detection: for example, an access request to the main.html file is sent to the httpserver. If an error message is received, the server is considered to be faulty.

In addition to the two factors mentioned above, the advantages and disadvantages of the Server Load balancer policy are also affected. In some cases, we need to allocate all requests from the same client to the same server, for example, when the server stores the client registration, shopping, and other service request information in a local database, it is vital to allocate the client's subrequests to the same server for processing. There are two ways to solve this problem. One is to allocate multiple requests from the same client to the same server based on the IP address, the corresponding information of the Client IP address and server is stored on the server Load balancer device. Second, the unique identifier is used in the client browser cookie to allocate multiple requests to the same server for processing, suitable for clients accessing the Internet through proxy servers.

There is also an out-of-path return mode. When a client connection request is sent to a server Load balancer device, the central server Load balancer device directs the request to a server, the server's response request is no longer returned to the central server Load balancer device, that is, bypassing the traffic distributor and directly returning it to the client. Therefore, the central server Load balancer device is only responsible for receiving and forwarding requests, the network load is much reduced, and the client provides a faster response time. This mode is generally used for HTTP Server clusters. A virtual network adapter is installed on each server and its IP address is set as the VIP of the server group, in this way, three handshakes can be successfully achieved when the server directly responds to the client request.
Load Balancing implementation elements

The server Load balancer solution should be considered at the initial stage of website construction. However, sometimes the explosive growth of access traffic exceeds the expectation of decision makers, which becomes a problem that must be faced. When we introduce a Server Load balancer solution or even implementation, like many other solutions, we first determine the current and future application requirements, and then weigh the costs and results.

Based on the current and future application requirements and the analysis of network bottlenecks, we need to determine which type of Server Load balancer technology is used and what type of balancing strategy is used, what are the requirements for availability, compatibility, and security.

Whether the Server Load balancer solution is implemented by using layer-4 switches, Server Load balancer, and other hardware modes with higher performance and functions, or other different types of balancing technology, the following are the issues that we may consider when introducing a balancing solution:
Performance: performance is the most difficult issue to grasp when we introduce a balanced solution. When balancing performance, the number of packets passing through the network per second can be used as a parameter, and the other parameter is the maximum number of concurrent connections that the server group can process in the balancing solution. However, assuming that a balanced system can process millions of concurrent connections, but it can only forward at the rate of 2 packets per second, this obviously has no effect. The advantages and disadvantages of performance are closely related to the processing capability of the Server Load balancer device and the adopted balancing policy. There are two points to note: 1. the overall performance of the Server Load balancer solution to the server cluster, this is the key to responding to client connection requests. 2. The performance of the Server Load balancer device prevents service bottlenecks due to insufficient performance when a large number of connection requests exist. Sometimes we can also consider using a hybrid load balancing policy to improve the overall performance of the server group, such as the combination of DNS load balancing and Nat load balancing. In addition, for websites with a large number of static file requests, you can also consider using the high-speed cache technology to save costs and improve response performance; for sites with a large number of SSL/XML content transmission, the SSL/XML acceleration technology should be considered.
Scalability: IT technology is changing with each passing day. The latest product a year ago may be the product with the lowest performance in the network. The rapid increase in business volume requires a new round of expansion for the network a year ago. Appropriate balancing solutions should be able to meet these needs and balance the load between different operating systems and hardware platforms, load Balancing of different servers such as HTTP, mail, news, proxy, database, firewall, and cache, and dynamic addition or deletion of some resources in a completely transparent manner to the client.

Flexibility: balanced solutions should be able to flexibly provide different application requirements to meet the changing application requirements. When different Server clusters have different application requirements, a wide range of balanced policies should be provided.

Reliability: In websites with high service quality requirements, the Server Load balancer solution should provide full fault tolerance and high availability for the server group. However, when the server Load balancer device fails, there should be good redundancy solutions to improve reliability. When redundancy is used, multiple Server Load balancer devices in the same redundancy unit must have an effective way to monitor each other and protect the system from losses caused by major faults as much as possible.

Easy to manage: whether it is a balance solution through software or hardware, we all hope it has a flexible, intuitive, and secure management method, which facilitates installation, configuration, maintenance, and monitoring, improve work efficiency and avoid errors. Currently, there are three management methods available for hardware Load Balancing Devices: 1. command line interface (CLI: command line interface ), you can use a Super Terminal to connect to the serial interface of the Server Load balancer device for management, or telnet remote logon management. The former is often used during configuration initialization. 2. graphical user interface (GUI: graphical User Interfaces) provides management based on common web pages and Java applet for security management. Generally, the Management Terminal must install a certain version of browsers; iii. SNMP (Simple Network Management Protocol) support, and manage devices that comply with SNMP standards through third-party network management software.