Implementing multi-server load Balancing with network address translation

Server | network | Conversion SUMMARY: This paper discusses the load balancing technology and load allocation strategy used by distributed network server, and realizes load balancing gateway on FreeBSD based on network address translation, which is applied to our Internet network server and distributes the load to multiple servers. To address the high CPU or I/O load problems caused by the large number of concurrent accesses to Internet servers. To achieve optimal load balancing, the load controller needs to allocate the load based on the current CPU and I/O status of each server, which requires dynamic monitoring of the server's load and the application of an optimized load allocation strategy to achieve the average distribution load.

Keywords: load balancing, network address translation, FreeBSD


1. Introduction

The rapid growth of the internet has led to a rapid increase in the number of accesses to multimedia network servers, the ability of servers to provide a large number of concurrent access services, and the processing and I/O capabilities of the server as a bottleneck for service delivery. Since the performance of a single server is always limited, multiple servers and load balancing techniques must be used to meet the needs of a large number of concurrent accesses.

The first load balancing technology is implemented through DNS, in DNS for multiple addresses to configure the same name, so the client queried this name will get one of the addresses, so that different customers access to different servers, to achieve load balancing purposes [1]. DNS load balancing is a simple and efficient method, but it does not differentiate between servers and the server's current state of operation.

The reverse proxy server can forward the request to the internal Web server, and if the proxy server can forward the request evenly to multiple internal servers, it can achieve the purpose of load balancing [2]. In the reverse proxy mode, an optimized load balancing strategy can be applied to provide services for each visit to the most idle internal server. However, as the number of concurrent connections increases, the load on the proxy server itself becomes very large, and finally the reverse proxy server itself becomes a service bottleneck.

A load-balanced address translation gateway can map an external IP address to multiple internal IP addresses and dynamically use one of the internal addresses for each TCP connection request to achieve load balancing purposes [3]. Many hardware vendors integrate this technology into their switches as a function of their layer fourth switching, typically by randomly choosing a load-balancing strategy based on the number of connections to the server or the response time. However, the hardware implementation of the load controller flexibility is not strong, can not support a more optimized load balancing strategy and more complex application protocols.

In addition to these three load balancing methods, some protocols internally support load-balancing-related functions, such as redirection capabilities in HTTP protocols, but they rely on specific protocols and are therefore limited in scope. According to the existing load balancing technology, we choose the way of using software to realize the load balance of network address translation, to make up the inflexible of the hardware load balancer, and apply the optimal equilibrium strategy to realize the optimal state of the load sharing of the backend server.


2. Load Balancing Strategy

In order to distribute the load evenly to multiple internal servers, it is necessary to apply a certain load balancing strategy. The traditional load balancing strategy does not take into account the different types of service requests, the different capabilities of the backend servers and the uneven load distribution caused by random selection. In order to make the load distribution very evenly, it is necessary to apply a load balancing strategy that correctly reflects the CPU and I/O state of each server [4].

Customer-initiated service request types are diverse and can be easily divided into two different categories, depending on the processor, network, and I/O resource requirements to apply different processing strategies:



Static document requests: such as ordinary text, images, and other static multimedia data, they have little impact on the processor load, resulting in the disk I/O load and the size of the document is proportional to the main network I/O pressure.


Dynamic document requests: more common requests often require a server to be processed in advance, such as searching the database, compressing the uncompressed multimedia files, and so on, which require considerable processor and disk I/O resources.


For static documents, each service process consumes roughly the same system resources, so the number of processes can be used to represent the system load. Dynamic Document services require additional processing, which consumes more system resources than processing static requests, and therefore needs to be represented using a weight. Such a simple server load representation formula is:

Where L is the load of the server, the NS is the number of static document service processes, ND is the dynamic document service process, and a is the weight of each dynamic document service relative to the static document service, which can be selected from 10 to 100.

In this formula does not consider the server hardware limitations, when the hardware constraints, due to resource constraints, the server load will be significantly increased. For example, due to the size of the server memory, some processes will be swapped to the hard disk, causing the system load to increase rapidly. Considering the system hardware limitations, the server load can be expressed as:

The newly added parameter ll represents the limits of the normal load on the server, which is set according to the hardware capabilities of each server itself. and b indicates that the weight that is used to limit the task assigned to the server when it exceeds the normal load should be set to a value greater than ll to indicate the hardware restriction. Usually in a server cluster, the worse the hardware settings, the greater the weight of the server, to avoid when all servers are overloaded to run, the hardware's worst servers load the most. So B is inversely proportional to the hardware limit of this server, then B can be set to:

Llmax the LL value of the server configured for the highest hardware in the server cluster. Once the load is determined for each server, the server that centers control the load allocation can distribute the load correctly to the most idle servers, thus not causing uneven load distribution, as other load-allocation policies do.


3. Realization method and experiment result

Our server system consists of multiple FreeBSD systems connected by using Fast Ethernet. Each back-end server runs a daemon to dynamically obtain its own load state, while the central control gateway using the FreeBSD implementation refreshes the load of the individual servers for proper load allocation through these daemons.

3.1 Support for load balancing gateways

Under the FreeBSD system, a divert interface is provided to support network address translation capabilities. IP packets are sent to the divert interface through the system kernel's IPFW filtering function, so that the external daemon natd can receive the original packet, and then send back the system kernel for normal IP distribution after processing [5].

Therefore, based on the FreeBSD address translation structure, you can create your own network address translation daemon to support load balancing so that the FreeBSD system can be used as a load-balancing gateway. Because it is the way of software implementation, it is easy to support non-standard protocols and application of optimized load balancing strategy, with great flexibility.

3.2 Experiment and analysis

To test the usability of this implementation, we conducted our test experiments against the most common HTTP protocols. To differentiate between different request types, three different types of tests were designed to test different aspects of performance.



A dynamic document generated by CGI programs: a load-balancing state used to test the processing power of a server.


Small static Documentation: use static documents of small size to test the state of load balancing under frequent connections;


Large static Document: Use large documents to test the load balancing status of disk and network I/O;


The test results are based on the performance of the request per second of a single server, showing the ratio of requests completed per second to the number of base requests for load balancing using multiple servers.



Figure 1: Load Balancing Performance



The first curve A in the above illustration deals with dynamic document requests, and the performance is multiplied as the number of servers increases, while the second curve B, which is requested for small-size static documents, is less performance-enhancing when using three servers , while the third curve C, which handles requests for large-size static documents, has little performance change. To find out why the load-balancing system is not up to the ideal state, we have examined the utilization of server resources:

Table 1. Utilization of server resources


Processing type
Load Balancing Gateway
Server 1
Server 2
Server 3

A
53%
97%
85D
98%

B
76%
43%
39%
41%

C
94%
32%
31%
35%




As you can see from this table, three servers are running at full speed when dealing with dynamic document A, and the load is evenly distributed, which is an ideal state. While working with static document types B and C, the load is evenly distributed to three servers, but each server is not running at full speed. Especially when dealing with large size documents, the NATD process in the load balancing device occupies most of the processing resources. Since all network traffic is converted through it, the load on the NATD process increases as the network traffic and the number of concurrent connections are quite large. When using different numbers of backend servers in the experiment, the actual network bandwidth flowing through the load-balanced gateway is:

Table 2: Bandwidth of server clusters when providing large size documents


Number of servers
1 units
2 units
3 units

Network speed (KB/S)
10042.14
11015.10
11442.67




You can see that the bandwidth limit is around 10mb/s, obviously this is the bandwidth limit of the load balancing process used by this test, in fact the program uses a linked list to maintain the state of the network address translation, which greatly limits its network performance, by improving the hardware performance and improving the algorithm, can further improve its performance.


4. Discussion

From the experiments above, it can be seen that the load balancer based on network address translation can effectively solve the CPU and disk I/O load of the server, but the performance of the load balancer is limited by the network I/O, and it has certain bandwidth limits under certain hardware conditions. However, this bandwidth limit can be improved by improving the algorithm and improving the hardware performance of the running load balancing program. At the same time, it can be seen that different service types occupy different server resources, and the load measurement strategy we use is evaluated using the same load, which is appropriate for most conditions, but the best approach is to monitor the server load separately for different resources, such as CPU, disk I/O, or network I/O, The central controller selects the most appropriate server to distribute the customer request. Our future work will be from these two aspects to improve this load balancing controller.


Reference documents:


[1] e.kata,m.butler, and R. McGrath. A Scalable HTTP server:the NCSA prototype. Computer Networks and ISDN systems. 1994. Vol. p155-164

[2] Ralf S.engelschall. Load balancing Your Web Site. Web techniques Magazine (http://www.WebTechniques.com), May 1998. Vol.3, Iss.5

[3] CICSO. LocalDirector Documents. Http://www.cisco.com, 1997

[4] H.zhu. T.yang, Q.zheng, D.watson, O.h.ibarra, Andt.smith, adaptive load Sharing for clustered digital library servers. Technical, CS, UCSB, 1998.

[5] FreeBSD core team. NATD and divert manual pages. Http://www.freebsd.org. 1995



Implement a load balancing gateway by NAT


Wang, Bo

Nongye Road, Zhengzhou, 450002, P.R.China

wb@email.online.ha.cn


Abstract:this paper investigates load balancing techniques and strategies, and implements a load balancing gateway based NAT for our Internet servers. The Internet servers involve the high load of CPU and I/O by simultaneous access requests, the symmetrical clustered serve RS can distribute the server load to solve the problem. To balance the "load in" the best way, the gateway distributes the load according to the status of server ' s CPU and I/O. the Gateway must monitor every server ' s load and apply the best scheme to delivery every request, so it can provide the Performance for Internet services.

Keywords:load balancing, NAT, FreeBSD