Linux-based Cluster System Environment Construction Process

1. design objectives
Design a Linux-based cluster system that provides the load balancing function. The system constantly monitors the load status of each server in the cluster, and forwards multiple requests from the external network to an actual server in the Intranet for execution.

Specifically, it must have the following features:

1) Service forwarding. Accept various TCP/IP-based service requests from the external network, such as FTP, TELNET, and HTTP, and forward them to the machine with the least load for execution.

2) dynamic load balancing. The balancer can monitor the Load Status of the actual servers on the Intranet and find the machine with the lightest load.

3) connection continuity. All requests from the same customer from the external network must be forwarded to the same actual server in the Intranet for processing.

2. Environment Settings
As shown in Figure 1-1, the cluster system consists of machines such as router, server 1, server 2, server 3, and server n. The router acts as an interface between the Intranet and the external network. It can receive user requests from the external network and send the requests to a machine from server1 to servern, for example, server1 ), after Server 1 processes user requests, it sends the processed results to the router, and then the router returns the Request Response to the user on the external network. This machine is also called the Server Load balancer because it undertakes the role of Server Load balancer, And the servers such as server1 that actually responds to user requests are called the actual servers. In the test environment, the IP address of the customer machine in the external network is 192.9.200.53, And the Server Load balancer router has two IP addresses: 192.9.200.56 and 10.1.1.1, the operating system kernel running on the server Load balancer is Linux 2.2.x. There are n actual servers in the Intranet. Their IP addresses are 10.1.1.2 and 10.1.1.3 ........ 10.1.1.n: the operating system running on them can be a linux operating system or a Windows operating system, such as Windows 98 or Windows NT. They both set 10.1.1.1 as a gateway and added routes to the 192.9.200.0 network. The purpose of Server Load balancer is to distribute client requests to the server Load balancer, such as telnet, ftp, and www, according to the current load on the Intranet.

Figure 1-1 Server Load balancer System Environment Settings 3. Build Process
In the building process section, we first introduce the principles of the IP camouflage technology, and then introduce the creation process of the cluster system. The establishment of the cluster system requires the establishment of an IP camouflage mechanism, and then the IP Portforwarding (IP port forwarding) mechanism is added, and then an application is designed, it can achieve real-time Dynamic Load Balancing Based on the status of machines in the cluster. The Linux kernel version on the server Load balancer in the cluster system is 2.2.x.

3.1 IP Camouflage Technology

The cluster system mainly uses the IP MasqueradeIP disguise) mechanism. The server Load balancer system adopts the network address translation mechanism. NAT is mainly used for communication between private and external networks. Private IP addresses such as 10.0.0.0/255.0.0.0, 172.16.0.0/255.240.0.0, and 192.168.0.0/255.255.0.0 cannot communicate directly with machines on the Internet. If they want to communicate with machines on the Internet, network Address Translation (NAT) is required.

NAT means to map IP addresses from one group to another, if this ing relationship is N-N, it is called static network address translation; if the ing is M-N (M> N, it is called dynamic network address translation; IP camouflage mechanism is actually a kind of dynamic network address translation of M-1, it can map IP addresses in multiple intranets to an external IP address connected to the Internet, in this way, machines with intranet IP addresses that cannot communicate directly with machines on the Internet can communicate with the outside world through this ing Machine. Network Address port translation is an extension of Network Address Translation. It translates many network addresses and their TCP/UDP ports into an IP address and a TCP/UDP port. The cluster system uses the network address and port translation mechanism.

3.2 creation process of IP camouflage Mechanism

1) Compile the core to support IP camouflage: pay attention to the following options when compiling the core.

2) After re-compiling the core, you should re-compile and install the IP camouflage module using the following command:

Make modules; make modules_install

3.3 IP port forwarding mechanism establishment process

Now we need to add an appropriate forwarding mechanism to forward data packets to appropriate machines. The first thing to note is that IPFWADM is no longer a tool for controlling IP camouflage rules in 2.1.x and 2.2.x core. These core tools are IPCHAINS.

1) Create the/etc/rc. d/rc. firewall file according to the following rules.

/sbin/depmod -a/sbin/modprobe ip_masq_ftp/sbin/modprobe ip_masq_portfw.oecho "1" > /proc/sys/net/ipv4/ip_forwardecho "1" > /proc/sys/net/ipv4/ip_always_defrag/sbin/ipchains -M -S 7200 10 160/sbin/ipchains -P forward DENY/sbin/ipchains -A forward -s 10.1.1.0/24 -j MASQ/usr/sbin/ipmasqadm portfw -f#port forwarding strategy#port forward the packet of interface 192.9.200.56 to 10.1.1.2 (server2)#telnet service:port 23/usr/sbin/ipmasqadm portfw -a -P tcp -L 192.9.200.56 23 -R 10.1.1.2 23 -p 1#ftp service:port 21/usr/sbin/ipmasqadm portfw -a -P tcp -L 192.9.200.56 21 -R 10.1.1.2 21 -p 1#www service:port 80/usr/sbin/ipmasqadm portfw -a -P tcp -L 192.9.200.56 80 -R 10.1.1.2 80 -p 1#port forward the packet of interface 192.9.200.56 to 10.1.1.3 (server3)/usr/sbin/ipmasqadm portfw -a -P tcp -L 192.9.200.56 23 -R 10.1.1.3 23 -p 1/usr/sbin/ipmasqadm portfw -a -P tcp -L 192.9.200.56 21 -R 10.1.1.3 21 -p 1/usr/sbin/ipmasqadm portfw -a -P tcp -L 192.9.200.56 80 -R 10.1.1.3 80 -p 1#port forward the packet of interface 192.9.200.56 to 10.1.1.4 (server4)/usr/sbin/ipmasqadm portfw -a -P tcp -L 192.9.200.56 23 -R 10.1.1.4 23 -p 1/usr/sbin/ipmasqadm portfw -a -P tcp -L 192.9.200.56 21 -R 10.1.1.4 21 -p 1/usr/sbin/ipmasqadm portfw -a -P tcp -L 192.9.200.56 80 -R 10.1.1.4 80 -p 1

2) After editing the/etc/rc. d/rc. firewall file, run the chmod 700/etc/rc. d/rc. firewall command to make the file executable.

3) add a line in the/etc/rc. d/rc. local file to activate the IP camouflage module after each restart.

#rc.firewall script - Start IPMASQ and the firewall /etc/rc.d/rc.firewall.

After completing operations in sections 3.2 and 3.3 in turn, a cluster system based on the Round-Robin scheduling algorithm has been established, if you want to assign the corresponding weight based on the computer performance, you only need to modify the rc. rules in firewall. For example, if you want to change the weight of server2 (10.1.1.2) to 2, you only need to change the original rules:

/Usr/sbin/ipmasqadm portfw-a-P tcp-L 192.9.200.56 23-R 10.1.1.2 23-p 1/usr/sbin/ipmasqadm portfw-a-P tcp-L 192.9.200.56 21- R 10.1.1.2 21-p 1/usr/sbin/ipmasqadm portfw-a-P tcp-L 192.9.200.56 80-R 10.1.1.2 80-p 1 changed: /usr/sbin/ipmasqadm portfw-a-P tcp-L 192.9.200.56 23-R 10.1.1.2 23-p 2/usr/sbin/ipmasqadm portfw-a-P tcp-L 192.9.200.56 21- R 10.1.1.2 21-p 2/usr/sbin/ipmasqadm portfw-a-P tcp-L 192.9.200.56 80-R 10.1.1.2 80-p 2

In this way, you can.

3.4 create an application to achieve dynamic load balancing

This application monitors the load of each actual server in the cluster and forwards user requests to the actual server with the lightest load. For specific implementation, refer to the implementation method of the scheduling module in the second part of the article.

4. Design Principles
The cluster system achieves IP-level load balancing. When a customer sends a request message to the balancer, the target address of the request message is replaced at the IP layer of the balancer, replace the target address with the IP address of the machine with the least load on the actual server in the intranet. Then, the packet is forwarded again. After the actual server in the Intranet processes the request, it sends the Request Response to the balancer. The balancer replaces the target address with the customer's IP address in the external network where the request is sent at the IP layer, then, the packet is forwarded to the customer again.

The replacement of the target address is implemented in the core of the operating system, and the IP address of the machine with the least load is selected at the application layer. This is because obtaining load data at the application layer can improve the scalability of the system. When a new actual server needs to be added to the Intranet, you only need to add one to the array variable of the application. In addition, the scheduling policy can be flexibly determined at the application layer. You can use static scheduling policies such as Round Robin and Weighted Round Robin, you can also use dynamic scheduling policies such as Least Connection and Weighted Least Connection. The target address rewriting of IP packets is mainly completed at the core, because it is fast and saves the communication process from the user to the core.

When a customer in the external network sends a service request (such as www, ftp, and telnet) to the Server Load balancer, the customer can obtain the IP address and port number of the external network machine laddr from this request, lport), and the Server Load balancer IP address. Based on this information, query the two-way linked list of IP port forwarding to see if there is a table that matches laddr and lport. If yes, obtain the value of (raddr, rport) in the table, that is, the IP address and port number of the machine in the Intranet, and replace the destination address and port number of the IP package with raddr and rport ), then, send the IP package to the corresponding machine on the Intranet. If no table item exists, a new IP port forwarding table item is created and the corresponding IP address disguised table item is created. Then, replace the target address and resend the package.

5. Functions of each module
This load balancing system is mainly divided into IP camouflage module, IP port forwarding module, and scheduling module. The IP camouflage module and IP port forwarding module are implemented at the IP layer, you can find the corresponding programs in the directory where the Linux source code is located. The scheduling module is implemented at the application layer.

Module name	Identifier	Description
Ip camouflage Module	Ip_masq	Rewrite the ip header to forward ip packets.
Ip port forwarding Module	Ip_portfw	When receiving external requests, the scheduling algorithm determines the actual server to which the ip packets are forwarded.
Scheduling module	Sched	The address of the machine with the lightest load is determined by the load information collection module to forward user requests to the machine with the lightest load.