Rhel6-lvs detail Configuration

For more information about lvs, please refer to the blog "LVS cluster system details" http://wangziyin.blog.51cto.com/blog/6948950/1297354

Or refer to lvs Chinese site: http://www.linuxvirtualserver.org/zh/lvs4.html

System Environment: rhel6 x86_64 iptables and selinux disabled
Yum repository Configuration:
[Rhel-source]
Name = Red Hat Enterprise Linux $ releasever-$ basearch-Source
Baseurl = ftp: // 192.168.122.1/pub/yum
Enabled = 1
Gpgcheck = 1
Gpgkey = file: // etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release

[HighAvailability]
Name = Instructor Server Repository
Baseurl = ftp: // 192.168.122.1/pub/yum/HighAvailability
Gpgcheck = 1
Gpgkey = file: // etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
Enabled = 1

[LoadBalancer]
Name = Instructor Server Repository
Baseurl = ftp: // 192.168.122.1/pub/yum/LoadBalancer
Gpgcheck = 1
Gpgkey = file: // etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
Enabled = 1

[ResilientStorage]
Name = Instructor Server Repository
Baseurl = ftp: // 192.168.122.1/pub/yum/ResilientStorage
Gpgcheck = 1
Gpgkey = file: // etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
Enabled = 1

[ScalableFileSystem]
Name = Instructor Server Repository
Baseurl = ftp: // 192.168.122.1/pub/yum/ScalableFileSystem
Gpgcheck = 1
Gpgkey = file: // etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
Enabled = 1
Load Balancer:
Kernel 2.6.x built-in LVS Module
Kernel 2.4.x or earlier kernel versions must be patched
Rhel5/rhel6 comes with LVS package to install ipvsadm package.

VS/NAT

Load Balance dual Nic 192.168.122.82 (Internet) 192.168.1.82 (Intranet)

Gateway 192.168.1.82 (note: the Gateway of Realserver should be the Intranet IP address of Load Balance)
Realserver1 192.168.1.33 255.33.example.com
Realserver2 192.168.1.34 255.34.example.com

Virtual IP 192.168.122.122

Perform the following steps on Route 82:
# Enable routing

# vim /etc/sysctl.conf     net.ipv4.ip_forward = 1# sysctl -p

# Loading the nat Module
# Modprobe iptable_nat
Note: If this module is not loaded, it can also be accessed successfully for the first time, but the latency or access timeout may occur when the module is accessed again.

Yum install ipvsadm-yipvsadm-Cipvsadm-A-t 192.168.122.82: 80-s rr1_sadm-a-t 192.168.122.82: 80-r 192.168.1.33: 80-m1_sadm-a-t 192.168.122.82: 80-r 192.168.1.34: 80-m # Save rule/etc/init. d/ipvsadm save # bind vip # ifconfig eth0: 0 192.168.122.122 netmask 255.255.255.0

Perform the following steps on instances 33 and 34:

# yum install httpd -y# echo `hostname` > /var/www/html/index.html# /etc/init.d/httpd start

Test:
Select a host with a CIDR block of 192.168.122.0/24 to access http: // repeatedly refresh the webpage at 192.168.122.122.

VS/DR


The following is the working principle of LVS-DR, including the data packet, data frame direction and conversion process:

Official principles: ctor receives user requests, selects a realserver Based on the server Load balancer algorithm, forwards packets, and then the realserver directly replies to the user.

Instance scenario device list:

650) this. width = 650; "src =" http://www.bkjia.com/uploads/allimg/131228/055133F38-0.png "title =" Screenshot-1.png "alt =" 154446737.png"/>

Note: For convenience, the client is a machine with the same network segment as the vip. If it is accessed by an external user, replace the client with the gateway, because the IP address header is unchanged and the source mac address is changed.
1. The client sends a request to the target vip and ctor receives the request. The IP header and data frame header information are as follows:
650) this. width = 650; "src =" http://www.bkjia.com/uploads/allimg/131228/055133B12-1.png "title =" Screenshot-2.png "alt =" 154648958.png"/>

2. VS selects an active realserver Based on the Load Balancing Algorithm (assuming 192.168.57.122), takes the mac address of the network card where the RIP is located as the target mac address, and sends it to the LAN. the IP header and data frame header information are as follows:
650) this. width = 650; "src =" http://www.bkjia.com/uploads/allimg/131228/05513345P-2.png "title =" Screenshot-3.png "alt =" 154750519.png"/>

3. the realserver (192.168.57.122) receives the frame in the LAN. After disassembling the frame, it finds that the destination IP address (VIP) matches with the local IP address, so it processes the packet. then the packets are re-encapsulated and sent to the LAN. the IP header and data frame header information are as follows:
650) this. width = 650; "src =" http://www.bkjia.com/uploads/allimg/131228/055133Ic-3.png "title =" Screenshot-4.png "alt =" 154846482.png"/>

4. If the client and VS are in the same network segment, the client (192.168.57.135) will receive the reply message. If the network segment is exceeded, the message is returned to the user through the gateway/router over the Internet.

Realserver 192.168.122.33 255.33.example.com
Realserver 192.168.122.34 255.34.example.com
Load Balance 192.168.122.82 255.82.example.com

Virtual IP 192.168.122.122

Perform the following steps on Route 82:

# Yum install ipvsadm-y # Load rule # ipvsadm-C (clear ipvs Forwarding Table) # ipvsadm-A-t 192.168.122.122: 80-s rr (-A: Add A virtual service; -t: tcp Service;-s Scheduling Algorithm) # ipvsadm-a-t 192.168.122.122: 80-r 192.168.122.33: 80-g # ipvsadm-a-t 192.168.122.122: 80-r 192.168.122.34: 80-g # Save rule #/etc/init. d/ipvsadm save # bind vip # ifconfig eth0: 0 192.168.122.122 netmask 255.255.255.0

Realserver must block arp cache:

Perform the following steps on ipv33:

# Disable arp method 1 # yum install arptables_jf-y # arptables-a in-d 192.168.122.122-j DROP # arptables-a out-s 192.168.122.122-j mangle -- mangle-ip-s 192.168.122.33 #/etc/init. d/arptables_jf save method 2 # vim/etc/sysctl. conf net. ipv4.conf. lo. arp_ignore = 1 net. ipv4.conf. lo. arp_announce = 2 net. ipv4.conf. all. arp_ignore = 1 net. ipv4.conf. all. arp_announce = 2 # sysctl-p # ifconfig eth0: 0 192.168.122.122 netmask 255.255.255.255 # yum install httpd-y # echo 'hostname'>/var/www/html/index.html #/etc/init. d/httpd start

Perform the following steps on ipv34:

# Disable arp method 1 # yum install arptables_jf-y # arptables-a in-d 192.168.122.122-j DROP # arptables-a out-s 192.168.122.122-j mangle -- mangle-ip-s 192.168.122.34 #/etc/init. d/arptables_jf save method 2 # vim/etc/sysctl. conf net. ipv4.conf. lo. arp_ignore = 1 net. ipv4.conf. lo. arp_announce = 2 net. ipv4.conf. all. arp_ignore = 1 net. ipv4.conf. all. arp_announce = 2 # sysctl-p # ifconfig eth0: 0 192.168.122.122 netmask 255.255.255.255 # yum install httpd-y # echo 'hostname'>/var/www/html/index.html #/etc/init. d/httpd start

Test:
Access 192.168.122.122 and refresh the webpage repeatedly. If the webpage appears different each time, the configuration is successful.

VS/TUN

VS/TUN technology requires servers, that is, all servers must support "IP Tunneling" or "IPEncapsulation" protocols. Currently, VS/TUN backend servers mainly run Linux operating systems. In the VS/TUN cluster system, the Server Load balancer only schedules requests to different backend servers, and the backend servers directly return the response data to users. In this way, the Server Load balancer can process a large number of requests. It can even tune hundreds or more servers (servers of the same size) without becoming a bottleneck in the system. Even if the load Scheduler only has Mbps full-duplex NICs, the maximum throughput of the entire system can exceed 1 Gbps. Therefore, VS/TUN can greatly increase the number of servers scheduled by the load scheduler. The VS/TUN scheduler can schedule hundreds of servers without becoming a System Bottleneck and can be used to build high-performance super servers.

Realserver 192.168.122.33 255.33.example.com
Realserver 192.168.122.34 255.34.example.com
Load Balance 192.168.122.82 255.82.example.com

Virtual IP 192.168.122.122

On route 82:

# Vi/etc/sysctl. conf net. ipv4.ip _ forward = 1 # sysctl-p load rule # ipvsadm-C # Clear previous rules # ipvsadm-A-t 192.168.122.122: 80-s rr # ipvsadm-a-t 192.168.122.122: 80-r 192.168.122.33: 80-I # ipvsadm-a-t 192.168.122.122: 80-r 192.168.122.34: 80-I save rule # service ipvsadm save ifconfig eth0: 0 192.168.122.122 netmask 255.255.255.255 up

RealServer settings
The Default Gateway points to the ctor lan ip address 192.168.122.82.

#vi /etc/sysctl.conf    net.ipv4.ip_forward = 1    net.ipv4.conf.all.arp_ignore = 1    net.ipv4.conf.all.arp_announce = 2#sysctl -p#ifconfig tunl0 192.168.122.122 netmask 255.255.255.255 up#route add -host 192.168.122.122 dev tunl0#echo `hostname` > /var/www/html/index.html#service httpd start

Test
Select a host to access http: // 192.168.122.122 and refresh the webpage repeatedly.

Lvs faq:

1. How does LVS/DR process request packets and modify the IP packet content?

1.1vs/dr itself does not care about the information above the IP layer, even if the port number is also the TCP/IP protocol stack to determine whether the correct, vs/dr itself mainly do the following:
1) receive client requests and select the ip address of a realserver Based on the server Load balancer algorithm you set;
2) use the mac address corresponding to the selected ip address as the target mac address, and then encapsulate the IP package into a frame and forward it to this RS;
3) record the connection information in hashtable. Vs/dr does very few things and is also very simple, so it is very efficient, not much worse than the Hardware load balancing device.
The general flow of data packets and data frames is as follows: client --> VS --> RS --> client

2. Why does RealServer need to configure the VIP on the lo interface and the VIP on the egress Nic?
2.1 To enable RS to process IP packets whose destination address is vip, RS must first receive the packet. Configure the vip address on lo to receive the packet and return the result to the client.
2.2 The answer is that the VIP cannot be set on the egress Nic. Otherwise, the client's arprequest will be responded, resulting in disorder of client/gateway/arp table, and the entire loadbalance will not work properly.

3. Why does RealServer suppress arp frames?
This issue has been explained in the previous issue. Here we will further discuss it with the Implementation command. We will make the following adjustments during implementation and deployment:
Echo "1">/proc/sys/net/ipv4/conf/lo/arp_ignore
Echo "2">/proc/sys/net/ipv4/conf/lo/arp_announce
Echo "1">/proc/sys/net/ipv4/conf/all/arp_ignore
Echo "2">/proc/sys/net/ipv4/conf/all/arp_announce
I believe that many people will not understand what their role is, but they must know that they must. I am not going to discuss it in detail here. I just want to make a few notes to add.
3.1
Echo "1">/proc/sys/net/ipv4/conf/lo/arp_ignore
Echo "2">/proc/sys/net/ipv4/conf/lo/arp_announce
You can skip these two steps because arp has no significance for logical interfaces.
3.2 If your RS's external network interface is eth0
Echo "1">/proc/sys/net/ipv4/conf/all/arp_ignore
Echo "2">/proc/sys/net/ipv4/conf/all/arp_announce
What is actually to be executed is:
Echo "1">/proc/sys/net/ipv4/conf/eth0/arp_ignore
Echo "2">/proc/sys/net/ipv4/conf/eth0/arp_announce
Therefore, I personally suggest adding the above two to your script, because in case the default values of the above two are not 0, there may be problems.

4. Why are LVS/DR loadbalancer (director) and RS in the same network segment?
From the first question, we should understand how vs/dr forwards requests to RS? It is implemented at the data link layer, and the ctor must be in the same network segment as RS.

5. Why does the eth0 interface have an ip address (DIP) in addition to the VIP address on ctor )?
5.1 if keepalived or other tools are used for HA or LoadBalance, DIP is required for health check.
5.2 HA or Load Balance without health check mechanism has no practical significance.

6. Do I need to enable LVS/DRip_forward?
No. Because director and realserver are in the same network segment, you do not need to enable forwarding.

7. In lvs/dr, the director vip netmask does not need to be set to 255.255.255.255 or

Route add-host $ VIP dev eth0: 0

Director's vip is intended to be announced as a normal ip address. Do not make such a special announcement.

If you have any questions, please correct them !!!!!!!!!!!!!!

Computer College, Xi'an University of Petroleum

Wang zyin

Mailofwzy@163.com

This article is from "Wang zyin's blog", please be sure to keep this source http://wangziyin.blog.51cto.com/6948950/1297377