High Availability-LVS (Linux Virtual Server)

High Availability-Linux Virtual Server (LVS) Linux Virtual Server terms: ctor: Also known as a scheduler, LVS front-end device; realserver: Also known as a real internal Server, is actually providing services; VIP: The published IP address, that is, the IP Address requested by the customer; DIP: the address for communication between the scheduler and the realserver; LVS working mode introduction LVS Server Cluster load balancing has three ways, NAT, DR And TUN, the following briefly talk about the differences between these three methods: LVS-NAT: the idea of this method is to implement network layer (IP layer) Data spoofing, which replaces the destination address sent from the client to the redirector data IP package. 1. In the network environment, one ctor + N realservers. director and realserver are in the same private network segment. director is the default gateway of realserver. Only director has public IP addresses and can be exposed on the wide area network. 2. The client requests a public IP (director) first. the IP address of the IP packet in the request packet is replaced with the IP address of a realserver selected by director according to the load balancing policy. 3. After the realserver responds to the request, it generates a returned data packet. The source address of the returned data packet is the IP address of the realserver, And the destination address is the IP address of the client server. Because the default gateway of realserver is redirector, although the destination address of the returned IP packet is the IP address of the client, the returned data packet is first sent back to redirector. Redire implements spoofing again and changes the source address of the returned IP packet to its own IP address. Then, it is forwarded to the vswitch and returned to the client. The redirector task is to implement two IP layer spoofing modifications, one is to modify the target address of the request data packet, the purpose of this modification is to achieve Load Balancing distribution of data. The other is to modify the source address of the response packet to hide the realserver, so that the user does not feel the existence of the realserver. 3. Restrictions: the throughput of the entire cluster is limited by the bandwidth of redirector (mainly the egress bandwidth ). LVS-DR: the idea of this method is to implement data spoofing on the data link layer and modify the Mac address of network frame data. 1. In the network environment, one ctor + N realservers, director and realserver both have public IP addresses and are exposed on the wide area network. In addition, realserver also has an ip alias that is the same as the director IP address. In other words, the realserver has two ip addresses, a real IP address, and an IP alias (that is, a public IP address) that is the same as the direcotr address. A public ip address is the IP address opened to the client. Realserver also needs to make a configuration so that they ignore all ARP broadcasts targeting public IP addresses. When the system arp broadcast asks which mac address has a public IP address, only the scheduling server will respond, data sent from outside will not be sent to the actual server. 2. The client requests a public IP address (director) first. Because the network environment has been configured that only director has responded to ARP broadcast, the mac address of the request packet will be changed to the mac address of the realserver. That is to say, redireimplements link layer spoofing to replace the target mac address of the frame data with the mac address of a realserver determined based on the load balancing policy. 3. The realserver responds to the request that the realserver receives the mac frame and then assembles the mac frame into an ip package. It finds that the destination address in the ip package is the same as its own ip address alias and there is no problem. Proceed with the process, (This is why the ip alias must be the same. If the ip alias is different, the operating system may directly ignore it.) generate a response data and send it back. At this time, because redirector is not the default gateway, the data will be directly sent to the wide area network, and the WAN will send the data to the client. 4. Advantage: The returned data packet does not need to pass through redirector, and there is no redirector Bandwidth bottleneck. In principle, the bandwidth of the cluster is the sum of the bandwidth of all servers. Of course, they cannot exceed the bandwidth of the connected Wan switch. 5. Restrictions: You need to purchase multiple public IP addresses. director and realserver must be in the same wan cidr block, that is, on the same vswitch. Why must I be in the same CIDR Block? If the realserver is in another CIDR Block, redirechanges the entire packet and mac frame before sending it to the switch. The switch finds that the mac address cannot be found in its wan and cannot forward it. LVS-TUN: This method is to break through the limitations of the LVS-DR within the same network segment. It does not cheat, but is a bright and bright communication, and is encapsulated twice at the network layer. 1. In the network environment, one ctor + N realservers, director and realserver both have public IP addresses and are exposed on the wide area network. Public ip addresses are different from each other, and there is no alias restriction or need not be in the same network segment. 2. The client requests the client to send data to redirector, and redireputs the IP package as a valid load into a new IP package, determine the ip address of a specific realserver as the destination address of the new ip package based on the scheduling policy. These new IP packets are completely compliant with network protocols, and there is no deception. Therefore, these IP packets must pass through the wan CIDR block to reach the specified realserver. 3. After the realserver responds to the data obtained by the realserver, it needs to do something, extract the payload of the IP packet, and then use these loads as the IP packet to form TCP, then form the final request data up. Based on the request data, after the realserver generates the returned data, it returns the result to the client. 4. Advantage: Like LVS-DR, there is no redirector outbound Bandwidth bottleneck. 5. Disadvantages: Additional packaging and package settlement are required. LVS configuration LVS-NAT configuration 1, realserver Configuration Requirements: configure the internal private network address, the default gateway pointing to the scheduling Server 2, Directer Configuration Requirements basic configuration: the scheduling server requires two NICs (one for external network and one for internal network. One module can also be used to configure Sub-interfaces. The external VIP and DIP are both configured on the same Nic, but this will further reduce the performance of the scheduler. We recommend that you use a dual Nic) disable selinux and iptablessetenforce 0 service iptables stop (to avoid unnecessary troubles, disable these two services on each server) enable the packet forwarding function echo "1">/proc/sys/net/ipv4/ip_forward1_sadm-A-t $ VIP: $ Port-s rr explanation: -A indicates adding A cluster service (multiple services can be added, for example, adding A web 80 and an https 443);-t indicates tcp; -s indicates that the scheduling algorithm is round robin (a total of 10 scheduling algorithms can be selected as needed) ipvsadm-a-t $ VIP: $ Port-r $ DIP: $ Port-m explanation:-a indicates adding a realserver, followed by the address Port of the previously defined Cluster service.-r indicates adding a specific real The address of the server,-m indicates that the mode is NAT mode LVS-DR configuration 1, Realserver Configuration Requirements basic configuration first configure the limit arp, otherwise after the address is configured, the address conflict occurs, you can modify the kernel parameters. In linux, by default, arp broadcasts of IP addresses on all interfaces are advertised on the interface, and arp requests of IP addresses on all interfaces are responded to. arp_announce: advertise arp broadcasts of IP addresses on all interfaces on the interface. 1. Restrict broadcast notification responses to arp requests from other devices as much as possible on the interface (not strict enough). 2: only the arp broadcast arp_ignore of the IP address on this interface is advertised. The maximum arp response level is 0. For arp requests from other devices, the arp response level of the IP address on all other interfaces is 1: for arp requests from other devices, only reply to the arp response echo "1">/proc/sys/net/ipv4/conf/lo/arp_ignoreecho "2">/proc/sys/net/ipv4 /conf // lo/arp_announceecho "1">/proc/sys/net/ipv4/conf/all/arp_ignoreecho "2">/proc/sys/net/ipv4/conf/ all/arp_announce configures VIP on the lo port of realserver, this configuration limits that the VIP will not generate a MAC address table on the physical switch, so as to avoid IP conflict ifconfig lo: 1 $ VIP broadcast $ VIP netmask broadcast bandwidth 255.255ifconfig eth0 $ DIP up note that the broadcast address of this VIP interface is still VIP, which limits its broadcast. The subnet mask is 32 bits, in the following configurations on the scheduler, pay attention to this configuration for special routes, so that the lo port with the source address as VIP of the destination VIP package goes out route add-host $ VIP dev lo: 1Directer configuration requires VIP and DIP configuration. VIP is configured on the sub-interface of the physical network card ifconfig eth0 $ DIP broadcast $ VIP netmask 255.255.0 upifconfig eth0: 1 $ VIP broadcast $ VIP netmask route 255.255.255 up configure a special route. The goal is to route the VIP package from the physical sub-interface configured with the VIP to route add-host $ VIP dev eth0: 1 cluster configuration ipvsadm-A-t $ VIP: $ Port-s rr explanation:-A indicates adding A cluster service (the configuration is the same as that in NAT) ipvsadm-a-t $ VIP: $ Port-r $ DIP: $ Port-g explanation: the other is similar to NAT. In the final mode, change to-g, that is, DR mode.