Web infrastructure: Load balancing and LVS

In large-scale Internet applications, load balancing devices are an essential node, originating from high concurrency and large traffic impact pressures in Internet applications, where we typically deploy multiple stateless application servers and several stateful storage servers (databases, caches, and so on) on the server side.

I. The role of load balancing

The task of the load balancing device is to take the application server traffic as the entrance, first select the most suitable server, then forward the client's request to this server processing, the client to the real service side of transparent forwarding. In recent years, the very fire of "cloud computing" and distributed architecture, is essentially the back-end server as a computing resource, storage resources, a management server encapsulated as a service to provide, the client does not need to care about the real service is which machine, in its view, It's as if it's a server with a near-limitless capacity, and essentially, a backend cluster that really serves.

The topology of a typical Internet application is this:

Ii. Types of load Balancing

Load balancing can be either a hardware device or a software load.

Commercial hardware load equipment costs are usually high (a hundreds of thousands of million is normal), so we will use the soft load when the conditions permit, the two core problems of soft load solution are: Who to choose, forwarding, the most famous is LVs (Linux Virtual Server).

Third, soft load--lvs

LVS is a four-layer load balancer, that is, based on the fourth layer of the OSI model-the transport layer, where our familiar Tcp/udp,lvs support tcp/udp load balancing.

The retransmission of LVS is implemented mainly by modifying the IP address (NAT mode, which is divided into the source address to modify the Snat and the target address to modify the Dnat) and the target Mac (Dr Mode).

So why is LVS doing load balancing on the fourth floor?

First of all, LVS is not like haproxy, such as seven-layer soft load for the HTTP packet, so the seven-layer load can do the URL parsing, etc., LVS can not be completed. Second, a user access is connected to the server after the exchange of data packets implemented, if the third layer of Network Load Balancing, then will lose the "connection" semantics. The soft-load object should be a user who has already established a connection, not a lone IP packet. As you'll see later, the LVs machine actually connects with the user through the TCP three handshake instead of the real server, so LVS needs to be concerned with the state of the "Connection" level.

There are 4 main types of LVS operating modes:

DR

Nat

Tunnel

Full-nat

Here's a quick introduction to the usual Dr, NAT, and Full-nat.

1. DR

The request is accepted by LVS and is returned directly to the user by the Real service server (Realserver, RS), which is returned without LVS.

In Dr Mode, LVS is required and the same VIP is bound (RS is implemented by binding the VIP to loopback).

When a request comes in, LVS only need to change the MAC address of the network frame to a certain Rs Mac, the package will be forwarded to the corresponding RS processing, note that at this time the source IP and destination IP have not changed, LVS just do a bit counter switch.

RS received the package of LVS forwarding, link layer found that the MAC is its own, to the network layer above, found that the IP is also their own, so this package is legally accepted, RS does not perceive the presence of LVS.

When RS returns to the response, it is returned directly to the source IP (that is, the user's IP) and is no longer passed LVS.

Dr Mode is one of the best performance modes.

2. NAT

NAT (Network address translation) is a technique for external network and intranet addresses mapping.

In NAT mode, the network report must be processed by LVS. LVS is required as a gateway for Rs.

When the package arrives at the LVS, the LVS does the destination address translation (DNAT) and changes the target IP to the IP of Rs. RS receives the packet as if it were sent directly to it by the client.

When the RS is processed, the source IP is the RS IP and the destination IP is the IP of the client when the response is returned.

At this time, the RS packet through the Gateway (LVS) relay, LVS will do the source address translation (SNAT), the source address of the package to the VIP, so that the package to the client appears as if the LVS directly returned to it. The client cannot perceive the presence of the back-end Rs.

3, Full-nat

Both the DR and Nat modes inevitably have a problem: LVs and RS must be in the same VLAN, otherwise LVS cannot be the gateway for Rs.

The two problems that arise are:

1, the same VLAN restrictions cause operation is inconvenient, RS cross VLAN can not access.

2. The level expansion of LVS is restricted. When RS expands horizontally, one day its single-point LVS will become a bottleneck.

Full-nat from this, the solution is LVS and Rs cross-VLAN problem, and after the cross-VLAN problem solved, LVS and RS no longer have a dependency on the VLAN, you can do multiple LVS corresponding to multiple RS, solve the problem of horizontal expansion.

Full-nat the main improvement over NAT is that, on the basis of Snat/dnat, plus another conversion, the conversion process is as follows:

In the process of transferring the package from LVs to RS, the source address is replaced with the LVS intranet IP from the client IP.

Inter-network IP can communicate across VLANs through multiple switches.

When RS finishes processing the received packet, it returns the packet back to the LVS intranet IP, which is not restricted to VLANs.

After the LVS receives the package, the NAT mode modifies the source address, then the target address from the RS packet is changed from the LVS intranet IP to the client IP.

The main idea of Full-nat is to change the communication between the gateway and its lower machine to the normal network communication, which solves the problem of cross-VLAN. In this way, the deployment of LVS and RS will no longer have any restrictions on VLANs, greatly improving the convenience of operational deployment.

4. Session

Client-to-server communication, a request may contain multiple TCP packets, LVS must ensure that the same connection TCP packet, must be forwarded to the same RS, otherwise it is a mess. To ensure this, the LVS internally maintains a session hash table that can be found on which RS should be forwarded via some information from the client.

5. LVS Clustering

After adopting Full-nat mode, we can build the cluster of LVS, and the topological structure such as:

6. Disaster tolerance

Disaster tolerance is divided into Rs disaster tolerance and LVS disaster tolerance. The disaster tolerance of the

Rs can be achieved through the LVS regular health check, and if an RS loses its heartbeat it is considered to be offline and will not be forwarded to the RS. The disaster tolerance of the

LVs can be achieved by means of the primary and Standby + heartbeat. When the main LVS loses its heartbeat, the LVS can be replaced immediately as a hot spare.

Disaster recovery is mainly done by keepalived.