How do Web sites like Facebook handle billions of of requests and maintain high availability?

How do Facebook's Web sites handle billions of of requests and maintain high availability, and the answer is load balancing, which this article will explore.

What is load balancing

Load balancing is an allocation strategy for many collaborative work resources (usually computers). They are often used to increase capacity and reliability.

To facilitate discussion of load balancing, for service extensions I assume the following two points:

I can run any number of instances

Any request can reach any instance

The first assumes that the service is stateless (or shared as a Redis cluster). The second hypothesis is not actually necessary (such as viscous load balancing), but the assumption in this article is easy to discuss.

Here is the load balancing technique I'm going to discuss:

Application Layer (OSI layer seventh) load balancing (HTTP, HTTPS, WS)

Transport Layer (OSI layer fourth) load balancing (TCP, UDP)

Network layer (OSI third layer) load balancing

DNS Load Balancing

Multiple child domains manual load Balancing

Anycast (anycast)

Finally, there are other points of knowledge:

Latency and throughput

The server returns directly

These techniques are roughly sorted according to the steps required to increase site traffic. For example, the application layer load balancing is the first thing to do (much earlier than the anycast). The first three technologies improve throughput and availability, but there are single points of failure. The remaining three technologies improve throughput while avoiding single points of failure.

To help us understand load balancing, let's look at a simple service extension.

Note: Each extension technique uses a non technical metaphor (shop shopping). These metaphors are merely descriptions of the ideas behind the technology and are not entirely accurate.

Service

Let's say we're building a massive service. As shown in the following image:

The system does not handle a lot of traffic, and if it goes down, the entire application stops.

metaphor

You go to the store and the only checkout line.

You buy your goods, if there is no cashier, you can not complete the purchase.
Application Layer (OSI layer seventh) load Balancing

In order to carry more traffic, the first use is the application layer load balancing. The application layer is the seventh layer of OSI. It includes HTTP, HTTPS, and WebSockets. A very popular and time-tested application layer load balancer is Nginx. Let's see how it helps us extend our services:

Note that with this technique, we can load a balanced dozens of or hundreds of server instances. The above picture shows only two as examples.

metaphor

The store staff leads you to a special checkout team (with a cashier).

You buy your goods .
Tools

Nginx

Haproxy
Attention

We're going to stop using SSL here (no SSL for distribution)
Transport Layer (OSI layer fourth) load balancing (TCP, UDP)

The last technology helps us to carry a lot of traffic, but it is very useful if we need to load a larger amount of traffic. The transport layer is the fourth layer of OSI, including TCP and UDP. The popular transport layer load balancer has haproxy (this is also used for application layer load Balancing) and Ipvs. Let's see how they help us extend our services:

Application Layer load Balancing + Transport Layer load Balancing can handle the traffic in most cases. But we still have to worry about usability. A single point of failure may occur at the transport layer of the load balancer. We address this issue in the following section of DNS load Balancing.

metaphor

According to customer membership card number has different checkout area. For example, if your membership card number is even, go to the checkout desk near the electrical area, or go to the checkout desk near the food area.

Once you get to the correct checkout area, the store staff leads you to a special checkout team.

You buy your goods .
Tools

Haproxy

Ipvs
Network layer (OSI third layer) load balancing

If we want to continue to expand, we need to increase the network layer load balancing. This is more complicated than the two technologies above. The network layer is the third layer of OSI, including IPv4 and IPV6. The following is the network layer load balancing:

In order to figure out how it works, we need a bit of equivalent routing knowledge (ECMP). When more than one equivalent link arrives at the same address, we use the equivalent route. In simple terms, it allows routers or switches to send packets via different links (which support high throughput) and eventually reach the same address.

We can use this to achieve network layer load balancing, because in our view, each transport layer load balancer is the same. This means that we can view the link from the network layer load balancer to the transport layer load balancer as a link to the same destination. If we bind all load balancers to the same IP address, we can use equivalent routing to distribute traffic between the transport layer load balancer.

metaphor

There are two separate but identical shops across the street, and it all depends on your habits.

Once you arrive at the store, there are different checkout areas according to the customer membership card number. For example, if your membership card number is even, go to the checkout desk near the electrical area, or go to the checkout desk near the food area.

Once you get to the correct checkout area, the store staff leads you to a special checkout team.

You buy your goods .
Tools