Architecture High-performance Web site tips (vi)--load balancing

What is load balancing?

When the performance of a single server reaches its limit, we can use a server cluster to improve the overall performance of the site. Then, in the server cluster, you need a server to act as the dispatcher's role, all of the user's requests will be first received by it, the dispatcher then according to the load situation of each server to assign the request to a back-end server to handle.

So in this process, dispatcher how to properly assign tasks, ensure that all back-end servers will be full performance, so as to maintain the overall performance of the server cluster is optimal, this is the load balancing problem.

Four ways to implement load balancing are detailed below.

HTTP redirection for load Balancing process description

When the user initiates a request to the server, the request is first intercepted by the cluster dispatcher, and the dispatcher chooses a server based on an allocation policy, encapsulates the IP address of the selected server in the Location field in the header of the HTTP response message, and sets the status code of the response message to 302. Finally, the response message is returned to the browser.

When the browser receives a response message, parses the Location field and initiates a request to the URL, then the specified server processes the user's request and finally returns the result to the user.

In the process of using HTTP redirection to implement server cluster load balancing, a server is required as a request dispatcher. One operation of the user needs to initiate two HTTP requests, send a request to the dispatch server at a time, obtain the IP of the backend server, send a request for the second back-end server, and obtain processing results.

Scheduling policy

After the dispatch server receives the user's request, it is up to the scheduling policy used by the dispatch server to choose which back-end server to process the request.

1. Random allocation policy
   When the dispatch server receives a user request, it is possible to randomly decide which backend server to use, and then encapsulate that server's IP in the Location property of the HTTP response message and return it to the browser.

2. Polling policy (RR)
   The dispatch server needs to maintain a value that records the IP of the last assigned back-end server. Then when a new request arrives, the dispatcher assigns the request to the next server in turn.

Because the polling policy requires the dispatcher to maintain a value for logging the server IP that was last allocated, it requires additional overhead, and since this value is a mutex, the mutex resource needs to be locked to avoid thread security issues when multiple requests come together, thereby reducing performance. The random allocation policy does not need to maintain additional values, and there is no thread safety issue, so performance is higher than polling.

Pros and Cons analysis

The use of HTTP redirection to achieve server cluster load balancing is easier to implement, logic is relatively simple, but the shortcomings are more obvious.

In the HTTP redirection method, the dispatch server acts only when the client initiates a request to the Web site for the first time. After the dispatch server returns the response information to the browser, the client's subsequent operations are based on the new URL (that is, the back-end server), after which the browser does not have a relationship with the dispatch server, resulting in several problems:

• Because of the different user's access time, the access page depth is different, thus each user to the respective back-end server caused the pressure also different. The dispatch server in the scheduling, can not know how much the current user will put pressure on the server, so this way can not achieve a true load balancing, but the number of requests evenly distributed to each server.
• If the back-end server assigned to the user fails, and if the page is cached by the browser, the request is sent to the failed server when the user accesses the site again, causing the access to fail.

What is DNS load balancing DNS?

Before we understand DNS load balancing, we first need to understand the process of DNS domain name resolution.

We know that packets are transmitted over the network using IP addresses, and we use domain names to access websites for the convenience of user memory. Then, before we access the site through the domain name, we first need to resolve the domain name to an IP address, this work is done by DNS. This is the domain name server.

The request we submit is not sent directly to the website you want to visit, but first to the name server, which will help us resolve the domain name to an IP address and return it to us. We will not initiate a request to the IP until we receive the IP.

Then, the DNS server has a natural advantage, if a domain name points to multiple IP addresses, each time the domain name resolution, DNS only select an IP to return to the user, you can achieve server cluster load balancing.

Specific practices

First, we need to point our domain name to a number of back-end servers (to resolve a domain name to multiple IP), and then set the scheduling policy, then our preparation is complete, the next load balancer is fully implemented by the DNS server.

When a user initiates a request to our domain name, the DNS server automatically returns the appropriate IP to the user based on our pre-set scheduling policy, and the user initiates the request to that IP.

Scheduling policy

The general DNS provider provides a number of scheduling strategies for us to choose from, such as random allocation, polling, and distribution of the nearest server to the requester's geographic location.

Pros and Cons analysis

The greatest advantage of DNS load balancing is the simplicity of the configuration. The dispatch of the server cluster is fully committed by the DNS server, so we can focus on the backend server to ensure their stability and throughput. And without worrying about the performance of the DNS server, even with a polling strategy, its throughput rate is still excellent.

In addition, DNS load balancing is highly scalable, and you can resolve more IPs for a single domain name without worrying about performance issues.

However, because the cluster scheduling authority to the DNS server, so we do not have the means to control the dispatcher, no way to customize the scheduling strategy.

DNS servers also have no way of knowing the load on each server, so there is no way to achieve true load balancing. It is just like HTTP redirection, except that all requests are evenly distributed to the backend server.

In addition, when we find a backend server failure, even if we immediately remove the server from the domain name resolution, but because the DNS server will have a cache, the IP will remain in the DNS for a period of time, it will cause some users can not access the site. This is a deadly problem! Fortunately, this problem can be solved with dynamic DNS.

Dynamic DNS

Dynamic DNS allows us to dynamically modify the DNS server's domain name resolution through the program. Thus, when our monitoring program discovers that a server has been hung, it can immediately notify DNS to delete it.

Sum up

DNS load balancing is a rugged approach to load balancing, which is only described here and is not recommended.

Reverse Proxy load Balancing What is reverse proxy load balancing?

The reverse proxy server is a server located in front of the actual server, all requests to our web site to go through the reverse proxy server, the server according to the user's request or directly return the results to the user, or the request to the back-end server processing, and then returned to the user.

Previously, we introduced the caching of static pages and commonly used dynamic pages using a reverse proxy server. Next we introduce the more common features of the reverse proxy server--load balancing.

We know that all requests sent to our website are first passed through the reverse proxy server. Then, the reverse Proxy Server can act as the dispatcher of the server cluster, which can forward the request to a suitable server based on the load of the current back-end server, and return the processing results to the user.

Advantages

1. Hides the back-end server.
   In contrast to HTTP redirection, reverse proxies can hide back-end servers, and all browsers do not interact directly with back-end servers, thereby ensuring the dispatcher's control and improving the overall performance of the cluster.
2. Fail over
   The reverse proxy is able to remove the failure node more quickly than DNS load balancing. When the Monitor discovers that a back-end server fails, it can notify the reverse proxy server in time and delete it immediately.
3. Assign Tasks reasonably
   Neither HTTP redirection nor DNS load balancing can achieve true load balancing, which means that the dispatch server cannot allocate tasks based on the actual load of the back-end servers. However, the reverse proxy server supports manually setting weights for each back-end server. We can set different weights according to the configuration of the server, different weights will lead to the probability of being selected by the dispatcher.
   
   

Disadvantages

1. Dispatcher pressure is too high
   Because all requests are processed by the reverse proxy server, the throughput reduction of the dispatch server reduces the overall performance of the cluster directly when the request volume exceeds the maximum load of the dispatch server.
2. Restriction expansion
   When the backend server does not meet the large throughput, it is necessary to increase the number of back-end servers, there is no way to increase indefinitely, because it will be constrained by the maximum throughput of the dispatch server.
   
   

Sticky sessions

A reverse proxy server can cause a problem. If a back-end server processes the user's request and saves the user's session or caches, then when the user sends the request again, there is no guarantee that the request will still be processed by the server that saved its session or cache, if it is handled by another server. The previous session or cache could not be found.

Workaround 1:
You can modify the task assignment policy for the reverse proxy server, which is more appropriate for the user IP as the identity. The same user IP is handled by the same back-end server, thus avoiding sticky session issues.

Workaround 2:
The server ID of the request can be labeled in the cookie, and when the request is submitted again, the dispatcher assigns the request to the server marked in the cookie for processing.

Architecture High-performance Web site tips (vi)--load balancing