A simple and thorough understanding of load balancing (Balance load)

Load Balancing (Balance) is one of the factors that must be considered in the design of distributed system architecture.
It usually means to allocate the request/data "evenly" to multiple operation units, and the key of load balancing is "uniformity".

The uniformity here is not the 10÷2 = 5 we imagined, which is even. No.
The Common Internet distributed architecture is divided into the client layer, the reverse proxy nginx layer, the site layer, the service layer, the data layer.

common load-balancing scenarios

The Common Internet distributed architecture is divided into the client layer, the reverse proxy nginx layer, the site layer, the service layer, the data layer. You can see that each downstream has more than one upstream call, just to do, each upstream is evenly access each downstream, you can achieve "the request/data" evenly distributed to multiple units of operation.

load balancing of "client layer-> Reverse proxy layer"

"Client layer" to "reverse proxy layer" load balancing, is through "DNS polling" implemented: Dns-server for a domain name configured with multiple resolution IP, each DNS resolution request to access Dns-server, will poll back to these IP, ensure that each IP resolution probability is the same. These IP is nginx IP, in order to achieve each Nginx request allocation is also balanced.

load balancing of "reverse proxy layer-> site layer"

The "Reverse proxy layer" to the "site layer" load balancing is implemented through "Nginx". By modifying nginx.conf, a variety of load balancing strategies can be implemented:

1 Request polling: Similar to DNS polling, requests are routed to individual web-server

2 Minimum connection route: which Web-server is less connected and which web-server to route to

3 IP Hash: According to the access of the user's IP hash by Web-server, as long as the user's IP distribution is uniform, the request is also uniform, the IP hash equalization method can be achieved, the same user's request fixed to the same web-server, this strategy is suitable for stateful services, For example, session (58 Shen Jianbei Note: You can do this, but strongly do not recommend this, the site layer stateless is one of the basic principles of distributed architecture design, session best put to the data tier storage)

4) ...

load balancing of the "Site Layer-> service layer"

The load balancing of the site layer to the service layer is implemented through the service connection pool.

The upstream connection pool establishes multiple connections to downstream services, and each request "randomly" selects the connection to access downstream services.

The previous article "Rpc-client implementation Details" has detailed load balancing, failover, timeout processing details of the description, welcome to click Link Lookup, no longer open here.

load balancing of the "Data tier"

In the case of large amount of data, because the data layer (Db,cache) involves the horizontal segmentation of the data, the load balance of the data layer is more complicated, it is divided into "Data balance" and "Request Balance".

The balance of data refers to: each service (Db,cache) after horizontal segmentation, the amount of data is similar.

The equalization of requests means that each service (Db,cache) after the horizontal segmentation is similar in volume.

There are several common horizontal segmentation methods in the industry:

one, according to the range level segmentation

For each data service, a range of data is stored, as in the example above:

USER0 Service, storage uid range 1-1kw

User1 service, storage uid range 1kw-2kw

The benefits of this program are:

(1) Simple rules, service only to judge the UID range can be routed to the corresponding storage services

(2) Good balance of data

(3) Easy to expand, can be added at any time a uid[2kw,3kw] data services

The disadvantages are:

(1) The requested load is not necessarily balanced, in general, the new registered users will be more active than the old users, large range of service request pressure will be greater

second, according to the ID hash level segmentation

For each data service, store some of the data in the hash of a key value, as shown in the above example:

USER0 service, storing even UID data

User1 service, storing odd uid data

The benefits of this program are:

(1) The rule is simple, the service only can be routed to the corresponding storage service with the UID hash

(2) Good balance of data

(3) Better uniformity of request

The disadvantages are:

(1) Not easy to expand, expand a data service, the hash method changes, you may need to do data migration

Summary

Load Balancing (Balance) is one of the factors that must be considered in the design of distributed system architecture, it usually means to allocate the request/data "evenly" to multiple operation units, and the key of load balancing is "uniformity".

(1) The load balancing of "client layer" to "reverse proxy layer" is implemented through "DNS polling"

(2) "Reverse proxy layer" to "site layer" load balance, is achieved through "Nginx"

(3) The load balancing of "site layer" to "service layer" is realized through "Service connection pool"

(4) "Data layer" load balancing, to consider "data balance" and "request balance" two points, the common way is "in accordance with the scope of horizontal segmentation" and "Hash level segmentation"