Fundamental Principles of server load balancer

1. What is load?

Load is the "resource" of the server: CPU and IO.

The former is suitable for computing-intensive tasks, and the latter corresponds to data-intensive tasks.

2. Prerequisites for server load balancer

Load (or CPU/IO resources)Split, Divided into different computers (or CPU cores.

To achieve this, the underlying operations should be interrupted and scheduled, and IO can be spread over multiple channels. Otherwise, a simple "this: jmp this" command (jumping back to itself) will make the computer unable to do anything, and I/O will interrupt and block the bus without interruption, computers can do the same thing (but normally I/O bus is not as fast as the CPU internal bus)

3. Essence of server load balancer

Since the load can be divided into different computers (distributed systems, how to maintain the consistency between these computers, how to maintain the ACID nature of transactions, how to improve availability, CAP/BASE is another topic ), the key to the rest is the externalLarge-scale concurrent requestsDoRouting.

First, assume that each request is processed by a computer on the server. If the external request load is evenly distributed to each computer node, the server load balancer will be successful.

Without considering the backend technology of distributed databases, the core of server load balancer is routing. This route is actually a Hash function.

What is the input of a function? All external user request parameters that can be differentiated, such as user ID, session ID, request parameters, network layer address, and so on.

What is output? Assume that the number of the computer that finally processes the request is 1 to N, then the output of the route Hash function is an integer between [1. N.

Obviously, if the input is even (random), the Hash function can adopt linear Hash operations to ensure that the load balancing falls on the N processing nodes. Undoubtedly, sometimes the input may be uneven and there are too many requests falling in a certain local range, which may lead to hot spots/bottlenecks, if the total number of input samples for this part is not large compared to the total input space, a non-linear Hash algorithm (such as SHA1/256) can be found theoretically ), so that the input can still fall evenly in the range [1 .. n.

4. Multi-layer routing

In practical application, the preceding Hash operation is not executed on only one point, but may be performed on multiple layers. For example, the network layer LB technology LVS (Linux Virtual Server) front-end proxy (HTTP reverse proxy with Round Robin, database connection driver proxy), DNS allocation

This is easily reminiscent of the multi-layer forward/feedback network in the neural network (that deep learning does not increase the layer depth of the intermediate node, multi-layer routing is actually a non-linear Hash ing in mathematics.

This is almost the essence of server load balancer. For practical applications, the key lies in the design of a Hash function, so that specific data of this application can be evenly distributed to the backend processing nodes.

Fundamental Principles of server load balancer