Design of server network architecture for a large number of user services (Server Load balancer)

Origin:
Because I plan to develop a server similar to instant messaging, I need to plan the framework for future development. The most important solution is to connect the client to the server. As the number of customers increases (for example, 1 million), online and offline (for example, 10 thousand) at the same time will put a lot of pressure on the system. In addition, Server Load balancer should be considered.

Ideas:
By learning the background load solution that MSN constantly improves when the number of users increases (Google
"Success Secrets of hundreds of millions of users website MySpace"), let us have a foresight. In software design, we will take into account the solutions to these expected problems. In the future, we will also be able to improve these problems.
By studying the QQ login method, we can prepare the online and offline user pressure.

After learning, we can get two solutions:
Solution 1: the client detects the connection response speed of each communication server and then connects to the server with the fastest response speed.
Solution 2: the client actively detects various Server Load balancer servers (hereinafter referred to as "Server Load balancer") and then connects to the Server Load balancer with the fastest response speed. Then, the Server Load balancer allocates the communications server with the smallest load in the managed area to the customer. Finally, the customer actively connects to the communication server allocated.

Analysis:
Solution 1:
Advantage: the client can always connect to the server with the fastest response for the customer.
Disadvantage: You need to detect all communication servers to find the fastest response.
The client connects to the fastest responding server. What advantages does communication bring to its main business? Text communication is generally only 1-2 k in size, and audio and video files can be used for P2P communication by means of holes. In addition, server response in a certain region (such as Mainland China and Hong Kong and Taiwan) is not slow. Therefore, the server speed response speed has little impact on its main business.
The advantage is not good, but the disadvantage will bring a lot of problems in the use process, especially when you need to increase the number of servers. First, the client needs to detect all communication servers. When the number of users is small, the user's online and offline detection of the response speed of each server does not impose any pressure on each server. However, considering the development in the later stages, when the number of users is large, or even huge, QQ, as it is now, will reach 0.1 million of the number of online and offline lines at the same time. If the detection method is still used, the load and network pressure on each server will be high. In addition, the pressure on each server is the same. Under this framework, you cannot eliminate this pressure. Some of your new servers are under this pressure when they are launched. Not worth it. The client must also have the latest communication server list. As the number of communication servers increases, the client will also be affected. This shortcoming will bring the entire development into a vicious circle.
Through analysis, solution 1 is not suitable for more and more business systems for such later users.

Solution 2 analysis:
Advantage: the Server Load balancer ("Server Load balancer" for short) makes full use of server resources to conveniently add new servers, which is transparent to clients.
Disadvantage: the client may be connected to a server with low load, but the response speed is not fast for the customer.
The advantages of this solution are exactly the solution in solution 1. The upgrade is transparent to the client. When the load is high, you can add new communication servers to share the load. You only need to add a server to register with a server balancer. The balancer has a single function, simple transactions, and high throughput. The chance to increase the Server Load balancer is small, so that the address to be tested by the client remains unchanged. To increase scalability, you can design the protocol to be tested multiple times when designing the protocol between the server and the client. That is, the address returned by the probe server can be the address of the Communication Server or the server address can be balanced. In this way, the scalability will be more powerful, and the client can be completely unaffected.
In addition, the client detects the balancer and allows the customer to find the network area with the fastest response speed.
As for the disadvantages, the opposite is the advantage of solution 1. Since the advantages of solution 1 are not an advantage, this disadvantage is naturally not a disadvantage. Can be ignored.

Load Balancing function requirements:
1. Respond to client detection;
2. Return the address of the server with the smallest load to the connected client;
3. Upgrade the old load server table to the client that still uses the old load server table (optional ).
Description: optional. Because the Server Load balancer has a single function, simple transactions, and high throughput. In addition, secondary detection can be performed. In addition, if you reserve some probe addresses on the client at the beginning (all of which point to the same server IP address in the early stage), and then convert them to other expanded Real Server IP addresses in the later stage, this basically eliminates the need to worry about the impact of the increase of the balancer on the client.

Extended thinking:
1. Prevent DDoS attacks. DDoS attacks will make some Server Load balancer unable to respond to the detection of the client. A large number of clients are connected to the Server Load balancer that is not under attack. Because each server is only responsible for the servers managed by itself (usually in the same network area), this will cause a large increase in the communication server load in the region and the network will be very busy. The attacked server cannot process new user requests, so the communication servers managed by the server will no longer have new client pressure, and their load and network pressure will remain unchanged. In this way, you are busy and idle. To prevent this, you can expand the functions of the Server Load balancer. When the load pressure or network pressure in the managed area of the Server Load balancer is too high (reaching a threshold), the Server Load balancer can communicate with other Server Load balancer instances, mutual resource adjustment.

(2009/01/09)
Supplement:
The preceding assumption is that all communication servers have independent IP addresses and are exposed to the Internet. Because the client can actively connect to the client only on the Internet.
In practical applications, this possibility is very small. On the basis of this, only a few fixed IP addresses are exposed, and then servers are added to the Intranet. In fact, such a balance mechanism is still useful. To ensure uninterrupted services, servers must be distributed in different regions. In this way, even if you only have one IP address in each region, this balance mechanism is also useful. First, it is possible to achieve load balancing between regions through communications between the balancer, and then to shoulder the customer's detection pressure. In addition, if there are any later changes to the framework and there are more layers, everything will be easier.

Limited thinking. Welcome to discuss it!