Load Balancing--the key to realize large-scale online system (Part One) (again talk about the technology realization of QQ game million people online)

The author of this article: Sodme
Source of this article: Http://blog.csdn.net/sodme
Disclaimer: This article can be reproduced without the consent of the author, but any reference to this article should be indicated by the author, the source and the information of this declaration. Thank you..

To learn about another article that is quoted in this article, please go to the following address:
Http://blog.csdn.net/sodme/archive/2004/12/12/213995.aspx
The above article was written as early as last year, when the leisure platform, has been thinking about how to achieve an expandable support for millions of people online game platform, and then the idea has, wrote the summary. The meaning of the article is to elaborate on how a millions online system is implemented, not really seriously investigate the QQ game is not the way to achieve.

In recent discussions with the industry, the introduction of the QQ game is not the way I thought, so, today and carefully grasp the QQ game bag, the result is indeed as this brother said, QQ game architecture and I originally conceived that the structure of the difference is indeed not small. Below, I give the QQ millions on-line technology realization plan, and in order to expand, talk about the large-scale on-line system load balance mechanism design.

QQ game from the login and game process, QQ game, also divided into at least three types of servers. They are:
First layer: Login/account Server (login server), responsible for authenticating the user identity, to send the initial information to the client, from the QQ chat software Packaging Common sense, these initial information may include "session key" this kind of information, Subsequent communication between the client and the subsequent server uses this session key for authentication and information encryption;
Second floor: Lobby server (estimated and so called, Game Hall Server), responsible for delivering all room information in the current game to the client, which includes: The ip,port of each room, the current number of people in each room, the room name and so on.
Third level: Game logic Server (Game Logic Server), responsible for handling room logic and room table logic.

From the static statement, the above three-layer structure seems to be compared with the article I have written before is not much different, in fact, the focus is its workflow, QQ game communication flow and my previous ideas are quite divergent, its design ideas and technical level is indeed very good. Specifically, the communication process of the QQ game is this:

1. By client to the login server to send account and password login message, login server based on the results of the verification returned the appropriate information. It is conceivable that, if the login server passes client authentication, it notifies other game hall servers or places where the game Hall server can take the authenticated message and the session key. In short, this checksum success message can be shared between Login server and game Hall server. Once the client receives a message that the login server returns a successful checksum, the login server actively disconnects the client to free up the socket resource. The IP information for the Login server is stored in the Qqgame/config/qqsvrinfo.ini.

2.Client received the login server verification success, and so on, began to enter the game lobby according to the first selection of the game hall, each game Hall game Hall server IP and port information, is stored in the Qqgame/dirconfig.ini. Game Hall server receives client-side logon messages and determines whether to accept client logins based on certain policies, and if the current Game Hall server is up to the upper limit or temporarily unable to process current player login messages, Game Hall The server sends a message to the client to have the client redirect to another game Hall server login. Redirected IP and port information, which is not saved locally, is obtained by means of a packet or a certain algorithm. If the current game Hall server accepts the player's login message, it sends the room directory information to the client, which I have mentioned above. After messages such as directories are sent, the Game Hall server disconnects from the client to free up the socket resource. During this time, the client will reconnect to game Hall server every 30 minutes and ask for the latest room directory information and online headcount information.

3.Client Select a room to enter the game according to the list of rooms listed. According to my bag result analysis, QQ game, not to each game room is allocated a separate port for processing. In the QQ game, there are many rooms are shared with the same IP and the same port. For example, in the Bucket landlord area, the first 50 rooms, with the same IP and port information. This means that these rooms, in the QQ game server, in fact, may be the same program in the processing ... The maximum number of QQ game rooms is 400 people, it is not difficult to calculate, QQ game a single server program's user load is 20,000, that is, a QQ game Logic server program can be at the same time with 20,000 players to keep the TCP connection and ensure the efficiency and quality of the game, more importantly, this can be for Tencent province how much money ah ... Oh, wow. QQ is really very cow. It is not easy to maintain such a good quality of the game with 20,000 online numbers. QQ Game of a single server program, the management is no longer just a logical sense of a single room, but may be a lot of logical sense of the room. In fact, for the server, it is a large area server or a part of the server, we can understand it as a huge game map, it is also the implementation of block processing. And for the logic of the cards on each table, there is a unified processing process, the 50-room 50*100 table is handled by this server program (I do not know the QQ game specific card logic is how to design, I think it is likely to be the subregion, the chunking). Of course, these are just server-made things, for the client, the client only in performance, will be a separate list of rooms, this is to facilitate the player to play and reduce the cost of the server, the large area of 400 people in a collection for processing (such as chat information, " Broadcasting to 400 people "and broadcasting to 20,000 people" is completely different from the two concepts.

4. A point that needs special clarification. Enter the QQ game room, until you click a location to open another program interface, the client program, no longer create a new socket, but still use the original lobby room client and Game logic server interaction with the socket. In other words, this is the same socket that two processes share. Do not underestimate this point. If you create a table client and then a new socket and the game logic server to communicate, then the resulting players to enter, exit, escape and other messages will bring a very troublesome data synchronization problem, I was at the beginning of the suffering. Once shared with the same socket, if you exit the table, the server does not involve the release of the socket problem, so there is a lot less data synchronization problems. For questions about how multiple processes share the same socket, go to Google below: WSADuplicateSocket.

The above is I according to the latest QQ game grab packet results analysis of the QQ game communication flow, of course, this process is more client how to interact with the server, but does not relate to the server and how the communication between each other and data synchronization. With regard to the communication process between servers, we can only draw the following ideas based on our own experience and conjecture:

1.Login server communication issues before game Hall server. The Login server is responsible for user authentication, and once validated, it tries to get game Hall server to know the message. These are the ways in which information is communicated before, and I think there might be a few: a.　Login server will store authenticated users in the staging database; Login server will authenticate the user in memory, of course, this information should be globally accessible, that is, all QQ game Hall server can be through the packet communication between servers to obtain such information.

Acquisition of the latest room catalog information for 2.Game Hall server. This information, is the global, that is, the entire game, only one directory. Its source of information, can be from the bottom of the room server step by step, reported to WHO. I think, like a saved global login list, it is reported to the server or database that holds the global login list.

3. In the QQ game, the same type of game, can not open more than two game rooms. This information can be judged according to the global information.

The above about how the communication between the contents of the server, is a personal guess, QQ in the end how to do, I am afraid only after one of us into the Tencent to know. Oh. However, one thing is for sure, in the entire server architecture, there should be a place that is dedicated to keeping a global list of logged-in players, only in order to ensure that players do not log on repeatedly and enter multiple rooms of the same type.

In the previous description, I have mentioned a problem: when the login current game Hall server is unsuccessful, the QQ game server will choose to let the client redirect to the other server to log in, in fact, QQ chat server and MSN Server login is similar, it also has a login redirection problem.

So, this leads to another question, who makes the choice of strategy. And who is going to provide such a selection of resources. Such a process is responsible for the load balancing of the server processing scope. A further discussion of the balanced and millions online system, derived from the communication process analysis of QQ games, will continue in the next article.

In this, special thanks to Netizen Tilly and some inconvenient to disclose the name of the Netizen's discussion, is you let me decide to take another bag carefully.

< to Be continued >