Computing and data storage scheme for large-scale IM online users

A simple computational model

1, if a second processing 1000 requests (each is stored), then the amount of data for the day is: 24*60*60*1000=8640 million; if 10,000 pens per second, the data is about 864 million.
2, the industry's general statistical method is the peak is the daily activity of One-fifth, the daily life is the total user 8%. According to, according to the peak 1万来 calculation, the total number of users is: 10,000 *5/0.08=62.5 million, in addition to pay users accounted for the total user generally around 5%, the specific view of the operation.
3, daily active users to generate peak calculation: The general message class, probably to 0.5% to 1% is good (a second at the same time, the network game may be a little different, will be higher), based on the 2 hypothesis, the total user volume is: 625,000/0.01=6250 million
4, the game itself frequency is relatively high, in step 3 calculation method may be different

Data storage Scenarios

This part of the data storage is mainly real-time message storage, for online real-time processing scheme, the current popular is the use of Redis, personally think the more successful scenarios are:
1. Redis cache, Database persistent storage
Program Reference: http://blog.codingnow.com/2014/03/mmzb_redis.html
?? Start a guardianship service on the physical machine of the data server. When the game server pushes the data to the data service and confirms the success, the ID of this set of data is sent to the guardianship service at the same time. It then reads the data back from Redis and stores it locally.
?? Because this guardianship service and Redis 1:1 are configured on the same machine, and the hard disk write speed is greater than the network bandwidth, it must not be overloaded. As for Redis, it's a pure memory database, no longer running BGSAVE
2. Redis Cache, LEVELDB storage
Reference: http://bbs.chinaunix.net/thread-3777230-1-1.html
?? Redisstorage is based on the Redis 2.6.2, plus the LEVELDB storage engine. This project is a Passport user certification transformation originating from the company project.
Summary : simply using Redis for caching and data storage is a pit.

Information about Redis

Two methods of data storage for Redis
?? The SAVE and BGSAVE two commands call the Rdbsave function, but they are called in different ways:? Save calls Rdbsave directly, blocking the Redis master process until the save is complete. During the main process blocking, the server cannot process any requests from the client. BGSAVE then fork out a child process, which is responsible for calling Rdbsave and sending a signal to the main process after the save is complete, informing that the save is complete. Because Rdbsave is called in a child process, the Redis server can continue to process client requests during bgsave execution.

Review of Redis key parameters
http://blog.itpub.net/29254281/viewspace-2099173/

Computing and data storage scheme for large-scale IM online users