Big Data Case Study

Preface

The company switched from browser games to mobile games. The company's data analysis needs to be designed for mobile games. Therefore, the original data analysis framework for browser games is not very suitable, on the one hand, mobile games and web games have different business logic, and on the other hand, they have changed the data magnitude, as well as connections between channels and district servers, as well as cross-query between channel and district servers in the mobile game Bi system. This makes it difficult to build a web game framework from Game 4399. Here we will analyze the adjustments made to big data analysis during the mobile Web Game process, as well as the analysis cases between them.

Web Game cooperation scenarios

In the browser game stage, if a browser game is to be pushed to a platform, you must connect to the platform and take the test on the platform to reach a certain score, the platform can provide resources for online service expansion. If you want to use another platform, you need to perform the same process with another platform. Most of the current webpage game companies and platform partners cooperate in this way. In fact, the platform and the Zone server are all one-to-one. A single platform can have multiple zones, and a single zone server only belongs to one platform (different from a mobile game ).

Web Game data collection and analysis process

As the game starts, the company needs to view the number of game analyses, such as the loss status, payment status, active users, and crashes, this is a subsystem of many BI systems (Data Analysis Systems: used for user behavior analysis, product quality monitoring, activity engagement statistics, etc ). The process is roughly as follows:

Analysis:
1. Data between platforms is isolated.
2. When using ETL to process project group data, keep the original game platform-zone server sub-database. If you are targeting some special tables, you can pull the summary table, such as the login table and the paid table.
3. Compile a statistical script and place it in a Linux scheduled task. The statistical results are placed in the result library. The result library is sharded based on the platform.
4. Displays charts of relevant indicator data based on the statistical results in the result library of Bi system data analysis.
5. There will be a small amount of real-time data analysis requirements in this process. This requires you to directly connect to the number of project groups in the Bi system or pull the result library through a short scheduled task.

Mobile Game data collection and analysis process

In the mobile game stage, if a game needs to be launched through another channel, it needs to cooperate with Channel providers. iOS and Android are platforms, in addition, game data from different channels on the same platform can be written to the same zone server. That is to say, there is a many-to-many relationship between the channel and the Zone server. In comparison with web games, data in one server may come from multiple channels, and data in one channel may be divided into multiple zones. The general analysis process is as follows:

Analysis:
1. Direct Multi-to-many relationship between channels and servers
2. During ETL pulling and cleaning, database shards are still performed based on the channel-zone server.
3. Compile a statistical script and save the statistical results to the result library. A game has only one result Library (including all channels and servers)
4. Because there is only one result library, you need to consider Table sharding to avoid slow Query when a single table is too large. The analysis is as follows:
Assume that there are 50 channels, 100 regional servers, and no more than 5 single-Table dimensions. Estimate the [10 W-15 W]/day increase based on experience. Then, perform table sharding on a quarterly basis, the data volume of each sub-Table ranges from million to million. In addition, for statistical tables with relatively active records, the data volume may need to be divided into tables by month or in more detail. All in all, it is to reduce the size of a single table as much as possible, but this will bring about a problem, such as how to query and pagination in the Bi system, however, this problem has been solved by corresponding components, such as MyCat of Alibaba and sharding-jdbc of Dangdang. I plan to use sharding-JDBC, because MyCat is a bit heavy for queries and paging. If you also need to use similar features, select one based on your needs.

Big Data Case Study