Complex query of big data tables

Now:

 

1. The data volume of the table to be queried is large;

2. A large amount of data is returned;

 

Complexity is reflected in:

1. Associate the data of multiple big data tables for query.

2. query conditions are also complex.

 

For example:

Tianyi reading has a user portrait group function.

Users must be filtered to different user groups based on their behaviors.

User table: about 10 million data

User subscription and consumption data: possibly hundreds of millions of data

User Login data: may also contain hundreds of millions of data

User reading e-book data: hundreds of millions

Users read e-book content data: hundreds of millions

 

We need to count the data of the above user behaviors within three months. For example, the purchase volume of consumption data is greater than 30, less than 50, and the consumption amount is greater than 100, less than 300. In this case, how can we implement queries?

 

You can use sharded table queries and write the results directly to oneTemporary table.

 

For example, if the number of orders for data consumption in three months is greater than 30, and less than 50 users are imported to a temporary table, users with query results of other conditions are also placed in a temporary table, finally, the final result is obtained after these temporary tables are joined.

 

FinallyStored ProcedureDirectly store the results in the database.