How to filter the statistics of high volume user log data?
such as 100W users
Now the table is designed like this:
Table A is 100W user basic information (UID unique)
Uid,name, region, service provider ....
Table B Each user has 100 or so data, that is, 100w*100=1 billion data. Each data content includes (user ID, open time, close time)
Uid,time1,time2
Uid,time1,time2
...
Requirements: Statistics of 24-hour users and can be based on a region and a service provider statistics
------Solution--------------------
24 hours The number of users refers to what, for example.
------Solution--------------------
100 million data is not too much, but to establish a reasonable index.
------Solution--------------------
Do it once a day, and then put the results into a table record, the next time the direct re-reading, not every time statistics.
------Solution--------------------
References:
do it once a day, and then put the results into a table record, the next time the direct re-reading, not every time statistics.
Indexing and caching the data are all an optimization method.
