Data caching-Data classification strategy

the data of the Internet system can be classified into the following 3 categories:
1, core business data: such as user order form, merchant order form
2, user-related data: User basic information, avatar, profile, hobbies, etc.
3, configuration information: Database configuration, sub-Library table configuration, etc.

If all of the above data is completed on the read Operation database, the performance of the system will be greatly compromised, and it is precisely in order to solve this problem, the memcached, Redis and other memory databases, how to use the memory database reasonably and efficiently. We have introduced a data classification strategy to solve this problem.

we divide the system's data cache into the following 3 levels:

level 1th: core business data; This data is very demanding in terms of real-time and accuracy, so no caching is used, and read and write operations operate the database directly.

level 2nd: User- related data, which is related to the user, has a feature of read-write-less, so we use Redis for caching.

level 3rd: configuration information, this data is not user-independent, with a small amount of data, frequent read, almost unmodified features, so we use local memory for caching.

There is a data synchronization problem with the local memory cache because the configuration information is cached in memory and local memory is not aware of the configuration information being modified in the database, which can cause inconsistencies in the data and local in-memory data in the database.

In order to solve this problem, we need a highly available message push platform, when the configuration information is modified, we can use the push platform to push the profile update message to all servers, the server receives the message will automatically update the configuration information, and give a successful feedback. Zookeeper can be a good solution to the problem of unified configuration, disconf is Baidu Open source of a development based on ZK distributed configuration management platform.