Thinking about re-designing data accessing layer (i)

Background: The data access layer, which has been running for 10 years in the project, has a number of architectural problems, in order to completely resolve and release the capability of parallel access, and to provide a complete set of object-oriented APIs that decide to redesign.

Consider the question:

1. How to ensure compatibility:

1) Replace the implementation of the old library with the implementation of the new library, ensure all-in-one modification and testing, do not modify the old library's API, and provide the user new API

2) The new library is completely isolated from the old library and exists at the same time, but the new library needs to be guaranteed to synchronize with the old library. The old library does not make any changes. Users can use both the new library and the old Library

2. How to ensure the efficiency of multi-threaded access:

1) There has always been a conflict between memory and CPU, especially in data access. When dynamic data has only one storage address in memory, it is bound to require various synchronization mechanisms, or asynchronous access, to affect efficiency in multi-threaded access. If the memory is sacrificed and the duplication of the memory is provided, then the existing problem becomes the amount of memory used in double, and the merge changes are required, or duplication. In fact, the design of the database, the problem provides a variety of solutions, such as snapshot. In fact, this idea in the old data accessing layer design, also embodies some, when the first data exists in memory only one copy, but the runtime found that in multi-threaded or multi-process conditions, the use of locking mechanism, can not fully play the advantages of multithreading. Later, you can only take multiple backups through the data to ensure the efficiency of multithreading. Of course, each backup needs to synchronize data with each other, which creates a problem, that is, what to do when there is conflict. At first, I thought it would be a most convenient and efficient way to put the data together in share memory, but it is increasingly found that if you want to make the most of multi-threaded or multi-process efficiencies, you should consider how to make a tradeoff between memory usage and CPU efficiency.

2) Reduce the mutex of data, especially for dynamic data. Modify the data Model in the database to ensure that the table is the dynamic data generated by the runtime (80% of the existing data is static data), the static data is stored in sharememory, shared with all processes, and using a separate process preload. For dynamic data, especially string data, the ability to map with an enum should be removed from the database.

3) Each thread, when accessed, creates a snapshot of unique data, so that when the thread needs to commit the modification, it will decide how to handle the conflict.

adjourned