How to implement DBMGR data caching

In the development of game server, a data server is often used to cache game data in the front-end of the database, and the whole system performance can be improved by submitting the transaction in batches.
Please refer to the dbmgr described in one of the < system architectures (RPG Games common architectures) >
Here I describe a caching strategy used by Dbmgr, which is practiced and used in large game projects:


Dbmgr uses the memory database Fastdb to cache non log class data.
Such data as player status is always cached in the FASTDB memory database
For log-class water tables, those that are updated into DB are not saved in Fastdb.


Dbmgr when a transaction request is received, a transaction object is organized, which includes an update operation on all library table objects in the current transaction
The transaction is also executed in the FASTDB memory database and the transaction object is saved to the transaction log queue.
Dbmgr initiate a separate thread to periodically remove a transaction object from the transaction log queue to update to DB



The problem with this scenario is that if DB fails, the Dbmgr memory is written more and more, and there is a risk that a large amount of data will not be synchronized to DB,
Another dbmgr program crashes, causing data loss that is not synchronized to DB. Often some items may be commercially possible to allow for a certain period of time the data back to the file failure,
We can design that we have to sync to db in this time range, and once we get over this time, we can only call the police and stop the system from running until we're back to normal.
For a project that does not allow any data loss to be tolerated, it must be synchronized to DB in real time, while synchronizing to DB in real time, you can also
A large number of transactions are brought together to improve the throughput of the whole system by using bulk transaction submissions.


Dbmgr when synchronizing data to DB, the number of threads in which multiple threads commit transactions at the same time in batches, and the maximum number of transactions allowed in each batch transaction.
These can be adjusted by configuration parameters according to the system environment, hardware and software configuration to achieve optimal results.



For simplicity in this project all transaction control is controlled by Dbmgr, Dbmgr does not allow request callers start transaction, end transaction, each request is a transaction,
If you want to do more complex to allow the requester to control the transaction, Dbmgr is added to the transaction state management. and handling the various transaction exceptions, such as to
There is a feature that periodically checks the state of the transaction, for example, in some cases, the client initiates a transaction, but there is no call to the end transaction, or the network is broken,
These conditions will automatically recognize and roll back the transaction.