Lightning MDB source Code Analysis (5)-Transaction control

This blog series has explored the Lmdb system architecture, MMAP mapping, b-tree operations, and so on, and this article will attempt to describe the implementation of transaction control in Lmdb.

Basic characteristics of a transaction:

Transactions are the basic unit of recovery and concurrency control. It is a sequence of operations that are either executed or not executed, which is an inseparable unit of work.

A transaction is a unit of data consistency maintained by a database that maintains data consistency at the end of each transaction.

A transaction should have 4 properties: atomicity, consistency, isolation, persistence. These four properties are often called acid properties .

Atomicity (atomicity). A transaction is an inseparable unit of work, and the operations included in the transaction are either done or not.

Consistency (consistency). The transaction must be to change the database from one consistency state to another. Consistency is closely related to atomicity.

Isolation (Isolation). Execution of one transaction cannot be disturbed by other transactions. That is, the operations inside a transaction and the data used are isolated from other transactions that are concurrently executing, and cannot interfere with each other concurrently.

Persistence (durability). Persistence, also known as permanence (permanence), refers to the fact that once a transaction is committed, its changes to the data in the database should be permanent. The next operation or failure should not have any effect on it.

The basic idea of lmdb in the implementation:

Atom (A): Lmdb is controlled by TXN data structures and cursor data structures by placing a list of dirty pages in Dirtylist, and then once again unified flush to disk when TXN commits

Or abort, do not commit to ensure that the transaction is unsuccessful or all fails. For long transactions, if the page spill to disk, because of cow technology, these pages are not associated with the root of the entire b-tree

Page is associated so that subsequent transactions are not accessible to these pages, which also guarantees the atomicity of the transaction.

The data is consistent, and there is no case where data is generated incorrectly because of simultaneous multithreading of data.

Isolation (I): Transaction Isolation through lock control (mutex), Lmdb-supported lock mutexes are process-level/thread-level, supported isolation mode for lock table support, read-read not locked, write waiting to be read after completion of the start,

Read wait write finish start

Duration (D): Lmdb, without the use of technologies such as Wal, Undo/redo Log, to ensure the availability of the database in the case of a system crash, the technology that ensures that data is continuously available is cow technology and only one-thread write technology.

If the Lmdb or system crashes, only the read operation, then the data will not have changed, so the data is not likely to be compromised. If a thread is writing in the event of a crash, it is only necessary to determine the last

The page number is consistent with the page number successfully submitted to the database, if the inconsistency indicates that the write operation is not completed, the last transaction fails, the data is correct in front of the last successful page, and the following belongs to

A crash transaction is not available, which guarantees that the data will be usable as long as it is serialized to disk, or that it is not yet serialized to disk in accordance with the ACI principle

By the way, because Mmap technology, only a write thread implementation, so the database is particularly simple to backup, as long as the regular online hot-standby entire database can be completed. At the same time recovery will be relatively fast. Of course because

It uses the reuse of the old page technology, Lmdb restore only to the latest state, can not be restored to any time.

Implementation method:

Lmdb supports nested transactions and does not expect any read and write operations on the parent transaction before the child transaction is complete, thus avoiding data inconsistency between parent and child transactions.

Lmdb does not support cross-threading transactions, a transaction can belong to only one thread, and a thread can hold only one transaction at any one time.

Mdb_txn_begin:

Opens a transaction that determines whether the parent transaction is a child transaction, based on the incoming parameter, and whether it is a read-only transaction. Nested transaction support supports only one child transaction, and the child transaction is a write transaction, and the parent transaction must also be a write transaction.

And the database cannot be mmap writable. Transaction opening process: Allocate memory, set variable, if child transaction, set parent-child correlation variable and shadow father all cursor to reduce IO read. Otherwise call renew0 complete

The final transaction opens the job.

Mdb_txn_abort:

Discarding a transaction, having a child transaction discarding the child transaction, and then calling Reset0 to actually perform the end operation.
Mdb_txn_commit:

Commits a transaction, the child transaction commits the child transaction, if it is a read-only transaction, closes all open database handles and remains open, then discards the transaction, and if it is a writable transaction, determines whether the transaction state is correct, if the error

State, can not be submitted, if it is not based on the existence of a parent transaction processing, no parent transaction first update the database root node, and then save the reusable space to freedb for space reuse, and release the MIDL space after the

The page refreshes, synchronizes the related environment variables, frees the memory, and finally releases the write lock, so that no parent transaction is committed. If there is a parent transaction, it merges the MIDL list with the MIDL of the parent transaction, and the cursor is also merged into the parent transaction

Finally closes, merges the dirtylist into the parent transaction, and after the related merge and the variable memory of this transaction is released, the child transaction commits successfully, that is, the child transaction mainly completes the memory release, and the other actions such as disk refresh are merged into the parent transaction once.

Mdb_txn_reset:

Discards a transaction, but retains a handle that is valid only for a read-only transaction, and calls Reset0 for a true transaction end operation.
Mdb_txn_reset0:

The public code that discards the transaction. First close the open database handle in the transaction. For a read-only transaction, set the transaction-dependent variable, and if it is a writable transaction, you need to close all cursors, release the MIDL Space, and finally release the write lock. This transaction

Closed.
Mdb_txn_renew:

Reuse a read-only transaction handle, avoid a memory allocation, check for serious errors, and if there is a failure, call renew0 to complete.
Mdb_txn_renew0:

Renew0 is the common code for renew and begin. If you write a transaction, apply for the inter-process mutex, if the Read transaction, first check whether this thread has read transactions, there is no support return error, no, start to apply read table mutex,

After success, the thread ID is recorded in the Reading table, then the Read table lock is immediately released. Then confirm again that there is a transaction in the thread. After the successful application of the transaction (read/write), the Env meta page is switched according to Txnid and used in turn.

Finally, the caller is notified of the successful application after setting some variables again.
Mdb_txn_env:

Returns the Env object associated with the transaction

The basic flow of how Lmdb implements transaction control and the main interface method is explained above, and if fine-grained transactions like relational databases are implemented, finer-grained locks and complex page-waiting queue mechanisms are required to ensure row or table locks

Correctness and finally implement the transaction control mechanism, and in the database application can fall into a deadlock state, and in the Lmdb, read and write locks are separated, and the process crashes, the system will release the relevant kernel variables, so as to ensure that the process is not normal,

The lock is released successfully, or the process crashes, the system releases the lock, so the database never falls into a deadlock state, but if the transaction waits for a write lock, it may wait for a longer time.

I hope you can actively criticize and reprint.

Lightning MDB source Code Analysis (5)-Transaction control