Dig deeper into the MongoDB architecture

Ricky Ho, a software engineer, recently published a blog post on the MongoDB architecture (MongoDB Architecture), although this is a very broad-sounding topic, However, in the article, the author does an analysis of the internal-to-external architecture of MongoDB. This article captures a brief description of several key architectures in its article.

1. The internal structure of the MongoDB data file

• MongoDB is partitioned by namespaces on the Datastore, a collection is a namespace, and an index is a namespace.

• Data from the same namespace is partitioned into multiple extent,extent using a doubly linked list connection.

• In each extent, the data for each row is saved, which is also connected by a two-way link.

• Each row of data storage includes not only the data footprint, but also some additional space, which makes it possible to not move the location after the data update has become large.

• The index is implemented in the BTREE structure.

2. Implement transactions in MongoDB

It is well known that MongoDB only supports atomic modification of single-line records and does not support atomic manipulation of multiple rows of data . But through the incredible steps in the process, you can actually implement the transaction yourself. the steps are as follows :

• The 1th step: Record a transaction record, the modified value of the multiline record to be modified is written inside, and set its state to Init (if the operation is interrupted at this time, then at the restart, it will be determined that it is in the Init state, so that its saved multi-line modification operation to apply to the specific line).

• 2nd step: Then update the specific row to be modified and write the identity of the transaction record that you just wrote to its Tran field.

• 3rd Step: Change the state of the transaction record from Init to Pending (if at this time the operation is interrupted, then at the reboot, it will be determined that its state is pending, then see all of its corresponding records to be modified, if its Tran value is not empty, then 4th step; If the value is null, Note that the 4th step has been executed, and the state is directly changed from pending to commited on the line).

• 4th step: Modify the corresponding values of the multiple records that need to be modified, and unset the previous Tran fields.

• 5th step: Change the state of the transaction record from pending to commited, and the transaction is complete.

In fact, the above steps are not uncommon, and in a DBMS that supports transactions, the assurances of atomicity of their transactions are mostly similar to the above. The Tran record of the transaction records is similar to the redolog in these DBMS.

3. MongoDB Data Synchronization

MongoDB uses the replica sets mode synchronization process

This process can be briefly described as follows:

• A red arrow indicates that the write operation can be written to primary and then asynchronously synchronized to multiple secondary.

• A blue arrow indicates that the read operation can be read from either primary or secondary.

• The heartbeat synchronization is maintained between each primary and secondary to determine the state of the replica sets.

4. Fragmentation mechanism

• MongoDB shards are specified by a shard key, and the data is divided into different chunk by range, with a limit of size for each chunk.

• There are multiple shard nodes that save these chunk, and each node holds a portion of the chunk.

• Each shard node is a replica sets, which guarantees the security of the data.

• When a chunk exceeds its limit of maximum volume, it splits into two small chunk.

• The chunk migration operation is raised when the chunk is unevenly distributed across the Shard nodes.

5. Server Role

The mechanism of sharding is described earlier, and the following are the roles of several nodes at the time of sharding:

• The client accesses the routing node MONGOs to read and write data.

• The config server holds two mappings, one is the mapping of which chunk is the interval of the key value, and the other is the mapping of which shard node exists in chunk.

• The routing node obtains the data information through the config server, and through this information, it finds the corresponding operation of the Shard node that actually holds the data.

• The routing node also determines whether the current chunk exceeds the qualified size when writing. If exceeded, it is divided into two chunk.

• For queries and update operations by Shard Key, the routing node will find the specific chunk and then do the related work.

• For queries and update operations that do not press shard Key, MONGOs sends a request to all subordinate nodes and then merges the returned results.

Delving into the MongoDB schema