Working principle of MongoDB journaling)

This was originally posted to Kristina chodorow's blog, snail in a turtleneck

I participated in writing the journaling section in the definitive guide, but I realized that many of the implementation details were not paid enough attention to, so I will illustrate it with easy-to-understand diagrams in this blog.

So how does journaling work? Assume that there are data files and log files on your hard disk.

When you start mongod, it maps your data files to a shared view. For example, the operating system will map your 2,000 bytes data files to the memory address 1,000,000-1,002,000, in this way, you can access the address 1,000,042, that is, the 42nd-bit address of the file. (In fact, it is the memory ing ...). Of course, in fact, this data file will be loaded into the memory only when it is accessed for the first time.

If you change the data in the memory, the operating system will refresh and change it to the actual file. The operating system refresh MongoDB every 60 seconds if Journaling is not enabled. If Journaling is enabled, mongod maps to another private view, which is why the virtual memory usage doubles when journaling is enabled.

Now, each time a shared view is written, it is written to a private view. Note that because the private view is not mapped to the actual file, the OS will not write the changes to the private view back to the hard disk each time it is refreshed.

Mongod will write a brief description to the journal file about which file the bytes have changed.

Each change information is appended to the Journal File.

If the data change has not been written to the disk when mongod crashes, you can play back the records in journal to the shared view.

 

After the playback is successful, mongod will rely on the operating system to refresh the changes to the disk file. The default value is 60 seconds.

Finally, mongod will re-map the shared view to the private view. This is to prevent private views from becoming too "dirty" (in fact, it is to prevent redundant changes from being written during playback next time ).