MongoDB Memory management mechanism

Currently, MongoDB uses a memory-mapped storage engine, it will convert the disk IO operation into memory operations, if it is read operation, in-memory data play a role in the cache, if it is a write operation, memory can also convert random write operations into sequential write operations, in short, can greatly improve performance. MongoDB does not interfere with memory management work, but to leave this work to the operating system's virtual Cache manager to deal with, the benefit is to simplify the work of MongoDB, but the downside is that you have no way to control how much memory MongoDB, In fact MongoDB consumes all the memory it can use, so it's best not to put other services and MongoDB together.

Sometimes, even if MongoDB is using a 64-bit operating system, it may encounter the infamous Oom problem, which is mostly due to limiting the size of the virtual memory, so you can see the current value:

[Email protected] bin]# Ulimit-a | grep ' virtual ' virtual memory          (Kbytes,-V) Unlimited

　　

Most operating system defaults are set to unlimited, if your operating system is not, you can modify this:

　　

However, it is important to note that the use of ulimit is contextual, preferably in a MongoDB startup script.

In addition, the cache can also be freed by adjusting the kernel parameter drop_caches:

[Email protected] bin]# sysctl-w vm.drop_caches=1vm.drop_caches = 1

　　

You can usually monitor the memory usage of MongoDB using the MONGO command line as follows:

Repl:primary> db.serverstatus (). mem{"bits": +, "resident":, "Vsan", "virtual": 5316, "Supported": TRUE, "mapped": 2287, "M Appedwithjournal ": 4574}

　　

You can also use the Mongostat command to monitor the memory usage of MongoDB as follows:

[Email protected] bin]#./mongostat

Mapped vsize   res faults   4.2G  9.2G 84.0M      0   4.2G  9.2G 84.0M      0   4.2G  9.2G 84.0M      0   4.2G  9.2G 84.0M      0   4.2G  9.2G 84.0M      0   4.2G  9.2G 84.0M      0   4.2G  9.2G 84.0M      0   4.2G  9.2G 84.0M      0   4.2G  9.2G 84.0M      0   4.2G  9.2G 84.0M      

Where the memory-related fields mean:
Mapped: size of data mapped to memory
Visze: The amount of virtual memory occupied
Res: The actual memory size used
Note: If the operation is no longer in memory, the value of the faults column will not be 0, and the apparent size may have a performance problem.
In the above results, vsize is mapped twice times, and mapped equals the size of the data file, so vsize is the data file twice times, the reason is this, because in this case, MongoDB opened the journal, need to map the data file in memory, If journal is turned off, vsize and mapped are roughly equal.
If you want to verify this, you can observe the file mapping using the PMAP command after you turn journal on or off:

[Email protected] bin]# Pmap $ (pidof mongod) 19300:   ./mongod--config/root/software/mongodb/ mongo.conf0000000000400000  20396K r-x--  /root/software/mongodb/bin/mongod00000000019ea000    560K rw---  /root/software/mongodb/bin/mongod

　　

How much memory is appropriate for MongoDB? Broadly speaking, the more the better, if you want to be exact, it depends on your data and index size, memory can be loaded with all the data and index is the best case, but many times, the data will be larger than memory, such as this article said the MongoDB instance involved:

Repl:primary>  db.stats () {"db": "Admin", "collections": 0, "Objects": 0, "avgobjsize": 0, "DataSize": 0, " Storagesize ": 0," numextents ": 0," Indexes ": 0," indexsize ": 0," fileSize ": 0," OK ": 1}

Another example:

Mongo> db.stats () {"DataSize": 1004862191980, "Indexsize": 1335929664}

　　

In this example, the index is only more than 1G, the memory can be fully loaded, and the data file reached 1T, it is difficult to find such a large memory, at this time to ensure that the memory can be installed thermal data can, as for the thermal data, this is a proportional problem, depending on the specific application. As a result, the memory size is clear: Memory > Index + Thermal data.
For MongoDB and memory topics, you can also refer to the official documentation for more information.

MongoDB Memory management mechanism