Use MongoDB as a pure memory database (Redis style)

Basic Ideas

MongoDB is used as an in-memory database, that is, it does not allow MongoDB to save data to a disk at all, which has aroused more and more people's interest. This method is extremely useful for the following applications:

• Write-intensive high-speed cache placed before a slow RDBMS System
• Embedded System
• PCI compatible systems without persistent data
• Unit testing, which requires a lightweight database and can easily erase data in the database)

If all this can be achieved, it is really elegant: We can skillfully use the MongoDB query/retrieval function without involving disk operations. You may also know that in 99% cases, disk I/O (especially random I/O) is the bottleneck of the system, and disk operations cannot be avoided if you want to write data.

MongoDB has a cool design decision, that is, it can use memory-mapped file to process read/write requests to data in disk files. That is to say, MongoDB does not treat RAM and disk differently. It only regards files as a huge array and then accesses the data in bytes, the rest are handled by the operating system (OS! This design decision allows MongoDB to run in RAM without any modifications.

Implementation Method

 

All of this is achieved by using a special type of file system called tmpfs. In Linux, it looks like a conventional File System (FS), but it is completely in RAM (unless its size exceeds the RAM size, it can also perform swap, this is very useful !). My server has 32 gb ram. Let's create a 16 GB tmpfs:

# mkdir /ramdata# mount -t tmpfs -o size=16000M tmpfs /ramdata/# dfFilesystem           1K-blocks      Used Available Use% Mounted on/dev/xvde1             5905712   4973924    871792  86% /none                  15344936         0  15344936   0% /dev/shmtmpfs                 16384000         0  16384000   0% /ramdata

 

Next, use the appropriate settings to start MongoDB. To reduce the number of wasted RAM resourcesSmallfilesAndNopreallocSet to true. Since it is now based on RAM, this will not reduce performance at all. Using journal is meaningless, so we shouldNojournalSet to true.

dbpath=/ramdatanojournal = truesmallFiles = truenoprealloc = true

 

After MongoDB is started, you will find that it runs very well and the files in the file system appear as expected:

# mongoMongoDB shell version: 2.3.2connecting to: test> db.test.insert({a:1})> db.test.find(){ "_id" : ObjectId("51802115eafa5d80b5d2c145"), "a" : 1 }# ls -l /ramdata/total 65684-rw-------. 1 root root 16777216 Apr 30 15:52 local.0-rw-------. 1 root root 16777216 Apr 30 15:52 local.ns-rwxr-xr-x. 1 root root        5 Apr 30 15:52 mongod.lock-rw-------. 1 root root 16777216 Apr 30 15:52 test.0-rw-------. 1 root root 16777216 Apr 30 15:52 test.nsdrwxr-xr-x. 2 root root       40 Apr 30 15:52 _tmp

 

Now let's add some data to verify that it runs completely normally. Now let's add some data and make sure it behaves properly. First we create a 1KB document and then add it to MongoDB for 4 million times:

> str = ""> aaa = "aaaaaaaaaa"aaaaaaaaaa> for (var i = 0; i < 100; ++i) { str += aaa; }> for (var i = 0; i < 4000000; ++i) { db.foo.insert({a: Math.random(), s: str});}> db.foo.stats(){        "ns" : "test.foo",        "count" : 4000000,        "size" : 4544000160,        "avgObjSize" : 1136.00004,        "storageSize" : 5030768544,        "numExtents" : 26,        "nindexes" : 1,        "lastExtentSize" : 536600560,        "paddingFactor" : 1,        "systemFlags" : 1,        "userFlags" : 0,        "totalIndexSize" : 129794000,        "indexSizes" : {                "_id_" : 129794000        },        "ok" : 1}

 

1 2 3 Next Page