Some pits in the MongoDB (preferably not) _mongodb

MongoDB is currently the hottest NoSQL document database, it provides some good features: such as automatic failover mechanism, automatic sharding, modeless schemaless, most of the performance is also good. But in the deep use of mint in the process of MongoDB, encountered a lot of problems, the following summed up a few of the pits we encountered. Special statement: We currently use the MongoDB version is 2.4.10, has been upgraded to MongoDB 2.6.0 version, the problem persists, and back to the 2.4.10 version.

MongoDB Database-level locks

Pit Dad index: 5 stars (highest 5 stars)

MongoDB lock mechanism and general relational database such as MySQL (InnoDB), Oracle is very different, InnoDB and Oracle can provide row-level granularity lock, and MongoDB can only provide library-level granularity lock, which means that when the MongoDB a write lock is occupied state , all the other reading and writing operations have to wait.

At first glance, the library-level lock has serious problems in the large concurrency environment, but MongoDB can still maintain large concurrency and high performance, because the MongoDB lock granularity is very extensive, but there are great differences in lock processing mechanism and relational database lock, which are mainly shown in:

MongoDB does not have full transaction support, operating atomicity only to a single document level, so usually the operation granularity is smaller;
MongoDB Lock actual time is the memory data calculation and change time, usually quickly;
MongoDB Lock has a temporary waiver mechanism, when there is a need to wait for the slow IO read and write data, you can temporarily discard, and so on after the IO complete and then regain the lock.
Usually no problem is not equal to no problems, if the data is not properly manipulated, will still lead to a long time to occupy write locks, such as the following mentioned in the foreground index operation, when this situation, the entire database is in a completely blocked state, can not do any read and write operation, the situation is very serious

To solve the problem, try to avoid long time to occupy the write-lock operation, if there are some set operation is difficult to avoid, you can consider to put this set into a separate MongoDB library, because MongoDB different library locks are separated from each other, separate sets can avoid a set operation to cause global blocking problems.

Indexing causes database blocking

Pit Dad index: 3 stars

The problem with MongoDB library-level locks is mentioned above. Indexing is a problem that can easily cause long time to write locks, MongoDB need to use a write lock when building an index in the foreground (and will not give up temporarily), if the data volume of the collection is large, it usually takes a long time to build the index, which is especially easy to cause problems.

The solution is simple, MongoDB provides two kinds of indexed access, one is background way, do not need to occupy write lock for a long time, the other is background way, need to occupy the lock for a long time. Use the background method to solve the problem. For example, to create an index for an oversized table posts, never use

Copy Code code as follows:

Db.posts.ensureIndex ({user_id:1})

Instead, you should use

Copy Code code as follows:

Db.posts.ensureIndex ({user_id:1}, {background:1})

Unreasonable use of embedded embed document

Pit Dad index: 5 stars

Embed document is a place where the difference between relational databases is MongoDB, and other child document can be embedded in a document, which can be kept in a single collection in a parent-child document, which is more convenient to retrieve and modify.

For example, there is a group document in the Mint application scenario where the user applies to join group modeling for Grouprequest document, and we initially use the Embed method to place the grouprequest in the group. The Ruby code looks like this (using Mongoid ORM):

Copy Code code as follows:

Class Group
Include mongoid::D ocument
...
Embeds_many:group_requests
...
End

Class Grouprequest
Include mongoid::D ocument
...
Embedded_in:group
...
End

This use way let us fall into the pit, almost can not climb out, it causes nearly two weeks of time system problem, peak period often a few minutes of system cotton, the most serious once even caused MongoDB downtime.

After careful analysis, it is found that some active Group group_requests increases (when there are new applications) and changes (when a user request is passed or rejected) are unusually frequent, and these operations often occupy write locks for a long time, causing the entire database to block. The reason is that when there is an increase in group_request operations, the group does not have enough space to allocate, need to reallocate space (both memory and hard drive), takes a long time, there are many indexes built on the group, the move group location causes a large number of index update operations is also time-consuming, Combined to cause a long time to occupy the lock problem.

The solution to the problem, it is simple to say, is to change the embed association to the Common Foreign Key association, is similar to the practice of relational database, so that group_request add or modify only occurs on the grouprequest, simple and fast, avoid long time to occupy write lock problem. When the data of the associated object is not fixed or changes frequently, it is necessary to avoid using embed association, otherwise it will die miserably.

Unreasonable use of the Array field

Pit Dad index: 4 stars

The MongoDB Array field is a unique feature that can store simple one-to-many relationships in a single document.

Mint has an application scenario using a serious performance problem, directly on the code as follows:

Copy Code code as follows:

Class User
Include mongoid::D ocument
...
Field:follower_user_ids, Type:array, default: []
...
End

User to save the ID of the person who is concerned by the follower_user_ids of an Array type field, the user pays attention to the person from 10 to 3,000, the change is more frequent, and the problem that the above embed raises is similar, frequent follower_user_id s increased modification operation resulted in a large number of long time database write locks, which caused the MongoDB database performance to drop dramatically.

The solution to the problem: we moved the follower_user_ids to the memory database Redis, avoiding the frequent change of user in MongoDB to solve the problem thoroughly. If you do not use Redis, you can also create a Userfollower collection, which is associated with a foreign key form.

First listed above a few pits, are hairenbujian traps, use the MongoDB process must pay more attention to avoid falling into the pit.