The Writeconcern of MongoDB learning notes

Writeconcern:

Reprint: MongoDB Writeconcern (write attention) mechanism

Http://www.ywnds.com/?p=3688&viewuser=40

MongoDB Deployment Mode

There are three types of MongoDB deployment modes: The first is stand-alone mode (development test), the second is a highly available replication set, and the third is an extensible shard cluster. As shown in.

Now that we know a few common deployment patterns for MongoDB, let's look at the write process for each deployment pattern.

MongoDB single-point write operation

As can be seen, where primary is an instance of MongoDB, there are two memory areas, one is data buffer (buffer), one is journal buffer (log buffer). The two memory regions correspond to the physical file data file and journal file respectively.

When data is written in, the following order of execution occurs:

1) The client's data comes in;

2) Data operations are written to the log buffer;

3) data is written to the data buffer;

4) Returns the result of the operation to the client (async);

5) Background thread to log buffer in the data brush disk, very frequent (default 100) milliseconds, can also set their own (30-60);

6) Background thread to buffer data in the data brush disk, the default is 60 seconds;

MongoDB Copy Set write operation

Replication set is also the most common mode of deployment of MongoDB, if the replication set is enabled, there will be more than one oplog zone in memory, is a means of synchronizing between nodes, it will put the operation log in Oplog, and then Oplog will be copied to the slave node. The operation log in Oplog is received and executed from the node to achieve synchronization of the data.

1) The client's data comes in;

2) Data operations are written to the log buffer;

3) data is written to the data buffer;

4) Put the operation log in the log buffer into the Oplog;

5) Returns the result of the operation to the client (async);

6) Background thread to oplog copy to from the node, this frequency is very high, than the log brush disk frequency is also higher, from the node will always listen to the master node, oplog a change will be replicated;

7) Background thread to log buffer in the data brush disk, very frequent (default 100) milliseconds, can also set their own (30-60);

8) Background thread to buffer data in the data brush disk, the default is 60 seconds;

The role of recovery log Journal

1) Recover memory data when system is down (cannot be used to restore historical data like MySQL)

2) default to asynchronous brush disk

3) Brush disk interval, mmap engine for 30-100ms,wiredtiger 100MB or checkpoint

4) J:1 can be used to force the synchronization of the brush disc

Write the mechanism of attention the role of write concern

1) used to specify the receipt behavior of Mongod for write operations.

2) can be specified at connection level or at level of write operation.

3) The Write Conern supports the following values.

w:0 | 1 | N | majority | Tag

J:1

Wtimeout:millis

1) when w:0 is unacknowledged

Testing (data loss scenarios occur)

Here, write a loop to insert 10 data, insert the data {_ID:10,A:I}, and then set Writeconcern to 0. According to We can see print return 10 insert successful information, but we db.test.count () view only one piece of data, the remaining 9 is not inserted successfully but also returned the correct information (because the first Data _id:10 is a unique key, and then insert the same data can not be inserted, Two IDs cannot be the same). No error message was returned to the client because {writeconcern:{w:0}} caused an exception to occur.

2) when W:1 is acknowledged

Testing (data loss scenarios occur)

When W:1, it solves the problem that w:0 appears, from can see the first data insert succeeds, the remaining 9 data will insert to fail and return error message to the client.

However, when W:1, MongoDB will return the information after the log is finished, although the w:0 data loss problem, but the w:1 if a system crash will also lead to data loss, that is, the log information has not been flushed to the disk at the moment of a system outage, At this point the memory log is actually written successfully, so MongoDB will return the deterministic information.

We can write the data by w:1 and then high speed, then through killall-9 Mongod to simulate a system crash (remember to delete the Mongod.lock file in the data directory when restarting the system, or not to start), and finally check the data written by the program and the actual inserted data (this is not a very likely occurrence).

As you can see, the data is missing 731. Doing the above experiment when executing the Journaldataloss function, you need to open another session to simulate server outage (killall-9 mongod)

The problem of data loss in the event of a server outage in W:1 can be resolved using j:1, J:1 does not return the determined information until after the log brush disk.

1	> db. Test. Validate(true)

function script

1234567891011121314

function journaldataloss(){ var count=0, start = new Date(); try{ var docs=[]; For (var i=0; I<; I+ +) docs.push ({a:i}); While (true){ var res=db. Test. Insert(docs); count + = res. ninserted; If(Count % 100000 == 0) print ( "inserted" + count+ "time used:" Span class= "crayon-h" > + ( new date (.gettime ( - start . Gettime () /1000 /span> +"seconds"); }}Catch(error){ print("Total doc inserted successfully:"+ Count); }}

3) when J:1 is journal

Instance (data loss in the event of a test)

Another situation

The problem of w:1 data loss can be solved when j:1, but it comes down to the performance of MongoDB. While resolving the server outage data loss problem (will lose about 60s of data, that is, one interval is not flushed to disk data).

However, it is not possible to solve the other situation, whether w:1 or j:1 can not solve the data loss caused by primary and standby displacement. Above we have introduced the MongoDB three deployment mode and each mode of the write operation flow. Look at the picture below,

When the j:1 can guarantee the security of the primary node data, the log after the disk to return the determination of information to the client, then the next copy Oplog to the replication set from the node if the primary node is down, Oplog not replicated to the slave node and when the primary node down, MongoDB copy Rally re-election station secondary to primary. When the secondary for primary after assuming that the original primary was restored, after the start of the secondary node, when the new primary found secondary have X data and do not own, then there will be data rollback situation, Secondary will roll the X data back to a disk file (Rollback_db_name), back to manual processing, from the user's perspective this is also a data loss situation. So how do we solve this situation? That's what we're going to mention here. W:2/n/majority can resolve this data loss situation, W:2 determines that the data must be copied to a minimum from a node when the deterministic information is returned to the client.

4) when w:2/n/majority replica acknowledged

Obviously this approach guarantees data consistency, but it will undoubtedly slow down MongoDB's performance. So in MongoDB 3.2 after the official gave a readconcern mechanism, specifically to be studied.

Write Concern Summary

The interpretation of write concern by the above figures shows that this is a very important parameter for data security, whether it is a development or ops person should be very aware of the write concern mechanism. For write concern, the higher the level, the more secure the data, but the slower the performance, and the data security and performance have always been so. You can determine the level of write concern (by default, W:1) based on your own application scenario. In fact, j:1 is not particularly important, if you use a good copy set at the same time the data is particularly important, you can use w:majority, because the data in MongoDB copy set is often faster than the log run, but also a more efficient way. is a basic summary of the wirte concern.

The Writeconcern of MongoDB learning notes