MongoDB master-slave copy source code analysis basics

(1) Description of the Master/Slave copy lock section on the MongoDB Official Website

How Does concurrency affect a replica Set primary?

In replication, when MongoDB writes to a collection on the primary,
MongoDB also writes to the primary's oplog, which is a special collection inLocalDatabase.
Therefore, MongoDB must lock both the collection's database andLocalDatabase.MongodMust
Lock both databases at the same time keep both data consistent and ensure that write operations, even with replication, are "all-or-nothing" operations.

How Does concurrency affect secondaries?

In replication, MongoDB does not apply writes serially to secondaries.
Secondaries collect oplog entries in batches and then apply those batches in parallel. secondaries do not allow reads while applying the write operations, and apply write operations in the order that they appear in the oplog.

MongoDB can apply several writes in parallel on replica set secondaries, in two phases:

1. During the firstPreferPhase, under a read lock,MongodEnsures
   That all clients affected by the operations are in memory. During this phase, other clients may execute queries against this member.
2. A thread pool using Write locks applies all write operations in the batch as part of a coordinated write phase.

(2) understand the source code based on the theory on the Mong Official Website

1. mongdb startup code process

------- DB. cpp -----
Main
-- Mongodbmain
---- Initandlisten
------ _ Initandlisten
--------- Listen
------------ Createserver (options, new mymessagehandler ());
------------ Startreplication

Above, we mainly focus on the last two points

(1) The mymessagehandler class is the message after mongdb receives the request. The process function in this class is used to process the request. Let's look at the call process of the process function.

Process
-- Assumeresponse
If (OP = dbquery) {query data
}
Else if (OP = dbgetmore ){Query
}
Else if (OP = dbmsg ){
}
Else {
Try {
If (OP = dbkillcursors ){
}
Else if (! Nsstring. isvalid ()){
}
Else if (OP = dbinsert ){
Receivedinsert (M, currentop); insert data
}
Else if (OP = dbupdate ){
Descriedupdate (M, currentop); update data
}
Else if (OP = dbdelete ){
Receiveddelete (M, currentop); Delete data
}
Else {
}

Here, we only analyze the subject of data writing.

Receivedinsert (M, currentop );

While (true ){
Try {
Lock: dbwrite LK (NS); applied for the database write lock
...................
If (multi. Size ()> 1 ){
Const bool Keepgoing = D. reservedfield () & insertoption_continueonerror;
Insertmulti (Keepgoing, NS, multi, OP); write data and oplog
} Else {
Checkandinsert (NS, multi [0]);Write Data and oplog
Globalopcounters. incinsertinwritelock (1 );
Op. debug (). ninserted = 1;
}
Return;
}

The bold part of the above Code is the focus, while the insertmulti function still calls checkandinsert,
Void checkandinsert (const char * NS,/* modifies */bsonobj & JS ){
{
While (I. More ()){
Thedatafilemgr. insertwithobjmod () data inserted into the database
Logop ("I", NS, JS); write oplog
}
}

The role of the logop function is to write oplog (the oplog table of the local database). The main part of this function is as follows:

If (replsettings. Master ){Determine master nodes in master-slave mode

_ Logop (opstr, NS, 0, OBJ, patt, B, frommigrate); Subject:

}

_ Logopold
---- Lock: dbwrite LK ("local"); lock the local database
---- Write oplog to write the oplog table

2) summarize the write operations on the master node

1. The database to write the write lock
2. Write Data
3. Write lock local database
4. Write oplog

(3) Main procedures and functions of the startreplication () function

The main function code is as follows:

If (replsettings. Slave ){
Boost: thread repl_thread (replslavethread)Start a replslavethread thread from the node.
Replslavethread );
}

If (replsettings. Master ){

Replsettings. Master = true;
Createoplog ();Related to the primary node's oplog table Creation
Boost: thread t (replmasterthread );Start a replmasterthread
}
1. analyze the role of the master node replmasterthread thread
Static void replmasterthread (){
Int tosleep = 10;
While (1 ){
Sleepsecs (tosleep );Sleep for 10 seconds
{
Writelocktry LK (1 );
Logkeepalive (); focus, which is analyzed below
}
}

LogkeepaliveThe function body is as follows:
Void logkeepalive (){
_ Logop ("N", "", 0, bsonobj (), 0, 0, false );See the previous analysis of the data written by the master node to lock the local database.
}

Now it is clear that this thread writes a data record to the oplog every 10 seconds, as shown below:
{"Ts": Timestamp (1373347524, 1), "op": "N", "ns": "", "O ":{}}

2. Slave partReplslavethread thread Analysis

Void replslavethread (){
Sleepsecs (1 );
Client: initthread ("replslave ");
While (1 ){
Try {
Replmain (); the slave thread calls this function in a loop. The function is analyzed below.
Sleepsecs (5 );
}
}
}
The main body of the replmain function is as follows:
While (1 ){
S = _ replmain (sources, napplied );
}

_ Replmain the main part of the function is as follows:
_ Replmain
{
Lock: globalwrite LK;Obtain global locks
Replsource: loadall (sources );One slaver node can be configured with multiple master nodes
}Released
For (replsource: sourcevector: iterator I = sources. Begin (); I! = Sources. End (); I ++ ){
Res = s-> sync (napplied );
Sync
---- Sync_pulloplog

--------- Obtain data from the oplog on the master node
--------- Lock: globalwrite> LK (justone? 0: New lock: globalwrite (); obtain the global write lock.
-------- Sync_pulloplog_applyoperation (bsonobj & OP, bool alreadylocked) writes oplog data to its own database

3. Summary of slave

Open a thread to read oplog information from the master node cyclically, obtain the global write lock, and write data to the database.

Consistent with the description in the official document, the slave node can still serve the master node information. When oplog is applied to its own database, the global write lock is obtained and cannot serve the client.