MongoDB Build Replica set

A replica set (Replica set) is a cluster of MongoDB instances consisting of a primary (Primary) server and multiple backup (secondary) servers. With replication, updates to the data are pushed from primary to other instances, and after a certain delay, each MongoDB instance maintains the same copy of the DataSet. By maintaining redundant database replicas, you can achieve offsite backup of data, read-write separation, and automatic failover.

One, MongoDB version and Environment

Create a replica set with three nodes on Windows, using the environment:

• Database: MongoDB version 3.2.9
• Server environment: Windows Server R2
• Visual Programming Environment: Robomongo version 0.08.4
• Three Windows Server:srv1,srv2,srv3

Second, topological structure analysis

Create a replica Set that contains three members: a primary member and two secondary members, primary is used to process client requests and secondary to hold primary copies of the data. The client application reads and writes on the Primary node, synchronizes the data operation to the secondary member through the asynchronous synchronization mechanism of replication, and after a certain delay, three members have the same data set.

Theoretically, each member can be distributed in a different data center room, these data centers may be far apart, to achieve offsite backup of data; You can set the primary node to be responsible for the write operation, and the secondary node is responsible for the read operation, to achieve the data set read and write separation; if primary If the connection is interrupted more than 10s, the other nodes will automatically elect a primary node, which is responsible for responding to client application's request and realizing automatic data failover.

Three, technical principle explanation

1, start the MongoDB instance in replica set mode

MongoDB instance has two different startup modes: stand-alone mode (Standalone) and replica set mode (Replica set), and if you set the Replset parameter when you start Mongod, the MongoDB instance starts in replica set mode If you do not set the Replset parameter, MongoDB starts in stand-alone mode.

A stand-alone instance is a Mongod process running on a server that is not a member of any replica set, so a standalone instance cannot be automatically failed over, the risk is high in the product environment, and if the server crashes, the client app is inaccessible for at least some time, If there is a problem with the hardware, it can cause permanent loss of data. It is recommended that you retain at least two copies of the data set using replica set.

2, election of primary members

There are two types of members in the replica set: Primary members and secondary members, and a replica set can have only one primary member, but there may be more than one secondary member. The primary is used to process client requests, and secondary is used to hold primary copies of the data. If the primary crashes, Replica set detects that the primary is inaccessible, initiates an automatic failover process, and, from the remaining secondary members, polls a member as primary, receiving and processing the client's request.

When electing primary members, the principle of "majority" and "one vote veto" is used. In replica set, each member can only request that he be elected as the primary node. When a secondary member is unable to communicate with the primary member, the member will initiate an election, requesting the other Members to elect themselves as primary members, who shall be elected as primary members only if they are supported by a "majority" member; The election will be canceled.

Not every member has the right to vote, and in a replica set, there are up to 7 members for the right to vote, and primary members are elected by these 7 members. A member with voting rights, whose attribute: "votes" is 1, and if 0, indicates that the member has no voting rights.

3, Operation Log

MongoDB uses the operations log (Oplog) to implement replication (Replication) functionality, Oplog contains every update operation for primary members, by passing oplog to other secondary members. Redo (Redo) The committed operation in the other member to achieve asynchronous synchronization of the data. Each member of the Replica set maintains its own oplog that records the data of each copy operation from the primary member. The procedure for a copy operation is to copy the data first and then write the operation to Oplog. If a member fails while performing an operation, the last action from Oplog is automatically synchronized when the member restarts. Because the process of copying is to replicate the data before writing to Oplog, the member may perform the copy operation again on the data that has already been synchronized, and MongoDB takes this into account when designing Oplog: The same operation is executed multiple times in Oplog, the same as the result of one execution.

Oplog saves the update operation log for each doc, if a command updates only one doc, then the replication process inserts a log into Oplog, and if one command updates more than one doc, the replication process inserts multiple logs into Oplog. Each log updates only one doc.

The size of the oplog is fixed, only a specific number of operation logs can be saved, if the primary member update the amount of data is particularly large, oplog is quickly filled up, secondary too late to synchronize data, primary members will oplog in the log, so, Secondary members will become obsolete (Stale). It is recommended to have primary members use larger oplog to save enough log of operations.

Iv. Creating a configuration document

1. Create a Mongod-initiated configuration file

Create a profile on each server and store the configuration file in directory C:\data\. All members in the same replica set must have the same replica set Name, which is set to "Rs0".

--srv1    config_file_name:rs0_1.conf
dbpath=C:\data\db\db_rs0
logpath=C:\data\db\db_rs0\rs0_1.log
port=40001
replSet=rs0

--srv2   config_file_name:rs0_2.conf
dbpath=C:\data\db\db_rs0
logpath=C:\data\db\db_rs0\rs0_2.log
port=40002
replSet=rs0

--srv3  config_file_name:rs0_3.conf
dbpath=C:\data\db\db_rs0
logpath=C:\data\db\db_rs0\rs0_3.log
port=40003
replSet=rs0

Configuration parameter meaning:

• Replset: Sets the name of the replica set, which must have the same value in each configuration file.
• Dbpath:mongodb the directory used to store data, the default value is C:\data\db
• LogPath: Log data used to record Mongod
• PORT: Specifies the ports that MongoDB listens on, the default value is 27017

2, start Mongod in profile mode

In general, Mongod parameter values are invariant, and writing these parameters to the configuration file simplifies MongoDB management.

The Mongod command has parameters:--config or-F, which is used to reference the configuration file.

--srv1
mongod -f C:\data\rs0_1.conf

--srv2
mongod -f C:\data\rs0_2.conf

--srv3
mongod -f C:\data\rs0_3.conf

3, start MONGO shell

Open three MONGO shells on any server, use the parameter--host to specify the server Name, and use--port to specify the port number. Because Mongod does not use the default listening port 27017, you must explicitly specify the port to listen using the--port parameter in the MONGO shell.

--connect srv1
mongo --host srv1 --port 40001

--connect srv2
mongo --host srv2 --port 40002

--connect srv3 
mongo --host srv3 --port 40003

Five, configuring replica Set

Different MongoDB Instance are running on different servers, but each mongod does not know the existence of other mongod, in order for each mongod to be able to perceive each other's existence, it is necessary to configure replica set and add members.

1, adding members to replica Set using the configuration document

In Srv1 's MONGO shell, create a configuration document, invoke the Rs.initiate () function, and initialize the replica Set according to the configuration document.

conf=
{
    "_id" : "rs0",
    "members" : [
        { "_id" : 0,  "host" : "srv1:40001"  },
        { "_id" : 1,  "host" : "srv2:40002"  },
        { "_id" : 2,  "host" : "srv3:40003"  }
    ]
}
rs.initiate(conf)

In configuration doc, use "_id": "Rs0" to specify the name,members array of Replica set to specify the ID of the member of the Replica set and the host ("Host:port"). When all member configurations are complete, Replica Set automatically elects a primary node, two secondary nodes. With the update operation on the primary node, you can synchronize to the secondary node.

2, modify replica Set

If replica set is initialized in Rs.initiate (), MongoDB initializes the replica set with the default configuration document, which can be increased by adding the member through the Add () function.

2.1 Adding a member to the replica set

Rs.add ("Host:port")

2.2 Remove a member from the replica set

Rs.remove ("host")

2.3 Viewing the configuration of the replica set

Rs.conf ()

2.4 Reconfigure Replica Set

var conf=rs.conf () conf.members[1].priority =5
--at Primary Member
rs.reconf (CONF)

--at secondary memberrs.reconf (Conf,{force:true})

2.5 Viewing the status of replica set

Rs.status ()

Six, maintenance replica Set

1. View the configuration information for the replica set

Rs.conf ()

The configuration document is divided into three main blocks: the ID of the Replica Set and the version,members array and Settings, and the following are simplified configuration information.

{
    "_id" : "rs0",
    "version" : 202997,
    "members" : [ 
        {
            "_id" : 1,
            "host" : "srv1:40001",
            "arbiterOnly" : false,
            "hidden" : false,
            "priority" : 5,
            "votes" : 1
        }, {...}
    ],
    "settings" : {  .....  }
}

The ID field of the Replica set uniquely identifies a Replica set, each Replica set has an auto-increment version number, identified by the version field, and identifies different versions of the Replica set. The initial value of the version field is 1, and the version field is self-increment each time you modify the configuration of the replica set.

The value of the Settings field is the configuration information that is applied to all members in the replica set.

The most critical is the field of the members array, which identifies the configuration information for each member.

• arbiteronly:0 or 1, which identifies a quorum (arbiter), Arbiter's sole role is to participate in primary elections, arbiter not save the data, and does not provide services to the client, and its significance is to elect primary.
• hidden:0 or 1, which indicates that the member is a hidden member, the primary role of the hidden member is to back up the data and can use a poorly performing server as a hidden member. Hidden members do not receive client requests and do not become primary. When setting the hidden member, you must set the Members[n].priorty property to 0;
• Priority: A numeric type that sets the precedence of a member as a primary. Higher priority members have the opportunity to become primary. If priority=0, then the member will never become primary.
• Votes:1 or 0, which represents the number of votes for the member, in each replica set, has a maximum of 7 members whose votes property value is 1. The Votes property is a member of 1 (voting members) who has the right to elect primary. A member who wants to be a primary must be supported by the majority of voting members.

In replica set, if the number of voting members is 5, then a member becomes primary with the support of more than 2 voting, and there is no voting opposition. As long as any one voting member against the member becomes primary, then the member cannot be primary.

2, forcing a member to become primary

If the priority property of a member is the highest in all members of the replica set, then that member is most likely to become primary.

Set the priority of member 0 to 5, and the other member's priorities to 1, so that member 0 has the highest precedence for primary.

cfg = rs.conf()
cfg.members[0].priority = 5
cfg.members[1].priority = 1
cfg.members[2].priority = 1
rs.reconfig(cfg)

Seven, test data

1. Read and write data on primary

rs.setSalveOk()
db.users.find({_id:1})

2, read the data on the secondary

By default, clients cannot read data from secondary members. After the Setslaveok is explicitly executed on the secondary member, the data can be read from the secondary node.

Rs.setsalveok ()

Db.users.find ({_id:1})

Eight, view the command line parameters of the Mongod server

Db.servercmdlineopts ()

/* 0 */
{
    "argv" : [ 
        "mongod", 
        "-f", 
        "C:\\data\\rs0_1.conf"
    ],
    "parsed" : {
        "config" : "C:\\data\\rs0_1.conf",
        "net" : {
            "port" : 40001
        },
        "replication" : {
            "replSet" : "rs0"
        },
        "storage" : {
            "dbPath" : "C:\\data\\db\\db_rs0"
        },
        "systemLog" : {
            "destination" : "file",
            "path" : "C:\\data\\db\\db_rs0\\rs0_1.log"
        }
    },
    "ok" : 1
}

Reference doc:

Replication

Replica Set Tutorials

Mongodb-replication

Replica Set Configuration

Force a Member to become Primary

MongoDB Build Replica set