MongoDB replica set (for election replication, failover, upgrade Oplog size, certified replication)

What is a replication set?

• A replica set (replica sets) is an additional copy of the data that is the process of synchronizing data across multiple servers, which provides redundancy and increases data availability, and enables recovery of hardware failures and outage services through replication sets.

Benefits of replication Sets

• Make your data more secure.
• High data availability.
• Disaster recovery.
• Maintenance without downtime (e.g. backup, index rebuild, failover)
• Read scaling (extra copy read)
• The replica set is transparent to the application.

Replication Set Overview

• The MongoDB replica set is an additional copy of the data, and the replica sets provide redundancy and increase the availability of data.
• MongoDB's replica set requires at least two nodes, where the primary node is responsible for processing client requests, from which the node is responsible for replicating data on the master node.
• The MongoDB replica set enables high availability of the cluster and automatically switches when the primary node fails.
• Replication is based on the operation Log Oplog, which is equivalent to the binary log in MySQL and records only the changes that have occurred. Replication is the process of synchronizing and applying the Oplog log of the master node to other nodes.
• The node types are divided into standard nodes, passive nodes, and quorum nodes. Only standard nodes may be elected as active (primary) nodes.
• Try to ensure that the oplog of the master node is large enough to hold an operation record for quite a long time.

Replica set service configuration 1, creating multi-instance

Mkdir-p/data/mongodb/mongodb{2,3,4}//Create data Directory
mkdir logs
Touch Logs/mongodb{2,3,4}.log//create log file
CD logs/
chmod 777 *.log//give permission

• Modifying a multi-instance configuration file

Vim/etc/mongod2.conf

# where to write logging data.
systemLog:
   destination: file
   logAppend: true
   path: /data/mongodb/logs/mongodb2.log // log file storage location

# Where and how to store data.
storage:
   dbPath: / data / mongodb / mongodb2 // Data file storage location
   journal:
     enabled: true
# engine:
# mmapv1:
# wiredTiger:

# how the process runs
processManagement:
   fork: true # fork and run in background
   pidFilePath: /var/run/mongodb/mongod.pid # location of pidfile
   timeZoneInfo: / usr / share / zoneinfo

# network interfaces
net:
   port: 27018 // Listen port and IP address
   bindIp: 0.0.0.0 # Listen to local interface only, comment to listen on all interfaces.

#security:

#operationProfiling:

replication:
   replSetName: abc // Enable the master-slave replication set function, each node must be configured with the same name

2. Configure replication sets for three nodes

• Once you have created a multi-instance, after modifying the completed profile, restart each service in turn.

Mongod-f/etc/mongod.conf--shutdown
Mongod-f/etc/mongod.conf
Mongod-f/etc/mongod2.conf
Mongod-f/etc/mongod3.conf
Mongod-f/etc/mongod4.conf

• Go to master node configuration replica set

Mongo

> show dbs // View database

> rs.status () // View replication set status

> cfg = {"_ id": "abc", "members": [{"_ id": 0, "host": "192.168.144.112:27017"},{"_id":1,"host":"192.168 .144.112: 27018 "}, {" _ id ": 2," host ":" 192.168.144.112:27019 "}]} // Configure the replica set node IP

> rs.initiate (cfg) // Ensure that there is no data from the slave node when initializing the configuration

• When you are able to see the status, you can see that each node state represents the replication set configuration complete.

3. Node additions and deletions for replica sets

abc: PRIMARY> rs.add ("192.168.144.112:27020") // Add node

abc: PRIMARY> rs.remove ("192.168.144.112:27020") // Remove node

rs.status ()

4, Failover switch automatic switch

PS aux | grep Mongod

• Analog failure, replication set complete automatic switchover

Kill-9 46374

Manually switch

abc: PRIMARY> rs.freeze (30) // Stop 30s from participating in elections

abc: PRIMARY> rs.stepDown (60,30) // hand over the position of the master node, maintain the status of the slave node for not less than 60 seconds, wait 30 seconds to synchronize the logs of the master node and the slave

MongoDB copy election 1, copy principle

• Replication is based on the operation Log Oplog, which is equivalent to the binary log in MySQL, which records only the changed records, and the replication is the process of synchronizing and applying the Oplog log of the master node to other slave nodes.

2. Principles of Election

• The node types are divided into: standard (host) node, passive (passive), and quorum (arbiter) nodes.

• (1) Only the standard node can be elected as active (primary) node, have the right to vote, the passive node has a complete copy, cannot become an active node, has the right to vote, the quorum node does not copy data, cannot become an active node, only the right to vote.
• The difference between the standard node and the passive node is that the higher priority value is the standard node, the lower is the passive node.
• The election rule is that the votes are high and the priority is the value of 0~100, which is equivalent to the additional 0~100 of the ticket number.

Copy an election configuration

• As in the previous section, open the Copy feature in the configuration file:

replication:  replSetName: abc

Election node settings

MONGO//Enter the database

cfg = {"_ id": "abc", "members": [{"_ id": 0, "host": "192.168.144.112:27017","priority":100},{"_id":1," host ":" 192.168.144.112:27018","priority":100},{"_id":2,"host":"192.168.144.112:27019","priority":0},{"_id":3 , "host": "192.168.144.112:27020","arbiterOnly":true}]}
// Set node IP port and node type

> rs.initiate (cfg) // Initialize the database

> rs.isMaster () // View status

• All modifications to the master node will be recorded in the Oplog log, which will simulate the modifications and view the Oplog log records.

abc: PRIMARY> use kgc // Create database

abc: PRIMARY> db.t1.insert ({"id": 1, "name": "tom"}) // Create collection t1 and insert data

abc: PRIMARY> db.t1.insert ({"id": 2, "name": "jerry"})

abc: PRIMARY> db.t1.find () // View collection data

abc: PRIMARY> db.t1.update ({"id": 2}, {$ set: {"name": "jack"}}) // Modify the collection data

abc: PRIMARY> db.t1.remove ({"id": 1}) // Delete data

abc: PRIMARY> use local // Enter the database where oplog is located

abc: PRIMARY> show collections // View all collections
     oplog.rs // oplog collection

abc: PRIMARY> db.oplog.rs.find ()
// View the log to record all operations.At this time, the slave node will synchronize data from the oplog.

Analog Standard node failure

• When the standard node 1 election is primary, in order to simulate the failure, directly select the Shutdown node 1

Mongod-f/etc/mongod.conf--shutdown

• Go to Node 2 and view node 2 status
• This will elect the second standard node as the primary node

MONGO--port 27018

• When continuing to simulate the standard node 2 failure, at this time two standard nodes can not work, at this time into the passive node three, found that the passive node three can not become the primary node

MONGO--port 27019

Allow data to be read from a node

• Write data on a standard node, replication is also synchronous on other nodes, how can I read data from the slave node?

• Go to the Slave node database

[[Email protected]]# MONGO--port 27018

abc: SECONDARY> show dbs // At this time, the slave node is not allowed to read data

abc: SECONDARY> rs.slaveOk () // Allow reading data from the node by default

abc: SECONDARY> show dbs // When you check again

Viewing replication status information

abc: SECONDARY> rs.help () // View command help manual
abc: SECONDARY> rs.printReplicationInfo ()
configured oplog size: 990MB
log length start to end: 1544secs (0.43hrs)
oplog first event time: Mon Jul 16 2018 05:49:12 GMT + 0800 (CST)
oplog last event time: Mon Jul 16 2018 06:14:56 GMT + 0800 (CST)
now: Mon Jul 16 2018 06:14:59 GMT + 0800 (CST)

abc: SECONDARY> rs.printSlaveReplicationInfo ()
source: 192.168.235.200:27018
     syncedTo: Mon Jul 16 2018 06:16:16 GMT + 0800 (CST)
     0 secs (0 hrs) behind the primary
source: 192.168.235.200:27019
     syncedTo: Mon Jul 16 2018 06:16:16 GMT + 0800 (CST)
     0 secs (0 hrs) behind the primary

abc: ARBITER> rs.printReplicationInfo ()
cannot provide replication status from an arbiter.
   // It will be found that the arbitration node does not have data replication

Change the Oplog log size

• Oplog is the shorthand for operation log, which is stored in the local database. The new operation in Oplog automatically replaces the old operation to ensure that the oplog does not exceed the preset size. By default, the Oplog size consumes 64-bit disk space for instance 5%.
• In the process of MongoDB replication, the master node applies business operations to the database, then records these operations into the oplog, copies these oplog from the nodes, and then applies these modifications, which are asynchronous if the operation from the node has been dropped very far by the master node, Oplog log in from the node has not finished, Oplog may have rolled round, from the node to keep up with synchronization, replication will stop, from the node needs to do a full synchronization, in order to avoid this situation, try to ensure that the main node oplog large enough to hold a considerable amount of time to record operations.

To view the current Oplog log file size

abc: PRIMARY> db.printReplicationInfo ()
configured oplog size: 1613.301513671875MB // default size
log length start to end: 18650secs (5.18hrs)
oplog first event time: Tue Jul 17 2018 11:08:30 GMT + 0800 (CST)
oplog last event time: Tue Jul 17 2018 16:19:20 GMT + 0800 (CST)
now: Tue Jul 17 2018 16:19:21 GMT + 0800 (CST)

Offline upgrade, change Oplog log size

• For the primary node server oplog log size, if the primary node server exists in the replication set, we need to close the primary node service, then close the replication replication related options in the configuration file, and modify the port number, because if the port number is not changed, when the service starts, Will still be added to the replica set queue. Start its service as a single instance.

Mongo

abc: PRIMARY> use admin
switched to db admin
abc: PRIMARY> db.shutdownServer ()

You can also use mongod -f /etc/mongod.conf --shutdown to shut down the service

Vim/etc/mongod.conf

• Logoff replication: Related startup parameters, and modify port port number 27027

...
# network interfaces
net:
   port: 27027 // Modify the port
   bindIp: 0.0.0.0 # Listen to local interface only, comment to listen on all interfaces.

#security:

#operationProfiling:

#replication: // Replication replication function
  # replSetName: abc
...

• Single instance startup of the master node

Mongo-f/etc/mongod.conf

• Back up the current Oplog log

Mongodump--port 27027--db local--collection ' oplog.rs '//Full backup current node Oplog record

• Enter the current node MongoDB

MONGO--port 27027

> use local // Enter the local database
> db.oplog.rs.drop () // Delete the original collection of oplog
> db.runCommand ({create: "oplog.rs", capped: true, size: (2 * 1024 * 1024 * 1024)}) // Rebuild the oplog and specify the size
> use admin
> db.shutdownServer () // shut down the service

• Change the configuration file back

...
# network interfaces
net:
   port: 27017 // Modify the port back to the original port
   bindIp: 0.0.0.0 # Listen to local interface only, comment to listen on all interfaces.

#security:

#operationProfiling:

replication: // Uncomment replication
   replSetName: abc
   oplogSizeMB: 2048 // Add the specified oplog log size
...

• Start the node service

Mongod-f/etc/mongod.conf
MONGO--port 27017

abc: SECONDARY> db.printReplicationInfo ()
configured oplog size: 2048MB // oplog log size has changed
log length start to end: 30secs (0.01hrs)
oplog first event time: Tue Jul 17 2018 17:33:18 GMT + 0800 (CST)
oplog last event time: Tue Jul 17 2018 17:33:48 GMT + 0800 (CST)
now: Tue Jul 17 2018 17:33:54 GMT + 0800 (CST)
abc: SECONDARY>

Deploying certified Replication

Mongo

• Enter primary

  kgcrs:PRIMARY> use adminkgcrs:PRIMARY> db.createUser({"user":"root","pwd":"123","roles":["root"]})   //创建管理用户，并且使用管理用户认证

• To turn on the authentication feature in each instance configuration file

[[Email protected]]# vim/etc/mongod.conf
[[Email protected]]# vim/etc/mongod2.conf
[[Email protected]]# vim/etc/mongod3.conf
[[Email protected]]# vim/etc/mongod4.conf

...
security:
   keyFile: /usr/bin/kgcrskey1
   clusterAuthMode: keyFile
...

cd/usr/bin/

• Create the authentication file and enter the unified authentication key

  [[email protected] bin] # echo "kgcrs key"> kgcrskey1
  [[email protected] bin] # echo "kgcrs key"> kgcrskey2
  [[email protected] bin] # echo "kgcrs key"> kgcrskey3
  [[email protected] bin] # echo "kgcrs key"> kgcrskey4
  [[email protected] bin] # chmod 600 kgcrskey {1..4} // Set permissions, only the owner can view
  

• Restart four instances

Enter primary
kgcrs: PRIMARY> show dbs // Unable to view the database
kgcrs: PRIMARY> rs.status () /// Cannot view replication set

kgcrs: PRIMARY> use admin // Identity login verification
kgcrs: PRIMARY> db.auth ("root", "123")

kgcrs: PRIMARY> rs.status () // You can view the database
kgcrs: PRIMARY> show dbs // You can view the replication set 

MongoDB replica set (for election replication, failover, upgrade Oplog size, certified replication)