MongoDB build replica Set Shard cluster step

This document records the process of building a shard cluster from a MongoDB replica set from scratch.

We're going to create a distributed cluster like this: There are two Shard, each shard a replica set, each with two copies (a aribiter in the actual product application), three config servers, and a mongos. The steps are as follows:

1. Replica set

Replica set A
Mkdir-p./replset_shard1/node1
Mkdir-p./replset_shard1/node2
Numactl--interleave=all mongod--port 20001--dbpath./replset_shard1/node1--replset set_a--oplogsize 1024x768--logpath. Replset_shard1/node1/rs20001.log--fork
Numactl--interleave=all mongod--port 20002--dbpath./replset_shard1/node2--replset set_a--oplogsize 1024x768--logpath. Replset_shard1/node2/rs20002.log--fork
Initialization
Rs.initiate ({"_id": "Set_a", "members": [{_id:0, Host: "xxxhost:20001"}, {_id:1, Host: "Xxxhost:20002"}]})

Replica set B
Mkdir-p./replset_shard2/node1
Mkdir-p./replset_shard2/node2
Numactl--interleave=all mongod--port 30001--dbpath./replset_shard2/node1--replset set_b--oplogsize 1024x768--logpath. Replset_shard2/node1/rs30001.log--fork
Numactl--interleave=all mongod--port 30002--dbpath./replset_shard2/node2--replset set_b--oplogsize 1024x768--logpath. Replset_shard2/node2/rs30002.log--fork

Initialization
Rs.initiate ({"_id": "Set_a", "members": [{_id:0, Host: "xxxhost:30001"}, {_id:1, Host: "Xxxhost:30002"}]})

2. config server

Mkdir-p./data/configdb
Mongod--configsvr--fork--logpath/data/configdb/mongo17019.log--dbpath/data/configdb1--port 17019
Mongod--configsvr--fork--logpath/data/configdb/mongo27019.log--dbpath/data/configdb2--port 27019
Mongod--configsvr--fork--logpath/data/configdb/mongo37019.log--dbpath/data/configdb3--port 37019

3, MONGOs

Mkdir-p./data/mongosdb
MONGOs--configdb xxxhost:17019,xxxhost:27019,xxxhost:37019--logpath./mongosdb/mongos.log--fork--port 8100

When you start MONGOs, configuration information for config server does not use localhost, and you do not use 127.0.0.1, otherwise the shard of adding other machines will be error-prompted:
"Can ' t use localhost as a shard since all shards need to communicate. Either use all shards and configdbs on localhost or all in actual IPs host:xxxxx Islocalhost "

4. Replica set added to Shard cluster

test> Use admin
Switched to DB admin
Admin> Db.runcommand ({addshard: "set_a/xxxhost:20001"})
{"shardadded": "Set_a", "OK": 1}

Admin> Db.runcommand ({addshard: "set_b/xxxhost:30001"})
{"shardadded": "Set_b", "OK": 1}

View Config.databases:
Config> Db.databases.find ()
{"_id": "admin", "partitioned": false, "PRIMARY": "Config"}
{"_id": "Cswuyg", "partitioned": false, "PRIMARY": "Set_a"}

View Shards:
Config> Db.shards.find ()
{"_id": "Set_a", "host": "Set_a/xxxhost:20001,xxxhost:20002"}
{"_id": "Set_b", "host": "Set_b/xxxhost:30001,xxxhost:30002"}

5. Use Shard function for document

cswuyg> Use admin
Switched to DB admin
Admin> Db.runcommand ({"enablesharding": "Cswuyg"})
{"OK": 1}
Admin> Db.runcommand ({"Shardcollection": "Cswuyg.a", "key": {"_id": 1}})
{"collectionsharded": "Cswuyg.a", "OK": 1}

6. Insert Data test

var a = 10000
for (var i = 0; i < 1000000; ++i) {
Db.a.save ({"B": i})
}
The distribution effect of chunks after set auto-equalization:
Config> Db.chunks.find ()
{"_id": "Cswuyg.a-_id_minkey", "Lastmod": Timestamp (2, 0), "Lastmodepoch": ObjectId ("54f54f0a59b0d8e1cbf0784e"), "NS" : "Cswuyg.a", "min": {"_id": {"$minKey": 1}, "Max": {"_id": ObjectId ("54f477859a27767875b03801")}, "Shard": "Set_b"}
{"_id": "Cswuyg.a-_id_objectid (' 54f477859a27767875b03801 ')", "Lastmod": Timestamp (3, 0), "Lastmodepoch": ObjectId (" 54f54f0a59b0d8e1cbf0784e ")," ns ":" Cswuyg.a "," min ": {" _id ": ObjectId (" 54f477859a27767875b03801 ")}," Max ": {" _id ": ObjectId ("54f5507a86d364ad1c3f125f")}, "Shard": "Set_b"}
{"_id": "Cswuyg.a-_id_objectid (' 54f5507a86d364ad1c3f125f ')", "Lastmod": Timestamp (4, 1), "Lastmodepoch": ObjectId (" 54f54f0a59b0d8e1cbf0784e ")," ns ":" Cswuyg.a "," min ": {" _id ": ObjectId (" 54f5507a86d364ad1c3f125f ")}," Max ": {" _id ": ObjectId ("54f551fe86d364ad1c44a844")}, "Shard": "Set_a"}
{"_id": "Cswuyg.a-_id_objectid (' 54f551fe86d364ad1c44a844 ')", "Lastmod": Timestamp (3, 2), "Lastmodepoch": ObjectId (" 54f54f0a59b0d8e1cbf0784e ")," ns ":" Cswuyg.a "," min ": {" _id ": ObjectId (" 54f551fe86d364ad1c44a844 ")}," Max ": {" _id ": ObjectId ("54f552f086d364ad1c4aee1f")}, "Shard": "Set_a"}
{"_id": "Cswuyg.a-_id_objectid (' 54f552f086d364ad1c4aee1f ')", "Lastmod": Timestamp (4, 0), "Lastmodepoch": ObjectId (" 54f54f0a59b0d8e1cbf0784e ")," ns ":" Cswuyg.a "," min ": {" _id ": ObjectId (" 54f552f086d364ad1c4aee1f ")}," Max ": {" _id ": {"$maxKey": 1}, "Shard": "Set_b"}

7. Add a new replica for replica set set_a

Start a new replica instance
Mkdir-p./replset_shard1/node3
Numactl--interleave=all mongod--port 20003--dbpath./replset_shard1/node3--replset set_a--oplogsize 1024x768--logpath. Replset_shard1/node3/rs20003.log--fork
Adding a new replica instance to the replica set
Go to primary instance execution:
Test> Rs.add ("xxxhost:20003")
{"OK": 1}
The new replica instance after joining takes time to initialize the synchronization data, and the large data volume data initialization process can be long and have a significant impact on the service. And if the synchronization initialization process takes too long, and causes the Oplog space to be written full round, then again trigger synchronous initialization, in this case can be implemented in another way to implement the Add replica: Copy the disk file of the primary instance to the new directory and then start with a copy, then add to the replica Set, there is no need to have a synchronous initialization process in this way.

Reference:

http://docs.mongodb.org/manual/tutorial/convert-standalone-to-replica-set/http://docs.mongodb.org/manual/tutorial/restore-replica-set-from-backup/

8. Other

If you want to convert a shard cluster to a replica set cluster, you need to dump the data and restore it back;
If you want to convert a shard cluster to a replica set shard cluster, refer to:
http://docs.mongodb.org/manual/tutorial/convert-standalone-to-replica-set/

If you are a multi-machine cluster, do not use localhost, 127.0.0.1 in replica, or you will not be able to use multi-machine deployments.

This article is located in: http://www.cnblogs.com/cswuyg/p/4356637.html

MongoDB build replica Set Shard cluster step