MongoDB Cluster--sharding

1, the structure and principle of the Shard
Shard Cluster Structure Distribution:

The Shard cluster consists of three components: Mongos,config Server,shard
1) MONGOS
The entrance to the database cluster request, all requests are coordinated through MONGOs, do not need to add a route selector in the application, MONGOs itself is a request distribution center, it is responsible for the corresponding data request request forwarded to the corresponding Shard server. In a production environment there are usually multiple MONGOs as the ingress of the request, preventing one from losing all MongoDB requests without a way to operate.
2) CONFIG SERVER
As the name implies, configure the server to store the configuration of all database meta information (routing, sharding). The mongos itself does not have a physical storage Shard server and data routing information, but is cached in memory, and the configuration server actually stores the data.
MONGOs the first boot or shutdown reboot will load configuration information from config server, and if configuration server information changes will notify all MONGOs to update their status, so that MONGOs can continue to route accurately. In a production environment, there are usually multiple config server configuration servers, because it stores the metadata for the Shard route, which can not be lost. Even if you hang one of them, as long as there is inventory, the MongoDB cluster will not hang off.
3) SHARD
This is the legendary shard.

As shown in the figure, one of the machine's data tables Collection1 stored 1T of data, too much pressure. After the 4 machines are divided, each machine is 256G, and the pressure that is concentrated on a single machine is apportioned.

In fact, the above figure of 4 shards if there is no replica set (replica set) is an incomplete schema, assuming that one of the shards hangs out of that one-fourth of the data is lost, so in a highly available Shard architecture It is also necessary to build replica set replica set for each shard to ensure the reliability of the Shard. The production environment is typically 2 copies + 1 arbitrations.

2. Building Shards
Suppose we want to configure a shard cluster as shown in the figure below

Because there are not enough machines, they are now deployed on the same machine, divided by different ports:

	"Machine" a	"Machine" two	"Machine" three
MONGOs	20000	20001	20002
Config	30000	30001	30002
Shard	40000,40001 (Dungeon), 40002 (Arbitration)	41000,41001 (Dungeon), 42002 (Arbitration)	42000,42001 (Dungeon), 42002 (Arbitration)

1) Start the configuration server
Example, start the configuration server for 3 "machines":
Mongod--configsvr--dbpath config\1\data--port 30000--logpath Config\1\log\config.log
Mongod--configsvr--dbpath config\2\data--port 30001--logpath config\2\log\config.log
Mongod--configsvr--dbpath config\3\data--port 30002--logpath config\3\log\config.log
2) Start MONGOs server
example, starting 3 "machine" mongos,jsj-1306-201 is the machine name under the computer:
MONGOs--configdb jsj-1306-201:30000,jsj-1306-201:30001,jsj-1306-201:30002--port 20000--logpath mongos\1\log \ Mongos.log
MONGOs--configdb jsj-1306-201:30000,jsj-1306-201:30001,jsj-1306-201:30002--port 20001--logpath mongos\2\log \ Mongos.log
MONGOs--configdb jsj-1306-201:30000,jsj-1306-201:30001,jsj-1306-201:30002--port 20002--logpath mongos\3\log \ Mongos.log
3) Start the Shard and its replica set instance
Shard1:
Mongod--shardsvr--replset shard1--port 40000--dbpath shard\1\data--logpath shard\1\log\shard.log--nojournal--opl Ogsize 10
Mongod--shardsvr--replset shard1--port 40001--dbpath shard\1_1\data--logpath shard\1_1\log\shard.log--nojournal- -oplogsize 10
Mongod--shardsvr--replset shard1--port 40002--dbpath shard\1_2\data--logpath shard\1_2\log\shard.log--nojournal- -oplogsize 10

Shard2:
Mongod--shardsvr--replset shard2--port 41000--dbpath shard\2\data--logpath shard\2\log\shard.log--nojournal--opl Ogsize 10
Mongod--shardsvr--replset shard2--port 41001--dbpath shard\2_1\data--logpath shard\2_1\log\shard.log--nojournal- -oplogsize 10
Mongod--shardsvr--replset shard2--port 41002--dbpath shard\2_2\data--logpath shard\2_2\log\shard.log--nojournal- -oplogsize 10

Shard3:
Mongod--shardsvr--replset shard3--port 42000--dbpath shard\3\data--logpath shard\3\log\shard.log--nojournal--opl Ogsize 10
Mongod--shardsvr--replset shard3--port 42001--dbpath shard\3_1\data--logpath shard\3_1\log\shard.log--nojournal- -oplogsize 10
Mongod--shardsvr--replset shard3--port 42002--dbpath shard\3_2\data--logpath shard\3_2\log\shard.log--nojournal- -oplogsize 10

4) Start the Shard replica set
Log on to any instance in each replica set to initialize the replica sets
config={_id: "Shard1", members:[
{_id:0,host: "jsj-1306-201:40000", "Priority": 2},
{_id:1,host: "jsj-1306-201:40001"},
{_id:2,host: "jsj-1306-201:40002", arbiteronly:true}
]
}

config={_id: "Shard2", members:[
{_id:0,host: "jsj-1306-201:41000", arbiteronly:true},
{_id:1,host: "jsj-1306-201:41001", "Priority": 2},
{_id:2,host: "jsj-1306-201:41002"}
]
}

config={_id: "Shard3", members:[
{_id:0,host: "jsj-1306-201:42000"},
{_id:1,host: "jsj-1306-201:42001", arbiteronly:true},
{_id:2,host: "jsj-1306-201:42002", "Priority": 2}
]
}
Detailed operation method reference replica set
5) Set the Shard for the Shard to take effect
Connect MONGOs, such as
MONGO jsj-1306-201:20000
Use admin
Db.runcommand ({addshard: "shard1/jsj-1306-201:40000,jsj-1306-201:40001,jsj-1306-201:40002"});
Db.runcommand ({addshard: "shard2/jsj-1306-201:41000,jsj-1306-201:41001,jsj-1306-201:41002"});
Db.runcommand ({addshard: "shard3/jsj-1306-201:42000,jsj-1306-201:42001,jsj-1306-201:42002"});
6) View Shards
Under the MONGOs
Db.runcommand ({listshards:1});
3, test the Shard
Under the MONGOs:
mongos> Use test
Switched to DB test
mongos> Use admin
Switched to DB admin
Mongos> Db.runcommand ({enablesharding: "TestDB"});
{"OK": 1}
Mongos> Db.runcommand ({shardcollection: "Testdb.table1", Key:{id:1}});
{"collectionsharded": "Testdb.table1", "OK": 1}
mongos> use TestDB;
Switched to DB TestDB
Mongos> for (var i=1;i<=100000;i++) Db.table1.save ({id:i, "test1": "Testval1"});
Check Data distribution:
Mongos> db.table1.stats ();

4, Lookup shard information
Shard cluster information is stored in the Congfig server, the database named CONFIG. Access is available through MONGOs.
mongos> use config
1) queries the current version of
Mongos> db.getcollection ("version"). FindOne ()
2) Query the size of the current configuration chunksize
mongos> db.settings.find ()
3) queries the members of the entire shard cluster
mongos> db.shards.find ()
4) Query a collection that is split horizontally:
mongos>  db.collections.find ()
5) Query the chunk distribution divided by the horizontally split
mongos> db.chunks.find ()
6) Query The Shard information of the database in the current MongoDB:
mongos> db.databases.find ()
7) Query MONGOs collection
mongos> db.mongos.find ( )
8) View details of the entire MongoDB database shard
mongos> printshardingstatus ();


Reference article:
http://www.lanceyan.com/tech/arch/mongodb_shard1.html
http://blog.itpub.net/22664653/ viewspace-710281/