Shard configuration for MongoDB distributed deployment

Http://eshilin.blog.163.com/blog/static/13288033020106215227346/

Shard configuration for MongoDB distributed deployment

17:22:07 | category: MongoDB | font size subscription
This section describes the shard configuration of MongoDB. If you want to configure a master-slave or master-slave pair for data backup, you can see the distributed data replication of MongoDB.

1. Start related processes
Start mongod on the shard server and use the -- shardsvr command line parameter. For master-slave pairs, use the -- pairwith command line option. We recommend that a shard Server Run only one mongod process.

Start mongod on config server and use the -- configsvr command line parameter. If config server is not an independent server, assign it an independent dbpath, -- dbpath command line parameter.

Start mongos and use the -- configdb parameter to specify the configured database address.

2. Configure shell parts.

Run these commands on the started mongos server and then run all the configuration commands. Note: You should use a specific admin database to store these commands, even though the data is stored on the config server during mongos running, therefore, you only need to run these commands once in the lifecycle of the cluster.

./Mongo <mongos-hostname>: <mongos-port>/admin
> DB
Admin
>

3. Add one slice. Each slice includes two servers (one Master/Slave) or a single server (alpha 2 only supports one server ).

> DB. runcommand ({addshard: "<serverhostname> [: <port>]"});
{"OK": 1, "added ":...}

Multiple configurations are separated by commas ).

The optional parameter maxsize allows you to set the disk space that can be used by the shard. The default parameter is the entire disk. This parameter is currently not valid in versions earlier than 1.4 and 1.4, and is valid in Versions later than 1.5.

> DB. runcommand ({listshards: 1 });

View the existing parts.

4. Enable a database, which must be a multipart enable database; otherwise, all data will be stored in the shard.

> DB. runcommand ({enablesharding: "<dbname> "});

Once a database is enabled, mongos places different data sets in the database on different shards. Unless the dataset is partitioned, all data in a dataset is placed in one partition.

5. Use the shardcollection command to separate datasets. Keys are automatically generated.

> DB. runcommand ({shardcollection: "<namespace> ",
Key: <shardkeypatternobject>
})

For example, the gridfs block set in the sharding test database.

> DB. runcommand ({shardcollection: "test. fs. chunks", key: {_ ID: 1 }})
{"OK": 1}

Configure the shard key to be unique:

DB. runcommand ({shardcollection: "test. Users", key: {Email: 1}, unique: true });

6. Case study:

Two groups (one group, one group), three config databases, one mongos, and one test server.

First, set up the database storage location. Here it is only for testing, so it is in the current directory, mkdir data
Go to the bin/directory of the MongoDB installation directory to start MongoDB.
$./Mongod -- pairwith 10.13.127.212: 18020 -- dbpath data -- Port 18020 // start the first part (10.13.127.211: 18020)
$./Mongod -- pairwith 10.13.127.211: 18020 -- dbpath data -- Port 18020 // start the first part (10.13.127.212: 18020)
$./Mongod -- dbpath data -- Port 18020 // start the second part (10.13.127.210: 18020)
$./Mongod -- configsvr -- dbpath/var/MongoDB/sky/18022 -- Port 18022 // start config server (10.13.127.210: 18022, 10.13.127.211: 18022, 10.13.127.212: 18022)
$./Mongos -- configdb 10.13.127.210: 18022, 10.13.127.211: 18022, 10.13.127.212: 18022 // start mongos and specify config Server

PS: You can configure multiple config Servers. When mongos is started, specify the config server. Separate multiple config servers with commas (,). Note that no space exists between them, in addition, you will find that the number of config servers can only be a singular number, not a double number. It should be mentioned that v1.4 has a bug. If mongos configures multiple configurations at the beginning, there is a problem. When you execute the./Mongo command, an error will be reported when you execute the show DBS command:

JS error: uncaught exception: error {"$ Err": "mongos: syncclusterconnection: Call can't handle $ cmd "}.

There are two solutions:
1) When mongos is started, only one config is connected first. It seems that you can initialize the config, kill mongos, and then start multiple configs.
(2) This bug has been fixed in the version. When downloading the version, you can download the v1.4 version.
To put it bluntly, after the service is started, you can use PS-Ef | grep Mongo to view the running status and start using MongoDB.
$./Mongo -- Host 10.13.127.211 // connect to mongos. You can use -- port to specify the port.

> Show DBS
Admin
Config
Local
> Use Admin
> DB. runcommand ({addshard: "10.13.127.211: 18020, 10.13.127.212: 18020", allowlocal: true}) // Add the first group of shards.
> DB. runcommand ({addshard: "10.13.127.210: 18020", allowlocal: true}) // Add the second shard.

> DB. runcommand ({listshards: 1}) // you can view the shard information.

{
"_ Id": objectid ("4bd27bb39e2b00a5f7d5f0dd "),
"Host": "10.13.127.210: 18020"
},
{
"_ Id": objectid ("4bd27bd39e2b00a5f7d5f0de "),
"Host": "10.13.127.211: 18020, 10.13.127.212: 18020"
}

> Config = connect ("10.13.127.210: 18022") // connect to the config server, which must be operated on the mongos server.
> Config = config. getsisterdb ("Config") // get the configuration information

> Test = dB. getsisterdb ("test") // Add a database named test.

> DB. runcommand ({enablesharding: "test "})
> DB. runcommand ({shardcollection: "test. People", key: {Name: 1}) // create a sharded dataset people

> DB. printshardingstatus (); // view the shard status information. The details of shards and databases are displayed, including the primary shard information of the created test and the shard information of test. People.

My chunks
{"Name": "test", "partitioned": True, "primary": "10.13.127.211: 18020, 10.13.127.212: 18020", "sharded": {"test. people ": {" key ": {" name ": 1}," unique ": false }}," _ id ": objectid (" 4bd2813fbaaa646853a4b33a ")}
My chunks
Test. people {"name": {$ Minkey: 1 }}-- >>{ "name": {$ maxkey: 1 }}on: 10.13.127.211: 18020, 10.13.127.212: 18020 {"T": 1272086852000, "I": 1} 10.13.126.173: 18020 {"T": 1272086192000, "I": 1}

> Use test // enter the Test Database
Switched to DB Test
> DB. createcollection ("user_001") // create a dataset user_001, which is different from the people dataset and is not sharded.
{"OK": 1}
> Show collections // view the dataset
People
System. Indexes
User_001

> DB. People. insert ({name: "sky_people", sex: "male", age: 25}); // insert a record
> DB. People. Find ()
{"_ Id": objectid ("4bbc38a39e8b5e76dcf4c5a3"), "name": "sky_people", "sex": "male", "Age": 25}
> DB. user_001.insert ({name: "falcon. c", sex: "male", age: 25 });
> DB. user_001.find (); // view records
{"_ Id": objectid ("4bbc371c9e8b5e76dcf4c5a2"), "name": "sky", "sex": "female", "Age": 25}

> DB. People. Stats () // view the people dataset information. The inserted data is placed on the 10.13.127.210: 18020 partition.
{
"Sharded": True,
"Ns": "test. People ",
"Count": 1,
"Size": 72,
"Storagesize": 8192,
"Nindexes": 2,
"Nchunks": 1,
"Shards ":{
"10.13.127.211: 18020, 10.13.127.212: 18020 ":{
"Ns": "test. People ",
"Count": 1,
......

> DB. user_001.stats () // view user_001 dataset Information
{
"Sharded": false,
"Ns": "test. user_001 ",
"Count": 1,
"Size": 68,
......
}

When the data volume increases to a certain extent (it seems to be 64 MB, that is, the size of a data block), the chunks start to split the chunks from the first block to multiple blocks, the chunks of the data block will begin to be migrated to other shards.

> DB. printshardingstatus (); // view the shard status.

{"Name": "test", "partitioned": True, "primary": "10.13.127.211: 18020, 10.13.127.212: 18020", "sharded": {"test. people ": {" key ": {" name ": 1}," unique ": false }}," _ id ": objectid (" 4bbc3524314b8edcaebd990f ")}
My chunks
Test. people {"name": {$ Minkey: 1 }}-- >>{ "name": 1103} on: 10.13.127.211: 18020, 10.13.127.212: 18020 {"T": 1270627397000, "I": 1}
Test. people {"name": 1103} -- >>{ "name": 1259408} on: 10.13.127.211: 18020, 10.13.127.212: 18020 {"T": 1270627921000, "I": 2}
Test. people {"name": "sky_people"} -- >>{ "name": {$ maxkey: 1 }}on: 10.13.127.210: 18020 {"T": 1270627404000, "I": 3}
Test. People {"name": 20130082} -- >>{ "name": 21392803} on: 10.13.127.210: 18020 {"T": 1270627924000, "I": 2}
....

> DB. People. Stats ()
{
"Sharded": True,
"Ns": "test. People ",
"Count": 3384868,
"Size": 5293932968,
"Storagesize": 5942890496,
"Nindexes": 2,
"Nchunks": 33,
"Shards ":{
"10.13.127.211: 18020, 10.13.127.212: 18020 ":{
"Ns": "test. People ",
"Count": 3265179,
"Size": 5106739036,
"Storagesize": 5722345728,
"Numextents": 32,
"Nindexes": 2,
"Lastextentsize": 960247296,
"Paddingfactor": 1,
"Flags": 1,
"Totalindexsize": 312467456,
"Indexsizes ":{
"_ Id _": 177045504,
"Name_1": 135421952
},
"OK": 1
},
"10.13.127.210: 18020 ":{
"Ns": "test. People ",
"Count": 119689,
"Size": 187193932,
"Storagesize": 220544768,
"Numextents": 15,
"Nindexes": 2,
"Lastextentsize": 43281920,
"Paddingfactor": 1,
"Flags": 1,
"Totalindexsize": 12025856,
"Indexsizes ":{
"_ Id _": 7159808,
"Name_1": 4866048
},
"OK": 1
}
},
"OK": 1
}