Research on Auto-sharding

Skip the theory. I have introduced automatic sharding on the internet... directly go to the configuration. Please leave a message if you have any questions.

Or test with million data

Insert million pieces of data to a database without sharding.

Data storage:

 
 

  
  
^_^[root@:/usr/local/mongodb/data/mongodb/test]#ls -lh 
  
  
total 4.0G 
  
  
-rw------- 1 root root  64M Jul 25 11:25 test.0 
  
  
-rw------- 1 root root 128M Jul 25 11:22 test.1 
  
  
-rw------- 1 root root 256M Jul 25 11:23 test.2 
  
  
-rw------- 1 root root 512M Jul 25 11:25 test.3 
  
  
-rw------- 1 root root 1.0G Jul 25 11:25 test.4 
  
  
-rw------- 1 root root 2.0G Jul 25 11:24 test.5 
  
  
-rw------- 1 root root  16M Jul 25 11:25 test.ns 
 
 

Perform automatic sharding configuration as follows:

Here we select two servers for testing.

Test Machine IP Address:

10. X. X.163

10. X. X.228

Start the Shardsvr process on the 163,228 server.

 
 

  
  
^_^[root@:/usr/local/mongodb/bin]#cat runServerShard.sh  
  
  
./mongod --shardsvr -dbpath=../data/mongodb --logpath=../data/shardsvr_logs.txt --logappend --fork 
 
 

Start the configsvr and mongos routing processes on the 163 server (the following 27019 is the ip address of configsvr, And the ip address of mongos can be viewed in the log)

 
 

  
  
[root@localhost bin]# cat runServerConfig.sh  
  
  
./mongod --configsvr --dbpath=../data/mongodb/config_db/ --logpath=../data/mongodb/log/configsvr_logs.txt --logappend --fork  
  
  
[root@localhost bin]# cat runServerMongos.sh  
  
  
./mongos --configdb 10.10.21.163:27019 --logpath=../data/mongodb/log/mongos_logs.txt --logappend --fork 
 
 

Check whether all startup is successful:

 
 

  
  
[root@localhost bin]# ps -ef | grep 'mongo' 
  
  
root     17939     1  2 18:47 ?        00:01:26 ./mongod --shardsvr -dbpath=../data/mongodb --logpath=../data/shardsvr_logs.txt --logappend --fork 
  
  
root     18250     1  0 19:05 ?        00:00:00 ./mongod --configsvr --dbpath=../data/mongodb/config_db/ --logpath=../data/mongodb/log/configsvr_logs.txt --logappend --fork 
  
  
root     18290     1 12 19:08 ?        00:03:35 ./mongos --configdb 10.10.21.163:27019 --logpath=../data/mongodb/log/mongos_logs.txt --logappend --fork 
 
 

It can be seen that all three services have been started successfully.

Configure Automatic parts as follows:

Use mongo to connect to the mongos process (see the log) for configuration management:

 
 

  
  
> use admin 
  
  
switched to db admin 
  
  
> db.runCommand({addshard:"10.10.21.163:27018"}); 
  
  
{ "ok" : 0, "errmsg" : "host already used" } 
  
  
> db.runCommand({addshard:"10.7.3.228:27018"}); 
  
  
{ "shardAdded" : "shard0001", "ok" : 1 } 
  
  
> db.runCommand({enableSharding:"test"}) 
  
  
{ "ok" : 1 } 
  
  
> db.runCommand({shardcollection:"test.users",key:{_id:1}}) 
  
  
{ "collectionsharded" : "test.users", "ok" : 1 } 
 
 

Insert million data into the mongos Process

Let's see the data size:

163:

 
 

  
  
^_^[root@:/usr/local/mongodb/data/mongodb]#ls -lh 
  
  
total 2.0G 
  
  
-rw-r--r-- 1 root root 1011 Jul 25 12:33 mongodb.log 
  
  
-rwxr-xr-x 1 root root    6 Jul 25 12:34 mongod.lock 
  
  
drwxr-xr-x 3 root root 4.0K Jul 25 12:39 moveChunk 
  
  
-rw------- 1 root root  64M Jul 25 12:42 test.0 
  
  
-rw------- 1 root root 128M Jul 25 12:40 test.1 
  
  
-rw------- 1 root root 256M Jul 25 12:41 test.2 
  
  
-rw------- 1 root root 512M Jul 25 12:42 test.3 
  
  
-rw------- 1 root root 1.0G Jul 25 12:41 test.4 
  
  
-rw------- 1 root root  16M Jul 25 12:42 test.ns 
  
  
drwxr-xr-x 2 root root 4.0K Jul 25 12:38 _tmp 
 
 

228:

 
 

  
  
[Root @ localhost mongodb] # ls-lh
  
  
Total 2.0 GB
  
  
Drwxr-xr-x 3 root 4.0 K 07-25 19:39 config_db
  
  
Drwxr-xr-x 2 root 4.0 K 07-25 18:51 config_repair_db
  
  
Drwxr-xr-x 2 root 4.0 K 07-25 log
  
  
-Rwxr-xr-x 1 root 6 07-25 19:39 mongod. lock
  
  
Drwxr-xr-x 3 root 4.0 K 07-25 19:13 moveChunk
  
  
-Rw ------- 1 root 64 M 07-25 20:38 test.0
  
  
-Rw ------- 1 root 128 M 07-25 20:38 test.1
  
  
-Rw ------- 1 root 256 M 07-25 20:40 test.2
  
  
-Rw ------- 1 root 512 M 07-25 20:41 test.3
  
  
-Rw ------- 1 root 1.0G 07-25 20:42 test.4
  
  
-Rw ------- 1 root 16 M 07-25 20:38 test. ns
  
  
Drwxr-xr-x 2 root 4.0 K 07-25 20:38 _ tmp
 
 

Auto-sharding into every 2 GB.