Shard Storage of a MongoDB cluster

We left two questions in the last section that haven't been solved yet:

• From each node above the data are all copies of the database, from the node pressure will not be too large?
• Data pressure is large enough to automatically expand when the machine doesn't support it.

early in the system, the amount of data will not cause too much problem, but as the volume of data continues to increase, the subsequent sooner or later there will be a machine hardware bottleneck problem. and MongoDB Main is the massive data architecture,"Shard" Use this to solve the problem.

Sharding is the splitting of a logical data into different servers. Each split of data is independent,

Shard Benefits:

1 sharding reduces the amount of data per tile operation in the cluster while increasing the operational capacity of the cluster increases throughput

2 fragmentation reduces the amount of data that each server needs to store. As the cluster grows, each shard stores less data

How to Shard

Mongodb support for Shard clustering with configuration files

We can see through a picture

A shard cluster consists of three parts :Router (mongos) config Server, shard (Replica sets, replica set)

Shard: This is sharding, which stores data for high availability and consistency.

Mongos : Routing server, ingress of DB cluster, all requests through MONGOs do not need to add a route selection in the application, MONGOs itself is a request distribution center, responsible for forwarding object requests to the corresponding Shard server.

Configserver: as the name implies, configure the server to store the configuration of all database meta information (routing, sharding). the MONGOs itself does not have a physical storage Shard server and data routing information, but is cached in memory, and the configuration server actually stores the data. MONGOs the first boot or shutdown reboot will load configuration information from config server, and if configuration server information changes will notify all MONGOs to update their status, so that MONGOs can continue to route accurately. In a production environment there are usually multiple config server configuration servers, because it stores the metadata of the Shard route, this can not be lost! Even if you hang one of them, as long as there is inventory, the MongoDB cluster will not hang off.

Actual combat

1 First Prepare 3 each MongoDB database, respectively, as a configuration server , shard 1 and shard 2, are stored on one machine, corresponding ports are different

2 planning ports, configuring Servers - , the routing server + , Shard A 8091 , Shard B 8092,  

3 configuration information for individual servers  

3.1 the first is to create a configuration server

Configuration.confdbpath =j:\java\mongopointcut\conf\databaseport =2000bind_ip=127.0.0.1 Configuration.batmongod- -configconfiguration.conf

3.2 Create a router server, and connect the configuration server, the router is called MONGOs Command

MONGOs--port 1000--configdb 127.0.0.1:2000connectonmongos.batmongo127.0.0.1:1000/admin

3.3 Create a shard server configuration

sharda.confdbpath=j:\java\mongopointcut\pointcut2\databaseport=8092bind_ip=127.0.0.1 ConnictShartA.batmongod-- Config sharda.conf loginsharta.batmongo127.0.0.1:8091/admin shartb.confdbpath=j:\java\mongopointcut\pointcut2\ databaseport=8092bind_ip=127.0.0.1 Mongod--config shartb.conf

4 Logon Server

4.1 Log on to the configuration server first , log on to the routing server . When you start each shard server

Add a shard server to the router server. However, the MONGOs itself does not physically store the Shard server and data routing information, but only caches it in memory, and the configuration server actually stores the data .

Db.runcommand ({addshard: "127.0.0.1:8091", Allowlocal:true}) Db.runcommand ({addshard: "127.0.0.1:8092", allowLocal: True})

It is important to note that the Addshard , we can also add replica sets to achieve higher stability

4.2 The pieces are clustered, but MONGOs don't know how to slice data

Open the database sharding function, the command is very simple enablesharding (), here I opened the Foobar database.

Specifies the slice key of the Shard in the collection, where I designate the bar.id field , the slice key, to set the slice key in the mogondb

By looking at the configuration server, you can see

5 View Effects

We add The data of the million

function Add () {   vari=0;   for (; i<200000;i++) {  Db.bar.insert ({"Age": i+10, "name": "Jim"})  

through Printshardingsttus View Shard Condition

Mongos> showcollectionsbarpersonsystem.indexesmongos>db.bar.stats () {"sharded": True, "systemFlags": 1, "UserFlags": 1, "ns": "Foobar.bar", "Count": 931672,//Total number of bars "numextents": +, "s                Ize ": 104347264," storagesize ": 144752640," totalindexsize ": 36334144," indexsizes ": {  "_id_": 36334144}, "Avgobjsize":, "nindexes": 1, "Nchunks": 7, "shards":                         {"shard0000": {"ns": "Foobar.bar", "Count": 352282,//shard 1 "Size": 39455584, "avgobjsize": "Storagesize":                        58441728, "userflags": 1, "totalindexsize": 13441344,                        "indexsizes": {"_id_": 13441344},  "OK": 1              }, "shard0001": {"ns": "Foobar.bar", "Count":                                               579390,//Shard 2 "size": 64891680, "avgobjsize": 112, "Totalindexsize": 22892800, "indexsizes": {" _id_ ": 22892800}," OK ": 1}," OK ": 1}

you can see that the data is split into two shards, shard1352282 , Shard2 579390, Total 931672 , has been successful.

Summary: 

Testing with shards we recognize that shard storage reduces the copy of the database while increasing the throughput of the cluster, while fragmentation is also an enhancement of the replica set, which is improved on the basis of the replica set.

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.

Shard Storage of a MongoDB cluster