Big Data MongoDB 3.2.1 Shard

MongoDB Shard

• In MongoDB there is another cluster, that is, the Shard technology, can meet the requirements of a large number of MONGODB data volume growth.

• When MongoDB stores massive amounts of data, a machine may not be enough to store data, or it may not be sufficient to provide acceptable read and write throughput. At this point, we can divide the data on multiple machines so that the database system can store and process more data.

Purpose of Sharding

High data volume and throughput of the database application will be more pressure on the performance of the single machine, large query volume will be a single CPU exhausted, large amount of data on the single-machine storage pressure, and eventually will exhaust the system's memory and transfer the pressure to disk IO.

WORKAROUND: There are two basic methods: vertical scaling and horizontal scaling.

• Vertical scaling: Add more CPU and storage resources to scale capacity.

• Horizontal scaling: Distributes datasets across multiple servers. Horizontal expansion is sharding

Shard Chart (image from Network):

The composition of the MongoDB shard cluster (image from the network): three main components of a MongoDB shard cluster:

Shard:
Used to store actual blocks of data, a Shard server role in the actual production environment can be assumed by several machines, one replica set, to prevent a single point of failure of the host

Config Server:
The Mongod instance, which stores the entire clustermetadata, including chunk information.

Query Routers:
Front-end routing, which the client accesses, and makes the entire cluster look like a single database, the front-end application can be used transparently.

Simple deployment of a shard cluster: Experimental environment: 1 Routing instances (port 27017). 1 Configuration instances (port 37017). 2 Shard Instances (ports 47017, 47018). 1. Configure the configuration server:

vim mongodb1.confport=37017dbpath=/data/mongodb/mongodb1logpath=/data/logs/mongodb1.loglogappend=truefork=truemaxConns=5000storageEngine=mmapv1configsvr=true        #开启配置服务mongod -f /usr/local/mongodb/bin/mongodb1.conf  #开启配置实例

2. Configure the Shard server:

vim mongodb2.confport=47017dbpath=/data/mongodb/mongodb2logpath=/data/logs/mongodb2.loglogappend=truefork=truemaxConns=5000storageEngine=mmapv1shardsvr=true    #开启分片服务vim mongodb3.confport=47018dbpath=/data/mongodb/mongodb3logpath=/data/logs/mongodb3.loglogappend=truefork=truemaxConns=5000storageEngine=mmapv1shardsvr=true    #开启分片服务mongod -f /usr/local/mongodb/bin/mongodb2.conf   #开启分片实例mongod -f /usr/local/mongodb/bin/mongodb3.conf

3. Start the routing server:

[[email protected] bin]# ./mongos --port 27017 --fork  --logpath=/usr/local/mongodb/bin/route.log --configdb 192.168.217.134:37017 --chunkSize 12018-07-23T14:15:28.185+0800 W SHARDING [main] Running a sharded cluster with fewer than 3 config servers should only be done for testing purposes and is not recommended for production.about to fork child process, waiting until server is ready for connections.forked process: 15337child process started successfully, parent exiting

4. Add a shard server:

[[email protected] bin]# mongomongodb shell version:3.2.1......mongos> show Dbsconfig 0.031GBmongos> Sh.status () #查看分片状态---sharding status---sharding version: {"_id": 1, "mincompatibleversion": 5, "cur Rentversion ": 6," Clusterid ": ObjectId (" 5b557280f9effb757fd31cdb ")} shards: #分片为空 active mongoses:" 3     .2.1 ": 1 balancer:currently enabled:yes currently running:no Failed balancer rounds in last 5 attempts:0 Migration Results for the last Hours:no recent migrations databases:mongos> Sh.addshard ("192.168.217.1 34:47017 ") #添加分片 {" shardadded ":" shard0000 "," OK ": 1}mongos> sh.addshard (" 192.168.217.134:47018 ") {" shardadded ":    "shard0001", "OK": 1}mongos> sh.status () #查看分片状态---sharding status---sharding version: {"_id": 1,      "Mincompatibleversion": 5, "CurrentVersion": 6, "Clusterid": ObjectId ("5b557280f9effb757fd31cdb")} Shards: #分片信息 {"_id": "shard0000",  "Host": "192.168.217.134:47017"} {"_id": "shard0001", "host": "192.168.217.134:47018"} Active Mongoses: "  3.2.1 ": 1 balancer:currently enabled:yes currently running:no Failed balancer rounds in last 5 attempts: 0 migration Results for the last Hours:no recent migrations databases:

4. Enable the Shard server:

Mongos> use testswitched to DB Testmongos> for (var i=1;i<=10000;i++) Db.users.insert ({"id": I, "name": "Tom" +i}) #添加数据WriteResult ({"ninserted": 1}) mongos> Sh.status () ... databases: {"_id": "Test", "PRIMARY": "Shard 0000 "," partitioned ": false} #partitioned value of False indicates that the database is not yet fragmented. Mongos> sh.enablesharding ("test") #启用数据库分片mongos > Db.users.createIndex ({"id": 1}) #创建索引mongos > Sh.shardcollection ("Test.users", {"id": 1}) #表分片 {"collectionsharded": "Test.users", "OK": 1}mongos> sh.status ()            ...... {"id": {"$minKey": 1}}-->> {"id": 2341} on:shard0001 Timestamp (5, 1) {"id": 2341}-->& Gt {"id": 4682} on:shard0001 Timestamp (3, 0) {"id": 4682}-->> {"id": 7023} on:shard0000 times  Tamp (6, 1) {"id": 7023}-->> {"id": 9364} on:shard0000 Timestamp (1, 3) {"id": 9364 }-->> {"id": 13407} on:shard0000 Timestamp (3, 2) {"id": 13407}-->> {"id": 21295} on:shard0000 Timestamp (3, 3) {"id": 21295}-->> {"id": 2597              6} on:shard0001 Timestamp (4, 2) {"id": 25976}-->> {"id": 33545} on:shard0001 Timestamp (4, 3) {"id": 33545}-->> {"id": 38226} on:shard0000 Timestamp (5, 2) {"id": 38226}--&G t;> {"id": 45910} on:shard0000 Timestamp (5, 3) {"id": 45910}-->> {"id": {"$maxKey": 1} } on:shard0001 Timestamp (6, 0) #数据存放在两个分片服务器上即: shard0000, shard0001.

5. Add tags to shards:

mongos> sh.status()......  shards:    {  "_id" : "shard0000",  "host" : "192.168.217.134:47017" }    {  "_id" : "shard0001",  "host" : "192.168.217.134:47018" }mongos> sh.addShardTag("shard0000","sales00")  #添加标签mongos> sh.addShardTag("shard0001","sales01")mongos> sh.status()......  shards:    {  "_id" : "shard0000",  "host" : "192.168.217.134:47017",  "tags" : [ "sales00" ] }    {  "_id" : "shard0001",  "host" : "192.168.217.134:47018",  "tags" : [ "sales01" ] }

6. Delete The Shard node:

mongos> use adminmongos> db.runCommand({"removeshard":"192.168.217.134:47018"})   #删除分片节点

Ps:mongodb more than 4 of the version of the Shard, you need to first make a copy set.

Big Data MongoDB 3.2.1 Shard