Deploying MongoDB shard clusters and shard management

MongoDB Shard

• In MongoDB there is another cluster, that is, the Shard technology, can meet the requirements of a large number of MONGODB data volume growth.
• When MongoDB stores massive amounts of data, a machine may not be enough to store data, or it may not be sufficient to provide acceptable read and write throughput. At this point, we can divide the data on multiple machines so that the database system can store and process more data.

• Advantages of Sharding

Sharding provides a way to cope with high throughput and large data volumes.

• Using sharding reduces the number of requests that each shard needs to process, so the cluster can increase its storage capacity and throughput by horizontally scaling. For example, when inserting a piece of data, the app only needs to access the shards that store the data. Using shards reduces the data stored for each shard.
• The advantage of sharding is to provide a similar linear growth architecture, improve data availability, improve the performance of large database query servers, and when MongoDB single-point database server storage becomes a bottleneck, the performance of a single-point database server becomes a bottleneck or you need to deploy large-scale applications to make full use of memory, you can use sharding technology.

• The composition of a MongoDB shard cluster:

• Shard: A shard server for storing actual blocks of data, a Shard server role in the actual production environment can be assumed by a few machines, one replica set, to prevent a single point of failure of the host
• Config server: Configures the servers, Mongod instances, and stores the entire clustermetadata, including chunk information.
• Query routers: Front-end routing, where the client is connected and makes the entire cluster look like a single database, the front-end application can be used transparently.

• Deploying a MongoDB Shard cluster
• Installing mongodb3.2
• Create multi-instance data directories and log files

• Modify the value of Ulimit-n and Ulimit-u to 25000

  # yum install openssl-devel -y# tar zxvf mongodb-linux-x86_64-3.2.1.tgz -C /opt/# mv mongodb-linux-x86_64-3.2.1/ /usr/local/mongodb# mkdir -p /data/mongodb/mongodb{1,2,3,4}# mkdir /data/mongodb/logs# touch /data/mongodb/logs/mongodb{1,2,3,4}.log# chmod -R 777 /data/mongodb/logs/*.log# ulimit -n 25000# ulimit -u 25000

Configure the server

# cd /usr/local/mongodb/bin/# vim mongodb1.conf    port=37017    dbpath=/data/mongodb/mongodb1    logpath=/data/mongodb/logs/mongodb1.log    logappend=true    fork=true    maxConns=5000    storageEngine=mmapv1    configsvr=true

• Optimization: allocating memory from other nodes when a node is low on memory

  # sysctl -w vm.zone_reclaim_mode=0# echo never > /sys/kernel/mm/transparent_hugepage/enabled# echo never > /sys/kernel/mm/transparent_hugepage/defrag# ln -s /usr/local/mongodb/bin/mongo /usr/bin/mongo# ln -s /usr/local/mongodb/bin/mongod /usr/bin/mongod

• Start login The first instance confirms that it is correct

  # mongod -f /usr/local/mongodb/bin/mongodb1.conf # mongo --port 37017

• Shard Server

  # cp -p mongodb1.conf mongodb2.conf# vim mongodb2.conf    port=47017    dbpath=/data/mongodb/mongodb2    logpath=/data/mongodb/logs/mongodb2.log    logappend=true    fork=true    maxConns=5000    storageEngine=mmapv1    shardsvr=true# cp -p mongodb1.conf mongodb3.conf# vim mongodb3.conf    port=47018    dbpath=/data/mongodb/mongodb3    logpath=/data/mongodb/logs/mongodb3.log    logappend=true    fork=true    maxConns=5000    storageEngine=mmapv1    shardsvr=true# mongod -f mongodb2.conf# mongod -f mongodb3.conf# mongo --port 47017# mongo --port 47018# ./mongos --help   //查看参数选项

• Start the routing server
• --port Specify the connection entry 27017
• --fork Background Run
• --logpath specifying the log file storage path

• --configdb assigned to whom to deal with

  # ./mongos --port 27017 --fork --logpath=/usr/local/mongodb/bin/route.log --configdb 192.168.86.129:37017 --chunkSize 1

• Enable a shard server

  > mongo> mongos> show dbs> mongos> sh.status()    #shards下为空，没有分片服务器> mongos> sh.addShard("192.168.86.129 :47017")    //添加分片服务器> mongos> sh.addShard("192.168.86.129 :47018")> mongos> sh.status()       // 查看数据库分片信息

• Shard Feature Management

  > use kgc> for(var i=1;i<=10000;i++) db.users.insert({"id":i,"name":"tom"+i})mongos> show dbsmongos> use kgcmongos> show collections    //查看集合mongos> db.users.find().limit(5)mongos> sh.status()          //查看数据库分片信息

  
  

?
?

• Enable Database sharding

  mongos> sh.enableSharding("kgc")   //启用数据库分片mongos> sh.status()     //再查看数据库分片信息

• Now the database shard is open, but your collection is not developing shards
• Below to open the Shard of the collection
• To create an index on the Users collection first

• Then shard the index of the collection

  mongos> use kgc  mongos> db.users.createIndex({"id":1})       //对users表创建索引mongos> sh.shardCollection("kgc.users",{"id":1})    //表分片mongos> sh.status()

• • MongoDB Shard Management

• Select the TAB key, view the status information for the slice key, add multiple shards, view the configuration server information, and add the delete shard server.

  mongos> mongomongos> use kgcmongos> for(var i=1;i<=50000;i++)db.users2.insert({"id":i,"name":"jerry"+i})mongos> db.users2.createIndex({"id":1})mongos> sh.shardCollection("kgc.users2",{"id":1})mongos> db.users2.stats()

• Add tags

  mongos> sh.addShardTag("shard0000","sales00")mongos> sh.addShardTag("shard0001","sales01")

• Connection Configuration Server

  # mongo --port 37017configsvr> use config              //打开配置数据库configsvr> show collections        //查看集合

  configsvr> db.chunks.findOne()     //记录所有块的信息configsvr> db.collections.find()    //分片集合信息configsvr> db.databases.find()     //分片中所有数据库信息

• Adding a shard server

  # cd /usr/local/mongodb/bin/# cp -p mongodb2.conf mongodb4.conf# vim mongodb4.conf        port=47019        dbpath=/data/mongodb/mongodb4        logpath=/data/mongodb/logs/mongodb4.log        logappend=true        fork=true        maxConns=5000        storageEngine=mmapv1        shardsvr=true# mongod -f mongodb4.conf# mongo

• To add a shard node
  Mongos> Sh.addshard ("192.168.86.129:47019")//Add Shard Server
  Mongos> Sh.status ()
  

• Delete a shard node

  mongos> use adminmongos> db.runCommand({"removeshard":"192.168.86.129:47019"})    //删除节点

Deploying MongoDB shard clusters and shard management