MongoDB Shard Cluster

MongoDB Shard Cluster What is a shard

High data volume and throughput of the database application will be a single-machine performance of a large pressure, large query volume will be a single CPU exhausted, large data volume on the single-machine storage pressure, and eventually consume the system's memory and transfer the pressure to disk IO

The benefits of MongoDB sharding

Sharding provides a way to cope with high throughput and large data volumes

• Using sharding reduces the need to process requests, so by expanding horizontally, the cluster can increase its own storage and throughput.
• Using shards reduces the data stored for each shard

The Shard server consists of several servers

• Routing Server (Router)
• Configuration Server (CONFIGSVR)
• Shard Server (SHARDSVR)

We send the request to the configuration server through the routing server, and the configuration server calls the following Shard server structure diagram below

MongoDB Shard Cluster Deployment = = = (Note: The Shard cluster is different in versions 3.2 and 3.2) 3.2 versions can be a shard server for a single database when making a shard server, but at more than 3.2 it requires that each shard server must be clustered) = = This way, we're using the 3.2 version of the source code package. Experimental steps: Do Configuration server

---------------------安装MongoDB 3.2----------------------------#安装开发包yum install openssl-devel -y#解压源代码包tar zxvf mongodb-linux-x86_64-3.2.1.tgz -C /opt/#重命名并移动到/usr/local文件里面mv mongodb-linux-x86_64-3.2.1/ /usr/local/mongodb#创建数据存放位置mkdir -p /data/mongodb/mongodb{1,2,3,4}#创建日志存放位置mkdir /data/mongodb/logs创建以log为结尾的日志存放touch /data/mongodb/logs/mongodb{1,2,3,4}.log#给日志文件最大全权限chmod -R 777 /data/mongodb/logs/*.log#修改最大并发连接ulimit -n 25000ulimit -u 25000

Above is to start the installation of MongoDB, configure the server configuration file below to start configuration

cd /usr/local/mongodb/bin/vim mongodb1.conf#配置服务器端口号port=37017       #数据文件储存位置dbpath=/data/mongodb/mongodb1#日志文件储存位置logpath=/data/mongodb/logs/mongodb1.loglogappend=truefork=true#同时承受的并发连接数maxConns=5000storageEngine=mmapv1#配置为配置服务器configsvr=true

allocating memory from other nodes when a node is low on memory

sysctl -w vm.zone_reclaim_mode=0echo never > /sys/kernel/mm/transparent_hugepage/enabledecho never > /sys/kernel/mm/transparent_hugepage/defrag#建立软连接便于管理ln -s /usr/local/mongodb/bin/mongo /usr/bin/mongoln -s /usr/local/mongodb/bin/mongod /usr/bin/mongod#开启配置服务器mongod -f /usr/local/mongodb/bin/mongodb1.conf

Shard server 1 and Shard server 2 are also configured to change the port Number data file log file location join Shardsvr=true Delete the original configuration server Configsvr=true

#端口号改为47017port=37017        #数据文件储存位置dbpath=/data/mongodb/mongodb2#日志文件储存位置logpath=/data/mongodb/logs/mongodb.loglogappend=truefork=true#同时承受的并发连接数maxConns=5000storageEngine=mmapv1#配置为分片服务器shardsvr=true#分别开启分片服务器1和2

Start the routing server

cd /usr/local/mongodb/bin/#启动mongos 路由服务器的端口号为27017日志存放目录在/usr/local/mongodb/bin/下面的route.log里面配置服务IP：端口号 --chunkSize 1 块大小为1./mongos --port 27017 --fork --logpath=/usr/local/mongodb/bin/route.log --configdb 192.168.235.204:37017 --chunkSize 1

To start the MONGO routing server directly

[[email protected] bin]# mongomongodb Shell version:3.2.1connecting to:testwelcome to the MongoDB shell. For interactive help, type ' help '. For more comprehensive documentation, see http://docs.mongodb.org/Questions? Try the support group Http://groups.google.com/group/mongodb-userServer have startup warnings:2018-07-18t14:52:06.622+ 0800 I CONTROL [main] * * Warning:you was running this process as the root user, which was not recommended.2018-07-18t14:5 2:06.622+0800 I CONTROL [main] mongos> #查看数据库mongos > Show Dbsconfig 0.031gbmongos> sh.status () #shards下为空, no shards    Server---sharding Status---sharding version: {"_id": 1, "mincompatibleversion": 5, "CurrentVersion": 6, "Clusterid": ObjectId ("5b4ee3961c914459f603260f")} shards:active mongoses: "3.2.1": 1 balancer:currently Enab Led:yes currently Running:no Failed balancer rounds in last 5 attempts:0 migration Results for the last 24 Hours:no Recent migrations Databases: #添加分片服务器 mongos> sh.addshard ("192.168.32.207:47018") {"shardadded": "shard0000", "OK": 1}mongos> Sh.addshar D ("192.168.32.207:47017") {"shardadded": "shard0001", "OK": 1}mongos> sh.status ()---sharding status---sharding Version: {"_id": 1, "mincompatibleversion": 5, "CurrentVersion": 6, "Clusterid": ObjectId ("5b4ee3961c9144 59f603260f ")} shards: #下面两个是分片服务器 {" _id ":" shard0000 "," host ":" 192.168.32.207:47018 "} {" _id ":" shard0001 "," host ":" 192.168.32.207:47017 "} Active Mongoses:" 3.2.1 ": 1 balancer:currently Enabled:yes currently Running:no Failed balancer rounds in last 5 attempts:0 migration Results for the last Hours:no Recen T migrations databases:

Experimental Shard function

After you add two shard servers, the databases and collections have not enabled sharding use the Mongoimport command to import the Sales.txt data to the sales table of the KGC database

Mongos> db.createcollection (' users ') {"OK": 1}mongos> show collectionssystem.indexesusersmongos> exitbye[[ Email protected] bin]# mongomongodb Shell version:3.2.1connecting To:testserver has startup warnings:2018-07-18t1 4:52:06.622+0800 I CONTROL [main] * * Warning:you was running this process as the root user, which was not recommended.201 8-07-18t14:52:06.622+0800 I CONTROL [main] mongos> show Dbsconfig 0.031GBKGC 0.078gbmongos> for (Var i=1;i< =10000;i++) Db.users.insert ({"id": 1, "name": "Jaap" +i}) #插入10000条信息WriteResult ({"ninserted": 1}) mongos> Show Tablessystem.indexesusersmongos> Db.users.find () #查看信息 {"_id": ObjectId ("5b4ef398b83d741f7c50491a"), "id": 1, " Name ":" {"_id": ObjectId ("5b4ef407b83d741f7c50491b"), "id": 1, "name": "Jaap1"} {"_id": ObjectId ("5b4ef407b 83d741f7c50491c ")," id ": 1," name ":" Jaap2 "} {" _id ": ObjectId (" 5b4ef407b83d741f7c50491d ")," id ": 1," name ":" JAAP3 " } {"_id": ObjectId ("5b4ef407b83d741f7c50491e")), "id": 1, "name": "JAAP4"} {"_id": ObjectId ("5b4ef407b83d741f7c50491f"), "id": 1, "name": "Jaap5"} {"_id": objec TId ("5b4ef407b83d741f7c504920"), "id": 1, "name": "Jaap6"} {"_id": ObjectId ("5b4ef407b83d741f7c504921"), "id": 1, "NA Me ":" JAAP7 "} {" _id ": ObjectId (" 5b4ef407b83d741f7c504922 ")," id ": 1," name ":" Jaap8 "} {" _id ": ObjectId (" 5b4ef407b83 d741f7c504923 ")," id ": 1," name ":" Jaap9 "} {" _id ": ObjectId (" 5b4ef407b83d741f7c504924 ")," id ": 1," name ":" Jaap10 "} {"_id": ObjectId ("5b4ef407b83d741f7c504925"), "id": 1, "name": "Jaap11"} {"_id": ObjectId ("5b4ef407b83d741f7c504926"), "id": 1, "name": "JAAP12"} {"_id": ObjectId ("5b4ef407b83d741f7c504927"), "id": 1, "name": "Jaap13"} {"_id": ObjectId ("5b4ef407b83d741f7c504928"), "id": 1, "name": "Jaap14"} {"_id": ObjectId ("5b4ef407b83d741f7c504929"), "id": 1, "name": "JAAP15"} {"_id": ObjectId ("5b4ef407b83d741f7c50492a"), "id": 1, "name": "JAAP16"} {"_id": ObjectId ("5b4ef407b83d741f7c50492b"), "ID ": 1," name ":" Jaap17 "} {" _id ": ObjectId (" 5b4ef407b83d741f7c50492c ")," id ": 1," name ":" Jaap18 "} {" _id ": ObjectId  ("5b4ef407b83d741f7c50492d"), "id": 1, "name": "Jaap19"}type "it" for moremongos> db.users.find (). Limit (5) #查看指定信息 {  "_id": ObjectId ("5b4ef398b83d741f7c50491a"), "id": 1, "name": "Zhang"} {"_id": ObjectId ("5b4ef407b83d741f7c50491b"), "id": 1, "name": "Jaap1"} {"_id": ObjectId ("5b4ef407b83d741f7c50491c"), "id": 1, "name": "Jaap2"} {"_id": objecti D ("5b4ef407b83d741f7c50491d"), "id": 1, "name": "Jaap3"} {"_id": ObjectId ("5b4ef407b83d741f7c50491e"), "id": 1, "name" ":" JAAP4 "}mongos> sh.status () #查看数据库分片信息---sharding status---sharding version: {" _id ": 1," Mincompatib Leversion ": 5," CurrentVersion ": 6," Clusterid ": ObjectId (" 5b4ee3961c914459f603260f ")} shards: {" _id ":" Sh ard0000 "," host ":" 192.168.32.207:47018 "} {" _id ":" shard0001 "," host ":" 192.168.32.207:47017 "} Active Mongose S: "3.2.1": 1 Balancer:currently Enabled:yes currently Running:no Failed balancer rounds in last 5 attempts:0 migration Re Sults for the last Hours:no recent migrations databases: {"_id": "KGC", "PRIMARY": "shard0000", "par Titioned ": false}

Enable Sharding

Mongos> sh.enablesharding ("KGC") {"OK": 1}mongos> sh.status ()---sharding status---sharding version: {"_id ": 1," mincompatibleversion ": 5," CurrentVersion ": 6," Clusterid ": ObjectId (" 5b4ee3961c914459f603260f ")} sha RDS: {"_id": "shard0000", "host": "192.168.32.207:47018"} {"_id": "shard0001", "host": "192.168.32.207:47  017 "} Active Mongoses:" 3.2.1 ": 1 balancer:currently enabled:yes currently running:no Failed balancer  Rounds 5 attempts:0 Migration Results for the last Hours:no recent migrations databases: { "_id": "KGC", "PRIMARY": "shard0000", "partitioned": true# Create primary key index to the users table ID mongos> db.users.createIndex ({"id": 1}) # Table Shard mongos> sh.shardcollection ("Kgc.users", {"id": 1}) #查看分片情况mongos > Sh.status ()---sharding status---sharding Version: {"_id": 1, "mincompatibleversion": 5, "CurrentVersion": 6, "Clusterid": ObjectId ("5b4ee3961c9144  59f603260f ")} shards: {"_id": "shard0000", "host": "192.168.32.207:47018"} {"_id": "shard0001", "host": "192.168.32.207:47017"} Act Ive mongoses: "3.2.1": 1 balancer:currently enabled:yes currently running:no Failed balancer rounds in Last 5 attempts:0 migration Results for the last Hours:no recent migrations databases: {"_id": "Kg C "," PRIMARY ":" shard0000 "," Partitioned ": true} kgc.users Shard key: {" id ": 1} uniqu E:false balancing:true chunks:shard0000 1 {"id": {"$minKey": 1} }-->> {"id": {"$maxKey": 1}} on:shard0000 Timestamp (1, 0)

Add tags

#添加分片服务器mongod -f mongodb4.confmongomongos> sh.addShard("192.168.235.205:47019")mongos> sh.status()chunks:    shard0000   4    shard0001   4    shard0002   3#删除分片节点mongos> use adminmongos> db.runCommand({"removeshard":"192.168.235.205:47019"})

MongoDB Shard Cluster