The deployment of a MongoDB shard cluster (carefully described, detailed and understandable)!!

Concept:

MongoDB shards are a way to store data using multiple servers to support huge data storage and data storage

Advantage:

1, reduce the number of requests per shard Ouch processing, the cluster can improve their storage capacity and throughput

2. Reduce the data stored per shard

Three main components:

shard：分片服务器，用于存储实际的数据块，由多台服务器组成一个复制集承担，防止主机单点故障config server：配置服务器，存储整个分片群集的配置信息，包括块信息routers：前端路由，客户端由此进入，让整个群集看上去像单一数据库

How to deploy a MongoDB shard cluster!!!

1. Install MongoDB3.2 version package and Openssl-devel package

[[email protected] ~]# yum install openssl-devel-y[[email protected] ~]# mkdir/abc[[email protected] ~ ]# mount.cifs//192.168.200.1/orc/abcpassword for [EMAIL PROTECTED]//192.168.200.1/ORC: [[email protected] ] ~]# cd/abc[[email protected] abc]# lsmongodb-linux-x86_64-3.2.1.tgz//mongodb version 3.2 [[email protected] ABC ]# tar zxvf mongodb-linux-x86_64-3.2.1.tgz-c/opt[[email protected] abc]# cd/opt[[email protected] opt]#                 lsmongodb-linux-x86_64-3.2.1 rh[[email protected] opt]# mv MONGODB-LINUX-X86_64-3.2.1//usr/local/mongodb [[email protected] opt]# cd/usr/local/bin/[[email protected] bin]# ln-s/usr/local/mongodb/bin/mongo/us R/bin/mongo//Establish soft connection [[email protected] bin]# ln-s/usr/local/mongodb/bin/mongod/usr/bin/mongod[[email  Protected] bin]# mkdir-p/data/mongodb/mongodb{1,2,3,4}[[email protected] bin]# mkdir/data/mongodb/logs[[email  protected] bin]# Touch/data/mongodb/logs/mongodB{1,2,3,4}.log[[email protected] bin]# chmod-r 777/data/mongodb/logs/*.log[[email protected] bin]# ulimit -N 25000//number of open files [[email protected] bin]# ulimit-u 25000//process concurrency

2. Deploying the Configuration Server

Edit mongodb1.conf configuration file, Port 37017, set Configsvr=true, start configuration server

[[email protected] bin]# vim mongodb1.conport=37017dbpath=/data/mongodb/mongodb1logpath=/data/mongodb/logs/mongodb1.loglogappend=truefork=truemaxConns=5000storageEngine=mmapv1configsvr=true                //开启配置服务器

3. A node has insufficient memory to allocate memory from other nodes

[[email protected] bin]# sysctl -w vm.zone_reclaim_mode=0vm.zone_reclaim_mode = 0[[email protected] bin]# echo never > /sys/kernel/mm/transparent_hugepage/enabled[[email protected] bin]# echo never > /sys/kernel/mm/transparent_hugepage/defrag[[email protected] bin]# mongod -f mongodb1.conf about to fork child process, waiting until server is ready for connections.forked process: 6895child process started successfully, parent exiting

4. Deploying a shard Server

Edit the mongodb2.conf configuration file, Port 47017, set shardsvr=true, edit mongodb3.conf profile, Port 47018, start two shard server

[[email protected] bin]# cp-p mongodb1.conf mongodb2.conf [[email protected] bin]# vim mongodb2.conf port= 47017//Modify Port Dbpath=/data/mongodb/mongodb2logpath=/data/mongodb/logs/mongodb2.loglogappend=truefork=truem Axconns=5000storageengine=mmapv1shardsvr=true//Turn on the Shard server [[email protected] bin]# cp-p mongodb2.conf mo Ngodb3.conf[[email protected] bin]# vim mongodb3.conf port=47018dbpath=/data/mongodb/mongodb3logpath=/data/ mongodb/logs/mongodb3.loglogappend=truefork=truemaxconns=5000storageengine=mmapv1shardsvr=true[[email   Protected] bin]# mongod-f mongodb2.conf about to fork child process, waiting until server was ready for connections.forked Process:7080child process started successfully, parent exiting[[email protected] bin]# mongod-f mongodb3.conf Abou T to fork child process, waiting until server was ready for connections.forked Process:7107child process started Successfu Lly, parent exiting[[email protected] bin]# netstAT-ANTP |  grep mongodtcp 0 0 0.0.0.0:37017 0.0.0.0:* LISTEN God TCP 0 0 0.0.0.0:47017 0.0.0.0:* LISTEN God TCP 0 0 0.0.0.0:47018 0.0.0   .0:* LISTEN God

5. Start the routing server

The./mongos--help command can view routing-related parameter information. Chunksize is the block size, default is 200M, here the value is set to 1

[[email protected] bin]# ./mongos --port 27017 --fork --logpath=/usr/local/mongodb/bin/route.log --configdb 192.168.200.142:37017 --chunkSize 1

6. Enable the Shard server

mongos> sh.addShard("192.168.200.142:47017"){ "shardAdded" : "shard0000", "ok" : 1 }mongos> sh.addShard("192.168.200.142:47018"){ "shardAdded" : "shard0001", "ok" : 1 }mongos> sh.status()--- Sharding Status ---   sharding version: {    "_id" : 1,    "minCompatibleVersion" : 5,    "currentVersion" : 6,    "clusterId" : ObjectId("5b4f68599dcf397cc4e7c598")}  shards:                 //分片服务器开启    {  "_id" : "shard0000",  "host" : "192.168.200.142:47017" }    {  "_id" : "shard0001",  "host" : "192.168.200.142:47018" }  active mongoses:    "3.2.1" : 1  balancer:    Currently enabled:  yes    Currently running:  no    Failed balancer rounds in last 5 attempts:  0    Migration Results for the last 24 hours:         No recent migratio

7. Shard function

[[email protected] bin]# mongomongos> use schoolswitched to DB Schoolmongos> for (var i=1;i<=10000;i++) Db.users.insert ({"id": I, "name": "Jack" +i}) Writeresult ({"ninserted": 1}) mongos> show Dbsconfig 0.031GBscho  OL 0.078GB mongos> use schoolswitched to DB schoolmongos> show collectionssystem.indexesusersmongos>  Db.users.find (). Limit (10)//view first ten lines of content {"_id": ObjectId ("5b4f6a24ef33d588c0dbc4ae"), "id": 1, "name": "Jack1"} {"_id": ObjectId ("5b4f6a24ef33d588c0dbc4af"), "id": 2, "name": "Jack2"} {"_id": ObjectId ("5b4f6a24ef33d588c0dbc4b0"), "id": 3, "name": "Jack3"} {"_id": ObjectId ("5b4f6a24ef33d588c0dbc4b1"), "id": 4, "name": "Jack4"} {"_id": ObjectId ("5b4f6 A24ef33d588c0dbc4b2 ")," id ": 5," name ":" Jack5 "} {" _id ": ObjectId (" 5b4f6a24ef33d588c0dbc4b3 ")," id ": 6," name ":" Jac K6 "} {" _id ": ObjectId (" 5b4f6a24ef33d588c0dbc4b4 ")," id ": 7," name ":" Jack7 "} {" _id ": ObjectId (" 5b4f6a24ef33d588c0dbc 4b5 ")," id ": 8," name ":" JacK8 "} {" _id ": ObjectId (" 5b4f6a24ef33d588c0dbc4b6 ")," id ": 9," name ":" Jack9 "} {" _id ": ObjectId (" 5b4f6a24ef33d588c0dbc 4b7 ")," id ": Ten," Name ":" Jack10 "}mongos> sh.status ()//View database shard information---sharding Status---sharding vers Ion: {"_id": 1, "mincompatibleversion": 5, "CurrentVersion": 6, "Clusterid": ObjectId ("5b4f68599dcf397cc4 e7c598 ")} shards: {" _id ":" shard0000 "," host ":" 192.168.200.142:47017 "} {" _id ":" shard0001 "," host ":" 19    2.168.200.142:47018 "} Active Mongoses:" 3.2.1 ": 1 balancer:currently Enabled:yes currently Running:no Failed balancer rounds in last 5 attempts:0 migration Results for the last Hours:no recent migrations D  Atabases: {"_id": "School", "PRIMARY": "shard0000", "partitioned": false}mongos> sh.enablesharding ("school") Enable Database Shard {"OK": 1}mongos> sh.status ()---sharding status---sharding version: {"_id": 1, "Mincompatiblev Ersion ": 5," currentVersion ": 6," Clusterid ": ObjectId (" 5b4f68599dcf397cc4e7c598 ")} shards: {" _id ":" shard0000 "," host ":" 192.16 8.200.142:47017 "} {" _id ":" shard0001 "," host ":" 192.168.200.142:47018 "} Active Mongoses:" 3.2.1 ": 1 Balanc er:currently Enabled:yes currently Running:no Failed balancer rounds in last 5 attempts:0 migration Res Ults for the last Hours:no recent migrations databases: {"_id": "School", "PRIMARY": "shard0000", "p Artitioned ": True}mongos> db.users.createIndex ({" id ": 1})//CREATE INDEX {" Raw "for the users table: {" 192.168.200.142:47017 "  : {"createdcollectionautomatically": false, "Numindexesbefore": 1, "Numindexesafter": 2, "OK": 1}, "OK": 1}mongos> sh.shardcollection ("School.users", {"id": 1})//Table Shard {"Collectio    Nsharded ":" School.users "," OK ": 1}mongos> sh.status ()---sharding status---sharding version: {" _id ": 1, "Mincompatibleversion": 5, "CurrentVersion": 6, "Clusterid": ObjectId ("5b4f68599dcf397cc4e7c598")} shards: {"_id": "shard0000", "Host": "192.168.200.142:47017"} {"_id": "shard0001", "host": "192.168.200.142:47018"} Active Mongoses: "3.    2.1 ": 1 balancer:currently enabled:yes currently running:no Failed balancer rounds in the last 5 attempts:0 Migration Results for the last Hours:no recent migrations databases: {"_id": "School", "PRIMARY":            "shard0000", "Partitioned": true} school.users Shard key: {"id": 1} unique:false  Balancing:true chunks:shard0000 3 {"id": {"$minKey": 1}}-->> { "id": 4682} on:shard0000 Timestamp (1, 0) {"id": 4682}-->> {"id": 9364} on:shard0000 timest  AMP (1, 1) {"id": 9364}-->> {"id": {"$maxKey": 1}} on:shard0000 Timestamp (1, 2)//Shard succeeded

The deployment of a MongoDB shard cluster (carefully described, detailed and understandable)!!