MongoDB3.6 Shard Replication Cluster

Concept

• MONGOs
  The portal of the database cluster request, all requests are coordinated by MONGOs, and no route selector needs to be added to the application, MONGOs is a request distribution center. It is responsible for forwarding the corresponding data request request to the corresponding Shard server. In a production environment there is usually more mongos as the entrance to the request, preventing one of the other MongoDB requests from being hung out of operation. The
• config server
  , as the name implies, configures the servers to store the configuration of all database meta information (routing, sharding). The mongos itself does not have a physical storage Shard server and data routing information, but is cached in memory, and the configuration server actually stores the data. MONGOs the first boot or shutdown reboot will load configuration information from config server, and if configuration server information changes will notify all MONGOs to update their status, so that MONGOs can continue to route accurately. In a production environment, there are typically multiple config server configuration servers because it stores metadata for fragmented routes to prevent data loss!
• Shard
  sharding (sharding) refers to the process of splitting a database and dispersing it across different machines. Spread the data across different machines without the need for powerful servers to store more data and handle larger loads. The basic idea is to cut the set into small pieces, which are scattered across several slices, each of which is responsible for only a portion of the total data, and finally a equalizer to equalize each shard (data migration). The
• replica set
  Chinese translation copy set is actually a shard backup that prevents data loss after Shard hangs. Replication provides redundant backups of data, stores copies of data on multiple servers, improves data availability, and guarantees data security. The
• arbiter (arbiter)
  is a MongoDB instance in a replication set and does not hold data. The quorum node uses minimal resources and does not require hardware devices, cannot deploy arbiter in the same dataset node, can be deployed on other application servers or monitoring servers, or can be deployed in a separate virtual machine. To ensure that there are an odd number of voting members (including primary) in the replication set, the quorum node needs to be added as a vote, otherwise primary will not automatically switch primary when it is not running.

Summing up, the application request MONGOs to manipulate MongoDB additions and deletions, configure the server storage database meta-information, and MONGOs do synchronization, the data is finally deposited on The Shard (Shard), in order to prevent data loss synchronization in the replica set stored a copy, The quorum determines which node to store when the data is stored to the Shard.

Environment preparation

system : cenos7.4
Server : 10.10.16.52/53/54
MongoDB Package : mongodb-linux-x86_64-rhel70-3.6.3.tgz

Planning

mongodb-16-52	mongodb-16-53	mongodb-16-54
mongos/20000	mongos/20000	mongos/20000
Config server/21000	Config server/21000	Config server/21000
Shard Server1 main node (27001)	Shard Server1 Sub-node (27001)	Shard Server1 arbitration (27001)
Shard Server2 arbitration (27002)	Shard Server2 main node (27002)	Shard Server2 Sub-node (27002)
Shard Server3 Sub-node (27003)	Shard Server1 arbitration (27003)	Shard Server3 main node (27003)

Cluster build and install MongoDB

wget https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-rhel70-3.6.3.tgzcd /usr/lcoal/ln -sf mongodb-linux-x86_64-rhel70-3.6.3/ mongodb

Set up conf, MONGOs, config, Shard1, Shard2, shard3 six directories on each machine respectively, because MONGOs does not store data, only need to establish log file directory.

mkdir -p /data/mongodb/confmkdir -p /data/mongodb/mongos/logmkdir -p /data/mongodb/config/datamkdir -p /data/mongodb/config/logmkdir -p /data/mongodb/shard1/datamkdir -p /data/mongodb/shard1/logmkdir -p /data/mongodb/shard2/datamkdir -p /data/mongodb/shard2/logmkdir -p /data/mongodb/shard3/datamkdir -p /data/mongodb/shard3/log

Adding users, configuring environment variables

useradd -M -u 8000 -s /sbin/nologin mongochown -R mongo.mongo /usr/local/mongodbchown -R mongo.mongo /data/mongodb

Config Server Configuration service

mongodb3.4 later requires that the configuration server also create replica sets, or the cluster is not built successfully

Add a configuration file

cat >> /data/mongodb/conf/config.conf << EOF## 配置文件内容pidfilepath = /data/mongodb/config/log/config-srv.piddbpath = /data/mongodb/config/datalogpath = /data/mongodb/config/log/config-srv.loglogappend = true bind_ip = 0.0.0.0port = 21000fork = true #declare this is a config db of a cluster;configsvr = true#副本集名称replSet=configs #设置最大连接数maxConns=20000EOF

Systemd Startup scripts

cat >> /usr/lib/systemd/system/mongo-config.service << EOF[Unit]    Description=mongodb-config   After=network.target remote-fs.target nss-lookup.target    [Service]  Type=forkingUser=mongoGroup=mongoExecStart=/usr/local/mongodb/bin/mongod -f /data/mongodb/conf/config.confExecReload=/bin/kill -s HUP $MAINPID  ExecStop=/usr/local/mongodb/bin/mongod --shutdown -f /data/mongodb/conf/config.conf PrivateTmp=true      [Install]  WantedBy=multi-user.target EOF

Start config server for three servers

systemctl daemon-reload systemctl start mongo-configsystemctl enable mongo-config

Initializing a replica set
Log on to any configuration server, initialize the configuration replica set

mongo --port 21000   //登陆#config变量config = {...    _id : "configs",...     members : [...         {_id : 0, host : "10.10.16.52:21000" },...         {_id : 1, host : "10.10.16.53:21000" },...         {_id : 2, host : "10.10.16.54:21000" }...     ]... }#初始化副本集rs.initiate(config)# 查看集群状态rs.status()

Where, _id : configs should be consistent with the Replicaction.replsetname configured in the configuration file, members in the host three-node ip andport

Configure a shard replica set (three machines) set the first shard replica set

Configuration file

cat >> /data/mongodb/conf/shard1.conf << EOFpidfilepath = /data/mongodb/shard1/log/shard1.piddbpath = /data/mongodb/shard1/datalogpath = /data/mongodb/shard1/log/shard1.loglogappend = truejournal = truequiet = truebind_ip = 0.0.0.0port = 27001fork = true#副本集名称replSet=shard1 #declare this is a shard db of a cluster;shardsvr = true #设置最大连接数maxConns=20000EOF

Systemd Startup scripts

cat >> /usr/lib/systemd/system/mongo-shard1.service << EOF[Unit]    Description=mongodb-shard1  After= mongo-config.target network.target  [Service]  Type=forkingUser=mongoGroup=mongoExecStart=/usr/local/mongodb/bin/mongod -f /data/mongodb/conf/shard1.confExecReload=/bin/kill -s HUP $MAINPID  ExecStop=/usr/local/mongodb/bin/mongod --shutdown -f /data/mongodb/conf/shard1.conf PrivateTmp=true      [Install]  WantedBy=multi-user.target EOF

Shard1 server that starts three servers

systemctl daemon-reload systemctl start mongo-shard1systemctl enable mongo-shard1

Initializing a replica set
Log on to any configuration server, initialize the configuration replica set

mongo --port 27001#使用admin数据库use admin#定义副本集配置，第三个节点的 "arbiterOnly":true 代表其为仲裁节点。不可为当前初始化设置的节点config = {...    _id : "shard1",...     members : [...         {_id : 0, host : "10.10.16.52:27001" },...         {_id : 1, host : "10.10.16.53:27001" },...         {_id : 2, host : "10.10.16.54:27001" , arbiterOnly: true }...     ]... }#初始化副本集配置rs.initiate(config);rs.status()

Set up a second shard replica set

Configuration file

cat >> /data/mongodb/conf/shard2.conf << EOFpidfilepath = /data/mongodb/shard2/log/shard2.piddbpath = /data/mongodb/shard2/datalogpath = /data/mongodb/shard2/log/shard2.loglogappend = truejournal = truequiet = truebind_ip = 0.0.0.0port = 27002fork = true#副本集名称replSet=shard2 #declare this is a shard db of a cluster;shardsvr = true #设置最大连接数maxConns=20000EOF

Systemd Startup scripts

cat >> /usr/lib/systemd/system/mongo-shard2.service << EOF[Unit]    Description=mongodb-shard2  After= mongo-config.target network.target  [Service]  Type=forkingUser=mongoGroup=mongoExecStart=/usr/local/mongodb/bin/mongod -f /data/mongodb/conf/shard2.confExecReload=/bin/kill -s HUP $MAINPID  ExecStop=/usr/local/mongodb/bin/mongod --shutdown -f /data/mongodb/conf/shard2.conf PrivateTmp=true      [Install]  WantedBy=multi-user.target EOF

Shard2 server that starts three servers

systemctl daemon-reload systemctl start mongo-shard2systemctl enable mongo-shard2

Initializing a replica set
Log on to any of the configuration servers, select any server (note: Do not select the server that will be set as the quorum node)

mongo --port 27002#使用admin数据库use admin#定义副本集配置，第三个节点的 "arbiterOnly":true 代表其为仲裁节点。不可为当前初始化设置的节点config = {...    _id : "shard2",...     members : [...         {_id : 0, host : "10.10.16.52:27002", arbiterOnly: true  },...         {_id : 1, host : "10.10.16.53:27002" },...         {_id : 2, host : "10.10.16.54:27002" }...     ]... }#初始化副本集配置rs.initiate(config);rs.status()

Set up a third shard replica set

Configuration file

cat >> /data/mongodb/conf/shard3.conf << EOFpidfilepath = /data/mongodb/shard3/log/shard3.piddbpath = /data/mongodb/shard3/datalogpath = /data/mongodb/shard3/log/shard3.loglogappend = truejournal = truequiet = truebind_ip = 0.0.0.0port = 27003fork = true#副本集名称replSet=shard3 #declare this is a shard db of a cluster;shardsvr = true #设置最大连接数maxConns=20000EOF

Systemd Startup scripts

cat >> /usr/lib/systemd/system/mongo-shard3.service << EOF[Unit]    Description=mongodb-shard3  After= mongo-config.target network.target  [Service]  Type=forkingUser=mongoGroup=mongoExecStart=/usr/local/mongodb/bin/mongod -f /data/mongodb/conf/shard3.confExecReload=/bin/kill -s HUP $MAINPID  ExecStop=/usr/local/mongodb/bin/mongod --shutdown -f /data/mongodb/conf/shard3.conf PrivateTmp=true      [Install]  WantedBy=multi-user.target EOF

Shard3 server that starts three servers

systemctl daemon-reload systemctl start mongo-shard3systemctl enable mongo-shard3

Initializing a replica set
Log on to any of the configuration servers, select any server (note: Do not select the server that will be set as the quorum node)

mongo --port 27003#使用admin数据库use admin#定义副本集配置，第三个节点的 "arbiterOnly":true 代表其为仲裁节点。不可为当前初始化设置的节点config = {...    _id : "shard3",...     members : [...         {_id : 0, host : "10.10.16.52:27003" },...         {_id : 1, host : "10.10.16.53:27003",arbiterOnly: true },...         {_id : 2, host : "10.10.16.54:27003" }...     ]... }#初始化副本集配置rs.initiate(config);rs.status()

Configuring the Routing server MONGOs

Start the configuration server and the Shard server before starting the routing instance: (Three machines)
Configuration file

cat >> /data/mongodb/conf/mongos.conf << EOFpidfilepath = /data/mongodb/mongos/log/mongos.pidlogpath = /data/mongodb/mongos/log/mongos.loglogappend = truebind_ip = 0.0.0.0port = 20000fork = true#监听的配置服务器,只能有1个或者3个 configs为配置服务器的副本集名字configdb = configs/10.10.16.52:21000,10.10.16.53:21000,10.10.16.54:21000 #设置最大连接数maxConns=20000EOF

Systemd Startup scripts

cat >> /usr/lib/systemd/system/mongos.service << EOF[Unit]Description=Mongo Router ServiceAfter=mongo-config.service[Service]Type=forkingUser=mongoGroup=mongoExecStart=/usr/local/mongodb/bin/mongos --config /data/mongodb/conf/mongos.confRestart=on-failure[Install]WantedBy=multi-user.targetEOF

MONGOs server that starts three servers

systemctl daemon-reload systemctl start mongossystemctl enable mongos

Enable Sharding

Currently has a MongoDB configuration server, routing server, each shard server, but the application to connect to the MONGOs routing server does not use the Shard mechanism, also need to set the Shard configuration in the program, let the Shard take effect.
Login to any MONGOs

MONGO--port 20000# uses the admin database use admin# in-line routing server and allocation replica set Sh.addshard ("shard1/ 10.10.16.52:27001,10.10.16.53:27001,10.10.16.54:27001 ") Sh.addshard (" shard2/ 10.10.16.52:27002,10.10.16.53:27002,10.10.16.54:27002 ") Sh.addshard (" shard3/   10.10.16.52:27003,10.10.16.53:27003,10.10.16.54:27003 ") #查看集群状态sh. Status () Sh.status ()---sharding status---  Sharding version: {"_id": 1, "mincompatibleversion": 5, "CurrentVersion": 6, "Clusterid": ObjectId ("5a9a97f70d2d5d358998988d")} shards: {"_id": "Shard1", "host": "Shard1/10.10.16.52:27001,10.10.16 .53:27001 "," state ": 1} {" _id ":" Shard2 "," host ":" shard2/10.10.16.53:27002,10.10.16.54:27002 "," state ":        1} {"_id": "Shard3", "host": "shard3/10.10.16.52:27003,10.10.16.54:27003", "state": 1} Active Mongoses: "3.6.3": 3 autosplit:currently enabled:yes balancer:currently Enabled:yes currently run Ning:no Failed balancer rounds in last 5 attempts: 0 Migration Results for the last Hours:no recent migrations databases: {"_id": "C Onfig "," primary ":" config "," partitioned ": true}

Test

The current configuration service, routing service, Shard service, replica set service are all concatenated, but our goal is to insert the data and the data can be automatically fragmented. Connected on MONGOs, ready to have the specified database, specified collection shards in effect.

#指定testdb分片生效db.runCommand( { enablesharding :"testdb"});#指定数据库里需要分片的集合和片键db.runCommand( { shardcollection : "testdb.table1",key : {id: 1} } )

We set the Table1 table for TestDB to be fragmented, automatically sharding to shard1 according to the ID, shard2,shard3 above. This is set because not all MONGODB databases and tables require sharding!
Test Shard Configuration Results

mongo 127.0.0.1 --port 20000#使用testdbuse  testdb;#插入测试数据for(i=1;i<=1000;i++){db.table1.insert({"id":i,"name":"lovego"})};# 总条数db.table1.aggregate([{$group : {_id : "$name", totle : {$sum : 1}}}]){ "_id" : "lovego", "totle" : 100000 }#查看分片情况如下，db.table1.stats();

Maintenance

Boot order

systemctl start mongo-configsystemctl start mongo-shard1systemctl start mongo-shard2systemctl start mongo-shard3systemctl start mongos

Close Order

systemctl stop mongossystemctl stop mongo-shard1systemctl stop mongo-shard2systemctl stop mongo-shard3systemctl stop mongo-config

Reference

https://cloud.tencent.com/developer/article/1034843
Http://www.ityouknow.com/mongodb/2017/08/05/mongodb-cluster-setup.html
Http://www.cnblogs.com/clsn/p/8214345.html

MongoDB3.6 Shard Replication Cluster