MongoDB4.0 Building a distributed shard cluster

MongoDB Shard Brief

• High data volume and throughput of the database application will be more pressure on the performance of the single machine, large query volume will be a single CPU exhausted, large amount of data on the single-machine storage pressure, and eventually will exhaust the system's memory and transfer the pressure to disk IO.
• MongoDB shards are a way to store data using multiple servers to support huge data stores and manipulate data. Shard Technology can meet the demand of large amount of MONGODB data, when a MongoDB server is not enough to store massive data or not enough to provide acceptable read and write throughput, we can divide the data on multiple servers, so that the database system can store and process more data.

MongoDB Shard Advantage

• Sharding provides a way to cope with high throughput and large data volumes:

1. Using sharding reduces the number of requests that each shard needs to process, so the cluster can increase its storage capacity by horizontally scaling. For example, when inserting a piece of data, the app only needs to access the shards that store the data.
2. Using sharding reduces the data stored in each Shard village.

The advantage of sharding is to provide a similar linear growth architecture, improve data availability, and improve the performance of large database query servers. Sharding technology is used when MongoDB single-point database server storage becomes a bottleneck, the performance of a single-point database server becomes a bottleneck, or a large application needs to be deployed to take advantage of memory.

The composition of a MongoDB shard cluster

The MongoDB shard cluster consists of the following three components:

• Shard: A shard server for storing actual blocks of data, a Shard server role in a real-world production environment can be made up of several servers, one Peplica Set, to prevent a single point of failure of the host.
• Config server: Configures servers that store configuration information for the entire shard cluster, including chunk information.
• Routers: Front-end routing, where the client is connected and makes the entire cluster look like a single database, and front-end applications can be used transparently.

System environment

• System: CentOS 7.4 x86_64
• Software version: 4.0
• Shutting down firewalls and SELinux

IP Address	routing Server (routers)	Configuring the Server (config-server)	Shard1	Shard2	Shard3
192.168.125.119	27017	27018	27001	27002	27003
192.168.125.120	27017	27018	27001	27002	27003
192.168.125.121	27017	27018	27001	27002	27003

Deploying a shard Cluster

　

Three physical server installation and configuration

2. Download Unzip MongoDB

   wget https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-4.0.0.tgztar zxvf mongodb-linux-x86_64-4.0.0.tgz -C /optmv /opt/mongodb-linux-x86_64-4.0.0/ /usr/local/mongodb

4. Create a routing, configuration, data storage directory for the Shard server, and log management
   The routing server does not store data, so you do not need to create a data store directory, and you need to give permissions to log file creation to completion.

   mkdir -p /data/mongodb/configmkdir -p /data/mongodb/shard{1,2,3}mkdir -p /data/mongodb/logstouch /data/mongodb/logs/shard{1,2,3}.logtouch /data/mongodb/logs/mongos.logtouch /data/mongodb/logs/config.logchmod 777 /data/mongodb/logs/*.log

6. Create an administrative user, modify directory permissions

   useradd -M -s /sbin/nologin mongochown -R mongo:mongo /usr/local/mongodbchown -R mongo:mongo /data/mongodb

8. Add environment variables for ease of use

   echo ‘export MONGODB_HOME=/usr/local/mongodb‘ >> /etc/profileecho ‘export PATH=$PATH:$MONGODB_HOME/bin‘ >> /etc/profilesource /etc/profile

10. System parameter Optimization

    ulimit -n 25000    //可以打开的最大文件数量ulimit -u 25000    //用户最大可用的进程数sysctl -w vm.zone_reclaim_mode=0  //内存不足时，从其他节点分配内存# 从CentOS7开始，MongoDB会建议关闭系统的THP特性，否则可能会导致性能下降echo never > /sys/kernel/mm/transparent_hugepage/enabledecho never > /sys/kernel/mm/transparent_hugepage/defrag  //*注意*这些优化都是临时的，重启失效

Deployment Configuration server (three physical server configuration steps are the same)

1. Write the configuration file, we can use the SCP command to send the configuration file to the other two physical servers

   # vim config.confdbpath=/data/mongodb/config  //数据文件存放位置logpath=/data/logs/config.log    //日志文件port=27018    //端口号logappend=truefork=truemaxConns=5000  storageEngine=mmapv1replSet=configs  //复制集名称configsvr=true    //设置参数为true# mongod -f config.conf  //启动config实例scp /usr/local/mongodb/bin/config.conf [email protected]:/usr/local/mongodb/binscp /usr/local/mongodb/bin/config.conf [email protected]:/usr/local/mongodb/bin

2. Configure a replication set (on any physical machine)

   mongo --port 27018config={_id:"configs",members:[{_id:0,host:"192.168.125.119:27018"},{_id:1,host:"192.168.125.120:27018"},{_id:2,host:"192.168.125.121:27018"}]}    //创建复制集rs.initiate(config)                //初始化复制集

Deploying a shard Server

• Edit shard{1,2,3}.conf configuration file, Port 27001,27002,27003, set Shardsvr=true, start Shard server

  # vim shard1.confdbpath=/data/mongodb/shard1logpath=/data/logs/shard1.logport=27001logappend=truefork=truemaxConns=5000  storageEngine=mmapv1shardsvr=true# mongod -f shard1.conf# 与另外两台配置实例配置文件相同，仅端口、数据文件存放及日志文件要改，只需配置完成后启动即可

• Configure Shard1 as a replica set (it is important to note that the replication assembly error is created on the server that is pre-set as the quorum node.) )

  mongo --port 27001use adminconfig={_id:"shard1",members:[{_id:0,host:"192.168.125.119:27001"},{_id:1,host:"192.168.125.120:27001"},{_id:2,host:"192.168.125.121:27001"}]}    //创建复制集rs.initiate(config)                //初始化复制集

• The remaining two shard server Shard2, shard3 settings are the same, note port and quorum node issues can be deployed routing server
• Create a configuration file to send the configuration file to another physical server. Note that the routing server does not need to store the data directory

  # vim mongos.conflogpath=/data/mongodb/logs/mongos.loglogappend = trueport = 27017fork = trueconfigdb = configs/192.168.125.119:27018,192.168.125.120:27018,192.168.125.121:27018maxConns=20000

• Launch MONGOs Instance

  mongs -f /usr/local/mongodb/bin/mongos.conf# 注意，这边启动mongos实例用的是mongos命令

Start the Shard feature

mongo  //默认进入27017端口mongos> use adminmongos> sh.addShard("shard1/192.168.125.119:27001,192.168.125.120:27001,172.16.10.29:27001")mongos> sh.addShard("shard2/192.168.125.119:27002,192.168.125.120:27002,172.16.10.29:27002")mongos> sh.status()           //查看群集状态# 此处为添加两台分片服务器，后续添加的也会均匀分配分片数据

Implementing sharding Capabilities

• Set the Shard chunk size

  mongos> use configmongos> db.settings.save({"_id":"chunksize","value":1})# 设置块大小为1M是方便实验，不然需要插入海量数据

• Analog Write Data

  mongos> use schoolmongos> show collectionsmongos> for(i=1;i<=50000;i++){db.user.insert({"id":i,"name":"jack"+i})}# 在school库的user表中循环写入五万条数据

• Starting a Database Shard

  mongos>sh.enableSharding("school")# 我们可以自定义需要分片的库或表

• Create an index for the user collection in the school library and then shard the table

  mongos> db.user.createIndex({"id":1})# 以"id"作为索引mongos> sh.shardCollection("school.user",{"id":1})# 根据"id"对user表进行分片mongos> sh.status()# 查看分片情况mongos> sh.help()# 查看分片相关的命令

MongoDB4.0 Building a distributed shard cluster