CentOS7-based MongoDB shard cluster

Introduction to MongoDB Shard cluster based on CentOS7

High data volume and throughput of the database application of the performance of a single machine will be more pressure, large query volume will be a single CPU exhausted, large amount of data on the single-machine storage pressure, and eventually will exhaust the system's memory and transfer the pressure to disk IO.
MongoDB shards are a way to store data using multiple servers to support huge data stores and manipulate data. Shard Technology can meet the demand of large growth of MONGODB data volume. , when a MongoDB server is not enough to store massive amounts of data or is insufficient to provide acceptable read and write throughput, it is possible to divide the data on multiple servers so that the database system can store and process more data.

Advantages of Sharding

• Using sharding reduces the number of requests per shard that needs to be processed, and by scaling horizontally, the cluster can increase its storage capacity and throughput.

• Using shards reduces the data stored for each shard

Composition of a shard cluster

1. Shard: Shard server for storing actual blocks of data
2. Configserver: Configure the server to store configuration information for the entire shard cluster
3. Routers: Routing Server

Experiment of Shard Cluster management

Experimental configuration diagram above

Let's start with the experiment, where I open multiple instances on a single machine instead of multiple servers.

Install the mongodb3.2 version (manually compiled installation)

• Install the Software Environment pack

Yum-y Install Openssl-devel

• Unzip the MongoDB package

Tar zxvf mongodb-linux-x86_64-3.2.1.tgz-c/opt

• Move package to System-aware directory

MV MONGODB-LINUX-X86_64-3.2.1//usr/local/mongodb

• Create a MongoDB data store (/DATA/MONGODB1, 2, 3, 4,) and log storage directory (/data/logs)

Mkdir-p/data/mongodb/mongodb{1,2,3,4}

Mkdir/data/mongodb/logs

Touch/data/mongodb/logs/mongodb{1,2,3,4}.log

Chmod-r 777/data/mongodb/logs/*.log

• Set values for Ulimit-n and Ulimit-u

When MongoDB is in a state of frequent access, if the shell startup process consumes too much resource settings, an error will result in the inability to connect to the MongoDB instance, so it is necessary to set ulimit-n and Ulimit-u values greater than 20000
Ulimit-n 25000
Ulimit-u 25000

• To create a configuration file for a configuration server

cd/usr/local/mongodb/bin/
Vim mongodb1.conf

port=37017           #端口dbpath=/data/mongodb/mongodb1    #数据存储位置logpath=/data/mongodb/logs/mongodb1.log    #日志存储位置logappend=true   #错误日志采用追加模式，配置这个选项后mongodb的日志会追加到现有的日志文件，而不是从新创建一个新文件     fork=true      #后台运行maxConns=5000    #最大同时连接数storageEngine=mmapv1    #指定存储引擎为内存映射文件configsvr=true    #指定配置服务器的模式

• Setting Kernel parameters
  When a node is low on memory, the system allocates memory from other nodes

Sysctl-w vm.zone_reclaim_mode=0 #永久设置
echo Never >/sys/kernel/mm/transparent_hugepage/enabled
echo Never >/sys/kernel/mm/transparent_hugepage/defrag

• Create soft links for easy management

Ln-s/usr/local/mongodb/bin/mongo/usr/bin/mongo
Ln-s/usr/local/mongodb/bin/mongod/usr/bin/mongod

• Turn on the first instance (configuration server)

• To configure an instance of a shard server
  Copy build configuration file

  Cp-p mongodb1.conf mongodb2.conf #复制配置服务器的配置文件生成实例2的配置文件
  Cp-p mongodb1.conf mongodb3.conf #复制配置服务器的配置文件生成实例2的配置文件

• Modify the configuration file for instance 2, 3

Vim mongodb2.conf

port=47017               #修改端口号 dbpath=/data/mongodb/mongodb2       #修改数据存放目录logpath=/data/mongodb/logs/mongodb2.log      #修改日志存放目录logappend=true  fork=truemaxConns=5000storageEngine=mmapv1shardsvr=true          #指定分片服务器的模式

Vim mongodb3.conf

port=47018dbpath=/data/mongodb/mongodb3logpath=/data/mongodb/logs/mongodb3.loglogappend=truefork=truemaxConns=5000storageEngine=mmapv1shardsvr=true

• Open an instance of two shard servers

Mongod-f mongodb2.conf
Mongod-f mongodb3.conf

• Start the routing server

./mongos--port 27017--fork--logpath=/usr/local/mongodb/bin/route.log--configdb 192.168.234.177:37017--chunkSize 1

Here about the MONGOs command do not know how to use, you can view the Help information

• Enable a shard server

Mongo

• Adding a shard server

Sh.addshard ("192.168.234.177:47017")
Sh.addshard ("192.168.234.177:47018")

View again after adding a shard server

• Test Shard Functionality

  Mongos> Show DBS
  Config 0.031GB
  mongos> Use KGC #进入并创建一个use的集合
  Switched to DB KGC
  Mongos> Db.users.insert ({"id": 1, "name": "Zhangsan"}) #添加一条数据
  Writeresult ({"ninserted": 1})
  Mongos> for (var i=2;i<=20000;i++) Db.users.insert ({"id": I, "name": "Zyc" +i}) #使用for循环添加20000条数据
  Writeresult ({"ninserted": 1})
  Mongos> Show DBS #查看表空间就会有一个kgc
  Config 0.031GB
  KGC 0.078GB
  Mongos> Show Tables #查看表信息也可以看到users表
  System.indexes
  Users

• Turn on the Shard feature

Sh.enablesharding ("KGC")

• To turn on Sharding on a collection in a database

Db.users.createIndex ({"id": 1}) #对users表创建索引
Sh.shardcollection ("Kgc.users", {"id": 1}) #表分片
Sh.status () #此时再次查看, it can already be shard processed.

The introduction of the MongoDB Shard Cluster Management is complete, if you feel that it is helpful to remember to praise the little brother Oh!!!

CentOS7-based MongoDB shard cluster