MongoDB Shard Technology

MongoDB Shard

Sharding

In MongoDB there is another cluster, that is, the Shard technology, can meet the requirements of a large number of MONGODB data volume growth.

When MongoDB stores massive amounts of data, a machine may not be enough to store data, or it may not be sufficient to provide acceptable read and write throughput. At this point, we can divide the data on multiple machines so that the database system can store and process more data.

Why use Shards

Copy all write operations to the master node

Deferred sensitive data is queried on the master node

A single replica set is limited to 12 nodes

When the volume of requests is large, there is insufficient memory.

Insufficient Local Disk

Vertical expansion is expensive

MongoDB Shard

Shows the use of shard cluster structure distribution in MongoDB:

There are three main components as described below:

Shard:

Used to store actual blocks of data, a Shard server role in the actual production environment can be assumed by several machines, one replica set, to prevent a single point of failure of the host

Config Server:

The Mongod instance, which stores the entire clustermetadata, including chunk information.

Query Routers:

Front-end routing, which the client accesses, and makes the entire cluster look like a single database, the front-end application can be used transparently.

Shard Instance

The Shard structure port is distributed as follows:

Shard Server 1:27,020

Shard Server 2:27,021

Shard Server 3:27,022

Shard Server 4:27,023

Config server:27100

Route process:40000

Step one: Start Shard Server

[Email protected]/]# mkdir-p/www/mongodb/shard/s0

[Email protected]/]# mkdir-p/www/mongodb/shard/s1

[Email protected]/]# mkdir-p/www/mongodb/shard/s2

[Email protected]/]# mkdir-p/www/mongodb/shard/s3

[Email protected]/]# mkdir-p/www/mongodb/shard/log

[Email protected]/]# /usr/local/mongodb/bin/mongod--port 27020--dbpath=/www/mongodb/shard/s0--logpath=/www/ Mongodb/shard/log/s0.log--logappend--fork

....

[Email protected]/]# /usr/local/mongodb/bin/mongod--port 27023--dbpath=/www/mongodb/shard/s3--logpath=/www/ Mongodb/shard/log/s3.log--logappend--fork

Step two: Start config Server

[Email protected]/]# mkdir-p/www/mongodb/shard/config

[Email protected]/]# /usr/local/mongodb/bin/mongod--port 27100--dbpath=/www/mongodb/shard/config--logpath=/ Www/mongodb/shard/log/config.log--logappend--fork

Note: Here we can boot up like a normal mongodb service, without adding-SHARDSVR and Configsvr parameters. Because these two parameters are used to change the boot port, we specify the port on our own.

Step three: Start the route Process

/Usr/local/mongodb/bin/mongos--port 40000--configdb localhost:27100--fork--logpath=/www/mongodb/shard/log/ Route.log--chunksize

MONGOs start parameter, chunksize this item is used to specify the size of the chunk, in megabytes, the default size is 200MB.

Step Four: Configure Sharding

Next, we use the MongoDB shell to log in to MONGOs and add the Shard node

[Email protected] shard]# /usr/local/mongodb/bin/mongo admin--port 40000

MongoDB Shell version:2.0.7

Connecting To:127.0.0.1:40000/admin

Mongos> Db.runcommand ({addshard: "localhost:27020"})

{"shardadded": "shard0000", "OK": 1}

......

Mongos> Db.runcommand ({addshard: "localhost:27029"})

{"shardadded": "shard0009", "OK": 1}

Mongos> Db.runcommand ({enablesharding: "Test"}) #设置分片存储的数据库

{"OK": 1}

Mongos> Db.runcommand ({shardcollection: "Test.log", key: {id:1,time:1}})

{"collectionsharded": "Test.log", "OK": 1}

Step five: In the program code without too much change, directly follow the connection to the normal MONGO database, connect the database to the interface 40000

Add:

1. Create a sharding replica set Rs0

# Mkdir/data/log

# MKDIR/DATA/DB1

# nohup Mongod--port 27020--dbpath=/data/db1--logpath=/data/log/rs0-1.log--logappend--fork--shardsvr--replSet=rs0 &

# MKDIR/DATA/DB2

# nohup Mongod--port 27021--dbpath=/data/db2--logpath=/data/log/rs0-2.log--logappend--fork--shardsvr--replSet=rs0 &

1.1 Replica set RS0 configuration

# MONGO localhost:27020 > Rs.initiate ({_id: ' Rs0 ', Members: [{_id:0, Host: ' localhost:27020 '}, {_id:1, host: ' Localho st:27021 '}]}) > Rs.ismaster () #查看主从关系

2. Create a sharding replica set rs1

# MKDIR/DATA/DB3

# nohup Mongod--port 27030--dbpath=/data/db3--logpath=/data/log/rs1-1.log--logappend--fork--shardsvr--replSet=rs1 &

# MKDIR/DATA/DB4

# nohup Mongod--port 27031--dbpath=/data/db4--logpath=/data/log/rs1-2.log--logappend--fork--shardsvr--replSet=rs1 &

2.1 Replica Set RS1 configuration

# MONGO localhost:27030

> rs.initiate ({_id: ' rs1 ', Members: [{_id:0, Host: ' localhost:27030 '}, {_id:1, Host: ' localhost:27031 '}]})

> Rs.ismaster () #查看主从关系

3. Create config replica set conf

# MKDIR/DATA/CONF1

# nohup Mongod--port 27100--dbpath=/data/conf1--logpath=/data/log/conf-1.log--logappend--fork--configsvr--replSet =conf &

# MKDIR/DATA/CONF2

# nohup Mongod--port 27101--dbpath=/data/conf2--logpath=/data/log/conf-2.log--logappend--fork--configsvr--replSet =conf &

3.1 Replica Set conf configuration

# MONGO LOCALHOST:27100

> rs.initiate ({_id: ' conf ', Members: [{_id:0, Host: ' Localhost:27100 '}, {_id:1, Host: ' Localhost:27101 '}]})

> Rs.ismaster () #查看主从关系

4. Create route

# nohup MONGOs--port 40000--configdb conf/localhost:27100,localhost:27101--fork--logpath=/data/log/route.log-- Logappend &

4.1 Setting up Shards

# MONGO localhost:40000

> Use admin

> Db.runcommand ({addshard: ' rs0/localhost:27020,localhost:27021 '})

> Db.runcommand ({addshard: ' rs1/localhost:27030,localhost:27031 '})

> Db.runcommand ({enablesharding: ' Test '})

> Db.runcommand ({shardcollection: ' Test.user ', key: {name:1}})

MongoDB Shard Technology