MongoDB Finishing Note のsharding Shard

It is a kind of database cluster system that expands the massive data horizontally, the data table is stored on each node of sharding, and the user can easily construct a distributed MongoDB cluster by simple configuration.
The data chunking of MongoDB is called Chunk. Each chunk is a contiguous data record in the Collection, usually with a maximum size of 200MB, and a new chunk is generated beyond that. To build a MongoDB sharding Cluster, you need three roles:
Shard Server
That is, the shards that store the actual data, each shard can be an Mongod instance, or a set of replica set that consists of Mongod instances. In order to implement each shard internal Auto-failover,mongodb the official recommendation for each shard is a set of replica set.
Config Server
In order to store a particular collection in multiple Shard, you need to specify a shard key for the collection, for example {age:1}, and Shard key determines which chunk the record belongs to. Config Servers is used to store the configuration information for all shard nodes, the Shard key range for each chunk, chunk distribution in each Shard, and collection configuration information for all DB and sharding in the cluster.
Route Process
This is a front-end route that the client accesses, then asks Config Servers which shard to query or save the record, and then connect the corresponding shard to operate, and finally return the results to the client. The client simply sends the query or update request that was originally sent to mongod to routing Process without worrying about which shard the record is stored on.
Below we build a simple sharding Cluster on the same physical machine:

The architecture diagram is as follows:

Shard Server 1:20,000
Shard Server 2:20,001
Config server:30000
Route process:40000

(1) Start three services

Start Shard Server

Mkdir-p/data/shard/s0---p/data/shard/-p/data/shard/log-- Create log directory /apps/mongo/bin/ Mongod--shardsvr--port 20000--dbpath/data/shard/s0--fork--logpath/data/shard/log/s0.log-- Directoryperdb-- start shard Server instance 1/apps/mongo/bin/mongod--shardsvr--port 20001--dbpath/data/shard/s1-- Fork--logpath/data/shard/log/s1.log--directoryperdb--Start Shard Server Instance 2

View Code

Start Config Server

Mkdir-p/data/shard/config-- Create Data Catalog /apps/mongo/bin/mongod--configsvr--port 30000--dbpath/data/shard/ Config--fork--logpath/data/shard/log/config.log--directoryperdb--start config Server instance

Start Route Process

/apps/mongo/bin/mongos--port 40000--configdb localhost:30000--fork--logpath/data/shard/log/route.log-- ChunkSize 1--Start a route Server instance

MONGOs startup parameters, ChunkSize This is used to specify the size of the chunk, the unit is MB, the default size is 200MB, in order to facilitate testing sharding effect, we specify ChunkSize as 1MB.
(2) configuration sharding

Next, we use the MongoDB Shell to log in to MONGOs and add the Shard node

 [[email protected] ~]#/apps/mongo/bin/mongo admin--port 40000-- This operation requires connection to the Admin Library MongoDB Shell version:  1.8.1connecting to:  127.0.0.1:40000 /admin  > Db.runcommand ({addshard: "localhost:20000"})-- add Shard server{ "shardadded": "shard0000", "OK": 1 > Db.runcommand ({ Addshard: "localhost:20001"  "shardadded": "shard0001", "OK": 1}  > Db.runcommand ({enablesharding: "Test"})-- set the Shard store's database { "OK": 1< Span style= "color: #000000;" >}  > Db.runcommand ({shardcollection: "Test.users", key: {_id:1}})-- Set the collection name of the Shard, and you must specify Shard Key, the system automatically creates the index { "collectionsharded": "Test.users", "OK": 1; 

View Code

(3) Verify that sharding is working properly
We have partitioned the Test.users table, so let's insert some data to see the results.

>Use testswitched to DB test> for(var i = 1; I <= 500000; i++) Db.users.insert ({age:i, Name: "Wangwenlong", Addr: "Beijing", Country:"China"})>db.users.stats () {"Sharded":true, --Description This table has been Shard"NS": "Test.users","Count": 500000,"Size": 48000000,"Avgobjsize": 96,"Storagesize": 66655232,"Nindexes": 1,"Nchunks": 43,"Shards" : {"shard0000": {--approximately 24.5M data on this shard instance"NS": "Test.users","Count": 254889,"Size": 24469344,"Avgobjsize": 96,"Storagesize": 33327616,"Numextents": 8,"Nindexes": 1,"Lastextentsize": 12079360,"Paddingfactor": 1,"Flags": 1,"Totalindexsize": 11468800,"Indexsizes" : {"_id_": 11468800},"OK": 1},"shard0001": {--approximately 23.5M data on this shard instance"NS": "Test.users","Count": 245111,"Size": 23530656,"Avgobjsize": 96,"Storagesize": 33327616,"Numextents": 8,"Nindexes": 1,"Lastextentsize": 12079360,"Paddingfactor": 1,"Flags": 1,"Totalindexsize": 10649600,"Indexsizes" : {"_id_": 10649600},"OK": 1}},"OK": 1}>

View Code

Let's take a look at the physical files on the disk.

[Email protected] bin]# ll/data/shard/s0/test--262420-rw-------1 root root 16777216 06-03 15:21 test.0-rw---- ---1 root root 33554432 06-03 15:21 test.1-rw-------1 root root 67108864 06-03 15:22 test.2-rw-------1 root root 134217 728 06-03 15:24 test.3-rw-------1 root root 16777216 06-03 15:21/data/shard/s1/test--262420- RW-------1 root root 16777216 06-03 15:21 test.0-rw-------1 root root 33554432 06-03 15:21 test.1-rw-------1 root root 67108864 06-03 15:22 test.2-rw-------1 root root 134217728 06-03 15:23 test.3-rw-------1 root root 16777216 06-03 15:21< c3> Test.ns[[email protected] bin]#

View Code

Looking at the above results, it is shown that the Test.users collection has been fragmented, but through MONGOs routing, we do not feel that the data is stored on which Shard Chunk, which is an advantage of MongoDB user experience, that is, transparent to the user.

MongoDB Finishing Note のsharding Shard