MongoDB Shards (9)

What is a shard

Sharding refers to the process of splitting a database and dispersing it across different machines. Spread the data across different machines without the need for powerful servers to store more data and handle larger loads. The basic idea is to cut the set into small pieces, which are scattered across several slices, each of which is responsible for only a portion of the total data, and finally a equalizer to equalize each shard. Through a routing process called MONGOs, MONGOs knows the correspondence between the data and the slices by configuring the server. Most usage scenarios are solving disk space problems, and the query will avoid cross-shard queries as far as the write is likely to become worse.

Time to use shards:

1, the machine's disk is not enough. Use sharding to resolve disk space issues.

2, a single mongod can no longer meet the performance requirements for writing data. Partitioning allows write pressure to spread across shards, using the resources of the Shard server itself.

3, want to put a lot of data in memory to improve performance. As above, the Shard server's own resources are used through sharding.

Before you build a shard, understand the role of each role in the Shard.

① Configure the server. is a separate mongod process that holds the metadata for clusters and shards, which is information about what data each shard contains. Start building first, enable logging. Start the configuration server like normal Mongod, and specify the configsvr option. Without too much space and resources, the 1KB space of the configuration server corresponds to 200MB of real data. Only the distribution table of the data is saved. When the service is unavailable, it becomes read-only, unable to block, migrate data.

② Routing Server . That is, MONGOs, a routing function for the program to connect. Do not save the data itself, load the cluster information from the configuration server at startup, open the MONGOs process needs to know the address of the configuration server, specify the configdb option.

③ The Shard server . is an independent normal mongod process that holds data information. Can be a replica set or it can be a separate server.

1. Create a new Folder

DATA/CONFIG20 Configuring server Configsvr 27020 Port

Data/shard17 Shard Server 27017 Port

Data/shard18 Shard Server 27018 Port

2. Start the configuration server

Mongod--dbpath C:\Users\yuxl\Desktop\mongodb\data\config20--port 27020

3. Establish the MONGOs process for application connectivity

MONGOs--port 30000--configdb 127.0.0.1:27020

4. Start the Shard server

Mongod--dbpath C:\Users\yuxl\Desktop\mongodb\data\shard17--port 27017

Mongod--dbpath C:\Users\yuxl\Desktop\mongodb\data\shard18--port 27018

5. Adding shards

Connecting the MONGOs process MONGO 127.0.0.1:30000/admin

Adding shards

Sh.addshard ("127.0.0.1:27017")

Sh.addshard ("127.0.0.1:27018")

View status Sh.status (), we found two shards

Shards:

{"_id": "shard0000", "host": "127.0.0.1:27017"}

{"_id": "shard0001", "host": "127.0.0.1:27018"}

6. Add a library to be fragmented

Sh.enablesharding (DatabaseName);

Sh.enablesharding ("Shop");

7: Add table to be fragmented

Sh.shardcollection (' Dbname.collectionname ', {field:1});

field is a field of collection, and the system will use the value of filed to calculate which slice should be divided.

This filed is called "Chip Key", Shard key

Sh.shardcollection ("Shop.goods", {goodsid:1});

Then insert the data to test, in order to easily see the effect, use the operation

Use Config

Db.settings.update ({"_id": "Chunksize"},{$set: {"value": 1}})

Use Shop

for (Var i=1;i<=10000;i++)

{

Db.goods.insert ({"Goodsid": I, "goodsname": "ABDDSDFSDFSDFSDFSDDDDDDDDDDDDD"})

}

And then we connect to two shard servers, and we find data, but it's not even.

Reason:

MongoDB is not from a single document level, the absolute average scattered on each piece, but the N document, form a block "chunk", priority on a piece, when this piece of chunk, than the chunk of another film, the difference is relatively large, (>=3), will put on this film chunk , move to another slice, and maintain data equalization between slices in chunk

Q: Why did I insert 100,000 data, only 2 chunk?

A: Description chunk is relatively large (default is 64M)

In the config database, modify the value of the chunksize.

Q: Since priority is inserted on a piece, when the chunk is unbalanced, then move the chunk,

Naturally, with the increase of data, there is a phenomenon of chunk moving back and forth between instances of Shard, what problems will this bring?

Answer: The increase of IO between servers,

Q: Can I define a rule in which n data is formed into 1 blocks, with the advance of assigning M chunk,

M chunk trailers are assigned on different slices.

Future data directly into their pre-allocated chunk, no longer moving back and forth?

Answer: Yes, manual pre-shard! Note: Pre-shard collection are empty

Take the Shop.user table as an example

1:sh.shardcollection (' Shop.user ', {userid:1}); User table with UserID do shard key

2:for (var i=1;i<=40;i++) {sh.splitat (' Shop.user ', {userid:i*1000})}//Pre-1K 2K ... 40K such a boundary is cut well chunk (although chunk is empty), these chunk will be evenly moved to each piece.

3: Add user Data via MONGOs. The data is added to the pre-allocated chunk, and chunk does not move back and forth.

In the build environment we use the Replcation set with the Shard Shard

1: On 3 stand-alone servers, run 27017,27018,27019 instances separately, each with a replica set, forming 3 sets of repl set

2: On 3 servers, configure Config server, run on port 27020

3: Configure MONGOs

./bin/mongos--port 30000 \

--dbconfig 192.168.1.201:27020,192.168.1.202:27020,192.168.1.203:27020

4: Connect the router

./bin/mongo--port 30000

5: Add Repl Set as Slice

>sh.addshard (' 192.168.1.201:27017 ');

>sh.addshard (' 192.168.1.202:27017 ');

>sh.addshard (' 192.168.1.203:27017 ');

6: Add the library to be fragmented

>sh.enablesharding (DatabaseName);

7: Add table to be fragmented

>sh.shardcollection (' Dbname.collectionname ', {field:1});

MongoDB Shards (9)