MongoDB Shard Management

In MongoDB (version 3.2.9), a shard cluster (sharded cluster) is a way to scale the performance of a database system horizontally, with the ability to store datasets distributed on different shards (shard), each of which only holds part of the data set. MongoDB guarantees that there will be no duplicate data between the shards, and that the sum of the data stored by all shards is the complete data set. The Shard cluster will distribute the data set and distribute the load on multiple shards, and each shard is responsible for reading and writing part of the data, making full use of the system resources of each shard and improving the throughput of the database system.

The datasets are split into chunks (chunk), each containing multiple doc, and the data blocks distributed in the Shard cluster (Tengyun technology ty300.com). MongoDB is responsible for tracking the data block on the Shard distribution information, each shard store which data blocks, called shards of metadata, stored in the config Server database config, generally use 3 config server, all config The config database in the server must be exactly the same. The MONGOs can directly access the database config to view the metadata of the Shard, and the MONGO shell provides the SH helper function to safely view the metadata information of the Shard cluster.

Querying on any one shard only gets a subset of the data collection on the current Shard, not the entire dataset. Application only needs to connect to MONGOs, read and write to it, MONGOs automatically routes read and write requests to the appropriate shard (diligent qkxue.net). MongoDB MONGOs The underlying implementation of the Shard to application transparently, and, in application's view, accesses the entire data set.

One, primary shard

In a shard cluster, not every collection is distributed, and the collection is distributed in different Shard only after the collection shard is explicitly used with sh.shardcollection (). For non-shard collections (Un-sharded collection), their data is stored only in the primary shard (Primary shard), by default, the primary shard is the Shard originally created by the database, which stores data for non-shard collections in the database. Each database has a primary shard.

Each database in a sharded cluster have a primary shard that holds all the un-sharded collections for that database. Each database have its own primary shard.

For example, a shard cluster has three shards: Shard1,shard2,shard3, which creates a database blog in a shard shard1. If the database Bolg shards, then MongoDB will automatically create a shard2,shard3 on the same structure of the database blog, database blog primary shard is Shard1.

As shown, the primary shard of the Collection2 is Sharda.

Using the moveprimary command to change the default primary shard of the database, the non-Shard collection will move from the current Shard to the new primary shard.

Db.runcommand ({moveprimary: "Test", To: "shard0001"})

After you change the primary shard of the database using the Moveprimary command, the configuration information in config server is up-to-date and the configuration information for the MONGOs cache becomes obsolete. MongoDB provides commands: Flushrouterconfig forces MONGOs to get the latest configuration information from config server, refreshing the MONGOs cache.

Db.admincommand ({"Flushrouterconfig": 1})

Second, the metadata of the Shard

Do not go directly to the Config server to view the metadata information of the Shard cluster, which is very important and secure by connecting to the Config Data View via MONGOs, or by using the SH helper function.

Use the SH helper function to view

Sh.status ()

Connect to MONGOs view a collection in the Config database

mongos> use Config

1,shards Collection Saving Shard information

Db.shards.find ()

Shard data is stored in the replica set or standalone Mongod specified by the host.

{"_id": "Shard_name", "host": "Replica_set_name/host:port", "tag": [Shard_tag1,shard_tag2]}

2,databases Collection holds information for all databases in a fragmented cluster, regardless of whether the database is fragmented

Db.databases.find ()

If Sh.enablesharding ("Db_name") is executed on the database, the field partitioned field value is the primary shard (primary shard) of the True;primary field that specifies the database.

{"_id": "Test", "PRIMARY": "Rs0", "Partitioned": true}

3,collections Collection holds information for all fragmented collections, excluding non-Shard collections (un-sharded collections)

Key is: The slice key of the Shard

Db.collections.find () {"_id": "Test.foo", "Lastmodepoch": ObjectId ("57dcd4899bd7f7111ec15f16"), "Lastmod": ISO Date ("1970-02-19t17:02:47.296z"), "dropped": false, "key": {"_id": 1}, "unique": true}

The 4,chunks collection holds the data block information,

NS: The collection of shards, the structure is: db_name.collection_name

Min and Max: The minimum and maximum values of the slice keys

Shard: The Shard where the block resides

Db.chunks.find () {"_id": "Test.foo-_id_minkey", "Lastmod": Timestamp (1, 1), "Lastmodepoch": ObjectId ("57dcd489 9bd7f7111ec15f16 ")," ns ":" Test.foo "," min ": {" _id ": 1}," Max ": {" _id ": 3087}," s Hard ":" Rs0 "}

5,changelog Collection records the operation of a shard cluster, including chunk split and migration operations, shard Add or remove operations

What field: Represents the type of operation, for example: Multi-split represents a split of chunk,

' What ': ' Addshard ', ' What ': ' Shardcollection.start ', ' What ': ' shardcollection.end ', ' What ': ' Multi-split ',

6,tags record the tag of the Shard and the corresponding chip key range

{"_id": {"ns": "Records.users", "min": {"ZipCode": "10001"}}, "ns": "Records.users", "min": {"ZipCode ":" 10001 "}," Max ": {" ZipCode ":" 10281 "}," tag ":" NYC "}

The 7,settings collection records the size of the equalizer state and chunk, the default chunk size is 64MB.

"Chunksize"}

"Balancer"}

8,locks Collection records the distribution lock (distributed lock), guaranteeing that only one MONGOs instance can perform administrative tasks in the Shard cluster.

MONGOs when serving as balancer, it acquires a distribution lock and inserts a doc into the config.locks.

Manuscripts: Diligent Learning qkxue.net

Read the full version of MongoDB Shard Management

MongoDB Shard Management