A summary of the common issues in deployment and maintenance management of MongoDB shards

Shards (sharding is a method used by MongoDB to split large collections into different servers (or clusters), mainly for scenarios that address high throughput and large data volumes.

Compared with the existing sub-database sub-table, partitioning scheme, MongoDB is the biggest difference is that it can almost automatically do everything, just tell MongoDB to allocate data, it can automatically maintain the data between the different server balance.

I. Clustered components of a shard

1.Mongos "Routing"

As the requested access gate, all requests are routed, distributed, merged by MONGOs, driver transparent to the client, and the user connection MONGOs as if it were a connection mongod. The MONGOs will route the request to the corresponding shard based on the request type and Shard key.

2.Config Server "configuration Server"

Stores all metadata for sharding Cluster, all of which are stored in the Config database;

* Save the chunk information on each Shard * Save the Slice key range on the chunk.

3. Shard "Shard"

Store app data records.

two. Sharding Advantages

1. Abstract the cluster so that the cluster is "invisible" and the Shard is transparent to the application system.

MONGOs is a proprietary routing process that routes requests from clients to one or a group of servers in the cluster and sends the received responses back together to the client.

2. Ensure that the cluster is always readable and writable

The use of MongoDB shards and replica sets, while ensuring that data is fragmented to multiple servers, ensures that every data is backed up to ensure that when a server is broken, the rest of the library can immediately take over from the broken part to continue working. Increased availability and reliability of the cluster.

3. Make the cluster easy to extend

When the system needs more space and resources, MongoDB allows us to expand the system capacity as needed.

Three. Shard Deployment considerations (Common Errors)

1. Configuring a replicable set as a shard node is basically the same as configuring a replicable set that is used separately. However, the-SHARDSVR parameter must be specified in the startup parameter.

Otherwise, start the database shard times wrong:{"Code": 193, "OK": 0, "errmsg": "Cannot accept sharding commands if not started with--shardsvr"}.

2. When creating a configuration server cluster, the witness node cannot be set.

Otherwise, the error "ErrMsg": "Arbiters is not allowed in the replica set configurations being used for config servers".

3. When configuring MONGOs instances, do not configure the DBPath parameter.

Otherwise, set the DBPath parameter, the service does not start properly, error:error parsing INI config file:unrecognised option ' dbpath '.

4. When configuring MONGOs instances, set keyfile.

Otherwise, do not set Keyfile,service does not start properly, error: 2018-05-10t15:30:26.791+0800 W sharding [Mongosmain] Error initializing sharding state , sleeping for 2 seconds and trying again:: Caused by:: Unauthorized:error loading Clusterid:: Caused by:: not au thorized on config to execute command {find: ' Version ', Readconcern: {level: ' Majority ', Afteroptime: {ts:timestamp 15 25937413000|2, T:1}}, maxtimems:30000

5. Shard Collection settings.

Shards are not generated by default, you need to start the Shard (sh.enablesharding ("DBName") in the database before setting the collection shard (Sh.shardcollection ("Collection" {Slice key})

Four. Considerations for Shard Management (common commands)

1. Check the shards configuration and status

Db.runcommand ({listshards:1})

2. Check the address of the database main slice and whether it is partitioned

Db.getsiblingdb ("config"). Databases.find ()

3. Check the number of data blocks

Db.chunks.count ()-- you need to switch to the configuration database (config )

4. View the details of the Shard, including database information and scope information

Sh.status ()

5. Indexing is an important means of optimizing query performance. When an index is declared on a shard collection, each shard defines a separate index for its own collection part. The Shard collection only allows a unique index to be established on the _id field and the Sharding key.

6. Partitioning and migrating MongoDB bottom-up relies on 2 mechanisms to maintain the balance of the cluster: partitioning and migration.

Segmentation is the process of splitting a large chunk of data into 2 smaller chunks. Migration is the process of moving chunks of data between shards. The migration process is triggered when some shard servers contain much more data block data than other shard servers, which is called the Migration Round (migration round)

6.1 Migration Trigger conditions

6.2 See if the balancer process is turned on sh.getbalancerstate ()

6.3 Stop balancer Process sh.stopbalancer () and open balancer process

6.4. By default, the Balancer process is running, in order to reduce the Balancer process to the system, you can set the run time window for the Balancer process and let the Balancer process operate in the specified time window.

6.4.1 For example, set the balancer process to execute within the 23:00 to 6:00 time window. db.settings.update ({_id: "balancer"}, {$set: {ActiveWindow: {start: "23:00", Stop: "6:00PM"}}}, True);

6.4.2 Delete Balancer process run time window

6.5. View the scope of the block

6.5.1 If the collection data volume is small, you can view it directly through Sh.status ()

6.5.2 If the collection data is large, sh.status () cannot reflect the chunking information for this collection. At this point, you can view Printshardingstatus (db.getsisterdb ("config"), 1) by executing the following command:

6.5.3 can also switch commands to the Config database and perform db.chunks.find () viewing. You can enter a set of parameters, for example, View Shard Repsms2, set Cloud-docs. Pushmessagerecord Block case (Cloud-docs for database name)

Db.chunks.find ({"Shard": "Repsms2", "ns": "Cloud-docs. Pushmessagerecord "}). Pretty ()

Five. Notes

Shard Cluster Management has a relatively large amount of data, and the schema of the Shard is comparatively complex. Therefore, must be in the business needs of fragmentation, and then on the Shard, and can not be quasi-"dazzle" technology and on the Shard. In addition, after the line, the relevant monitoring must be deployed, and gradually improve.

This article copyright belongs to the author, without the consent of the author, thank you!!!