Mongo DB 2.6 needs to know some of its own limits

In the real world, everything has two sides, in the process of the world, and so on! Whether we are in the use of a new language, or a new technology, in the understanding of how exciting it is, make people relaxed, all of the excellent as for, it is necessary to understand some of his limitations, it is convenient for you in the actual development process encountered these when you understand how to deal with these areas, when it comes to these places, Can advance judgment, the article translated from MONGO DB official website, translation is not good, but also hope you understand, at the same time I also use MONGO DB to do some of the spatial geocoding operations, I hope we have a lot of exchange advice, in this first thanked!!!

This note provides some limitations on hardware and software when using MONGO db (the limitation of MongoDB itself)

Translation to: http://docs.mongodb.org/manual/reference/limits/#shard-key-limitations

BSON File Size Limits

The maximum limit for a 1.BSON file is no more than 16MB for a single document.

The maximum file size limit helps ensure that a single file does not consume too much memory and does not consume too much bandwidth during the transfer process. MongoDB provides an additional set of Api-grid FS when a single file size that needs to be stored in MongoDB exceeds the maximum file size of the MONGO db limit.

2. Deep nesting of Bson documents

Since the MONGO DB 2.2 update, MongoDB supports a single file with no more than 100 Bson file nesting.

Name space

the length of the namespace: in each namespace, including the database name, the collection name, must not exceed 123KB.

limits on the number of namespaces: the limit for the number of namespaces is the size of the namespace divided by 628. For example: A 16MB namespace can provide approximately 24,000 namespaces, and a single collection name or index name is a namespace.

namespace File Size: namespace file cannot exceed 2047MB. The default namespace size is 16MB, but you can configure the size of the namespace by using the Nssize parameter.

When the database is created, a file named "DB name].0" is created, and when more than half of that file is used, a file named "DB name].1" is created again, which is twice times the size of the file. This situation will continue to occur, so files of size 256, 512, 1024, 2048 will be written to disk. Finally, when you create the file again, the size will be 2048Mb. If storage space is a limitation of the project, then you have to consider this situation.

Index

1. Index key Restrictions 2.6 version update comparison

The total size of the index entries, including the index structure, depends on the data type used (see--bson type), but must be less than 1024KB.

　　

After the MongoDB 2.6 version is updated, MongoDB has a strong restriction on the creation of the index keys, with the following limitations:

A. If the number of indexes for the collection has reached the maximum limit, Mongo DB will no longer create an index entry for it. Before 2.6, MongoDB creates an index for it, but no longer indexes the documents.

B. An error occurs if you rebuild the index on a field that has an index entry that exceeds the index key limit. Rebuilding an index occurs when the Db.collection.reIndex () method is called when the Compact and RepairDatabase commands are executed. Because these operations delete all existing indexes that are already in the collection, and then try to recreate the index, the index key restriction (occurrence) error prevents any index rebuild operations from being made on the collection. However, if you use the RepairDatabase command, the program will continue to execute down.

C. MongoDB will no longer insert any documents that have been indexed since there is already a collection with indexes that exceed the index key limit. Instead, an error is prompted. Before 2.6, MongoDB inserted a document with an index, but did not index the inserted document.

D. In the process of updating an index, an error message is returned if the updated index value raises an index that exceeds the index key limit (the error).

E. If a document contains a slice key that exceeds the index key limit, an error will occur for any updates to the file that are moved.

If an existing document contains a indexed field whose index entry exceeds the limit, Any update this results in The relocation of this document on disk would error.

F.mongorestore and Mongoimport do not insert new documents for any collection that appears to have indexes that exceed the index key limit.

G. In version 2.6, if the index exceeds the number of index limits, a level two replica set member continues to perform a file copy operation when an integrated initial synchronization occurs, but a warning message is printed in the log.

If MongoDB in the replica set is not the same version, the main level version is 2.4, the secondary version is 2.6, and the secondary level copies the files that have been inserted or updated to the master server in accordance with the 2.4 version. However, MongoDB prints the error message to the log file if there is a document in the document that has an indexed entry that exceeds the index creation limit.

H. For an existing Shard collection, if the data block has a document that contains a slice key, the index entry exceeds the index key limit. Failed while performing chunk migration (block migration).

2. Number of indexes per collection: Each collection can create (no more than) 64 indexes.

3. Index name length: Fully qualified name (including name and Dot delimiter, for example Databse.collection.indexName) no more than 128 characters.

4. Slice key for composite index: When creating a composite index, the fields in the composite index must not exceed 31.

5. When querying, you cannot use both a text index and a geospatial index: When you perform a query operation, you need a different type of special index, and you cannot mix the text index with the text command to perform the special. For example, you cannot query using both text and $near commands ($near spatial queries used in geocoding, text for general textual retrieval)

Data

Maximum number of documents in a collection

Mongo DB 2.4, if you use the max parameter to set the maximum number of documents that are allowed to be created in the collection, the maximum number of documents must be guaranteed to be within 32 of 2, and the number of documents you can create is unrestricted if you do not specify the maximum number of documents when you create the collection.

Data Size

A single Mongod instance cannot manage a data set that exceeds the maximum virtual memory address space provided by the operating system.

The number of collections in the database

The maximum number of collections in a database is determined by the size of the namespace file and the number of indexes in the collection. See the namespace for more details .

Replica set

Number of members in a replica set: A replica set can contain up to 12 members (nodes), and if you have more than 12 nodes in your application, you should consider using master re-replication.

number of members who can participate in the election in a replica set: In the replica set, only 7 members can have the right to vote at any given time, for more information see- replica set elections

Maximum value of the Automatically created action log: After 2.6, MongoDB will automatically create an operation log of no more than 50GB if you do not explicitly specify the size of the operation log (for example, by setting OPLOGSIZEMB or--oplogsize).

Shard Cluster

Only the limits and thresholds of the Shard cluster are provided here

Shard Operation Limit

operations that cannot be used in a sharding environment:

1. In the Shard, group is unusable, you should use mapReduce or aggregate instead of group.

2.db.eval () is incompatible in the Shard collection, and you may only use Db.eval () in a collection that has not yet been fragmented in the Shard cluster.

3. $where function will not allow objects referencing DB, which is rare in non-shard collections.

4. $isolated updating modifiers in a fragmented cluster environment is not effective.

5. The $snapshot fails in this environment.

The 6.geoSearch command also does not support sharding.

overwrite queries in a shard cluster: overwriting queries in a Shard collection are not supported in MongoDB.

data size for an existing (Shard) collection:

for existing collections, MongoDB supports sharding of file collections that are less than 256GB. MongoDB can shard up to 400GB of collection files based on the distribution of file size.

MongoDB can precisely limit the size of the block and the size of the data

Important: When Sharding is successful, The Shard collection itself takes up a certain amount of space.

Single-document modification in a shard collection

All update () and remove () operations in the Shard collection must explicitly specify Justone or Multi:false, and must include a slice key (Shard key) or _id field (a unique identifier for MONGODB) in the query parameter.  If the update () and remove () operations specify Justone or Multi:false but do not specify a slice key (Shard key) or a _id field, an error message is returned. Note: The Tablet key (Shard key) can contain multiple fields in the collection (field)

Unique index key in the Shard collection

MongoDB does not support creating unique index keys through shards, unless the unique index key is prefixed with the full Shard field as an index, in which case MongoDB forces a unique index to be created by this complete field instead of creating an index on a single field.

Reference: Enforce Unique Keys for sharded collections see another way to do this.

restrictions on the chip key

chip Key size limit: The total length of the tablet key must not exceed 512bytes.

The type of the slice key: the type of the slice key can be an ascending index or a composite index or hash index that starts with a slice key and specifies the sort order.

The index of the slice key cannot be specified as the following types of indexes: Multikey index, text index, geospatial index.

The tablet key is immutable

In the collection of shards that have been completed, you can no longer modify the slice key if you have to modify it:

1. Turn all data in MongoDB into other formats

2. Delete the previous Shard collection.

3. Configure the Shard to use the new (need to modify) the chip key;

4. Pre-dividing the range of the chip key to ensure that it can be evenly distributed at initialization time

5. Restore the previous data to MongoDB.

The value of the slice key in the document is immutable

After a successful new document is added to the partitioned collection, you cannot modify the value of the field used to create the index in this document, and the update () operation will no longer update the value of the field in the document that is used to create the index.

The increment of the chip key can effectively limit the throughput of the insertion

for clusters with a large number of new operations, the incremental slice key can effectively limit the throughput of the insertion. If your slice key is _id, you need to be aware of the Objectid type of _id, whose value is usually self-increasing.

By incrementing the index to insert the document, all the inserted documents belong to the block in the same shard, and the system eventually receives all the write operations and allocates the written content reasonably to a different block in order to distribute the data evenly. However, at any time the cluster inserts the new document directly into a shard, which creates bottlenecks in the throughput at insert time.

If the cluster is primarily responsible for read and update operations, then this restriction does not affect the cluster.

to avoid this constraint, you can use the hash pad key or select a non-self-increment or reduce the self-increment field (select a field that does not increase or decrease monotonically.).

In version 2.4: The hash key and the hash index store the index key in the order in which the values are listed.

operation

Storing Documents

If the sort operation uses less than 32MB of memory, MongoDB will return only the sort results without the slice key

Aggregation Pipeline Operations

in a 2.6 version update, the pipeline stage will have a 100MB limit on memory, and if the segment exceeds this limit, MongoDB will produce an error, in order to allow processing of large datasets, you can use the allowdiskuse parameter to ensure that data can be written to a temporary file during the aggregation pipeline phase. See $sort and memory restrictions and $group Operator and memory.

When using 2d for spatial queries, you cannot use the $or operator, see $or and 2d Index internals.

Spherical polygons must be suitable for hemisphere operation

Any geometry query using the $geointersects or $geoWithin in Geojson must be suitable for the hemisphere. When MongoDB parses the geometry, more than half of the spheres are queried as complementary smaller geometries.

Limit size for write command operations

The Write command can receive no more than 1000 operations. There is no limit to the bulk () and similar methods in the MONGO script.

Naming restrictions

database names are strictly case-sensitive: MongoDB does not allow the case of letters only, but the names of the databases. For example, test and test

Window Limitations of database names in the operating system (updated in version 2.2 ):

If MongoDB's development environment is under Windows system, MongoDB does not allow the following characters to appear in the database name:/\. "*<>:|?"

Also, the database name cannot contain null characters

Unix/linux restrictions on naming under the system

If MongoDB's development environment is under Windows system, MongoDB does not allow the following characters to appear in the database name:/\. "

Also, the database name cannot contain null characters

length limit for database names: database names cannot be empty and must be controlled within 64-character ranges.

restrictions on Collection names (updated in version 2.2):

The name of the collection should begin with an underscore or a letter, and the name cannot contain: $, an empty string (a space), a null character (null character), and cannot be prefixed with system. In the MONGO script, using Db.getcollection () to specify the collection name may conflict with the script or be used as a JavaScript that is not validated.

restrictions on field names: field names cannot contain ". "," $ ", or a null character (null character) see Dollar sign Operator escaping for another way.

Mongo DB 2.6 needs to know some of its own limits