The pit of the MongoDB data update

Statistics MongoDB slow query, found that some set slow query a lot, and then notify the development of the field to look at the index,

And after the development of the discussion with a unique index, add the time to find duplicate data, and then use the aggregate command to count 24w of data have 10w+ duplicate data,

Development said update operation when added {upsert:true}, should be the query not to add a new, no duplicate data,

Then look at the official documentation for MongoDB to view Db.collection.update, which has the following explanations

Use Unique Indexes

WARNING

To avoid inserting the same document more than once, only use Upsert: True if the query field is uniquely indexed.

Given A collection named people where no documents has a name field that holds the value Andy. Consider when multiple clients issue the following update with Upsert: true at the same time:

" Andy " " Andy " 1 1 true

If all Update () operations complete the query portion before any client successfully inserts data, Andthere is no unique I Ndex on the Name field and then each update operation may result in an insert.

To prevent MongoDB from inserting the same document more than once, create a unique index on the Namefield. With a unique index, if multiple applications issue the same update with Upsert: true, exactlyone update ( ) would successfully insert a new document.

The remaining operations would either:

• Update the newly inserted document, or

• Fail when they attempted to insert a duplicate.

  If the operation fails because of a duplicate index key error, applications may retry the operation which would succeed as An update operation.

The meaning is generally said: At the same time high concurrency upsert words, query operation completed, but not insert, at the same time insert more than the same data, in order to avoid this problem can increase the unique index, that is, the uniqueness of the data can only be guaranteed by a unique index,

At the same time, an example of a unique index and then high concurrency Update operation:

1. Update the new data that has been inserted
2. Because the unique index insert fails, it is best to add a retry mechanism so that the update succeeds.

The pit of the MongoDB data update