resolves a MongoDB cursor ID is not a valid at server issue
When you use Db.collection.find (), it returns not all data, but actually a "cursor". Its default behavior is to query the database for the first 101 documents, or 1 MB of documents, depending on which condition is met first, and then query 4 MB of documents each time the document in the cursor is exhausted. In addition, the default behavior of Find () is to return a cursor that expires after 10 minutes of no action. If I have a batch document not finished in 10 minutes, and then processed, and then with the same cursor ID to the server to remove a batch, the cursor ID of course has expired, which can explain why I got the cursor ID invalid error.
A workaround on Stack Overflow is to pass in Timeout=false at Find () to disable the 10-minute timeout protection. But I think this is a very bad way, because if you have an exception in the loop, or even a power outage or disconnection, will cause the MongoDB server resources can never be released. And the better way is (I also sent on the Stack Overflow), estimated a batch size, so that the MongoDB client each crawl of the document within 10 minutes to run out, so that the client has to contact the server at least 10 minutes to ensure that the cursor does not time out.
Specific usage:
For document in Db.collection.find (). Batch_size (30):
