Official MongoDB documentation translation series: Official mongodb documentation

Official MongoDB documentation translation series: Official mongodb documentation
Introduction

This document provides a method for writing data into multiple documents using two-phase commit to process multi-document updates or "Multi-document transactions ". On this basis, you can expand and implement functions similar to data rollback.

Background

In MongoDB databases, operations acting on a single document are always Atomic. However, operations involving multiple documents are what we often call "Multi-document transactions ", it is non-atomic. Since documents can be designed very complex and contain multiple "embedded" documents, the atomicity of a single document provides necessary support for many practical scenarios. (Translator's note: for example, if you want to update the factory date of a batch of products in batches, You can embed the product information in the same document. However, I have hardly used this method, and there will be many additional problems. For example, frequent operations may cause document move .)

Although Single-document atomic operations can meet many requirements, multi-document transactions are still required in many scenarios. Some problems may occur when executing a transaction composed of several sequential operations, such:

• Atomicity: If an operation fails, all operations that occur before it in the same transaction must be rolled back to the initial state (that is, "Either all OK, or do nothing ").
• Consistency: if a serious fault occurs, the transaction is interrupted (network or hardware faults), and the database must be restored to a consistent state.

For scenarios that require multi-document transactions, You can implement two-phase commit in the application to provide support. Two-phase commit ensures data consistency. If an error occurs, the status before the transaction can be restored. In the transaction execution process, data and status can be restored to the preparation phase no matter what happens.

Note:

Because only single-document operations in MongoDB are atomic, two-phase commit can only provide semantics similar to transactions. During a two-phase commit or rollback, the application can return intermediate data at any step point.

 

Mode

Overview

In this scenario, you want to transfer money from account A to account B. In A relational database system, you can first reduce the funds of account A in A single multi-statement transaction and then increase the funds for account B. In MongoDB, you can simulate a two-phase commit to get the same result.

All examples in this section use the following two sets:

1. Set accounts to save account information.

2. Set transactions to save the transfer transaction information.

Initialize the source account and target account

Write the information of accounts A and B to the collection accounts.

Db. accounts. insert (

[

{_ Id: "A", balance: 1000, pendingTransactions: []},

{_ Id: "B", balance: 1000, pendingTransactions: []}

]

)

The preceding statement returns a BulkWriteResult () object, which contains the status information of this operation. If the data is successfully written, the value of nInserted in the BulkWriteResult () object is 2. (Note: all write operations after version 2.6 will return the WriteResult object. BulkWriteResult will be returned for batch write operations. For details, see the relevant chapter)

Initialize Transfer Data

Write each transfer information to the transactions table. The transfer data includes the following fields:

• The source and destination fields point to the _ id value in the accounts set.
• Value Field, indicating the transfer amount, affecting the balance of the source and target accounts
• The state field, indicating the current status of the transfer operation. The optional values of the state field include initial, pending, applied, done, canceling, and canceled.
• LastModified field, indicating the last update time

Initialize the operation information of account A transferring 100 to account B to the transactions set. The value of the state field is "initial", and the value of the lastModified field is set to the current time:

Db. transactions. insert (

{_ Id: 1, source: "A", destination: "B", value: 100, state: "initial", lastModified: new Date ()}

)

The preceding statement returns a WriteResult () object that contains the status information of this operation. If the write is successful, the value of nInserted of the WriteResult () object is 1.

Use two-phase transfer submission

? Get the data of the transaction set

Query the data whose state field value is initial from the transactions set. Currently, the transactions set contains only one piece of data, that is, we only write one piece of data in the above initialization Transfer Data step. If the set contains other data, the following query will return any data whose state field is initial unless you attach other query conditions.

Var t = db. transactions. findOne ({state: "initial "})

Define the variable t in mongo shell to print the returned content. The preceding statement will get the following output:

{"_ Id": 1, "source": "A", "destination": "B", "value": 100, "state": "initial ", "lastModified": ISODate ("2014-07-11T20: 39: 26.345Z ")}

 

? Set the state field of the transaction data to pending

Set the state field of the transaction data from initial to pending and use the $ currentDate operation to set the lastModified field to the current time.

Db. transactions. update (

{_ Id: t. _ id, state: "initial "},

{

$ Set: {state: "pending "},

$ CurrentDate: {lastModified: true}

}

)

This update operation returns a WriteResult () object that contains the status information of this update operation. If the update is successful, nMatched and nModified are displayed as 1.

In this update statement, the state: "initial" condition ensures that no other thread has updated this data. If nMatched and nModified are 0, return to the first step to obtain a new data record and continue to follow the steps.

 

? Transfer an account

If the account does not contain the transaction information, use the update () method to update the account information, with pendingTransactions :{$ ne: t. _ id} to avoid repeated transfers.

Update the balance field and pendingTransactions field to transfer funds.

Update the source account information, subtract the value of the transaction data from the balance field, and write the transaction _ id to the array of the pendingTransactions field.

Db. accounts. update (

{_ Id: t. source, pendingTransactions: {$ ne: t. _ id }},

{$ Inc: {balance:-t. value}, $ push: {pendingTransactions: t. _ id }}

)

After the operation is successful, the method returns the WriteResult () object. The values of nMatched and nModified are 1.

Update the target account information, add the value of transaction data to the balance field, and write the _ id of transaction to the array of the pendingTransactions field.

Db. accounts. update (

{_ Id: t. destination, pendingTransactions: {$ ne: t. _ id }},

{$ Inc: {balance: t. value}, $ push: {pendingTransactions: t. _ id }}

)

After the operation is successful, the method returns the WriteResult () object. The values of nMatched and nModified are 1.

 

? Set the state of the transaction data to applied.

Use the following update () operation to set the state value of the transaction data to applied operation to set the transaction's state to applied, and update the value of the lastModified field to the current time:

Db. transactions. update (

{_ Id: t. _ id, state: "pending "},

{

$ Set: {state: "applied "},

$ CurrentDate: {lastModified: true}

}

)

After the operation is successful, the method returns the WriteResult () object. The values of nMatched and nModified are 1.

 

? Remove the _ id value of the transaction data from the pendingTransactions field of the two accounts.

Remove the _ id value of the transaction data whose state value is applied from the pendingTransactions field of the two accounts.

Update source account

Db. accounts. update (

{_ Id: t. source, pendingTransactions: t. _ id },

{$ Pull: {pendingTransactions: t. _ id }}

)

After the operation is successful, the method returns the WriteResult () object. The values of nMatched and nModified are 1.

Update target account

Db. accounts. update (

{_ Id: t. destination, pendingTransactions: t. _ id },

{$ Pull: {pendingTransactions: t. _ id }}

)

After the operation is successful, the method returns the WriteResult () object. The values of nMatched and nModified are 1.

 

? Update the state value of transaction data to done.

Set the state of the transaction data to done and update lastModified to the current time, which also indicates the end of the transaction.

Db. transactions. update (

{_ Id: t. _ id, state: "applied "},

{

$ Set: {state: "done "},

$ CurrentDate: {lastModified: true}

}

)

After the operation is successful, the method returns the WriteResult () object. The values of nMatched and nModified are 1.

Recover from failure scenarios

In fact, the most important part is not the applicable scenario in the preceding example. It is important that the transaction cannot be recovered from various failures when it is not completed successfully. This section summarizes various possible failure scenarios and teaches you how to recover from these events.

Restore operation

The two-phase commit mode allows the application to run some operations in an orderly manner to restore transactions and achieve consistency. When the application starts, the recovery program may be a program that is periodically executed to capture any unfinished transactions.

The time required for consistency depends on how long the application interval is to restore each transaction.

The Recovery Program in the following example uses the lastModified field as an indicator to determine whether the pending state transaction needs to be restored. Then, if the pending or applied state transaction has not been updated within 30 minutes, the Recovery Program considers that these transactions need to be recovered. You can use different conditions to determine whether the transaction needs to be restored.

Pending State transactions

The state of the transaction data is set to pending. "Step after, but occurs in" set the state of the transaction data to applied. "An error occurred before the step. Obtain a pending data from the transactions collection:

Var dateThreshold = new Date ();

DateThreshold. setMinutes (dateThreshold. getMinutes ()-30 );

Var t = db. transactions. findOne ({state: "pending", lastModified :{$ lt: dateThreshold }});

Then proceed from the above "transfer an account" step

Applied State transactions

To restore the state of the transaction data after the "set the state of the transaction data to applied" step in the preceding example, but the state of the transaction data is set to done in "but before. "An error occurred before the step. Obtain an applied state data from the transactions collection:

Var dateThreshold = new Date ();

DateThreshold. setMinutes (dateThreshold. getMinutes ()-30 );

Var t = db. transactions. findOne ({state: "applied", lastModified :{$ lt: dateThreshold }});

Then, remove the _ id value of transactions data from the pendingTransactions field of the two accounts from the preceding "U "."

Rollback

In some cases, you may need to "Roll Back" or cancel the transaction. For example, the application needs to "cancel" the transaction subjectively, or an account in the transaction does not exist, or the account no longer exists in the transaction.

Applied State transactions

After the "SET transaction data state to applied." Step, you 'd better not roll back the transaction. Instead, complete the transaction, start a new transaction, change the source account and target account of the previous transaction, and make another transfer.

Pending State transactions

After the "SET transaction data state to pending." Step, before "SET transaction data state to applied.", you can roll back and forth the transaction according to the following process:

? Set the transaction data state to canceling.

Set the state of transaction from pending to canceling.

Db. transactions. update (

{_ Id: t. _ id, state: "pending "},

{

$ Set: {state: "canceling "},

$ CurrentDate: {lastModified: true}

}

)

After the operation is successful, the method returns the WriteResult () object. The values of nMatched and nModified are 1.

? Undo transactions on two accounts

Perform a reverse operation on the two accounts to cancel the transaction. In the update condition, add pendingTransactions: t. _ id to filter the data that meets the condition.

Update the target account information, subtract the value of transaction data from the balance field, and remove the _ id of transaction data from the pendingTransactions array.

Db. accounts. update (

{_ Id: t. destination, pendingTransactions: t. _ id },

{

$ Inc: {balance:-t. value },

$ Pull: {pendingTransactions: t. _ id}

}

)

After the operation is successful, the method returns the WriteResult () object. The values of nMatched and nModified are 1. If the transfer transaction has not occurred to this account before, the above update operation will not match the data, and the nMatched and nModified values will be 0.

Update the source account information, add the value of transaction data to the balance field, and remove the _ id of transaction data from the pendingTransactions array.

Db. accounts. update (

{_ Id: t. source, pendingTransactions: t. _ id },

{

$ Inc: {balance: t. value },

$ Pull: {pendingTransactions: t. _ id}

}

)

After the operation is successful, the method returns the WriteResult () object. The values of nMatched and nModified are 1. If the transfer transaction has not occurred to this account before, the above update operation will not match the data, and the nMatched and nModified values will be 0.

 

? Set the transaction data state to canceled.

Set the state of the transaction data from canceling to canceled to complete the final rollback.

Db. transactions. update (

{_ Id: t. _ id, state: "canceling "},

{

$ Set: {state: "canceled "},

$ CurrentDate: {lastModified: true}

}

)

After the operation is successful, the method returns the WriteResult () object. The values of nMatched and nModified are 1.

 

Multiple applications

To some extent, the existence of transactions aims to facilitate concurrent creation and execution of multiple applications without causing data inconsistency and data conflict. In our program, when updating or retrieving data in the transaction set, the update condition contains the state field condition, which can prevent multiple applications from applying for transaction data in conflict.

For example, App1 and App2 both obtain the transaction data with the same state as initial. Before App2 starts, App1 executes the complete transaction. When App2 tries to execute the step "set the transaction data state to pending. ", because the update condition contains the state:" initial "Statement, the value of nMatched and nModified is 0 because the update operation does not match the data. This will allow App2 to return to the first step to obtain another transaction data and re-start the transaction process.

When multiple applications run, the most important thing is that at any time, only one application can operate on a given transaction data. Similarly, in addition to the expected transaction status in the update condition, you can also create a tag for the transaction data to identify the application that is operating on the transaction data. Atomic modification using findAndModify () method and return transaction data:

T = db. transactions. findAndModify (

{

Query: {state: "initial", application: {$ exists: false }},

Update:

{

$ Set: {state: "pending", application: "App1 "},

$ CurrentDate: {lastModified: true}

},

New: true

}

)

Modify the transaction operation in the previous example to ensure that only the application that matches the application field can operate the corresponding transaction data.

If App1 fails during the transaction execution, you can use the Recovery Program to recover, but before the restoration, the application must determine that they "own" the corresponding transaction data. For example, to locate and continue executing a pending state transaction, use a query similar to the following:

Var dateThreshold = new Date ();

DateThreshold. setMinutes (dateThreshold. getMinutes ()-30 );

 

Db. transactions. find (

{

Application: "App1 ",

State: "pending ",

LastModified: {$ lt: dateThreshold}

}

)

Use two-phase commit in the production environment

The example of Account Transaction in this article is intentionally simple. For example, we assume that the account can always be rolled back and the account balance is negative.

The implementation in the production environment may be more complex. For example, in real scenarios, the information required by the account also includes the current balance, to be transferred out, and to be transferred in.

For all transactions, you need to set a proper write mode during deployment. (Translator's note: I think it is better to use secure writing to deal with transactions)

 

Copyright Disclaimer: This article is an original article by the blogger and cannot be reproduced without the permission of the blogger.