perform a two-step commit
OverviewThis section provides a multi-record update or multi-record transaction, using a two-step commit to complete a multi-record write template. In addition, this method can be extended to provide rollback-like functionality.
BackgroundMongoDB's operations on a single record are atomic, but operations involving multiple records are not atomic in nature. Because records can be quite complex and have embedded records, single-record atomic operations provide the necessary support that is commonly used in practice.
In addition to the atomic operation of a single record, there are many situations that require more logging of operational transactions, and when a transaction that contains some column operations is performed, the following requirements are available:
Atomicity: If an operation fails, the previous operation in the transaction needs to be rolled back to its previous state
Consistency: If a major error, such as a network failure, a hardware failure, a transaction is interrupted, the database must be able to revert to its previous state
For transactions that require multiple logging operations, you can implement a two-step commit method in your app to provide multi-record update support. This method guarantees consistency, and in the event of an error, the execution state of the transaction is recoverable. In this process, however, the records are in undecided data and state.
Note: Because MongoDB has only single-record operations that are atomic, two-step commits can only provide semantic "class transaction" functionality. For an application, enable it to return to intermediate data or rollback data in a state in a two-step commit.
TemplatesConsider the following scenario:
To transfer funds from account A to account B, in a relational database, you can subtract funds from a in a transaction and add them in B. In MongoDB, you can simulate two-step commit to get the same result.
This example uses two sets of
1.accounts for storing account information
2.transactions, information for storing money transfer transactions
initializing account information Db.accounts.insert (
[
{_id: "A", balance:1000, Pendingtransactions: []},
{_id: "B", balance:1000, Pendingtransactions: []}
]
);
Initialize transfer recordfor each money transfer operation, the transfer information is added to the Transactions collection, and the inserted record contains the following information:
Source and destination fields, referencing _id fields from the Ccounts collection
Value field, declaring the transfer value
The State field, which indicates the current transfer status, can be a value of initial,pending, applied, done, canceling, or canceled.
LastModified field, reaction last modified date
From a transfer of 100 to B, initialize the transactions record:
Db.transactions.insert ({_id:1, Source: "A", Destination: "B", value:100, State: "Initial", Lastmodified:new Date ()});
use a two-step commit to transfer funds 1. From the Transactions collection, locate the record with state initial. At this point there is only one record in the Transactions collection, the one that was just inserted. In a collection that contains other records, the query returns any records with state initial unless you declare other query criteria.
var t = db.transactions.findOne ({state: "initial"});
Enter T in the MongoDB shell to see the contents of T, similar to:
{"_id": 1, "source": "A", "Destination": "B", "value": +, "state": "Initial", "LastModified":??}
2. Update the transaction status to Pending
Set state to pending,lastmodified as current time
Db.transactions.update (
{_id:t._id, State: "Initial"},
{
$set: {state: "Pending"},
$currentDate: {lastmodified:true}
}
)
In the update operation, state: ' initial ' ensures that no other process has updated this record. If nmatched and nmodified are 0, go back to the first step, get a new transaction, and start the process again.
3. Apply the transaction to two accounts
Use the Update method to apply a transaction T to two accounts. In the update condition, include the condition pendingtransactions:{$ne: t._id} To avoid repeating the transaction.
Db.accounts.update (
{_id:t.source, pendingtransactions: {$ne: t._id}},
{$inc: {balance:-t.value}, $push: {pendingtransactions:t._id}}
)
Db.accounts.update (
{_id:t.destination, pendingtransactions: {$ne: t._id}},
{$inc: {balance:t.value}, $push: {pendingtransactions:t._id}}
)
Subtract t.value from a account, add T.value to the B account, plus transaction ID for each account's pendingtransactions array
4. Update the transaction status to applied
Db.transactions.update (
{_id:t._id, State: "Pending"},
{
$set: {state: "Applied"},
$currentDate: {lastmodified:true}
}
)
5. Update the account pendingtransactions array
Db.accounts.update (
{_id:t.source, pendingtransactions:t._id},
{$pull: {pendingtransactions:t._id}}
)
Db.accounts.update (
{_id:t.destination, pendingtransactions:t._id},
{$pull: {pendingtransactions:t._id}}
)
Removes an applied transaction from two accounts.
6. Update the transaction status to Done
Db.transactions.update (
{_id:t._id, State: "Applied"},
{
$set: {state: ' Done '},
$currentDate: {lastmodified:true}
}
)
recovering data from a failed scenario The most important thing about a transaction is not the prototype provided by the above example, but the possibility of recovering data from various failure scenarios when the transaction is not fully executed.
Recovery operations
The two-step commit model allows the application to re-execute the sequence of transaction operations to ensure data consistency. Crawl any outstanding transactions at the time of program startup or by performing a scheduled recovery operation.
The time to recover to the consistent state of the data depends on how long the application needs to recover each transaction.
The following recovery operation uses the LastModified date as the identifier for whether a transaction in the pending state needs to be rolled back. If a transaction with pending or applied status has not been updated within the last 30 minutes, these transactions need to be restored. You can use different conditions to determine whether a recovery is required.
pending state of the transaction Restore the transaction state after pending, before applied
Cases:
Get an unsuccessful transaction record in 30 minutes
var datethreshold = new Date ();
Datethreshold.setminutes (Datethreshold.getminutes ()-30);
var t = Db.transactions.findOne ({state: "Pending", LastModified: {$lt: Datethreshold}});
Then go back to "3. Apply this transaction to two accounts" step
applied State of the transaction Cases:
Get an unsuccessful transaction record in 30 minutes
var datethreshold = new Date ();
Datethreshold.setminutes (Datethreshold.getminutes ()-30);
var t = Db.transactions.findOne ({state: "Applied", LastModified: {$lt: Datethreshold}});
Then go back to "5. Update account pendingtransactions Array" This step
rollback operationIn some cases, you may need to rollback, or undo, for example, the application needs to cancel the transaction, or one of the accounts does not exist or is frozen.
applied State of the transaction After the "4. Update transaction status is applied" step, you should not roll back the transaction, but instead should complete the current transaction and then create a new transaction to modify the data back.
pending state of the transaction After the "2. Update transaction status is pending" step, after the "4. Update transaction status to applied" step, you can roll back the transaction by following these steps:
1. Update the transaction status to cancel
Db.transactions.update (
{_id:t._id, State: "Pending"},
{
$set: {state: "Canceling"},
$currentDate: {lastmodified:true}
}
)
2. Cancel the operation in two accounts
If the transaction has already been applied, the transaction needs to be rolled back to cancel operations on two accounts. In the updated condition, include pendingtransactions:t._id to update the account when pending transaction has been applied.
Update the target account, minus the value added to it in the transaction, remove the transaction from the Cong pendingtransactions array _id
Db.accounts.update (
{_id:t.destination, pendingtransactions:t._id},
{
$inc: {balance:-t.value},
$pull: {pendingtransactions:t._id}
}
)
If pending transaction has not been applied to this account, there will be no record matching query criteria.
3. Update the transaction status to Canceled
Db.transactions.update (
{_id:t._id, State: "Canceling"},
{
$set: {state: "cancelled"},
$currentDate: {lastmodified:true}
}
)
The update transaction state is cancelled to flag that the transaction has been canceled.
Multi-application Scenarios Because of the existence of transactions, multiple applications can create and execute operations at the same time without data inconsistencies or conflicts. In the previous example, updating or rolling back a record that contains the state field's update condition prevents different applications from repeating the transaction
For example, App1 and APP2 acquire a transaction in the initial state at the same time. App1 commits the entire transaction before the APP2 begins. When App2 tries to update the transaction status to Pending, the update condition that contains state: ' Initial ' will not match any records.
At the same time nmatched and nmodified will be 0. This shows that APP2 needs to go back to the first step and restart a different transaction process.
When multiple applications are running, the key is that only one application can handle the specified transaction in a timely manner. In this case, even if there are records that meet the update criteria, you can create a tag in the transaction record to flag that the application is processing the transaction. Use the Findandmodify () method to modify the transaction and fallback.
t = db.transactions.findAndModify (
{
Query: {state: "initial", application: {$exists: false}},
Update
{
$set: {state: "Pending", Application: "App1"},
$currentDate: {lastmodified:true}
},
New:true
}
)
A modified transaction operation ensures that only an app with an identifier match can commit the transaction.
If App1 fails in transaction execution, the previous recovery operation can be used, but the application needs to ensure that the transaction is "owned" before the transaction is applied.
For example, a job that finds and restores a pending state
var datethreshold = new Date ();
Datethreshold.setminutes (Datethreshold.getminutes ()-30);
Db.transactions.find (
{
Application: "App1",
State: "Pending",
LastModified: {$lt: Datethreshold}
}
)
use two-step commit in a production environment The above example is intended to be very simple to write. For example, it assumes that a rollback operation for an account is always possible, and that the account can hold negative values.
The production environment may be more negative, usually the account needs a variety of information such as current account value, credit, arrears, etc.
for all transactions, make sure that you are using the Write concern permission level.
MongoDB Operations Manual CRUD transaction Two-step commit
