Two-step commit of the MongoDB operation manual CRUD transaction, mongodbcrud
Perform Two-Step submission
Overview This section provides a multi-record update or multi-record transaction template that uses two-step commit to complete multi-record writing. In addition, you can extend this method to provide the rollback-like function.
Background MongoDB performs atomic operations on a single record, but operations involving multiple records are not atomic. As records may be quite complex and contain embedded records, single-record atomic operations provide necessary support in practice.
In addition to the atomic operations of a single record, there are also many cases that require multiple record operation transactions. When executing a transaction that contains some column operations, there are the following requirements:
Atomicity: If an operation fails, the previous operations in the transaction must be rolled back to the previous state.
Consistency: if a major mistake, such as a network fault or hardware fault, interrupts the transaction, the database must be able to restore to the previous state.
For transactions that require multi-record operations, you can implement two-step commit in the application to support multi-Record Update. This method ensures consistency, and the execution status of the transaction can be restored in case of an error. However, in this process, records are in undetermined data and status.
Note: Because MongoDB only has single-record operations as atomic, the two-step commit can only provide the "class transaction" function in semantics. For an application, enable it to return to intermediate data or roll back data in a certain state in the two-step commit.
The template should consider the following scenarios:
To transfer funds from account A to account B, you can deduct funds from account A in A transaction and add them to account B. In MongoDB, you can simulate two-step commit to obtain the same result.
This example uses two sets
1. accounts, used to store account information
2. transactions, used to store information about fund transfer transactions
Initialize account information db. accounts. insert (
[
{_ Id: "A", balance: 1000, pendingTransactions: []},
{_ Id: "B", balance: 1000, pendingTransactions: []}
]
);
Initialize the transfer record to add the transfer information to the transactions collection for each transfer operation. The inserted record contains the following information:
Source and destination fields, referenced from the _ id field in the ccounts set
Value Field, indicating the transfer value
State field, indicating the current transfer status. The value can be initial, pending, applied, done, canceling, or canceled.
LastModified field to reflect the last modification date
From A transfer 100 to B, initialize the transactions record:
Db. transactions. insert ({_ id: 1, source: "A", destination: "B", value: 100, state: "initial", lastModified: new Date ()});
Use two-step commit for transfer 1. From the transactions collection, find the record whose state is initial. In this case, there is only one record in the transactions set, that is, the one just inserted. In a set that contains other records, unless you declare other query conditions, this query will return any records whose state is initial.
Var t = db. transactions. findOne ({state: "initial "});
Input t in MongoDB shell to view t content, which is similar:
{"_ Id": 1, "source": "A", "destination": "B", "value": 100, "state": "initial ", "lastModified ":??}
2. Update the transaction status to pending.
Set state to pending and lastModified to the current time.
Db. transactions. update (
{_ Id: t. _ id, state: "initial "},
{
$ Set: {state: "pending "},
$ CurrentDate: {lastModified: true}
}
)
In the update operation, state: 'initial' to ensure that no other process has updated this record. If nMatched and nModified are 0, return to the first step, obtain a new transaction, and start the process again.
3. Apply the transaction in two accounts
Use the update method to apply transaction t to two accounts. The update condition contains the condition pendingTransactions :{$ ne: t. _ id} to avoid repeated application of the transaction.
Db. accounts. update (
{_ Id: t. source, pendingTransactions: {$ ne: t. _ id }},
{$ Inc: {balance:-t. value}, $ push: {pendingTransactions: t. _ id }}
)
Db. accounts. update (
{_ Id: t. destination, pendingTransactions: {$ ne: t. _ id }},
{$ Inc: {balance: t. value}, $ push: {pendingTransactions: t. _ id }}
)
Subtract t. value from account A, add t. value to account B, and add the transaction id to the pendingTransactions array of each account.
4. Update the transaction status to applied.
Db. transactions. update (
{_ Id: t. _ id, state: "pending "},
{
$ Set: {state: "applied "},
$ CurrentDate: {lastModified: true}
}
)
5. Update the account pendingTransactions Array
Db. accounts. update (
{_ Id: t. source, pendingTransactions: t. _ id },
{$ Pull: {pendingTransactions: t. _ id }}
)
Db. accounts. update (
{_ Id: t. destination, pendingTransactions: t. _ id },
{$ Pull: {pendingTransactions: t. _ id }}
)
Remove applied transactions from two accounts.
6. Update the transaction status to done.
Db. transactions. update (
{_ Id: t. _ id, state: "applied "},
{
$ Set: {state: "done "},
$ CurrentDate: {lastModified: true}
}
)
The most important thing to recover a data transaction from a failure scenario is not the prototype provided in the preceding example, but the possibility of restoring data from various failure scenarios when the transaction is not fully executed successfully.
Restore operation
The two-step commit model allows the application to re-execute the transaction operation sequence to ensure data consistency. When the program is started, or the recovery operation is scheduled to capture any unfinished transactions.
The time to restore to the data consistency state depends on how long the application needs to recover each transaction.
The following recovery operation uses the lastModified date as the identifier of whether the pending state transaction needs to be rolled back. If a pending or applied state transaction is not updated in the last 30 minutes, it indicates that the transaction needs to be restored. You can use different conditions to determine whether to restore the database.
Transactions in Pending state resume the transaction state after pending, before applied
Example:
Obtain unsuccessful transaction records within 30 minutes
Var dateThreshold = new Date ();
DateThreshold. setMinutes (dateThreshold. getMinutes ()-30 );
Var t = db. transactions. findOne ({state: "pending", lastModified :{$ lt: dateThreshold }});
Then return to "3. Apply the transaction in two accounts ".
Applied Status transaction example:
Obtain unsuccessful transaction records within 30 minutes
Var dateThreshold = new Date ();
DateThreshold. setMinutes (dateThreshold. getMinutes ()-30 );
Var t = db. transactions. findOne ({state: "applied", lastModified :{$ lt: dateThreshold }});
Go back to "5. Update account pendingTransactions array"
In some cases, you may need to roll back or cancel the operation. For example, the application needs to cancel the transaction, or one of the accounts does not exist or is frozen.
The transaction in the Applied status is in the "4. after updating the transaction status to applied, you should not roll back the transaction, but complete the current transaction and create a new transaction to modify the data back.
After the Pending state transaction step "2. Update the transaction state to pending", before "4. Update the transaction state to applied", perform the following steps to roll back the transaction:
1. Update the transaction status to canceled
Db. transactions. update (
{_ Id: t. _ id, state: "pending "},
{
$ Set: {state: "canceling "},
$ CurrentDate: {lastModified: true}
}
)
2. Cancel the operation in two accounts
If the transaction has been applied, you need to roll back the transaction to cancel the operation on both accounts. The updated condition contains pendingTransactions: t. _ id to update the account when the pending transaction has been applied.
Update the target account, subtract the value added to the transaction, and remove the transaction _ id from the cong pendingTransactions array.
Db. accounts. update (
{_ Id: t. destination, pendingTransactions: t. _ id },
{
$ Inc: {balance:-t. value },
$ Pull: {pendingTransactions: t. _ id}
}
)
If pending transaction has not been applied to this account, no matching query conditions will be found.
3. Update the transaction status to canceled
Db. transactions. update (
{_ Id: t. _ id, state: "canceling "},
{
$ Set: {state: "canceled "},
$ CurrentDate: {lastModified: true}
}
)
Update the transaction status to canceled to indicate that the transaction has been canceled.
Due to the existence of transactions, multiple applications can create and execute operations simultaneously without data inconsistency or conflict. In the previous example, the update or rollback record contains the update condition of the state field to prevent repeated transactions from being committed by different applications.
For example, app1 and app2 both obtain a transaction in the initial state. App1 committed the entire transaction before app2. When app2 tries to update the transaction state to pending, the update condition containing state: 'initial' will not match any records,
Both nMatched and nModified will be 0. This indicates that app2 needs to return to the first step and restart a different transaction process.
When multiple applications run, only one application can process specified transactions in a timely manner. In this way, you can create a tag in the transaction record even if there is a record that meets the update condition to indicate that the application is processing the transaction. Use the findAndModify () method to modify the transaction and roll back the transaction.
T = db. transactions. findAndModify (
{
Query: {state: "initial", application: {$ exists: false }},
Update:
{
$ Set: {state: "pending", application: "App1 "},
$ CurrentDate: {lastModified: true}
},
New: true
}
)
The corrected transaction operation ensures that only the application with the matching identifier can submit the transaction.
If app1 fails to be executed in the transaction, you can use the Restoration Operation described earlier, but the application must ensure that the transaction is "owned" before the application transaction.
For example, find and restore a pending job
Var dateThreshold = new Date ();
DateThreshold. setMinutes (dateThreshold. getMinutes ()-30 );
Db. transactions. find (
{
Application: "App1 ",
State: "pending ",
LastModified: {$ lt: dateThreshold}
}
)
In the production environment, it is easy to use the two-step commit example. For example, it is assumed that the rollback operation of an account is always possible, and the account can save negative values.
The production environment may be more negative. Generally, the account requires the current account value, credit, overdue payment, and other information.
For all transactions, make sure that the write concern permission level is used.
Why does MongoDB replace MySQL?
MongoDB is a document-oriented database, which is currently developed and maintained by 10gen. It has rich functions and is complete and can completely replace MySQL. In the process of using MongoDB as a product prototype, we summarized some highlights of MonogDB: The JSON-style syntax is easy to understand and understand: mongoDB uses the JSON variant BSON as the internal storage format and syntax. All operations on MongoDB use JSON-style syntax, and the data submitted or received by the client is displayed in JSON format. Compared with SQL, it is more intuitive and easy to understand and master. Schema-less: supports embedding sub-documents: MongoDB is a Schema-free document database. A database can have multiple collections, each of which is a Collection of Documents. The Table and Row of Collection and Document are not equal to those of traditional databases. Collection can be created at any time without prior definition. Collection can contain document records with different schemas. This means that the document in your previous record has three attributes, and the document in the next record can have 10 attributes, attribute types can be basic data types (such as numbers, strings, dates, etc.), arrays or hash, or even a sub-document (embed document ). In this way, you can implement the denormalizing data model to improve the query speed. Figure 1 MongoDB is a Schema-free document database. Figure 2 is an example. Works and comments can be designed as a collection. comments are embedded as sub-documents in the comments attribute of art, comments are embedded in the replies attribute as subdocuments of comment sub-documents. According to this design pattern, you only need to retrieve all the relevant information by file id. In MongoDB, we do not emphasize that data must be Normalize. In many cases, we recommend De-normalize. developers can discard the limitations of various paradigms of traditional relational databases, you do not need to map all objects to a Collection. You only need to define the top class. The document model of MongoDB allows us to easily map our own objects to collections for storage. Figure 2 MongoDB supports the simple and easy-to-use query method for Embedded sub-documents: the query in MongoDB is quite comfortable, and the JSON is directly used without the SQL-hard-to-remember syntax, which is quite intuitive. For different development languages, you can use its most basic array or hash format for query. With the added operator, MongoDB supports range query, regular expression query, and subdocument attribute query, which can replace the SQL query of most previous tasks. CRUD is simpler and supports in-place update: you only need to define an array and the insert/update method passed to MongoDB can be automatically inserted or updated. For the update mode, mongoDB supports an upsert option, that is, "If a record exists, it will be updated; otherwise, it will be inserted ". MongoDB's update method also supports Modifier, which allows immediate updates on the server end, saving communication between the client and the server end. These modifer allows MongoDB to have a function similar to Redis, Memcached, and other KV features: Compared with MySQL, MonoDB is simpler and faster.
