Distributed transaction log

The log guarantees the durability of the data and the atomicity of the transaction. You can simply think of a log as a file that is constantly appending log records. A single log record is a binary buffer. Here are a few common log records that are used in this article: marking the start of the Trasaction Transcatoin successfully submitted, all changes to the data have been successful. Because of the cache, seeing a commit in the log does not necessarily mean that the modification of the data has persisted. The purpose of the log is to ensure that all commit transactions are preserved in the event that the program program exits unexpectedly, and that any changes to transactions that do not commit are not preserved in the event that the program exits unexpectedly, as if these transactions do not have a start at all. None of the changes to the transaction T can be preserved, and the final manifestation of something like a transaction t does not start at all. The main types of logs are: Undo Log,redo Log,undo/redo Log. The most commonly used types are redo log and Undo/redo log. If the system does not appear abnormal, then the log is not necessary, so in the description of most of the scene refers to the system abnormal exit restart, in the narrative process is not repeated description of the scene premise.

UNDO LOG Log Contents

The Undo log rolls back all transactions without a commit to the state before the transaction started and does nothing with the commit transaction. Because no processing is done for a COMMIT transaction, the transaction's modifications to the data must be persisted before the commit log is written, and if only a portion of it is persisted, the transaction modification data is in an inconsistent state. In addition, because of the need to do the undo operation, therefore, the log record must contain the value before the data modification, a single undo log, which means that the value of X is v before the transaction T starts running, and that the undo operation is required for the uncommitted transaction before the data for the database is modified. You must first ensure that the transaction log is persisted and that if the log is not persisted and the last transaction has no commit, then the data cannot be rolled back to the state before the transaction started. This concludes the two rules that must be met by the Undo log: U1:if transaction T modifies database element X, then the log record of the form must be Writte N to disk before the new value of X was written to disk. U2:if a transaction commits, then its COMMIT log record must is Witten to disk only after all database elements changed B Y the transaction have been written to disk, but as soon thereafter as possible therefore, the order of the log and the data itself to be persisted is: all the data that the transaction modifies, the reaction to the data Logging data itself commit log here we can see that the modification of the data must be persisted before each transaction commit, which can cause performance problems because each transaction brings back a sync write to the data file, and the main purpose of using the log is to reduce the disk sync operation. As a result, the undo log has a large performance problem, so there is not much use in practice.

Log Playback

The system restarts the playback log, only to scan the log file from the back forward, and to rollback the data in the log record for all transactions that do not commit.

CheckPoint

To replay the log as described above, all log files need to be scanned, and the log files cannot be deleted. But in practice, we can see that if a transaction has already been commit, the previous log record can be done without playback. Therefore, the concept of checkpoint is introduced.

The simplest checkpoint approach is to block all updates while doing checkpoint until all pending transactions are abort and recorded. When the log is replayed, stop replaying the log if you encounter it. Blocking updates is certainly not an elegant way to handle it. A checkpoint method that does not block updates is described below.

The process of generating checkpoint is: Record start_ckpt<t1,t2,..., tn>, where ti indicates that pending transactions (no committed transactions) are waiting for all transactions to be committed while the checkpoint is starting to be generated end_ckpt

Log playback, if you first encounter end_ckpt, only need to replay the log to the next start_ckpt so far, start_ckpt before the log can be discarded; if the first encounter is start_ckpt, just play back all the transactions recorded in the first step, the first place to start, The previous log records can be deleted directly.

REDO LOG

Redo log is to redo a transaction that has already been made while replaying the log, and to deal with a transaction without a commit according to abort. Log playback does not handle any transactions that do not commit, so the modification of the data cannot be persisted until the commit log is persisted. Because if the data is persisted before a commit, this partially modified transaction is in an inconsistent state when the system exits unexpectedly. Also, because of the redo transaction, the transaction log must record the value of the transaction after the modification. The redo log must meet the following rules: R1:before modifying any database element X on disk, which is necessary this all log records pertaining[attached to To this modification of X. including both the update record and the record must appear on disk.

Redo log and data modifications in order: Record changes to the database (update log) write commit log Modify database entries

log playback and checkpoint for log playback with no checkpoint: First scan the log from the back: Record all the transactions that have been commit, append the abort to the end of the log for transactions that do not commit, and then scan the log from the Redo all the transaction logs that have been commit.

Redo Log Generation Checkpoint method: Write Log start_ckpt<t1, T2, ..., tn>; where TI indicates that a transaction that is currently in progress that has not been commit has persisted the modification of all the commit transactions to the database data. This requires a transaction to be written to the cache after committing a commit log record. Write Log End_ckpt

When we scan the log file from the back, if we encounter end_ckpt, we know that all the commit transactions before start_ckpt need not be redo, so the log can be scanned from the back to the Start_ The oldest transaction in an ongoing transaction that is recorded in Ckpt. Of course, it is possible to string up all the logs of a transaction in a particular way, reducing the number of scans for the log.

The simplified example of redo log Redlog is widely used in NoSQL system, the main reason is simple implementation. In the case of a row-level transaction, a transaction degrades to only one log, and the operation is the simplest: write Log update database

The problem with this is how the steps to update the database fail. Here are two ways: use Copy-on-write for transaction preprocessing (pre-update is performed prior to writing the log) to ensure that the update database is guaranteed to succeed in the event that the database failed to update, the program exits. Obviously, the first approach is more graceful and harmonious.

Transaction preprocessing is mentioned here, which is a key technology for optimizing transaction performance. Since the operations on transactions are often sequential, the TPS of the system cannot be improved if the reading of the data is handled in a deliberate manner. The simple transaction preprocessing is to read the data in parallel, to increase the concurrency of the transaction, and to improve the TPS of the system, the real memory operation can be done in addition to the log in the transaction processing.

For this simplified version of the redo log checkpoint is very simple: in memory generation of snapshot, there are many techniques can be generated in the instantaneous snapshot, the generation of snapshot log point (checkpoint) Persisting the generated snapshot record checkpoint log points

In engineering practice, many technical points need to be easy to rely on.

Undo/redo Log As mentioned earlier, the drawback of the UNDO log is obvious: it is obvious that no one can accept the fact that all of the database data that the transaction involves after it has been completed is immediately persisted. The redo log is widely used in the NoSQL system, but it still has its drawbacks when dealing with complex transactions: it is necessary to update the data in a transaction-local buffer because the transaction is committed before the real commit, which may cause the transaction to occupy a large amount of memory.

In addition, when solving the problem of distributed consistency, it is true that there is a need to roll back, this time redo log can do nothing. Specific examples of this site to chubby a possible implementation of the analysis.

and Undo/redo log can solve these problems very well. Undo/redo log is the rollback of all no commit transactions, as in the case of the undo log when the log is replayed, and redo log redo all the transactions that have been commit. Because both redo and undo are required, both the modified value and the modified value must be recorded in the log record. <t, X, V, W>, which indicates that transaction T modifies the contents of element x, the value of x before modification is V, and the value is w after modification. At the same time, Undo/redo log requires very low data and log persistence Order: URl before modifying any database element X on disk because of changes made by some Nsaction T, it is necessary this update record <t, X, V., w> appear on disk It means that you must write the transaction update log before you read the database for modification. Undo/redo log is also called the write ahead log.

The process of logging and Data Update: Log update log updating database (not necessarily persisted, write cache) write commit log at appropriate times

log playback and checkpoint No checkpoint log playback is simple: first iterate through all the logs, undo all uncommitted transactions, append abort, record all committed transactions, and then traverse the log from the time you go. Redo all transactions that have been committed.

The checkpoint way of Redo/undo log is similar to redo log: Write log start_ckpt<t1, T2, ..., tn>; where TI indicates that a transaction that is currently in progress without a commit persists all dirty data in the cache to disk Write Log End_ckpt

The difference from the redo log checkpoint is that in the second step, Undo/redo log can persist all data to disk, including transactions that have not yet been commit, and redo Log will only be able to persist the dirty data of all the transactions that have been commit to disk in the second step.

In Database systems:the Complete book, the checkpoint of Undo/redo log has one such description: A transaction must not write any values (Eve N to memory buffers) until it are certain not to abort. The author has not yet understood the meaning of this paragraph, welcome to discuss. My understanding is that since the second step in generating checkpoint is to persist all dirty data to disk, then all updates (even if no commit transactions are updated) are seen by other transactions, and if a transaction without a commit finally abort, This allows other transactions to see inconsistent data.  If my understanding is correct, then Undo/redo log and redo log all updates to the data must be in the transaction-local buffer before the transaction is actually committed, then Undo/redo Log has no advantage over redo log. So what are the advantages of Undo/redo log relative to redo log in a stand-alone environment?