MySQL Series: InnoDB source analysis Redo Log Recovery

In the previous article "InnoDB Source analysis of the redo log structure" We know the basic structure of redo log and log writing steps, then redo log is how to recover data? When is the log deduction for redo log? Redo log deduction only after the database is abnormal or shut down, the database restarts when the log deduction, the database state back to the state before the shutdown. So how does this process proceed? Let's step through the analysis.

1.RECV_SYS_T structureInnoDB When MySQL is started, redo log files will be redo log, redo log is through a recv_sys_t structure for data recoverycomplex and controlled. It is structured as follows:

struct recv_sys_struct{mutex_t mutex;                        /* Protection Lock */Ibool apply_log_recs;                     /* The log record is being applied to page */Ibool apply_batch_on; /* Bulk application of log record flag */Dulint LSN; Ulint last_log_buf_size;                             Byte* Last_block;             /* Restore the last block memory buffer */byte* Last_block_buf_start;                                        /* The starting position of the last block memory buffer, since LAST_BLOCK is 512 address aligned, this variable is required to record the address location of the free */byte* buf; /* Redo log information data read from the log block */Ulint Len;                       /*buf valid log data length */Dulint PARSE_START_LSN;                           /* Start the parse lsn*/dulint scanned_lsn;          /* The LSN sequence number has been scanned */Ulint scanned_checkpoint_no;                       /* Checkpoint serial number of the recovery log */Ulint recovered_offset;                         /* Offset to recovery position */Dulint RECOVERED_LSN;                                  /* Recovery of LSN location */Dulint LIMIT_LSN;                   /* Log recovery maximum LSN, temporarily in the process of log redo did not use */Ibool found_corrupt_log; /* Whether to turn on log recovery diagnostics */log_group_t*archive_group;    mem_heap_t* Heap;                         /*RECV SYS's memory allocation heap, used to manage the memory footprint of the recovery process */hash_table_t*addr_hash;                                        /*RECV_ADDR hash table, with Space ID and page no for key*/ulint N_addrs; /*addr_hash contains the number of recv_addr *};

In this structure, the more complex is addr_hash this hash table, this hash table is used sapce_id and page_no as a hash key, which stores the corresponding record content in the recovery. After the recovery log is read from the log file, it is parsed into several recv_t and stored in a hash table. After a read parsing cycle, log recovery writes data from the recv_t in the hash table to the IBUF and page. Why do we use hash tables here? Personally think it is for the same page of data batch recovery, so that you can reduce the page random insertion and modification. Here are a few data structures related to this process:

/* Set of data recovery operations for the corresponding page */struct recv_addr_struct{ulint state;          /* Status, Recv_not_processed, recv_being_processed, recv_processed*/ulint space;         /*space of id*/ulint page_no;    /* Page number */ut_list_base_node_t (recv_t) rec_list; hash_node_t addr_hash;};/ * Current record operation */struct recv_struct{byte type;             /*log type */Ulint len;               /* Current Record Data length */recv_data_t*data; /* Current record data list*/dulint START_LSN;     /*mtr initiation lsn*/dulint end_lsn;      /*mtr End lns*/ut_list_node_t (recv_t) rec_list;};/ * Specific data body */struct recv_data_struct  {recv_data_t*next;/* The next Recv_data_t,next address is followed by a large chunk of memory, used to store rec body*/};

Their memory relationship structure is shown below:
2. LSN relationship of redo log deduction processIn addition to this restored hash table, the various LSN in the recv_sys_t is closely related to log recovery. The following are explanations of the various LSN: PARSE_START_LSN This log redo the start of the LSN, if the recovery from checkpoint, is equal to CHECKPOINT_LSN. SCANNED_LSN In the recovery process, the recovery log is stored recv_sys->buf log LSN from the LOG_SYS->BUF parsing block. RECOVERED_LSN has restored the data to the page or has stored log operations in the Addr_hash log LSN;When the log begins to resume:
PARSE_START_LSN = SCANNED_LSN = Recovered_lsn = LSN of the checkpoint.
When the log completes recovery:
PARSE_START_LSN = LSN of checkpoint
SCANNED_LSN = RECOVERED_LSN = Log_sys->lsn.
in the log deduction process, the LSN size relationship is as follows:

3. Main interfaces and processes for log recoveryrecovery Log Main interface functions: Recv_recovery_from_checkpoint_start recovering data from the most recent checkpoint in the Redo log group
Recv_recovery_from_checkpoint_finish End data recovery operation from checkpoint in redo log group
Recv_recovery_from_archive_start data recovery from an archive log file
Recv_recovery_from_archive_finish End Data recovery operation from archive log
Recv_reset_logsintercepts the last section of the redo log as the starting position for the new redo log and may lose data。
Redo Log Recovery Data flow (checkpoint mode)1. When MySQL starts, it first reads out the last-saved LSN from the database file.
2. Then call Recv_recovery_from_checkpoint_start and pass the largest LSN as a parameter into the function.
3. The function will recently establish the Checkpoint log group and read out the corresponding checkpoint information
4. Compare by the checkpoint LSN and the maximum LSN passed in, if equal, do not log recovery data, if not equal, log recovery.
5. The archive archive status of each log group is synchronized before starting the recovery
6. At the beginning of the recovery, the log file will be read from the 2M log data to Log_sys->buf, and then scan the 2M data, verify its legitimacy, and then remove the block header log into the Recv_sys->buf, This process, called scan, changes the scanned LSN.
7. After a scan of the log data for 2M, InnoDB will parse the log for MTR operations and perform the relevant MTR functions. If the MTR is legal, the corresponding record data will be credited to Recv_sys->addr_hash as a key in space page_no.
8. When the log data for scan is resolved by MTR, InnoDB will call Recv_apply_hashed_log_recs to scan the entire recv_sys->addr_hash. And according to the operation of the log corresponding page data recovery. This process will change the RECOVERED_LSN.
9. If after completing the 8th step, the 2M data is read again from the log group file, skip to step 6 to continue the corresponding processing until the log file does not need to recover the log data.
10.innodb after recovering the data in the log file, the Recv_recovery_from_checkpoint_finish end log recovery operation is called, primarily freeing up some open memory. and the processing of transactions and Binlog.
The above process is as follows:

MySQL Series: InnoDB source analysis Redo Log Recovery