Take a note of the xtrabackup code

Take a note of the xtrabackup code. The two most important processes of xtrabackup binary are backup and prepare. the corresponding functions are xtrabackup_backup_func () and xtrabackup_prepare_func.

Thread model of xtrabackup backup:

1. a log copy thread;

2. n ibd file copy threads;

3. an I/O monitoring thread;

4. use the suspend_start/suspend_end file to indicate whether to start the termination thread;

Typedef struct {

Datafiles_iter_t * it;

Uint num;

Uint * count;

OS _ib_mutex_t count_mutex;

OS _thread_id_t id;

} Data_thread_ctxt_t;

Switch the working directory of the data thread context;

116/** Set if InnoDB must operate in read-only mode. We don't do any

117 recovery and open all tables in RO mode instead of RW mode. We don't

118 sync the max trx id to disk either .*/

Xb_set_innodb_read_only (): Set innodb to read-only mode.

Srv_backup_mode = TRUE; set innodb to backup mode;

Set a series of innodb parameters

Innodb_init_param ()

Xb_normalize_init_values (void)

Modify srv_unix_file_flush_method

Adjust the srv_max_n_threads parameter based on the bp size.

1017 /************************************** *******************************//**

1018 Initializes the synchronization primitives, memory system, and the thread

1019 local storage .*/

Srv_general_init ()

Ut_crc32_init ()

Xb_filters_init ()

2567 /************************************** **********************************

2568 Initializes the I/O and tablespace cache subsystems .*/

Xb_fil_io_init (void)

838 /************************************** ****************//**

839 Initializes the log .*/

Log_init (void)

585 /************************************** *******************************//**

586 Creates the lock system at database start .*/

Lock_sys_create ()

Open_or_create_log_file

Create xtrabackup_extra_lsndir/extrabackup_traget_dir

Tablespace memory cache

Fil_system_t * f_system = fil_system;

Recv_find_max_checkpoint (& max_cp_group, & max_cp_field)

Log_group_read_checkpoint_info (max_cp_group, max_cp_field)

Checkpoint_lsn_start/checkpoint_no_start

Confirm the consistent checkpoint status;

Create an XB_LOG_FILENAME file and write the file header information;

Create io_watching_thread;

Copy log files from the checkpoint position;

Xtrabackup_copy_logfile (checkpoint_lsn_start, FALSE)

Log_copying/log_copying_stop

Create a log copy thread

OS _thread_create (log_copying_thread, NULL, & log_copying_thread_id );

2591 /************************************** **************************************

2592 Populates the tablespace memory cache by scanning for and opening data files.

2593 @ returns DB_SUCCESS or error code .*/

Xb_load_tablespaces ()

Pending. wait until the XB_FN_SUSPENDED_AT_START file is deleted.

Xtrabackup_suspend

Xb_page_bitmap_init ()

Create a data_copy_thread_func thread based on the xtrabackup_parallel settings

Wait until all data_copy_thread_func threads exit

Pending. wait until the XB_FN_SUSPENDED_AT_END file is deleted.

Xtrabackup_suspend

Read the latest checkpoint and record it in the to_lsn field of metadata;

Stop log_copying_thread by setting log_copying = FALSE & set log_copying_stop;

Create a file XB_FN_LOG_COPIED to notify the external script that log_copying_thread has ended;

Write metadata; Prepare_func

1. switch to xtrabackup_real_target_dir.

2. read XTRABACKUP_METADATA_FILENAME and obtain the original information;

Metadata_type

3. xtrabackup_init_temp_log ()

4. innodb_init_param ()

2670 /************************************** **********************************

2671 Initialize the tablespace memory cache and populate it by scanning for and

2672 opening data files.

2673 @ returns DB_SUCCESS or error code .*/

Xb_data_files_init ()

Application incremental to full

Xtrabackup_apply_deltas ()

Resetting innodb initialization parameters

Innodb_init_param ()

Innodb_init () file traversal

Mtr_start-> mtr_commit

Trx_sys_print_mysql_binlog_offset ()

Output the binlog location information to the xtrabackup_binlog_pos_innodb file.

Xtrabackup_close_temp_log (TRUE)

Output record metadata_log

Backup

$./Xtrabackup_56 -- defaults-file =/u01/my3928/my. cnf -- backup -- target_dir =/u01/xianlin. lh/backup_dir/

./Xtrabackup_56 version 2.1.8 for MySQL server 5.6.15 Linux (x86_64) (revision id: undefined)

Xtrabackup: uses posix_fadvise ().

Xtrabackup: cd to/u01/my3928/data

Xtrabackup: using the following InnoDB configuration:

Xtrabackup: innodb_data_home_dir =/u01/my3928/data

Xtrabackup: innodb_data_file_path = ibdata1: 4G; ibdata2: 16 M: autoextend

Xtrabackup: innodb_log_group_home_dir =/u01/my3928/data

Xtrabackup: innodb_log_files_in_group = 4

Xtrabackup: innodb_log_file_size = 1073741824

17:29:35 2ac06bc4a2c0 InnoDB: Using Linux native AIO

Xtrabackup: using O_DIRECT

> Log scanned up to (1451746590)

[01] Copying/u01/my3928/data/ibdata1 to/u01/xianlin. lh/backup_dir/ibdata1

> Log scanned up to (1451746590)

> Log scanned up to (1451746590)

> Log scanned up to (1451746590)

> Log scanned up to (1451746590)

> Log scanned up to (1451746590)

> Log scanned up to (1451746590)

> Log scanned up to (1451746590)

> Log scanned up to (1451746590)

> Log scanned up to (1451746590)

> Log scanned up to (1451746590)

> Log scanned up to (1451746590)

> Log scanned up to (1451746590)

> Log scanned up to (1451746590)

> Log scanned up to (1451746590)

> Log scanned up to (1451746590)

> Log scanned up to (1451746590)

> Log scanned up to (1451746590)

> Log scanned up to (1451746590)

[01]... done

[01] Copying/u01/my3928/data/ibdata2 to/u01/xianlin. lh/backup_dir/ibdata2

[01]... done

[01] Copying./test/t2.ibd to/u01/xianlin. lh/backup_dir/test/t2.ibd

[01]... done

[01] Copying./test/t4.ibd to/u01/xianlin. lh/backup_dir/test/t4.ibd

[01]... done

[01] Copying./test/t3.ibd to/u01/xianlin. lh/backup_dir/test/t3.ibd

[01]... done

[01] Copying./test/sbtest1.ibd to/u01/xianlin. lh/backup_dir/test/sbtest1.ibd

> Log scanned up to (1451746590)

> Log scanned up to (1451746590)

[01]... done

[01] Copying./test/t1.ibd to/u01/xianlin. lh/backup_dir/test/t1.ibd

[01]... done

[01] Copying./mysql/innodb_index_stats.ibd to/u01/xianlin. lh/backup_dir/mysql/innodb_index_stats.ibd

[01]... done

[01] Copying./mysql/slave_worker_info.ibd to/u01/xianlin. lh/backup_dir/mysql/slave_worker_info.ibd

[01]... done

[01] Copying./mysql/innodb_table_stats.ibd to/u01/xianlin. lh/backup_dir/mysql/innodb_table_stats.ibd

[01]... done

[01] Copying./mysql/slave_relay_log_info.ibd to/u01/xianlin. lh/backup_dir/mysql/slave_relay_log_info.ibd

[01]... done

[01] Copying./mysql/slave_master_info.ibd to/u01/xianlin. lh/backup_dir/mysql/slave_master_info.ibd

[01]... done

> Log scanned up to (1451746590)

Xtrabackup: The latest check point (for incremental): '123'

Xtrabackup: Stopping log copying thread.

.> Log scanned up to (1451746590)

Xtrabackup: Transaction log of lsn (1451746590) to (1451746590) was copied.

Backup creation process

Http://www.percona.com/doc/percona-xtrabackup/2.1/xtrabackup_bin/creating_a_backup.html

1 backup_type = full-backuped

2 from_lsn = 0

3 to_lsn = 1451746590

4 last_lsn = 1451746590

5 compact = 0 Prepare

$./Xtrabackup_56 -- defaults-file =/u01/my3928/my. cnf -- prepare -- target-dir =/u01/xianlin. lh/backup_dir/

./Xtrabackup_56 version 2.1.8 for MySQL server 5.6.15 Linux (x86_64) (revision id: undefined)

Xtrabackup: cd to/u01/xianlin. lh/backup_dir/

Xtrabackup: This target seems to be not prepared yet.

Xtrabackup: xtrabackup_logfile detected: size = 2097152, start_lsn = (1451746590)

Xtrabackup: using the following InnoDB configuration for recovery:

Xtrabackup: innodb_data_home_dir = ./

Xtrabackup: innodb_data_file_path = ibdata1: 4G; ibdata2: 16 M: autoextend

Xtrabackup: innodb_log_group_home_dir = ./

Xtrabackup: innodb_log_files_in_group = 1

Xtrabackup: innodb_log_file_size = 2097152

2014-05-05 18:38:32 2b7f2f5202c0 InnoDB: Using Linux native AIO

Xtrabackup: using the following InnoDB configuration for recovery:

Xtrabackup: innodb_data_home_dir = ./

Xtrabackup: innodb_data_file_path = ibdata1: 4G; ibdata2: 16 M: autoextend

Xtrabackup: innodb_log_group_home_dir = ./

Xtrabackup: innodb_log_files_in_group = 1

Xtrabackup: innodb_log_file_size = 2097152

2014-05-05 18:38:32 2b7f2f5202c0 InnoDB: Using Linux native AIO

Xtrabackup: Starting InnoDB instance for recovery.

Xtrabackup: Using 104857600 bytes for buffer pool (set by -- use-memory parameter)

InnoDB: The InnoDB memory heap is disabled

InnoDB: Mutexes and rw_locks use GCC atomic builtins

InnoDB: Compressed tables use zlib 1.2.3

InnoDB: Using Linux native AIO

InnoDB: Using CPU crc32 instructions

InnoDB: Initializing buffer pool, size = 100.0 M

InnoDB: Completed initialization of buffer pool

InnoDB: Highest supported file format is Barracuda.

InnoDB: The log sequence numbers 552524932 and 552524932 in ibdata files do not match the log sequence number 1451746590 in the ib_logfiles!

InnoDB: Database was not shutdown normally!

InnoDB: Starting crash recovery.

InnoDB: Reading tablespace information from the. ibd files...

InnoDB: Restoring possible half-written data pages

InnoDB: from the doublewrite buffer...

InnoDB: Last MySQL binlog file position 0 26767310, file name mysql-bin.000023

InnoDB: 128 rollback segment (s) are active.

InnoDB: Waiting for purge to start

18:38:33 2b7f49056700 InnoDB: Warning: table 'test/sbtest1'

InnoDB: in InnoDB data dictionary has unknown flags 50.

InnoDB: 5.6.15 started; log sequence number 1451746590

[Notice (again)]

If you use binary log and don't use any hack of group commit,

The binary log position seems to be:

InnoDB: Last MySQL binlog file position 0 26767310, file name mysql-bin.000023

Xtrabackup: starting shutdown with innodb_fast_shutdown = 1

InnoDB: FTS optimize thread exiting.

InnoDB: Starting shutdown...

InnoDB: Shutdown completed; log sequence number 1451747281

Prepare process:

Http://www.percona.com/doc/percona-xtrabackup/2.1/xtrabackup_bin/preparing_the_backup.html

1 backup_type = full-prepared

2 from_lsn = 0

3 to_lsn = 1451746590

4 last_lsn = 1451746590

5 compact = 0