Use the earlier version of Xtrabackup to restore the full backup file created by the later version of Xtrabackup.

Use the earlier version of Xtrabackup to restore the full backup file created by the later version of Xtrabackup.
Recently, we want to restore the data of multiple MySQL servers backed up by xtrabackup to another MySQL Server and start multiple instances using different ports as the review environment. Several database instances fail to be started when the utility executes auto-Restore. Check the error log in the data directory and find the following startup error:

2015-02-02 12:31:36 27876 [Note] Plugin 'FEDERATED' is disabled.2015-02-02 12:31:36 27876 [Note] InnoDB: The InnoDB memory heap is disabled2015-02-02 12:31:36 27876 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins2015-02-02 12:31:36 27876 [Note] InnoDB: Compressed tables use zlib 1.2.32015-02-02 12:31:36 27876 [Note] InnoDB: Using CPU crc32 instructions2015-02-02 12:31:36 27876 [Note] InnoDB: Initializing buffer pool, size = 500.0M2015-02-02 12:31:36 27876 [Note] InnoDB: Completed initialization of buffer pool2015-02-02 12:31:36 27876 [Note] InnoDB: Highest supported file format is Barracuda.2015-02-02 12:31:36 27876 [Note] InnoDB: The log sequence numbers 13637542590703 and 13637542590703 in ibdata files do not match the log sequence number 13637542595176 in the ib_logfiles!2015-02-02 12:31:36 27876 [Note] InnoDB: Database was not shutdown normally!2015-02-02 12:31:36 27876 [Note] InnoDB: Starting crash recovery.2015-02-02 12:31:36 27876 [Note] InnoDB: Reading tablespace information from the .ibd files...2015-02-02 12:31:37 27876 [Note] InnoDB: Restoring possible half-written data pages2015-02-02 12:31:37 27876 [Note] InnoDB: from the doublewrite buffer...2015-02-02 12:31:37 2aeb2d96a590 InnoDB: Error: page 7 log sequence number 36755345241838InnoDB: is in the future! Current system log sequence number 13637542595176.InnoDB: Your database may be corrupt or you may have copied the InnoDBInnoDB: tablespace but not the InnoDB log files. SeeInnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.htmlInnoDB: for more information.2015-02-02 12:31:37 2aeb2d96a590 InnoDB: Error: page 1 log sequence number 35468208055287InnoDB: is in the future! Current system log sequence number 13637542595176.InnoDB: Your database may be corrupt or you may have copied the InnoDBInnoDB: tablespace but not the InnoDB log files. SeeInnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.htmlInnoDB: for more information.……

It means that the log serial number in the ibdata file is inconsistent with the log serial number in the ib_logfiles file, and the crash recovery process is started. The tablespace information is read from the. ibd file and the data page is restored from the dual-write buffer. However, the log serial number on the page is later than the current system log serial number. Indicates that the database may be damaged. Or only the tablespace is backup and no log files are backup. Therefore, the hypothetical error may occur in two phases: one is that the backup file itself is damaged, and the other is that the backup file itself is normal, and the application log is faulty.

Verify from a simple situation
Manually pull the xtrabackup full backup file from the previous day to perform the restoration step by step. The following error is reported:

……150202 12:36:35  innobackupex: Connecting to MySQL server with DSN 'dbi:mysql:;mysql_read_default_file=/etc/my.cnf;mysql_read_default_group=xtrabackup' as 'usvr_serveradmin'  (using password: YES).150202 12:36:35  innobackupex: Connected to MySQL serverConnected successfullyCan't call method "disconnect" on an undefined value at /usr/bin/innobackupex line 1482.……

Normally, when the output is complete OK! So it was determined that there was a problem. All machines use the same restoration program for restoration. Why are there only a few such problems? As you can see, these machines are installed with MySQL version 5.6.21 and xtrabackup version 2.2.24. The MySQL Server instance to be restored is 5.6.12, And the xtrabackup version is 2.1.3.

Replace 2.1.3 with xtrabackup2.2.24 and then apply logs to the full backup file again. So the final problem is located here. That is to say, if you use a later version of xtrabackup to back up the full backup file, there will be problems when you use the earlier version of xtrabackup application logs. (It must be related to the MySQL Server version)

However, the problem persists. There are a few MySQL5.6.21 machines in some other machines, and the xtrabackup version is 2.2.24. Why is the restoration successful ?? Observe the databases of these machines and find that the databases are empty, so that there is no situation for full backup application logs. Therefore, the full backup recovery is successful.

To sum up, there is a problem when using the xtrabackup application logs of a later version.