Mysql log file usage and data recovery bitsCN.com
Use and data recovery of mysql log files
I. mysql log type
MYSQL has different types of log files (each storing different types of logs), from which you can query what MYSQL has done and manage MYSQL, these log files are indispensable.
1. The error log records The error information during database startup, operation, and stop;
2. ISAM operation log (The isam log): records all changes to The ISAM table. This log is only used to debug The ISAM mode;
3. SQL execution log (The query log): records The client connection and The executed SQL statements;
4. update log: The statement that records data changes. it is not recommended to use this log instead of binary logs;
5. binary log: records all statements for modifying database data;
6. The time-out log (The slow log): records all statements whose execution time exceeds The maximum SQL execution time (long_query_time) or where no index is used;
7. relay log: If you are using mysql's copy and backup functions, the slave server also provides a log file called relay log.
By default, all log files are recorded in the MYSQL data directory. you can force mysql to close and re-open a file to record the logs. of course, the system will automatically add a suffix (such. 00001 ,. 00002). you can run the mysql> flush logs statement in the mysql environment or run the mysqladmin management program # mysqladmin flush-logs or # mysqladmin refresh.
II. mysql log configuration
The startup method of these logs can be followed by the option parameters when the mysqld_safe method is used to start the database, or configured in the configuration file. The second method is recommended. the configuration method is very simple, I only configured three types of logs:
[Mysqld]
Log =/var/log/mysqld_common.log
Log-error =/var/log/mysqld_err.log
Log-bin =/var/log/mysqld_bin.bin
III. View mysql logs
Log viewing is very simple. most of the logs are text. you can directly view the logs using tools such as vim, less, and more. it is worth noting that you can view the binary files:
1. first determine whether the binary file record function is enabled
SQL code
Mysql> show variables like 'log _ bin ';
2. if you want to know the details of the file that records binary data, you can use the following statement to see which file is being recorded and the current location of the record:
SQL code
Mysql> show master status;
3. you need to use the program mysqlbinlog to view the binary data and use it as needed.
SQL code
Mysqlbinlog/var/log/mysql/mysql-bin.000040;
4. export the binary log file to the txt text file for viewing.
SQL code
Mysql> mysqlbinlog/var/log/mysql/mysql-bin.000040>/var/log/mysql/41540.txt;
5. if you query a time range, you can execute the following statements. if there are many records, you can direct the results to a file and read them :-):
SQL code
Mysql> mysqlbinlog -- start-datetime = '2017-01-01 00:00:00 '-- stop-datetime = '2017-08-08 00:00:00'/var/log/mysql/mysql-bin.000040>. /tmp. log
4. use binary logs to restore data
For the use of the mysqlbinlog tool, you can refer to the MySQL help manual, which provides detailed use. In this example, the focus is on the use of the -- start-position parameter and -- stop-position parameter.
-- Start-position = N: read from the event where the position in the binary log is equal to N.
-- Stop-position = N: stop reading from an event where the position in the binary log is equal to or greater than N.
1. create an experiment environment
In a test database, create a table, add records, and generate a log file.
SQL code
Mysql> create table test (id int auto_increment not null primary key, val int, data varchar (20 ));
Mysql> insert into test (val, data) values (10, 'Shanghai ');
Query OK, 1 row affected (0.03 sec)
Mysql> insert into test (val, data) values (20, 'Jia ');
Query OK, 1 row affected (0.08 sec)
Mysql> insert into test (val, data) values (30, 'hui ');
Query OK, 1 row affected (0.03 sec)
Mysql> flush logs; -- generate the second log file
Query OK, 0 rows affected (0.09 sec)
Mysql> insert into test (val, data) values (40, 'AAA ');
Query OK, 1 row affected (0.05 sec)
Mysql> insert into test (val, data) values (50, 'BBB ');
Query OK, 1 row affected (0.03 sec)
Mysql> insert into test (val, data) values (60, 'CCC ');
Query OK, 1 row affected (0.03 sec)
Mysql> delete from test where id between 4 and 5; -- delete a record
Query OK, 2 rows affected (0.05 sec)
Mysql> insert into test (val, data) values (70, 'ddd ');
Query OK, 1 row affected (0.03 sec)
Mysql> flush logs; -- Generate the third file
Query OK, 0 rows affected (0.11 sec)
Mysql> insert into test (val, data) values (80, 'ddddd ');
Query OK, 1 row affected (0.05 sec)
Mysql> insert into test (val, data) values (90, 'eeeee ');
Query OK, 1 row affected (0.03 sec)
Mysql> drop table test; -- delete a table
Query OK, 0 row affected (0.05 sec)
2. restore data
Use mysqlbinlog to generate a txt file for analysis.
SQL code
Mysqlbinlog/var/log/mysql/mysql-bin.000001>/var/log/mysql/000001.txt;
Mysqlbinlog/var/log/mysql/mysql-bin.000002>/var/log/mysql/000002.txt;
Mysqlbinlog/var/log/mysql/mysql-bin.000003>/var/log/mysql/000003.txt;
With these three commands, you can generate the log file content, that is, the user operation steps.
Because we need to redo all the operations on the first log file, we only need to restore the first log file.
SQL code
Mysql & gt; mysqlbinlog/var/log/mysql/mysql-bin.000001 | mysql-uroot-p
OK. Next, we need to analyze the second log file. Why do we need to analyze it, because it executes a DELETE operation halfway, because what we need to do is restore all the data, that is, we do not want to redo this statement. So here we have to find a way to bypass it.
Open the. txt file for analysis.
SQL code
/*
/*! 40019 SET @ session. max_insert_delayed_threads = 0 */;
/*! 50003 SET @ OLD_COMPLETION_TYPE = @ COMPLETION_TYPE, COMPLETION_TYPE = 0 */;
DELIMITER /*! */;
# At 4
#090427 15:27:56 server id 1 end_log_pos 106 Start: binlog v 4, server v 5.1.32-community-log created 090427 15:27:56
Binlog'
Bytes
AAAAAAAAAAAAAAAAAAAAAAAAEzgNAAgAEgAEBAQEEgAAUwAEGggAAAAICAgC
'/*! */;
# At 106
#090427 15:28:37 server id 1 end_log_pos 176 Query thread_id = 1 exec_time = 0 error_code = 0
Use mytest /*! */;
Set timestamp = 1240817317 /*! */;
SET @ session. pseudo do_thread_id = 1 /*! */;
SET @ session. foreign_key_checks = 1, @ session. SQL _auto_is_null = 1, @ session. unique_checks = 1, @ session. autocommit = 1 /*! */;
SET @ session. SQL _mode = 1344274432 /*! */;
SET @ session. auto_increment_increment = 1, @ session. auto_increment_offset = 1 /*! */;
/*! /C gbk *//*! */;
SET @ session. character_set_client = 28, @ session. collation_connection = 28, @ session. collation_server = 28 /*! */;
SET @ session. lc_time_names = 0 /*! */;
SET @ session. collation_database = DEFAULT /*! */;
BEGIN
/*! */;
# At 176
#090427 15:28:37 server id 1 end_log_pos 204 Intvar
SET INSERT_ID = 4 /*! */;
# At 204
#090427 15:28:37 server id 1 end_log_pos 312 Query thread_id = 1 exec_time = 0 error_code = 0
Set timestamp = 1240817317 /*! */;
Insert into test (val, data) values (40, 'AAA ')
/*! */;
# At 312
#090427 15:28:37 server id 1 end_log_pos 339 Xid = 12
COMMIT /*! */;
# At 339
#090427 15:28:46 server id 1 end_log_pos 409 Query thread_id = 1 exec_time = 0 error_code = 0
Set timestamp = 1240817326 /*! */;
BEGIN
/*! */;
# At 409
#090427 15:28:46 server id 1 end_log_pos 437 Intvar
SET INSERT_ID = 5 /*! */;
# At 437
#090427 15:28:46 server id 1 end_log_pos 545 Query thread_id = 1 exec_time = 0 error_code = 0
Set timestamp = 1240817326 /*! */;
Insert into test (val, data) values (50, 'BBB ')
/*! */;
# At 545
#090427 15:28:46 server id 1 end_log_pos 572 Xid = 13
COMMIT /*! */;
# At 572
#090427 15:29:35 server id 1 end_log_pos 642 Query thread_id = 1 exec_time = 0 error_code = 0
Set timestamp = 1240817375 /*! */;
BEGIN
/*! */;
# At 642
#090427 15:29:35 server id 1 end_log_pos 670 Intvar
SET INSERT_ID = 6 /*! */;
# At 670
#090427 15:29:35 server id 1 end_log_pos 778 Query thread_id = 1 exec_time = 0 error_code = 0
Set timestamp = 1240817375 /*! */;
Insert into test (val, data) values (60, 'CCC ')
/*! */;
# At 778
#090427 15:29:35 server id 1 end_log_pos 805 Xid = 14
COMMIT /*! */;
# At 805
#090427 15:30:21 server id 1 end_log_pos 875 Query thread_id = 1 exec_time = 0 error_code = 0
Set timestamp = 1240817421 /*! */;
BEGIN
/*! */;
# At 875
#090427 15:30:21 server id 1 end_log_pos 981 Query thread_id = 1 exec_time = 0 error_code = 0
Set timestamp = 1240817421 /*! */;
Delete from test where id between 4 and 5
/*! */;
# At 981
#090427 15:30:21 server id 1 end_log_pos 1008 Xid = 15
COMMIT /*! */;
# At 1008
#090427 15:30:34 server id 1 end_log_pos 1078 Query thread_id = 1 exec_time = 0 error_code = 0
Set timestamp = 1240817434 /*! */;
BEGIN
/*! */;
# At 1078
#090427 15:30:34 server id 1 end_log_pos 1106 Intvar
SET INSERT_ID = 7 /*! */;
# At 1106
#090427 15:30:34 server id 1 end_log_pos 1214 Query thread_id = 1 exec_time = 0 error_code = 0
Set timestamp = 1240817434 /*! */;
Insert into test (val, data) values (70, 'ddd ')
/*! */;
# At 1214
#090427 15:30:34 server id 1 end_log_pos 1241 Xid = 16
COMMIT /*! */;
# At 1241
#090427 15:30:41 server id 1 end_log_pos 1282 Rotate to mysql-bin.000003 pos: 4
DELIMITER;
# End of log file
ROLLBACK/* added by mysqlbinlog */;
/*! 50003 SET COMPLETION_TYPE = @ OLD_COMPLETION_TYPE */;