is MySQL purge log simple?

is MySQL purge log simple?

simple: The Purge log function is simple, purge master logs to specify a binlog, then delete the Binlog file on the previous file system and update the Binlog.index file.

Complex: need to guarantee purge atomic operation, how to guarantee that there is a crash recovery?

I didn't read the code before and thought about the problem with the code implementation:

1: If you delete more files, how to balance the lock time?
2: How do I ensure that OS files are deleted and the index files are synchronized?
Question 3: How to guarantee that the index file write is complete?
Question 4: What happens if the index is updated, the file is not deleted, or the reverse occurs?
5: How do I finally clean up the first n bytes of the index file and then change the file size?

In general, that is, in the absence of a transaction engine to guarantee a series of operations (purge log includes multi-step operation), how to ensure acid or what trade-offs?

1. Purge Log test:

Purge master logs to ' binlog.000012 ';

2. The main data structure:

struct st_log_info{char  log_file_name[fn_reflen];my_off_t Index_file_offset, Index_file_start_ offset;my_off_t Pos; BOOL  lock;}

3. Brief steps for Purge:

　　

Step 1: Add the binlog.000012 to the command line as an absolute path file.
Function: Mysql_bin_log::make_log_name
Based on the current log_file_name, the length of the directory is computed, and the complete To_log is generated at the end.

Step 2: Find the binlog.000012 file in the Binlog.index file.
Function: Find_log_pos: If you do not find this file, you will get an error. For example, if the text file is not formatted, it cannot be deleted.

Step 3: Initialize the IO cache of the index file. Finally find the position of binlog.000012 at index.

Step 4: Create purge Index temp file
Create binlog.~rec~ Save the Binlog file that you want to delete.

Step 5: Re-init Index_file, read from the starting position. Matches each binlog, written to the purge index file.

Step 6: Write to purge index file and refresh
1. Flush IO writes the data in memory to the file system standard IO cache.
2. Sync purge_index_file, use Fdatasync. persisted to disk.

Step 7: Update the index file.
1. Mysql_file_seek,
2. Mysql_file_read,
3. Mysql_file_seek,
4. Mysql_file_write
Deletes the first n bytes of a file using 1->2->3->4 's timing.
5. Ftruncate change the size of the index file

Step 8: Delete the files on the OS and delete the purge index log temp file

4. Now take a look at these questions:

Question 1:
The Binlog.index file is share, and if the entire process of purge is locked, the main library will not be able to acquire the lock when flush log.
So using a policy like copy on write, after releasing the lock, asynchronously slowly deletes the file on the OS.

That is to ensure that the index file lock full write, and there is no long time to affect the main library switch Binlog log.

Question 2:
Because of the asynchronous operation of OS file deletion, it is not possible to guarantee the synchronization of index file deletion and OS file deletion. This relies on the purge index file
To complete synchronization and crash recovery.

The problem with writing a 3:index file is that there is no guarantee that a row is a full Binlog file, which relies on the file system.
But the index file is a text file, and when purge parsing error, operations can modify the file to fix it.

Note: If it is a binary file, is this a trade-off feasible, I am afraid to use the transaction engine to ensure.

Question 4:
1. If the index file is updated, the OS file is not deleted
When MySQL crash resumes, it reads purge index file to continue deleting
2. If the file is deleted, the index file is not updated
From the above steps, it is not allowed to happen in this case.

Question 5:
The file deletes the previous n-byte problem, itself can use another strategy, create a new table, copy data, rename.
But Binlog.index is share and does not allow online rename, so the process of using step 7 is done on a file handler.

Trade- Offs: index file as a data dictionary for MySQL binlog files, first of all to ensure the consistency of the data, and the OS files can be removed to make a trade-offs.

Do not know when the implementation of the command is simple.

It's a little more complicated when you think about the realization of the problem.

After reading the source code, I think it is more complicated.