InnoDB main data structure and call flow
InnoDB is a common data engine in MySQL. This article will analyze the INNODB key data structure and call process from the source level.
- Main data structure (BUF0BUF.H)
- Buf_pool
Buf_pool is the core data structure in the entire buffer system, and all operations in the database are reflected in this buffer layer. We can specify the size of the buffer pool in the configuration file (innodb_buffer_pool_size).
The Buffer pool also contains multi-layer data structures: In order to realize the online sizing of Buffer_pool, the chunk data structure is introduced;
?
- Chunk
The chunk data structure is a more specific memory cache, which consists mainly of block structures that control memory blocks.
? Block
The main data structure of block is page and frame, which are used to store the database page on the hard disk. When the hard disk storage is not using compression, the data is read into the frame, and when compression is used, it is stored in the page.zip.
The page structure must be placed in the first position of the struct, making it easy to convert between page and block when using pointers later.
For example:(buf_page_t *) buf_block_t *p
You can get a pointer of type Buf_page
?
- Page
The page structure is primarily to store files on the hard disk, which is unique to the database file on the hard disk via space and offset; Io_fix indicates the type of the page (read or write); Zip stores the compressed page.
- Buf_pool
- Io_threads
Within InnoDB, there are 10 threads when using the default configuration parameters, where:
4 read threads, 4 write threads, are responsible for asynchronous read and write operations.
1 Log Write threads responsible for logging operations into the log file
1 Srv_master threads that are responsible for writing data to disk on a timed basis.
When the InnoDB plug-in is initialized, Innobase_start_or_create_for_mysql is called, and this function completes the initialization of the InnoDB storage engine. Create part of IO threads in function Yes
os_thread_create(io_handler_thread, n + i, thread_ids + i)
for (i = 0;; i++) {
fil_aio_wait(segment);
Mutex_enter (&ios_mutex);
ios++;
Mutex_exit (&ios_mutex);
}
The main function of fil_aio_wait is Os_aio_simulated_handle, which simulates asynchronous IO settings based on the number of threads in the InnoDB.
Segment = Os_aio_get_array_and_local_segment (&array, global_segment);
AIO and Sync Aio
When MySQL was born, there was no AIO mechanism on Linux, so InnoDB himself implemented a set of asynchronous IO frameworks.
For an introduction to Linux AIO, AIO first entered the Linux kernel in 2.5 and was now a standard feature of 2.6 production kernels. http://www.ibm.com/developerworks/linux/library/l-async/
Recently, Innobase has realized the real AIO mechanism in InnoDB plugin. Http://blogs.innodb.com/wp/2010/04/innodb-performance-aio-linux/we will introduce him in a later time.
- Arrays
- Os_aio_read_array//asynchronous read
- Os_aio_write_array//Asynchronous Write
- Os_aio_ibuf_array//insert Bufer
- Os_aio_log_array//log array
- Os_aio_sync_array//Synchronous IO Array
Sync Aio was introduced in InnoDB, with the goal of maintaining consistency with AIO at the code level.
In order to implement the array introduced by InnoDB asynchronous Io, there are altogether five types of arrays in the system:
When an IO request occurs, the asynchronous mechanism inserts the request into one slot (slot) and waits for the requested signal to be executed, and the IO is completed and removed from the slot.
?
- Segment
In array, segment is used as an identifier for the array, and according to the segment value, the type of array can be obtained. For example, the default number of arrays (4 read,4 Write) 3rd indicates the read array
?
- Slots
The slot that holds the asynchronous read and write request.
- Invoke process
In InnoDB, in order to improve the response speed, sometimes need to return data immediately, at this time using synchronous reading mechanism, while others are not very high speed requirements, such as pre-read, InnoDB will be asynchronous read to load the data into memory. In the case where double write buffer is turned on, InnoDB writes the data to a section of the hard disk called a double write buffer, and then writes to disk at some point in the future, depending on the current load of the system, There are different latencies for the number of dirty pages in the current memory.
Wait for the request when asynchronous read and write:
When a cache block/file is read and written, a signal is sent:
fil_io() -> os_aio()->
os_aio_simulated_wake_handler_threads()->os_aio_simulated_wake_handler_thread->
os_event_set()->pthread_cond_broadcast()
InnoDB main data structure and call flow
