MySQL series: Table space management for innodb source code analysis, mysqlinnodb
Innodb implements a logical storage space management layer based on file IO for table space. table space adopts a logical hierarchical structure: space, segment inode, extent, and page. the logic at the implementation layer uses the disk linked list structure to manage the logical relationship. Let's first introduce the disk linked list.
1. implement fut0lst for disk linked list. * In the file, innodb defines a disk-based linked list to manage tablespaces and index modules. It is mainly used to store the relationship between disk data structures. This linked list is not based on memory pointers, but based on page no and boffset for location binding. Define a fil_addr_t structure in innodb to describe it:
Typedef struct fil_addr_struct {ulintpage;/* The number of pages in space */ulintboffset;/* The Byte offset in page, which is expressed by */} fil_addr_t in memory;Fil_addr_t can use the fut_get_ptr function to obtain the memory location of the corresponding node (flst_node_t)
Flst_node_t you can use buf_ptr_get_fsp_addr to determine fil_addr_t. Flst_node_t contains 12 bytes of content. The first 6 bytes (page: 4 boffset: 2) indicate the fil_addr_t information relative to the previous node, the last six bytes indicate the fil_addr_t relative to the last node. In addition to flst_node_t, the disk linked list also has a header information flst_base_node_t. the header information is the number of nodes FLST_LEN (4 bytes) + FLST_FIRST (6 bytes) + FLST_LAST (6 bytes ). 1.1 Structure of the disk linked list
2. in the tablespace of innodb, space Structure Analysis stores all data in pages. There are two types of pages in space: FSP_HDR/XDES page and fseg inodes page. Each page is stored in 16 KB by default. innodb allocates 64 pages at a time in the unit of extent when allocating pages. 2.1 fsp hdr/XDES Page2.1.1XDES Structure Analysis (extent) this type of page mainly stores two types of information, the first 112 bytes store the File Space header information, the remaining space is used to store multiple extent descriptions (XDES). The specific storage structure is as follows:
Only the first page of space will save the FSP header, and other pages will be filled with 0. Each XDES Page contains a maximum of 256 XDES descritptors entries, and each XDES descritptors Entry corresponds to an extent. The structure of the XDES descritptors Entry is described as follows:
The File Segment ID is the ID of the segment to which the current extent belongs.
XDES list is a node of the disk two-way linked list, which respectively refers to the page position of the forward XDES entry and the page location of the next XDES entry.
State extent status, XDES_FREE, release, XDES_FULL_FRAG, XDES_FSEG. When XDES_FSEG is set, the extent is already affiliated with a Segment, and the extent is specified as XDES_FSEG when it is created. The status XDES_FREE of an extent instance when it is just allocated.
Status indexes of all pages in the current extent of bitmap. One page occupies 2 bits, the first bit indicates whether the page is in use, and the second bit indicates whether the page is cleared, all are TRUE.
2.1.2 FSP Header space id ID of the current tablespace size the maximum number of pages that can be accommodated in the current space. This value is changed only when the file is expanded.
The number of pages allocated for initialization in the current space, including idle and used pages.
Flag does not work
Number of pages used in the frage used FSP_FREE_FRAG list
List of available extent objects in free list space. No page in extent is used.
Frag free list has the extent list of available broken leaf pages, and some pages in exntent are used
The frag full list does not have an extent list with available pages. All pages in the exntent are used.
The next available segment id of the segment id
Full inode list space the list of fully occupied segment inode pages
Free inode list space the list of fully occupied segment inode pages
2.2 Fseg inode Page the page type is used to store the pages used by fseg inode. Each inode occupies 192 bytes, and one Page stores 85 inode objects. The structure is as follows:
The FIL Header is followed by 12 bytes. The 12 bytes are actually the full inode list or the list in the free inode list. Therefore, they represent the fil_addr_t before and after the FIL Header. Each inode occupies 192 bytes and manages the corresponding extent and fragment pages respectively. The inode structure is as follows:
Fseg id segment ID
Not full used FSEG_NOT_FULL number of pages in the list
List of idle extent in FSEG_FREE inode
FSEG_NOT_FULL extent some pages are occupied and some pages are idle extent list
List of extent fully occupied by FSEG_FULL
FSEG_MAGIC_N verify the magic word
Fragment array an array of 32-bit fragmented page index storage, if the data is full. the main function is to save space. For example, when a table is created, a complete extent is not assigned to the table and only six PAGE pages are allocated. In this case, fragment array is used for management.
3. space Structure Diagram 3.1space framework Diagram
3.2 Module relationship
4. in the inode, extent, and page allocation process of space in innodb space, the relationship between inode, extent, and page is interlocking, inode corresponds to segment, and extent corresponds to zone, page is the page, which is also the smallest Distribution Unit of the tablespace. A page is 16 KB by default in MySQL. An extent manages 64 pages with a size of 1 MB. inode can manage many extent pages and 32 frag pages (broken pages ). Frag page is defined to save space. After learning about the above basic concepts, we began to analyze inode allocation, extent allocation, and page allocation processes.
4.1 inode distribution process we can know through the introduction of inode page that inode information must be stored in inode page. When inode is assigned, the inode must be obtained from the inode page. If no inode page is available, an inode page will be obtained first in the free list of space (in the fsp_alloc_seg_inode_page function), and then an idle inode will be obtained in this inode page. In this process, two disk linked lists are involved: FSP_SEG_INODES_FREE and FSP_SEG_INODES_FULL. The two queues manage inode pages. If there is no idle inode, inode pages are placed in FSP_SEG_INODES_FULL, if inode pages with idle inode are stored in FSP_SEG_INODES_FREE. An inode page contains 85 inode information. Inode allocation is as follows:
Step 2: When FSP_SEG_INODES_FREE is null, obtain an inode page from the default space header page, corresponding to the function fsp_alloc_seg_inode_page.
Step 2: When applying for inode, if FSP_SEG_INODES_FREE has an inode page available, the corresponding function fsp_alloc_seg_inode is defined from inode page or an inode.
Step 2: If the inode page in which inode is located does not have any idle inode after applying for inode, The inode page will be put into FSP_SEG_INODE_FULL and deleted from FSP_SEG_INODES_FREE.
Step 1: If all the pages managed by inode are idle, The inode status will be set to idle. At this time, the inode page will be moved from FSP_SEG_INODE_FULL to FSP_SEG_INODES_FREE; this process is called only when the segment is deleted. Corresponding function fsp_free_seg_inode
4.2extent there are two ways to allocate extent, one is to apply for distribution through inode, and the other is to apply for distribution through fragment fragmentation. Inode is allocated when there is no idle extent available in inode, one or five extent files will be applied to the space free list for management, if the number of extent managed by inode is less than 40, only one extent will be applied each time. If the number exceeds this size, five extent will be applied at a time, this process involves three disk linked lists: FSEG_FREE, FSEG_NOT_FULL, and FSEG_FULL of inode. The second method is to apply for extent directly when the frag page is allocated, which involves two linked lists: FSP_FREE_FRAG and FSP_FULL_FRAG. Distribution is as follows:
Medium, 1 ~ 7 is the process of applying for distribution by inode, 8 ~ 12 is the extent Application Method for frag page.
1: When the free list of inode is empty, if you need to apply for a new extent, innodb will obtain the idle extent from the space free list and add it to the inode free list.
2: when there is extent in the inode free list, if you apply to use the new extent, you only need to retrieve it from the inode free list and move the extent to the inode not full.
3: only apply for page operations through inode. At this time, extent has enough free pages.
4: when there is no idle page in extent, the extent will be transferred from inode not full to inode full.
5: When a page is released, the extent of the page is a fully occupied and managed by inode. After the page is released, the extent will be moved from inode full to inode not full.
6: When a page is released, the extent of this page has only one occupied page. After the page is released, the extent will be returned to the inode list. in addition, extent is directly returned to space free list in Step 7.
8 ~ 12. Similar to the above steps
4.3page allocation process page application distribution is based on inode application and extent application. The application on the page includes external application through inode and fragment page application. Fragment method is relatively simple to apply, so it is not described, and the source code is very clear. Inode allocation is complicated. Its main implementation is the fseg_alloc_free_page_low and fseg_free_page_low functions. In the fseg_alloc_free_page_low function, page in inode is obtained in seven cases.
1. The page at the hint position of the specified inode is idle and the corresponding page is directly returned.
2. descr is idle, but the number of idle pages in the segment inode is less than 1/8, and all the fragment pages are used up. assign an extent to it and obtain the page corresponding to the hint.
3. If descr is not idle and the number of idle pages in the segment inode is less than 1/8, an idle extent is obtained in the inode and the page corresponding to the extent descr is returned.
4. descr is in XDES_FSEG State, and there is an idle page in the extent, and a page is obtained from it.
5. In addition to the above cases, if descr is not idle, but inode has other idle extent, obtain one from other extent.
6. If the other extent does not have a free page, but the fragment array still has a free fragment page, obtain a free page from the free fragment page.
7. If there is no broken page, apply to allocate a new extent and obtain an idle page.
5. Summary The implementation of table space is also dependent on files such as page0page. * fil0fil. * In the fsp0fsp. * file. In terms of storage, innodb defines the smallest storage unit as page. space Designs these layer relationships to manage pages more efficiently and reasonably. Space can exist in the same database file as other tables, or one table can be stored in one file. This depends on the MySQL configuration. Analyzing the structure and working principle of space helps us to understand the storage method of innodb, which provides a powerful foundation for understanding indexes, locks, and transactions. As mentioned above, the smallest storage unit is page. I will introduce the storage method and working principle of data page separately in the next chapter.
