MySQL series: The basic data structure of INNODB engine analysis

Nearly a year has been in the analysis of the database-related source code, the previous period of time to analyze the implementation of the LEVELDB and the implementation of BEANSDB, the two database network analysis of many articles, but also comparative analysis of the relatively deep, so there is not much need to repeat labor. Recently began to focus on the relational database and MySQL, of course, mainly the database storage engine, first of all, I still from InnoDB this most popular open source relational database engine started to step through analysis and understanding. I generally analyze the source code is from the basic data structure and algorithm to the analysis, encountered do not understand the place, I re-enter according to the source code and do the corresponding unit test, so easy to understand. For the big project such as InnoDB, should also be so, I will gradually in the specific details and implementation of writing to the blog. Here is my understanding of the data structures and algorithms of the InnoDB Foundation.

1.vectorThe vector of InnoDB is a data structure of a dynamic array, similar to the STL usage of C + +, and it is worth mentioning that the vector's memory allocation can be specified by a function pointer to allocate memory from the heap memory pool stack or with the OS's own malloc. The structure of the memory allocator is:

        struct ib_alloc_t {             ib_mem_alloc_tmem_malloc;//Allocator's malloc function pointer             ib_mem_free_tmem_release;  The free function pointer of the allocator is             ib_mem_resize_tmem_resize;  The allocator's redefined heap size pointer             void* arg;     Heap handle, if it is the system malloc mode, this value is Null<span style= "White-space:pre" ></span>};

The sorting function is integrated within the vector, and the sorting algorithm is sorted by qsort (FAST).Vector Memory Structure:

2. Memory ListInnoDB's list data structure is a standard doubly-linked list structure, with ib_list_node_t pointing to the previous node prev and pointing to the latternode's next,the memory allocation of a list can be allocated either through the heap heap of memory or by malloc of the system. It depends on the useib_list_create_heap to create List love is forever ib_list_create to create a list. However, the internal ib_list_node_t memory allocation isheap to allocate.
the memory structure of the IST:

3.fifo-queuethe InnoDB FIFO queue is a multi-threaded message queue that can have multiple threads adding messages to the queue, and multiple threads can read and process messages in the queue at the same time. The mutex of a queue is to ensure that at the same time only one thread is in operation (read or write) of the queue's items list, os_event is a write thread that notifies all read threads that a read event can be made to queue, that is, only write to the queue to complete a message, The event signal is sent to the read thread. The queue's message buffer is stored using ib_list_t, which is usually written at the end of the list, while reading always reads the first one of the list. Queue processing provides a way to read messages all the time, and it also provides the longest way to wait for the message to be read, so that the read thread does not have to wait for the message, and can wait a while to handle other tasks. Its c structure is defined as follows:

struct Ib_wqueue_t{ib_mutex_tmutex;/* Mutex */ib_list_t*items;/* Use list as the carrier of queue */os_event_tevent;/* semaphore */};

4. Hash tablethe basic structure of the hash table in InnoDB is similar to that of the traditional hash table, and the difference is that the INNODB hash table uses a custom chain bucket structure, instead of using a traditional list to do collision management with each bucket unit. because of this feature, the hash table operation in InnoDB uses a series of operation macros to operate the hash table so that it can operate on a generic type, because in InnoDB, the data in the insinuate hard disk is manipulated in addition to the data in memory. The following is an action macro for InnoDB:
Hash_insert                               &NBSP ;     Insert action         Hash_delete                   &N Bsp                 Delete action         Hash_get_first                               Get the first data cell of the specified hash key corresponding to the cell   &NBS P     Hash_get_next                           &NB Sp     Get the next cell for Cell_node         Hash_search               & nbsp                   Find the value of the corresponding key         Hash_search_all                             traverse the entire hash table and execute each data unit as a parameter Asse RtioN Operations         hash_delete_and_compact         Delete operations and optimizes and adjusts memory allocation layouts on the heap to make the heap more efficient and nbsp       hash_migrate                         &NBS P       Merging old_table data units into new_tableThese macros specify the type of data and the next function name when they are called.
The InnoDB hash table also provides a cell-level granularity lock in multithreaded concurrency mode, with a mutex-type lock, and a rw_lock-type lock. During the Hash_create_sync_obj_func function call, a n_sync_obj lock data unit is created, and the n_sync_obj must be 2 of the n-th party. That is, if N_sync_obj = 8, n_cells = 19 of the hash table, then at least two cells will have a common lock. This is unmatched by other hash tables. The following is the structure definition of hash table:

struct Hash_table_t{enum hash_table_sync_ttype;/*hash table synchronization type */ulintn_cells;/*hash bucket number */hash_cell_t*array;/* Hash bucket array */#ifndef univ_hotbackupulintn_sync_obj;union{/* sync lock */ib_mutex_t*mutexes;rw_lock_t*rw_locks;} Sync_obj;/*heaps the number of units and n_sync_obj the same */mem_heap_t**heaps; #endifmem_heap_t *heap;ulintmagic_n;/* Check the magic word */#endif};

5. SummaryInnoDB There are other data structures, such as the minimum heap, these are generic packages, do not do too much description, you can go to see the source of InnoDB related to it. InnoDB Special handling when defining data structures, such as control of thread concurrency, control of memory allocations. This is done for the purpose of unified management. The code for INNODB is C, but it supports C + +. There is no use of the traditional STL data structure and algorithms, to a large extent, is the problem of suitability. MySQL 5.7 is said to start using boost and STL heavily. Personal feeling STL is still reluctant to use boost a little step to the feeling of big.

MySQL series: The basic data structure of INNODB engine analysis