MySQL series: Basic Data Structure of innodb Engine Analysis
Over the past year, I have been analyzing database-related source code. Some time ago I analyzed the implementation of levelDB and BeansDB. There are many articles on the network analysis of these two databases, it is also relatively in-depth analysis, so there is no need to repeat the work. Recently, I began to focus on relational databases and MYSQL. Of course, I mainly focused on the database storage engine. First of all, I started to analyze and understand innodb, the most popular open source relational database engine. I usually analyze the source code step by step from the basic data structure and algorithms. If I do not understand the source code, I enter the source code again and perform the corresponding unit test, this is easy to understand. This should also be the case for large projects such as Innodb. In the future, I will gradually write the details and implementations to the BLOG. I analyzed Innodb is based on the MySQL-3.23 as the analysis object, and then compare the changes in version 5.6 for analysis. The advantage of doing so is that it is easy to first understand the relatively basic code and then understand the latest changes after the basic concepts are introduced. The following is my understanding of the basic data structures and algorithms of innodb.
1. vectorinnodb vector is a dynamic array data structure, similar to c ++ STL usage, it is worth mentioning that the memory allocation of the vector can be specified through the function pointer whether to allocate memory from the heap memory pool heap stack or use the memory allocation of the memory with the built-in OS malloc. The memory distributor structure is as follows:
Struct ib_alloc_t {ib_mem_alloc_tmem_malloc; // alloc function pointer of the Allocator ib_mem_free_tmem_release; // The free function pointer of the Allocator defines the heap size pointer void * arg; // heap handle. If the system malloc mode is used, the value is NULL <span style = "white-space: pre" ></ span> };The sorting function is integrated in vector, and its sorting algorithm is sorted by qsort (FAST. Vector memory structure:
2. the list data structure of the listinnodb memory is a standard two-way linked list structure. ib_list_node_t indicates the prev of the previous node and the next of the next node, you can allocate the list memory through heap memory heap or by using the system's malloc. We can see that using ib_list_create_heap to create a list is always ib_list_create to create a list. However, the internal memory allocation of ib_list_node_t is based on heap.
Memory Structure of ist:
3. The FIFO queue of FIFO-queueinnodb is a multi-threaded message queue. Multiple Threads can add messages to the queue. Multiple Threads can simultaneously read and process messages in the queue. The mutex of queue ensures that only one thread is in the items linked list of the queue for operation (read or write? Http://www.bkjia.com/kf/ware/vc/ "target =" _ blank "class =" keylink "> examples/fs8y/examples/JxdWV1ZdC0zeqzydK7uPbP + 8 + examples/vPoru6s + XH + examples/examples + examples/bbByKHP + 8 + itcS3vbeoo6zV4tH5tsHIoc/fs8zDu9PQsdjSqtK71rG1yLT9z/vPoqOsv8nS1NTatci0/rotate "brush: SQL; "> struct ib_wqueue_t {ib_mutex_tmutex;/* mutex */ib_list_t * items;/* use list as the carrier of queue */OS _event_tevent;/* semaphore */};4. the basic structure of the hash table in the hash table innodb is similar to that of the traditional hash table. The difference is that the hash table in innodb uses a custom chain bucket structure, instead of using a traditional list for collision management for each bucket unit. Due to this feature, the hash table operations in innodb use a series of Operation macros for operations. The purpose of this operation is to perform operations on the hash table in a generic manner, because in innodb, in addition to operating the data in the memory, the operation also hides the data in the hard disk. The operating macros of innodb are as follows:
HASH_INSERT insert operation HASH_DELETE delete operation HASH_GET_FIRST get the first data unit of the cell corresponding to the specified HASH key HASH_GET_NEXT get the value of the corresponding key HASH_SEARCH for the next unit corresponding to cell_node traverse the entire hash table and set each data the Unit is to execute the ASSERTION operation HASH_DELETE_AND_COMPACT to delete the operation and optimize and adjust the heap memory allocation layout, this makes heap more efficient. HASH_MIGRATE combines OLD_TABLE data units into NEW_TABLE. These macros will specify the data type and Next function name when calling.
The innodb hash table also provides cell-level locks in multi-thread concurrency mode, including mutex locks and rw_lock locks. During the call of the hash_create_sync_obj_func function, a lock data unit of n_sync_obj is created, and n_sync_obj must be the N power of 2. That is to say, if n_sync_obj = 8 and n_cells = 19 in the hash table, at least two cells share one lock. This is unmatched by other hash tables. The following is the structure definition of the hash table:
Struct hash_table_t {enum hash_table_sync_ttype;/* synchronization type of hash table */ulintn_cells;/* Number of hash buckets */hash_cell_t * array;/* hash bucket array */# ifndef buckets; union {/* synchronization lock */ib_mutex_t * mutexes; Limit * rw_locks;} sync_obj;/* The number of heaps units is the same as n_sync_obj */mem_heap_t ** heaps; # limit * heap; ulintmagic_n;/* Verify the magic word */# endif };
5. Summary there are other data structures in Innodb, such as the smallest heap, which are all common encapsulation and will not be described too much. You can check the source code of innodb. Innodb makes special processing when defining the data structure, such as controlling thread concurrency and memory allocation. This is intended for unified management. The innodb code is C, but C ++ is supported. The traditional data structures and algorithms such as STL are not used in it, which is very suitable. It is said that MYSQL 5.7 began to use boost and STL in large quantities. I personally feel that STL is barely used, and boost is a little too big.
