MySQL column: InnoDB the underlying data structure for the analysis of source code

Database-related source code analysis over the past year. Previous time Analysis LEVELDB implementation and BEANSDB implementation, database network analysis These two articles are very many. They are also more deeply comparative analysis, so there is no need to repeat a lot of labor. MYSQL, of course, is mainly the database storage engine, first of all, I still start from InnoDB, the most popular open source relational database engine to gradually analyze and understand.

I usually analyze the source code from the basis of the data structure and algorithm to step up the analysis. Encountered ambiguous place, oneself according to source code again input again and do the corresponding unit test, so easy to understand. For InnoDB this big project, should also be so, later I will gradually put detailed details and implementation of the blog. I analyze InnoDB is based on MySQL-3.23 as the object of analysis, and then to compare the revision of the 5.6 version to do the analysis. One of the advantages of this is that you first understand the relative basics of code easy. Understand the latest changes after you have a basic concept. Here is my understanding of the data structures and algorithms of the InnoDB Foundation.

1.vectorThe vector of InnoDB is the data structure of a dynamic array. Similar to the STL usage of C + +, it is worth mentioning that the vector's memory allocation can be specified by a function pointer to allocate memory from the heap memory pool stack or with the OS's own malloc. The structure of the memory allocator is:

        struct ib_alloc_t {             ib_mem_alloc_tmem_malloc;//Allocator's malloc function pointer             ib_mem_free_tmem_release;  The free function pointer of the allocator is             ib_mem_resize_tmem_resize;  The allocator again defines the heap size pointer             void* arg;     Heap handle, assumed to be the system malloc mode, this value is Null<span style= "White-space:pre" ></span>};

The sorting function is integrated within the vector. The sorting algorithm is sorted by qsort (high speed).Vector Memory Structure:

2. Memory Listthe list data structure of INNODB is a standard doubly-linked list structure. ib_list_node_t which has a prev pointing to the previous node and pointing to the latter onenode's next,The memory allocation of the list can be allocated through the heap memory heap, or by the system malloc.

It looks like it's used .ib_list_create_heap to create List love is forever ib_list_create to create a list. However, the internal ib_list_node_t memory allocation isheap to allocate.
the memory structure of the IST:

watermark/2/text/ahr0cdovl2jsb2cuy3nkbi5uzxqvexvhbnj4zhu=/font/5a6l5l2t/fontsize/400/fill/i0jbqkfcma==/ Dissolve/70/gravity/center ">
The FIFO queue for the 3.fifo-queue InnoDB is a multi-threaded message queue. The ability to have multiple threads add messages to a queue can have multiple threads reading and processing messages in the queue at the same time. The mutex of a queue is a guarantee that at the same time only one thread is manipulating (reading or writing) the queue's items list, os_event is the write thread that notifies all read threads that the read event is capable of queue. In other words, only one message is written to the queue. The event signal is sent to the read thread. The queue's message buffer is made of ib_list_t for storage. Writing is usually written at the end of the list, while reading always reads the first one of the list.

Queue processing provides a way to read messages all the time, and provides the longest way to wait for a message to be read. This makes it unnecessary for the read thread to wait for the message. Be able to wait for a period of time to deal with other tasks. Its C structure definition is as follows:

struct Ib_wqueue_t{ib_mutex_tmutex;/* mutually exclusive */ib_list_t*items;/* Use list as the carrier of the queue */os_event_tevent;/* semaphore */};

4. Hash tablethe basic structure of the hash table in InnoDB is similar to that of the traditional hash table, and the difference is that the INNODB hash table uses its own definition of the chain-bucket structure. Instead of using a traditional list for every bucket unit to do collision management. due to this feature. The hash table operation in InnoDB uses a series of operation macros to do the operation. The purpose of this is to be able to manipulate the hash table in a generic way, because in InnoDB, the data in the insinuate hard disk is manipulated in addition to manipulating the data in memory. Here is the operation macro for InnoDB:
Hash_insert                               &NBSP ;     Insert action         Hash_delete                   &N Bsp                 Delete action         Hash_get_first                               Get a specified hash key the first data unit of the corresponding cell   &NBS P     Hash_get_next                           &NB Sp     Get cell_node the corresponding next cell         Hash_search               & nbsp                   Find the value of the key         Hash_search_all                             go through the hash table and run every data unit for the number of ErtiOn actions         hash_delete_and_compact         Delete operations and optimizes and adjusts memory allocation layouts on the heap to make the heap more efficient         hash_migrate                       &NBS P         merging old_table data units into new_tableThese macros specify the type of data and the next function name when they are called.
The InnoDB hash table also provides a cell-level granularity lock with a mutex type in multi-threaded concurrency mode. There are also rw_lock types of locks.

During the Hash_create_sync_obj_func function call, a n_sync_obj lock data unit is created. The n_sync_obj must be 2 of the n-th party. That is to say N_sync_obj = 8, n_cells = 19 of the hash table. Then at least two cells are common to a lock.

This is unmatched by other hash tables. Here is the structure definition of hash table:

struct Hash_table_t{enum hash_table_sync_ttype;/*hash table synchronization type */ulintn_cells;/*hash bucket number */hash_cell_t*array;/* Hash bucket array */#ifndef univ_hotbackupulintn_sync_obj;union{/* sync lock */ib_mutex_t*mutexes;rw_lock_t*rw_locks;} Sync_obj;/*heaps the number of units and n_sync_obj the same */mem_heap_t**heaps; #endifmem_heap_t *heap;ulintmagic_n;/* Check the magic word */#endif};

5. SummaryInnoDB There are other data structures, such as the minimum heap, these are generic packages, and do not do too much descriptive narrative, in the ability to see InnoDB source code related to be able. InnoDB Special handling when defining data structures, such as the control of thread concurrency. The control of memory allocations.

This is done for the purpose of unified management.

The code for INNODB is C. But support C + +. There is no traditional data structure and algorithms, such as STL, to a large extent, the problem of suitability.

MySQL 5.7 is said to have started using boost and STL extensively.

Personal feeling STL is still reluctant. Use boost a bit to feel strides.

Copyright notice: This article Bo Master original articles, blogs, without consent may not be reproduced.

MySQL column: InnoDB the underlying data structure for the analysis of source code