MySQL MEM_ROOT explanation and instance code, mysqlmem_root

MySQL MEM_ROOT explanation and instance code, mysqlmem_root

MySQL MEM_ROOT

This article will describe in detail the widely used MEM_ROOT struct in MySQL, while saving the debugging information. It will only analyze the memory allocation in mysql under normal conditions.

Before the specific analysis, we set a precedent for some macros used in the use of this struct:

# Define MALLOC_OVERHEAD 8 // During allocation, A part of extra space needs to be retained # define ALLOC_MAX_BLOCK_TO_DROP 4096 // the usage of the macro will be analyzed later # define ALLOC_MAX_BLOCK_USAGE_BEFORE_DROP 10 // the usage of the macro will be analyzed later # define ALIGN_SIZE) MY_ALIGN (A), sizeof (double) # define MY_ALIGN (A, L) (A) + (L)-1 )&~ (L)-1) # define ALLOC_ROOT_MIN_BLOCK_SIZE (MALLOC_OVERHEAD + sizeof (USED_MEM) + 8)/* Define some useful general macros (shocould be done after all headers ). * // * Author: www.manongjc.com */# define MY_MAX (a, B) (a)> (B )? (A): (B) // calculate the maximum value between two values # define MY_MIN (a, B) (a) <(B )? (A): (B) // calculates the minimum value between two values.

Next let's take a look at the MEM_ROOT struct information:

Typedef struct st_mem_root {USED_MEM * free;/* free block link list linked list header pointer */USED_MEM * used;/* used block link list linked list header pointer */USED_MEM * pre_alloc; /* pre-allocated block */size_t min_malloc;/* If the remaining available space of the block is smaller than this value, it will be moved from free list to used list */size_t block_size; /* size of Space initialized each time */unsigned int block_num;/* record the actual number of blocks, initialized to 4 */unsigned int first_block_usage; /* number of times that the first block test in the free list does not meet the size of the allocated space */void (* error_handler) (void);/* error handling function for failed allocation */} MEM_ROOT;

The following describes how to allocate specific block information.

Typedef struct st_used_mem {struct st_used_mem * next; // point to the unsigned int left of the next allocated block; // The unsigned int size of the remaining space of the block; // total size of the block} USED_MEM;

In fact, MEM_ROOT manages used and free blocks through a two-way linked list during the allocation process:

The initialization process of MEM_ROOT is as follows:

void init_alloc_root( MEM_ROOT *mem_root, size_t block_size, size_t pre_alloc_size __attribute__( (unused) ) ){ mem_root->free  = mem_root->used = mem_root->pre_alloc = 0; mem_root->min_malloc = 32; mem_root->block_size = block_size - ALLOC_ROOT_MIN_BLOCK_SIZE; mem_root->error_handler = 0; mem_root->block_num = 4; /* We shift this with >>2 */ mem_root->first_block_usage = 0;}

Block_size space is a block_size-ALLOC_ROOT_MIN_BLOCK_SIZE during initialization. Because the memory is not enough and you need to resize it through mem_root-> block_num> 2 * block_size, mem_root-> block_num> 2 must be at least 1, therefore, during the initialization process, mem_root-> block_num = 4 (Note: 4> 2 = 1 ).

Next let's take a look at the specific steps to allocate memory:

Void * alloc_root (MEM_ROOT * mem_root, size_t length) {size_t get_size, block_size; uchar * point; reg1 USED_MEM * next = 0; reg2 USED_MEM ** prev; length = ALIGN_SIZE (length); if (* (prev = & mem_root-> free ))! = NULL) {if (* prev)-> left <length & mem_root-> first_block_usage ++> = ALLOC_MAX_BLOCK_USAGE_BEFORE_DROP & (* prev)-> left <ALLOC_MAX_BLOCK_TO_DROP) {next = * prev; * prev = next-> next;/* Remove block from list */next-> next = mem_root-> used; mem_root-> used = next; mem_root-> first_block_usage = 0;} for (next = * prev; next & next-> left <length; next = next-> next) prev = & next-> next;} if (! Next) {/* Time to alloc new block */block_size = mem_root-> block_size * (mem_root-> block_num> 2 ); get_size = length + ALIGN_SIZE (sizeof (USED_MEM); get_size = MY_MAX (get_size, block_size); if (! (Next = (USED_MEM *) my_malloc (get_size, MYF (MY_WME | ME_FATALERROR) {if (mem_root-> error_handler) (* mem_root-> error_handler )(); DBUG_RETURN (void *) 0);/* purecov: inspected */} mem_root-> block_num ++; next-> next = * prev; next-> size = get_size; next-> left = get_size-ALIGN_SIZE (sizeof (USED_MEM);/* bug: If the block uses mem_root-> block_size * (mem_root-> block_num> 2) after calculation, ALIGN is removed. _ SIZE (sizeof (USED_MEM), which is repeated here. */* Prev = next;} point = (uchar *) (char *) next + (next-> size-next-> left);/* TODO: next part may be unneded due to mem_root-> first_block_usage counter * // Author: www.manongjc.com */if (next-> left-= length) <mem_root-> min_malloc) {/* Full block */* prev = next-> next;/* Remove block from list */next-> next = mem_root-> used; mem_root-> used = next; mem_root-> first_block_usage = 0 ;}}

The specific logic of the above Code is as follows:

1. view the free linked list to find blocks that meet the requirements of the space. If a proper block is found:

1.1 directly return the initial address of the block from size-left. Of course, in the process of free list traversal, we will judge the free list
The left space in the first block does not meet the space to be allocated, and the block has been searched for 10 times.
(ALLOC_MAX_BLOCK_USAGE_BEFORE_DROP) does not meet the allocation length and the remaining space of the block is smaller
4 k (ALLOC_MAX_BLOCK_TO_DROP), move the block to the used linked list.

2. If there is no proper block in the free linked list, then:

2.1 allocate mem_root-> block_size * (mem_root-> block_num> 2) and length + ALIGN_SIZE (sizeof (USED_MEM ))
As the new block memory space.

2.2 According to the usage of the block, the block is mounted on the used or free linked list.

Note the usage of level 2 pointers:

for (next= *prev ; next && next->left < length ; next= next->next)prev= &next->next;}

Prev points to the address pointed to by the next of the last block:

Therefore, replace the prev address with the address of the new block to add the new block to the end of the free list: * prev = next;

Summary:

MEM_ROOT uses a heuristic algorithm for memory allocation. As the number of blocks increases, the memory size of a single block increases: block_size = mem_root-> block_size * (mem_root-> block_num> 2 ).

Thank you for reading this article. I hope it will help you. Thank you for your support for this site!