MEM_ROOT for MySQL

MEM_ROOT for MySQL
This article will describe in detail the widely used MEM_ROOT struct in MySQL, while saving the debugging information. It will only analyze the memory allocation in mysql under normal conditions.

Before the specific analysis, we set a precedent for some macros used in the use of this struct:

 
 

  
  
# Define MALLOC_OVERHEAD 8 // some additional space needs to be retained during the allocation process
 
  
  
# Define ALLOC_MAX_BLOCK_TO_DROP 4096 // the usage of the macro will be analyzed later

  
  
# Define ALLOC_MAX_BLOCK_USAGE_BEFORE_DROP 10 // The Macro will be analyzed later

  
  


  
  
# Define ALIGN_SIZE (A) MY_ALIGN (A), sizeof (double ))

  
  
# Define MY_ALIGN (A, L) (A) + (L)-1 )&~ (L)-1 ))

  
  


  
  
# Define ALLOC_ROOT_MIN_BLOCK_SIZE (MALLOC_OVERHEAD + sizeof (USED_MEM) + 8)

  
  
/* Define some useful general macros (shocould be done after all headers ).*/

  
  
# Define MY_MAX (a, B) (a)> (B )? (A): (B) // calculate the maximum value between two values.

  
  
# Define MY_MIN (a, B) (a) <(B )? (A): (B) // calculates the minimum value between two values.
 
 

Next let's take a look at the MEM_ROOT struct information:

 
 

  
  
Typedef struct st_mem_root
 
  
  
{

  
  
USED_MEM * free; // The linked list header pointer of the free block link list.

  
  
USED_MEM * used; // chain table head pointer of used block link list

  
  
USED_MEM * pre_alloc; // pre-allocated block

  
  
Size_t min_malloc; // if the remaining available space of the block is smaller than this value, it will be moved from free list to used list

  
  
Size_t block_size; // size of the Space initialized each time

  
  
Unsigned int block_num; // record the actual number of blocks, initialized to 4

  
  
Unsigned int first_block_usage; // number of times the first block test in the free list does not meet the size of the allocated space

  
  
Void (* error_handler) (void); // error handling function for allocation failure

  
  
} MEM_ROOT;
 
 

The following describes how to allocate specific block information.

 
 

  
  
Typedef struct st_used_mem
 
  
  
{

  
  
Struct st_used_mem * next; // point to the next allocated block

  
  
Unsigned int left; // The remaining space of the block

  
  
Unsigned int size; // The total size of the block

  
  
} USED_MEM;
 
 

In fact, MEM_ROOT manages used and free blocks through a two-way linked list during the allocation process:


The initialization process of MEM_ROOT is as follows:

 
 

  
  
void init_alloc_root(MEM_ROOT *mem_root, size_t block_size,size_t pre_alloc_size __attribute__((unused)))
 
  
  
{

  
  
mem_root->free= mem_root->used= mem_root->pre_alloc= 0;

  
  
mem_root->min_malloc= 32;

  
  
mem_root->block_size= block_size - ALLOC_ROOT_MIN_BLOCK_SIZE;

  
  
mem_root->error_handler= 0;

  
  
mem_root->block_num= 4; /* We shift this with >>2 */

  
  
mem_root->first_block_usage= 0;

  
  
} 
 
 

Block_size space is a block_size-ALLOC_ROOT_MIN_BLOCK_SIZE during initialization. Because the memory is not enough and you need to resize it through mem_root-> block_num> 2 * block_size, mem_root-> block_num> 2 must be at least 1, therefore, during the initialization process, mem_root-> block_num = 4 (Note: 4> 2 = 1 ).

Next let's take a look at the specific steps to allocate memory:

 
 

  
  
Void * alloc_root (MEM_ROOT * mem_root, size_t length)
 
  
  
{

  
  
Size_t get_size, block_size;

  
  
Uchar * point;

  
  
Reg1 USED_MEM * next = 0;

  
  
Reg2 USED_MEM ** prev;

  
  


  
  
Length = ALIGN_SIZE (length );

  
  
If (* (prev = & mem_root-> free ))! = NULL)

  
  
{

  
  
If (* prev)-> left <length &&

  
  
Mem_root-> first_block_usage ++> = ALLOC_MAX_BLOCK_USAGE_BEFORE_DROP &&

  
  
(* Prev)-> left <ALLOC_MAX_BLOCK_TO_DROP)

  
  
{

  
  
Next = * prev;

  
  
* Prev = next-> next;/* Remove block from list */

  
  
Next-> next = mem_root-> used;

  
  
Mem_root-> used = next;

  
  
Mem_root-> first_block_usage = 0;

  
  
}

  
  
For (next = * prev; next & next-> left <length; next = next-> next)

  
  
Prev = & next-> next;

  
  
}

  
  
If (! Next)

  
  
{/* Time to alloc new block */

  
  
Block_size = mem_root-> block_size * (mem_root-> block_num> 2 );

  
  
Get_size = length + ALIGN_SIZE (sizeof (USED_MEM ));

  
  
Get_size = MY_MAX (get_size, block_size );

  
  


  
  
If (! (Next = (USED_MEM *) my_malloc (get_size, MYF (MY_WME | ME_FATALERROR ))))

  
  
{

  
  
If (mem_root-> error_handler)

  
  
(* Mem_root-> error_handler )();

  
  
DBUG_RETURN (void *) 0);/* purecov: inspected */

  
  
}

  
  
Mem_root-> block_num ++;

  
  
Next-> next = * prev;

  
  
Next-> size = get_size;

  
  
Next-> left = get_size-ALIGN_SIZE (sizeof (USED_MEM); // bug: If the block is calculated through mem_root-> block_size * (mem_root-> block_num> 2, ALIGN_SIZE (sizeof (USED_MEM) has been removed, which is repeated here.

  
  
* Prev = next;

  
  
}

  
  


  
  
Point = (uchar *) (char *) next + (next-> size-next-> left ));

  
  
/* TODO: next part may be unneded due to mem_root-> first_block_usage counter */

  
  
If (next-> left-= length) <mem_root-> min_malloc)

  
  
{/* Full block */

  
  
* Prev = next-> next;/* Remove block from list */

  
  
Next-> next = mem_root-> used;

  
  
Mem_root-> used = next;

  
  
Mem_root-> first_block_usage = 0;

  
  
}

  
  
}
 
 

The specific logic of the above Code is as follows: 1. view the free linked list to find a block that satisfies the space. If a proper block is found, 1.1 returns the initial address of the block from size-left. Of course, in the free list traversal process, the left space in the first block in freelist is determined not to meet the space to be allocated, this block has been searched for 10 times (ALLOC_MAX_BLOCK_USAGE_BEFORE_DROP) and does not meet the allocation length. If the remaining space of this block is less than 4 k (ALLOC_MAX_BLOCK_TO_DROP), the block is moved to the used linked list. 2. if no suitable block exists in the free linked list, mem_root-> block_size * (mem_root-> block_num> 2) and length + ALIGN_SIZE (sizeof (USED_MEM) are allocated to 2.1 )) as the new block memory space. 2.2 According to the usage of the block, the block is mounted on the used or free linked list.
Note the usage of level 2 pointers:

 
 

  
  
for (next= *prev ; next && next->left < length ; next= next->next)
 
  
  
prev= &next->next;

  
  
} 
 
 

Prev points to the address pointed to by the next of the last block:

Therefore, replace the prev address with the newblock address, and add the new block to the end of the free list: * prev = next;


Conclusion: MEM_ROOT uses a heuristic Allocation Algorithm for memory allocation. As the number of blocks increases, the memory size of a single block increases: block_size = mem_root-> block_size * (mem_root-> block_num> 2 ).