STL Source ANALYSIS--second level space Configurator

This article explains the second level configurator of the SGI STL Space Configurator.

Compared with the first level configurator, the second level configurator has some more mechanisms to avoid fragmentation of memory in small chunks. Not only the problem of fragmentation, but also the additional burden of configuration is a big problem. Because the smaller the block, the greater the burden.

An additional burden is the extra information at the head of a dynamically allocated block of memory, including information about the size of the memory block and the memory reserve (to determine whether it is out of bounds). For more information, refer to MSVC or other malloc implementations.

SGI STL Second Level Configurator realizes the idea concretely

As follows: If the chunk to be allocated is greater than 128bytes, it is handed over to the first level configurator for processing. If the chunk to be allocated is less than 128bytes, the Memory pool management (memory pool), also known as the secondary layer configuration (sub-allocation), configures one chunk of memory at a time and maintains the corresponding free list (free-list). The next time you have the same size of memory requirements, you can take it directly from the free-list. If a small chunk is released, it is reclaimed by the Configurator into the free-list.

The following sections describe memory pool management techniques.

In the second level configurator, the size of the small block memory requirements are raised to multiples of 8, such as the size of the need to allocate 30bytes, automatically adjusted to 32bytes. The system maintains a total of 16 free-lists, each managing small chunks of 8,16,...,128bytes size.

In order to maintain a linked list, additional pointers are required, and in order to avoid an additional burden, a technique is used to represent a linked list node structure by union:

[CPP] view plain copy print?    Union obj {Union obj * free_list_link;//refers to the next node char client_data[1]; /* The client sees this. */   };

Union can achieve the effect of one thing and two, when the node refers to the memory block is a free block, obj is treated as a pointer to another node. When a node is assigned, it is treated as a pointer to the actual block.

The following is an overview of the second level configurator overall implementation code:[CPP] View Plain copy print? template <bool threads, int inst>   class __default_alloc_template  {      private:     //  We should actually use  static const  int x = n     //  to replace  enum { x = n    But there are still not many translators supporting that quality.    # ifndef __sunpro_cc       enum {__align = 8 };       enum {__MAX_BYTES = 128};        enum {__NFREELISTS = __MAX_BYTES/__ALIGN};   # endif      static size_t round_up (size_t bytes)  {            return  (((bytes)  + __align-1)  & ~ (__align - 1));      }   __private:     union obj {           union obj * free_list_link;            char client_data[1];    / * the client sees this. */     };   private:   #  ifdef __SUNPRO_CC       static obj * __VOLATILE  free_list[];            // Specifying a  size results in duplicate def for 4.1   # else        static obj * __VOLATILE free_list[__NFREELISTS];    #  endif     static  size_t freelist_index (size_t bytes)  {            return  ((bytes)  + __align-1)/__ Align - 1);     }        // returns an object of  size n, and optionally adds to size n free list.      static void *refill (size_t n);     // Allocates  a chunk for nobjs of size  "Size".   nobjs may be reduced      // if it is inconvenient to allocate the  requested number.     static char *chunk_alloc (size_t size, int &NBSP;&AMP;NOBJS);        // Chunk allocation state.     static char *start_free;     static char *end_free;      static size_t heap_size;       /* n must be  > 0      */     static void * allocate (size_t  n) {...}        /* p may not be 0 */     static  void deallocate (void *p, size_t n) {...}     static void * reallocate (void *p, size_t old_sz, size_t &NBSP;NEW_SZ);      template <bool threads, int inst>   char *__default_alloc_template<threads, inst>::start_free = 0;//Memory Pool Start location       template <bool threads, int inst>   CHAR&NBSP;*__DEFAULT_ALLOC_ template<threads, inst>::end_free = 0;//Memory pool End position       template < bool threads, int inst>   size_t __default_alloc_template<threads,  Inst>::heap_size = 0;   template <bool threads, int inst>   __default_alloc_template< threads, inst>::obj * __volatile   __default_alloc_template<threads, inst > ::free_list[   # ifdef __sunpro_cc       __ nfreelists   # else       __default_alloc_template<threads,  inst>::__NFREELISTS   # endif  ] = {0, 0, 0, 0,  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, };   

Space Configuration function allocate ()

The implementation is as follows: the chunk to be allocated is less than 128bytes, and the first level Configurator is invoked. Otherwise, seek help from the corresponding free-list. The corresponding free list has a usable block, which can be used directly. If no blocks are available, the call function refill () fills the space with the free list.

The code is as follows:

[CPP] View Plain copy print?  /* n must be > 0      */      static void * allocate (size_t n)      {        obj * __VOLATILE * my_free_list;       obj *  __RESTRICT result;          if  (n >  size_t )  {           return (malloc_alloc::  __max_bytes) Allocate (n));       }       my_free_list =  free_list + freelist_index (n);       // acquire the  lock here with a constructor call.       //  This ensures that it is released in exit or during stack       // unwinding.   #        ifndef _NOTHREADS           /*REFERENCED*/            lock lock_instance;   #        endif       result = *my_free_list;       if  (result == 0)  {            void *r = refill (ROUND_UP (n));            return r;       }       *my_free_ list = result -> free_list_link;       return  (Result ;     };   Note here is that each time you remove the available blocks of memory from the header of the corresponding free list.
This is illustrated below:

Figure I remove the free block sketch

refill ()-Fills space for free list

This function needs to be called to repopulate the space when it is found that the corresponding free list does not have an available block. The new space will be taken from the memory pool. The management of the memory pool is described later.

20 new blocks are obtained by default, but if there is not enough memory pool, the number of nodes may be less than 20. The following is the source code in the SGI STL:

[CPP] View Plain copy print? /* returns an object of size n, and optionally adds to  size n free list.*/  /* we assume that n is properly  aligned.                                 */   /* we hold the allocation lock.                                           */   Template  <bool threads, int inst>   Void* __default_alloc_template<threads,  inst>::refill (size_t n)    {       int nobjs = 20;       char * chunk = chunk_alloc (n,  NOBJS);       obj * __VOLATILE * my_free_list;       obj * result;       obj * current_obj , * next_obj;       int i;           if  (1&NBSP;==&NBSP;NOBJS)  return (chunk);       my_free_list  = free_list + freelist_index (n);          /*  build free list in chunk */         result  =  (obj *) chunk;         *my_free_list = next_ obj =  (obj *) (chunk + n);         for  (i  = 1; ; i++)  {//The nodes together (note that the index 0 is returned to the client)             current_obj = next_obj;           next_obj  =  (obj *) ((char *) next_obj + n);            if  (nobjs - 1 == i)  {                current_obj -> free_list_link = 0;                break;            } else {                current_obj -> free_list_link = next_obj;            }         }    &NBSP;&NBSP;&NBSP;&NBSp;return (Result);  }  

chunk_alloc-take space from the memory pool for free list use

The concrete realization thought is as follows: The memory pool remaining space completely satisfies 20 blocks the demand, then takes out the corresponding size space directly. The remaining space of the memory pool can not fully meet the demand of 20 blocks, but enough to supply one or more blocks, then take out the space of the number of blocks that can satisfy the condition. The memory pool space can not meet the size of a block, it is first to determine whether there are residual memory space in the memory pool, if there is a recovery, into the free list. Then apply space to the heap to replenish the memory pool. Heap space to meet, space allocation success. Insufficient heap space, malloc () call failed. Search for the appropriate free list (as appropriate: there are unused blocks, and the blocks are large enough) to be adjusted for release and into the memory pool. Then recursively calls the Chunk_alloc function to take space from the memory pool for free list. Searching free list freeing space also fails to resolve the problem by invoking the first level configurator and using the out-of-memory mechanism to try to solve the out-of-memory problem. If you can succeed, exclude bad_alloc exceptions.

The source code is as follows:

[CPP] View Plain copy print? /* we allocate memory in large chunks in order to avoid  fragmenting     */  /* the malloc heap too  much.                                              */  /* we assume that size  is properly aligned.                              */   /* we hold the allocation lock.                                          */   template <bool threads, int inst>   char*   __DEFAULT_ALLOC_ Template<threads, inst>::chunk_alloc (SIZE_T&NBSP;SIZE,&NBSP;INT&AMP;&NBSP;NOBJS)    {        char * result;       size_t total_ bytes = size * nobjs;       size_t bytes_left =  end_free - start_free;          if  (bytes_left  >= total_bytes)  {           result =  start_free;           start_free += total_ bytes;           return (Result);        } else if  (bytes_left >= size)  {            nobjs = bytes_left/size;           total_ bytes = size * nobjs;           result  = start_free;           start_free +=  total_bytes;           return (Result);        }&