Primary knowledge nginx--Memory Pool Chapter

First, Memory pool overview

A memory pool is a pre-applied allocation of a certain number of memory blocks of equal size (in general) to be reserved before the memory is actually used. When there is a new memory requirement, a portion of the memory block is separated from the memory pool, and if the memory block is not enough, then continue to request new memory.

The benefits of a memory pool include reducing the time overhead of requesting and releasing memory to the system, resolving fragmentation caused by frequent memory allocations, prompting program performance, and reducing the memory concerns of programmers in writing code.

Some common memory pool implementations include the memory allocation area in the STL, the Ngx_pool_t,google Open source project Tcmalloc in boost Object_pool,nginx, etc.

Second, Nginx Memory pool Summary

Nginx is a TCP connection, HTTP requests, modules are allocated a pool of memory, at the end of the time will destroy the entire memory pool, the allocated memory is returned to the operating system at once.

In the allocated memory, Nginx has the concept of small chunks of memory and large chunks of memory, the small block of memory Nginx will attempt to allocate in the current memory pool node, while large chunks of memory will call system functions malloc to the operating system request

When releasing memory, Nginx does not specifically provide for the release of small memory functions, small pieces of memory will be released in Ngx_destory_pool and Ngx_reset_pool

There are 2 reasons to differentiate between small chunks of memory and large chunks of memory,

1, for large chunks of memory if its life cycle is much shorter than its own memory pool, it makes sense to provide a separate release function, but it does not differentiate between large chunks of memory and small chunks of memory, and it cannot be released prematurely for larger chunks of memory.

2. The limit of chunk memory and small block memory is one page of memory (P->max = (Size < Ngx_max_alloc_from_pool)? SIZE:NGX_MAX_ALLOC_FROM_POOL, Macro Ngx_max_alloc_from_pool by calling getpagesize activity), is more than one page in existence physically not necessarily contiguous

So if you allocate more memory than one page, use it from the memory pool, and re-apply to the operating system for the same efficiency.

Nginx Memory Pool provides a number of functions mainly include the following

Three, nginx memory pool detailed

Nginx uses ngx_pool_s to represent the entire memory pool object, ngx_pool_data_t represents the allocation information for a single memory pool node, ngx_pool_large_s represents large chunks of memory

Their structure and meaning are as follows

struct NGX_POOL_LARGE_S {
ngx_pool_large_t *next;
void *alloc;
};

Next: Point to the next chunk of memory

Alloc: Point to allocated chunk memory

struct ngx_pool_s {
ngx_pool_data_t D;
size_t Max;
ngx_pool_t *current;
Ngx_chain_t *chain;
ngx_pool_large_t *large;
ngx_pool_cleanup_t *cleanup;
ngx_log_t *log;
};

D: Data distribution of nodes in the memory pool

Max: Maximum size of the memory pool

Current: Point to the present memory pool node

Chain: Point to a ngx_chain_t structure

Large: Pointing to the large memory chain list

Cleanup: Releasing the callback of the memory pool

LOG: For output log

typedef struct {
U_char *last;
U_char *end;
ngx_pool_t *next;
Ngx_uint_t failed;
} ngx_pool_data_t;

Last: The current memory pool has been assigned a final address, and the next assignment will try to start

End: Where the memory pool node ends

Next:next refers to the next piece of memory pool node

Failed: Number of current memory pool node allocation failures

Nginx Memory Pool 1

When allocating memory, Nginx will determine whether the current memory allocated is small block of memory or large chunks of memory, large memory calls Ngx_palloc_large to allocate, small block of memory, Nginx will first try to allocate from the memory pool's current node (p->current), If there is not enough space left in the current node of the memory pool, Nginx calls Ngx_palloc_block to create a new memory pool node and allocate it from there.

If the allocation failure of the current node of the memory pool is greater than or equal to 6 times (p->d.failed++ > 4), the current memory pool node is moved forward one

(There is a need to note that when the remaining space of the current memory node is not enough allocated, Nginx will re-create a ngx_pool_t object, and will Pool.d->next point to the new ngx_pool_t, the newly assigned Ngx_pool_ The T object uses only the ngx_pool_data_t area, and there is no header information, and the Head Information section has been used as the memory allocation area.

Ngx_palloc Code

Nginx Memory Pool 2 (creates a new memory pool node and allocates 2 large chunks of memory, one of which has been freed)

Iv. Sample Code

Here is a direct replacement of the original Nginx code of the main function (SRC/CORE/NGINX.C)

void Print_pool (ngx_pool_t *pool) {if (Pool->large! = NULL) {printf ("has large memory\n"); for (ngx_pool_large_t* i = pool->large; i!=null; i = i->next) {printf ("\t\tlarge next=0x%x\n", i-&            Gt;next);        printf ("\t\tlarge alloc=0x%x\n", I->alloc);    }} int i=1;        while (pool) {printf ("pool=0x%x,index:%d\n", pool, i++);        printf ("\t\tlast=0x%x\n", (Pool->d). last);        printf ("\t\tend=0x%x\n", (pool->d). end);        printf ("\t\tnext=0x%x\n", (Pool->d). Next);        printf ("\t\tfailed=%d\n", pool->d.failed);        printf ("\t\tmax=%d\n", Pool->max);        printf ("\t\tcurrent=0x%x\n", pool->current);        printf ("\t\tchain=0x%x\n", Pool->chain);        printf ("\t\tlarge=0x%x\n", Pool->large);        printf ("\t\tcleanup=0x%x\n", pool->cleanup);        printf ("\t\tlog=0x%x\n", Pool->log);        printf ("\t\tavailable pool memory=%d\n", pool->d.end-pool->d.last); Printf("\ n");    pool=pool->d.next; }}

void Print_array (int *a,int size) {for    (int i=0; i<size; i++)

    {

printf ("%d", a[i]);

    }

    printf ("\ n");

}

int main ()

{ngx_pool_t *pool;    int array_size = 128;    int array_size_large = 1024;    int page_size = GetPageSize ();    printf ("page_size:%d\n", page_size);    printf ("----------------------------\ n");    printf ("Create a new pool");    Pool = Ngx_create_pool (1024x768, NULL);    Print_pool (pool);    printf ("----------------------------\ n");    printf ("Alloc block 1 from the pool:\n");    int *a1 = Ngx_palloc (pool, sizeof (int) * array_size);    for (int i=0; i< array_size; i++) {a1[i] = i+1;    } print_pool (pool);    printf ("----------------------------\ n");    printf ("Alloc block 2 from the pool:\n");    int *a2 = Ngx_palloc (pool, sizeof (int) * array_size);    for (int i=0; i< array_size; i++) {a2[i] = 12345678;    } print_pool (pool);    printf ("----------------------------\ n");    printf ("Alloc large memory:\n");    int * a3 = Ngx_palloc (pool, sizeof (int) * array_size_large);    for (int i=0; i< array_size_large; i++) {a3[i] = i+1; } prinT_pool (pool);    Print_array (a1,array_size);    Print_array (a2,array_size);    Print_array (A3,array_size_large);    Ngx_destroy_pool (pool); return 0;}

Operation Result:

With the red box you can see that only the header information of the first memory pool node in ngx_pool_t is meaningful, and the header information of the nodes created by subsequent calls to Ngx_palloc_block has been overwritten by the data.

Primary knowledge nginx--Memory Pool Chapter