The lifecycle of Nginx cache mappings in memory

M

For the most recent work, I need to add a custom field (Ngx_queue) to the Nginx cache map in the data structure (ngx_http_file_cache_node_t) of the node in memory. To add this field, you must understand the life cycle of these memory nodes, from generation to extinction; otherwise, the new fields cannot be initialized when they are generated, and subsequent custom queue nodes cannot be inserted correctly.

1. The generation of memory nodes

So far, I've only found two entry functions that generate memory nodes, and there may be some missing. At the moment the node is generated, I need to initialize the custom fields in the nodes.

(Build point 1)

If proxy cache is enabled, the master process starts two of the admin caches at startup

Subprocess (cache manager process, cache loader process). Which, the latter only after the initial start-up of the 60s

, and its role is to map the cache in the periodic disk into memory, which is invoked multiple times by calling the Ngx_cache_loader

_process_handler (NGX_PROCESS_CYCLE.C) to achieve. This function mainly calls the ngx_http_file_cache_

Loader (void *data)

static void

Ngx_http_file_cache_loader (void *data) {

......

Initialize Tree

Tree.init_handler = NULL;

Tree.file_handler = ngx_http_file_cache_manager_file;//to locate the cached file, establish the memory node

Tree.pre_tree_handler = Ngx_http_file_cache_noop;

Tree.post_tree_handler = Ngx_http_file_cache_noop;

Tree.spec_handler = delete cache on disk after ngx_http_file_cache_delete_file;//memory node is established ...

if (Ngx_walk_tree (&tree,&cache_path_name) = = Ngx_abort) {}//ngx_walk_tree Complete cache

Main features to memory mapping

}

ngx_int_t

Ngx_walk_tree (ngx_tree_ctx_t *ctx, ngx_str_t *tree) {

......

for (;;) {

.......

if (Ngx_de_is_file ($dir)) {

Ctx->file_handler (CTX, $file);//The Ngx_http_file_cache_manager_file function is called here

}

}

}

In the Ngx_http_file_cache,

Static ngx_int_t

Ngx_http_file_cache_manage_file (ngx_tree_ctx_t *ctx, ngx_str_t *path) {

....

if (Ngx_http_file_cache_add_file (Ctx,path)!= Ngx_ok) {

(void) ngx_http_file_cache_delete_file (ctx,path);//If not successfully mapped to memory, delete the cached file ...

}

return Ngx_http_file_cache_manager_sleep (cache); If the current server IO resource is tight, suspend the process for a period of time

}

In the ngx_http_file_cache.c,

Static ngx_int_t

Ngx_http_file_cache_add_file () {

......

Return Ngx_http_file_cache_add (cache, &c);

}

In the ngx_http_file_cache.c,

Static ngx_init_t

Ngx_http_file_cache_add (ngx_http_file_cache_t *cache, ngx_http_cache_t *c) {//build point in this function

....

FCN = Ngx_http_file_cache_lookup (cache, C->key);//Find in memory

if (FCN = = NULL) {//If not found in memory, create a node

FCN-> uses = 1;//node initialization, my custom content should also be initialized here

FCN-> count = 0;

FCN-> exists = 1;

....

}

}

(Build point 2)

After the client request arrives, Nginx will actively retrieve the memory node, if it does not exist (regardless of whether the disk cache exists).

Static ngx_int_t

Ngx_thhp_file_cache_exists (ngx_http_file_cache_t *cache, ngx_http_cache_t *c) {

FCN = Ngx_http_file_cache_lookup (cache,c-> key);

if (FCN) {

FCN-> uses++;

FCN-> count++;

......

}

If executed here, the node does not exist.

FCN = ngx_slab_alloc_locked ()/New node
This completes the initialization of the custom structure in the node.

} 2. Logoff of Memory nodes

I need to free up the space for the custom field request while the node is logged off. Because I just need to use the node.

Complete the custom function, so I only care about how to make the node be recycled after I use it. According to my simple

Observation, it seems that only in the case of Fcn->count = 0, the system will be invoked. So, before you finish customizing the functionality,

You can prevent the node from being cleaned up by other processes by performing the FCN-> count++, and then executing the fcn->count– after completing the task.

Off.

3. Some behaviors of memory node participation

I need to understand some of the behavior of the client requesting resources to Nginx for the corresponding, memory-node involvement. Because I

You will be in the right place to achieve the functions you need.

(1) query whether the memory node exists

In the ngx_http_file_cache.c,

Static ngx_int_t

Ngx_http_file_cache_exists (Ngx_http_file_open *cache, ngx_http_cache_t *c) {

......

FCN = Ngx_http_file_cache_lookup (Cache,c->key);

if (FCN) {

FCN-> uses++;//uses seems to indicate how many times the node has been used.

FCN-> Count++;//count appears to be used to indicate how many units the node is currently in. Process. ) use

This field can be used to prevent the node from being deleted by other nodes, for example, if the Mutux lock

After unlocking still do not want the node to be deleted, need FCN-> count++; After using the end point

Need fcn->count–;

......

}

If executed here, the node does not exist.

FCN = ngx_slab_alloc_locked ()//New node

FCN-> uses = 1;

FCN-> count = 1;

Done

C->node = FCN;

}

(2) to determine whether the need to update from the source server or add a node corresponding to the cache

In the ngx_http_file_cache.c,

ngx_int_t

Ngx_http_file_cache_open () {

rc = ngx_http_file_cache_exists (CACHE.C);

Cold = cache->sh->cold;

if (rc = = NGX_OK) {//node already exists

Test = c->exists?1:0//If the cache does not exist, you will need to contact the source server

RV = ngx_declined;//Even if a memory node indicates a cache presence, if the save file on the disk cannot be successfully opened

You also need to contact the source server

else {//Memory node has just been created or failed

if (c-> min_uses > 1) {//Unclear what this branch represents

if (!cold) {//May be if C-> min_uses > 1 and currently must have c->uses = = 1, so

//temporarily do not update cache.

Test = 1;

RV = Ngx_http_cache_scarce;

} else {

C-> temp_file = 1;

Test = cold?1:0;//If cold = 0, contact the source server, I do not understand what this cold is meant ...

RV = ngx_declined;

}

}

if (!test) {

Return declined;//update point one (code execution here, will necessarily contact source Server update data)

}

if (Ngx_open_cached_file (dlcf->open_file_cache,&c->file.name,&c->of,r->pool)!= NGX_OK) {

Switch (of.err) {

Case .....

Case Ngx_enoent:

Case NGX_ENOTDIR:

Return rv;;/ /Update point two, (Code execution here, if rv==ngx_declide, will necessarily contact source Server update data)

}

}

......

Return Ngx_http_file_cache_read (R,C);

}

In the ngx_http_file_cache.c,

Static ngx_int_t

Ngx_http_file_cache_read (ngx_http_request *r, ngx_http_cache_t *c) {

......

if (H->crc32!= c->crc32) {

Return ngx_declined;//update point three, but in reality, rarely executes this

}

......

if (C->valid_sec < now) {

if (c->node->updating) {

rc = ngx_http_cache_updating; Update point four, cache expired, but not yet updated

} else {

...

rc = ngx_http_cache_stale;//Update point five, cache expired

}

}

}

(3) If you do not need to update the cache, respond directly to the client

In the NGX_HTTP_UPSTREAM.C,

Static mgx_int_t

Ngx_http_upstream_cache () {

......

Switch (RC) {

Case NGX_OK:

rc = Ngx_http_upstream_cache_send (r,u);

}

}

Static ngx_int_t

Ngx_http_upstream_cache_send () {

......

if (rc = = NGX_OK) {

if (Ngx_http_upstream_process_headers (r,u)!= Ngx_ok) {

Return Ngx_done

}

return Ngx_http_cache_send (R);//Send an existing cache back to the client

}

}

Ngx_http_upstream_process_headers () {

if (...) {

Ngx_http_upstream_finalize_request ();

}

}

Ngx_http_upstream_finalize_request () {

......

Ngx_http_file_cache_free (r->cache,u->pipe->temp_file);//This function will release the reference to the memory node and the temporary file

}