Analysis of nginx memcached Module

1 nginx memcached Module
1.1 Overview
Nginx memcached is a module that uses memory to accelerate page access. When a client request arrives at the nginx server, nginx first accesses the memcached server through a key value (such as URI, when the data can be obtained from the memcached server, the data is directly encapsulated and returned to the client. Otherwise, the system continues to access related services such as PHP and obtains the content from related applications and sends it to the client, at the same time, related applications actively write content to the memcached server, so that the next access can accelerate the performance.
The interaction between the nginx memcached module and the memcached server is implemented through the nginx upstream mechanism. Note that the official memcached module only has the ability to access and read the memcached server, but does not write the content to the memcached server. some third-party modules have this function.


1.2 Configuration
A typical memcached access configuration is as follows:
Server {
Listen 80;
SERVER_NAME 192.168.3.139;
Root HTML;
 
Location ~ * \. (GIF | JPG | JPEG | PNG | BMP | SWF | JS | CSS) $ {
Set $ memcached_key $ URI;
Default_type text/html;
Memcached_pass 192.168.3.139: 11211; // The memchaced server is on 192.168.3.139. when accessing the URI, the server first obtains the resource. When not, the server jumps to the @ goto404 branch for processing.
Error_page 404 = @ goto404; // when a 404 error occurs, redirect to @ goto404 for processing.
}
 
Location @ goto404 {
Fastcgi_pass 127.0.0.1: 9000;
Fastcgi_index index. php;
Include FastCGI. conf;
Rewrite ^ (.*)? /Goto. php? Q = $1 break;
}
 
}


1.3 source code analysis
The memcached module relies on the nginx upstream module for implementation. The main processing part is also embedded into the memcached module and registered in various callback functions of upstream. The following uses version 1.0.15 for source code analysis:

The configuration of the memcached module is mainly the pass_proxy keyword, and the related processing function is ngx_http_memcached_pass. The processing process includes creating the data structure of upstream, but the most important thing is the following sentence:

Clcf-> handler = ngx_http_memcached_handler;
Ngx_http_memcached_handler; the callback function is registered to the content Phase Stage in 11 stages of request packet processing. At this stage, the processing function is called.
So what did we do in this processing function? In terms of code, we mainly initialize the various data structures used by the upstream mechanism and set the various callback functions called by the upstream mechanism, prepare for connecting to the upstream memcached server. The code is parsed as follows:

Static aggregate (ngx_http_request_t * r) {ngx_int_t RC; ngx_http_upstream_t * U; ngx_http_memcached_ctx_t * CTX; ngx_http_memcached_loc_conf_t * mlcf; If (! (R-> Method & (ngx_http_get | ngx_http_head) {return ngx_http_not_allowed;} // discard the Request body. Rc = ngx_http_discard_request_body (R) is not concerned with the request body ); if (RC! = Ngx_ OK) {return RC;} If (ngx_http_set_content_type (r )! = Ngx_ OK) {return ngx_http_internal_server_error;} // create an upstream data structure if (ngx_http_upstream_create (r )! = Ngx_ OK) {return ngx_http_internal_server_error;} u = r-> upstream; ngx_str_set (& U-> schema, "memcached: //"); U-> output. tag = (ngx_buf_tag_t) & ngx_http_memcached_module; mlcf = ngx_http_get_module_loc_conf (R, ngx_http_memcached_module); U-> conf = & mlcf-> upstream; // The following are the callback functions used by upstream. // The callback function creates a GET request, which is sent to the upstream server U-> create_request = ngx_http_memcached_create_request; u-> reinit_request = ngx_http_memcached_reinit_request; // process the response packet header returned by the memcache server, calculate its header, and calculate the response packet // Data Length U-> process_header = ngx_http_memcached_process_header; u-> abort_request = callback; U-> finalize_request = ngx_http_memcached_finalize_request; CTX = ngx_palloc (R-> pool, sizeof (ngx_http_memcached_ctx_t); If (CTX = NULL) {return ngx_http_internal_server_error;} // CTX-> the rest is initialized here. The rest is used in a single reception, // part of \ r \ nend \ r \ n has been received, but not completely received. It is used to record // The Rest Of The unreceived part CTX-> rest = ngx_http_memcached_end; CTX-> request = r; ngx_http_set_ctx (R, CTX, ngx_http_memcached_module); U-> input_filter_init = ngx_http_memcached_filter_init; // This function is used to process the response package body, mount the Buf package to the out linked list of the request, U-> input_filter = ngx_http_memcached_filter; U-> input_filter_ctx = CTX; r-> main-> count ++; // initiate a connection to the upstream memcached server ngx_http_upstream_init (r); Return ngx_done ;}

In the above Code, we mainly focus on the functions set for the create_request, process_header, and input_filter fields set in upstream:
Ngx_http_memcached_create_request: used to construct request packets sent to the memcached server.
Ngx_http_memcached_process_header: process the corresponding headers returned by the memcached server, calculate the length of the response body, and determine the request access status (whether the request is successful ).
Ngx_http_memcached_filter: package all the responses received by upstream into out_buf of reuqest, and check whether the receipt of the package body ends.
The three functions will be parsed in detail below. Before that, let's take a brief look at the interactive packets of the memcached Server:

Authorization: this function is mainly used to construct packets sent to the memcached server. The message format is get URI \ r \ nstatic Digest (ngx_http_request_t * r) {size_t Len; uintptr_t escape; ngx_buf_t * B; ngx_chain_t * Cl; ngx_http_memcached_ctx_t * CTX; ngx_http_variable_value_t * vv; Memory * mlcf; mlcf = memory (R, ngx_http_memcached_module); VV = ngx_http_get_inde Xed_variable (R, mlcf-> index); If (VV = NULL | vv-> not_found | vv-> Len = 0) {return ngx_error ;} escape = 2 * ngx_escape_uri (null, VV-> data, VV-> Len, ngx_escape_memcached); // calculate the length of the message sent to the memcached server. Len = sizeof ("get ") -1 + VV-> Len + escape + sizeof (CRLF)-1; // allocate the relevant Buf, chain, it is used to write data packets into it B = ngx_create_temp_buf (R-> pool, Len); If (B = NULL) {return ngx_error;} Cl = ngx_alloc_chain_link (R-> Pool); If (CL = NULL) {return ngx_error;} Cl-> Buf = B; CL-> next = NULL; r-> upstream-> request_bufs = Cl; // write the get keyword * B-> last ++ = 'G'; * B-> last ++ = 'E '; * B-> last ++ = 'T'; * B-> last ++ = ''; CTX = ngx_http_get_module_ctx (R, ngx_http_memcached_module); CTX-> key. data = B-> last; // write URL if (Escape = 0) {B-> last = ngx_copy (B-> last, VV-> data, VV-> Len);} else {B-> last = (u_char *) ngx_escape_uri (B-> L AST, VV-> data, VV-> Len, ngx_escape_memcached);} CTX-> key. len = B-> last-CTX-> key. data; // set the end mark of the message \ r \ n * B-> last ++ = CR; * B-> last ++ = lf; return ngx_ OK ;} 2 encode (ngx_http_request_t * r) {u_char * P, * Len; ngx_str_t line; ngx_http_upstream_t * U; ngx_http_memcached_ctx_t * CTX; u = r-> upstream; for (P = u-> buffer. pos; P <u-> B Uffer. last; P ++) {If (* P = lf) {goto found;} return ngx_again; found: * P = '\ 0'; line. len = p-u-> buffer. pos-1; line. data = u-> buffer. pos; P = u-> buffer. pos; CTX = ngx_http_get_module_ctx (R, ngx_http_memcached_module); // The value keyword returned by the packet header indicates that the memcached server has the relevant data content if (ngx_strncmp (P, "value ", sizeof ("value")-1) = 0) {P + = sizeof ("value")-1; if (ngx_strncmp (p, CTX-> key. data, CTX-> key. len )! = 0) {return ngx_http_upstream_invalid_header;} p + = CTX-> key. Len; If (* P ++! = '') {Goto no_valid;}/* Skip flags */while (* P) {If (* P ++ ='') {goto length ;}} goto no_valid; Length: Len = P; while (* P & * P ++! = Cr) {/* void * // calculate the length of the corresponding message r-> headers_out.content_length_n = ngx_atoof (Len, p-len-1 ); if (R-> headers_out.content_length_n =-1) {return ngx_http_upstream_invalid_header;} U-> headers_in.status_n = 200; U-> state-> Status = 200; U-> buffer. pos = p + 1; return ngx_ OK;} // when the end \ x0d is returned, it indicates that the memcached server does not have the content if (ngx_strcmp (P, "end \ x0d ") = 0) {u-> headers_in.status_n = 404; U-> state-> Status = 404; Return ngx_ OK;} no_valid: Return variable;} 3 ngx_http_memcached_filter static variable (void * data, ssize_t bytes) {optional * CTX = data; u_char * last; ngx_buf_t * B; ngx_chain_t * Cl, ** ll; ngx_http_upstream_t * U; u = CTX-> request-> upstream; B = & U-> buffer; // This branch means that some ngx_http_memcached_end nodes have not yet received the request. // If (u-> length = CTX-> r Est) {If (ngx_strncmp (B-> last, ngx_http_memcached_end + ngx_http_memcached_end-CTX-> rest, bytes )! = 0) {u-> length = 0; CTX-> rest = 0; return ngx_ OK;} U-> length-= bytes; CTX-> rest-= bytes; return ngx_ OK;} For (CL = u-> out_bufs, LL = & U-> out_bufs; CL = Cl-> next) {ll = & CL-> next ;} CL = ngx_chain_get_free_buf (CTX-> request-> pool, & U-> free_bufs); If (CL = NULL) {return ngx_error ;} CL-> Buf-> flush = 1; CL-> Buf-> memory = 1; * LL = Cl; // B is the Buf that upstream currently uses to receive data. last = B-> last; // the starting position of the data is B-> las T address Cl-> Buf-> Pos = last; // reset the value of B-> last to use B-> last + = bytes when receiving subsequent data; CL-> Buf-> last = B-> last; CL-> Buf-> tag = u-> output. tag; // here the data is not received. If (Bytes <= (ssize_t) (u-> length-ngx_http_memcached_end) {u-> length-= bytes; return ngx_ OK;} // find the start position of \ r \ nend \ r \ n last + = u-> length-ngx_http_memcached_end; // compare whether the received data ends with if (ngx_strncmp (last, ngx_http_memcached_end, B-> last-last )! = 0) {ngx_log_error (ngx_log_err, CTX-> request-> connection-> log, 0, "memcached sent invalid trailer ");} // It indicates that the data has been received. Of course, some of the data may have been received. // The ngx_http_memcached_end data has not been received. // CTX-> rest records the data that has not been fully received, adjust CTX-> rest-= B-> last-last; // update the address used by the worker for the next data reception, this also indicates that the end // operator does not need to be passed to downstream client B-> last = last; CL-> Buf-> last = last; U-> length = CTX-> rest; return ngx_ OK ;}

When memcached hits, the above three functions can basically deal with everything, but when the server returns the end \ x0d string, it indicates that the memcached server does not have this content. Check the relevant code. nginx intends to return the Error 404. At this time, error_page is usually used to redirect it to the real resources of the Real Server. These tasks are done in the ngx_http_upstream_finalize_request function. This function is complex and can be processed in many cases. Taking 404 redirection as an example, the function call sequence is:
Ngx_http_upstream_finalize_request ---> ngx_http_finalize_request -----> ngx_http_sepecial_response_handler ---> ngx_http_send_error_page ----> ngx_http_internal_redirect ---> ngx_http_handler


In the ngx_http_handler function:
Cproductname = ngx_http_get_module_main_conf (R, ngx_http_core_module );
R-> phase_handler = colap-> phase_engine.server_rewrite_index;
The request is redirected to the URL redirection phase in 11 phases and continues to be executed.

Analysis of nginx memcached Module