Detailed Nginx the configuration function for the request body read _nginx

The Nginx core itself does not actively read the request body, which is given to the module at the request processing stage, but the Nginx core provides a ngx_http_read_client_request_body () interface to read the request body. In addition, an interface-ngx_http_discard_request_body () for discarding the request body is provided, in which the modules of any stage, if they are interested in the request body or if they wish to lose the requested body sent by the client, can be called to complete each of the two interfaces. These two interfaces are standard interfaces for the processing request body provided by the Nginx core, and if you want some of the request-body-related directives (such as client_body_in_file_only,client_body_buffer_size, etc.) in the configuration file to work, As well as some of the request-related variables (such as $request_body and $request_body_file) that are built into the normal use of nginx, all modules must call these interfaces to complete the operation, and if a custom interface is required to handle the request body, You should also try to be compatible with nginx default behavior.

1, read the request body

The request body reads generally occurs in the Nginx content handler, some Nginx built-in module, for instance proxy module, fastcgi module, Uwsgi module and so on, The behavior of these modules must be forwarded to the backend service process by the requesting body (if any) from the client, all of which call the Ngx_http_read_client_request_body () interface to complete the request body read. It is noteworthy that these modules will be the client's request body after the full read to start forwarding data back.

Due to memory limitations, the request read by the Ngx_http_read_client_request_body () interface is partially or wholly written to a temporary file, depending on the size of the request body and the associated instruction configuration, the request body may be completely placed in a contiguous memory. It can also be placed in two different pieces of memory, possibly all in one temporary file, and finally some in memory, and the remainder in temporary files. Here are some instructions related to these different storage behaviors:

Client_body_buffer_size: Sets the buffer size of the cache request body, defaults to twice times the system page size, and when the request body size exceeds this size, Nginx writes the request body to the temporary file. Can be set to the appropriate size according to business requirements, as far as possible to avoid disk IO operation;

Client_body_in_single_buffer: Indicates whether the body of the request is stored in a contiguous memory, the default is off, and if this instruction is set to ON, then Nginx will ensure that the requested body is not in the Client_body_buffer_ Size set, is stored in a contiguous memory, but when the size is exceeded, it is written to a temporary file;

Client_body_in_file_only: Sets whether the request body is always saved in a temporary file, the default is off, and when this designation is set to ON, Nginx still creates a temporary file for the request, even if the client display indicates that the request body length is 0 o'clock.

It then introduces the implementation of the Ngx_http_read_client_request_body () interface, which is defined as follows:

ngx_int_t 
ngx_http_read_client_request_body (ngx_http_request_t *r, 
 

The interface has 2 parameters, the 1th is a pointer to the request structure, and the 2nd is a function pointer, which is invoked when the request body is read. It was also mentioned that according to Nginx existing behavior, the module logic will be executed after the request body is read, and this callback function is usually the logic processing function of the module. The Ngx_http_read_client_request_body () function first adds 1 to the reference of the main request corresponding to the parameter r, and the purpose of doing so is related to the context in which the interface is invoked, and in general, the module calls this interface in the content handler. A typical call is as follows:

Static ngx_int_t 
Ngx_http_proxy_handler (ngx_http_request_t *r) 
{ 
 ... 
 rc = Ngx_http_read_client_request_body (R, Ngx_http_upstream_init); 
 
 
 if (RC >= ngx_http_special_response) {return 
 rc; 
 } 
 
 return ngx_done; 
}

The above code invokes the Ngx_http_read_client_request_body () function in the content Handler,ngx_http_proxy_handler () of the Porxy module, where ngx_http_ Upstream_init () is passed into the interface as a callback function, and the context of the content handler invocation of the module in Nginx is as follows:

ngx_int_t 
ngx_http_core_content_phase (ngx_http_request_t *r, 
 ngx_http_phase_handler_t *ph) 
{ 
 ... 
 if (r->content_handler) { 
 r->write_event_handler = Ngx_http_request_empty_handler; 
 Ngx_http_finalize_request (R, R->content_handler (r)); 
 return NGX_OK; 
 } 
 ... 

In the above code, after the content handler call, its return value calls the Ngx_http_finalize_request () function as an argument, and the request body is not received completely, ngx_http_read_client_request_ The body () function returns a value of Ngx_again, at which point the content handler, such as Ngx_http_proxy_handler (), returns Ngx_done, and Ngx_done is passed as a parameter to Ngx_http_finalize_ The request () function causes the reference count of the main request to be reduced by 1, which exactly offsets the addition of the Ngx_http_read_client_request_body () function to the main request count of 1.

Next go back to the Ngx_http_read_client_request_body () function, which checks whether the requested body of the request has been read or discarded, and if so, calls the callback function directly and returns NGX_OK, which is actually a check for the child request, A child request is a concept in nginx that Nginx can initiate another or more entirely new child requests in the current request to access other location, and the specific description of the child request is detailed in the later chapters, and generally the child request does not need to read the request itself.

The function then calls Ngx_http_test_expect () to check whether the client sent the expect:100-continue header, then to the client to reply "http/1.1 Continue", according to the HTTP 1.1 protocol, The client can send a expect header to indicate to the server that it expects to send the request body, and if the server allows the client to send the request body, it will reply to "http/1.1 Continue" and the client will begin sending the request body when it receives it.

It then continues to prepare for the receiving body, allocating a ngx_http_request_body_t structure and saving it in R->request_body, which is used to hold information such as cache references, temporary file references, remaining request body sizes used in the request body read process, It is defined as follows.

<span style= "FONT-FAMILY:SIMSUN;FONT-SIZE:18PX;" >typedef struct { 
 ngx_temp_file_t   *temp_file; 
 ngx_chain_t   *bufs; 
 ngx_buf_t   *buf; 
 off_t    rest; 
 ngx_chain_t   *to_write; 
 Ngx_http_client_body_handler_pt Post_handler; 
} Ngx_http_request_body_t;</span> 

• Temp_file: A pointer to a temporary file that stores the requested body;
• Bufs: Point to the chain header that holds the request body;
• BUF: Point to the memory cache that is currently used to hold the request body;
• Rest: The size of the currently remaining request body;
• Post_handler: Saves the callback function passed to the Ngx_http_read_client_request_body () function.

When you are ready, the function begins checking to see if the request has a content_length header, or if the client sends a CONTENT_LENGTH header with a value of 0, indicating that there is no request body, call the callback function directly and return to NGX_OK. Of course, if the client_body_in_file_only instruction is set to ON and Content_length is 0 o'clock, the function creates an empty temporary file before calling the callback function.

Into the lower part of the function, which indicates that the client request does indicate that the request body is to be sent, the function checks to see if the request body was read in while reading the request header, where the check is to determine if there are any unhandled data in the cache (R->HEADER_IN) that holds the request headers. If there is read-ahead data, a ngx_buf_t structure is assigned and the read-only data in the r->header_in is saved, and if there is space left in the r->header_in, and the remaining unread request body can be contained, the space will continue to be used. Instead of allocating a new cache, of course, even if the request body has been read all over, you do not need to continue processing, when the callback function is invoked and returned.

If there is no prefetch data or incomplete prefetching, the function allocates a new memory (unless the r->header_in has enough space left), and if the request_body_in_single_buf instruction is set to No, The prefetch data is copied into the newly opened memory block, and the actual read request body is in the ngx_http_do_read_client_request_body () function, which loops through the request body and is stored in the cache, if the cache is fully written, The data will be emptied and written back into the temporary file. Of course, there's probably no time to read the data at once, the function hangs the read event and sets the read event handler to Ngx_http_read_client_request_body_handler, and the Nginx core also makes time-out settings between the read events of the two request bodies , the client_body_timeout instruction can set this timeout, the default is 60s, and if the next read event times out, Nginx will return 408 to the client.

After the final reading of the request body, ngx_http_do_read_client_request_body () adjusts the request body to the desired location (memory or file) according to the configuration, and in all cases the request body is available from the R->request_ The body of the Bufs linked list, the list may have a maximum of 2 nodes, each node is a buffer, but the contents of this buffer may be stored in memory, or may be saved in the disk file. In addition, the $request_body variable can get the appropriate data only when the requesting body has been read and is all stored in memory.

2, discard the request body

A module wants to actively discard the client-sent request body, you can invoke the Ngx_http_discard_request_body () interface provided by the Nginx core, there may be many reasons for active discarding, such as the module's business logic does not require the request body at all, The client sends an too large request body, and in addition to the pipeline request for compatibility with the HTTP1.1 protocol, the module has the obligation to actively discard unwanted request bodies. In a nutshell, to maintain good client compatibility, Nginx must actively discard unwanted request bodies. The following starts parsing the Ngx_http_discard_request_body () function:

ngx_int_t ngx_http_discard_request_body (ngx_http_request_t *r) {ssize_t size; 
 
 ngx_event_t *rev; 
 if (r!= R->main | | r->discard_body) {return NGX_OK; 
 } if (Ngx_http_test_expect (r)!= Ngx_ok) {return ngx_http_internal_server_error; 
 
 Rev = r->connection->read; 
 
 Ngx_log_debug0 (ngx_log_debug_http, Rev->log, 0, "HTTP set Discard Body"); 
 if (rev->timer_set) {Ngx_del_timer (rev); 
 } if (r->headers_in.content_length_n <= 0 | | | r->request_body) {return NGX_OK; 
 
 Size = r->header_in->last-r->header_in->pos; 
  if (size) {if (R->headers_in.content_length_n > Size) {r->header_in->pos = = size; 
 
 R->headers_in.content_length_n-= size; 
  else {R->header_in->pos = = (size_t) r->headers_in.content_length_n; 
  R->headers_in.content_length_n = 0; 
 return NGX_OK; 
 
 } R->read_event_handler = Ngx_http_discarded_request_body_handler; if (Ngx_handle_read_evENT (rev, 0)!= Ngx_ok) {return ngx_http_internal_server_error; 
 
 } if (Ngx_http_read_discarded_request_body (r) = = NGX_OK) {r->lingering_close = 0; 
 else {r->count++; 
 R->discard_body = 1; 
return NGX_OK; 

 }

Because the function is not long, here it is fully listed, the beginning of the function also first judged that no need to do the processing of the case: the child request does not need processing, has called the function does not need to be processed. Then call Ngx_http_test_expect () to handle the http1.1 expect situation, according to the http1.1 expect mechanism, if the client sends the expect header, and the server does not want to receive the request body, must return 417 ( Expectation Failed) error. Nginx did not do this, it simply lets the client send the request body and discard it. Next, the function deletes the timer on the read event, because at this time does not need to request the body, so it does not matter whether the client sent fast or slow, of course, will come later, when Nginx has processed the request but the client has not sent out the useless request body, Nginx will be on the read event and then hang up the timer.
The function also checks the Content-length header in the request header, and if the client intends to send the request body, it must send a content-length header and see if the request body has been read elsewhere. If the requested body is really waiting to be processed, the function then checks the prefetch data in the request header buffer, and the read-ahead data is discarded directly, although the function is returned directly if the request body has been read all over.

Next, if there is still the remaining request body unhandled, the function calls Ngx_handle_read_event () to mount the read event in the event-handling mechanism and set the processing function of the read event to Ngx_http_discarded_request_body_ Handler After doing these preparations, the function finally calls the Ngx_http_read_discarded_request_body () interface to read the client's request body and discard it. If the client does not send the body of the request at once, the function returns, and the remaining data is passed to Ngx_http_discarded_request_body_handler () when the next read event is over, and the requested Discard_ The body will be set to 1 to identify the situation. The additional requested reference count (count) is also added by 1, so that the client may not be able to send the requested body in full after the request has been processed by Nginx, and the addition of the reference is to prevent the Nginx core from releasing the requested resource directly after processing the request.

The

Ngx_http_read_discarded_request_body () function is so simple that it loops through the link to read the data and discard it until all the data in the receiving buffer is read, and if the request body has been read, the function sets the Read event's processing function to Ngx_ Http_block_reading, this function simply deletes the level-triggered read event to prevent the same event from being triggered continuously.
Take a look at the processing function of the Read event Ngx_http_discarded_request_body_handler, which is invoked each time the event is read, and first look at its source code:

void Ngx_http_discarded_request_body_handler (ngx_http_request_t *r) {... c = r->connection; 
 
 Rev = c->read; 
 if (rev->timedout) {c->timedout = 1; 
 C->error = 1; 
 Ngx_http_finalize_request (R, Ngx_error); 
 Return 
 
 } if (r->lingering_time) {timer = (ngx_msec_t) (R->lingering_time-ngx_time ()); 
  if (timer <= 0) {r->discard_body = 0; 
  R->lingering_close = 0; 
  Ngx_http_finalize_request (R, Ngx_error); 
 Return 
 } else {timer = 0; 
 
 rc = Ngx_http_read_discarded_request_body (r); 
 if (rc = = NGX_OK) {r->discard_body = 0; 
 R->lingering_close = 0; 
 Ngx_http_finalize_request (R, Ngx_done); 
 Return 
 }/* rc = = Ngx_again */if (Ngx_handle_read_event (rev. 0)!= ngx_ok) {c->error = 1; 
 Ngx_http_finalize_request (R, Ngx_error); 
 Return 
 
 } if (timer) {CLCF = ngx_http_get_module_loc_conf (R, Ngx_http_core_module); 
 
 Timer *= 1000; if (Timer > Clcf->lingering_timeout) {timer = clcf->lingering_timeout; 
 Ngx_add_timer (rev, timer); 
 } 
}

The function begins with a read event timeout, and when it comes to a timer that has been deleted in the Ngx_http_discard_request_body () function, when will the timer be set? The answer is that when Nginx has finished processing the request, but has not completely discarded the requested body of the request (the client may not have sent it over), in the Ngx_http_finalize_connection () function, if you check that there are no discarded request bodies, Nginx adds a read event timer, which is specified by the lingering_timeout instruction, defaults to 5 seconds, but this time is just two times the timeout between read events, waiting for the total length of the request body to be specified by the lingering_time instruction, default is 30 seconds. In this case, the function returns directly and disconnects if a timeout event is detected. Also, it is necessary to control that the length of the entire discard request body cannot exceed the time of the Lingering_time setting, and if the maximum length is exceeded, the connection is returned directly and disconnected.
If the Read event occurs before the request is processed, the timeout event is not handled, and the timer is not set, and the function simply calls Ngx_http_read_discarded_request_body () to read and discard the data.