Nginx of upstream module data forwarding process and flow control analysis

1. Basic Introduction

The basic principle and design of the upstream module of Nginx can be referenced by clicking on the Open link. In fact it is a special handler module, the SET function of various xx_proxy instructions, will set the Content_handler of the location, such as Xx_memcached_module ngx_http_memcached_ Handler (), Xx_proxy_module Ngx_http_proxy_handler (). You can simply read the code and find that they are almost

1. First create a upstream structure, simple initialization;

2. Mount some hook functions, such as create_request,reinit_request,process_header,finalize_request, etc. (this hook is implemented in a way that Very good unified the overall structure of various upstream modules, only in the specific agreement-related part of the difference);

3. Call Ngx_http_upstream_init (R) to initialize the upstream connection.

where the Ngx_http_upstream_init () function is a key, it finds an upstream server (based on a specific algorithm), establishes a TCP connection, and sets the downstream, upstream two connections read/write_ Handler, and added to the Epoll. There are a lot of places to learn, such as finding peer.connection methods, non-blocking TCP connections, and so on. Here, the whole content_handler is over, and then things are completely driven by events.

2. Work Flow

Look at the following work flow. A larger file transfer typically requires multiple epoll_wait to complete the request, such as requesting a picture (150K) from the upstream server, working as follows:

where Xx_upstream_process_upstream () is the key to control the flow of data, it is called ngx_event_pipe (), the specific look at this function

ngx_int_t ngx_event_pipe (ngx_event_pipe_t *p, ngx_int_t do_write) {U_int flags;
    ngx_int_t RC;

    ngx_event_t *rev, *wev; for (;;)

            {//cyclic processing, forwarding data from upstream to downstream if (do_write) {p->log->action = "sending to client";

            rc = Ngx_event_pipe_write_to_downstream (p);
            if (rc = = ngx_abort) {return ngx_abort;
            }//To downstream, then there is no need to read from the upstream if (rc = = ngx_busy) {return NGX_OK;
        } p->read = 0;

        p->upstream_blocked = 0;

        P->log->action = "reading upstream";
        if (Ngx_event_pipe_read_upstream (p) = = Ngx_abort) {return ngx_abort; }/* did not read the data, but if because of p->upstream_blocked, that is, the upstream application layer read cache is not enough, then can continue to write downstream, thereby releasing the upstream read cache, but if it is not because of this, it may be an error, or error, then exit/if (!p-
        >read &&!p->upstream_blocked) {break;
    } do_write = 1; } if (P->UPSTREAM->FD !=-1) {Continue to add read event to Epoll, and add timeout} if (p->downstream->fd!=-1 && p->downstream->
data = = P->output_ctx) {ibid.} return NGX_OK; }

where the Ngx_event_pipe_read_upstream (p) body is a for (;;) Loop, call Ngx_readv_chain (C,chain) to read the upstream data to the application layer, then through the Ngx_http_proxy_copy_filter (), filter the data, and eventually hang on the P->in chain.

The Ngx_event_pipe_write_to_downstream (p) body is also a for (;;) Loops, consolidates p->busy,p->out,p->in to form a chain out, calls Ngx_http_top_body_filter (R,out), enters the filter chain, until the last Ngx_http_write_ Filter, send the data to the client, if it is too late to send, then put the remaining data on the r->out.

3. Data Flow

Here, upstream basic work flow is clear. As mentioned earlier, you can see that in the process of transferring data, the flow control of data is often the cause of the complication of the process, and because it is difficult to predict the behavior of the two connections at the same time, especially the back-end up-server in some special cases (such as blocking), we should deal with each situation carefully. The following data flow control as an opportunity to look again at the transformation of the data process.

Data traffic control, involving both sides of the TCP connection, and with the underlying protocol is very related, there will be many problems, and then continue to see.