Thoughts on upstream mechanism in nginx

Those who have searched for upstream related analysis on the Internet may already have a rough understanding of it, and many of them are doing a great job in analysis. Why do I have to say a few more here? One is to summarize some of your understanding, and the other is to supplement the analysis of the predecessors, hoping to bring more inspiration to everyone.
First, we will give a summary and analysis of Alibaba's technical experts for your reference:
Http://tengine.taobao.org/book/chapter_05.html

Http://www.pagefault.info /? P = 251

See the configuration example provided by the official nginx Wiki (slightly modified ):

upstream backend {     server 211.20.10.11 weight=5;    server 198.172.10.9:8080 weight=10;}server {    location / {        proxy_pass http://backend;    }}

(It is stated in advance that the backend connection Selection Algorithm in upstream is not the focus of this discussion. We will only involve the round Robbin algorithm, such as ip_hash or the update mechanism, such as keepalive, which is not discussed here)
For the above upstream configuration, a structure will be created in the system initialization phase to save it. Note that this structure is the memory allocated during the initialization phase, and its lifecycle does not follow a specific request. It must be emphasized that, when analyzing nginx code, we need to remember what structure the lifetime follows the request and what structure is not. This is useful for you to grasp the system framework in general. Refer to the ngx_http_upstream_init_round_robin function.

In fact, the main driver of the upstream module is xxx_pass, for example, the above proxy_pass. In addition, there are also memcache_pass, fastcgi_pass, and so on. The establishment of Upstream-related configuration structures does not necessarily require the upstream command to be configured. Sometimes proxy_pass is directly followed by a DNS domain name, at this time, the initialization of upstream will also run normally. We will not discuss this situation here.

How can a specific request be associated with the upstream Configuration System? Generally, the Peer. init function pointer is used. Here, round Robbin uses ngx_http_upstream_init_round_assist_peer. Of course, ip_hash has its own processing function. From the configuration point of view, all requests to location
/Request, they are all associated with the same upstream configuration, this is no problem, we will do the same design, but since everyone shares a structure, do not need to mutex? If you want to modify a member... In fact, a request in nginx will not be processed in multiple processes at the same time, and a request is always in the same worker. Second, because nginx does not block requests in a single process, requests are processed asynchronously. Specifically, when a request is processed, eagain occurs (some system calls, or voluntarily give up) previously, it was not transferred to another request, so there was no mutex problem. It seems to be far away... We will continue our discussion.

Through the sample configuration, we can see that the upstream structure manages two backend servers. in use, you can select a backend with the highest priority (round Robbin) to initiate a connection. For how to select algorithms, you can directly read the source code. Here we are concerned about the subsequent handling of connection exceptions. After all, the handling of exceptions and their details directly determines whether a service is stable.

Case 1: the connection to the selected backend machine is abnormal (timeout or error)
Because the sockets here are non-blocking, Direct Connect is often not successful immediately. Generally, the classic einprogress error code is returned. In this case, you need to add a timer, if you haven't added a read event to epoll before, add a read
Event. Why is this necessary (in fact nginx does )? First, if the connection succeeds or an error occurs, the read event we register will be reported by epoll, And we will handle it in the event processing function; if the connection times out (for example, if the peer network is physically disconnected), our timeout timer will be triggered. In this way, no matter whether the connection fails or succeeds, we can properly handle the connection. You cannot ignore the exception.

What does upstream need to do after a timeout or error occurs? Nginx uses ngx_http_upstream_next for processing. For timeout and error, ngx_http_upstream_ft_timeout and ngx_http_upstream_ft_error are used respectively. In ngx_http_upstream_next, if a backend connection is faulty, nginx will call ngx_http_upstream_connect again to find other available backend connections.

It is worth mentioning that if a backend connection fails, nginx will make a small note. When there is a problem with the backend (obtain the back-end connection and return ngx_busy), nginx has to find a backup, if any. If there is no spare tire, or there is a problem with the spare tire, that's all right,
502 is for you.

Case 2: the backend connection is successful, but the response header received by nginx is abnormal.
The exception here is simply considered as a status code that is not what we expect. For example, if we want to get "200 OK", we get "404 Not Found" or something else. What Will nginx do in the future? I think there are two types of processing. The first is to send the response directly to the client. The problem is that nginx cannot perceive the content on the backend machine. That is to say, there is no file on the backend of the current connection, but other machines may have it. The backend is a cluster on the content, which is obviously not suitable for content processing. So it is more appropriate to let nginx try again at the backend... However, this is not a solution. If you have a lot of backend machines and only have the content that a client wants, you are lucky...

We will not discuss cluster optimization first. Let's look at how nginx works.

When upstream receives an abnormal response, it has to submit a command "proxy_next_upstream". See the official introduction.

The usage is clearly described. Let's look at the corresponding code:

static ngx_conf_bitmask_t ngx_http_proxy_next_upstream_masks[] = {    { ngx_string("error"), NGX_HTTP_UPSTREAM_FT_ERROR },    { ngx_string("timeout"), NGX_HTTP_UPSTREAM_FT_TIMEOUT },    { ngx_string("invalid_header"), NGX_HTTP_UPSTREAM_FT_INVALID_HEADER },    { ngx_string("http_500"), NGX_HTTP_UPSTREAM_FT_HTTP_500 },    { ngx_string("http_502"), NGX_HTTP_UPSTREAM_FT_HTTP_502 },    { ngx_string("http_503"), NGX_HTTP_UPSTREAM_FT_HTTP_503 },    { ngx_string("http_504"), NGX_HTTP_UPSTREAM_FT_HTTP_504 },    { ngx_string("http_404"), NGX_HTTP_UPSTREAM_FT_HTTP_404 },    { ngx_string("updating"), NGX_HTTP_UPSTREAM_FT_UPDATING },    { ngx_string("off"), NGX_HTTP_UPSTREAM_FT_OFF },    { ngx_null_string, 0 }};

The above information tells us what special response code processing is supported in the proxy module. Of course, FastCGI, memcache, and other modules that use upstream all have similar arrays. For the http_xxx in the proxy_next_upstream command configuration, it means that nginx will try other backend (function ngx_http_upstream_test_next) when the backend returns these responses ). Of course, nginx native supports this. If you have your own needs, you can try to modify this code. My personal suggestion is that you should be careful when changing the core code of nginx.