Nginx Learning Note Five (Nginx event module definition)

In the field of Linux background server development, Epoll's name is early to hear. "In-depth understanding of Nginx" in the 9th chapter-the event module details how Epoll related system calls are embedded in the Nginx framework.

Below is a description of some of the modules in the Nginx framework related to event handling.

I. Ngx_events_module

Ngx_events_module is one of the core modules. It's not always clear what the core module means, but for now, the core module of the event module should be the first event-related module to start. This module does not deal with the actual event business, but rather does some basic initialization operations. Ngx_events_module is defined as follows:

ngx_module_t Ngx_events_module = {NGX_MODULE_V1, &NGX_EVENTS_MODULE_CTX,/* Module context */ Ngx_events_commands,/* Module directives */Ngx_core_module,/* Module type     */NULL,/* INIT master */NULL,/* INIT module */    NULL,/* init process */NULL,/* init thread */    NULL,/* EXIT thread */NULL,/* Exit process */ NULL,/* Exit Master */ngx_module_v1_padding};static ngx_command_t Ngx_events_comm Ands[] = {{ngx_string ("events"), ngx_main_conf| ngx_conf_block| Ngx_conf_noargs, Ngx_events_block, 0, 0, NULL}, ngx_null_command};static ngx_core_module_t ngx_ Events_module_ctx = {ngx_string ("events"), NULL, NULL};

From the above program can be seen, this module do things very limited, in Ngx_events_module_ctx in the init_conf and create_conf methods are empty. In Ngx_events_block, the other event modules are called to parse the configuration in events, and other event modules are called to build and populate the configuration item structure. Some of the core code is as follows:

conf is conf in Ngx_conf_handler = confp[ngx_modules[i]->ctx_index]; is conf pointing to ngx_cycle_s->conf_ctx[],// So the Conf assignment is the CONF_CTX assignment to the ngx_cycle_s, and ultimately ngx_cycle_s Conf_ctx[ngx_events_module=>index] points to CTX * (void * *) conf =    CTx        for (i = 0; ngx_modules[i]; i++) {if (Ngx_modules[i]->type! = Ngx_event_module) {continue;        } m = ngx_modules[i]->ctx;            if (m->create_conf) {(*CTX) [Ngx_modules[i]->ctx_index] = m->create_conf (cf->cycle);            if ((*CTX) [ngx_modules[i]->ctx_index] = = NULL) {return ngx_conf_error;    }}}//zero before saving CF, after parsing event{} configuration, after restoring PCF = *CF;    Cf->ctx = CTX;    Cf->module_type = Ngx_event_module;    Cf->cmd_type = ngx_event_conf; RV = Ngx_conf_parse (cf, NULL);//This time the context ctx inside the CF is Ngx_event_module module create_conf for storing event{} space *cf = PCF;    After parsing event{} configuration, recover if (rv! = NGX_CONF_OK) {return RV; } for (i = 0; ngx_modules[i]; I+ +) {if (Ngx_modules[i]->type! = Ngx_event_module) {continue;        } m = ngx_modules[i]->ctx;            if (m->init_conf) {RV = m->init_conf (cf->cycle, (*CTX) [Ngx_modules[i]->ctx_index]);            if (rv! = NGX_CONF_OK) {return RV; }        }    }

Two. The Ngx_event_core_module event module Ngx_event_core_module is the first order in the event module (which is the core module, which in principle does not belong to the event module). This module is responsible for responding to many of the configuration items in events, completing the appropriate initialization, and preparing for the actual event handling module behind it. This module is defined as follows:

Related configuration See ngx_event_core_commands ngx_http_core_commands ngx_stream_commands ngx_http_core_commands ngx_core_commands Ngx_mail_commandsstatic ngx_command_t ngx_event_core_commands[] = {//per worker process can concurrently process the maximum number of connections//connection pool size, that is, each worker process The number of TCP maximum connections supported in, which is duplicated with the meaning of the following connections configuration items, can be referenced in the 9.3.3 section to understand the concept of connection pooling {ngx_string ("worker_connections"), ngx_event_conf| Ngx_conf_take1, ngx_event_connections, 0, 0, NULL},//Set the event model. Use [kqueue | rtsig | epoll | dev/poll | select | poll | eventport] Linux system only supports select poll Epoll three//freebsd kqueue,l Inux no//determines which event module is selected as the event-driven mechanism {ngx_string ("use"), ngx_event_conf| Ngx_conf_take1, Ngx_event_use, 0, 0, NULL},//When the event module notifies a TCP connection, try to establish a connection to all client TCP connection requests in this schedule whenever possible//corresponds to The available field for the event definition. For the Epoll event-driven mode, it means that when a new connection event is received, the accept is called to receive as many connections as possible {ngx_string ("multi_accept"), ngx_event_conf| Ngx_conf_flag, Ngx_conf_set_flag_slot, 0, Offsetof (ngx_event_conf_t, multi_accept), NULL},//Accept_mutex On|off whether to open the accept process lock, is to implement the worker process receive connection load balancing, open after the number of worker process rotation of serial numbers to receive TCP connection//default is open, if it is closed TCP connection is faster,    But the connection between workers is not so uniform. {ngx_string ("Accept_mutex"), ngx_event_conf| Ngx_conf_flag, Ngx_conf_set_flag_slot, 0, Offsetof (ngx_event_conf_t, Accept_mutex), NULL},//accep    T_mutex_delay time, if set to Accpt_mutex on, then only one process in the worker at the same time can acquire the accept lock, the accept lock is not blocked, if you do not marry to//immediately return, and then wait to get back. After you enable the Accept_mutex load Balancer Lock, delay accept_mutex_delay milliseconds before attempting to process the new connection event {ngx_string ("Accept_mutex_delay"), ngx_event_conf|    Ngx_conf_take1, Ngx_conf_set_msec_slot, 0, Offsetof (ngx_event_conf_t, Accept_mutex_delay), NULL}, Debug_connection 1.2.2.2 uses the debug level to print when the IP address request is received. The other is to follow the settings in the Error_log//need to print the debug-level bin log {ngx_string ("debug_connection") to the TCP connection from the specified IP, ngx_event_conf| Ngx_conf_take1, ngx_event_debug_connection, 0, 0, NULL}, Ngx_null_command};//ngx_event_core_modu The Le module only implements the Create_conf method and the Init_conf method, because it is not really responsible for the TCPNetwork event driver,//So does not implement the method in ngx_event_actions_t ngx_event_module_t ngx_event_core_module_ctx = {&event_core_name, ngx_e vent_core_create_conf,/* Create configuration */ngx_event_core_init_conf,/* init Configurati On/{NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL}};/*nginx defines a series of event-driven modules (currently 9) running on different operating systems and different kernel versions, including: N Gx_epoll_module, Ngx_kqueue_module, Ngx_poll_module, Ngx_select_module, Ngx_devpoll_module, Ngx_eventport_module, Ngx_aio_module, Ngx_rtsig_module, and Windows-based ngx_select_module modules. During the initialization of the Ngx_event_core_module module, 1 event-driven modules are selected from the above 9 modules as nginx processes. */ngx_module_t Ngx_event_core_module = {NGX_MODULE_V1, &NGX_EVENT_CORE_MODULE_CTX,/* Module Contex  T */ngx_event_core_commands,/* Module directives */Ngx_event_module,/* module Type */NULL,/* init master */ngx_event_module_init,/* init mod Ule *//After parsing the configuration file, execute NGX_EVENT_PROcess_init,/* init process *//execute ngx_worker_process_init NULL inside the creation subprocess,                                  /* INIT thread */NULL,/* EXIT thread */NULL, /* Exit Process */NULL,/* Exit Master */ngx_module_v1_padding};

Note that the Ngx_event_process_init function here actually does a lot of things, and this function is called when the child process is created. See the book description for details.
Three. Ngx_epoll_module module when the use configuration item in events is selected for Epoll, it is later corresponding to the execution of the Ngx_epoll_module module. This module is defined as follows:

Static ngx_command_t ngx_epoll_commands[] = {/* when calling Epoll_wait, the 2nd and 3rd parameters tell the Linux kernel how many events can be returned at a time. This configuration item represents the maximum number of events that can be returned when epoll_wait is called, and, of course, it will also pre-allocate so many epoll_event structures to store events */{ngx_string ("epoll_events"), Ngx_even T_conf| Ngx_conf_take1, Ngx_conf_set_num_slot, 0, Offsetof (ngx_epoll_conf_t, events), NULL},/* in open async i/ O and when initializing the asynchronous I/O context environment with the Io_setup system call, the number of asynchronous I/O events initially allocated */{ngx_string ("worker_aio_requests"), ngx_event_conf| Ngx_conf_take1, Ngx_conf_set_num_slot, 0, Offsetof (ngx_epoll_conf_t, aio_requests), NULL}, Ngx_n ull_command};ngx_event_module_t Ngx_epoll_module_ctx = {&epoll_name, ngx_epoll_create_conf,/* C             reate configuration */ngx_epoll_init_conf,/* INIT configuration */{ngx_epoll_add_event, /* Add an event *///ngx_add_event ngx_epoll_del_event,/* Delete an event */Ngx_ep Oll_add_event,/* Enable an event *///ngx_add_conn ngx_epoll_del_event,/* Disable an event */ngx_epoll_add_connection, /* Add an connection */ngx_epoll_del_connection,/* Delete an connection */#if (NGX_HAVE_EVENTFD) ng X_epoll_notify,/* trigger a notify */#else NULL,/* trigger a notify */#  endif ngx_epoll_process_events,/* Process the events */ngx_epoll_init,/* init the Events */Ngx_epoll_done,/* Done the events */}};ngx_module_t Ngx_epoll_module = {Ngx_m ODULE_V1, &ngx_epoll_module_ctx,/* Module context */Ngx_epoll_commands,/* Modu  Le directives */ngx_event_module,/* MODULE type */NULL,/* init Master */NULL,/* INIT module */NULL,/* init proce          SS */NULL,                      /* INIT thread */NULL,/* EXIT thread */NULL, /* Exit Process */NULL,/* Exit Master */ngx_module_v1_padding};

Ngx_epoll_init, Ngx_epoll_addevent, Ngx_epoll-process_events is the corresponding epoll_create,epoll_ctl, epoll_wait these epoll system call function.
Let's take a look at the implementation of the Ngx_epoll_process_events program:

Static ngx_int_tngx_epoll_process_events (ngx_cycle_t *cycle, ngx_msec_t timer, ngx_uint_t flags)//flags parameter contains Ngx_post    _events said this batch of events will be deferred processing {int events;    uint32_t revents;    ngx_int_t instance, I;    ngx_uint_t level;    ngx_err_t err;    ngx_event_t *rev, *wev;    ngx_queue_t *queue;    Ngx_connection_t *c;        Char epollbuf[256]; /* Ngx_timer_infinite = = Inftim *///ngx_log_debug1 (ngx_log_debug_event, Cycle->log, 0, "Begin to Epoll_wait, Epoll    Timer:%M ", timer); /* Call epoll_wait to get the event. Note that the timer parameter is passed in when Process_events is called, and this parameter is mentioned in sections 9.7 and 9.8 *///the call is interrupted by a signal handler before any of    The requested events occurred or the timeout expired; If a signal occurs (see function Ngx_timer_signal_handler), such as a timer, it returns-1//required and ngx_add_event with Ngx_add_conn used//event_list stored is ready to do , if the select is an incoming user-registered event, a traverse check is required, and the event set needs to be reset every time the select returns, Epoll do not use/* This waits for events including client connection events (this is an EP inherited from the parent process. Then the Ngx_even before the sub-process whileT_process_init->ngx_add_event added), add the FD read-write event that has established the connection to the Ngx_event_accept->ngx_http_init_connection->ngx_ Handle_read_event *//*ngx_notify->ngx_epoll_notify only triggers epoll_in, does not raise epoll_out at the same time, and if it is a network read event epoll_in, it will also cause Epoll_  out*/events = epoll_wait (EP, Event_list, (int) nevents, timer);  A timer of 1 means that infinite wait nevents indicates the maximum number of events to listen to, must be greater than 0//epoll_wait if there is no read-write event or timer Timeout event occurs, will go to sleep, this process will give up CPU err = (events = 1)?    ngx_errno:0;  When the flags flag indicates the time to update, it is updated here//to touch ngx_timer_resolution milliseconds timeout followed by new time, to touch Epoll read-write event timeout followed by new time if (Flags & Ngx_update_time | |    Ngx_event_timer_alarm) {ngx_time_update ();                } if (err) {if (err = = Ngx_eintr) {if (ngx_event_timer_alarm) {////Timer timeout caused by epoll_wait return                Ngx_event_timer_alarm = 0;            return NGX_OK;        } level = Ngx_log_info;        } else {level = Ngx_log_alert;        } ngx_log_error (level, Cycle->log, err, "epoll_wait () failed");    return ngx_error;} if (events = = 0) {if (Timer! = ngx_timer_infinite) {return NGX_OK; } ngx_log_error (Ngx_log_alert, Cycle->log, 0, "epoll_wait () returned no events without time        Out ");    return ngx_error;  }//Traverse all events returned by this epoll_wait for (i = 0; i < events; i++) {//And ngx_epoll_add_event with/* c = Event_list[i].data.ptr; Through this OK is that connection instance = (uintptr_t) C & 1; Take out the last one of the address, using the instance variable identifier, see Ngx_epoll_add_event/* Whether it is a 32-bit or 64-bit machine, the last 1 bits of the address is definitely 0, you can use the following line of statements to ngx_connection_t  Address to the true address value *//Note here C is likely to be the C before the accept, to detect whether the client initiates a TCP connection event, the accept return succeeds will recreate a ngx_connection_t to read and write to the client's data C = (ngx_connection_t *)        ((uintptr_t) C & (uintptr_t) to); Rev = c->read; Take out Read Event//Note here C is likely to be the C before the accept, to detect whether the client initiates a TCP connection event, and after the accept returns successfully, a ngx_connection_t is recreated to read and write to the client's data if (C-&GT;FD = =-1 | | Rev->instance! = instance) {//To determine if this read event is an out-of-date event//When the FD socket descriptor is-l or the instance flag bit is not equal, which indicates that the event has expired and does not have to process/* * The stale event from a file descriptor * is just closed in                           This iteration */NGX_LOG_DEBUG1 (ngx_log_debug_event, Cycle->log, 0,            "Epoll:stale event%p", c);        Continue } revents = event_list[i].events;        Remove Event Type Ngx_epoll_event_2str (revents, epollbuf);        memset (epollbuf, 0, sizeof (EPOLLBUF));        Ngx_epoll_event_2str (revents, epollbuf);                       Ngx_log_debug4 (ngx_log_debug_event, Cycle->log, 0, "epoll:fd:%d%s (ev:%04xd) d:%p",        C-&GT;FD, Epollbuf, revents, event_list[i].data.ptr); if (Revents & (epollerr|                           Epollhup) {///For example, the other side close-off socket, here will sense Ngx_log_debug2 (ngx_log_debug_event, Cycle->log, 0,        "Epoll_wait () error on fd:%d ev:%04xd", C-&GT;FD, revents); } #if 0 if (revents & ~ (epollin| Epollout| Epollerr| Epollhup) {ngx_log_error (Ngx_log_alert, Cycle->log, 0, "strange epoll_wait () even        TS fd:%d ev:%04xd ", C-&GT;FD, revents); } #endif if (Revents & (epollerr| Epollhup) && (Revents & (epollin|             epollout) = = 0) {/* * If the error events were returned without Epollin or epollout,            * Then add these flags to handle the events at least in one * active handler */ Revents |= epollin| Epollout; Epoll epollerr| Epollhup is actually a read-write operation that triggers a read-write event recv write to detect a connection exception} if ((Revents & Epollin) && rev->active) {//If yes            Read event and the event is active if # if (ngx_have_epollrdhup) if (revents & epollrdhup) {rev->pending_eof = 1;            } #endif//Note that C here is likely to be the C before the accept, to detect whether the client initiates a TCP connection event, and when the accept returns successfully, it re-creates a ngx_connection_t to read and write the client's data Rev->ready = 1; Which means there's already data here.Pre-Ngx_connection_t->read->ready 1,accept returns a ngx_connection_t from the connection pool when the//flags parameter contains ngx_post_events indicating that the batch Event deferred processing if (Flags & ngx_post_events) {/* If you want to defer processing of the event in the POST queue, first determine that it is a new connection event or an ordinary event to decide to add it to the ngx_posted_accept_events queue or the Ngx_postedl events queue. About when events in the Post queue Execute */queue = rev->accept?                &ngx_posted_accept_events: &ngx_posted_events;             Ngx_post_event (rev, queue); } else {//If the client data is received, this is Ngx_http_wait_request_handler Rev->handler (rev); , it is set to Ngx_event_accept in Ngx_event_process_init.        If the connection is already established, the read data is Ngx_http_process_request_line}} Wev = c->write;  if ((Revents & epollout) && wev->active) {if (C-&GT;FD = =-1 | | Wev->instance! = instance) { Determines whether this read event is an out-of-date event//When the FD socket descriptor is-1 or instanceWhen the flag bit is not equal, it means that the event has expired without processing/* * The stale event from a file descriptor * that                               Was just closed in this iteration */NGX_LOG_DEBUG1 (ngx_log_debug_event, Cycle->log, 0,                "Epoll:stale event%p", c);            Continue            } Wev->ready = 1;            if (Flags & ngx_post_events) {ngx_post_event (Wev, &ngx_posted_events);//Add this event to the POST queue for deferred processing            } else {//Call this write event callback method immediately to handle this event Wev->handler (WEV); }}} return NGX_OK;}

This module is only responsible for the distribution of events, the specific business logic processing is in each read and write event corresponding to the handler method.

Nginx Learning Note Five (Nginx event module definition)