Implementation of nginx timer and usage of Scheduled Events

Nginx provides a set of efficient timer implementation. In addition to the timer used by the nginx core, we can also use the timer to complete some scheduled tasks during module development. The core of nginx timer implementation is to use a red/black tree to store various scheduled events. During each cycle, timeout events are found from the tree and triggered one by one to complete scheduled task operations. The following describes several key points of nginx timer implementation. This article describes the implementation of a timer based on epoll in Linux.

Timer Initialization

When nginx is blocked in epoll_wait, it may be awakened by three types of events: read/write events, wait time timeout, and signal interruption. Wait for timeout and signal interruption are related to the timer implementation. Their initialization takes place in the process initialization phase of the ngx_event_core_module module. The code segment is as follows:

    if (ngx_event_timer_init(cycle->log) == NGX_ERROR) {        return NGX_ERROR;    } 

Call the ngx_event_timer_init function to build the timer red/black tree. This red/black tree also provides a waiting time for epoll_wait while storing the timer.

    if (ngx_timer_resolution && !(ngx_event_flags & NGX_USE_TIMER_EVENT)) {        struct sigaction  sa;        struct itimerval  itv;        ngx_memzero(&sa, sizeof(struct sigaction));        sa.sa_handler = ngx_timer_signal_handler;        sigemptyset(&sa.sa_mask);        if (sigaction(SIGALRM, &sa, NULL) == -1) {            ngx_log_error(NGX_LOG_ALERT, cycle->log, ngx_errno,                          "sigaction(SIGALRM) failed");            return NGX_ERROR;        }        itv.it_interval.tv_sec = ngx_timer_resolution / 1000;        itv.it_interval.tv_usec = (ngx_timer_resolution % 1000) * 1000;        itv.it_value.tv_sec = ngx_timer_resolution / 1000;        itv.it_value.tv_usec = (ngx_timer_resolution % 1000 ) * 1000;        if (setitimer(ITIMER_REAL, &itv, NULL) == -1) {            ngx_log_error(NGX_LOG_ALERT, cycle->log, ngx_errno,                          "setitimer() failed");        }    }

The setitimer system call is used to set the system timer. A sigalrm signal will occur every time it reaches the time point. At the same time, the epoll_wait blocking will be interrupted by the signal, and the scheduled event will be awakened. In fact, this initialization is not always executed. Its condition ngx_timer_resolution is set through the configuration command timer_resolution. If this command is not configured, this initialization code will not be executed. That is to say, after the timer_resolution command is used in the configuration file, epoll_wait will use the signal interruption mechanism to drive the timer. Otherwise, the minimum time of the timer redtree will be used as the epoll_wait timeout time to drive the timer.

Timed wake-up of epoll_wait

The timer execution is to check the red and black trees of the timer every time the timer is executed in the event loop, locate all timed events that have timed out, and execute them one by one. An event loop cannot be an infinite loop, otherwise it is equivalent that an infinite loop will consume most of the CPU. Therefore, a blocking point in an event loop is epoll_wait. With wait, the problem of loop idling is solved. But how long does it take? 1 second, 2 seconds, 1 minute, 2 minutes... If the wait time is too long, the timer will be inaccurate. If the wait time is too short or short enough, the process will degrade to a non-Wait loop. Nginx introduces the above two mechanisms to set the wait time. The code snippet is as follows:

   if (ngx_timer_resolution) {            timer = NGX_TIMER_INFINITE;            flags = 0;        } else {        timer = ngx_event_find_timer();        flags = NGX_UPDATE_TIME;           }

This code (in the ngx_process_events_and_timers function of ngx_event.c) clearly shows two timer mechanisms. If the timer_resolution command is used, the timer is set to-1. Otherwise, the ngx_event_find_timer () function is called to find the minimum timer time in the red/black tree of the timer. This timer value is used as the epoll_wait timeout (timeout ). Note that the use of the timer_resolution command sets the epoll_wait timeout time to-1, which means epoll_wait will be blocked forever and will not be automatically woken up, therefore, the setitimer operation in Initialization will play its role-regularly generate a sigalrm signal to interrupt epoll_wait and wake up.

Scheduled event execution

At this time, epoll_wait is awakened, indicating that the event loop will start a new round, therefore, nginx checks whether there are timed-out or scheduled events in the red/black tree of the timer. If yes, it executes them one by one. The code snippet involved is as follows:

    if (delta) {        ngx_event_expire_timers();    }

Epoll_wait will run this code (in the ngx_process_events_and_timers function of ngx_event.c) after waking up the returned result. The ngx_event_expire_timers function traverses the red/black tree of the timer to find out the timed events and execute callback. You may say that this code has execution conditions. Yes, the delta here is actually used to reflect how long epoll_wait is blocked. So when Delta is equal to 0, it indicates that epoll_wait is almost not blocked, therefore, the last and current event cycles are completed within almost zero latency. The current time has not changed, so you do not need to check the scheduled event. Nginx is well positioned in this subtle optimization, and its performance is truly deducted from 1.1 drops.

Use of Scheduled Events

static ngx_connection_t dummy;static ngx_event_t ev; static voidngx_http_hello_print(ngx_event_t *ev) {    printf("hello world\n");    ngx_add_timer(ev, 1000);}static ngx_int_tngx_http_hello_process_init(ngx_cycle_t *cycle){    dummy.fd = (ngx_socket_t) -1;     ngx_memzero(&ev, sizeof(ngx_event_t));    ev.handler = ngx_http_hello_print;    ev.log = cycle->log;    ev.data = &dummy;    ngx_add_timer(&ev, 1000);    return NGX_OK;}

This code registers a scheduled event-prints Hello world every second. The ngx_add_timer function is used to add a new scheduled event to the red/black tree of the timer. After the scheduled event is executed, it will be removed from the tree. Therefore, to continuously print Hello world in a loop, after the event callback function is called, you need to add the event to the red/black tree of the timer. Ngx_http_hello_process_init is a callback function registered in the initialization phase of the module process. Because the ngx_even_core_module module is in front of the custom module, the timer has been initialized when we add a scheduled event in the process initialization phase.

This article only briefly introduces the timing of nginx, and also needs to read the code in details, such as the implementation of the nginx red/black tree.