Learning redis source code [event mechanism]

Introduction

Redis implements a set of event-driven mechanisms similar to libevent, mainly used to process time events and file events. The underlying layer of file events mainly refers to the processing of network I/O events. The underlying layer may be select, epoll, or kqueue. Redis uses its own AE instead of the libevent used by memcache to improve its performance, because libevent has added a lot of extended functions for its versatility, which obviously reduces its performance.

Source code

AE. h AE. c

Analyze the overall process

Simply put, the event processing mechanism used by redis is to run in a single thread through a main aemain loop. In each loop, first check whether other blocked clients or aof clients need to be executed. Then, the specific aeprocessevents determines how to process file events and time events based on the passed parameters. For more information, see the following description.

Description of main functions: aeprocessevents

/* Process every pending time event, then every pending file event * (that may be registered by time event callbacks just processed ). ** process all arrived time events and all ready file events. ** Without special flags the function sleeps until some file event * fires, or when the next time event occurrs (if any ). ** if no special flags is input, the function sleep until the file event is ready * or the event arrives at the next time (if any ). ** If flags is 0, the function does nothing and returns. * If flags is 0, the function returns directly without any action. ** If flags has AE _all_events set, all the kind of events are processed. * If flags contain AE _all_events, all types of events are processed. ** If flags has AE _file_events set, file events are processed. * If flags contains AE _file_events, file events are processed. ** If flags has AE _time_events set, time events are processed. * If flags contains AE _time_events, the time event is processed. ** If flags has AE _dont_wait set the function returns ASAP until all * the events that's possible to process without to wait are processed. * If flags contains AE _dont_wait, * the function returns immediately after processing all events that are not blocked. ** The function returns the number of events processed. * The Return Value of the function is the number of processed events */INT aeprocessevents (aeeventloop * eventloop, int flags) {int processed = 0, numevents;/* nothing to do? Return ASAP */If (! (Flags & AE _time_events )&&! (Flags & AE _file_events) return 0;/* Note that we want call select () even if there are no * file events to process as long as we want to process time * events, in order to sleep until the next time event is ready * to fire. */If (eventloop-> maxfd! =-1 | (flags & AE _time_events )&&! (Flags & AE _dont_wait) {Int J; aetimeevent * shortest = NULL; struct timeval TV, * TVP; // obtain the latest time event if (flags & AE _time_events &&! (Flags & AE _dont_wait) Shortest = aesearchnearesttimer (eventloop); If (shortest) {// if a time event exists, // The blocking time of the file event is long now_sec and now_ms based on the time difference between the last executable time event and the current time; /* calculate the time missing for the nearest * timer to fire. * /// calculate how long it will take for a recent time event to be reached // and save the time from aegettime (& now_sec, & now_ms) in the TV structure; TVP = & TV; TVP-> TV _sec = shortest-> when_sec-now_sec; If (Shortest-> when_ms <now_ms) {TVP-> TV _usec = (Shortest-> when_ms + 1000)-now_ms) * 1000; TVP-> TV _sec --;} else {TVP-> TV _usec = (Shortest-> when_ms-now_ms) * 1000;} // The time difference is less than 0, indicating that the event can be executed. Set the seconds and milliseconds to 0 (not blocked) if (TVP-> TV _sec <0) TVP-> TV _sec = 0; If (TVP-> TV _usec <0) TVP-> TV _usec = 0;} else {// execute this step, it indicates that there is no time event // determine whether to block based on whether AE _dont_wait is set, and the blocking duration/* if we have to check for events but need to return * ASAP because of AE _dont_wait we need To se the timeout * to zero */If (flags & AE _dont_wait) {// set file events not to block TV. TV _sec = TV. TV _usec = 0; TVP = & TV;} else {/* Otherwise we can block * // file events can be blocked until an event arrives TVP = NULL; /* wait forever */} // process file events. The blocking time is determined by TVP numevents = aeapipoll (eventloop, TVP); For (j = 0; j <numevents; j ++) {// obtain the event aefileevent * Fe = & eventloop-> events [eventloop-> fired [J] from the ready array. FD]; int mask = eventloop-> Fired [J]. mask; int FD = eventloop-> fired [J]. FD; int rfired = 0;/* Note the Fe-> mask & Mask &... code: maybe an already processed * event removed an element that fired and we still didn't * processed, so we check if the event is still valid. */If (Fe-> mask & Mask & AE _readable) {// read event rfired = 1; // ensure that only one of the Fe-> rfileproc (eventloop, FD, Fe-> clientdata, mask);} If (Fe-> mask & Mask & AE _wri Table) {// write event if (! Rfired | Fe-> wfileproc! = Fe-> rfileproc) Fe-> wfileproc (eventloop, FD, Fe-> clientdata, mask);} processed ++ ;}} /* check time events * // execution time event if (flags & AE _time_events) processed + = processtimeevents (eventloop); Return processed; /* return the number of processed file/time events */}

Note: Different flag labels are used to perform different operations. Note the following methods when handling file events:

If a file FD is under monitoring, the system determines whether to block the uploaded flag based on whether it is waiting. If you do not need to wait, aepoll is not blocked. If you need to wait, aepoll is blocked until an event occurs.

If the FD is not in monitoring, determine whether to handle the time event. If necessary, determine the blocking time of the file event based on the time difference between the last executable time event and the current time.

Prcesstimeevents

/* Process time events ** process all arrived time events */static int processtimeevents (aeeventloop * eventloop) {int processed = 0; aetimeevent * te; long maxid; time_t now = Time (null);/* If the system clock is moved to the future, and then set back to the * right value, time events may be delayed in a random way. often this * means that scheduled operations will not be saved med soon enough. ** here we try To detect system clock skews, and force all the time * events to be processed ASAP when this happens: the idea is that * processing events earlier is less dangerous than delaying them * indefinitely, and practice suggests it is. * /// reset the running time of the event, // prevent event processing confusion caused by time interspersed (Skew) if (now <eventloop-> lasttime) {Te = eventloop-> timeeventhead; while (TE) {Te-> when_sec = 0; Te = te-> next ;}// Last Update Processing Time event time eventloop-> lasttime = now; Te = eventloop-> timeeventhead; maxid = eventloop-> timeEventNextId-1; while (TE) {long now_sec, now_ms; long ID; // skip invalid event if (Te-> ID> maxid) {Te = te-> next; continue;} // obtain the current time aegettime (& now_sec, & now_ms ); // if the current time is equal to or equal to the event execution time, then execute this event if (now_sec> Te-> when_sec | (now_sec = te-> when_sec & now_ms> = te-> when_ms) {int retval; id = te-> ID; retval = Te-> timeproc (eventloop, ID, te-> clientdata); processed ++;/* after an event is processed our time event list may * no longer be the same, so we restart from head. * Still we make sure to don't process events registered * By Event Handlers itself in order to don't loop forever. * To do so we saved the max ID we want to handle. ** future optimizations: * Note that this is not great algorithmically. Redis uses * a single time event so it's not a problem but the right * way to do this is to add the new elements on head, and * to flag deleted elements in a special way for later * Deletion (putting references to the nodes to delete into * Another linked list ). * /// record whether this event needs to be executed cyclically if (retval! = AE _nomore) {// Yes, retval continues to execute this time event in milliseconds. aeaddmillisecondstonow (retval, & Te-> when_sec, & Te-> when_ms);} else {// No, delete this event aedeletetimeevent (eventloop, ID);} // The event list may have been changed after the event is executed. // Therefore, you need to put the Te back into the header, continue to run the event TE = eventloop-> timeeventhead;} else {Te = te-> next ;}} return processed ;}

Note:

After processing an event, our event list may change. Therefore, we need to traverse and process it from the beginning.

At the same time, to ensure that the next event we process is not registered by the currently processed event, we can save the handle currently processed as the largest handle to avoid this.

Summary

The understanding of AE event processing mainly depends on the understanding of the event-based asynchronous programming framework. The underlying Network poll will be further described in the following articles.