Redis source code analysis (20) --- AE event-driven

The term "event-driven" appears more and more frequently. It sounds very tall. Today I have studied the internal drive model of redis, And I feel quite fruitful. An AE. C main program, coupled with four event files, allows you to thoroughly figure out how redis handles these events. Epoll, select, kqueue, evport, and evport may be used in redis event processing. The first three events are very common and also appear in the libevent event network library. When writing this event-driven model, the author also said that this was just for simple reuse. A small processing model was designed:

/* A simple event-driven programming library. originally I wrote this Code * For the Jim's event-loop (Jim is a Tcl interpreter) but later translated * it in form of a library for easy reuse. ** AE is a simple event-driven library written by the author. It is converted later to be reused more easily.

So it is not very complicated. Before learning about the entire event-driven model, you should first understand some defined event structures. The event type has a total of two fileevents, timeevent:

/* File event structure * // * file event structure */typedef struct aefileevent {// only one int mask of the read or write events; /* One of AE _ (readable | writable) * // read aefileproc * rfileproc; // write method aefileproc * wfileproc; // client data void * clientdata;} aefileevent; /* time event structure * // * time event structure */typedef struct aetimeevent {// time event Id long ID;/* time event identifier. * // time in seconds long when_sec;/* seconds * // time in milliseconds long when_ms;/* milliseconds * // The processing function aetimeproc * timeproc in the time event; // The method aeeventfinalizerproc * finalizerproc will be called when the object is deleted; // The client data void * clientdata; // The next struct in the Time Structure Body struct aetimeevent * Next;} aetimeevent; /* A fired event * // * fired struct, used to indicate the file event to be processed */typedef struct aefiredevent {// file descriptor ID int FD; int mask;} aefiredevent;

Fireevent is only used to mark the event of the file to be processed.

These events exist in the structure of an aeeventloop:

/* State of an event based program */typedef struct aeeventloop {// currently created highest file descriptor int maxfd;/* highest file descriptor currently registered */INT setsize; /* max number of file descriptors tracked * // the ID of the next time event long timeeventnextid; time_t lasttime; /* used to detect system clock skew * // three event types: aefileevent * events;/* registered events */aefiredevent * fired;/* fired events */aetimeevent * timeeventhead; // The Event stop identifier int stop; // The Event API data is stored here, including epoll, select, and other event void * apidata; /* This is used for polling API specific data */aebeforesleepproc * beforesleep;} aeeventloop;

In each event, corresponding processing functions are defined, and functions are stored in struct like variables. The following describes the composition of Some APIs in AE. C:

/* Prototypes */aeeventloop * aecreateeventloop (INT setsize);/* Create aeeventloop. The number of internal fileevents and fired events is setsize */void aedeleteeventloop (aeeventloop * eventloop ); /* Delete eventloop and release the space occupied by the corresponding event */void aestop (aeeventloop * eventloop);/* set the stop attribute in eventloop to 1 */INT aecreatefileevent (aeeventloop * eventloop, int FD, int mask, aefileproc * proc, void * clientdata);/* create a file event in eventloop */void aedeletefileevent (aeeventloop * eventloop, int FD, int mask ); /* delete file events */INT aegetfileevents (aeeventloop * eventloop, int FD); // locate the file attributes based on the file descriptor ID, whether it is a read event or write event long aecreatetimeevent (aeeventloop * eventloop, long milliseconds, aetimeproc * proc, void * clientdata, aeeventfinalizerproc * finalizerproc);/* add time events in eventloop, the creation time is the current time plus the input time */INT aedeletetimeevent (aeeventloop * eventloop, long ID); // Delete the time event based on the time Id, int aeprocessevents (aeeventloop * eventloop, int flags) involving linked lists;/* process all types of events in eventloop */INT aewait (int fd, int mask, long milliseconds ); /* wait for an event */void aemain (aeeventloop * eventloop);/* execute the main program for the AE event */char * aegetapiname (void); void aesetbeforesleepproc (aeeventloop * eventloop, aebeforesleepproc * beforesleep);/* call */INT aegetsetsize (aeeventloop * eventloop) when the eventloop event is re-executed after it is executed ); /* Get the eventloop size */INT aeresizesetsize (aeeventloop * eventloop, int setsize);/* eventloop re-resize */

It involves only some files, time event addition, modification, and so on, all of which are internal modifications in eventloop. Let's look at the most important and core methods:

/* AE event execution main program */void aemain (aeeventloop * eventloop) {eventloop-> stop = 0; // If the stop flag in eventloop is not 1, process the while (! Eventloop-> stop) {// If (eventloop-> beforesleep! = NULL) eventloop-> beforesleep (eventloop); // while loop processes all evetloop events aeprocessevents (eventloop, AE _all_events );}}

The principle is simple: Through a while loop, process all types of events in eventloop, and intercept some processevents () code:

Numevents = aeapipoll (eventloop, TVP); For (j = 0; j <numevents; j ++) {aefileevent * Fe = & eventloop-> events [eventloop-> fired [J]. FD]; int mask = eventloop-> fired [J]. mask; int FD = eventloop-> fired [J]. FD; int rfired = 0;/* Note the Fe-> mask & Mask &... code: maybe an already processed * event removed an element that fired and we still didn't * processed, so we check if the event is still valid. * /If (Fe-> mask & Mask & AE _readable) {rfired = 1; // determines whether it is an AE read event based on mask calculation, processing Method of Reading in call time Fe-> rfileproc (eventloop, FD, Fe-> clientdata, mask);} If (Fe-> mask & Mask & AE _writable) {If (! Rfired | Fe-> wfileproc! = Fe-> rfileproc) Fe-> wfileproc (eventloop, FD, Fe-> clientdata, mask);} processed ++ ;}}

The time events created in AE are created based on the current time;

/* Add a time event to eventloop. The creation time is the current time plus the input time */long aecreatetimeevent (aeeventloop * eventloop, long milliseconds, aetimeproc * proc, void * clientdata, aeeventfinalizerproc * finalizerproc) {long id = eventloop-> timeeventnextid ++; aetimeevent * te; Te = zmalloc (sizeof (* te )); if (TE = NULL) return AE _err; Te-> id = ID; aeaddmillisecondstonow (milliseconds, & Te-> when_sec, & Te-> when_ms ); te-> timeproc = proc; Te-> finalizerproc = finalizerproc; Te-> clientdata = clientdata; // the newly added header te to timeevent-> next = eventloop-> timeeventhead; eventloop-> timeeventhead = tE; // return the ID return ID of the newly created time event ;}

The following describes how to call the method in the event api library. First, we will introduce epoll, poll, select, kqueu, and evport. These are all event models.

Select event model

(1) create a collection of descriptors (fd_set) for the event of interest. For a descriptor, you can follow the read, write, and exception events on it, therefore, three fd_sets are usually created. One is used to collect the descriptor that follows the read event, the other is used to collect the descriptor that follows the write event, and the other is used to collect the descriptor set that follows the exception event.
(2) round-robin of each FD in all fd_set and check whether a corresponding event occurs. If yes, process it.

The difference between poll and above is that file descriptors can be reused. In the previous scenario, a file needs to poll three file descriptor sets, while poll only needs one, which is more efficient.
Epoll is an upgraded version of poll. It submits the descriptor list to the kernel. Once an event occurs, the kernel notifies the process of the List of event descriptors, which avoids polling the entire descriptor list. High Efficiency

The evport appears rarely, which means that evport associates a specific event of an object with the event Port:

After learning about the principles of the three event models, let's take a look at how AE. C is called in redis,

// The Event API data is stored here, including epoll, select, and other event void * apidata;/* This is used for polling API specific data */

In the preceding four events, three files are corresponding to AE _poll.c and AE _select.c, but their API structures are similar. I will give an example, in the epoll example, the event-specific struct is first available:

typedef struct aeApiState {    int epfd;    struct epoll_event *events;} aeApiState;

There are also common template methods:

static int aeApiCreate(aeEventLoop *eventLoop)static int aeApiResize(aeEventLoop *eventLoop, int setsize)static void aeApiFree(aeEventLoop *eventLoop)static int aeApiAddEvent(aeEventLoop *eventLoop, int fd, int mask)static void aeApiDelEvent(aeEventLoop *eventLoop, int fd, int delmask)static int aeApiPoll(aeEventLoop *eventLoop, struct timeval *tvp)static char *aeApiName(void)

Assign values to the API data of eventloop during creation:

State-> epfd = epoll_create (1024);/* 1024 is just a hint for the kernel */If (State-> epfd =-1) {zfree (State-> events); zfree (State); Return-1 ;}// Finally, assign the State data to eventloop API data> apidata = State; return 0;

The poll methods of events are distinguished by the following methods:

static int aeApiPoll(aeEventLoop *eventLoop, struct timeval *tvp) {    aeApiState *state = eventLoop->apidata;    int retval, numevents = 0;    retval = epoll_wait(state->epfd,state->events,eventLoop->setsize,            tvp ? (tvp->tv_sec*1000 + tvp->tv_usec/1000) : -1);    if (retval > 0) {.....

The poll method in the SELECT statement is as follows:

static int aeApiPoll(aeEventLoop *eventLoop, struct timeval *tvp) {    aeApiState *state = eventLoop->apidata;    int retval, j, numevents = 0;    memcpy(&state->_rfds,&state->rfds,sizeof(fd_set));    memcpy(&state->_wfds,&state->wfds,sizeof(fd_set));    retval = select(eventLoop->maxfd+1,                &state->_rfds,&state->_wfds,NULL,tvp);......

Finally, it is implemented based on the conversion between events in the state and eventloop. Input the information in eventloop, input the state information, and obtain the final event result after internal processing. The call is so simple.

Redis source code analysis (20) --- AE event-driven