Redis Implementation Events

Event

The Redis server is an event driver and the server needs to handle the following two categories of things:

• File event: The Redis server connects to the client (or other Redis server) through a socket, and the file event is the server's abstraction of the socket operation. The server communicates with the client (or other server) to generate a corresponding file event, and the server completes a series of network traffic operations by listening for and processing these events
• Time event: Some operations in the Redis server (such as the Servercron function) need to be performed at a given point in time, and the time event is the server's abstraction of such timed operations

File events

Redis has developed its own network event processor based on the reactor pattern, which is known as the file event handler:

• The file event handler uses the I/O multiplexing program to listen for multiple sockets at the same time and correlate different event handlers for sockets based on the tasks currently performed by the socket
• When a listening socket is ready to perform an operation such as a connection answer (accept), read (read), write (write), close (close), the file event corresponding to the operation is generated, and the file event handler invokes the associated event handler before the socket to handle these events

Although the file event handler runs single-threaded, the file event handler achieves a high-performance network communication model by using the I/O multiplexing program to listen for multiple sockets, and it is well connected to other modules in the Redis server that are also run as single-threaded. This keeps the simplicity of the Redis internal single-threaded design

The composition of the file event handler

Figure 1-1 shows the four components of the file event handler, which are sockets, I/O multiplexer, file event dispatcher, and event handler

Figure 1-1 Four components of a file event handler

A file event is an abstraction of a socket operation that produces a file event whenever a socket is ready to perform operations such as a connection answer (accept), write, read, close, and so on. Because a server typically connects multiple sockets, multiple file events can occur concurrently. The I/O multiplexer is responsible for listening to multiple sockets and transmitting the socket that generated the event to the file event dispatcher

Although multiple file events may occur concurrently, the I/O multiplexer always places all sockets that generate events into a queue, and through this queue, passes the socket to the file event dispatcher in an orderly, synchronized, one-socket manner. The I/O multiplexer will continue to transmit the next socket to the file event dispatcher, as shown in 1-2, after the practice of the previous socket has been processed (the socket is executed for the event handler associated with the event)

Figure 1-2 I/O multiplexing program transmits sockets through a queue to a file event dispatcher

The file event dispatcher receives the socket from the I/O multiplexing program and invokes the appropriate event handler based on the type of event that the socket generates. The server associates different event handlers for sockets that perform different tasks, which are functions that define the actions that the server should perform when an event occurs

Implementation of the I/O multiplexing program

All of the functions of the Redis I/O multiplexing program are implemented by wrapping the common Select, Epoll, Evport, and kqueue I/O multiplexing libraries, each of which corresponds to a separate file in the Redis source code, such as Ae_ Select.c, AE_EPOLL.C, ae_kqueue.c, etcetera. Because Redis implements the same API for each I/O multiplexing function library, the underlying implementation of the I/O multiplexing program is interchangeable, as shown in 1-3

Figure 1-3 Redis's I/O multiplexing program has multiple I/O multiplexing libraries to implement optional

The implementation of Redis in the I/O multiplexing program source code is defined with an # include macro, and the program will automatically select the highest system-neutral I/O multiplexing function library at compile time as the underlying implementation of the Redis I/O multiplexing program

Ae.c

/* Include the best multiplexing layer supported by this system. * The following should is ordered by performances, descending. */#ifdef Have_evport#include "ae_evport.c" #else    #ifdef have_epoll    #include "ae_epoll.c"    #else        # ifdef have_kqueue        #include "ae_kqueue.c"        #else        #include "ae_select.c"        #endif    #endif #endif

　　

Type of Event　　

The/O multiplexing program can listen for multiple sockets of ae.h/ae_readable events and ae.h/ae_writable events, and the correspondence between these two types of events and socket operations is as follows:

• Sockets generate Ae_readable events when sockets become readable (client performs write operations on sockets, or performs a close operation), or when a new acceptable socket appears (client performs a connect operation on the server's listening socket)
• Socket generates ae_writable event when socket becomes writable (client performs read operation on socket)

The/O multiplexing program allows the server to listen for both the Ae_readable event and the Ae_writable event of the socket, and if a socket produces both events, the file dispatcher takes precedence over the Ae_readable event before processing Ae_ Writable event. That is, if a socket is readable and writable, then the server will read the socket first, then write the socket

Api

• The Ae.c/aecreatefileevent function accepts a socket descriptor, an event type, and an event handler as a parameter, joins the event of a given socket into the listening range of the I/O multiplexer and associates the event and event handlers
• The Ae.c/aedeletefileevent function accepts a socket descriptor and a listener event type as a parameter, allowing the I/O multiplexer to suppress the listener of a given socket and to cancel the association between the event and the event handler
• The Ae.c/aegetfileevents function accepts a socket descriptor that returns the type of event that the socket is listening on:
  • If the socket does not have any events being monitored, then the function returns Ae_none
  • If the read event of the socket is being monitored, then the function returns ae_readable
  • If the write event of the socket is being monitored, then the function returns ae_writable
  • If the read and write events of the socket are being monitored, then the function returns ae_readable| Ae_writable
• The ae.c/aewait function accepts a socket descriptor, an event type, and a number of milliseconds for a parameter, blocks and waits for a given type of socket event to occur within a given time, when the event succeeds, or after a wait time-out, the function returns
• The Ae.c/aeapipoll function accepts a SYS/TIME.H/STRUCT timeval structure as a parameter and, within a specified time, blocks and waits for all sockets that are set to listen state by the aecreatefileevent function to produce a file event. When at least one event is generated, or the wait times out, the function returns
• The Ae.c/aeprocessevents function is a file event dispatcher that calls the Aeapipoll function to wait for the event to occur, then iterates through all the resulting events and invokes the appropriate event handlers to handle these events
• The Ae.c/aegetapiname function returns the name of the I/O multiplexing function library used by the I/O multiplexer bottom: Returning "select" means the base is the Select library, and so on

Processor for file events

Redis has written multiple processors for file events that are used to implement different network communication requirements, such as:

• In order to answer each client that connects to the server, the server will associate a connection answering processor for the listening socket
• In order to receive a command request from a client, the server requests the processor for the Client Socket Association command
• In order to return the execution result of the command to the client, the server responds to the handler for the Client Socket Association command
• When both the primary and the server replicate, the master and slave servers need to correlate with the replication processor specifically written for the replication function

In these event handlers, the server is most commonly used by the connection answering processor, the command request processor, and the command reply processor to communicate with the client

Connect the answering processor

The Networking.c/accepttcphandler function is a Redis connection answering processor that is used to respond to clients that connect to the server listening sockets, specifically as a wrapper for the sys/socket.h/accept function. When the Redis server is initialized, the program associates this connection answering processor with the Ae_readable event of the service listener socket, which generates AE_ when a client uses the Sys/socket.h/connect function to connect to the service-side listener socket. The readable event, which raises the connection answering processor, executes and performs the corresponding socket answer operation, as shown in 1-4

Figure 1-4 the server responds to a client's connection request

Command Request processor

The Networking.c/readqueryfromclient function is a redis command request processor, which is responsible for reading the command request content sent from the client from the socket, which is implemented as a wrapper for the Unistd.h/read function. When a client successfully connects to the server through a connection answering processor, the server associates the Ae_readable event of the client socket with the command request processor, and the socket generates a Ae_readable event when the client sends a command request to the server. Raises the command request processor to execute, and performs the corresponding socket read-in operation, as shown in 1-5. During the client connection to the server, the server will always have the Ae_readable event associated with the client socket command request processor

Figure 1-5 the server receives a command request sent by the client

Command Reply handler

The Networking.c/sendreplytoclient function is a redis command reply handler, which is responsible for returning the command reply from the server to the client via a socket, which is implemented as a wrapper for the Unistd.h/write function. When a server has a command reply that needs to be routed to the client, the server associates the Ae_writable event of the client socket with the command reply handler, and when the client is ready to receive the command reply returned by the server, it generates a ae_writable event that causes the command reply handler to execute. and perform the corresponding socket write operation, as shown in 1-6. When the command reply is sent, the server will dismiss the association between the command reply processor and the Ae_writable event of the client socket

Figure 1-6 Server sends command reply to client

Example of a complete client-server connection event

Assuming a Redis server is working, the Ae_readable event for the listener socket on this server should be in the listening state, and the corresponding processor for that event is the connection answering processor. If at this point a Redis client initiates a connection to the server, the listener socket generates a ae_readable event that triggers the connection answering processor to execute. The processor answers the client's connection request, then creates the client socket, and the client state, and associates the Ae_readable event of the client socket with the command request processor, allowing the client to send a command request to the primary server

Then, assuming that the client sends a command request to the primary server, the client socket generates the Ae_readable event, raises the command request processor execution, the processor reads the client command content, and then passes it to the relevant program to execute. The execute command will produce a corresponding command reply, in order to send these commands back to the client, the server associates the Ae_writable event of the client socket with the command reply processor. When a client attempts to read a command reply, the client socket generates a Ae_writable event that triggers the command reply handler to execute, and when the command reply processor writes the command reply to the socket, the server unlocks the client socket Ae_ The association between the writable event and the command reply processor

Figure 1-7 summarizes the entire communication process described above, and the event handlers used to communicate

Figure 1-7 Communication process between the client and the server

Time Events

The Redis time event is divided into the following two categories:

• Timed event: Lets a program execute once after a specified time. For example, let program x execute once in 30 milliseconds of the current time
• Recurring event: Allows a program to be executed once every specified time. For example, let program y do it every 30 milliseconds.

A time event consists mainly of the following three attributes:

• ID: The globally unique ID (identification number) that the server created for the time event. ID numbers are incremented in order from small to large, and the ID number of the new event is greater than the ID number of the old event
• When: millisecond-precision Unix timestamp, recording time event arrival (arrive) time
• Timeproc: Time event handler, a function. When an event arrives, the server calls the appropriate processor to handle the event

Whether a time event is a timed event or a recurring event depends on the return value of the time event handler:

• If the event handler returns Ae.h/ae_nomore, then this event is a timed event: The event is deleted after it has been reached once, and then no longer arrives
• If the event handler returns a non-ae_nomore integer value, then this event is a recurring time: When a time event arrives, the server updates the time event's when property based on the value returned by the event handler, allowing the event to arrive again after a period of time, and continues to be updated and run in this way. For example, if the processor of a time event returns an integer value of 30, then the server should update the time event so that the event arrives again after 30 milliseconds

Realize

The server places all time events in an unordered list, and whenever the time event executor runs, it traverses the entire list, finds all the time events that have elapsed, and invokes the appropriate event handler. Figure 1-8 shows an example of a linked list of time-saving events, with three different time events in the list: Because the new time event is always inserted into the table header of the list, three time events are sorted in reverse order by ID, the ID of the header event is 3, the ID of the intermediate event is 2, and the ID of the footer event is 1

Figure 1-8 three time events connected with a linked list

We say that the list of time-saving events is a list of unordered lists, not that the list is not sorted by ID, but that the list does not sort by the size of the When property. Because the list is not sorted by the When property, when the event executor runs, it must traverse all the time events in the list to ensure that all the time events in the server are processed

Api

• The Ae.c/aecreatetimeevent function takes a number of milliseconds milliseconds and a time event handler proc as a parameter to add a new time event to the server, This new time event will arrive after milliseconds milliseconds for the current time, and the event's processor is Proc.

For example, if the server is currently saving time event 1-9, then when the program takes 50 milliseconds and the Handler_3 processor as parameters, the Aecreatetimeevent function is called at time 1385877599980 (20 milliseconds before 0 o'clock December 1, 2013). The server creates a time event with an ID of 3, and the time event that the server saves is shown in 1-8

Figure 1-9 two time events connected with a linked list

• The Ae.c/aedeletefileevent function takes a time event ID as a parameter and then deletes the time event for that ID from the server
• The Ae.c/aesearchnearesttimer function returns the time event that is closest to the current time of arrival time
• The Ae.c/processtimeevents function is the executor of a time event that iterates through all the time events that have arrived and invokes the handlers of those events. Reached refers to the time event when property records a Unix timestamp that is equal to or less than the current time of the Unix time stamp

For a chestnut, if the server is saving the time event 1-8, and the current time is 1385877600010 (10 milliseconds after December 1, 2013 0 o'clock), then the Processtimeevents function will process the time events with IDs 2 and 1 in the graph. Because the arrival time of both time is greater than or equal to 1385877600010

The definition of the Processtimeevents function can be described by the following pseudo-code:

Def processtimeevents ():    #遍历服务器中的所有时间事件 for    time_event in All_time_event ():        #检查事件是否已经到达        if time_ Event.when <= Unix_ts_now ():            #事件已到达            #执行事件处理器 and get the return value            retval = Time_event.timeproc ()            #如果这是一个定时事件            if retval = = Ae_nomore:                #那么将该事件从服务器中删除                delete_time_event_from_server (time_event)        #如果这是一个周期性事件        else:            #那么按照事件处理器的返回值更新时间事件的when属性            #让这个事件在指定的时间之后再次到达            update_when (time_event, retval)

　　

Time Event Application Example: Servercron function

A continuously running Redis server needs to periodically check and adjust its own resources and state to ensure that the server can run for long periods of time, and that these periodic operations are performed by the Redis.c/servercron function, whose main work includes:

• Update server statistics such as time, memory usage, database occupancy, etc.
• To clean up expired key-value pairs in the database
• Shutting down and cleaning up clients with failed connections
• Try a aof or RDB persistence operation
• If the server is the primary server, synchronize periodically from the server
• If you are in cluster mode, periodically synchronize and connect your cluster to test

The Redis server runs the Servercron function as a recurring event, and every once in a while while the server is running, Servercron executes once until the server shuts down. In the Redis2.6 version, the server defaults to Servercron run 10 times per second, averaging 100 milliseconds per interval. Starting with Redis2.8, the user can modify the Hz option to adjust the number of executions per second for Servercron, refer to the sample configuration file redis.conf for a description of the HZ option

Scheduling and execution of events

Because the server has both a file event and a time event of two event types, the server must schedule both events, decide when to handle file events, when to handle time events, and how much time to process them, and so on. The dispatch and execution of events is the responsibility of the ae.c/aeprocessevents function. You can do this with the following source code:

Def aeprocessevents ():    #获取到达时间离当前时间最接近的时间事件    time_event = Aesearchnearesttimer ()    #计算最接近的时间事件距离到达还有多少毫秒    Remaind_ms = Time_event.when-unix_ts_now ()    #如果事件已到达, then the value of Remaind_ms may be negative, set it to 0    if Remaind_ms < 0:        Remaind_ms = 0    #根据remaind_ms的值, creating timeval structure    timeval = Create_timeval_with_ms (Remaind_ms)    # Blocking and waiting for file events to occur, maximum blocking time is determined by the incoming Timeval structure    #如果remaind_ms的值为0, then Aeapipoll call returns immediately, without blocking    aeapipoll (timeval)    #处理所有已产生的文件事件 (In fact, there is no such function)    processfileevents ()    #处理所有已到达的时间事件    processtimeevents ()

　　

Redis Implementation Events