C + + server Design (i): Reactor mode based on I/O multiplexing

I/O model selection

In the network server programming, a common scenario is that servers need to determine whether multiple connected sockets are readable, and if a socket is readable, the socket data is read and processed further.

In the most commonly used blocking I/O model, we get readable data for each connection socket by rotating the read system call. As shown in 3-1, the read system call will block the thread until the datagram arrives and is copied into the buffer of the application process.

Figure 3-1 Blocking I/O model

In the blocking I/O model, data-readable and read-data are combined in a system call, and for a single socket to be readable, it must wait until the actual data is received and the blocking time is uncertain. Given this scenario, if there are 10 connected sockets in the server, the server calls read for one of the sockets, and the next time the connection client sends the data to the server is 2 hours later. The server will also block the read of this system call for 2 hours, during which time the server cannot process even if it receives nine other connected socket data. This kind of server I/O model that can only be connected to one customer at a time is obviously not what we should consider.

We can modify it on the basis of a blocking I/O model, using a non-blocking I/O model. As shown in 3-2, when we execute the Read system call, if we do not receive the datagram immediately, we will not put the thread to sleep, but return an error.

Figure 3-2 Non-blocking I/O model

In a non-blocking I/O model, although both data-readable and read data are still in a system call, the system call returns immediately if there is no data to read. At this point we can call read for multiple connection sockets in turn, until the actual data is received by a call, and we process the data we receive. In this way, we can initially solve the problem that the server reads multiple customer data simultaneously. But this is the way in which a line range loops through multiple non-blocking descriptors, which we call polling. The app continually polls the kernel to see if an operation is ready, which tends to consume a lot of CPU time, while also giving the entire server a great deal of extra overhead. Therefore, this server I/O model does not satisfy our pursuit of high performance very well.

One of the improved schemes is to add multithreading support, namely Thread-per-connection scheme, on the basis of the above two I/O models. In this scenario, the server assigns a thread to each connected client, and the read and write of each connection is performed in a separate thread. Therefore, a socket within a client thread is read, at most, only blocking the client thread, without affecting other connections to other threads. This is a common scenario for Java network programming. The overhead of thread creation and destruction in this way is not suitable for short connection services that may be frequently connected and disconnected, and of course frequent threading creation and destruction can be improved through the thread pool. At the same time, the scalability of this scheme is also limited by the number of threads, the number of one hundred or two hundred threads created for the existence of one hundred or two hundred connections, the system can barely support, but if there are thousands of threads at the same time, this will have a great burden on the operating system scheduler. At the same time, more threads also make high demands on memory size.

As we can see from the above examples, the key to solving the server's reading of multiple connection sockets is to separate the readable judgment from the actual read data, and the other is to support multiple sockets readable judgments at the same time. So we need a capability to inform the kernel beforehand so that once the kernel discovers that one or more of the I/O conditions specified by the process is in place, the input is ready to be read, it notifies the process. This behavior is called I/O multiplexing. On the Linux platform, several system calls such as SELECT, poll, and Epoll are provided as I/O multiplexing.

The Epoll is an improved poll of the Linux kernel for processing large-volume file descriptors and is an enhanced version of the multiplexed I/O interface under Linux that supports both horizontal and edge triggering. Compared to I/O multiplexing, such as SELECT, which has a large number of descriptors, I/O efficiency does not decrease linearly with the number of registered descriptors (the efficiency of traditional Select and poll results in a two or even three descent due to the linear increment of the number of registered descriptors). And the use of mmap to accelerate the core and user space of the message delivery and other advantages. The system will use the level-triggered epoll as a system call for specific I/O multiplexing.

Figure 3-3 I/O multiplexing model

As shown in 3-3, we register multiple connection sockets in an I/O multiplexed epoll system call and register read events, at which time the system blocks the Epoll call and waits for a datagram socket to become readable. When Epoll returns, a readable set of sockets is returned, and we simply iterate through the readable sockets, then invoke the read system call on each of the readable sockets, read out the specific data, and proceed with the corresponding processing.

By using this I/O multiplexing model of epoll, we can implement the server's read support for multiple connection sockets without introducing the cost of creating new threads. The number of connections here is only related to the size of the system memory. And because the Linux kernel is optimized for epoll, the increase in the number of connection descriptors does not result in a linear decrease in performance. Therefore, the Epoll-based I/O multiplexing model can meet our high concurrency and high performance requirements for server systems.

Reactor Mode Introduction

In previous studies, we chose Epoll as the I/O multiplexing model for server systems. But Epoll is too low, it's just a Linux system call that needs to be further encapsulated by some sort of event-handling mechanism.

In the ordinary event processing mechanism, the program first calls a function, then the function executes, the program waits, when the function completes the function to return the result and the control to the program, finally the program continues to process. In the study of the previous I/O multiplexing model, we also used this sequence of event processing.

We can abstract the whole problem. Each connected socket descriptor is an event source, and each socket receives data after the further processing operation as an event handler. We will need to register the event processing source and its event handler for processing to a epoll-like event splitter. The event separator is responsible for waiting for events to occur. Once an event is sent, the event splitter passes the event to the corresponding processor registered by the event, and finally the processor is responsible for completing the actual read and write work. This is how the event is handled in reactor mode.

The reactor pattern is an event-driven mechanism, relative to the event-handling approach of the previous normal function call. In reactor mode, the application is not actively calling an API to complete processing, but instead reverse the event processing flow, the application needs to provide the corresponding event interface and register to reactor, if the corresponding event occurs, reactor will actively invoke the application registration interface, Specific event handling is done through the registered interface.

Reactor Mode Advantages

Reactor mode is one of the necessary techniques to write high-performance network server, and some common network libraries such as Libevent, Muduo and so on have realized the core of network library by using reactor mode. It has the following advantages:

• The response is fast and does not have to be blocked for a single synchronization time, although the reactor itself still needs to be synchronized;
• Simple programming, can avoid complex multithreading and synchronization problems, and avoid multi-threading/process switching overhead;
• Scalability is strong, can be easily by increasing the number of reactor instances to make full use of CPU resources;
• Reusability is strong, the reactor mode itself is independent of the specific event processing logic, and has high reusability.

Reactor pattern composition

Figure 3-4 is a reactor class diagram, which shows that the reactor pattern is composed of event sources, event reactors, event separators, event handlers, and the like, specifically:

Figure 3-4 Reactor class diagram

• L Event Source (handle): provided by the operating system to identify each event, such as the socket descriptor, file descriptor, and so on. In a server-side system with an integer representation. The event may come from outside, such as connection requests, data, and so on from the client. It may also come from inside, such as timer events.
• L Event Reactors (reactor): Definitions and application control event scheduling, as well as application registration, removal of event handlers, and related descriptor interfaces. It is the dispatch core of the event handler and uses the event splitter to wait for the event to occur. Once an event occurs, the reactor first separates each event and then dispatches the callback function in the event handler for the specific event to handle the event.
• L Event Separator (demultiplexer): is an I/O multiplexing function provided by the operating system, where we choose Epoll. Used to wait for one or more events to occur. The caller will be blocked until an event occurs on the descriptor set that separates the splitter.
• L event handler (even handler): Event handlers provide a set of interfaces, each of which corresponds to a type of event that the reactor invokes when the corresponding event occurs and performs the corresponding event handling. Typically, each specific event handler will always bind a valid descriptor handle to identify events and services.

Reactor Event Processing Flow

Reactor event processing Flow 3-5, it is divided into two parts, one is the event registration part, and the other is the event Distribution section, specifically described below.

Figure 3-5 reactor event processing sequence diagram

In the Event Registration section, the application first expects the registered socket descriptor as the event source, encapsulates the descriptor and the event-handling callback function corresponding to the event into the specific event handler, and registers the event handler with the event reactor. The event reactor receives the event, processes it, and registers the registration information with the event separator Epoll again. Finally, in the Epoll separator, add descriptors and their events through EPOLL_CTL, and return the registration results in layers.

In the Event handling section, the first event reactor is called by the epoll_wait of the event splitter, which causes the thread to block waiting for the registration event to occur. At this point, if a registered event occurs, Epoll_wait will return and return the set of events containing the registration event to the event reactor. After the event is received by the reactor, the event handler for the event is found based on the event source, and the event type is determined, and the event processing is done in this specific callback function based on the specific callback function encapsulated by the event type prior to the event handler invocation.

Depending on the event-handling flow of the reactor model, the application participates only in the initial Event Registration section. The application is not directly involved in the subsequent process of waiting and processing the entire event, and the final event processing is also delegated to the event reactor. So by using the reactor mode, the application doesn't have to worry about how the event came and when it comes, we just have to set up the appropriate processing when registering the event. This also reflects the "Hollywood principles" in design patterns, and the process of handling events is reversed by the event reactor control.

Use of the reactor mode

So far, the reactor event processing mode has been preliminarily formed, we can gradually realize the bottom of the service-end network framework by further using and perfecting the reactor mode.

We return the reactor event splitter Epoll to the next call to the server Epoll block, called an event loop. As shown in 3-6, we will conduct a detailed analysis of the entire business process from the point of view of the business to the server system listening to the client connected to the readable data that processes each connection.

Figure 3-6 Event loops in reactor mode

As soon as the server was started, we completed the initialization and registered the server listening socket and the corresponding listener socket processor in the reactor reactor. The system then enters the reactor's event loop waiting for the registration event to occur.

At this point, if an event occurs and the event is a readable event listening on the socket, a new connection is generated. Reactor callback before registering the listener socket handler. In this handler process, we obtain a new connection socket via the Accept system call and register the newly connected socket with its corresponding connection socket processor in the reactor reactor. After executing the listener socket handler, if there is still an event unhandled, we proceed to callback processing in the handler of the event, otherwise we go into the next event loop and continue to call Epoll blocking to wait for the new event.

At this point we have registered two types of events in the reactor reactor, one is to listen for socket readable events, to represent a new connection, and the other is to connect a socket-readable event that receives data from the client on behalf of the connection socket.

If another event occurs and you are listening for a socket-readable event, continue with the above processing. If the socket is connected, the reactor callback registers the connection socket handler. In this handler callback, we get the data sent by the client by invoking the read system call and do further processing according to our specific business requirements. Until all the events have been processed, we go into the next event loop again, and continue calling Epoll blocking to wait for the new event.

Through the above business implementation, we have completed a single-threaded server to establish a new client connection, and to receive and process client data work. With the reactor model, we not only guarantee high concurrency and high performance, but also separate the network details from the business logic. We only need to implement specific business logic in the different event handler that we register.

C + + server Design (i): Reactor mode based on I/O multiplexing