Introduction and comparison of several network server Models

The title of the original article is "using the event-driven model for efficient and stable network server programs"

 

---------------------------------------------------------------- Gorgeous split line --------------------------------------------------------------

Preface

Event drivers are familiar to a large number of programmers. The most interesting thing is the application in graphic interface programming. In fact, event drivers are also widely used in network programming, and large-scale deployment in High-connection high-throughput server programs, such as HTTP server programs, FTP server programs and so on. Compared with the traditional network programming method, event-driven greatly reduces resource usage, increases service reception capability, and improves network transmission efficiency.

As for the server model mentioned in this article, you can refer to a lot of implementation code on the search network. Therefore, this article will not stick to the display and analysis of source code, but focus on the introduction and comparison of models. The server model using the libev event-driven Library provides the implementation code.

This article involves the thread/time legend, which only indicates that the thread does have a blocking latency on each Io, but does not guarantee the correctness of the latency ratio and IO execution sequence. In addition, the interfaces mentioned in this article are only Unix/Linux interfaces that I am familiar with. Windows interfaces are not recommended. You can check the corresponding Windows interfaces on your own.

 

Blocked Network Programming Interface

Almost all the network programming that programmers first came into contact with started from interfaces such as listen (), send (), and Recv. By using these interfaces, you can easily build server/client models.

We assume that we want to create a simple server program to provide a single client with a content service similar to "one question and one answer.

Figure 1. Simple Server/client Model
 

We noticed that most of the socket interfaces are blocking. The so-called blocking interface means that a system call (generally an I/O interface) does not return the call result and keeps the current thread congested. It is returned only when the system call gets the result or times out and an error occurs.

In fact, almost all I/O interfaces (including socket interfaces) are blocked unless otherwise specified. This brings a big problem to network programming. For example, when sending () is called, the thread will be blocked. During this period, the thread cannot perform any operations or respond to any network requests. This poses a challenge to network programming with multiple clients and multiple business logic. At this time, many programmers may choose multiple threads to solve this problem.

 

Multi-thread server program

The simplest solution for multi-client network applications is to use multithreading (or multi-process) on the server side ). Multi-thread (or multi-process) is designed to give each connection an independent thread (or process), so that the blocking of any connection will not affect other connections.

The specific use of multi-process or multi-thread does not have a specific mode. Traditionally, the process overhead is much greater than the thread. Therefore, if you need to provide services for a large number of clients at the same time, multi-process is not recommended; if a single service execution body consumes a large amount of CPU resources, such as large-scale or long-term data operations or file access, the process is safer. Generally, use pthread_create () to create a new thread and fork () to create a new process.

We assume that we have higher requirements for the above server/client model, that is, to allow the server to provide Q & A services for multiple clients at the same time. So we have the following model.

Figure 2. multi-threaded Server Model
 

In the preceding thread/time legend, the main thread continues to wait for client connection requests. If there is a connection, a new thread is created and the same Q & A service is provided for the queue in the new thread.

Many beginners may not understand why a socket can be accept multiple times. In fact, the socket designer may leave a foreshadowing for the case of multiple clients, so that accept () can return a new socket. The following is a prototype of the accept interface:

int accept(int s, struct sockaddr *addr, socklen_t *addrlen);

The input parameter S is the socket handle value inherited from socket (), BIND (), and listen. After BIND () and listen () are executed, the operating system has started to listen to all connection requests at the specified port. If there is a request, the connection request is added to the Request queue. The accept () interface is called to extract the first connection information from the request queue of socket S and create a new socket return handle similar to that of socket S. The new socket handle is the input parameter of the subsequent read () and Recv. Accept ()
Will enter the blocking status until a request enters the queue.

The preceding multi-threaded server model seems to perfectly meet the requirements of providing Q & A services for multiple clients, but it is not all at all. If you want to respond to hundreds of thousands of connection requests at the same time, both multi-thread and multi-process will occupy system resources and reduce system response efficiency to the outside world, the thread and process itself are more likely to enter the false state.

Many programmers may considerThread Pool"Or"Connection Pool". The "Thread Pool" is designed to reduce the frequency of thread creation and destruction. It maintains a reasonable number of threads and allows Idle threads to re-undertake new execution tasks. The "connection pool" maintains the connection cache pool, reusing existing connections as much as possible, and reducing the frequency of creating and disabling connections. Both technologies can effectively reduce system overhead and are widely used in many large systems, such as websphere, tomcat, and various databases.

However, the "Thread Pool" and "connection pool" technologies only alleviate the resource occupation caused by frequent IO Interface calls. In addition, the so-called "pool" always has its upper limit. When the request exceeds the upper limit, the system composed of "pool" does not respond much better to the outside world than when there is no pool. Therefore, when using the "pool", you must consider the response scale and adjust the size of the "pool" based on the response scale.

Corresponding to thousands or even tens of thousands of client requests that may occur at the same time in the preceding example, the "Thread Pool" or "connection pool" may relieve some of the pressure, but it cannot solve all the problems.

In short, the multi-threaded model can easily and efficiently solve small-scale service requests, but the multi-threaded model is not the best solution in the face of large-scale service requests. In the next chapter, we will discuss how to solve this problem by using non-blocking interfaces.

 

Non-blocking server programs

The above problems are caused by the blocking feature of Io interfaces to some extent. Multithreading is a solution. Another solution is to use non-blocking interfaces.

Non-blocking interfaces are significantly different from blocking interfaces in that they are returned immediately after being called. You can use the following function to set a handle FD to a non-blocking state.

fcntl( fd, F_SETFL, O_NONBLOCK );

The following describes how to use only one thread, but check whether data is delivered from multiple connections at the same time and accept data.

Figure 3. Use a non-blocking receiving data model
 

In the non-blocking status, the Recv () interface returns immediately after being called. The returned value indicates different meanings. In this example,

• The value of Recv () is greater than 0, indicating that the data is accepted. The returned value is the number of bytes received;
• Recv () returns 0, indicating that the connection has been normally disconnected;
• Recv () returns-1, and errno is equal to eagain, indicating that the Recv operation has not been completed;
• -1 is returned for Recv (), and errno is not equal to eagain, which indicates that the Recv operation encounters a system error errno.

The server thread can call the Recv () interface cyclically to receive data from all connections in a single thread.

However, the above models are never recommended. Because loop calling Recv () will greatly increase the CPU usage. In this solution, Recv () is more used to detect whether the operation is complete, the actual operating system provides more efficient interfaces for detecting whether an operation is completed, such as select ().

 

Event-driven server model using the select () Interface

Most Unix/Linux systems support the select function, which is used to detect the status changes of multiple file handles. The following is a prototype of the Select Interface:

FD_ZERO(int fd, fd_set* fds) FD_SET(int fd, fd_set* fds) FD_ISSET(int fd, fd_set* fds) FD_CLR(int fd, fd_set* fds) int select(int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, struct timeval *timeout)

Here, the fd_set type can be simply understood as a queue that marks the handle by bit. For example, to mark a handle with a value of 16 in an fd_set, the 16th bits of the fd_set are marked as 1. You can use a macro such as fd_set and fd_isset to implement specific positions and verification. In the select () function, readfds, writefds, and limit TFDs are both input and output parameters. If the input readfds is marked with a 16th handle, select () checks whether the 16th handle is readable. After Select () returns, you can check
Whether readfds marks the handle number 16 to determine whether the "readable" event has occurred. In addition, you can set the timeout time.

Next, we will re-simulate the model for receiving data from multiple clients in the previous example.

Figure 4. Receiving data model using select ()
 

The preceding model only describes the process of using the select () interface to receive data from multiple clients at the same time () the interface can detect the read, write, and error statuses of multiple handles at the same time, so it is easy to build a server system that provides independent Q & A services for multiple clients.

Figure 5. event-driven server model using the select () Interface
 

It should be noted that a connect () operation on the client will inspire a "readable event" on the server side, so select () can also detect the connect () behavior from the client.

In the above model, the most important thing is how to dynamically maintain the three select () parameters readfds, writefds, and limit TFDs. As the input parameter, readfds should mark all the "readable event" handles to be detected, which will always include the "parent" handle of the test connect (). At the same time, writefds and writable TFDs should mark all the "writable events" and "error events" handles to be detected (marked using fd_set ).

As an output parameter, the handles of all events captured by select () are saved in readfds, writefds, and limit TFDs. The programmer needs to check all the flag bits (using fd_isset () to determine which handles have an event.

The above model mainly simulates the "one-question-one-answer" service process. Therefore, if select () finds that a handle captures a "readable event", the server program should perform Recv () operations in a timely manner, prepare the data to be sent based on the received data, add the corresponding handle value to writefds, and prepare the select () test for the next "writable event. Similarly, if select () finds that a handle captures a "writable event", the program should promptly perform the send () operation and prepare for the next "readable event" detection. Describes an execution cycle in the preceding model.

Figure 6. One execution cycle
 

This model features that each execution cycle detects one or more events, and a specific event triggers a specific response. We can classify this model as"Event-driven model".

Compared with other models, the Select () event-driven model only uses a single thread (process) for execution, consumes less resources, does not consume too much CPU, and can provide services for multiple clients. If you try to build a simple event-driven server program, this model has some reference value.

However, this model still has many problems.

First, the Select () interface is not the best choice for implementing "event-driven. When the handle value to be tested is large, the Select () interface itself consumes a lot of time to poll each handle. Many operating systems provide more efficient interfaces, such as epoll in Linux, kqueue in BSD, And/dev/poll in Solaris .... If you want to implement more efficient server programs, Apis like epoll are more recommended. Unfortunately, the epoll interfaces provided by different operating systems are quite different. Therefore, it is difficult to use the epoll-like interfaces to implement servers with better cross-platform capabilities.

Secondly, this model integrates event detection and Event Response. Once the execution body of the Event Response is large, it is disastrous for the entire model. As shown in the following example, the large execution body 1 will directly lead to a delay in the execution body responding to event 2, and greatly reduce the timeliness of event detection.

Figure 7. Impact of a large execution body on the event-driven model using select ()
 

Fortunately, there are many efficient event-driven libraries that can block the above difficulties. Common event-driven libraries include the libevent Library and the libev library as a replacement for libevent. These libraries will select the most appropriate event detection interface based on the characteristics of the operating system, and add signal and other technologies to support asynchronous response, this makes these libraries the best choice for building an event-driven model. The following chapter describes how to use the libev database to replace the select or epoll interface to implement an efficient and stable server model.

 

Server model using the event-driven library libev

Libev is a high-performance event loop/event-driven library. As an alternative to libevent, the first version of libevent was released in November 2007. Libev designers claim that libev has advantages such as faster speed, smaller volume, and more functions, which are proven in many evaluations. Because of its good performance, many systems begin to use the libev library. This chapter describes how to use libev to implement the server that provides the Q & A service.

(In fact, there are many existing event loops/event-driven libraries, and the author does not intend to recommend readers to use the libev library, it is only to illustrate the convenience and benefits of the event-driven model for Network Server programming. Most event-driven libraries have interfaces similar to the libev library. As long as you understand the general principles, you can flexibly select the appropriate library .)

Similar to the model in the previous chapter, libev also needs to cyclically detect whether an event is generated. The libev loop body is expressed in the ev_loop structure and started with ev_loop.

void ev_loop( ev_loop* loop, int flags )

Libev supports eight types of events, including Io events. An IO event is characterized by ev_io and initialized using the ev_io_init () function:

void ev_io_init(ev_io *io, callback, int fd, int events)

The initialization content includes the callback function callback, the tested handle FD and the event to be detected, the ev_read table "readable event", and the ev_write table "writable event ".

Now, you only need to add or remove some ev_io from ev_loop when appropriate. Once added, the next loop will check whether the event specified by ev_io has occurred. If the event is detected, ev_loop will automatically execute the callback function callback () of ev_io (); if ev_io is deregistered, the corresponding event is not detected.

Whether an ev_loop is started or not, you can add or delete one or more ev_io. The added and deleted interfaces are ev_io_start () and ev_io_stop ().

void ev_io_start( ev_loop *loop, ev_io* io ) void ev_io_stop( EV_A_* )

As a result, we can easily draw the following "one question and one answer" server model. Because the server does not take the initiative to terminate the connection mechanism, each connection can be maintained at any time, and the client can freely choose to exit.

Figure 8. Server Model Using libev Library
 

The above model can accept any number of connections and provides a completely independent Q & A service for each connection. With the event loop/event-driven interfaces provided by libev, the above model has the opportunity to provide high efficiency, low resource usage, good stability, and simple writing features that other models cannot provide.

Because traditional Web servers, FTP servers, and other network applications all have the communication logic of "one question and one answer, therefore, the above "one-question-one-answer" model using the libev library has reference value for building similar server programs. In addition, for applications that require remote monitoring or remote control, the above model also provides a feasible implementation scheme.

 

Summary

This article focuses on how to build a server program that provides "one-question-one-answer", discusses the model implemented with the blocking Socket interface, uses the multi-threaded model, and uses select () the event-driven Server Model of the interface until the server model of the libev event-driven library is used. The article compares the advantages and disadvantages of various models and draws a conclusion from the comparison that the "event-driven model" can be used to implement more efficient and stable server programs. The various models described in this article can provide readers with reference value for network programming.

---------------------------------------------------------------- Gorgeous split line --------------------------------------------------------------

 

From: http://www.ibm.com/developerworks/cn/linux/l-cn-edntwk/index.html? CA = Drs-