36. IO model, 36io Model

36. IO model, 36io Model

In particular, this essay is copied to egon (Lin Haifeng ).

 

I. IO model Introduction

To better understand the IO model, we need to review in advance: synchronous, asynchronous, blocking, non-blocking

What are the differences between synchronous I/O and asynchronous I/O, blocking I/O, and non-blocking I/O? Different people may give different answers to this question. For example, wiki considers asynchronous IO and non-blocking IO as one thing. This is because different people have different knowledge backgrounds and the context is different when discussing this issue. Therefore, to better answer this question, I will first limit the context of this article.

The background of this article is network IO in Linux. The most important references in this article are Richard Stevens's "UNIX Network Programming Volume 1, Third Edition: The Sockets Networking", section 6.2 "I/O Models ", this section describes in detail the features and differences of various IO. If the English is good enough, we recommend that you read it directly. Steven S's style is famous for its depth, so you don't have to worry about it. The flowchart in this article is also taken from references.

In this article, Steven s compares five IO models:
* Blocking IO
* Nonblocking IO
* IO multiplexing
* Signal driven IO
* Asynchronous IO
Signal driven I/O (signal-driven I/O) is not commonly used in practice, so we mainly introduce the remaining four I/O models.

Let's talk about the objects and steps involved when I/O occurs. For a network IO (Here we use read as an example), it involves two system objects, one is to call the IO process (or thread), and the other is the system kernel (kernel ). When a read operation occurs, the operation goes through two phases:

#1) Waiting for data preparation (Waiting for the data to be ready) #2) copy the data from the kernel to the process (Copying the data from the kernel to the process)

It is important to remember these two points, because the differences between these I/O models are different in the two phases.

2. blocking IO)

In linux, all sockets are blocking by default. A typical read operation process is like this:

When the user process calls the recvfrom system call, the kernel starts the first stage of IO: Prepare data. For network io, data has not arrived at the beginning (for example, a complete UDP packet has not yet been received). At this time, the kernel will wait for enough data to arrive.

On the user process side, the whole process will be blocked. When the kernel waits until the data is ready, it will copy the data from the kernel to the user memory, and then the kernel returns the result, the user process will unblock the status and run it again.
Therefore, blocking IO is characterized by being block in both stages of IO execution (waiting for data and copying data.

Almost all the network programming that programmers first came into contact with started from interfaces such as listen (), send (), and recv, by using these interfaces, you can easily build server/client models. However, most socket interfaces are blocking. For example

Ps: the so-called blocking interface means that the system call (usually the IO Interface) does not return the call result and keeps the current thread congested, it is returned only when the system call gets the result or a timeout error occurs.

In fact, almost all I/O interfaces (including socket interfaces) are blocked unless otherwise specified. This brings a big problem to network programming. For example, when calling recv (1024), the thread will be blocked. During this period, the thread cannot perform any operations or respond to any network requests.

A simple solution:

# Use multiple threads (or multiple processes) on the server ). Multi-thread (or multi-process) is designed to give each connection an independent thread (or process), so that the blocking of any connection will not affect other connections.

The problem with this solution is:

# Enable multi-process or multi-thread mode. If you want to respond to hundreds of connection requests at the same time, both multi-thread and multi-process will occupy system resources and reduce system response efficiency to the outside world, in addition, threads and processes are more likely to enter the suspended state.

Improvement Plan:

# Many programmers may consider using the "Thread Pool" or "connection pool ". The "Thread Pool" is designed to reduce the frequency of thread creation and destruction. It maintains a reasonable number of threads and allows Idle threads to re-undertake new execution tasks. The "connection pool" maintains the connection cache pool, reusing existing connections as much as possible, and reducing the frequency of creating and disabling connections. Both technologies can effectively reduce system overhead and are widely used in many large systems, such as websphere, tomcat, and various databases.

After improvement, the solution also has the following problems:

# The "Thread Pool" and "connection pool" technologies only alleviate the resource occupation caused by frequent io api calls. In addition, the so-called "pool" always has its upper limit. When the request exceeds the upper limit, the system composed of "pool" does not respond much better to the outside world than when there is no pool. Therefore, when using the "pool", you must consider the response scale and adjust the size of the "pool" based on the response scale.

Corresponding to thousands or even tens of thousands of client requests that may occur at the same time in the preceding example, the "Thread Pool" or "connection pool" may relieve some of the pressure, but it cannot solve all the problems. In short, the multi-threaded model can easily and efficiently solve small-scale service requests. However, in the face of large-scale service requests, the multi-threaded model may also encounter bottlenecks. You can try to solve this problem by using non-blocking interfaces.

 

Three non-blocking IO (non-blocking IO)

In Linux, you can set socket to non-blocking. When a read operation is performed on a non-blocking socket, the process looks like this:

It can be seen that when a user process sends a read operation, if the data in the kernel is not ready, it does not block the user process, but immediately returns an error. From the perspective of the user process, after initiating a read operation, it does not need to wait, but immediately gets a result. When the user process determines that the result is an error, it will know that the data is not ready, so the user can do other things within the interval of the next read query, or directly resend the read operation. Once the data in the kernel is ready and the system call of the user process is received again, it immediately copies the data to the user memory (this phase is still blocked ), then return.

That is to say, after a non-blocking recvform system call, the process is not blocked, and the kernel immediately returns to the process. If the data is not ready, an error is returned. After the process returns, it can do something else and then initiate a recvform system call. Recvform system calls are carried out cyclically. This process is usually called round robin. Round Robin checks kernel data until the data is ready, and then copies the data to the process for data processing. Note that the process of copying data is still blocked.

Therefore, in non-blocking IO, user processes need to actively ask whether the kernel data is ready.

Non-blocking IO instances

However, the non-blocking I/O model is never recommended.

We can't do this. Advantages: we can do other jobs (including submitting other jobs, that is, multiple tasks can be executed simultaneously in the "background ).

However, it is hard to hide its disadvantages:

#1. loop call recv () will greatly increase the CPU usage. This is also the time in the code. sleep (2) reasons, otherwise it is very easy to get stuck in the low-configuration host #2. the response delay of the task is increased because the read operation is polling once every time, and the task may be completed at any time between two polling times. This will reduce the overall data throughput.

In addition, in this solution, recv () is used to detect whether the operation is completed, the actual operating system provides more efficient interfaces for detecting whether operations are completed. For example, the select () multiplexing mode can detect multiple connections at a time.

 

IO multiplexing)

IO multiplexing may be a bit unfamiliar, but if I say select/epoll, I will probably understand it. This IO method is also called in some placesEvent-driven IO(Event driven IO ). We all know that the benefit of select/epoll is that a single process can process the IO of multiple network connections at the same time. The basic principle of this function is that the select/epoll function will continuously poll all the sockets in charge. When a socket has data, it will notify the user process. Its Process

When a user process calls the select statement, the entire process will be blocked. At the same time, the kernel will "Monitor" All sockets under the select statement. When the data in any socket is ready, select returns. At this time, the user process then calls the read operation to copy data from the kernel to the user process.
This graph is not much different from the blocking IO graph, but it is actually worse. Here we need to use two system calls (select and recvfrom), while blocking IO only calls one System Call (recvfrom ). However, the advantage of using select is that it can process multiple connections at the same time.

Note:

1. If the number of connections to be processed is not very high, the web server using select/epoll may not have better performance than the web server using multi-threading + blocking IO, and may have a greater latency. The advantage of select/epoll is not that it can process a single connection faster, but that it can process more connections.

2. In the multiplexing model, each socket is generally set to non-blocking. However, as shown in, the entire user process is always blocked. However, process is block by the select function, rather than block by socket IO.

Conclusion: The advantage of select is that it can process multiple connections and is not applicable to a single connection. 

Select network I/O model

 

Analysis of the Process of select monitoring fd changes:

# The user process creates a socket object and copies the listening fd to the kernel space. Each fd corresponds to a system file table. After the fd of the kernel space responds to the data, it sends a signal to the user's process that the data has arrived; # The user process then sends a system call, for example, (accept) to copy the data in the kernel space to the user space, at the same time, it is used to clear data from the kernel space of the receiver, in this way, new data of fd can be returned when the listener is re-monitored (the sender needs to receive the response to clear the response because it is based on the TCP protocol ).

Advantages of this model:

# Compared with other models, the select () event-driven model only uses a single thread (process) for execution, consumes less resources, does not consume too much CPU, and can provide services for multiple clients. If you try to build a simple event-driven server program, this model has some reference value.

Disadvantages of this model:

# First, the select () interface is not the best choice to implement "event-driven. When the handle value to be tested is large, the select () interface itself consumes a lot of time to poll each handle. Many operating systems provide more efficient interfaces, such as epoll in linux, kqueue in BSD, And/dev/poll in Solaris ,.... If you want to implement more efficient server programs, Apis like epoll are more recommended. Unfortunately, the epoll interfaces provided by different operating systems are quite different. Therefore, it is difficult to use the epoll-like interfaces to implement servers with better cross-platform capabilities. # Second, this model integrates event detection and Event Response. Once the Event Response execution body is large, it is disastrous for the entire model.

Five Asynchronous IO (Asynchronous I/O)

In Linux, asynchronous IO is not used much. It was introduced from kernel 2.6. Let's take a look at its process:

After the user process initiates the read operation, it can immediately start to do other things. On the other hand, from the perspective of kernel, when it receives an asynchronous read, it will first return immediately, so it will not generate any block to the user process. Then, the kernel will wait for the data preparation to complete and then copy the data to the user memory. After all this is done, the kernel will send a signal to the user process to tell it that the read operation is complete.

 

Comparison and Analysis of six IO Models

So far, four I/O models have been introduced. Now let's look back at the first few questions: What is the difference between blocking and non-blocking? What is the difference between synchronous IO and asynchronous IO.
Answer the simplest one: blocking vs non-blocking. The difference between the two is clearly described in the previous introduction. Calling blocking IO will block the corresponding process until the operation is completed, while non-blocking IO will return immediately when the kernel still prepares data.

Before describing the differences between synchronous IO and asynchronous IO, you must first define the two. The definitions provided by Steven S (actually the POSIX definition) are like this:
A synchronous I/O operation causes the requesting process to be blocked until that I/O operationcompletes;
An asynchronous I/O operation does not cause the requesting process to be blocked;
The difference between the two is that synchronous IO blocks process when performing "IO operation. According to this definition, the four IO models can be divided into two categories. The previously described blocking IO, non-blocking IO, and IO multiplexing all belong to the synchronous IO class, while the asynchronous I/O class.

Some may say that non-blocking IO is not blocked. Here is a very "Tricky" place. The "IO operation" in the definition refers to the real IO operation, that is, the recvfrom system call in the example. When non-blocking IO executes the recvfrom system call, if the kernel data is not ready, the process will not be blocked. However, when the data in the kernel is ready, recvfrom will copy the data from the kernel to the user memory. At this time, the process is blocked. During this time, the process is blocked. Asynchronous IO is different. When a process initiates an I/O operation, it directly returns the result and ignores it again until the kernel sends a signal telling the process that I/O is complete. In this process, the process is not blocked at all.

Comparison of IO models:

After the above introduction, we will find that the difference between non-blocking IO and asynchronous IO is quite obvious. In non-blocking IO, although the process is not blocked for most of the time, it still requires the process to take the initiative to check, and after the data preparation is complete, the process also needs to actively call recvfrom again to copy data to the user memory. Asynchronous IO is completely different. It is like a user process handing over the entire IO operation to another person (kernel) to complete, and then the other person will send a signal after completion. During this period, the user process does not need to check the I/O operation status or actively copy data.

 

Seven selectors module select, poll, epoll

These three IO multiplexing models have different support on different platforms, but epoll is not supported in windows. Fortunately, we have the selectors module, which helps us select the most suitable one on the current platform by default.

View Code