Five types of I/O modes for Linux

1) Blocking I/O (blocking I/O)
2) non-blocking I/O (nonblocking I/O)
3) I/O multiplexing (SELECT and poll) (I/O multiplexing)
4) Signal-driven I/O (signal driven I/O (SIGIO))
5) asynchronous I/O (asynchronous I/O (the POSIX aio_functions))

The first four types are synchronous, and only the last is asynchronous IO.

blocking I/O Model:

Summary: The process will block until the data copy is complete

The application calls an IO function that causes the application to block and wait for the data to be ready. If the data is not ready, wait .... The data is ready to be copied from the kernel to the user space, and the IO function returns a success indicator.

Our first exposure to network programming began with interfaces such as listen (), send (), recv (), and so on. Using these interfaces makes it easy to build a server/client model.

When the recv () function is called, the system first checks to see if there is ready data. If the data is not ready, then the system is in a wait state. when the data is ready, the data is copied from the system buffer to the user space , and then the function returns. In a socket application, when the recv () function is called, the data is not necessarily present in the user space, then the recv () function is in a wait state.

Sockets API calls are divided into the following four types:

1. Input operations: recv (), Recvfrom (), WSARecv (), and WSARecvFrom () functions. Call the function to receive data as a blocking socket for the parameter. If no data is readable in the socket buffer at this point, the calling thread sleeps until the data arrives.

2. Output operations: Send (), sendto (), WSASend (), and WSASendTo () functions. Calls the function to send data with a blocking socket as a parameter. If the socket buffer does not have free space, the thread will sleep until there is space.

3. Accepts connections: the Accept () and wsaacept () functions. Call the function as a blocking socket, waiting for the connection request to be accepted. If there is no connection request at this time, the thread goes to sleep.

Complements the function of a listen function: The Listen function uses an active connection socket interface to become a connected socket interface, which allows a process to accept requests from other processes, thus becoming a server process. In TCP server programming, the Listen function transforms a process into a server and specifies that the corresponding socket becomes a passive connection.

4. Outgoing connections: Connect () and WSAConnect () functions. For a TCP connection, the client invokes the function to initiate a connection to the server, with the blocking socket as the parameter. The function does not return until it receives a reply from the server. This means that TCP connections always wait at least one round trip to the server.

Blocking mode poses a big problem for network programming, such as when calling Send (), the thread will be blocked, during which time the thread will be unable to perform any operations or respond to any network requests. this poses a challenge to network programming for multi-client and multi-business logic. At this point, we may choose the multi-threaded way to solve this problem.

For Multi-Client network applications, the simplest solution is to use multithreading (or multiple processes) on the server side. The purpose of multithreading (or multi-process) is to have separate threads (or processes) for each connection, so that blocking of any one connection does not affect other connections .

The specific use of multi-process or multi-threading, and does not have a specific pattern. Traditionally, processes are much more expensive than threads, so if you need to serve more clients at the same time, it is not recommended to use multiple processes, and if a single service executor needs to consume more CPU resources, such as large-scale or long-time data operations or file access, The process is more secure (threads need to be synchronized). Typically, a new thread is created with Pthread_create (), andfork () creates a new process.

Multi-threaded/Process servers provide answering services to multiple clients at the same time. The model is as follows:

The main thread continues to wait for the client's connection request and, if there is a connection, creates a new thread and provides the same question and answer service as in the new thread.

The multi-threaded server model seems to be a perfect solution to the requirements for answering questions and answers for multiple clients, but not really. If you want to respond to hundreds or thousands of connection requests at the same time, no matter how many threads or processes will take up the system resources, reduce the system to the external response efficiency, and the thread and process itself more easily into the suspended animation state .

You might consider using the thread pool or connection pool . The thread pool is designed to reduce the frequency of creating and destroying threads, maintaining a reasonable number of threads, and allowing idle threads to re-assume new execution tasks. Connection pooling maintains a connected cache pool, reusing existing connections as much as possible, and reducing the frequency with which connections are created and closed. Both of these technologies can be very good to reduce the system overhead, are widely used in many large-scale systems, such as Apache,mysql database.

However, the thread pool and connection pooling techniques are only to some extent mitigated by the frequent invocation of the IO interface for resource consumption. Moreover, the so-called "pool" always has its upper limit, when the request greatly exceeds the upper limit, the "pool" composed of the system response to the outside world is not much better than when there is no pool. So using the pool must consider the scale of the response it faces and adjust the size of the pool based on the response scale. The "thread pool" or "Connection pool" may alleviate some of the stress, but not all of them, in response to the thousands or even thousands of client requests that may appear in the previous example.

Add to the connection pool:  The application of CS method is inevitably to encounter the problem of socket connection, many times, the use of the socket library in the programming language, the use of some are not accustomed to, although the system comes with the library in a lot of exception handling, stability and a lot of effort, But to understand and use those libraries, such as the socket connection pool to go a lot of detours. Here I'll talk to you about how to create and use the socket link pool.     There are two ways to get a generic socket Link: long (regular) links and short links.     Long link : The socket link continues to open when the data is sent. Keep until the exception or the program exits, the advantage of this way is not every time to initiate a connection disconnect, at a faster speed than a short connection, but relatively to the server resource pressure is also greater. Long links with a wide range, such as game system, QQ and so on, long (often) links generally also need to periodically ping data to the server to ensure that the socket link unblocked. You need to re-open the link when the ping does not connect to the server.     Short link : When the data is sent, the active disconnection, each time the data sent to a link, disconnect operation, this way the advantage is: the server resource consumption is relatively small, but because each time to relink, the speed overhead is also relatively large, This approach is useful for situations where it is not necessary to interact with the server frequently. The     above two methods in the case of a very large number of users, there are a lot of shortcomings, so we can consider a compromise approach, that is, the connection pool using the socket.    The program starts by initializing the creation of a number of long links. Give them an identity bit that indicates whether the link is idle. When the data needs to be sent, the system assigns it a currently idle link. At the same time, the resulting link is set to "busy", when the data is sent, the link identifier bit set to "free", so that the system can be assigned to the next user, so that the advantages of both methods are fully played out. 

Non-blocking IO model:

Summary: Non-blocking IO calls the IO function repeatedly through the process (multiple system calls and returns immediately); In the process of data copying, the process is blocked;

We set a socket interface to non-blocking to tell the kernel that when the requested I/O operation cannot be completed, do not sleep the process , but return an error. This way our I/O operations function will constantly test whether the data is ready, and if not, continue testing until the data is ready. In this continuous testing process, the CPU will be a lot of time.

Set the socket to non-blocking mode, that is, notify the system kernel: when calling the Windows Sockets API, do not let the thread sleep, but should let the function return immediately. On return, the function returns an error code. As shown in the figure, a non-blocking pattern socket calls the recv () function multiple times. The kernel data is not ready when you call the recv () function the first three times. Therefore, the function immediately returns the WSAEWOULDBLOCK error code. The fourth time the recv () function is called, the data is ready to be copied into the application's buffer, and the recv () function returns a successful indication that the application is starting to process the data.

When you create a socket using the socket () function and the WSASocket () function, the default is blocked. After the socket is created, the socket is set to non-blocking mode by calling the Ioctlsocket () function. The functions under Linux are: Fcntl ().
When the socket is set to non-blocking mode, the calling function returns immediately when calling the Windows Sockets API function. In most cases, these function calls will call "failed" and return the Wsaewouldblock error code. Indicates that the requested operation did not have time to complete during the call. Typically, the application needs to call the function repeatedly until a successful return code is obtained.

It is necessary to note that not all Windows Sockets APIs are called in nonblocking mode and will return a wsaewouldblock error. For example, when you call the bind () function in a non-blocking socket for a parameter, the error code is not returned. Of course, the error code is not returned when the WSAStartup () function is called, because the function is the first call of the application and certainly does not return such an error code.

To set the socket to non-blocking mode, you can use the WSAAsyncSelect () and WSAEventSelect () functions in addition to the ioctlsocket () function. When the function is called, the socket is automatically set to non-blocking mode.

Because a non-blocking socket is used when calling a function, wsaewouldblock errors are often returned. So at any time, be careful to check the return code and prepare for the "failure". The application calls this function continuously until it returns to the successful instruction. In the above program manifest, the recv () function is constantly called in the while loop to read in 1024-byte data. This approach is a waste of system resources.

To do this, someone calls the recv () function using the MSG_PEEK flag to see if there is data readable in the buffer. Similarly, this method is not good. Because this practice is costly to the system, the application must call the recv () function at least two times to actually read the data. It is a good practice to use the "I/O model" of sockets to determine whether non-blocking sockets are readable and writable.

Non-blocking mode sockets are not easy to use compared to blocking mode sockets. With non-blocking mode sockets, you need to write more code to process the received Wsaewouldblock errors in each Windows Sockets API function call. Therefore, non-blocking sockets appear to be difficult to use.

However, the non-blocking sockets in the control of the establishment of multiple connections, the data received and received the amount of uneven, time is uncertain, obviously has the advantage. This kind of socket is difficult to use, but as long as the difficulty is eliminated, it is very powerful in function. In general, consider using the "I/O model" of sockets, which helps the application to manage the communication of one or more sockets asynchronously.

IO multiplexing Model:

Introduction: Mainly select and Epoll; for an IO port, two calls, two return, there is no advantage than blocking IO, the key is to enable the simultaneous monitoring of multiple IO ports;

The I/O multiplexing model uses the Select, poll, Epoll functions, which also block the process, but unlike blocking I/O, these two functions can block multiple I/O operations at the same time. I/O functions can be detected at the same time for multiple read operations, multiple write operations, and I/O operation functions are not actually invoked until there is data readable or writable.

Signal-driven IO

Introduction: Two calls, two returns;

First we allow the socket interface to drive the signal-driven I/O and install a signal processing function, and the process continues to run without blocking. When the data is ready, the process receives a sigio signal that can be called by the I/O operation function in the signal processing function to process the data.

Asynchronous IO Model

Summary: The process does not need to block when copying data.

When an asynchronous procedure call is made, the caller cannot get the result immediately. The part that actually handles the call notifies the caller of the input-output operation by state, notification, and callback after completion

Synchronous IO causes the process to block until the IO operation is complete.
Asynchronous IO does not cause the process to block.
Io multiplexing is blocked by a select call first.

Comparison of 5 I/O models:

Introduction to select, poll, Epoll

Epoll and select provide a multi-channel I/O multiplexing solution. In the current Linux kernel can be supported, where Epoll is unique to Linux, and select should be POSIX rules, the general operating system has implemented

Select

Select essentially processes the next step by setting or checking the data structure that holds the FD flag bit. The disadvantages of this are:

1, a single process can monitor the number of FD is limited, that is, the size of the listening port is limited.

In general, this number and system memory relationship is very large, the specific number can be cat/proc/sys/fs/file-max. A 32-bit machine defaults to 1024. The 64-bit machine defaults to 2048.

2, the socket scan is a linear scan, that is, the use of polling method, low efficiency:

When the socket is more, each time the select () through the traversal of the fd_setsize socket to complete the dispatch, regardless of which socket is active, are traversed again. This can waste a lot of CPU time. If you can register a callback function with the socket, and when they are active, the related actions are automatically done, then polling is avoided, which is exactly what Epoll and Kqueue do.

3, the need to maintain a large number of FD data structure, which will make the user space and kernel space in the transfer of the structure when the replication cost is large

Poll

Poll is essentially not the same as SELECT, it copies the user's incoming array to the kernel space, and then queries each FD corresponding device state, if the device is ready to add an entry in the device waiting queue and continue to traverse, if not found ready device after traversing all FD, the current process is suspended, Until the device is ready or the active timeout is awakened, it again iterates over the FD. This process has gone through many meaningless loops.

It does not have a limit of the maximum number of connections because it is stored based on a linked list, but there is also a disadvantage:

1, a large number of FD arrays are copied in the whole between the user State and the kernel address space, regardless of whether such replication is meaningful. 2, poll also has a feature is "horizontal trigger", if the FD is reported, is not processed, then the next poll will report the FD again.

Epoll:

The epoll supports both horizontal and edge triggering, and the biggest feature is the Edge trigger, which only tells the process which FD has just become the desired state and only notifies once. Another feature is that Epoll uses the "event" ready notification method to register FD through EPOLL_CTL, once the FD is ready, the kernel will use a callback mechanism similar to callback to activate the fd,epoll_wait to receive notification

Advantages of Epoll:

1, there is no limit of the maximum concurrent connection,The upper limit of the FD that can be opened is much larger than the maximum (1G memory can listen to about 100,000 ports);
2. Efficiency improvement, not polling, does not decrease as the number of FD increases in efficiency. Only active FD will invoke the callback function;
The biggest advantage of Epoll is that it's just your "active" connection, which has nothing to do with the total number of connections, so in a real network environment, Epoll is much more efficient than select and poll. 3. Memory Copy, using the mmap () file to map memory to accelerate message delivery to kernel space, that is, Epoll uses mmap to reduce replication overhead.

Select, poll, Epoll difference summary:

1. Support the maximum number of connections that a process can open

Select	The maximum number of connections that a single process can open is defined by the Fd_setsize macro, the size of which is 32 integers (on a 32-bit machine, the size is 32*32, and the 64-bit machine fd_setsize is 32*64), of course, we can modify it, and then recompile the kernel, However, performance may be impacted, which requires further testing.
Poll	Poll is essentially no different from Select, but it does not have the maximum number of connections because it is stored based on a linked list
Epoll	Although the number of connections is capped, but large, 1G of memory on the machine can open about 100,000 of the connection, 2G memory of the machine can open about 200,000 of the connection

2. The IO efficiency problem caused by FD surge

Select	Because the connection is linearly traversed each time it is invoked, the increase in FD results in a "linear descent performance problem" with slow traversal.
Poll	Ditto
Epoll	Because the implementation in the Epoll kernel is implemented according to the callback function on each FD, only the active socket will actively invoke callback, so in the case of less active sockets, using Epoll does not have a performance problem with the linear descent of the preceding two. However, when all sockets are active, there may be performance issues.

3. Message Delivery method

Select	The kernel needs to pass the message to the user space, requiring the kernel copy action
Poll	Ditto
Epoll	Epoll is implemented by sharing a piece of memory with the kernel and user space.

Summarize:

In summary, the choice of Select,poll,epoll should be based on the specific use of the occasion and the three ways of their own characteristics.

1, the surface of the Epoll performance is the best, but the number of connections and connections are very active, select and poll performance may be better than epoll, after all, Epoll notification mechanism requires a lot of function callbacks.

2. Select is inefficient because it requires polling every time. But inefficient is also relative, depending on the situation, but also through a good design to improve

Five types of I/O modes for Linux