Summary of the network I/O model and the Python example of the select Model

Synchronous, asynchronous, blocking, and non-blocking. when these network I/O terms are combined, it will inevitably confuse programming beginners, here we will summarize the Python instances of the network I/O model and select model for you: Network I/O model
If there are too many people, there will be problems. When the web first appeared, few people visited it. In recent years, the scale of network applications has gradually expanded, and the application architecture needs to change. The C10k problem allows engineers to think about service performance and application concurrency.

Network applications need to deal with only two types of problems: network I/O and data computing. Compared with the latter, network I/O latency brings more performance bottlenecks to applications than the latter. The network I/O models are roughly as follows:

• Synchronous model (synchronous I/O)
• Blocking I/O (bloking I/O)
• Non-blocking I/O (non-blocking I/O)
• Multiplexing I/O (multiplexing I/O)
• Signal-driven I/O (signal-driven I/O)
• Asynchronous I/O (asynchronous I/O)

The essence of network I/O is socket reading. socket is abstracted as a stream in linux, and I/O can be understood as a stream operation. This operation is divided into two phases:

Wait for the data to be ready ).
Copy the data from the kernel to the process ).
For socket streams,

The first step usually involves waiting for data groups on the network to arrive and then being copied to a buffer zone of the kernel.
Step 2: copy data from the kernel buffer to the application process buffer.
I/O model:
Let's give a simple analogy to understand these models. Network I/O is like phishing. waiting for a fish bait is the process of waiting for data preparation in the network. when a fish bait is pulled ashore, it is the data replication phase in the kernel. Phishing is an application process.

Blocking I/O (bloking I/O)
Blocking I/O is the most popular I/O model. It conforms to the most common thinking logic. Blocking means that the process is "taken" and the CPU processes other processes. During network I/O, the process initiates a recvform system call, and then the process is blocked and does nothing until the data is ready and the data is copied from the kernel to the user process, finally, the process processes the data again. The whole process is blocked while waiting for the data to be processed. Cannot process other network I/O. For example:

This is like waiting for the fish to hook up. Then, let's throw the rod again and wait for the next bait. when you wait, you will probably think about nothing.

The feature of IO blocking is that both stages of IO execution are blocked.
Non-blocking I/O (non-bloking I/O)
During network I/O, non-blocking I/O will also call the recvform system to check whether the data is ready, unlike blocking I/O, "non-blocking divides a large block time into N-plus small blocks, so the process continuously has the opportunity to 'be patronized by the CPU ".

That is to say, after a non-blocking recvform system call, the process is not blocked, and the kernel immediately returns to the process. if the data is not ready, an error is returned. After the process returns, it can do something else and then initiate a recvform system call. Recvform system calls are carried out cyclically. This process is usually called round robin. Round Robin checks kernel data until the data is ready, and then copies the data to the process for data processing. Note that the process of copying data is still blocked.

We will use the phishing method for classification. when we throw the rod into the water, we will check whether there is any movement of the fish float. if there is no fish bait, we will do something else, such as digging a few more cockroaches. Then I will try again later to see if fish are hooked. In this way, the round-trip checks and leaves until the fish are hooked and then processed.

The non-blocking IO feature is that the user process needs to actively ask whether the kernel data is ready.
Multiplexing I/O (multiplexing I/O)
As can be seen, due to non-blocking calls, polling occupies a large part of the process, which consumes a lot of CPU time. Combine the previous two modes. If the polling is not the user State of the process, it would be nice to have some help. Multiplexing handles such problems.

Multiplexing has two special systems that call select or poll. The select call is kernel-level. The difference between select round-robin and non-blocking round-robin is that the former can wait for multiple sockets. when the data of any socket is good, and then the process calls the recvform system to copy data from the kernel to the user process. of course, this process is blocked. Multiplexing has two types of blocking. after the select or poll call, the process will be blocked. different from the first blocking, the select statement does not wait until all socket data arrives before processing, instead, a part of the data will be called by the user process for processing. How can I know that some data has arrived? The monitoring task is handed over to the kernel, which is responsible for processing data arrival. It can also be understood as "non-blocking.

For multiplexing, that is, polling multiple sockets. When we were fishing, we hired a helper who could discard multiple fishing rods at the same time. when any one of them hooked up, he would draw a tie. He is only responsible for helping us fish, and does not help us, so we have to wait and wait for him to take over. Let's try again. Since multiplexing can process multiple I/O operations, it brings about a new problem. The order between multiple I/O operations becomes uncertain. of course, it can also be tailored to different numbers.

Multiplexing is characterized by a mechanism in which a process can wait for the IO file descriptor at the same time. the kernel monitors these file descriptors (socket descriptors), and any of them enters the read-ready state, select, poll and epoll functions can be returned. You can also select, poll, and epoll monitoring methods.
I learned about the first three modes. when a user process calls a system, when they are waiting for the arrival of data, the processing method is different. they directly wait, round robin, select, or poll for polling, the first process is blocked, some are not blocked, and some can be blocked and not blocked. At that time, the second process was congested. From the perspective of the entire I/O process, they are executed sequentially, so they can be classified as asynchronous ). All processes are actively checking the kernel.

Asynchronous I/O (asynchronous I/O)
Compared with synchronous I/O, asynchronous I/O is not executed sequentially. After a user process calls the aio_read system, it will directly return to the user process regardless of whether the kernel data is ready or not. then, the user-mode process can do other things. When the socket data is ready, the kernel directly copies the data to the process and sends a notification to the process from the kernel. In the I/O phase, both processes are non-blocking.

The method is different from the previous method. this time we hired a phishing guru. He will not only fish, but also send us a text message after the fish are hooked to notify us that the fish is ready. We only need to entrust him to give a shot, and then we can continue to do other things until his text message. Let's go back and process the fish that have already landed.

Differences between synchronous and asynchronous
Through the discussion of the above models, we need to distinguish between blocking and non-blocking, synchronous and asynchronous. They are actually two groups of concepts. It is easier to distinguish the previous group, and the latter is often easier to mix with the previous one. In my opinion, the so-called synchronization is in the whole I/O process. In particular, the process of copying data blocks the process, and the kernel state is checked by the application process state. Asynchronous mode means that the user process in the whole process I/O process is not blocked, and when the data is copied, the kernel sends a notification to the user process.

For the synchronization model, the first-stage processing method is different. The asynchronous model is different in both stages. The signal driving mode is ignored here. These terms are still confusing. blocking and non-blocking are considered only for the synchronous model, because asynchronization is definitely not blocking, and the asynchronous non-blocking statement is superfluous.

Select Model
In the synchronization model, multiplexing I/O can improve the server performance.
The select Model and poll model are commonly used in multiplexing models. Both are system interfaces provided by the operating system. Of course, the select module of Python is more advanced encapsulation. The underlying principles of select and poll are similar. This article focuses on the select Model.
1. select principle
Network communication is abstracted as file read/write by Unix systems. it is usually a device provided by the device driver to know whether its data is available. Device drivers that support blocking operations usually implement a group of their own waiting queues. for example, read/write waiting queues are used to support block or non-block operations required by the upper layer (user layer. If the file resources of the device are available (readable or writable), the process will be notified. Otherwise, the process will sleep and wake up when data is available.

The file descriptors of these devices are put in an array, and then traverse the array when the select statement is called. if the file descriptor is readable, the file descriptor is returned. After the traversal is complete, if there is still no available device file descriptor, select will let the user process sleep until the system wakes up when the resource is available and traverses the previously monitored array. Each traversal is linear.

2. select echo server
The select statement involves the knowledge of system calls and operating systems. Therefore, it is boring to simply understand the principle. It is better to use code to demonstrate it. Using the select module of python, you can easily write the following echo server:

Import selectimport socketimport sysHOST = 'localhost' PORT = 5000BUFFER_SIZE = 1024 server = socket. socket (socket. AF_INET, socket. SOCK_STREAM) server. bind (HOST, PORT) server. listen (5) inputs = [server, sys. stdin] running = Truewhile True: try: # Call the select function to block waiting for readable, writeable, predictional = select. select (inputs, [], []) limit T select. error, e: break # data arrival, loop for sock in readable: # establish a connection if sock = server: conn, addr = server. accept () # socket inputs monitored by select. append (conn) elif sock = sys. stdin: junk = sys. stdin. readlines () running = False else: try: # read the data sent by the client connection = sock. recv (BUFFER_SIZE) if data: sock. send (data) if data. endswith ('\ r \ n \ r \ n'): # Remove the socket inputs of the select listener. remove (sock) sock. close () else: # Remove socket inputs from the select listener. remove (sock) sock. close () handle T socket. error, e: inputs. remove (sock) server. close ()

Run the preceding code and use curl to access http: // localhost: 5000. you can view the HTTP request information returned by the command line.

The following describes the principles of the above code.

server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)server.bind((HOST, PORT))server.listen(5)

The above code uses socket to initialize a TCP socket, bind the host address and port, and then set the server listener.

inputs = [server, sys.stdin]

A list of select listeners is defined here. the list contains the objects to be listened to (equivalent to the file descriptor listened to by the system ). Listen for socket and user input.

Then the code performs a wireless loop on the server.

Try: # Call the select function to block waiting for readable, writeable, predictional = select. select (inputs, [], []) blocks T select. error, e: break

The select function is called to start looping through the inputs list passed in by the listener. If there is no curl server, no tcp client connection is established at this time, so the objects in the list are all data resources unavailable. So select blocking does not return.

After the client inputs curl http: // localhost: 5000, a socket communication starts. in this case, the first object server in the input is changed from unavailable to available. Therefore, when the select function is called, the readable has a socket object (the file descriptor is readable ).

For sock in readable: # establish a connection if sock = server: conn, addr = server. accept () # socket inputs. append (conn) of the select listener)

After the select statement is returned, the system then traverses the readable object. at this time, only one socket connection exists in the readable object, and calls the socket's accept () method to establish a TCP three-way handshake connection, append the connection object to the inputs monitoring list, indicating that we want to monitor whether the connection has data IO operations.

At this time, because readable only has one available object, the traversal ends. Return to the main loop and call select again. during the call, the system not only checks whether a new connection needs to be established, but also monitors the newly added connection. If the curl data arrives, select returns to readable, and then performs a for loop. If there is no new socket, the following code will be executed:

Try: # read the data sent by the client connection = sock. recv (BUFFER_SIZE) if data: sock. send (data) if data. endswith ('\ r \ n \ r \ n'): # Remove the socket inputs of the select listener. remove (sock) sock. close () else: # Remove socket inputs from the select listener. remove (sock) sock. close () handle T socket. error, e: inputs. remove (sock)

Call the recv function through a socket connection to obtain the data sent by the client. after the data transmission is complete, remove the connection from the monitored inputs list. Close the connection.

The entire network interaction process is like this. of course, if the user inputs an interrupt in the command line, the sys. stdin monitored in the inputs list will also let the select return, and finally the following code will be executed:

elif sock == sys.stdin:  junk = sys.stdin.readlines()  running = False

Some people may wonder what will happen if a program processes a sock connection and enters a curl request to the server? At this time, there is no doubt that the server socket in inputs will become available. After the current for loop processing is complete, the select call will return the server. If the inputs contains the conn connection of the previous process, it will traverse the inputs cyclically and monitor the new socket accept to the inputs list again, then, the conn connection is processed cyclically. So methodically, until the for loop ends, the main loop calls select.

At any time, the object monitored by inputs has data. when the select statement is called next time, the system returns the readable variable. if the result is returned, the system performs a for loop on the readable, the next select operation is performed until the for loop ends.

It is important to note that the connection established by the socket is an IO, and the arrival of the connected data is also an IO.

3. limitations of select
Although select is quite easy to use, it is a cross-platform feature. However, select still has some problems.
Select needs to traverse the monitored file descriptor, and the array of this descriptor has the maximum limit. As the number of file descriptors increases, the overhead caused by the replication of the user state and kernel address space also increases linearly. Even if the monitored file descriptor is inactive for a long time, the select statement performs linear scanning.

To solve these problems, the operating system provides the poll solution. However, the poll model is roughly the same as the select model, but only changes some restrictions. Currently, the epoll model is the most advanced method in Linux.

Many high-performance software such as nginx and nodejs are asynchronous based on epoll.