Python io multiplexing

Co-process: switch on IO operation.
But when do you cut it back? How do I know I'm done with IO?

Many programmers might consider using a "thread pool" or "Connection pool." The thread pool is designed to reduce the frequency of creating and destroying threads, maintaining a reasonable number of threads, and allowing idle threads to re-assume new execution tasks. Connection pooling maintains a connected cache pool, reusing existing connections as much as possible, and reducing the frequency with which connections are created and closed.

Both of these technologies can reduce system overhead and are widely used in many large systems, such as WebSphere, Tomcat, and various databases. However, the thread pool and connection pooling techniques are only to some extent mitigated by the frequent invocation of the IO interface for resource consumption. Moreover, the so-called "pool" always has its upper limit, when the request greatly exceeds the upper limit, the "pool" composed of the system response to the outside world is not much better than when there is no pool. So using the pool must consider the scale of the response it faces and adjust the size of the pool based on the response scale.
The "thread pool" or "Connection pool" may alleviate some of the stress, but not all of them, in response to the thousands or even thousands of client requests that may appear in the previous example. In short, multithreaded models can easily and efficiently solve small-scale service requests, but in the face of large-scale service requests, multithreading model will encounter bottlenecks, you can use non-blocking interface to try to solve the problem

first, the event-driven model introduction

A. Linear mode

The traditional programming is in the following linear mode:

Start---> code block a---> code block b---> block c---> Block d---...---> End

Each code block is a code that accomplishes a variety of things, but the programmer knows the code block a,b,c,d ... Order of execution, the only thing that can change this process is data. Enter different data, judging by the conditional statement, the process may be changed to a--->c--->e ...---> End. Each time the program may run in a different order, but its control flow is determined by the input data and the program you are writing. If you know the current running state of the program (including the input data and the program itself), then you know that the next step is even until the end of its running process.

For the event-driven program model, the process is roughly as follows:

Start---> Initialize---> Wait

Unlike the traditional programming model above, the event driver is waiting there after startup, waiting for what? Wait for the event to be triggered. Traditional programming also has a "wait" time, such as in code block D, you define an input (), the user needs to enter data. But this is different from the wait for the following, traditional programming "Wait", such as input (), you as a program writer know or force the user to enter something, perhaps the number, perhaps the file name, if the user input error, you need to remind him, and ask him to re-enter. The wait for the event driver is completely unknown and does not force the user to enter or do anything. As long as an event occurs, the program will respond accordingly. These events include input information, a mouse, a key on the keyboard, and a system internal timer trigger.

B. Event-driven model

Typically, when writing a program that the server processes the model, there are several models:

(1) Each request is received, a new process is created to process the request, and (2) Each request is received, a new thread is created to process the request, and (3) Each request is received, a list of events is placed, allowing the main process to process the request in a non-blocking I/O manner

The third is the co-process, event-driven approach, which is generally considered to be (3) the way most Web servers are used

Event-driven model mouse click events

In UI programming, it is often necessary to click on the mouse to the corresponding, first how to get a mouse click?

Two different ways:

1. Create a thread loop to detect if there is a mouse click

Then there are several drawbacks to this approach:

1. CPU resources wasted, may be mouse click frequency is very small, but the scan thread will still be cyclic detection, which will cause a lot of CPU waste, if the scan mouse click on the interface is blocked?
2. If it is blocked, and will appear the following problems, if we not only to scan the mouse click, but also to scan the keyboard is pressed, because scanning the mouse is blocked, then may never go to scan the keyboard;
3. If a loop needs to scan a lot of devices, this will lead to response time problems;
   So, the way is very bad.

2, is the event-driven model

Most of the current UI programming is an event-driven model, as many UI platforms provide the OnClick () event, which represents the mouse down event. The event-driven model is broadly thought of as follows:

1. There is an event (message) queue;
2. When the mouse is pressed, add a click event (message) to this queue;
3. There is a loop that constantly takes events out of the queue, and calls different functions, such as onclick (), OnKeyDown (), according to different events;
4. Events (messages) typically hold their own handler pointers, so that each message has its own handler function;

The above-mentioned event-driven model, then what is the event-driven model

Event-driven programming is a programming paradigm where the execution flow of a program is determined by an external event. It is characterized by the inclusion of an event loop that uses a callback mechanism to trigger the corresponding processing when an external event occurs . Two other common programming paradigms are (single-threaded) synchronization and multithreaded programming.

1. Let's use examples to compare and contrast single-threaded, multithreaded, and event-driven programming models. Shows the work done by the programs in these three modes over time. This program has 3 tasks to complete, each of which blocks itself while waiting for an I/O operation. The time it takes to block the I/O operation has been marked with a gray box.

The initial problem: How to determine the IO operation is finished cutting back? through the callback function

1. To understand event drivers and programs, you need to compare them with non-event-driven programs. In fact, modern programs are mostly event-driven, such as multithreaded programs, which are definitely event-driven. In the early days there are many non-event-driven programs that, when waiting for a certain condition to trigger, constantly check the condition until the condition is met, which is a waste of CPU time. Event-driven programs have the opportunity to release the CPU to sleep (note that there is a chance, of course, that the program can decide not to release the CPU), and when the event is triggered by the operating system, it is more efficient to use cpu.2. What is an event-driven program? A typical event-driven program is a dead loop that exists as a thread, the dead loop consists of two parts, the first part is receiving and selecting an event to be handled according to certain conditions, and the second part is the process of handling the event. The execution of the program is to select events and handle events, and when there is no event triggered, the program will go to sleep because the query event queue failure, thereby freeing the CPU. 3. Event-driven programs must have an event queue directly or indirectly to store events that are not handled in a timely manner. 4. Event-driven program behavior is completely controlled by external input events, so there is a large number of such programs in the event-driven system, with events as the primary means of communication. 5. One of the biggest benefits of event-driven programs is the ability to process events in the queue in a certain order, which is determined by the order in which events are triggered, which is often used to ensure that certain processes are atomized. 6. Currently Windows,linux,nucleus,vxworks are event-driven, and only some microcontroller may be non-event-driven.

Note that event-driven listener events are done by the CPU that is called by the operating system

Second, IO model

With the implementation of the IO blocking automatic switching, then how the association process is implemented, in the principle is how to achieve. How to achieve the event-driven situation of IO automatic blocking switch, what is the name of this study? = = IO multiplexing
For example, Socketserver, multiple client connections, single-threaded implementation of the concurrency effect, called multiplexing.

The IO model is divided into: blocking IO, non-blocking IO, synchronous io, asynchronous IO How are they defined, and what is the difference between them?

Before explaining, declare some concepts:

• User space and kernel space
• Process switching
• Blocking of processes
• File descriptor
• Cache I/O

User space and kernel space

Now that the operating system is using virtual memory, the 32-bit operating system, its addressing space (virtual storage space) is 4G (2 of 32).
The core of the operating system is the kernel, which is independent of the normal application, has access to protected memory space, and has all the permissions to access the underlying hardware device.
In order to ensure that the user process can not directly manipulate the kernel (kernel), to ensure the security of the kernel, worry about the system to divide the virtual space into two parts, part of the kernel space, part of the user space.
For the Linux operating system, the highest 1G bytes (from the virtual address 0xc0000000 to 0xFFFFFFFF) for the kernel to use, called the kernel space, and the lower 3G bytes (from the virtual address 0x00000000 to 0xBFFFFFFF) for each process to use, Called User space.

Process switching

To control the execution of a process, the kernel must have the ability to suspend a process that is running on the CPU and resume execution of a previously suspended process. This behavior is referred to as process switching, which is done by the operating system. So it can be said that any process that runs under the support of the operating system kernel is closely related to the kernel.
The process of moving from one process to another runs through the following changes:

Save the processor context, including program counters and other registers.

Update PCB information.

The PCB of the process is moved into the appropriate queue, such as ready, in an event blocking queue.

Select another process to execute and update its PCB.

Update the data structure of memory management.

Restore the processing machine context.
Note: In short, it is very resource-intensive

Blocking of processes

The executing process, because some expected events did not occur, such as requesting system resources failed, waiting for the completion of an operation, new data has not arrived or no new work to do, etc., the system automatically executes the blocking primitive (block), making itself from the running state into a blocking state. It can be seen that the blocking of a process is an active behavior of the process itself, and therefore it is possible to turn it into a blocking state only if the process is in a running state (acquiring the CPU). When a process goes into a blocking state, it does not consume CPU resources.

File descriptor

File descriptor, a term in computer science, is an abstraction that describes a reference to a file.
The file descriptor is formally a non-negative integer. In fact, it is an index value that points to the record table in which the kernel opens a file for each process maintained by the process. When a program opens an existing file or creates a new file, the kernel returns a file descriptor to the process. In programming, some of the underlying programming often revolves around file descriptors. However, the concept of file descriptors is often applied only to operating systems such as UNIX and Linux.

Cache I/O

Cache I/O is also known as standard I/O, and most file system default I/O operations are cache I/O. In the Linux cache I/O mechanism, the operating system caches the I/O data in the file system's page cache, which means that the data is copied into the buffer of the operating system kernel before it is copied from the operating system kernel buffer to the application's address space. User space is not able to directly access the kernel space, the kernel state to the user state of the data copy

Think: Why data must first go to the kernel area, directly to the user's memory is not more direct?
Disadvantages of Cache I/O:

Data is required to perform multiple copies of data in the application address space and the kernel during transmission, and the CPU and memory overhead of these data copy operations is very large.

What is the difference between synchronous (synchronous) IO and asynchronous (asynchronous) Io, what is blocking (blocking) IO and non-blocking (non-blocking) IO respectively? The problem is that different people may give different answers.

Since signal driven IO (signal driven IO model) is not commonly used in practice, only the remaining four IO models are mentioned.

Again, the objects and steps involved in the IO occur.
For a network IO (here we read for example), it involves two system objects, one that calls the IO process (or thread), and the other is the system kernel (kernel). When a read operation occurs, it goes through two stages:
1 Waiting for data preparation (waiting for the
2 copying data from the kernel to the process (Copying the data from the kernel to the)
It is important to remember these two points because the difference between these IO model is that there are different situations in both phases.

Blocking io (blocking IO)

In Linux, all sockets are blocking by default, and a typical read operation flow is probably this:

Non-blocking io (non-blocking IO)

Under Linux, you can make it non-blocking by setting the socket. When you perform a read operation on a non-blocking socket, the process looks like this:

As you can see, when the user process issues a read operation, if the data in kernel is not ready, it does not block the user process, but returns an error immediately. From the user process point of view, it initiates a read operation and does not need to wait, but immediately gets a result. When the user process determines that the result is an error, it knows that the data is not ready, so it can send the read operation again. Once the data in the kernel is ready and again receives the system call of the user process, it immediately copies the data to the user's memory and then returns.
Therefore, the user process is in fact need to constantly actively ask kernel data well no.

Attention:

In the network IO, the non-blocking IO will also make recvform system calls, check whether the data is ready, and blocking IO is not the same, "non-blocking the large block of time is divided into more than n small blocking, so the process has a chance to ' be ' CPU patronage." That is, each time the Recvform system calls, the CPU's permissions are still in the hands of the process, this time can do other things.

That is, after a non-blocking recvform system call, the process is not blocked, the kernel is returned to the process immediately, and if the data is not ready, an error is returned. After the process returns, it can do something else before initiating the recvform system call. Repeat the above process and iterate through the recvform system calls. This process is often called polling. Poll the kernel data until the data is ready, and then copy the data to the process for data processing. It is important to note that the process of copying data is still a blocking state.

IO multiplexing (IO multiplexing)

The word IO multiplexing may be a bit unfamiliar, but if I say select,epoll, I'll probably get it. Some places also call this IO mode for event driven IO. As we all know, the benefit of Select/epoll is that a single process can simultaneously handle multiple network connections of IO. The basic principle of the select/epoll is that the function will constantly poll all sockets that are responsible, and when a socket has data arrives, notifies the user of the process. It's process

When the user process invokes select, the entire process is blocked, and at the same time, kernel "monitors" all select-responsible sockets, and when the data in any one socket is ready, select returns. This time the user process then invokes the read operation, copying the data from the kernel to the user process.
This figure is not much different from the blocking IO diagram, in fact, it's even worse. Because two system calls (select and Recvfrom) are required, blocking IO only invokes one system call (Recvfrom). However, the advantage of using select is that it can handle multiple connection at the same time. (Say one more word.) Therefore, if the number of connections processed is not high, Web server using Select/epoll does not necessarily perform better than the Web server using multi-threading + blocking IO, and may be more delayed. The advantage of Select/epoll is not that a single connection can be processed faster, but that it can handle more connections. ）
In the IO multiplexing model, the actual, for each socket, is generally set to become non-blocking, but, as shown, the entire user's process is actually always block. Only the process is the block of the Select function, not the socket IO.

Note that if a file is readable in the return result of the 1:select function, the process can let kernel copy the data that is in the kernel to the user area by calling accept () or recv ().

Note The advantage of 2:select is that it can handle multiple connections, not a single connection

asynchronous I/O (asynchronous IO)

The asynchronous IO under Linux is actually used very little. Let's take a look at its process:

After the user process initiates the read operation, you can begin to do other things immediately. On the other hand, from the perspective of kernel, when it receives a asynchronous read, first it returns immediately, so no block is generated for the user process. Then, kernel waits for the data to be ready and then copies the data to the user's memory, and when all this is done, kernel sends a signal to the user process to tell it that the read operation is complete.

Four IO models have been introduced in a simple way

Review the above question difference call blocking IO will block the corresponding process until the operation is complete, and non-blocking IO will return immediately if the kernel is ready for the data.

Asynchronous IO is a model with no blocking, while synchronous IO has a blocking

Comparison of each IO model:

As described above, the difference between non-blocking io and asynchronous io is obvious. In non-blocking io, although the process will not be blocked for most of the time, it still requires the process to go to the active check, and when the data is ready, it is also necessary for the process to proactively call Recvfrom to copy the data to the user's memory. and asynchronous Io is completely different. It's like a user process handing over an entire IO operation to someone else (kernel) and then sending a signal notification when someone finishes it. During this time, the user process does not need to check the status of the IO operation, nor does it need to actively copy the data.

Comparison of five IO models:

Here, the different IO models are explained and differentiated, but they have some conceptual understanding of them, want to integrate them, but also need to deepen understanding in the future practice

The topic of this chapter is IO multiplexing, so now we're going to go into the chapter's content

Three, select poll Epoll io multiplexing Introduction

The details are open.

Select and Epoll

View Code

IO multiplexing Trigger mode

horizontal trigger and Edge trigger

Instance to attack ...

Instance One

non-blocking IO

Pros: Ability to do other work while waiting for the task to complete (including submitting other tasks, i.e. "backstage" can have multiple tasks at the same time).

Disadvantage: The response delay for task completion is increased because each time a read operation is polled, the task may be completed at any time between polling two times. This can result in a decrease in overall data throughput.

Example Two

Io multiplexing (multiplexed io):

In a non-blocking instance, the subject of the poll is a process, and "backstage" may have multiple tasks at the same time, people think of the loop to query the completion status of multiple tasks, as long as any one task is completed, to deal with it. However, the task of this listener is given to the kernel by invoking functions such as SELECT. IO multiplexing has two special system calls to select, poll, and Epoll functions. Select calls are kernel-level, and select polls are the difference between non-blocking polling-the former can wait for multiple sockets to be able to listen to multiple IO ports at the same time, and when any one of the sockets is ready, the data can be returned for readability. The process then makes the recvfrom system call, copying the data from the kernel to the user process, of course, the process is blocked.

Select multiplexed io

Question, why get rid of the accept will repeatedly fall into the dead loop select is the horizontal trigger

Example Three

Select to implement concurrent Chat

Server side Concurrent Chat

The file descriptor is actually the handle we normally call, except that the file descriptor is a concept in Linux. Note that the recvfrom request is made to the system when our accept or recv call

(1) If the kernel buffer does not have data---> wait for---> data to the kernel buffer, go to the user process buffer;

(2) If you first use Select to listen to a file descriptor corresponding to the kernel buffer has data, when we call accept or recv, the data is directly transferred to the user buffer.

Think 1: Open 5 clients, respectively, in 54321 order to send messages, then the server side is in what order to return the message?

For:......

Think 2: How to exit the client side without affecting the server side and other clients to communicate normally

A: After a client exits, set up an exception handler, catch the exception that the client exited, and delete the Select Listener's conn

code such as

Iv. Asynchronous IO

Asynchronous IO Example

V. Explanation of IO Programming

Io is the input/output in the computer, that is, input and output. Because the program and runtime data resides in memory and is executed by the CPU, an ultra-fast compute core, where the data is exchanged, usually a disk, a network, and so on, requires an IO interface. For example, you open a browser, visit the Sina homepage, the browser this program needs to get Sina's web page via network IO. The browser first sends the data to the Sina server, tells it I want the homepage HTML, this action is outward sends the data, is called the output, then the Sina server sends over the webpage, this action is receives the data from the outside, is called input. So, typically, the program completes IO operations with input and output two data streams. Of course there is only one case, for example, to read from disk to memory, only input operation, in turn, writes the data to the disk file, is just an output operation. In IO programming, stream (stream) is a very important concept that can imagine a stream as a water pipe, and the data is a pipe, but it can only flow in one direction. Input Stream is the data that flows from the outside (disk, network) into memory, and the Output stream is the data flowing from the memory to the outside. For web browsing, the browser and Sina server need to establish at least two water pipes, before they can both send data and collect data. Because the CPU and memory speed is much higher than the speed of the peripheral, so in IO programming, there is a serious problem of speed mismatch. For example, for example, to write 100M data to disk, the CPU output 100M of data only need 0.01 seconds, but the disk to receive this 100M data may take 10 seconds, how to do? There are two ways: the first is the CPU waiting, that is, the program suspends execution of subsequent code, such as 100M of data in 10 seconds after writing to disk, and then execute, this mode is called synchronous IO, the other way is the CPU does not wait, just tell the disk, "you old slowly write, not anxious, I went on to do something else" The subsequent code can then be executed immediately, which is called asynchronous IO. The difference between synchronous and asynchronous is whether to wait for the results of IO execution. Like you go to McDonald's order, you say "to a hamburger", waiter told you, sorry, hamburger to do now, need to wait 5 minutes, so you stand in front of the cashier waiting for 5 minutes, get hamburger and then go shopping mall, this is synchronous IO. You say "Come a hamburger", the waiter tells you, Hamburg needs to wait 5 minutes, you can go to the mall, wait for the good, we will inform you, so you can immediately do other things (shopping mall), this is asynchronous IO. Obviously, using asynchronous IO to write program performance is much higher than synchronous IO, but the disadvantage of asynchronous IO is the complexity of the programming model. Think about it, you need to know when to tell you that "burger is ready", and the method of informing you is different. If it's a waiter running over to find you, this is the callback mode,If the waiter sends you a text message, you'll have to keep checking your phone, which is the polling mode. In summary, the complexity of asynchronous IO is much higher than that of synchronous IO. The ability to operate IO is provided by the operating system, and each programming language encapsulates the low-level C interface provided by the operating system for ease of use, and Python is no exception. IO programming is synchronous mode, and asynchronous IO is too complex.

Python io multiplexing