Python I/O operations

Io is the input/output in the computer, that is, input and output. Because the program and runtime data resides in memory and is executed by the CPU, an ultra-fast compute core, where the data is exchanged, usually a disk, a network, and so on, requires an IO interface.

For example, you open a browser, visit the Sina homepage, the browser this program needs to get Sina's web page via network IO. The browser first sends the data to the Sina server, tells it I want the homepage HTML, this action is outward sends the data, is called the output, then the Sina server sends over the webpage, this action is receives the data from the outside, is called input. So, typically, the program completes IO operations with input and output two data streams. Of course there is only one case, for example, to read from disk to memory, only input operation, in turn, writes the data to the disk file, is just an output operation.

Event-driven models

Typically, there are several case models:

1. Each time a request is received, a new process is created to process the request.
2. Each time a request is received, a new thread is created to process the request.
3. Each time a request is received, a list of times is placed to allow the main process to process the request through non-blocking IO.

In general, the third approach is generally considered to be the way most Web servers are used.

For example, in UI programming, often use mouse click to operate, then how, when to get a mouse click to deal with it?

In the thread we learned earlier, we can create a thread to detect the mouse. So that's the problem?

1. CPU resources wasted, may be mouse click frequency is very small, but the scan thread will still be cyclic detection, which will cause a lot of CPU waste, if the scan mouse click on the interface is blocked?
2. If it is blocked, and will appear the following problems, if we not only to scan the mouse click, but also to scan the keyboard is pressed, because scanning the mouse is blocked, then may never go to scan the keyboard;
3. If a loop needs to scan a lot of devices, this will lead to response time problems;

So this way is not advisable

The second is the event-driven model just mentioned.

Most of the current UI programming is an event-driven model. As many UI platforms provide the OnClick () event, this event represents the mouse down event. The event-driven model is broadly thought of as follows:

1. There is a message queue
2. When you click on the mouse, add a click event to this queue.
3. There is a loop that constantly pulls events out of the queue and invokes different functions based on different events.
4. Events typically hold their own handler pointers, so that each message has a separate handler function.

A simple code example is shown below

<! DOCTYPE html>
Graphic explanation
Event-driven programming is a programming paradigm where the execution flow of a program is determined by an external event. It is characterized by the inclusion of an event loop that uses a callback mechanism to trigger the corresponding processing when an external event occurs. Two other common programming paradigms are (single-threaded) synchronization and multithreaded programming.
Use a simple illustration to illustrate the three relationships
The gray section of the figure indicates when the I/O operation was blocked
Synchronous IO and Asynchronous IO
CPU and memory speed is much higher than the speed of the peripheral, so in IO programming, there is a serious problem of speed mismatch. For example, for example, to write 100M data to disk, the CPU output 100M of data only need 0.01 seconds, but the disk to receive this 100M data may take 10 seconds, how to do? There are two ways to do this:

 
 

  
  
 
   
The first is the CPU waiting, that is, the program suspends execution of subsequent code, such as 100M of data in 10 seconds after writing to the disk, and then down execution, this mode is called synchronous IO;
 
   
Another way is that the CPU does not wait, just tell the disk, "you old slowly write, do not worry, I went on to do something else," then, the subsequent code can be executed immediately, this mode is called asynchronous IO.
 
  
 
 
The difference between synchronous and asynchronous is whether to wait for the results of IO execution. For instance.
Like you go to McDonald's order, you say "to a hamburger", waiter told you, sorry, hamburger to do now, need to wait 5 minutes, so you stand in front of the cashier waiting for 5 minutes, get hamburger and then go shopping mall, this is synchronous IO.
You say "Come a hamburger", the waiter tells you, Hamburg needs to wait 5 minutes, you can go to the mall, wait for the good, we will inform you, so you can immediately do other things (shopping mall), this is asynchronous IO.
Asynchronous IO Model
Asynchronous IO process without blocking state.
Note: Asynchronous IO Module 3.0 only, called Asyncio
Blocking io (blocking IO)
In the above example you go to McDonald's burger, but there is no, it takes 5 minutes to come out, and you bite to wait 5 minutes, the moment of 5 minutes wasted, this is the typical blocking IO. Probably the process can be represented as such
Non-blocking IO (non-blocking io)
If you do not want to wait there, go out and do other things, but you worry about the hamburger you ordered to be taken away, you come back and forth many times, finally finished. The approximate process is as follows
It is important to note that the process of copying data is still a blocked state
IO multiplexing (IO multiplexing)
The basic principle of the select/epoll is that the function will constantly poll all sockets that are responsible, and when a socket has data arrives, notifies the user of the process. As we order a good meal, do not have to ask the waiter every time, to see the special display can be known, so that each customer knows their waiting time. It's process
Note: When a user process invokes select, the entire process is blocked. There is not much difference between seemingly and blocked Io, but this can be monitored simultaneously to handle multiple connection. The entire user's process is actually always blocked. Only the process is the block of the Select function, not the socket IO.
In summary, several IO models are compared as follows:
IO multiplexing Select,poll,epoll
Select
The Select function monitors file descriptors in 3 categories, Writefds, Readfds, and Exceptfds, respectively. After the call, the Select function blocks until a descriptor is ready (readable, writable, or except), or the timeout (timeout wait time, such as immediate return is set to null), the function returns. When the Select function returns, you can find the ready descriptor by traversing Fdset.
The select advantage is that almost all platforms are supported.
The downside is

 
 

  
  
 
   
There is a maximum limit on the number of file descriptors that a single process can monitor, which defaults to 1024.
 
   
Every time the Select () is polled to traverse fd_setsize sockets to complete the dispatch, the efficiency is low.
 
   
There is a need to maintain a data structure that holds a large amount of FD, making the user space and kernel space more expensive to replicate during delivery.
 
  
 
 
Poll
Equivalent to the excessive between select and Epoll, the only change is that there is no limit to the number of file descriptors
Epoll (Linux-specific implementation method, currently not supported by Windows)
Epoll has two modes, the difference in when Epoll_wait detects that a descriptor event occurs and notifies the application of this event.

 
 

  
  
 
   
Lt Mode: The application can not handle the event immediately. The next time you call Epoll_wait, the application will respond again and notify you of this event.
 
   
ET mode: The application must handle the event immediately. If it is not processed, the next time you call Epoll_wait, the application will not respond again and notify this event. A non-blocking socket interface must be used
 
  
 
 
The last example of a concurrent chat is
ImportSocketImportSelectsk=socket.socket () Sk.bind ('127.0.0.1', 8080)) Sk.listen (3) Sk_list=[SK,] whiletrue:r_list, w_list, X_list=Select.select (Sk_list, [], [],) forIinchr_list:ifi = =Sk:conn, addr= I.accept ()#receive the first to come in is the SK, then began to execute Conn.sk_list.append (conn)elifi = =Conn:data= CONN.RECV (1024) Data= Data.decode ('UTF8')            Try:                #data is False                ifData is  notFalse:Print(data) INP= Input ("reply to client%s >>>>"%Sk_list.index (i)) Conn.sendall (Inp.encode ('UTF8'))            exceptException:#print (e)Sk_list.remove (i)
Server
ImportSocketsk=Socket.socket (socket.af_inet, socket. Sock_stream) Sk.connect (('127.0.0.1', 8080)) whileTRUE:INP= Input (">>>>") Sk.sendall (Inp.encode ('UTF8')) Data= SK.RECV (1024)    Print(Data.decode ('UTF8'))
Client
Python I/O operations