Basic Primer _python-network programming. minutes to master blocking/non-blocking/synchronous/asynchronous io models?

Concept Grooming:

Story monologue: full of love tea, nonsense not much to say, began to boil boiling water.

Characters: full, ordinary kettle, Superior Kettle (water meeting Ring)

1. Put the kettle on the fire, stand there and wait for the water to open (synchronous Blocking)

I feel a little bit silly

2. Put the kettle on the fire, go to the living room to watch tv, and occasionally go to the kitchen to see the water open wood has (synchronous Non-blocking)

Feel full of oneself still a little silly ~, so bought a high-class kettle, water will ring after open ~

3. Put the kettle on the fire and stand there waiting for the water to open (asynchronous Blocking)

Full of high-grade kettle water to meet their own call ~ for hair not to see a tv?

4. Put the high water bottle on the fire, go to the living room to watch tv, no longer go to see the kettle ring, and then go to get the kettle (asynchronous Non-blocking)

And suddenly I feel so smart

Summary: a small story in detail describes the synchronous blocking/synchronous nonblocking/asynchronous blocking/asynchronous non-blocking scenarios, and finally said that a bit of blocking/non-blocking is relative to full, and synchronous/asynchronous is relative to the kettle, so the former is more inclined to the program itself, and the latter more in favor of the message notification mechanism

Common Models:

Description: The essence of network I/O is socket read, socket is abstracted as stream under Linux system, I/O is the operation of convection, but the operation is divided into two stages: waiting for stream data preparation phase and copying data from kernel to Process. In fact, the whole process is that the data on the network is copied to the kernel buffer when it arrives, and then the data is copied from the kernel buffer to the user space application process buffer

Synchronous blocking I/O

650) this.width=650; "src=" http://s5.51cto.com/wyfs02/M00/88/A5/wKioL1f-PNmR0fANAAFL-hoaIuM508.png "style=" float: none; "title=" 1.png "alt=" wkiol1f-pnmr0fanaafl-hoaium508.png "/>

Description: the application calls an IO function that causes the application to block until the data is prepared and copied to the User-space process buffer, and the IO function returns a success indicator, often encountering an input operation (recv/recvfrom) when no data is readable in the Buffer. The calling thread is dormant until the data arrives, and the output operation (send/sendto) is dormant when there is no free space in the buffer, accepting the connection (accept) the thread has been dormant while there is no connection request, outgoing connection (connect) The Front-thread has been dormant while receiving the server response

Problem: developing a network program with blocking sockets is simple and suitable for scenarios where you want to be able to understand sending accepted data and handle a small number of sockets, when you want to handle a large number of sockets at the same time ~

Synchronous non-blocking I/O

650) this.width=650; "src=" http://s5.51cto.com/wyfs02/M02/88/A5/wKioL1f-PN-ysOKbAAI85mpH2dA313.png "style=" float: none; "title=" 2.png "alt=" wkiol1f-pn-ysokbaai85mph2da313.png "/>

Description: the application repeatedly calls the IO function, and immediately return the error, the kernel data is not ready, not let the thread sleep, and let the function immediately return an error, so the data is not ready for the front-end process is non-blocking, but from the moment the data is ready, the thread is blocked until the data copy is Completed.

Problem: using non-blocking sockets, when the data received and received uneven, time-varying scenarios do have advantages, but due to periodic polling to check the kernel data is ready, very wasteful system resources, and need to properly handle kernel errors

Synchronous multiplexing I/O

650) this.width=650; "src=" http://s2.51cto.com/wyfs02/M00/88/A8/wKiom1f-POWy1rZmAAIMDkRrhTI896.png "style=" float: none; "title=" 3.png "alt=" wkiom1f-powy1rzmaaimdkrrhti896.png "/>

Description: the application listens to the kernel data on multiple blocking sockets through Kernel-level Select/poll to be ready, and any one of the socket data is ready to be returned, and the process calls the Recvfrom system call to copy the data from the kernel buffer to the User-space application Buffer. Say one more thing in the process of data copying is blocking state

Asynchronous I/O

650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M00/88/A5/wKioL1f-POqhOGMZAAGa3apTclA514.png "style=" float: none; "title=" 4.png "alt=" wkiol1f-poqhogmzaaga3aptcla514.png "/>

Description: the application does not immediately get results after the call, the actual processing of the part of the call after completion, through the state, notification and callback to notify the caller results, the entire process, including waiting for data and copy data are non-blocking

Extended Knowledge:

1. Select essence is to determine whether the FD corresponding kernel data is ready by checking the data structure of the FD flag bit, and the default FD storage mode results in a limited number of FD quantities that can be monitored by a single process, and the size of the port that can listen is also limited, generally and memory close is larger/proc/sys/fs/ file-max, and the FD scan is a line-scan, regardless of whether there is data are scanned, wasting CPU time, poll and select similar, but because of the linked list to store FD so there is no limit to the maximum number of connections. but they have a common flaw, A large number of FD data structures are repeatedly passed along with the user space and kernel space.

2. Epoll essentially solves the Select senseless line scan, which only tells the data to be ready for fd, and there is basically no limit to the maximum concurrent connections (1G memory can listen on 100,000 ports), and the memory copy uses MMAP to reduce replication overhead

This article is from the "ζ automated operation and maintenance Development Road ζ" blog, Please be sure to keep this source http://xmdevops.blog.51cto.com/11144840/1861280

Basic Primer _python-network programming. minutes to master blocking/non-blocking/synchronous/asynchronous io models?