Python 3.x Study Notes 17 (coroutine and I/O mode), python3.x

Python 3.x Study Notes 17 (coroutine and I/O mode), python3.x

1. coroutine (microthread)
Coroutine is a lightweight thread in user mode.
Coroutine has its own register context and stack. During the coroutine scheduling switch, the register context and stack are saved to other places. When the switch is back, the previously saved register context and stack are restored. Therefore:

The association can retain the status of the last call (that is, a specific combination of all local States). When each process is re-accessed, it is equivalent to entering the status of the last call. In other words: the location of the logical stream at the last exit.

2. greenlet Module
Greenlet is a coroutine module implemented in C. Compared with the yield provided by python, it allows you to switch between any functions without the need to declare this function as a generator.

Example

From greenlet import greenletdef fun1 (): print (6) gar2.switch () # convert to gar2 print (58) def fun2 (): print (54) gar1.switch () gar1 = greenlet (fun1) # Start coroutine gar2 = greenlet (fun2) gar1.switch ()

 

3. gevent Module
Gevent is a third-party library that can easily implement concurrent synchronization or asynchronous programming through gevent. Greenlet is the main mode used in gevent. It is a lightweight coroutine that connects to Python in the form of a C extension module. Greenlet runs all inside the operating system processes of the main program, but they are collaboratively scheduled.

Import geventdef fun1 (): print ('first run fun1') gevent. sleep (2) # Switch to the gevent of fun2. sleep (1) print ('second run fun1') def fun2 (): print ('first run fun2') gevent. sleep (1) # continue to switch to the gevent of fun3. sleep (2) print ('second run fun2') def fun3 (): print ('first run fun3') gevent. sleep (2) print ('second run fun3') gevent. joinall ([gevent. spawn (fun1), gevent. spawn (fun2), gevent. spawn (fun3),])

Result

First Run fun1 first run fun2 first run fun3 second run fun2 second run fun1 second run fun3

 

4. gevent cannot detect urllib I/o operations by default.

 

5. To perform asynchronous crawler operations, you must add monkey. patch_all (), which means to mark all io operations of the current program separately.

For example

From urllib import requestimport gevent, timefrom gevent import monkeymonkey. patch_all () # mark all io operations of the current program as def f (url): print ('get % s' % url) resp = request. urlopen (url) data = resp. read () print ('% d data is received from % s. '% (len (data), url) start_time = time. time () gevent. joinall ([gevent. spawn (f, 'https: // www.python.org/'), gevent. spawn (f, 'https: // www.baidu.com/'), gevent. spawn (f, 'https: // github.com/'),]) print ('total time:', time. time ()-start_time)

 

6. event-driven model
Currently, most of the UI programming models are event-driven models. For example, many UI Platforms provide onClick () events, which represent mouse-pressed events. The general idea of the event-driven model is as follows:
1). There is an event (Message) queue;
2. When you press the mouse, add a click event (Message) to the queue );
3) There is a loop in which events are constantly retrieved from the queue. different functions, such as onClick () and onKeyDown (), are called based on different events;
4) Events (messages) generally store their respective processing function pointers. In this way, each message has an independent processing function;

7. event-driven programming is a programming paradigm where the execution flow of programs is determined by external events. It is characterized by an event loop. When an external event occurs, the callback mechanism is used to trigger corresponding processing. The other two common programming paradigms are (single-threaded) synchronization and multi-threaded programming.

8. cache I/O

Cache I/O is also referred to as standard I/O. Most file systems use cache I/O by default. In Linux's cache I/O mechanism, the operating system will cache the I/O data in the file system's page cache, that is, the data is first copied to the buffer zone of the operating system kernel before it is copied to the address space of the application from the buffer zone of the operating system kernel.

Cache I/ODisadvantages:
During data transmission, you need to perform multiple data copy operations in the application address space and kernel. the CPU and memory overhead caused by these data copy operations are very large.

Note: This cache I/O I/O in linux
Details: http://www.cnblogs.com/alex3714/articles/5876749.html

9. IO Mode

Block I/O (blocking IO)
Non-blocking I/O (nonblocking IO)
I/O multiplexing (IO multiplexing)
Signal-driven I/O (signal driven IO)
Asynchronous I/O (asynchronous IO)