Python Concurrent Programming---co-process

1. What is a co-process

Co-process: is a single-threaded concurrency, also known as micro-threading, fiber.

The process is a kind of user-state lightweight thread, and the process is controlled by the user program itself.

2. Points to note:

It should be emphasized that: # 1. Python threads are at the kernel level, that is, scheduled by the operating system control (such as single-threaded encounters IO or too long execution time will be forced to hand over the CPU execution permissions, switch other threads run)#2. Single-wire range opening process, once IO is encountered, The switch is controlled from the application level (not the operating system) to increase efficiency (!!!) Non-IO switching vs. efficiency) compared to the operating system control thread switching, the user in the single-threaded control of the process of the switch advantages are as follows:#1. The transition cost of the association is smaller, it belongs to the program level switch, the operating system is completely unaware, Thus more lightweight #2. Concurrency can be achieved within a single thread, maximizing CPU disadvantage as follows:#1. The nature of the process is single-threaded, unable to take advantage of multicore , can be a program to open more than one process, each process to open multiple threads, each line range to open the coprocessor #2. The association refers to a single thread, so once the association is blocked, it will block the entire thread

3. The characteristics of the co-process are as follows:

(1) Concurrency must be implemented in only one single thread

(2) Modify the shared data without the need to lock

(3) The context stack in the user program to save multiple control flows

(4) Attach: A co-process encountered IO operation automatically switch to other co-process (how to implement detection Io,yield, Greenlet can not be achieved, the use of the Gevent module (select mechanism))

4.Greenlet

If we had 20 tasks within a single thread, it would be cumbersome to use the yield generator in order to switch between multiple tasks (we need to get the generator initialized once before calling send ...). Very cumbersome), while using the Greenlet module can be very simple to achieve these 20 tasks directly switching

# installing PIP3 Install Greenlet

 fromGreenletImportGreenletdefEat (name):Print('%s Eat 1'%name) G2.switch ('Egon')    Print('%s Eat 2'%name) G2.switch ()defPlay (name):Print('%s Play 1'%name) G1.switch ()Print('%s Play 2'%name) G1=Greenlet (Eat) G2=Greenlet (play) G1.switch ('Egon')#parameters can be passed in at the first switch, and will not be required at any later time

Greenlet just provides a more convenient way to switch than generator, when cutting to a task execution if you encounter Io, then blocking in place, still does not solve the problem of automatically switching to the IO to improve efficiency.

The code for these 20 tasks for single-line thread usually has both a computational and a blocking operation, and we can get stuck in the execution of Task 1 o'clock, using blocking time to perform task 2 .... In order to improve efficiency, this will use the Gevent model.

5.Gevent Introduction

# installing PIP3 Install gevent

Gevent is a third-party library that makes it easy to implement concurrent or asynchronous programming through Gevent, and the main pattern used in Gevent is Greenlet, which is a lightweight coprocessor that accesses Python in the form of a C extension module. Greenlet all run inside the main program operating system process, but they are dispatched in a collaborative manner.

# usage g1=gevent.spawn (func,1,,2,3,x=4,y=5) creates a co-object g1,spawn the first argument in parentheses is the function name, such as Eat, which can be followed by multiple arguments, which can be positional arguments or keyword arguments. Are all passed to the function eat g2=# wait for the end   of the G1 # wait for the G2 to end #  or two-step cooperation step: Gevent.joinall ([g1,g2])g1.value# get func1 return value

To use gevent, you need to place the from Gevent import Monkey;monkey.patch_all () to the beginning of the file

 fromGeventImportMonkey;monkey.patch_all ()ImportgeventImport Timedefeat ():Print('Eat food 1') Time.sleep (2)    Print('Eat Food 2')defPlay ():Print('Play 1') Time.sleep (1)    Print('Play 2') G1=Gevent.spawn (Eat) G2=gevent.spawn (Play_phone) Gevent.joinall ([g1,g2])Print('Master')

Python Concurrent Programming---co-process