Python coprocessor (asynchronous invocation in single thread) (Liaoche teacher Python tutorial)

Co-process, also known as micro-threading, fiber. English name Coroutine.

The concept of the co-process was raised early, but it has not been widely used in some languages (such as LUA) in recent years.

subroutines, or functions, are hierarchical calls in all languages, such as a call to b,b in the execution of a call to C,c execution complete return, b execution is finished, and finally a executes.

So the subroutine call is implemented through the stack, and a thread is executing a subroutine.

Subroutine calls are always one entry, one return, and the call order is clear. And the call of the association and subroutine is different.

The process appears to be a subroutine, but during execution, it can be interrupted inside the subroutine, then execute another subroutine, and then return to execute at the appropriate time.

Note that interrupts in one subroutine, to execute other subroutines, are not function calls, a bit like CPU interrupts. such as subroutines A, B:

def A():    print ‘1‘ print ‘2‘ print ‘3‘def B(): print ‘x‘ print ‘y‘ print ‘z‘

Assuming that the process is executed by the coprocessor, it can be interrupted at any time during the execution of a, and the execution of b,b may be interrupted during execution to execute a, and the result may be:

12xy3z

But in a there is no call to B, so the call of the association is more difficult to understand than the function call.

It seems that a, B's execution is a bit like multi-threading, but the process is characterized by a thread execution, and multithreading ratio, what is the advantage of the co-process?

The greatest advantage is the high execution efficiency of the process. Because the subroutine switch is not a thread switch, but is controlled by the program itself, therefore, there is no thread switching overhead, and multithreading ratio, the more the number of threads, the performance advantage of the association is more obvious.

The second big advantage is that there is no need for a multi-threaded lock mechanism, because there is only one thread, there is no simultaneous write variable conflict, in the process of controlling shared resources without locking, only need to judge the state is good, so the execution efficiency is much higher than multithreading.

Because the co-process is a thread execution, how do you take advantage of multicore CPUs? The simplest method is multi-process + co-progression, which takes full advantage of multicore, and maximizes the high-efficiency of the co-processes, which can achieve very good performance.

Python's support for the process is very limited, and yield in generator can be implemented to some extent. Although the support is not complete, but already can play the quite big power. (gevent)

See Example:

The traditional producer-consumer model is a thread-write message, a thread that takes messages, controls the queue through a lock mechanism, and waits, but can be accidentally deadlocked.

If the use of the process, the producer after the production of the message, directly through the yield jump to the consumer to start execution, after the consumer execution, switch back to the producer to continue production, efficiency is very high:

Import timeDefConsumer (): R =  "while true:n = span class= "keyword" >yield r if not N: return print ( [CONSUMER] consuming%s ... '% n) time.sleep (1) R = def produce (c): C.next () n = 0 while n < 5:N = n + 1 Print (" [PRODUCER] producing%s ... '% n) r = c.send (n) print ( "[PRODUCER] Consumer return:%s '% r) c.close () if __name__== ' __main__ ': c = Consumer () Produce (c)               

Execution Result:

[PRODUCER]Producing 1 ...[CONSUMER]Consuming 1 ...[PRODUCER]Consumerreturn:200Ok[PRODUCER]Producing 2 ...[CONSUMER]Consuming 2 ...[PRODUCER]Consumerreturn:200ok[producer] producing 3... [CONSUMER] consuming 3 ... [producer] consumer return:200  Ok[producer] producing 4... [CONSUMER] consuming 4 ... [producer] consumer return:200  Ok[producer] producing 5... [CONSUMER] consuming 5 ... [producer] consumer return:200  OK                 

Note that the consumer function is a generator (generator) that puts a consumer into the produce after:

1. First call C.next () to start the generator;

2. Then, once the thing is produced, switch to consumer execution by C.send (n);

3. Consumer through yield to the message, processing, and through yield to return the results;

4. Produce get consumer processing results, continue to produce the next message;

5. Produce decided not to produce, through C.close () Close the consumer, the entire process ended.

The entire process is unlocked, executed by one thread, and produce and consumer collaborate to complete the task, so called "co-process", rather than the thread's preemptive multi-tasking.

Finally, apply Donald Knuth's sentence to summarize the features of the process:

"Subroutine is a special case of the process. ”

Python coprocessor (asynchronous invocation in single thread) (Liaoche teacher Python tutorial)