Python process in-depth understanding

Syntactically, the Cheng Builder is similar to a function that defines the yield keyword in the body.
The use of yield in a process:

• yield in the association usually appears to the right of the expression , for example: Datum = yield, can produce a value or not output-if there is no expression after the yield keyword, then the generator produces none.
• The process may accept data from the caller, and the caller is using Send (datum) to provide the data to the coprocessor instead of next (...). function, usually the caller pushes the value to the co-process.
• The co-process can give the controller to the Central scheduler, thus activating other co-processes

So, in general, yield is seen as a way to control processes in the process.

Understanding the process of co-processes

Let's start with an example of a simple co-process:

An analysis of the above examples:
There is no expression on the right-hand side of yield, so the default output value is None
The first call to Next (...) begins. It is because this time the generator has not started, does not stop at yield there, this time is also unable to send data through send. So when we pass next (...) when the process is activated, the program runs to x = yield, and here's a question we need to be aware that thecalculation of the expression x = yield is calculated by first calculating the content to the right of the equal sign, and then assigning it, so when the generator is activated, the program stops at yield. But there is no value assigned to X.
When we call the Send method, yield receives this value and assigns it to X, and when the program runs to the end of the thread definition body and throws a Stopiteration exception as it does with the generator.

If the co-process does not pass next (...) Activation (also we can activate by Send (None), but we send directly, will prompt the following error:

About calling Next (...) This step of the function is often referred to as the "pre-excitation (prime)" process, which allows the process to move forward to the first yield expression, ready to be used as an active co-process

The process has four states during the run:

1. Gen_create: Waiting to start execution
2. Gen_running: Interpreter is executing, this state is generally not visible
3. gen_suspended: Pause at yield expression
4. Gen_closed: End of execution

Use the following example to view the status of the process:

Then continue to understand with an example of calculating the mean:

Here is a dead loop, as long as the send value to the co-process, you can always calculate down.
Through the above examples we find that if we want to start using the process, we must pass the next (...). mode activation of the process, if not pre-excitation, the process can not be used, if one day in the code is forgotten then there is a problem, so there is a pre-excitation association decorator, can help us do this thing

Pre-excitation co-process decorator

The following is a demonstration example of a pre-excitation adorner:

1  fromFunctoolsImportWraps2 3 4 defCoroutine (func):5 @wraps (func)6     defPrimer (*args,**Kwargs):7Gen = func (*args,**Kwargs)8 Next (gen)9         returnGenTen     returnPrimer One  A  - @coroutine - defAverager (): theTotal = 0.0 -Count =0 -Average =None -      whileTrue: +term =yieldAverage -Total + = Term +Count + = 1 AAverage = total/Count at  -  -Coro_avg =Averager () -  fromInspectImportgetgeneratorstate - Print(Getgeneratorstate (coro_avg)) - Print(Coro_avg.send (10)) in Print(Coro_avg.send (30)) - Print(Coro_avg.send (5))

With respect to the pre-excitation, it is automatically pre-activated when the yield from syntax is called, which is not compatible with the Coroutine adorner defined above. The Asyncio.coroutine adorner inside the python3.4 does not have a pre-excitation process, so it is compatible with yield from

Terminating the association and exception handling

Exceptions handled in the process are bubbled up and passed to the caller of the next function or the Send function (that is, the object that triggered the association)
Take the above code example, if we send a string instead of an integer, it will be an error, and this time the process is terminated

Starting from python2.5 the client code calls two methods on the generator object, displaying the exception sent to the coprocessor
are:throw and close
Generator.throw: Causes the generator to throw the specified exception at the paused yield expression, and if the generator handles the thrown exception, the code forwards to the next yield expression, and the output value becomes the return value of the calling Generator.throw method code. If the generator does not handle the thrown exception, the exception is bubbled up and passed to the caller's context.
Generator.close: Causes the generator to throw an Generatorexit exception at the paused yield expression. If the generator does not handle this exception, or throws a Stopiteration exception, the caller will not error, if the Generatorexit exception is received, the generator must not produce a value, otherwise the interpreter throws a RuntimeError exception. The exception thrown by the generator bubbles up and passes to the caller.
Here is an example:

When passing the exception that we define does not affect the co-process, the co-process does not stop, it can continue to send, but if it is not handled by the exception, it will be an error, and the association will be terminated

Let the co-process return a value

The following example shows how to get the return value of a co-process:

1  fromCollectionsImportNamedtuple2 3 4Result = Namedtuple ("Result","Colunt Average")5 6 7 defAverager ():8Total = 0.09Count =0TenAverage =None One      whileTrue: Aterm =yield -         ifTerm isNone: -              Break theTotal + = Term -Count+=1 -Average = total/Count -     returnResult (count,average) +  -Coro_avg =Averager () + Next (CORO_AVG) ACoro_avg.send (10) atCoro_avg.send (30) -Coro_avg.send (5) - Try: - coro_avg.send (None) - exceptstopiteration as E: -result =E.value in     Print(Result)

This allows you to obtain the final result:

In fact, it is relatively troublesome to get the return value in this way, and the yield from structure automatically captures the stopiteration exception, which is handled in the same way that the for loop handles stopiteration exceptions, and the looping mechanism makes it easier to understand handling exceptions. For yield from, the interpreter will not only catch the stopiteration exception, but will also change the Value property to the value of the yield from expression

About yield from

When using yield from Subgen () in Generator Gen, Subgen will gain control and pass the output value to Gen's caller, which means that the caller can directly control the Subgen while Gen blocks and waits for Subgen to terminate.

The first thing a yield from x expression does to an X object is to call ITER (x) to get an iterator, so x can be any object that can be iterated

Here is an example of yield from which you can simplify yield expressions:

1 defGen ():2      forCinch "AB":3         yieldC4      forIinchRange (1,3):5         yieldI6 7 Print(List (gen ()))8 9 defGen2 ():Ten     yield  from "AB" One     yield  fromRange (1,3) A  - Print(List (Gen2 ()))

The results of both are the same, but the yield from is more concise, but the yield from is not just a nested for loop to replace the output value.
The main function of yield from is to open the bidirectional channel, the outermost caller with the most inner layer of the sub-generator, so that both can send and output values directly, but also directly into the exception, and no longer as before in the middle of the process to add a lot of code to handle the exception

You can also link an iterative object through yield from

When the delegate builder pauses at the yield from expression, the caller can send the data directly to the child generator, the sub-generator then sends the output output value to the caller, and after the child generator returns, the interpreter throws the stopiteration exception and appends the return value to the exception object. The delegation generator is restored at this time.

The following is a complete example code

1  fromCollectionsImportNamedtuple2 3 4Result = Namedtuple ('Result','count Average')5 6 7 #Child Generators8 defAverager ():9Total = 0.0TenCount =0 OneAverage =None A      whileTrue: -term =yield -         ifTerm isNone: the              Break -Total + = Term -Count + = 1 -Average = total/Count +     returnResult (count, average) -  +  A #Delegation Builder at defgrouper (result, key): -      whileTrue: -Result[key] =yield  fromAverager () -  -  - #client-side code, which is the caller in defMain (data): -Results = {} to      forKey,valuesinchData.items (): +Group =grouper (Results,key) - Next (group) the          forValueinchvalues: * group.send (value) $Group.send (None)#This means we're going to abort.Panax Notoginseng  - Report (Results) the  +  A #Output Report the defReport (Results): +      forKey, resultinchSorted (Results.items ()): -Group, Unit = Key.split (';') $         Print('{: 2} {: 5} averaging {:. 2f}{}'. Format ( $ Result.count, group, result.average, Unit -         )) -  thedata = { -     'girls;kg':Wuyi[40.9, 38.5, 44.3, 42.2, 45.2, 41.7, 44.5, 38.0, 40.6, 44.5], the     'girls;m': -[1.6, 1.51, 1.4, 1.3, 1.41, 1.39, 1.33, 1.46, 1.45, 1.43], Wu     'boys;kg': -[39.0, 40.8, 43.2, 40.8, 43.1, 38.6, 41.4, 40.6, 36.3], About     'boys;m': $[1.38, 1.5, 1.32, 1.25, 1.37, 1.48, 1.25, 1.49, 1.46], - } -  -  A if __name__=='__main__': +Main (data)

The above code focuses on the Delegation Builder section, where each iteration of the loop creates a new Averager instance, each of which is the generator object used as the coprocessor.

Each value sent by the grouper is passed through the yield from processing to the Averager instance through the pipeline. Grouper pauses at the yield from expression, waiting for the Averager instance to process the value sent by the client. When the Averager instance finishes running, the returned value is bound to Results[key], and the while loop continuously creates the Averager instance, processing more values

and the child generator in the code above can return a value using return, and the returned value becomes the value of the yield from expression.

On the meaning of yield from

An important note about yield from six points:

1. The output value of the child generator is passed directly to the caller of the delegate builder (that is, the client code)
2. The values sent to the delegate builder using the Send () method are passed directly to the child generator. If the value sent is None, then the __next__ () method of the sub-generator is called to the delegate. If the value sent is not none, then the child generator's Send method is called, and if the calling method throws a Stopiteration exception, the delegate generator resumes running, and any other exception is bubbled up to the delegation builder
3. When the generator exits, the return expr expression in the generator (or sub-builder) starts with the stopiteration (expr) exception thrown
4. The value of the yield from expression is the first parameter passed to the stopiteration exception when the child generator terminates. The other two characteristics of the yield from structure are related to the exception and termination.
5. The exception to the incoming delegation generator, except for Generatorexit, is passed to the child generator's throw () method. If a stopiteration exception is thrown when the throw () method is called, the delegate generator resumes running. Exceptions outside stopiteration are bubbled up and passed to the delegation builder
6. If the generatorexit exception is passed into the delegation generator, or the close () method is called on the delegation builder, the Clsoe () method is called on the child generator, if it has one. If the call to the close () method causes an exception to be thrown, the exception is bubbled up and passed to the delegate builder, otherwise the delegate builder throws a Generatorexit exception

Python process in-depth understanding