Python:generator, yield, yield from the detailed

1.Generator Expressions

A generator expression is a simple generator notation represented by parentheses:

Generator_expression:: = "(" Expression comp_for ")"

The generator expression produces a generator object whose syntax and for are similar, out of which it is contained by "()", not [] or {};

The calculation of a variable in the generator expression is deferred to the call of the __next__ () function, but the leftmost for loop clause is immediately computed so that if he has an error, it can be immediately seen. The later for loop clauses cannot be evaluated immediately because they may depend on the preceding for loop, for example (x*y for x in range) for y in Bar (x) )

After python3.6, if the generator appears in the Async def function, then the async for clause and the await expression can be understood to be asynchronous. If the generator expression contains an async for clause or an await expression, it is called an asynchronous generator expression. The asynchronous generator expression produces a new asynchronous generator object, which is an asynchronous iterator.

2. Yield Expressions

yield_atom :: = "(" yield_expression ")"
yield_expression :: = "yield" [expression_list | "from" expression]

Yield expressions are used to define a generator function or asynchronous generator function, so they can only be used within a function definition body. Use yield expressions in a function definition body to make it a generator,
Using yield expressions within an async def function body makes the coroutine function an asynchronous generator. E.g:
def gen (): // define a generator function
Yield 123
async def agen (): // Define an asynchronous generator function
Yield 123

When a generator function is called, it returns an iterator, also called a generator. This generator controls the execution of generator functions. When a generator function is called, the generator function starts to execute.
When the first yield expression is executed, it hangs and returns the value of expression_list to the caller. For suspend, we mean that all local state is preserved, including the binding of the current local variable,
Instruction pointer, internal call stack, any exception handling status. When execution is resumed by calling a function of the generator, the execution of the function is as if the yield expression was called again from the outside. After resuming execution,
The value of the yield expression depends on the method called. If __next __ () is called (usually through a for loop or the built-in next () function), then the value is None. If send () is called,
The value is the value of the parameter passed to send.

All of this makes the generator function very much like a coroutine; it generates multiple values, has multiple entry points and execution can be suspended. The only difference is that after the generator function yields, it cannot control where the program continues to execute,
Control is always passed back to the caller of the generator.

The yield expression can be anywhere in the try block. If the generator is not resumed before being ended (reaching zero reference or because of the garbage collection mechanism), the generator's close function is called, so finally
The clause is executed.
When yield from <expr> is used, it treats the additional expression as a sub-iterator, and the values produced by all sub-iterators are passed directly back to the caller of the current generator function. The parameter value of the current generator call send
And the exception parameters that call throw will be passed to the underlying iterator (sub-iterator), if it has a corresponding method. Otherwise, send causes AttributeError or TypeError, and throw immediately
Raise the exception passed to him.
When the execution of the underlying iterator is completed, the value of the StopIteration object becomes the value of the yield from expression. This value can be set explicitly when StopIteration is generated, or automatically set if
The child iterator is a generator (the child generator returns a value)
3. The generator-iterator method
This section introduces generator iterator methods, which can be used to control the execution of generator functions. Calling these functions while the generator is executing will result in ValueError.
(1) generator .__ next __ ()
    Start the execution of the generator function or resume the execution from the last yield expression that was executed. If it is resumed, the value of the yield expression is None, and continue to the next yield expression
    Suspend and return the value of expression_list to the caller of __next __ (). If the generator does not yield another value, a StopIteration exception is generated
    This method is generally implicitly called, such as for loop, next ()
(2) generator.send (value)
    Resume execution and send a value to the generator function. The value parameter is the result of the current yield expression. The send method returns the value of the next generator yield
    If there is no yield, return to StopIteration. When using send to start the generator, his parameter must be None, because there is no yield expression to receive the value.
(3) generator.throw (type [, value [, traceback]])
    An exception of type type is generated where the generator hangs, and the value of the next generator function yield is returned. If there is no yield, StopIteration is returned.
    If the generator function does not catch the incoming exception, or generates a different exception, then pass the exception to the caller.
(4) generator.close ()
    A GeneratorExit () is generated where the generator hangs. If the generator exits gracefully after it has been closed or a GeneartorExit is generated (does not catch the exception), close will return to its
    Caller. If the generator yields a value, then a RuntimeError is generated. If the generator generates any other exception, it will be passed to the caller. in case
    The generator exited because of an exception or normally, so close does nothing.
Examples:
>>> def echo (value = None):
... print ("Execution starts when‘ next () ’is called for the first time.")
... try:
... while True:
... try:
... value = (yield value)
... except Exception as e:
... value = e
... finally:
... print ("Do n‘t forget to clean up when‘ close () ‘is called.")
...
>>> generator = echo (1)
>>> print (next (generator))
Execution starts when ‘next ()’ is called for the first time.
1
>>> print (next (generator))
None
>>> print (generator.send (2))
2
>>> generator.throw (TypeError, "spam")
TypeError (‘spam’,)
>>> generator.close ()
Do n‘t forget to clean up when ‘close ()’ is called.

PEP380 added a yield from expression, allowing one generator to delegate some operations to another generator. This can remove part of the code containing yield and put it in another generator. In addition,
The child generator can return a value, which is also available to the delegate generator.
Although it mainly involves delegating a child generator, the yield from expression can actually delegate any child iterator.
Below a simple iterator, yield from iterable is essentially a short form: for item in iterable: yield item, for example:
>>> def g (x):
... yield from range (x, 0, -1)
... yield from range (x)
...
>>> list (g (5))
[5, 4, 3, 2, 1, 0, 1, 2, 3, 4]
However, unlike ordinary loops, yield from allows the sub-generator to receive the send and throw values directly from the calling area and return a final value to the outer generator. Examples are as follows:
>>> def accumulate ():
... tally = 0
... while 1:
... next = yield
... if next is None:
... return tally
... tally + = next
...
>>> def gather_tallies (tallies):
... while 1:
... tally = yield from accumulate ()
... tallies.append (tally)
...
>>> tallies = []
>>> acc = gather_tallies (tallies)
>>> next (acc) # Ensure the accumulator is ready to accept values
>>> for i in range (4):
... acc.send (i)
...
>>> acc.send (None) # Finish the first tally
>>> for i in range (5):
... acc.send (i)
...
>>> acc.send (None) # Finish the second tally
>>> tallies
[6, 10]
 




 
Python: generator, yield, yield from