In-depth understanding of the generators in Python

Generator (generator) concept

Instead of saving the result in a series, the generator saves the state of the generator and returns a value each time it is iterated until the end of the stopiteration exception is encountered.

Builder syntax

Generator expression: pass-through list parsing syntax, just replace the list parsing [] with ()
The list of things that the builder expression can do is basically handled, but the list parsing compares the memory when the sequence that needs to be processed is relatively large.

>>> Gen = (x**2 for x in range (5))
>>> Gen
<generator Object <genexPR> at 0x0000000002fb7b40>
>>> for G in Gen:
... print (g, end= '-')
...
0-1-4-9-16-
>>> for x in [0,1,2,3,4,5]:
... print (x, end= '-')
...
0-1-2-3-4-5-

Generator function: If the yield keyword appears in the function, then the function is no longer a normal function, but a generator function.
But the generator function can produce a wireless sequence so that the list has no way of handling it at all.
The effect of yield is to turn a function into a generator, and the function with yield is no longer a normal function, and the Python interpreter treats it as a generator.

The following is a generator function that can produce an infinite number of odd numbers.

Def odd ():
N=1
While True:
Yield n
n+=2
Odd_num = Odd ()
Count = 0
For O in Odd_num:
If Count >=5:break
Print (o)
Count +=1

Of course, a similar effect can be achieved by manually writing iterators, but the generator is more intuitive to understand

Class Iter:
def __init__ (self):
Self.start=-1
def __iter__ (self):
return self
def __next__ (self):
Self.start +=2
Return Self.start
I = Iter ()
For count in range (5):
Print (Next (I))

Off-topic: The generator is comprised of the __iter () and next__ () methods, so you can iterate directly using for, instead of the self-compiled ITER containing the stopiteration, which can only be iterated by a manual loop.

>>> from collections Import iterable
>>> from collections Import Iterator
>>> isinstance (Odd_num, iterable)
True
>>> isinstance (Odd_num, Iterator)
True
>>> iter (odd_num) is Odd_num
True
>>> Help (Odd_num)
Help on Generator object:

Odd = Class Generator (object)
| Methods defined here:
|
| __iter__ (self,/)
| Implement iter (self).
|
| __next__ (self,/)
| Implement Next (self).
......

See the above results, now you can be very confident in accordance with the iterator way to cycle it!

When the For loop executes, each loop executes the code inside the FAB function, and when it executes to yield B, the FAB function returns an iteration value, and the next iteration, the code proceeds from the next statement of Yield B, and the local variable of the function looks exactly the same as before the last break, so the function Continue execution until yield is encountered again. It looks as if a function was interrupted several times by yield during normal execution, and each break returns the current iteration value through yield.

Yield and return

In a generator, if there is no return, the default execution is returned to the stopiteration when the function is complete;

>>> def G1 ():
... yield 1
...
>>> G=g1 ()
>>> Next (g) #第一次调用next (g) Hangs after the yield statement is executed, so the program does not end at this point.
1
>>> Next (g) #程序试图从yield语句的下一条语句开始执行, the discovery has reached the end, so throw a stopiteration exception.
Traceback (most recent):
File "<stdin>", line 1, in <module>
Stopiteration
>>>

If a return is encountered, the Stopiteration abort iteration is thrown directly if the return is executed during execution.

>>> def g2 ():
... yield ' a '
.. return
... yield ' B '
...
>>> G=g2 ()
>>> Next (g) position after #程序停留在执行完yield ' a ' statement.
A
>>> Next (g) #程序发现下一条语句是return, so throw the stopiteration exception so that the yield ' B ' statement will never execute.
Traceback (most recent):
File "<stdin>", line 1, in <module>
Stopiteration

If a value is returned after return, then this value is the description of the stopiteration exception, not the return value of the program.

The generator has no way to return a value using return.

>>> def g3 ():
... yield ' hello '
... return ' world '
...
>>> G=g3 ()
>>> Next (g)
' Hello '
>>> Next (g)
Traceback (most recent):
File "<stdin>", line 1, in <module>
Stopiteration:world

Methods supported by the generator

>>> Help (Odd_num)
Help on Generator object:

Odd = Class Generator (object)
| Methods defined here:
......
| Close (...)
| Close (), raise generatorexit inside generator.
|
| Send (...)
| Send (ARG), send ' arg ' into generator,
| Return next yielded value or raise stopiteration.
|
| Throw (...)
| Throw (TYP[,VAL[,TB]]), raise exception in generator,
| Return next yielded value or raise stopiteration.
......

Close ()

Close the generator function manually, and the subsequent call will return the stopiteration exception directly.

>>> def g4 ():
... yield 1
... yield 2
... yield 3
...
>>> G=g4 ()
>>> Next (g)
1
>>> G.close ()
>>> Next (g) #关闭后, yield 2 and yield 3 statements will no longer work
Traceback (most recent):
File "<stdin>", line 1, in <module>
Stopiteration

Send ()

The biggest feature of the generator function is the ability to accept an externally passed-in variable and return it based on the contents of the variable.
This is the most difficult place to understand the generator function, but also the most important place, the implementation of the later I will talk about the process of it all depend on it.

Def gen ():
Value=0
While True:
Receive=yield value
If receive== ' E ':
Break
Value = ' Got:%s '% receive

G=gen ()
Print (G.send (None))
Print (G.send (' AAA '))
Print (G.send (3))
Print (G.send (' e '))

Execution process:

by G.send (None) or Next (g), you can start the generator function and execute to the point where the first yield statement ends.
At this point, the yield statement is executed, but receive is not assigned a value.
Yield value outputs the initial value of 0
Note: You can only send (None) when you start the generator function, and you will get an error message if you try to enter a different value.

by G.send (' AAA '), the AAA is passed in and assigned to receive, then the value of value is computed and returned to the while header, and the yield value statement is stopped.
The yield value now outputs "GOT:AAA" and then hangs.

Through G.send (3), the 2nd step is repeated and the result is "Got:3"

When we g.send (' e '), the program executes a break and then rolls out the loop, and finally the entire function is finished, so we get the stopiteration exception.
The results of the final implementation are as follows:

0
Got:aaa
Got:3
Traceback (most recent):
File "h.py", line +, in <module>
Print (G.send (' e '))
Stopiteration

Throw ()

Used to feed an exception to the generator function, which can end a system-defined exception, or a custom exception.
Throw () runs directly out of the exception and ends the program, either consumes a yield, or goes directly to the end of the program without the next yield.

Def gen ():
While True:
Try
Yield ' normal value '
Yield ' normal value 2 '
Print (' here ')
Except ValueError:
Print (' We got valueerror here ')
Except TypeError:
Break

G=gen ()
Print (Next (g))
Print (G.throw (valueerror))
Print (Next (g))
Print (G.throw (TypeError))

The output is:

Normal value
We got valueerror here
Normal value
Normal value 2
Traceback (most recent):
File "h.py", line <module>
Print (G.throw (TypeError))
Stopiteration

Explain:

Print (Next (g)): The normal value is output and stays before yield ' normal value 2 '.

Because G.throw (valueerror) is executed, all subsequent try statements are skipped, which means that the yield ' normal value 2 ' will not be executed, and then go into the except statement and print out the We got valueerror here.
It then enters the while statement part again, consumes a yield, and outputs the normal value.

Print (Next (g)) executes the yield ' normal value 2 ' statement and stays where it was after the statement was executed.

G.throw (TypeError): Will jump out of the try statement so that print (' here ') will not be executed, then execute the break statement, jump out of the while loop, and then reach the end of the program, so run out of the stopiteration exception.

A comprehensive example is given below to expand a multidimensional list, or to flatten a multidimensional list.

def flatten (nested):

Try
#如果是字符串, then manually throw the TypeError.
If isinstance (nested, str):
Raise TypeError
For sublist in nested:
#yield Flatten (sublist)
for element in Flatten (sublist):
#yield element
Print (' Got: ', Element)
Except TypeError:
#print (' here ')
Yield nested

l=[' AAADF ', [1,2,3],2,4,[5,[6,[8,[9]], ' DDF '],7]
For num in Flatten (L):
Print (num)

If it's difficult to understand, it's clear that the comments on the print statement are open for viewing.

Yield from

The function produced by yield is an iterator, so we usually put it in a looping statement to output the result.
Sometimes we need to put this yield-generated iterator in another generator function, which is the generator nesting.
For example, the following:

def inner ():
For I in range (10):
Yield I
def outer ():
G_inner=inner () #这是一个生成器
While True:
res = G_inner.send (None)
Yield Res

G_outer=outer ()
While True:
Try
Print (G_outer.send (None))
Except stopiteration:
Break

At this point, we can use the yield from statement to reduce my workload.

Def outer2 ():
Yield from inner ()

Of course, the focus of the yield from statement is to help us automatically deal with the anomalies between the inner and outer layers, there are 2 good articles written here, so I'm no longer wordy.
Http://blog.theerrorlog.com/yield-from-in-python-3.html
Http://stackoverflow.com/questions/9708902/in-practice-what-are-the-main-uses-for-the-new-yield-from-syntax-in-python-3

Summarize

According to duck model theory, a generator is an iterator that can be iterated using for.

The first time you execute next (generator), the program suspends after the yield statement is executed, and all parameters and states are saved.
Once again, when you execute next (generator), it is executed from the suspended state.
The loop ends when the end of the program is encountered or when Stopiteration is encountered.

Parameters can be passed in by Generator.send (ARG), which is the model of the co-process.

You can pass an exception by Generator.throw (Exception). The throw statement consumes a yield.
You can manually turn off the generator by Generator.close ().

Next () equivalent to send (None)

The above is a deep understanding of the content of the builder in Python, more related articles please pay attention to topic.alibabacloud.com (www.php.cn)!

