Python generator and yield, Python generator yield

Python generator and yield, Python generator yield
List derivation and generator expression

When we create a list, we create an object that can be iterated:

>>> squares=[n*n for n in range(3)]>>> for i in squares:print i014

This kind of list creation operation is very common, called list derivation. However, the list iterators, such as str and file, are easy to use, but they are stored in the memory. If the value is large, it will be very troublesome.

The generator expression is different. It executes the same computation as the list, but iteratively generates results. Its syntax is the same as list derivation, but it should replace brackets with parentheses:

>>> squares=(n*n for n in range(3))>>> for i in squares:print i014

Generator expressions do not create objects in sequence form, and do not read all values into the memory. Instead, they create a Generator object (Generator) That is iterated and generates values as needed ).

Is there any other way to generate a generator?

Example: Fibonacci Series

For example, to generate the first 10 digits of the Fibonacci series, we can write as follows:

def fib(n):    result=[]    a=1    b=1    result.append(a)    for i in range(n-1):        a,b=b,a+b        result.append(a)    return resultif __name__=='__main__':    print fib(10)

When there are few numbers, the function runs well, but when there are many numbers, the problem arises. It is not a good idea to generate a list of thousands or tens of thousands of lengths.

In this way, the requirement becomes: Write a function that can generate iteratable objects, or do not let the function return all values at a time, but return a value at a time.

This seems to be contrary to our common sense. When we call a common Python function, it is generally executed from the first line of code of the function, it ends with the return statement, exception, or function (which can be considered as an implicit return of None ):

Def fib (n): a = 1 B = 1 for I in range (n-1): a, B = B, a + B return aif _ name __= = '_ main _': print fib (10) >>> 1 # gets stuck when the first value is returned

Once the function returns the control to the caller, it means the operation is complete. All the work done in the function and the data stored in the local variable will be lost. When you call this function again, everything will be created from scratch. A function has only one chance to return results, so all results must be returned at a time. We usually think so. But what if they are not? See the magic yield:

def fib(n):    a=1    yield a    b=1    for i in range(n-1):        a,b=b,a+b        yield aif __name__=='__main__':    for i in fib(10):        print i>>> 112358132134

Generator

The definition of a generator in python is very simple. A function using the yield keyword can be called a generator, which generates a sequence of values:

def countdown(n):    while n>0:        yield n        n-=1if __name__=='__main__':    for i in countdown(10):        print i

Generator function return generator. Note that the generator is a special iterator. As an iterator, the generator must define some methods, one of which is _ next __(). Like the iterator, we can use the next () function (Python3 is _ next _ () to obtain the next value:

>>> c=countdown(10)>>> c.next()10>>> c.next()9

Whenever the generator is called, it returns a value to the caller. Use yield in the generator to complete this action. To remember what yield did, the simplest way is to treat it as a special return for the generator function. When next () is called, the generator function continues to execute statements until yield is encountered. At this time, the "State" of the generator function will be frozen and the values of all variables will be retained, the location of the code to be executed in the next line will also be recorded until the next () command is called again to continue executing the statement after yield.

Next () cannot be executed infinitely. When iteration ends, a StopIteration exception is thrown. If you want to end the generator when iteration is not over, you can use the close () method.

>>> c.next()1>>> c.next()StopIteration>>> c=countdown(10)>>> c.next()10>>> c.close()>>> c.next()StopIteration

Coroutine and yield expressions

Yield statements have more powerful functions. As a statement that appears on the right of the value assignment operator, it accepts a value, or generates a value at the same time and accepts a value.

def recv():    print 'Ready'    while True:        n=yield        print 'Go %s'%n>>> c=recv()>>> c.next()Ready>>> c.send(1)Go 1>>> c.send(2)Go 2

The functions that use the yield statement in this way are called coroutines. In this example, the initial call to next () is essential, so that the coroutine can execute the statement that can lead to the first yield expression. Here, the coroutine will be suspended and wait for the relevant generator object to send a value to it by the send () method. The value passed to send () is returned by the yield Expression in the coroutine.

The coroutine generally runs indefinitely. You can use close () to explicitly close it.

If a value is provided in the yield expression, the coroutine can use the yield statement to receive and send return values simultaneously.

def split_line():    print 'ready to split'    result=None    while True:        line=yield result        result=line.split()>>> s=split_line()>>> s.next()ready to split>>> s.send('1 2 3')['1', '2', '3']>>> s.send('a b c')['a', 'b', 'c']

Note:: Understanding the order in this example is very important. The first next () method allows the coroutine to execute the yield result, which returns the value of result None. In the next send () call, the received value is placed in line and split into result. The Return Value of the send () method is the value of the next yield statement. That is to say, the send () method can pass a value to the yield expression, but its return value comes from the next yield expression, rather than the yield expression that receives the value passed by send.

If you want to use the send () method to enable coroutine execution, you must first send a None value, because there is no yield statement to accept the value at this time, otherwise an exception will be thrown.

>>> s=split_line()>>> s.send('1 2 3')TypeError: can't send non-None value to a just-started generator>>> s=split_line()>>> s.send(None)ready to split

Use the generator and coroutine

At first glance, it does not seem obvious how to use generators and coroutines to solve practical problems. However, generators and coroutines are particularly useful in solving system, network, and distributed computing problems. In fact, yield has become one of the most powerful keywords in Python.

For example, to create a file processing pipeline:

import os,sysdef default_next(func):    def start(*args,**kwargs):        f=func(*args,**kwargs)        f.next()        return f    return start@default_nextdef find_files(target):    topdir=yield    while True:        for path,dirname,filelist in os.walk(topdir):            for filename in filelist:                target.send(os.path.join(path,filename))@default_nextdef opener(target):    while True:        name=yield        f=open(name)        target.send(f)    @default_nextdef catch(target):    while True:        f=yield        for line in f:            target.send(line)            @default_nextdef printer():    while True:        line=yield        print line

Connect these coroutines to create a data stream processing pipeline:

finder=find_files(opener(catch(printer())))finder.send(toppath)

The execution of the program is completely driven by sending data to the first coroutine find_files (). The coroutine pipeline will remain active forever until it explicitly calls close ().

In short, the generator is very powerful. Coroutine can be used to achieve some form of concurrency. In some types of applications, you can use a task scheduler and some generators or coroutines to implement collaborative user space multithreading, that is, greenlet. Yield's power will be truly reflected in coroutine, cooperative multitasking, and asynchronous IO.

In python, how does return and yield work? What are the differences between the two?

I often see that there are not many Chinese interpretations that can be found when others use or discuss the yield syntax. Today I am determined to fix yield and paste my understanding here.

Before yield: iterator)

Yield: generator (constructor) found)

Use yield: recursive call

1. iterator

The simplest example of the stack generator should be the array subscript. See the following c ++ code:

Int array [10];

For (int I = 0; I <10; I ++)

Printf ("% d", array [I]);

The stacked generator works in a container (array [10]), which extracts values from the container in a certain order (I ++) (array [I]) and perform operations (printf ("% d", array [I]).

The above code is translated into python:

Array = [I for I in range (10)]

For I in array:

Print I,

For I in array )? First, array is a container as a list, and secondly, the list built-in type has the default next action. What is the secret acquired after python finds that is not seen by everyone: take out the array, which is the container's heap generator. next from the inside, give the value to I for processing by the for loop entity. for, print the value.

The problem now is that data can be stacked in containers. Can code be used?

Why not? The dishes can be used to store dishes. Won't the wk think of using nt to store dishes? Of course, our yield won't be so yellow + bt.

2. constructor

How to change a function into a constructor? You just need yield in the function body!

Def gen ():
Print 'enter'
Yield 1
Print 'Next'
Yield 2
Print 'Next again'

For I in gen ():
Print I

Everybody! In python, yield appears in the gen function, and you can use next. The problem is, how to play next to the Code container?

When I get the iterator from the container, it does not have anything. It is at the container entrance. For arrays, it is the place where the subscript is-1. For functions, it is the function entry, however, if everything is ready, it will be less than next.

Start for I in g, next let itreator crawl to the place where the yield statement exists and return the value,

Next, go to the place where the next yield statement exists and return the values until the function returns (the end of the container ).
The output of the above Code is:

Enter
1
Next
2
Next again

If you do not see it, please do not read it down to avoid yield fixing it.

3. Use yield

Yield's code stacks not only interrupt function execution, but also write down the data at the breakpoint. next time the next book is connected, this is exactly what recursive functions need.

For example, traverse a binary tree in the middle order:

(It should have been written by David Mertz)

Def inorder (t ):
If t:
For x in inorder (t. left ):
Yield x
Yield t. label
For x in inorder (t. right ):
Yield x

For n in inorder (tree)

Print n

Of course, yield's ability to code next can also be used in other aspects, and it is found that the case is being posted... the remaining full text>

For python beginners, I would like to ask questions about yield and so on.

About yield, the kanchi240 downstairs is completely correct. I will not add it.

I want to talk about it. The logic of your yield function may be faulty. The entry parameter nested is an array. When the for sublist in nested in the function is executed, it is unreasonable if the nested is changed and an exception occurs. Maybe the original design is like this.
Def flatten (nested): try: for sublist in nested: for element in flatten (sublist): yield element failed t TypeError: yield nestedif _ name __= = "_ main _": for x in flatten (nested = [1, [2, [3]): print x, the output result of type (x) is
1 <type 'int'>
2 <type 'int'>
3 <type 'int'>
You seem to be a recursive function. The next step is like this.
Flatten (nested = [1, [2, [3])
Sublist = 1
For element in flatten (sublist.
And so on.

I modified the program and it may seem clearer.
Def flatten (nested, level = 0): try: for sublist in nested: print "sublist:", sublist, type (sublist) for element in flatten (sublist, level + 1): print level, "yield", element yield element failed t TypeError: print level, 'yield type error', nested, type (nested) yield nestedif _ name __= = "_ main _": for x in flatten (nested = [1, [2, [3]): print x, the output result of type (x) is
Sublist: 1 <type 'int'> 1 yield type error 1 <type 'int'> 0 yield 11 <type 'int'> sublist: [2, [3] <type 'LIST'> sublist: 2 <type 'int'> 2 yield type error 2 <type 'int'> 1 yield 20 yield 22 <type 'int'> sublist :[...... remaining full text>