The builder and yield details in Python

list derivation and builder expressions

When we create a list, we create an object that we can iterate over:

The code is as follows:

>>> squares=[n*n for N in range (3)]
>>> for i in Squares:
Print I

0
1
4

This creates a list of operations that are common, called list inference. But iterators like lists such as STR, file, and so on are handy, but one thing is that they are stored in memory, which can be cumbersome if the values are large.

Unlike a generator expression, it performs calculations that contain the same, but iterative, build results as the list. Its syntax is the same as a list derivation, except that brackets are used instead of brackets:

The code is as follows:

>>> squares= (n*n for N in range (3))
>>> for i in Squares:
Print I

0
1
4

The generator expression does not create a sequence of objects, does not read all the values into memory, but creates a generator object (Generator) that iterates through and generates values as required.

So, is there any other way to generate the generator?

Example: Fibonacci sequence

For example, there is a requirement to generate the first 10 bits of the Fibonacci sequence, which we can write:

The code is as follows:

def fib (n):
Result=[]
A=1
B=1
Result.append (a)
For I in Range (n-1):
A,b=b,a+b
Result.append (a)
return result
If __name__== ' __main__ ':
Print FIB (10)

When the numbers are small, the function works fine, but when the numbers are large, the problem comes, and obviously generating a thousands of-tens of thousands of-length list is not a good idea.

In this way, the requirement becomes: Write a function that can generate an iterative object, or let the function return a value at a time instead of returning all the values at once.

This seems to be contrary to our common sense, when we call a normal Python function, it is usually executed from the first line of the function, ending with a return statement, an exception, or the end of the function (which can be considered an implicit return of none):

The code is as follows:

def fib (n):
A=1
B=1
For I in Range (n-1):
A,b=b,a+b
Return a
If __name__== ' __main__ ':
Print FIB (10)
>>>
1 #返回第一个值时就卡住了

Once a function returns control to the caller, it means that it is all over. All the work done in the function and the data saved in the local variables will be lost. When you call this function again, everything will be created from scratch. The function has only one chance to return the result, so all results must be returned at once. Usually we all think so. But what if they are not? Please see the magic Yield:

The code is as follows:

def fib (n):
A=1
Yield a
B=1
For I in Range (n-1):
A,b=b,a+b
Yield a
If __name__== ' __main__ ':
For I in FIB (10):
Print I
>>>
1
1
2
3
5
8
13
21st
34

Generator generator

The definition of a generator in Python is simple, and a function that uses the yield keyword can be called a generator, which generates a sequence of values:

The code is as follows:

def countdown (N):
While n>0:
Yield n
N-=1
If __name__== ' __main__ ':
For I in Countdown (10):
Print I

The generator function returns the generator. It is important to note that the generator is a special kind of iterator. As an iterator, the generator must define some methods, one of which is __next__ (). As with iterators, we can use the next () function (Python3 is __next__ ()) to get the next value:

The code is as follows:

>>> C=countdown (10)
>>> C.next ()
10
>>> C.next ()
9

Whenever the generator is called, it returns a value to the caller. Use yield inside the generator to do this. The simplest way to remember what yield actually did is to use it as a special return for the generator function. When you call next (), the generator function continuously executes the statement until yield is encountered, at which point the "state" of the generator function is frozen, the values of all variables are preserved, and the next line of code to be executed is recorded until another call to next () continues to execute the statement after yield.

Next () cannot be executed indefinitely, and when the iteration ends, a stopiteration exception is thrown. If you want to end the generator at the end of the iteration, you can use the close () method.

The code is as follows:

>>> C.next ()
1
>>> C.next ()
Stopiteration
>>> C=countdown (10)
>>> C.next ()
10
>>> C.close ()
>>> C.next ()
Stopiteration

co-and yield expressions

The yield statement also has a more powerful function, as a statement appears to the right of the assignment operator, accepts a value, or generates a value at the same time and accepts a value.

The code is as follows:

Def recv ():
print ' Ready '
While True:
N=yield
print ' Go%s '%n
>>> C=recv ()
>>> C.next ()
Ready
>>> C.send (1)
Go 1
>>> C.send (2)
Go 2

A function that uses the yield statement in this way is called a co-process. In this example, the initial call to next () is necessary so that the process can execute the statement that leads to the first yield expression. Here the coprocessor hangs, waiting for the related generator object Send () method to send a value to it. The value passed to send () is returned by the yield expression in the association.

The run of the process is generally indefinite, and the method close () can be used to close it explicitly.

If a value is provided in the yield expression, the coprocessor can use the yield statement to receive and emit the return value at the same time.

The code is as follows:

Def split_line ():
print ' Ready to split '
Result=none
While True:
Line=yield result
Result=line.split ()
>>> S=split_line ()
>>> S.next ()
Ready to split
>>> s.send (' 1 2 3 ')
[' 1 ', ' 2 ', ' 3 ']
>>> S.send (' a b C ')
[' A ', ' B ', ' C ']

Note: It is important to understand the sequencing in this example. The first next () method allows the coprocessor to execute to yield result, which returns the value of result none. In the next send () call, the received value is placed in line and split into result. The return value of the Send () method is the value of the next yield statement. That is, the Send () method can pass a value to the yield expression, but its return value comes from the next yield expression, not the yield expression that receives the value passed by Send ().

If you want to use the Send () method to open the execution of the process, you must first send a value of none, because there is no yield statement to accept the value, otherwise it throws an exception.

The code is as follows:

>>> S=split_line ()
>>> s.send (' 1 2 3 ')
Typeerror:can ' t send non-none value to a just-started generator
>>> S=split_line ()
>>> S.send (None)
Ready to split

Using the generator and the co-process

At first glance, it doesn't seem obvious how to use generators and co-routines to solve real-world problems. However, generators and co-processes are particularly useful in solving certain problems in system, network, and distributed computing. In fact, yield has become one of the most powerful keywords in python.

For example, to create a pipeline that processes files:

The code is as follows:

Import Os,sys
def default_next (func):
def start (*args,**kwargs):
F=func (*args,**kwargs)
F.next ()
return F
return start
@default_next
def find_files (target):
Topdir=yield
While True:
For path,dirname,filelist in Os.walk (topdir):
For filename in filelist:
Target.send (Os.path.join (path,filename))

@default_next
DEF opener (target):
While True:
Name=yield
F=open (name)
Target.send (f)

@default_next
def catch (target):
While True:
F=yield
For line in F:
Target.send (line)

@default_next
def printer ():
While True:
Line=yield
Print Line

Then, you can create a data flow processing pipeline by connecting these processes together:

The code is as follows:

Finder=find_files (Opener (catch (Printer ())))
Finder.send (Toppath)

The execution of the program is driven entirely by sending the data to the first find_files (), which is always active until it explicitly calls Close ().

In short, the generator has a very powerful function. The co-process can be used to implement some form of concurrency. In some types of applications, a multi-threaded, Greenlet, collaborative user space can be implemented with a task scheduler and some generators or processes. Yield's power will be true in the process, collaborative multitasking (cooperative multitasking), and asynchronous Io.

