List derivation and generator expression
When we create a list, we create an object that can be iterated:
Copy Code code as follows:
>>> squares=[n*n for N in range (3)]
>>> for i in Squares:
Print I
0
1
4
This is a common form of creating lists, called List derivations. But like a list of iterators, such as str, file, etc., although it is convenient to use, but there is a point, they are stored in memory, if the value is very large, it will be troublesome.
Instead of a generator expression, it performs the same calculations as the list, but it iterates over the build results. Its syntax is the same as list derivation, except that parentheses are used instead of brackets:
Copy Code code as follows:
>>> squares= (n*n for N in range (3))
>>> for i in Squares:
Print I
0
1
4
A builder expression does not create an object in the form of a sequence, and does not read all the values into memory, but instead creates a generator object (generator) that iterates through and generates a value on demand.
So, is there any other way to generate the generator?
Example: Fibonacci sequence
For example, there is a need to generate the first 10 bits of the Fibonacci series, which we can write:
Copy Code code as follows:
def fib (n):
Result=[]
A=1
B=1
Result.append (a)
For I in Range (n-1):
A,b=b,a+b
Result.append (a)
return result
If __name__== ' __main__ ':
Print FIB (10)
The function works fine when the numbers are low, but the problem comes when there are many numbers, and it's clear that generating a thousands of tens of thousands of-length list is not a good idea.
In this way, the requirement becomes: Write a function that can generate an iterative object, or, instead of having the function return all the values at once, return a value at a time.
This seems to contradict our common sense when we call an ordinary Python function, usually from the first line of the function, ending with the return statement, the exception, or the end of the function (which can be thought of as implicitly returning none):
Copy Code code as follows:
def fib (n):
A=1
B=1
For I in Range (n-1):
A,b=b,a+b
Return a
If __name__== ' __main__ ':
Print FIB (10)
>>>
1 #返回第一个值时就卡住了
Once a function returns control to the caller, it means all ends. All the work done in the function and the data saved in the local variable will be lost. When you call this function again, everything is created from scratch. A function has only one chance to return results, and therefore must return all results at once. Usually we all think so. But what if they are not? Please see the magical yield:
Copy Code code as follows:
def fib (n):
A=1
Yield a
B=1
For I in Range (n-1):
A,b=b,a+b
Yield a
If __name__== ' __main__ ':
For I in FIB (10):
Print I
>>>
1
1
2
3
5
8
13
21st
34
Builder generator
The definition of a generator in Python is simple, a function that uses the yield keyword can be called a generator, and it generates a sequence of values:
Copy Code code as follows:
def countdown (N):
While n>0:
Yield n
N-=1
If __name__== ' __main__ ':
For I in Countdown (10):
Print I
The generator function returns a generator. Note that the generator is a special kind of iterator. As an iterator, the builder must define some methods, one of which is __next__ (). As with iterators, we can use the next () function (Python3 is __next__ ()) to get the next value:
Copy Code code as follows:
>>> C=countdown (10)
>>> C.next ()
10
>>> C.next ()
9
Whenever the generator is invoked, it returns a value to the caller. Use yield inside the generator to complete this action. The easiest way to remember what yield is doing is to use it as a special return specifically for the generator function. When you call next (), the generator function keeps executing the statement, until yield is encountered, the "state" of the generator function is frozen, the value of all the variables is preserved, and the position of the next line of code to be executed is recorded until you call Next again () to continue executing the statement after yield.
Next () cannot be executed indefinitely, and when the iteration ends, a stopiteration exception is thrown. If you want to end the generator at the end of the iteration, you can use the close () method.
Copy Code code as follows:
>>> C.next ()
1
>>> C.next ()
Stopiteration
>>> C=countdown (10)
>>> C.next ()
10
>>> C.close ()
>>> C.next ()
Stopiteration
co-yield and expressions
The yield statement also has a more powerful function, appearing as a statement to the right of the assignment operator, accepting a value, or simultaneously generating a value and accepting a value.
Copy Code code as follows:
Def recv ():
print ' Ready '
While True:
N=yield
print ' Go%s '%n
>>> C=recv ()
>>> C.next ()
Ready
>>> C.send (1)
Go 1
>>> C.send (2)
Go 2
A function that uses the yield statement in this way is called a coprocessor. In this example, the initial call to next () is necessary so that the coprocessor can execute statements that lead to the first yield expression. Here the coprocessor hangs and waits for the related generator object Send () method to send a value to it. The value passed to send () is returned by the yield expression in the coprocessor.
The operation of a coprocessor is generally indefinite, and the method close () can be used to explicitly turn it off.
If a value is provided in the yield expression, the coprocessor can use the yield statement to receive and emit the return value at the same time.
Copy Code code as follows:
Def split_line ():
print ' Ready to split '
Result=none
While True:
Line=yield result
Result=line.split ()
>>> S=split_line ()
>>> S.next ()
Ready to split
>>> s.send (' 1 2 3 ')
[' 1 ', ' 2 ', ' 3 ']
>>> S.send (' a b C ')
[' A ', ' B ', ' C ']
Note: It is important to understand the order of precedence in this example. The first next () method allows the coprocessor to execute to yield result, which returns the value of result none. In the next send () call, the received value is placed in line and split into result. The return value of the Send () method is the value of the next yield statement. That is, the Send () method can pass a value to the yield expression, but its return value comes from the next yield expression, rather than the yield expression that receives the value passed by Send ().
If you want to use the Send () method to open the execution of the coprocessor, you must first send a none value, because there is no yield statement to accept the value, otherwise it throws an exception.
Copy Code code as follows:
>>> S=split_line ()
>>> s.send (' 1 2 3 ')
Typeerror:can ' t send non-none value to a just-started generator
>>> S=split_line ()
>>> S.send (None)
Ready to split
Using the builder and the coprocessor
At first glance, it does not seem obvious how to use the builder and the coprocessor to solve the actual problem. However, generators and processes are particularly useful when addressing some of the problems with systems, networks, and distributed computing. In fact, yield has become one of the most powerful keywords in python.
For example, to create a pipeline for processing files:
Copy Code code as follows:
Import Os,sys
def default_next (func):
def start (*args,**kwargs):
F=func (*args,**kwargs)
F.next ()
return F
return start
@default_next
def find_files (target):
Topdir=yield
While True:
For path,dirname,filelist in Os.walk (topdir):
For filename in filelist:
Target.send (Os.path.join (path,filename))
@default_next
DEF opener (target):
While True:
Name=yield
F=open (name)
Target.send (f)
@default_next
def catch (target):
While True:
F=yield
For line in F:
Target.send (line)
@default_next
def printer ():
While True:
Line=yield
Print Line
Then, by connecting these threads, you can create a data flow processing pipeline:
Copy Code code as follows:
Finder=find_files (opener) (catch (printer ()))
Finder.send (Toppath)
The execution of a program is driven entirely by the sending of data to the first find_files (), and the covariant pipe remains active until it explicitly calls Close ().
In short, the generator is very powerful. A coprocessor can be used to implement some form of concurrency. In some types of applications, you can use a Task scheduler and some generators or a coprocessor to implement collaborative user-space multithreading, or Greenlet. The power of yield will be truly embodied in the co-process, collaborative multitasking (cooperative multitasking), and asynchronous Io.
