Python 3 Builder detailed

------Generator------------------------------------------------------------------

Today Python provides more support for latency-it provides the tools to produce results when needed, rather than producing results immediately. In particular, there are two language constructs that delay the creation of the results as much as possible.

• Generator functions: Write as regular def statements, but use the yield statement to return one result at a time, suspending and continuing their state between each result.

• Generator expressions: Similar to list parsing, however, they return an object that produces results on demand, rather than building a list of results.

Because neither of them builds a single list at a time, they save memory space and allow the calculation to spread across the result requests. We will see that both end up with the magic of implementing their deferred results by implementing the iteration protocol we described earlier.

Generator function: Yield VS return

We have learned to write general functions that receive input parameters and immediately send back a single result. However, it is also possible to write a function that can return a value and then continue from where it left off. Such functions are called generator functions because they produce a sequence of values over time.

In general, generator functions are the same as regular functions, and are actually written with regular def statements, however, when created, they automatically implement iterative protocols so that they can appear in an iterative context.

Status hangs

Unlike regular functions that return a value and exit, the generator function automatically hangs at the moment the value is generated and resumes execution of the function. Therefore, they are useful for calculating the entire series of values in advance and for the total manual save and restore state of the Thunder. because the generator functions hold their entire local scope when they are suspended, their local variables persist information and make them available when the function resumes .

The main code between the generator function and the regular function differs in that the generator yields a value instead of returning a value. The yield statement suspends the function and sends a value back to the caller, but retains enough state to allow the function to continue from where it left off. When it continues, the function resumes execution as soon as the last yield returns. From a function's point of view, it allows its code to produce a series of values over time instead of calculating them at once and returning them in content such as lists.

Iterative Protocol Integration

To really understand the generator functions, we need to know that they are closely related to the concept of iterative protocols in Python. As we can see, an iterative object defines a __next__ method that either returns the next item in the iteration or throws a special Stopiteration exception to terminate the iteration. An iterator for an object is received with the ITER built-in function.

If this protocol is supported, Python's for loop and other iterative backgrounds use this iterative protocol to traverse a sequence or value generator, and if not, iterate back to repeat the index sequence.

to support this protocol, the function contains a yield statement, which is specifically compiled into a generator . When called, they return an iterator object that supports an interface that continues execution with an automatically created method named __next__. The generator function may also have a return statement, which always terminates the generation of the value directly at the end of the DEF statement block. Technically, Grams throws a Stopiteration exception after any regular function exits execution. From the caller's point of view, the __next__ method of the generator continues the function and runs to the next yield result to return or throw a stopiteration exception.

The direct effect is a generator function, written as a DEF statement containing yield statements, that automatically supports iterative protocols, and thus may be used in any iteration of the environment to produce results over time and as needed.

Generator function Application

To get a clear idea of the basics, consider the following code, which defines a generator function that will be used to continuously generate the squares of numbers for some columns.

>>> def gensquares (n): For I in range (n): Yield i * * 2

This function generates a value for Each loop and then returns it to its caller. When it is paused, its previous state is saved and the controller is immediately recycled after the yield statement. For example, when used in a For loop, control is returned to the function every time the yield statement for the function is completed in the loop.

>>> for I in Gensquares (5):p rint (i, end= ":") 0:1: 4:9: 16:

In order to terminate the generated value, the function can either use a return statement with no value, or simply disengage the controller at the end of the function body.

If you want to see what happens in for, call a generator function directly:

>>> x = gensquares (4) >>> X<generator object gensquares at 0x0000014ef59fedb0>

The result is a generator object that supports the iterator protocol, that is, the generator object has a __next__ method, which can start the function, or recover from its last yield value, and when the last of a series of values is obtained, Generates a Stopiteration exception. For convenience, the next (x) built-in function invokes an object's x.__next__ () method for us:

>>> Next (x) # py3-like x.__next__ () 0>>> Next (x) # similar method in Py2 for X.next () or Next () 1>>&G T  Next (x) 4>>> next (x) 9>>> next (x) Traceback (most recent call last): File ' <pyshell#52> ', line 1, in <module> Next (x) stopiteration

As previously learned, the For loop (and other iterative environments) works the same way with the generator: by repeating the __next__ method, you know to catch an exception. if an object that does not support this kind of protocol iterates, the For loop uses the index protocol to iterate .

Note In this example, we can simply build a list of the values we get at once.

>>> def buildsquares (n): res = []for i in Range (n): res.append (i * * 2) return res>>> for x in Buildsquares (5): Print (x, end = ":") 0:1: 4:9: 16:

For this example, we can also use a for loop ,map or list parsing techniques.

>>> for x in [n * 2 for N in range (5)]:p rint (x, end= ":") 0:1: 4:9: +: >>> for x in Map ((LAMBD A. N = * 2), range (5)):p rint (x, end= ":") 0:1: 4:9: 16:

However, the generator is better in terms of memory usage and performance. They allow the function to avoid doing all the work temporarily, which is especially useful when the list of results is large or when it takes a lot of time to process each result. The generator processes the time distribution of a series of values in the loop iteration.

However, for more advanced applications, they provide a simpler alternative to manually saving the class's objects to the state in the iteration. With generators, function variables can be saved and restored automatically.

Extension Generator function protocol: send and Next

In Python2.5, a Send method is added to the Generator function protocol. The Send method produces the next element of a series of results, just like the __next__ method, but it also provides a way for the caller to communicate with the generator, which can affect its operation.

Technically, yield is now in the form of an expression that can be returned to an incoming element to be sent, rather than a statement [although it can be either: as Yield x or a = (yield x)]. The expression must be included in parentheses unless it is the only item to the right of the assignment statement. For example, x = yield y is no problem, just like x = (yield y) + 42.

When this additional protocol is used, the value can be sent to a generator G by calling G.send (value). The code of the generator is then restored, and the yield expression in the generator returns the value passed in in order to send. If the normal g.__next__ () method (or its equivalent Next (G)) is called in advance, yield returns none. For example:

>>> def Gen (): For I in range: x = yield iprint (x) >>> G = gen () >>> Next (g) # Next () Start Generator 0>>> G.send (77) # Advanced Send method sends parameters to generator expression 771>>> g.send () 882>>> Next (G) # return Nonen One3

For example, using the Send method, write a generator that can be terminated by its callers. In addition, in version 2.5, the generator also supports the throw (type) method, which produces an exception and a close method within the generator's last yield, which produces a new Generatorexit exception within the generator that terminates the iteration. These are some of the advanced features that we don't learn in depth, and see Python's standard library for more details.

Note that although Python 3 provides a handy built-in function for next (X), it invokes the X.__next__ method of an object, but other generator methods, such as send, must be called directly as a method of the generator object (for example, G.send (x)). It makes sense, you know, that these extra methods are only implemented on built-in generator objects, and the __next__ method is applied to all of the iterative objects (including built-in types and user-defined classes).

Generator expression: Iterator encounters list resolution

in the latest version of Python, the concept of iterators and list parsing forms a new feature of this language, the generator expression . Syntactically, generator expressions are like General List parsing, but they are enclosed in parentheses instead of square brackets.

>>> [x * * 2 for X in range (4)][0, 1, 4, 9]>>> (x * * 2 for X in range (4)) # Generator Expression <generator OB Ject <genexpr> at 0x0000014ef59fedb0>

In fact, writing a list resolution, at least on the basis of a function, is basically equivalent to including a generator expression in a list built-in call to force it to generate all the results in the list at once.

>>> List (x * * 2 for X in range (4)) [0, 1, 4, 9]

However, the generator expression is quite different from the execution process: instead of building the result in memory, it returns a generator object that will support the iterative protocol and operate in any iterative context.

>>> G = (x * * 2 for X in range (4)) >>> Next (g) 0>>> next (g) 1>>> next (g) 4>>> NEX T (g) 9>>> next (g) Traceback (most recent): File "<pyshell#99>", line 1, <module> next ( G) stopiteration

We generally do not mechanically use the next iterator to manipulate the generator expression, because the For loop is automatically triggered.

>>> for Num IN (x * * 2 for X in range (4)):p rint ("%s,%s"% (num, num/2.0)) 0, 0.01, 0.54, 2.09, 4.5

In fact, the context of every iteration will be this, including built-in functions such as Sum, Map, and sorted, as well as other iterative contexts previously involved, such as any, all, and list built-in functions.

Note that if the generator expression is within other parentheses, as in the case of those function calls, the parentheses of the generator itself are not necessary. In spite of this, additional parentheses are required in the second sorted call below.

>>> SUM (x * * 2 for X in range (4)) 14>>> sorted (x * * 2 for X in range (4)) [0, 1, 4, 9]>>> sorted ( X * * 2 for X in range (4), reverse=true) [9, 4, 1, 0]>>> import math>>> list (map (math.sqrt, (x * * 2 for X in range (4))) [0.0, 1.0, 2.0, 3.0]

Generator expressions can generally be thought of as memory-space optimizations, which do not require a list resolution like square brackets to construct the entire list at once. They may be slightly slower to run in practice, so they may be the best choice for operations with very large result sets. For a more authoritative evaluation of performance, you must wait until the last time the timing script is written.

Generator function VS Builder expression

Interestingly, the same iteration of Mong Wang can be written with a generator function or a generator expression. For example, the resulting expression, repeat 4 times for each letter in a string.

>>> g = (c * 4 for C in "SPAM") >>> list (g) [' SSSS ', ' pppp ', ' AAAA ', ' MMMM ']

The equivalent generator function requires a bit more code, but, as a function of multiple statements, it will be able to write more logic and use more state information if needed.

>>> def timesfour (S): For C in S:yield c * 4>>> G = timesfour ("spam") >>> list (G) [' SSSs ', ' pppp ', ' AAAA ', ' mmmm ']

Expressions and functions support automatic iterations and manual iterations ... The previous list automatically calls the iteration, as the next iteration is done manually.

>>> G = (c * 4 for C in "SPAM") >>> i = iter (G) >>> Next (i) ' SSSS ' >>> next (i) ' PPPP ' >&G t;> G = timesfour (' spam ') >>> I = iter (G) >>> Next (i) ' SSSs ' >>> next (i) ' PPPP '

Note that we are iterating over the new generator here, as described in the next section, where the generator is a single iterator.

Generator is a single Iterator object

Both the generator function and the generator expression itself are iterators, and thus only support one active iteration ... Unlike some built-in types, we cannot have multiple iterators that are located in different locations in the result set. For example, using the builder expression in the preceding section, the iterator to a generator is the god of the generator (in fact, the call to ITER on a generator has no effect).

>>> G = (c * 4 for C in "SPAM") >>> iter (g) is gtrue

If you use multiple iterators manually to iterate over the result stream, they will point to the same location.

>>> G = (c * 4 for C in "SPAM") # New generator expression >>> I1 = iter (G) >>> Next (I1) ' SSSS ' >>> Next (I1) ' pppp ' >>> I2 = iter (G) #---->>> Next (I2) ' AAAA '

Also, once any iterator runs to completion, the even iterator will be exhausted, and we must produce a new generator to start again.

>>> list (I1)                              #  automatic iteration [' MMMM ']> >> next (I2)                Manual iterative Traceback for               # i2   (most recent call last):  file  "<pyshell#156>",  line 1,  in <module>    next (I2) stopiteration                             #  exception >>> i3 = iter (G)                          #  Generate a new iterator (it will not actually generate a new) >>>  next (I3) traceback  (most recent call last):  file  "<pyshell#158 > ",  line 1, in <module>    next (I3) stopiteration                              #  still iterates over abnormal >>> i3 = iter (c *  4 for c in  "SPAM")      #  New iterator >>> next ( I3)                               #  start iteration ' SSSS '

The same is true for the generator function, where the statement-based DEF equivalent form supports only one active generator and is exhausted after one iteration.

>>> def timesfour (S): For C in S:yield c * 4>>> G = timesfour ("spam") >>> iter (G) is Gtrue>> ;> I1, I2 = iter (g), ITER (g) >>> Next (I1) ' SSSs ' >>> Next (I1) ' PPPP ' >>> Next (I2) ' AAAA '

This differs from the behavior of some built-in types, which support multiple iterators and pass in an active iterator and reflect their original modifications.

>>> L = [1, 2, 3, 4]>>> I1, I2 = iter (L), ITER (L) >>> next (I1) 1>>> next (I1) 2>>> ; Next (I2) 1>>> del l[2:]>>> Next (I1) Traceback (most recent call last): File ' <pyshell#180> ', line 1 , in <module> next (I1) stopiteration

When we start not to write class-based iterators, we will see that we decide that we want to support multiple iterators for our own objects.

This article from "Professor elder brother" blog, declined reprint!

Python 3 Builder detailed