A brief analysis of Python yield usage

A brief analysis of Python yield usage

Liu Xuefeng, software engineer, HP

November 22, 2012

Novice Python developers often find a lot of Python functions used in the yield keyword, however, with the yield of the function execution process is not the same as the normal function, yield exactly what to do, why to design yield? This article will explain the concept and usage of yield, and help readers to realize the simple and powerful function of yield in Python.

As you may have heard, the function with yield is called generator (generator) in Python, what is generator?

Let's throw away the generator and show the concept of yield with a common programming topic.

How to generate Fibonacci columns

The Fibonacci (FIBONACCI) Number column is a very simple recursive sequence, in addition to the first and second numbers, any one of the numbers can be added by the first two numbers. Using a computer program to output the first N number of Fibonacci numbers is a very simple question, and many beginners can easily write the following functions:

Listing 1. The number of first N in a simple output Fibonacci array

Def FAB (max):     N, a, b = 0, 0, 1 while     n < max:         print B         A, B = B, a + b         n = n + 1

Executing fab (5), we can get the following output:

>>> Fab (5)  1  1  2  3  5

The result is no problem, but an experienced developer will point out that printing a number directly in the FAB function results in a poor reusability of the function because the FAB function returns None and the other functions cannot get the sequence generated by the function.

To improve the reusability of fab functions, it is best not to print the columns directly, but instead to return a list. The following is the second version after the FAB function is overwritten:

Listing 2. Output Fibonacci Number The first N numbers second edition

Def FAB (max):     N, a, b = 0, 0, 1     L = [] while     n < max:         l.append (b)         A, B = B, a + b         n = n + 1
   return L

The List returned by the FAB function can be printed using the following method:

>>> for N in Fab (5): ...     Print N ...  1  1  2)  3  5

The rewritten fab function can meet the requirements of reusability by returning the List, but more experienced developers will point out that the memory consumed by this function will increase with the parameter max, and if you want to control the memory footprint, it is best not to use the list

To save the intermediate results, instead of iterating through the Iterable object. For example, in python2.x, the code:

Listing 3. Iterate through Iterable objects

For I in range (+): Pass

Causes a List of 1000 elements to be generated, and the code:

For I in Xrange (£): Pass

A List of 1000 elements is not generated, but the next value is returned in each iteration, and the memory footprint is small. Because Xrange does not return a List, it returns a Iterable object.

With iterable we can rewrite the FAB function to a class that supports iterable, and the following is the third version of Fab:

Listing 4. A third version

Class Fab (object):     def __init__ (self, max):         Self.max = Max         SELF.N, self.a, self.b = 0, 0, 1     def __iter__ (s  ELF):         return self     def next:         if SELF.N < self.max:             r = self.b             self.a, self.b = self.b, SELF.A + self.b             SELF.N = SELF.N + 1             return R         raise Stopiteration ()

The Fab class continuously returns the next number of columns through next (), and memory consumption is always constant:

>>> for N in Fab (5): ...     Print N ...  1  1  2)  3  5

However, with this version of the class rewrite, the code is far less concise than the first version of the Fab function. Yield comes in handy if we want to maintain the simplicity of the first version of the Fab function, while also gaining the iterable effect:

Listing 5. Version fourth using yield

Def FAB (max):     N, a, b = 0, 0, 1 while     n < max:         yield b         # Print B         A, B = B, a + b         n = n + 1 ""

In the fourth version of Fab and the first edition, only print B was changed to yield B, and the iterable effect was achieved while maintaining simplicity.

The Fab and the second version of Fabs that call version fourth are exactly the same:

>>> for N in Fab (5): ...     Print N ...  1  1  2)  3  5

Simply put, the function of yield is to turn a function into a generator, the function with yield is no longer a normal function, the Python interpreter treats it as a generator, and the Call to FAB (5) does not execute the FAB function, but instead returns a Iterab Le Object! When the For loop executes, each loop executes the code inside the FAB function, and when it executes to yield B, the FAB function returns an iteration value, and the next iteration, the code proceeds from the next statement of Yield B, and the local variable of the function looks exactly the same as before the last break, so the function Continue execution until yield is encountered again.

You can also call the next () Method of Fab (5) Manually (because Fab (5) is a generator object with the next () method) so that we can see more clearly the execution process of the Fab:

Listing 6. Execution process

>>> f = Fab (5)  >>> f.next ()  1  >>> f.next ()  1  >>> f.next ()  2  >>> f.next ()  3  >>> f.next ()  5  >>> f.next ()  Traceback ( Most recent:   File ' <stdin> ', line 1, <module>  stopiteration

When the function execution finishes, generator automatically throws the Stopiteration exception, indicating that the iteration is complete. In the For loop, there is no need to handle the stopiteration exception and the loop will end normally.

We can draw the following conclusions:

A function with yield is a generator, which, unlike a normal function, generates a generator that looks like a function call, but does not execute any function code until it calls next () (which is automatically called next () in the For loop) to begin execution. Although the execution process still executes according to the process of the function, each execution to a yield statement is interrupted and an iteration value is returned, and the next execution proceeds from the next statement of yield. It looks as if a function was interrupted several times by yield during normal execution, and each break returns the current iteration value through yield.

The benefits of yield are obvious, and the ability to rewrite a function into a generator yields an iterative capability that computes the value of the next next () rather than the instance of the class, not only the code is concise, but the execution process is exceptionally clear.

How can I tell if a function is a special generator function? You can use Isgeneratorfunction to judge:

Listing 7. Use Isgeneratorfunction to judge

>>> from inspect import isgeneratorfunction  >>> isgeneratorfunction (FAB)  True

To distinguish between Fab and fab (5), Fab is a generator function, and fab (5) is a generator returned by a call to fab, like the definition of a class and the difference between an instance of a class:

Listing 8. Class definitions and instances of classes

>>> Import Types  >>> isinstance (fab, types. Generatortype)  False  >>> isinstance (Fab (5), types. Generatortype)  True

Fab is not iterative, and Fab (5) is iterative:

>>> from collections import iterable  >>> isinstance (Fab, iterable)  False  >>> Isinstance (Fab (5), iterable)  True

Each call to the FAB function generates a new instance of generator, each of which does not affect each other:

>>> f1 = Fab (3)  >>> F2 = Fab (5)  >>> print ' F1: ', F1.next ()  f1:1  >>> print ' F2: ', F2.next ()  f2:1  >>> print ' F1: ', F1.next ()  f1:1  >>> print ' F2: ', F2.next ()  f2:1  >>> print ' F1: ', F1.next ()  f1:2  >>> print ' F2: ', F2.next ()  f2:2  >>> print ' F2: ', F2.next ()  f2:3  >>> print ' F2: ', F2.next ()  F2:5

Back to top of page

The function of return

In a generator function, if there is no return, the default execution to the function is complete, if the return in the execution process, then directly throws Stopiteration terminates the iteration.

Back to top of page

Another example

Another example of yield is from file reads. Calling the Read () method directly on a file object causes unpredictable memory consumption. A good approach is to use fixed-length buffers to continuously read the contents of the file. With yield, we no longer need to write an iterative class of read files to easily implement file reads:

Listing 9. Another example of yield

def read_file (Fpath):     block_size = 1024x768     with open (Fpath, ' RB ') as F: While         True:             BLOCK = F.read (block_ SIZE)             if block:                 yield block             else:                 return

The above simply introduces the basic concepts and usage of yield, and yield is more powerful in Python 3, which we will discuss in subsequent articles.

Note: The code in this article is debugged in Python 2.7 through

Go to [http://www.ibm.com/developerworks/cn/opensource/os-cn-python-yield/]

A brief analysis of Python yield usage