Analysis on the use of Python yield

Introduction:Developers who are new to Python often find that the yield keyword is used in many Python functions. However, the execution process of functions with yield is different from that of common functions. What is yield used for? Why design yield? This article will explain the concept and usage of yield in a simple and in-depth way, and help readers understand the simple and powerful yield functions in Python.

 

You may have heard that functions with yield are called generator (generator) in Python and what is generator?

Let's move away from generator to present the concept of yield with a common programming question.

How to generate a Fibonacci data column

The Fibonacci data column is a simple recursive series. Except for the first and second numbers, any number can be obtained by adding the first two numbers. Using a computer program to output the first n of the Fibonacci number column is a very simple problem. Many beginners can easily write the following functions:

Listing 1. Simple output of the first n of the Fibonacci number column

Def FAB (max ): N, a, B = 0, 0, 1 While n <Max: Print B A, B = B, A + B N = n + 1

Run FAB (5) and we can get the following output:

>>> FAB (5) 1 1 2 3 5

There is no problem with the results, but experienced developers will point out that printing numbers with print in the FAB function will lead to poor reusability of the function, because the FAB function returns none, other functions cannot obtain the sequence generated by this function.

To improve the reusability of FAB functions, it is best not to print the series directly, but to return a list. The second version after the FAB function is rewritten is as follows:

Listing 2. output the second version of the first n of the Fibonacci number column

Def FAB (max ): N, a, B = 0, 0, 1 L = [] While n <Max: L. append (B) A, B = B, A + B N = n + 1 Return L

You can print out the list returned by the FAB function as follows:

>>> For N in FAB (5 ): ... Print n ... 1 1 2 3 5

The rewritten Fab function can meet the reusability requirement by returning the list, but more experienced developers will point out that the memory occupied by the function during running will increase with the increase of the max parameter, if you want to control memory usage, it is best not to use list

To save the intermediate results, but to iterate through the iterable object. For example, in python2.x, the Code is as follows:

Listing 3. Iteration through the iterable object

For I in range (0, 1000): Pass

This will generate a list of 1000 elements, and the code:

For I in xrange (0, 1000): Pass

Instead of generating a list of 1000 elements, the next value is returned in each iteration, and the memory space is very small. Xrange does not return list, but returns an iterable object.

With iterable, we can rewrite the FAB function into a class that supports iterable. below is the third version of FAB:

Listing 4. Third version

Class FAB (object ):   Def _ init _ (self, max ): Self. max = max Self. N, self. A, self. B = 0, 0, 1   Def _ ITER _ (Self ): Return self   Def next (Self ): If self. n <self. MAX: R = self. B Self. A, self. B = self. B, self. A + self. B Self. n = self. n + 1 Return R Raise stopiteration ()

The Fab class continuously returns the next number of columns through next (), and the memory usage is always constant:

>>> For N in FAB (5 ): ... Print n ... 1 1 2 3 5

However, the code of this version rewritten by class is far less concise than the FAB function of the first version. If we want to keep the simplicity of the first version of the FAB function and get the iterable effect, yield will come in handy:

Listing 5. Use version 4 of Yield

Def FAB (max ): N, a, B = 0, 0, 1 While n <Max: Yield B # Print B A, B = B, A + B N = n + 1   '''

Compared with the first version, FAB of the fourth version only changes print B to yield B, which achieves the iterable effect while keeping it simple.

Calling the fourth edition of FAB is exactly the same as the second edition of FAB:

>>> For N in FAB (5 ): ... Print n ... 1 1 2 3 5

In short, yield is used to convert a function into a generator. A function with yield is no longer a common function. The Python interpreter regards it as a generator and calls FAB (5) instead of executing the FAB function, an iterable object is returned! During for loop execution, code in the FAB function is executed in each loop. When yield B is executed, the FAB function returns an iteration value. During the next iteration, the Code continues to run from the next statement of yield B, and the local variable of the function looks exactly the same as before the last interrupted execution, so the function continues to run
Yield.

You can also manually call the next () method of FAB (5) (because FAB (5) is a generator object and this object has the next () method ), in this way, we can see more clearly the FAB Execution Process :,

Listing 6. Execution Process

>>> F = FAB (5) >>> F. Next () 1 >>> F. Next () 1 >>> F. Next () 2 >>> F. Next () 3 >>> F. Next () 5 >>> F. Next () Traceback (most recent call last ): File "<stdin>", line 1, in <module> Stopiteration

When the function execution ends, generator automatically throws a stopiteration exception, indicating that the iteration is complete. In a for loop, the loop ends normally without handling the stopiteration exception.

We can draw the following conclusions:

A function with yield is a generator, which is different from a common function. Generating a generator looks like a function call, but does not execute any function code until it calls next () (next () is automatically called in the for loop to start execution. Although the execution process is still executed according to the function process, every execution of a yield statement is interrupted and an iteration value is returned. The next statement of yield continues to be executed during the next execution. It seems that a function is interrupted several times by yield during normal execution, and the current iteration value is returned through yield for each interruption.

The advantage of yield is obvious. By Rewriting a function as a generator, the iteration capability is achieved. Compared to calculating the next () value by using the Instance Storage status of the class, the Code is not only concise, the execution process is clear.

How can I determine whether a function is a special generator function? You can use isgeneratorfunction to determine:

Listing 7. Use isgeneratorfunction to judge

>>> From inspect import isgeneratorfunction >>> Isgeneratorfunction (FAB) True

Note that FAB and FAB (5) are distinguished. fab is a generator function, and FAB (5) is a generator returned by calling FAB, like the difference between class definition and class instance:

Listing 8. Class Definition and class instance

>>> Import types >>> Isinstance (FAB, types. generatortype) False >>> Isinstance (FAB (5), types. generatortype) True

Fab cannot be iterated, While FAB (5) can be iterated:

>>> From collections import iterable >>> Isinstance (FAB, iterable) False >>> Isinstance (FAB (5), iterable) True

Each time you call the FAB function, a new generator instance is generated, and each instance has no influence on each other:

>>> F1 = FAB (3) >>> F2 = FAB (5) >>> Print 'f1: ', f1.next () F1: 1 >>> Print 'f2: ', f2.next () F2: 1 >>> Print 'f1: ', f1.next () F1: 1 >>> Print 'f2: ', f2.next () F2: 1 >>> Print 'f1: ', f1.next () F1: 2 >>> Print 'f2: ', f2.next () F2: 2 >>> Print 'f2: ', f2.next () F2: 3 >>> Print 'f2: ', f2.next () F2: 5

Function of return

In a generator function, if there is no return, it is executed until the function is complete by default. If return is executed, the stopiteration is directly thrown to terminate the iteration.

Another example

Another yield example is from File Reading. If you call the read () method directly on a file object, unpredictable memory usage may occur. A good way is to use a fixed-length buffer to constantly read the file content. With yield, you can easily read files without having to write iteration classes for reading files:

Listing 9. Another yield example

Def read_file (fpath ): Blocking _size = 1024 With open (fpath, 'rb') as F: While true: Block = f. Read (block_size) If block: Yield Block Else: Return

The above briefly introduces the basic concepts and usage of yield. yield has more powerful usage in Python 3, which will be discussed in subsequent articles.

Note: All the code in this article has been debugged in Python 2.7

References

Learning

• Stay tuned to developerworks technical events and network broadcasts.
• Visit developerworks Open Source
  The zone provides a wealth of how-to information, tools and project updates, as well as the most popular articles and Tutorials that help you develop with open source code technology and use them with IBM products.

Discussion

• Join developerworks
  The Chinese community, the developerworks community is a professional social networking community that targets global IT professionals and provides community functions such as blogs, bookmarks, wiki, groups, contacts, sharing and collaboration.
• Join IBM
  Software downloads and technical exchange groups to participate in online communication.

About the author

Liao Xuefeng is proficient in Java/Java EE/Java me/Android/Python/C #/Visual Basic and has in-depth research on open-source frameworks, author of spring 2.0 core technology and best practices book, created with open source framework jopenid, its official blog is http://www.liaoxuefeng.com/and http://michael-liao.appspot.com /.