Analysis on the use of Python yield

Analysis on the use of Python yield

This article mainly introduces how to use Python yield. This article provides multiple examples to analyze how to use yield. For more information, see

Developers who are new to Python often find that the yield keyword is used in many Python functions. However, the execution process of functions with yield is different from that of common functions. What is yield used for? Why design yield? This article will explain the concept and usage of yield in a simple and in-depth way, and help readers understand the simple and powerful yield functions in Python.

You may have heard that functions with yield are called generator (generator) in Python and what is generator?

Let's move away from generator to present the concept of yield with a common programming question.

How to generate a Fibonacci data column

The Fibonacci data column is a simple recursive series. Except for the first and second numbers, any number can be obtained by adding the first two numbers. Using a computer program to output the first N of the Fibonacci number column is a very simple problem. Many beginners can easily write the following functions:

Listing 1. Simple output of the first N of the Fibonacci number column

The Code is as follows:

Def fab (max ):

N, a, B = 0, 0, 1

While n <max:

Print B

A, B = B, a + B

N = n + 1

Run fab (5) and we can get the following output:

The Code is as follows:

>>> Fab (5)

1

1

2

3

5

There is no problem with the results, but experienced developers will point out that printing numbers with print in the fab function will lead to poor reusability of the function, because the fab function returns None, other functions cannot obtain the sequence generated by this function.

To improve the reusability of fab functions, it is best not to print the series directly, but to return a List. The second version after the fab function is rewritten is as follows:

Listing 2. output the second version of the first N of the Fibonacci number column

The Code is as follows:

Def fab (max ):

N, a, B = 0, 0, 1

L = []

While n <max:

L. append (B)

A, B = B, a + B

N = n + 1

Return L

You can print out the List returned by the fab function as follows:

The Code is as follows:

>>> For n in fab (5 ):

... Print n

...

1

1

2

3

5

The rewritten fab function can meet the reusability requirement by returning the List, but more experienced developers will point out that the memory occupied by the function during running will increase with the increase of the max parameter, if you want to control memory usage, it is best not to use List

To save the intermediate results, but to iterate through the iterable object. For example, in Python2.x, the Code is as follows:

Listing 3. Iteration through the iterable object

The Code is as follows:

For I in range (0, 1000): pass

This will generate a List of 1000 elements, and the code:

The Code is as follows:

For I in xrange (0, 1000): pass

Instead of generating a List of 1000 elements, the next value is returned in each iteration, and the memory space is very small. Xrange does not return List, but returns an iterable object.

With iterable, we can rewrite the fab function into a class that supports iterable. below is the third version of Fab:

Listing 4. Third version

The Code is as follows:

Class Fab (object ):

Def _ init _ (self, max ):

Self. max = max

Self. n, self. a, self. B = 0, 0, 1

Def _ iter _ (self ):

Return self

Def next (self ):

If self. n <self. max:

R = self. B

Self. a, self. B = self. B, self. a + self. B

Self. n = self. n + 1

Return r

Raise StopIteration ()

The Fab class continuously returns the next number of columns through next (), and the memory usage is always constant:

The Code is as follows:

>>> For n in Fab (5 ):

... Print n

...

1

1

2

3

5

However, the code of this version rewritten by class is far less concise than the fab function of the first version. If we want to keep the simplicity of the first version of the fab function and get the iterable effect, yield will come in handy:

Listing 5. Use version 4 of yield

The Code is as follows:

Def fab (max ):

N, a, B = 0, 0, 1

While n <max:

Yield B

# Print B

A, B = B, a + B

N = n + 1

'''

Compared with the first version, fab of the fourth version only changes print B to yield B, which achieves the iterable effect while keeping it simple.

Calling the fourth edition of fab is exactly the same as the second edition of fab:

The Code is as follows:

>>> For n in fab (5 ):

... Print n

...

1

1

2

3

5

In short, yield is used to convert a function into a generator. A function with yield is no longer a common function. The Python interpreter regards it as a generator and calls fab (5) instead of executing the fab function, an iterable object is returned! During for loop execution, code in the fab function is executed in each loop. When yield B is executed, the fab function returns an iteration value. During the next iteration, the Code continues to run from the next statement of yield B, and the local variable of the function looks exactly the same as before the last interrupted execution, so the function continues to run until yield is encountered again.

You can also manually call the next () method of fab (5) (because fab (5) is a generator object and this object has the next () method ), in this way, we can see the fab execution process more clearly:

Listing 6. Execution Process

The Code is as follows:

>>> F = fab (5)

>>> F. next ()

1

>>> F. next ()

1

>>> F. next ()

2

>>> F. next ()

3

>>> F. next ()

5

>>> F. next ()

Traceback (most recent call last ):

File" ", Line 1, in

StopIteration

When the function execution ends, generator automatically throws a StopIteration exception, indicating that the iteration is complete. In a for loop, the loop ends normally without handling the StopIteration exception.

We can draw the following conclusions:

A function with yield is a generator, which is different from a common function. Generating a generator looks like a function call, but does not execute any function code until it calls next () (next () is automatically called in the for loop to start execution. Although the execution process is still executed according to the function process, every execution of a yield statement is interrupted and an iteration value is returned. The next statement of yield continues to be executed during the next execution. It seems that a function is interrupted several times by yield during normal execution, and the current iteration value is returned through yield for each interruption.

The advantage of yield is obvious. By Rewriting a function as a generator, the iteration capability is achieved. Compared to calculating the next () value by using the Instance Storage status of the class, the Code is not only concise, the execution process is clear.

How can I determine whether a function is a special generator function? You can use isgeneratorfunction to determine:

Listing 7. Use isgeneratorfunction to judge

The Code is as follows:

>>> From inspect import isgeneratorfunction

>>> Isgeneratorfunction (fab)

True

Note that fab and fab (5) are distinguished. fab is a generator function, and fab (5) is a generator returned by calling fab, like the difference between class definition and class instance:

Listing 8. Class Definition and class instance

The Code is as follows:

>>> Import types

>>> Isinstance (fab, types. GeneratorType)

False

>>> Isinstance (fab (5), types. GeneratorType)

True

Fab cannot be iterated, While fab (5) can be iterated:

The Code is as follows:

>>> From collections import Iterable

>>> Isinstance (fab, Iterable)

False

>>> Isinstance (fab (5), Iterable)

True

Each time you call the fab function, a new generator instance is generated, and each instance has no influence on each other:

The Code is as follows:

>>> F1 = fab (3)

>>> F2 = fab (5)

>>> Print 'f1: ', f1.next ()

F1: 1

>>> Print 'f2: ', f2.next ()

F2: 1

>>> Print 'f1: ', f1.next ()

F1: 1

>>> Print 'f2: ', f2.next ()

F2: 1

>>> Print 'f1: ', f1.next ()

F1: 2

>>> Print 'f2: ', f2.next ()

F2: 2

>>> Print 'f2: ', f2.next ()

F2: 3

>>> Print 'f2: ', f2.next ()

F2: 5

Function of return

In a generator function, if there is no return, it is executed until the function is complete by default. If return is executed, the StopIteration is directly thrown to terminate the iteration.

Another example

Another yield example is from File Reading. If you call the read () method directly on a file object, unpredictable memory usage may occur. A good way is to use a fixed-length buffer to constantly read the file content. With yield, you can easily read files without having to write iteration classes for reading files:

Listing 9. Another yield example

The Code is as follows:

Def read_file (fpath ):

Blocking _size = 1024

With open (fpath, 'rb') as f:

While True:

Block = f. read (BLOCK_SIZE)

If block:

Yield block

Else:

Return

The above briefly introduces the basic concepts and usage of yield. yield has more powerful usage in Python 3, which will be discussed in subsequent articles.

Note: All the code in this article has been debugged in Python 2.7