A deep understanding of the Python Generator (Generator)

With a deep understanding of the Python Generator (Generator), we can simply create a list using the list generation method, but the list capacity is limited due to memory restrictions. In addition, creating a list containing 1 million elements not only occupies a large storage space, but if we only need to access the first few elements, the space occupied by the vast majority of elements is wasted.

Therefore, if the list elements can be calculated by some algorithm, can we continue to calculate the subsequent elements in the loop process? In this way, you do not need to create a complete list to save a lot of space. In Python, this type of computing mechanism is called a Generator ).

There are many ways to create a generator. The first method is very simple. you only need to change [] of a list generator type to (), and a generator is created:

>>> mylist = [ x for x in range(1, 10)]>>> mylist[1, 2, 3, 4, 5, 6, 7, 8, 9]>>> gen = (x for x in range(1,10))>>> gen
 
   at 0x7f1d7fd0f5a0>
 

The difference between mylist and gen is that the [] and () of the outermost layer, mylist is a list, and gen is a generator (generator ).

We can print every element of the list directly, but how can we print every element of generator?

To print them one by one, you can use the next () method of generator:

>>> gen.next()1>>> gen.next()2>>> gen.next()3...>>> gen.next()9>>> gen.next()Traceback (most recent call last):  File "
 
  ", line 1, in 
  
   StopIteration
  
 

As we have mentioned, generator stores algorithms. every time next () is called, the value of the next element is calculated until the last element is computed and no more elements exist, throw the StopIteration error.

In fact, we can use the for loop instead of the next () method to better adapt to the efficient programming ideas:

>>> gen = ( x for x in range(1, 10))>>> for num in gen:...     print num... 123456789

Generator is very powerful. If the calculation algorithm is complex and cannot be implemented using a for loop similar to the list generation type, you can also use functions.

For example, in the famous onacci series, except for the first and second numbers, any number can be obtained by adding the first two numbers:

1, 1, 2, 3, 5, 8, 13, 21, 34, ...

The Fibonacci series cannot be written in the form of list generation, but it is easy to print it out using functions:

def fib(max):    n = 0     a, b = 0, 1    while n < max:        print b        a, b = b, a + b        n = n + 1

The above function can output the first N of the Fibonacci series:

>>> fib(6)112358

After careful observation, we can see that the fib function actually defines the calculation rule of the Fibonacci series. it can start from the first element and calculate any subsequent elements. this logic is very similar to generator.

That is to say, the above functions and generator are only one step away. To change the fib function to generator, you only need to change print B to yield B:

def fib(max):    n = 0     a, b = 0, 1    while n < max:        yield b        a, b = b, a + b        n = n + 1

This is another method for defining generator. If a function definition contains the yield keyword, this function is no longer a common function, but a generator:

>>> fib(6)
 

The most difficult thing to understand here is that the execution process of generator and function is different. The function is executed sequentially. if a return statement or the last row of the function statement is returned. The generator function is executed every time next () is called. when the yield statement is returned, the yield statement returned last time is executed again.

For example, define a generator and return numbers 1, 3, and 5 in sequence:

>>> def odd():...     print 'step 1'...     yield 1...     print 'step 2'...     yield 3...     print 'step 3'...     yield 5...>>> o = odd()>>> o.next()step 11>>> o.next()step 23>>> o.next()step 35>>> o.next()Traceback (most recent call last):  File "
 
  ", line 1, in 
  
   StopIteration
  
 

We can see that odd is not a common function, but a generator. during the execution process, yield is interrupted and the next execution will continue. After three times of yield execution, no yield can be executed. Therefore, if you call next () for the first time, an error is returned.

Back to the fib example, we call yield continuously in the loop process, and it will be interrupted. Of course, you must set a condition for the loop to exit the loop. Otherwise, an infinite number of columns will be generated.

Similarly, after the function is changed to generator, we basically never use next () to call it, but directly use the for loop for iteration:

>>> for n in fib(6):...     print n...112358

Generator is a very powerful tool. in Python, you can simply change the list generator type to generator, or implement a complex logic generator through functions.

To understand the working principle of generator, it is to constantly calculate the next element in the for loop process and end the for loop with appropriate conditions. For the generator to which the function is changed, if you encounter a return statement or execute the last row of the function body, the command of the generator is ended and the for loop ends.