How about Python iterators and generators?

How about Python iterators and generators?

An iterator is a way to access collection elements. The iterator object is accessed from the first element of the set until all elements are accessed. The iterator can only move forward without moving back, but there is nothing to do with it, because people seldom go back on the way of iteration.

Advantages of using iterator

For data structures (such as tuple and list) that support random access in the native, The iterator has no advantage over index access in the classic for loop, instead, the index value is lost (you can use the built-in function enumerate () to retrieve the index value ). However, for data structures (such as set) that cannot be accessed randomly, the iterator is the only way to access elements.

In addition, the major advantage of the iterator is that it is not required to prepare all elements in the entire iteration process in advance. The iterator calculates an element only when it iterates to an element. Before or after this, the element may not exist or be destroyed. This feature makes it especially suitable for Traversing large or infinite sets, such as several G files or Fibonacci series.

The primary benefit of the iterator is that it provides a unified interface for accessing the set. As long as the _ iter _ () method object is defined, the iterator can be used for access.

The iterator has two basic methods:

Next method: return the next element of the iterator.

_ Iter _ method: return the iterator object itself

I. Iterators

The iterator is only a container object and implements the iterator protocol. It has two basic methods:

1) next Method

Returns the next element of the container.

2) _ iter _ Method

Return iterator itself

The iterator can be created using the built-in iter method. See the example below:

>>> I = iter ('abc ')

>>> I. next ()

'A'

>>> I. next ()

'B'

>>> I. next ()

'C'

>>> I. next ()

Traceback (most recent call last ):

File "", line 1, in

StopIteration:

Class MyIterator (object ):

Def _ init _ (self, step ):

Self. step = step

Def next (self ):

"Returns the next element ."""

If self. step = 0:

Raise StopIteration

Self. step-= 1

Return self. step

Def _ iter _ (self ):

"Returns the iterator itself ."""

Return self

For el in MyIterator (4 ):

Print el

--------------------

Result:

3

2

1

0

Ii. Generators

Since Python2.2, the generator provides a simple way to return functions of list elements to complete simple and effective code.

It allows you to stop a function and return results immediately based on the yield command.

This function saves the execution context. If necessary, you can continue execution immediately.

For example, the Fibonacci function:

Def maid ():

A, B = 0, 1

While True:

Yield B

A, B = B, a + B

Fib = maid ()

Print fib. next ()

Print fib. next ()

Print fib. next ()

Print [fib. next () for I in range (10)]

--------------------

Result:

1

1

2

[3, 5, 8, 13, 21, 34,55, 89,144,233]

PEP Python Enhancement Proposal Python Enhancement suggestions

Tokenize Module

>>> Import tokenize

>>> Reader = open ('C:/temp/py1.py'). next

>>> Tokens = tokenize. generate_tokens (reader)

>>> Tokens. next ()

(1, 'class', (1, 0), (1, 5), 'class MyIterator (object):/N ')

>>> Tokens. next ()

(1, 'myiterator', (1, 6), (1, 16), 'class MyIterator (object):/N ')

>>> Tokens. next ()

(51, '(', (1, 16), (1, 17), 'class MyIterator (object):/N ')

Example:

Def power (values ):

For value in values:

Print 'powering % s' % value

Yield value

Def adder (values ):

For value in values:

Print 'adding to % s' % value

If value % 2 = 0:

Yield value + 3

Else:

Yield value + 2

Elements = [,]

Res = adder (power (elements ))

Print res. next ()

Print res. next ()

--------------------

Result:

Powering 1

Adding to 1

3

Powering 4

Adding to 4

7

Keep code simple, not data.

Note: You would rather have a large number of simple iteratable functions than a complex function that calculates only one value at a time.

Example:

Def psychologist ():

Print 'Please tell me your problems'

While True:

Answer = (yield)

If answer is not None:

If answer. endswith ('? '):

Print ("Don't ask yourself too much questions ")

Elif 'good' in answer:

Print "A that's good, go on"

Elif 'bad' in answer:

Print "Don't be so negative"

Free = psychologist ()

Print free. next ()

Print free. send ('I feel bad ')

Print free. send ("Why I shouldn't? ")

Print free. send ("OK then I shocould find what is good for me ")

--------------------

Result:

Please tell me your problems

None

Don't be so negative

None

Don't ask yourself too much questions

None

A that's good, go on

None

Although I have been familiar with the word yield for a long time, it has always been a bit confusing. Take the time to study it.

A function containing yield indicates that it is a generator rather than a common function. When the program runs on the yield Line, the function returns the value and stores the State of all the variables in the current domain. When the function is called next time, it is executed from the last interrupted place, the next yield is always encountered, the program returns the value, and saves the current state here; this is repeated until the function is executed normally.

At the beginning, I still wondered how the function stack between the caller and the generator was implemented. Later I realized that the principle of 'corout' was used. Coroutine can be viewed as a micro-thread. The following uses examples to illustrate the yield and coroutine running process. Assume that the test method is defined:

[Python]

Def test (len ):

I = 0

While I <len:

Yield I

I + = 1

Let's call it to see the output:

>>> For I in test (5 ):

Print I

Output:

0

1

2

3

4

This scenario is similar to for I in xrange (len); yes, xrange does. The for... in operation actually calls the next () method of the generator. The above call process can be equivalent:

[Python]

F = test (5)

Print f. next ()

Print f. next ()

Print f. next ()

Print f. next ()

Print f. next ()

The output result is consistent with the previous output.

In addition, during this call, the coroutine was created once, awakened five times (through next), suspended five times (through yield), and finally exited and destroyed. These are some of the points, so I have a deeper understanding of them and I will try again.

Generator)

If the list elements can be calculated by some algorithm, can we continue to calculate the subsequent elements in the loop process? In this way, you do not need to create a complete list to save a lot of space. In Python, this type of computing mechanism is called a Generator ).

There are many ways to create a generator. The first method is very simple. You only need to change [] of a list generator type to (), and a generator is created:

>>> L = [x * x for x in range (10)]

>>> L

[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

>>> G = (x * x for x in range (10 ))

>>> G

At 0x0000feab40>

The difference between L and g is that the [] and () of the outermost layer, L is a list, and g is a generator.

We can print every element of the list directly, but how can we print every element of generator?

To print them one by one, you can use the next () method of generator:

>>> G. next ()

0

>>> G. next ()

1

>>> G. next ()

4

>>> G. next ()

9

>>> G. next ()

16

>>> G. next ()

25

>>> G. next ()

36

>>> G. next ()

49

>>> G. next ()

64

>>> G. next ()

81

>>> G. next ()

Traceback (most recent call last ):

File "", line 1, in

StopIteration

As we have mentioned, generator stores algorithms. Every time next () is called, the value of the next element is calculated until the last element is computed and no more elements exist, throw the StopIteration error.

Of course, the above constant call to the next () method is too abnormal. The correct method is to use the for loop, because the generator is also an object that can be iterated:

>>> G = (x * x for x in range (10 ))

>>> For n in g:

... Print n

...

0

1

4

9

16

25

36

49

64

81

Therefore, after we create a generator, we will never call the next () method, but iterate it through the for loop.

Generator is very powerful. If the calculation algorithm is complex and cannot be implemented using a for loop similar to the list generation type, you can also use functions.

For example, in the famous onacci series, except for the first and second numbers, any number can be obtained by adding the first two numbers:

1, 1, 2, 3, 5, 8, 13, 21, 34 ,...

The Fibonacci series cannot be written in the form of list generation, but it is easy to print it out using functions:

Def fib (max ):

N, a, B = 0, 0, 1

While n <max:

Print B

A, B = B, a + B

N = n + 1

The above function can output the first N of the Fibonacci series:

>>> Fib (6)

1

1

2

3

5

8

After careful observation, we can see that the fib function actually defines the calculation rule of the Fibonacci series. It can start from the first element and calculate any subsequent elements. This logic is very similar to generator.

That is to say, the above functions and generator are only one step away. To change the fib function to generator, you only need to change print B to yield B:

Def fib (max ):

N, a, B = 0, 0, 1

While n <max:

Yield B

A, B = B, a + B

N = n + 1

This is another method for defining generator. If a function definition contains the yield keyword, this function is no longer a common function, but a generator:

>>> Fib (6)

The most difficult thing to understand here is that the execution process of generator and function is different. The function is executed sequentially. If a return statement or the last row of the function statement is returned. The generator function is executed every time next () is called. When the yield statement is returned, the yield statement returned last time is executed again.

For example, define a generator and return numbers 1, 3, and 5 in sequence:

>>> Def odd ():

... Print 'step 1'

... Yield 1

... Print 'step 2'

... Yield 3

... Print 'step 3'

... Yield 5

...

>>> O = odd ()

>>> O. next ()

Step 1

1

>>> O. next ()

Step 2

3

>>> O. next ()

Step 3

5

>>> O. next ()

Traceback (most recent call last ):

File "", line 1, in

StopIteration

We can see that odd is not a common function, but a generator. During the execution process, yield is interrupted and the next execution will continue. After three times of yield execution, no yield can be executed. Therefore, if you call next () for the first time, an error is returned.

Back to the fib example, we call yield continuously in the loop process, and it will be interrupted. Of course, you must set a condition for the loop to exit the loop. Otherwise, an infinite number of columns will be generated.

Similarly, after the function is changed to generator, we basically never use next () to call it, but directly use the for loop for iteration:

>>> For n in fib (6 ):

... Print n

...

1

1

2

3

5

8

Summary

Generator is a very powerful tool. In Python, you can simply change the list generator type to generator, or implement a complex logic generator through functions.

To understand the working principle of generator, it is to constantly calculate the next element in the for loop process and end the for loop with appropriate conditions. For the generator to which the function is changed, if you encounter a return statement or execute the last row of the function body, the command of the generator is ended and the for loop ends.