Python function Programming Guide (3): iterator details

This article mainly introduces the Python function Programming Guide (3): Iterator details. This article describes the Iterator overview, the use of Iterator, generator expression (Generatorexpression) and ListComprehension. For more information, see 3. iterator

3.1. Iterator overview

An iterator is a way to access elements in a set. The iterator object is accessed from the first element of the set until all elements are accessed.

The iterator cannot be rolled back and can only be iterated forward. This is not a big drawback, because people almost do not need to perform rollback during iteration.

The iterator is not thread-safe. it is a dangerous operation to use the iterator for variable sets in a multi-threaded environment. However, if you are cautious, or simply stick to the immutable set by implementing functional thinking, that is not a big problem.

For data structures (such as tuple and list) that support random access in the native, the iterator has no advantage over Index access in the classic for loop, instead, the index value is lost (you can use the built-in function enumerate () to retrieve the index value ). However, for data structures (such as set) that cannot be accessed randomly, the iterator is the only way to access elements.

Another advantage of the iterator is that it does not require you to prepare all the elements in the entire iteration process in advance. The iterator calculates the element only when it iterates to an element. before or after this, the element may not exist or be destroyed. This feature makes it especially suitable for traversing large or infinite sets, such as several G files or Fibonacci series. This feature is called Lazy evaluation ).

The primary benefit of the iterator is that it provides a unified interface for accessing the set. You can use the iterator to access the object that implements the _ iter _ () method.

3.2. use the iterator

You can use the built-in factory function iter (iterable) to obtain the iterator object:

The code is as follows:

>>> Lst = range (2)
>>> It = iter (lst)
>>> It

You can use the next () method of the iterator to access the next element:

The code is as follows:

>>> It. next ()
0

For Python 2.6 +, there is also the built-in function next (iterator) to complete this function:

The code is as follows:

>>> Next (it)
1

How can we determine whether the iterator has more elements to access? The iterator in Python does not provide methods like has_next.
In this example, we have accessed the last element 1. what if we use the next () method?

The code is as follows:

>>> It. next ()
Traceback (most recent call last ):
File" ", Line 1, in
StopIteration

When Python encounters such a situation, it will throw a StopIteration exception. In fact, Python determines whether to stop iteration based on whether this exception is detected.
This method has advantages over manual checks before iteration. However, Python is always suspected of using exceptions for process control.

After learning about these situations, we can use the iterator to traverse them.

The code is as follows:

It = iter (lst)
Try:
While True:
Val = it. next ()
Print val
Optional T StopIteration:
Pass

In fact, because iterative operations are so common, Python uses the keyword for as the syntactic sugar of the iterator. In the for loop, Python will automatically call the factory function iter () to obtain the iterator, call next () to obtain the element, and check for StopIteration exceptions. The above code can be written as follows, so you must be familiar with it:

The code is as follows:

For val in lst:
Print val

First, Python will call the iter function for the object after the keyword in to obtain the iterator, and then call the next method of the iterator to obtain the element until a StopIteration exception is thrown. When the iter function is called for the iterator, the iterator itself is returned. Therefore, the iterator can also be used in the for statement without special processing.

Several commonly used built-in data structures, such as tuple, list, set, and dict, support iterators. character strings can also be iterated. You can also implement an iterator by yourself. as mentioned above, you only need to return an object in the _ iter _ method of the class. this object has a next () method, this method can throw a StopIteration exception when appropriate. However, there is not much time to implement the iterator by yourself. it is easier to use the generator even if needed. In the next article, we will discuss the generator section.

* Exceptions are not non-throws. the iterator that does not throw the exception performs infinite iterations. in some cases, such an iterator is useful. In this case, you need to determine the element and stop it. Otherwise, it will be an endless loop!

Using the iterator loop can avoid indexes, but sometimes we still need indexes for some operations. At this time, the built-in function enumerate will be used. it can add an index before the result of the iter function and return it as a tuples. it is used like this:

The code is as follows:

For idx, ele in enumerate (lst ):
Print idx, ele

3.3. Generator expression and List Comprehension)
In most cases, traversing a set is to apply an action to an element or filter it. If you have read the second part of this article, you should remember that the built-in functions map and filter provide these functions, but Python still provides language-level support for these operations.

The code is as follows:

(X + 1 for x in lst) # generator expression, which returns the iterator. The external brackets can be omitted when used as parameters.
[X + 1 for x in lst] # list parsing, returns list

As you can see, generator expressions and list parsing (note: There are many types of translation here, such as list expansion and list derivation, which refer to the same meaning) have little difference, when people mention this feature, it is often described as list parsing only for the sake of simplicity. However, since the returned iterator does not calculate all elements at the beginning, this gives more flexibility and avoids many unnecessary computations, therefore, unless you explicitly want to return the list, you should always use the generator expression. In the following text, I will not distinguish these two forms :)

You can also provide the if clause for list parsing for filtering:

The code is as follows:

(X + 1 for x in lst if x! = 0)

Or multiple for clauses can be provided for nested loops. the nesting order is the order of for clauses:

The code is as follows:

(X, y) for x in range (3) for y in range (x ))

List parsing is a clear Pythonic. I often encounter two problems of using list resolution, which should belong to the best practice, but these two problems are very typical, so I would like to mention them here:

The first problem is that, because the action on the element application is too complex to be written using an expression, list resolution is not used. This is a typical example of no change in thinking. if we encapsulate an action into a function, isn't it an expression?

The second problem is that, because the conditions in the if clause need to be calculated, and the results also need to be calculated twice, as shown in the following figure:

The code is as follows:

(X. doSomething () for x in lst if x. doSomething ()> 0)

This write is really bad, but it can be solved by combining the list parsing:

The code is as follows:

(X for x in (y. doSomething () for y in lst) if x> 0)

The internal list parsing variable can also use x, but for clarity, we changed to y. Or, you can write two expressions:

The code is as follows:

Tmp = (x. doSomething () for x in lst)
(X for x in tmp if x> 0)

List parsing can replace the vast majority of scenarios where map and filter are needed. For this reason, the famous static check tool pylint uses the columns of map and filter as warnings.

3.4. Related libraries

Python has a built-in module itertools, which contains many functions for creating iterators for efficient looping (creating a more efficient loop iterator), which means it is very domineering, in this section, you can browse these functions and leave an impression on them. when you need these functions, just remember them. The content of this section is translated from the official documentation of the itertools module.

3.4.1. unlimited iteration

Count (start, [step])

Start from start and add step to each element later. The default value of step is 1.
Count (10) --> 10 11 12 13 14...

Cycle (p)

After iteration to the last element of the sequence p, start from the first element of p again.
Cycle ('ABC') --> a B C D...

Repeat (elem [, n])

Repeat elem n times. If n is not specified, there are no duplicates.
Repeat (10, 3) --> 10 10 10

3.4.2. stop iteration at the end of the shortest sequence parameter
Chain (p, q ,...)
After iteration to the last element of the p sequence, start from the first element of q until all sequences are terminated.
Chain ('ABC', 'Def ') --> A B C D E F
Compress (data, selectors)
If bool (selectors [n]) is True, next () returns data [n]; otherwise, data [n] is skipped.
Compress ('abcdef', [,]) --> A C E F
Dropwhile (pred, seq)
Iteration starts only when pred calls seq [n] returns False.
Dropwhile (lambda x: x <5, [, 1]) --> 6 4 1
Takewhile (pred, seq)
The opposite version of dropwhile.
Takewhile (lambda x: x <5, [, 1]) --> 1 4
Ifilter (pred, seq)
The iterator version of the built-in function filter.
Ifilter (lambda x: x % 2, range (10) --> 1 3 5 7 9
Ifilterfalse (pred, seq)
The opposite version of ifilter.
Ifilterfalse (lambda x: x % 2, range (10) --> 0 2 4 6 8
Imap (func, p, q ,...)
The version of the iterator for the built-in function map.
Imap (pow, (2, 3, 10), (5, 2, 3) --> 32 9 1000
Starmap (func, seq)
Call func as a variable-length parameter (* args) for each element of seq.
Starmap (pow, [(2, 5), (3, 2), (10, 3)]) --> 32 9 1000
Izip (p, q ,...)
The version of the built-in function zip iterator.
Izip ('abcd', 'XY') --> Ax
Izip_longest (p, q,..., fillvalue = None)
The version of the izip that obtains the longest sequence. The short sequence is filled with fillvalue.
Izip_longest ('abcd', 'XY', fillvalue = '-') --> Ax By C-D-
Tee (it, n)
Returns the replica iterator of n iterators it.
Groupby (iterable [, keyfunc])
This function is similar to SQL grouping. Before using groupby, use the same keyfunc to sort iterable, such as calling the built-in sorted function. Then, groupby returns the iterator. the elements of each iteration are tuples (key values, subiterators of the set of elements with the same key value in iterable ). Maybe it is helpful to understand this function by looking at the Python sorting guide.
Groupby ([0, 0, 0, 1, 1, 1, 2, 2]) --> (0, (0 0 0) (1, (1 1) (2, (2 2 2 ))

3.4.3. combined iterator
Product (p, q,... [repeat = 1])
Cartesian product.
Product ('abcd', repeat = 2) --> AA AB AC AD BA BB BC BD CA CB CC CD DA DB DC DD
Permutations (p [, r])
Remove repeated elements.
Permutations ('abcd', 2) --> AB AC AD BA BC BD CA CB CD DA DB DC
Combinations (p, r)
Remove repeated elements after sorting.
Combinations ('abcd', 2) --> AB AC AD BC BD CD
Combinations_with_replacement ()
After sorting, it contains repeated elements.
Combinations_with_replacement ('abcd', 2) --> AA AB AC AD BB BC BD CC CD DD
This article ends.