Python Functional Programming Guide (III): An explanation of an iterator

3. Iterators

3.1. iterators (Iterator) Overview

Iterators are a way to access elements within a collection. The iterator object is accessed from the first element of the collection until all the elements have been accessed one after the other.

Iterators cannot be rolled back and can only be iterated forward. This is not a big drawback because there is little need for a fallback in the middle of an iteration.

Iterators are also not thread-safe, and using iterators for mutable collections in a multithreaded environment is a risky operation. But it's not a big problem if you're cautious, or simply implement functional thinking and stick with immutable collections.

For primitive data structures that support random access (such as tuple, list), there is no advantage over the index access of the iterator and the classic for loop, but the index value is lost (you can use the built-in function enumerate () to retrieve the index value, which is something). But for data structures that cannot be accessed randomly (such as set), iterators are the only way to access elements.

Another advantage of iterators is that it does not require you to prepare all the elements of the entire iteration in advance. An iterator computes an element only when it iterates over it, and before or after that, the element may not exist or be destroyed. This feature makes it particularly useful for traversing large or infinite collections, such as several G files, or Fibonacci sequences, and so on. This feature is called lazy evaluation or lazy evaluation.

The greater credit for iterators is the interface that provides a unified access to the collection. As long as the object that implements the __iter__ () method is implemented, it can be accessed using an iterator.

3.2. Using iterators

Use the built-in factory function iter (iterable) to get an iterator object:

The code is as follows:

>>> LST = range (2)
>>> it = iter (LST)
>>> it

The next element can be accessed using the next () method of the iterator:

The code is as follows:

>>> It.next ()
0

In the case of Python 2.6+, there is also the built-in function next (iterator) to accomplish this function:

The code is as follows:

>>> Next (IT)
1

How can I tell if an iterator has more elements to access? The iterator in Python does not provide a method like Has_next ().
So in this example, what happens if we have access to the last element 1 and then the next () method?

The code is as follows:

>>> It.next ()
Traceback (most recent):
File " ", line 1, in
Stopiteration

Python throws a stopiteration exception when it encounters such a situation. In fact, Python determines whether to stop iterating based on whether the exception is checked.
This approach has advantages over whether the Manual check is out of bounds before the iteration. But Python's approach always has some suspicion of using exceptions for Process control.

With this in view, we are able to iterate with iterators.

The code is as follows:

it = iter (LST)
Try
While True:
val = It.next ()
Print Val
Except stopiteration:
Pass

In fact, because iterative operations are so common, python specifically uses the keyword for as the syntax sugar for iterators. In the For loop, Python automatically calls the Factory function iter () to get the iterator, automatically calls next () to get the element, and completes the work of checking the stopiteration exception. The above code can be written in the following form, you must be very familiar with:

The code is as follows:

For Val in LST:
Print Val

First Python will call the ITER function to get the iterator for the keyword in after object, and then call the iterator's next method to get the element until the stopiteration exception is thrown. When you call the ITER function on an iterator, the iterator itself is returned, so the iterator can also be used in the For statement without special handling.

Several commonly used built-in data structures tuple, list, set, dict all support iterators, and strings can also use iterative operations. You can also implement an iterator yourself, as described above, by simply returning an object in the class's __iter__ method, which has a next () method that throws the stopiteration exception at the appropriate time. But there are few times when you need to implement an iterator yourself, and even if you need it, it's easier to use the generator. Next we will discuss the parts of the generator.

* Exceptions are not non-throwing, iterators that do not throw the exception will iterate indefinitely, and in some cases such iterators can be useful. In this case, you need to judge the element and abort it, otherwise it will be dead loop!

Loops with Iterators can bypass the index, but sometimes we still need the index to do something. This time the built-in function enumerate comes in handy, it can be indexed before the results of the ITER function, returned in tuples, and used like this:

The code is as follows:

For IDX, ele in Enumerate (LST):
Print idx, ele

3.3. Generator expressions (Generator expression) and list parsing (lists comprehension)
In most cases, traversing a collection is done to apply an action to an element or to filter. If you read the second part of this article, you should remember that the built-in function map and filter provide these features, but Python still provides language-level support for these operations.

The code is as follows:

(X+1 for x in LST) #生成器表达式, returns an iterator. External parentheses can be omitted when used with parameters.
[X+1 for x in LST] #列表解析, return to list

As you can see, generator expressions and list parsing (note: There are many kinds of translations here, such as list expansion, List derivation, etc., refers to the same meaning) the difference is very small, so when people mention this feature, the simplicity is often only described as a list of parsing. However, since the return iterator does not compute all the elements at the outset, it gives you more flexibility and avoids many unnecessary calculations, so you should always use a builder expression unless you explicitly want to return the list. In the following text I will not distinguish between the two forms:)

You can also filter for list parsing by providing an IF clause:

The code is as follows:

(X+1 for x in LST if x!=0)

or provide multiple for clauses for nested loops, which are the order of the FOR clauses:

The code is as follows:

((x, y) for x in range (3) for Y in range (x))

List parsing is a stark pythonic. I often encounter two problems using list parsing, which should be attributed to best practices, but these two issues are typical, so here's what you might want to mention:

The first problem is that because the action applied to an element is too complex to be written out with an expression, list parsing is not used. This is a typical example of a thought that doesn't change, and if we encapsulate the action as a function, isn't that an expression?

The second problem is that because the conditions in the IF clause need to be computed, and the result needs to be calculated the same, you do not want to calculate two times, like this:

The code is as follows:

(X.dosomething () for x in LST if x.dosomething () >0)

This is really bad writing, but a combination of list resolution can be solved:

The code is as follows:

(X for X in (Y.dosomething () to Y in lst) if x>0)

The internal list parsing variable can actually be used with X, but for clarity we change to Y. Or more clearly, you can write two expressions:

The code is as follows:

TMP = (x.dosomething () for x in LST)
(x for X in TMP if x > 0)

List parsing can replace the vast majority of situations where map and filter are needed, perhaps because of this, the famous static check tool pylint the use of map and filter as a warning.

3.4. Related libraries

Python has a built-in module Itertools, which contains a number of functions for the creating iterators for efficient looping (creating a more efficient loop iterator), which means it's domineering, This section will go through these functions and make an impression, and when you need them, vaguely remember that there is a good. The content of this section is translated from the official documentation of the Itertools module.

3.4.1. Infinite iterations

Count (start, [step])

Starting with start, each element is added with step. The step default value is 1.
Count---10 11 12 13 14 ...

Cycle (P)

After iterating to the last element of the sequence p, start again from the first element of P.
Cycle (' ABCD ')--a B c d A b c D ...

Repeat (Elem [, N])

Repeats the Elem n times. If n is not specified, the infinite repeats.
Repeat (3)--10 10 10

3.4.2. Stopping iterations when the shortest sequence parameter is terminated
chain (p, Q, ...)
After iterating to the last element of the sequence p, start with the first element of Q until all the sequences are terminated.
Chain (' ABC ', ' DEF ')--A B C D E F
Compress (data, selectors)
if BOOL (Selectors[n]) is true, Next () returns DATA[N], otherwise skips data[n].
Compress (' ABCDEF ', [1,0,1,0,1,1])--A C E F
dropwhile (pred, seq)
The iteration is only started when the call of Pred to Seq[n] returns false.
Dropwhile (Lambda x:x<5, [1,4,6,4,1])--6 4 1
takewhile (pred, seq)
The opposite version of Dropwhile.
TakeWhile (Lambda x:x<5, [1,4,6,4,1])--1 4
IFilter (pred, seq)
The iterator version of the built-in function filter.
IFilter (Lambda x:x%2, Range)--1 3 5 7 9
ifilterfalse (pred, seq)
The opposite version of IFilter.
Ifilterfalse (Lambda x:x%2, Range)--0 2 4 6 8
IMAP (func, p, Q, ...)
The iterator version of the built-in function map.
IMAP (POW, (2,3,10), (5,2,3))--32 9 1000
Starmap (func, seq)
Invokes func for each element of the SEQ in the form of a variable-length parameter (*args).
Starmap (POW, [(2,5), (3,2), (10,3)])--32 9 1000
Izip (P, Q, ...)
The iterator version of the built-in function zip.
Izip (' ABCD ', ' xy ')--Ax by
Izip_longest (P, q, ..., fillvalue=none)
Izip takes the longest sequence of versions, and the short sequence is filled with fillvalue.
Izip_longest (' ABCD ', ' xy ', fillvalue= '-')--Ax by c-d-
Tee (it, n)
Returns a replication iterator for n iterator it.
GroupBy (iterable[, Keyfunc])
This function functions like a grouping of SQL. Before using GroupBy, you first need to sort the iterable using the same keyfunc, such as calling the built-in sorted function. Then, GroupBy returns the iterator, and each iteration of the element is a tuple (a key value, a sub-iterator to the collection of elements in the iterable that has the same key value). Perhaps a look at Python's sort guide is helpful in understanding this function.
GroupBy ([0, 0, 0, 1, 1, 1, 2, 2, 2])--(0, (+ 0 0 0)) (1, (+ 1 1 1)) (2, (2 2 2))

3.4.3. Composite iterators
Product (p, q, ...) [Repeat=1])
Cartesian product.
Product (' ABCD ', repeat=2)--AA AB AC AD BA BB BC BD CA CB CC CD DA DB DC DD
Permutations (p[, R])
Removes the duplicate element.
Permutations (' ABCD ', 2)--AB AC AD BA BC BD CA CB CD DA DB DC
Combinations (p, R)
Removes duplicate elements after sorting.
Combinations (' ABCD ', 2)--AB AC AD BC BD CD
Combinations_with_replacement ()
After sorting, contains repeating elements.
Combinations_with_replacement (' ABCD ', 2)--AA AB AC AD BB BC BD CC CD DD
The end of this article.

