Python Functional Programming Guide (III): An iterative Walkthrough

This article introduces Python functional Programming Guidelines (III): Iterator details, this article explains the iterator (iterator) overview, use iterators, builder expressions (generator expression), and list resolution (lists comprehension) And so on content, need friends can refer to the following

3. iterators

3.1. Iterator (iterator) overview

Iterators are a way to access elements within a collection. The iterator object is accessed from the first element of the collection until all the elements have been accessed once and end.

Iterators cannot be rolled back and can only be iterated forward. This is not a big drawback because people rarely need to rollback on the iteration.

Iterators are also not thread-safe, and using iterators for mutable collections in a multithreaded environment is a risky operation. But it's not a big problem if you're cautious, or simply implement a functional mindset and stick with immutable collections.

For native support random access data structures (such as tuple, list), the iterator and the classic for loop index Access does not have the advantage, but the index value is lost (you can use the built-in function enumerate () to retrieve this index value, this is something). But for data structures that cannot be randomly accessed (such as set), iterators are the only way to access elements.

Another advantage of iterators is that it does not require you to prepare all the elements of the entire iteration in advance. An iterator evaluates the element only when it is iterated to an element, and before or after that element can be either non-existent or destroyed. This feature makes it especially useful for traversing a large or infinite set, such as a few G files, or Fibonacci sequences, and so on. This feature is called deferred computation or lazy evaluation (Lazy evaluation).

The greater merit of the iterator is the provision of a unified interface to access the collection. As long as it is an object that implements the __iter__ () method, it can be accessed using an iterator.

3.2. Using iterators

Use the built-in factory function iter (iterable) to get the iterator object:

The code is as follows:

>>> LST = range (2)

>>> it = iter (LST)

>>> it

Use the next () method of the iterator to access the next element:

The code is as follows:

>>> It.next ()

0

In the case of Python 2.6+, there is built-in function next (iterator) to do this:

The code is as follows:

>>> Next (IT)

1

How can I tell if there are more elements to be accessed by iterators? The iterator in Python does not provide a method like Has_next ().

So in this case, we have access to the last element 1, and then what about using the next () method?

The code is as follows:

>>> It.next ()

Traceback (most recent call last):

File " ", line 1, in

Stopiteration

When Python encounters such a situation, it throws a Stopiteration exception. In fact, Python decides whether to stop iterating based on whether or not to check this exception.

This practice has the advantage of having to manually check for boundaries before iterating. But Python's approach always has some suspicion of using exceptions for Process control.

Once we know these things, we can iterate over them using iterators.

The code is as follows:

it = iter (LST)

Try

While True:

val = It.next ()

Print Val

Except stopiteration:

Pass

In fact, because of the ubiquity of iterations, Python specifically uses the keyword for the syntax of the iterator. In the For loop, Python automatically invokes the factory function iter () to get the iterator, automatically call next () to get the element, and also completes the work of checking stopiteration exceptions. The above code can be written in the following form, you must be very familiar with:

The code is as follows:

For Val in LST:

Print Val

First Python invokes the ITER function on the keyword in object to obtain the iterator, and then calls the iterator's next method to get the element until the stopiteration exception is thrown. When you call the ITER function on an iterator, it returns the iterator itself, so the iterator can also be used in the For statement and does not require special handling.

Several commonly used built-in data structures tuple, list, set, dict all support iterators, strings can also use the iterative operation. You can also implement an iterator yourself, as described above, by simply returning an object in the __iter__ method of the class, which has a next () method that throws the stopiteration exception at the right time. But there are not many times when you need to implement iterators yourself, and it's easier to use a builder even if you need to. We'll discuss the parts of the generator in the next article.

* An exception is not a throw, and an iterator that does not throw the exception is infinitely iterative, and in some cases such an iterator is useful. In this case, you need to judge the element and abort it, otherwise the loop will die!

Loops using Iterators can bypass the index, but sometimes we still need an index to do something. This time, the built-in function enumerate comes in handy, it can be indexed before the result of the ITER function, and returned in tuples, and used like this:

The code is as follows:

For IDX, ele in Enumerate (LST):

Print idx, ele

3.3. Builder expression (generator expression) and list resolution (listing comprehension)

In most cases, traversing a collection is all about applying an action to an element or filtering it. If you've seen the second part of this article, you should remember that built-in functions map and filter provide these features, but Python still provides language-level support for these operations.

The code is as follows:

(X+1 for x in LST) #生成器表达式, returns the iterator. External parentheses can be omitted when used for parameters.

[X+1 for x in LST] #列表解析, return to list

As you can see, generator expressions and List resolution (note: There are many translations here, such as list expansion, List derivation, etc., refer to the same meaning) the difference is very small, so when people mention this feature, simplicity is often described as a list parsing. However, since returning iterators does not compute all the elements at the outset, this gives you more flexibility and avoids many unnecessary computations, so you should always use the builder expression unless you explicitly want to return to the list. In the following words I do not distinguish between the two forms:

You can also filter for list parsing by providing an IF clause:

The code is as follows:

(X+1 for x in LST if x!=0)

Or you may have multiple for clauses nested loops, in the order of the FOR clause:

The code is as follows:

((x, y) for x in range (3) for Y in range (x))

List parsing is a stark pythonic. I've often encountered two problems using list parsing, which should belong to best practices, but these two are very typical, so here are some things to mention:

The first problem is that because the action applied to the element is too complex to be written out with an expression, no list parsing is used. This is an example of a typical idea without a change, and if we encapsulate the action as a function, isn't that an expression?

The second problem is that because the conditions in the IF clause need to be computed, and the results need to be calculated the same way, do not want to compute two times, like this:

The code is as follows:

(X.dosomething () for x in LST if x.dosomething () >0)

This is really bad writing, but the combination of list resolution can be resolved:

The code is as follows:

(X for X in (Y.dosomething () to Y in lst) if x>0)

The internal list resolution variable can actually be x, but for clarity we've changed to Y. Or more clearly, you can write two expressions:

The code is as follows:

TMP = (x.dosomething () for x in LST)

(x for X in TMP if x > 0)

List parsing can replace the vast majority of situations where you need to use map and filter, perhaps because of this, the famous static check tool pylint the use of map and filter as a warning.

3.4. Related libraries

Python has a modular itertools that contains a number of functions for the creating iterators for efficient looping (creating a more efficient loop iterator), which shows that it's domineering, This section is to browse through these functions and make an impression, when the need for these functions vaguely remember that there is good. The contents of this section are translated from the official documentation of the Itertools module.

3.4.1. Infinite iterations

Count (start, [step])

Starting with start, each element is then added with step. The step default value is 1.

Count (-->) 10 11 12 13 14 ...

Cycle (P)

Iteration to the last element of the sequence p, starting from the first element of P.

Cycle (' ABCD ')--> a b c d a b c D ...

Repeat (Elem [, N])

Repeat the Elem n times. If n is not specified, an infinite repetition occurs.

Repeat (3)--> 10 10 10

3.4.2. Stop the iteration when the shortest sequence parameter terminates

Chain (P, Q, ...)

Iteration to the last element of the sequence p, starting with the first element of Q until all the sequences terminate.

Chain (' ABC ', ' DEF ')--> A B C D E F

Compress (data, selectors)

if BOOL (Selectors[n]) is true, Next () returns DATA[N] or skips data[n].

Compress (' ABCDEF ', [1,0,1,0,1,1])--> A C E F

Dropwhile (pred, seq)

The iteration is not started until the call to Pred on Seq[n returns false.

Dropwhile (Lambda x:x<5, [1,4,6,4,1])--> 6 4 1

TakeWhile (pred, seq)

The reverse version of Dropwhile.

TakeWhile (Lambda x:x<5, [1,4,6,4,1])--> 1 4

IFilter (pred, seq)

The iterator version of the built-in function filter.

IFilter (Lambda x:x%2, Range ())--> 1 3 5 7 9

Ifilterfalse (pred, seq)

The reverse version of IFilter.

Ifilterfalse (Lambda x:x%2, Range ())--> 0 2 4 6 8

IMAP (func, p, Q, ...)

The iterator version of the built-in function map.

IMAP (POW, (2,3,10), (5,2,3))--> 32 9 1000

Starmap (func, seq)

Call Func for each element of SEQ as a variable-length parameter (*args).

Starmap (POW, [(2,5), (3,2), (10,3)])--> 32 9 1000

Izip (P, q, ...)

The iterator version of the built-in function zip.

Izip (' ABCD ', ' xy ')--> Ax by

Izip_longest (P, q, ..., fillvalue=none)

Izip the version of the longest sequence, the short sequence will be filled into the fillvalue.

Izip_longest (' ABCD ', ' xy ', fillvalue= '-')--> Ax by c-d-

Tee (it, N)

Returns the replication iterator for n iterator it.

GroupBy (iterable[, Keyfunc])

This function functions like a grouping of SQL. Before using GroupBy, you first need to use the same keyfunc to sort iterable, such as calling the built-in sorted function. Then, GroupBy returns the iterator, and the element of each iteration is the tuple (the key value, the child iterator of the collection of elements with the same key value in the iterable). Maybe a look at Python's sort guide is helpful in understanding this function.

GroupBy ([0, 0, 0, 1, 1, 1, 2, 2, 2])--> (0, 0 0 0)) (1, (1 1 1)) (2, 2 2 2))

3.4.3. Combined iterator

Product (p, q, ...) [Repeat=1])

Cartesian product.

Product (' ABCD ', repeat=2)--> AA AB AC AD BA BB BC BD CA CB CC CD DA DB DC DD

Permutations (p[, R])

Remove the duplicated elements.

Permutations (' ABCD ', 2)--> AB AC AD BA BC BD CA CB CD DA DB DC

Combinations (p, R)

Removes duplicate elements after sorting.

Combinations (' ABCD ', 2)--> AB AC AD BC BD CD

Combinations_with_replacement ()

After sorting, contains the repeating elements.

Combinations_with_replacement (' ABCD ', 2)--> AA AB AC AD BB BC BD CC CD DD

This article ends.