Fluent Python 14th iterative object, iterator and generator learning record

In Python, all collections can be iterated, and within the Python language, iterators are used to support

For loop

Building and extending collection types

Traverse a text file line by row

List derivation, dictionary derivation, and set derivation

Tuple unpacking

When calling a function, use the * unpacking argument

Topics covered in this chapter

Language internal use of ITER (...) built-in functions to manipulate the way an object can be iterated
How to implement the classic iterator pattern using Python
Detailed description of how the generator function works
How to use a generator function or builder expression instead of a classic iterator
How to use common generator functions in the standard library
How to use the yield from statement to merge generators
Case study: Using generator functions in a database conversion tool to process large datasets
Why generators and co-processes look the same, but the difference is very large, can not be confused

Why sequences can be iterated: the ITER function

When an interpreter needs to iterate over an object x, it automatically calls ITER (x).
The built-in ITER function has the following effects.
(1) Check whether the object implements the __iter__ method, call it if implemented, and get an iterator.
(2) If the __iter__ method is not implemented, but the __getitem__ method is implemented, Python creates an iterative
To get the element in order (starting at index 0).
(3) If the attempt fails, Python throws a TypeError exception, usually prompting "C object is not iterable" (c
object is not iterative), where C is the class to which the target object belongs.

Starting with Python 3.4, the most accurate way to check whether an object X can iterate is to call the ITER (x) Letter
Number, if not iterative, and then handle the TypeError exception. This is more than using Isinstance (x,
Abc. iterable) is more accurate because the ITER (x) function takes into account the legacy __getitem__ method,
and ABC. The Iterable class is not considered.

The comparison between an iterative object and an iterator

Objects that can be iterated
Use the ITER built-in function to get an iterator object. If an object implements an iterator that can return
__iter__ method, then the object is iterative. The sequence can be iterated, and the __getitem__ Square is realized.
parameter is a zero-based index, and this object can also iterate

>>> s = ' ABC ' >>> it = iter (s) #?>>> while True:     ... Try:         ... Print (Next (IT)) #?...     Except stopiteration: #?...         Del it #?...         Break #?... A
B
C

? Build iterator it with an object that can be iterated.
? Always call the next function on the iterator to get the next character.
? If there are no characters, the iterator throws an stopiteration exception.
? Releases a reference to it, which is an obsolete iterator object.
? Exits the loop.

An stopiteration exception indicates that the iterator is at its head. The Python language internally handles the for loop and other iterations
Stopiteration Exceptions in the following (such as list derivation, tuple unpacking, and so on).

The standard iterator interface has two methods.
__next__
Returns the next available element, if there is no element, throws a Stopiteration exception.
__iter__
Return self to use iterators where you should use an iterative object, for example, in a For loop.

In Python 3, the abstract method defined by the Iterator abstract base class is it.__next__ (), and
In Python 2, it is It.next (). As always, we should avoid calling special methods directly, using
Next (it), this built-in function can be used in both Python 2 and Python 3.

the best way to check whether an object x is an iterator is to call Isinstance (x, ABC. Iterator). Benefit

Iterator.__subclasshook__ method, even if object x belongs to a class other than the Iterator class.
True subclass or virtual subclass, you can also check this

Because iterators only need to __next__ and __iter__ two methods, so in addition to calling the next () method, and catching
In addition to stopiteration anomalies, there is no way to check if there are any remaining elements. In addition, there is no
"Restore" iterator. If you want to iterate again, call ITER (...) and pass in the previous build iterator to
The Iteration object. The incoming iterator itself is useless because the iterator.__iter__ method was previously said to be implemented in a way that
Returns the instance itself, so an incoming iterator cannot restore an already exhausted iterator.

Iterators
An iterator is an object that implements a parameterless __next__ method that returns the next element in the sequence;
If there are no elements, throw a stopiteration exception. The iterators in Python also implement the
The __iter__ method, so iterators can also iterate.

Errors are often encountered when building objects and iterators that can be iterated, because they are confused. You know, an iterative pair of
Like there is a __iter__ method that instantiates a new iterator each time, and the iterator implements the __next__ side
Method, returns a single element, and also implements the __iter__ method, returning the iterator itself.

As a result, iterators can iterate, but objects that can be iterated are not iterators.

An iterative object must not be an iterator of its own. In other words, an object that can be iterated must implement
__iter__ method, but cannot implement the __next__ method.

On the other hand, iterators should always be iterative. The __iter__ method of the iterator should return itself.

How the Generator function works
As long as there is a yield keyword in the definition body of a Python function, the function is a generator function. Call builder function
, a generator object is returned. In other words, the generator function is the generator factory.

The generator function creates a generator object that wraps the definition body of the generator function. Pass the generator to
Next (...) function, the generator function forwards, executes the next yield statement in the function definition body, and returns
The value of the output and pauses at the current position of the function definition body. Finally, when the function's definition body returns, the outer generator
Object throws a Stopiteration exception-this is consistent with the iterator protocol.

When to use a builder expression

A builder expression is a concise syntax for creating a generator so that you do not need to define function re-tune
Use. However, the generator function is much more flexible, you can use multiple statements to implement complex logic, or as a co-process
Use

Choosing which syntax to use is easy to judge: If the generator expression is to be divided into multiple lines of writing, I tend to
To define the generator functions to improve readability. In addition, the generator function has a name and can therefore be reused.

Generate arithmetic progression using the Itertools module
The Itertools module in Python 3.4 provides 19 generator functions, which can be used together to achieve many interesting
The usage.

The generator returned by the Itertools.count function can generate multiple numbers. If you do not pass in the parameter
Number, the Itertools.count function generates a zero-based integer sequence. However, we can provide an optional
Start and step values

However, the Itertools.takewhile function is different, and it generates a generator that uses another generator.
Stops when the specified condition evaluates to False. Therefore, the two functions can be combined to use the

Generator functions in the standard library

Think of the generator as a co-process

As with the. __next__ () method, the. Send () method causes the generator to advance to the next yield statement. No
The. Send () method also allows the client using the generator to send the data to itself, regardless of the pass-through to the. Send () method
What argument, that parameter becomes the value of the yield expression in the Generator function definition body. That is
Said, the. Send () method allows data to be exchanged in two directions between the customer code and the generator. and the. __next__ () method only
Allows the customer to fetch data from the generator.

Fluent Python 14th iterative object, iterator and generator learning record