Introduction
This article will introduce iterators and
generators in
Python 3, describe the relationship between iterable and iterators, and implement a custom class iterator pattern.
The concept of iteration
The result of the last output is the initial value of the next input. The repeated process is called iteration. Each iteration is an iteration, and the result of each iteration is the initial value of the next iteration.
Note: Loop is not iterative
while True: #Only meet the repetition, so it is not iterative
print('====>')
Iterator
1. Why is there an iterator?
For data types without indexes, an iterative method that does not rely on indexes must be provided.
2. Iterator definition:
Iterator: The iterable object executes the __iter__ method, the result is the iterator, and the iterator object has the __next__ method
It is a stateful object. It can return the next value in the container when you call the next() method. Any object that implements the __iter__ and __next__() methods is an iterator, and __iter__ returns iterative The device itself, __next__ returns the next value in the container, if there are no more elements in the container, a StopIteration exception is thrown
Iterable
There are some iterable objects in the Python standard library, such as list, tuple, dict, set, str, etc.
You can perform for-in and other iterative operations on these iterative objects, for example:
for s in "helloworld":
print(s)
If the compiler wants to iterate an object a, it will automatically call iter(a) to obtain the iterator of the object. If iter(a) throws an exception, the object a cannot be iterated.
Determine if the object is iterable
The native function iter(instance) can determine whether an object is iterable. Its workflow is roughly divided into the following three steps:
Check whether the object instance implements the __iter__ method, and call it to get the iterator returned.
If the object does not implement the __iter__ method, but implements the __getitem__ method, Python will generate an iterator.
If the above fails, the compiler throws a TypeError error, ‘xxx’ Object is not iterable.
Custom class implements __iter__ method
According to the first article, our custom class Iter1 implements the __iter__ method to make objects of this class iterable.
class Iter1:
def __init__(self, text):
self.text = text
def __iter__(self):
return iter(self.text)
iter1 = Iter1("hello")
for s in iter1:
print(s)
The Iter1 class implements the __iter__ method. Iter() is called to get an iterator of the iterable object text and return it. The iterator protocol is implemented, so the object can be iterated through for-in and other methods.
The second one is usually defined for sequences in Python, such as list, in order to implement the sequence protocol, the __getitem__ method needs to be implemented.
class Iter2:
def __init__(self, sequence):
self.sequence = sequence
def __getitem__(self, item):
return self.sequence[item]
iter2 = Iter2([1, 2, 3, 4])
for s in iter2:
print(s)
In fact, in order to avoid post-version changes, the sequence in the Python standard library not only implements the __getitem__ method, but also implements the __iter__ method, so we should also implement __iter__ when defining the sequence.
In summary, if it is displayed to determine whether an object is iterable, iter(instance) should be called to throw an exception, because the sequence that only implements __getitem__ is also iterable (Iter2 objects in the example are iterable, but isinstance (iter2, abc.Iterator) The return result is False). At the same time, if iterative operation after calling iter does not need to display judgment, you can use try/except to wrap the code block.
iterable vs iterator (iterable vs iterator)
iterable definition
Any object that can get the iterator by the native function iter
Any object that implements the __iter__ method and returns an iterator
All sequences (implemented __getitem__)
Python implements iteration through an iterator that obtains an iterable object. For example, the implementation of for-in actually obtains an iterator for operation internally. The for-in mechanism can be understood as the following code:
s ='hello'
it = iter(s)
while (True):
try:
print(next(it))
except StopIteration:
del it
break
The StopIteration exception will be thrown after the iterator is exhausted, and iterative operations such as for-in, generation (comprehension), tuple unpacking (tuple unpacking) will handle this exception.
The iterator is an iterative value production factory, which saves the iterative state and generates the next iterative value through the next() function. Implementing an iterator requires implementing the following two methods:
__iter__
Return self
__next__
Returns the next available element, if there is no available element, a StopIteration exception is thrown
The iterator implements __iter__, so all iterators are iterable. The following figure shows the structure of iterable and iterator.
Iterator pattern
Implementing a custom iterator pattern requires two classes, namely the class that implements the __iter__ method and the iterator instance class returned by __iter__ (implements the __iter__ and __next__ methods). The following example simply implements the above functions.
class IterText:
def __init__(self, text):
self.text = text
def __iter__(self):
return IteratorText(self.text)
class IteratorText:
def __init__(self, text):
self.text = text
self.index = 0
def __iter__(self):
return self
def __next__(self):
try:
letter = self.text[self.index]
except IndexError:
raise StopIteration
self.index += 1
return letter
text = IterText("hey")
for l in text:
print(l)
The iterable IterText implements the __iter__ method, returning an IteratorText instance of the iterator. IteratorText implements the __next__ method to return the next iteration element until an exception is thrown, and IteratorText implements the __iter__ method to return its own object for iteration.
IterText and IteratorText here are easy to confuse. If the __next__ method is implemented in IterText and returns its own instance self in __iter__, the above function can also be achieved, but usually the iterable object and the iterator should be separated, so that iterable Different iterator objects can be returned in __iter__ in the object, making the function independent.
Generator
According to the above article, the iterator continues to produce the next element through next() until the iterator is exhausted, and the generator in Python can be understood as a more elegant iterator (no need to implement the __iter__ and __next__ methods ), which implements the iterator protocol, it can also produce elements through next().
There are two main types of generators in Python:
The generator function returns the resulting generator:
Functions containing the yield keyword are called generator functions
def gen_func():
yield 1
yield 2
yield 3
g = gen_func()
The generator expression returns the resulting generator
g = (i for i in (1, 2, 3))
We can use the generator to iterate:
for e in g:
print(e)
## The generator g has been exhausted, if you need to iterate again you need to get a new generator object
g = gen_func()
for e in g:
print(e)
Use generators to replace iterable __iter__ iterators
In the Iterator Mode chapter, we return the IteratorText instance of Iterator in the iterable IterText __iter__, however, the use of the generator will make the code more elegant.
class IterText:
def __init__(self, text):
self.text = text
def __iter__(self):
for letter in self.text:
yield letter
Because yield exists in __iter__, __iter__ becomes a generator function. Calling it returns a generator, and the generator implements the iterator protocol, so IterText meets the need for iterable.
