Python standard modules--iterators and generators

1 Introduction to Modules

When you start using Python programming, you may have used iterators (iterators) and generators (generators), which you might not have realized at the time. In this blog post, we will learn what iterators and generators are. Of course, we also know how to create them, and we can create our own iterators and generators when we need them.

2 Module uses 2.1 iterators

An iterator is an object that allows you to iterate over a container. Python's iterators are implemented primarily in two ways: __iter__ and __next__. __ITER__ requires that your container support iterations. It returns the Iterator object itself. If you want to create an iterator object, you also need to define the __next__ method, which will return the next element of the container.

Note: in Python 2, the naming habits are different, __iter__ remains the same, __next__ changes to next.

To make these concepts clearer, let's review the following two definitions:

• An iterative object (iterable) that defines only the __iter__ method;
• Iterator (iterator), defines the __iter__ and __next__ two methods, __ITER__ returns the iterator itself, and the __next__ method returns the next element;

It is amazing that there are double underlines in all function names, and you do not need to call __iter__ or __next__ directly. You can use the For loop or convert to a list, and Python will automatically call these methods for you. Of course you might want to call them, so you can use Python's built-in functions like the ITER and next methods.

Python 3 provides a number of sequence types, such as list, tuple, and range. The list is an iterative object, but not an iterator because it does not implement the __next__ method. We can easily find this by using the following example.

>>> my_list = [1,2,3]>>> next(my_list)Traceback (most recent call last): File "<stdin>", line 1, in <module>TypeError: ‘list‘ object is not an iterator

When we try to invoke the next method of the list in this example, we finally receive a TYPEERROR exception message, and we can see that the list object is not an iterator. Let's take a look at the following code:

 >>> iter (my_list) <list_iterator object at 0x7f6ec6b00f98>>>> list_iterator = iter (my_list) >>> Next (list_iterator) 1>> > Next (list_iterator) 2>>> Next (List_iterator) 3>>> Next (List_iterator) Traceback (most recent call last) : File  "<stdin>", line 1, in <module>stopiteration   

You can convert a list to an iterator simply by invoking the Python built-in ITER function on the list. When you call the next method on it, until it iterates through all the elements, you receive the stopiteration exception message. Let's try to convert the list to an iterator and then iterate through it in the loop.

>>> for item in iter(my_list):...     print(item)... 123

When you use loops to iterate over an iterator, you don't need to call the next method, and you don't have to worry about getting stopiteration exception information.

2.2 Creating an iterator that belongs to you

Sometimes you also want to create an iterator that belongs to you. Python is easily able to do this task. As mentioned in the previous section, all you need to do is implement the two methods of __iter__ and __next__ in your class. Let's create an iterator that iterates through the string.

ClassMyiterator:Def__init__(self,letters): self.letters = Letters Self.position =0 def  __iter__return self < Span class= "Hljs-keyword" >def __next__ (self): if self.position >= len (self.letters): raise stopiteration letter = Self.letters[self.position] Self.position + = 1 return letter< Span class= "Hljs-keyword" >if __name__ = =  "__main__": i = Myiterator (  "ABCD") for item in I: print (item)   

For this example, in our class, we just need to implement three methods. In the initialization function, we pass in the string and create a class variable to reference the string. We also initialized a positional variable so that we know where we are in the string. The __iter__ method simply returns itself, which is what it has to do. __NEXT__ is the most important method in this class, we first check if the position exceeds the length of the string and, if exceeded, throws the stopiteration exception information. We then extract the character from that position on the string, add 1 to the position, and finally return the character.

In the following example, we create an infinite iterator. An infinite iterator is one that can be traversed all the time. You should be aware that when you call this iterator, they will cause a dead loop if you do not add any restrictions.

ClassDoubler():Def__init__(self): Self.number =0 def  __iter__return self < Span class= "Hljs-keyword" >def __next__ (self): Self.number + = Span class= "Hljs-number" >1 return self.number * self.numberif __name__ = =  "__main__": Doubler = Doubler () Count = 0 for number in doubler: print (number) if count >  5: break count + = 1              

In this code, we don't pass any parameters to the iterator. We only use this as an example to illustrate. In order not to get into a dead loop, we add a counter before we start traversing the iterator. Finally, we start traversing, and when the timer exceeds 5, the loop is interrupted.

2.3 Generators

A common Python function often returns a value, whether it is a list, an integer, or another object. But if you want to invoke a function, can this function produce a series of values? This is why the generator was born. The generator works by saving the location where it stopped (or it has been), and then returns a value to the keynote function. Instead of returning the execution of a calling function, the generator simply returns a temporary control return. In order to complete this function, the generator function needs to use the Python yield statement.

Note: in other programming languages, the generator may be called a synergistic program;

Let's start by creating a simple generator,

>>>Defdoubler_generator ():  ... number = 2 ... while true: ... yield number ... number *= Number ... Span class= "Hljs-meta" >>>> doubler = Doubler_generator () >>> Next (Doubler) 2>>> Next (doubler) 4>>> Next (doubler) 16>>> Type (doubler) <class ' generator '                 

This generator is simply creating an infinite sequence. You can always call the next method on it, and it will never run out of all values. Because you can traverse the generator, a generator can also be considered a class of iterators, but no one really ties them together. In fact, the generator also defines the __next__ method, and we look at the example above, which is why the next function can execute normally.

Let's look at another example in which only 3 elements are generated, not infinite sequences.

>>>DefSilly_generator(): ... yield  "Python"  ... yield  ... yield  "so do you!"  ... >>> gen = Silly_generator () >>> Next (gen)  ' Python ' >>> Next (gen)  ' Rocks ' Span class= "Hljs-meta" >>>> Next (gen)  ' so do you! ' >>> Next (Gen) Traceback (most recent call last): File  < Stdin> ", line 1, in <module>stopiteration   

In this example, we created a generator that uses the yield statement 3 times. In each instance, it produces a different string. You can think of yield as the generator's return statement. Whenever you call yield, the function aborts and then saves its state. Then it produces a value, which is the output you see in the above example through the terminal. If we have some variables in the function, then these variables will be saved as well.

When you see stopiteration, you should know that you have exhausted this iterator. This means that the iterator is exhausted. You can also see this feature in the iterator section above.

Whenever we call the next method, the generator starts where it left off, then produces the next value or we end the function, and the generator ends. If you don't call the next method all the time, the state will remain forever.

Let's instantiate the generator and use loops to traverse it.

>>> gen = silly_generator()>>> for item in gen:... print(item)... PythonRocksSo do you!

We created a new generator instance because we didn't want anything to happen when we walked through the generator. This is because we have traversed all the values of the instance of this generator. So in this example, we create an instance and then iterate through it and print the resulting value. When the generator traverses, the for loop handles the stopiteration exception for us, and then interrupts the loop.

One advantage of the generator is that it can traverse large datasets and then return only a subset of the data at a time. The following example is when we open a file and then return the data row by line,

with open("temp.txt") as fobj:    for line in fobj: print line

When we traverse this way, Python converts the file object to a generator. This approach allows us to deal with files that are too large to read into memory. For any large data set, you need a block operation for these big data sets or when you need to generate a large data set but it will fill your memory and you will find the generator useful.

2.4 Summary

Here, you've learned what an iterator is and how to use it. You also know the difference between an iterative object and an iterator. Finally, we learned what a generator is and how you use it. For example, the generator is good for working with memory-efficient data.

Python standard modules--iterators and generators