Python iterators and generators

Iterators

Iterator is the meaning of an iterator, and its function is to produce one data item at a time until it is not. This allows the loop to be processed in the for loop. What is the difference between it and the general sequence type (list, tuple, etc.)? It only returns one data item at a time, consuming less memory. But it needs to remember the current state in order to return to the next data item. It is an object that has a next () method. The sequence type holds all the data items, and their access is indexed.

An iterator is an object that implements an iterator protocol, and the iterator protocol in Python is that an object with the next method advances to the next result, and Stopiteration is thrown at the end of a series of results.

In the For loop, Python automatically calls the Factory function iter () to get the iterator, automatically calls next () to get the element, and completes the work of checking the stopiteration exception.

Several commonly used built-in data structures tuple, list, set, dict all support iterators, and strings can also use iterative operations.

You can also implement an iterator yourself, as described above, by simply returning an object in the class's __iter__ method, which has a next () method that throws the stopiteration exception at the appropriate time. But there are few times when you need to implement an iterator yourself, and even if you need it, it's easier to use the generator.

#!/usr/bin/env python# coding=utf-8class Test:    def __init__ (self, input_list):        self.list = input_list        SELF.I = 0    def __iter__ (self):        return self        def next (self):        if self.i = = Len (self.list):            self.i = 0< C8/>raise stopiteration        self.i + = 1        return  self.list[self.i-1]

One of the obvious benefits of using iterators is that you only read one piece of data from the object at a time, without causing too much memory overhead.

For example:

/* Load the file into memory one at a time and print it on a line-by-row basis. When the file size is large, the memory overhead of this method is very large */for line in open ("Test.txt"). ReadLines (): Print line/* This is the simplest and fastest way to do    it, and he doesn't read the file explicitly. Instead, use the iterator to read one line at a time ("Test.txt"):   #use file iterators    print lines */for

Generator

Generators are an easy way to create iterators. The generator is a special function. We can understand generator functions from both static and dynamic two angles.

First, from a static point of view, the generator function behaves in code as:

1. Contains yield statements (whether or not yield is likely to be executed)
2. No return or only no value return (once the yield statement exists in the function, the value return is considered a syntax error)

Second, from a dynamic point of view, the generator function is in the process of running:

1. When the generator function is called, the generator function does not execute any internal code and immediately returns an iterator.
2. When the returned iterator calls next for the first time, the generator function executes from the beginning, and if it encounters execution yield x,next immediately returns the yield value x.
3. When the returned iterator continues to call next, the generator function executes from the next sentence of the last yield statement until the next execution yield is encountered
4. Any time a function end is encountered, or a return statement throws a Stopiteration exception

In particular, the generator returns an iterator whose __iter__ returns itself.

The generator is written in a similar way to the function definition, except that it is changed to yield in the return place.

There can be more than one yield in the generator. When the generator encounters a yield, it pauses to run the generator, returning the value after yield. When the generator is called again, it will continue to run from where it was just paused until the next yield.

The generator itself forms an iterator that uses the value returned by a yield at each iteration.

It is important to note that the return statement is not required in the generator, there is no need to specify a return value, and the default returns statement already exists in the generator

Builder expression

(I for I in range (5))//Return iterator <generator object <genexpr> at 0x7ff3e8f0d960>

列表解析，返回list

[I for I in range (5)]//return list[0, 1, 2, 3, 4]

One problem here is that range (5) returns a 5-length data, and if it is a range (1000) it takes up a 1000-size array space, and if we use the ' generator ' to generate a number when needed, the space occupancy will be reduced. Here we can use the xrange () function to implement.

Xrange    Function Description: The usage is exactly the same as the range, and the difference is that it is not an array, but a generator. xrange example: "' >>> xrange (5) xrange (5) >>> list (xrange (5)) [0, 1, 2, 3, 4]>>> xrange (1,5) xrange ( 1, 5) >>> list (xrange (1,5)) [1, 2, 3, 4]>>> xrange (0,6,2) xrange (0, 6, 2) >>> list (xrange (0,6,2) ) [0, 2, 4]

So xrange do loop performance better than range, especially when returning very large, try to use Xrange bar, unless you are going to return a list.

So, what is the difference between range and xrange for Python 2.X? The answer is that the return value of range is a list, and when you call range, Python produces all the elements. The xrange is a specially designed iterative object that only stores the terminating value when it is created. You can compare the actual running results of the following two types of wording:

For-V in Range (1000000000000): #possible Memory Error    if v = = 2: Break for        v in    xrange (1000000000000): #fine 
   if v = = 2: Break        

　　

In Python 3.X, there is no more built-in xrange, whose range is equivalent to Python 2.X's xrange

Python iterators and generators