Python Iteration object, iterator, Generator Detailed

in understanding Python Data structure, container (container), Iterationobject (iterable), iterator (iterator) , Generator (generator) , List / Collection / Dictionary Derivation type (list,set,dict Comprehension) a lot of concepts mixed together, It is inevitable for beginners to confused, This article will be one for everyone to introduce, hope to learn python to help You.

Container (container)

a container is a data structure that organizes multiple elements together, and the elements in a container can be iterated over one after another, using in, the not keyword, to determine whether an element is contained in a container. Usually such data structures store all the elements in memory (and some are not all elements in Memory) in Python , common container objects are:

list, deque, ....

set, frozensets, ....

dict, defaultdict, ordereddict, Counter, ....

tuple, namedtuple, ...

Str

the container is easier to understand because you can think of it as a box, a house, a cupboard, which can be stuffed with anything. technically, when it can be used to ask if an element is contained in it, then the object can be considered a container, such as list,set,tuples are container objects:

>>> assert 1 in [1, 2, 3] # Lists

>>> assert 4 not in [1, 2, 3]

>>> assert 1 in {1, 2, 3} # Sets

>>> assert 4 not in {1, 2, 3}

>>> assert 1 in (1, 2, 3) # tuples

>>> assert 4 not in (1, 2, 3)

ask if an element is in dict with the dict key:

>>> d = {1: ' foo ', 2: ' bar ', 3: ' Qux '}

>>> assert 1 in D

>>> assert ' foo ' not in D # ' foo ' is not an element in dict

ask if a substring is in string :

>>> s = ' foobar '

>>> assert ' B ' in S

>>> assert ' X ' not in S

>>> assert ' foo ' in S

While the vast majority of containers provide some way to get each of these elements, this is not the ability provided by the container itself, but rather can iterate over objects the ability to give a container, of course, is not all containers are iterative, such as: Bloom filter , although Bloom filter can be used to detect whether an element is contained in a container, it is not possible to obtain each of these values in a single device, because Bloom filter instead of storing the element in a container, it is mapped to a value stored in the array by a hash function.

can iterate over objects (iterable)

as mentioned earlier, Many containers are iterative objects, and there are more objects that are also iterative objects, such as files that are open,sockets , and so On. But the object that can return an iterator can be called an iterative object, which may sound a bit confusing, but let's look at an example:

>>> x = [1, 2, 3]

>>> y = iter (x)

>>> z = iter (x)

>>> Next (y)

1

>>> Next (y)

2

>>> Next (z)

1

>>> type (x)

<class ' list ' >

>>> type (y)

<class ' List_iterator ' >

here x is an iterative object, iterative objects and containers is a popular term, not a specific data type,list is an iterative object,dict  is an iterative object, andset is also an iterative object. y and z are two independent iterators that hold a state inside the iterator that records where the current iteration is located, so that the correct element is available for the next Iteration. Iterators have a specific type of iterator, such as list_iterator , set_iterator . the __iter__ and __next__ methods are implemented in an iterative object (python2 C22>next method,python3 is the __next__ method), These two methods correspond to the built-in functions  ITER () and next () . the __iter__ method returns an iterator that iterates over the object itself, which makes him both an iterative object and an Iterator.

When running Code:

x = [1, 2, 3]

For Elem in X:

...

The actual performance is:

to decompile The code, you can see that the interpreter appears to call The get_iter instruction is equivalent to calling ITER (x) , and the for_iter instruction is invoking the next () method, Gets the next element in the iterator continuously, but you can't see it directly from the command, because he is optimized by the Interpreter.

>>> Import Dis

>>> x = [1, 2, 3]

>>> Dis.dis (' for _ in X:pass ')

1 0 setup_loop (to 17)

3 Load_name 0 (x)

6 Get_iter

>> 7 For_iter 6 (to 16)

Ten Store_name 1 (_)

7 Jump_absolute

>> Pop_block

>> Load_const 0 (None)

Return_value

iterators (iterator)

So what about iterators?  It is a stateful object that can return the next value in the container when you call the next () method, any implementation of __next__ () ( implemented in Python2 next () ) method is an iterator, and it does not matter how it is Implemented.

so, The iterator is the object that implements the factory pattern, and it returns you every time you ask for the next Value. There are many examples of iterators, such as the Itertools function that returns an iterator Object.

Generate an infinite sequence:

>>> from Itertools Import count

>>> counter = Count (start=13)

>>> Next (counter)

13

>>> Next (counter)

14

To generate an infinite sequence from a finite sequence:

>>> from Itertools Import cycle

>>> colors = Cycle ([' red ', ' white ', ' blue ')

>>> Next (colors)

' Red '

>>> Next (colors)

' White '

>>> Next (colors)

' Blue '

>>> Next (colors)

' Red '

To generate a finite sequence from an infinite sequence:

>>> from Itertools import islice

>>> colors = Cycle ([' red ', ' white ', ' Blue ']) # Infinite

>>> Limited = Islice (colors, 0, 4) # finite

>>> for X in Limited:

... print (x)

Red

White

Blue

Red

To get a more intuitive sense of the execution process inside an iterator, we customize an iterator that takes the Fibonacci sequence as an example:

Class Fib:

def __init__ (self):

Self.prev = 0

Self.curr = 1

def __iter__ (self):

return self

def __next__ (self):

Value = Self.curr

Self.curr + = Self.prev

Self.prev = value

return value

>>> f = Fib ()

>>> list (islice (f, 0, 10))

[1, 1, 2, 3, 5, 8, 13, 21, 34, 55]

The Fib is both an iterative object (because it implements the __iter__ METHOD) and an iterator (because the __next__ method is implemented). Instance variables prev and curr users maintain the state inside the Iterator. Do two things each time you call the next () method:

Modify the state for the next call to the next () method

Generates a return result for the current call

An iterator is like a lazy-loaded factory that returns a value when someone needs it, and waits for the next call when it is not called.

Generator (generator)

Generator is a One of the most appealing features of the Python language, the generator is actually a special iterator, but this iterator is more elegant. It does not need to write the __iter__ () and __next__ () methods as in the class above , only one yiled keyword is required. The generator must be an iterator (and vice versa), so any generator also generates values in a lazy-loaded pattern. Examples of Fibonacci sequences implemented with generators are:

def fib ():

prev, Curr = 0, 1

While True:

Yield Curr

prev, Curr = curr, Curr + prev

>>> f = fib ()

>>> list (islice (f, 0, 10))

[1, 1, 2, 3, 5, 8, 13, 21, 34, 55]

The FIB is an ordinary python function, where the function is not a return keyword, and the return value of the function is a generator object. When execution f=fib () Returns a generator object, the code in the body of the function does not execute, and the code is actually executed only when the next is shown or implicitly called .

Generator in Python is a very powerful programming structure that can write streaming code with less intermediate variables, and in addition, it can save memory and CPU compared to other container objects , of course, it can use less code to achieve similar Functions. Now it's time to refactor your code and see something like This:

def something ():

result = []

For ... in ...:

Result.append (x)

return result

Can be replaced with a generator function:

def iter_something ():

For ... in ...:

Yield x

Builder Expression (generator Expression)

A builder expression is a list-down version of the builder that looks like a list derivation, but it returns a generator object instead of a list object.

>>> A = (x*x for x in range (10))

>>> A

<generator Object <genexpr> at 0x401f08>

>>> sum (a)

285

Source: Public Account

Python Iteration object, iterator, Generator Detailed