Introduction to Python Generators and iterators

Before formally contacting the generator, Let's start by understanding some concepts

Container ( container )

A container is a data structure that organizes multiple elements together, and the elements in a container can be iterated over one after another, and the in and not keywords can be used to determine whether an element is contained in a container. Usually such data structures store all the elements in memory (and there are some special).

Common container objects in python:

1. list, deque, ...
2. set, frozensets, ...
3. dict, defaultdict, ordereddict, Counter, ...
4. tuple, nametuple, ...
5. Str

Tip: 1. An iterative object gives the container the ability to extract elements

2. Not all containers are iterative

an iterative object (iterable)

Given a list or tuple, we can traverse the list or tuple through a for loop, which we call Iteration (iteration).

An iterative object is not confined to a container, such as: files,sockets, etc. (in the open state). As long as it is available I returns an object that has an iteration which can be called an iterative Object.

Cases:

x = [1, 2, 3]y = iter (x) print (next (y)) print (next (y)) print (type (x), type (y))

Analysis:

Here x is an iterative object that can iterate over objects and containers as a popular salutation, and does not refer to a specific data type. List,set,dict are iterative objects and Y is a stand-alone iterator that has a state inside the iterator that records where the current iteration sits, so that the correct element is obtained for the next Iteration.

Iterators have a specific type of iterator: list_iterator,set_iterator, .... An iterative object implements the __iter__ () and __next__ () methods (the Next () method in python2, python3.x is the __next__ () method), which correspond to the built-in functions ITER () and Next (). The __iter__ method returns an iterator that iterates over the object itself, making it both an iterator and an Iterator.

iterators (iterator)

An iterator (iterator) is an object that can be used to traverse some or all of the elements in a standard Template's container, and each iterator object represents a definite address in the Containers. Iterators modify the interface of regular pointers, so-called iterators are a conceptual abstraction: things that behave like iterators can be called Iterators. however, iterators have many different abilities, which can unify the abstract container and the general algorithm organically.

Iterators have the following two ways:

• __iter__ ()
• __next__ ()

An iterative object must have:

• __iter__ ()

tips: 1. For iterators, one __next__ () is Enough. When using a for loop, the program automatically invokes the iterator object that is about to be processed, and then uses the next () function until a stopiteration exception is Detected.

The 2.next () built-in function is the method __next__ () that invokes the object, and the ITER () built-in function is the __iter__ () method of the calling Object.

Case:

• Call the next () method directly

• First Use ITER (), then call next ()

In the above example, we all know that the list T can be used for the value operation for the loop, but not by the built-in function next () to take the value, therefore, the judge T is an iterative object (iterable); When T is wrapped through ITER (), it is possible to invoke the next () query to fetch a value, so I is an iterator (iterator).

In addition to the above methods can be judged, there are collections module judgment function:

• Can iterate object judgment

• Iterator judgment

Use ITER () to convert an iterative object into an iterator

Summary: we can infer that the inside of the For loop is the first call to ITER () to turn the iterable into a iterator and then loop through the Iterations.

The above for loop and while loop are equivalent

Why is the list, dict, set, str, and other data types not iterators (Iterator)?

Because the Python Iterator object represents a data stream, the Iterator object can be called by the next () built-in function and continuously return to the next data until the Stopiteration exception is thrown when no data is Available. This data stream can be viewed as an ordered sequence, but the length of the sequence cannot be known in advance, and the next data is calculated on demand by the next () built-in function, so the calculation of the iterator is inert and is calculated only if the next data needs to be Returned. Iterator can be seen as an infinite stream of data, eg: all natural numbers, while list,set, str, etc. are impossible to store all the natural numbers.

add: iterator inherits from the iterable, it is easy to see iterator contains the __iter__ () and next () methods from the following tests, and iteratble only contains __iter__ ().

From collections import Iterator, iterable

Iterator summary:

• Objects that are iterated with a for loop are iterable types
• The object that uses the next () built-in function is the iterator type, which represents a sequence of lazy computations
• Iterable can be obtained by using the built-in function ITER () to obtain a iterator object
• Internal implementation mechanism for the For loop

Generator

The concept of a generator is slightly more complex than an iterator, because the generator is a function that returns an iterator, and its greatest effect is to return the input object as an Iterator. The concept of iteration is used in Python because the traditional memory load method consumes a lot of memory when it is necessary to iterate through a larger object, which is less expensive than reading an element when Needed.

A generator is a special type of function (a special Iterator) that generates a value at a Time. It can be treated as a recoverable function. Calling this function returns a generator generator that can be used to generate successive X-values.

There are two points to be clear first:

• Any generator is an iterator (conversely, not valid)
• Any generator is a factory that can delay the creation of values (controllability)

Creation of generators

• Change the list in the generated form to []()

With list generation, You can create a list directly. however, with memory limitations, The list capacity is certainly limited. also, Creating a list that contains millions of elements is not only taking up a lot of memory space, such as: we only need to access the previous elements, and most of the space behind the elements is Wasted.

therefore, It is not necessary to create a complete list (saving a lot of memory space). In python, we can take the generator: edge loop, edge calculation Mechanism->generator

How do we print the elements?

Just mentioned that the generator is a special iterator, you can get the next return value for generator by using the next () built-in function:

Generator is the saved algorithm, and each time the next () built-in function is called, the next element can be Computed.

Next's Way is definitely not working, because generator is also an iterative object, and we can use a for loop.

Tip: The For loop is used here, with no exception, as with iterators, because of the internal mechanism of the for Loop.

Simply put, in the execution of the function, the yield statement will return the value you need to the calling generator, and then exit the function, and the next time the generator function is called from where it was last interrupted, and all the variable parameters in the generator are saved for the next Use.

Generator summary:

• The generator object is a special iterator that satisfies the iterator protocol and can call the next () built-in function, and the Iter () method is called when the generator for loop returns the generator object and then the next () iteration, and Iter () and next () are implemented within YIELD.
• Builder creation: Two common (list-generated, function-keyword).

Introduction to Python Generators and iterators