Python builder and iterator this is enough.

This section focuses on the list generation, generator, and iterator knowledge points

List Builder

Let's start with an example.

Now there is a need to look at the list [0,1,2,3,4,5,6,7,8,9], asking you to add 1 to each value in the list, how do you achieve it?

Method One (Simple):

info = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]b = []# for index,i in Enumerate (info): #     print (i+1) #     B.append (i+1) # print (b) F or index,i in enumerate (info):    Info[index] +=1print (info)

Method Two (general):

info = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]a = map (lambda x:x+1,info) print (a) for I in A:    print (i)

Method Three (Advanced):

info = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]a = [i+1 for i in Range]]print (a)

Generator What is a generator?

With a list build, we can create a list directly, but with memory constraints, the list size is limited, and a list of 1 million elements is created, which takes up a lot of storage space, and if we just need to access the first few elements, then the vast majority of the space used by the elements is wasted.

So, if the list element can be calculated according to an algorithm, can we continue to calculate the subsequent elements in the process of the loop? This eliminates the need to create a complete list, which saves a lot of space, and in Python, this side loop computes the mechanism called the generator: generator

The generator is a special program that can be used as an iterative behavior for controlling loops, where the generator in Python is one of the iterators, using the yield return value function, and each call to yield is paused, and the generator can be restored using the next () function and the Send () function.

The generator is similar to a function that returns an array of values, which can accept parameters that can be called, however, unlike normal functions that return an array of all values at once, the generator can produce only one value at a time, so the amount of memory consumed will be greatly reduced. and allowing the calling function to quickly process the first few return values, so the generator looks like a function, but behaves like an iterator

The builder in Python

To create a generator, there are a number of methods, the first method is simple, only a list of the generated [] brackets to () parentheses, create a generator

　　Examples are as follows:

#列表生成式lis = [x*x for x in range]]print (lis) #生成器generator_ex = (x*x for x in range) print (GENERATOR_EX) results: [0, 1, 4, 9 , +, 81]<generator object <genexpr> at 0x000002a4cbf9eba0>

So what's the difference between creating an LIS and generator_ex? From the surface is [] and (), but the result is not the same, one print out is the list (because it is a list generated), and the second print is <generator object <genexpr> at 0x000002a4cbf9eba0> So how do you print out every element of GENERATOR_EX?

If you want to print it out, you can get the next return value for generator by using the next () function:

#生成器generator_ex = (x*x for x in range) print (Next (GENERATOR_EX)) print (Next (GENERATOR_EX)) print (Next (generator_ex ) Print (Next (GENERATOR_EX)) print (Next (GENERATOR_EX)) print (Next (GENERATOR_EX)) print (Next (GENERATOR_EX)) print ( Next (GENERATOR_EX)) print (Next (GENERATOR_EX)) print (Next (GENERATOR_EX)) print (Next (GENERATOR_EX)) Results: 0149162536496481Traceback (most recent call last):  File "List-generated. py", line, in <module>    print (Next ( GENERATOR_EX)) stopiteration

As you can see, Generator saves the algorithm, each time it calls next (GENERAOTR_EX) calculates the value of his next element until the last element is calculated, no more elements are thrown stopiteration the error, And it's a bad habit to keep calling like this, and the right way is to use a For loop because generator is also an iterative object:

#生成器generator_ex = (x*x for x in range) for I in GENERATOR_EX:    print (i)    result: 0149162536496481

So after we create a generator, we basically never call next (), but iterate through the for loop, and don't need to care about Stopiteration's error, generator is very powerful, if the algorithm is more complex, When a for loop with a similar list generation is not possible, it can also be implemented with functions.

For example, the famous Fibonacci sequence, in addition to the first and second numbers, any number can be added by the first two:

1,1,2,3,5,8,12,21,34 .....

The Fibonacci sequence is not written in list generation, but it is easy to print it out with a function:

#fibonacci数列def fib (max):    n,a,b =0,0,1 while    n < max:        A, b =b,a+b        n = n+1    return ' done ' a = fib (Ten) p Rint (FIB (10))

A, B, a+b is actually equivalent to T =a+b, a =b, b =t, so you can output the first n digits of the Fibonacci sequence without writing a temporary variable T. The results of the above output are as follows:

1123581321345511235813213455done

Looking closely, it can be seen that the fib function is actually a calculation rule that defines the Fibonacci sequence, which can be derived from the first element, and the subsequent arbitrary elements, which are actually very similar to generator.

That is, the above function can also be implemented with generator, above we found that print (b) Every time the function is printed, accounting for memory, so in order not to account for memory, we can also use the generator, this is called yield. As follows:

def fib (max):    n,a,b =0,0,1 while    n < max:        yield B,        =b,a+b        n = n+1    return ' done ' A = fib (10) Print (FIB (10))

But the return is no longer a value, but a generator, as in the example above, you can look at the results:

<generator Object fib at 0x000001c03ac34fc0>

So this does not account for the memory, here to say the generator and functions of the execution flow, the function is executed sequentially, encountered a return statement or the last line function statement returned. The function that becomes generator, executes at each call to next (), encounters yield statement return, again by next () call time from the last return yield statement is urgently executed, that is, with how much, take how much, do not occupy memory.

def fib (max):    n,a,b =0,0,1 while    n < max:        yield B,        a =b,a+b n = n+1    return ' done ' a = FIB (TEN) PR int (fib) print (a.__next__ ()) print (a.__next__ ()) print (a.__next__ ()) print ("Can do other things") print (a.__next__ ()) Print (a.__next__ ()) Result: <generator object fib at 0x0000023a21a34fc0>112 can do other things by the way 35

　　in the above fib example, we continue to call during the loop yield , and we are constantly interrupted. Of course, you have to set a condition for the loop to exit the loop, or it will produce an infinite sequence. Similarly, after changing a function to generator, we basically never use it next() to get the next return value, but instead use the for loop directly to iterate:

def fib (max):    n,a,b =0,0,1 while    n < max:        yield B, a        =b,a+b        n = n+1    return ' done ' for I in FIB ( 6):    print (i)    results: 112358

However, when you call generator with a For loop, you find that you cannot get the return value of the generator return statement. If you do not get the return value, then you will get an error, so in order not to let the error, we should do exception handling, get the return value, if you want to get the return value, you must catch the Stopiteration error, the return value is contained in the value of stopiteration:

def fib (max):    n,a,b =0,0,1 while    n < max:        yield B, a        =b,a+b        n = n+1    return ' done ' G = fib (6 While True:    try:        x = Next (g)        print (' Generator: ', x)    except stopiteration as E:        print (" Generator return value: ", e.value) break        Result: Generator:  1generator:  1generator:  2generator:  3generator:  5generator:  8 Generator return value: Done

You can also achieve the effect of concurrency in single-threaded scenarios with yield

Import timedef Consumer (name):    print ("%s ready to learn!"%name)    while True:       lesson = yield       Print ("Start [%s], [%s] The teacher came to lecture! "% (Lesson,name)) def producer (name):    C = Consumer (' A ')    c2 = consumer (' B ') c.__next__    ()    c2.__ next__ ()    Print ("The class has begun!")    For I in range:        time.sleep (1)        print ("to two classmates!")        C.send (i)        c2.send (i) Result: A ready to learn! B 准备学习啦!同学们开始上课 了!到了两个同学!开始[0]了,[A]老师来讲课了!开始[0]了,[B]老师来讲课了!到了两个同学!开始[1]了,[A]老师来讲课了!开始[1]了,[B]老师来讲课了!到了两个同学!开始[2] 了,[A]老师来讲课了!开始[2]了,[B]老师来讲课了!到了两个同学!开始[3]了,[A]老师来讲课了!开始[3]了,[B]老师来讲课了!到了两个同学!开始[4]了,[A]老师来讲课了!开始[4]了,[B]老师来讲课了 to two classmates! [5], [a] the teacher came to lecture! [5], [b] The teacher came to lecture! to two classmates! [6], [a] the teacher came to lecture! [6], [b] The teacher came to lecture!

From the above example, I can see that Python offers two basic ways

Generator functions: Also defined with Def, using the keyword yield to return one result at a time, blocking, restarting

Generator expression: Returns an object that produces results only when needed

--Generator function

Why is it called a generator function? Because it generates a numeric queue over time. The general function returns a value after execution and exits, but the generator function automatically hangs and then picks up the badly needed execution, and he uses the yield keyword to close the function, return a value to the caller, and keep the current enough state so that the function can continue to execute. The generator and iteration protocol are closely related, and the iterated object has a __next__ () __ Member method that either returns the next item of the iteration and buys the iteration that causes the exception to end.

# After the function has yield, the function name + () becomes the generator # return in the generator to represent the abort of the generator, direct error # Next function is to wake up and continue to execute # Send the role is to wake up and continue to execute, send a message to the generator internal "generator" "Def Create_counter (n):    print ("Create_counter") while    True:        yield n        print ("Increment n")        n +=1gen = Create_counter (2) print (gen) print (Next gen) Result: <generator object Create_counter at 0x0000023a1694a938>create_counter2increment N3process finished with exit code 0

　　

--Generator expression

The generator expression comes from a combination of iteration and list resolution, similar to the generator and list parsing, but it uses angle brackets instead of square brackets

>>> # List resolution build list >>> [x * * 3 for X in range (5)][0, 1, 8, 64]>>> >>> # Builder Expression >> > (x * * 3 for X in range (5)) <generator object <genexpr> at 0x000000000315f678>>>> # Convert between two >> > List (x * * 3 for X in range (5)) [0, 1, 8, 27, 64]

　　An iteration can be written either as a generator function or a generates builder expression that supports both automatic and manual iterations. And these generators only support one active iteration, which means that the generator's iterator is the generator itself.

Iterators (iterations are loops)

We already know that there are several types of data that can be directly applied to a For loop:

A class is a collection of data types, such as List,tuple,dict,set,str

A class of generator, including generators and generator function with yield

These objects that can directly act on a for loop are called iterative objects: iterable

You can use Isinstance () to determine whether an object is an iterable object

>>> from Collections Import iterable>>> isinstance ([], iterable) true>>> isinstance ({}, iterable) true>>> isinstance (' abc ', iterable) true>>> isinstance ((x for X in range), iterable) True >>> isinstance (iterable) False

The generator can not only be used for a for loop, but it can also be called by the next () function and return the next value until the last throw Stopiteration error indicates that the next value cannot be returned again.

So here's a bit of an iterator

An object that can be called by the next () function and continually returns the next value is called an iterator: Iterator.

You can use Isinstance () to determine whether an object is a Iterator object:

>>> from Collections Import iterator>>> isinstance ((x to X in range), Iterator) true>>> are Instance ([], Iterator) false>>> isinstance ({}, Iterator) false>>> isinstance (' abc ', Iterator) False

　　

Generators are Iterator objects, but,, list dict str Though Iterable（可迭代对象） they are, they are not Iterator（迭代器） .

turn list , dict and str wait Iterable for Iterator the function to be used iter() :

>>> Isinstance (ITER ([]), Iterator) true>>> isinstance (ITER (' abc '), Iterator) True

　　

You may ask, why, list dict , str etc. data types are not Iterator ?

This is because the Python Iterator object represents a data stream , and the iterator object can be next() called by the function and will return the next data continuously until there is no data to throw an StopIteration error. You can think of this data stream as an ordered sequence, but we can't know the length of the sequence in advance, only by continuously using the next() function to calculate the next data on demand, so Iterator the calculation is lazy, and it will only be calculated when the next data needs to be returned.

IteratorIt can even represent an infinitely large stream of data, such as the whole natural number. Using list is never possible to store all natural numbers.

Summary:

• Any object that can be used for for the loop is a Iterable type;
• All objects that can be used for next() functions are Iterator types, which represent a sequence of lazy computations;
• Collection data types such as list , dict ,, and str so on are Iterable not Iterator , however, you can iter() get an object from a function Iterator .

Python3 for loops are essentially implemented by calling next() functions, for example:

For x in [1, 2, 3, 4, 5]:    Pass

is actually exactly equivalent to

# Get Iterator Object first: it = iter ([1, 2, 3, 4, 5]) # loop: While True:    try:        # Get Next value:        x = Next (it)    except Stopite Ration:        # exit Loop break when you encounter Stopiteration        

　　

A summary of yield

(1): usual for. In ... Loop, in followed by an array, this array is an iterative object, similar to the linked list, string, file. He can be a = [X-ray], or a = [x*x for x in range (3)].

Its shortcomings are also obvious, that is, all the data in memory, if there is a huge amount of data, will be very memory consumption.

(2) The generator can be iterated, but it can only be read once. It is generated when used, such as a = (x*x for x in range (3)).!!!! Note that this is a parenthesis instead of square brackets.

(3) The key to the iteration of the generator (generator) is that he has the next () method, which works by repeatedly invoking the next () method until an exception is caught.

(4) A function with yield is no longer a normal function, but a generator generator, which can be used for iterative

(5) yield is a keyword similar to return, which returns the value of yield behind or to the right when the yield is encountered in the iteration. And the next iteration, the code that follows the yield from the last iteration starts executing

(6) Yield is a value returned by return and remembers where the return is. The next iteration starts from this position.

(7) A function with yield is not just for a for loop, but for a function parameter, as long as the parameter of the function also allows the iteration parameter.

(8) The difference between send () and next () is that the send can pass parameters to the yield expression, which is passed as the value of the yield expression, and the yield argument is the value returned to the caller, that is, send can forcibly modify the previous yield expression value.

(9) both send () and next () have return values, and their return value is the yield that is encountered by the current iteration, and the value of the expression after yield is actually the argument behind the current iteration yield.

(10) First call must first next () or send (), or will be error, send after the reason is none because there is no previous yield, so you can also think next () equivalent to send (None)

Python builder and iterator this is enough.