List-Generated
Now there's a need to look at the list [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
and ask you to add 1 to each value in the table, how do you do that? You might think of 2 ways
Two-Force Youth edition
>>> a[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]>>> b = []>>> for i in a:b.append(i+1)... >>> b[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]>>> a = b>>> a[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
Normal Youth Edition
a = [1,3,4,6,7,7,8,9,11]for index,i in enumerate(a): a[index] +=1print(a)
Literary Youth Edition
>>> a[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]>>> a = map(lambda x:x+1, a)>>> a<map object at 0x101d2c630>>>> for i in a:print(i)... 357911
In fact, there is a way of writing, as follows
Forced Youth Edition
>>> a = [i+1 for i in range(10)]>>> a[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
This is called a list-generation
Generator
With list generation, we can create a list directly. However, with memory limitations, the list capacity is certainly limited. Also, creating a list of 1 million elements takes up a lot of storage space, and if we just need to access the first few elements, the vast majority of the space behind it is wasted.
So, if the list element can be calculated according to an algorithm, can we continue to calculate the subsequent elements in the process of the loop? This eliminates the need to create a complete list, which saves a lot of space. In Python, this side loop computes the mechanism, called the generator: Generator.
There are a number of ways to create a generator. The first method is simple, as long as you change a list-generated form to []
(),
create a generator:
>>> L = [x * x for x in range(10)]>>> L[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]>>> g = (x * x for x in range(10))>>> g<generator object <genexpr> at 0x1022ef630>
The L
difference between creating and making is g
only the outermost []
and, a ()
L
list, and g
a generator.
We can print out every element of the list directly, but how do we print out every element of generator?
If you want to print out one, you can next()
get the next return value for generator by using a function:
>>> next(g)0>>> next(g)1>>> next(g)4>>> next(g)9>>> next(g)16>>> next(g)25>>> next(g)36>>> next(g)49>>> next(g)64>>> next(g)81>>> next(g)Traceback (most recent call last): File "<stdin>", line 1, in <module>StopIteration
As we've said, generator holds the algorithm, each time the call next(g)
computes g
the value of the next element, until the last element is computed, and no more elements are thrown when the StopIteration
error occurs.
Of course, this constant invocation is next(g)
so perverted that the correct approach is to use for
loops, because generator is also an iterative object:
>>> g = (x * x for x in range(10))>>> for n in g:... print(n)...0149162536496481
So, after we create a generator, we basically never call next (), but instead iterate over it with a for loop and don't need to care about Stopiteration's error.
Generator is very powerful. If the algorithm is more complex, it can also be implemented by using a function that is not possible with a For loop that resembles a list-generated type.
For example, the famous Fibonacci sequence (Fibonacci), except for the first and second numbers, can be summed up by the top two numbers:
1, 1, 2, 3, 5, 8, 13, 21, 34, ...
The Fibonacci sequence is not written in a list, but it is easy to print it out with a function:
def fib(max): n, a, b = 0, 0, 1 while n < max: print(b) a, b = b, a + b n = n + 1 return ‘done‘
Note that the assignment statement:
a, b = b, a + b
Equivalent:
t = a + b a = b b = t
However, you do not have to explicitly write out a temporary variable T to assign a value.
The above function can output the number of the first n of the Fibonacci sequence:
>>> fib(10)11235813213455done
Looking closely, it can be seen that the fib
function is actually a calculation rule that defines the Fibonacci sequence, which can be derived from the first element, and the subsequent arbitrary elements, which are actually very similar to generator.
In other words, the above functions and generator are only a step away. To turn fib
a function into a generator, just print(b)
change yield b
it to:
def fib(max): n,a,b = 0,0,1 while n < max: #print(b) yield b a,b = b,a+b n += 1 return ‘done‘
This is another way to define generator. If a function definition contains a yield
keyword, then the function is no longer a normal function, but a generator:
>>> f = fib(6)>>> f<generator object fib at 0x104feaaa0>
The most difficult thing to understand here is that the generator is not the same as the execution flow of the function. The function is executed sequentially, the return statement is encountered, or the last line function statement is returned. The function that becomes generator, executes at each call to next (), encounters a yield statement return, and is again called by next () to continue execution from the last yield statement returned.
data = fib(10)print(data)print(data.__next__())print(data.__next__())print("干点别的事")print(data.__next__())print(data.__next__())print(data.__next__())print(data.__next__())print(data.__next__())#输出<generator object fib at 0x000002E33EEFFCA8>11干点别的事235813
在上面fib
example, we continue to call in the loop process yield
, will continue to interrupt. Of course, you have to set a condition for the loop to exit the loop, or it will produce an infinite sequence. Similarly, after changing a function to generator, we basically never use it next()
to get the next return value, but instead use the for
loop directly to iterate:
>>> for n in fib(6):... print(n)...112358
However, when you call generator with a For loop, you find that you cannot get the return value of the generator return statement. If you want to get the return value, you must catch the Stopiteration error, and the return value is contained in the value of stopiteration:
>>> g = fib(6)>>> while True:... try:... x = next(g)... print(‘g:‘, x)... except StopIteration as e:... print(‘Generator return value:‘, e.value)... break...g: 1g: 1g: 2g: 3g: 5g: 8Generator return value: done
The following error handling will also be explained in detail on how to catch errors.
The effect of concurrency can also be achieved in single-threaded case with yield
#_*_coding:utf-8_*___author__ = ‘Alex Li‘import timedef consumer(name): print("%s 准备吃包子啦!" %name) while True: baozi = yield print("包子[%s]来了,被[%s]吃了!" %(baozi,name))def producer(name): c = consumer(‘A‘) c2 = consumer(‘B‘) c.__next__() c2.__next__() print("老子开始准备做包子啦!") for i in range(10): time.sleep(1) print("做了2个包子!") c.send(i) c2.send(i)producer("alex")通过生成器实现协程并行运算
Iterators
We already know that for
there are several types of data that can be directly acting on a loop:
A class is a collection of data types, such as,,, list
tuple
, and dict
set
str
so on;
One is generator
to include the generator and yield
the generator function with the band.
These objects, which can be directly applied to for
the loop, are called iterative objects: Iterable
.
You can use to isinstance()
determine whether an object is an Iterable
object:
>>> from collections import Iterable>>> isinstance([], Iterable)True>>> isinstance({}, Iterable)True>>> isinstance(‘abc‘, Iterable)True>>> isinstance((x for x in range(10)), Iterable)True>>> isinstance(100, Iterable)False
The generator can not only be used for a for loop, but it can also be called by the next () function and return the next value until the last throw Stopiteration error indicates that the next value cannot be returned again.
An object that can be called by the next () function and continually returns the next value is called an iterator: Iterator.
You can use Isinstance () to determine whether an object is a iterator object:
>>> from collections import Iterator>>> isinstance((x for x in range(10)), Iterator)True>>> isinstance([], Iterator)False>>> isinstance({}, Iterator)False>>> isinstance(‘abc‘, Iterator)False
Generators are Iterator
objects, but,, list
dict
str
Though Iterable
they are, they are not Iterator
.
Turn list
, dict
and str
wait for the Iterable
Iterator
function to be used iter()
:
>>> isinstance(iter([]), Iterator)True>>> isinstance(iter(‘abc‘), Iterator)True
You may ask, why, list
dict
, str
etc. data types are not Iterator
?
This is because the Python Iterator
object represents a data stream, and the iterator object can be next()
called by the function and will return the next data continuously until there is no data to throw an StopIteration
error. You can think of this data stream as an ordered sequence, but we can't know the length of the sequence in advance, only by continuously using the next()
function to calculate the next data on demand, so Iterator
the calculation is lazy, and it will only be calculated when the next data needs to be returned.
Iterator
It can even represent an infinitely large stream of data, such as the whole natural number. Using list is never possible to store all natural numbers.
Summary
Any object that can be used for for
the loop is a Iterable
type;
All objects that can be used for next()
functions are Iterator
types, which represent a sequence of lazy computations;
Collection data types such as list
, dict
,, and str
so on are Iterable
not Iterator
, however, you can iter()
get an object from a function Iterator
.
Python3 for
loops are essentially implemented by calling next()
functions, for example:
for x in [1, 2, 3, 4, 5]: pass
is actually exactly equivalent to:
# 首先获得Iterator对象:it = iter([1, 2, 3, 4, 5])# 循环:while True: try: # 获得下一个值: x = next(it) except StopIteration: # 遇到StopIteration就退出循环 break
Python Path Builder & iterator