"Python" iterator, generator, yield single-threaded asynchronous concurrency implementation detailed

Transfer from http://blog.itpub.net/29018063/viewspace-2079767

Everyone in the study of Python development may often have doubts about the use of iterators, generators, yield keyword usage, in this article will be from the theory + program debugging verification of the way in detail this part of the knowledge, not much to say, directly into the subject.

One, iterator (iterater):
First, an iterator is introduced, an iterator is a way to access the elements of the collection, and the iterator object is accessed from the first element of the collection until all the elements have been accessed and finished. Does it feel like a for-loop? However, there are several characteristics of iterators that you should remember:
1, visitors do not need to care about the internal structure of the iterator, just keep executing next () method to get the next content;
2, can not randomly access a value in the collection, can only read from beginning to end sequential;
3, access to half can not be returned, can not access the previous value;
4. It is suitable for traversing large data collection, saving memory and improving speed.

According to the above characteristics should have some understanding of the iterator, I would like to give an example to deepen understanding, everyone has played Linux, in Linux there are two commands, VI and CAT, the role is to display the contents of the file, and when a file is very large (such as 1G), Using the VI command to open the file is significantly slower, have realized it, will be a long time, and the use of the Cat command is not the problem, no matter how large files, the execution of the cat command will be immediately from the first row of data printing, the reason is very simple, the use of VI need to load the entire file into memory and then display it When using cat, the first line of the file starts with a row-by-line read to the memory display, but the read content is released directly, so that each read into memory only a row of records of the response speed of course fast.
In fact, the cat here is the use of the idea of an iterator, each time the iterator to take a value in the collection to memory, the end is obsolete, then remove a value, corresponding to the characteristics of 1, for a large file traversal is fast, corresponding to the characteristics of 4; The previous value is recovered after reading the memory, so it cannot be read repeatedly, corresponding to the attribute 3; This is where you can already fully understand the principle of iterators, and the following code demonstrates:
1. Create an iterator:
A = ITER (["Wang", "Xuqian", "Xiaozhuzi"]) # An Iterator object has been created and the values that need to be iterated are set.
2. Iterate through the iterator data:
Print (a.__next__ ())
Print (a.__next__ ())
Print (a.__next__ ()) # has three values, so execute three times next () method
3. Results:
Wang
Xuqian
Xiaozhuzi
It is so simple, because of the characteristics of the iterator, we can only sequentially take the value, not as a list can take any value in the collection, where three values are removed after the execution of a.__next__ (), will be an error: "Stopped iteration"
Traceback (most recent):
Print (a.__next__ ())
Stopiteration

second, generator (Generator) and yield keywords:
Generator definition: When a function is called and an iterator is returned, the function is called the generator, and if the function includes the yield syntax, the function is a generator
Yield: The effect is to break the function, and save the state of the interrupt, after the interruption, the code can continue to execute, after a period of time can also recall the function, and can be from the last yield of the following code to start execution; yield can return a value, you can also receive the parameters of send.
The explanation of the generator and yield is rather abstract and difficult to understand, so don't worry, let's show them how to use it by debugging code, and the charm of yield! ~
Def cash (account): # First define a function to receive a parameter
While account > 0: # when parameter >0
Account-= #减去100
Yield #中断函数吗, return 100
Print ("Come and fetch the money again!") ") #打印
According to the generator definition, at this point the cash function is a generator, as a generator, when called after the resulting generator object is definitely to support the iterator interface, that is, the generator returns an iterator, the following we verify:
A = Cash (500) # Call generator, build iterator
Print (a.__next__ ()) # Print the first iteration value using the next () method of the iterator
Execution results are: 100
The above verifies that the call generator cash does generate an iterator, there should be no doubt about the iterator, you can continue next () to the value of the account = = 0, and we must now have a question, what is the effect of the yield keyword? Why does print result without print ("Come again to get the money!") ")？ Then I'll run through the program, analyzing the Order of code execution through the results:
Def cash (account):
While account > 0:
Account-= 100
Yield 100
Print ("Start from me!") ") Print (" The Code is done ")

A = Cash (500)

Print ("No. 0 time to execute Me")

Print (a.__next__ ())

Print ("Execute me for the first time")

Print (a.__next__ ())

The result of the operation is:
No. 0 Time to execute my
100
The first time I executed
From the beginning I began to execute!
The code is done!
100

From the running result it can be inferred that when the program goes to a = Cash (500), the function does not make a call but continues to go down print "No. 0 time Execute Me", why? It is precisely because at the time of the function definition, when the function is detected that the yield keyword is written, the function becomes a generator, and the function pauses to run; when it executes to the first print (a.__next__ ()), it starts the real calling function, and the function executes to yield 100. , the function breaks back 100 and prints, according to yield characteristics, and the program continues to print "First execute me"; When executing to the second print (a.__next__ ()), the function continues to be called, according to the yield characteristics,resumes execution of the function at the last interrupted position,So print "Start from me down!" "The code is finished," and according to the while loop in the function, the loop code is executed again until yield 100, and the function breaks back 100 again and prints. Here, the program is finished!

Iii. yieldsingle-thread asynchronous concurrency implementation
According to the above program demonstration, we should have more profound understanding of yield and generator characteristics, the yield broke the regular code execution order, even can jump out of the loop, and the next call to the function (generator) can also go down the last time the code execution, is not very bull X? So what is the nature of yield?
For example, I go to the bank to withdraw money, add a large amount of withdrawal, 1 million ~, the bank needs to review, the allocation of funds and other formalities, if it takes 1 days, then this day I will wait there, this is obviously unreasonable it. The best way is that I continue to do other things, and then notify me after the bank approval to withdraw money. The same is true for the code we write, since we are single-threaded serial execution code, what if the function does not return the result when a function is called? According to the previous thinking, then the program can only die there has been waiting, and to go to collect money and other banks to approve a truth; the yield characteristics, breaking this constraint, so that we in the process of single-threaded programming, can still achieve asynchronous concurrency! Let's look at a piece of code below:
Import time
DEF consumer (name):
Print ("%s ready to eat barbecue!") "%name)
While True:
Shaokao = yield
Print ("BBQ%s was eaten by%s!") "% (Shaokao,name))

def producter (name):
c = Consumer ("A")
C1 = Consumer ("B")
C.__next__ ()
C1.__next__ ()
Print ("Start firing skewers!") ")
For I in range (3):
Time.sleep (1)
Print ("Roast two skewers of lamb")
C.send (i)
C1.send (i)

Producter ("Jiaqi")

According to the previous knowledge, it should be possible to obtain the result of execution:
A get ready to have A barbecue!
B prepare to have a barbecue!
Start the fire and grill the skewers!
Grilled two strings of lamb
BBQ 0 was eaten by a!
Grill 0 was eaten by B!
Grilled two strings of lamb
BBQ 1 was eaten by a!
Grill 1 was eaten by B!
Grilled two strings of lamb
BBQ 2 was eaten by a!
Grill 2 was eaten by B!

In this case, asynchronous concurrency control is implemented, and a call is made between a function and a generator, which is achieved through yield; there is also a knowledge point, C.send (i), that the yield can not only return the value, but also receive the value, where send () can be understood as next (), Is the code that triggers the call generator, but next () can be understood as passing a null value to Yield,send () to pass an actual value to yield. The above code will pass the I value to yield in generator consumer

"Python" iterator, generator, yield single-threaded asynchronous concurrency implementation detailed