Concurrent programming with generators implemented in Python

This article introduces the simple introduction of Python in the use of the generator implementation of concurrent programming, the use of yield generator function for multiple process programming is a Python learning the important knowledge of the advanced, the need for friends can refer to the

We all know that concurrent (not parallel) programming currently has four ways, multiple processes, multiple threads, Asynchrony, and coprocessor.

Multi-process programming has a C-like os.fork in Python, and, of course, a higher-layer multiprocessing standard library, providing a similar nginx master process and worker in a previously written Python highly available programming approach The method of signal processing between processes ensures that the exit of the business process can be perceived by the main process.

Multithreaded programming python has thread and threading, and the so-called threads in Linux are actually LWP lightweight processes that have the same scheduling as processes in the kernel, about Lwp,cow (write-time copies), Fork,vfork, Clone and so on more information, here no longer repeat.

There are three implementations of asynchronous Select,poll,epoll in Linux, and asynchronous is not the focus of this article.

To say that the yield is definitely going to say, let's take a look at an example:

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

#coding =utf-8 Import time Import sys # producer def produce (L): i=0 while 1:if i < 5:l.append (i) yield I i=i+1 time.sleep ( 1 Else:return # Consumer def consume (l): p = Produce (L) while 1:try:p.next () while Len (l) > 0:print l.pop () except St OpIteration:sys.exit (0) L = [] consume (L)

In the above example, when the program executes to the yield I of produce, a generator is returned, and when we call P.next () in custom, the program returns to produce I continue to execute, so l yield the element. Then we print l.pop () until P.next () throws a Stopiteration exception.

We can see from the example above that the scheduling of the coprocessor is not visible to the kernel, and the coordination between the processes is coordinated, which makes the concurrent volume in the tens of thousands of times, the performance of the coprocessor is far higher than the thread.

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

Import stackless Import urllib2 def output (): While 1:url=chan.receive () Print URL f=urllib2.urlopen (URL) #print f.read () Print stackless.getcurrent () def input (): F=open (' Url.txt ') l=f.readlines () for I in L:chan.send (i) Chan=stackless.cha Nnel () [Stackless.tasklet (output) () for I in Xrange ()] Stackless.tasklet (input) () Stackless.run ()

About the process, you can refer to the implementation of Greenlet,stackless,gevent,eventlet.