An introductory tutorial on Python's gevent framework

This article mainly introduces the introduction of Python's gevent framework, sample code based on the python2.x version, the need for friends can refer to the following

Python provides basic support for the coprocessor through yield, but not completely. Third-party gevent provides a more complete support for Python.

Gevent is the third party library, through the Greenlet implementation of the process, the basic idea is:

When a greenlet encounters an IO operation, such as accessing the network, it automatically switches to the other Greenlet, waits until the IO operation completes, and then switches back to continue execution at the appropriate time. Because IO operation is very time-consuming, often put the program in a waiting state, with gevent for us to automatically switch the coprocessor, it is guaranteed that there is always greenlet running, rather than waiting for IO.

Because switching is done automatically during IO operations, Gevent needs to modify some of the standard libraries that Python has brought with it, which is done through monkey patch at startup:

?

1 2 3 4 5 6 7 8 9 10 11 12-13

from gevent import Monkey; Monkey.patch_socket () Import gevent def f (n): For I in range (n): Print gevent.getcurrent (), I g1 = Gevent.spawn (f, 5) G2 = Gevent.spawn (f, 5) G3 = Gevent.spawn (f, 5) G1.join () G2.join () () G3.join ()

Run Result:

?

1 2 3 4, 5 6 7 8 9 10 11 12 13 14 15

<greenlet at 0x10e49f550:f (5) > 0 <greenlet at 0x10e49f550:f (5) > 1 <greenlet at 0x10e49f550:f (5) > 2 &L T Greenlet at 0x10e49f550:f (5) > 3 <greenlet at 0x10e49f550:f (5) > 4 <greenlet at 0x10e49f910:f (5) > 0 <gr Eenlet at 0x10e49f910:f (5) > 1 <greenlet at 0x10e49f910:f (5) > 2 <greenlet at 0x10e49f910:f (5) > 3 <gree Nlet at 0x10e49f910:f (5) > 4 <greenlet at 0x10e49f4b0:f (5) > 0 <greenlet at 0x10e49f4b0:f (5) > 1 <greenl ET at 0x10e49f4b0:f (5) > 2 <greenlet at 0x10e49f4b0:f (5) > 3 <greenlet at 0x10e49f4b0:f (5) > 4

As you can see, 3 Greenlet are run sequentially rather than alternately.

To allow Greenlet to run alternately, you can surrender control by Gevent.sleep ():

?

1 2 3 4

def f (n): For I in range (n): Print gevent.getcurrent (), I gevent.sleep (0)

Execution results:

?

1 2 3 4, 5 6 7 8 9 10 11 12 13 14 15

<greenlet at 0x10cd58550:f (5) > 0 <greenlet at 0x10cd58910:f (5) > 0 <greenlet at 0x10cd584b0:f (5) > 0 &L T Greenlet at 0x10cd58550:f (5) > 1 <greenlet at 0x10cd584b0:f (5) > 1 <greenlet at 0x10cd58910:f (5) > 1 <gr Eenlet at 0x10cd58550:f (5) > 2 <greenlet at 0x10cd58910:f (5) > 2 <greenlet at 0x10cd584b0:f (5) > 2 <gree Nlet at 0x10cd58550:f (5) > 3 <greenlet at 0x10cd584b0:f (5) > 3 <greenlet at 0x10cd58910:f (5) > 3 <greenl ET at 0x10cd58550:f (5) > 4 <greenlet at 0x10cd58910:f (5) > 4 <greenlet at 0x10cd584b0:f (5) > 4

3 Greenlet running Alternately,

Change the number of loops to 500000 to allow them to run longer, and then see only 1 threads in the operating system's process manager.

Of course, in the actual code, we do not use Gevent.sleep () to switch the coprocessor, but when performing to IO operation, Gevent automatically switch, the code is as follows:

?

1 2 3 4, 5 6 7 8 9 10 11 12 13 14 15

from gevent import Monkey; Monkey.patch_all () Import gevent import urllib2 def f (URL): Print (' Get:%s '% url) resp = urllib2.urlopen (URL) data = RE Sp.read () print ('%d bytes received from%s '% (len (data), URL)) Gevent.joinall ([Gevent.spawn (F, ' https://www.python.or g/'), Gevent.spawn (F, ' https://www.yahoo.com/'), Gevent.spawn (F, ' https://github.com/'),]

Run Result:

?

1 2 3 4 5 6

get:https://www.python.org/get:https://www.yahoo.com/get:https://github.com/45661 bytes received from https:// www.python.org/. 14823 bytes received from https://github.com/. 304034 bytes received from https://www.yahoo.com/.

As a result, 3 network operations are executed concurrently, and the ending order is different, but there is only one thread.

Summary

With Gevent, you can achieve extremely high concurrency performance, but gevent can only run under Unix/linux and does not guarantee normal installation and operation under Windows.

Since Gevent is a coprocessor based on Io switching, so the most amazing thing is, we write the Web App code, do not need to introduce gevent package, and do not need to change any code, only in the deployment, with a support gevent WSGI server, immediately achieved several times the performance of the upgrade. Specific deployment methods can refer to the following "combat"-"Deployment Web App" section.