[Reprint]python Gevent

Original address:

Http://www.liaoxuefeng.com/wiki/001374738125095c955c1e6d8bb493182103fac9270762a000/ 001407503089986d175822da68d4d6685fbe849a0e0ca35000

Thank Liaoche Teacher.

Python provides basic support for the process through yield, but not entirely. Third-party gevent provide Python with a more complete range of support.

Gevent is a third-party library, through the Greenlet implementation of the process, the basic idea is:

When an greenlet encounters an IO operation, such as accessing the network, it automatically switches to the other Greenlet, waits until the IO operation is complete, and then switches back to execution at the appropriate time. Because the IO operation is very time-consuming and often puts the program in a waiting state, with gevent automatically switching the co-process for us, it is guaranteed that there will always be greenlet running, rather than waiting for IO.

Since the switchover is done automatically during IO operations, Gevent needs to modify some of the standard libraries that Python comes with, which is done at startup with Monkey Patch:

from gevent import monkey; monkey.patch_socket()import geventdef f(n): for i in range(n): print gevent.getcurrent(), ig1 = gevent.spawn(f, 5)g2 = gevent.spawn(f, 5)g3 = gevent.spawn(f, 5)g1.join()g2.join()g3.join()

Operation Result:

<GreenletAt0X10E49F550:F5) > 0<GreenletAt0X10E49F550:F5) > 1<GreenletAt0X10E49F550:F5) > 2<GreenletAt0X10E49F550:F5) > 3<GreenletAt0X10E49F550:F5) > 4<GreenletAt0x10e49f910:F5) > 0<GreenletAt0x10e49f910:F5) > 1<GreenletAt0x10e49f910:F5) > 2<GreenletAt0x10e49f910:F5) > 3<GreenletAt0x10e49f910:F5) > 4<GreenletAt0x10e49f4b0: f (5) > 0 <greenlet at  0x10e49f4b0: f (5) > 1< greenlet at 0x10e49f4b0: f (5) > 2<greenlet span class= "attribute" >at 0x10e49f4b0: f ( 5) > 3<greenlet at  0x10e49f4b0: f (5) > 4       

As you can see, 3 Greenlet are run sequentially instead of alternately.

For the Greenlet to run alternately, it can be gevent.sleep() controlled by handing over control:

def f(n):    for i in range(n): print gevent.getcurrent(), i gevent.sleep(0)

Execution Result:

<GreenletAt0X10CD58550:F5) > 0<GreenletAt0x10cd58910:F5) > 0<GreenletAt0X10CD584B0:F5) > 0<GreenletAt0X10CD58550:F5) > 1<GreenletAt0X10CD584B0:F5) > 1<GreenletAt0x10cd58910:F5) > 1<GreenletAt0X10CD58550:F5) > 2<GreenletAt0x10cd58910:F5) > 2<GreenletAt0X10CD584B0:F5) > 2<GreenletAt0X10CD58550:F5) > 3<GreenletAt0x10cd584b0: f (5) > 3 <greenlet at  0x10cd58910: f (5) > 3< greenlet at 0x10cd58550: f (5) > 4<greenlet span class= "attribute" >at 0x10cd58910: f ( 5) > 4<greenlet at  0x10cd584b0: f (5) > 4       

3 Greenlet alternately run,

Change the number of cycles to 500000, let them run longer, and then, in the process manager of the operating system, there are only 1 threads.

Of course, in the actual code, we will not use gevent.sleep() to switch the co-process, but in the implementation of the IO operation, Gevent automatically switch, the code is as follows:

from gevent import monkey; monkey.patch_all()import geventimport urllib2def f(url): print(‘GET: %s‘ % url) resp = urllib2.urlopen(url) data = resp.read() print(‘%d bytes received from %s.‘ % (len(data), url))gevent.joinall([ gevent.spawn(f, ‘https://www.python.org/‘), gevent.spawn(f, ‘https://www.yahoo.com/‘), gevent.spawn(f, ‘https://github.com/‘),])

Operation Result:

GET: https://www.python.org/GET: https://www.yahoo.com/GET: https://github.com/45661 bytes received from https://www.python.org/.14823 bytes received from https://github.com/.304034 bytes received from https://www.yahoo.com/.

From the result, 3 network operations are executed concurrently, and the end order is different, but only one thread.

Summary

With Gevent, you can get very high concurrency performance, but gevent can only run under Unix/linux and is not guaranteed to install and run properly under Windows.

Since Gevent is based on the IO switchover, the most amazing thing is that the Web App code we're writing does not need to introduce gevent packages or any code, and in the deployment, with a WSGI server that supports gevent, it gains several times faster performance. Specific deployment methods can refer to the following "actual combat"-"Deploy Web App" section.

[Reprint]python Gevent