An introductory tutorial on Python's gevent framework

Source: Internet
Author: User
Tags join sleep switches

This article mainly introduces the introduction of Python's gevent framework, sample code based on the python2.x version, the need for friends can refer to the following

Python provides basic support for the coprocessor through yield, but not completely. Third-party gevent provides a more complete support for Python.

Gevent is the third party library, through the Greenlet implementation of the process, the basic idea is:

When a greenlet encounters an IO operation, such as accessing the network, it automatically switches to the other Greenlet, waits until the IO operation completes, and then switches back to continue execution at the appropriate time. Because IO operation is very time-consuming, often put the program in a waiting state, with gevent for us to automatically switch the coprocessor, it is guaranteed that there is always greenlet running, rather than waiting for IO.

Because switching is done automatically during IO operations, Gevent needs to modify some of the standard libraries that Python has brought with it, which is done through monkey patch at startup:

?

1 2 3 4 5 6 7 8 9 10 11 12-13 from gevent import Monkey; Monkey.patch_socket () Import gevent def f (n): For I in range (n): Print gevent.getcurrent (), I g1 = Gevent.spawn (f, 5) G2 = Gevent.spawn (f, 5) G3 = Gevent.spawn (f, 5) G1.join () G2.join () () G3.join ()

Run Result:

?

1 2 3 4, 5 6 7 8 9 10 11 12 13 14 15 <greenlet at 0x10e49f550:f (5) > 0 <greenlet at 0x10e49f550:f (5) > 1 <greenlet at 0x10e49f550:f (5) > 2 &L T Greenlet at 0x10e49f550:f (5) > 3 <greenlet at 0x10e49f550:f (5) > 4 <greenlet at 0x10e49f910:f (5) > 0 <gr Eenlet at 0x10e49f910:f (5) > 1 <greenlet at 0x10e49f910:f (5) > 2 <greenlet at 0x10e49f910:f (5) > 3 <gree Nlet at 0x10e49f910:f (5) > 4 <greenlet at 0x10e49f4b0:f (5) > 0 <greenlet at 0x10e49f4b0:f (5) > 1 <greenl ET at 0x10e49f4b0:f (5) > 2 <greenlet at 0x10e49f4b0:f (5) > 3 <greenlet at 0x10e49f4b0:f (5) > 4

As you can see, 3 Greenlet are run sequentially rather than alternately.

To allow Greenlet to run alternately, you can surrender control by Gevent.sleep ():

?

1 2 3 4 def f (n): For I in range (n): Print gevent.getcurrent (), I gevent.sleep (0)

Execution results:

?

1 2 3 4, 5 6 7 8 9 10 11 12 13 14 15 <greenlet at 0x10cd58550:f (5) > 0 <greenlet at 0x10cd58910:f (5) > 0 <greenlet at 0x10cd584b0:f (5) > 0 &L T Greenlet at 0x10cd58550:f (5) > 1 <greenlet at 0x10cd584b0:f (5) > 1 <greenlet at 0x10cd58910:f (5) > 1 <gr Eenlet at 0x10cd58550:f (5) > 2 <greenlet at 0x10cd58910:f (5) > 2 <greenlet at 0x10cd584b0:f (5) > 2 <gree Nlet at 0x10cd58550:f (5) > 3 <greenlet at 0x10cd584b0:f (5) > 3 <greenlet at 0x10cd58910:f (5) > 3 <greenl ET at 0x10cd58550:f (5) > 4 <greenlet at 0x10cd58910:f (5) > 4 <greenlet at 0x10cd584b0:f (5) > 4

3 Greenlet running Alternately,

Change the number of loops to 500000 to allow them to run longer, and then see only 1 threads in the operating system's process manager.

Of course, in the actual code, we do not use Gevent.sleep () to switch the coprocessor, but when performing to IO operation, Gevent automatically switch, the code is as follows:

?

1 2 3 4, 5 6 7 8 9 10 11 12 13 14 15 from gevent import Monkey; Monkey.patch_all () Import gevent import urllib2 def f (URL): Print (' Get:%s '% url) resp = urllib2.urlopen (URL) data = RE Sp.read () print ('%d bytes received from%s '% (len (data), URL)) Gevent.joinall ([Gevent.spawn (F, ' https://www.python.or g/'), Gevent.spawn (F, ' https://www.yahoo.com/'), Gevent.spawn (F, ' https://github.com/'),]

Run Result:

?

1 2 3 4 5 6 get:https://www.python.org/get:https://www.yahoo.com/get:https://github.com/45661 bytes received from https:// www.python.org/. 14823 bytes received from https://github.com/. 304034 bytes received from https://www.yahoo.com/.

As a result, 3 network operations are executed concurrently, and the ending order is different, but there is only one thread.

Summary

With Gevent, you can achieve extremely high concurrency performance, but gevent can only run under Unix/linux and does not guarantee normal installation and operation under Windows.

Since Gevent is a coprocessor based on Io switching, so the most amazing thing is, we write the Web App code, do not need to introduce gevent package, and do not need to change any code, only in the deployment, with a support gevent WSGI server, immediately achieved several times the performance of the upgrade. Specific deployment methods can refer to the following "combat"-"Deployment Web App" section.

Related Article

E-Commerce Solutions

Leverage the same tools powering the Alibaba Ecosystem

Learn more >

Apsara Conference 2019

The Rise of Data Intelligence, September 25th - 27th, Hangzhou, China

Learn more >

Alibaba Cloud Free Trial

Learn and experience the power of Alibaba Cloud with a free trial worth $300-1200 USD

Learn more >

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.