Getting started with the Python gevent framework
This article mainly introduces the Python gevent framework getting started tutorial. The sample code is based on Python2.x. For more information, see
Python provides basic support for coroutine through yield, but not completely. Third-party gevent provides comprehensive coroutine support for Python.
Gevent is a third-party library that implements coroutine through greenlet. Its basic idea is:
When a greenlet encounters an I/O operation, such as accessing the network, it will automatically switch to another greenlet. After the I/O operation is completed, it will switch back to continue execution when appropriate. Since IO operations are very time-consuming, the program is often waiting. With gevent, We can automatically switch the coroutine to ensure that greenlet is always running, rather than waiting for IO.
Because the switchover is automatically completed during I/O operations, gevent needs to modify some standard libraries that come with Python. This process is completed through monkey patch at startup:
?
|
1
2
3
4
5
6
7
8
9
10
11
12
13 |
From gevent import monkey; monkey. patch_socket ()
Import gevent
Def f (n ):
For I in range (n ):
Print gevent. getcurrent (), I
G1 = gevent. spawn (f, 5)
G2 = gevent. spawn (f, 5)
G3 = gevent. spawn (f, 5)
G1.join ()
G2.join ()
G3.join ()
|
Running result:
?
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15 |
<Greenlet at 0x10e49f550: f (5)> 0
<Greenlet at 0x10e49f550: f (5)> 1
<Greenlet at 0x10e49f550: f (5)> 2
<Greenlet at 0x10e49f550: f (5)> 3
<Greenlet at 0x10e49f550: f (5)> 4
<Greenlet at 0x10e49f910: f (5)> 0
<Greenlet at 0x10e49f910: f (5)> 1
<Greenlet at 0x10e49f910: f (5)> 2
<Greenlet at 0x10e49f910: f (5)> 3
<Greenlet at 0x10e49f910: f (5)> 4
<Greenlet at 0x10e49f4b0: f (5)> 0
<Greenlet at 0x10e49f4b0: f (5)> 1
<Greenlet at 0x10e49f4b0: f (5)> 2
<Greenlet at 0x10e49f4b0: f (5)> 3
<Greenlet at 0x10e49f4b0: f (5)> 4
|
As you can see, the three greenlets run sequentially rather than alternately.
To enable greenlet to run alternately, you can use gevent. sleep () to hand over control:
?
|
1
2
3
4 |
Def f (n ):
For I in range (n ):
Print gevent. getcurrent (), I
Gevent. sleep (0)
|
Execution result:
?
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15 |
<Greenlet at 0x10cd58550: f (5)> 0
<Greenlet at 0x10cd58910: f (5)> 0
<Greenlet at 0x10cd584b0: f (5)> 0
<Greenlet at 0x10cd58550: f (5)> 1
<Greenlet at 0x10cd584b0: f (5)> 1
<Greenlet at 0x10cd58910: f (5)> 1
<Greenlet at 0x10cd58550: f (5)> 2
<Greenlet at 0x10cd58910: f (5)> 2
<Greenlet at 0x10cd584b0: f (5)> 2
<Greenlet at 0x10cd58550: f (5)> 3
<Greenlet at 0x10cd584b0: f (5)> 3
<Greenlet at 0x10cd58910: f (5)> 3
<Greenlet at 0x10cd58550: f (5)> 4
<Greenlet at 0x10cd58910: f (5)> 4
<Greenlet at 0x10cd584b0: f (5)> 4
|
Three greenlets run alternately,
Change the number of cycles to 500000, so that the running time is longer. Then, in the Process Manager of the operating system, the number of threads is only one.
Of course, in the actual code, we will not use gevent. sleep () to switch the coroutine. Instead, gevent will automatically switch during IO operations. The Code is as follows:
?
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15 |
From gevent import monkey; monkey. patch_all ()
Import gevent
Import urllib2
Def f (url ):
Print ('get: % s' % url)
Resp = urllib2.urlopen (url)
Data = resp. read ()
Print ('% d bytes encoded ed from % s.' % (len (data), url ))
Gevent. joinall ([
Gevent. spawn (f, 'https: // www.python.org /'),
Gevent. spawn (f, 'https: // www.yahoo.com /'),
Gevent. spawn (f, 'https: // github.com /'),
])
|
Running result:
?
|
1
2
3
4
5
6 |
GET: https://www.python.org/
GET: https://www.yahoo.com/
GET: https://github.com/
45661 bytes encoded ed from https://www.python.org /.
14823 bytes encoded ed from https://github.com /.
304034 bytes encoded ed from https://www.yahoo.com /.
|
The results show that the three network operations are executed concurrently, and the end sequence is different, but there is only one thread.
Summary
Gevent can achieve extremely high concurrency performance, but gevent can only run in Unix/Linux, and cannot be installed and run normally in Windows.
Since gevent is a coroutine Based on IO switching, the most amazing thing is that the Web App code we write does not need to introduce the gevent package or change any code, when deploying a WSGI server that supports gevent, it immediately achieved several times the performance improvement. For details about the deployment method, refer to "practice"-"deploy Web apps.