Getting started with the Python gevent framework

Source: Internet
Author: User

Getting started with the Python gevent framework

This article mainly introduces the Python gevent framework getting started tutorial. The sample code is based on Python2.x. For more information, see

Python provides basic support for coroutine through yield, but not completely. Third-party gevent provides comprehensive coroutine support for Python.

Gevent is a third-party library that implements coroutine through greenlet. Its basic idea is:

When a greenlet encounters an I/O operation, such as accessing the network, it will automatically switch to another greenlet. After the I/O operation is completed, it will switch back to continue execution when appropriate. Since IO operations are very time-consuming, the program is often waiting. With gevent, We can automatically switch the coroutine to ensure that greenlet is always running, rather than waiting for IO.

Because the switchover is automatically completed during I/O operations, gevent needs to modify some standard libraries that come with Python. This process is completed through monkey patch at startup:

?

1

2

3

4

5

6

7

8

9

10

11

12

13

From gevent import monkey; monkey. patch_socket ()

Import gevent

 

Def f (n ):

For I in range (n ):

Print gevent. getcurrent (), I

 

G1 = gevent. spawn (f, 5)

G2 = gevent. spawn (f, 5)

G3 = gevent. spawn (f, 5)

G1.join ()

G2.join ()

G3.join ()

Running result:

?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

<Greenlet at 0x10e49f550: f (5)> 0

<Greenlet at 0x10e49f550: f (5)> 1

<Greenlet at 0x10e49f550: f (5)> 2

<Greenlet at 0x10e49f550: f (5)> 3

<Greenlet at 0x10e49f550: f (5)> 4

<Greenlet at 0x10e49f910: f (5)> 0

<Greenlet at 0x10e49f910: f (5)> 1

<Greenlet at 0x10e49f910: f (5)> 2

<Greenlet at 0x10e49f910: f (5)> 3

<Greenlet at 0x10e49f910: f (5)> 4

<Greenlet at 0x10e49f4b0: f (5)> 0

<Greenlet at 0x10e49f4b0: f (5)> 1

<Greenlet at 0x10e49f4b0: f (5)> 2

<Greenlet at 0x10e49f4b0: f (5)> 3

<Greenlet at 0x10e49f4b0: f (5)> 4

As you can see, the three greenlets run sequentially rather than alternately.

To enable greenlet to run alternately, you can use gevent. sleep () to hand over control:

?

1

2

3

4

Def f (n ):

For I in range (n ):

Print gevent. getcurrent (), I

Gevent. sleep (0)

Execution result:

?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

<Greenlet at 0x10cd58550: f (5)> 0

<Greenlet at 0x10cd58910: f (5)> 0

<Greenlet at 0x10cd584b0: f (5)> 0

<Greenlet at 0x10cd58550: f (5)> 1

<Greenlet at 0x10cd584b0: f (5)> 1

<Greenlet at 0x10cd58910: f (5)> 1

<Greenlet at 0x10cd58550: f (5)> 2

<Greenlet at 0x10cd58910: f (5)> 2

<Greenlet at 0x10cd584b0: f (5)> 2

<Greenlet at 0x10cd58550: f (5)> 3

<Greenlet at 0x10cd584b0: f (5)> 3

<Greenlet at 0x10cd58910: f (5)> 3

<Greenlet at 0x10cd58550: f (5)> 4

<Greenlet at 0x10cd58910: f (5)> 4

<Greenlet at 0x10cd584b0: f (5)> 4

Three greenlets run alternately,

Change the number of cycles to 500000, so that the running time is longer. Then, in the Process Manager of the operating system, the number of threads is only one.

Of course, in the actual code, we will not use gevent. sleep () to switch the coroutine. Instead, gevent will automatically switch during IO operations. The Code is as follows:

?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

From gevent import monkey; monkey. patch_all ()

Import gevent

Import urllib2

 

Def f (url ):

Print ('get: % s' % url)

Resp = urllib2.urlopen (url)

Data = resp. read ()

Print ('% d bytes encoded ed from % s.' % (len (data), url ))

 

Gevent. joinall ([

Gevent. spawn (f, 'https: // www.python.org /'),

Gevent. spawn (f, 'https: // www.yahoo.com /'),

Gevent. spawn (f, 'https: // github.com /'),

])

Running result:

?

1

2

3

4

5

6

GET: https://www.python.org/

GET: https://www.yahoo.com/

GET: https://github.com/

45661 bytes encoded ed from https://www.python.org /.

14823 bytes encoded ed from https://github.com /.

304034 bytes encoded ed from https://www.yahoo.com /.

The results show that the three network operations are executed concurrently, and the end sequence is different, but there is only one thread.

Summary

Gevent can achieve extremely high concurrency performance, but gevent can only run in Unix/Linux, and cannot be installed and run normally in Windows.

Since gevent is a coroutine Based on IO switching, the most amazing thing is that the Web App code we write does not need to introduce the gevent package or change any code, when deploying a WSGI server that supports gevent, it immediately achieved several times the performance improvement. For details about the deployment method, refer to "practice"-"deploy Web apps.

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.