Day 35 About Threads

For single-threaded, we inevitably have IO operations in the program, but if we can control multiple tasks in our own program (that is, at the user program level, not at the operating system level), you can switch to another task to calculate when one task encounters io blocking. This ensures that the thread is in the best possible state, that it can be executed at any time by the CPU, and that we can hide our IO operations to the maximum extent possible at the user program level, thus confusing the operating system to see that the thread seems to have been calculating, the IO is relatively small, This allows more CPU execution permissions to be allocated to our threads.

the nature of the process is that in a single thread, the user controls a task by itself when the IO block is switched on to another task to execute, to improve efficiency . To achieve this, we need to find a solution that can control the switch between multiple tasks, save the state of the task before switching, and then resume execution based on the paused position, so that the IO operation can be detected and the switch occurs in the event of an IO operation.

I. Introduction of the Association Process

1. Definition

The co-process is a single-threaded concurrency, also known as micro-Threading. The process is a kind of user-state lightweight thread, that is, the process is controlled by the user program itself scheduling. emphasized as follows:

(1) The Python thread is at the kernel level, that is, the operating system control scheduling, if the IO or the execution time is too long, the operating system will force the request to surrender the CPU execution permissions, execute other threads;

(2) Single-wire range open the process, once encountered IO will be switched from the application level, thereby improving efficiency. Note: must be encountered IO before cutting

Greenlet Module

2. Advantages

The advantages are as follows:

(1) The switching cost of the co-process is small, it belongs to the program-level switch, the operating system is not aware, thus more lightweight;

(2) Implementation of single-threaded concurrency , the maximum use of the CPU.

3. Disadvantages

Disadvantages are as follows:

(1) The nature of the process is in single-threaded, can not use multi-core;

(2) The co-process refers to a single thread, which blocks the entire thread once it encounters a blocking process.

4, summarize the characteristics of the process:

1. Concurrency must be implemented in only one single thread
2. No lock required to modify shared data
3. The context stack in the user program that holds multiple control flows
4. Additional: A co-process encountered IO operation automatically switch to other Io,yield (how to implement detection, Greenlet can not be implemented, the use of the Gevent module (select mechanism))

Second, Greenlet module

The Greenlet module provides a convenient way to switch back and forth between multiple tasks. The following example:

From Greenlet import Greenletdef Eat (name):    print ('%s is eating1 '%name)    g2.switch (' Alex ')                  #2. Switch execution Play (' Alex ')    print ('%s is eating2 '%name)    g2.switch ()                        #4. Toggle to continue play (' Alex ') def play (name):    print ('%s ' is Playing1 '%name)    g1.switch ()                        #3. Toggle to continue execution eat (' Alex ')    print ('%s is playing2 '%name) g1=greenlet (EAT) g2= Greenlet (play) "Switch only needs to pass the" G1.switch (' Alex ') #1 for the first switch                      . Toggle execution Eat (' Alex ') ' Output: Alex is Eating1alex is Playing1alex is Eating2alex "is Playing2"

The above example Gevent.sleep () simulates the IO that the gevent can recognize, which is meaningless for the actual application. How to solve the blocking of time.sleep () such as gevent unrecognized IO? Take a look at the example below.

2. Gevent Unrecognized IO blocking condition

Instance:

From gevent import Monkey;monkey.patch_all () "From gevent import Monkey;monkey.patch_all () must be placed at the beginning of the file" "Import time, Geventfrom Threading Import Current_threaddef Eat (name):    print ('%s is running1:%s '% (Name,current_thread (). GetName ()))    Time.sleep (2)    print ('%s is running2:%s '% (Name,current_thread (). GetName ())) def play (name):    print ('%s is playing1:%s '% (Name,current_thread (). GetName ()))    Time.sleep (1)    print ('%s is playing2:%s '% (name, Current_thread (). GetName ())) Start=time.time () G1=gevent.spawn ( Eat, ' Alex ')   g2=gevent.spawn (play,name= ' Alex ') Gevent.joinall ([g1,g2]) Stop=time.time () print (Stop-start) " Alex is Running1:dummythread-1alex is Playing1:dummythread-2alex are Playing2:dummythread-2alex is running2:dummythread-12.0029842853546143 ""

The above example is our final implementation of the process, in order to solve the problem of identifying the normal blocking, we use patching way: from gevent import Monkey;monkey.patch_all (), the patch must be placed at the beginning of the file. Results in

Dummythread-n translates to the meaning of pseudo-threading, not to turn on multithreading.

Iv. examples of co-processes

1, the implementation of the crawler instance of the process

From gevent import monkeymonkey.patch_all () Import geventimport requestsfrom threading import current_threaddef get (URL) :    print ('%s get%s '% (Current_thread (). GetName (), URL))    response=requests.get (URL)  #存在IO    if response.status_code==2:        return {' URL ': Len (response.text)}g1=gevent.spawn (GET, ' www.baidu.com ') g2= Gevent.spawn (GET, ' www.qq.com ') g3=gevent.spawn (GET, ' www.jd.com ') G1.join () G2.join () G3.join () print (G1.value)  #value为取值print (g2.value) print (G3.value)

2, the association process to achieve socket communication

Service side:

From gevent import monkeymonkey.patch_all () Import Geventfrom sockets import *from Threading Import Current_threadsever = So Cket (af_inet,sock_stream)  # Default parameters can be sever.setsockopt (sol_socket,so_reuseaddr,1) sever.bind (' 127.0.0.1 ', 8090 ) Sever.listen () def talk (conn,addr):    print ('%s got a connect from%s:%s '% (Current_thread (). GetName (), addr[0], ADDR[1]) while    True:        data = CONN.RECV (1024x768). Decode (' Utf-8 ')        if not data:break        conn.send (data.upper (). Encode ()) if __name__ = = ' __main__ ':    while True:        conn,addr = sever.accept ()        g = Gevent.spawn (Talk,conn, Addr

Client

From socket import *client=socket () Client.connect ((' 127.0.0.1 ', 8090)) while True:    msg=input (' >>> '). Strip ()    if not msg:continue    client.send (Msg.encode ())    Res=client.recv (1024x768). Decode ()    print (res) # Multithreaded open Client # from threading import thread# from socket import *# client=socket () # Client.connect ((' 127.0.0.1 ', 8090)) # Def Talk (client): #     client.send (' Hello '. Encode ()) #     Res=client.recv (1024x768). Decode () #     Print (res) # # if __name__ = = ' __main__ ': # for     i in range: #         T=thread (target=talk,args= (client)) #         T.start ()

Day 35 About Threads