Python process, thread, coprocessor, io multiplexing

Threading Process Introduction

1. Working minimum unit is thread
2. At least one of the processes---at least one thread
3. Application Scenarios:

IO-Intensive: Threading

Compute Intensive: Process

4. GIL, Global interpreter lock.

Ensure that only one thread in the same process is scheduled at the same time

Thread

1. Basic use

def Task (ARG):    time.sleep (Arg    )print(ARG) for in range (5):     = Threading. Thread (target=task,args=[I,])    #  T.setdaemon (True) # The main thread terminates without waiting for the child thread     #  T.setdaemon (False)    t.start ()    #  t.join () # Always waiting    # T.join (1) # Wait for the maximum time

2. Lock

ImportThreadingImportTimev= 10# 1. Only one person can use lock locks= Threading. Lock ()#You can only open one .Lock = Threading. Rlock ()#you can drive more .defTask (ARG): Time.sleep (2)    #apply for lock, other people, etc.Lock.acquire () lock.acquire ( )Globalv v-= 1Print(v)#Releaselock.release () lock.release ( ) forIinchRange (10): T= Threading. Thread (target=task,args=(i,)) T.start ()

1. Only one person can use a lock
# lock = Threading. Lock () # can only open one
# lock = Threading. Rlock () # can open a lot
2. Multiple people using locks at the same time
# lock = Threading. Boundedsemaphore (3)
3. Restrictions on the release of all locks
#lock = Threading. Event ()
4. Arbitrary
#lock = Threading. Condition ()

Under what circumstances does the thread pool use the thread pool? 1. Single task processing time is relatively short 2. The benefits of using the thread pool for a large number of tasks to be processed: 1. Reduce the time spent on creating and destroying threads and the overhead of system resources 2. If you do not use a thread pool, it can cause the system to create a large number of threads that consume system memory and "over-cut Change ".

Mode one: Direct processing

defTask (URL):"""The task performs two actions: download; save a local"""    #all data for HTTP request response encapsulated in response    #-URL of Response.url request    #-Response.status_code Response status Code    #-Response.text response content (string format)    #-response.content response content (byte format)    #DownloadResponse =requests.get (URL)#download content to localf = open ('A.log','WB') F.write (response.content) f.close () Pool= Threadpoolexecutor (2) Url_list= [    'http://www.oldboyedu.com',    'http://www.autohome.com.cn',    'http://www.baidu.com',] forUrlinchurl_list:Print('Start Request', URL)#go to the connection pool to get the linkPool.submit (Task,url)

Mode two: Step-up processing

defSave (future):"""do save only # future contains response"""Response=Future.result ()#download content to localf = open ('A.log','WB') F.write (response.content) f.close ()defTask (URL):"""only do download requests"""    #all data for HTTP request response encapsulated in response    #-URL of Response.url request    #-Response.status_code Response status Code    #-Response.text response content (string format)    #-response.content response content (byte format)    #DownloadResponse =requests.get (URL)returnResponsepool= Threadpoolexecutor (2) Url_list= [    'http://www.oldboyedu.com',    'http://www.autohome.com.cn',    'http://www.baidu.com',] forUrlinchurl_list:Print('Start Request', URL)#go to the connection pool to get the link    #The future contains responseFuture =pool.submit (Task,url)#after the download succeeds, the Save method is called automaticallyFuture.add_done_callback (Save)

Process

1. Basic use

 fromMultiprocessingImportProcessImport TimedefTask (ARG): Time.sleep (ARG)Print(ARG)if __name__=='__main__':     forIinchRange (10): P= Process (target=task,args=(i,)) P.daemon=True#P.daemon = FalseP.start ()#p.join (1)Print('The main process finally ...')

2. Data sharing between processes
Something special.
-Array (' type ', length)
-Manager (). List ()/manager (). Dict ()

3. Process Pool

Same as the thread pool.

The operation of the thread and process is triggered by the program to trigger the system interface, the final performer is the system, and the operation of the coprocessor is the programmer. The significance of the existence of the process: for multi-threaded applications, the CPU by slicing the way to switch between threads of execution, thread switching takes time (save state, next continue).　　, only one thread is used, and a code block execution order is specified in one thread. Application scenario: When there are a large number of operations in the program that do not require the CPU (IO), it is suitable for the association process;

#co-process#From greenlet import Greenlet##def test1 ():#print (All)#Gr2.switch ()#print ($)#Gr2.switch ()##def test2 ():#print (+)#Gr1.switch ()#Print (%)##Gr1 = Greenlet (test1)#GR2 = Greenlet (test2)#Gr1.switch ()# Two development according to the Association process:+IO fromGeventImportmonkey; Monkey.patch_all ()ImportgeventImportRequestsdeff (URL): Response=requests.get (URL)Print(Response.url,response.status_code) Gevent.joinall ([Gevent.spawn (F,'http://www.oldboyedu.com/'), Gevent.spawn (F,'http://www.baidu.com/'), Gevent.spawn (F,'http://github.com/'),])

IO multiplexing

Python process, thread, coprocessor, io multiplexing