Python's process pool and thread pool

1. When do I use the pool?

The function of a pool is to limit the number of processes started or the number of threads.

When should I limit it?

When the number of concurrent tasks far exceeds the capacity of the computer, that is, the number of processes or threads cannot be opened at once

You should use the pool concept to limit the number of open processes or the number of threads to the extent that the computer can tolerate


2. Synchronous vs Asynchronous

Synchronous, asynchronous refers to two ways of submitting a task

Sync: After submitting a task, wait in place until the task is finished and get the return value of the task before continuing to run the next line of code

Async: After committing a task (binding a callback function), it is not waiting at all, run the next line of code directly, wait until the task has a return value will automatically trigger the callback function.



Process Pool:

 # from Concurrent.futures import processpoolexecutor,threadpoolexecutor 
 # import OS 
 # import Time 
 # Import Random 
 # 
 # def task (n): 
 # print ('%s run ... '%os.getpid ()) 
 # Time.sleep (5) 
 # return n**2< br># 
 # Def parse (future): 
 # time.sleep (1) 
 # Res=future.result () 
 # print ('%s processed%s '% (Os.getpid (), RES)) 
 # 
 # if __name__ = = ' __main__ ': 
 # Pool=processpoolexecutor (4) 
 # # Pool.submit (task,1) 
 # #     Pool.submit (task,2) 
 # # Pool.submit (task,3) 
 # # Pool.submit (task,4) 
 # 
 # start=time.time () 
 # For I in range (1,5): 
 # Future=pool.submit (task,i) 
 # Future.add_done_callback (parse) # Parse will be in Futrue Trigger immediately when there is a return value, and pass the future as a parameter to parse 
 # Pool.shutdown (wait=true) 
 # stop=time.time () 
 # print (' Master ', Os.getpid (), (Stop-start)) 
 
 
 
 
 thread pool: 
 
 
 

From concurrent.futures import Processpoolexecutor,threadpoolexecutor
From threading Import Current_thread
Import OS
Import time
Import Random

def task (n):
    Print ('%s run ... '%current_thread (). Name)
    Time.sleep (5)
    Return n**2

Def parse (future):
    Time.sleep (1)
    Res=future.result ()
    Print ('%s processed%s '% (Current_thread (). Name,res))

if __name__ = = ' __main__ ':
    Pool=threadpoolexecutor (4)
    Start=time.time ()
    For I in range (1,5):
        Future=pool.submit (Task,i)
        Future.add_done_callback (parse) # Parse will trigger immediately when the Futrue has a return value and pass the future as a parameter to the parse
    Pool.shutdown (Wait=true)
    Stop=time.time ()
    Print (' Master ', Current_thread (). Name, (Stop-start))

Python's process pool and thread pool