Use Concurrent.futures and processpoolexecutor to replace threads and processes

Concurrent.futures and Processpoolexecutor These two classes implement the pretext of executing callable objects in different threads or processes respectively, the two classes in the internal maintainer of a worker thread or process pool, and the queue to be executed, both of which have high levels of abstraction that do not require attention to implementation details

Common method for downloading the national flag

Import Osimport timeimport sysimport requestspop20_cc= (' CN in US ID BR PK NG BD RU JP '          MX PH VN ET EG DE IR TR CD FR ' ). Split () base_url= ' http://flupy.org/data/flags ' dest_dir= ' downloads/' def save_flag (img,filename):    path= Os.path.join (dest_dir,filename)    with open (path, ' WB ') as FP:        Fp.write (IMG) def get_flag (cc):    url= ' {}/{cc }/{cc}.gif '. Format (Base_url,cc=cc.lower ())    Resp=requests.get (URL)    return resp.contentdef Show (text):    Print (text,end= "")    Sys.stdout.flush () def download_many (cc_list): For    cc in sorted (cc_list):        Image=get_flag (CC)        show (cc)        Save_flag (image,cc.lower () + '. gif ')    return Len (cc_list) def main ( Download_many):    to=time.time ()    count=download_many (POP20_CC)    elapsed=time.time ()    msg= ' \n{} Flags downloaded in {:. 2f}s '    print (Msg.format (count,elapsed)) if __name__== ' __main__ ':    Main (Download_many )

Alternative multithreading methods

#!/usr/bin/env python#-*-coding:utf-8-*-from Concurrent Import futuresimport sys# sys.path.append (r "F:\regtest\ Asyncio! ") # from QW import save_flag,get_flag,show, mainimport osimport timeimport sysmax_workers=20import requestspop20_cc= (' CN in US ID BR PK NG BD RU JP ' MX PH VN ET EG DE IR TR CD FR '). Split () base_url= ' Http://flupy.org/data/flags ' DEST        _dir= ' downloads/' def save_flag (img,filename): Path=os.path.join (Dest_dir,filename) with open (path, ' WB ') as FP: Fp.write (IMG) def get_flag (cc): Url= ' {}/{cc}/{cc}.gif '. Format (Base_url,cc=cc.lower ()) resp=requests.get (URL) retu RN resp.contentdef Show (text): Print (text,end= "") Sys.stdout.flush () def download_one (cc): Image=get_flag (CC) s How (CC) Save_flag (image,cc.lower () + '. gif ') return ccdef Download_many (cc_list): Workers=min (Max_workers,len (cc_li ST)) with futures.  Threadpoolexecutor (workers) as Executor:res=executor.map (download_one,sorted (cc_list)) Return Len (List (res)) def Main (dOwnload_many): To=time.time () Count=download_many (POP20_CC) elapsed=time.time () msg= ' \n{} flags downloaded in {:. 2f}s ' Print (Msg.format (count,elapsed)) if __name__== ' __main__ ': Main (Download_many)

With this artifact, it is no longer necessary to create a thread pool or process pool and queue to handle the problem. ~

about how to create thread pool process pools, in the Python basic article ~

Use Concurrent.futures and processpoolexecutor to replace threads and processes