Python third-party library series 17--multiprocessing Library

When it comes to concurrency, we think of multithreading and multi-process.

Does it use multiple processes or multithreading? It depends on the situation. Our procedures are generally divided into:

1) consume the network (a large part of the time is in the network interaction);

2) CPU consumption (full use of multicore)

In the first case, most of the time is occupied by the network delay, so the use of multithreading and multi-process are similar.

In the second case, the length of time is determined by whether or not the CPU is fully utilized, see the Python third-party library series 12-Multi-threaded threading library, the multi-threaded will appear lock mechanism, so more process better, take full advantage of the CPU, save time.

Here is a multi-process example:

#coding =utf-8import osimport tracebackimport timefrom multiprocessing import Process, managercase_counts = 100PROCESS_ COUNTS = 10def Assign_task_index (index): try:if case_counts% process_counts = = 0:section_num = CAS E_counts/process_counts Else:section_num = case_counts/process_counts + 1 if case_counts% s Ection_num = = 0:process_count = Case_counts/section_num Else:process_count = case_counts /Section_num + 1 Start_index = Section_num * (index-1) + 1 End_index = section_num * index if Inde x = = Process_count:end_index = case_counts #print start_index, End_index return start_index, end _index except Exception as E:print Traceback.format_exc () def do_single_process (index, Q): Start_index, Stop_  index = Assign_task_index (index) for I in range (Start_index, stop_index+1): Try:is_even_number = i% 2 if Is_even_number = = 0:q.put (i) Else: #print I pass except Exception as        E:print Traceback.format_exc () def do_multi_process (): Try:process_queue = [] m = Manager () Q = M.queue () # for interprocess communication, record run result for index in range (process_counts): Try:pid = Pr                Ocess (Target=do_single_process, args= (index+1, q)) Except:print Traceback.format_exc ()  Os._exit ( -1) else:process_queue.append (pid) Time_start = Time.time () for        PID in Process_queue:pid.start () # waits for all processes to complete for the PID in Process_queue:pid.join ()               Time_end = Time.time () Time_delta = time_end-time_start print "Total number:%s, even number:%s, odd number:%s, time elapsed (seconds):%s"%        (Str (case_counts), str (q.qsize ()), str (case_counts-q.qsize ()), str (TIME_DELTA)) except Exception as E: Print Traceback.foRmat_exc () if __name__ = = ' __main__ ': do_multi_process () 

Python third-party library series 17--multiprocessing Library