Python: Process pool, python process pool

Python: Process pool, python process pool

1. Can a process be opened infinitely?

Processes cannot be opened infinitely. The cpu has 16 cores, which means that 16 tasks can be processed simultaneously. If there are 160 processes, each process

0.01 s for processing, and 160 s for a 16-core processor to process 0.5 processes, because the more processes, the more time required.

2. Process pool

Import osimport timeimport randomfrom multiprocessing import Poolfrom multiprocessing import Processdef func (I): I + = 1if _ name _ = '_ main _': p = Pool (5) # Five processes are created. Generally, the value here is the number of cores # + 1. If you do not know the number of cpu cores, you can call the OS. cpu_count () view start = time. time () p. map (func, range (1000) # target = func args = next (iterable) # [(, 3),] p. close () # It is not allowed to add task p to the process pool. join () print (time. time ()-start) start = time. time () l = [] for I in range (1000): p = Process (target = func, args = (I,) # created one hundred processes p. start () l. append (p) [I. join () for I in l] print (time. time ()-start)

3,

Import timefrom multiprocessing import Pool # applydef func (I): time. sleep (1) I + = 1 # print (I) return I + 1if _ name _ = '_ main _': p = Pool (5) res_l = [] for I in range (20): # p. apply (func, args = (I,) # apply is the mechanism for synchronously submitting tasks res = p. apply_async (func, args = (I,) # apply_async is the mechanism for asynchronous task submission res_l.append (res) # print (res. get () # blocking: waiting for the task result p. close () # close must be added to join. New tasks cannot be added to p. join () # Wait until the sub-process ends and execute [print (I. get () for I in res_l]