Python process, python
Reference blog: https://www.cnblogs.com/vamei/archive/2012/10/12/2721484.html
I. Preface
Python threads or processes call the native threads or processes of the operating system. However, due to the existence of GIL, python multithreading cannot take advantage of multiple cpu cores. The python process does not have GIL, and the data between processes is independent and secure. All python multi-process can take advantage of multiple cores.
Applicability:
Python multithreading: I/O-intensive tasks
Python multi-process: CPU-intensive tasks
Ii. multiprocessing
Multiprocessing is a package that supports spawning processes using an API similar to the threading module. the multiprocessing package offers both local and remote concurrency, please tively side-stepping the Global Interpreter Lock by using subprocesses instead of threads. due to this, the multiprocessing module allows the programmer to fully leverage multiple processors on a given machine. it runs On both Unix and Windows. the multiprocessing package is a multi-process management package in Python. It uses APIs similar to the thread module to create processes (a large part of multiprocessing uses the same set of Apis as threading, but changed to the multi-process situation ). The multi-process management package provides local and remote concurrency. By using sub-processes instead of threads, it effectively avoids GIL. For this reason, multiprocessing allows programmers to take full advantage of multiple processors on a given machine. It can run on Unix and Windows.
Similar to threading. Thread, it can use the multiprocessing. Process object to create a Process. This process can run functions written inside the Python program. This Process object has the same usage as the Thread object, and also has the start (), run (), join () methods. In addition, the multiprocessing package also contains the Lock/Event/Semaphore/Condition class.
However, when using these shared APIs, pay attention to the following points:
- On UNIX platforms, when a process ends, the process needs to be called by its parent process wait; otherwise, the process becomes a Zombie process (Zombie ). Therefore, it is necessary to call the join () method (actually equivalent to wait) for each Process object ). For multithreading, this necessity does not exist because there is only one process.
- Multiprocessing provides IPC (such as Pipe and Queue) that are not included in the threading package, which is more efficient. Pipe and Queue should be given priority to avoid synchronization methods such as Lock/Event/Semaphore/Condition (because they occupy resources of user processes ).
- Multi-process should avoid sharing resources. In multithreading, we can easily share resources, such as using global variables or passing parameters. In the case of multiple processes, the above method is not suitable because each process has its own memory space. In this case, we can share resources by sharing memory and Manager. However, this improves the complexity of the program and reduces the efficiency of the program due to synchronization needs.
Process. PID contains a PID. If the Process does not have start (), the PID is None.
Iii. Example 3.1 define a process
Import multiprocessing import timeimport osdef run (name): time. sleep (2) print ('hello', name) if _ name _ = '_ main _': start_time = time. time () p_list = [] for I in range (10): p = multiprocessing. process (target = run, args = ('Bob % s' % I,) # create a Process object p. start () print (p. name, p. pid) # process name and pid p_list.append (p) # p. join () # Like a thread, join will block for p in p_list: p. join () print ('parent process', OS. getppid () print ('total time', time. time ()-start_time)3.2 embedded thread in process
#-*-Coding: UTF-8-*-import multiprocessingimport timeimport osimport threadingdef threading_run (name): print ('name: ', name) def run (name): time. sleep (2) t = threading. thread (target = threading_run, args = (name,) # define Thread t. start () # print ('hello', name) if _ name _ = '_ main _': start_time = time. time () for I in range (10): p = multiprocessing. process (target = run, args = ('Bob % s' % I,) # Process p. start () # print (p. name, p. pid) print ('parent process', OS. getppid ())
Note: Like a thread, the main process does not wait until the child process ends by default.
3.3 Parent and Child Processes
from multiprocessing import Processimport osdef info(title): print(title) print('module name:', __name__) print('parent process:', os.getppid()) print('process id:', os.getpid()) print("\n\n")def f(name): info('\033[31;1mfunction f\033[0m') print('hello', name)if __name__ == '__main__': info('\033[32;1mmain process line\033[0m') p = Process(target=f, args=('bob',)) p.start() p.join()
Result:
main process linemodule name: __main__parent process: 2464process id: 13628function fmodule name: __mp_main__parent process: 13628process id: 17480hello bob
Note:
Parent process: 2464 is the process Number of PyCharm in my computer
Process id: 13628 is the process number of the main process ('_ main _')
Process id: 17480 is the process number of sub-process p
It can be seen that all processes are started by the parent process.