Python process, python

Source: Internet
Author: User

Python process, python

Reference blog: https://www.cnblogs.com/vamei/archive/2012/10/12/2721484.html

I. Preface

Python threads or processes call the native threads or processes of the operating system. However, due to the existence of GIL, python multithreading cannot take advantage of multiple cpu cores. The python process does not have GIL, and the data between processes is independent and secure. All python multi-process can take advantage of multiple cores.

Applicability:

Python multithreading: I/O-intensive tasks

Python multi-process: CPU-intensive tasks

Ii. multiprocessing
Multiprocessing is a package that supports spawning processes using an API similar to the threading module. the multiprocessing package offers both local and remote concurrency, please tively side-stepping the Global Interpreter Lock by using subprocesses instead of threads. due to this, the multiprocessing module allows the programmer to fully leverage multiple processors on a given machine. it runs On both Unix and Windows. the multiprocessing package is a multi-process management package in Python. It uses APIs similar to the thread module to create processes (a large part of multiprocessing uses the same set of Apis as threading, but changed to the multi-process situation ). The multi-process management package provides local and remote concurrency. By using sub-processes instead of threads, it effectively avoids GIL. For this reason, multiprocessing allows programmers to take full advantage of multiple processors on a given machine. It can run on Unix and Windows.

Similar to threading. Thread, it can use the multiprocessing. Process object to create a Process. This process can run functions written inside the Python program. This Process object has the same usage as the Thread object, and also has the start (), run (), join () methods. In addition, the multiprocessing package also contains the Lock/Event/Semaphore/Condition class.

However, when using these shared APIs, pay attention to the following points:

  • On UNIX platforms, when a process ends, the process needs to be called by its parent process wait; otherwise, the process becomes a Zombie process (Zombie ). Therefore, it is necessary to call the join () method (actually equivalent to wait) for each Process object ). For multithreading, this necessity does not exist because there is only one process.
  • Multiprocessing provides IPC (such as Pipe and Queue) that are not included in the threading package, which is more efficient. Pipe and Queue should be given priority to avoid synchronization methods such as Lock/Event/Semaphore/Condition (because they occupy resources of user processes ).
  • Multi-process should avoid sharing resources. In multithreading, we can easily share resources, such as using global variables or passing parameters. In the case of multiple processes, the above method is not suitable because each process has its own memory space. In this case, we can share resources by sharing memory and Manager. However, this improves the complexity of the program and reduces the efficiency of the program due to synchronization needs.

Process. PID contains a PID. If the Process does not have start (), the PID is None.

Iii. Example 3.1 define a process
Import multiprocessing import timeimport osdef run (name): time. sleep (2) print ('hello', name) if _ name _ = '_ main _': start_time = time. time () p_list = [] for I in range (10): p = multiprocessing. process (target = run, args = ('Bob % s' % I,) # create a Process object p. start () print (p. name, p. pid) # process name and pid p_list.append (p) # p. join () # Like a thread, join will block for p in p_list: p. join () print ('parent process', OS. getppid () print ('total time', time. time ()-start_time)
3.2 embedded thread in process
#-*-Coding: UTF-8-*-import multiprocessingimport timeimport osimport threadingdef threading_run (name): print ('name: ', name) def run (name): time. sleep (2) t = threading. thread (target = threading_run, args = (name,) # define Thread t. start () # print ('hello', name) if _ name _ = '_ main _': start_time = time. time () for I in range (10): p = multiprocessing. process (target = run, args = ('Bob % s' % I,) # Process p. start () # print (p. name, p. pid) print ('parent process', OS. getppid ())

Note: Like a thread, the main process does not wait until the child process ends by default.

3.3 Parent and Child Processes

  

from multiprocessing import Processimport osdef info(title):    print(title)    print('module name:', __name__)    print('parent process:', os.getppid())    print('process id:', os.getpid())    print("\n\n")def f(name):    info('\033[31;1mfunction f\033[0m')    print('hello', name)if __name__ == '__main__':    info('\033[32;1mmain process line\033[0m')    p = Process(target=f, args=('bob',))    p.start()    p.join()

Result:

main process linemodule name: __main__parent process: 2464process id: 13628function fmodule name: __mp_main__parent process: 13628process id: 17480hello bob

Note:

Parent process: 2464 is the process Number of PyCharm in my computer

Process id: 13628 is the process number of the main process ('_ main _')

Process id: 17480 is the process number of sub-process p

It can be seen that all processes are started by the parent process.

    

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.