Python multi-process development and multi-thread Development

Source: Internet
Author: User

Python multi-process development and multi-thread Development
Let's first understand what a process is? The program cannot run independently. It can only run when it is loaded into the memory, and the system allocates resources for it. Such a program is called a process. The difference between a program and a process is that a program is a set of commands, and it is a static description text of the process running. A process is an execution activity of a program and is a dynamic concept. In multi-channel programming, we allow multiple programs to be loaded to the memory at the same time. Under the scheduling of the operating system, the program can be executed concurrently. This design greatly improves the CPU utilization. The emergence of processes makes every user feel that they have exclusive CPU. Therefore, the process is proposed to implement multi-channel programming on the CPU. Why thread? The process provides multi-channel programming, giving full play to the concurrency of computer components, and improving the computer's utilization. Since the process is so good, why do we need threads? In fact, there are still many defects, mainly reflected in two points: the process can only do one thing at a time, if you want to do two or more at the same time, the process will be powerless. If the process is blocked during execution, such as waiting for input, the entire process will be suspended, even if some work in the process is not dependent on the input data, it will not be executed. The solution is to let a single process, accept requests, wait for I/O, and process computing in parallel. This obviously avoids synchronization waiting and improves execution efficiency, in the actual operating system, such a mechanism is thread. Because of the concurrency, we have invented the process and the thread. But the process and thread concurrency levels are different: the process is an abstraction provided at the processor level; the thread is a concurrency abstraction provided at the process level. If we enter the computer architecture, we will find that the pipeline also provides a kind of concurrency, but it is a command-level concurrency. In this way, the pipelines, threads, and processes provide the concurrency we urgently need at three levels from low to high! In addition to improving the concurrency of processes, threads also have the advantage of effectively utilizing multi-processor and multi-core computers. The current trend of the processor is the development of multi-core. Before there is no thread, multi-core cannot improve the execution speed of a process, because it is still the above two restrictions. However, if a process is divided into several threads, different threads can run on different cores, thus improving the process execution speed. For example, we often use Microsoft Word for Text Formatting. In fact, multiple threads are opened. Each thread is responsible for displaying, receiving input from the keyboard, and storing disks. These threads run together to make us feel that our input and screen display are happening at the same time, rather than entering some characters, so that we can see the display after a while. We accidentally stored the disk automatically. This is the convenience that the thread brings to us. The difference between a process and a thread is a running activity of a program with certain independent functions about a dataset. A process is an independent unit for the system to allocate and schedule resources. A thread is an entity of a process. It is the basic unit for CPU scheduling and scheduling. It is smaller than a process and can run independently. the thread itself basically does not have system resources, and only has a few resources (such as program counters, a set of registers and stacks) that are essential for running ), however, it can share all resources of a process with other threads of the same process. One thread can create and withdraw another thread, and multiple threads in the same process can be concurrently executed. The main difference between processes and threads is that they are different operating system resource management methods. A process has an independent address space. After a process crashes, it will not affect other processes in the protection mode, but the thread is only a different execution path in the process. A thread has its own stack and local variables, but there is no separate address space between threads. If a thread dies, the whole process dies. Therefore, multi-process programs are more robust than multi-threaded programs, however, during process switching, resources are consumed and the efficiency is lower. But for some concurrent operations that require simultaneous and shared variables, you can only use threads, not processes. The Python multiprocessing Unix/Linux operating system provides a fork () system call, which is very special. A common function is called once and returns once, But fork () is called once and returns twice because the operating system automatically calls the current process (called the parent process) copy a copy (called a child process), and then return the results in the parent process and child process respectively. The child process always returns 0, and the parent process returns the child process ID. The reason for doing so is that a parent process can fork many sub-processes, so the parent process needs to write down the ID of each sub-process, and the sub-process only needs to call getppid () you can get the ID of the parent process. The Python OS module encapsulates common system calls, including fork. You can easily create sub-processes in a Python program: 123456789 # multiprocessing. pyimport OS print 'process (% s) start... '% OS. getpid () pid = OS. fork () if pid = 0: print 'I am child process (% s) and my parent is % s. '% (OS. getpid (), OS. getppid () else: print 'I (% s) just created a child process (% s ). '% (OS. getpid (), pid) Because Windows does not have a fork call, the above Code cannot run on Windows. Multiprocessing is a multi-Process module in a cross-platform version. It provides a Process class to represent a Process object. The following is the sample code:

#!/usr/local/python27/bin/python2.7# coding=utf8# noinspection PyUnresolvedReferencesfrom multiprocessing import Processimport timedef f(n):    time.sleep(1)    print n*nfor i in range(10):    p = Process(target=f,args=[i,])    p.start()

 

If this program is written by a single process, it takes more than 10 seconds to execute the program. If multiple processes are used, the program starts 10 processes for parallel execution, which takes more than one second. In general, the memory resources of multiple processes are independent from each other, while multithreading can share the memory resources of the same process. Example code:
#! /Usr/local/python27/bin/python2.7 # coding = utf8 # noinspection PyUnresolvedReferences # inter-process memory cannot be shared through multi-process and multi-thread comparison, memory sharing between threads from multiprocessing import Processimport threadingimport timelock = threading. lock () def run (info_list, n): lock. acquire () info_list.append (n) lock. release () print ('% s \ n' % info_list) info = [] for I in range (10): ''' target is the function executed by the sub-process, args is the parameter '''p = Process (target = run, args = [info, I]) p to be passed to the function. start () ''' here is to make the output neat so that the execution of the main process and other sudden processes '''time. sleep (1) print ('------------ threading ------------') for I in range (10): p = threading. thread (target = run, args = [info, I]) p. start ()

 


Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.