Python concurrent programming Multi-process

Python concurrent programming Multi-process

I. What is a process

Process: a process in progress or a task that performs a task is the CPU.

Principle: Single-core plus multi-channel technology

Ii. the difference between process and procedure

Process refers to the process of running a program

It should be stressed that: the same program executes two times is two processes, such as the opening of the Storm video, although all the same software, but one can play Cang Jing empty, another can play Enrique.

Third, concurrency and parallelism

Whether it is parallel or concurrent, in the view of the user is ' simultaneously ' running, whether it is a process or a thread, is just a task, really work is cpu,cpu to do these tasks, and a CPU can only perform one task at a time.

(1) Concurrency

Concurrency is pseudo-parallel, which appears to be running concurrently. Concurrency can be achieved with a single CPU plus multi-channel technology.

(2) Parallel

Run concurrently, only with multiple CPUs to achieve parallelism

If four cores, six missions, so that at the same time there are four tasks to be executed

Multi-Channel technology: in-memory multi-channel (multiple) programs, the CPU from one process quickly switch to another, so that each process each run dozens of or hundreds of milliseconds, so that, although in a moment, a CPU can only perform a task, but in 1 seconds, the CPU could run multiple processes, This gives rise to a parallel illusion, pseudo concurrency, to differentiate the true hardware parallelism of a multiprocessor operating system (multiple CPUs share the same physical memory)

Iv. Synchronous vs. asynchronous

Synchronous execution: When a process executes a task, another process must wait for it to complete before it can proceed with the asynchronous execution: When a process executes a task, another process does not have to wait for its execution to finish, and when a message returns, the system notifies the latter to process it. This can improve execution efficiency

For example, the telephone is synchronous communication, the short interest is asynchronous communication.

V. Creation of processes

But all hardware, need to have the operating system to manage, as long as there is an operating system, there is the concept of process, you need to have a way to create processes, some operating systems for a single application program design, such as the microwave oven controller, once the microwave oven, all processes are already there.

For general-purpose systems (running many applications), there is a need to have the ability to create or revoke processes in the process of running the system, mainly divided into 4 forms to create new processes

1. System initialization (view process Linux in the PS command, Windows with Task Manager, the foreground process is responsible for interacting with the user, the background running process is not user-independent, running in the background and only when needed to wake up the process, called daemons, such as e-mail, Web pages, news, printing)

2. A process in the process of running a child process (such as Nginx open multi-process, os.fork,subprocess. Popen, etc.)

3. User's interactive request, and create a new process (such as user double-click Storm Video)

4. Initialization of a batch job (applied only in a mainframe batch system)

Either way, the creation of a new process is created by an already existing process that executes a system call to create the process:

1. In Unix the system call is: Fork,fork creates a copy that is identical to the parent process, with the same storage image, the same environment string, and the same open file (in the shell interpreter process, executing a command creates a child process)

2. In Windows, the system call is: Createprocess,createprocess processes both the creation of the process and the process of loading the correct program into a new one.

About creating child processes, UNIX and Windows

1. The same is true: After a process is created, the parent and child processes have their own different address spaces (the multi-channel technology requires the physical plane to implement the isolation of the memory between processes ), and the modification of any one process in its address space does not affect another process.

2. The difference is that, in Unix, the initial address space of a child process is a copy of the parent process, which indicates that the child process and the parent process can have read-only shared memory areas. However, for Windows systems, the address space of the parent process and the child process is different from the beginning.

VI. termination of the process

1. Normal exit (voluntary, such as the user clicks on the interactive page of the cross, or the completion of the program call to initiate system calls to exit normally, in Linux with exit, in Windows with ExitProcess)

2. Error exiting (voluntary, a.py does not exist in Python a.py)

3. Serious error (involuntary, execution of illegal instructions, such as referencing non-existent memory, 1/0, etc., can catch the exception, try...except ... ）

4. Killed by other processes (involuntary, e.g. kill-9)

VII. Hierarchical structure of processes

Regardless of Unix or Windows, the process has only one parent process, and the difference is:

1. All processes in Unix are rooted in the init process and are formed into a tree structure. Parent-child processes collectively form a process group so that when a signal is emitted from the keyboard, the signal is sent to all members of the current keyboard-related process group.

2. In Windows, there is no process hierarchy concept, all processes are of the same status, and the only hint similar to the process level is that when the process is created, the parent process gets a special token ( called a handle) that can be used to control the child process, But the parent process has the right to pass the handle to the other child processes, so there is no hierarchy.

Viii. Status of the process

Execute the program tail, open a subprocess, execute the program grep, open another subprocess, two processes based on pipe ' | ' Communication, tail the result as input to grep.

The state of process grep while waiting for input (i.e. I/O) is called blocking, when the grep command cannot be run

In fact, in both cases, a process cannot be logically run,

1. Process hangs is its own cause, encountering I/O blocking, it will let the CPU to allow other processes to execute, so that the CPU has been working

2. Regardless of the process, it is the operating system level that may invoke other processes to use the CPU because one process takes up too much time, or a priority, and so on.

Thus a process consists of three states

IX. implementation of the process

The implementation of the concurrency of the process is that the hardware interrupts a running process, saving all the state at which the process is running, and for this reason, the operating system maintains a table, the process table, which consumes a process table entry (these are also referred to as Process Control blocks)

This table holds important information about the status of the process: program counters, stack pointers, memory allocations, status of all open files, account and dispatch information, and other information that must be saved when a process is turned into a ready or blocked state, ensuring that the process starts again as if it had never been interrupted.

Python concurrent programming Multi-process