The process of Python

Simple Introduction

While building Macaca parallel testing using Python's multi-process approach, the goal is to test multiple devices at the same time, if you wait for a single device to run and then proceed to the next device, imagine that there are 20 devices, run the test script that will take a long time ah? Using a multi-process approach is a lot easier, but problems are also found in the use process. I am using three simulator and a real machine test, three simulators have run out, the real machine has not started, the time gap is still there, it may be because the computer's CPU core is not enough.

Actual operation

Because Python is cross-platform, nature should also provide a cross-platform, multi-process support. The multiprocessing module is a multi-process module with cross-platform versions.

First, Process

The multiprocessing module provides a process class to represent a processing object, and the following example demonstrates starting a child process and waiting for it to end:

The code is as follows:

From multiprocessing import Process

Import OS

# code to be executed by the child process

def run_proc (name):

Print (' Run child process%s (%s) ... '% (name, Os.getpid ()))

If __name__== ' __main__ ':

Print (' Parent process%s. '% Os.getpid ())

p = Process (Target=run_proc, args= (' Test ',))

Print (' child process'll start. ')

P.start ()

P.join ()

Print (' child process end. ')

The results are as follows:

Parent process 928.

Process would start.

Run Child process Test (929) ...

Process end.

It is important to note that starting the process with start (), the Join () method can wait for the child process to end and then continue down, typically for inter-process synchronization.

Second, Pool

I used the pool method when building Macaca multi-process, and if you want to start a lot of sub-processes, you can batch create the child processes in the way of the process pool:

The

Code is as follows:

From multiprocessing import Pool

Import os, time, random

Def long_time_task (name):

    Print (' Run task%s (%s) ... '% (name, Os.getpid ()))

    start = Time.time ()

    Time.sleep (Random.random () * 3)

    end = Time.time ()

    Print (' Task%s runs%0.2f seconds. '% (name, (End-start))

If __name__== ' __main__ ':

    Print (' Parent process%s. '% Os.getpid ())

    p = Pool (4)

    for I in range (5):

        P.apply_async (Long_time_task, args= (i,))

    Print (' Waiting for all subprocesses do ... ')

    P.close ()

    P.join ()

    Print (' All subprocesses do. ')

The results are as follows:

Parent process 669.

Waiting-subprocesses done ...

Run Task 0 (671) ...

Run Task 1 (672) ...

Run Task 2 (673) ...

Run Task 3 (674) ...

Task 2 runs 0.14 seconds.

Run Task 4 (673) ...

Task 1 runs 0.27 seconds.

Task 3 runs 0.86 seconds.

Task 0 runs 1.41 seconds.

Task 4 runs 1.91 seconds.

All subprocesses is done.

The pool class can provide a specified number of processes for the user to call, and when a new request is submitted to the pools, a new process will be created to execute the request if it is not full. If the pool is full, the request tells you to wait until the process ends in the pool before a new process is created to execute the requests.

Here are a few methods under the pool class under the multiprocessing module:

Apply ()

Function Prototypes:

Apply (func[, args= () [, kwds={}])

This function is used to pass an indeterminate parameter, and the main process is blocked until the function execution is finished (not recommended, and the 3.x does not appear later).

Apply_async ()

Function Prototypes:

Apply_async (func[, args= () [, kwds={}[, Callback=none]])

Same as apply usage, but it is non-blocking and supports the return of the result for callbacks.

Map ()

Function Prototypes:

Map (func, iterable[, Chunksize=none])

The map method in the pool class is basically consistent with the built-in map function usage behavior, which causes the process to block until the result is returned.

Note that although the second parameter is an iterator, in practice, the program will not run the child process until the entire queue is ready.

Close ()

Close the process pool so that it is not accepting new tasks.

Terminate ()

Ends a worker process and is not processing an unhandled task.

Join ()

The main process blocks waiting for the child process to exit, and the Join method must be used after close or terminate.

Reference:

Multi-process

Python multi-process multiprocessing. The pool class is detailed

The process of Python