A case study of Python multi-process programming technology

In this paper, the Python multi-process programming technique is analyzed in the instance form, which is helpful for further Python programming techniques. Share to everyone for your reference. The specific analysis is as follows:

In general, because of the limitations of Python threads, such as multithreading does not take full advantage of multi-core CPUs, we prefer to use multiple processes in Python. But we also use multithreading in scenarios such as non-blocking asynchronous UIs. This article focuses on Python multi-process issues.

Python introduces a multi-process mechanism in 2.6 and provides a rich set of components and APIs to facilitate the writing of concurrent applications. Components of the multiprocessing package process, Queue, Pipe, lock and other components provide functionality similar to multithreading. These components make it easy to write multi-process concurrent programs.

Process

The use of process is a bit like java.lang.Thread, but thread is threading. The Start method is used to start a process. A simple example:

From multiprocessing import Processimport osimport timedef sleeper (name, seconds):  print "Process id#%s"% (os.getpid ())  print "Parent Process id#%s"% (Os.getppid ())  print "%s would sleep for%s seconds"% (name, seconds)  time . Sleep (seconds) if __name__ = = "__main__":  Child_proc = Process (Target=sleeper, args= (' Bob ', 5))  Child_ Proc.start ()  print "in parent process after child process start"  print "parent process Abount to join child proces S "  child_proc.join ()  print" in parent process after child process join "  print" the parent process:% S "% (Os.getppid ())

The instantiation of a process must specify target and args. Target is the new process's entry method, which can be thought of as the main method. Args is the parameter list for this method. The START process is similar to starting thread, and you must call the Start method. You can also inherit the process, override the Run method, and implement the logic of the procedure in the Run method. Calling the Join method blocks the current calling process until the called process finishes running.
Manually terminating a process can call the Terminate method, which in a UNIX system sends a sigterm semaphore, and in a Windows system, the TerminateProcess method is used. It is important to note that the exit processing logic will not be executed, the process's child processes will not be terminated, they will only become orphan processes.

Queue

A queue is a multi-process-safe queuing that enables data transfer between multiple processes using the queue. The Put method is used to insert data into the queue, and the put method has two optional parameters: Blocked and timeout. If blocked is true (the default) and timeout is a positive value, the method blocks the time specified by timeout until the queue has the remaining space. If timed out, a Queue.full exception is thrown. If blocked is false, but the queue is full, an Queue.full exception is thrown immediately.

The Get method can read from the queue and delete an element. Similarly, the Get method has two optional parameters: Blocked and timeout. If blocked is true (the default) and timeout is a positive value, then no element is taken within the wait time, and a Queue.empty exception is thrown. If blocked is false, there are two cases where the queue has a value that is available, and the value is immediately returned, otherwise the Queue.empty exception is thrown immediately if it is empty. A sample code for the queue:

From multiprocessing import Process, Queuedef offer (queue):  queue.put ("Hello World") def test (queue, num):  Queue.put ("Hello World:" + str (num)) if __name__ = = ' __main__ ':  q = Queue ()  p1 = Process (target=test, args= (q, 1))  P1.start ()  p = Process (Target=offer, args= (Q,))  P.start ()  p2 = Process (Target=test, args= (q, 2))  P2.start ()  P2 = Process (target=test, args= (q, 3))  P2.start ()  print q.get ()  print q.get ()  print q.get ()  print q.get ()  print q.close ()

Output:

Hello world:1
Hello World
Hello World:2
None

Pipes

Pipe method Return (CONN1, CONN2) represents the two end of a pipeline. The pipe method has the duplex parameter, if the duplex parameter is true (the default), then the pipeline is full duplex, that is, both CONN1 and CONN2 can send and receive. Duplex is only responsible for receiving messages for FALSE,CONN1, CONN2 is only responsible for sending messages.

The Send and Recv methods are methods for sending and receiving messages, respectively. For example, in full-duplex mode, you can call Conn1.send to send a message conn1.recv receive a message. If there is no message to receive, the Recv method is blocked. If the pipe has been closed, then the Recv method throws Eoferror.

From multiprocessing import Process, Pipedef Send (conn):  conn.send ("Hello World")  conn.close () if __name__ = = ' _ _main__ ':  parent_conn, child_conn = Pipe ()  p = Process (Target=send, args= (Child_conn,))  P.start ()  Print Parent_conn.recv ()

Synchronous

The multiprocessing package provides condition, Event, Lock, Rlock, semaphore and other components that can be used for synchronization. Here is an example of using lock:

From multiprocessing import Process, Lockdef L (lock, num):  lock.acquire ()   print "Hello num:%s" (num)  lock. Release () if __name__ = = ' __main__ ':  lock = lock () for NUM in range:  Process (target=l, args= (lock, Num)). Start ()

Summarize

The above is a simple introduction and examples of Python multiprocessing library, familiar with the Java multithreaded development of the classmate is not very familiar with the Java concurrency API very much like, but Javaconcurrency is to deal with multi-threaded only, We can directly follow the previous Java multi-threading experience with these APIs.

Interested friends can test running this example to deepen understanding. It is believed that this article has some reference value to the study of Python program design.

