Python's multiprocessing,queue,process

In the multithreaded multiprocessing module, there are two classes, queue (queues) and process (process);

There is also a queue class in queue.py, which is the difference between the two queue?

From multiprocessing import queue,process introduction of queues and process classes in multiprocessing modules

Queue queues:

The queue is a standard library in Python, can be directly referenced in the queue, Queue.queue (maxsize) creates a queue object, and if MaxSize is not provided, the number of queues is unlimited.

# _*_ encoding:utf-8 _*_ Import  = queue.queue (q.put) ('SB') q.put ('you  ')print ( q.get ())print (Q.get ())

When a queue is empty, with get back up the blockage, so generally take the queue time will use, get_nowait () method, this method when the value of an empty queue will throw an empty exception, so the general will first determine whether the queue is empty, if not empty to take value;

Take a queue in a non-blocking manner

Determines whether the queue is empty, returns true for null, and does not return FALSE for null

Returns the length of the queue

Queue.get ([block[, timeout]]) Get queue, timeout wait time
Queue.get_nowait () quite queue.get (False)
Non-blocking Queue.put (item) write queue, timeout wait time
Queue.put_nowait (item) quite Queue.put (item, False)

The concept process of using child processes in multiprocessing:

From multiprocessing import Process

A process can be used to construct a subprocess

P=process (target=fun,args= (args))

Then start the subprocess by P.start ().

The P.join () method is then used to make the child process run and then the parent process is executed.

To use the pool in multiprocessing:

If you need more than a child process, consider using a process pool to manage

Pool creates sub-processes differently than process, which is achieved by P.apply_async (func,args= (args)), where a pool can run at the same time depending on the number of CPUs in your computer, and if it is 4 CPUs, there will be Task0,task1 , Task2,task3 starts at the same time, TASK4 needs to start at the end of a process.

Communication between multiple sub-processes:

Communication between multiple sub-processes takes the queue in the first step, for example, with the following requirements, one child process writing data to the queue, and another process fetching data from the queue.

#_*_ encoding:utf-8 _*_ fromMultiprocessingImportProcess,queue,pool,pipeImportOs,time,random#code to write the data Process execution:defWrite (P): forValueinch['A','B','C']:        Print('Write---Before put value---put%s to queue ...'%value) p.put (value)Print('Write---After Put value') Time.sleep (Random.random ())Print('Write---After sleep')#read the code that the data process executes:defRead (p): whileTrue:Print('Read---before get value') Value=p.get (True)Print('Read---After get value---The Get%s from queue.'%value)if __name__=='__main__':    #The parent process creates a queue and passes it to each child process:p =Queue () PW= Process (target=write,args=(P,)) PR= Process (target=read,args=(P,))#Start child process PW, write:Pw.start ()#start child process PR, read:Pr.start ()#wait for PW to end:Pw.join ()#PR process is a dead loop, can not wait for its end, can only forcibly terminate:Pr.terminate ()

With regard to the application of locks, if there are simultaneous operations on the same queue between different programs, in order to avoid errors, it is possible to add a lock to a function when the queue is operated so that only one child process can operate on the queue at the same time, and the lock is also locked in the manager object.

Python's multiprocessing,queue,process