Threads and processes in Python

Processes and Threads

In multitasking, each task has its own process, and a task has many subtasks that open threads in the process to perform these subtasks. In general, you can run the base unit of a separate dispatch and allocation as a thread, while a process is the basic unit of resource ownership.

Python supports multi-process multiprocessing, as well as multithreaded threading.

Multi-process

The Os.fork () function can open a process. The function returns two values, returns the ID of the child process in the parent process, and returns 0 in the child process forever.

The Os.getpid () function can return the ID of the process. Os.getppid () can return the ID of the parent process.

With Os.fork (), you can open a process at any time and return the process ID, as well as use the Os.getpid (), Os.getppid () function, to see at any time the current process ID and the ID of the parent process.

The fork () function works only under Unix/linux, and Windows does not support fork (), using multiprocessing. The multi-process invocation provided by the process class reflects the entire process more completely.

From multiprocessing Imort Process
def proc_worker (args):
Pass
def main ():
p = Process (target=proc_worker,args= (Some_args,))
P.start ()
P.join ()

The process instance specifies the functions that the child process runs and the associated parameters, and the start () and join () methods can control the start and wait of the process.

It is important to note that since a process only has one task at a time, after the child process has been called, the join () method must be called to wait for its parent process, or the process will become a zombie process. For threads, it is not necessary to join ().

Processes can also share memory, but in design, the process should be the basic unit of resource ownership, so try to avoid sharing memory between processes, because of the need for synchronization, which can reduce the efficiency of the program.

Synchronization between processes through locks (lock): The lock instance has two main methods: Acquire (), released (), and a lock can only be occupied by one process.

Between processes can be communicated through pipe and queue,

When the pipe instance, the default is a bidirectional channel, any end of the pipeline can send and receive messages, when instantiated by specifying Duplex=false can create a one-way channel, the instance will be a tuple containing two elements, representing the ends of the pipeline:

From multiprocessing import pipe,process
def proc1 (pipe):
Pipe.send (' There is Proc1 ')
Print (' Proc1 recv: ', Pipe.recv ())
def proc2 (pipe):
Pipe.send (' There is proc2 ')
Print (' proc2 recv: ', Pipe.recv ())
def main ():
Pipe = pipe ()
P1 = Process (target=proc1,args= (pipe[0],))
P2 = Process (target=proc2,args= (pipe[1],))
P1.start ()
P2.start ()
P1.join ()
P2.join ()

And the queue is a queuing, to meet the FIFO principle (pipe is also an advanced first-out structure, if you send more than one message, it will be in the first in order to receive), and the queue to support multiple processes simultaneously incoming messages, and support multiple processes simultaneously read messages, when instantiated, Specify an integer to limit the maximum number of processes allowed:

From multiprocessing import queue,process
Import OS
def inQueue (queue):
info = ' Put from proc:%s '%os.getpid ()
Queue.put (Info)
def outqueue (queue):
info = Queue.get ()
Print ('%s get a info of%s '% (Os.getpid (), info))
def main ():
Queue = Queue (5)
Processes1 = []
Processes2 = []
For I in range (10):
Process = Process (target = inqueue,args= (queue,))
Process.Start ()
Processes1.append (Process)
For I in range (10):
Process = Process (target = outqueue,args= (queue,))
Process.Start ()
Processes2.append (Process)
for proc in Processes1:
Proc.join ()
Queue.close ()
for proc in Processes2:
Proc.join ()

Although the FIFO principle is followed in the queue, the lock should be added if you want to truthfully react to the situation in the queue because the above code does not synchronize the process:

From multiprocessing import Process,queue,lock
Import OS
def inQueue (Queue,lock):
Lock.acquire ()
info = ' Put from proc:%s '%os.getpid ()
print '%s put a info '%os.getpid ()
Queue.put (Info)
Lock.release ()
def outqueue (Queue,lock):
Lock.acquire ()
info = Queue.get ()
Print ('%s get a info of%s '% (Os.getpid (), info))
Lock.release ()
Def main1 ():
Queue = Queue (10)
Lock = Lock ()
Processes1 = []
Processes2 = []
For I in range (10):
Process = Process (target = inqueue,args= (Queue,lock))
Process.Start ()
Processes1.append (Process)
For I in range (10):
Process = Process (target = outqueue,args= (Queue,lock))
Process.Start ()
Processes2.append (Process)
for proc in Processes1:
Proc.join ()
Queue.close ()
for proc in Processes2:
Proc.join ()

Multithreading

Multithreading is a process in which multiple subtasks of a task are performed, and should be designed to be a basic unit of independent dispatch and allocation. Multithreading in Python is implemented using the threading module.

threading, like multiprocessing, has a thread class that instantiates thread objects, and the thread instance, like process, has a start (), Join () method, and so on.

As mentioned above, it is best not to have the exchange of resources in the process, so the lock model is less used in the process, and the thread is more common, the threading and locking model is the basic concurrency model, the lock in thread is the same as in the process, also has the acquire () method and the release () method.

However, in Python, multithreaded concurrency does not perform perfectly, because the official Python interpreter, CPython, adds a Gil lock at design time, and any Python thread must get a Gil lock before execution, and every 100 byte-code Gil lock will be released. So the multithreading in Python is still alternating, multi-threaded in Python can only use the same CPU, if you want to use multiple CPUs, you need to use a multi-process to complete.

However, this does not mean that multithreading in Python is meaningless, although multi-core hardware cannot be used, but it is much more efficient to perform multithreaded execution of the same task than to open multiple processes.

Threads and processes in Python