Python development [Article 9]: process, thread, coroutine, python Article 9

Python development [Article 9]: process, thread, coroutine, python Article 9
What is process )?

The program cannot run independently. It can only run when it is loaded into the memory, and the system allocates resources for it. Such a program is called a process. The difference between a program and a process is that a program is a set of commands, and it is a static description text of the process running. A process is an execution activity of a program and is a dynamic concept.

What is thread )?

A thread is the smallest unit that the operating system can schedule operations. It is included in the process and is the actual operating unit of the process. A thread refers to a single-Order Control Flow in a process. A process can be concurrently executed by multiple threads, and each thread executes different tasks in parallel.

What is the difference between a process and a thread?

The thread shares the memory space, and the memory of the process is independent.

The threads of the same process can communicate directly, but the two processes must communicate with each other through an intermediate proxy.

It is easy to create a new thread. To create a new process, you need to clone its parent process.

One thread can control and operate other threads in the same process, but the process can only operate sub-processes.

Python GIL (Global Interpreter Lock)

No matter how many threads are enabled and how many CPUs are there, python allows only one thread at a time during execution.

The Python threading module directly calls the inherited call Join and DaemonJoin

The function of Join is to block the main process and the program following join cannot be executed.

In the case of multi-threaded and multi-join, the join methods of each thread are executed in sequence. Only after the execution of the previous thread ends can the execution of the next thread be executed.

If no parameter exists, the subsequent program is executed only after the thread ends.

After the parameter is set, the main process following the thread is executed after the set time, regardless of whether the thread ends.

Set the parameters as follows:

Daemon

By default, when the main thread exits, it waits for the end of all sub-threads. If you want the main thread to automatically end all the sub-threads when exiting without waiting for the sub-thread, you need to set the sub-thread as the background thread (daemon ). The method is to call the setDaemon () method of the thread class.

Thread lock (Mutex)

Multiple Threads can be started under a process. multiple threads share the memory space of the parent process, which means that each thread can access the same data, if two threads need to modify the same data at the same time, the thread lock is required.

Lock version

The Lock operation blocks access from other threads to shared resources, and the same thread can only be acquire once. If a deadlock occurs more than once, the program cannot continue to execute.

Gil vs Lock

GIL ensures that only one thread can be executed at a time. A lock is a user-level lock and has nothing to do with GIL.

RLock)

Rlock is allowed to be acquire multiple times in the same thread. To release shared resources, the thread needs to release all locks. That is, n times of acquire, and n times of release are required.

The main difference between the two locks is that RLock allows multiple acquire requests in the same thread. But Lock does not allow this situation. NOTE: If RLock is used, the acquire and release must appear in pairs. That is, the acquire and release must be called n times to release the occupied lock.

Semaphore (Semaphore)

The mutex lock allows only one thread to change data at the same time, while Semaphore allows a certain number of threads to change data at the same time. For example, if there are three windows at the ticket office, only three people can buy tickets at the same time, the people behind the ticket can only buy tickets when they leave any window in front of the ticket.

Timer (Timer)

Timer calls a function at a certain time. If you want to call a function at intervals, you need to set Timer again in the function called by Timer. Timer is a derived class of Thread.

Event

Python provides the Event object for inter-thread communication. It has a signal flag set by the thread. If the signal flag is false, the thread waits for the guidance signal to be set to true by other threads. The Event object implements a simple thread communication mechanism. It provides setting signals, clearing signals, and waiting for inter-thread communication.

You can use the set () method of Event to set the signal flag inside the Event object to true. The Event object provides the isSet () method to determine the conversion of its internal signal signs. When the set () method of the event object is used, the isSet () method returns true.

The clear () method of Event can be used to clear the signal signs inside the Event object and set it to false. When the clear () method of Event is used, the isSet () method returns false.

The Event wait () method can be executed quickly and returned only when the internal signal is true. When the internal signal of the Event object is false, the wait () method will not return until it is true.

Event is used to implement interaction between two or more threads. The following uses traffic lights as an example to start a thread for traffic control lights and generate several threads for vehicles, the rules for the vehicle to stop green in red.

Queue

In Python, a queue is the most common form of data exchange among threads. The Queue module provides Queue operations.

Create a queue object

The queue. Queue class is a queue synchronization implementation. The queue length can be unlimited or limited. You can set the Queue length through the optional parameter maxsize of the Queue constructor. If maxsize is smaller than 1, the queue length is infinite.

Put a value into the queue

Call the put () method of the queue object to insert a project at the end of the team. Put () has two parameters. The first item is required and is the value of the inserted project. The second block is an optional parameter. The default value is 1. If the queue is empty and the block is 1, The put () method suspends the calling thread until a data unit is empty. If the block is 0, the put () method will cause a Full exception.

Extract A value from the queue

Call the get () method of the queue object to delete the queue header and return a project. The optional parameter is block. The default value is True. If the queue is empty and the block is True, get () will suspend the calling thread until a project is available. If the queue is Empty and the block is False, the queue will cause an Empty exception.

The Python Queue module has three queues and constructor common methods: producer consumer model

In development and programming, the use of producer and consumer models can solve the vast majority of concurrency problems. This mode improves the overall data processing speed of the program by balancing the working capabilities of the production line and consumption thread.

Why use producer and consumer models?

In the thread world, the producer is the thread that produces data, and the consumer is the thread that consumes data. In multi-threaded development, if the producer processing speed is very fast and the Consumer processing speed is very slow, the producer must wait until the consumer completes processing before continuing to produce data. Similarly, if the processing capability of a consumer is greater than that of a producer, the consumer must wait for the producer. To solve this problem, the producer and consumer models were introduced.

What is the producer-consumer model?

The producer consumer model uses a container to solve the strong coupling problem between producers and consumers. The producer and consumer do not communicate directly with each other, but communicate by blocking the queue. Therefore, after the producer completes data production, it no longer waits for the consumer to process the data and directly throws it to the blocking queue. the consumer does not seek data from the producer, instead, the block queue is taken directly from the blocking queue, which is equivalent to a buffer zone, balancing the processing capabilities of producers and consumers.

An example of the most basic producer-consumer model.