Python multi-thread Programming 1

Some basic concepts that must be understood in multithreaded programming are applicable to all programming languages. Content: Concurrent programming multi-task operating system multi-thread vs multi-process thread secure thread life cycle thread type... some basic concepts that must be understood in multi-thread programming, suitable for all programming languages. Content:

Concurrent programming

Multi-task operating system

Multithreading vs multi-process

Thread security

Thread lifecycle

Thread type

Concurrent programming

Different programming paradigms have different perspectives on software. Concurrent programming treats software as a combination of tasks and resources-competing and sharing resources between tasks. when resources meet the requirement, the task is executed; otherwise, the system waits for resources.

Concurrent programming makes software easy to understand and reuse, and greatly improves performance in some scenarios.

Multi-task operating system

To achieve concurrency, you must first support the operating system. Most of the current operating systems are multi-task operating systems that can "execute multiple tasks at the same time.

Multi-task execution can be performed at the process or thread level.

A process is an application running in memory. each process has its own memory space. The multi-task operating system can "concurrently" execute these processes.

A thread is a code block that is executed in disorder and multiple times. multiple threads can run simultaneously. Therefore, multiple threads are considered to be "concurrent. Multithreading aims to maximize the use of CPU resources. For example, in a JVM process, all program code runs in the thread mode.

The "concurrent" and "concurrent" here are just macro feelings. In fact, at the micro level, it is only the process/thread rotation, but the switching time is very short, therefore, it produces the feeling of "parallelism.

Multithreading vs multi-process

The operating system allocates different memory blocks for each process, while multiple threads share the memory blocks of the process. The most direct difference is that the thread creation overhead is much smaller than the overhead of the creation process.

At the same time, the communication between processes is relatively difficult due to different memory blocks. Pipe/named pipe, signal, message queue, shared memory, and socket must be used. Inter-thread communication is simple and fast, that is, sharing global variables in processes.

However, the operating system is responsible for Process Scheduling. we need to consider thread scheduling to avoid deadlock, hunger, live locks, and resource depletion, this will increase complexity. In addition, because threads share memory, we also need to consider thread security issues.

Thread security

It is assumed that the global variables in the process are shared among threads. Therefore, when other threads change the shared variables, this thread may be affected. The so-called thread security constraint means that a function must always produce correct results when it is repeatedly called by multiple concurrent threads. To ensure thread security, lock the shared variables to ensure correct access.

The stricter constraint than thread security is "reentrant", that is, the function is paused during execution in one thread, and then called in another thread, and then continue execution in the original returned thread. The correct execution can be ensured throughout the process. To ensure the reusability, we usually make a local copy of global variables.

Thread lifecycle

The so-called xx life cycle is actually a state chart containing the generation and destruction of an object. Shows the thread lifecycle:

The statuses are described as follows:

New. The newly created thread enters the Runnable state after initialization.

Runnable is ready. Wait for thread scheduling. Enter the running status after scheduling.

Running.

Blocked blocking. Stop running. after blocking is removed, the system enters the Runnable state and waits for scheduling again.

Dead disappears. After the thread method is executed, return or terminate abnormally.

There may be three cases from Running to Blocked:

Synchronization: the synchronization Lock is obtained in the thread, but the resource has been Locked by other threads and enters the Locked state until the resource can be obtained (the acquisition order is controlled by the Lock queue)

Sleep: after the thread runs the sleep () or join () method, the thread enters the Sleeping state. The difference is that sleep waits for a fixed period of time, while join waits for the sub-thread to finish running. Of course, you can also specify a "timeout" for join ". In terms of semantics, if two threads a and B call B. join () in a, it is equivalent to merging (join) into a thread. The most common case is to join all sub-threads in the main thread.

Wait: after the wait () method is executed in the thread, the thread enters the Waiting state and waits for notifications from other threads ).

Thread type

Main thread: When a program starts, a process is created by the operating system (OS), and a thread runs immediately, this Thread is usually called the Main Thread of the program ). Each process has at least one main thread, which is usually closed at the end.

Sub-thread: other threads created in the program, which is the sub-thread of the main thread as opposed to the main thread.

Daemon thread indicates a thread. The daemon thread provides services for other threads, such as JVM garbage collection threads. When all the remaining threads are daemon threads, the process exits.

Foreground thread: other threads relative to the daemon thread are called foreground threads.

Support for multiple threads in python

Virtual Machine layer

The Python virtual machine uses GIL (Global Interpreter Lock, Global Interpreter Lock) to mutex the thread's access to shared resources. At present, it cannot take advantage of the multi-processor.

Language Level

At the language level, Python provides excellent support for multithreading. in Python, multithreading-related modules include thread, threading, and Queue. It can easily support features such as thread creation, mutex lock, Semaphore, and synchronization.

Thread: the underlying support module of multithreading, which is generally not recommended.

Threading: encapsulates threads to visualize operations of some threads and provides the following classes:

Thread class

Timer is similar to Thread, but it takes a while to start running.

Lock primitive

RLock can be reentrant. So that a single thread can obtain the acquired lock again

The Condition variable can stop a thread and wait for other threads to meet a certain Condition"

The condition variable used by the Event. Multiple threads can wait for an event. after the event occurs, all threads are activated.

Semaphore provides a structure similar to "waiting room" for lock-waiting threads

BoundedSemaphore is similar to semaphore, but it cannot exceed the initial value.

Queue: implements queues of multiple producers and consumers. it supports lock primitives and provides good synchronization support between multiple threads. Provided classes:

Queue

LifoQueue (LIFO) queue

PriorityQueue priority queue

The Thread class is your main Thread class and you can create process instances. The functions provided by this class include:

GetName (self) returns the thread name

The isAlive (self) Boolean flag indicates whether the thread is still running.

The daemon flag of the thread returned by isDaemon (self)

The join (self, timeout = None) program is suspended until the thread ends. if a timeout is provided, a maximum of timeout seconds can be blocked.

Run (self) defines the function of a thread.

SetDaemon (self, daemonic) sets the daemon flag of the thread to daemonic

SetName (self, name) sets the thread name

Start (self) starts Thread Execution

Third-party support

If you are particularly concerned about performance, you can also consider the implementation of some "micro-threads:

Stackless Python: an enhanced version of Python that provides support for micro-threads. A micro-thread is a lightweight thread. it takes more time to switch between multiple threads and consumes less resources.

Greenlet: a by-product of Stackless. it calls microthreads "tasklet ". Tasklet runs in pseudo-concurrency and uses channel for synchronous data exchange. "Greenlet" is a more primitive micro-thread concept without scheduling. You can construct a micro-thread scheduler by yourself, or use greenlet to implement advanced control flow.

In the next section, create and start a thread using python.