Python Threads & Processes & co-process [0]-Basic concepts

Basic Concept /basic Concept

0 Introduction and Motivation /Why Multi-thread/multi-process/coroutine

before multithreading (multithreaded, MT) programming occurs, the execution of a computer program is made up of a sequence of individual steps, which is performed in the host's CPU in a synchronized order. That is, regardless of the number of tasks, whether or not to include subtasks, should be done sequentially.

However, assuming that the subtasks are independent of each other and that there is no causal relationship, this parallel processing can significantly improve the performance of the entire task, which is multithreaded programming, if the separate tasks can be run concurrently.

For Python, though controlled by the Gil (Global Interpreter Lock), multithreading may not improve performance when dealing with compute-intensive programs, but for I/O intensive programs, Python's multithreaded mode can be a good performance boost.

1 related nouns /Relevant Noun

1.0 Process /Process

is a computer program on a data set on a running activity, the system is the basic unit of resource allocation and scheduling, is the basis of the operating system structure. In the early process design-oriented computer architecture, the process is the basic execution entity of the program, and in the contemporary thread-oriented computer architecture, the process is the container of the thread . A program is a description of the instruction, the data and its organization, the process is the entity of the program , is an executing program , also known as the heavyweight process .

1.1 Threads /Thread

A thread, sometimes called a lightweight process (lightweight PROCESS,LWP), is the smallest unit of program execution Flow . A standard thread consists of a thread ID, a current instruction pointer (PC), a collection of registers , and a stack . In addition, the thread is an entity in the process, the basic unit that is dispatched and dispatched independently by the system, the thread itself does not own the system resources , only has a bit of resources that are essential in the operation, but it can share all the resources owned by the process with other threads belonging to one process. . One thread can create and revoke another thread, which can be executed concurrently between multiple threads in the same process. Because of the mutual constraints between threads, the thread is running in a discontinuous. Threads also have three basic states of readiness, blocking, and running . A ready state is a thread that has all of the conditions running, is logically capable of running, is waiting on a processing machine, and a running state is a thread-owning processor that is running; a blocking state is a thread waiting for an event, such as a semaphore, to be logically unenforceable. Each program has at least one thread, and if the program has only one thread, it is the program itself.

1.2 co-process /Coroutine

A process can be interrupted at a predetermined or random location of a subroutine during a thread's execution, and then execute another subroutine, returning at the appropriate time and then executing. It is a special subroutine or function called.

A program can contain multiple threads, which can be compared with a process that contains more than one thread. We know that multiple threads are relatively independent, have their own context, that the switchover is controlled by the system, and that the process is relatively independent, has its own context, but that its switchover is controlled by itself , and that the current coprocessor is controlled by the current process.

1.3 Global Interpreter Lock /Global Interpreter Lock

Global interpreter Lock Gil is a tool that the computer programming language interpreter uses to synchronize threads so that only one thread executes at any moment in the interpreter . Common examples are CPython (Jpython not using Gil) with Ruby MRI.

Formally due to the existence of the global interpreter lock, the Python interpreter can only be run as a single thread at any moment, that is, the multithreading in Python is actually a program that executes a certain amount of memory per thread in a multi-thread, and then switches to another thread to continue execution until it is again cut back. As a result, Python has a greater advantage when it comes to cpu-bound computation, while processing i/o-intensive computation like crawlers.

1.4 Daemon Threads /Daemon Thread

A daemon thread is a thread that provides a generic service in the background while the program is running, such as a garbage collection thread that is a competent guardian, and that thread is not an integral part of the program. Therefore, when all non-daemon threads end , the program terminates and kills all the daemon threads in the process . Conversely, the program will not terminate as long as any non-daemon threads are still running. That is, the presence of a daemon does not affect program exit.

The only difference between the user thread and the daemon thread is that the virtual machine leaves , and if the user thread is all out of operation, only the daemon thread is present, and the virtual machine exits. Because there is no guardian, the daemon will have no work to do, there is no need to continue to run the program.

1.5 Signal Volume /Semaphore

Semaphore is one of the oldest synchronization primitives, invented by Dutch computer scientist Edsger W. Dijkstra. (He first used a function called P () and V () to correspond to acquire () and release ()). In the threading module, the semaphore manages a counter internally. Calling acquire () causes the counter to -1,release () to be +1. The value of the counter is never less than 0, and when the counter is 0 o'clock, the call to acquire () is blocked until other threads call Release (). Semaphore also supports context management protocols.

1.6 bounded signal Volume /Boundedsemaphore

A factory function in the threading module that returns a new bounded semaphore object. A bounded semaphore ensures that its current value does not exceed its initial value . If it is exceeded, the valueerror is raised. In most cases, semaphores are used to guard resources with limited capacity. If the semaphore is released too many times, it is a sign of a bug. If not given, value defaults to 1.

1.7 Synchronization Primitives /Synchronization Primitive

When a process invokes a send primitive, the sending process is blocked until the message begins to send, and the subsequent statement of the send primitive can continue until the message is completely sent. When a process invokes a receive primitive, it does not return control immediately, but waits until the message is actually received and puts it into the specified receiving area before returning to control and continuing with the subsequent instruction of the primitive. During this time it has been in a blocking state. The above send and receive are referred to as synchronous communication primitives or blocking communication primitives . Events, as a synchronous primitive, are a synchronization mechanism in computer science that indicates that the process-specific conditions in a wait have become true.

1.8 Lock /Lock

In the case of locks, the essence of a lock is the contractual control between a thread, that is, the role of a lock does not actually lock a resource so that other threads cannot be modified. The essence of a lock is that a lock is contracted between multiple threads, and the Common resource of the lock is agreed upon, and must have this lock when it is necessary to modify the resource corresponding to the lock. Permission to do so. Therefore, when you need to modify a resource, you need to try to obtain the corresponding lock, and when the permission of the lock is obtained by other threads, the remaining threads that need to get into the blocking wait state. And when the lock is released, the access to the permission is random, regardless of the time of entry into the block.

1.9 mutexes and Reentrant locks /Mutex Lock and Reentrant lock

For mutexes and reusable locks, the mutex can only be fetched once, and if multiple fetches are blocked, wait for the original lock to be released before the lock can be re-entered. A reentrant lock can be acquire into a lock multiple times by this thread, but requires the same number of locks and releases to be fully unlocked, and the release of the lock needs to be done in the same thread

Note: for a reentrant lock, the multiple-lock feature is only valid on this thread, that is, even if a reentrant lock is acquired by one thread, the other thread cannot enter again, only this thread can.

1.10 dead Lock /Deadlock

Deadlocks occur when a resource is called multiple times and the caller fails to release the resource. Deadlocks can be broadly divided into two forms, iterative deadlocks and mutual invocation of deadlocks. A generic deadlock is caused by a mutex, and a partial deadlock problem can be avoided by using a reentrant lock.

1.11 Thread Safety /Thread Safety

Thread Safety is a multi-threaded access, with a locking mechanism, when a thread accesses one of the class's data, protection, other threads can not access until the thread is finished reading, other threads are available. There is no data inconsistency or data contamination. thread Insecurity is not providing data access protection, it is possible that multiple threads change the data successively, causing the resulting data to be dirty data.

Reference Links

The third edition of Python core programming

https://stackoverflow.com/questions/24744739/ Multi-threading-in-python-is-it-really-performance-effiicient-most-of-the-time

Python Threads & Processes & co-process [0]-Basic concepts