Python concurrent programming Multi-Threading

One, what is a thread

A thread is an entity in a process, a process can have multiple threads, and a thread must have a parent process. Threads do not own system resources, only some data structures that must be run;
It shares all of the resources owned by the process with other threads of the parent process. Threads can create and revoke threads to implement concurrent execution of programs. In general, threads have three basic states of readiness, blocking, and running.

Second, the difference between thread and process

1) In short, a program has at least one process, and a process has at least one thread.

2) The thread partition scale is smaller than the process, which makes the multi-thread procedure high concurrency.

3) In addition, the process has a separate memory unit during execution, while multiple threads share memory, which greatly improves the efficiency of the program operation.

4) threads are still different from the process in the execution process. Each separate thread has a program run entry, sequence of sequence execution, and exit of the program. However, threads cannot be executed independently, and must be dependent on the application, which provides multiple threads of execution control.

5) from a logical point of view, the meaning of multithreading is that in an application, there are multiple execution parts that can be executed concurrently. However, the operating system does not consider multiple threads as separate applications to implement scheduling and management of processes and resource allocation. This is the important difference between processes and threads.

4. Pros and cons

Threads and processes have advantages and disadvantages in use: the overhead of thread execution is small but not conducive to the management and protection of resources, while the process is the opposite.

Third, why to use multi-threaded

Multithreading refers to the opening of multiple threads in a process, simply speaking: If multiple tasks share a single address space, then multiple threads must be opened within a process. The detailed lecture is divided into 4 points:

1. Multithreading share the address space of a process

2. Threads are more lightweight than processes, threads are easier to create revocable than processes, and in many operating systems, creating a line turndown creates a process 10-100 times faster, which is useful when a large number of threads require dynamic and rapid modification

3. If multiple threads are CPU-intensive, there is no performance gain, but if there is a lot of computation and a lot of I/O processing, having multiple threads allows these activities to overlap each other, which speeds up the execution of the program.

4. In a multi-CPU system, in order to maximize the use of multicore, you can open multiple threads, much less than the open process overhead. (This one does not apply to Python)

Four, when to use multi-threaded

1 , CPU-intensive code (various loop processing, counting, etc.), in this case, because of the computation work, and then trigger the Gil's release and re-competition (multiple threads to switch back and forth is of course the need to consume resources), so the multi-threading under Python is not friendly to CPU-intensive code, So it is recommended to use multiple processes at this time.

2, IO-intensive code (file processing, network crawler, etc.), multi-threading can effectively improve efficiency (single-threaded IO operation will be IO wait, resulting in unnecessary time wasted, and open multi-threaded can thread a wait, automatically switch to the threads B, can not waste the resources of the CPU, which can improve program execution efficiency). So Python's multithreading is more friendly to IO-intensive code. This fact is multi-threaded rather than multi-process.

Python concurrent programming Multi-Threading