Python Learning note __10.4 process vs thread

# This is a learning note for the Liaoche teacher Python tutorial

1 , overview

We have introduced multi-process and multi-threading, which are the two most common ways to achieve multitasking. Now, let's discuss the pros and cons of both approaches.

to achieve multitasking, we typically design Master-worker mode , where Master is responsible for assigning tasks, and the worker is responsible for performing tasks, so in a multitasking environment, it is usually a master and multiple workers.

• If you implement master-worker with multiple processes, the master process is master and the other process is Worker.

• If the Master-worker is implemented with multithreading, the main thread is master, and the other threads are worker.

1.1 and the advantages and disadvantages of multi-process mode

1) Advantages

high stability because one child process crashes without affecting the master and other child processes. ( of course , the main process hangs up all the processes, but the master process is only responsible for assigning tasks, the probability of hanging off is low)

2) Disadvantages

The cost of creating a process is large , and under the Unix/linux system, it is OK to make a fork call, and the process overhead under Windows is huge. In addition, the operating system can run simultaneously the number of processes is also limited, under memory and CPU constraints, if there are thousands of processes running simultaneously, the operating system even scheduling will be problematic.

1.2 , the pros and cons of multithreaded mode

1) Advantages

Multithreaded mode is usually a little faster (only a little) than a multi-process. Under Windows, multithreading is more efficient than multiple processes, so Microsoft's IIS server uses multithreaded mode by default.

2) Disadvantages

The fatal drawback is that any thread that hangs up can directly cause the entire process to crash because all threads share the memory of the process. Because of the stability of multithreading, IIS is less stable than Apache.

2 , process switching, and thread switching

the process switches in two steps, the thread only makes the first 2 Step:

1. Switch the page directory to use the new address space

2. Switch between the kernel stack and the hardware context.

one of the main differences between thread context switching and process context switching is that the switching of virtual memory space for threads is still the same, but process switching is different . The processing of both context switches is done through the operating system kernel. The most significant performance loss associated with this switching process of the kernel is to switch the contents of the register out.

regardless of the process / thread switching involves:

1. Saves the currently executing field environment (CPU register status, memory pages, etc.)

2. Prepare the execution environment for the new task (restore the last register state, switch memory pages, etc.)

This switching process is fast, but it also takes time. If there are thousands of tasks at the same time, the operating system may be mostly busy switching tasks, there is not much time to perform the task. Therefore, once the multi-tasking to a limit, it will consume all the resources of the system, resulting in a sharp decline in efficiency, all tasks are not good.

3 , compute-intensive vs. IO Intensive

The second consideration with multitasking is the type of task. We can divide the task into compute-intensive and IO-intensive .

3.1 , compute-intensive

to do a lot of computing, consumption of CPU resources, such as the calculation of pi, video, such as HD decoding, all rely on the CPU's computing power . This computationally intensive task can be accomplished with multitasking, but the more tasks, the more time it takes to switch tasks, the less efficient the CPU is to perform the task, so the most efficient use of the CPU should be equal to the number of cores in the CPU .

Compute-intensive tasks are critical to the efficiency of your code because they consume CPU resources primarily . Scripting languages like Python are inefficient and are completely unsuitable for compute-intensive tasks. For computationally intensive tasks, it is best to write in C .

3.2 , IO Intensive

tasks related to network and disk IO are IO-intensive tasks that are characterized by a low CPU consumption and most of the time the task is waiting for the IO operation to complete (because IO is much slower than CPU and memory). for IO-intensive tasks, the more tasks you have, the higher the CPU efficiency, but there is a limit. Most of the tasks that are common are IO-intensive tasks, such as Web applications.

IO-intensive task execution, 99% of the time spent on the IO, the time spent on the CPU is very small, so the fast-running C language to replace the very low-speed scripting language with Python, completely unable to improve operational efficiency. for IO-intensive tasks , the most appropriate language is the most efficient (least code) language, the scripting language is preferred , and the C language is the worst.

4 , asynchronous IO

Considering the huge speed difference between CPU and IO , a task is waiting for the IO operation most of the time during execution, and the single-process single- threaded model causes other tasks not to execute in parallel , so we Requires a multi-process model or multithreaded model to support multi-tasking concurrency execution.

The modern operating system has made great improvements to IO operations, with the biggest feature being the support for asynchronous IO. If you take advantage of the asynchronous IO support provided by the operating system, you can use a single-process single-threaded model to perform multitasking, a new model called Event-driven model ,nginx is a Web server that supports asynchronous IO , and it can efficiently support multitasking by adopting a single-process model on a single-core CPU. On multi-core CPUs, you can run multiple processes (the same number as the number of CPU cores) to take advantage of multicore CPUs. Because the total number of processes in the system is very limited, operating system scheduling is very efficient. using the asynchronous IO programming model to achieve multi-tasking is a major trend.

corresponding to the Python language, the single-threaded asynchronous programming model is called the co- process , and with the support of the coprocessor, an efficient multitasking program can be written based on event-driven.

Python Learning note __10.4 process vs thread