Python Multithreading & Process

One, Thread & process

For the operating system, a task is a process, such as open a browser is to start a browser process, open a notepad started a Notepad process, open two Notepad started the two Notepad process, open a word started a word process. A process is a collection of many resources.

Some processes do more than one thing at the same time, such as word, which can be typed, spell-checked, and printed at the same time. Within a process, to do multiple tasks at the same time, you need to run multiple "subtasks" at the same time, and we refer to these "subtasks" in the process as threads (thread).

Because each process has at least one thing to do, a process has at least a single thread. Of course, a complex process such as word can have multiple threads, multiple threads can execute simultaneously, multithreading is performed the same way as multiple processes, and the operating system quickly switches between multiple threads, allowing each thread to run briefly alternately, seemingly as if it were executing concurrently. Of course, a multi-core CPU is required to actually execute multiple threads at the same time. A thread is the smallest execution unit, and a process consists of at least one thread.

We are doing things, a person do is relatively slow, if more people together to do, it is faster, the program is the same, we want to run a little faster, you have to use multi-process, or multi-threading, in Python, multithreading is a lot of people criticized, why? Because the Python interpreter uses a Gil called the Global interpreter lock, it can't take advantage of multicore CPUs, it can only run on one CPU, but when you run the program it looks like it's still running together because the operating system turns the tasks around, Task 1 executes 0.01 seconds, switches to Task 2, Task 2 executes 0.01 seconds, then switches to Task 3, executes 0.01 seconds ... This is done repeatedly. On the surface, each task is executed alternately, but because the CPU is executing too fast, we feel as if all the tasks are executing at the same time. This is called context switching.

Two, multi-threaded, multi-threaded in Python using the Theading module

Here is a simple multithreaded

The following is another way to start multithreading, an inherited

There is no difference between the two ways, I personally like to use the first, more simple.

Thread waiting, multi-threaded at run time, each thread is running independently, not affected by other threads, if you want to run after which thread, and then do other things, you have to wait for it to complete, then how to wait, use join, wait for the end of the thread

Daemon thread, what is a daemon thread, it's like you're a king (not a daemon), then you have a lot of servants (Guardian threads), these servants are for you, and once you die, your servants are buried with you.

Thread lock, the thread lock is that many threads together in the operation of a data, there may be problems, it is necessary to add a lock, only one thread at a time to manipulate the data.

Here is a simple crawler, look at the effect of multithreading:

Three, multi-process, said Python inside the multi-threaded, is not able to take advantage of multi-core CPU, if you want to use multi-core CPU, you have to use multi-process, Python multi-process use multiprocessing module.

Python Multithreading & Process