Study notes-Liao Xuefeng <python 2.7 Tutorials >

Basic knowledge of processes and threads

The CPU execution code is executed sequentially, and the single-core CPU is executed alternately by making the task, "impersonation" in addition to multitasking concurrency. True multitasking is on multi-core CPUs, where each CPU is responsible for performing a task. But the actual number of tasks is much more than the number of CPU cores, so ultimately the operating system will be multi-tasking rotation to different core execution.

Association of process/thread and physical memory (register)/CPU: A function call that allocates a block of space in the stack, stores local variables and parameters, invokes the end, and the stack space is freed. Each thread has a separate stack, register. All threads in the same process share files, code, and data.

Process independence: In the CPU of the program counter PC , for example, the physical PC only one, but each process has a separate logical PC, save the program running values, but the CPU turn to the process, the logical PC value is copied to the physical PC.

Python supports multiple processes across platforms

Unix/linux platform Python supports fork (): Parent process replicates a child process

Windows platform Python supports process (Target,args): "Simulates the effect of fork ()", that is, all Python objects of the parent process are passed to the child process pickle

Data exchange methods for Python interprocess communication

Queue provided by the multiprocessing module, Pipes

Python multithread Basics

The standard library provides two modules: Thread and threading (recommended).

The task process starts by default a main thread called Mainthread, and the main thread can start a new child thread.

Multi-threaded variable lock: A statement in a high-level language is a number of statements when the CPU executes, and the results in the calculation are stored in temporary variables, each with its own temporary variables.

Be sure to release after acquiring the lock, otherwise the thread waiting for the lock will continue to block and become a dead thread. Python can be try...finally to ensure release.

The disadvantage of the Lock: (1) lock code can only be executed in single-threaded mode, prevent multithreading concurrency, reduce efficiency; (2) may cause a deadlock .

Python's Gil lock causes multithreading to not take full advantage of multicore CPUs

The Python interpreter has a Gil (Global interpreter Lock) lock that, before any thread executes, obtains the Gil Lock, and each execution of 100 lines of code, the interpreter automatically releases the Gil lock, allowing other threads to execute. So even if 100 threads run on a 100-core CPU, only 1 cores can be used.

The Gil is a legacy of the Python interpreter design, making it impossible for Python to make efficient use of multicore, but can be implemented in multiple processes.

Multiple Python processes have separate Gil locks that do not affect each other.

Select the lock global variable under multi-threaded or have to pass down the local variable?

Locking and passing local variables are all old methods, but local variables are cumbersome to pass them on. If you encounter a dedicated object per thread and cannot use global variables, the author thinks of using a global dict, but the corresponding object is saved by the thread itself as key, for example:

1 #-*-coding:utf-8-*-2 ImportTime , threading3 4Global_dict = {}5 6 classStudent (object):7     8     def __init__(self, name):9Self.name =nameTen      One     defReturn_name (self): A         returnSelf.name -  -  the defdo_task_1 (): -std = Student ('Elsa') -Global_dict[threading.currentthread ()] =STD -std =Global_dict[threading.current_thread ()] +     Print 'thread%s, do_task_1 () Std.name =%s\n'%(Threading.currentthread (). Name, Std.return_name ()) -  + defdo_task_2 (): Astd = Student ('Anna') atGlobal_dict[threading.currentthread ()] =STD -std =Global_dict[threading.current_thread ()] -     Print 'thread%s, do_task_2 () Std.name =%s\n'%(Threading.currentthread (). Name, Std.return_name ()) -          -  - if __name__=='__main__': in     Print 'thread%s is running ...'%Threading.currentthread (). Name -T1 = Threading. Thread (Target=do_task_1, name='T1') toT2 = Threading. Thread (Target=do_task_2, name='T2') + T1.start () - T2.start () the T1.join () * T2.join () $     Print 'thread%s ended.'% Threading.currentthread (). Name

Global_dict = {} as a global variable that holds the thread and its own object. Operation Result:

d:\workspacevscode>is == annathread mainthread ended.

Python then simplifies the storage and access of each thread-specific object, which encapsulates the above global_dict = {}. As can be understood, Local_school = Threading.local () is a global variable, but it is stored in the local variables of each thread, it is not necessary to manage the thread lock, feel very convenient appearance.

Use Threadlocal to save thread local variables with global variables when multi-threading

Often used to bind a database connection, HTTP requests, user identity information, and so on for each thread.

processes, threads and compute-intensive, IO-intensive tasks

Multiple processes are more stable, the child processes of the system are independent of each other, and the child process crashes without affecting other processes (except the main process)

Multithreading is more efficient, but because all threads share the memory of the process, a crash can cause the system to force the end of the entire process

Compute-intensive tasks such as video HD decoding , major overhead in CPU computing, task Count = best number of cores

IO-intensive tasks such as network, disk IO, etc., CPU consumption is very small, most of the time is waiting for the IO operation to complete , the more tasks, the higher the CPU efficiency (limited)

Cannot see the huge speed differences between CPU and IO operations in isolation

Since a task is executed most of the time is CPU waiting for IO, single-process single-threaded and can not fully utilize the CPU resources, when the CPU waits for IO, do not spend idle time to perform CPU computing tasks. "Modern operating systems have made great improvements to IO operations, and the biggest feature is the support for asynchronous IO. If you take advantage of asynchronous IO support provided by the operating system, you can perform multiple tasks with a single-process single-threaded model. " (Android Asynctask is a kind of asynchronous, but it is essentially another small thread, so asynchronous is multithreaded?)

Study notes-Liao Xuefeng <python 2.7 Tutorials >