The scheduling principle of process, thread, and association of Python

Summary of scheduling and running principles for processes, threads, and co-routines.

Introduction to scheduling policies for processes and threads

Linux operating system detailed scheduling policy can be consulted: http://blog.csdn.net/gatieme/article/details/51872659

There are three main scheduling strategies in Linux:

1. Priority scheduling: The process is divided into ordinary processes and real-time processes;

2. FIFO (queue) Scheduling: real-time processes are created first and executed until

3. Rotation scheduling (time slice): to achieve a certain amount of CPU execution time after the forced switchover;

The scheduling of multi-process programs is still the thread scheduling, the thread is the basic unit of CPU scheduling; In the same process, inline switching does not result in process switching, which causes the process to switch when a thread in one process switches to a thread in another process.

Causes of process or thread scheduling

• The executing process is complete;

• A process in progress has been blocked, such as calling sleep

• The executing process invokes the P primitive operation, which is blocked due to insufficient resources, or invokes a V primitive operation to activate the process queue waiting for the resource;

• The process of executing the I/O request is blocked;

• CPU allocated time slices run out; (default 10ms)

• The priority of a process in the ready queue becomes higher than the priority of the current execution process, triggering a forced switchover;

Python process or thread scheduling under Linux

If we use Python to create multi-process or multi-threading, we can assume that these processes or threads are real-time processes in a fair queue (that is, the same priority), then the scheduling policy is FIFO and RR.

For example, suppose you now have a single-core Cpu,python program that creates 5 threads, the five lines that Cheng created are entered into a fair queue, the CPU starts executing the first thread on the FIFO principle, and if it encounters an IO operation or hibernation, or if it takes more than 10ms to execute the thread The cup will stop the current thread, tune to the second thread to execute until the fifth thread, and then start the loop from the first thread until all the threads have finished executing the resource that is recycled by the operating system.

Of course, switching processes or threads also pays a price, and process switching costs more than threads.

Resource comparisons for processes, threads, and co-routines

Process:

• After creating a process, each process has its own independent memory address space, code snippets, data segments, BSS segments, heaps, stacks and other information for all user spaces;

• In multi-process, the sub-process replicates almost all the information of the main process, out of the PID and other special information;

Thread:

• One process under multiple threads, multiple threads share process code snippets for processes, public data (heaps) of processes, and other auxiliary resources owned by the process;

• The resources owned by each thread are: Thread ID, program counter, a stack, counter register, and stack to hold the thread's execution history and execution state.

Co-process:

• The process can be considered as a lightweight thread, that is, the co-process is open on the threads, multi-process implementation in a single-threaded concurrency, and the operating system can only sense the thread, that is, the switch of the association is not aware of the operating system, is a program-level switch;

• Multiple co-processes share single-threaded code snippets, public data (heaps), and so on;

• Each process has its own stack to hold the context state, the transition cost of the association is smaller, for the operating system, it will be considered that a multi-threaded thread has been in the calculation;

• The advantage of the co-process is that the cost of switching is smaller, so the efficient utilization of the CPU is improved.

• Python's process mainstream is implemented through the Gevent and Asyncio modules, whose core principle is to create event loops in the underlying code to dispatch the context of multiple threads;

Application scenarios for processes, threads, and co-routines

• Python's code avoids using multithreading as much as possible;

• If the context has a section of code can be divided into relatively independent two parts, if the independent two parts are CPU-intensive, then use multi-process, if it is IO-intensive, then use the coprocessor, if both are involved, you can consider using a child process to run the coprocessor.

• Business code to quickly create a thread or process, while enhancing the readability of the code, it is recommended to use anonymous functions.

from redis import StrictRedisrs = StrictRedis(host=‘192.168.1.20‘, port=6390, db=1)def get_rs1():    t = time.time()    res = gevent.joinall([gevent.spawn(lambda x: gevent.sleep(2), x=i) for i in range(2)])    ls = [x.get() if x.kwargs[‘x‘] == 1 else x.get() for x in res]    print(ls)    print(time.time() - t)if __name__ == ‘__main__‘:    get_rs1()# gevent需要判断返回的结果的顺序

• Reference

• Https://www.cnblogs.com/njczy2010/p/5800072.html

• Http://www.gevent.org/intro.html

The scheduling principle of process, thread, and association of Python