Linux kernel scheduling algorithm (1)--quickly find the highest priority process

Why should we understand the scheduling strategy of the kernel? Hehe, because it is worthy of our study, not a nonsense. The kernel scheduler is very advanced and powerful, managing a large number of messy processes running on your Linux, while maintaining a highly responsive response to user actions, and if possible, why not put this idea into your own application? Or, is it possible to better implement your own application, allowing the operating system to allocate resources to its own processes with its own will?

Look at kernel with these two questions. First of all, we developed the application, basically two types, 1, IO consumption type: such as the trunk service on Hadoop, it is obvious that its consumption mainly on the IO, including network IO disk IO and so on. 2. CPU consumption, such as mapreduce or other components that require a large amount of data to be processed, is like compressing high-definition video into a process suitable for the phone's viewing resolution, and their consumption is mainly on the CPU. How will the operating system dispatch the two classes of processes when they are running on a single server? Now that the servers are SMP-multicore, will a process switch back and forth on multiple CPUs? If I have a program, both IO consumption and CPU consumption, how to make multicore better scheduling my program?


A few more questions. Take a look at the kernel scheduler, let's start with its priority queue. The dispatcher code is in the schedule function of the kernel source KERNEL/SCHED.C.
First look at the following priority queue, each of which has a runqueue. What is Runqueue? The following will be described in detail, now you can understand that the kernel for each CPU assigned a runqueue, to maintain the CPU can run processes. Runqueue, there are several members of the Prio_array type, this is the priority queue, first look at its definition:

See how bitmap_size is calculated: #define Bitmap_size (((((max_prio+1+7)/8) +sizeof (long)-1)/sizeof (long))
So, the Linux default configuration (if you compile the kernel with the default option) Max_prio is 140, which means that altogether the kernel defines 140 priority levels for the process. A process that waits for a CPU to process may contain many priority processes, but Linux is the operating system of a preemptive scheduling algorithm, which means that the highest-priority process execution must be found when scheduling. The above bitmap_size value is calculated according to Max_prio 5, then bitmap is actually 32*5=160 bit, so it contains 140 bits of Max_prio. How is the priority queue used? Look at 2649 lines of code: Idx=sched_find_first_bit (Array->bitmap); This method is used to quickly find the highest-priority queue. See how it's implemented to make it easy for us to understand the design of this priority bit:

So what does __ffs do?

The Sched_find_first_bit return value is the ordinal of the queue with the highest priority, which corresponds to the use of HA, Queue=array->queue + idx, so that the process queue to be processed is taken. This design is very fast when looking for priority, it is worth learning.


OK, the priority queue is clear, now take a look at Runqueue, each runqueue contains three priority queues.

Linux is a time-multiplexed system, which means that by dividing the CPU execution time into many slices and then allocating it to the process, it seems that even a single CPU system can allow multiple tasks to execute simultaneously. Then, the time slice size is assumed to be 100ms, too short too long, too long is not sensitive, too short, even when the process can be switched to consume a few milliseconds. 100 process execution, after all the processes have used their own time slices, need to re-allocate time slices to all processes, how to allocate it? The For loop iterates through all the run state processes, resetting the time slices? This performance is not tolerated! Too slow, related to the current number of system processes. So what does the 2.6 kernel do? It uses the above mentioned two priority queue active and expired, as the name implies, active is still a time slice of the process queue, and expired is the time slice exhaustion must reallocate the time slice of the process queue.


The advantage of this design is that you don't have to cycle through all the processes to reset the time slices and see how the scheduling function is played:

When all the time slices of the running process are used up, the active and expired queues are swapped for pointers, no traversal, and the time-consuming process is individually reassigned when the ACITVE team is included in the expired queue.


Look again at the schedule (void) dispatch function, when a process sleeps or is preempted, the system starts debugging schedule (void) to decide which process to run next. The above-mentioned things are reflected in this function ha.

Of course, in our program, you can also change the priority of your processes by executing the following system calls. Nice system call can change the base priority of a process, and setpriority can change the priority of a set of processes.

Linux kernel scheduling algorithm (1)--quickly find the highest priority process