Linux time Subsystem (13) Tick Device Layer Overview

First, preface

The tick device layer in the time subsystem mainly involves kernel/time/tick-* related files, and the main content of this article is to describe the operating logic of the tick device layer from the level of high (not entangled in specific lines of code).

If each. c file is a module, we can first briefly describe the individual modules of the tick device layer. TICK-COMMON.C describes some common operations for tick device, and the file also includes code for periodic ticks. To get the system to work in tickless mode (more accurately should be the dynamic tick module, which means that depending on the current state of the system, dynamic start-stop periodic tick) requires two modules of support, namely TICK-ONESHOT.C and TICK-SCHED.C. TICK-ONESHOT.C is primarily an operator interface function that provides the one shot mode associated with the tick device. Literally, TICK-SCHED.C is related to tick scheduling, the so-called tick scheduling includes two aspects, on the one hand, in the normal operation of the system, how to generate periodic tick event, on the other hand, when the system has no task to execute, enter the idle state, How to stop the periodic tick, and how to update the system state (for example: jiffies, etc.) when recovering. Tick-broadcast.c and TICK-BROADCAST-HRTIMER.C are related to tick broadcast, this article will not cover this part of the content, there will be a special document describing it.

The second chapter of this article describes the overview of tick device, and then in chapter three describes how the tick device layer is initialized, since the tick device always works in periodic mode, This chapter also describes, incidentally, the operation of periodic ticks. If the hardware and system configuration allow, the tick device in the system switches one shot mode into tickless mode, so the fourth chapter describes how dynamic tick works in the case of a high-precision timer. The fifth and fourth chapters are similar, but describe a situation where a high-precision timer is not configured.

II. Overview of tick device and software structure

Although there are some descriptions of tick device in the periodic tick document, it is reviewed here again, this time not to detail the data structure, but to describe the software structure of the tick device from a higher level.

1. What is tick

To understand what tick device is, what is tickless kernel, first of all, of course to understand what tick is? To understand what tick is, the first thing to understand is how OS kernel works. There are a lot of everyday things in the system that need to be dealt with, such as:

---Update the system time

---Handle Low precision timer

---processing the time slice information of the running process

The system uses polling methods when dealing with these things, which means doing them at a fixed frequency. HW is required to assist, in general, the hardware will have a HW timer (called the system timer) can periodically trigger interrupt, let the system to deal with the above-mentioned daily sexual affairs. Each time the timer breaks, the kernel's modules know that a fixed time slice is over. For daily life, the concept of tick is associated with a clock: the clock will emit a periodic tick, and the sound is called a tick. The CPU and OS kernel extend this concept: A recurring timer interrupt event is called a tick, and the device that generates the tick is called a tick.

How to select the tick cycle is required to balance power comsuption, time accuracy, and system time. We consider the low-precision tick-based timer module in the system, and selecting a higher tick frequency increases the time accuracy, for example, the 10ms tick period means that the time accuracy of the low-precision timer is 10ms, and setting the 3MS low-precision timer does not make any sense. In order to improve the time accuracy, we can increase the tick frequency, for example, can be raised to 1ms tick, but, when more CPU time is spent in the timer interrupt processing, in fact, when the system is not busy, not every tick is so meaningful, In fact most of the tick comes when OS kernel is often just idling and actually doing something that is harmful to the system's power consumption. For embedded devices, the cyclical tick is more harmful to power consumption because the standby time is an important indicator for embedded devices, while a cyclical tick means that the system cannot actually enter the idle state, but is periodically wakeup. These actions will consume the battery power. Similarly, for schedulers. If you set the tick of 10ms, the time slice precision allocated for each process is only 10ms, and the scheduler calculates that each process consumes CPU time only in 10ms units. To improve the process time slice accuracy, we can increase the tick frequency, for example, can be raised to 1ms tick, but, when more CPU time is spent on the process context switch, but the corresponding benefit is that the system response time will be shorter.

2. What is Tickless?

Tickless essentially removes the ticking of that annoyance. I'm not afraid of noise while I'm sleeping, but I'm afraid of a fixed-cycle tick, so I need a tickless clock. For OS kernel, tickless means there is no fixed-cycle timer interrupt event, but how does the fixed tick,os kernel work? We will also select the three topics in the previous section for each analysis.

First look at how to handle the timer. Various drivers and kernel modules (such as the TCP module for the network subsystem) have the need for a timer, so the time subsystem needs to manage all the timer registrations to the system. For systems with ticks, it is a logical idea to scan all the timer in each tick, and if the timer is checked for an extended (or expiring) system, the timer's callback function is called. Of course, because it is important to check the timer at the time each tick arrives, the kernel has some interesting designs, and interested readers can look at the scan process of the low-precision timer. What if there is no tick? This time need to find all the timer in the recently overdue timer, set its value to the actual HW timer is OK, of course, the need for the bottom of the HW timer to support one shot, that is, the timer interrupt to come once, In the interrupt processing of the timer in addition to processing the extended function, you also need to scan all the timer, to find the most recently extended timer, set its time value to the actual HW timer is OK, and then repeatedly repeat the above process is OK. Assuming that the system is registered with 1200ns, 1870ns, 2980ns, 4500ns, 5000ns and 6250ns of the timer, on a hz=1000 system, the overtime of the timer is in the regular tick point of time, for the tickless system, The interrupt for the timer is not uniform, as shown in the following:

Let's take a look at updating the system time. For a system with tick, it is very simple to call update_wall_time at each tick to update the system time, of course, because it is a cyclical tick, this time each time is summed up the same times. For the tickless system, we can choose to call Update_wall_time to update the system time during each timer's extended interrupt, however, this design is not good, on the one hand, if the number of timer in the system is too many, then Update_wall_ Is the time call too frequent, and does the system actually require such a high-precision value? More importantly: The timer interrupt arrival is uncertain, and the system timer settings related, some time period timer interrupt more frequent, get the system time accuracy is high, some time period, timer interrupt is sparse, then get the system time accuracy is lower.

Finally, let's see how the scheduler adapts to tickless. We know that unless you are a completely priority-based scheduler, the system assigns a time slice to the process, slice, when the CPU-consuming time slice quota is used, the process hangs in the queue, waits for the scheduler to allocate the next time slice, and dispatches the run. Having a tick is, of course, relatively simple, subtracting the current process's time slice from the timer interrupt of the tick. No tick is always troublesome, I can think of the method is: Suppose we give the process to allocate 40ms of time slices, then when scheduling the process needs to set a 40ms Timer,timer expires, the scheduler select another process, and then set the timer again. Of course, if there is no concept of process priority (or if the priority is only in the case of allocating more time slices), and there are fewer processes in the runnable state in the system, the overall operation is OK. What if I have a priority concept? What happens if the process is interrupted during execution and switches to another process? If the number of processes in the system is many how to guarantee the performance of the scheduler? Well, it's too complicated, and it's better to have a tick, so in practice, Linux kernel will start a cyclical tick when there is a task to execute. Of course, there is no absolute right design in the world, and any elegant design is suitable for a certain application scenario. In fact, the law of nature is not the same? Newton's law is not absolutely correct, only suitable for low-speed scenes, when the movement of objects near the speed of light, Newton's classical laws of mechanics have failed.

3. Tickless in the kernel

In this section we mainly look at the tickless in the kernel. Traditional UNIX and old Linux (before the beginning of 2000) are all tick (for the new kernel, the configuration of Config_hz_periodic is also tick), the new Linux kernel has added the tickless option:

---config_no_hz_idle

---config_no_hz_full

Config_no_hz_idle is that there is no tick when the system is Dile, of course, there is tick when the system is running, so we also call it dynamic tick or no HZ mode. After the 3.10 release, a full tickless mode was introduced, which sounded as if there was no tick in any case, but in fact it was not that strong, except that the CPU was idle to stop the tick, and when there was only one process running on the CPU, You can also stop the periodic tick, and in other cases, for example, there are multiple processes waiting to be scheduled for execution, there are still ticks. This configuration is actually only meaningful for high-performance computing (HPC) and is therefore not the focus of this article.

4. Tick Device Overview

The tick device is one that can provide a continuous tick event. Currently, there are two tick device periodic tick and one-shot tick in Linux kernel. Periodic tick generates a tick event at regular intervals. One-Shot tick is set to generate only one tick event, if you want to generate the tick event continuously, it needs to be set every time.

Each CPU has its own tick device. Defined as Tick_cpu_device. Each tick device has its own type (periodic or one-shot), and each tick device is actually a clock event device (which adds a member that represents mode), with different types of ticks The device has different event handler. For a tick device of type periodic, the event handler of its clock event device is tick_handle_periodic (no high precision timer is configured) or Hrtimer_ Interrupt (High precision timer configured). For a tick device of type one-shot, the event handler of its clock event device is hrtimer_interrupt (with a high precision timer) or tick_nohz_ Handler (no high-precision timer configured).

The Tick device module is responsible for managing all tick devices in the system, in SMP environments, each CPU has its own tick device, one of which is selected to be the global tick device, The device is responsible for maintaining the jiffies of the entire system and updating which system-wide statistics are based on jiffies.

Iii. kernel How to initialize the tick device layer and the operation of the periodic tick?

If the logic of tick device was originally a story, the story begins with the Clockevent device layer. Whenever a new clockevent device is added to the system at the bottom, it calls Clockevents_register_device or clockevents_config_and_register to the generic clockevent Layer to register a new clockevent device, this time, call Tick_check_new_device to notify the tick device layer that a new shipment has arrived. If you love me with tick device and clockevent device, then you'll call the Tick_setup_device function Setup, which is the tick device. In general, when the system initializes, all CPU tick device does not match the clock event device, so the CPU's local tick device is the global tick device. Also, if the tick device is married (the clock event device of the tick device equals null before the match, the Tick device mode is set to Tickdev_mode_periodic, even if the clock The event has one shot capability, even if the system is configured with no HZ. Well, anyway you need to start with a cyclical tick, so look at how the periodic tick is initialized.

The Tick_setup_periodic function is used to set a periodic tick device. Of course, the most important setting for event handler, for periodic tick device, the handler of its clock event device is set to Tick_handle_periodic. Light has handler also not, also have to kick off the hardware, let it periodically generate clock event, so as to promote the operation of the system (this is done by calling the interface function of the clockevent device layer).

Finally, let's think of a question: When does the system start with tick? multicore system, the BSP starts first, and during its initialization it calls Time_init, which initiates the Clocksource initialization process. At this point, the cyclical tick will begin. At some stage, the other processor will start and then register its own local timer, so that the tick on each CPU is started.

Iv. How does dynamic tick work when a high-precision timer is set?

1. Software level

The following diagram, which is centered on tick device, describes the interaction between the module and other time subsystem modules (configuring high-precision timer and dynamic tick conditions):

, the red Border module is per CPU module, so-called per CPU means that each CPU will maintain belong to a own object. For example, for a tick device, each CPU maintains its own tick device, but in order to keep the picture from getting too large and drawing only one CPU, the other CPU's actions are similar. Why is the clock event not painted with a red border? In fact, the clock event device is not per CPU, some per CPU local timer, there is a global timer, if the hardware designer is willing, a CPU can have multiple local timer, All timer hardware in the system is abstracted into one clock event device for system level management, and each CPU does not specifically maintain a clock event device that belongs to itself. Weak water 3,000, take only one scoop. Each CPU will only select the most appropriate clock event device in the many clock event device to build the CPU local tick device.

The driving force of the tick device system comes from the interrupt subsystem, and when the HW timer (the one used by the tick device) is triggered by an interrupt, the hrtimer_interrupt is called to drive the high-precision timer (call to execute the extended timer) Back function). And in the Hrtimer_interrupt will be scanned to save high-precision timer red black tree, find the next time to set the expiration, call Tick_program_event to set the next extended event, you know, this is our tick Device works in one shot mode and requires constant set next expire time to drive the entire system forward.

The traditional low-precision timer is a cyclical tick driver, but what if the tick is at one shot mode? Can only be simulated, Tick device layer needs to set a periodic triggered high-precision timer, in the timer's extended function (Tick_sched_timer) to perform the original periodic Tick work, such as triggering timer_ SOFTIRQ to drive the system's low-precision timer operation, update the real clock in the timekeeping module.

2. How to switch to Tickless

We know that the start tick device always works in the periodic tick mode, and everything is just like in the past, whenever the system always has that periodic tick coming. This cyclical tick is driven by the interruption of the hardware timer, which registers the soft IRQ, so the HW timer will periodically trigger the execution of the soft IRQ, which is the RUN_TIMER_SOFTIRQ function. In this function, the Hrtimer is switched to high-precision mode as appropriate (Hrtimer also has two modes, a high-precision mode, and a low-precision mode, which always starts with a low-precision mode). When the system switches to the high-precision timer mode (hrtimer_switch_to_hres), because the high-precision timer must require the underlying tick device to run in one shot mode, this is called tick_switch_to The _oneshot function switches the mode of the tick device on the CPU to the one shot (note: The event handler is set to Hrtimer_interrupt). Similarly, the underlying clock event device will be set to one shot mode. Once into the one shot mode, the periodic timer interrupt will disappear, from this system will only be based on the system's Hrtimer settings to set the underlying HW timer trigger.

3. How to generate periodic tick

While the tick device and the underlying HW timer are all working in one shot mode, it seems that the system's HW timer interrupts are generated on demand and how wonderful. However, for a variety of reasons (omitted here 3000 words), during the system operation, the periodic tick also needs to be maintained, so when switching to the one shot mode, the Tick_setup_sched_timer function is also called to create a sched Timer (a common hrtimer only), the characteristics of the timer is that each time after the extended call Hrtimer_forward, and constantly will be hung back to hrtimer red and black trees, so, tick_sched_do_ The timer interface is called continuously according to the tick cycle, thus simulating the periodic tick.

4. How to stop the tick when idle

We know that the swapper process on each CPU (process No. 0, also known as the Idle process) will eventually execute the Cpu_idle_loop function, which calls Tick_nohz_idle_enter before actually executing the CPU idle instruction, where the function Sched timer will be stopped, therefore, periodic HW timer will not come back, when the CPU from idle wake up to the actual system in the Hrtimer of the most recent extended time.

5. How to restore tick

The concept goes as above, when waking from idle, the Tick_nohz_idle_exit function is called, the sched timer is rebuilt, and everything is restored to its original state.

How does dynamic tick work without setting a high-precision timer?

This section is reserved for interested readers to learn for themselves.

Linux time Subsystem (13) Tick Device Layer Overview