New Tricks: Extreme Optimization of Linux Startup time

◆ 1) First, tracking and analyzing the Linux Startup Process and generating a detailed startup time report.

The simple and feasible method is to add a timestamp for all kernel information during the startup process through the PrintkTime function, which facilitates summary and analysis. PrintkTime was one of the earliest Kernel patches provided by CELF and was formally incorporated into the standard Kernel in later Kernel 2.6.11. Therefore, you may directly enable this function in the new kernel version. If your Linux kernel cannot be updated to Versions later than 2.6.11 for some reason, you can refer to the methods provided by CELF to modify or directly download the patches they provide: http://tree.celinuxforum.org/CelfPubWiki/PrintkTimes

You can simply enable the PrintkTime function by adding "time" to the kernel startup parameters. Of course, you can also choose to directly specify "Show timing information on printks" in "Kernel hacking" during Kernel compilation to force the timestamp to be added to the Kernel information at each startup. This method also has another advantage: You can get all the information of the kernel before parsing the startup parameter. Therefore, I chose the next method.

After completing the preceding configuration, restart Linux and run the following command to output the kernel startup information to the file:

Dmesg-s 131072> ktime

Then, use the script "show_delta" located in the scripts folder of the Linux Source Code) to convert the output file to the time increment display format:

/Usr/src/linux-x.xx.xx/scripts/show_delta ktime> dtime

In this way, you get a detailed report on Linux Startup time consumption.

◆ 2) then, we will use this report to find out the relatively time-consuming process during startup.

It must be clear that there is no necessary correspondence between the time increment and kernel information in the report. The real time consumption must be analyzed from the kernel source code.

This is not difficult for a friend who is a little familiar with programming, because the time increment is only the time difference between two calls to printk. Generally, during kernel startup, some time-consuming tasks, such as creating hash indexes and probe hardware devices, are printed through printk. In this case, the time increment usually reflects the time consumed by the information-related process. However, in some cases, the kernel starts the corresponding process only after the printk output information is called, in the report, the time consumed for the corresponding kernel information process corresponds to the time increment of the next line. In other cases, the time is consumed in an uncertain period between two kernel information outputs, in this way, the time increment may be completely unable to be reflected through the kernel information.

Therefore, in order to accurately determine the real time consumption, we need to analyze it with the kernel source code. When necessary, for example, in the third case above, You have to insert printk printing in the source code to further determine the actual time consumption process.

The Linux kernel startup analysis after my last reduction is as follows:

Total kernel Start Time: 6.188 s

Key time-consuming part:

1) initialization of core components such as 0.652 s-Timer, IRQ, Cache, And Mem Pages
2) 0.611 s-kernel and RTC Clock Synchronization
3) 0.328 s-calculate the total consumption of Calibrating Delay4 CPU cores)
4) 0.144 s-calibration APIC clock
5) 0.312 s-calibration Migration Cost
6) 3.520 s-Intel E1000 Nic Initialization

Next, we will analyze and resolve the above parts one by one.

◆ 3) Next, we will conduct specific sub-item Optimization

CELF has already proposed a complete set of startup optimization solutions for Embedded Linux for consumer electronic products. However, due to different applications, we can only use some of their experiences for reference, analyze and try your own problems.

Kernel key parts: Timer, IRQ, Cache, Mem Pages ......) Currently, there are no reliable and feasible optimization solutions for initialization.

For items 2 and 3 in the above analysis results, CELF has a special optimization solution: "RTCNoSync" and "PresetLPJ ".

The former is easier to implement by shielding the RTC clock synchronization during the startup process or placing the process after startup based on the requirements of specific applications for clock accuracy, however, the kernel must be patched. It seems that CELF's current job is only to remove this process, rather than implementing the aforementioned "delay" to process RTC clock synchronization. For this reason, I have not introduced this optimization in my solution. After all, the time drift it brings has reached the "second" level.

The latter skips the actual computation process by forcibly specifying the LPJ value in the startup parameter, which is based on the fact that the LPJ value does not change without changing the hardware conditions. Therefore, after normal startup, record the "Calibrating Delay" value in the kernel information, and then you can forcibly specify the LPJ value in the following form in the startup parameters:

Lpj = 1, 9600700

The 4 and 5 items in the above analysis results are part of SMP initialization, so they are not in the scope of CELF research. Maybe the multi-core MP4 will appear in the future ?......), You must be self-reliant. After studying the SMP initialization code, we found that "Migration Cost" can also be used as "Calibrating Delay" to skip the calibration time in a preset way. The method is similar. Add the following to the kernel startup parameters:

Migration_cost = 4000,4000

However, it is troublesome for Intel to initialize and optimize Nic drivers. Although it is also open-source, reading hardware drivers is no better than reading General C code, moreover, the "optimization" modification based on such a superficial understanding is hard to maintain. Based on Reliability, I finally gave up this path after both attempts failed. From another perspective, we can use the "parallel initialization" idea of CELF in the "ParallelRCScripts" solution to compile the NIC driver into a module independently, load it in the initialization script in sync with other modules and applications to eliminate the impact of Probe blocking on startup time. Considering that the application initialization may also use the network, in our actual hardware environment, only eth0 is available for use. Therefore, we need to take the 0.3s of the first network port initialization time into consideration.

In addition to the Optimization points mentioned above in my solution, CELF also proposes some special optimizations that you may be interested in, such:

ShortIDEDelays-shorten IDE test duration my application scenario does not contain hard disks, so it cannot be used)

KernelXIP-run the kernel directly in ROM or Flash. Considering the compatibility, the kernel is not used)

IDENoProbe-Skip the IDE port of the unconnected device

OptimizeRCScripts-optimize the linuxrc script in initrd. I used the busrc script in BusyBox, which is more concise)

And other optimization solutions that are still in the hypothetical phase. If you are interested, visit CELF Developer Wiki to learn more.

◆ 4) optimization results

After the special optimization and redundant reduction of inittab and rcS scripts, the startup time of the Linux kernel is reduced from 6.188 s before the optimization to 2.016 s. If eth0 Initialization is not included, only 1.708seth0 Initialization is required to load data in parallel with system middleware and some applications. In combination with Kexec, it can greatly reduce the reset time caused by software faults and effectively improve product reliability.

Related Articles]

• Newbie School: instructions on how to start the Linux operating system

• Start and install Ubuntu Linux with USB flash

• Linux 2.6 kernel startup and command line Transfer Process Analysis

• Detailed analysis of the boot guide process for Linux operating system programs