Cutting-edge design concept: looking forward to the performance of the systemd Program

In short, systemd is an init program. According to the previous Upstart introduction, you should briefly understand the shortcomings of the traditional Sysvinit system. However, the design concept of systemd is more advanced. It can be said that it draws on the idea of launchd on OSX.

The init process is started by the kernel. Is it the parent process of all other processes? As a result, more tasks can be done than other processes, such as loading service processes during startup.

The traditional sysinit system does not conform to an effective and fast init system standard:

1. Start as few processes as possible

2. Try to start more processes in parallel

Clause 1 means that the instance is not started unless necessary. Unlike syslog and swap, the Bluetooth service is only required when the Bluetooth adapter is inserted. The printing service is only required when the printer is connected or the program is to be printed. Even sshd is not required, the Administrator often runs sshd for listening for ssh that is only connected once in a few months. Clause 2 means that the startup process should not be executed serially like sysvinit to maximize the efficiency of CPU and IO bandwidth.

Dynamic Changes in software and hardware

Modern Operating systems should have a high degree of dynamic characteristics: different programs start and stop, different hardware is added and removed, so services need to be dynamically started when needed. Most of the current systems continue to synchronize the calls of some daemon when trying to parallelize the startup process as much as possible: for example, Avahi requires failover, therefore, Avahi is started only after the supervisor is started and the signal preparation is complete. Similarly, libmongod and X11 require HAL though HAL will be removed). libmongod and X11 won't start before HAL starts.

Parallel Socket Service

The above synchronization process leads to the serialization of the startup process. Can we get rid of the disadvantages of synchronization and serialization? The answer is yes! First, we need to understand what the real dependency of daemon on each other is and why they are delayed at startup. For traditional Unix daemon, the answer is: they remain in the waiting state until the socket provided by the required service is ready for connection. AFUNIX is common, but it may also be AFINET. For example, the customer program that requires the Guest Service will wait before/var/run/guest/systembussocket can be connected. The customer program of the syslog service will wait on/dev/log... this is what they only need! This means that as long as this condition is met, the customer program does not have to wait, regardless of whether the actual service has been started. Details are provided below)

The above is the true meaning of waiting. If we can set up the socket necessary for the customer program as soon as possible and keep the customer program in a waiting state, rather than starting the customer program after the service program is fully started, we can speed up the process and further run the parallel process. How can we achieve this? In Unix systems, it is very simple: listen to its socket before you really need to start the service, and then pass the socket to exec () (meaning is not understood here, in the init system, you can create all the sockets required by all daemon and run all daemon once in step 2. If one service needs another, but the number of the service is not fully started by the user, there is no problem. Before the user queries the socket, the customer process will be blocked due to this request before processing the request, however, only the client process blocked by this request is blocked. In addition, the dependency between services will no longer need to be manually configured): If all sockets required by all services are created at a time and all related services are started ), all the requirements of all services will be met at the end.

The following will be important: If syslog is started at the same time and multiple of its customer processes mean that syslog has not yet started to prepare for request processing), and the customer process starts to send requests, the request will be written to the/dev/log Socket cache. Unless the cache is full, the customer process can continue to complete the startup process without waiting. When syslog is fully started, all messages of the processor are sent. Another example: the consumer starts with the customer process. When a synchronous request is sent but does not receive the expected response, the customer process will be blocked. When the consumer starts the post-processing request, the customer process continues.

The socket cache of the kernel will help achieve maximum parallelism. Because the kernel has completed synchronization, no additional management of user space is required! If all the required sockets are available before daemon is started, the dependency will become redundant at least secondary ). Daemon A needs another daemon B, which is simply connected to B. If B has started, A's request is successful. If B is not started yet, if A does not send A synchronous request, it does not even need to care that B is not fully started ). Even if B does not execute such as crash at all, it is not too late to restart B. For A, there is no difference between the two. In this way, we can achieve the best parallelism and random demand when loading on-demand. On this basis, it is more robust. Because the socket is available even when the corresponding service is temporarily unavailable, such as crash), the client process will not lose any request into the buffer, after the service is restarted)

More interesting things

First, this is not a brand new logic. This is just like what Apple's launchd system does: All the sockets required by daemon in OSX are created by launchd, and then the service itself starts in parallel, you do not need to manually specify related dependencies. This design idea is unique, which is also the reason for the rapid startup of OSX. However, this idea is not reflected in any system outside Apple.

Before launchd, this thought was intended to be an earlier embodiment: The old inetd worked like this: the socket was established by daemon, and the actual service in daemon through exec () the file descriptor of the socket is passed when the related process is started. However, inetd is not a local service, but a service on the network. It is not a proper tool to start or hide dependencies in parallel.

For TCP socket inetd, a new daemon instance is created for each incoming connection. This means that each connection will lead to a process initialization, which does not help high-performance servers. However, since then, inetd supports another mode Ticket: when the first connection comes, the first instance ticket is started and will continue to accept subsequent requests.

Parallel Bus service

In Linux, modern daemon interacts with each other through sockets instead of sockets. The question now is, can these services be implemented in parallel with the same logic as the traditional socket-Based Service Startup? The answer is yes. The Handler already provides all the necessary hooks: using the handler will load the service during the first access, in addition, the minimum synchronization mechanism, one per request, and initiated by both the consumer and producer is provided. For example, Avahi and CUPSCUPS require Avahi for mDNS/printer scanning on the DNS-SD) to start at the same time, simply start both at the same time, if CUPS is faster than Avahi to start, then the slave will cache the request into the queue, until the Avahi service is processed.

Therefore, the following is a summary: Socket-based and callback-based services can start all processes at a time without any additional synchronization. Activation-based policies can also delay loading services. If the service is rarely used, we can start it when it is accessed for the first time, instead of starting it during the startup process.

Parallel File System Tasks

The startup sequence diagram of the current release shows that the synchronization point of a file system task is larger than that of a daemon task: mount fsck. At present, it takes a lot of time to start the process of waiting for all the file systems listed in fstab to be loaded and fsck to be idle. The process will continue only after it is fully started. How can we improve it? The answer is The autofs system. Just as a connect () call means that one service is interested in another service, while an open () call or similar call means that the service has a need for a specific file or file system. Therefore, we should only block services that access the file system that are not yet available for mount and fsck), thus improving the concurrency. First load autofs and then continue the normal mount process. When the accessed file system is not ready, the kernel will cache the request for calling and the access process will be blocked, an unsuccessful access may only block one process. After a real file system is loaded, the startup process is completed as normal without any missing files, so that we can load the service before all file systems are ready.

It is not surprising if someone suggested that it would be inappropriate to build autofs into init, which may lead to more crash. However, after the experiment, I can say this is not true. Using autofs means that you do not need to immediately provide the real File System in the background, which will only lead to access delays. If the process tries to access autofs but the real file system fails to load for a long time, the process will be suspended by a sleep interruption, which means you can safely cancel it. If the file system fails to load at last, The autofs returns an error Code, such as ENOENT ). So I think the built-in autofs to init is correct. The experiment Code shows that this idea works well in practice.

Keep the PID of the First Login User very small, that is, to start less processes)

Another thing we can learn from the MacOS startup logic is that shell scripts are harmful. Shell is easy to hack, but execution is too slow. The traditional sysvinit startup logic is a mode around shell scripts, whether it is/bin/bash or other shells used to execute shell scripts faster.) the final results are destined to be slow in my system, /etc/init. the system script in d calls grep 77 times, awk92 times, cut23 times, And sed74 times at least. This process is generated once during each call, and the database is searched once. Some i18n processes load more. These processes based on the comparison of trivial strings lead to incredible slowness. Only shell scripts work this way. In addition, shell scripts are also very fragile, and environment variables can thoroughly change the script behavior behaviour), which is difficult to check and control.

So get rid of shell scripts during startup! Before doing so, we need to clarify the actual purpose of using shell scripts: In general, shell scripts are doing some boring work for trivial setup and uninstallation services. The content should be completed in C, the executable program located in the separation, or moved to the daemon itself, or handed over to init. It seems that we still cannot get rid of the script in the startup process, and it takes time to rewrite it with C, and sometimes shell scripts are too flexible to be detached, but we certainly can make them not as prominent as they are now ).

Simple measurement of the harm of shell scripts to the startup process can be seen from the PID of the first process loaded after the system is fully started: Start, log on, open the terminal, input echo $, linux and MacOS are compared as follows: Linux PID: 1823, MacOS PID: 154. Test it by yourself.

Next, the author analyzes some of Upstart's solutions and shortcomings for these problems, and then introduces the final conclusion of the author: putting it all together: systemd

The feature is that the core idea of translation has been explained in the above text, so I will not introduce the detailed concepts here. It is worth mentioning that systemd parses the traditional SysVinit script and converts its dependency to its native dependency. Therefore, it is compatible with the Sysvinit script while still providing concurrency, however, Upstart cannot parallelize traditional scripts. And can be controlled through the butler interface.

Current status: it can be used as an alternative to Sysinit or Upstart. A modified Fedora 13 qemu image is available for download. The prepared stopwatch timed Fedora13 uses Upstart to start 27 s, and systemd24s starts from grub to Em. In the same system, with the same settings, fewer parameters are obtained during the two consecutive start processes ).