LINUX PID 1 and Systemd

LINUX PID 1 and Systemd
Http://coolshell.cn/articles/17998.html

To say clear Systemd, you have to start with the Linux operating system started speaking. The startup of the Linux operating system starts with the BIOS, then the boot Loader loads into the kernel and initializes the kernel. The final step in kernel initialization is to start the init process. This process is the first process of the system, PID 1, also known as the super process, called the root process. It is responsible for generating all other user processes. All processes will be suspended under this process, and if the process exits, then all processes are killed. If the parent process of a child process is retired, the child process is hung under PID 1. (Note: PID 0 is part of the kernel, mainly for internal feed-in, see: Process identifier)

SysV Init

PID 1 This process is very special, its main task is to bring the entire operating system into an operational state. For example: Start Ui–shell for human-computer interaction, or go to the X graphics window. Traditionally, PID 1 is compatible with the traditional Unix System V, so also called Sysvinit, which is the oldest implementation of the init. Unix System V was released in 1983.

Under the Sysvint, there are several modes of operation, also called RunLevel. For example: Common level 3 Specifies the command line interface to launch to a multi-user character, level 5 Specifies the start to graphical interface, 0 indicates shutdown, and 6 means reboot. It is configured in the/etc/inittab file.

There are also/etc/init.d/and/ETC/RC[X].D, the former store various processes of the start-stop script (need to follow the specification support START,STOP subcommand), the latter of the X represents a different runlevel under the corresponding background process services, such as:/ETC/RC3.D It's runlevel=3. The file is mainly the start-stop script in link to/etc/init.d/. There are also certain naming conventions: S or K, followed by a number, and followed by a custom name, such as: S01rsyslog,s02ssh. S means start, k means stop, number indicates the order of execution.

Upstart

Unix and Linux have been operating in sysvint for many years, about 2006 years after the Linux kernel entered the 2.6 era, Linux has a lot of updates. And, Linux began to enter the desktop system, and desktop systems and server systems are not the same, the desktop system is faced with frequent reboots, and users will be very frequent use of hardware hot-plug technology. As a result, these new scenes make Sysvint a lot of challenges.

For example, the printer needs CUPS and other service processes, but if the user does not play machine printing, start the service is a waste, and if not start, if you want to use a printer, you can not use, because Sysvint does not automatically detect the mechanism, it can only start all services at once. In addition, there is a problem with the network disk mount. In the/etc/fstab, responsible for the hard drive mount, and sometimes network hard disk (NFS or ISCSI) in which, but on the desktop, there is probably no network when the boot, so the network hard disk can not access, and cannot mount, which will greatly affect the startup speed. Sysvinit adopt Netdev way to solve this problem, that is, need the user in/etc/fstab to the corresponding hard disk configuration on the Netdev attribute, so sysvint boot will not mount it, only after the network is available, by a dedicated NETFS service process to hang Contained This management method is more difficult to manage, it is easy to let people fall out of the pit.

So, after evaluating several optional init systems, the Ubuntu developer decided to redesign the system, so that's what we saw behind the upstart. Upstart is an event-driven mechanism that changes the way in which the previously fully serial synchronous start-up service has been changed to an event-driven asynchronous approach. For example, if a USB stick is inserted and Udev is notified, upstart senses the event and triggers the corresponding service program, such as mounting the file system, and so on. Because of the use of an event-driven gameplay, so, when the operating system starts, many unnecessary services can not start, but wait for notification, lazy boot. And the benefit of event-driven is that the service can be started in parallel, and the dependencies between them are completed by the corresponding event notification.

Upstart has a very good design, the two most important concepts are Job and Event.

Job has a general job, there is service job, and, upstart management of the entire job life cycle, such as: waiting, starting, Pre-start, spawned, Post-start, Running, PR E-stop, stopping, killed, post-stop, etc., and maintain the state machine of this life cycle.

The Event is divided into three categories, signal, method, and hooks. Signal is an asynchronous message, method is synchronous blocking. Hooks is also synchronous, but between the preceding two, the process that issues the hook event must wait for the event to complete, but not check for success.

However, the events of upstart are very complex and chaotic, with a variety of events (events not well-categorized) causing a bit of clutter. However, because the whole event-driven design is much better than the previous sysvinit, it is also very welcome.

Systemd

Until 2010 one day, an engineer working at RedHat Lennart Poettering and Kay Sievers, began introducing a new init system--systemd. It was a very, very ambitious project that almost changed everything, and systemd not only wanted to replace the existing init system, but also wanted to do more.

Lennart agreed upstart did well, the code quality is good, the event-based design is also very good. But he feels upstart also has problems, the biggest problem is not fast enough, although upstart with events can achieve a certain degree of start-up parallelism, but, in essence, these events will let the startup process serially together. such as: NetworkManager in the D-bus of the startup event, while the D-bus in the startup events such as Syslog.

Lennart that, in terms of implementation, upstart is actually managing a logical service dependency tree, but the service dependency tree is relatively simple in its presentation, and you just need to configure--"Start B OK and start a" or "stop a after a" to stop B. But, Lennart says, this simplicity is actually harmful (this simplification is actually detrimental). He believes that

From a system management point of view, he will initially set up the entire system to start the service dependency tree, but this system administrator to human flesh of this is a very clean service according to the whole tree to translate into a computer to see the Event/action form, and Event/action This configuration is run-time, so you need to run it to know what it is.
The event logic is everywhere from head to toe, which expands the complexity of operations rather than the previous sysvint. That is, when the user has configured "start D-bus after startup NetworkManager", this upstart can dry, but conversely, if the user starts NetworkManager, we should first go to start his predecessor dependency D-bus, However, you should also configure the corresponding reverse Event. Originally, I only need to configure a dependency, the result now I want to configure a lot of situations in the event.
Finally, the Event in upstart is not standard, chaotic, and not well defined. For example: Existing, process start, run, stop events, USB device plug-in, available, unplugged events, file system devices being mounted, mounted and umounted events, and AC power cord connection and disconnection events. You will find that this process is starting to stop, USB, file system, power cord events, looks very similar, but not by the standardized pumping out, because the vast majority of events are triples: start, condition, stop. This conceptual design model does not appear in the upstart. Because upstart is designed as a single event, it ignores logical dependencies.
Of course, if systemd just solve upstart problem, he will change upstart, but Lennart ambition not only want to do such a thing, he wants to do more.

First of all, systemd aware of the init process of the primary goal is to allow users to quickly enter the operating system can operate the environment, so this speed must be fast, the faster the better, so, systemd design concept is two:

To start less.
And to start more in parallel.
That is, on-demand startup, can not start without booting, if you want to start, can be started in parallel parallel start, including you have dependencies between, I also started in parallel. Start on Demand OK understand, then, there is dependency on the parallel start, how does it do? Here, Systemd has borrowed from MacOS's launchd gameplay (on Youtube there is a sharing--launchd:one program to Rule them all, and there are related design documents on Apple's open source site--about Daemo NS and Services)

To address these dependencies, SYSTEMD needs to address three underlying dependencies--socket, D-bus, and file systems.

Socket dependent. If service C relies on the socket of the service s, start s before starting C, because C fails if the socket of S is not found at c startup. SYSTEMD can help you to set up a socket to receive all C requests and data when S is not ready, and to cache it, once S is all started, replace the cached data and the socket descriptor with the SYSTEMD.
D-bus dependent. D-bus Full name Desktop Bus, is a service used to communicate between processes. In addition to the process used for user-state processes and kernel-state process communication, also used for user-state processes. Now, many of the current service processes use D-bus instead of sockets to communicate. For example: NetworkManager is communicated through D-bus and other service processes, that is, if a process needs to know the state of the network, it will need to communicate through D-bus. D-bus supports the "Bus Activation" feature. In other words, a to D-bus service and B communication, but B does not start, then D-bus can put B up, in the process of B startup, D-bus to help you cache data. SYSTEMD can help you use this feature to start A and B in parallel.
File system dependent. During system startup, file system-related activity is the most time consuming, such as mounting the file system, disk checking on the file system (FSCK), disk quota checking, and so on are very time-consuming operations. While waiting for these jobs to complete, the system is in an idle state. Services that want to use the file system seem to have to wait for the file system to initialize before it can be started. SYSTEMD reference to the design of AutoFS, so that the dependent file system services and the file system itself initialization can work concurrently. AutoFS can monitor that a file system mount point is actually accessed to trigger a mount operation, which is implemented through the support of the kernel Automounter module. For example, when an open () system call is in "/misc/cd/file1",/MISC/CD has not yet performed a mount operation, when the open () call is pending, the Linux kernel notifies autofs,autofs to perform the mount. At this point, control is returned to the open () system call, and the file is opened normally.
A page of PPT from Lennart's presentation shows the start-up of different init systems.

In addition, SYSTEMD also managed some of the following when it was launched.

Replace the traditional scripted start-up with the C language. As I said earlier, Sysvint starts with a variety of scripts under/ETC/RCX.D. However, these scripts need to use awk, sed, grep, find, Xargs, and so on these operating system commands, these commands need to build the process, the cost of the build process is very large, the key is to generate these processes, the process will be a bit of a fart to go back. In other words, my operating system has done so many things for you to pull a process up, the result you turn a string into lowercase to retire, what is my operating system?

In a normal sysvinit script, there may be hundreds or thousands of such commands. So, slow death. As a result, SYSTEMD is completely replaced with the C language. Generally speaking, sysvinit, after the operating system started, with the echo $$ can be seen, the PID is assigned to thousands of appearance, and SYSTEMD system is just hundreds.

In addition, SYSTEMD is a truly managed service process that can track all the processes fork/exec on the service process.

We know that the best practices for traditional Unix/linux Daemon service processes are essentially this way (see this article "SysV Daemon")--when the process starts, close all open file descriptors (in addition to the standard descriptor 0,1,2), Then reset all the signal processing.
Call Fork () to create the child process, Setsid () in the child process, and then the parent process exits (for background execution)
again, calls the fork (), creates the grandson process, and determines that there are no interactive terminals. The child process then exits.
in the grandson process, the standard input standard output standard error is connected to the/dev/null, but also to create PID files, log files, processing related signals ... The
is the last thing to start providing services.
in the above process, the service process, in addition to two Fork fork will be a lot of sub-process (such as some WEB services process, according to the user's request link to fork sub-process), the process tree is quite difficult to manage, because once the parent process exits, The child process will be suspended to PID 1, so, basically, you can't find all the related processes from a given PID file by the service process (this is too high for developers), so it's quite Buggy to start the service in a traditional way, Because can't do to all the waiter out of the descendants to do monitoring.
to solve this problem, upstart tracks the related system calls such as fork () and exec () or exit () in the process through a perverted strace. This method is rather clumsy. Systemd used a very interesting gameplay to tracking all the processes that the service process was born with Cgroup (I spoke about this in the basic technology "cgroup" of Docker). Cgroup is mainly used to manage the process group resource quotas, so, regardless of how the service starts a new subprocess, all of these related processes will belong to a cgroup, so systemd just need to simply go through the corresponding cgroup of the virtual file system directory of the file, will be able to find all the relevant processes correctly and stop them all.
In addition, SYSTEMD simplifies the entire daemon development process:

You do not need to fork () two times, just implement the main logic of the service itself.
Don't need Setsid (), SYSTEMD will do it for you.
Do not need to maintain the PID file, SYSTEMD will help handle.
No need to manage log files or use syslog, or handle HUP log reload signals. Put the log on the stderr, SYSTEMD help you manage.
Processing the SIGTERM signal, this signal is to correctly exit the current service, do not do anything else.
......
Besides, SYSTEMD can also--

Automatic detection of the start-up between the services there is no ring dependency.
Built-in AutoFS automatic mount management function.
Log service. SYSTEMD has transformed the traditional syslog problem by saving the log in binary format and making the log index faster.
Snapshots and recoveries. A snapshot of the collection of services running on the current system and can be resumed.
......
There are many more, he took over a lot of things, so let a lot of people unhappy, because he is doing a lot of things that do not belong to PID 1.

Systemd Controversy and gossip

So systemd this thing is probably the most spit war ever in the open source software. Systemd is plagued by controversy, and the biggest controversy is that he has broken the design philosophy of Unix (the related philosophy can read "UNIX Programming Art"), and has done a chatty and rather complex thing. Of course, Lennart does not agree with such a statement, he later wrote a blog "The biggest Myths" to explain systemd is not so, we can go to the first reading.

What's the big controversy like? In 2014, Debian Linux was ready to use SYSTEMD as the standard init daemon to replace Sysvinit. The controversy surrounding the matter reached an unprecedented heat, the controversy was full of hatred, systemd supporters and opponents were all insulting each other, resulting in the Debian camp began to split. Others sent Lennart a death-threatening message, used Bitcoin to hire a killer, threatened to take his life, posted insulting songs in Youbute, and sent him nasty and insulting messages on IRC and various social channels. This is no longer a dispute, but a kind of outright hatred!

So, Lennart on Google Plus on the post, criticizing the entire Linux open source community and Linus himself. He said to the effect that

The community is so sick, it's all about the holes, and you're constantly insulting and abusing me in different languages and ways in various places. I am still a young man, I have never experienced such a scene, but today I have been very familiar with the scene. I may not be accurate at times, but I will not say that as he did, and I am not affected by these things because I am thick-skinned, so why I can make systemd successful in the face of big objections, but your Linux community is horrible. There are too many mentally ill people in you. In addition, for Linus Torvalds, you are the role model of this community, but unfortunately you are a bad Role model, you are in the community of the caustic and insulting words and deeds, basically to some extent, encourage others like you, of course, not just you a person's problem, Instead, a group of people like you have gathered around you to do this. Give you a word--a fish rots from the head down! A fish is rotting from the ground down ...

This flung is very long, and the students who like to gossip can go to the first reading. Feel the Lennart at the time (I think it is very smooth).

Linus was also asked by the media about SYSTEMD (see "Torvalds says he has no strong opinions on Systemd"), Linus said in an interview,

I have no strong idea of the SYSTEMD and Lennart posts. While the traditional Unix design philosophy-"Do one thing and does it well"-is good, and most of us have practiced it for years, it doesn't mean all the real world. In history, it's not just systemd. However, I am personally a old-fashioned person, at least I like the text-based log, not the binary log. But there is no need for systemd to have such taste. Oh, I said the details ...

Today, Systemd occupies almost all of the default configurations of the mainstream Linux distributions, including: Arch Linux, CentOS, CoreOS, Debian, Fedora, Megeia, OpenSUSE, RHEL, SUSE Enterprise Edition, and Ubuntu. And, for CentOS, CoreOS, Fedora, RHEL, SUSE, these distributions cannot be systemd. (Ubuntu also has a nice wiki–systemd for upstart Users How to switch between the two)

Other

Remember in the article "Cache update routines," I said, if you're going to be a good architect, first you have to thoroughly understand the computer architecture and many of the old-fashioned basic technologies. Because there will be a lot of things that can be borrowed and communicated. So, did you see something similar to distributed architecture in this article?

For example: from Sysvinit to upstart to SYSTEMD, like service governance? Are these service processes under the Linux system much like microservices in a distributed architecture? And the D-bus, isn't it like an ESB in SOA? and is the init system much like a control system? Even like a service orchestration (Orchestration) system?

There are also a lot of dependencies between services in a distributed system, so when we start a schema, if we can do it in parallel like SYSTEMD, is it just like a micro-service play?

Well, you will find that a lot of technical things are interlinked, but also each other's shadow, so, in fact, the technology is not much. The key is that we learn on the surface or see the essence.

LINUX PID 1 and Systemd