This is the case for Linux.

I have reposted it in my blog.How to build your own Linux system (LFS Quick Guide),In fact, LFS is not completed with patience and time. We finally have the opportunity to make another LFS tour with time. Using the vmwarevm, we performed step-by-step practices (haha, there are still a lot of tests not carried out), except for ReferenceHow to build your own Linux system (LFS Quick Guide)For more information, seeLFS, Mainly through which you can understand the specific meaning of each command, and there is no detailed explanation in the speed-up process.

It took about two days to complete the compilation of the entire LFS, but I still had some regrets:GrubThere is a problem with how to install it. You cannot automatically load the compiled LFS, but you can only manually input the command to load it. But in any case, I am very happy ,:-).

Looking back at the LFS production process, the obvious progress is the obvious proficiency in Linux commands, and the parameter understanding is more in-depth. I have a clearer understanding of the internal principles of Linux than before. Next we will talk about our current understanding of Linux (which is better than before, but may still be incorrect, and we have been improving ).

1. Composition of the entire Linux operating system

As an operating system, Linux has a lot of content, but in terms of external performance, I think there are two layers: the core layer and the application layer. The core layer mainly includes process management, storage management, and I/O management. I/O management is actually a variety of drivers. I also manage files to I/O management.

There are two main types of core layer files: kernel (image) files and driver files. The kernel file contains basic modules of the operating system such as process management. You can also compile some drivers, such as file drivers. All kernel files are loaded to the memory when the operating system is running. The driver file is a driver module for a specific Io device. It can be compiled into a separate kernel module or a kernel file, as a separate module, the kernel can automatically load or manually load the module when detecting the device ). In extreme cases, the kernel layer has only one large Kernel File but no other driver file, which is widely used in embedded systems because the hardware system of the embedded system is fixed, the drivers required by the hardware module are also fixed. Generally, all the drivers are loaded to the memory during system operation, improving the system running speed.

The core layer runs in the core State (ring0 ). The Application Layer runs on ring3 and cannot directly run core-state code. It can only communicate with the core-state through the CPU call gate and other mechanisms. Therefore, the core layer generally communicates with the application layer by calling the API of the library file function. The Application Layer Program only needs to call the API without worrying about the specific call details. The details are encapsulated in the API function implementation.

The Application Layer mainly involves various applications, including special init and shell. Init is the first application of the system. Shell is only a special application used to interpret and call other programs. They call library functions to complete their tasks, which is essentially different from other programs we write.

During the compilation process, LFS gradually generates tool chains (compilers, etc.) for the target system from the existing host operating system ), remove the dependency on the existing system and gradually adapt the system to the new system, which is similar to cross-compilation. LFS has been adjusting the tool chain most of the time, and then compiling various tools (including libraries). These tools are all at the application layer. At the end of LFS, the kernel is compiled. So it seems that there is no need to spend so much time compiling those applications. But without those applications, the kernel still cannot be compiled purely.

2. Linux boot

After preparing the kernel files and applications, it is important to "Activate" these files so that the hardware runs according to the new kernel file. This is the boot part of the system. This can be done through Lilo, grub, and other boot devices. The specific process is omitted. You can refer to the grub installation section behind how to build your own Linux system (LFS quick Manual.

3. Summary

The above analysis shows that Linux is composed of kernel files and applications, and then the boot program "activates" the kernel files. In the past, I was confused about how much space I used to install Linux, and what files are required and can be omitted. Through LFS, we finally understand that the simplest Linux system can run only one kernel file (the benefit of the macro kernel is:-), but it may not be able to complete any task, other files are for specific applications and can be customized based on the application.

But simply speaking, it is necessary to make full use of the powerful functions of Linux and the complexity of applications.

4. Linux reduction

Since a kernel file can meet our requirements by adding necessary library files and necessary programs, we can try to cut the existing huge Linux operating system into a very small system.

4.1 create a root file system

It is easier to set up a root file system: Step 1: format the host device that holds the root file system (format the device according to the required file system format); Step 2, copy the required file to the root file system.

The detailed structure of the Linux File System can be found online. The root file system we set up here does not need to contain too much content, but only provides the most basic directory structure and system commands. The configuration file should also be kept as concise and clear as possible.

The root file system must contain the following content:

• Contains/dev,/proc,/bin,/sbin,/etc,/lib,/usr,/root, and other directories.
• Contains a set of basic commands, such as LS and insmod.
• Supports the running database functions of the preceding commands and related database functions for system login authentication.
• This includes compiling the modules generated by the kernel.
• Required device files.
• Some necessary configuration files and service management scripts.

4.2 create a root file system

Step 1:Mount the hard disk of the target host to the host system. We assume that I will not install the hard disk on the second disk. Mount/dev/hdb1/rootfs

Step 2:Create required subdirectories in the root directory of the file system under the rootfs directory:

# Cd rootfs

# Mkdir Dev, Proc, bin, etc, Lib, USR, sbin, root, usr/bin, usr/sbin, usr/lib

Step 3:Copy the kernel file and the required commands to rootfs, such as the LS command.

Kernel files are generally/boot/kernel-2.6.x, we only need to copy all the files under/boot (Map Files, image files, initd files, etc.) to the target directory/boot.

For an application or command, we must first determine its location in the system (use the command whereis ls to find it in the/bin/ls directory ), then copy the command to the same directory structure under rootfs (CP/bin/ls/rootfs/bin/LS ). It is not enough to copy the command file. You must also copy the dynamic link library used by the command. How can I find the dynamic link libraries used by commands? It is easy to use the command LDD/bin/ls to do this, and the output is as follows:

Libtermcap. so.2 =>/lib/libtermcap. so.2 (0x4001f000)

Libacl. so.1 =>/lib/libacl. so.1 (0x40023000)

Libc. so.6 =>/lib/libc. so.6 (0x40029000)

Libattr. so.1 =>/lib/libattr. so.1 (0x40149000)

/Lib/ld-linux.so.2 =>/lib/ld-linux.so.2 (0x40000000)

The files with. So names are the dynamic link libraries used. Find these dynamic link libraries and copy them to the location corresponding to the original location under rootfs. Generally, they should be in lib or lib/i386 under the rootfs directory. (The commands used by the user are mostly in/bin and/sbin. In addition, some scripts use commands and tools that appear in/usr/bin and/usr/sbin. If you want to use these scripts, these command tools and the libraries they need should not be less ).

 

But don't be too busy. These file names are not what we actually want. They are just a symbolic link to the actual library file. The system uses a symbolic link, it is used to facilitate the upgrading of library files without directly affecting the applications using it (the library path used in the application is the connection file of the library, the name of the connection file will not be changed ). Therefore, if it is meaningless to copy a single symbolic link, you must copy the actual library file pointed to by the symbolic link to the workdir/lib directory.

The/lib directory also has an important component, the modules directory. It contains the modules generated by kernel compilation. The modules corresponding to different kernel versions are stored in the file named after the version number. Don't forget to copy our new kernel module to our rootfs/lib/modules.

A long time ago, these libraries under/lib were enough, but the current Linux system has many new requirements compared to the past, especially in terms of network security, A lot of verification techniques are added, so you must also have these additional libraries related to security verification. Although these libraries are not directly used in commands, they are indirectly called by the system's security mechanism. In most cases, the specific library used by the security mechanism is defined in the configuration file. The security framework checks the configuration file and selects the authentication library to call. I will not discuss the security framework much. If you are interested, you can check the usage of PAM and NSS. They are all security-related libraries that must be used during system login. Here, we leave it alone for November 21, to integrate all the libraries related to Pam under/lib/security/and all libnss * libraries related to NSS under/lib, are copied to/Security and/lib under rootfs/lib (without these verification libraries, you will not be able to log on, and the system will repeatedly prompt you to log on, but it does not give you a chance to enter your password ). The method is stupid, but it is easy.

In Linux, most library functions that are frequently used by applications are not statically compiled to connect to the application. Dynamic Links are usually used, for centralized storage management. In this way, if multiple programs use a shared library, the library file will only be transferred to the memory once and will reside in the memory for all applications to call. Therefore, the use of shared libraries can significantly save memory space and reduce the volume of execution files. Of course, there is no free lunch in the world. Although the shared library is much more flexible than the static library, path search is required, and the transfer time is more time-consuming.

Linked files are one of the features provided by UNIX operating systems. They can be divided into soft links and hard links. Soft links are also called symbolic links. The unique content of such files is the path containing the actual file system. Therefore, even if the actual file has been deleted or transferred, the symbolic link still exists. Of course, it is no longer valid. On the contrary, hard links share the index nodes with the linked files. Deleting the files corresponding to the hard links reduces the number of index nodes, without damaging the hard link files.

Step 4:It is very important to create a device file, but it is quite easy. Linux inherits a good UNIX feature and abstracts devices into special files for use and management. It wants to use system peripherals, such as hard disks, clocks, system terminals, and even memory, can be accessed through device files. Therefore, we need to create the device files corresponding to all the devices that the system may use. You can open the/dev/directory to see which device files you need cannot be generalized. Of course, there will certainly be a lot of files that will make you dazzled.

But do not be afraid, because most of them are useless. For the experiment system we want to build, there are only a handful of device files:

Console-console device;

Tty * -- virtual terminal managed by the console (we use ctrl-[1-7] to switch to this device );

Sda1 -- SCSI interface device;

Ram-memory virtual disk device;

Null -- null device (a very useful character device file, which is ignored for everything sent to this device. If the output result of any program is redirected to/dev/null, no output information is visible. Therefore, it is often used in scripts to remove information that should have been displayed on the screen );

Zero-zero device (read this device and only get blank content, so sometimes it is often used to obtain a high compression rate when a space is fully charged );

Initrd-this is a special character device that is used to send information about the switch operation level to the system kernel from the user space. It belongs to a virtual character device (for example, the init 1-6 command that changes the running level, are passed to the kernel through this device ).

There are probably so many necessary device files. It is clear that you need all the device files, and you can use the mknod command to create these required device files one by one. If this method is used, note that mknod requires parameters. You can run the LS ?? To check whether a device belongs to a block device or a character device, view the Master/Slave Device Number of the device. If you think this is too annoying, use the copy command to directly copy these files from the/dev/directory of the standard system, but use the parameter-R. Otherwise, what you copied is all the files of the device, that is, the entire content of the device, not just a device file. Copying the device content is like stuffing yourself into your clothes and pockets, and you will never get in.

 

In Linux, devices are divided into Block devices and character devices. Block devices can be randomly accessed (described by symbol B). Character devices can only be accessed in sequence (described by symbol C ). In addition, a driver may control multiple devices, so there are master device numbers and slave device numbers. The driver corresponding to the master device number, which is used to differentiate specific devices.

Step 5:The configuration files and scripts required for system running. We should simply copy the existing files of the standard system and then modify the files as needed. The following are the required configuration files and script files.

Our test system must meet the needs of multiple tasks and users. Therefore, we must first copy the passwd and group files in/etc, if the system uses shadow to protect the user's password, the shadow file must be copied along with it.

After the system starts, the INIT program is executed first, and the configuration files required by the system are not small. The inittab file specifies many basic attributes of system operation. Next, the INIT program executes the RC referenced in inittab. run the sysinit script to initialize the system boot later. The fstab configuration file is used, which contains the file system to be installed after the system is started and the directory to which it is installed; there are only two items for our system: one is to install/dev/Ram as the root file system to/, and the other is to install the proc file system to the/proc directory, this can be adjusted as needed.

After executing RC. sysinit in init, enter the corresponding/etc/rc? According to the running level defined in inittab ?. D/(1 is/etc/rc1.d/, of course, the most common is Level 3,/etc/rc3.d/) directory to execute the system service script starting with S. You can use man init to obtain relevant information. Now we need to copy all the initab, RC. sysinit, and RC. D files under/etc/to the corresponding location of the new root file system.

Assume that the system runs at 3 and only needs to start the network service? Network and S? All s scripts except local are deleted. (Of course, you can also change the default system startup process to execute your own initialization script. All you need to do is modify the Script Name in "sysinit: XXX" in the inittab ).

The above Initialization is executed. After the related services are started, the system will execute RC. local file. Here you can put some commands that you want to execute during startup. Here we will put "OK you are welcome !!!" As a greeting before you enter the system (because our system initialization does not have much operation, so the startup process is very fast, the virtual terminal device in the system (TTY ?) It is often too late for initialization to complete, so it may appear after the system prompts ("id xxx respawning too fast: disabled for 5 minutes"), in order to avoid this error, we also sleep the system for 20 seconds in local ).

In addition, login of the login program often uses Pam to verify the validity of the user. Therefore, the PAM Configuration file must be copied to the new system. Many systems will also use the NSS (Name Service switch, which has been mentioned earlier. This service helps the customer machine or application obtain network information from the local device or from somewhere on the network-from DNS or NIS. Functions such as getxbyy () often use this service. login is likely to be used when users log on, depending on your libc version ), therefore, the NSS configuration file/etc/nsswitch. conf also needs to be copied. For details about the script content, you can use man nsswitch. conf to understand.

You also need to copy the terminfo/termcap (new and old versions are different) file and set the term terminal environment variable to use this file.

Finally, do not forget to copy the module configuration file modules. conf, which contains information about the module.

Put those in the/root/directory. the hidden files used for Bash configuration at the beginning are also copied to the root directory of the new system. These are all bash environment parameters (if not, the relationship is not big, it is not convenient ).

Last: ldconfig ?? Cr rootfs (directory of the target file system) creates the library file path cache. From then on, when the commands on the new root file system use dynamic link libraries, you do not need to specify the library directory, because their paths are cached. (Ldconfig uses the dynamic library configuration file lD. So. conf, which specifies the library path to be used. For example, if your program uses the Kerberos library, you need to include the/usr/lkerberos/lib path in LD. So. conf ).

4.3 system guidance

If grub is used for system guidance, you can follow the instructions to teach you how to build your own Linux system (LFS quick manual) and finally perform GRUB boot.

Grub

Enter root (hd1, 1)

Enter setup (hd1)

Quit

Set the grub Startup Menu

Code:

Cat>/boot/GRUB/menu. lst <"EOF"

# Begin/boot/GRUB/menu. lst

# By default boot the first menu entry.

Default 0

# Allow 30 seconds before booting the default.

Timeout 30

# Use prettier colors.

Color green/black light-green/Black

# The first entry is for LFS.

Title LFS 6.3

Root (hd0, 1)

Kernel/boot/lfskernel-2.6.22.5 root =/dev/hda2

EOF

Note: The disk partition after root needs to be adjusted according to the actual situation.

Connect menu. lst to the/etc directory

Code:

Mkdir-V/etc/GRUB

Ln-SV/boot/GRUB/menu. LST/etc/GRUB

To restart the system, we should be able to boot from the new Kernel configuration, and it is the system we just used ,:-)

5. SummaryI explained the essential structure of the Linux system with my own points of view, and provided the ways and methods to cut down the minimum system from the existing system. Why do you need to explore? Because many books use a lot of space to talk about the characteristics, structure, source code, and so on of the Linux kernel, but they do not use a macro perspective to show the features of the Linux "system" macro kernel, the so-many C source code, which does not emphasize fainting, generates only one kernel file. After learning for a long time, the source code of the kernel has not figured out where the generated binary code is, in the end, how is it used by the system can be realized only through practice. When we intuitively understand the status and performance of the kernel, we can learn and modify the source code to be more targeted!

6. References

How to build your own Linux system (LFS Quick Guide)

Ande yitian taobaojian-building an experimental system

Teach you how to build your own Linux Version 2

 

Thank you for your selfless dedication to our beautiful online life!