Introduction to vmlinuz, system. Map, and initrd

In the network, many servers use Linux systems. To further improve the server performance, you may need to recompile the Linux kernel based on specific hardware and requirements. To compile the Linux kernel, you must follow the required steps. Several important files are involved in compiling the kernel. For example, for Redhat Linux, there are some Linux kernel-related files in the/boot directory and the files are executed in/boot.

 

People who have compiled the RedHat Linux kernel may be impressed with system. Map, vmlinuz, and initrd-2.4.7-10.img, because operations such as file creation are involved during kernel compilation. How are these files generated? What role does it play? This article introduces this topic.

 

I. vmlinuz

 

Vmlinuz is a bootable and compressed kernel. "VM" represents "Virtual Memory ". Linux supports Virtual Memory. Unlike earlier operating systems such as DOS, there is a limit of KB memory. Linux can use hard disk space as virtual memory, so it is named "VM ". Vmlinuz is an executable Linux kernel located in/boot/vmlinuz, which is generally a soft link than a soft link in the vmlinuz-2.4.7-10.

 

There are two ways to establish vmlinuz. One is to compile the kernel through "make zimage" to create, and then through: "CP/usr/src/linux-2.4/ARCH/i386/Linux/boot/zimage/boot/vmlinuz" generated. Zimage is suitable for small kernels. It exists for backward compatibility.

 

The second is when the kernel is compiled by the command make bzimage to create, and then through: "CP/usr/src/linux-2.4/ARCH/i386/Linux/boot/bzimage/boot/vmlinuz" generated. Bzimage is a compressed kernel image. It should be noted that bzimage is not compressed with Bzip2, and BZ in bzimage is easy to misunderstand. BZ indicates "Big zimage ". B In bzimage means "big. Both zimage (vmlinuz) and bzimage (vmlinuz) are compressed using gzip. They are not only a compressed file, but also embedded at the beginning of the two files
Decompress the gzip code. Therefore, you cannot use gunzip or gzip-DC to unpackage vmlinuz.

 

The kernel file contains a micro gzip used to extract the kernel and boot it. The difference between the two is that the old zimage decompress the kernel to the low-end memory (the first 640 K), and The bzimage decompress the kernel to the high-end memory (more than 1 MB ). If the kernel is small, zimage or bzimage can be used. The two methods guide the same system runtime. Bzimage is used for large kernels, and zimage cannot be used. Vmlinux is an uncompressed kernel and vmlinuz is a compressed file of vmlinux.

 

2. initrd-x.x.x.img

 

Initrd is short for "Initial ramdisk. Initrd is generally used to temporarily boot the hardware to the State where the actual kernel vmlinuz can take over and continue to boot. The initrd-2.4.7-10.img in the figure is mainly used to load the driver of file systems and SCSI devices such as ext3.

 

For example, if a SCSI hard disk is used, and the kernel vmlinuz does not have the SCSI hardware driver, the kernel cannot load the root file system before the SCSI module is loaded, however, the SCSI module is stored in/lib/modules of the root file system. To solve this problem, we can guide an initrd kernel that can read the actual kernel and correct the SCSI Boot question with initrd. Initrd-2.4.7-10.img is compressed with gzip files, initrd to load some modules and install file systems and other functions.

 

The initrd image file is created using mkinitrd. The mkinitrd utility can create an initrd image file. This command is proprietary to RedHat. Other Linux distributions may have corresponding commands. This is a very convenient utility. For more information, see help: run the following command under man mkinitrd to create an initrd image file.

 

3. system. Map

 

System. Map is a kernel symbol table of a specific kernel. It is the link to the system. Map of your currently running kernel.

 

How is the kernel symbol table created? System. MAP is generated by "nm vmlinux" and irrelevant symbols are filtered out.

 

For the example in this article, when compiling the kernel, system. MAP is created at/usr/src/linux-2.4/system. Map. As shown below:

 

Nm/boot/vmlinux-2.4.7-10> system. Map

 

The following lines are from/usr/src/linux-2.4/makefile:

 

NM vmlinux | grep-V '(Compiled) | (. o $) | ([auw]) | (.. ng $) | (lash [RL] Di) '| sort> system. map

 

Then copy to/boot:

 

CP/usr/src/Linux/system. MAP/boot/system. Map-2.4.7-10

 

Is part of the system. map file:

 

Some symbols such as variable names or function names are named during program design. Linux Kernel is a complex code block with many global symbols.

 

The Linux kernel uses a variable or function name instead of a symbolic name. For example, instead of using a symbol like size_t bytesread, the variable is referenced like c0343f20.

 

For people who use computers, they prefer names like size_t bytesread rather than c0343f20. The kernel is mainly written in C, so the compiler/connector allows us to use a symbolic name for encoding and an address for Kernel running.

 

However, in some cases, we need to know the address of the symbol or the symbol corresponding to the address. This is done by the symbol table, which is a list of all symbols along with their addresses. It is a kernel symbol table. It can be seen that the variable name checkcputype is at the kernel address c01000a5.

 

The Linux symbol table uses two files:

 

/Proc/ksyms

 

System. Map

 

/Proc/ksyms is a "proc file" created during kernel boot. In fact, it is not really a file. It is just a representation of kernel data, but it gives people the illusion of a disk file, which can be seen from its file size being 0. However, system. map exists in the actual file system.

 

When you compile a new kernel, the address of each symbol name changes. Your old system. MAP has incorrect symbol information. Each time the kernel is compiled, a new system. MAP is generated. You should replace the old system. map with the new system. Map.

 

Although the kernel itself does not actually use system. Map, other programs such as klogd, lsof, and PS need a correct system. Map. If you use incorrect or do not have system. MAP, the output of klogd will be unreliable, which will cause difficulties in troubleshooting the program. Without system. map, you may face annoying prompts.

 

In addition, a few drivers need system. Map to parse the symbols. If system. map is not created for the specific kernel you are running, they cannot work normally.

 

For Linux kernel log daemon klogd to perform name-Address Resolution, klogd needs to use system. Map. System. map should be placed where the software using it can find it. Run: Man klogd. If system. map is not given to klogd as a variable, it searches for system. map in the following order:

 

/Boot/system. Map

 

/System. Map

 

/Usr/src/Linux/system. Map

 

System. MAP also has version information. klogd can intelligently find the correct map file.

 

Author: IHH

 

Introduction to vmlinuz, initrd, and system. MAP:

Vmlinuz: This is the compiled compressed Kernel File.

Introduction to initrd:

Initrd is a temporary root file system used in Linux during system boot. It is used to support the two-phase boot process.

To put it bluntly, initrd is a virtual ramdisk with a root file system, which contains the root directory '/' and other directories, such as bin, Dev, Proc, sbin, sys and other directories required for Linux Startup, as well as the required executable commands added to the bin directory.

The Linux kernel of the PC or server uses this initrd to mount the real root file system, and then unmounts the initrd from the memory. In this case, initrd is actually a transitional application. In many simple embedded Linux systems, the initrd is not uninstalled, but directly used as the root file system. Before that, you need to set the required program, command and other files are installed in this file system. In fact, most embedded systems also have their own disks. Therefore, initrd is just a transitional use of most embedded systems, just like Linux.

Initrd boot process: 'Phase 2 Bootstrap '. Grub decompress the kernel and copy it To the memory. Then the kernel takes over the CPU and starts execution. Then the kernel calls Init () function. Note that this init function is not a later INIT process !!! Then the kernel calls the initrd_load () function to load the initrd root file system in the memory. The initrd_load () function calls other functions to allocate space for the RAM disk and calculate CRC. Decompress the RAM disk and load it To the memory. Now, the memory has an initrd image.

Then the kernel will call the mount_root () function to create a true root partition file system, and then call the sys_mount () function to load the real root file system, then chdir to the real root file system.

Finally, the init function calls the run_init_process function and uses execve to start the INIT process to enter the init running process.

System. Map introduction:

The kernel symbol ing table, as its name implies, is a list that associates the symbols in the kernel (that is, the functions in the kernel) with its address. Is a list of all symbols and their corresponding addresses. In this way, function symbols can be used directly for the convenience of programming, rather than the address of the function.

When you compile a new kernel, the symbolic information in the original system. MAP is incorrect. Each time the kernel is compiled, a new system. Map File is generated, and the original file must be replaced with this file.

 

Functions of system. Map Files

 

Information about the system. Map File seems to be lacking. In fact, it is not mysterious at all, and it is not as important as it looks. However, the lack of necessary documentation makes it mysterious. It is like an earlobe, which we all have, but we don't know what to do. This page is used to illustrate this problem.

Note that I am not 100% correct. For example, a system may not support the/proc file system, but most systems certainly do. Here I assume that you are "with the big stream" and have a typical configuration system.

Some explanations about Kernel errors (Oops) come from Alessandro Rubini's book "Linux device drivers", where I learned most about Kernel programming.
What is a symbol (symbols )?

In programming, a symbol is a program creation block: it is a variable name or function name. Just like your own program, it should not be surprising that the kernel has various symbols. Of course, the difference is that the kernel is a very complex code block and contains many and many global symbols.

What is the kernel symbol table?

The kernel does not use a symbolic name. It uses variables or functions through the address (pointer) of variables or functions, rather than using size_t bytesread. The kernel prefers to use (for example) c0343f20 to reference this variable.

On the other hand, people do not like names like c0343f20. We like to use a representation like size_t bytesread. Generally, this does not cause any problems. The kernel is mainly written in C language. Therefore, during programming, the compiler/Connection Program allows us to use the symbol name and make the kernel express the address at runtime. So everyone is satisfied.

However, there is a situation where we need to know the address of a symbol (or the symbol corresponding to an address ). This is done through the symbol table, which is similar to the situation where GDB can give the function name (or the address of a function name) from an address. A symbol table is a list of all symbols and their corresponding addresses. Here is an example of a symbol table:

C03441a0 B dmi_broken
C03441a4 B is_sony_vaio_laptop
C03441c0 B dmi_ident
C0344200 B pci_bios_present
C0344204 B pirq_table
C0344208 B pirq_router
C034420c B pirq_router_dev
C0344220 B ascii_buffer
C0344224 B ascii_buf_bytes

You can see that the variable named dmi_broken is located at the kernel address c03441a0.

What is a system. map file?

Two files are used as symbol tables:

1./proc/ksyms
2. system. Map

Here, you can now know what the system. Map File is.

Each time you compile a new kernel, the address of each symbol name changes.

/Proc/ksyms is a "proc file" created at kernel startup. In fact, it is not a real file; it is just a simple representation of the kernel data, showing like a disk file. If you don't believe me, try to find the file size of/proc/ksyms. Therefore, for the currently running kernel, it is always correct ..

However, system. Map is a real file on the file system. When you compile a new kernel, the symbolic information in your original system. MAP is incorrect. Each time the kernel is compiled, a new system. Map File is generated, and the original file must be replaced with this file.

What is a Oops?

What are the most common errors in self-developed programs? It is a segment error (segfault), signal 11.

What are the most common bugs in Linux kernel? It is also a segment error. In addition, as you might imagine, the problem of incorrect segments is very complicated and serious. When the kernel references an Invalid Pointer, it is not called a segment error -- it is called "oops ". An oops indicates that there is a bug in the kernel, and it should always be reported and corrected.

Note that an oops is not the same as a segment error. Your program cannot be recovered from a segment error. When an oops occurs, it does not mean that the kernel is definitely unstable. The Linux kernel is very robust. One oops may only kill the current process and keep the remaining kernel in a good and stable State.

An oops is not a kernel endless loop (panic ). After the kernel calls the panic () function, the kernel cannot continue to run. At this time, the system is paused and must be restarted. If the key part of the system is damaged, an oops may also cause the kernel to enter an endless loop (panic ). For example, the oops in the device driver will hardly lead to an endless loop of the system.

When an oops occurs, the system displays information about debugging problems, such as the content in all CPU registers and the location of the page Descriptor Table, in particular, the EIP (command pointer) content is printed as follows:

EIP: 0010: [<00000000>]
Call trace: []

What is the relationship between an oops and the system. map file?

I think you will also think that the EIP and call trace do not provide much information, but it is important that this information is not enough for Kernel developers. Because a symbol does not have a fixed address, c010b860 can point to any place.

To help us use oops vague output, Linux uses a background program called klogd (kernel log Background Program). klogd intercepts the kernel Oops and uses syslogd to record it, and convert some information like c010b860 into information that we can recognize and use. In other words, klogd is a kernel message recorder (logger) that can be used for name-to-address resolution. Once klogd starts to convert the kernel message, it uses the recorder at hand to record the message of the entire system, usually using the syslogd recorder.

To perform name-Address Resolution, the klogd must use the system. Map File. I think you now know the relationship between Oops and system. Map.

For more information, see klogd.

* For static conversions, The system. map file is used.
* Dynamic conversion. This method is used to load modules without using system. Map. Therefore, it has nothing to do with this discussion, but I still briefly describe it.

Dynamic conversion of klogd

Suppose you have loaded a kernel module that generates oops. Therefore, a oops message is generated, and klogd intercepts it and finds that the oops occurs at d00cf810. Because the address belongs to the dynamic loading module, there is no corresponding entry in the system. Map File. Klogd will be searched for and will not be obtained, so it is determined that a loadable module produces oops. At this time, klogd will query the kernel for the symbols output by the loaded module. Even if the compiler of this module does not output its symbols, klogd will at least know which module generates oops, which is better than a oops.

Other software will use system. Map, which will be described later.

Where should system. Map be located?

System. map should be located where the software that uses it can find, that is, where klogd will look for it. When the system is started, if the location of system. map is not given by klogd in the form of a parameter, klogd searches for system. Map in three places. In sequence:

1./boot/system. Map
2./system. Map
3./usr/src/Linux/system. Map

System. MAP also contains version information, and klogd can intelligently search for correct map files. For example, assume that you are running kernel 2.4.18 and the corresponding map file is located in/boot/system. Map. Now you compile a new kernel 2.5.1 In the/usr/src/Linux directory. During compilation, the file/usr/src/Linux/system. map will be created. When you start the new kernel, klogd will first query/boot/system. map to check whether it is the correct map file to start the kernel, and then query/usr/src/Linux/system. map, make sure the file is the correct map file to start the kernel and start to read the symbol information.

Notes:

* In a version of the 2.5.x series kernel, the Linux kernel starts to untar into Linux-version, not just Linux (please raise your hand to vote-how many people have been waiting for this ?). I don't know if klogd has been changed to search in/usr/src/Linux-version/system. Map. Todo: view the klogd source code.
* This is not fully described in the online manual. Please refer:

# Strace-F/sbin/klogd | grep 'System. map'
31208 open ("/boot/system. Map-2.4.18", o_rdonly | o_largefile) = 2

Obviously, not only does klogd search for the correct map file in the three search directories, but klogd also knows that the search name is "system. map "added"-kernel version ", such as system. map-2.4.18. this is an undisclosed feature of klogd.

Some drivers will use system. Map to parse the symbols (because they are connected to the kernel header rather than the glibc library). Without the system. map file, they will not work correctly. This is the same as a module that does not get loaded due to kernel version mismatch. Module loading is related to the kernel version, but not to the compiled kernel that changes even the symbol table of the kernel of the same version.

Who else uses system. Map?

Do not think that the system. Map File is only useful for the kernel oops. Although the kernel itself does not actually use system. Map, other programs, such as klogd, lsof,

Satan # strace lsof 2> & 1 1>/dev/null | grep System
Readlink ("/proc/22711/FD/4", "/boot/system. Map-2.4.18", 4095) = 23

PS,

Satan # strace PS 2> & 1 1>/dev/null | grep System
Open ("/boot/system. Map-2.4.18", o_rdonly | o_nonblock | o_noctty) = 6

And many other software, such as dosemu, must have a correct system. Map File.

If I don't have a good system. Map, what will happen?

Suppose you have multiple kernels on the same machine. Each kernel requires an independent system. Map File! If the started kernel does not have the corresponding system. map file, you will see this information regularly:

System. map does not match actual kernel (system. map does not match the actual kernel)

It is not a fatal error, but it will be annoying whenever you execute PS ax. Some software, such as dosemu, may not work normally. Finally, when a kernel oops occurs, the output of klogd or ksymoops may be unreliable.

How can I remedy the above situation?

You can place all your system. Map Files in the directory/boot and rename them using the kernel version number. Assume that you have the following kernels:

*/Boot/vmlinuz-2.2.14
*/Boot/vmlinuz-2.2.13

You only need to rename the map file for each kernel version and put it under/boot, for example:

/Boot/system. Map-2.2.14
/Boot/system. Map-2.2.13

What if you have two copies of the same kernel? For example:

*/Boot/vmlinuz-2.2.14
*/Boot/vmlinuz-2.2.14.nosound

The best solution is that all software can find the following files:

/Boot/system. Map-2.2.14
/Boot/system. map-2.2.14.nosound

But to be honest, I don't know if this is the best case. I have seen the search for "system. Map-kernelversion", but what about the search for "system. Map-kernelversion. othertext? I don't know. What I can do at this point is to take advantage of the fact that/usr/src/Linux is the search path of the Standard map file, so your map file will be placed in:

*/Boot/system. Map-2.2.14
*/Usr/src/Linux/system. Map (for nosound Version)

You can also use a symbolic connection:

System. Map-2.2.14
System. map-2.2.14.sound
System. Map-> system. map-2.2.14.sound