Introduction to Linux kernel engineering-process: How ELF files are executed

Introduction to Linux kernel engineering-process: How ELF files are executed

1. Process Execution

We all know that windows processes cannot be opened by double-clicking in linux, and vice versa. However, programs written in C or golang can be compiled in linux and compiled in windows. Of course, if you call an OS-specific system call, it cannot be executed. Specifically, the compilation fails. We will discuss the use of standard C-library functions for system calls that do not call the operating system. Standard C library functions are also called by the system in the background, but the conversion is completed in different C library implementations of different operating systems.

Why can't I execute system calls related to operating systems that have not been called? Some people may say that the elf format is compiled in linux and the exe format is compiled in windows. This binary format is supported in the kernel, because when the system that calls the loaded executable program is called, the kernel must know the format of the executable file it loads, in order to identify information (such as 32-bit or 64-bit architecture, data is stored on the big end or small end, where the symbol table is stored, and where the program entry is ). This recognition process for storage formats is a bit like a file system. The kernel must have a clear understanding of the organization formats of different file systems in order to correctly index and modify the data in it. A mechanism that enables the kernel to recognize specific formats is called a driver. Similarly, the NIC needs different drivers to send data, the file system also needs different drivers to read data, and binary files to execute code. Windows does not have the elf driver and Linux kernel does not have the exe driver. Thanks to the open-source features of linux, you can completely write an exe driver of the kernel so that the exe program can be executed directly in linux.

But is it that simple? None. The basic library used for compiling code in windows can only be run on windows. This basic library specifies the order in which function parameters should be pushed into the stack and system calls when a process is called. (The way in which linux and windows fall into system calls is different ). That is to say, if the basic library design is good enough, it can be compatible between the two operating systems (the symbol table is the same). If the processing is different, the basic library can also be processed. It cannot be forgotten that a program depends on many dynamic libraries, which are also related to the system. Some codes may even bypass the operating system calls of the basic library. There are also processes that need to be loaded, and the loader must be able to recognize the formats of other platforms. This is also the basis for wine to work. In linux, the wine program converts all the differences in the underlying layer to make the exe binary program compatible in linux. Therefore, we can see that if the kernel system calls enough to be consistent with windows, and then implement some compatible basic libraries, linux can also be highly compatible with the exe program. However, at present, most of the wine conversions are completed in the user space, which inevitably results in loss of efficiency. Even if the kernel state is completed, the conversion cost will not be small due to different logic designs.

Such a situation as Linux and windows is called binary incompatibility, that is, ABI is different. ABI specifies the underlying call and parameter transfer, and the specific format of the binary file layout. If a kernel supports one ABI, a single compilation can be performed everywhere No matter what operating system it is in.

2. elf File Format

The disk storage structure usually has a header, and the same is true for elf. The elf header, segment header, and section header have three parts. One binary file has only one elf header, multiple segment headers, and multiple section headers. A segment logically contains multiple sections.

What are segment and section concepts? Common segment types include PT_LOAD, PT_DYNAMIC, PT_INTERP, PT_NOTE, and PT_PHDR. Let's think about the necessary conditions for process execution.

The layout of binary files on disks is not in memory. Therefore, a ing and a program are required to implement the ing, in addition, binary files on linux usually need to load shared libraries (for example, libc is almost essential). This is not done in the kernel, because the kernel does not know the concept of a library. In the kernel view, all programs are executable code segments, and some code segments can be mapped and relocated. The program that executes External library search and loading is called the loader, the elf format is ld-linux.so, and the. out format is ld. so. Since the loader may have multiple implementations or versions, you must specify which loader to use in each binary file, specify which loader is implemented by segment. This segment is of the PT_INTERP type. Since golang generally uses static links, you will find that only golang elf formats do not have PT_INTERP segment.

A program generally has a. dynamic segment, which is placed in a segment of the PT_DYNAMIC type. Why do we need to create one separately? This segment, including this segment, is also used to serve dynamic loading. We can use the ldd command to read a binary dependent database. This dependency is written here. That is to say, the name of the library required for the current elf execution is recorded here. As for where to find these libraries, is the ld-linux.so thing.

PT_NOTE is used to record some auxiliary information of the program. What auxiliary information does the program have? For example, the program type, the owner of the program, and the description of the program. This information is not involved in the execution of the program. It only serves as a description.

PT_LOAD is the place where real programs are stored. This constitutes the main body of the program.

Section is the format of the specific organization data in the segment. Each section has a name, which is given by the compiler. You can also customize the name, which starts with a decimal point. For example, text. data. The connector and the loader recognize some segments together, so we can discuss the operation. For example, if the loader sees the. text Segment, it knows that it is a code segment, and the creator of this. text Segment is a linker.

3. Process Loader

As mentioned above, the loader of the elf file is a ld-linux.so, And the loader of the. out file is ld. so. However, the configuration paths used for loading are the same:/etc/ld. so. conf file. This file is generally all files under the include ld. so. conf. d directory, so it is best to add a library path to create a file under the directory. Because the file name is a good description of the usage of this library. Run ldconfig after adding, because the actual ld-linux.so is not one by one to search for paths, it will be very slow. Instead, it is directly queried from the cache. This cache file is ld. so. cache, which contains the path of each database. It is calculated using the ldconfig program using the ld. so. conf file. Therefore, you need to execute this command every time you modify the database configuration.

You can also make an experiment where all linux processes can run effectively because the ld-linux.so is located under the same directory/lib. If this file is moved or renamed, almost all programs cannot be executed (a program compiled with golang can still be executed without being loaded using a ld-linux.so ). At this point, if you want to resume the execution, you have to copy the ld-linux.so to the/lib/directory, but you will find that the mv command cannot be executed. However, builtin commands such as cd are acceptable. The way to recover is to display the use of./ld-linux.so mv a B, and of course add the necessary parameters. Here, we just want to demonstrate that if all gcc compilation processes are to be executed, the loader program is actually executed first, and then the loader calls the actual process for execution. It seems that the python program cannot be executed directly, but after the shell settings, the python program can be automatically found for execution.

In addition, you may have multiple versions in the same database. This does not conflict, as long as you add all the paths.