program linking and loading and implementation of dynamic link under Linux

http://www.ibm.com/developerworks/cn/linux/l-dynlink/

program linking and loading and implementation of dynamic link under Linux

There are many ways to link and load a program, and today the most popular are dynamic-link, dynamic-loading methods. This paper first reviews the basic working principle of linker and loader and the development history of this technology, and then analyzes the implementation of dynamic link under Linux system through practical examples. Understanding the implementation details of the underlying key technologies is undoubtedly necessary for system analysis and designers, especially when we are confronted with real-time systems that require precise measurement and mastery of the time and space efficiency of program execution.

The basics of how the linker and the loader work

A program that wants to run in memory, in addition to being compiled, is linked and mounted in both steps. From the programmer's point of view, the advantage of introducing these two steps is that the meaningful function name and variable name of printf and errno can be used directly in the program without explicitly specifying the address of printf and errno in the standard C library. Of course, compilers and assemblers have made a revolutionary contribution to freeing programmers from the nightmare of early, direct use of address programming. The advent of compilers and assemblers makes it possible for programmers to use more meaningful symbols in their programs to name functions and variables, which greatly improves the correctness and readability of the program. But with the popularity of the C language, which supports the compilation of programming languages, a complete program is often divided into several separate parts for parallel development, and each module communicates through a function interface or global variable. This leads to the problem that the compiler can only perform the conversion of symbolic names to addresses within a module, and who does the symbolic parsing between different modules? For example, the user program that invokes printf and the standard C library that implements printf are obviously two different modules. In fact, this work is done by the linker.

In order to solve the problem of link between different modules, the linker has two main tasks to do-symbolic parsing and relocation:

Symbolic parsing: When a module uses a function or global variable that is not defined in the module, the compiler-generated symbol table marks all such functions or global variables, and the responsibility of the linker is to go to other modules to find their definition, if no suitable definition is found or the appropriate definition is found, Symbolic parsing does not complete properly.

Relocation: The compiler typically uses a zero-based relative address when compiling a build target file. However, during the linking process, the linker will start with a specified address, assembling them one after the other based on the order of the input destination files. In addition to the target file assembly, in the relocation process has completed two tasks: first, the final symbol table is generated, and the second is to modify some locations in the code snippet, all the locations that need to be modified are indicated by the compiler-generated relocation table.

For a simple example, the concept above is straightforward for the reader. If we have a program consisting of two parts, the main function in M.C calls the function sum implemented in F.C:

/* M.C */int i = 1;int j = 2;extern int sum (); void Main () {        int s;        s = SUM (i, j);/* F.C */int sum (int i, int j) {        return i + j;}

In Linux, the two-segment source program is compiled into a target file using gcc:

$ gcc-c m.c$ gcc-c F.C

Let's take a look at the symbol table and the relocation table generated during the compilation by Objdump:

$ objdump-x M.O ... SYMBOL TABLE: ... 00000000 go. Data  00000004 i00000004 go. Data  00000004 j00000000 gF. Text  00000021 main00000000         *und*< c4/>00000000 sumrelocation RECORDS for [. Text]:offset   TYPE              VALUE00000007 r_386_32 j0000000d          r_386_32          i00000013 r_386_pc32        sum

First, we notice that the sum in the symbol table is marked as und (undefined), which is not defined in M.O, so it is possible to find the definition of function sum in other modules through the symbolic parsing function of the LD (Linux linker). In addition, there are three records in the relocation table that indicate the three locations in the code snippet that need to be modified during the relocation process, at 7, D, and 13, respectively. Here's a more intuitive way to take a look at these three locations:

$ objdump-dx m.odisassembly of section. text:00000000 <main>:   0:   55push   %ebp   1:   E5mov    %ESP,%EBP   3:   EC 04sub    $0x4,%esp   6:   A1 00mov    0x0,%eax7:r_386_32     J   B:   50push   %eax   C:   A1 xx 00mov    0x0,%eaxd:r_386_32     i  :   50push   %eax  :   E8 FC FF FF ffcall   <main+0x13>13:r_386_pc32  sum  :   c4 08add    $0X8,%ESP  1a:   c0mov    %eax,%eax  1c:   Fcmov    %EAX,0XFFFFFFFC (%EBP)  1f:   c9leave  :   c3ret

Take sum as an example, the call to function sum is implemented by the Invoke command, using the IP relative addressing method. As you can see, in the target file M.O, the call instruction is located at the location of the zero-based relative address 12, where the E8 is the operation code of call, and 4 bytes starting from 13 hold the offset of the next instruction add of sum relative to call. Obviously, the offset before the link is not known, so in the future to modify the code 13 here. Now why is this place stored in 0XFFFFFFFC (note that Intel's CPU uses little endian's addressing method)? This is probably for security reasons, because 0XFFFFFFFC is 4 of the complement representation (readers can use p/x-4 view in gdb), and the call instruction itself occupies 5 bytes, so in any case the call instruction in the offset can not be-4. Let's look at what this offset in the call command has been modified to after the relocation:

$ gcc m.o f.o$ objdump-dj. Text A.out | lessdisassembly of section. Text: ... 080482C4 <main> 80482D6:       E8 0d 00call   80482e8 <sum>80482db:       c4 08add    $0x8,%esp ... 080482e8 <sum>

You can see that after repositioning, the offset in the call instruction is modified to 0x0000000d, and a simple calculation tells us: 0x080482e8-0x80482db=0xd. In this way, the final executable program is generated after the relocation.

After the executable program is generated, the next step is to load it into memory run. Linux compiler (C language) is CC1, assembler is as, linker is LD, but there is no actual program corresponding to the concept of loader. In fact, the ability to load executable programs into memory runs is implemented by the system call of Execve (2). Simply put, the loading of a program consists of the following steps:

• Read the header information of the executable file to determine the size of its file format and address space;
• Dividing the address space in the form of segments;
• The executable program is read into the various segments in the address space, and the mapping relationship between the actual and virtual addresses is established.
• Will BBS Chingqing 0;
• Create a stack segment;
• Establish program parameters, environment variables and other procedures to run the necessary information;
• Start the run.

Back to top of page

History of linking and loading technologies

A program to load the memory run must be compiled, linked and loaded in the three stages, although it is such a familiar concept, in the process of operating system development has undergone several major changes. In simple terms, it can be divided into the following three stages:

1. static linking, static loading

This approach was first adopted and was characterized by its simplicity and the need for no additional support from the operating system. Programming languages like C have been compiled from a very early time, and different modules of the program can be developed in parallel, and then compiled independently into the corresponding target files. After all the target files have been obtained, statically linked, static loading is done by linking all the target files into an executable image and then loading the executable image into memory all at once when the process is created. As a simple example, suppose we developed two programs Prog1 and PROG2,PROG1 are composed of main1.c, UTILITIES.C, and errhdl1.c, respectively, corresponding to the main frame of the program, some common auxiliary functions (which are equivalent to the library) and the error handling part , the three parts of the code are compiled with their respective target files main1.o, UTILITIES.O, and ERRHDL1.O respectively. Similarly, PROG2 is composed of main2.c, UTILITIES.C, and errhdl2.c, and three parts of the code are compiled to get their corresponding target files main2.o, UTILITIES.O and ERRHDL2.O respectively. It is important to note that Prog1 and Prog2 use the same common auxiliary function UTILITIES.O. When we use statically linked, statically loaded methods, the memory and hard disk usage 1 are shown when running both programs:

As you can see, first of all, the use of the hard disk, although the two programs share the use of utilities, but this is not on the hard disk to save the executable image of the embodiment. Instead, UTILITIES.O is linked into the executable image of every program that uses it. This is also the case with memory, where the operating system loads the executable image of the program into memory at the time the process is created, before the process can start running. As mentioned earlier, using this approach makes the implementation of the operating system very simple, but its drawbacks are obvious. First of all, since two programs use the same UTILITIES.O, it should be sufficient to save a copy of UTILITIES.O on the hard disk, and if the program does not have any errors during operation, then the code of the error handling section should not be loaded into memory. So static link, static loading method not only wasted hard disk space, but also wasted memory space. Because the memory resources of the early system are very valuable, the latter is more deadly for earlier systems.

2. Static linking, dynamic loading

Since static linking and static loading can do more harm than benefit, let's take a look at how people solve this problem. Because of the memory tension problem in the early system appears more prominent, so people first think of to solve the problem of low memory use efficiency, and then put forward the idea of dynamic loading. The idea is very simple, that a function is loaded into memory only when it is called. All modules are stored on disk in a relocatable mount format. First, the main program is loaded into memory and starts running. When a module needs to invoke a function in another module, the first thing to check is whether the module containing the called function is loaded into memory. If the module is not already loaded into memory, the module is loaded into memory by the link loader responsible for relocation, and the Address table of the program is updated to reflect this change. After that, the control is transferred to the function that is called in the newly installed module.

The advantage of dynamic loading is that it never loads an unused module. If there is a lot of code like error-handling functions in the program to handle small probability events, using this method is undoubtedly fruitful. In this case, the part that is actually used (and therefore loaded into memory) may actually be very small, even though the entire program may be large.

Still with the above mentioned two programs Prog1 and PROG2 for example, if Prog1 run the error and PROG2 in the process of running without any errors. When we use statically linked, dynamic loading methods, the memory and hard disk usage 2 are shown when running both programs:

Figure 2 Using static links, dynamic mount methods, memory and hard disk usage while running PROG1 and PROG2

It can be seen that when there are a lot of modules such as error handling in the program, the use of static link, dynamic loading method in the memory efficiency of the use of a considerable advantage. So far, people have moved to the ideal goal, but the problem has not been fully solved-the use of memory efficiency increased, hard disk?

3. Dynamic linking, dynamic loading

Using static link, dynamic loading method seems to be only the hard disk space usage is not very efficient problem, in fact, the problem of inefficient memory usage is still not fully resolved. In Figure 2, since two programs use the same UTILITIES.O, the ideal scenario is that only one copy of the UTILITIES.O is stored in the system, whether in memory or on the hard disk, so people think of dynamic linking.

When using dynamic linking, you need to hit a stub in the program image where every call to the library function is made. A stub is a small piece of code that locates the appropriate library of mounted memory, and if the required library is not already in memory, the stub will indicate how to load the library in memory for that function.

When executing to such a stub, first check that the required function is already in memory. If the desired function is not already in memory, it needs to be mounted first. Anyway, the stub will eventually be replaced by the address of the called function. This allows the same library function to run directly the next time the same code snippet is run, eliminating the overhead of dynamic linking. As a result, all processes that use the same library are running with the same copy of the library.

Let's take a look at the above mentioned two programs Prog1 and PROG2 in the dynamic link, dynamic loading method, while running both programs memory and hard disk usage (see Figure 3). It is still assumed that an error occurred during the PROG1 operation and that no errors occurred during the Prog2 operation.

Figure 3 Using dynamic link, dynamic mount method, memory and hard disk usage while running PROG1 and PROG2

Figure, there is only one copy of UTILITIES.O in both the hard disk and in memory. In memory, two processes are shared by mapping addresses to the same UTILITIES.O. This feature of dynamic linking is critical for library upgrades (such as bug fixes). When a library is upgraded to a new version, all programs that use the library will automatically use the new version. If you do not use dynamic link technology, all of these programs need to be re-linked to access the new version of the library. In order to prevent the program from accidentally using some incompatible new versions of the library, the program and library usually contain their own version information. There may be several versions of a library in memory, but each program can decide which one it should use by version information. If you make minor changes to the library, the version number of the library remains the same, and if the changes are large, the version number is incremented accordingly. Therefore, if the new version of the library contains changes that are incompatible with earlier versions, only those programs that are compiled with the new version of the library will be affected, and the program that has been linked before the new library installation will continue to use the previous library. Such a system is called a shared library system.

Back to top of page

Implementation of dynamic link under Linux

Most of the libraries we use to program Linux (like LIBC, QT, etc.) now provide both dynamic-link libraries and two versions of the static-link library, whereas GCC uses the dynamic-link library in the system when it compiles links without the-static option. For the principle of dynamic link library Most of the books are just a general introduction, the author will be in the actual system by the disassembly of the code to the reader to show the implementation of this technology under Linux.

Here is a simple C program hello.c:

#include <stdio.h>int main () {    printf ("Hello, world\n");    return 0;}

Under Linux we can use GCC to compile it into an executable file a.out:

$ gcc hello.c

The program used printf, which is located in the standard C library, if you compile with gcc without-static, the default is to use Libc.so, which is the dynamic link standard C library. In GDB, you can see that the compiled printf corresponds to the following code:

$ gdb-q a.out (GDB) disassemble printfdump of assembler code for function printf:0x8048310 <printf>:     jmp    *0x 80495a40x8048316 <printf+6>:   push   $0x180x804831b <printf+11>:  jmp    0x80482d0 <_init +48>

This is usually the process of piling on books and the previous mention, obviously this is not really a printf function. The purpose of this stub code is to go to libc.so to find the real printf.

(gdb) x/w 0x80495a40x80495a4 <_global_offset_table_+24>:   0x08048316

You can see that the 0x08048316 stored at 0X80495A4 is exactly the address of the PUSHL $0x18, so the first jmp instruction does not play any role, it acts like a null operation instruction NOP. Of course this was the first time we called printf, and its real purpose was to be reflected in the next call to printf. The purpose of the second JMP directive is the PLT, which is procedure linkage table, whose contents can be viewed through the objdump command, which is of interest to the following two commands that have an impact on the control flow of the program:

$ OBJDUMP-DX a.out ... 080482d0 >.plt>: 80482d0:       ff       PUSHL  0x8049590 80482D6:       FF 94,       jmp    * 0x8049594 ...

The first push instruction presses the Got-related table entry address in the Global offset table into the stack, and then jmp to the address 0x4000a960 in the memory unit 0x8049594. One thing to note here is that you must start the program a.out before you view the got, otherwise the result from the X command in GDB is not correct at 0x8049594.

(GDB) b mainbreakpoint 1 at 0x8048406 (gdb) rstarting Program:a.outbreakpoint 1, 0x08048406 in Main () (GDB) x/w 0x8049594 0x8049594 <_global_offset_table_+8>: 0x4000a960 (GDB) disassemble 0x4000a960dump of assembler code for function _d l_runtime_resolve:0x4000a960 <_dl_runtime_resolve>: Pushl%eax0x4000a961 <_dl_runtime_resolve+1>: P USHL%ecx0x4000a962 <_dl_runtime_resolve+2>: Pushl%edx0x4000a963 <_dl_runtime_resolve+3>: movl 0x Ten (%esp,1),%edx0x4000a967 <_dl_runtime_resolve+7>: Movl 0xc (%esp,1),%eax0x4000a96b <_dl_runtime_resolve+ 11&gt: Call 0x4000a740 <fixup>0x4000a970 <_dl_runtime_resolve+16>: popl%edx0x4000a971 <_dl_runt Ime_resolve+17&gt: Popl%ecx0x4000a972 <_dl_runtime_resolve+18>: Xchgl%eax, (%esp,1) 0x4000a975 <_dl_run TIME_RESOLVE+21&GT: Ret $0x80x4000a978 <_dl_runtime_resolve+24>: nop0x4000a979 <_dl_runtime_resolve+25& gt;: Leal 0x0 (%esi,1),%esiend of assembler dump. 

The contents of the stack after the previous three push instructions are executed are as follows:

The following will 0x18 deposit edx,0x8049590 deposit eax, with these two parameters, fixup can find the address of printf in libc.so. When fixup returns, the address is saved in EAX. After the XCHG instruction executes, the contents of the stack are as follows:

The best thing to do is to count the usage of the next RET instruction, where RET is actually used as a call. After the RET $0x8 control is transferred to the real printf function, and the 0x18 and 0x8049584 on the stack are cleared, the stack becomes the following:

And that's exactly what we're looking for. It should be said that the use of RET here and the Linux kernel after the boot through the Iret instructions from the kernel state switch to the user state of the same way. Many people have heard that the interrupt instruction int can achieve a user-state-to-kernel-state priority from low to high, and after receiving the system service, the iret instruction is responsible for re-prioritizing the priority to the user state. However, the system starts with a kernel-high priority, and the Intel i386 does not provide a separate special instruction to lower the priority to run the user program after the system starts. In fact, the problem is very simple, as long as the anti-iret can be used, as if RET as a call to use the same. Also, a side effect of the fixup function is to fill in the address of the found printf function in the dynamic link library in got with the printf-related table entry (that is, the memory unit with address 0X80495A4). So when we call the printf function again, its address can be obtained directly from got, thus eliminating the process of finding through fixup. That is to say, got plays the role of the cache here.

Back to top of page

A little Feeling

In fact, there are a lot of things as long as diligent thinking, or can realize some of the truth. Some of the foreign masters are through to be able to see a little bit of information, their own to explore a lot of unknown secrets. Like the author of "Undocument Dos" and "Undocment Windows", he set this example for us!

Learning computer is a key point is to be rich in the spirit of exploration, to let oneself know it and know its why. Hou in the "STL Source Analysis," the beginning of the book preface wrote "The Source, no secret", of course, this is in our hands to grasp the source of the case, if not, do not forget that Linux also provides us with a large number of such as GDB, objdump and other practical tools. With these powerful assistants, even without the source code, we can do "no secret."

Resources

• John R. Levine "Linkers & Loaders".
• "Executable and Linkable Format".
• Intel. Intel Architecture software Developer ' s Manual. Intel Corporation, 1997.

Http://www.cnblogs.com/Anker/p/3527677.html

Use of the Linux dynamic link library

1. Preface

In the actual development process, each module will involve some common functions, such as reading and writing files, searching, sorting. In order to reduce the redundancy of the Code and improve the quality of the code, these common parts can be extracted to make a common module library. A dynamic link library enables you to share common functions among multiple modules. Before reading "Programmer self-accomplishment" in the program link and loading process, these things are the bottom, for the understanding of the program's compilation process is beneficial. http://www.ibm.com/developerworks/cn/linux/l-dynlink/Blog Describes the program's link and loading process. This paper focuses on application, how to write and use dynamic link library, and then use dynamic link library to implement a plug-in program.

2. Dynamic Link Library production

Dynamic link library compared with ordinary programs, there is no main function, which is a series of functions. Production of so dynamic link library files through shared and fpic compilation parameters. When the program calls the library function, it only needs to connect to the library. For example, the following implements a simple integer four transport dynamic link library, which defines the caculate.h and caculate.c two files, producing libcac.so dynamic link libraries.

The program code is as follows:

/*caculate.h*/#ifndef caculate_head_#define caculate_head_//addition int Add (int a, int b);//subtraction int sub (int a, int b);//Division int div (int a, int b);//multiplication int mul (int a, int b); #endif

/*CACULATE.C file */
#include "caculate.h"//For two numbers and int add (int a, int b) {    return (A + b);} Subtract an int sub (int a, int b) {    return (a);} Division int div (int a, int b) {    return (int) (A/b);} multiply int mul (int a, int b) {    return (A * b);}

Compile the production libcac.so file as follows: gcc-shared-fpic caculate.c-o libcac.so
Write a function that the test program calls this dynamic-link library, as shown in the following procedure:

#include <stdio.h> #include "caculate.h" int main () {    int a = +;    int B = Ten;    printf ("%d +%d =%d\n", A, B, add (A, b));    printf ("%d-%d =%d\n", A, B, sub (A, b));    printf ("%d/%d =%d\n", a, B, Div (A, b));    printf ("%d *%d =%d\n", A, B, Mul (A, b));    return 0;}

Compiling the production executable main is as follows:gcc main.c-o main-l./-LCAC (where-L indicates the path of the dynamic-link library,-L is the name of the link library, omitting Lib)
The program execution results are as follows:

3. Get the function of the dynamic link library
Linux provides functions for Dlopen, Dlsym, Dlerror, and dlcolose functions to get a dynamic-link library. Through this four function can implement a plug-in program, convenient program extension and maintenance. The function format is as follows:

#include <dlfcn.h>void *dlopen (const char *filename, int flag), char *dlerror (void), void *dlsym (void *handle, const char *symbol); int dlclose (void *handle); Link WITH-LDL.

Dlopen () is a powerful library function. The function opens a new library and loads it into memory. This function is primarily used to load symbols in the library, which are not known at compile time. Write a test program called the above production Libcac.so library as follows:

#include <stdio.h> #include <dlfcn.h> #define DLL_FILE_NAME "libcac.so" int main () {    void *handle;    Int (*func) (int, int);    char *error;    int a =;    int b = 5;    Handle = Dlopen (Dll_file_name, rtld_now);    if (handle = = NULL)    {    fprintf (stderr, "Failed to open Libaray%s error:%s\n", Dll_file_name, Dlerror ());    return-1;    }    Func = Dlsym (handle, "add");    printf ("%d +%d =%d\n", A, B, Func (A, b));    Func = Dlsym (handle, "sub");    printf ("%d +%d =%d\n", A, B, Func (A, b));    Func = Dlsym (handle, "div");    printf ("%d +%d =%d\n", A, B, Func (A, b));        Func = Dlsym (handle, "mul");    printf ("%d +%d =%d\n", A, B, Func (A, b));    Dlclose (handle);    return 0;}

The program execution results are as follows:gcc call_main.c-o call_main-ldl

4. Reference website

Http://www.cnblogs.com/xuxm2007/archive/2010/12/08/1900608.html

http://blog.csdn.net/leichelle/article/details/7465763

program linking and loading and implementation of dynamic link under Linux