Some issues related to compilation and linking (-FPIC)

Issues related to compilation and links (zz)
Address-independent code. When compiling a dynamic library in 64-Bit mode, the following errors are often encountered:

/Usr/bin/ld:/tmp/ccq1dkqh. O: Relocation r_x86_64_32 against 'a local symbol' can not be used when making a shared object; recompile with-FPIC

It is prompted that-FPIC compilation is required, and the result of-FPIC parameter compilation is added to the link to the dynamic library. An error is returned, all the static databases used by the shared library must be compiled using-FPIC to compile the dynamic library in a 64-bit environment.

Here-FPIC refers to the address-independent code.

Here, we first describe the problem of migration during loading. If a program does not use any dynamic library, it knows all the code,
Then, when the loader loads the program into the memory, it can directly install the static library and set the position of the code segment during the link to directly load it into the corresponding position in the memory.
But in the face of dynamic databases, this method will not work. Assume that the shared library a needs to be loaded, but the shared library used during the compilation of the link is not necessarily the same library that is run at the end,
During the compilation period, you cannot know the specific library length, and you cannot determine the specific location of the library or other dynamic libraries when linking. Some global symbols are also used in the dynamic library,
These symbols may come from other dynamic libraries, which cannot be assumed in the compiler (if possible, they are all static libraries)

Based on the above reasons, it is required to relocate the used symbolic address when loading the dynamic library. In terms of implementation, no relocation is performed during link compilation, and all addresses use relative addresses,
1. When loading is required, the final absolute address is calculated based on the offset of the relative address and loaded into the memory.

However, the same library code (excluding the data) is a problem in this method of migration during loading) it cannot be shared among multiple processes (each code is stored in its own process space ),
This loss of the advantage of saving memory in the dynamic library.

To solve this problem, elf creates a point in the data segment that needs to be used (internal location independence can be achieved simply by using relative address access) address List (also known as global offset table, global offset table, got ). you can use the location corresponding to got for indirect reference.

For our 32-bit environment, whether-FPIC is added during compilation will not affect the link, there are several copies of a piece of code in the memory (and the results are the same for the static library ). however, there is a problem in the method of relocation when loading in a 64-bit environment, that is, in our 64-bit environment, the CPU command for location offset locating only supports 32-bit offsets, however, the actual location offset may exceed 64-bit. Therefore, in this case, the compiler requires that you compile the program using FPIC to use it in the shared library.

Theoretically, the performance of FPIC may be compromised due to the multiple calls of memory address fetch. however, from some tests, we cannot see the significant loss on the database performance after-FPIC is added, this may be related to the machine cache we are currently using and the existence of a large number of registers.

TIPS:

-The above introduction of FPIC and-FPIC shows that GCC needs to add-FPIC with address-independent code, however, in the GCC manual, we can see a-FPIC (the difference is in upper case and lower case) parameter, which is functionally the same. -FPIC can be optimized in some specific environments (including hardware environments) to generate less and faster Code. However, due to platform restrictions, such as our compiling environment, when the development environment and running environment are not completely uniform, there is a certain unknown risk to use FPIC. In most cases, we use-FPIC to generate address-independent code.
Shared Memory Efficiency

In the case of read-only in the shared memory, the performance is the same as that of the read common memory (if not the first load consumption), and because multiple processes share the CPU cache, it is relatively friendly.

Both static and dynamic libraries exist.

As mentioned above, when compiling a dynamic library, we mentioned that compiling a dynamic library can be done in the-lpath-LXX mode just like compiling a static library, but there is a problem here, what is the behavior when there are both dynamic and static libraries in the path directory? In fact, in this case, the linker gives priority to dynamic library compilation. for example, libx exists in the same directory. A And libx. so, the libx. so. this is why most of the third-party libraries (third, third-64) maintained by the COM group have only one output. A is mainly used to avoid using it by default. so library,
This causes trouble when going online (especially databases that exist in some systems but differ from the version we need ).

To control the compilation of dynamic and static libraries, the following methods are provided:

Directly use the library to be compiled
As mentioned above, there are three methods for compiling static libraries.
Direct use of the target file. o
Static library file. A directly compiled
Compile in-l-L Mode

If-lpath-LXX is not used for compilation, and path/libx is written directly. A or path/libx. so for compilation, we will use the specified link. A or. so does not cause the so-called priority of dynamic databases or static databases. however, this solution requires the path of the compiling database, which is not suitable for use in some cases.

-- Static parameter

Add the -- static parameter during GCC compilation. In this way, the static library will be preferentially selected during compilation, rather than dynamic library compilation by default.

However, using the -- static parameter will bring about another problem, which is not recommended. It will mainly bring about the following problems:

If there is only a dynamic library and no static library with the same name exists, no error will be reported during the link, but an error may occur during running/lib/ld64.so. 1: bad elf Interpreter:
Because the program itself is running requires the support of some libraries in the system, after the -- Static compilation method is adopted, the static compilation versions of these libraries are linked, it means that the database on the compiler is used, but our runtime environment may be different from that on the compiler, the objective of glibc dynamic libraries is to make it easier for compiled libraries on one machine to be moved to another machine. The program only needs to focus on interfaces, from the interface to the bottom layer. so to process. however, this problem is not so absolute. In some special cases (for example, when there are major differences in glibc and GCC versions, it is mainly because gcc2 and gcc3 are not well done in some places, the problem of ABI incompatibility is quite prominent. In this case, you need to change the compiler.) -- Static compilation can run normally. but it is not recommended,
These can be described in the following 6th points using other method specifications. In addition, glibc -- Static compilation may generate the following warning:
Warning: using 'getservbyport _ R' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking. This is mainly because interfaces such as getservbyport_r still require the support of dynamic libraries. run, many glibc functions have such problems, especially network programming interfaces.
It is unfriendly to some third-party tools, similar to valgrind checking for Memory leakage. In order to avoid false positives in some special circumstances, it needs to replace the function in glibc with a dynamic library, if static compilation is performed, valgrind cannot replace these functions, resulting in false positives or even errors. tcmalloc is not supported in this case.
The pthread library used in the 64-bit environment, if it is a dynamic library, it uses the ntpl library. If it is a linuxthread library used by the static library, using -- Static will lead to performance degradation.
-- After static, the code size will increase, which is unfriendly to the CPU code cache and a waste of memory space. However, this is not a problem with small code.

Link parameter control
The-DN-dy parameter is provided in the linker to control whether a dynamic library or a static library is used.-DN indicates that a static library is used later, and-dy indicates that a dynamic library is used.

Example:

G ++-lpath-wl,-DN-lx-wl,-dy-lpthread

In this way, if the path contains libx. So and libx. A, only libx..

Note that if-WL is not found at the end and-dy enables dynamic libraries to be used in subsequent databases, the error "cannot find-lgcc_s" may be reported because of glibc. database A and. if the so library name is different, -- Static will be automatically processed, but-WL and-dy won't identify this problem.

TIPS:

If -- Static is used, the following libraries are all shared libraries due to the use of-dy (dy forcibly blocks static libraries ), an error will be reported when the compiled program is forced to use -- Static compilation only when the dynamic library is used.

Running error "undefined reference to 'xxx ()'"

For dynamic link libraries, the actual symbols are located at runtime. in the compilation. in so, if the library required by the library is not associated with it, for example, libx. so you need to use uldict, but forget to compile libx. if-luldict is added to so, libx is compiled. so does not report an error, because libx. so is considered to be a library, which contains some symbols that do not know the specific implementation is legal, it can be specified at runtime or when another binary program is compiled.

If G ++-lpath-lx is used for compilation, the linker will find that the required uldict symbol table cannot be found and an error is reported. However, if the program uses dlopen for loading, because the program runs at runtime, an error is reported during this process. another case is that an external interface has been declared and defined in the dynamic library, but the implementation is forgotten, and similar errors will occur at this time.

If such an error is reported at runtime, check whether the error is caused by the fact that some libraries are not linked or some interfaces are not implemented.

========================================================== ==========

I have collected some compilation and link related questions. If you have any questions, please feel free to ask.

How to Use ullib libraries compiled with G ++ for pure C Programs

As mentioned above, directly compiling results in the G ++ environment will lead to different results that conform to the table and GCC compilation, and thus cannot be mixed compilation.

Library principles compiled by GCC using G ++:

1. when compiling the library, G ++ needs to package the interfaces used by the outside world in a pure C ++ acceptable way with extern "C", and add the _ cplusplus macro to judge, for more information, see public/mcpack and public/nshead. for some special cases, such as libraries compiled by G ++, which are not suitable for modification, such as ullib and word segmentation, you can write a XXX by yourself. CPP program, in XXX. CPP encapsulate the required interfaces with pure C interfaces again, and use extern "c" to export and use pure C interfaces. compile with G ++ and add libraries such as ullib to the link. 2. to compile the G ++ library with GCC, add-lstdc ++ at the end of our 64-bit environment.

Most of the libraries compiled by GCC using G ++ are seen in the need to expand the basic library and PHP, And the Apache Mod is integrated;

G ++ uses the library compiled by GCC: This is relatively simple. Only the header file compiled by GCC adopts the extern "C" encapsulation.

Whether or not the MD5 values of programs compiled in the same way are the same in the same environment

If the environment is exactly the same, including the compilation path and environment variables, it is usually the same, but it is difficult for us to do the same in many environments, for example, if the program uses some data, the time-related macro will lead to different compilation results. Sometimes, even the amount of memory may affect the compilation results.

When linking and running, what is the search order of static and dynamic library paths?

Search Order of links:

-L the specified path, search from left to right
The path specified by the Environment Variable LIBRARY_PATH. Use ":" To split and search from left to right.
The path sequence specified by/etc/lD. So. conf
/Lib and/usr/lib (64-bit/lib64 and/usr/lib64)

Search sequence of dynamic library calls:

The path specified by the-rpath parameter of LD, which is written to the code
Path specified by the LD script
Path specified by LD_LIBRARY_PATH
Path specified by/etc/lD. So. conf
/Lib and/usr/lib (64-bit/lib64 and/usr/lib64)

In general, we use the-l method to specify the search path and LD_LIBRARY_PATH to specify the link path when calling the dynamic link library.

In addition, if the first one is found, the system will return the result, and the following will not be searched. for example,-l. /A-L. /B-lx has libx in. a B contains libx. A And libx. so, this will be used in. /A libx. A does not follow the dynamic library priority principle, because. /A is first found, and there is no dynamic inventory with the same name in.
In what situations will the "undefined reference error" error occur?

Here we will summarize the possible scenarios of this problem:

No corresponding library specified (. O /. a /. so) the object defined in the library is used, but the Library (-LXXX) is not specified or the library path (-lyyy) is not specified, this error will occur,
The parameter sequence of the connected database is incorrect by default. The more basic the database is used, the more dynamic the database is.
The GCC/LD version does not match the GCC/LD version. Due to compatibility issues between gcc2 and gcc3 (in fact, gcc3.2 to 3.4 also has such problems to some extent) this error occurs when you use an earlier version of a machine on a later version. This problem is common in a 32-bit environment, in addition, the 64-bit database is accidentally used in the 32-bit environment or the 32-bit database is used in the 64-bit environment in turn.
C/C ++ interdependencies and links the GCC and G ++ compilation results to ensure that both sides of the extern "C" interface can be used, in our 64-bit environment, the GCC link g ++ Library also needs to be added with-lstdc ++. For details, see the previous section on mixed compilation.
The error is reported during runtime because the program uses the dlopen Method for loading. so,. so all the required databases are not linked. For details, refer to the preceding description of hybrid use of static and dynamic databases.

Can two. O files be directly merged into a. o file?

Yes. The command is LD-r. o B. o-o x. o, but this is not recommended. The only advantage of doing so is that when the static library is linked, if. the symbol in O can also put B. to avoid the application of -- whole-archive.

However, it is not recommended to do so, which increases the trouble of maintaining source files.

Why didn't I add the code inline into the program when using inline?

The compiler determines whether the inline function can be used by the inline function. In many cases, the inline function is specified immediately but cannot be used by the inline function.

Note that in GCC, inline takes effect only after-O and above optimization is used. There are no-o,-O2,-O3 optimization methods, no matter whether-Finline-functions GCC is added or not, inline optimization is not performed. At this time, inline is equivalent to a common function (in fact, there is a difference, it is different in the symbol table ). -Finline-functions is added to the program during compilation, but if there is no-ox (x> = 1) combination,-Finline-functions is actually invalid and does not work or report errors.

In GCC, in order to support the function inline without-ox (x> = 1), an extension always_inline is provided. Write the function as follows:

_ Attribute _ (always_inline) int Foo ()
{
...
}

You can add Foo inline to the program without-ox (x> = 1). However, the always_inline extension is only supported after gcc3, 2.96 GCC used in a 32-bit environment is not supported.

Can a 32-bit program be compiled on a 64-bit machine?

Theoretically, yes. The-M32 parameter is provided in the 64-bit GCC on a 64-bit machine, which can be specified for 32-bit compilation, however, the compilation problem still exists although it solves the link problem. There are two libraries that can be used for link on 64-bit machines for 64-bit programs, the other one is for gcc2.96 compilation programs to run on 64-bit machines. Neither of these libraries can provide a link environment for GCC-M32 results (32-bit libraries cannot be connected to 64-bit libraries, the library gcc2.96 is too old to be incompatible. Therefore, you cannot compile the executable 32-bit program (compiled. o file)

Why cannot I directly run the dynamic link library?

In the summary of the shared library, we introduced how to implement the shared library to run on your own. However, sometimes the error of undefined reference error occurs, and the shared library cannot be run.

The reason for this is that the dynamic library adopts a statement similar to static int val = func (XXX); where Val is a global variable (or static global variable ). When a dynamic library is loaded into the memory for use, the func function will be run first. If func is from another library (for example, in some cases, the main program uses-rdynamic for compilation, dynamic library uses the space of the main program). This problem occurs when the library for Dynamic Link compilation is not linked.

For such problems, consider the following solutions:

1. Do not use static int val = func (XXX );
Link the used static library to the shared library, but pay attention to the impact of-rdynamic. When necessary, ensure that the library version used by the main program is the same.
Making the shared library unavailable is also a solution

Can the program be executed before the main function starts?

If the code is executed before the main function starts, there are two methods below:

Run int val = func (XXX) in func (XXX ).
Declare a class, write the function to be run in class, and define a global (or static) Class variable.

In implementation, the compiler puts them in a special symbol _ init, which is executed when the program is loaded into the memory.

However, this method is not recommended, especially when there is a dependency between the library and the library in the Execution code, such as the following scenario:

LIBA. cpp
Class Aclass
{
Public:
Aclass ()
{
Int * u = bfunc (); // This is a function in the libb of another library.
Int c = U [0];
}

}

Static Aclass s_test;

Libb. cpp

Static int * s_test = test_init (); // initialize s_test

Int * bfunc ()
{
Return s_test;
}

The above program has two libraries. A static variable constructor in database a depends on a function in database B, this function in database B operates another static variable initialized by the function test_init.

According to the program requirements, we must make the test_init () function run before the Aclass function, but it is a pity that in some cases it is difficult to do this, this involves the question about the library links and initialization sequence of the linker.

By default, the execution sequence of the constructors test_init () and s_test is from right to left in the order of-l at the link time, for example, if-LB-La is used, the Aclass constructor will execute before test_init (). In this case, problems may occur. Ensure that the order of-la-lb is normal.

Another problem is involved here, that is, in normal cases, since a depends on B, you must ensure that-La is in-lb when linking. but here we can only say that the more basic the database needs to be placed behind, rather than at the end. or the above example. if there is a test. CPP uses database A and does not directly use the items in database B in test. If-lb is placed in front of-la, the linker reports an error, because test does not use B when the symbol is expanded from left to right, it is not expanded. From this perspective, the symbol cannot be found when a is linked. however, if a function associated with test_init is used in test, test_init will be exported if-lb is placed before-LA to expand function B,
In this way, a will think that test_init already exists, so that no compilation error is reported. However, the result is that the initialization sequence of test_init is put after Aclass, which may cause errors when the program is running.

There are several main considerations for solving this problem.

The Singleton mode is used. If (null = PTR) PTR = new XXX; return PTR is used for control through user State judgment. However, some multithreading issues need to be considered in some cases,
Understand the dependency and put-lb behind-la
This method is not allowed.

Pay special attention to the initialization sequence when using global variables. TIPS: constructor initialization and so on, which are processed in _ init. In another aspect, _ fini is an operation that exists before the program exits.