Classical programming: DLL hell and Its Solutions

Original Author: Ivan S zapreev
Translator: Lu Qiming

From http://tech.acnow.net/Html/Program/VC/DLL/2004-5/17/041435538.shtml
　　
Summary
　　
This article will introduce the backward compatibility of DLL, which is also known as "DLL hell. First, I will list my own research results, including the results of some other researchers. At the end of this article, I will also provide a solution to the "DLL hell" problem.
　　
Introduction
　　
I once accepted a task to solve a dll version update problem-a company provides users with a set of sdks, which are composed of a series of DLL; DLL exports many classes. You can use these classes (directly use or derive new subclasses) to continue their C ++ program development. The user does not get detailed instructions when using these DLL (such as restrictions on the Classes exported from these DLL ). When these DLL versions are updated to a new version, they find that their DLL-based applications often crash (their applications derive a new subclass from the export class of the SDK ). To solve this problem, users must recompile their applications and reconnect to the new sdk dll.
　　
I will give my research results on this issue, as well as information I have collected from other places. Finally, I will solve this "DLL hell" problem in the future.
　　
Study results
　　
In my personal understanding, this problem is caused by the changes to the base class exported in the sdk dll. After reading some articles, I found that some people have raised backward compatibility issues with DLL. But as a real researcher, I decided to do some experiments on my own. As a result, I found the following problems:
　　
1. adding a new virtual function to the DLL export class will cause the following problems:
(1) If this class previously had a virtual function B, a new virtual function A will be added before it. In this way, we changed the virtual function table of the class. Therefore, the first function in the table points to function a (instead of the original B ). At this time, the client program (assuming that the new DLL is not re-compiled and connected) calls function B, and an exception occurs. At this time, function B actually calls function a. If function a and function B have different parameter types and return value types, the problem arises!
(2) If this class does not have a virtual function (its parent class does not have a virtual function ), adding a new virtual function to this class (or adding a virtual function to its parent class) will lead to a new class member, which is of the pointer type, point to the virtual function table. Therefore, the size of this class will be changed (because a member variable is added ). In this case, if the customer program creates an instance of this class and needs to directly or indirectly modify the value of the class member, the problem may occur. Because the pointer of the virtual function table is added as the first member of the class, that is to say, the member defined in this class generates an address offset because of the addition of the virtual function table pointer. The operations performed by the customer program on the original members are naturally abnormal.
(3) If the class has a virtual function (or as long as its parent class has a virtual function), and the class is exported, it will be used as a parent class by the client program. So, we should not add virtual functions to this class! It cannot be added at the beginning of the class declaration, even at the end. Adding a virtual function will lead to an offset in the function ing in the virtual function table. Even if you add the virtual function to the end of the class declaration, the virtual function table of the derived class of this class also produces an offset.
　　
2. adding a new member variable to the DLL export class will cause the following problems:
(1) adding a member variable to a class will lead to a class size change (adding a virtual function to a class with a virtual function table will not change the class size ). Suppose this member is added at the end of the class declaration. If the client program allocates less memory for the instance that creates this class, the memory may be out of bounds when this member is accessed.
(2) It would be worse if a new member is added to the original class member. This causes the address of the original Class Members to be offset. The client program operates on an incorrect address table. This is especially true for new members (they all change their offset in the class due to the addition of new members ).
　　
(Note: The above customer program refers to the application that uses sdk dll .)
　　
In addition to the above reasons, there are other operations that may cause DLL backward compatibility problems. The solutions to most of these problems are listed below.
　　
DLL coding conventions
　　
The following are all the solutions I have collected, some of which are obtained through online articles and some after communication with Different developers.
　　
The following conventions mainly aim at DLL development and solve the backward compatibility problem of DLL:
　　
1. Encoding conventions:
(1) Each DLL export class (or its parent class) contains at least one virtual function. In this way, this class will always save a pointer member pointing to the virtual function table. This can facilitate the addition of new virtual functions.
(2) If you want to add a virtual function to a class, add it to the end of all other virtual functions. In this way, the address ing sequence of the original functions in the virtual function table will not be changed.
(3) If you plan to expand a class member in the future, reserve a pointer pointing to a data structure. In this case, add a member to directly modify the data structure, instead of modifying the member in the class. Therefore, adding new members will not change the class size. Of course, several operation functions need to be defined for this class to access new members. In this case, the DLL must be implicitly connected by the Client Program (implicitly.
(4) to solve the previous problem, you can also design a pure interface class for all export classes. However, at this time, the client program cannot continue to derive from these export classes, the levels of DLL export classes cannot be maintained.
(5) Release two versions of DLL and Lib files (debug and release versions ). If only the release version is released, developers will not be able to debug their programs, because the release version and the debug version use different heap managers, therefore, when the client program of the debug version releases the memory applied for by the DLL of the release version, it will cause a runtime error (runtime failure ). There is one way to solve this problem, that is, the DLL also provides functions to apply for and release the memory for the client program to call; the DLL also ensures that the content requested by the client program is not released. It is not that easy to follow this convention!
(6) during compilation, do not change the default parameters of the DLL export class function, if these parameters will be passed to the client program.
(7) Pay attention to the changes to the inline function.
(8) check that all enumeration items do not have default element values. Because when you add/delete a new enumerated member, you may move the value of the old enumerated member. This is why each member should have a unique ID value. If enumeration can be extended, you should also document it. In this way, the customer program developers will be noticed.
(9) do not change the macro defined in the header file provided by DLL.
　　
2. Version Control of the DLL: if the main DLL changes, it is best to change the DLL file name at the same time, just like Microsoft's mfc dll. For example, the DLL file can be named in the following format: dll_name_xx.dll, where XX is the dll version number. Sometimes a lot of changes are made in the DLL, so that the backward compatibility problem cannot be solved. A new dll should be generated. The old DLL is retained when the new DLL is installed to the system. Therefore, the old client program can still use the old DLL, while the new client program (using the new DLL to compile and connect) can use the new DLL, which does not interfere with each other.
　　
3. dll backward compatibility test: there are many other problems that may damage the DLL backward compatibility. Therefore, it is necessary to implement the DLL backward compatibility test!
　　
Next, I will discuss a question about a virtual function and a corresponding solution.
　　
Virtual functions and inheritance
　　
First, let's take a look at the following virtual functions and inheritance structures:
　　
/*********** DLL export class **********/
Class export_dll_prefix extends functclass {
Public:
Extends functclass (){}
~ Extends functclass (){}
Virtual void dosmth (){
// This-> doanything ();
// Uncomment of this line after the corresponding method
// Will be added to the class declaration
}
// Virtual void doanything (){}
// Adding of this virtual method will make shift in
// Table of Virtual Methods
};
　　
/********** The client program derives a new subclass from the DLL export class **********/
Class extends functclasschild: Public extends functclass {
Public:
Extends functclasschild (): extends functclass (){}
~ Using functclasschild (){};
Virtual void dosomething (){}
};
　　
Assume that the above two classes are implemented by using javasfunctclass in my. dll, while javasfunctclasschild is implemented in the client program. Next, let's make some changes and release the following two comments:
// Virtual void doanything (){}
And
// This-> doanything ();
　　
That is to say, the DLL export class has been modified! If the client program is not re-compiled, then the javasfunctclasschild in the client program will not know that the javasfunctclass class in the DLL has changed: added a virtual function void doanything (). Therefore, the virtual function table of the javasfunctclasschild class still contains the ing of the two functions:
1. Void dosmth ()
2. Void dosomething ()
　　
In fact, this is already incorrect. The correct virtual function table should be:
1. Void dosmth ()
2. Void doanything ()
3. Void dosomething ()
　　
The problem is that after instantiating javasfunctclasschild, if you call its void dosmth () function, the dosmth () function calls the void doanything () function instead, however, the base class javasfunctclass only knows to call the second function in the virtual function table, while the second function in the virtual function table of the javasfunctclasschild class is still void dosomething (), the problem arises!
　　
In addition, adding a virtual function to the derived class of the DLL export class (in the previous example, javasfunctclasschild) is not helpful. Because, if the javasfunctclasschild class does not have the virtual void dosomething () function, the void doanything () function in the base class (the second function in the virtual function table) the call will point to an empty memory address (because the empty functclasschild class maintains only one function address for the virtual function table ).
Now we can see how serious it is to add a virtual function to the DLL export class! However, if the virtual function is used to handle callback events, we can solve this problem (as described below ).
　　
COM and others
　　
It can be seen that the backward compatibility of DLL is a very famous issue. To solve these problems, we can not only use some conventions, but also use other advanced technologies, such as COM technology. Therefore, if you want to get rid of the "DLL hell" problem, use com or some other suitable technologies.
　　
Let's go back to the task I accept (the task I mentioned at the beginning of this article)-to solve the backward compatibility problem of a DLL product.
　　
I have some knowledge about com, so my first suggestion is to use COM technology to overcome all the problems in that project. However, this suggestion was rejected for the following reasons:
1. The product already has a COM server in an internal layer.
2. rewrite a large number of interface classes to the form of COM, with a large investment.
3. Because the product is a dll library, and many applications are using it. Therefore, they do not want to force their customers to rewrite their applications.
　　
In other words, the task I want to accomplish is to solve the problem of backward compatibility of the DLL at the minimum cost. Of course, I should point out that the main problem of this project is to add new members and virtual callback functions on the Interface Class. The first problem can be solved simply by adding a pointer to a data structure in the class declaration (so that new members can be added at will ). I have mentioned this method above. But the second problem is that the question of the virtual callback function is newly raised. Therefore, I have proposed the following minimum price and the most effective solution.
　　
Virtual callback functions and inheritance
　　
Imagine that we have a DLL that exports several classes. The client application will derive new classes from these export classes to implement virtual functions to handle callback events. We want to make a small change in the DLL. This change allows us to add a new virtual callback function to the export class "Painless" in the future. At the same time, we do not want to affect the application that uses the current version of DLL. We expect that these applications will re-compile with the new DLL only when they have. Therefore, I have provided the following solutions:
　　
We can retain each virtual callback function in the DLL export class. We only need to remember to add a new virtual function to any class definition. If the application does not re-compile with the new version of DLL, it will cause serious problems. What we do is to avoid this problem. Here we can use a "listener" mechanism. If the virtual functions defined and exported in the DLL export class are used for processing callback, we can transfer these virtual functions to an independent interface.
　　
Let's take a look at the following example:
　　
// If you want to test the modified DLL, release the following definition.
// # Define dll_example_modified
　　
# Ifdef dll_export
# Define dll_prefix _ declspec (dllexport)
# Else
# Define dll_prefix _ declspec (dllimport)
# Endif
　　
/*********** DLL export class **********/
# Define class_uiid_def static short getclassuiid () {return 0 ;}
# Define object_uiid_def virtual short
Getobjectuiid () {return this-> getclassuiid ();}
　　
// All basic callback Interfaces
Struct dll_prefix icallback
{
Class_uiid_def
Object_uiid_def
};
　　
# UNDEF class_uiid_def
　　
# Define class_uiid_def (x) Public: static
Short getclassuiid () {return X: getclassuiid () + 1 ;}
　　
// Interface extension only when the dll_example_modified macro has been defined
# If defined (dll_example_modified)
// New interface Extension
Struct dll_prefix icallback01: Public icallback
{
Class_uiid_def (icallback)
Object_uiid_def
Virtual void docallback01 (INT event) = 0; // new callback function
# Endif // defined (dll_example_modified)
　　
Class dll_prefix cexample {
Public:
Cexample () {mphandler = 0 ;}
Virtual ~ Cexample (){}
Virtual void docallback (INT event) = 0;
Icallback * setcallbackhandler (icallback * Handler );
Void run ();
PRIVATE:
Icallback * mphandler;
};
　　
Obviously, to facilitate the export class of the extended DLL (adding new virtual functions), we must do the following:
1. Added icallback * setcallbackhandler (icallback * Handler); function;
2. Add the corresponding pointer to the definition of each export class;
3. Define three macros;
4. define a general icallback interface.
　　
To demonstrate how to add a new virtual callback function to the cexample class, I added an icallback01 interface definition here. Obviously, the new virtual callback function should be added to the new interface. Each DLL update adds an interface (of course, multiple virtual callback functions can be added to a class each time the DLL is updated ).
　　
Note: Each new interface must be inherited from the interface of the previous version. In my example, I defined only one Extended Interface icallback01. If a new virtual callback function is added to the DLL in another version, we can define an icallback02 interface. Note that the icallback02 interface must be derived from the icallback01 interface, just like icallback01 is derived from icallback.
　　
The code above also defines several macros to define functions that need to check the interface version. For example, we need to add a new docallback01 function for the new interface icallback01. If we want to call icallback * mphandler;, we should check the function in the cexample class. This check should be implemented as follows:
　　
If (mphandler! = NULL & mphandler-> getobjectuiid ()> = icallback01: getclassuiid ()){
(Icallback01 *) mphandler)-> docallback01 (2 );
}
　　
We can see that after the new callback interface is added, you only need to insert a new callback call in the implementation of the cexample class.
　　
Now you can see that the above DLL changes will not affect the customer's application. The only thing that needs to be done is to use the first dll version after the new design (which adds the macro definition, the basic callback interface icallback, And the setcallbackhandler function for callback processing for the DLL export class, and icallback interface pointer) released, the application re-compilation. (New callback interfaces will be extended in the future. Re-compilation of applications is not necessary !)
　　
In the future, if someone wants to add a new callback, they can implement it by adding a new interface (we add icallback01 in the above example ). Obviously, this change will not cause any problems, because the order of virtual functions has not changed. Therefore, the application is still running in the previous way. The only thing you need to note is that, unless you implement a new interface in the application, you will not receive the newly added callback call.
　　
We should note that DLL users can still work with it easily. The following is an example of the implementation of a class in the client program:
　　
// If dll_example_modified is not defined, use the DLL of the previous version.
# If! Defined (dll_example_modified)
// The Extended Interface icallback01 is not used at this time
Class cclient: Public cexample {
Public:
Cclient ();
Void docallback (INT event );
};
　　
# Else //! Defined (dll_example_modified)
// After the new interface icallback01 is added to the DLL, the client program can modify its own class.
// (But not required, if he does not want to handle the new callback event)
Class cclient: Public cexample, public icallback01 {
Public:
Cclient ();
Void docallback (INT event );
　　
// Declare the docallback01 function (the client program must implement it to handle new callback events)
// (Docallback01 is the new virtual function of icallback01 Interface)
Void docallback01 (INT event );
};
# Endif // defined (dll_example_modified)
　　
Routine ---> code download (6.26 K)
　　
In concert with the content in this article, I provide the demo program dll_hell_solution.
　　
1. dll_example: DLL Implementation Project;
2. dll_client_example: client application project of DLL.
　　
Note: The dll_example_modified definition in the dll_hell_solution/dll_example/dll_example.h file is commented out. If you release this annotation, you can generate the updated dll version, and then test the customer application again.
　　
To ensure normal demonstration, follow these steps:
1. Do not change any code (the dll_example_modified is not defined at this time) to compile two projects: dll_example and dll_client_example. Run the customer program to experience the initial situation.
2. Release the comments of dll_example_modified and recompile dll_example. Re-run the client program (the new dll version is used at this time) and it should still run normally.
3. recompile dll_client_example to generate a new client program. We can see that the newly added callback function has been called!