Explore a hot update mechanism for C ++ binary modules

Nowadays, it is the basic responsibility of Internet service providers to provide services that are stable, reliable, and able to meet the increasing material and cultural needs of the people. Therefore, server-side software must be strong and flexible enough. Once the service program runs, it is better to stay around for 7x24 hours, and the changing and increasing user needs must be met as soon as possible. But the problem is, never expect programmers to write programs without Bugs. No architect can predict the future of the Ghost World, no matter
The code that looks perfect at the time will also be modified (or discarded) for various reasons in the future ?). In this case, we may want to add some evolutionary capabilities to the program so that it can never stop working and work tirelessly. At the same time, we also constantly reflect on ourselves, correct ourselves, and thrive.

This article is written to C ++ programmers. If your tool is a dynamic language such as LISP, Erlang, and Ruby, then because of their advanced dynamic features, you don't need to lick your blood on the binary layer.

In short, hot update means that the program is updated while running. Some people must think that I intentionally (like an expert) install B and make complicated simple problems, because the dynamic link library itself can be dynamically loaded and detached, you just need to notify the program to reload after the new dynamic library is built and deployed?

Here, I want to tell you with great enthusiasm: first, I didn't pretend to be B because I don't want to be hacked; second, this simple solution works in a few cases, but in most cases, it is not possible because the actual program is the correct combination of code and data structure, and the code almost always needs to operate the corresponding data structure. For example, the create function of database a creates the data object data, and the foo function can operate data correctly. Then we use database B to hot update database, in this way, both the create and foo functions are implemented by database B, and the data objects generated by the new CREATE FUNCTION are in different formats from data (Binary layout, the new Foo can only operate new data objects correctly. Assuming that the application needs to use (Database B) Foo to operate the data created by database a (which we cannot avoid, because the data lifecycle is closely related to the application logic), is it very likely that serious errors will happen at this time? Therefore, after a module is hot-replaced, all data objects created by it must be converted to a new version compatible (Binary layout) format. However, this raises a new question: how can we find all the data objects created by the old module? Like a lame mathematician, we turn a dirty problem into another dirty problem.

For another idea, if you are willing to follow certain specifications during programming (specification is a constraint, but reasonable constraints can often improve the overall freedom ), this specification allows us to avoid the thorny issue of finding all old data objects, so we can achieve secure hot updates.

The recommended specification in this article is to use component object models like COM and XPCOM: A program consists of component objects, each of which provides several functions, external functions can only be used through the interface of objects. Interfaces are usually built using C ++ abstract base classes. From the perspective of ABI (Application binary Interface, the most important thing about the C ++ abstract base class is to define the layout of the virtual function table of the subclass. You can also use other methods to build interface mechanisms, but you must ensure that the virtual function table model (G ++, mainstream compilers, such as VC ++, implement the same in this regard) and are compatible on the ABI layer. A module is a physical object container that can contain one or more component objects. The most common form of a module is the dynamic link library (so or DLL). The dynamic update mechanism explored in this article is the smallest unit of the module.

From the ABI layer, calling the relevant functions through the component object interface is actually to call the virtual function implementation pointed to by the corresponding items in the virtual table of this object, it is precisely because a look-up table is required to call the virtual function to find the intermediate operation of the real function address, so that we can hook the call of the component object, so as to have the opportunity to convert the old version of the object to the new version. So how can I hook the virtual function call of an object? The method is simple. Modify the virtual table so that each item of the virtual table points to us.
After this modification, the hook code will be executed whenever and wherever the external module calls the old version object. Some may think this is too hack and insecure: How can you determine the location of the virtual table and the number of items in the virtual table? Yes, you are right. If you do not follow any rules, it is indeed a very dangerous action to shake the strange C ++ compiler. Fortunately, if you use a component object model such as XPCOM, therefore, a rare consensus among the major C ++ compilers on the Implementation of Virtual tables in abstract classes can ensure that we can find the correct location of the virtual table, moreover, the extra (not available in Standard C ++) interface type information provided by the model ensures that we can safely modify the appropriate number of virtual table items.

Now we can use a simple example to test whether the above idea works. Here we provide a zip package for downloading code. Currently, only windows and FreeBSD platforms running on Intel ia32 architecture CPU are supported (of course, other UNIX platforms should be okay, only the header file inclusion path may need to be adjusted) implementation, Windows platform needs to install mingw. Decompress the package and run it in the corresponding directory.
Gmake plat = Windows
Or
Gmake plat = FreeBSD
The test program is generated. Remember to use FreeBSD first
Export LD_LIBRARY_PATH = ./
Add the current directory to the dynamic library search path before running.

The example includes the following source program files: nsibase. h nsimp1.h nsimp1.cpp nsimp2.h nsimp2.cpp dynahook. cpp test. cpp.

Nsibase. h defines an abstract base class nsibase, which represents an interface, which contains two interface methods: Hello and foo.

Nsimp1.h and nsimp1.cpp jointly form the implementation of the first version of the nsibase interface. They will be built into a dynamic link library named libimp1.dll (libimp1.so in UNIX, this is a module. The header file defines the nsimp1 class, which inherits from the nsibase abstract class. In addition to the specific implementation of interface methods, the CPP file also has three agreed export functions: <1> Create, which is equivalent to the factory method, because the external module does not know the specific implementation of the component object, therefore, you can only use it to create an object instance. <2> on_swapping, when this module is hot replaced by a new version of the module, this function is called, the format conversion function pointer passed to the module of the new version is used as a parameter. This function should change the function pointers in the virtual table of the current version object and point to the special hook code, after that, the external module will first be intercepted by the hook for any calls to these objects, and then the format conversion function of the new module will be executed before the normal interface function call is completed. <3> converter, format Conversion Function. When this module replaces another module, it is used to convert objects of earlier versions to objects of its own version, this includes setting new virtual table pointers and converting data blocks for objects. Because nsimp1 is the first version of implementation, there is no older implementation than it needs to be replaced, so its converter is an empty function.

Nsimp2.h and nsimp2.cpp form the implementation of the second version of the nsibase interface, which is built into libimp2.dll. The organizational structure of nsimp2 is similar to that of nsimp1, but its converter actually needs to be converted.

Dynahook. cpp is the most interesting part. It provides a function:
Void dynahook (void ** p_old_vtbl, int method_count, converter_t CF). p_old_vtbl points to the virtual function table of the old version object, and method_count indicates the number of interface methods in the table to be monitored, cf is (provided by the new module) The conversion function pointer used to convert the format of objects of the old version. The function is to generate special hook code to monitor corresponding interface method calls. The so-called special hook code is actually very simple, and its implementation is as follows (80 x86 compilation, GNU assembler format ):

Pushl % EBP; Save the stack frame base of caller
Movl % ESP, % EBP; set your own stack frame base
Pushl 8 (% EBP); this pointer goes into the stack
Call * converter; call the converter conversion function. It should set a new virtual function table pointer for the object.
Subl $4, % ESP; adjust the stack top
Movl 8 (% EBP), % eax; this pointer reads eax
Movl (% eax), % eax; (new version) read the first address of the virtual function table into eax
Addl $ method_offset, % eax "; plus the correct interface function offset
Movl (% eax), % eax; (new version) virtual function entry address read into eax
Leave; restore EBP and ESP, note that it is not followed by the normal RET command
JMP * % eax; jump directly to the implementation of (new version) interface functions

The above code is just a template. dynahook generates an almost identical machine code for each interface method to be hooked according to the template, but both the converter address and method_offset are reset, because they can only be determined at runtime, and then let the function pointer in the virtual function table point to these dynamically generated hook machine codes, does this achieve dynamic monitoring? Note that the implementation of the hook depends on caller passing the first (implicit) parameter of the interface method through the stack-This pointer. G ++ can meet this requirement, but the Visual C ++ series does not: even if the call specification such as fastcall is not used, it will also pass the this pointer in the ECX register (the above conclusion comes from the observation of the assembly code generated by the compiler. If there is an error, please correct it ). If you are interested, you can modify the hook code to make it suitable for the Visual C ++ compiler.

Test. cpp demonstrates how to use component objects and hot replacement. Some code is as follows:

Nsibase * O1 = (* create_v1) (); // create a component object with version 1
Nsibase * O2 = (* create_v1) (); // create a component object with version 1

Printf ("create objects of version 1: % P, % P/N ",
O1, O2 );

O1-> Hello (); // directly call the implementation of version 1
O2-> Foo (); // directly call the implementation of version 1

// Hot replacement
......

O1-> Hello (); // The Conversion Function is triggered first, and then the implementation of version 2 is called.
O2-> Foo (); // The Conversion Function is triggered first, and then the implementation of version 2 is called.

O1-> Hello (); // directly call the implementation of Version 2
O2-> Hello (); // directly call version 2.
O1-> Foo (); // directly call the implementation of Version 2
O2-> Foo (); // directly call version 2.


The program output is:

Create objects of version 1: 00032bc8, 00032ce0
OBJ (00032bc8), Version1, hello (), m_data-> A = 2008
OBJ (00032ce0), Version1, Foo (), m_data-> A = 2008
Convert object (00032bc8) from version 1 to 2
OBJ (00032bc8) Version 2, hello (), m_data-> A = 2008, m_data-> B: 77
Convert object (00032ce0) from version 1 to 2
OBJ (00032ce0) Version 2, Foo (), m_data-> A: 2008, m_data-> B: 77
OBJ (00032bc8) Version 2, hello (), m_data-> A = 2008, m_data-> B: 77
OBJ (00032ce0) Version 2, hello (), m_data-> A = 2008, m_data-> B: 77
OBJ (00032bc8) Version 2, Foo (), m_data-> A: 2008, m_data-> B: 77
OBJ (00032ce0) Version 2, Foo (), m_data-> A: 2008, m_data-> B: 77

Let's talk about the shortcomings and limitations of this solution.

First, the solution requires the new version module to know the binary layout of the old version object, which means that the definition file of the old component object should be available. If the old version of the object aggregates third-party (non-component) objects, and the internal status of the object does not provide interface replication, the implementation of the new version cannot replace the third-party objects. This is a huge limitation.

Second, even if you have the definition of the old component object, you must ensure that the key compilation parameters (such as the number of bytes alignment of the struct) of each module are the same, otherwise, it is very dangerous for the Conversion Function to access data blocks of old objects.

Furthermore, in order to facilitate the format conversion of data blocks, the implementation of each version requires that data members cannot be directly embedded, and only one pointer can be used to point to a struct, just like nsimp1:
Class nsimp1: Public nsibase
{
Public:
Nsimp1 ();

Virtual void Hello ();
Virtual void Foo ();

Protected:
Struct data_t
{
Int;
};

Data_t * m_data;
};
This makes code compilation more complicated.

Finally, to correctly change the virtual table, you need to know some type information such as the number of methods contained in the interface, and C ++ does not have a standard method to obtain it. Therefore, the component
The dynamic type information provided by the framework is crucial.