C ++ Engineering Practice (2): Do not reload the Global: operator new ()

Chen Shuo (giantchen_AT_gmail)

Blog.csdn.net/Solstice

This article only considers Linux x86 platform and server development (does not consider the issue of cross-DLL Memory Allocation and release in Windows ). This article assumes that the reader knows what: operator new () and: operator delete () are, and there are differences and relationships with the new/delete expressions, for more information about this, see Mr. Hou Jie's article "" "[1] Or this article.

The memory management of C ++ is a common topic. In section 7th, "When a destructor encounters multithreading", I wrote an episode: systematically avoiding various pointer errors "briefly reviews some common problems and solutions in modern C ++. Basically, the memory is managed by the modern C ++ method (RAII). It is hard for you to encounter memory errors. "No error" is a basic requirement and does not mean "good enough ". We often try to optimize the performance. If profiling indicates that the hot spot is in memory allocation and release, the Global: operator new () and: operator delete () are reloaded () it seems to be a good method once and for all (hereinafter referred to as "Overload: operator new ()"). This article attempts to explain that this method often does not work.

Basic Requirements for Memory Management
If you only want to allocate and release the files, the basic requirement for memory management is "No duplicate": Neither duplicate delete nor missing delete. That is to say, we often say that new/delete should be paired. "pairing" not only means that the number is equal, but also implies that the call of new and delete must match, do not "pay back the things borrowed by the East ". For example:

The memory allocated with the system's default malloc () should be handed over to the system's default free () for release;
Objects created using the default new expression must be parsed and released by the default delete expression;
Objects created using the default new [] expression must be handed over to the default delete [] Expression for structure and release;
The memory allocated by the system's default: operator new () should be handed over to the system's default: operator delete () for release;
Objects Created with placement new must be parsed using placement delete (for the sake of convenience, let's just say) (in fact, directly calling the destructor );
The memory allocated from A memory pool A should be returned to the memory pool.
If new/delete is customized, follow the rules. For more information, see Objective C ++.
It is not difficult to achieve this. It is the basic skill of every C ++ developer. However, if you want to reload the Global: operator new (), it will be troublesome.

Overload: operator new () Reason
Article 50th of the third edition of Objective C ++ lists the reasons for customizing new/delete:

Detect memory errors in the code
Optimized Performance
Obtain memory usage statistics
These are legitimate requirements. At the end of this article, we will see that: operator new () can achieve the same purpose without heavy load.

: Operator new ().
1. without changing its signature, the original version of the system can be seamlessly replaced, for example:

# Include

Void * operator new (size_t size );

Void operator delete (void * p );

In this way, the user does not need to include any special header file, that is, they do not need to see the two function declarations. This method is usually used for "performance optimization.

2. Add new parameters. These additional parameters are also provided during the call, for example:

Void * operator new (size_t size, const char * file, int line); // The returned pointer must be released by the common: operator delete (void *)

Void operator delete (void * p, const char * file, int line); // This function is called only when the Destructor throws an exception.

And then

Foo * p = new (_ FILE, _ LINE _) Foo; // This can be used to track the FILE and the memory allocated by the Code.

We can also use macros to replace new to save typing. The second method is used for reload. the user needs to see the two function declarations, that is, to actively include the header file you provided. This method is usually used for "Checking memory errors" and "counting memory usage. Of course, this is not absolute.

At the C ++ stage, everyone can write a 100 or 200-line program to verify the statement in the textbook. The reload is: operator new () it will not cause any trouble in such a toy program.

However, in actual product development, reload: operator new () is the best strategy. We have a simpler and safer way to achieve the above goals.

Realistic Development Environment
As a C ++ application developer, we usually use some libraries when writing programs of a relatively large scale. We can classify the library providers into the following categories:

The C-language standard library also includes Posix series functions provided by the Linux programming environment.
Third-party C language library, such as OpenSSL.
The standard library of C ++ language, mainly STL. (I think no one is using IOStream in the product, right ?)
Third-party General C ++ libraries, such as Boost. Regex or an XML library.
Other teams in the company develop internal infrastructure C ++ libraries, such as network communication and log infrastructure.
This project team's colleagues developed their own basic libraries for this application, such as a three-dimensional model's affine transformation module.
When using these libraries, it is inevitable to exchange data between libraries. For example, the output of library A is used as the input of library B, while the output of library A often uses the dynamically allocated memory (such as std: vector ).

If all C ++ libraries use the same memory distributor (new/delete by default), it is very convenient to release the memory, and you can just release it directly to delete. If this is not the case, you should always remember "which distributor does the memory belong to? Is this system default? Is it customized? Do not make a mistake when releasing it ".

(C library usually uses malloc/free to allocate and release memory by default, because C language does not mention so much customization as C ++ does, there is no "memory error" mentioned above. Some may consider that a more comprehensive C library will allow you to register two functions for internal allocation and release of memory. This gives you full control over the memory usage of the library. This method of dependency injection becomes fancy and useless in C ++. See allocator in the C ++ standard library written by Chen Shuo.)

However, if you reload: operator new (), it may not be that simple.

Overload: operator new () dilemma
First, reload: operator new () does not bring any trouble to the C language library. Of course, reload it to get three benefits that cannot be enjoyed by the C language library.

Only the C ++ library and C ++ main programs are considered below.

Rule 1: Never reload in the library: operator new ()
If you are the author of a library and your library must be provided to others, you do not have the permission to reload the Global: operator new (size_t) (note that this is the first overload method mentioned above), because it is very aggressive: any program that uses your library is forced to use your overloaded: operator new (), others may be reluctant to do so. In addition, if both libraries attempt to reload: operator new (size_t), they will fight. I guess duplicated symbol link error will occur. As the creator of the library, do not reload: operator new (size_t.

What about the second overload method? First, the void * pointer obtained by: operator new (size_t size, const char * file, int line) must be: operator delete (void *) and :: operator delete (void * p, const char * file, int line) functions are released. At this time, you need to decide whether the pointer returned by: operator new (size_t size, const char * file, int line) is compatible with the system's default: operator delete (void *).

If it is incompatible (that is, it cannot use the default: operator delete (void *) to release the memory), you must reload: operator delete (void *), make its behavior match your operator new (size_t size, const char * file, int line. Once you decide to reload: operator delete (void *), you must reload: operator new (size_t), which returns to situation 1: You do not have permission to reload the Global :: operator new (size_t ).
If you select the default compatible system: operator delete (void *), you can do very limited things in operator new (size_t size, const char * file, int line, for example, you cannot dynamically allocate additional memory for house keeping or save statistical data (whether explicit or implicit), because the default: operator delete (void *) will not release your extra memory. (Implicit memory allocation here refers to adding elements to a container like std: map <> .)
It is estimated that many people are dizzy, but this is not complete yet.

Second, reload operator new (size_t size, const char * file, int line) in the library also involves your overload and whether to expose it to the library's users (other libraries or main programs ). Here, "exposure" has two meanings: 1) Will the code containing your header file be reloaded with: operator new (), 2): operator new () after reload () can the allocated memory be safely released outside your library. If not, do you want to expose an interface function so that the user can safely release the memory? Or is shared_ptr returned to use its "capture" deleter feature? It sounds complicated? I will not discuss it one by one here. In short, as the author of the library, never use the idea of "reload operator new.

Fact 2: overloading in the main program: operator new () has little effect
This is not a rule, but an attempt to demonstrate that it does not make much sense.

If the first method is used to reload the Global: operator new (size_t), it will affect all C ++ libraries used in this program. This may not cause any problems, however, I suggest you use the simpler "alternative" described in the next section ".

If you use the second method to overload: operator new (size_t size, const char * file, int line), will your behavior benefit other C ++ libraries used in this program? For example, do you want to count the memory usage in C ++ library? If a library returns its own memory and objects allocated with new, so that you can release them after use, are you sure you want to check the memory released due to errors?

C ++ library has two types of Code Organization: 1) it is provided as a header file (such as a template library represented by STL and Boost); 2) provided by header files + binary library files (most non-template libraries are released in this way ).

For libraries implemented in the header file mode, you can. the first line of the cpp file contains the header file of the overload: operator new, so that other C ++ l files used in the program