A cross-platform C + + memory leak detector

March 01, 2004

memory leaks can also count as a permanent topic for C + + programmers. Under Windows, one of the most useful features of MFC is the ability to report a memory leak at the end of a program's run. Under Linux, there is relatively no easy-to-use solution: Existing tools like Mpatrol, usability, additional overhead, and performance are not ideal. This article implements a very easy-to-use, cross-platform, C + + memory leak detector. And the related technical problems are discussed.

Basic use

For a simple program like the following test.cpp:

int main () {int* P1 = new int;char* p2 = new Char[10];return 0;}

Our basic requirement is of course that there are two memory leaks in the program report. To do this, it is very simple, as long as the debug_new.cpp also compiled, link in it. Under Linux, we use:

g++ test.cpp debug_new.cpp-o Test

The output results are as follows:

Leaked object at 0x805e438 (size ten, <unknown>:0) leaked object at 0x805e410 (size 4, <unknown>:0)

If we need a clearer report, it's also easy to add a line at the beginning of test.cpp

#include "Debug_new.h"

Can. The output after adding the line is as follows:

Leaked object at 0x805e438 (size ten, Test.cpp:5) leaked object at 0x805e410 (size 4, test.cpp:4)

Very simple!

Back to top of page

Background knowledge

In the new/delete operation, C + + creates a call to the user for operator new and operator delete. This is the user cannot change. The prototypes for operator new and operator delete are as follows:

void *operator New (size_t) throw (Std::bad_alloc), void *operator new[] (size_t) throw (std::bad_alloc); void operator Delete (void*) throw (), void operator delete[] (void*) throw ();

for "New int", the compiler produces a call to "operator new (sizeof)", and for "new CHAR[10]" The compiler produces "operator new[" (sizeof (char) * 10) " (If new is followed by a class name, of course, the constructor of the class is called). Similarly, for "delete ptr" and "delete[" PTR, the compiler produces a "operator delete (PTR)" Call and a "operator delete[" (PTR) call (if the type of PTR is a pointer to an object, That also calls the object's destructor before operator delete. When the user does not provide these operators, the compilation system automatically provides its definition, and when the user provides these operators themselves, it overwrites the version provided by the compilation system, so as to obtain accurate tracking and control of dynamic memory allocation operations.

At the same time, we can also use the placement new operator to adjust the behavior of operator new. The so-called placement new, refers to the new operator with additional parameters, for example, when we provide a prototype for

void* operator new (size_t size, const char* file, int line);

operator, we can use "new (" Hello ", 123) int" to produce a call to "operator new (sizeof (int)," Hello ", 123)". This can be quite flexible. As another example, the C + + standard requires a compiler to provide a placement new operator that is

void* operator new (size_t size, const std::nothrow_t&);

Where nothrow_t is usually an empty structure (defined as "struct nothrow_t {};") ), whose sole purpose is to provide the compiler with a type that can identify specific calls based on overloaded rules. The user generally simply uses the "New (Std::nothrow) type" (Nothrow is a constant of type nothrow_t) to invoke the placement new operator. It differs from standard new in that new throws an exception when allocating memory fails, and new (Std::nothrow) returns a null pointer when allocating memory fails.

Note that there is no corresponding "delete (Std::nothrow) PTR" syntax, but the latter will refer to another related issue.

For more information about the features of the C + + language, see [Stroustrup1997], especially the 6.2.6, 10.4.11, 15.6, 19.4.5, and b.3.4 sections. These C + + language features are key to understanding this implementation.

Back to top of page

Detection principle

Similar to some other memory leak detection methods, the operator new overload is provided in Debug_new and a macro is used to replace it in the user program. The relevant sections in Debug_new.h are as follows:

void* operator new (size_t size, const char* file, int line); void* operator new[] (size_t size, const char* file, int. line); #define NEW Debug_new#define debug_new new (__file__, __line__)

With the inclusion of Debug_new.h included in the above Test.cpp, "new char[10" will become "new" ("Test.cpp", 4) char[10] "after preprocessing, the compiler will generate a" operator new["( sizeof (char) *, "Test.cpp", 4) "called. By customizing "operator new (size_t, const char*, int)" and "operator delete (void*)" in Debug_new.cpp (as well as "operator new[] ..." and "operator Delete[] ... "; to avoid being cumbersome, the following is not particularly noted, when it comes to operator new and operator delete both contain the array version, I can track all memory allocation calls, and alarms the mismatched new and delete operations on the specified checkpoint. The implementation can be quite simple, using a map to record all allocated memory pointers can be: new when you add a pointer to the map and its corresponding information, delete the pointer and corresponding information; Delete If the pointer does not exist in the map to delete the error When the program exits, there is a memory leak if there is a pointer in the map that is not deleted.

However, this approach does not work if you do not include debug_new.h. Not only that, some files contain debug_new.h, some do not contain debug_new.h are not feasible. Because although we used two different operator new--"operator new (size_t, const char*, int)" and "operator new"--but the available "size_t Delete" There is only one! Using our custom "operator delete", when we delete a pointer assigned by "operator New (size_t)", the program will think it is an illegal pointer! We are in a dilemma: either false positives for this situation, or no alarms for repeated deletion of the same pointer two times: neither is acceptable good behavior.

It seems that the custom global "operator new (size_t)" is also inevitable. In Debug_new, I did this:

void* operator new (size_t size) {return operator new (size, "<Unknown>", 0);}

But the way described earlier to implement a memory leak detector, in some C + + implementations (such as the SGI STL in GCC 2.95.3) works fine, but somehow crashes in other implementations. The reason is not complicated, and SGI STL uses a memory pool to allocate a large amount of memory at a time, thus making it possible to take advantage of the map, but in other implementations it may not do so, adding data to the map calls operator new, and operator new adds data to the map. This creates a dead loop that causes memory to overflow and the application immediately crashes. Therefore, we have to stop using the convenient STL templates and use the data structures that are built manually:

struct new_ptr_list_t{new_ptr_list_t*next;const char*file;intline;size_tsize;};

My initial implementation was to call malloc to allocate more than sizeof (new_ptr_list_t) bytes each time I allocated memory using new, and to string the allocated memory into a single list (using the next field), storing the file name, line number, and object size information separately in the file , line, and Size fields, and then returns (malloc returns the pointer + sizeof (NEW_PTR_LIST_T)). In the delete, the list is searched, if found ((char*) linked list pointer + sizeof (new_ptr_list_t) = = to release the pointer), the list is adjusted, free memory, not found the report delete illegal pointer and abort.

As for automatic detection of memory leaks, my approach is to generate a static global object (based on the lifetime of the C + + object, the constructor of the object is called when the program initializes, and the destructor of the object is called when it exits), and the function that detects the memory leak is called in its destructor. It is also possible for the user to manually invoke the memory leak detection function.

The basic implementation is generally the case.

Back to top of page

Usability improvements

The scheme worked quite well at first, until I started to create a large number of objects. Since each delete needs to be searched in the linked list, the average number of searches is (linked list length/2), and the program is soon as slow as the turtle climbs. Although only for debugging, the speed is too slow is unacceptable. So, I made a small change, changed the new_ptr_list of the head of the list to an array, and the object pointer in which linked list is determined by its hash value. --Users can change the definition of macro debug_new_hash and debug_new_hashtablesize to adjust debug_new behavior. Their current values are a fairly satisfactory definition of my test.

In the use we found that in some special cases (please refer directly to Debug_new.cpp in the Debug_new_filename_len section of the comments), the file name pointer will be invalidated. As a result, the current default behavior of Debug_new duplicates the first 20 characters of the file name, not just the pointer that stores the file name. Also, note that the original new_ptr_list_t has a length of 16 bytes and is now 32 bytes, ensuring that the memory is aligned normally.

In addition, in order to allow the program to work with new (Std::nothrow), I also overloaded operator new (size_t, const std::nothrow_t&) throw (); otherwise, debug_new would think it corresponds to The Delete call to New (nothrow) deletes an illegal pointer. Because the debug_new does not throw an exception (the program directly exits when the memory is low), the operation of this overload is simply calling operator new (size_t). That's no need to say more.

As mentioned earlier, to get an accurate memory leak detection report, you can include "Debug_new.h" at the beginning of the file. My usual practice can be used as a reference:

#ifdef _debug#include "Debug_new.h" #endif

The included location should be as early as possible unless there is a conflict with the system's header file (typically STL's header file). In some cases, you might not want debug_new to redefine new, so you can define debug_new_no_new_redefinition before you include debug_new.h, so you should use Debug_ in your user application. New replaces new (incidentally, you can also use debug_new instead of new when you don't define debug_new_no_new_redefinition). This may be the case in the source file:

#ifdef _debug#define debug_new_no_new_redefinition#include "Debug_new.h" #else # define DEBUG_NEW new#endif

and use debug_new when you need to track memory allocations (consider using global substitution).

The user can choose to define the Debug_new_emulate_malloc so that Debug_new.h uses debug_new and delete to simulate malloc and free operations, which can also be traced in the user program. When using certain compilers (such as digital Mars C + + Compiler 8.29 and Borland C + + Compiler 5.5.1), the user must define No_placement_delete or the compilation will not pass. Users can also use two global booleans to adjust the behavior of the debug_new: New_verbose_flag, the default is False, which is defined as true to display trace information to the standard error output at each new/delete; new_autocheck_ Flag, which is true by default, which automatically calls Check_leaks to check for memory leaks when the program exits, and False if the user must manually call Check_leaks to check for memory leaks.

It is important to note that since the automatic invocation of Check_leaks is a static object destructor in Debug_new.cpp, there is no guarantee that the destructor of the user's global object will occur before the Check_leaks call. For msvc on Windows, I used "#pragma init_seg (lib)" to adjust the order in which object allocations were freed, but unfortunately I didn't know how to do that in some other compilers (especially if I didn't successfully solve the problem in GCC). In order to reduce the false alarm, I take the method that is set New_verbose_flag to True after the automatic call of Check_leaks, so that if the memory leak is reported by mistake, then the delete operation will be printed and displayed. As long as the leak report and the delete report are consistent, we can still tell that no memory leaks have occurred.

Debug_new can also detect errors that repeatedly call Delete (or delete invalid pointer) to the same pointer. The program displays the wrong pointer value and forces call Abort to exit.

Another problem is exception handling. This is worth explaining in a special section.

Back to top of page

Exceptions in constructors

Let's take a look at the following simple program examples:

#include <stdexcept> #include <stdio.h>void* operator new (size_t size, int line) {printf ("Allocate%u bytes on Line%d//n ", size, line); return operator new (size);} Class OBJ {public:obj (int n);p rivate:int _n;};o Bj::obj (int n): _n (n) {if (n = = 0) {throw std::runtime_error ("0 not Allowed");}} int main () {try {obj* p = new (__line__) Obj (0);d elete p;} catch (const std::runtime_error& e) {printf ("Exception:%s//n ", E.what ());}}

See what's wrong with the code? In fact, if we compile with msvc, the compiler's warning message already tells us what happened:

Test.cpp: Warning C4291: ' void *__cdecl operator new (unsigned int,int) ': No matching operator delete found; Memory is not being freed if initialization throws an exception

OK, link the debug_new.cpp in. The results of the operation are as follows:

Ah oh, the memory leaks out no!

Of course, this is not a very common situation. However, as the object becomes more and more complex, who can guarantee that the constructor of a child object of an object or all the functions that an object calls in the constructor will not throw an exception? Also, the solution to this problem is not complicated, but requires the compiler to support the C + + standard Well, allowing the user to define the placement delete operator ([c++1998],5.3.4 section; You can find a 1996-year standard draft, such as the following URL, http:// www.comnets.rwth-aachen.de/doc/c++std/expr.html#expr.new). In the compiler I tested, GCC (2.95.3 or later, linux/windows) and MSVC (6.0 or later) did not have a problem, while Borland C + + Compiler 5.5.1 and digital Mars C + + Compiler (all versions up to v8.38) do not support this feature. In the example above, if the compiler supports it, we need to declare and implement operator delete (void*, int) to reclaim the memory allocated by new. If the compiler does not support it, you need to use a macro to let the compiler ignore the related declarations and implementations. If you want debug_new to compile under Borland C + + Compiler 5.5.1 or digital Mars C + + Compiler, the user must define the macro No_placement_delete; The user has to pay attention to the problem of throwing exceptions in the constructor.

Back to top of page

Scenario Comparison

The IBM developerworks contains a memory leak detection method ([Hong Kun 2003]) that was designed by Mr. Hong Kun to implement Linux. The main differences between my plans are the following:

Advantages:

• Cross-platform: Use only standard functions, and debug through a number of compilers, such as GCC 2.95.3/3.2 (linux/windows), MSVC 6, Digital Mars C + + 8.29, Borland C + + 5.5.1, and more. (Although Linux is my main development platform, I have found that sometimes it is very convenient to compile and run code under Windows.) ）
• Ease of use: due to overloading of operator new (size_t)--Hong Kun only overloads operator new (size_t, const char*, int)--it detects a memory leak even if it does not contain my header file, and automatically detects a memory leak when the program exits ; You can detect memory leaks from Malloc/free in the user program (excluding system/library files).
• Flexibility: There are several flexible configurable items that can be selected using the macro definition for compile time.
• Reentrant: Do not use global variables, no nested delete issues.
• Exception security: In the case of compiler support, you can handle exceptions thrown in constructors without memory leaks.

Disadvantages:

• Single-threaded model: cross-platform multithreading implementation is cumbersome, according to the actual needs of the project, but also for the sake of clarity and simplicity of the code, my scenario is not thread-safe, in other words, if the new or delete operations in multiple threads simultaneously, the consequences are undefined.
• Failed to implement in-run memory leak detection reporting mechanism: I did not encounter this requirement J; However, it is not difficult to call the Check_leaks function implementation manually, but cross-platform is a bit of a problem.
• Mismatched mismatches with [] operators and no [] operators are not detected: it is primarily a requirement problem (it is not difficult to modify the implementation).
• The file name and line number cannot be displayed on the wrong delete call: It should not be a big problem; because I overload operator new (size_t), I can guarantee that the delete error when the program has a problem, so I do not just display the warning message, and will force the program abort, You can know where the problem is by tracking the program and checking the call stack of the program at abort.

In addition, there are already many commercial and open source memory leak detectors, this article does not intend to compare one by one. Compared with them, debug_new is generally weaker in function, but its good usability and cross-platform, low-cost additional cost still have a great advantage.

Back to top of page

Summary and Discussion

The above paragraphs have basically explained the main features of Debug_new. Let's do a little summary below.

Overloaded operators:

• operator new (size_t, const char*, int)
• Operator new[] (size_t, const char*, int)
• operator new (size_t)
• Operator new[] (size_t)
• operator new (size_t, const std::nothrow_t&)
• Operator new[] (size_t, const std::nothrow_t&)
• operator Delete (void*)
• Operator delete[] (void*)
• operator delete (void*, const char*, int)
• Operator delete[] (void*, const char*, int)
• operator delete (void*, const std::nothrow_t&)
• Operator delete[] (void*, const std::nothrow_t&)

The functions provided are:

• Check_leaks ()
  Check for memory leaks

The provided global variables

• New_verbose_flag
  Whether to "wordy" display of information at new and delete
• New_autocheck_flag
  Whether to automatically detect a memory leak when the program exits

Macros that can be redefined:

• No_placement_delete
  Assuming the compiler does not support placement Delete (globally valid)
• Debug_new_no_new_redefinition
  Do not redefine new, assuming the user will use DEBUG_NEW (valid when including debug_new.h)
• Debug_new_emulate_malloc
  Redefine Malloc/free, simulate with new/delete (valid when Debug_new.h is included)
• Debug_new_hash
  Algorithm for changing the hash value of the memory block list (valid when compiling debug_new.cpp)
• Debug_new_hashtable_size
  Change the size of the memory block list hash bucket (valid when compiling debug_new.cpp)
• Debug_new_filename_len
  If the file name is copied when the memory is allocated, the length of the file name is retained, and the debug_new_no_filename_copy is automatically defined at 0 (valid when compiling the debug_new.cpp; see comments in the file)
• Debug_new_no_filename_copy
  When allocating memory, the file name is not copied, but only its pointer is saved; efficient (valid when compiling debug_new.cpp; see comments in file)

I personally believe that one of the main drawbacks of debug_new is that it does not support multithreading. For a particular platform, it is not difficult to join multi-threaded support, which is difficult to generalize (of course, conditional compilation is a method, though not elegant enough). When the C + + standard contains the threading model, the problem may be solved in a more perfect way. Another option is to use a thread wrapper class in a library like boost, but this will increase the dependency on other libraries-after all, boost is not part of the C + + standard. If the project itself is not using boost, it does not seem worthwhile to use another library for this purpose alone. Therefore, I will not do this further improvement for the time being.

Another possible modification is to keep the exception of standard operator new, either to throw an exception (normal) in case of low memory, or to return null (nothrow) instead of terminating the program as it is now (see source code for Debug_new.cpp). The difficulty of this approach lies mainly in the latter: I have not figured out a way to keep the syntax of new (nothrow), to report the file name and line number, and to use normal new. However, if you use Debug_new and Debug_new_nothrow without using standard syntax, it is very easy to implement.

If you have any suggestions for improvement or other ideas, you are welcome to write a discussion.

Debug_new source code can now be downloaded at dbg_new.zip.

After I finished writing this article, I finally implemented a thread-safe version. This version uses a lightweight cross-platform mutex class Fast_mutex (currently supports WIN32 and POSIX threads that can automatically detect thread types by command-line arguments when using gcc (LINUX/MINGW), msvc). If you are interested, you can download it at http://mywebpage.netscape.com/yongweiwu/dbg_new.tgz.

Resources

[c++1998] ISO/IEC 14882. Programming languages-c++, 1st Edition. International standardization Organization, International Electrotechnical Commission, American national standards Institute, and Information Technology Industry Council, 1998

[Stroustrup1997] Bjarne Stroustrup. The C + + programming Language, 3rd Edition. Addison-wesley, 1997

[Hong Kun 2003] Hong Kun. "How to detect memory leaks under Linux", IBM developerWorks China website.

About the author

		Wu Yongwei is currently engaged in the development of high-performance intrusion detection systems on Linux. Has a strong interest in developing cross-platform, high-performance, reusable C + + code. [email protected] can contact him.

http://blog.csdn.net/adcxf/article/details/3824209

A cross-platform C + + memory leak detector