Using Tcmalloc for stack analysis

In the previous translation, "Stack check with Tcmalloc", we introduced tcmalloc for stack checking and today translated "heap-profiling using Tcmalloc" to learn how to Tcmalloc stack analysis.

First, the use of stack analysis:

This technical document describes how to use a C + + program to parse a stack. Can be used to do three things:

• Understand the stack of a program at any time
• Locating memory leaks
• Location where a large number of memory allocations are found

1. Link Stack Analyzer

You can perform a stack analysis on any program that has links to Tcmalloc, and you do not need to recompile.

Linking Tcmalloc to your program is also safe if you don't want to use a stack analyzer to check it. Your program does not run any more slowly because you do not have to use any of the features of the stack analysis.

You can run a stack check through ld_preload in programs that are not compiled by you.

We do not recommend this use.

2. Turn on stack checking

Define the HEAPPROFILE environment variable to determine where the profiling file is generated, such as generated in/usr/local/nmetscape:

   1: $ heapprofile=/tmp/profile/usr/local/netscape # sh

   2:% setenv heapprofile/tmp/profile; /usr/local/netscape # CSH

Parsing a child process is also valid: Each subprocess gets its own parsing file (consisting of heapprofile and process ID) according to its own name

For security reasons, the stack check will not be written in the file, so the stack analysis is not available for setuid programs.

3. Unzip the analysis file

If stack analysis is turned on in the heap program, the program will generate the analysis file to the file system, and the name of the series of analysis files will be named as follows:

   1: <prefix>.0000.heap

   2: <prefix>.0001.heap

   3: <prefix>.0002.heap

   4: ...

<perfix> is the value defined in Heapprofile. Note that if there is no path to the defined file, the file will be generated directly into the program's current directory.

By default, a file is written full of 1GB for a new file. The frequency of writing files can be controlled by calling Heapprofilersetallocationinterval () in your program. It is concluded that the size of each file is a definite value.

You can also call Heapprofile to generate an analysis file at a specific location in your program, for example:

   1:extern Const char* heapprofile ();

   2:const char* profile = Heapprofile ();

   3:fputs (profile, stdout);

   4:free (const_cast<char*> (profile));

4. What was analyzed

This analysis system illustrates all memory requests and releases. It retains a series of information about each memory allocation. Memory allocations are defined as active calls within the stack: malloc,, calloc realloc , or, new .

5. Parsing profile

The analysis output can be obtained by passing the analysis file to the Pprof tool.pprof工具可以打印CPU和堆栈使用情况。解释如下：

这里是一些例子，这些例子假设二进制名字为gfs_master，一系列的堆栈分析文件名字如下：

Profile.0001.heapprofile.0002.heap...profile.0100.heap

6. Why the process is so large

% pprof--GV Gfs_master profile.0100.heap

This command will pop up a window showing the analysis information, as an example:

Some explanations:

• Gfs_masterchunk::addserver consumes 256M of memory, active memory is 25%
• Gfs_masterchunktable:updatestate consumes 176.2MB of active memory, in addition, it consumes 792MB by it. The memory of each callee is given on the label of the output edge.

7. Compare and analyze Files

You often want to skip the memory allocations in the initialization phase of the program to find a memory leak. An easy way to achieve this is by comparing the two analysis files, which are all two files from the beginning of the program to the run for a while. Use —base option to specify the first file, for example:

% pprof--base=profile.0004.heap Gfs_master profile.0100.heap

Memory usage in PROFILE.0004.HEAP will see the memory usage in PROFILE.0100.HEAP and show the results.

8. Text output

% pprof gfs_master profile.0100.heap 255.6 24.7% 24.7% 255.6 24.7% gfs_masterchunk::addserver 184.6 17.8% 42.5 % 298.8 28.8% gfs_masterchunktable::create 176.2 17.0% 59.5% 729.9 70.5% gfs_masterchunktable::updatestate 1 69.8 16.4% 75.9% 169.8 16.4% pendingclone::P endingclone 76.3 7.4% 83.3% 76.3 7.4% __default_alloc_templa Te::_s_chunk_alloc 49.5 4.8% 88.0% 49.5 4.8% hashtable::resize ...

• The first column contains the direct memory usage, in megabytes
• The fourth column contains the memory usage of the module it calls
• The second and fifth columns are the percentages of columns one and fourth.
• The third column contains the second row before the sum of the elements

9. Ignore or focus to a specific area

The command below will give the graphical display of the call, containing only those paths that contain the DataBuffer expression in the call graph:

% pprof--GV--focus=databuffer gfs_master profile.0100.heap

Similarly, the following command ignores all paths. All of the expressions that match the DataBuffer will be ignored:

% pprof--GV--ignore=databuffer gfs_master profile.0100.heap

10 All memory allocations + object information

All previous examples have shown how to display space usage, such as those that have been allocated but not released. Other information can be obtained using the following flags:

--inuse_space	Display the number of in-use megabytes (i.e. space) has been allocated and not freed). This is the default.
--inuse_objects	Display the number of In-use objects (i.e. number of objects that has been allocated but not freed).
--alloc_space	Display the number of allocated megabytes. This includes the space and has since been de-allocated. Use this if you want to find the main allocation sites in the program.
--alloc_objects	Display the number of allocated objects. This includes the objects, which has since been de-allocated. Use this if you want to find the main allocation sites in the program.

11. Precautions

• Stack analysis requires the use of Libcmalloc
• How the program links a library of sufficient symbolic information, all related samples will be responsible for the symbolic information that was discovered before the library, which creatively reduces the number of symbols.
• If you run the program on one machine, analyze it on another machine, and the two machines share a different library, the analysis output may not be allowed.
• Some libraries, such as STL implementations, are managed by their own memory. This time caused the analysis to be strange. You must also use Tcmalloc in the STL library. So the placement is only valid for a few STL implementations.

Using Tcmalloc for stack analysis