Use tcmalloc for Stack Analysis

Use tcmalloc for Stack Analysis

Author: Echo Chen (Chen Bin)

Email: [email protected]

Blog: blog.csdn.net/chen19870707

Date: October 10 th, 2014

 

In the previous article "Stack check with tcmalloc", we introduced tcmalloc for Stack check. Today we translate heap-profiling using tcmalloc to learn how to perform stack analysis with tcmalloc.

 

I. Usage of stack analysis:

 

This technical document describes how to use a C ++ program to analyze a stack. It can be used to do three things:

 

• Understand the stack of a program at any time
• Locate Memory leakage
• Find the location where a large amount of memory is allocated

1. Link stack Analyzer

 

You can perform stack Analysis on any program linked to tcmalloc without re-compiling.

 

It is safe to link tcmalloc to your program without using the stack analyzer. Your program does not run slowly because you do not use any stack analysis features.

 

You can run the stack check in programs that are not compiled by using ld_preload.

   1: $ LD_PRELOAD="/usr/lib/libtcmalloc.so" HEAPPROFILE=... 

This is not recommended.

 

2. Enable stack check

 

Define the heapprofile environment variable to determine the location of the generated analysis file, for example, in/usr/local/nmetscape:

   1: $ HEAPPROFILE=/tmp/profile /usr/local/netscape           # sh

   2: % setenv HEAPPROFILE /tmp/profile; /usr/local/netscape   # csh

 

Analysis is also effective for sub-processes: each sub-process obtains its own analysis file based on its own name (composed of heapprofile and process ID)

 

For security reasons, the stack check will not write in the file, so for programs with setuid, stack analysis cannot be used.

 

3. decompress the analysis File

 

If Stack analysis is enabled in the heap program, the program will generate the analysis file to the file system. The names of a series of analysis files will be named as follows:

   1: <prefix>.0000.heap

   2: <prefix>.0001.heap

   3: <prefix>.0002.heap

   4: ...

 

<Perfix> is the value defined in heapprofile. Note that if no file path is defined, the file will be directly generated to the current directory of the program.

 

By default, a file is filled with 1 GB and replaced with a new one. The frequency of file writing can be controlled by calling heapprofilersetallocationinterval () in your program. It is concluded that the size of each file is a definite value.

 

You can also call heapprofile to generate an analysis file at a specific location of your program, for example:

   1: extern const char* HeapProfile();

   2: const char* profile = HeapProfile();

   3: fputs(profile, stdout);

   4: free(const_cast<char*>(profile));

 

4. What have you analyzed?

 

This analysis system shows all memory requests and releases. It retains a series of information for each memory allocation. Memory Allocation is defined as an active call in the stack: malloc,calloc,realloc, Or,new.

 

5. parse Profile

 

You can pass the analysis file to the pprof tool to obtain the analysis output,Pprof can print CPU and stack usage. Explanation:

Here are some examples. These examples assume that the binary name is gfs_master, and the names of a series of stack analysis files are as follows:

profile.0001.heapprofile.0002.heap...profile.0100.heap

 

6. Why is the process so large?

 

% pprof --gv gfs_master profile.0100.heap

This command will pop up a window showing the analysis information, as shown in the following example:

 

 

Some explanations:

• Gfs_masterchunk: addserver consumes 25% MB of memory, and the active memory is.
• Gfs_masterchunktable: updatestate consumes 792 MB of active memory. In addition, it consumes MB for debugging. The output edge label shows the memory occupied by each caller.

7. Comparison Analysis File

 

You often want to skip the program's memory allocation during initialization to find memory leaks. A simple way to achieve this is by comparing two analysis files, both of which have been running from the beginning of the program for a while. Use—baseOption to specify the first file, for example:

% pprof --base=profile.0004.heap gfs_master profile.0100.heap

 

The memory usage in profile.0004.heap will be seen in profile0000100.heap and the results will be displayed.

 

8. Text output

 

% pprof gfs_master profile.0100.heap   255.6  24.7%  24.7%    255.6  24.7% GFS_MasterChunk::AddServer   184.6  17.8%  42.5%    298.8  28.8% GFS_MasterChunkTable::Create   176.2  17.0%  59.5%    729.9  70.5% GFS_MasterChunkTable::UpdateState   169.8  16.4%  75.9%    169.8  16.4% PendingClone::PendingClone    76.3   7.4%  83.3%     76.3   7.4% __default_alloc_template::_S_chunk_alloc    49.5   4.8%  88.0%     49.5   4.8% hashtable::resize   ...

 

• The first column contains direct memory usage, in MB
• The fourth column contains the memory usage of the module it calls.
• Percentage of the first and fourth columns in the second and fifth columns.
• The sum of elements before the row in the third column

9. ignore or focus on a specific area

 

The following command will display the calling graph. It only contains the paths that contain the databuffer expression in the call graph:

 

% pprof --gv --focus=DataBuffer gfs_master profile.0100.heap

Similarly, the following command ignores all paths. All expressions matching databuffer will be ignored:

% pprof --gv --ignore=DataBuffer gfs_master profile.0100.heap

 

10 all memory allocation + Object Information

 

All the previous examples show how to use the space, for example, the number of allocated but not released. To obtain other information, use the following mark:

--inuse_space	Display the number of in-use megabytes (I. e. space that has been allocated but not freed). This is the default.
--inuse_objects	Display the number of in-use objects (I. e. Number of objects that have been allocated but not freed ).
--alloc_space	Display the number of allocated megabytes. This parameter des the space that has since been de-allocated. Use this if you want to find the main allocation sites in the program.
--alloc_objects	Display the number of allocated objects. This parameter des the objects that have since been de-allocated. Use this if you want to find the main allocation sites in the program.

 

11. Notes

• Libcmalloc is required for Stack analysis.
• How can a program link to a library with sufficient symbolic information? All associated samples will be taken care of by the symbolic information found before the library, this reduces the number of symbols.
• If you run a program on one machine, analyze it on another machine, and the databases shared by the two machines are different, the analysis output may be inaccurate.
• Some libraries, such as STL implementation, are managed by their own memory. This caused a strange analysis. You must also use tcmalloc In the STL library. So it is only effective for a few STL implementations.

 

-

Echo Chen: blog.csdn.net/chen19870707

-

Use tcmalloc for Stack Analysis