Android debugging tool-mat

Android debugging tool-mat

I. Overview

For large Java applications, it is difficult to block all the vulnerabilities even if we perform a lot of effective work in the test phase, many problems will still be exposed in the production environment, and it is difficult to reproduce in the testing environment. The JVM can record part of the running status of the system when a problem occurs and store it in the heap dump file, which provides an important basis for us to analyze and diagnose the problem.

Generally, Memory Leak analysis is considered a very difficult task, which is generally conducted by senior personnel in the team. However, the mat (eclipse memory analyzer) we are going to introduce today is considered to be a "silly" heap dump file analysis tool, you only need to click to generate a professional analysis report. Compared with other memory leak analysis tools, mat is easy to use and can be easily used with one click. Even new users can use it quickly.

MAT is so easy to use. Are you also very interested to feel it yourself? The first step is to install mat first.

Ii. Installation

A memory analysis tool provided by eclipse. It is a function-rich Java heap dump file analysis tool that helps you discover memory vulnerabilities and reduce memory consumption.
Address: http://www.eclipse.org/mat

 

(If you are using motodev studio for Android, this tool is provided by default)
You can choose to install the original plug-in provided by eclipse or use the motodevstudio for Android plug-in

1. Original plug-in
1) Start eclipse and click Help-> intall new software ...;
2) Click the arrow on the right of the input column and select "Galileo" (my version is version 3.5. For different versions, find the corresponding version name );
3) wait until the list is updated. Find and expand general purpose tools ";
4) Select and download "Memory Analyser (incubation)" and "Memory Analyser (charts.

After installing the plug-in, you need to restart the eclipse working platform.

In comparison, the installation method of the standalone version is very simple. You only need to download the corresponding installation package and decompress it to run it. This is also a widely used installation method. In the following example, we use the single-host mat. For specific download requirements and addresses, see the product download page: http://www.eclipse.org/mat/downloads.php.

In addition, if you need to use mat to analyze the dump file generated by ibm jvm, you also need to install the IBM diagnostic toolframework. For detailed download and installation configuration steps, see: http://www.ibm.com/developerworks/java/jdk/tools/dtfj.html

2. motodev studio for Android plug-in
1) Start eclipse and click Help-> intall newsoftware ...;
2). Create an update through add (the latest address can be obtained here ):
Name: motodev studio for Android plugin
Location: https://studio-android.motodevupdate.com/android/2.0
3). Download and install the update.
* If it is mainly for Android development, we recommend that you use the motodevstudio for Android integrated plug-in, which provides many convenient tools. However, if you have installed ADT, you must manually uninstall it before installation, for more information, see install motodev studio as a plug-in.

 

3. Why mat?

During normal work, sometimes outofmemoryerror occurs. We know that an error usually indicates that the program has a serious problem, which may be disastrous. It is very important to find out the cause of outofmemoryerror. Previously, I thought that using tools such as real-time profiling/monitoring, it is correct to analyze the memory leakage in a very real-time manner. At the beginning of this year, a profiler tool was used to test the memory leakage in message-oriented middleware. It was a headache that the profiler tool could not respond to itself when the throughput was high. Later I learned that such a tool will consume performance and cannot be leaked under certain conditions. Therefore, it is very important to analyze offline data. mat is such a tool.

 

4. Why is memory overflow?

We know that JVM performs GC Based on Generation (generation). As shown in, it is divided into younggeneration (young generation), tenuredgeneration (old generation), permanentgeneration (permanent generation, Perm Gen ), permgen (or non-heap) is a different type. Note that heap space does not include perm Gen.

The vast majority of objects are allocated in young generation, and also in younggeneration. When the space of young generation is filled up, GC will perform minor collection (secondary collection ), this collection does not involve other generations in heap. The minor collection process assumes that a large number of objects in younggeneration are garbage collection based on weak generationalhypothesis (assumed in the weak age), and the minor collection process will be very fast. Young
The unrecycled objects in generation are transferred to tenuredgeneration. However, tenured generation is also filled, and major collection is triggered. This collection is targeted at the entire heap because a large number of objects are involved, so it is much slower than minor collection.

 

JVM has three garbage collectors: throughput collector for parallel young generation collection, which are started by the parameter-XX: + useparallelgc and concurrent low pausecollector for tenured generation concurrent collection, started by the parameter-XX: + useconcmarksweepgc; Incremental lowpause collector, which can be considered as the default garbage collector. It is not recommended to directly use a garbage collector. It is best to let the JVM decide on its own unless you are sure.

 

How are the generation spaces in heap divided? The maximum heap space can be specified through the-xmx = n parameter of JVM, while-XMS = n indicates the minimum heap space. During JVM initialization, if the minimum heap space is smaller than the maximum heap space, as shown in figure, the JVM will mark unused space as virtual. In addition, the two parameters-XX: minheapfreeratio = N and-XX: maxheapfreeratio = N are used to control the ratio of the maximum and minimum spaces to the active objects respectively. The default value is as follows:
In XP, the default value is similar.

 

Parameters	Default Value
Minheapfreeratio	40
Maxheapfreeratio	70
-XMS	3670 K
-Xmx	64 m

 

Because the majorcollection of tenured generation is slow, if the tenured generation space is smaller than young generation, frequent major collection will occur, affecting efficiency. The default ratio of young generation and tenured generation space in server JVM is. That is to say, the sum of the Eden and memory vor space of young generation is 1/3 of the total heap (excluding perm Gen, the ratio can be controlled by the-XX: newratio = n parameter, while the client
Default JVM-XX: newratio is 8. The newsize = N and maxnewsize = N parameters for adjusting the size of young generation space will not be discussed. Please refer to the following documents.

 

The surviving objects in young generation are transferred to tenured generation, but unfortunately the concurrent collector thread executes the major collection here, and the space is exhausted before the recycle task ends, at this time, full collections (full GC) will occur, and the entire application will stop until the collection is completed. Full GC is a nightmare in a high-load production environment ......

For now, the heterogeneous perm gen is used by JVM to store objects that cannot be described in the Java language. These objects are class and method data (related to Class Loader) and interned strings (string resident ). Generally, the 32-bit OS perm Gen defaults to 64 m, which can be specified by the parameter-XX: maxpermsize = n. In the JVM memory structure article, there is no more detailed document on this region, mysterious.

 

Back to the question "why is memory overflow ?".

To answer this question, we need to raise another question. What kind of object GC will be recycled? Of course, when GC finds that an object cannot be accessed through any reference chain, the object is recycled. The term GC roots is the starting point for analyzing this process. For example, the JVM ensures that the object is reachable (then the JVM is the GC roots ), therefore, GC roots keeps the object reachable in the memory. Once the object fails to arrive, it is recycled. Generally, GC roots is an object (such as method parameters and local variables) on the call stack of the current thread, or the thread itself or the system
Classes loaded by class loader and the active objects retained by native code. Therefore, GC roots is a powerful tool for analyzing why the object remains in the memory. After knowing what type of object GC will be recycled, let's learn what object references contain.

 

From the strongest to the weakest, different reference (reachable) levels reflect the object lifecycle.

L Strong REF (strong reference): Generally, the code we write is strong ref, which corresponds to strong accessibility. objects are recycled only when strong accessibility is removed.

L soft REF (soft reference): corresponding to soft accessibility, objects are kept as long as there is enough memory until it is found that the memory is tight and there is no strong ref. Generally, cache can be implemented through the java. Lang. Ref. softreference class.

L weak REF (weak reference): It is weaker than softref. When strong ref does not exist, the objects are immediately recycled instead of waiting for the memory to be tight. Implemented through java. Lang. Ref. weakreference and Java. util. weakhashmap classes.

L phantom REF (Virtual Reference): it does not keep any objects in the memory. You can only use phantom ref itself. It is generally used to perform a special cleaning process after entering the finalize () method, through java. Lang. Ref. phantomreference.

 

With all the above, I believe it is easy to break through heap and perm gen. Yes, it uses strong ref to store a large amount of data until heap breaks through; use interned strings (or class loader to load a large number of classes) to break perm Gen.

 

About shallow size and retained size

 

Shallow size is the size of the memory occupied by the object itself. It does not include references to other objects, that is, the sum of the object header and member variables (not the value of member variables. In 32-bit systems, the object header occupies 8 bytes, And the int occupies 4 bytes, regardless of whether the member variables (objects or arrays) reference other objects (instances) or if the value is null, it always occupies 4 bytes. Therefore, for a String object instance, it has three int members (3*4 = 12 bytes) and one char [] member (1*4 = 4 bytes) and an object header (8 bytes), a total of 3*4 + 1*4 + 8 = 24 bytes. According to this principle, for string a = "rosenjiang", the shallow of instance
The size is also 24 bytes (many people have disputes over this. Please check and leave a message to me ).

 

Retained size is the shallow size of the object, plus the sum of the shallow size from which the object can be accessed directly or indirectly. In other words, retained size is the total amount of memory that can be recycled after the object is GC. To better understand the retained size, let's look at an example.

 

The objects in the memory are considered as nodes, and the objects and objects are referenced from each other. Here is a special node GC roots! This is the starting point of reference chain.

Starting with obj1, the blue nodes represent objects that can be accessed directly or indirectly only through obj1. Because it can be accessed through GC roots, the obj3 in the left graph is not a blue node, but the right graph is blue because it has been included in the retained set.

Therefore, for the left image, the retained size of obj1 is the total shallow size of obj1, obj2, and obj4. The retained size of the right image is the total shallow size of obj1, obj2, obj3, and obj4. The retained size of obj2 can be calculated in the same way.

 

Heap dump

 

Heap dump is a memory snapshot of a Java Process at a specific time point. There are different formats for storing the data, which generally include the situation where the Java object and class are in heap when the snapshot is triggered. Because snapshot is only an instant task, heap dump cannot contain such information as when and where an object is allocated.

 

You can obtain heap dump in different platforms and different Java versions, while mat requires heapdump binary files in hprof format. To avoid manual intervention, configure the JVM parameter-XX:-heapdumponoutofmemoryerror as follows. When an error occurs, heap dump is automatically generated. This method is only used in the production environment. If you want to control when to generate heapdump, you can use the jconsole tool in Windows + jdk6, and use the jmap tool in Linux or Mac OS X. Of course, you can also configure the JVM parameter-XX: + heapdumponctrlbreak, that is, use the CTRL + break key on the console to generate heap
Dump. Because I am windows + jdk5, I chose-XX:-heapdumponoutofmemoryerror. For more configuration, see mat wiki.

 

References

Mat Wiki

Interned strings

Strong, soft, weak, phantom reference

Tuning garbagecollection with the 5.0 Java [Tm] Virtual Machine

Permanent generation

Understandingweak references Translation

Javahotspot VM options

Shallow and retained sizes

JVM Memory Structure

GC roots

 

5.Prepare heap dump

Please refer to the pilot class below, which is not special.

/**
* Pilot class
* @ Author Rosen Jiang
*/
Package org. rosenjiang. Bo;

Public class pilot {

String name;
Int age;

Public pilot (string a, int B ){
Name =;
Age = B;
}
}

Then let's look at the oomheaptest class, how it breaks through the heap dump.

/**
* Oomheaptest class
* @ Author Rosen Jiang
*/
Package org. rosenjiang. test;

Import java. util. date;
Import java. util. hashmap;
Import java. util. Map;
Import org. rosenjiang. Bo. Pilot;

Public class oomheaptest {
Public static void main (string [] ARGs ){
OOM ();
}

Private Static void OOM (){
Map <string, pilot> map = new hashmap <string, pilot> ();
Object [] array = new object [1000000];
For (INT I = 0; I <1000000; I ++ ){
String d = new date (). tostring ();
Pilot P = new pilot (D, I );
Map. Put (I + "Rosen Jiang", P );
Array [I] = P;
}
}
}

Yes. Many pilot class instances are constructed above and put in arrays and maps. Because it is strong ref, GC naturally does not recycle these objects and keeps them in heap until the overflow occurs. Of course, you must configure the VM parameter-XX: + heapdumponoutofmemoryerror in eclipse before running. Okay. After a while, the memory overflows and the console displays the following information.

Java. Lang. outofmemoryerror: Java heap Space
Dumping heap to java_pid3600.hprof
Heap Dump File Created[78233961 bytes in 1.995 secs]
Exception in thread "Main" Java. Lang. outofmemoryerror: Java heap Space

Java_pid3600.hprof is both heap dump and can be found in the project root directory where the oomheaptest class is located.

Mat Installation
In two cases, You have to install mat with heap dump. You can choose the appropriate installation method at http://www.eclipse.org/mat/downloads.php. After the installation is complete, switch to the memory analyzer view. There is the open heap dump button in the upper left corner of eclipse. Find and open the java_pid3600.hprof file in the path just described. It takes some time to parse the hprof file, and a wizard will pop up. Simply finish. You will see the interface shown later.

The mat tool analyzes heap dump and intuitively shows a pie chart on the interface. The dark area of this figure is suspected to have memory leakage. The entire heap memory is only 64 MB, the dark area accounts for 99.5%. Next is a brief description, which tells us that the main thread occupies a large amount of memory and clearly states that the "Java. lang. thread "the instance has memory aggregation, and the keyword" Java. lang. thread. Therefore, the mat illustrates the problem in two simple sentences, even if the user has no experience in dealing with memory problems. There is also a "details" link below. Before you click it, consider the following question: Why is the object instance clustered in the memory? Why is it alive (not GC )? Yes -- strong
Ref, so try again.

After clicking the "details" link, in addition to the description displayed on the previous page, there are also the shortest paths to the accumulation point and accumulated objects sections, which describe the shortest path from GC root to clustering point, and complete reference chain. Observe the accumulated Objects section, Java. util. hashmap and Java. lang. the retained heap (size) of the object [1000000] instance is the largest. in the previous article, we know that retained heap indicates that
The shallow heap (size) sum of other class instances that can be collected by chain, so it is obvious that class instances are clustered in the hashmap and object arrays. Here we find an interesting phenomenon, that is, the shallow heap and retained heap of the object array are the same. According to the Article shallow and retained sizes, we can see that the shallow heap of the array and the general object (non-array) different, depending on the length of the array and the type of the elements in it, evaluate shallow for the Array
Heap, that is, the sum of shallow heap of all objects in the array set. Okay. Let's take a look at Org. rosenjiang. bo. why is the shallow heap of the pilot object instance 16? Because the object header is 8 bytes, the member variable Int Is 4 bytes, and the string reference is 4 bytes, the total length is 16 bytes.

Next, let's look at the accumulated objects by class area. As the name suggests, we can find the Class Name of the clustered object instance. Org. rosenjiang. Bo. Pilot class toutiao.com was instantiated for 290,325 times and then returned to see the program. I admit that I did it on purpose. There are also many useful reports to help you analyze problems, but the examples in this article are too simple to use. If it is useful in the future, you must write a detailed description in the article.

 

Refer:

Http://www.ibm.com/developerworks/cn/opensource/os-cn-ecl-ma/index.html

Http://www.blogjava.net/rosen/archive/2010/06/13/323522.html