An example of analyzing and resolving Java memory leaks __java

These days, has been in the Java "memory leak" problem entangled. The memory consumed by Java applications continues to rise regularly, exceeding the monitoring threshold. Sherlock Holmes had to take a shot.

Memory overflow out of memory, which means that the program does not have enough memory space for it to use when applying for memory, and it appears out of the memory; For example, an integer is applied, but the number that has long to save is the memory overflow.

Memory leak memory leak, refers to the program in the application of memory, can not release the requested memory space, a memory leakage hazard can be ignored, but the memory leak accumulation consequences are very serious, no matter how much memory, sooner or later will be taken up by the light.

Memory leak will eventually lead out of the memory.

Memory overflow is that you ask to allocate more memory than the system can give you, the system does not meet the requirements, resulting in overflow.

A memory leak is when you apply to the system to allocate memory for use (new), but after the use is not returned (delete), the result of the piece of memory you apply to you can no longer access (perhaps you have lost its address), and the system can not assign it to the required program. A dish can only hold 4 fruits in every way, and you have 5, so you can't eat it on the ground. This is the overflow. For example, stack, stack when you do into the stack must produce space overflow, called overflow, stack empty when the stack also produces space overflow, called underflow.      Is that the allocated memory is not enough to drop the sequence of data items, called a memory overflow. From the user's point of view of the program, the memory leak itself will not produce any harm, as a general user, do not feel the existence of memory leaks. What really harms is the accumulation of memory leaks, which ultimately consumes all the memory of the system. In this sense, a one-time memory leak is harmless because it does not accumulate, and the implicit memory leak is very harmful because it is more difficult to detect than frequent and accidental memory leaks
General steps for analyzing memory leaks

If you find evidence that the memory used by Java applications is leaking, then we typically use the following steps to analyze

00001. Dump the heap used by the Java application

00002. Use the Java Heap Analysis tool to identify suspects in memory usage that exceeds expectations (typically because of too many)

00003. When necessary, the reference relationship between the suspect and other objects needs to be analyzed.

00004. View the source code of the program to find out the reason for the excessive number of suspected objects. Dump Heap

If there is a memory leak in the Java application, do not worry about killing the application, but save the site. If it is an Internet application, you can cut traffic to another server. The purpose of saving the site is to dump the heap of the running JVM.

JDK has its own jmap tool that can do this thing. It is implemented in the following ways:

Java code

00001. Jmap-dump:format=b,file=heap.bin <pid>

The meaning of format=b is that the dump comes out of the file in binary format.

The meaning of File-heap.bin is that the file name of the dump is heap.bin.

<pid> is the process number of the JVM.

(under Linux) perform PS aux first | grep Java, find the JVM's PID, then execute Jmap-dump:format=b,file=heap.bin <pid>, and get heap dump file. Analyze Heap

The binary heap dump file is parsed into human-readable information, which is naturally required with the help of a professional tool, which is recommended here memory Analyzer.

Memory Analyzer, referred to as Mat, is an open-source project of the Eclipse Foundation, donated by SAP and IBM. The software produced by giant companies is still very good, mat can analyze the heap of hundreds of millions of-level objects, quickly calculate the memory size of each object, the reference relationship between objects, and detect the suspect of memory leak, powerful and user-friendly.

The Mat interface is based on Eclipse Development and is published in two forms: the Eclipse plug-in and the Eclipe RCP. Mat's analysis results are provided in the form of pictures and statements, at a glance. In a word, the individual still likes this tool very much. Here are two official screenshots:

To start with, I opened the heap.bin with Mat, it is easy to see, char[] number out of its expectation, occupy more than 90% of the memory. In general, char[] does take up a lot of memory in the JVM, as well as a very large number, because string objects are char[as internal storage. But this time the char[] too greedy, careful observation, found that there are tens of thousands of char[], each occupies hundreds of K of memory. This phenomenon indicates that the Java program holds tens of thousands of large string objects. With the logic of the program, this is not supposed to be, there must be a problem somewhere.

Shuntengmogua

In suspicious char[], select one arbitrarily, using the path to GC root function, find the reference path to the char[, and find that the string object is referenced by a hashmap. This is also expected to happen, the Java memory leak is mostly because the object is left in the global HashMap can not be released. However, the HashMap is used as a cache, setting the threshold of the cache entry, which is automatically eliminated when the threshold is reached. From this logical analysis, there should be no memory leaks. Although the string object in the cache has reached tens of thousands of, it still does not have a pre-set threshold (the threshold setting is large because the estimated string object is smaller).

However, another question caught my attention: Why the cached string object is so large. The length of the internal char[] is up to hundreds of K. Although the number of string objects in the cache has not yet reached the threshold, the string object is much larger than we expected, resulting in a significant amount of memory consumption, which is a sign of memory leaks (and, to be precise, excessive memory consumption).

Take a further look at this question and see how the string object was put into the hashmap. By looking at the source code of the program, I found that there was a big string object, but instead of putting a large string object in the HashMap, I split the large string object (calling the String.Split method) and then split out The string small object came into the HashMap.

This is strange, put in the HashMap is obviously split after the string small object, how can occupy so much space. Is there a problem with the split method of the String class?

View Code

With that in view, I looked at the code for the string class in Sun JDK6, mainly the implementation of the Split method:

Java code

00001. Public

00002. string[] Split (String regex, int limit) {

00003. Return Pattern.compile (a regex). Split (this, limit);

00004.}

As you can see, the Stirng.split method calls the Pattern.split method. Continue to see the code for the Pattern.split method:

Java code

00001. Public

00002. string[] Split (charsequence input, int limit) {

00003. int index = 0;

00004. Boolean matchlimited = limit > 0;

00005. arraylist<string> matchlist = new

00006. arraylist<string> ();

00007. Matcher m = Matcher (input);

00008.//ADD segments before each match found

00009. while (M.find ()) {

00010. if (!matchlimited | | matchlist.size () < limit-1) {

00011. String match = input.subsequence (index,

00012. M.start ()). ToString ();

00013. Matchlist.add (Match);

00014. Index = M.end ();

00015.} else if (matchlist.size () = limit-1) {//last one

00016. String match = input.subsequence (index,

00017.

00018. Input.length ()). ToString ();

00019. Matchlist.add (Match);

00020. Index = M.end ();

00021.}

00022.}

00023.//If No match is found, return this

00024. if (index = = 0)

00025. return new string[] {input.tostring ()};

00026.//ADD remaining segment

00027. if (!matchlimited | | matchlist.size () < limit)

00028. Matchlist.add (input.subsequence Index,

00029. Input.length ()). ToString ());

00030.//Construct result

00031. int resultsize = Matchlist.size ();

00032. if (limit = 0)

00033. while (resultsize > 0 &&

00034. Matchlist.get (resultSize-1). Equals (""))

00035. resultsize--;

00036. string[] result = new String[resultsize];

00037. Return matchlist.sublist (0, resultsize). ToArray (result);

00038.}

Watch line 9th: stirng match = Input.subsequence (Intdex, M.start ()). ToString ();

The match here is a split string object, which is actually the result of a string object subsequence. Keep looking at the String.subsequence code:

Java code

00001. Public

00002. Charsequence subsequence (int beginindex, int endindex) {

00003. Return this.substring (Beginindex, endindex);

00004.}

String.subsequence had called the string.substring, and continued to look:

Java code

00001. Public String

00002. substring (int beginindex, int endindex) {

00003. if (Beginindex < 0) {

00004. throw new Stringindexoutofboundsexception (Beginindex);

00005.}

00006. if (Endindex > Count) {

00007. throw new Stringindexoutofboundsexception (Endindex);

00008.}

00009. if (Beginindex > Endindex) {

00010. throw new Stringindexoutofboundsexception (Endindex-beginindex);

00011.}

00012. Return ((beginindex = 0) && (endindex = count))? this:

00013. New String (offset + beginindex, endindex-beginindex, value);

00014.}

Look at the 11th and 12 lines, we finally see that, if the content of the substring is the complete original string, then the original string object is returned, otherwise a new string object is created, but this string object appears to use the original string object's char[]. We confirm this by using the constructor of string:

Java code

00001.//Package

00002. Private constructor which shares value array for speed.

00003. String (int offset, int count, Char value[]) {

00004. this.value = value;

00005. This.offset = offset;

00006. This.count = count;

00007.}

To avoid memory copying and speed, the Sun JDK directly reused the original string object's char[], offset and length to identify different string contents. In other words, the substring of a string object will still point to the char[],split of the original string large object. This explains why the char[of String objects in HashMap are so large. Explanation of Reason

In fact, the previous section has analyzed the reason, this section to organize again:

00001. The program obtains from each request a string large object, the object internal char[] length reaches hundreds of K.

00002. The program does a split on a string large object, placing the split string object in the HashMap as a cache.

00003. Sun JDK6 Optimized The String.Split method, and the split Stirng object used the char[of the original string object directly

Each string object in the 00004. HashMap actually points to a huge char[]

00005. HashMap's upper limit is million, so the total size of the cached sting objects = million * k=g level.

00006. G-Class memory is cached, and a lot of memory is wasted, causing memory leaks. Solution

The reason is found, the solution will have. Split is going to be used, but instead of putting the split string object directly into the HashMap, we'll call the copy constructor string (string original) of string, which is safe, and can look at the code:

Java code

00001./**

00002. * Initializes a newly created {@code String} object so that it

00003. Represents

00004. * The same sequence of characters as the argument; In the other words,

00005. The

00006. * Newly created string is a copy of the argument string. Unless an

00007. * Explicit copy of {@code original} is needed.

00008. Constructor is

00009. * Unnecessary since Strings are immutable.

00010. *

00011. * @param Original

00012. * A {@code String}

00013. */

00014. public string (string original) {

00015 int size = Original.count;

00016. char[] OriginalValue = Original.value;

00017. char[] v;

00018. if (originalvalue.length > Size) {

00019.//The array representing the String is bigger than the new

00020.//String itself. Perhaps this constructor is being called

00021.//In order to trim the baggage, so make a copy of the array.

00022. int off = Original.offset;

00023. v = arrays.copyofrange (OriginalValue, off, off+size);

00024. <