Write a memory leak analysis of Java

Current environment

1. JDK = = 1.8
2. Httpasyncclient = = 4.1.3

Code address

Git address: https://github.com/jasonGeng88/java-network-programming

Background

Not long ago, a new project was launched, the project is a pressure measurement system, can simply be seen through the playback of the Thesaurus (HTTP request data), and constantly send requests to the service to achieve the purpose of the pressure measurement service. In the test process, everything is smooth, fixed a few minor bugs, on the line. When used on-line to the first business party, a serious problem was found, and the application ran for about more than 10 minutes and received a large number of full GC alarms.

In response to this problem, we first and the business party confirmed the scene content of the test, playback of the number of vocabulary is about 100,000, the rate of playback of a single machine at about 100QPS, according to our previous estimate, this is far lower than the limit of the stand alone can withstand. As a matter of principle, memory problems are not generated.

On-Line troubleshooting

First, we need to troubleshoot on the server. Use the Jmap tool that comes with the JDK to see what the specific objects are in the JAVA application, as well as the number of cases and their size. The specific commands are as follows:

Jmap-histo:live ' pid of Java ' # for easy observation, or write output to file Jmap-histo:live ' pid of Java ' >/tmp/jmap00

After observation, it is found that there are objects instantiated more than 200,000, according to business logic, the most instantiated is the word list, that also more than 100,000, how there will be more than 200,000, we did not find in the code to display the declaration of the instantiation of the place. At this point, we need to dump the memory, offline for further analysis, the dump command is as follows:

Jmap-dump:format=b,file=heap.dump ' pid of Java '

Offline analysis

After we have downloaded dump heap.dump from the server, we need to do an in-depth analysis through the tool. The tools recommended here are mat, VisualVM.

I personally prefer to analyze using VisualVM, which can analyze offline dump files, integrate with idea, launch applications via idea, perform real-time analysis of application CPU, memory, and GC conditions (GC case, You need to install the visual GC plug- in in VISUALVM). The tools are shown below ( just to show the effect, the data is not true ):

Of course, Mat is also a very useful tool, it can help us quickly locate the memory leaks, easy to troubleshoot. Show below:

Scene reproduction

After analysis, we finally locate the memory leak problem that is caused by using httpasyncclient. Httpasyncclient is an HTTP toolkit provided by Apache, which mainly provides the IO nonblocking model of reactor, which realizes the function of sending HTTP requests asynchronously.

The following is a Demo to briefly explain the reasons for the specific memory leaks.

Httpasyncclient Usage Introduction:

• Maven Dependency

<dependency>    <groupId>org.apache.httpcomponents</groupId>    <artifactId> Httpasyncclient</artifactid>    <version>4.1.3</version></dependency>

• Httpasyncclient Client

public class Httpasyncclient {    private closeablehttpasyncclient httpclient;    Public httpasyncclient () {        httpclient = Httpasyncclients.createdefault ();        Httpclient.start ();    }    public void execute (httpurirequest request, futurecallback
Main logic:
The main logic of the Demo is this, first creating a cache list to hold the requested data that needs to be sent. The request that needs to be sent is then taken out of the cache list in a circular manner and sent to the Httpasyncclient client.
The specific code is as follows:
public class Replayapplication {public    static void Main (string[] args) throws Interruptedexception {//Create a memory-leaking playback client        Replaywithproblem replay1 = new Replaywithproblem ();         Load 10,000 request data into cache        list
Playback client implementation (memory leak):
Here to replay Baidu as an example, create 10,000 mock data into the cache list. When playing back, send a request out in a while loop every 100ms. The specific code is as follows:
/**
* Java Learning Exchange QQ Group: 589809992 We'll learn java! together.
*/
public class Replaywithproblem {public listMemory Analysis:
Start the Replayapplication app (you can start VisualVM directly after you install VisualVM launcher inidea) and watch it through VisualVM.

 
   

  
    
 
     
Startup situation:
 
    
 
   

 
   

  
    
 
     
The ratio of memory objects before and after 3 minutes in VisualVM:
 
    
 
   
Description: $ A represents the object itself, which represents the first inner class in the object. So replaywithproblem$1: Represents the callback class Futurecallback in the Replaywithproblem class.
From there, we can find that the Futurecallback class is constantly being created. Because each time an HTTP request is sent asynchronously, the result is received by creating a callback class, which logically looks normal as well. No hurry, we'll look down.

 
   

  
    
 
     
GC case for 3 minutes before and after VisualVM:
 
    
 
   
See, the memory of the old is constantly growing, this is not right. In memory should be only the cache list of HTTP request body, and now in constant growth, there is a constant object into the old area, combined with the above memory objects, the case of Futurecallback object is not timely recovery.
But the callback anonymous class after the HTTP callback ends, the reference relationship is gone, and the next GC should be recycled. We traced the source of the httpasyncclient send the request, and found that its internal implementation is to plug the callback class into the HTTP request class, and the request class is placed in the cache queue, so that the callback class is not dismissed the reference relationship, a large number of callback classes promoted to the old area, Eventually results in full GC generation.

 
   

  
    
 
     
Core Code Analysis:
 
    
 
   
Code optimization
To find the cause of the problem, let's now refine the code to validate our conclusions. Because List<HttpUriRequest> cache1 the callback object is saved, we cannot cache the request class, only the basic data can be cached, dynamically generated when used, to ensure the timely recovery of callback objects.
The code is as follows:
public class Replayapplication {public    static void Main (string[] args) throws Interruptedexception {        Replaywithoutproblem replay2 = new Replaywithoutproblem ();        list<string> cache2 = Replay2.loadmockrequest (10000);        Replay2.start (cache2);    }}
/**
* Java Learning Exchange QQ Group: 589809992 We'll learn java! together.
*/
public class Replaywithoutproblem {public list<string> loadmockrequest (int n) {list<string> cache =        New Arraylist<string> (n);        for (int i = 0; i < n; i++) {Cache.Add ("http://www.baidu.com?a=" +i);    } return cache; } public void Start (list<string> cache) throws Interruptedexception {httpasyncclient httpClient = new Htt        Pasyncclient ();        int i = 0;            while (true) {String URL = cache.get (I%cache.size ());            Final HttpGet request = new HttpGet (URL); Httpclient.execute (Request, new futurecallbackResult validation
 
    

  
     
 
      
Startup situation:
 
     
 
    

 
    

  
     
 
      
The ratio of memory objects before and after 3 minutes in VisualVM:
 
     
 
    

 
    

  
     
 
      
GC case for 3 minutes before and after VisualVM:
 
     
 
    
, we can prove that we have come to the conclusion that is correct. Callback classes will be recovered in the Eden area in a timely manner. There is no continuous growth in the old area. This time the memory leak problem is solved.
Summarize
The memory leak problem is often a bit overwhelming in the first troubleshooting. We need to have the right methods and means, with the use of good tools, so in order to solve the problem, to be able to do. Of course, the basic knowledge of Java memory is also essential, when you locate the key to the problem, or even if the tool tells you this piece of error, you can not locate the reason.
Finally, with regard to the use of httpasyncclient, there is no problem with the tool itself. But we have to understand its use of the scene, often produce a lot of problems, are used improperly caused. Therefore, when using the tool, the degree of understanding of it often determines the probability of the occurrence of a bug.
Write a memory leak analysis of Java