The location and analysis of Java memory leaks

1. Why memory leaks occur

How does Java detect internal leaks? We need some tools to detect and find memory leaks, otherwise it's easy to have a down-machine problem.

The most convenient place to write Java programs is that we do not need to manage the allocation and release of memory, everything is handled by the JVM, when the Java object is no longer applied, when the heap memory is not enough, the JVM will garbage collection, clean up the heap memory of these objects, if the object has been applied, The JVM cannot reclaim it, and when a new object is created, it cannot get enough memory from the heap to allocate to the object, which can cause memory overflow. Where memory leaks occur, it is common to keep objects in containers that do not have a corresponding size limit or cleanup mechanism. Easy to cause memory overflow.
How to quickly locate a problem when the server application is taking up too much memory? Now, the advent of Eclipse mat makes the problem very simple. Eclipsemat is a well-known SAP company contribution tool that can be downloaded to it on the Eclipse website, completely free of charge.
To locate the problem, first you need to get a memory snapshot of the server JVM. The JDK's own jmap can take a snapshot of the memory at a moment, and after exporting it to the DMP file, it can be analyzed with eclipse mat to find out if that object uses too much memory.

2, Memory leakage phenomenon:

Often, the initial signs of a program's memory leak occur after an error and get a outofmemoryerror in your program. This typically happens in a production environment where you want to have as little memory leaks as possible and minimize the likelihood of debugging. Perhaps your test environment and the product's system environment are different, resulting in leaks will only be exposed in the product. In this case, you need a low-load tool to monitor and look for memory leaks. At the same time, you need to connect the tool to your system without restarting or mechanization of your code. Perhaps more importantly, when you do the analysis, you need to be able to separate the tools so that the system will not be disturbed.
A outofmemoryerror is often a sign of a memory leak, and it is possible that the application does use too much memory; this time, you can neither increase the number of JVM heaps, nor change your program to make him reduce memory usage. However, in most cases, a outofmemoryerror is a memory leak flag. One solution is to continue to listen to GC activity, to see whether the memory usage increases over time, and if so, there must be a memory leak in the program.

3. Discovery of memory leaks

1. JSTAT-GC PID

You can display GC information, the number of times the GC is viewed, and the time.

The last five items were the number of young GC, the time of young GC, the number of full GC, the time of full GC, and the total time of GC.

2.jstat-gccapacity PID

can display the use and occupancy of three generation (Young,old,perm) objects in VM memory,

For example: PGCMN shows the minimum perm memory usage, PGCMX shows the maximum memory usage of perm,

The PGC is the current newly generated perm memory footprint, and the PC is but the pre-perm memory footprint.

The other can be based on this analogy, OC is the old inside the pure consumption.

3.jstat-gcutil PID

Statistical GC information Statistics.

4.jstat-gcnew PID

Information about the young generation object.

5.jstat-gcnewcapacity PID

The information of the young generation object and its consumption.

6.jstat-gcold PID

The information for the old generation object.

7.stat-gcoldcapacity PID

The information and consumption of the old generation object.

8.jstat-gcpermcapacity PID

The information of the Perm object and its consumption.

9.jstat-class PID

Displays information such as the number of loaded classes, and the amount of space occupied.
10.jstat-compiler PID

Displays information such as the number of real-time VMS compiled.

11.stat-printcompilation PID

Information about the current VM execution.

Some terms are explained in Chinese:

S0C: Capacity of the first Survivor (Survivor area) in the Young Generation (bytes)
S1C: Capacity of the second survivor (Survivor area) in the younger generation (bytes)
S0U: The first survivor (Survivor area) of the young generation has currently used space (bytes)
S1U: The second survivor (Survivor area) of the young generation has currently used space (bytes)
EC: The Capacity of Eden (Eden) in the Young Generation (bytes)
EU: Eden (Eden) in the young generation has currently used space (bytes)
Capacity of the Oc:old generation (bytes)
Ou:old currently used space (bytes)
Pc:perm (persistent generation) capacity (bytes)
Pu:perm (persistent generation) currently used space (bytes)
YGC: Number of GC times in young generations from application startup to sampling
YGCT: The time taken by the GC in the young generation from application startup to sampling (s)
FGC: Old (full GC) GC count from application boot to sampling
FGCT: Time taken from an old (full GC) GC when the application boots to sampling (s)
GCT: GC Total time from Application boot to sample (s)

NGCMN: initial (minimum) size in Young generation (bytes)

NGCMX: Max capacity of young generation (bytes)

NGC: Current capacity in young generation (bytes)

The size (in bytes) of the initialization (minimum) in ogcmn:old generation

Maximum capacity of the Ogcmx:old generation (bytes)

The current newly generated capacity of the Ogc:old generation (bytes)

The size (in bytes) of the initialization (minimum) in pgcmn:perm generation

Maximum capacity of the Pgcmx:perm generation (bytes)

The current newly generated capacity of the Pgc:perm generation (bytes)

S0: The first survivor (Survivor area) of the young generation has been used as a percentage of current capacity

S1: The second survivor (Survivor area) in the younger generation has been used as a percentage of current capacity

E: Eden (Eden) in the young generation has been used as a percentage of current capacity

Percentage of current capacity used by O:old generation

Percentage of current capacity used by P:perm generation

S0CMX: Maximum Capacity (bytes) of the first Survivor (Survivor area) in the young generation

S1CMX: Maximum capacity of the second survivor (Survivor area) in the younger generation (bytes)

ECMX: Maximum capacity of Eden (Eden) in the Young Generation (bytes)

DSS: Capacity (bytes) currently required for Survivor (surviving area) (Eden area Full)

TT: Limited number of holds

MTT: Maximum number of hold limit

If you are locating a memory leak problem, I usually use the following command:

jstat-gcutil15469 2500 70

[Email protected] logs]# Jstat-gcutil 15469 1000 300

S0 S1 E O P ygc ygct FGC fgct GCT

0.00 1.46 26.54 4.61 30.14 35 0.872 0 0.000 0.872

0.00 1.46 46.54 4.61 30.14 35 0.872 0 0.000 0.872

0.00 1.46 47.04 4.61 30.14 35 0.872 0 0.000 0.872

0.00 1.46 65.19 4.61 30.14 35 0.872 0 0.000 0.872

0.00 1.46 67.54 4.61 30.14 35 0.872 0 0.000 0.872

0.00 1.46 87.54 4.61 30.14 35 0.872 0 0.000 0.872

0.00 1.46 88.03 4.61 30.14 35 0.872 0 0.000 0.872

1.48 0.00 5.56 4.62 30.14 36 0.874 0 0.000 0.874

1000 represents how often intervals are displayed,

100 represents the display once.

Percentage of space used in Survivor space zone 0 on S0-heap

Percentage of space used in Survivor space Zone 1 on S1-heap

Percentage of space used in Eden space on E-heap

The percentage of space used in the old area on the O-heap

Percentage of space already used in the P-perm space area

ygc-number of young GC occurrences from application boot to sample

ygct– the time (in seconds) used by the young GC when booting from the application to sampling

fgc-the number of full GC occurrences from application startup to sampling

fgct– time (in seconds) for full GC from application boot to sampling

gct-total time (in seconds) for garbage collection from application startup to sampling

If there is a large number of FGC to query whether there is a memory leak problem, the figure in the number of FGC is relatively large, and the execution time is longer, this will lead to a longer response time, if the JVM memory settings are large, then the time to execute the FGC may be longer.

To better demonstrate the impact of FGC on server performance, we can use Java VISUALVM to see:

From the situation where the execution of the FGC can be found, there is no FGC before 3:10, and a large number of FGC occur.

Is the JVM heap memory usage, before 3:10 minutes of memory recycling is reasonable, but after a large amount of memory can not be recycled, resulting in less memory, resulting in a large number of full GC.

Here's a look at the impact of a lot of full GC on server performance, and here's how I used LoadRunner to stress test our project for the time:

It can be found that in the full GC after the corresponding time of the system has been significantly increased, CTR and throughput has also been significantly reduced. So the impact of Java memory leaks on system performance is not negligible.

3. Locating memory leaks

Of course, through the above several methods we can find Java memory leak problem, but as a qualified senior engineer, certainly not reconciled to this conclusion to the development, of course, this conclusion to the development, development is also difficult to locate the problem, in order to better provide their position in the company, We have to give the development engineers more in-depth test results, and here's a look at MemoryAnalyzer.exe. Java Memory leak Checker tool.

First we have to dump the JVM's heap memory, only to get this file can we analyze what is in the JVM heap memory, what exactly is it doing?

Memoryanalyzer users I do not explain this here, my blog also has a description, the following shows the success of my test diagram:

Where the dark blue part is the memory leak part, Java's heap memory is only 481.5M and the memory leak part occupies 336.2M so this time the memory leak is obvious, then I will look at the memory leak that the method caused:

From the way we can find that the red line around the method occupies 67.75% of the heap memory, if the test results can be given to the development, development should be well positioned. So as a senior test engineer, we need to learn too much stuff.

Although the uncertainty must be a memory leak, but can accurately tell the development of the cause of the problem, there is some convincing.

I have just completed the training of cloud storage architects (including Linux kernel Understanding, Shell advanced Programming, Linux Security Learning Focus iptables and TCP/IP and other protocols such as packet capture analysis, Linux clusters, Performance tuning and then there's the DBA's course waiting for me to challenge.

The location and analysis of Java memory leaks