Welcome visit: http://www.cloudnoter.com/?p=143
Since the platform upgrade to 3.0, the JVM of the application has become very unstable, mainly reflected in the following three issues:
1. Memory leak: 2G of jvm,2 days on the collapse.
2. Method area memory continues to soar, resulting in frequent triggering of FULLGC
3.class load frequently results in a 30% waste of CPU resources
Before writing the trough: This self-developed JPA components really TM deceptive, open source of the need to implement their own, three problems are self-developed JPA components introduced.
Solution:
Question 1:
Problem 1 relatively good solution, first with Jmap heap snapshot dump out, with mat analysis, according to gc-root find reference path, leakage reason: platform self-research JPA components sqlquery in the implementation of lazy load, due to cglib improper use ( When registering a callback method interceptor with the current thread, the query result cache is referenced by threads in the thread pool when it is not logged out in time, and eventually results in an Oom exception when the online pool capacity is larger.
Question 2, 3:
Never before encountered this situation, the method area of memory size should be in a relatively stable state after the application started (because most classes are loaded at startup, even if the use of Cglib dynamically generated proxy class should have an upper limit, up to the total class), but the problem 2 is obviously not in this case, Regardless of how much memory is given to the method area (set size by-X:MAXPERMSIZE=XXXM), the application can always rise to the highest value in a few minutes and trigger FULLGC,GC end, the method area takes up memory to close to 0M (the class unload that occurs here), And then into a new round of soaring cycles (where class loader occurs).
At first it was thought that the JPA component was still using cglib improperly, and that it was used to implement lazy load and privilege control with too many dynamic proxies (each action,model,service was created as a dynamic proxy, Even more unreasonable, each model's Get method uses the Proxymethodinterceptor, asks the litigant reason, its reply says in order to the lazy load, but actually only has the correlation field, the collection field is necessary the lazy load). As a result of this modification, the test results do not solve the problem: because JPA does not create a new proxy every time, it is cached based on class, so you can only find another way.
Since it is the problem of the method area, is it possible to dump the contents of the method area? So I looked at the next jmap parameter, where-permstat can be used: Jmap-permstat <pid>
The results are as follows (interception)
25007 intern Strings occupying 2799672 bytes.class_loader classesbytes parent_loaderalive? Type<bootstrap> 262415108248 null live <internal>0x000000076f045910 38758792 0x00000007617d4f70dead com/atomikos/util/[email protected]0x000000076f942f10 38758792 0x00000007617d4f70dead com/ Atomikos/util/[email protected]0x00000007d00e9ba0 40816816 0x00000007617d4f70dead com/atomikos/util/[email PROTECTED]0X00000007DD170C08 40816816 0x00000007617d4f70dead Com/atomikos/util/[email protected]0x000000079e2f0070 40816816 0x00000007617d4f70dead com/atomikos/util/[email protected]0x00000007cd6b1140 13112 null dead sun/ Reflect/[email protected]0x000000076fb5d130 38758792 0x00000007617d4f70dead com/atomikos/util/[email protected] 0X000000076FBE47C0 38758792 0x00000007617d4f70dead Com/atomikos/util/[email protected]
Com/atomikos/util/classloadinghelper$1: is an anonymous inner class (the class is an loader), which is used as a parameter to the JDK Proxy.newproxyinstance () method by this internal class loader. The latter produces a large number of dynamic classes prefixed with proxy$, and does not cache any.
Atomikos It should be clear to everyone: one implementation of JTA, but which component calls this tool class? Through the breakpoint analysis, the original is JPA and write what database connection pool, each connection in the pool is proxyconnection, and the pool seems to fail, frequent recycling, create ... (So far, I haven't figured out why to use a proxy type of connection, the role of the agent from the code does not see a doorway, nor for monitoring.) ）
The reason for the position, the solution is very simple: directly with Ali's Druid replaced. The test results prove that the previous analysis is correct.
Conclusion: Programmers create value instead of repeating wheels, embracing open source and not behind closed doors.
Java Case 3: Method Area (PermGen) memory rapid spike problem
