A product receives a CPU alarm every time it restarts Tomcat. the CPU usage can reach 80% ~ More than 90%, will continue to fall back for a period of time (the specific time was not investigated ). Analyze and locate the problem as required by the product team. Because online products cannot be restarted at will, they are used in the problem locating phase... "/> <scripttype =" text/javascript "src =" http: // A product receives a CPU alarm every time it restarts Tomcat. in the restart phase, the CPU usage can reach 80% ~ More than 90%, will continue to fall back for a period of time (the specific time was not investigated ). Analyze and locate the problem as required by the product team. Because online products cannot be restarted at will, QA19, a cloud host of the same type, is used in the problem locating phase. This environment has the same application as the online one, and when tomcat is restarted, there will also be high CPU usage.
Cause of high load:Monitor the resource usage during restart, and analyze the thread stack information that consumes too much CPU through top-H + jstack.
- By monitoring the resource usage during restart, the qa19 host restarts once, and the CPU usage lasts about 50-60 s.
- Using top-H + jstack to locate threads that consume too much CPU, three threads are found:
1. spring applicationContext loads resources in the web container :(Partial stack information)
"Main" prio = 10 tid = 0x000000004166f800 nid = 0x100f runnable [0x00007fbf42987000]
Java. lang. Thread. State: RUNNABLE
At org. springframework. web. context. ContextLoader. configureAndRefreshWebApplicationContext (ContextLoader. java: 384)
At org. springframework. web. context. ContextLoader. initWebApplicationContext (ContextLoader. java: 283)
At org. springframework. web. context. ContextLoaderListener. contextInitialized (ContextLoaderListener. java: 111)
At org. apache. catalina. core. StandardContext. listenerStart (StandardContext. java: 4206)
At org. apache. catalina. startup. Catalina. start (Catalina. java: 595)
At sun. reflect. NativeMethodAccessorImpl. invoke0 (Native Method)
At sun. reflect. NativeMethodAccessorImpl. invoke (NativeMethodAccessorImpl. java: 39)
At sun. reflect. DelegatingMethodAccessorImpl. invoke (DelegatingMethodAccessorImpl. java: 25)
At java. lang. reflect. Method. invoke (Method. java: 597)
At org. apache. catalina. startup. Bootstrap. start (Bootstrap. java: 289)
At org. apache. catalina. startup. Bootstrap. main (Bootstrap. java: 414)
2. Two compilation threads:
"C2 CompilerThread1" daemon prio = 10 tid = 0x00007fec48001800 nid = 0x13c4 runnable
Java. lang. Thread. State: RUNNABLE
"C2 CompilerThread0" daemon prio = 10 tid = 0x00000000407c1000 nid = 0x13c3 runnable
Java. lang. Thread. State: RUNNABLE
Solution:Two articles in helloJava provide two configuration methods: see: http://hellojava.info /? P = 195 and http://hellojava.info /? P = 201
- Try to add-XX: + TieredCompilation and expect multi-layer compilation to alleviate this problem:
This configuration item is added to the tomcat jvm parameter, and the CPU usage for resource monitoring is reduced to about 45 s, which is 10 s + less than the original one, with a slight effect.
- Try to add the-XX: CICompilerCount parameter. the default value is 2, that is, two c2 compilation threads for compilation:
Because QA19 and the online tomcat application server are both two-core cloud hosts, the default value 2 should be the optimal value. if you try to change this value to 1 or 4, the optimization effect is not achieved.
Problems:For the QA19 performance testing environment, no other requests are introduced after the restart, but requests are continuously sent after the online environment is restarted, so the CPU usage will take a longer time. For the performance testing environment, the first concurrent test after the restart will have a high load in the early stage. for example, after the Test starts, analyze the stack information through top-H + jstack, two reasons are obtained:
- Starting various threads to process requests, such as the AJP thread, data communication thread, and ActiveMQ communication thread will consume some resources in a short time.
- In addition, when the test is just started, the "C2 CompilerThread1" thread consumes too much resources and will last for 40% ~ 80% of CPU consumption
Add the-XX: + TieredCompilation parameter during the test. the resource usage is compared as follows:
- When 50 threads are concurrently added, add the parameter (left) and add the parameter (right)
- 100 concurrent threads, before adding parameters (left) and after adding parameters (right)
Conclusion:It can be seen that, even with the introduction of traffic, The-XX: + TieredCompilation parameter also plays a certain role, so that the CPU usage of the surging volume will fall back as soon as possible.
Appendix:-XX: + TieredCompilation
Enables a JIT compilation policy to make initial quick JIT compilation decisions analogous to optimizations made by the HotSpot VM's-client runtime and then continue to make more sophisticated JIT compilation extends similar to those made the VM's-server runtime for frequently called Java methods in the program. in short, it uses a combination of the best of both-client and-server runtimes, quick compilation along sophisticated optimizations for frequently called Java methods. at the time of this writing, it is not recommended to use this command line option as a replacement for the-server runtime since the-server runtime offers better peak performance. this recommendation may change in the future as the tiered compilation feature is enhanced. client applications running Java 6 Update 25 or later may consider using this command line option with the-server runtime (-server-XX: + TieredCompilation) as an alternative to the-client runtime. it is recommended you measure application startup performance and application responsiveness to assess whether the-server runtime with-XX: + TieredCompilation is better suited for the application than the-client runtime.
Http://www.techpaste.com/2012/02/java-command-line-options-jvm-performance-improvement/
