JBoss occupies 100% CPU usage

Through the analysis of Java thread dump find the most CPU-consuming source code classification: 9. Java2010-04-11 9272 people read comments (4) collection reports threadjavaeclipse plug-in redhatjbosslinux

Find the most CPU-consuming source code through Java thread dump Analysis

Hu Jiahui 2010-04-11

Recently, performance problems occur during the operation of the product. when the traffic is low, the CPU reaches 40%, and when the traffic is slightly higher, the CPU reaches 98%.

The product is written in Java and runs on the JBoss platform. The operating system is RedHat Linux. When you use the top command to find that the process of your application occupies 98% of the CPU, I think you must want to know where the CPU processing time is spent. This can be found through thread dump analysis, but this is only the first step to solve the problem, that is, locate the problem.

First, how do I generate thread dump logs?

Step 1: Find the process number where the application is located and use the top command.

Step 2: Run kill-3 PID to obtain the thread dump log (PID is obtained in the first step ). Note: logs output in different Linux environments may vary. Run the kill-3 pidcommand on Linux on the IBM PowerPC minicomputer. The file javacore.20100409.161739.7614.0001.txt is located in the working directory. In my environment, the thread dump information is output to the JBoss log file.

Second, get thread Information

Most server applications are multi-threaded, so you must check which threads occupy high CPU. The top-H command can be used to view the thread information of the application and CPU usage.

As follows:

PID user PR Ni virt res shr s % CPU % mem time + command

4280 NBG-syst 18 0 3608 M 2.0g 21 M r 93.6 25.9 5004: 49 Java

4279 NBG-syst 18 0 3608 M 2.0g 21 M r 92.6 25.9 4876: 40 Java

4281 NBG-syst 18 0 3608 M 2.0g 21 M r 92.6 25.9 3892: 54 Java

4282 NBG-syst 18 0 3608 M 2.0g 21 M r 91.2 25.9 4954: 40 Java

4244 NBG-syst 15 0 3608 M 2.0g 21 m s 3.3 25.9 168: 34.04 Java

The column where the PID is located is the corresponding thread ID, which is in decimal format.

Finally, find the CPU-consuming thread and the corresponding source code.

Take the PID 4280 of the first line with the highest CPU consumption above, convert it to hexadecimal to get 0x10b8. Search for 0x10b8 in the thread dump log, and the following information will be found:

"Stack. ClientSelector-1" daemon PRIO = 10 tid = 0x0000000000004baeec00 nid = 0x10b8 runnable [0x0000000000000000500009000 .. 0x00000000500009c90]

Java. Lang. thread. State: runnable

At sun. NiO. Ch. epollarraywrapper. epollwait (native method)

At sun. NiO. Ch. epollarraywrapper. Poll (unknown source)

At sun. NiO. Ch. epollselectorimpl. doselect (unknown source)

At sun. NiO. Ch. selectorimpl. lockanddoselect (unknown source)

-Locked <0x00002aaac00005468> (a sun. NiO. Ch. util $1)

-Locked <0x00002aaac4131670> (a java. util. Collections $ unmodifiableset)

-Locked <0x00002aaac3f79c78> (a sun. NiO. Ch. epollselectorimpl)

At sun. NiO. Ch. selectorimpl. Select (unknown source)

At com. *****. warlock. protocolstack. impl. layer2.nio. activeselectorimpl. callselect (activeselectorimpl. Java: 288)

At com. *****. warlock. protocolstack. impl. layer2.nio. activeselectorimpl. Run (activeselectorimpl. Java: 163)

The NID in the preceding log is the thread number. In this way, you can clearly see the specific location of the CPU-consuming source code, accurate to the line number.

Note: Some of the above uses ***** to block the company's copyright information, not ** haha. The above examples are based on HP blade hardware, RedHat Enterprise Operating System and Sun's JDK.

 

Conclusion: thread dump has more than one function. For example, you can also find many application program running problems, such as deadlocks. There is also a visual analysis tool lockness Of The Eclipse plug-in for this log. It feels good to try it. For more information, see (3 ).

 

The following is a good reference for this topic.

References:

Http://weblogic.sys-con.com/node/44027 analyzing Java application problems -- using Java thread dumps to assess the problem

(2) http://publib.boulder.ibm.com/infocenter/wasinfo/v4r0/index.jsp? Topic =/com.ibm.support.was40.doc/html/100w.cpu _ usage/swg21162381.html determining which Java thread is consuming CPU cycles on Solaris systems

(3) http://lockness.plugin.free.fr/home.php lockness eclipse plugin for thread dump GUI analysis tool