Troubleshoot high CPU usage of Java Virtual Machine

Problem description

One m4000 host, the operating system is solaris10, And the CPU usage of the resin process above is too high, reaching 70%, as shown below:

 
-Bash-3.00 $ PS-ef-o pid, pcpu, argS | grep Java
1511 0.1/usr/Java/bin/Java-dwebview. htdocs =/etc/opt/fjsvwvcnf/htdocs/fjsvwvbs-mx128m
2135 0.0/usr/Java/bin/Java-server-xmx128m-XX: + backgroundcompilation-XX: permsize = 32 m
15945 0.0 sh-C/SVI/jdk150/jdk1.5.0 _ 06/bin/Java-server-xms512m-xmx3072m-XX: maxpe
15946 70.7/SVI/jdk150/jdk1.5.0 _ 06/bin/Java-server-xms512m-xmx3072m-XX: maxpermsize =
 

Troubleshooting Process

1. First, you need to find the lwp with high CPU usage.

-Bash-3.00 $ prstat-l-P 15946 PID username size RSS state pri nice time CPU process/lwpid 15946 slview 3336 M 3301 M sleep 15 0 3:56:27 2.2% Java/49 15946 slview 3336 M 3301 M sleep 8 0 3:33:17 2.2% Java/52 15946 slview 3336 M 3301 M sleep 12 0 3:32:20 2.2% Java/50 15946 slview 3336 M 3301 M sleep 13 0 3:29:43 2.2% Java/51 15946 slview 3336 M 3301 M sleep 13 0 3:30:54 2.2% Java/47 15946 slview 3336 M 3301 M sleep 12 0 1:24:19 2.2% Java/64 15946 slview 3336 M 3301 M sleep 15 0 1:07:55 2.1% Java/ 144
 

 

2. Find the ing between lwp and Java threads

-Bash-3.00 $ pstack 15946 | grep lwp ----------------- lwp #47/thread #47 ------------------ ff2c49fc _ lwp_start (0, 0, 0, 0, 0, 0) ----------------- lwp #48/thread #48 ------------------ ff2c5cd0 lwp_cond_wait (1704928,170 4910, 0, 0) ff2c49fc _ lwp_start (0, 0, 0, 0, 0, 0, 0) ----------------- lwp #49/thread #49 effecff2c49fc _ lwp_start (0, 0, 0, 0, 0) --------------- lwp #50/thread #50 effecff2c49fc _ lwp_start (0, 0, 0, 0, 0) ----------------- lwp #51/thread #51 ------------------ ff2c49fc _ lwp_start (0, 0, 0, 0, 0, 0) ----------------- lwp #52/thread #52 ------------------ ff2c49fc _ lwp_start (0, 0, 0, 0, 0, 0)
 

 

3. Used the jstack <pid> Find the callstack of Thread

$ Jstack-M 15946 get the call heap quota of all threads

Hread t @ 50: (State = in_vm)-java. lang. abstractstringbuilder. expandcapacity (INT) @ BCI = 28, line = 99 (Compiled frame; information may be imprecise)-per. xwnmp. flux. report. rptfluxhisquery. getfluxdata (Java. lang. string [], Java. util. hashmap, Java. lang. string, Java. lang. string, Java. lang. string, Java. lang. string, Java. lang. string) @ BCI = 480, line = 509 (interpreted frame)-per. xwnmp. flux. report. rptfluxhisquery. genfluxreport (Java. lang. string, Java. lang. string [], Java. lang. string, Java. lang. string, Java. lang. string, Java. lang. string, Java. lang. string) @ BCI = 124, line = 82 (interpreted frame)-_ nos. _ flux. _ flux. _ fluxperfview_0excel _ JSP. _ jspservice (javax. servlet. HTTP. httpservletrequest, javax. servlet. HTTP. httpservletresponse) @ BCI = 930, line = 162 (interpreted frame)-Com. caucho. JSP. javapage. service (javax. servlet. servletrequest, javax. servlet. servletresponse) @ BCI = 9, line = 75 (interpreted frame)-Com. caucho. JSP. page. subservice (COM. caucho. server. HTTP. specified orequest, Com. caucho. server. HTTP. cauchoresponse) @ BCI = 214, line = 506 (I-Com. caucho. server. tcpconnection. run () @ BCI = 73, line = 139 (interpreted frame)-java. lang. thread. run () @ BCI = 11, line = 595 (interpreted frame) thread t @ 52: (State = in_vm)-per. xwnmp. flux. report. rptfluxhisquery. getfluxdata (Java. lang. string [], Java. util. hashmap, Java. lang. string, Java. lang. string, Java. lang. string, Java. lang. string, Java. lang. string) @ BCI = 435, line = 508 (Compiled frame; information may be imprecise)-per. xwnmp. flux. report. rptfluxhisquery. genfluxreport (Java. lang. string, Java. lang. string [], Java. lang. string, Java. lang. string, Java. lang. string, Java. lang. string, Java. lang. string) @ BCI = 124, line = 82 (interpreted frame)-_ nos. _ flux. _ flux. _ fluxperfview_0excel _ JSP. _ jspservice (javax. servlet. HTTP. httpservletrequest, javax. servlet. HTTP. httpservletresponse) @ BCI = 930, line = 162 (interpreted frame)-Com. caucho. JSP. javapage. service (javax. servlet. servletrequest, javax. servlet. servletresponse) @ BCI = 9, line = 75 (interpreted frame)-Com. caucho. JSP. page. subservice (COM. caucho. server. HTTP. specified orequest, Com. caucho. server. HTTP. cauchoresponse) @ BCI = 214, line = 506 (interpreted frame)