Today landed a colleague of the gateway began to think that the hive environment can not landing, a closer look at the next is because the machine is very card, I have to wait a few seconds without landing on the CTRL + C, see the next is a Java process cpu:340.4% mem:14.6%
The general workaround is to use the top command to identify the high-resource thread ID and to view all system calls to that thread using the Strace command
1. Find the suspect process PID with the top command
Top for a second
You can see that the Java Process CPU utilization has remained 100%, stable, found PID 24138
2. Identify the thread that consumes the most resources
Top-h-P 29580 You can view the most expensive thread without first stepping directly to the command top-h
top-20:42:01 up 633 days, 9:30am, 9 users, load average:6.75, 8.32, 15.86
Tasks:28 Total, 2 running, sleeping, 0 stopped, 0 zombie
Cpu (s): 42.4%us, 4.3%sy, 0.0%ni, 53.1%id, 0.0%wa, 0.0%hi, 0.1%si, 0.1%st
mem:7680000k Total, 5774940k used, 1905060k free, 400792k buffers
swap:2096472k Total, 876580k used, 1219892k free, 1727652k cached
PID USER PR NI VIRT RES SHR S%cpu%MEM time+ COMMAND
29679 Baishou 0 1560m 1.1g 18m R 72.0 15.3 115:41.08 java
29678 Baishou 0 1560m 1.1g 18m R 63.3 15.3 118:44.99 java
29673 Baishou 0 1560m 1.1g 18m S 1.0 15.3 0:59.72 java
29677 Baishou 0 1560m 1.1g 18m S 1.0 15.3 1:01.34 java
3. View all system calls for this thread
Strace-p 29679
Read114,"\0\225\0\0\6\0\0\0\0\0\10\0\0\4\3\0\0\0p\365\1_\0\0\0\0\0\0\0\0\0\4" ...,2064) =149Write114,"\7\333\0\0\6\0\0\0\0\0\3\201r\4\0\0\0\0\0\0\0 b\25\274\252*\0\0\275.\0" ...,2011) =2011Write114,"\7\333\0\0\6\0\0\0\0\0adddddc\2\301!\4\302dqq\3\300r\37\3\300c" ...,2011) =2011Write114,"\7\333\0\0\6\0\0\0\0\0000\6\305\6\1jdk\ttb1521841\0010\1\200" ...,2011) =2011Write114,"\7\333\0\0\6\0\0\0\0\0000\6\305\6\1k6\27\ttb1533113\0010\1\200" ...,2011) =2011Write114,"\7\333\0\0\6\0\0\0\0\0<\0|0\6\305\6\1l\23\20\ttb1518437\1" ...,2011) =2011Write114, "\7\333\0\0\6\0\0\0\0\0\0010\1\200\1\200\2\301" \1\200\2\301\2\1\200\1\200\1\200\1\ "..., 2011) = 2011write (114, \0\34\0\0\6\0\0\0\0\0\200\1\200\1\200\1\200\n\300\31\20f\10 \27#\23\23) ", 28) = 28read (114, \0\225\0\0\6\0\0\0\0\0\10\0\0\4\3\0\0\0q\365\1_\0\0\0\0\0\0 \0\0\0\4 "..., 2064) = 149write ( Span class= "Hljs-number" >114, "\7\333\0\0\6\0\0\0\0\0\3\201s\4\0\0\0\0\0\0\0 B\25\274\ 252*\0\0\311.\0 "..., 2011) = 2011
The discovery has a lot of write operations, which should be run by the Datax task.
+++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++ if it is a web app, you can continue to print the thread's stack information +++++++++
Convert the required thread ID to 16 binary format:
printf "%x\n" 29679
73ef
Last print thread's stack information:
Jstack 29679|grep 73ef-a 30
High CPU utilization of Java processes under Linux-analysis method