Remember the Java program consumes CPU superelevation troubleshooting

1, first through the top command to view the high CPU-occupied PID

#top

Top-18:07:25 up-Days, 1:07, 3 users, Load average:11.94, 11.90, 9.46
tasks:271 Total, 1 running, sleeping, 0 stopped, 0 zombie
%CPU (s): 74.2 us, 0.8 sy, 0.0 ni, 24.8 id, 0.0 wa, 0.0 hi, 0.0 si, 0.2nd
KiB mem:65808884 Total, 5901708 free, 46771732 used, 13135444 Buff/cache
KiB swap:524284 Total, 129448 free, 394836 used. 18397044 Avail Mem

PID USER PR NI VIRT RES SHR S%cpu%MEM time+ COMMAND
444 Root 0 20.181g 0.011t 28220 S 1181 17.2 207:46.21 Java
3631 Root 0 19.415g 1.443g 28184 S 19.6 2.3 1:42.24 Java
24528 Root 0 10.739g 671532 27128 S 3.7 1.0 8:58.51 Java
16952 Root 0 20.416g 340340 6496 S 3.3 0.5 158:35.14 java
15584 Root 0 144036 4036 2628 S 1.3 0.0 13:31.14 sshd
4717 Root 0 2942716 1.188g 7652 S 1.0 1.9 761:57.40 Java
Root 0 0 0 0 S 0.3 0.0 84:26.67 rcu_sched
Root 0 0 0 0 S 0.3 0.0 6:25.61 RCUOS/6
2759 Root 0 20.209g 1.985g 28308 S 0.3 3.2 1:46.17 Java
3474 nginx 0 124032 3680 452 S 0.3 0.0 1:49.80 nginx

PID has a 444,CPU usage rate of 1181%

2. Troubleshooting Threads

Use command: PS-MP pid-o thread,tid,time pid is the 444 found above.

#ps-mp 444-o Thread,tid,time

User%cpu PRI SCNT wchan user SYSTEM TID time
Root 792------01:18:40
Root 0.0 19-futex_--444 00:00:00
Root 4.0 19-futex_--480 00:00:24
Root 58.5 ----481 00:05:48
Root 58.6 ----482 00:05:49
Root 58.5 ----483 00:05:48
Root 58.6 ----484 00:05:49
Root 58.5 ----485 00:05:49
Root 58.6 ----486 00:05:49
Root 58.5 ----487 00:05:49
Root 58.5 ----488 00:05:49
Root 58.6 ----489 00:05:49
Root 58.9 ----490 00:05:51
Root 58.5 ----491 00:05:48
Root 58.5 ----492 00:05:48
Root 58.6 ----493 00:05:49
Root 3.4 19-futex_--494 00:00:20
Root 0.0 19-futex_--495 00:00:00
Root 0.0 19-futex_--496 00:00:00
Root 0.0 19-futex_--498 00:00:00
Root 1.2 19-futex_--499 00:00:07
Root 1.5 19-futex_--500 00:00:09
Root 1.2 19-futex_--501 00:00:07
Root 1.0 19-futex_--502 00:00:06
Root 1.3 19-futex_--503 00:00:07
Root 1.1 19-futex_--505 00:00:06

Red zone, it is clear that the created thread is not executing, and the GC is constantly being carried out, causing the CPU to soar. Found the red zone problem thread.

3. View the problem thread stack

Convert the tid of the problem thread to 16 binary:

#printf "%x\n" 485

1e5

Jstack Viewing thread stack information

#jstack PID | grep tid

#jstack 444 | grep 1e5

GC task Thread#4 (PARALLELGC) os_prio=0 tid=0x00007f87d402b800 nid=0x1e5 runnable

Jstat Viewing process memory status

Jstat-gcutil 444 2000 10
S0 S1 E O M CCS ygc ygct FGC fgct GCT
0.00 0.00 100.00 99.99 97.42 95.76 23 40.676 119 430.327 471.004
0.00 0.00 100.00 99.99 97.42 95.76 23 40.676 119 430.327 471.004
0.00 0.00 100.00 99.99 97.43 95.77 23 40.676 120 434.709 475.385
0.00 0.00 100.00 99.99 97.43 95.77 23 40.676 120 434.709 475.385
0.00 0.00 100.00 99.99 97.43 95.77 23 40.676 120 434.709 475.385
0.00 0.00 100.00 99.99 97.45 95.81 23 40.676 121 439.928 480.604
0.00 0.00 100.00 99.99 97.46 95.81 23 40.676 122 442.248 482.924
0.00 0.00 100.00 99.99 97.46 95.81 23 40.676 122 442.248 482.924
0.00 0.00 100.00 99.99 97.46 95.81 23 40.676 122 442.248 482.924
0.00 0.00 100.00 99.99 97.46 95.82 23 40.676 123 447.374 488.051

E represents Cenozoic memory usage, o indicates Laosheng memory usage

Remember the Java program consumes CPU superelevation troubleshooting