Linux CPU Share High debug

Today the debugger, using the top command, found that the CPU occupancy of the program is very high, has been in 99, which is very scary, so to debug.

Use the top command to get the following results

PID USER PR NI VIRT RES SHR S%cpu%MEM time+ COMMAND
1997 Root 0 358m 71m 3208 S 99.1 7.2 81:53.50 Test
1 root 0 24332 2044 1176 S 0.0 0.2 0:01.36 Init
2 Root 0 0 0 0 S 0.0 0.0 0:00.00 Kthreadd
3 Root 0 0 0 0 S 0.0 0.0 0:46.51 ksoftirqd/0
4 Root 0 0 0 0 S 0.0 0.0 0:38.53 kworker/0:0
6 root RT 0 0 0 0 S 0.0 0.0 0:00.00 migration/0

The PID of Test is 1997.

Then look at the CPU usage of the thread in test.

Using the Top-h-P 1997 command

[Email protected]:~# top-h-P 1997

Top-17:19:47 up Days, Min, 4 users, load average:1.02, 1.06, 1.06
Tasks:8 Total, 1 running, 7 sleeping, 0 stopped, 0 zombie
Cpu (s): 99.3%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.3%si, 0.3%st
mem:1017924k Total, 887500k used, 130424k free, 85928k buffers
swap:0k Total, 0k used, 0k free, 351280k cached

PID USER PR NI VIRT RES SHR S%cpu%MEM time+ COMMAND
2005 Root 0 358m 71m 3208 R 99.3 7.2 83:44.47 Test
2001 Root 0 358m 71m 3208 S 0.3 7.2 0:08.66 test
2004 Root 0 358m 71m 3208 S 0.3 7.2 0:19.61 test
1997 Root 0 358m 71m 3208 S 0.0 7.2 0:26.24 Test
1999 Root 0 358m 71m 3208 S 0.0 7.2 0:00.06 Test
Root 0 358m 71m 3208 S 0.0 7.2 0:00.00 Test
2002 Root 0 358m 71m 3208 S 0.0 7.2 0:05.86 Server Listen (2
2003 Root 0 358m 71m 3208 S 0.0 7.2 0:00.00 Server Accept (2

The PID of the thread that can get the highest CPU share in the thread is 2005

Using the GDB ICDN 2005 command

[Email protected]:~# gdb ICDN 2005
GNU gdb (Ubuntu/linaro 7.4-2012.04-0ubuntu2.1) 7.4-2012.04
Copyright (C) Free Software Foundation, Inc.
License gplv3+: GNU GPL version 3 or later This was free software:you was free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "Show copying"
and "Show warranty" for details.
This GDB is configured as "X86_64-linux-gnu".
For bugs reporting instructions, please see:
Icdn:no such file or directory.
Attaching to process 2005

Warning:process 2005 is a cloned process
Reading symbols From/home/slk/test/snmptrapd...done.
Reading symbols From/usr/lib/libnetsnmp.so.30...done.
Loaded symbols for/usr/lib/libnetsnmp.so.30
Reading symbols from/lib/x86_64-linux-gnu/libpthread.so.0 ... (No debugging symbols found) ... done.
[Thread debugging using libthread_db enabled]
Using host libthread_db Library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Loaded symbols for/lib/x86_64-linux-gnu/libpthread.so.0
Reading symbols from/usr/lib/x86_64-linux-gnu/libmysqlclient.so.18 ... (No debugging symbols found) ... done.
Loaded symbols for/usr/lib/x86_64-linux-gnu/libmysqlclient.so.18
Reading symbols from/usr/local/lib/libstd.so ... (No debugging symbols found) ... done.
Loaded symbols for/usr/local/lib/libstd.so
Reading symbols from/lib/x86_64-linux-gnu/libc.so.6 ... (No debugging symbols found) ... done.
Loaded symbols for/lib/x86_64-linux-gnu/libc.so.6
Reading symbols from/lib/x86_64-linux-gnu/librt.so.1 ... (No debugging symbols found) ... done.
Loaded symbols For/lib/x86_64-linux-gnu/librt.so.1
Reading symbols from/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 ... (No debugging symbols found) ... done.
Loaded symbols for/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
Reading symbols from/lib64/ld-linux-x86-64.so.2 ... (No debugging symbols found) ... done.
Loaded symbols for/lib64/ld-linux-x86-64.so.2
Reading symbols from/lib/x86_64-linux-gnu/libz.so.1 ... (No debugging symbols found) ... done.
Loaded symbols For/lib/x86_64-linux-gnu/libz.so.1
Reading symbols from/lib/x86_64-linux-gnu/libdl.so.2 ... (No debugging symbols found) ... done.
Loaded symbols for/lib/x86_64-linux-gnu/libdl.so.2
Reading symbols from/lib/x86_64-linux-gnu/libm.so.6 ... (No debugging symbols found) ... done.
Loaded symbols for/lib/x86_64-linux-gnu/libm.so.6
Reading symbols from/usr/local/lib/libudt.so ... (No debugging symbols found) ... done.
Loaded symbols for/usr/local/lib/libudt.so
Reading symbols from/usr/lib/x86_64-linux-gnu/libstdc++.so.6 ... (No debugging symbols found) ... done.
Loaded symbols for/usr/lib/x86_64-linux-gnu/libstdc++.so.6
Reading symbols from/lib/x86_64-linux-gnu/libgcc_s.so.1 ... (No debugging symbols found) ... done.
Loaded symbols For/lib/x86_64-linux-gnu/libgcc_s.so.1
Reading symbols from/lib/x86_64-linux-gnu/libnss_files.so.2 ... (No debugging symbols found) ... done.
Loaded symbols for/lib/x86_64-linux-gnu/libnss_files.so.2
Execute_search () at test.c:310

From the last line, we see that thread 2005 runs the function Execute_search, and then looks at the function and finds that there is a

while (g_search_state = = 1)
{
TMP = g_data_list->phead->next;

if (TMP)
{
Pthread_mutex_lock (&g_mysql_mutex);
Res.result = Search_mysql (TMP->SN, & (Res.machine));
Pthread_mutex_unlock (&g_mysql_mutex);

snprintf (res.sn, sizeof (RES.SN), "%s", TMP->SN);
}
}

Since TMP is NULL for a long time, this becomes a while dead loop, and the dead loop is very memory-intensive, so add usleep (50000) below to solve the problem

Linux CPU Share High debug