From: http://www.cnitblog.com/houcy/archive/2012/11/28/86801.html
1. Use the top command to see which process is consuming high CPU
The gateway process 14094 consumes up to 891% CPU, which is the cumulative value of each thread in the process that consumes the CPU.
Piduser PR NI VIRT RES SHR S%cpu%mem time+ COMMAND
14094root 0 315m 10m 7308 S891%2.2 1:49.01gateway
20642root 0 17784 4148 2220S 0.5 0.8 2:39.96microdasys
1679root 0 10984 1856 1556R 0.3 0.4 0:22.21sshd
22563root 18 0 24241060 r 0.3 0.2 0:00.03top
1root 18 0 2156 492 460s 0.0 0.1 0:01.59init
2. Use the Top-h-P PID command to see the percentage of CPU consumed by each thread in the process
#top-H-P 14094
You can see 107 threads in top, but the following 9 threads occupy high CPU, with thread 14086 as the main reason why the CPU
Piduser PR NI VIRT RES SHR S%cpumem time+ COMMAND
14086 Root 0 922m914m 538m R 10110.0 21:35.46gateway
14087root 0 922m914m 538m R 10110.0 10:50.22gateway
14081root 0 922m914m 538m S 10.0 8:57.36gateway
14082root 0 922m914m 538m R 10.0 11:51.92gateway
14089root 0 922m914m 538m R 10.0 21:21.77gateway
14092root 0 922m914m 538m R 10.0 19:55.47gateway
14094root 0 922m914m 538m R 10.0 21:02.21gateway
14083root 0 922m914m 538m R 10.0 21:32.39gateway
14088 Root 0 922m 914m 538m R 10.0 11:23.12 Gateway
3. Use the Gstack command to view the function call stacks for each thread in the process
#gstack 14094 > Gstack.log
Looking for thread ID14086 in Gstack.log, because the function stack exposes function details, only two function frames are shown, thread ID14086 corresponding thread number is 37
ThreadPanax Notoginseng (thread0x4696ab90 (LWP 14086)):
#0 0x40000410 in __kernel_vsyscall ()
#1 0x40241f33 in poll () from/lib/i686/nosegneg/libc.so.6
4. Use the Gcore command to dump the process image and memory context
#gcore 14094
This command generates a core file core.14094
5. Use the Strace command to view system calls and time spent
#strace-T-r-c-P 14094
The-c parameter displays statistics, removing this parameter to see the time and return value of each system call.
%time seconds Usecs/call calls errors syscall
----------------- ----------- --------- -------------------------------------
99.99 22.683879 3385 6702 poll
0.00 0.001132 0 6702 gettimeofday
0.00 0.000127 1 208 208 Accept
0.00 0.000022 1 Read
0.00 0.000000 0 1 Write
0.00 0.000000 0 1 Close
0.00 0.000000 0 Time
0.00 0.000000 0 2 Stat64
0.00 0.000000 0 4 Clock_gettime
0.00 0.000000 0 7 Send
0.00 0.000000 0 Recvfrom
----------------- ----------- --------- ---------------------------------------
100.00 22.685160 13652 218 Total
6. Debug the core file with GDB and switch thread to line Line 37
Gcore is almost the same as the core file produced by the actual core dump, except that some dynamic debugging is not possible with GDB
(GDB) GDB Gateway core.14094
(GDB) Thread 37
[Switching to Thread PNS (thread 0x4696ab90 (LWP14086))] #0 0x40000410 in __kernel_vsyscall ()
(GDB) where
#0 0x40000410 in __kernel_vsyscall ()
#1 0x40241f33 in poll () from/lib/i686/nosegneg/libc.so.6
GDB can be debugged according to the detailed function stack, printing some variable values, and combining the source code to analyze why the poll call high CPU.
Because the code involves the company's property rights, GU does not do detailed analysis here, need to understand is the analysis of the process and the use of the command.
Process is: Process id-> thread id-> thread function call stack, function time and call statistics--Source code Analysis
High Linux CPU Usage (RPM)