Debugging skills summary and debugging skills

Debugging skills summary and debugging skills

The following is a summary of the debugging technology of experts from the debug hacks book.

1. Use strace skills

Strace name: run the program in this way. You can see that the system call is only a system call when the program is running. You can see the parameter passing when the system call fails, or where the function is located.

-I option shows the address of each system call, so that you can add a breakpoint when using gdb for debugging.

-P option can attach a running program

-O can specify the output file

-T and-tt can specify the system call time in seconds and milliseconds, respectively.

2. Use objdump

After the objdump disassembly, it is often difficult to see which line of code corresponds to the c program. In this case, you can specify the-S and-l options to display the code and line number in the source file respectively, the program must contain debugging information, preferably a file without optimization options, but it may not be completely matched and can be used as a reference.

3. valgrind usage skills

Valgrind can evaluate the cache and heap, and detect POSIX thread conflicts.

The most common memory detection tool -- tool = memcheck is valgrind's default tool, which can be left unspecified.

The detected content includes memory leakage, illegal memory access, uninitialized areas read, released areas accessed, dual memory release, and illegal stack operations. However, valgrind is not very good at stack space detection.

4. Use of kprobe

This is a kernel debugging technology. You can add printing in any function or perform any other processing without re-compiling the kernel. Of course, you must have the kernel source code to perform a proper operation.

The better thing is that stack tracing can be displayed. This is a good technique in debugging.

5. Use of jprobe

Similar to kprobe, jprobe can detect the usage of any kernel function. However, jprobe has the advantage that the parameters of the detection function are the same as those of the function to be detected. It is convenient to print out the passing parameters, unlike kprobe, it requires stack or register reasoning.

In fact, I think the above two tools correspond to gdb as a breakpoint.

6. The power of kprobe

Kproble is powerful. He can insert the kernel at any position. Unlike jprobe, he can only insert it at the beginning of the function, including before executing a command after being inserted into a certain command.

7. kprobe replacing kernel functions

Kprobe can replace a function in the kernel, so that you can debug a function without re-compiling the kernel.

8. Replacing the application functions with KAHO

Similar to the previous kprobe function, this saves you the trouble of re-compiling large applications.

9. Use of systemtap

This tool is implemented using kprobe, but it is similar to the script language and is more convenient. Functions include viewing stacks and internal data. The gdb tool is used for application debugging.

10./proc/meminfo

This can be used as a memory detection. Compared with valgrind, valgrind must provide the test results at the end of the program running, but this can be seen directly in real time.

11./proc/<pid>/mem to quickly read process content

Like gdb or ptrace, it is used to view memory, but the speed is faster.

12. oom killer

When the memory is insufficient, the system scores each application process, and the highest score is disabled.

13. Error Injection

In general, malloc will be successful, but it is difficult to detect some errors caused by allocation failure. This function increases the probability of allocation failure, or specifies that the allocation fails.

You need to connect to a failmalloc library. This facilitates test failure.

This is very convenient. First download his code from the official website of failmalloc and compile and install it,

Specify the LD_PRELOAD parameter of env as the Directory and name of the database at each run,

The other database supports the specified options, which have four

FAILMALLOC_PROBABILITY

Specifies how often it shoshould fail between 0.0 and 1.0.

This option indicates the probability of failure.

FAILMALLOC_INTERVAL

Specifies the interval of failures.

This option indicates the number of malloc failures.

FAILMALLOC_TIMES

Specifies how many times failures may happen at most.

Maximum number of failures

FAILMALLOC_SPACE

Specifies the size of free space where memory can be allocated safely in bytes.

Specify the maximum number of failed applications for memory. If the value is lower than or equal to this value, the application fails. If the value is exceeded, the application is successful.

14. Use of oprofile

This tool can view the performance of a program, such as l2 cache hit and running time of each function, and can generate charts.

The most common is the running time of each function.

Similar Tools include gprof, but the functions are much inferior.

Additionally, oprofile does not support event-based counting under virtual machines. Obviously, the running time detection of each function is not supported.

The following describes how to use an oprofile:

Source code:

　　　　

# Include <stdio. h>

Int fun (int s, int I)
{
Printf ("s = % d \ n, I = % d \ n", s, I );
S = s + I;
Return s;
}

Int main ()
{
Int I = 0;
Int sum = 0;
For (; I <0x10000; I ++)
Sum = fun (sum, I );
Printf ("sum = % x \ n", sum );
Return 0;
}

Next, initialize the oprofile.

[Root @ localhost oprofile-1.1.0] # opcontrol -- init

Specifies the listening event. Here, the default event is used. sampling is performed under the cpu clock. Every 10000 clocks are collected once. The kernel is not recorded and only applications are recorded.

[Root @ localhost oprofile-1.1.0] # opcontrol -- event = CPU_CLK_UNHALTED: 10000: 0: 0: 1

Start Analysis

[Root @ localhost oprofile-1.1.0] # opcontrol -- start
Using 2.6 + OProfile kernel interface.
Using log file/var/lib/oprofile/samples/oprofiled. log
Daemon started.
Profiler running.

Run the program and stop the analysis.

[Root @ localhost oprofile-1.1.0] # opcontrol -- stop

View results

[Root @ localhost oprofile-1.1.0] # opreport -- merge = cpu-d a. out
CPU: Core 2, speed 2666.13 MHz (estimated)
Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a unit mask of 0x00 (Unhalted core cycles) count 10000
Processes with a thread ID of 32720
Processes with a thread ID of all
Vma samples % symbol name
004004c4 226 61.9178 756 59.9524 fun
004004c4 12 5.3097 34 4.4974
004004cc 4 1.7699 25 3.3069
004004da 2 0.8850 17 2.2487
004004e7 4 1.7699 20 2.6455
004004ec 23 10.1770 97 12.8307
004004ef 130 57.5221 378 50.0000
004004f2 32 14.1593 96 12.6984
004004f5 19 8.4071 89 11.7725
004004f7 139 38.0822 505 40.0476 main
0040050f 3 2.1583 18 3.5644
00400519 3 2.1583 23 4.5545
0040051e 7 5.0360 25 4.9505
00400521 92 66.1871 299 59.2079
00400525 9 6.4748 33 6.5347
0040052c 25 17.9856 107 21.1881

Here, a. out specifies the image and only views the function of the program. It can be seen that the main function and the fun function each account for the current running proportion. This is just a simple example. For a large program, you can optimize the function based on this result,

 

You can also view the code-level analysis results.

[Root @ localhost oprofile-1.1.0] # opannotate -- merge = cpu-s a. out

/*
* Command line: opannotate -- merge = cpu-s a. out
*
* Interpretation of command line:
* Output annotated source file with samples
* Output all files
*
* CPU: Core 2, speed 2666.13 MHz (estimated)
* Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a unit mask of 0x00 (Unhalted core cycles) count 10000
* Processes with a thread ID of 32720
* Processes with a thread ID of all
*/
/*
* Total samples for file: "/root/czh/oprofile-1.1.0/test. c"
*
* 365 100.000 1261 100.000
*/

: # Include <stdio. h>
:
: Int fun (int s, int I)
16 4.3836 59 4.6788: {/* fun total: 226 61.9178 756 */
6 1.6438 37 2.9342: printf ("s = % d \ n, I = % d \ n", s, I );
153 41.9178 475 37.6685: s = s + I;
32 8.7671 96 7.6130: return s;
19 5.2055 89 7.0579 :}
:
:
: Int main ()
: {/* Main total: 139 38.0822 505 40.0476 */
: Int I = 0;
: Int sum = 0;
126 34.5205 439 34.8136: for (; I <0x10000; I ++)
13 3.5616 66 5.2339: sum = fun (sum, I );
: Printf ("sum = % x \ n", sum );
: Return 0;
:}

You can clearly see which line of code occupies the most time.

 

NOTE: If running in a virtual machine, event-based sampling is not supported and can only be used based on time. However, the sampling rate is too low and the effect is poor.

Run modprobe oprofile timer = 1 before loading the module

You can use dmesg to check whether timer runs

 

15. vprobe

No relevant information found

16. Check whether the x86 machine supports 64-bit

In this case, you can view the register content of the cpu or the content in/proc/cpuinfo.