First, write a personal favorite debugging skills.
1. Open Config_debug_kernel in Menuconfig
2. objdump-d-S (uppercase) *.O > file can get mixed C and assembly code
Or make *.lst can get it.
3. Addr2line-f-E vmlinux address (0xcxxxxxxxx) to get the address corresponding to the function name and the number of rows in the file
4. Based on oops information, view the value of R13 (SP), R14 (LR), R15 (PC) register, ARM architecture
? printk
? Debug support in the kernel
? debug system failure by monitoring debug –oops message – System hangs? Debugger and related tools – Use GDB–KDB kernel debugger –kgdb Patch – User mode Linux virtual machine –linux Tracking Toolkit – dynamic probing
1. PRINTK (most basic used)
The robustness of the PRINTK () function – can be invoked anywhere, at any time. (Interrupt, process, hold lock, multi-processor) – vulnerability: cannot be used until terminal initialization. At this point, the possible method is to use EARLY_PRINTK () or direct serial operation. – If no level is specified, the default default_message_loglevel is kern_warning. Modifications can be "echo n >/proc/sys/kernel/printk"
2. Debugging support in the kernelKernel developers have built multiple debugging features in the kernel itself. The following is a list of configuration options commonly used by the development kernel, which typically appears in the kernel Configuration tool "kernel hacking" menu. Config_debug_kernel #这个选项相当于总开关? Config_debug_slab #这个重要的选项打开了内核内存分配函数的检查? Config_debug_pagealloc #页在释放时被从内核地址空间去除.? Config_debug_spinlock #内核捕捉对未初始化的自旋锁的操作, and a variety of other spin lock errors? Config_debug_spinlock_sleep #激活对持有自旋锁时进入睡眠的检查. ? Entries for the Config_init_debug #用__init (or __initdata) flag are discarded after the system is initialized or the module is loaded. Can be used to check for access attempts to initialize the memory space after initialization. Config_debug_info #使得内核在建立时包含完整的调试信息. If you want to debug the kernel using GDB, you will need this information. If you intend to use GDB, also activate Config_frame_pointer. Config_magic_sysrq #激活 "Magic SysRq" key.? Config_debug_stackoverflow? Config_debug_stack_usage #能帮助跟踪内核堆栈溢出.
The first option adds a clear overflow check to the kernel; The 2nd one makes the kernel monitoring stack use and makes some statistics, which can be obtained by magic SysRq key. Config_kallsyms #使得内核中包含符号信息; Symbol options are used in the debug context; Without it, a oops list can only give you a kernel trace in 16 binary format. Config_ikconfig? Config_ikconfig_proc is #使得完整的内核配置状态被建立到内核中 and can be accessed through/proc. The kernel you develop yourself is not needed. Config_acpi_debug #打开详细的 ACPI debugging information.? Config_debug_driver #打开了驱动核心的调试信息 that can be used to track low-level support code issues.? Config_scsi_constants #建立详细的 information about SCSI error messages.? Config_input_evbug #打开输入事件的详细日志. ? Config_profiling #通常用在系统性能调整, but also useful for tracing some kernel hangs and related issues.
3.
Debugging by monitoring
? The Strace command is a very powerful tool that can display all system calls made by user space. – not only the call is displayed, but also the parameters and return values of the call are displayed in symbolic form. When a system call fails, the wrong symbolic value (for example, Enomem) and the corresponding string (out of memory) are displayed. –strace has a lot of command-line options; Which of the most useful is? -T to show the time of each call execution? -T to show the time spent in the call? -E to limit the types of tracked calls? -O to redirect output to a file. By default, Strace prints the call information to stderr.
4. Debug system Failure? Oops message –
most bugs refer to NULL pointers or use other incorrect pointer values.The usual output of such a bug is a oops message. – A oops shows the processor state at the time of the error, including the CPU register contents and other seemingly incomprehensible information. – The call stack for symbols can be seen only when your kernel is turned on with the config_kallsyms option. Otherwise, if you see a bare 16-binary list, unless you decode it in a different way. System hangs – although most bugs in kernel code end with oops messages, sometimes they may completely suspend the system. If the system hangs, no messages can be printed out and the system does not respond to any action, including ctrl-alt-del– the "Magic SysRq key" is an essential tool for the above situation and is available on most systems. The Magic key SYSRQ is a combination of ALT and SYSRQ keys on a PC keyboard – turns on the SYSRQ function:? echo 0 >/proc/sys/kernel/sysrq
5. Debugger and related toolsUsing GDB–GDB is very useful for looking inside the system. At this level, proficiency in the use of the debugger requires mastering the GDB command, understanding the assembly code of the target platform, and the ability to match the source code and the optimized target. – Typical call:? gdb/usr/src/linux/vmlinux/proc/kcore– can use the GDB command to get information such as? P global_variable? Print * (Addre SS) – Compile Kernel-G to provide more information, such as printing members in a struct and tracking a pointer
? KDB Kernel Debugger – Why the kernel does not have more advanced debugging features in it. Is the Linus does not trust the interactive debugger. He feared that they would lead to undesirable changes that patched the problem rather than the real cause of the problem. –KDB built-in kernel debugger comes from an unofficial patch from oss.sgi.com. To use KDB, you must obtain this patch to apply it, recompile and install the kernel. Only x86 available – Start KDB: One is to add "Kdb=on" at startup, and the other is to enter the following command after the proc file system is loaded: #echo 1 ">/proc/sys/ Kernel/kdb.
Then you can press the "Pause" key to enter the debugging environment. –KDB provides a wealth of commands for running control, memory manipulation, register manipulation, breakpoint setting, stack trace, and many more
? Kgdb Patch – Isolate the system running the debug kernel from the system running the debugger. The two systems connect using a serial cable. – You can have the full-featured GDB run against the kernel. GDB needs to be set up and installed. – Support for X86,superh,ia64,x96_64,spar and 32-bit PPC architectures – detailed descriptions under Documentation/i386/kgdb
? user-mode Linux virtual machine –UML uses the Linux kernel to run as a standalone user-mode process on a Linux system – benefits:? Kernel errors do not break the "real" system? It's easy to use a debugger like GDB for user-mode Li Nux processing – Cons:? The user-mode kernel cannot access the hardware of the host system and cannot debug real hardware-related drivers –http://user-mode-linux.sf.net/more information about UML
? The Linux Tracking Toolkit –linux Trace Toolkit (LTT) is a kernel patch and a set of related tools that allow you to track events in the kernel. The trace includes time information that can create a complete description of what happened in a given time period.
Therefore, it can be used not only for debugging, but also for tracking performance issues. –LTT documents together, you can Http://www.opersys.com/LTTFound it
? Dynamic detection–dynamic Probes (dprobes) is a debug tool for Linux issued by IBM (under the GPL) for the IA-32 system. It allows the placement of a "probe" in almost any part of the system, both user and kernel space.– The probe is composed of some code (written in a special, stack-oriented language) that executes code when the control hits a given point. Code can report information to user space, change registers, or do many other things.– The probe can be inserted anywhere in a running system without a kernel rebuild or reboot. Dprobes can assist LTT to insert new trace events at any location.–dprobes tools can be downloaded from IBM's Open source website: http://oss.sof-ware.ibm.com.
LDD Reading Notes _ Debugging Technology
