Linux kernel debugging-I don't tell anyone (1)

Quietly go to Linux kernel debugging (1)

Address: http://blog.csdn.net/cugxueyu/archive/2007/12/21/1957740.aspx

※Debugging is difficult. It is a significant difference between kernel-level development and user-level development.
※The kernel debugging capability depends on experience and the entire operating system.
 
I. Preparations before debugging
Kernel-level bugs have many specific features such as unreliable behavior, unclear definitions, or difficult to reproduce, which makes it very difficult to track and debug kernel-level bugs.
※For bugs with unclear definitions, the key to the problem is to find the source of the bug. In many cases, when you precisely reproduce a bug, you will not be far from the success.
 
Ii. bugs in the kernel
From errors hidden in the source code to bugs displayed in front of witnesses, the attack is often triggered by a series of chain reactions.
Although kernel debugging is difficult, you may like this challenge through your efforts and understanding.
 
Iii. printk ()
The formatting function provided by the kernel.
1. robustness of the printk Function
Robustness is one of the most acceptable features of printk. It can be called by the kernel almost anywhere (Interrupt context, process context, hold lock, multi-processor processing ).
※During system startup, the terminal cannot be called in some places before initialization.
 
2. Record level
The printk function can specify a record level based on which the kernel determines whether messages are printed on the terminal.
The record level is defined in <Linux/kernel. h>:
# Define kern_emerg "<0>"/* system is unusable */
# Define kern_alert "<1>"/* action must be taken immediately */
# Define kern_crit "<2>"/* critical conditions */
# Define kern_err "<3>"/* error conditions */
# Define kern_warning "<4>"/* warning conditions */
# Define kern_notice "<5>"/* normal but significant condition */
# Define kern_info "<6>"/* informational */
# Define kern_debug "<7>"/* debug-level messages */
 
Call method: printk (ker_debug "This is a debug notice! /N ");
The kernel compares the specified record level with the record level console_loglevel of the current terminal to determine whether to print data to the terminal.
 
The console_loglevel definition of <Linux/kernel. h> is as follows:
# Define console_loglevel (console_printk [0])
<Printk. c> definition:
Int console_printk [4] = {
Default_console_loglevel,/* console_loglevel */
Default_message_loglevel,/* default_message_loglevel */
Minimum_console_loglevel,/* minimum_console_loglevel */
Default_console_loglevel,/* default_console_loglevel */
};
 
3. Record Buffer
Kernel messages are stored in a log_buf_len-sized annular queue.
Log_buf_len definition:
# DEFINE _ log_buf_len (1 <config_log_buf_shift)
※The config_log_buf_shift variable is defined by the configuration file during kernel compilation. For the i386 platform, its value is defined as follows (in linux26/ARCH/i386/defconfig ):
Config_log_buf_shift = 18
 
Record Buffer operations:
① When a message is read to the user space, the message will be deleted from the ring queue.
② When the message buffer is full, if another printk () call is made, the new message will overwrite the old message in the queue.
③ When reading and writing a circular queue, the synchronization problem is easily solved.
※This record buffer is called a ring because its read and write operations are performed in a ring queue.
 
4. syslogd and klogd
On a standard Linux system, the user space daemon klogd obtains kernel messages from the Record Buffer, and then stores these messages in the system log files through the syslogd daemon. The klogd process can read messages either from the/proc/kmsg file or through a syslog () System Call. By default, it is implemented in Read/proc mode. The klogd daemon remains in the blocking status until a new message exists in the message buffer. Once there is a new kernel message, klogd is awakened, read the kernel message and process it. By default, the processing routine is to pass the kernel message to the syslogd daemon.
The syslogd daemon generally writes received messages to the/var/log/messages file. However, you can still configure it through the/etc/syslog. conf file, and select other output files.
Figure 1 x-ray shows the process:


Iv. Oops
Oops (also called panic) messages contain details of system errors, such as the content of CPU registers. It is the most common way for the kernel to inform users of the unfortunate occurrence.
The kernel can only publish oops. This process includes outputting error messages to terminals, outputting information stored in registers, and outputting tracing clues for tracking. Generally, after oops is sent, the kernel is in an unstable state.
There are many possible causes for oops, including out-of-bounds memory access or invalid commands.
※As a kernel developer, oops is always processed.
※The important information contained in oops is exactly the same for all systems: register context and trace (trace shows the function call chain that leads to an error ).
 
1. ksymoops
In Linux, the traditional method for debugging system crashes is to analyze oops messages sent to the system console when a crash occurs. Once you have mastered the details, you can send the message to the ksymoops utility, which will try to convert the code into instructions and map the stack value to the kernel symbol.
※For example, the address in the trace will be converted to a visible function name through ksymoops.
Figure 2x the process of formatting the oops message is as follows:


Ksymoops requires several items: Oops message output, system. map file from the running kernel,/proc/ksyms, vmlinux, And/proc/modules.
For more information about how to use ksymoops, see the complete instructions on kernel source code/usr/src/Linux/documentation/oops-tracing.txt or on the ksymoops manual page. Ksymoops disassemble the Code Section to identify the wrong instruction and display a trace section to show how the code is called.
 
2. kallsyms
The development version 2.5 kernel introduces the kallsyms feature, which can be enabled by defining the config_kallsyms compilation option. This option can be used to load the symbolic name (that is, the function name) of the memory address corresponding to the kernel image. Therefore, the kernel can print the decoded tracing clues. Correspondingly, the system. Map and ksymoops tools are no longer required for decoding oops. In addition,
In this way, the kernel will become larger, because the symbol name corresponding to the address must always reside in the memory of the kernel.
# Cat/proc/kallsyms
C0100240 T _ stext
C0100240 t run_init_process
C0100240 t stext
C0100269 t init
...
 
V. kernel debugging configuration options
During kernel compilation, the kernel provides many configuration options to facilitate code debugging and testing.
※Enable options include:
Slab layer debugging (slab layer debugging options), high-memory debugging (high-end memory debugging options), I/O mapping debugging (I/O ing debugging options) spin-lock debugging, stack-overflow checking, and sleep-inside-spinlock checking.
 
1. debug atomic operations
Developed from kernel 2.5, the kernel provides excellent tools to check various problems caused by atomic operations.
The kernel provides an atomic operation counter, which can be configured as: once an atomic operation enters the city to sleep or performs some operations that may cause sleep, print the warning information and provide tracing clues.
Therefore, various potential bugs can be detected, including calling schedule () when using the lock and requesting memory allocation in blocking mode when using the lock.
The following options allow you to take advantage of this feature to the maximum extent:
Config_preempt = y
Config_debug_kernel = y
Config_kllsyms = y
Config_spinlock_sleep = y
 
6. Cause bugs and print information
1. Some kernel calls can be used to easily mark bugs, provide assertions, and output information. The two most common examples are bug () and bug_on ().
Defined in <include/ASM-generic>:

# Ifndef have_arch_bug
# Define bug () do {
Printk ("bug: failure at % s: % d/% s ()! ", _ File __, _ line __, _ function __);
Panic ("bug! ");/* Cause a more serious error, not only printing the error message, but also suspending the entire system industry */
} While (0)
# Endif

# Ifndef have_arch_bug_on
# Define bug_on (condition) do {If (unlikely (condition) Bug ();} while (0)
# Endif

When these two macros are called, they will lead to oops, leading to stack tracing and error message printing.
※These two calls can be used as assertions, for example, bug_on (bad_thing );
 
2. dump_stack ()
Sometimes, you only need to print stack tracing information on the terminal to help you debug the stack. In this case, you can use dump_stack (). This function only prints the register context and function trace clues on the terminal.
If (! Debug_check ){
Printk (kern_debug "provide some information... /N ");
Dump_stack ();
}
Note: Most of the content is introduced in Linux kernel design and implementation-version 2nd

This article from the csdn blog, reproduced please indicate the source: http://blog.csdn.net/cugxueyu/archive/2007/12/21/1957740.aspx