Linux kernel debugging tool

Address: http://blog.csdn.net/ariesjzj/article/details/8244333

Some Linux Kernel Analysis and debugging work, including changing qemu, kprobes, and trace for memo.

 

Qemu source code-level debugging Kernel

1. qemu compilation and Installation

Install the libsdl Development Library first

$./Configure

$ Make

# Make install

Qemu-1.2.1 has tried to work, and earlier versions may have problems at compilation startup. For fear of trouble, you can directly apt-Get install qemu... However, the qemu installed in this way may not work at the breakpoint.

 

2. Linux kernel Compilation

If you want to compile a 32-bit Kernel on a 64-bit machine, you must add the arch = xxx option. For example

# Make arch = i386 defconfig

# Make arch = i386 menuconfig

Then select:

Kernel hacking-> compile the kernel with debug info

Kernel hacking-> compile the kernel with frame pointers

Also selected

File System-> the extended 4 (ext4) filesystem

Otherwise, some IMG may fail to mount root at startup.

# Make arch = i386 bzimage

After the end, generate ARCH/x86/boot/bzimageand vmlinux, and then q-0.2.img.bz2 and unzip it to get the linux-0.2.img. Run

$ Qemu-kernel bzimage-hda linux-0.2.img-append root =/dev/sda rw-s

After-S is added, stop at Port 1234 and wait for the debugger connection. Therefore, start GDB client:

$ GDB vmlinux

(GDB) target remote localhost: 1234

Then you can call the kernel as you call the app.

Note that in addition to the test image provided by qemu, the root file system image can also be made using busybox. If it is too troublesome, you can use buildroot to automatically download and compile the Linux kernel and rootfs.

One advantage of qemu tuning is that you can also use qemu's monitor to view system information (http://doc.opensuse.org/products/draft/SLES/SLES-kvm_sd_draft/cha.qemu.monitor.html ). For example, viewing control register, TLB, or memory mapping is very useful during debugging.

Kprobes, jprobes, dprobes

The xprobes tool adds a hook to the specified address or function location as a module, and then you can call a Custom Handler. Kprobes is used to intercept the address (of course, more often it is based on the symbol location), jprobes is used to intercept the function, it is more convenient to view the function parameters, because its handler prototype is the same as the function to be intercepted. Dprobes can be executed dynamically.

 

Most of the time, we use the symbol to add a hook. the symbol of the kernel can be found in the following three ways:

1. The system. map file compiled by Kernel

2. $ nm vmlinuz

3./proc/kallsyms

 

View the current kprobes:

# Cat/sys/kernel/debug/kprobes/List

 

Close

# Echo "0">/sys/kernel/debug/kprobes/Enabled

Enable

# Echo "1">/sys/kernel/debug/kprobes/Enabled

 

Linux Kernel provides the Documentation/kprobes.txt and provides several examples under samples/kprobes. Use the following makefile for separate compilation:

OBJ-M + = jprobe_example.o kprobe_example.o

Kdir =/lib/modules/$ (shell uname-R)/build

ALL:

$ (Make)-C $ (kdir) subdirs = $ (PWD) Modules

Clean:

Rm-RF *. O *. Ko *. Mod. *. C *. T *

 

After compilation:

# Insmod./kprobes_example.ko

Then run the dmesg command to view the handler output, for example:

[2, 1251.789211] planted kprobe at c1058e30

[1256.716389] pre_handler: p-> ADDR = 0xc1058e30, IP = c1058e31, flags = 0x246

[1256.716394] post_handler: p-> ADDR = 0xc1058e30, flags = 0x246

[1262.680009] pre_handler: p-> ADDR = 0xc1058e30, IP = c1058e31, flags = 0x246

[1262.680024] post_handler: p-> ADDR = 0xc1058e30, flags = 0x246

...

[1468.928704] planted Jprobe at c1058e30, Handler ADDR e08ff000

[1471.688212] Jprobe: clone_flags = 0x1200011, stack_size = 0x0, regs = 0xcef87fb4

[1474.153652] Jprobe: clone_flags = 0x1200011, stack_size = 0x0, regs = 0xcef87fb4

 

Generate kernel core dump

Kernel support is required for Kernel core dump. The documentation/kdump.txt file in Linux kernel is described in detail.

Linux-crashdump can be installed on Ubuntu, including crash, kexec-tools, and makedumpfile. Restart the host after installation. Press shift to go to grub during startup, and press e to see that the startup option has more crashkernel = xxx. After entering the system

# Service kdump start

Enable the kdump process. When the system crashes, a dump file is generated under/var/crash. The simple way to make the system crash is to use magic sysrq:

Sysrqalt + sysrq + S // Sync

Sysrqalt + sysrq + C // force crash

Analyze core dump and Real-Time View System

Crash can be used to view the core dump or the running status in real time. Assuming the kernel core dump in/var/crash/linux-image-2.6.32-38-generic.0.crash. First, extract the vmcore file from the crash file:

# Apport-Unpack/var/crash/linux-image-2.6.32-38-generic.0.crash ./

Check with Crash:

# Crash vmlinux./vmcore

Here, vmlinux is the binary compiled by kernel source.

 

Crash can also be used to view the currently running system in real time and run directly:

# Crash vmlinux

Actually equivalent

# Crash vmlinux/proc/kcore,/proc/kcore is the core file virtualized by the system.

 

GDB and DDD can also be used for the same function, for example, to view the system in real time:

# DDD vmlinux/proc/kcore

You can perform a limited GDB debugging on the system.

 

Kernel trace

The trace function of the kernel depends on debugfs. If it is not mounted, it must be mounted first. The main trace files are in/sys/kernel/debug/tracing. Its usage is detailed under Documentation/trace in the kernel. At the same time, the kernel provides related examples sample/trace_events.c.

 

Several important files under/sys/kernel/debug/tracing:

Trace: Tracer output

Available_tracers: available Tracer

Current_tracer: Current enabled Tracer

Tracing_enabled: Switch

 

Example:

# Echo function> current_tracer

# Echo 1> tracing_enabled

# Cat trace> trace.txt

# Echo 0> tracing_enabled

# Cat./trace.txt

The result is as follows:

...

Xorg-896 [000] 212.518803: do_softirq <-irq_exit

Xorg-896 [000] 212.518806: _ do_softirq <-do_softirq

Xorg-896 [000] 212.518813: run_timer_softirq <-_ do_softirq

Xorg-896 [000] 212.518816: hrtimer_run_pending <-run_timer_softirq

Xorg-896 [000] 212.518818: _ raw_spin_lock_irq <-run_timer_softirq

Xorg-896 [000] 212.518821: rcu_bh_qs <-_ do_softirq

Xorg-896 [000] 212.518824: _ local_bh_enable <-_ do_softirq

Xorg-896 [000] 212.518826: rcu_irq_exit <-irq_exit

Xorg-896 [000] 212.518828: rcu_enter_nohz <-rcu_irq_exit

Xorg-896 [000] 212.518831: idle_cpu <-irq_exit

...

 

Note: The above tools depend on the feature of some kernel, such as debugfs and sysrq. to use these tools, you must add them during kernel compilation.

Debugfs presents kernel information in a memory-based file system, making it easy for users to interact with it. For more information, see the documentation/debugfs.txt file of kernel. Mount through

# Mount-T debugfs nodev/sys/kernel/debug

 

Sysrq provides some hotkeys for development and debugging of kernel. For details, see the documentation/sysrq.txt of kernel.

# Echo 1>/proc/sys/kernel/sysrq

Or set/etc/sysctl. conf to kernel. sysrq = 1.

 

The above is just a glimpse of the kernel debugging tool, and there are many artifacts that cannot be listed, such as kgdb, KDB remote debugging kernel, kmemcheck, faultinjection, and so on.