Deep parsing kgdb debugging Linux modules and kernels

Reprint article please specify the author and QR code and full text information.

Programmers who do not program, not good architects, programming and kernel debugging are also required for a great architect. Talking about the number of programmers, software engineers based on the Linux platform are certainly the most, not one. Today, let's take Linux as an example, in-depth talk about kernel modules and kernel debugging techniques and debugging Tools Kgdb.

1 Kgdb Background

KGDB is officially supported in the kernel 2.6.26 version, the corresponding release version is SLES11 and above, RHEL6 and above, before the kernel version by Linsyssoft Technologies Company to provide patches to support kgdb, but not all kernel versions have patches available , while patching operation is also more cumbersome and problematic, so the availability is not high.

2 Commissioning Environment Construction

Note: The following is called "the host being debugged" as the target machine, the host running GDB for debugging is the development machine

2.1 Target machine configuration

2.1.1 Configuring the serial port

Physical machine serial port according to the actual environment requirements configuration, the virtual machine is configured as follows, pipe name can be modified, but to ensure consistent with the development machine:

2.1.2 Update kernel to support KGDB

Note: This article takes SLES11SP1 as the target machine as an example, the kernel source code directly installed RPM package can be used, Rhel to a little trouble, need to download the source package, compiled after the installation.

Prepare before updating the kernel

After adding debugging information, the size of the kernel and each KO will increase several times, so before compiling the kernel must confirm that the disk has more than 7G of space (the insurance is recommended to reserve 10G), do make after the source directory space occupies more than 5G.

/lib/modules directory also takes 1.4G after executing make Modules_install

SLEs series default kernel source directory is/usr/src/linux-xxx/, but because the trial of the virtual machine created when the disk selection default size of only 8G, so the additional creation of a 20G disk mounted to the/home directory as the kernel compilation directory, you can directly directory usr/src/ linux-xxx/copying to/home/linux-xxx/does not affect compilation.

Update kernel steps

1, execute uname–r confirm the type of current running kernel, copy/boot/ The config file corresponding to the kernel type in the directory is renamed to the kernel source directory and then to. config, and most of the time the kernel fails to start because the kernel is improperly configured, so it is best to modify it based on the original configuration file of the system.

2, in the kernel source directory to execute make menuconfig kernel configuration;

• Enter the kernel hacking sub-option to confirm activation of the following items:

[*] Compile the kernel with debug info

[*] Compile the kernel with frame pointers

[*] Kgdb:kernel Debugging with remote GDB

• Clear the Write protect kernel read-only data structures option, which is activated by default, resulting in the subsequent use of GDB debugging without breakpoints;

The compilation error after removing the write protect kernel read-only data structures on SLES11SP1 is due to the function Mark_rodata_ro in init/ MAIN.C and CACHEFLUSH.H are defined repeatedly in the

The solution is to note the definition in MAIN.C:

3. Execute make all to compile the kernel, (it takes about 1 hours, can use Make–j x all to speed up the compilation speed, x indicates the number of threads)

4, the installation module, after the completion of the compilation, the newly generated module Ko is also in the source directory, not updated to the/lib/module/corresponding directory:

• Note that it is strongly recommended to back up the original module directory before installing the module so that when debugging is complete or the new compilation module has a problem, the environment is restored as follows.

• Perform make modules_install (note: Do not modules install) will copy Ko to/lib/module/

5. Create boot kernel and INITRD

• Note: It is still highly recommended to back up the original vmlinuz and initrd files in the/boot/directory, since the kernel install script will be automatically backed up, but if the install executes two or more times, the previous backup will be overwritten with the new backup.

• Set/etc/modprobe.d/unsupported-modules in Allow_unsupported_modules to 1, otherwise the newly compiled module KO may fail to load:

• Executing make install will copy the Vmlinux from the source directory to the/boot/directory and compress it to vmlinuz and create the INITRD:

6. Create a new startup item for the KGDB kernel

• Note: It is strongly recommended that you back up the original startup item first and specify the kernel and INITRD files used by the original boot entry as the previously backed up files:

• The new KGDB startup item has only one parameter added compared to the original startup item: KGDBOC=TTYS0,115200

• If the target machine needs to be disconnected (such as the code to debug the start-up phase), add another parameter kgdbwait

7. Restart the target machine and start with the KGDB option.

2.2 Development machine Configuration

The development machine does not need to be the same as the target hardware or the kernel, as long as the above version of GDB to meet the requirements of kgdb. This article uses a SLES10SP4 32-bit virtual machine as the development machine.

2.2.1 Configuring the serial port

Physical machine serial port according to the actual environment requirements configuration, the virtual machine is configured as follows:

Check the parameters to verify that the serial port configuration is correct:

• 1, in the target machine execution cat/dev/ttys0;

• 2. Execute echo Test >/DEV/TTYS0 in the development machine

• 3. Observe if the target machine prints test words;

2.2.2 Preparing the debug code and target binaries

Debug code

Since GDB debugging requires source files, it is necessary to copy the kernel source code to the development machine. It is recommended to copy the whole source directory to the development machine before compiling the target machine, otherwise the whole source directory is too large after compiling.

Target binary file

The target binaries are the files to be debugged, such as Vmlinux or Xxx.ko, which directly copy the compiled files from the target machine to the development machine, which is recommended to be placed in the kernel source directory.

3 Debugging Steps

3.1 Debug Kernel Vmlinux

Taking the function get_request_wait of the block layer of debugging function as an example

1. Performing echo g >/proc/sysrq-trigger on the target machine will trigger the target machine to hang to wait for the development machine input;

2. Start GDB in the development machine:

3. Set up Remote debugging

Enter the following two commands in the GDB interface, and the successful words will be broken in the Kgdb_breakpoint function:

• Set Remotebaud 115200

• Target REMOTE/DEV/TTYS0

4, enter B get_request_wait for the function we want to debug set a breakpoint (b means breakpoint), and then execute C (continue) to let the target machine continue to run until the breakpoint;

5, look at the call stack (BT) and single-step debugging (n) are more useful means;

To view the call stack for the function get_request_wait:

Single-Step Debugging:

• The example code executes to RQ = Get_request (q, Rw_flags, bio, Gfp_noio); before this line;

• Execute p RQ Print pointer variable RQ address display value optimized out represents null;

• Execute p *RQ print pointer variable RQ content display cannot access 0x0 address;

• Execute n Let RQ = get_request (q, Rw_flags, bio, Gfp_noio);

• Execute P RQ again to successfully print out the pointer variable RQ address;

• Execution p *RQ successfully prints out the contents of the pointer variable RQ;

6, after the completion of debugging to clear breakpoints to restore the target machine to normal operation;

• Execute Info B To view the current breakpoint;

• Execute D Breakpoint 1 clear breakpoint 1;

• Execute c to restore the target machine to run;

The network is interrupted before the target is suspended and can be re-logged after recovery:

3.2 Debug Module ko

Take the Debug module Scsi_mod.ko as an example:

1, first on the target machine to see the module offset address in the kernel, and then suspend the target machine:

2. Start GDB in the development machine and execute add-symbol-file [module KO] [kernel address] load module KO file:

The next steps are the same as debugging the kernel vmlinux: Start remote debugging, set breakpoints ...

4 Summary

Use Kgdb, on the one hand can help read kernel code, actually observe the process of code execution, on the other hand can help the non-self-research module related process problem localization, do not need to add the printing re-compilation kernel repeatedly, improve the problem location efficiency. This paper mainly describes the construction of the KGDB environment and the steps to start debugging, more GDB debugging tips please refer to the GDB manual.

The focus of the environment is to update the kernel, this is the whole process of the most time-consuming and error-prone, the project team can organize the division of the various versions, types of kernel kgdb update (such as SLES11 32-bit/64-bit, RHEL, etc.) and save, subsequent use can be copied directly. Please search "ict_architect" to join the public number "architect Technology Alliance" for more exciting content.

Deep parsing kgdb debugging Linux modules and kernels