Kernel debugging systemtap

ArticleDirectory

• 1. First check your kernel version

Related Technologies: utrace, probe, ftrace, dtrace, pstrace

• Install systemtap on centos
• Use instance
• Reference

Original Text connection

Reference connection

The key point is kernel matching.

1. First check your kernel version

Uname-

2.6.18-194. EL5

 

If your kernel version is old, you need to find the kernel-devel version you need.

If you want to use yum for installation, you can use Yum install kernel-devel

 

Note: The latest kernel-devel version is installed in the latest yum. The kernel-devel version must match the kernel version.

For example, centos release 5.5 (final) uses kernel version 2.6.18-194. EL5.

First download the corresponding

Then install

Rpm-IVHKernel-devel-2.6.18-194.el5-i686.rpm

 

After kernel-devel is installed, the kernel directory is located in

/Usr/src/kernels/2.6.18-194. el5-i686/

 

2. Install systemtap

Yum install systemtap (STAP-V 1.3/0.137 please install elfutils-devel to prompt which version of debuginfo you need to install)

 

3. Install debuginfo

Http://debuginfo.centos.org

Find and youKernel exact match

(For example, the local uname-r 2.6.18-194. EL5 can only be selected.

Kernel-debuginfo-2.6.18-194.el5.i686.rpm

Kernel-debuginfo-common-2.6.18-194.el5.i686.rpm

)

Kernel-debuginfo-common-XXXXX

Kernel-debuginfo-xxxx

 

If you cannot find the RPM of the corresponding kernel version, you can search for it by Google. You can use the debuginfo RPM package of RedHat.

 

RedHat kernel reference download

Http://rpm.pbone.net/index.php3/stat/4/idpl/13968571/dir/redhat_el_5/com/kernel-debuginfo-common-2.6.18-128.el5.x86_64.rpm.html

Ftp://ftp.pbone.net/mirror/ftp.redhat.com/pub/redhat/linux/enterprise/5Server/en/ OS /x86_64/Debuginfo/kernel-debuginfo-2.6.18-128.el5.x86_64.rpm

Install rpm-IVH kernel-debuginfo *. rpm

 

How to test:

STAP-ve 'probe begin {log ("Hello World") Exit ()}'

 

Remarks

  Officially recommended installation commands
2 Yum install systemtap kernel-devel Yum-utils
Debuginfo-install KernelBecause no kernel-debuginfo package exists on my centos Yum repository, You have to manually add the repository with this package. Of course, you can also download this package and its dependent package RPM file to your local device for installation.
In/Etc/yum. Repos. D to create a new file, run the command debuginfo
Then add it to debuginfo.
1
2
3
4
5
6 [debuginfo]
Name = centos-$ releasever-debuginfo
Base url = http://debuginfo.centos.org/5/
Gpgcheck = 0
Enabled = 1
Gpgkey = file: // etc/pki/rpm-GPG/RPM-GPG-KEY-CentOS-5 and then execute the officially recommended installation command. It takes a long time to download over 300 m files.

 

Official wiki:

Http://sourceware.org/systemtap/wiki/SystemTapOnCentOS

 

How does systemtap track libc. So?

In the afternoon, I had a hard timeProgramMemory leakage problem. valgrind, gogle perftools and other tools are not very easy to use, so it is easy to get rid of the application, so we plan to use systemtap to understand the memory usage at the libc. So layer. The main idea is to check the balance between the number of malloc/realloc and free calls.

First, prepare the environment. The system is the standard RHEL 5u4:

 

$ Uname - R  2.6 . 18 - 164  . EL5 $ STAP - Vsystemtap Translator /Driver (Version 1.3 / 0.137 Non- GIT sources) Copyright (c)  2005 - 2010  Red Hat, Inc. And othersthis is Free Software; see the source For  Copying conditions. enabled features: librpm libsqlite3 NSS boost_shared_ptr tr1_unordered_map $ STAP -L '  Kernel. function ("printk ")  '  Kernel.  Function ( "  Printk @ kernel/printk. C: 533  " ) $ FMT : Char Const *$ ARGs: va_list $ STAP -L '  Process ("/lib64/libc. so.6"). function ("malloc ")  '  Missing separate debuginfos, use: debuginfo - Install Glibc- 2.5 - 42 . X86_64

 

 

The kernel symbol is OK, and glibc does not have the installation symbol. The system prompts to use the debuginfo-install glibc-2.5-42.x86_64 command to install the symbolic information, but RHEL 5 does not pay for this service, you can only download the package installation.

 

$ Wget -C FTP Slave- 2.5 - 42  . X86_64.rpm $  Sudo Rpm-I glibc-debuginfo- 2.5 - 42  . X86_64.rpm $ STAP -L '  Process ("/lib64/libc. so.6"). function ("malloc ")  '  Process (  "  /Lib64/libc-2.5.so " ). Function ( "  _ Libc_malloc @/usr/src/debug/glibc-2.5-20061008T1257/malloc. C: 3560  " ) $ Bytes: size_t

 

 

With the glibc symbol, you can easily track the usage of malloc in libc. So.
Next we will simply write a C program to call malloc, and write a STAP script to track the call stack of malloc:

$ Cat  T. C # include <Stdlib. h> Void fun () {malloc (  1000  );}  Int Main (Int Argc, Char * Argv []) {fun (); Return  0  ;} $  Cat  M. STP probe process (  "  /Lib64/libc. so.6  " ). Function ( "  Malloc  "  ){  If (Target () = PID () {print_ubacktrace (); exit () ;}} probe begin {println (  "  ~  "  );} $  Gcc - G t. C $ STAP -L '  Process ("./A. Out"). function ("*")  '  Process (  "  /Home/chuba/a. Out  " ). Function ( "  Fun @/home/chuba/T. C: 3  "  ) Process (  "  /Home/chuba/a. Out  " ). Function ( "  Main @/home/chuba/T. C: 7  " ) $ Argc: Int $ Argv: Char **

 

 

Now that the program is ready, let's execute it to see where the memory leaks:

$SudoStap m. STP-C ./A. Out~0x33d5e74b96: Malloc +0x16/Zero X 230[Libc-2.5. So]0x4004a6[A. out +0x4a6/Zero X 1000]
 

 

We can see that malloc is called in 0x4004a6 of A. Out, but what is the specific line in the program? It is easy to find out with add2line:

$ Addr2line-E./A. Out 0x4004a6 /Home/chuba/T. C: 5  $  NL  T. C 1 # Include <stdlib. h> 2  Void fun (){  3 Malloc ( 1000  );  4  }  5    Int Main ( Int Argc, Char * Argv []) {  6  Fun (); 7 Return 0  ;  8 }

 

 

Reference connection

Erlang non-amateur research

IBM Linux self-check and systemtap

A new performance measurement and tuning diagnostic tool systemtap series in Linux

Systemtap Manual

RedHat systemtap introduce (PDF) Library Link

Ftp://ftp.redhat.com/pub/redhat/linux/enterprise/5Server/en/ OS /i386/Debuginfo