Use Strace and truss to assist in debugging programs

The Strace command is a powerful tool that can display all system calls made by user space programs.
Strace Displays the parameters of these calls and returns the value of the symbolic form. Strace receives information from the kernel and does not need to build the kernel in any particular way.
There are several common option records below.
1-f-F option tells Strace to track fork and vfork processes at the same time
2-o xxx.txt output to a file.
3-e Execve only records execve such system calls
—————————————————
The process does not start, the software slows down suddenly, the program "Segmentfault" and so on are the problem that each UNIX system user is having a headache,
This article demonstrates how to use the three common debugging tools truss, Strace, and ltrace to quickly diagnose the "incurable diseases" of software through three practical cases.
　　
　　
Truss and Strace are used to track the system call or signal generation of a process, while Ltrace is used to track the process call library functions. Truss is a debugger that was developed early for system V R4, and most Unix systems, including AIX and FreeBSD, have this tool in their own.
And Strace was originally written for the SunOS system, Ltrace first appeared in Gnu/debianlinux.
Both tools are now ported to most UNIX systems, and most Linux distributions come with Strace and Ltrace, and FreeBSD can also install them through ports.
　　
Not only can you debug a new starting program from the command line, you can also truss, strace, or ltrace bind to an existing PID to debug a running program. The basic usage of the three debugging tools is roughly the same, and the following are just a few of the three command-line parameters that are common to all three:
　　
-F: In addition to tracking the current process, it also tracks its child processes.
-o File: Writes output information to file files instead of the standard error output (stderr).
-P PID: binds to a running process that corresponds to a PID. This parameter is commonly used to debug background processes.
　　
With the above three parameters, you can basically do most of the debugging tasks, and here are a few command line examples:
Truss-o Ls.truss Ls-al: Tracks the operation of Ls-al, writes the output information to the file/tmp/ls.truss.
Strace-f-O vim.strace vim: Tracks the operation of Vim and its child processes, writing output information to file Vim.strace.
Ltrace-p 234: Tracking a process that is already running with a PID of 234.
　　
The output format of the three debugging tools is also similar, taking Strace as an example:
　　
BRK (0) = 0x8062aa8
BRK (0x8063000) = 0x8063000
MMAP2 (NULL, 4096, Prot_read, Map_private, 3, 0x92f) = 0x40016000
　　
Each row is a system call, and the left side of the equal sign is the function name of the system call and its parameters, and the return value of the call on the right. Truss, Strace and ltrace work the same principle, are using the Ptrace system call to track the process of debugging run, the detailed principle is not in the scope of this article, interested can refer to their source code.
Give two examples of how to use these three debugging tools to diagnose the "incurable diseases" of software:
　　
Case one: Run Clint appear segment fault error
　　
Operating system: Freebsd-5.2.1-release
Clint is a C + + static source code analysis tool that is installed by ports after it is run:
　　
# clint Foo.cpp
Segmentation fault (core dumped)
Meeting "Segmentation Fault" in a UNIX system is just as annoying as popping the illegal actions dialog box in MS Windows. OK, we use Truss to give Clint "pulse":
　　
# truss-f-O clint.truss Clint
Segmentation fault (core dumped)
# tail Clint.truss
739:read (0x6,0x806f000,0x1000) = 4096 (0x1000)
739:fstat (6,0xbfbfe4d0) = 0 (0x0)
739:fcntl (0x6,0x3,0x0) = 4 (0x4)
739:fcntl (0x6,0x4,0x0) = 0 (0x0)
739:close (6) = 0 (0x0)
739:stat ("/root/.clint/plugins", 0xbfbfe680) err#2 ' No such file or directory '
SIGNAL 11
SIGNAL 11
Process stopped because of:16
Process exit, Rval = 139
We use truss to track Clint system call execution, output the results to file Clint.truss, and then use tail to view the last few lines.
Note the last system call executed by Clint (line fifth): Stat ("/root/.clint/plugins", 0xbfbfe680) err#2 ' No such file or Directory, the problem is here: Clint cannot find the directory "/root/.clint/plugins", causing a segment error. How to solve. Very simple: mkdir-p/root/.clint/plugins, but this run Clint will still be "segmentation Fault" 9. Continue to use truss tracking, found Clint also need this directory "/root/.clint/plugins/python", built this directory Clint finally able to run normally.
　　
Case two: Vim startup speed is obviously slowing down
　　
Operating system: Freebsd-5.2.1-release
The VIM version is 6.2.154, and after running vim from the command line, wait nearly half a minute to get into the editing interface without any error output. Check it out. VIMRC and all Vim scripts are not misconfigured, and there is no way to find solutions to similar problems on the web, and it is difficult to hacking source code. There is no need to use truss to find the problem:
　　
# truss-f-d-o Vim.truss vim
　　
Here the-D argument works by adding a relative timestamp before each row of output, that is, the time spent executing a single system call. We just pay attention to which system calls take longer to spend, with less carefully look at the output file Vim.truss, quickly found a doubt:
　　
735:0.000021511 socket (0x2,0x1,0x0) = 4 (0x4)
735:0.000014248 setsockopt (0x4,0x6,0x1,0xbfbfe3c8,0x4) = 0 (0x0)
735:0.000013688 setsockopt (0x4,0xffff,0x8,0xbfbfe2ec,0x4) = 0 (0x0)
735:0.000203657 Connect (0x4,{af_inet 10.57.18.27:6000},16) err#61 ' Connection refused '
735:0.000017042 Close (4) = 0 (0x0)
735:1.009366553 nanosleep (0xbfbfe468,0xbfbfe460) = 0 (0x0)
735:0.000019556 socket (0x2,0x1,0x0) = 4 (0x4)
735:0.000013409 setsockopt (0x4,0x6,0x1,0xbfbfe3c8,0x4) = 0 (0x0)
735:0.000013130 setsockopt (0x4,0xffff,0x8,0xbfbfe2ec,0x4) = 0 (0x0)
735:0.000272102 Connect (0x4,{af_inet 10.57.18.27:6000},16) err#61 ' Connection refused '
735:0.000015924 Close (4) = 0 (0x0)
735:1.009338338 nanosleep (0xbfbfe468,0xbfbfe460) = 0 (0x0)
　　
Vim attempts to connect to the 10.57.18.27 6000 port (Connect () on line fourth), and after a failed connection, sleep one second to continue retrying (line 6th, Nanosleep ()). The above fragment loop appears more than 10 times, each time consumes a second more than a minute of time, this is the reason that vim obviously slows down. However, you will surely wonder: how can vim connect 6000 ports of other computers for no reason? "。 That's a good question, so please think back to what the 6000 service port is. Yes, that's X Server. It seems vim is going to direct the output to a remote X Server, then the shell definitely defines the DISPLAY variable, view. CSHRC, there is such a line: setenv DISPLAY ${remotehost}:0, comment it out, and then log in again, The problem is solved.
　　
　　
Case Three: Use Debugging tools to master the working principle of the software
　　
Operating system: Red Hat Linux 9.0
Using debugging tools to track the operation of the software in real time is not only an effective means to diagnose the software "difficult and incurable diseases", but also can help us to clarify the "thread" of software, that is, to quickly master the running process and working principle of software, which is an auxiliary method of learning source code. The following case demonstrates how to use Strace to "trigger inspiration" by tracking other software to solve the challenges of software development.
As you all know, there is a single file descriptor (Fd:file descriptor) that corresponds to this file when you open a document within the process. And I was in the development of a software process encountered such a problem:
Given an FD, how to obtain the full path of the file corresponding to this FD. What if Linux, FreeBSD, or other UNIX systems do not provide such an API? Let's think about it another way: What software is available under UNIX to get what files the process has opened. If you're experienced enough, it's easy to think of lsof, using it to know which files the process has opened, and what process a file is open to. OK, let's try a little program to test lsof and see how it gets the files that the process has opened. Lsof: Displays the files that the process opens.
　　
* TESTLSOF.C * *
#include #include #include #include #include
int main (void)
{
Open ("/tmp/foo", o_creat| O_RDONLY); /* Open File/tmp/foo * *
Sleep (1200); /* Sleep for 1200 seconds for subsequent operation * *
return 0;
}
　　
Put the testlsof into the background run, its PID is 3125. Command Lsof-p 3125 to see which files are open in process 3125, we use Strace to track lsof runs, and the output is saved in Lsof.strace:
　　
# gcc Testlsof.c-o testlsof
#./testlsof &
[1] 3125
# strace-o Lsof.strace lsof-p 3125
　　
We use "/tmp/foo" as the keyword search output file lsof.strace, the result is only one:
　　
　　
# grep '/tmp/foo ' lsof.strace
Readlink ("/proc/3125/fd/3", "/tmp/foo", 4096) = 8
　　
Originally lsof Clever use of the/proc/nnnn/fd/directory (nnnn for PID): The Linux kernel for each process in/proc/to establish a directory with its PID name to save the information about the process, and its subdirectory FD is saved by the process open all the files of the FD. The target is close to us. Okay, let's go to/proc/3125/fd/and see what happens:
　　
# cd/proc/3125/fd/
# ls-l
Total 0
Lrwx--1 root root Nov 5 09:50 0->/dev/pts/0
Lrwx--1 root root Nov 5 09:50 1->/dev/pts/0
Lrwx--1 root root Nov 5 09:50 2->/dev/pts/0
Lr-x--1 root root Nov 5 09:50 3->/tmp/foo
# READLINK/PROC/3125/FD/3
/tmp/foo
　　
The answer is clear: each fd file in the/proc/nnnn/fd/directory is a symbolic link that points to a file that was opened by the process. We just need to use Readlink () system call can get an FD corresponding file, the code is as follows:
　　
　　
#include #include #include #include #include #include
int get_pathname_from_fd (int fd, char pathname[], int n)
{
Char buf[1024];
pid_t pid;
Bzero (BUF, 1024);
PID = Getpid ();
snprintf (BUF, 1024, "/proc/%i/fd/%i", PID, FD);
Return Readlink (buf, pathname, N);
}
int main (void)
{
int FD;
Char pathname[4096];
Bzero (pathname, 4096);
FD = open ("/tmp/foo", o_creat| O_RDONLY);
GET_PATHNAME_FROM_FD (FD, pathname, 4096);
printf ("fd=%d; Pathname=%sn ", FD, pathname);
return 0;
}
　　
For security reasons, the system does not automatically load the proc file system by default after FreeBSD 5, so if you want to use truss or Strace tracker, you must manually load the proc file system: Mount-t PROCFS Proc/proc; Add a line to the Etc/fstab:
　　
Proc/proc procfs RW 0 0