Linux strace Commands

Introduction

Strace is commonly used to track system calls and received signals when a process executes. In the Linux world, processes cannot directly access hardware devices, and when a process requires access to a hardware device (such as reading a disk file, receiving network data, and so on), it must be switched from user mode to kernel mode and access to the hardware device through system calls. Strace can trace the system calls generated by a process, including parameters, return values, and time spent executing.

output parameter meaning

Execve ("/bin/cat", ["Cat", "/dev/null"], [/* for VARs */]) = 0
BRK (0)                                  = 0xab1000
Access ("/etc/ld.so.nohwcap", F_OK)      =-1 ENOENT (No such file or directory)
Mmap (NULL, 8192, prot_read| Prot_write, map_private| Map_anonymous,-1, 0) = 0x7f29379a7000
Access ("/etc/ld.so.preload", R_OK)      =-1 ENOENT (No such file or directory)
...
BRK (0) = 0xab1000
BRK (0xad2000) = 0xad2000
Fstat (1, {st_mode=s_ifchr|0620, St_rdev=makedev (136, 0), ...}) = 0
Open ("/dev/null", o_rdonly) = 3
Fstat (3, {st_mode=s_ifchr|0666, St_rdev=makedev (1, 3), ...}) = 0
Read (3, "", 32768) = 0
Close (3) = 0
Close (1) = 0
Close (2) = 0
Exit_group (0) =?

Each row is a system call, the left side of the equals sign is the function name of the system call and its arguments, and to the right is the return value of the call.
Strace Displays the parameters of these calls and returns the values in the form of symbols. Strace receives information from the kernel and does not need to build the kernel in any particular way.

Strace parameters

-C counts the time, number of times, and number of errors that are performed by each system call.
-D Output strace debug information about standard errors.
-F tracks the child processes that are generated by the fork call.
-FF If you provide-o filename, the trace results for all processes are output to the corresponding filename.pid, and the PID is the process number of each process.
-F attempts to trace vfork calls. At-F, vfork is not tracked.
-h outputs a brief help message.
The-I output system calls the entry pointer.
-Q suppresses the output of messages about secession.
-R prints out relative time about, every system call.
-T adds time information before each line in the output.
-tt The time information, in microseconds, before each line in the output.
-TTT microsecond output, in seconds, indicates the time.
-T displays the elapsed time for each call.
-V outputs all system calls. Some calls about environment variables, states, input and output calls due to frequent use, not output by default.
-V Output strace version information.
-X outputs non-standard strings in 16 binary form
-XX all strings are output in 16 binary form.
-A column
Sets the output location of the return value. Default is 40.
-E Expr
Specifies an expression that controls how to trace. The format is as follows:
[Qualifier=] [!] Value1[,value2] ...
Qualifier can only be one of the trace,abbrev,verbose,raw,signal,read,write. value is the symbol or number that is used to qualify. The default qualifier is trace. An exclamation point is a negative symbol. For example:
-eopen is equivalent to-e Trace=open, which means that only the open call is tracked. Instead of-etrace!=open, the trace is tracked except for open calls. There are two special symbols all and none.
Note that some shells use! To perform commands in the history, so use \ \.
-E Trace=set
Only the specified system calls are tracked. For example, the-e trace=open,close,rean,write indicates that only the four system calls are tracked. The default is Set=all.
-E Trace=file
Only system calls about file operations are tracked.
-E trace=process
Only system calls about process control are tracked.
-E Trace=network
Tracks all system calls related to the network.
-E strace=signal
Track all system calls related to system signals
-E TRACE=IPC
Track all system calls related to process communication
-E Abbrev=set
Sets the result set of the system call for the strace output.-V et abbrev=none. Default is Abbrev=all.
-E Raw=set
Displays the parameters of the specified system call in hexadecimal.
-E Signal=set
Specifies the system signal for the trace. The default is all. such as signal=! SIGIO (or Signal=!io), indicating that the SIGIO signal is not tracked.
-E Read=set
Outputs the data that is read from the specified file. For example:
-E read=3,5
-E Write=set
The output is written to the data in the specified file.
-O filename
Writes the output of the strace to the file filename
-P PID
Tracks the specified process PID.
-S Strsize
Specifies the maximum length of the output string. The default is 32. The file name is always output.
-u username
Executes the tracked command with the UID and GID of the username

Command Instance

Full usage of general:

Strace-o output.txt-t-tt-e trace=all-p 28979

The above meaning is to trace all system calls (-e Trace=all) of the 28979 process, and to count the time spent on the system call, as well as the start time (and display in the visual hour-in-seconds format), and finally the recorded result exists in the Output.txt file.

Strace Case

Debug the program with Strace

In an ideal world, whenever a program fails to perform a function properly, it gives a useful error message that tells you enough clues to correct the error. Unfortunately, we do not live in an ideal world, at least not always in the ideal world. Sometimes a program has a problem and you can't find the reason.
This is why the debugger appears. Strace is an essential debugging tool that Strace uses to monitor system calls. Not only can you debug a newly started program, you can also debug a program that is already running (bind the strace to an existing PID).
First, let's look at a real example: there was a problem starting KDE
Some time ago, I had a problem starting KDE, and KDE's error message could not give me any helpful clues.

_kde_icetranssocketcreatelistener:failed to bind Listener
_kde_icetranssocketunixcreatelistener: ... Socketcreatelistener () failed
_kde_icetransmakeallcotsserverlisteners:failed to create listener for local

Cannot establish any listening sockets Dcopserver self-test failed.

For me, this error message doesn't make much sense, but a program that is critical to KDE that is responsible for interprocess communication does not start. I can also tell that this error is related to the Ice protocol (Inter Client Exchange), and I don't know what is the cause of the KDE startup error.

I decided to use Strace to see what the program did when it started Dcopserver:

Strace-f-f-o ~/dcop-strace.txt Dcopserver

Here the-f-f option tells Strace to track both the fork and the vfork process, and the-o option writes all strace output to ~/dcop-strace.txt, Dcopserver is the program to start and Debug.

After the error again, I checked the error output file Dcop-strace.txt, there are many system call records in the file. The relevant records before the program run error are as follows:

27207 mkdir ("/tmp/. Ice-unix ", 0777) =-1 eexist (File exists)
27207 Lstat64 ("/tmp/. Ice-unix ", {st_mode=s_ifdir| s_isvtx|0755, st_size=4096, ...}) = 0
27207 unlink ("/tmp/. ice-unix/dcop27207-1066844596 ") =-1 ENOENT (No such file or directory)

27207 Write (2, "_kde_icetrans", 13) = 13
27207 Write (2, "socketcreatelistener:failed to" ..., 46) = 46
27207 Close (3) = 0 27207 Write (2, "_kde_icetrans", 13) = 13
27207 Write (2, "Socketunixcreatelistener: ... Soc "..., 59) = 59
27207 umask (0) = 0 27207 Write (2, "_kde_icetrans", 13) = 13
27207 Write (2, "Makeallcotsserverlisteners:fail" ..., 64) = 64
27207 Write (2, "cannot establish any listening S" ..., 39) = 39

The first line shows that the program is trying to create/tmp/. Ice-unix directory with permissions of 0777, this operation failed because the directory already exists. The second system call (Lstat64) checks the directory status and shows that the permission for this directory is 0755, where the first program run error occurred: The program tried to create a directory with a property of 0777, but a directory with a property of 0755 already exists. A third system call (unlink) attempted to delete a file, but the file does not exist. This is not surprising, because this operation only attempts to delete the old files that may exist.

However, line fourth confirms where the error is. He tried to bind to/tmp/. ice-unix/dcop27207-1066844596, but an access denied error has occurred: The users and groups of the Ice_unix directory are root, and only the owner has write permissions. A non-root user cannot create a file under this directory, and if the directory attribute is changed to 0777, the previous operation may be possible, and this is the operation that occurred when the first step was wrong.

So I ran the chmod 0777/tmp/. KDE can start normally after Ice-unix, the problem is solved, it only takes a few minutes to trace the program to run with Strace, then check and analyze the output file.

Description: Running chmod 0777 is only a test, generally do not set a directory for all users to read and write, and do not set the sticky bit (sticky bit). Setting a sticky bit on a directory can prevent a user from arbitrarily deleting files from other people under the writable directory. In general you will find that the/tmp directory has a sticky bit set for this reason. After KDE can start normally, run chmod +t/tmp/. Ice-unix set the sticky bit to. Ice_unix.

Resolve Library Dependency Issues

Another use of starce is to solve problems related to dynamic libraries. When you run LDD on an executable file, it tells you which dynamic library The program uses and where to find the dynamic library. But if you're using an older version of GLIBC (2.2 or earlier), you might have a bug-LDD program that might report a dynamic library found in a directory, but actually run the program dynamically (/lib/ ld-linux.so.2) may go to another directory to find a dynamic connection library. This is usually because the/etc/ld.so.conf and/etc/ld.so.cache files are inconsistent, or the/etc/ld.so.cache is corrupted. This error does not occur on GLIBC 2.3.2, and it is possible that the Ld-linux bug has been resolved.

In spite of this, LDD does not list all the dynamic libraries that the program relies on, and the system calls Dlopen to automatically call into the required dynamic libraries when needed, and these libraries may not be listed by LDD. The NSS (Name Server Switch) library, as part of GLIBC, is a typical example of what NSS does to tell the application where to look for the System account database. The application is not connected directly to the NSS library, and GLIBC is automatically transferred to the NSS library via Dlopen. If such a library is accidentally lost, you will not be told that there is a library dependency problem, but such a program will not be able to resolve the user ID by username. Let's look at an example:
The WhoAmI program will give you your own username, which is very useful in some scripts that need to know the actual user who runs the program, and a sample output from WhoAmI is as follows:

# WhoAmI
Root

Let's say that for some reason the library NSS, which is responsible for user name and user ID conversion during the upgrade of GLIBC, can simulate this environment by renaming the NSS library:

# WhoAmI
Whoami:cannot find username for UID 0

As you can see here, there was an error running WhoAmI, and the output of the LDD program did not provide useful help:

# Ldd/usr/bin/whoami
libc.so.6 =/lib/libc.so.6 (0x4001f000)
/lib/ld-linux.so.2 =/lib/ld-linux.so.2 (0x40000000)

You will only see WhoAmI dependent on libc.so.6 and ld-linux.so.2, which does not give the other libraries necessary to run the WhoAmI. Here when using Strace to trace the output of WhoAmI:

Strace-o Whoami-strace.txt WhoAmI

Open ("/lib/libnss_files.so.2", o_rdonly) =-1 ENOENT (No such file or directory)
Open ("/lib/i686/mmx/libnss_files.so.2", o_rdonly) =-1 ENOENT (No such file or directory)
Stat64 ("/lib/i686/mmx", 0xbffff190) =-1 ENOENT (No such file or directory)
Open ("/lib/i686/libnss_files.so.2", o_rdonly) =-1 ENOENT (No such file or directory)
Stat64 ("/lib/i686", 0xbffff190) =-1 ENOENT (No such file or directory)
Open ("/lib/mmx/libnss_files.so.2", o_rdonly) =-1 ENOENT (No such file or directory)
Stat64 ("/lib/mmx", 0xbffff190) =-1 ENOENT (No such file or directory)
Open ("/lib/libnss_files.so.2", o_rdonly) =-1 ENOENT (No such file or directory)
Stat64 ("/lib", {st_mode=s_ifdir|0755, st_size=2352, ...}) = 0
Open ("/usr/lib/i686/mmx/libnss_files.so.2", o_rdonly) =-1 ENOENT (No such file or directory)
Stat64 ("/usr/lib/i686/mmx", 0xbffff190) =-1 ENOENT (No such file or directory)
Open ("/usr/lib/i686/libnss_files.so.2", o_rdonly) =-1 ENOENT (No such file or directory)

You can find an attempt to find libnss.so.2 under a different directory, but all failed. Without a tool such as strace, it is hard to see that this error is due to the lack of a dynamic library. Now just find the libnss.so.2 and put it back in the right place.

Restricting Strace to track only specific system calls

If you already know what you're looking for, you can let strace only track some types of system calls. For example, you need to look at the programs executed in the Configure script, and the system calls you need to monitor are execve. Let Strace only record Execve calls with this command:

Strace-f-o configure-strace.txt-e execve./configure

Linux strace Commands