Linux Debug Analysis and Diagnosis tool--strace

Strace is a powerful Linux debug Analysis Diagnostic tool that can be used to track process system calls and received signals during program execution, especially for programs that are unreadable by the source code or are not compiled by source code.

In a Linux system, the user program runs in a sandbox, and the user process does not have direct access to the computer's hardware devices. When a process requires access to a hardware device (such as reading a disk file or receiving network data), it must be switched from user mode to kernel mode and access to the hardware device through system calls. Strace can track the system calls generated by the process, including parameters, return values, and time spent executing. If the strace does not have any output, it does not mean that the process is blocked at this time, or that the program process is doing something in its own sandbox that does not need to communicate with other parts of the system. Strace receives information from the kernel and does not need to build the kernel in any special way.

The strace command format is as follows:

Strace [-DFFHIQRTTTTVVXX] [-a column] [-e expr] [-o file] [-P PID] [-S strsize] [-u username] [-e Var=val] [command [ARG]. ..]] Or strace-c [-E expr] [-o overhead] [-s sortby] [-e Var=val] [command [ARG ...]

The Strace offers a very rich tracking capability through different options switches. The simplest application is to trace the entire lifecycle of an executable program runtime, outputting the name, parameters, and return value of the called system call to the file specified by the standard error output stderr (that is, the screen) or the-o option. Note that the command must be located after the list of options .

The detailed strace command options are listed below:

Options	Meaning
-C	Statistics and reporting on the time, number of calls, and number of errors performed per system call
-D	Output strace debug information about standard errors
-F	Tracks the current process and its child processes created through the fork system call
-ff	Often used in conjunction with the-O option, the trace results for different processes (sub-processes) are exported to the corresponding filename respectively. PID file, PID is the number of each process
-F	Try to trace the vfork system call. Otherwise, vfork is not tracked even if the-f option is turned on
-H	Display Help information
-I.	Displays the instruction pointer (IP) register value at the time the system call occurred
-Q	Suppresses (suppresses output) messages about binding (attaching), Detachment (detaching). Automatic suppression of such messages when output is redirected to a file
-R	Displays the relative timestamp at which each system call occurred, that is, the time difference between successive system call start points
-T -tt -ttt	-T adds an absolute timestamp (current clock) information before each line of output, accurate to the second level -tt Add absolute timestamp information before each line of output, accurate to microsecond level The-TTT adds relative time information before each line of output, in the format "number of seconds since the epoch time." Number of microseconds
-T	Displays the time spent on each system call, with the time overhead being within the angle brackets at the far right of the output line
-V	Display version information for Strace
-V	Redundant display mode: Displays all element/member contents of array/struct parameters such as argv[], envp[], stat, Termio (s) in system call. These arrays/structs are used frequently, and only a reasonable subset of their elements/members is displayed by default
-X	Displays non-standard (NON-ASCII) strings in 16 binary form, such as "/x08". The default is 8, such as "/10"
-xx	Displays all strings in 16 binary form
-A column	Sets the column position that displays the return value of the system call, which defaults to 40 (starting at 0), that is, "=" appears in the 40th column
-E Expr	Specifies an expression that is used to limit which events are tracked and how to track them. The format is [qualifier=][!] All or [qualifier=][!] Value1[,value2] .... The qualifier value is trace (default), abbrev, verbose, raw, signal, read, or write. Value is used for qualifying symbols (both all and none) or numbers. The exclamation mark is a negative symbol, such as-eopen equivalent to-e Trace=open, which indicates that only the open system call is tracked, while the-e Trace=!open indicates that other system calls are tracked except open. Note that some shells are used! Represents the command in the execution history, with the use of quotation marks, escape symbols (/). -E Trace=set: Tracks the specified list of system calls, such as-e Trace=open,close,read,write, that only tracks these four system calls. The default is Set=all. -E Trace=file: Tracks all system calls that make parameters with the specified file name. -E trace=process: Tracks all system calls involving process management and can be used to observe the fork, wait, and exec stages of a process. -E Trace=network: Tracks all system calls related to the network. -E strace=signal: Tracks all system calls related to system signals. -E TRACE=IPC: Tracks all system calls related to interprocess communication. -E Abbrev=set: Specifies which system calls are in a large array or struct-body content reduction display, such as Strace-e Abbrev=execve./test shows only parts of Execve] and argv[in envp[calls). The default is Abbrev=all, and Abbrev=none is equivalent to the-v option. -E Verbose=set: Specifies which system calls are in the full display of large arrays or struct-body content, and calls outside the collection whose arrays or structs appear as addresses. The default is Verbose=all. -E Raw=set: Specifies which system calls in the parameters are displayed in the original unencrypted form (that is, 16 binary). Useful when the user does not trust Strace decoding or needs to know the actual value of the parameter. -E Signal=set: Tracks the specified list of signals and defaults to Signal=all. such as signal=! SIGIO (or Signal=!io) indicates that the SIGIO signal is not tracked. -E Read=set: Displays all data read from the specified file descriptor in 16-binary and ASCII-coded controls, such as-e read=3,5 to observe input actions on file descriptors 3 and 5. This option is independent of the general tracing of the system call to read (controlled by the-e trace=read option). -E Write=set: Displays all data written to the specified file descriptor as a 16-binary and ASCII-coded control.
-O File	The Strace output information is displayed by default to the standard error output, which writes output information to the file. The following two commands are equivalent: Strace-c-o test.txt./test Strace-c./test 2>test.txt
-O Overhead	Set the overhead for tracing system calls to overhead microseconds. This was useful for overriding the default heuristic for guessing how much time was spent in mere measuring when timing syst EM calls using THE-C option. The acuracy of the heuristic can be gauged by timing a given program run without tracing (using the time command) and comparing the ACCU mulated system call time to the total produced using-c.
-P PID	Specifies the process number (PID) to be traced and the trace process continues to run by using CTRL-C to terminate the trace. Up to 32 (-p PID) options can be specified to track more than one process at a time. This option is commonly used to debug background processes
-S Strsize	Limits the maximum display length of a string (such as the read parameter) in each row of output, which defaults to 32 bytes. But the file name is always fully displayed
-S SortBy	Sorts the output histogram of the-c option by the specified rule. SortBy values can be time, calls, name, and nothing (default time)
-u username	Run a pending program with the UID and/or GID identity of the specified user
-E Var=val	List of environment variables to put Var=val into the command
-e var	Remove Var from the list of environment variables for the command

For example, the command strace-o out.txt-t-tt-e trace=all-p 2899 indicates that all system calls to the 2899 process are tracked and the time overhead of the system call is counted, as well as the invocation start time (shown in the visual hour-in-seconds format). Finally, the recorded results are deposited into the OUT.txt file.

For illustrative purposes, use the strace Trace ls-l command execution process (strace ls-l) with the first few lines output as follows:

Strace each system call produced by the logger and appears as a separate line in a C-like format, regardless of the programming language used when the program was created. Each line starts with a function name that is called by the system, and the return value of the call is displayed to the right of the equals sign in parentheses. When a parameter is an array or struct, the contents of its elements (brackets) or members (curly braces) are displayed, see EXECVE and Fstat64. When the argument is bit, square brackets are used to separate each parameter with a space, such as Sigprocmask (sig_block,[chld ttou],[]) = 0, the second parameter represents the signal sigchld and Sigttou, and if the bit type parameter is all set, The output, such as Sigprocmask (sig_unblock,~[],null) = 0, where the second parameter is all set.

By using the-C option, Strace can statistically analyze all system calls (strace-c./changesystime) of the process, such as:

You can see which system functions are called by the program, the number of calls, the time spent and the number of errors, etc., which helps to analyze the program speed bottleneck. Also notice that the Settimeofday call error two times, and the function in the Changesystime program is explicitly called two times, that is, the two calls are error! However, the settimeofday call is not handled incorrectly in the Changesystime program, so no error prompts are output during the run. Assuming the program source code is not modifiable, then you can use Strace to find out where the error (strace-e trace=settimeofday./changesystime):

The truth is that the original call to the Settimeofday function is denied due to insufficient operational permissions (Root permission required)! Note that lines 2nd, 3, and 5 output the Changesystime program printout.