Shell script debugging technology

Shell script debugging technology Cao Yuzhong (caoyuz@cn.ibm.com ),
Software Engineer, IBM China Development Center

I. Preface

Shell programming is widely used in the Unix/Linux World. Mastering shell programming is also the only way to become an excellent Unix/Linux developer and system administrator. The main task of script debugging is to find the cause of the script error and locate the wrong row in the script source code. Common means include analyzing the output error information, add debugging statements to the script and output debugging information to help diagnose errors and use debugging tools. However, compared with other advanced languages, the shell interpreter lacks the corresponding debugging mechanism and debugging tool support, and the output error information is often ambiguous. When a beginner debugs a script, in addition to knowing that echo statements are used to output some information, there is no such method. However, it is really complicated to simply rely on a large number of echo statements to diagnose errors, therefore, it is common for beginners to complain that shell scripts are too difficult to debug. This article will systematically introduce some important shell script debugging technologies, hoping to benefit beginners of shell.

The target readers of this article are developers, testers and system administrators in Unix/Linux environments, and require the readers to have basic shell programming knowledge. The example used in this article passes the test under bash3.1 + RedHat Enterprise Server 4.0, but the debugging skills described here should also apply to other shells.

2. Output debugging information in shell scripts

It is the most common debugging method to display the relevant information of some key or error points by adding debugging statements to the program. Shell programmers usually use echo (Ksh programmers often use print) statements to output information, but it is troublesome to rely only on the output tracing information of echo statements, A large number of echo statements added to the script in the debugging phase have to be removed one by one during product delivery. This section describes how to output debugging information conveniently and effectively.

1. Use the Trap Command

The trap command is used to capture specified signals and execute predefined commands.
The basic syntax is:
Trap 'command' Signal
Signal is the signal to be captured, and command is the command to be executed after the specified signal is captured. You can use the kill-l command to view all available signal names in the system. The command executed after the signal is captured can be any one or more valid shell statements or a function name.
When a shell script is executed, three so-called "pseudo signals" are generated. (the reason is that these three signals are generated by shell, other signals are generated by the operating system). It is very helpful for debugging to capture these three "pseudo signals" by using the trap command and output relevant information.

Table 1. Shell pseudo Signal

Signal name	When to generate
Exit	Exit from a function or complete Script Execution
Err	When a command returns a non-zero status (indicating that the command fails to be executed)
Debug	Before each command in the script is executed

By capturing the exit signal, we can output the values of certain variables to be tracked when the shell script is aborted or exited from the function, and then determine the script execution status and cause of error, the usage is as follows:
Trap 'command' exit or trap 'command' 0

By capturing the err signal, we can easily track unsuccessful commands or functions and output relevant debugging information. Below is a sample program for capturing the err signal, $ lineno is a built-in shell variable, representing the current row number of the shell script.

$ cat -n exp1.sh 1 ERRTRAP() 2 { 3 echo "[LINE:$1] Error: Command or function exited with status $?" 4 } 5 foo() 6 { 7 return 1; 8 } 9 trap 'ERRTRAP $LINENO' ERR 10 abc 11 foo

The output result is as follows:

$ sh exp1.sh exp1.sh: line 10: abc: command not found [LINE:10] Error: Command or function exited with status 127 [LINE:11] Error: Command or function exited with status 1

In the debugging process, to track the values of some variables, we often need to insert the same echo statement in many places of the shell script to print the values of related variables. This is cumbersome and clumsy. By capturing the debug signal, we only need a trap statement to track the entire process of related variables.

The following is an example program that tracks variables by capturing debug signals:

$ cat –n exp2.sh 1 #!/bin/bash 2 trap 'echo “before execute line:$LINENO, a=$a,b=$b,c=$c”' DEBUG 3 a=1 4 if [ "$a" -eq 1 ] 5 then 6 b=2 7 else 8 b=1 9 fi 10 c=3 11 echo "end"

The output result is as follows:

$ sh exp2.sh before execute line:3, a=,b=,c= before execute line:4, a=1,b=,c= before execute line:6, a=1,b=,c= before execute line:10, a=1,b=2,c= before execute line:11, a=1,b=2,c=3 end

From the running results, we can clearly see the changes in the values of related variables after each command is executed. At the same time, we can analyze the row numbers printed in the running results to see the execution track of the entire script and determine which condition branches are executed and which condition branches are not.

2. Use the tee command

In shell scripts, pipelines and input/output redirection are used a lot. The execution results of some commands directly become the input of the next command. If we find that the execution results of a batch of commands connected by pipelines are not as expected, we need to gradually check the execution results of each command to determine where the problem is, but because pipelines are used, these intermediate results are not displayed on the screen, which makes debugging difficult. In this case, we can use the tee command.

The Tee command reads data from the standard input, outputs its content to the standard output device, and saves the content as a file. For example, the following script snippet is used to obtain the IP address of the local machine:

ipaddr = `/ sbin / ifconfig | grep 'inet addr:' | grep -v '127.0.0.1' | cut -d: -f3 | awk '{print $ 1}' ` The entire sentence after the # note = sign is enclosed in backticks (the key to the left of the number 1 key). echo $ ipaddr

Run this script, but the actual output is not the IP address of the local machine, but the broadcast address. In this case, we can use the tee command to output some intermediate results and change the above script segment:

ipaddr=`/sbin/ifconfig | grep 'inet addr:' | grep -v '127.0.0.1' | tee temp.txt | cut -d : -f3 | awk '{print $1}'` echo $ipaddr

After that, execute the script again, and then copy the content of the temp.txt file:

$ cat temp.txt inet addr:192.168.0.1 Bcast:192.168.0.255 Mask:255.255.255.0

We can find that the second column of the intermediate result (separated by a: sign) contains the IP address, and the cut command in the script above intercepts the third column, therefore, you only need to change cut-D:-F3 in the script to cut-D:-F2 to get the correct result.

In the above script example, we may not need the help of the tee command. For example, we can run the commands connected by the pipeline in segments and view the output results of each command to diagnose errors, however, in some complex shell scripts, these commands connected by pipelines may depend on some other variables defined in the script, at this time, it is very troublesome to run each command in segments at the prompt. simply inserting a tee command between pipelines to view the intermediate results is more convenient.

3. Use "Debug hook"

In C language programs, we often use the debug macro to control whether to output debugging information. In shell scripts, we can also use this mechanism, as shown in the following code:

if [“$ DEBUG” = “true”]; then
echo "debugging" #Debug information can be output here
fi

Such a code block is usually called a debugging hook or a debugging block ". You can output any debugging information you want to output within the debugging hook. The advantage of using the debugging hook is that it can be controlled by the debug variable. In the development and debugging phase of the script, you can run the export DEBUG = true command to open the debugging hook and output the debugging information. When the script is delivered for use, you do not need to delete the debugging statements in the script one by one.

If the if statement is used to judge the value of the debug variable where debugging information needs to be output in each place, it still seems complicated, you can define a debug function to simplify the debugging hook Inserting Process, as shown in the following code:

$ cat –n exp3.sh 1 DEBUG() 2 { 3 if [ "$DEBUG" = "true" ]; then 4 $@   5 fi 6 } 7 a=1 8 DEBUG echo "a=$a" 9 if [ "$a" -eq 1 ] 10 then 11 b=2 12 else 13 b=1 14 fi 15 DEBUG echo "b=$b" 16 c=3 17 DEBUG echo "c=$c"

In the debug function shown above, any command sent to it will be executed, and the execution process can be controlled by the value of the debug variable, we can call all debugging-related commands as parameters of the debug function, which is very convenient.

Back to Top

3. Use shell execution options

The debugging method described in the previous section is to modify the source code of the shell script and output relevant debugging information to locate the error. Is there any way to debug the shell script without modifying the source code? The answer is to use shell execution options. This section describes the usage of some common options:

-N: only read shell scripts, but not actually executed.
-X: Enter the tracking mode to display each command executed.
-C "string" reads commands from strings

"-N" can be used to test whether the shell script has a syntax error, but does not actually execute the command. After the shell script is compiled and executed, it is a good habit to use the "-n" option to test whether the script has syntax errors. Some shell scripts may affect the system environment during execution, such as generating or moving files. If a syntax error is found during actual execution, you have to manually restore the system environment to continue testing this script.

The "-c" option enables the shell interpreter to read from a string rather than from a file and execute shell commands. You can use this option when you need to temporarily test the execution result of a small script, as shown below:
Sh-C 'a = 1; B = 2; let C = $ A + $ B; echo "c = $ C "'

The "-X" option can be used to track script execution. It is a powerful tool for debugging shell scripts. The "-X" option enables shell to display every command line actually executed by Shell during script execution, and display a "+" number at the beginning of the line. "+" Indicates the content of the command line after the variable is replaced, which helps to analyze what commands are actually executed. The "-X" option is easy to use and can easily deal with most shell debugging tasks. It should be considered as the preferred debugging method.

If we combine the trap 'command' debug mechanism described earlier in this article with the "-X" option, we can output every command actually executed, trace the values of related variables line by line, which is quite helpful for debugging.

For example, if you still use exp2.sh as described earlier, add the "-X" option to execute it:

$ sh –x exp2.sh + trap 'echo "before execute line:$LINENO, a=$a,b=$b,c=$c"' DEBUG ++ echo 'before execute line:3, a=,b=,c=' before execute line:3, a=,b=,c= + a=1 ++ echo 'before execute line:4, a=1,b=,c=' before execute line:4, a=1,b=,c= + '[' 1 -eq 1 ']' ++ echo 'before execute line:6, a=1,b=,c=' before execute line:6, a=1,b=,c= + b=2 ++ echo 'before execute line:10, a=1,b=2,c=' before execute line:10, a=1,b=2,c= + c=3 ++ echo 'before execute line:11, a=1,b=2,c=3' before execute line:11, a=1,b=2,c=3 + echo end end

In the preceding results, the "+" line is the command actually executed by the shell script, and the "+ +" line is the command specified in the execution trap mechanism, other rows are output information.

In addition to the shell startup, the shell execution options can be specified by using the set command in the script. "Set-parameter" indicates that an option is enabled, and "set + parameter" indicates that an option is disabled. Sometimes we do not need to use the "-X" option to trace all command lines at startup. In this case, we can use the set command in the script, as shown in the following script snippet:

set -x #Enable "-x" option Block to be traced set + x #Turn off the "-x" option

The SET command can also be called using the debug hook-debug function described in the previous section. This avoids the trouble of deleting these debugging statements when the script is delivered and used, as shown in the following script snippet:

DEBUG set -x #Enable "-x" option Block to be traced DEBUG set + x #Turn off the "-x" option

Back to Top

4. Enhancement of the "-X" option

The "-X" execution option is currently the most common method for tracking and debugging shell scripts, however, the output debugging information is only limited to each actually executed command after the variable is replaced and the "+" prompt at the beginning of the line. Even the important information such as the row number does not exist, debugging of complex shell scripts is very inconvenient. Fortunately, we can skillfully use some built-in environment variables of shell to enhance the output information of the "-X" option. The following describes several built-in environment variables of shell:

$ Lineno
Represents the current row number of the shell script, similar to the built-in macro _ line _ in C Language __

$ Funcname
The function name is similar to the built-in macro _ FUNC __in C language. However, macro _ FUNC _ can only represent the name of the current function, while $ funcname is more powerful, it is an array variable that contains the names of all functions in the call chain. Therefore, the variable $ {funcname [0]} represents the name of the function currently being executed by the shell script, the variable $ {funcname [1]} indicates the name of the function that calls the function $ {funcname [0]}. The remainder can be pushed accordingly.

$ PS4
The primary prompt variables $ PS1 and the second-level prompt variables $ PS2 are common, but few people notice the role of the fourth-level prompt variable $ PS4. We know that using the "-X" execution option will display every actually executed command in the shell script, the value of $ PS4 is displayed before each command output by the "-X" option. In bash shell, the default $ PS4 value is "+. (Why is there a "+" number before the output command when the "-X" option is used ?).

With the $ PS4 feature, we can enhance the output information of the "-X" option by using some built-in variables to redefine the value of $ PS4. For example, first execute export PS4 = '+ {$ lineno :$ {funcname [0]}', and then use the "-X" option to execute the script, the row number and function name of each actually executed command are displayed.

The following is an example of a shell script with a bug. This document uses this script to demonstrate how to use "-n" and the enhanced "-X" execution options to debug the shell script. This script defines the isroot () function to determine whether the current user is a root user. If not, stop the execution of the script.

$ cat –n exp4.sh 1 #!/bin/bash 2 isRoot() 3 { 4 if [ "$UID" -ne 0 ] 5 return 1 6 else 7 return 0 8 fi 9 } 10 isRoot 11 if ["$?" -ne 0 ] 12 then 13 echo "Must be root to run this script" 14 exit 1 15 else 16 echo "welcome root user" 17 #do something 18 fi

Run sh-N exp4.sh to check the syntax. The output is as follows:

$ sh –n exp4.sh exp4.sh: line 6: syntax error near unexpected token `else' exp4.sh: line 6: ` else'

A syntax error was found. Check the commands before and after the first line carefully, we found that 4th rows of IF Statements lack the then keyword (people who are familiar with the C program can easily make this mistake ). We can modify row 4th to if ["$ uid"-Ne 0]; then to correct this error. Run sh-N exp4.sh again to check the syntax. No errors are reported. Next, you can execute the script. The execution result is as follows:

$ sh exp4.sh exp2.sh: line 11: [1: command not found welcome root user

Although the script has no syntax errors, the error is reported during execution. The error message is also very strange: "[1: Command not found ". Now we can try to customize the $ PS4 value and use the "-X" option for tracking:

$ export PS4='+{$LINENO:${FUNCNAME[0]}} ' $ sh –x exp4.sh +{10:} isRoot +{4:isRoot} '[' 503 -ne 0 ']' +{5:isRoot} return 1 +{11:} '[1' -ne 0 ']' exp4.sh: line 11: [1: command not found +{16:} echo 'welcome root user' welcome root user

From the output result, we can see the statement actually executed by the script. The row number and function name of the statement are also printed, the execution track of the script and the internal execution of the called function can be clearly analyzed. Because the error is reported when the execution is 11th rows, this is an if statement. We will compare and analyze the tracking results of the 4th rows that are the same as the if statement:

+{4:isRoot} '[' 503 -ne 0 ']' +{11:} '[1' -ne 0 ']'

We can see that because there is a space missing behind the [number in line 11th, the [number is next to its variable $? The value 1 of is regarded as a whole by the shell interpreter and tries to regard it as a command for execution. Therefore, there is an error message such as "[1: Command not found. You only need to insert a space after the [No.

There are other built-in variables that are helpful for debugging, such as bash_source and bash_subshell in bash shell, you can use man Sh or man Bash to view and use these built-in variables to customize $ PS4 based on your debugging purposes, to enhance the output information of the "-X" option.

Back to Top

V. Summary

Now let's summarize the shell script debugging process:
First, use the "-n" option to check for syntax errors, and then use the "-X" option to track script execution. before using the "-X" option, do not forget to first customize the PS4 variable value to enhance the output information of the "-X" option. At least make it output the line number information (first execute the export PS4 = '+ [$ lineno]', the more permanent way is to add this statement to your user's home directory. bash_profile file), which makes debugging easier. You can also use traps, debugging hooks, and other means to output key debugging information to quickly narrow down the scope of troubleshooting errors, in the script, use "Set-X" and "set + X" to focus on some code blocks. We believe that you can easily capture bugs in your shell scripts. If your script is complex enough and requires more debugging capabilities, you can use the shell debugger bashdb. This is a debugging tool similar to GDB that can complete breakpoint settings for shell scripts, many functions such as single-step execution and variable observation are of great benefit to reading and understanding complex shell scripts using bashdb. Bashdb installation and use do not fall into the scope of this article. For more information, see http://bashdb.sourceforge.net.