"Go" Linux tips: Bash parameters and Parameter extensions

Focus on Listing 7

Now, many Linux® and UNIX® systems have bash shells, which are the common default shells on Linux. In this article, you'll learn how to handle parameters and options in a bash script, and how to use the shell's parameter extensions to check or modify parameters. This article focuses on bash, where the examples are run on a bash-shell Linux system. However, many other shells also have these extensions, such as Ksh, Ash, or dash, which you can use in other UNIX systems or even Cygwin environments. An earlier article on Linux tricks: Bash Testing and comparison functions have been described in the building tools in this article. Some of the materials in this article are excerpted from the DeveloperWorks tutorial LPI 102 Exam Preparation, subject 109:shell, scripting, programming, and compiling, which introduces a number of basic scripting techniques.

Passed parameters

One of the beauties of function and shell scripting is that passing parameters to a single function or script can make them behave differently. In this section, you will learn how to identify and use parameters that are passed.

In a function or script, you can use the Bash special variables listed in table 1 to refer to parameters. You can append these variables to the $ symbol prefix and then reference them as you would reference other shell variables.

table 1. Shell parameter of functionThe

parameter	purpose
0, 1, 2, ...	position parameter starts with parameter 0. The parameter 0 references the name of the program that launched bash, and if the function is running in a shell script, the name of the shell script is referenced. For additional information about this parameter, such as bash by  -C  parameters, see the Bash manual page. A string surrounded by single or double quotes is passed as a parameter, and the quotation marks are removed when passed. In double quotes, shell variables such as $HOME are extended before the function is called. For parameters that contain embedded white space or other characters that may have special meaning to the shell, you need to use either single or double quotation marks for the pass. The
*	position parameter starts with parameter 1. If the extension is in double quotation marks, the extension is a word, separated by the first character of the IFS special variable, and there are no spacing spaces if ifs is empty. The default values for IFS are white space, tab characters, and line breaks. If IFS is not set, whitespace is used as the delimiter (only for the default IFS). The
@	position parameter starts with parameter 1. If you extend in double quotes, each parameter becomes a word, so "[email protected]" is equivalent to "$" "$". You will need to use this form if the parameters may contain embedded whitespace.
#	Number of parameters (not including parameter 0).

Note: If you have more than 9 parameters, you cannot refer to the tenth parameter by using $ $. First, you must process or save the first parameter ($), and then use theshiftcommand to delete the parameter 1 and move all remaining parameters down one bit, so that $ A is $9, and so on. The value of the $# is updated to reflect the remaining quantity of the parameter. In practice, the most common scenario is to iterate over a parameter to a function or shell script, or to iterate over a command to replacefora list created with a statement, so this constraint is basically not a problem.

Now you can define a simple function that simply tells you how many parameters it has and displays the parameters, as shown in Listing 1.

Listing 1. function parameters

[[email protected] ~]$ testfunc () { echo "$# parameters"; echo "[email protected]"; }
[[email protected] ~]$ testfunc
0 parameters

[[email protected] ~]$ testfunc a b c
3 parameters
a b c
[[email protected] ~]$ testfunc a "b c"
2 parameters
a b c

The Shell script handles the parameters in the same way as the function handles the parameters. In fact, you will often find that scripts are often assembled from a number of small functions. Listing 2 shows a shell script testfunc.sh for the same simple task, and the result is to use one of the inputs above to run the script. Rememberchmod +xto mark the script as executable using.

Listing 2. Shell Script Parameters

[[email protected] ~]$ cat testfunc.sh
#!/bin/bash
echo "$# parameters"
echo "[email protected]";
[[email protected] ~]$ ./testfunc.sh a "b c"
2 parameters
a b c


  

	
   


  

In table 1 you will find that the shell may refer to the list of passed arguments as $* or [email protected], and whether these expressions are quoted will affect how they are interpreted. For the above function, the result of using $*, "$*", [email protected] or "[email protected]" Output is not very different, but if the function is more complicated, it is not so sure, When you want to parse parameters or pass some parameters to other functions or scripts, the difference between using or not quoting is obvious. Listing 3 shows a function that prints the number of parameters and then prints the parameters according to these four options. Listing 4 shows the functions in use. The IFS default variable uses a space as its first character, so listing 4 adds a vertical bar as the first character of the IFS variable, showing more clearly where to use the character in the "$*" extension.

Listing 3. A function to explore the differences in parameter handling

[[email protected] ~]$ type testfunc2
testfunc2 is a function
testfunc2 ()
{
    echo "$# parameters";
    echo Using ‘$*‘;
    for p in $*;
    do
        echo "[$p]";
    done;
    echo Using ‘"$*"‘;
    for p in "$*";
    do
        echo "[$p]";
    done;
    echo Using ‘[email protected]‘;
    for p in [email protected];
    do
        echo "[$p]";
    done;
    echo Using ‘"[email protected]"‘;
    for p in "[email protected]";
    do
        echo "[$p]";
    done
}

Listing 4. Printing parameter information using TESTFUNC2

[[email protected] ~]$ IFS="|${IFS}" testfunc2 abc "a bc" "1 2
> 3"
3 parameters
Using $*
[abc]
[a]
[bc]
[1]
[2]
[3]
Using "$*"
[abc|a bc|1 2
3]
Using [email protected]
[abc]
[a]
[bc]
[1]
[2]
[3]
Using "[email protected]"
[abc]
[a bc]
[1 2
3]

Take a closer look at the differences between the two, especially if you want to pay attention to the form of quotation marks and the parameters that contain whitespace, such as space characters and line breaks. In a [] character pair, note: the "$*" extension is actually a word.

Back to top of page

Options and Getopts

Traditional UNIX and Linux commands treat some of the passed parameters as options . In the past, these parameters were a single character switch, which differs from other parameters in having a leading hyphen or minus sign. For convenience, several options can be incorporated into thels -lrtcommand, which provides a-t-rlist of long (option) catalogs sorted by the reverse (option) of the modified time (option)-l.

You can use the same techniques for shell scripts, andgetoptsbuilt-in commands can simplify your tasks. To see how this command works, consider the example script testopt.sh shown in Listing 5.

Listing 5. testopt.sh Script

#!/bin/bash
echo "OPTIND starts at $OPTIND"
while getopts ":pq:" optname
  do
    case "$optname" in
      "p")
        echo "Option $optname is specified"
        ;;
      "q")
        echo "Option $optname has value $OPTARG"
        ;;
      "?")
        echo "Unknown option $OPTARG"
        ;;
      ":")
        echo "No argument value for option $OPTARG"
        ;;
      *)
      # Should not occur
        echo "Unknown error while processing options"
        ;;
    esac
    echo "OPTIND is now $OPTIND"
  done

getoptsThe command uses two pre-determined variables. The Optind variable starts to be set to 1. It then contains the index of the next parameter to be processed. If an option is found, thegetoptscommand returnstrue, so the common option handling paradigm usescaseloops with statementswhile, as in this case.getoptsThe first parameter is a list of the option letters to be identified, in this case thepandr. The colon after the option letter (:) Indicates that the option requires a value; For example, an-foption might be used to represent a file name, astaris the case in the command. The leading colon in this example tells Getopts to remain silent (silent) and suppress the normal error message, because this script will provide its own error handling.

The second parameter in this exampleoptnameis a variable name that will receive the name of the found option. If an option should be expected to have a value and the value is present, the value is placed in the OPTARG variable. In silent mode, the following two error conditions may occur.

1. If you find an option that is not recognized, optname will contain one? The optarg will contain the unknown option.
2. If an option is found that requires a value, but the value is not found, optname will contain a: and Optarg will contain the name of the option for the missing parameter.

If not in silent mode, these errors will cause a diagnostic error message and OPTARG will not be set. Can the script be used in optname? Or: value to detect errors (and possibly handle errors).

Listing 6 shows the two examples of running this simple script.

Listing 6. Run the testopt.sh script

[[email protected] ~]$ ./testopt.sh -p -q
OPTIND starts at 1
Option p is specified
OPTIND is now 2
No argument value for option q
OPTIND is now 3
[[email protected] ~]$ ./testopt.sh -p -q -r -s tuv
OPTIND starts at 1
Option p is specified
OPTIND is now 2
Option q has value -r
OPTIND is now 4
Unknown option s
OPTIND is now 5

If you need to do this, you can pass a set of parameters to thegetoptscalculation. If you have already called with a set of parameters in your scriptgetoptsand now call it with a different set of parameters, you will need to reset the Optind to 1 yourself. For more details, please refer to the Bash manual or information page.

Back to top of page

Parameter extension

Now that you've learned how to pass parameters to a function or script and how to identify options, start working with options and parameters. It should be a good thing if you know what parameters are left after processing the options. Next you may need to verify the parameter values or assign a default value to the missing parameters. This section describes some of the parameter extensions in bash. Of course, you still have all the functionality of Linux or UNIX commands (such assedorawk) to perform more complex work, but you should also know how to use shell extensions.

We started using the option analysis and parametric analysis functions described above to build a script. The testargs.sh script is given in Listing 7.

Listing 7. testargs.sh Script

#!/bin/bash

showopts () {
  while getopts ":pq:" optname
    do
      case "$optname" in
        "p")
          echo "Option $optname is specified"
          ;;
        "q")
          echo "Option $optname has value $OPTARG"
          ;;
        "?")
          echo "Unknown option $OPTARG"
          ;;
        ":")
          echo "No argument value for option $OPTARG"
          ;;
        *)
        # Should not occur
          echo "Unknown error while processing options"
          ;;
      esac
    done
  return $OPTIND
}

showargs () {
  for p in "[email protected]"
    do
      echo "[$p]"
    done
}

optinfo=$(showopts "[email protected]")
argstart=$?
arginfo=$(showargs "${@:$argstart}")
echo "Arguments are:"
echo "$arginfo"
echo "Options are:"
echo "$optinfo"

Try running this script several times to see how it works and then examine it in detail. Listing 8 shows some examples of output.

Listing 8. Run the testargs.sh script

[[email protected] ~]$ ./testargs.sh -p -q qoptval abc "def ghi"
Arguments are:
[abc]
[def ghi]
Options are:
Option p is specified
Option q has value qoptval
[[email protected] ~]$ ./testargs.sh -q qoptval -p -r abc "def ghi"
Arguments are:
[abc]
[def ghi]
Options are:
Option q has value qoptval
Option p is specified
Unknown option r
[[email protected] ~]$ ./testargs.sh "def ghi"
Arguments are:
[def ghi]
Options are:

Notice how these parameters are separated from the options. Theshowoptsfunction parses the option as before, but uses the return statement to return the value of the Optind variable to the calling statement. Call processing assigns this value to the variable Argstart. This value is then used to select a subset of the original parameters, including those that are not processed as options. This process uses parameter extensions
${@:$argstart}To complete.
Remember to use quotation marks on both sides of this expression to keep parameters and embedded spaces together, as shown in the rear of Listing 2.

If you're new to using scripts and functions, keep in mind the following instructions:

1. returnStatement returns the exit value of the Showopts function, and the caller uses $? To access the function. You can also use commands such as the return value of a function andtestorwhilethe like to control branching and looping.
2. The Bash function includes some optional words-"functions", such as:
   function showopts ()
   This is not part of the POSIX Standard and is not supported by shells such as dash, so if you want to use it, do not set the shebang line to
   #!/bin/sh
   Because this will give you the default shell for your system, it may not run the way you want it to.
3. The output from the function output, such as the statements of the two functions here,echois not printed, but the caller can access the output. If the output is not assigned to a variable, or if it is not used elsewhere as part of the invocation statement, the shell will attempt to execute the output instead of displaying the output.

Subsets and substrings

The general form of this extension is ${parameter:offset:length}, where the LENGTH parameter is an optional parameter. Therefore, if you only want to select a specific subset of the script parameters, you can use the full version to specify how many parameters to select. For example,${@:4:3}refer to the 3 parameters starting with parameter 4, which are parameters 4, 5, and 6. You can use this extension to select a single parameter in addition to those parameters that are immediately accessible by using $ $9.${@:15:1}is a method of directly accessing parameter 15.

You can use this extension in conjunction with a single parameter or with the entire set of parameters represented by $* or [email protected]. In this case, the parameter is treated as a string and a reference to the offset and length of the number. For example, if the value of the variable x is "some value", the
${x:3:5}
The value is "E Val", as shown in Listing 9.

Listing 9. Substring of the shell parameter value

[[email protected] ~]$ x="some value"
[[email protected] ~]$ echo "${x:3:5}"
e val

Length

You already know that $# represents the number of parameters, while the ${parameter:offset:length} extension applies to a single parameter as well as $* and [email protected], so you can use a similar struct ${#PARAMETER} It is not surprising to determine the length of a single parameter. This is illustrated by the simple function shown in Listing 10testlength. Try to use it yourself.

Listing 10. Parameter length

[[email protected] ~]$ testlength () { for p in "[email protected]"; do echo ${#p};done }
[[email protected] ~]$ testlength 1 abc "def ghi"
1
3
7

Pattern matching

The parameter extension also includes some pattern-matching features with wildcard features used in file name extensions or globbing. Note: This is notgrepused for regular expression matching.

table 2. Shell extension pattern matching

Extended	Purpose
${parameter#word}	The shell extends WORD in the same way as the file name extension and removes the shortest matching pattern from the beginning of the value of the PARAMETER extension (if there is a matching pattern). Use ' @ ' or ' $ ' to delete the pattern for each parameter in the list.
${parameter# #WORD}	Causes the longest matching pattern to be deleted from the beginning rather than the shortest matching pattern.
${parameter%word}	The shell extends WORD in the same way as the file name extension and removes the shortest matching pattern from the end of the PARAMETER extended value (if there is a matching pattern). Use ' @ ' or ' $ ' to delete the pattern for each parameter in the list.
${parameter%%word}	Causes the longest matching pattern to be removed from the end rather than the shortest matching pattern.
${parameter/pattern/string}	The shell expands the pattern as in the filename extension and replaces the longest matching pattern in the value of the PARAMETER extension (if there is a matching pattern). In order to match the pattern at the beginning of the value of the PARAMETER extension, the pattern can be appended with the prefix #, and the prefix% is appended if you want to match the pattern at the end of the value. If the STRING is empty, then/may be ignored at the end, and the match will be deleted. Use ' @ ' or ' $ ' to replace each parameter in the list with a pattern.
${parameter//pattern/string}	The substitution is performed on all matches (not just the first match).

Listing 11 shows some basic uses for pattern-matching extensions.

Listing 11. Pattern matching Example

[[email protected] ~]$ x="a1 b1 c2 d2"
[[email protected] ~]$ echo ${x#*1}
b1 c2 d2
[[email protected] ~]$ echo ${x##*1}
c2 d2
[[email protected] ~]$ echo ${x%1*}
a1 b
[[email protected] ~]$ echo ${x%%1*}
a
[[email protected] ~]$ echo ${x/1/3}
a3 b1 c2 d2
[[email protected] ~]$ echo ${x//1/3}
a3 b3 c2 d2
[[email protected] ~]$ echo ${x//?1/z3}
z3 z3 c2 d2

Back to top of page

Integration

Before you introduce the rest of the points, take a look at the actual examples of parameter handling. I built the DeveloperWorks author package (see Resources for information on Linux systems using bash scripts). We store the various files needed in the subdirectories of the Developerworks/library library. The latest release of the library is version 5.7, so the schema file is in developerworks/library/schema/5.7, the XSL file is in developerworks/library/xsl/5.7, and the sample template is located in the The developerworks/library/schema/5.7/templates. Obviously, a parameter that provides a version (5.7 in this case) satisfies the need for the script to build the path to all of these files. So the script gets the-v parameter that must have a value. Validation is performed later on this parameter by building the path and then using the[ -d "$pathname" ]check to see if it exists.

This approach is very effective for product construction, but during development, files are stored in different directories:

• developerworks/library/schema/5.8/archive/test-5.8/merge-0430
• Developerworks/library/xsl/5.8/archive/test-5.8/merge-0430 and
• developerworks/library/schema/5.8/archive/test-5.8/merge-0430/templates-0430

The current version in these directories is 5.8,0430, which represents the date of the latest beta version.

To handle this, I added a parameter-pthat contains a supplemental path information-archive/test-5.8/merge-0430. Now, I (or someone else) may have forgotten the leading slash or trailing slash, and some Windows users might use backslashes instead of forward slashes, so I decided to work with this in my script. Also, you notice that the path to the template directory contains two dates, so you need to try to remove the date 0430 at run time.

Listing 12 shows the code used to process two parameters and clean up some paths based on those requirements. The value of the-v option is stored in the ssversion variable, and the clean-p variable is stored in Pathsuffix, and the date (along with the leading hyphen) is stored in the Datesuffix. The comments explain the actions that are performed at each step. Even in such a small piece of script, you can find some parameter extensions, including length, substring, pattern matching, and pattern substitution.

Listing 12. Analyze parameters for DeveloperWorks author package builds

while getopts ":v:p:" optval "[email protected]"
  do
    case $optval in
      "v")
        ssversion="$OPTARG"
      ;;
      "p")
        # Convert any backslashes to forward slashes
        pathsuffix="${OPTARG//\\//}"
        # Ensure this is a leading / and no trailing one
        [ ${pathsuffix:0:1} != "/" ] && pathsuffix="/$pathsuffix"
        pathsuffix=${pathsuffix%/}
        # Strip off the last hyphen and what follows
        dateprefix=${pathsuffix%-*}
        # Use the length of what remains to get the hyphen and what follows
        [ "$dateprefix" != "$pathsuffix" ] && datesuffix="${pathsuffix:${#dateprefix}}"
        ;;
      *)
        errormsg="Unknown parameter or option error with option - $OPTARG"
        ;;
    esac
  done

Like most content in Linux, this is probably not the only solution for programming, but it does show a more practical use of the extensions you know.

Back to top of page

Default value

In the previous section, you learned how to assign option values to variables such as ssversion or Pathsuffix. In this case, null values will be detected later, and a null path suffix will appear when the product is built, so this is acceptable. What if I need to assign a default value to a parameter that has not been specified? The shell extensions shown in table 3 will help you accomplish this task.

Table 3. Default value-dependent Shell extensions

Extended	Purpose
${parameter:-word}	If PARAMETER is not set or empty, the shell expands WORD and replaces the result. The value of PARAMETER has not changed.
' ${parameter:=word}	If PARAMETER is not set or empty, the shell expands WORD and assigns the result to PARAMETER. This value is then replaced. You cannot assign values for positional or special parameters in this manner.
${parameter:? WORD}	If PARAMETER is not set or empty, the shell extends WORD and writes the result to a standard error. Writes a message if no WORD is in the line. If the shell is not interactive, it means that the extension exists.
${parameter:+word}	If the PARAMETER is not set or is empty, no substitution is made. Otherwise, the shell expands WORD and replaces the result.

Listing 13 illustrates these extensions and the differences between them.

Listing 13. Replace an empty variable or a variable that is not set.

[[email protected] ~]$ unset x;y="abc def"; echo "/${x:-‘XYZ‘}/${y:-‘XYZ‘}/$x/$y/"
/‘XYZ‘/abc def//abc def/
[[email protected] ~]$ unset x;y="abc def"; echo "/${x:=‘XYZ‘}/${y:=‘XYZ‘}/$x/$y/"
/‘XYZ‘/abc def/‘XYZ‘/abc def/
[[[email protected] ~]$ ( unset x;y="abc def"; echo "/${x:?‘XYZ‘}/${y:?‘XYZ‘}/$x/$y/" )>  >so.txt 2>se.txt
[[email protected] ~]$ cat so.txt
[[email protected] ~]$ cat se.txt
-bash: x: XYZ
[[[email protected] ~]$ unset x;y="abc def"; echo "/${x:+‘XYZ‘}/${y:+‘XYZ‘}/$x/$y/"
//‘XYZ‘//abc def/

Back to top of page

Passing parameters

There are some subtleties about parameter passing, which can make mistakes if you are not careful. You have already learned the importance of using quotes and the effect of quotes on using $* and [email protected], but consider the following example. Suppose you want a script or function to manipulate all the files or directories in the current working directory. To illustrate this example, consider the ll-1.sh and ll-2.sh scripts shown in Listing 14.

Listing 14. Two examples of scripts

#!/bin/bash
# ll-1.sh
for f in "[email protected]"
  do
    ll-2.sh "$f"
  done

#!/bin/bash
ls -l "[email protected]"

The script ll-1.sh simply passes each of its arguments to the script ll-2.sh and ll-2.sh a long list of the arguments passed. My test directory contains two empty files "file1" and "File 2". Listing 15 shows the output of the script.

Listing 15. Run the script-1

[[email protected] test]$ ll-1.sh *
-rw-rw-r-- 1 ian ian 0 May 16 15:15 file1
-rw-rw-r-- 1 ian ian 0 May 16 15:15 file 2

So far, everything has progressed well. However, if you forget to use the * parameter, the script will not take any action. It does notlsautomatically execute the contents of the current working directory as commands do. You can make a simple fix that adds a check for this condition in ll-1.sh and useslsthe output of the command to generate ll-2.sh input when no data is provided to Ll1-sh. Listing 16 shows a possible solution.

Listing 16. Ll-1.sh after the correction

#!/bin/bash
# ll-1.sh - revision 1
for f in "[email protected]"
  do
    ll-2.sh "$f"
  done
[ $# -eq 0 ] && for f in "$(ls)"
  do
    ll-2.sh "$f"
  done

Note: We carefully quote thelsresult of the command in quotation marks to ensure that "file 2" is handled correctly. Listing 17 shows the results of running a new ll-1.sh with * and without *.

Listing 17. Run the script-2

[[email protected] test]$ ll-1.sh *
-rw-rw-r-- 1 ian ian 0 May 16 15:15 file1
-rw-rw-r-- 1 ian ian 0 May 16 15:15 file 2
[[email protected] test]$ ll-1.sh
ls: file1
file 2: No such file or directory

That's weird, isn't it? When you pass parameters, especially if they are the output of the command, it may take some skill to process them. The error message indicates that the file name is delimited by a newline character, which gives us a clue. There are a number of ways to deal with this problem, but there is an easy way to do this, using the built-in Listing 18read. You can try it yourself.

Listing 17. Run the script-2

#!/bin/bash
# ll-1.sh - revision 2
for f in "[email protected]"
  do
    ll-2.sh "$f"
  done
[ $# -eq 0 ] && ls | while read f
  do
    ll-2.sh "$f"
  done

The purpose of this example is to illustrate that the script can be more reliable by paying attention to the details and using some unusual input to test. I wish you a smooth programming!

"Go" Linux tips: Bash parameters and Parameter extensions