Shell script programming Basics

Shell programming BASICS (from the UBUNTU Forum)

(12:49:20)

Reprinted

Tags: Shell

Classification: programming (C ++)

Http://wiki.ubuntu.org.cn/Shell%E7%BC%96%E7%A8%8B%E5%9F%BA%E7%A1%80

Why shell programming?
In Linux, although there are various graphical interfaces, shell is still a very flexible tool. Shell not only collects a large number of commands, but also a very good programming language. With shell, a large number of tasks can be automated. Shell is especially good at system management tasks, especially for those tasks that are easier to use, maintainability, and portability than efficiency.

Next, let's take a look at how shell works:
Create a script
Linux has many different shells. Usually we use Bash (Bourne again shell) for shell programming, because Bash is free (free) and easy to use. The scripts provided in this article are all executed using bash (of course, in most cases, these scripts can also be run in Bash's predecessors and Bourne shell.
Like other languages, we can use any text editor, such as nedit, kedit, emacs, and VI to write shell scripts, it must start with the following line (the first line of the file must be placed ):

Code:
#! /Bin/sh

Symbol #! The program used to tell the system to execute the script. This example uses/bin/sh. The script has been edited. to execute this script, you must make it executable:

Code:
Chmod + x filename

Enter./filename to execute the script.
Note
In the shell script, the line starting with # indicates the comment until the end of the line. We sincerely recommend that you use annotations in the script so that you can understand the role and working principle of the script in a short time even if you haven't used the script for a long time; there are also important reasons-with comments, you can share your scripts with others.

Variable
In other programming languages, you must use variables. In shell programming, all variables are composed of strings and do not need to be declared. To assign a value to a variable, you can write as follows:

Code:
Variable name = Value

Add $ before the variable name to retrieve the variable value:

Code:
#! /Bin/sh
# Assign values to variables:
A = "Hello World"

# Print the value of variable:
Echo "A is:" $

Select your favorite editor, enter the above content, save it as file first, execute chmod + X first to make it executable, and finally input./first to execute the script. The output result is as follows:

Reference:
A is: Hello World

Sometimes the variable name may be confused with other words, such:

Code:
Num = 2
Echo "this is the $ numnd"

The above script does not output "this is the 2nd", and only prints "this is the"; this is because shell will search for the value of the variable numnd, in fact, this variable has no value at this time. We can use curly braces to tell shell that we want to print the num variable:

Code:
Num = 2
Echo "this is the $ {num} Nd"

The output result of the above script is: This is the 2nd

Many variables are automatically set by the system. We will explain them later when using these variables. If you need to process mathematical expressions, you must use programs such as expr.
In addition to common shell variables that are valid only in the script, there are also environment variables, that is, those that have been processed by the Export keyword. This article does not discuss environment variables, because only environment variables are used in the login script.

Shell commands and Process Control
Three types of commands can be used in shell scripts:

1) Unix command:
Any Unix command can be used in shell scripts, but in fact, the most common commands are those related to file and text operations. The following describes some common command syntaxes and functions:
Echo "some text": Output Information on the screen
Ls: file list
WC-l file WC-W file WC-C file: calculate the number of lines, word, and character of the file, respectively)
CP sourcefile destfile: file copy
MV oldname newname: rename a file or move a file
RM file: delete an object
Grep 'pattern' file: searches for strings in the file or strings matching the regular expression.
Cut-B column file: output the file content within the specified range to the standard output device (screen. For example: output line 5th to 9 Characters cut-b5-9 file.txt, be sure not to be confused with the cat Command, this is two completely different commands
Cat file.txt: output file content to the standard output device (screen)
File somefile: gets the file type of the file somefile.
Read var: prompt the user to input and assign the input content to the variable VAR
Sort file.txt: sorts all rows of the file.txt file.
Uniq: Only output rows with inconsistent content in the file, such as: Sort file.txt | uniq
Expr: performs mathematical operations. For example, to perform 2 + 3 operations, the command is: expr 2 "+" 3
Find: search for a file. For example, search by file name: find.-Name filename-print
Tee: Outputs Data to standard output devices (screens) and files, for example, somecommand | tee OUTFILE.
Basename file: returns the file name that does not contain the path. For example, basename/bin/tux returns tux.
Dirname file: the path of the returned file. For example, dirname/bin/tux returns/bin.
Head file: prints the first few lines of a text file.
Tail file: number of rows at the end of a text file
Sed: SED is a basic search replacement program. You can read text from a standard input (such as a command pipeline) and output the results to the standard output (screen). This command uses regular expressions for search. Do not confuse with wildcards in shell. For example, replace Ubuntu with Ubuntu: CAT text. File | SED's/Ubuntu/'> newtext. File
Awk: awk is used to extract fields from text files. The default field delimiter is a space. You can use-F to specify other separators. Cat file.txt | awk-F, '{print $1 "," $3}', which is used here as a field delimiter to print both the First and Third fields. If the file contains Adam Bor, 34, indiakerry Miller, 22, USA, the output of the preceding command is Adam Bor, indiakerry Miller, USA.

2) concept: pipelines, redirection, and backtick
Although these are not system commands, they play a very important role.
The pipeline (|) uses the output of a command as the input of another command.

Code:
Grep "hello" file.txt | WC-l

The preceding command searches for rows containing "hello" in file.txt and calculates the number of rows. Here, the grep command output is the WC command input. Of course, you can use multiple commands.
Redirection: output the command result to a file instead of a standard output (screen ).

Code:
> Write the file and overwrite the old file
> Add it to the end of the file to retain the content of the old file.

Backlash
You can use a backslash to output a command as a command line parameter of another command.

Code:
Find.-mtime-1-type F-print

The preceding command can be used to search for files modified in the past 24 hours (-mtime-2 indicates the past 48 hours. If you want to pack all the searched files, you can use the following script:

Code:
#! /Bin/sh
# The ticks are backticks (') not normal quotes ('):
Tar-zcvf lastmod.tar.gz 'Find.-mtime-1-type F-print'

3) Process Control
If the expression "if" is true, the part after then is executed:

Code:
If...; then
....
Elif...; then
....
Else
....
Fi

In most cases, you can use test commands to test the conditions. For example, you can compare strings, determine whether a file exists, and whether the file is readable... Generally, "[]" is used for conditional testing. Note that spaces are important. Make sure that the spaces before and after square brackets are used.

Code:
[-F "somefile"]: determines whether it is a file.
[-X "/bin/ls"]: determines whether/bin/ls exists and has the executable permission.
[-N "$ Var"]: determines whether the $ var variable has a value.
["$ A" = "$ B"]: determines whether $ A and $ B are equal.

Run man test to view all types of test expressions that can be compared and judged.
Directly execute the following script:

Code:
#! /Bin/sh
If ["$ shell" = "/bin/bash"]; then
Echo "your login shell is the bash (Bourne again shell )"
Else
Echo "your login shell is not bash but $ shell"
Fi

The variable $ shell contains the name of the logon shell. We can compare it with/bin/bash.

Shortcut Operators
If you are familiar with the C language, you may like the following expressions:

Code:
[-F "/etc/shadow"] & Echo "this computer uses shadow passwors"

& Is a shortcut operator. If the expression on the left is true, execute the Statement on the right. You can also regard it as a logical operation and operation. The above script indicates that if the/etc/shadow file exists, print "this computer uses shadow passwors ". The same or operation (|) can also be used in shell programming, for example:

Code:
#! /Bin/sh
Mailfolder =/var/spool/mail/James
[-R "$ mailfolder"] | {echo "can not read $ mailfolder"; Exit 1 ;}
Echo "$ mailfolder has mail from :"
Grep "^ From" $ mailfolder

The script first checks whether mailfolder is readable. If it is readable, it prints a line of "from" in the file. If it is not readable or the operation takes effect, print the error message and exit the script. There is a problem here, that is, we must have two commands:
-Print the error message.
-Exit the program.
We use curly braces to put the two commands together as one command in the form of an anonymous function. General functions will be mentioned below. We can use the if expression to do anything without the sum or operator, but it is much more convenient to use the sum or operator.
A case expression can be used to match a given string, not a number.

Code:
Case... in
...) Do something here ;;
Esac

Let's take an example. The file command can identify the file type of a given file, for example, file lf.gz. The output result of this command is:

Reference:
Lf.gz: gzip compressed data, deflated, original filename,
Last modified: Mon Aug 27 23:09:18 2001, OS: Unix

We use this to write a script named smartzip, which can automatically decompress Bzip2, Gzip, and zip compressed files:

#! /Bin/sh
FTYPE = 'file "$1 "'
Case "$ FTYPE" in
"$1: ZIP Archive "*)
Unzip "$1 ";;
"$1: gzip COMPRESSED "*)
Gunzip "$1 ";;
"$1: Bzip2 COMPRESSED "*)
Bunzip2 "$1 ";;
*) Error "File $1 can not be uncompressed with smartzip ";;
Esac

You may notice that we use a special variable $1 here. The variable contains the first parameter value passed to the program. That is, when we run:

Smartzip articles.zip
$1 is the string articles.zip

The Select expression is a bash extension application, especially for interactive use. You can select from a group of different values.

Select VaR in...; do
Break
Done
... Now $ VaR can be used ....
The following is an example:
#! /Bin/sh
Echo "What is your favorite OS? "
Select VaR in "Linux" "GNU Hurd" "Free BSD" "other"; do
Break
Done
Echo "you have selected $ Var"

The following is the result of running the script:

What is your favorite OS?
1) Linux
2) GNU Hurd
3) Free BSD
4) Other
#? 1
You have selected Linux

You can also use the following loop expression in shell:

While...; do
....
Done

While-loop will run until the expression test is true. Will run while the expression that we test for is true. The keyword "break" is used to jump out of the loop. The keyword "continue" is used to directly jump to the next loop without executing the remaining part.

The for-loop expression is used to view a string list (strings are separated by spaces) and then assigned to a variable:

For VaR in...; do
....
Done

In the following example, ABC is printed to the screen:

#! /Bin/sh
For var in a B C; do
Echo "Var is $ Var"
Done

The following is a more useful script showrpm. Its function is to print statistics of some RPM packages:

#! /Bin/sh
# List a Content summary of a number of RPM packages
# Usage: showrpm rpmfile1 rpmfile2...
# Example: showrpm/CDROM/RedHat/RPMS/*. rpm
For rpmpackage in $ *; do
If [-R "$ rpmpackage"]; then
Echo "=====================$ rpmpackage ===================="
Rpm-qi-p $ rpmpackage
Else
Echo "error: cannot read file $ rpmpackage"
Fi
Done

The second special variable $ * is displayed, which contains all input command line parameter values. If you run showrpm OpenSSH. RPM w3m. RPM webgrep. rpm

$ * Contains three strings: OpenSSH. rpm, w3m. rpm, and webgrep. rpm.

Quotation marks

Before passing any parameters to a program, the program extends the wildcards and variables. Here, the extension means that the program will replace the wildcard (for example, *) with the appropriate file name, and replace the variable with the variable value. To prevent such replacement, you can use quotation marks: Let's take a look at an example. Suppose there are some files in the current directory, two JPG files, mail.jpg and tux.jpg.

#! /Bin/sh
Echo *. jpg

This will print the result of "mail.jpg tux.jpg.

Quotation marks (single quotation marks and double quotation marks) will prevent such wildcard extension:

#! /Bin/sh
Echo "*. jpg"
Echo '*. jpg'

This will print "*. jpg" twice.

Single quotes are stricter. It can prevent any variable extension. Double quotation marks can prevent wildcard extension but allow variable extension.

#! /Bin/sh
Echo $ Shell
Echo "$ shell"
Echo '$ shell'

The running result is:

/Bin/bash
/Bin/bash
$ Shell

Finally, there is also a method to prevent this extension, that is, to use the Escape Character -- reverse oblique ROD:

Echo *. jpg
Echo $ Shell

This will output:

*. Jpg
$ Shell
Here documents

To pass several lines of text to a command, here documents is a good method. It is very useful to write a helpful text for each script. If we have the here documents, we do not need to use the echo function to output a row. A "Here document" starts with <, followed by a string, which must also appear at the end of the here document. The following is an example. In this example, we rename multiple files and print the help using here documents:

#! /Bin/sh
# We have less than 3 arguments. Print the help text:
If [$ #-lt 3]; then
Cat <Ren -- renames a number of files using sed Regular Expressions

Usage: Ren 'regexp' 'replace 'files...

Example: rename all *. HTM files in *. html:
Ren 'htm $ ''html' *. htm

Help
Exit 0
Fi
Old = "$1"
New = "$2"
# The shift command removes one argument from the list
# Command line arguments.
Shift
Shift
# $ * Contains now all the files:
For file in $ *; do
If [-F "$ file"]; then
Newfile = 'echo "$ file" | sed "s/$ {old}/$ {New}/g "'
If [-F "$ newfile"]; then
Echo "error: $ newfile exists already"
Else
Echo "Renaming $ file to $ newfile ..."
Mv "$ file" "$ newfile"
Fi
Fi
Done

This is a complex example. Let's discuss it in detail. The first if expression determines whether the number of input command line parameters is smaller than 3 (special variable $ # indicates the number of parameters included ). If the number of input parameters is less than three, the help text is passed to the cat command and then printed on the screen by the cat command. Print the help text and exit the program. If the input parameter is equal to or greater than three, we assign the first parameter to the variable old, and the second parameter to the variable new. Next, we use the shift command to delete the first and second parameters from the parameter list, so that the original third parameter becomes the first parameter in the parameter list $. Then we start the loop. The command line parameter list is assigned to the variable $ file one by one. Then we determine whether the file exists. If yes, we use the SED command to search for and replace the file to generate a new file name. Then, assign the command result in the backslash to newfile. In this way, we can achieve our goal: Get the old and new file names. Use the MV command to rename the file.

Function

If you write a slightly more complex program, you will find that the same code may be used in several places in the program, and you will also find that if we use a function, it is much more convenient. A function looks like this:

Functionname ()
{
# Inside the body $1 is the first argument given to the Function
#$2 the second...
Body
}

You must declare the function at the beginning of each program.

The following is a script named xtitlebar. You can use this script to change the name of the terminal window. Here we use a function called help. As you can see, this defined function is used twice.

#! /Bin/sh
# VIM: Set Sw = 4 ts = 4 et:

Help ()
{
Cat <xtitlebar -- change the name of an xterm, gnome-terminal or KDE konsole

Usage: xtitlebar [-H] "string_for_titelbar"

Options:-H help text

Example: xtitlebar "CVS"

Help
Exit 0
}

# In case of error or if-H is given we call the function help:
[-Z "$1"] & Help
["$1" = "-h"] & Help

# Send the escape sequence to change the xterm titelbar:
Echo-e "33] 0; $107"
#

It is a good programming habit to help other users (and you) use and understand scripts.

Command Line Parameters

We have seen Special variables such as $ * and $1, $2... $9. These special variables include the parameters you input from the command line. So far, we have only learned some simple command line syntax (such as some mandatory parameters and the-H option for viewing help ). However, when writing more complex programs, you may find that you need more custom options. The common practice is to add a minus sign before all optional parameters, followed by a parameter value (such as a file name ).

There are many ways to analyze input parameters, but the following example using the case expression is a good method.

#! /Bin/sh
Help ()
{
Cat <this is a generic command line parser demo.
Usage example: extends parser-l Hello-f ---somefile1 somefile2
Help
Exit 0
}

While [-n "$1"]; do
Case $1 in
-H) Help; shift 1; # function help is called
-F) opt_f = 1; shift 1; # variable opt_f is set
-L) opt_l = $2; Shift 2; #-l takes an argument-> shift by 2
--) Shift; break; # End of options
-*) Echo "error: no such option $1.-H for help"; Exit 1 ;;
*) Break ;;
Esac
Done

Echo "opt_f is $ opt_f"
Echo "opt_l is $ opt_l"
Echo "first Arg is $1"
Echo "2nd Arg is $2"

You can run the script as follows:

Extends parser-l Hello-f ---somefile1 somefile2

The returned result is:

Opt_f is 1
Opt_l is hello
First Arg is-somefile1
2nd Arg is somefile2

How does this script work? The script first loops through all input command line parameters and compares the input parameters with the case expression. If the input parameters match, a variable is set and the parameter is removed. According to the Convention of the UNIX system, the first input should be the parameter containing the minus sign.

Instance

General programming steps

Now let's discuss the general steps for writing a script. Any excellent script should have help and input parameters. And write a pseudo script (Framework. Sh) that contains the framework structure required by most scripts, which is a very good idea. At this time, when writing a new script, we only need to execute the Copy command:

CP framework. Sh myscript

Then insert your own function.

Let's look at two more examples:

Binary to decimal conversion

The script b2d converts the binary number (such as 1101) to the corresponding decimal number. This is also an example of a mathematical operation using the expr command:

#! /Bin/sh
# VIM: Set Sw = 4 ts = 4 et:
Help ()
{
Cat <B2H -- convert binary to decimal

Usage: B2H [-H] binarynum

Options:-H help text

Example: B2H 111010
Will return 58
Help
Exit 0
}

Error ()
{
# Print an error and exit
Echo "$1"
Exit 1
}

Lastchar ()
{
# Return the last character of a string in $ rval
If [-z "$1"]; then
# Empty string
Rval = ""
Return
Fi
# WC puts some space behind the output this is why we need sed:
Numofchar = 'echo-n "$1" | WC-c | SED's // g''
# Now cut out the last char
Rval = 'echo-n "$1" | cut-B $ numofchar'
}

Chop ()
{
# Remove the last character in string and return it in $ rval
If [-z "$1"]; then
# Empty string
Rval = ""
Return
Fi
# WC puts some space behind the output this is why we need sed:
Numofchar = 'echo-n "$1" | WC-c | SED's // g''
If ["$ numofchar" = "1"]; then
# Only One char in string
Rval = ""
Return
Fi
Numofcharminus1 = 'expr $ numofchar "-" 1'
# Now cut all but the last CHAR:
Rval = 'echo-n "$1" | cut-B 0-$ {numofcharminus1 }'
}

While [-n "$1"]; do
Case $1 in
-H) Help; shift 1; # function help is called
--) Shift; break; # End of options
-*) Error "error: no such option $1.-H for help ";;
*) Break ;;
Esac
Done

# The main program
Sum = 0
Weight = 1
# One Arg must be given:
[-Z "$1"] & Help
Binnum = "$1"
Binnumorig = "$1"

While [-n "$ binnum"]; do
Lastchar "$ binnum"
If ["$ rval" = "1"]; then
Sum = 'expr "$ weight" "+" "$ sum "'
Fi
# Remove the last position in $ binnum
Chop "$ binnum"
Binnum = "$ rval"
Weight = 'expr "$ weight" "*" 2'
Done

Echo "binary $ binnumorig is decimal $ sum"
#

The script uses decimal and binary weights (, 16,...). For example, binary "10" can be converted to decimal:

0*1 + 1*2 = 2

To obtain a single binary number, we use the lastchar function. This function uses WC-C to calculate the number of characters, and then uses the cut command to retrieve the last character. The Chop function removes the last character.

File loop Program

Maybe you want to save all emails to one of the people in a file, but after a few months, this file may become so large that the access to this file may be slowed down. The following script rotatefile can solve this problem. This script can rename the email storage file (assuming outmail) to outmail.1, and change outmail.1 to outmail.2 and so on...

#! /Bin/sh
# VIM: Set Sw = 4 ts = 4 et:
Ver = "0.1"
Help ()
{
Cat <rotatefile -- rotate the file name

Usage: rotatefile [-H] filename

Options:-H help text

Example: rotatefile out
This will e. g rename out.2 to out.3, out.1 to out.2, out to out.1
And create an empty out-File

The Max number is 10

Version $ ver
Help
Exit 0
}

Error ()
{
Echo "$1"
Exit 1
}
While [-n "$1"]; do
Case $1 in
-H) Help; shift 1 ;;
--) Break ;;
-*) Echo "error: no such option $1.-H for help"; Exit 1 ;;
*) Break ;;
Esac
Done

# Input check:
If [-z "$1"]; then
Error "error: You must specify a file, use-H for help"
Fi
Filen = "$1"
# Rename any. 1,. 2 etc file:
For N in 9 8 7 6 5 4 3 2 1; do
If [-F "$ filen. $ N"]; then
P = 'expr $ n + 1'
Echo "Mv $ filen. $ N $ filen. $ P"
MV $ filen. $ N $ filen. $ P
Fi
Done
# Rename the original file:
If [-F "$ filen"]; then
Echo "Mv $ filen $ filen.1"
MV $ filen $ filen.1
Fi
Echo touch $ filen
Touch $ filen

How does this script work? After the user provides a file name, we perform a 9-1 loop. File 9 is named 10, file 8 is renamed to 9, and so on. After the loop is completed, we name the original file as file 1 and create an empty file with the same name as the original file.

Debugging

The simplest debugging command is the echo command. You can use echo to print any variable value in any suspected error. This is why most shell Programmers spend 80% of their time debugging programs. The benefit of a shell program is that it does not need to be re-compiled, and it does not take much time to insert an echo command.

Shell also has a real debugging mode. If an error occurs in the script "strangescript", you can debug it as follows:

Sh-x strangescript

This will execute the script and display the values of all variables.

Shell also has a mode that only checks the syntax without executing the script. It can be used as follows:

Sh-N your_script

This will return all syntax errors.