Linux Shell Programming

Shell can be used to automate a large number of tasks. Shell is especially good at system management tasks, especially for tasks that are easier to use, maintainability, and portability than efficiency.

Next, let's take a look at how shell works:

Create a script

Linux has many different shells, but we usually use Bash (Bourne again shell) for shell programming, because Bash is free and easy to use. So in this article, all the scripts provided by the author use Bash (but in most cases, these scripts can also be run in Bash's big sister, Bourne shell ).

Like other languages, we can use any text editor, such as nedit, kedit, emacs, and VI to compile our shell.Program. The program must start with the following line (must begin with the first line of the file ):

#! /Bin/sh

Symbol #! The parameter used to tell the system that the program is used to execute the file. In this example, we use/bin/sh to execute the program. When editing a script, you must make it executable if you want to execute it.

To make the script executable:

Chmod + x filename

Then, you can run your script by entering:./filename.

Note

During shell programming, a sentence starting with # represents a comment until the end of this line. We sincerely recommend that you use annotations in your program. If you have used annotations, you can understand the role and working principle of the script in a short time even if the script is not used for a long time.

Variable

In otherProgramming LanguageYou must use the variable. In shell programming, all variables are composed of strings, and you do not need to declare variables. To assign a value to a variable, you can write as follows:

Variable name = Value

You can add a dollar sign ($) before the variable to retrieve the variable value:

#! /Bin/sh

# Assign values to variables:

A = "Hello World"

# Print the content of variable:

Echo "A is :"
Echo $

Enter the preceding content in your editor and save it as a file first. Then run chmod + X first. Make it executable, and enter./first to execute the script.

This script will output:

A is:
Hello World

Sometimes the variable name is easily confused with other words, such:

Num = 2
Echo "this is the $ numnd"

This does not print "this is the 2nd", but only prints "this is the", because shell will search for the value of the variable numnd, but there is no value for this variable. We can use curly braces to tell shell that we want to print the num variable:

Num = 2
Echo "this is the $ nd"

This will print: This is the 2nd

Many variables are automatically set by the system, which will be discussed later when these variables are used.

If you need to handleMathematicsExpression, you need to use programs such as expr (see below ). In addition to the shell variables that are generally valid in the program, there are also environment variables. Variables processed by the Export keyword are called environment variables. We will not discuss environment variables, because generally, only environment variables are used in the login script.

Shell commands and Process Control

Three types of commands can be used in shell scripts:

1) Unix command:

Although any Unix command can be used in shell scripts, some more common commands are used. These commands are usually used for file and text operations.

Common commandsSyntaxAnd functions:

Echo "some text": print the text on the screen.

Ls: file list.

WC-l file WC-W file WC-C file: calculate the number of file lines. Calculate the number of words in the file. Calculate the number of characters in the file.

CP sourcefile destfile: copy a file.

MV oldname newname: rename a file or move a file.

RM file: delete an object.

Grep 'pattern' file: searches for strings in a file, for example, grep 'searchstring' file.txt.

Cut-B colnum file: specify the content range of the file to be displayed, and output them to the standard output device, for example: output 5th to 9th characters in each line cut-B 5-9 file.txt. Do not confuse it with the cat command, which is two completely different commands.

Cat file.txt: output file content to the standard output device (screen.

File somefile: Obtain the file type.

Read var: prompts the user to enter the input and assigns the input value to the variable.

Sort file.txt: sorts the rows in the file.txt file.

Uniq: Delete the columns in a text file, for example, sort file.txt | uniq.

Expr: perform mathematical operations example: Add 2 and 3 expr 2 "+" 3.

Find: search for a file. For example, search for find.-Name filename-print based on the file name.

Tee: outputs data to the standard output device (screen) and files such as: somecommand | tee OUTFILE.

Basename file: returns a file name that does not contain a path. For example, basename/bin/tux returns tux.

Dirname file: the path of the returned file, for example, dirname/bin/tux, returns/bin.

Head file: prints the first few lines of a text file.

Tail file: number of rows at the end of the text file.

Sed: SED is a basic search replacement program. You can read text from standard input (such as command pipeline) and output the results to standard output (screen ). This command uses a regular expression (see references) for search. Do not confuse with wildcards in shell. For example, replace linuxfocus with linuxfocus: CAT text. File | SED's/linuxfocus/'> newtext. File.

Awk: awk is used to extract fields from text files. By default, the field delimiter is a space. You can use-F to specify other separators. Cat file.txt | awk-F, '{print ","}', which is used here as a field delimiter and prints the first and third fields at the same time. If the file content is as follows:Adam Bor, 34, indiakerry Miller, 22, USA

Command output result:

Adam Bor, indiakerry Miller.

2) concept: pipelines, redirection, and backtick

These are not system commands, but they are really important.

The pipeline (|) uses the output of a command as the input of another command.

Grep "hello" file.txt | WC-l

Search for a row containing "hello" in file.txt and calculate the number of rows. Here, the grep command output serves as the WC command input. Of course, you can use multiple commands.

Redirection: output the command result to a file instead of a standard output (screen ).

> Write the file and overwrite the old file.

> Add it to the end of the file to retain the content of the old file.

Use a backslash to output a command as a command line parameter of another command.

Command:

Find.-mtime-1-type F-print

Used to search for files modified in the past 24 hours (-mtime-2 indicates the past 48 hours. If you want to pack all the searched files, you can use the following script:

#! /Bin/sh
# The ticks are backticks (') not normal quotes ('):
Tar-zcvf lastmod.tar.gz 'Find.-mtime-1-type F-print'

3) Process Control

If the expression "if" is true, the part after then is executed:

If...; then

________________________________________________________________________________________________________

Shell basic syntax

Like an advanced programming language, shell also provides instructions and the ability to use variables. For shell, the values of all variables are a string. The shell program uses $ VaR to reference the value of a variable named var.

Shell has the following basic types of variables.

(1) environment variables defined by shell:

At the beginning of execution, shell has defined some variables related to the system's working environment. You can also redefine these variables. Common shell environment variables include:

Home is used to save the full path name of the registration directory.

PATH is used to save the directory path names separated by colons. Shell searches these directories in the order given in the PATH variable, and the first executable file with the same name as the command will be executed.

The type of the term terminal.

The identifier of the current user. The value is a string consisting of digits.

The absolute path name of the current working directory of PWD. The value of this variable varies with the use of the CD command.

PS1 primary prompt. for privileged users, the default primary prompt is #. For common users, the default primary prompt is $.

When the PS2 shell receives the user's input command, if the user enters "\" at the end of the input line, and then press enter, or when the shell determines that the command entered by the user is not over when the user presses the Enter key, the auxiliary prompt is displayed, prompting the user to continue entering the rest of the command. The default auxiliary prompt is>.

(2) User-Defined variables:

You can define your own variables according to the following syntax rules:

Variable name = variable value

Note that $ should not be added before the variable name when defining the variable, and $ should be added before the variable name When referencing the variable content; when assigning values to the variable, no space must be left on both sides of the equal sign. If the variable itself contains spaces, the entire string should be enclosed in double quotation marks.

When writing a shell program, we recommend that you use uppercase letters to indicate all variable names to distinguish them from command names.

Sometimes, when we want to describe a variable and set it to a specific value without changing its value, we can use the following command to ensure the read-only of a variable:

Readonly variable name

At any time, the created variables are only local variables of the Current Shell, so they cannot be used by other commands or Shell programs run by the shell, the Export command can provide a local variable to other commands executed by shell in the format:

Export variable name

You can also use the Export command when assigning values to variables:

Export variable name = variable value

Variables described in export can be accessed in all commands or programs run after shell.

(3) location parameters:

A location parameter is a variable that is determined by its location in the command line that calls the shell program. It is a parameter entered after the program name. Location parameters are separated by spaces. Shell replaces $1 with the first location parameter in the program file, and the second with $2, and so on. $0 is a special variable whose content is the file name of the current shell program. Therefore, $0 is not a location parameter, $0 is not included when all the current location parameters are displayed.

(4) predefined variables:

Predefine variables are similar to environment variables and are also defined at the beginning of shell. The difference is that you can only use these variables according to shell definitions, rather than redefinition. All predefined variables are composed of the $ operator and another symbol. Common shell predefined variables include:

$ # Number of location parameters.

$ * Content of all location parameters.

$? Status returned after the command is executed.

$ Process ID of the current process.

$! The last process number that runs in the background.

$0 indicates the name of the currently executed process.

Where, $? Used to check whether the previous command is correctly executed. (In Linux, if the exit status of a command is 0, the command is correctly executed. If the value is not 0, an error occurs .)

$ The most common purpose of a variable is to use the name of the temporary file to ensure that the temporary file will not be repeated.

(5) variable for parameter replacement:

Shell provides the parameter replacement function so that you can assign different values to variables based on different conditions. There are four types of parameters for parameter substitution. These variables are usually associated with a location parameter and determine the value of the variable based on whether the specified location parameter has been set to a class, their syntax and functions are as follows.

A. Variable =$ {parameter-word}: If a parameter is set, replace the value of the parameter with the value of the parameter. Otherwise, replace the value with word. That is, the value of this variable is equal to the value of a parameter. If this parameter is not set, the variable is equal to the value of word.

B. variable =$ {parameter = word}: If a parameter is set, replace the value with the value of the parameter. Otherwise, set the variable to word and replace the value with word. Note that location parameters cannot be used in this way, because location parameters cannot be assigned values in Shell programs.

C. Variable =$ {parameter? Word}: If a parameter is set, replace the value of the variable with the value of the parameter. Otherwise, word is displayed and exited from shell. If word is omitted, standard information is displayed. This variable must be equal to the value of a parameter. If this parameter is not set, a message is displayed and then exited. Therefore, this method is often used to indicate errors.

D. Variable =$ {parameter + word}: If the parameter is set, replace the variable with word. Otherwise, do not replace the variable.

In all the four forms, the "parameter" can be either a location parameter or another variable, but there are many situations where the location parameter is used.

Process Control for Shell Program Design

Like other advanced programming languages, shell provides commands used to control program execution processes, including conditional branches and loop structures. You can use these commands to create very complex programs.

Unlike traditional languages, shell is used to specify condition values instead of Boolean operations, but commands and strings.

1. Test command

The test command is used to check whether a condition is true. It can be used to test values, characters, and files. The test characters and corresponding functions are as follows.

(1) numerical test:

-If EQ is equal to, it is true.

-If the ne value is not equal to the ne value, it is true.

-A value greater than-GT is true.

-If Ge is greater than or equal to, it is true.

-If it is less than, it is true.

-If the value of-Le is less than or equal to, it is true.

(2) string test:

= Equals to true.

! = Not equal to true.

The-Z string length is false.

-If the length of the N string is not pseudo, it is true.

(3) file test:

-E: the file name is true if the file exists.

-R: The file name is true if the file exists and is readable.

-W: The file name is true if the file exists and can be written.

-X: the file name is true if the file exists and can be executed.

-S file name is true if the file exists and contains at least one character.

-D. The file name is true if the file exists and is a directory.

-F: The file name is true if the file exists and is a normal file.

-C: The file name is true if the file exists and is a special character file.

-B: The file name is true if the file exists and is a special file.

In addition, Linux also provides (!) , Or (-O), non-(-a) logical operators, used to connect test conditions, the priority is :! Highest, followed by-A and-o.

Bash can also perform simple arithmetic operations in the following format:

$ [Expression]

For example:

Var1 = 2

Var2 = $[var1*10 + 1]

Then the value of var2 is 21.

2. If Condition Statement

The condition branch in the shell program is implemented through the IF Condition Statement. The general format is:

If condition command string

Then

Command string when the condition is true

Else

Command string when the condition is false

Fi

3. For Loop

The for loop executes a command sequence for all possible values of a variable. The values assigned to the variable can be provided in the form of a numerical list in the program, or in the form of a positional parameter outside the program. The general format of the For Loop is:

For variable name [in Value List]

Do

Several command lines

Done

The variable name can be any string selected by the user. If the variable name is Var, the value given after in replaces $ VaR in the circular command list in sequence. If in is omitted, the value of VaR is the location parameter. Every possible value assignment to a variable will execute the command list between do and done.

4. While and until Loops

Both the while and until commands use the return status value of the command to control the loop. The general format of the while loop is:

While

Several command lines 1

Do

Several command lines 2

Done

As long as the return status of the last command in "Several command line 1" in while is true, the while loop continues to execute "Several command line 2" between do... done ".

The until command is another loop structure, which is similar to the while command in the following format:

Until

Several command lines 1

Do

Several command lines 2

Done

The difference between an until loop and a while loop is that a while loop continues to run when the condition is true, while an until loop continues to run when the condition is false.

Shell also provides the "true" and "false" commands to create an infinite loop structure. Their return states are always 0 or not 0.

5. Select case conditions

The IF Condition Statement is used to select one of the two options, and the case condition selection provides you with a method to select one from multiple options based on the value of the string or variable. The format is as follows:

Case string in

Exp-1)

Several command lines 1

;;

Exp-2)

Several command lines 2

;;

......

*)

Other command lines

Esac

By calculating the string value, shell compares the results in sequence with the formula exp-1, exp-2, etc. until a matching formula is found. If a match is found, run the following command until a semicolon (;) is encountered.

You can also use shell wildcards ("*", "?" , "[]"). * Is usually used as the final formula of the Case command to execute the "other command line" command if no matching item is found.

6. Unconditional control statements break and continue

Break is used to terminate the execution of the current loop immediately, while contiune is used to start the execution of the next loop immediately without executing the statements following the loop. These two statements are valid only when they are placed between do and done.

7. Function Definition

You can also define functions in shell. A function is actually composed of several shell commands, so it is similar to a shell program in form. The difference is that it is not a separate process, but a part of the shell program. The basic format of the function definition is:

Functionname

{

Several command lines

}

The format of the called function is:

Functionname param1 param2...

Shell functions can complete some routine work and have their own exit States. Therefore, functions can also be used as conditions for control structures such as if and while.

Parameters are not required for function definition, but can be included when a function is called. In this case, shell will assign these parameters to the corresponding location parameters $1, $2 ,... and $ *.

8. Command Group

There are two command grouping methods in shell: () and {}. The former creates a new sub-process when the shell executes the commands in (), and then the sub-process executes the commands in the circular arc. When a user does not want to change the status set (such as location parameters, environment variables, and current working directory) during command execution to affect the execution of the following statements, these commands should be placed in the circular arc to ensure that all changes only affect sub-processes, and the parent process is not disturbed. {} Is used to use the output results of the commands executed in sequence for the input (pipe mode) of another command ). When we really want to use circular arc and curly arc (such as computing formula priority), we need to add an escape character (\) before it (\) so that the shell knows that they are not used for command execution control.

9. Signal

The trap command is used to capture signals in Shell programs. There are three response methods:

(1) execute a program to process this signal.

(2) receive the default signal operation.

(3) Ignore this signal.

Trap provides three basic forms for the above three methods:

The first form of Trap Command will execute the command string in double quotation marks when shell receives the signal with the same value as the signal list.

Trap 'commands' signal-list

Trap "commands" signal-list

To restore the default signal, use the second form of trap command:

Trap signal-list

The third form of the trap command allows the ignore signal:

Trap "" signal-list

Note:

(1) The signal 11 (segment violation) cannot be captured, because the shell itself needs to capture the signal for memory dump.

(2) In the trap, you can define the processing of the signal 0 (in fact, there is no such signal). The shell program sends this signal when it terminates (such as executing the exit Statement.

(3) After capturing the signal specified in the signal-list and executing the corresponding commands, if these commands do not terminate the shell program, the shell program will continue to execute the command after the command received when the signal is received, which will easily cause the shell program to be unable to terminate.

In TRAP statements, single quotes and double quotation marks are different. When the shell program first encounters a trap statement, it will scan the commands in commands. If commands is enclosed in single quotes, shell will not replace the variables and commands in commands. Otherwise, the variables and commands in commands will be replaced with the specific values at that time.

How to run a shell program

You can use any editing program to write Shell programs. Because the shell program is interpreted and executed, it does not need to be compiled into the target program. According to the shell programming Convention, for example, the first line of the program is generally "#! /Bin/bash ", where # indicates that this row is a comment and an exclamation point! Tell the shell to run the command after the exclamation mark and use the rest of the document as the input, that is, run/bin/bash and let/bin/bash execute the shell program content.

There are three methods for executing Shell programs.

1. sh shell program file name

The command format for this method is:

BASH Shell program file name

This is actually to call a new bash command to explain the program, and pass the shell program file name as a parameter to it. The newly started shell reads the specified file and runs the commands listed in the executable file. When all the commands are executed, the execution ends. The advantage of this method is that the shell debugging function can be used.

2. Sh

Format:

Bash <shell program name

In this way, the shell command interpreter uses the specified program file to redirect input.

3. Run the CHMOD command to make the shell program executable.

Whether a file can run depends on whether the content of the file is executable and the file has the execution right. For Shell programs, when a file is generated using the editor, the system grants 644 (RW-r --) Permission. Therefore, when you need to run this file, you only need to directly type the file name.

Among the three methods for running Shell programs, it is best to choose the following method: when a shell program is just created and its correctness is not yet grasped, the first method should be used for debugging. When a shell program has been debugged, use the third method to fix it. Later, you only need to enter the corresponding file name and can be called by another program.

4. Bash program debugging

Errors are inevitable during programming. Sometimes, debugging takes more time than programming, as does shell.

Shell program debugging mainly uses bash commands to explain Program Selection items. Bash is called in the form:

Bash-select the shell program file name

Several common options are:

-E. Exit immediately if a command fails.

-N reads commands but does not execute them.

-During U replacement, unconfigured variables are considered as errors.

-V displays the shell input rows.

-X displays commands and their parameters when executing commands.

All the options above can also be referenced in the shell program in the form of "Set-select item", and "set + select item" will disable this option. If you only want to use some selection items for a part of the program, you can enclose this part with the preceding two statements.

(1) unset variable exit and exit now

The unset variable exit feature allows you to check all variables. If an unassigned variable is referenced, shell program execution is terminated. Shell usually allows the use of unspecified variables. In this case, the variable value is null. If you have set an unset variable to exit the selection item, once you have used the unset variable, the error message is displayed and the program is terminated. Unset variable exit select item-U.

When a shell is running, if a command does not exist or cannot be executed, the redirection fails, or the command ends abnormally, the error message is displayed on the terminal screen without redirection, the shell program continues to run. To force the shell program to end immediately when an error occurs, you can use the-e option to terminate the execution of the shell program immediately.

(2) SHELL Program tracking

The main method to debug a shell program is to use shell commands to explain the program's-V or-x options to track program execution. -V selects the item so that the shell will display every command line it reads during the execution of the program, the-x option enables shell to display every command it executes in the process of executing the program with a + + command name at the beginning of the line. And display each variable and the value of the variable. Therefore, the main difference between them is that-V is not available before the command line is executed, and the original contents of the command line are displayed, if-V exists, the replaced command line content is displayed.

In addition to shell-V and-x options, you can also take some auxiliary debugging measures within the shell program. For example, you can use the echo command to display the necessary information in some key aspects of the shell program. Its function is equivalent to the printf statement in the C language, in this way, you can know where the program runs and the current state of the program.

Bash Internal commands

The bash command explains that the package contains some internal commands. Internal commands are invisible in the directory list. They are provided by the shell itself. Common internal Commands include echo, Eval, exec, export, readonly, read, shift, wait, and point (.). The following describes the Command Format and functions.

1. Echo

Command Format: Echo ARG

Function: displays the string specified by Arg on the screen.

2. Eval

Command Format: eval ARGs

Function: When the shell program runs the eval statement, the shell reads The args parameter and combines them into a new command and then runs it.

3. Exec

Command Format: exec Command Parameters

Function: When the shell executes the exec statement, it does not create a new sub-process, but instead executes the specified command. When the specified command is executed, the process (that is, the initial shell) is terminated, so the statements after exec in the shell program will not be executed.

4. Export

Command Format: Export variable name or: Export variable name = variable value

Function: shell can use export to bring its variables down to the sub-shell, so that the sub-process inherits the environment variables in the parent process. However, the sub-shell cannot use export to bring its variables up to the parent shell.

Note: The export statement without any variable name will display all the current export variables.

5. readonly

Command Format: readonly variable name

Function: identifies a user-defined shell variable as unchangeable. The readonly command without any parameters will display all read-only shell variables.

6. Read

Command Format: Read variable table

Function: Read a row from a standard input device, break it into several words, and assign values to variables defined in the shell program.

7. Shift statement

Function: The Shift statement renames all location parameter variables as follows, that is, $2 becomes $1, $3 becomes $2... Every shift statement used in the program makes all the location parameters move one position to the left in sequence, and the location parameter $ # Minus 1 until it is reduced to 0.

8. Wait

Function: enables shell to wait for all sub-processes started in the background to end. The return value of wait is always true.

9. Exit

Function: exit the shell program. After exit, you can selectively specify a digit as the return status.

10. "." (point)

Command Format:. shell program file name

Function: enables shell to read the specified shell program file and execute all statements in the file in sequence.