Linux application Environment Practical 10:bash aesthetics and Philosophy in script programming language

Read Catalogue

• First, everything is a string
• Two, references and metacharacters
• Three, where does the string come from, where to go
• Four, plus a little bit of definition, you can push to export the entire Bash scripting language syntax
• V. Input/Output redirection
• Vi. The aesthetics of the Bash scripting language: Boulevard to Jane
• Summarize:

I confess that I once again became the title party. But admittedly, this must be an essential essay. In this article, I'll explore the aesthetics and philosophy of the Bash scripting language. This is not a tutorial for bash scripting, but it gives you a more in-depth look at Bash scripting programming and faster learning of bash scripting. Reading this essay does not require you to have experience with bash programming, but be as passionate about exploring the nature of programming languages as I do, and feeling their charms.

I wanted to write something about bash long ago. We usually like to classify programming languages, such as process-oriented programming languages, object-oriented programming languages, functional programming languages, and so on. In my mind, I think bash is a string-oriented programming language. The nature of the Bash scripting language: Everything is a string. All the philosophies of the Bash scripting language revolve around strings: Where do they come from? Where to? What is the mission? All the aesthetics of the Bash scripting language originate from strings: almost all the symbols on the keyboard "$ ~!" # & () [] {} | > <-. , ; * @ ' ' \ ^ ' is a combination of very visually appealing, extremely complex strings.

Back to top one, everything is a string

Bash is a Shell,shell the original intention is to glue the various tools in the system together, so its most fundamental function is to invoke various commands. But Bash also offers rich programming features. We often classify programming languages, such as process-oriented languages, object-oriented languages, function-oriented languages, and so on. You can think of the Bash scripting language as a string-oriented language. The essence of the Bash language is that everything is a string. Look at the variables in these:

Is some of the demos I did in the Interactive Bash command line. In, I assign a value to the variable, whether the right side of the equal sign is a string without quotation marks, a string with quotation marks, or even a number, or a mathematical expression, the final result is that the variables are stored in strings. I use a For loop to show all the variables, and I can see that the mathematical expression is stored as a string and is not evaluated.

Back to top two, references and metacharacters

If everything is a trivial string with no special features, it cannot form a programming language. In bash, there are a number of symbols with special meanings, such as the "$" symbol being used for string expansion, and the "&" symbol for the command to execute in the background, "|" Use as a pipe, > < for input and output redirection, and so on. So in bash, although it's also a string, the quoted string is not the same as the string enclosed by quotation marks, and the string enclosed in quotation marks is not the same as the string surrounded by double quotes.

What is the difference between a quoted string and a string without a quoted number? Are some of the more typical examples I've built:

In, I showed 7 ways to generate a string in bash: Curly brace expansion, tilde expansion, parameter expansion, command substitution, arithmetic expansion, word segmentation, and file path expansion. There are also two ways to generate strings that are not spoken (Process substitution and History commands are expanded). When you're programming with bash scripting, it's enough to understand how the 7 types of strings are generated. When you use the Bash command line interactively, you also need to know the history command to expand, skilled use of the history of command deployment can make people more effective.

As you can see in the above picture, there are some unwinding methods that are not working in a string surrounded by double quotes, such as curly brace expansion, tilde expansion, word segmentation, and file path expansion, but only parameter expansion, command substitution, and arithmetic expansion work. As you can see from the picture, parameter expansion, command substitution, and arithmetic expansion in a string are guided by the "$" symbol, and command substitution can also be guided by "'". Therefore, it can be further summed up that in a string surrounded by double quotes, only the "$, ', \" Three characters have special meanings.

If you want any one character to have no special meaning, enclose the string in single quotation marks. For example, when using regular expressions, such as when using tools such as SED or awk, the commands of SED and awk tend to be surrounded by single quotes because they often contain many special characters. For example, using the awk command to display the user name and full name of each user in the/etc/passwd file, you can use this command awk-e '{print $1,$5}', where the command passed to Awk is surrounded by single quotes, Indicates that bash does not perform any substitution or expansion.

Another special character is "\", which is also a reference. It can remove the special meaning (reference) of a special character immediately following it. The need for "\" exists because in bash, some characters are called metacharacters, and once they occur, a string is split into multiple substrings. If you need to include these metacharacters in a string, you must refer to them. Such as:

The most common meta characters are spaces. As can be seen from the above pictures, if you want to assign a string containing a space to a variable, either enclose the string in double quotation marks, or use "\" to reference the space. As you can see, there are only 9 metacharacters in bash, each of which is "| & (); < > Space tab ", while in other programming languages often appear metacharacters". {} [] "and subtraction as a mathematical operation, are not metacharacters in bash.

Back to the top three, where the strings come from, where to go

After describing the string, describing the references and metacharacters, the next goal is to explore the philosophical question: Where does the string come from and where to go? Through the discussion of this philosophical question, we can deduce the whole syntax of the Bash scripting language. Where does the string come from? Obviously, one of the very direct sources is that we knocked on the keyboard. In addition, the 789 kinds of string expansion methods I mentioned earlier.

The process of string expansion is as follows:

1. First use the meta-character to divide a string into substrings;

2. If the string is used to assign a value to a variable, it is considered to be surrounded by double quotes, regardless of whether it is surrounded by double quotes;

3. If the string is not enclosed in quotation marks and double quotes, the curly brace expands, and the {a,b}c expands to AB AC;

The above three processes can be proven by:

4. If the string is not enclosed in quotation marks or double quotes, the tilde is expanded, the ~/is expanded to the user's home directory, ~+/is expanded to the current working directory (PWD), ~-/is expanded to the previous working directory (OLDPWD);

5. If the string is not enclosed in quotation marks, the arguments and variables are expanded; This type of expansion begins with "$", which is the most complex of the entire bash string expansion, including user-defined variables, including all environment variables, both of which are "$" followed by variable names, and also include positional variables "$ 1, ..., $9, ... ", other special variables:" [email protected], $*, $#, $-, $!, $ A, $, $_, and even arrays: "${var[i]}", you can also perform various complex operations on strings during the unfolding process, such as: "${parameter:-wo RD}, ${parameter:=word}, ${parameter:+word},; ${parameter:?word}, ${parameter:offset}, ${parameter:offset:length}, $ {!prefix*}, ${[email protected]}, ${name[@]}, ${!name[*]}, ${#parameter}, ${parameter#word}, ${parameter# #word}, ${ Parameter%word}, ${parameter%%word}, ${parameter/pattern/string}, ${parameter^pattern}, ${parameter^^pattern}, ${ Parameter,pattern}, ${parameter,,pattern} ";

6. If the string is not enclosed in quotation marks, the command substitution is made, and the command substitution has two formats, one is $ (...), the other is ' ... '; that is, the output of the command as the content of the string;

7. If the string is not enclosed in quotation marks, the arithmetic expansion is performed, and the format of the arithmetic expansion is $ (...). ；

8. If the string is not enclosed in quotation marks or double quotes, the word is split;

9. If the string is not enclosed in quotation marks or double quotation marks, the file path is expanded;

10. When the above process is complete, the quotation marks (if any) outside the string are finally removed. The above process is done only once in the order above. For example, you do not expand the curly brace after the variable is expanded, and do not perform any of the preceding steps after the 10th step of removing the reference. If you need to go through the process again, please use Eval.

After discussing where the string came from, let's see where the string goes. That is how to use these strings. There are several ways to use the string:

1. Take it as a command execution; This is the most fundamental use of bash, after all, the shell exists to glue the various commands. If a string appears where the command should appear (such as at the beginning of a line, or after the keyword then, do, etc.), it will be executed as a command, and if it is not a valid command, it will be an error;

2. Take it as an expression; there is no expression in bash, but there is ((...)) and [[...]], there is an expression; ((...)) You can think of the string in it as an arithmetic expression, and [[...]] It will take the string inside it as a logical expression, only these two special cases;

3. Assigning a value to a variable; this is also a special case that undermines the integrity of the syntax philosophy of the Bash programming language. Why do you say that? Because "=" is not a meta-character, there are no spaces on either side, and only the 1th equals sign is treated as an assignment operator.

The following image gives evidence for the above view:

Back to the top four, plus a little bit of definition, you can push the syntax of the entire Bash scripting language.

I've already shown my understanding of where the strings come from and where to go. As for the whereabouts of the string, except for two expressions and a three special case for assigning a variable, the rest is only when the command is executed. In the front, I mentioned the concept of meta-characters and references, and here we have to add a little bit more definition:

The

Definition 1: Control Operator refers to metacharacters in order to divide a string into substrings, and the control operator is to split a series of strings into multiple commands. For example, in Bash, a string cat/etc/ PASSWD is a command, the first word cat is the command, the 2nd word/etc/passwd is the argument of the command, and the string < Span style= "color: #0000ff;" >cat/etc/PASSWD | grep Youxia is two commands, each of which is cat and grep, between them through the "|" Split, so here's the "|" is the control operator. A friend familiar with the shell must Know "|" Represents a pipe, so its function is 1. Divide a string into two commands, 2. The output of the first command as input to the second command. In bash, there are only 10 control operators, each of which are "| | & && |;;; () |& <newline> ". As long as you see these control operators, you can think of the string in front of it as a complete command.

Definition 2: Keywords (Reserved Words) I did not translate them into reserved words, and it is clear that, as a programming language, they should be called keywords. A programming language must provide a Process control statement, such as selection, looping, and the ability to define functions. These features can only be implemented with keywords. In bash, there are only 22 keywords, and they are "! Case Coproc Do-elif else Esac fi for function if in select and until while {} time [[]] ". There are many special things in this, such as "! Symbols such as {} [[]] are keywords, meaning that they are used as a word when using keywords, meaning that they and other words must be separated by metacharacters (otherwise they cannot be independent words). That's why it's used in bash "! {} [[]] "is often the reason for leaving spaces around them. (Again proving "=" is a very perverted exception, because it is neither a meta-character nor a control operator, more than a keyword, what exactly is it?) ）

The syntax for the Bash scripting language is deduced below:

The

Derivation 1: Simple command is a simple command that can be a string ending with the above control operator. For example, a single line of uname-r command (a single line of commands actually ends with <newline>, <newline> is the control operator), or although not placed on a single line, but with ";" or "&" end, such as UNAME-R; who; PWD; gvim& each command is a simple command (of course, this line of four commands is not called a simple command), ";" is to simply split the command, and "&" has the ability to make the command perform in the background. The Special one here is the double semicolon ";;", which is used only in case statements.

Derivation 2: Pipe line pipe is the essence of the shell, that is, the output of the previous command to become the input of the latter command. The complete syntax for the pipeline is this [time[-P]] [!] command1 | Command2 or This [time [-p]] [!] Command1 |& Command2. Where the time keyword and! Keywords are optional (use [...] Indicates which parts are optional, and the time keyword calculates when the command runs, and the keyword is to reverse the return state of the command. Look clearly! The spaces around the keyword OH. If you use "|", the standard output of the first command is used as the standard input for the second command, and if you use "|&", the standard output of the first command and the standard error output are treated as inputs to the second command.

Derivation 3: Command sequence (List) if multiple simple commands or multiple pipes are put together, between them with "; & <newline> | | && "Control operators separate, called a sequence of commands. About "; & <newline> "It's been said before, no need to repeat. About "| |" and "&&", familiar with C, C + +, Java and other programming languages are certainly not unfamiliar friends, they follow the same short-circuit evaluation of the idea. For example Command1 | | Command2 executes COMMAND2 only when Command1 execution is unsuccessful, Command1 && Command2 Command2 is executed only when Command1 executes successfully.

Derivation 4: Compound command (Compound Commands) If you combine the preceding simple commands, pipelines, or command sequences in a more complex way, you can form a composite command. In bash, there are 4 forms of compound commands, each of which is (list), {list;}, (expression), [[expression]]. Note that the 2nd form and the 4th form braces and the spaces around the brackets, also note the ";" in the 2nd form of the list, but if "}" is another line, then ";" is not required because <newline> and ";" is the same effect. In the 4 composite commands above, the (list) executes a sequence of commands in a new shell that does not affect the environment variables of the current shell, and {list;} simply groups the command sequence. The following two expressions have been evaluated before, and this is not the case. Options for evaluating logical expressions may be listed later in detail.

The above 4-step deduction is a step further, from simple to complex, the simplest commands can be combined into a slightly more complex sequence of commands, combined into more complex command sequences, and finally compose the most complex compound commands.

Here is the Process Control statement for the Bash scripting language, as follows:

1. For name [[in [Word ...]]; Do list; Done ;

2. For (expr1; expr2; expr3); Do list; Done ;

3. select name [in Word]; Do list; Done ;

4. Case Word in [[(] pattern [| pattern] ...) list;;] ..... Esac;

5. if list; then list; [ elif list; then list;] ... [ else list;] fi;

6. while list-1; Do list-2; Done ;

7. until list-1; Do list-2; Done .

The above formula Everyone can understand, I believe you can certainly understand. One of the [...] Represents a part that can or may not be true. In the above formula, notice the double brackets in the 2nd formula for Loop, which performs the arithmetic operation of the expression within it, which is most similar to the for loop for other high-level languages, but unfortunately, the arithmetic expression in bash can only calculate integers at this time. Again, notice the 3rd formula, select Syntax, and for...in ... The syntax of the loop is similar, but it can display a menu on the screen. If I remember correctly, this function should be available in the basic language. Other control structures are common in other high-level languages, and there is no need for me to be verbose here.

Finally, let's show you how to define a function:

　　　　 Name () Compound-command [redirection]

Or

　　　　 function name [()] compound-command [redirection]

As you can see, if there is a function keyword, then "()" is optional and "()" is required if there is no function keyword. What needs to be pointed out here is that the function body is only required to be compound-command, and I have previously summed up that there are four forms of compound-command, so sometimes defining a function does not appear "{}". For example, such a function is also valid:

That's all. This is the full syntax for the Bash scripting language. It's so simple.

Seems to have forgotten something? Yes, there are also input and output redirects not speaking. Input and output redirection is another great invention in the shell, its existence has its unique philosophical significance. See the next section for this.

Back to top v. Input and output redirection

The Unix world has a great philosophy: everything is a document. (This is a bit far.) The Unix world also has a great philosophy: the creation process is more convenient. (This is a bit far too.) And, each process is created, it automatically opens three files, which are standard input, standard output, standard error output, and normally, they are connected to the user's console. In the shell, a number is used to identify an open file, called a file descriptor, and the numbers 0, 1, and 2 represent standard input, standard output, and standard error output, respectively. In the shell, you can redirect the input and output of the command through >, <. Combined with the EXEC command, you can easily open and close files. It is important to note that when the file descriptor appears on the right side of ">", "<", the "&" symbol is used before, which may be to distinguish between greater than and less than in a mathematical expression. Use "&-" to close the file descriptor.

> < & Digital EXEC-This is all of the input and output redirects. In the following formula, I use N to represent a number, and if it is two different numbers, use N1, N2, using [...] Represents an optional parameter. The syntax for the input and output redirection is as follows:

1 [n]>File #重定向标准输出 (or n) to file. 2 [n]>> file # Redirects the standard output (or n) to file, appended to the end of file. 3 [n]< file # redirect file to standard input (or n). 4 [N1]>&n2 #重定向标准输出 (or N1) to N2. 5 2> file >&< Span style= "color: #800080;" >2 #重定向标准输出和错误输出到file. 6 |  command #将标准输出通过管道传递给command. 7 2>&1 | command #将标准输出和错误输出一起通过管道传递给command, equivalent to |&.  

Note that there is no space between the number and the >, < symbol. Combined with Exec, it is very convenient to use a file descriptor to open and close the file as follows:

1echo Hello >File12 exec3<file14>File2 #打开文件3Cat <&3 >&4 #重定向标准输入到3, standard output to4, which is equivalent to reading the contents of file1 and then writing File24 exec3<&-4>&-#关闭文件5CatFile2 #显示结果为 hello 7  8  #还可以暂存和恢复文件描述符, as follows:  9 exec 5>&2 # Save the original standard error output to the file descriptor 5 10 exec 2>/tmp/$0.log #重定向标准错误输出 11 12 exec 2>&5< Span style= "color: #000000;" > #恢复标准错误输出 13 exec 5>&-#关闭文件描述符5, because you do not need the   

You can also use "<>" to open a file for reading and writing.

In addition to exec, the input and output redirection and the Read command mates are also good, and the read command reads one line of the file at a time. It is important to note, however, that the input-output redirection is not the same as the loop body and loop of the for and while loops. Such as:

In addition, the input and output redirection symbols >, < can also be used with the () to represent process substitution, such as > (list), < (list). Combining the meaning of "<", ">" and "(list)" mentioned earlier, the role of process substitution is easy to guess oh.

Back to the top six, the aesthetics of the Bash scripting language: Boulevard to Jane

If you ask me, where is the bash scripting language? I will answer: simplicity is beauty. Please see the following article by clause:

1. Use concise abstract symbols. The Bash scripting language uses almost all the symbols that can be found on the keyboard, such as "$" as a string expansion, "|" Used as a pipe, "<", ">" as input and output redirection, not wasted at all;

2. A complete, string-oriented language is constructed using only 9 metacharacters, 10 control operators, and 22 keywords;

3. Conceptually good consistency; for example, the function of the (list) compound command is to execute a sequence of commands within parentheses, while "$" is used to boot the string expansion, so $ (list) is used for command substitution (so I said "$ ()" In the form of a command replacement than "" " The form of command substitution is more consistent). For example ((expresion)) is used for mathematical expression evaluation, so $ (expression) represents an arithmetic expansion. For example, "{}" and "," are used together, and there are no spaces in the middle, which represents curly braces, but when you need to use "{}" to define a compound command, you must put "{}" as a keyword, they and its contents must be separated by a space, and "}" and a command before it must have a " or "<newline>". These conceptual consistency is designed to be very subtle, and it is natural to use it to make people feel a sense of beauty;

4. Perfectly solves the separation of output and running state when a command is executed. People with other programming language experience often encounter the problem: when we call a function, the function may produce two results, one is the return value of the function, and the other is the success of the function call. In high-level languages such as C # and Java, it is often used to catch exceptions such as Try...catch to determine the success of a function call, but there are still programmers who let the function return NULL for failure, while the C language has no exception mechanism. It is difficult to determine exactly how a function's return value indicates the success of the function call (for example, there are many APIs that allow the function to return 1 for failure, and some function fails to set the errno global variable). In bash, the state of the command run and the standard output of the command are clearly distinguishable, if you need the standard output of the command, use command substitution to generate the string, if you only need the command's running state, write the command directly in the IF statement, or use the $? Special variable to check the running state of the previous command. If you do not want the standard output of the command to affect the user when checking the status of the command, you can redirect it to/dev/null, such as

cat/etc/grep youxia >/dev/'youxia is existfi   

5. Using piping and input-output redirection makes it easy to read and write files. Think about how to read the file in C, in addition to the trouble of open, close does not say, each read a string must be prepared a buffer, ready to waste space, prepared for a short fear of buffer overflow, abuse heart. With bash, that's really handy.

6. It's a no-no dynamic language. Oh, the eval command is too powerful, see, the analog pointer to look up the table:

Of course, since Bash 3, bash itself provides the ability to refer indirectly (using "${!var}").

　　Exception:

The bash language is not perfect in all respects, and there are a few special exceptions. Like the "=" mentioned earlier. In addition to "=", "()" There is also a use of inconsistencies, that is, the initialization of the array, such as array= (a b c D E F), which and the preceding "()" for the execution of the command sequence in the child shell is really inconsistent.

Back to the top of the summary:

The above content is my nonsense, because the above content can not teach you the full bash grammar, and can not teach you to use bash to do any meaningful work.

If you want to do something with bash, I'll send you a copy of O ' Reilly's Shell Scripting Learning Guide: A Shell Scripting Learning Guide. Part1.rar Shell Scripting Learning Guide. Part2.rar Shell Scripting Learning Guide. Part3.rar Shell Scripting Learning Guide. Part4.rar (why only upload 10M at a time Ah, a good book has been split into 4 parts.) ）

In addition, my main reference is from the Bash User manual. You can run man bash in your own system .

Http://www.cnblogs.com/youxia/p/linux010.html

Linux application Environment Practical 10:bash aesthetics and Philosophy in scripting languages (turn)