The 9th chapter is the regular extension

Extended One//
14th Regular Expression http://www.apelearn.com/study_v2/chapter14.html
Grep
Syntax: grep [-CINVABC] ' word ' filename
-C: Print the number of lines that meet the requirements
-I: Ignore case
-N: Output with the same number of lines as required
-V: Print rows that do not meet the requirements
-A: followed by a number (with or without spaces), for example, –A2 to print the line that meets the requirements and the following two lines
-B: followed by a number, such as –B2, to print the line that meets the requirements and the above two lines
-C: followed by a number, such as –C2, to print the line that meets the requirements and two rows above and below

1. Filter out the line with a keyword and enter the travel number
Grep-n ' Root '/etc/passwd
2. Filter the line without a keyword, and lose the travel number
Grep-nv ' Nologin '/etc/passwd
3. Filter out all rows that contain numbers
grep ' [0-9] '/etc/inittab
4. Filter out all rows that do not contain numbers
Grep-v ' [0-9] '/etc/inittab
5. Remove all lines beginning with ' # '
Grep-v ' ^# '/etc/inittab
6. Remove all empty lines and lines beginning with ' # '
Grep-v ' ^# '/etc/crontab |grep-v ' ^$ '
In the regular expression, "^" means the beginning of the line, "$" means the end of the line, then the empty row can be represented by "^$"
7. Filter any character and repeat character
grep ' R.. O '/etc/passwd
8. Specify the number of times to filter the occurrences of a character
grep ' o{2} '/etc/passwd
Egrep
1. Filter one or more of the preceding characters
Egrep ' o+ ' test.txt
2. Filter 0 or one of the preceding characters
Egrep ' O? ' test.txt
3. Filter string 1 or String 2
Egrep ' aaa|111|ooo ' test.txt
Applications in 4.egrep ()
Egrep ' R (oo|at) o ' test.txt
Use () to denote a whole, for example (OO) + means 1 ' oo ' or multiple ' oo '
Egrep ' (oo) + ' test.txt
Use of SED tools
GREP implements only the lookup function, which does not replace the found content. The SED tool, along with the awk tool below, enables the ability to output the replaced text to the screen, along with other richer features. Both SED and awk are streaming editors that operate on the lines of the document.
1. Print a line
Sed-n ' n ' p filename The n in the single quotation mark is a number that represents the first few lines:
Sed-n ' 1,$ ' p filename prints out all the lines to use
2. Print a line that contains a string
Sed-n '/root/' P test.txt
3.-E can implement multiple behaviors
Sed-e ' 1 ' p-e '/111/' p-n test.txt
4. Delete a row or multiple rows
Sed ' 1,3 ' d test.txt
The ' d ' character is the delete action, not only can delete the specified single row and multiple lines, but also can delete the line matching a character, in addition to delete from a row until the end of the document line.
5. Replacing characters or strings
Sed ' 1,2s/ot/to/g ' test.txt
Sed ' s#ot#to#g ' test.txt
' s ' is the replacement command, ' G ' is the global replacement of the bank, if not added ' G ' only to the first occurrence of the line. There are no problems with other special characters such as ' # ' or ' @ ', except that you can use '/' as a delimiter.
6. Swap the position of two strings
Sed ' s/(Rot) (.) (bash)/\3\2\1/' test.txt
Use () to enclose the character that you want to replace as a whole, because the parentheses belong to the special symbol in the SED, so you need to precede the meaning character ' ', and replace it with the form ' 1 ', ' 2 ', ' 3 '. In addition to changing the position of two strings, it is often used to add the specified content before or after a row.
Sed ' s/^.$/123&/' Test.txt
7. Directly modify the contents of the file
Sed-i ' s/ot/to/g ' test.txt
This command can modify the file directly.

Exercise: Copy/etc/passwd to/root/test.txt, print all lines with SED
Print 3 to 10 lines of test.txt
Print the line containing ' root ' in test.txt
Delete Test.txt 15 rows and all subsequent rows
Delete rows containing ' bash ' in Test.txt
Replace ' root ' as ' Toor ' in Test.txt
Replace Test.txt '/sbin/nologin ' as '/bin/login '
Delete all the numbers in rows 5 through 10 in Test.txt
Delete all special characters in test.txt (except for numbers and uppercase and lowercase letters)
Swap the first and last words in the test.txt position
Replace the first number and the last word that appear in the test.txt position
Move the first number in the Test.txt to the end of a line
Test.txt 20 to the end of the line Plus ' AAA: '
Amin hope you can think as much as possible, the answer is only used as a reference.
SED exercise answers

2. /bin/cp/etc/passwd/root/test.txt; Sed-n ' 1,$ ' P test.txt
4. Sed-n ' 3,10 ' P test.txt
6. Sed-n '/root/' P test.txt
8. Sed ' 15,$ ' d test.txt
10. Sed '/bash/' d test.txt
12. Sed ' s/root/toor/g ' test.txt
14. Sed ' s#sbin/nologin#bin/login#g ' test.txt
16. Sed ' 5,10s/[0-9]//g ' test.txt
18. Sed ' s/[^0-9a-za-z]//g ' test.txt
20. Sed ' s/(^[a-za-z][a-za-z]) ([^a-za-z].) ([^a-za-z]) ([a-za-z][a-za-z]*$)/\4\2\3\1/' Test.txt
22. Sed ' s# ([^0-9][^0-9]) ([0-9][0-9]) ([^0-9].) ([^a-za-z]) ([a-za-z][a-za-z]$) #\1\5\3\4\2# ' Test.txt
24. Sed ' s# ([^0-9][^0-9]) ([0-9][0-9]) ([^0-9].*$) #\1\3\2# ' Test.txt
25. Sed '/^.*$/aaa:&/, $s ' test.txt
    Use of the awk tool
    Awk is more powerful than SED, it can do what sed can do, and it can do the same with SED. The awk tool is actually very complex.
    1. Intercepting a segment in a document
    HEAD-N2/ETC/PASSWD |awk-f ': ' {print $} '
    Head-n2 test.txt |awk-f ': ' {print $} '
    The function of the-f option is to specify a delimiter, or a space or tab as a separator if no-f is specified. Print is the printed action used to print out a field. $ $ for the first field, and $ for the second field, and so on, there is a special that is $ A, which represents the entire row.
    Note The format of awk, followed by the single quotation mark after-F, and then the delimiter inside, the print action to be enclosed in {}, otherwise it will be an error. Print can also print custom content, but the custom content is enclosed in double quotation marks.
    Head-n2 test.txt |awk-f ': ' {print $ "#" $ "#" $ $ "#" $4} '
    2. Match characters or strings
    awk '/oo/' test.txt
    Awk-f ': ' $ ~/oo/' test.txt
    Can make a segment to match, where the ' ~ ' is the meaning of the match
    Awk-f ': '/root/{print $1,$3}/test/{print $1,$3} '/etc/passwd
    You can also match multiple times, as in the previous example, after matching root, and then matching test, it can also print only the matching segments.
    3. Conditional operators
    Awk-f ': ' $3== ' 0 "'/etc/passwd
    Awk can be judged by logical notation, such as ' = = ' is equal to, can also be understood as ' exact match ' In addition, ' >=, ' <, ' <=, '! ', etc., it is noteworthy that, when compared with the number, if the comparison of the number in double quotation marks, Then awk does not think of the numbers, but the characters, which are considered numbers without double quotes.
    Awk treats all numbers as characters.
    Awk-f ': ' $3<$4 '/etc/passwd
    ! = is a mismatch and can be logically compared between two segments in addition to a logical comparison of the characters for a segment.
    Awk-f ': ' $3> ' 5 "&& $3<" 7 "'/etc/passwd
    Awk-f ': ' $3> ' 5 ' | | $7== "/bin/bash" '/etc/passwd
    In addition, you can use the && | | means "and" and "or".
    Built-in variables for 4.awk
    The variables commonly used by AWK are:
    NF: How many segments are separated by separators
    NR: Number of rows
    head-n3/etc/passwd | Awk-f ': ' {print NF} '
    head-n3/etc/passwd | Awk-f ': ' {print $NF} '
    NF is the number of segments, while the $NF is the last segment of the value, and NR is the line number.
    Mathematical operations in 5.awk
    Awk can change the value of a segment
    Head-n 3/etc/passwd |awk-f ': ' $1= ' root ';
    Of course, you can also calculate the sum of a segment
    Awk-f ': ' {(tot=tot+$3)}; END {print tot} '/etc/passwd

Practice:
Print the entire test.txt with awk (the following is done with the awk tool for Test.txt)
Find all rows that contain ' bash '
Use ': ' as a delimiter to find a line with a third paragraph equal to 0
Use ': ' as a delimiter to find the first line of ' root ' and replace the ' root ' of that segment with ' Toor ' (which can be used together with SED)
Use ': ' As a delimiter to print the last paragraph
Print all rows with a number of rows greater than 20
Use ': ' As a delimiter to print all third paragraphs less than the fourth paragraph
Use ': ' As a delimiter, print the first paragraph and the last paragraph, and the middle with ' @ ' connection (for example, the first line should be the form of ' [email Protected]/bin/bash ')
Use ': ' As a delimiter to add the fourth paragraph of the entire document, summing
awk Exercise Answers

1. awk ' {print $} ' test.txt
2. awk '/bash/' test.txt
3. Awk-f ': ' $3== ' 0 "' test.txt
4. Awk-f ': ' $1== ' root ' test.txt |sed ' s/root/toor/'
5. Awk-f ': ' {print $NF} ' test.txt
6. Awk-f ': ' nr>20 ' test.txt
7. Awk-f ': ' $3<$4 ' test.txt
8. Awk-f ': ' {print $ ' @ ' $NF} ' test.txt
9. Awk-f ': ' {(sum+=$4)}; END {print sum} ' test.txt

Extension Two//
(1) Print the contents of a line to a line http://ask.apelearn.com/question/559
Sed-n '/[abcfd]/,/[rty]/p ' test intercepts the contents of [ABCFD] to [Rty]
(2) SED conversion case http://ask.apelearn.com/question/7758
In sed, uppercase is denoted with \u, \l is lowercase

1. Capitalize the first lowercase letter of each word: sed ' s/\b[a-z]/\u&/g ' filename
2. Capitalize all lowercase: sed ' s/[a-z]/\u&/g ' filename
3. Uppercase to lowercase: sed ' s/[a-z]/\l&/g ' filename
   (3) SED adds a number at the end of a line http://ask.apelearn.com/question/288
   (4) Delete a line to the last line http://ask.apelearn.com/question/213
   Sed '/c/{p;:a; N;$!BA;D} ' Test
   Define a label A, match C, then n add the next line to the pattern space, match the last row, exit the label loop, and then command D to clear all the contents of this pattern space.
   If matches "C"
   : A
   Append Next line
   If does not match "$"
   Goto A
   Finally exits the loop and the D command is removed.
   (5) Print rows from 1 to 100 lines with a string http://ask.apelearn.com/question/1048
   SED specifies row range matching sed-n ' 1,100{/abc/p} ' 1.txt
   Extension Three//
   awk usage (Getting started) http://www.cnblogs.com/emanlee/p/3327576.html
   Extension Four//
   Using external shell variables in awk http://ask.apelearn.com/question/199

awk merges a file http://ask.apelearn.com/question/493
awk ' Nr==fnr{a[$1]=$2}nr>fnr{print $0,a[$1]} ' 1.txt 2.txt
Explanation: Merge the same rows in the first column of the two files into the same row.
NR represents the number of rows read, FNR represents the current number of rows read, you can run this command awk ' {print Nr,fnr} ' 1.txt 2.txt, compare NR and FNR, so actually NR==FNR is the time to read 1.txt. The same nr>fnr means when reading 2.txt
Array A is actually the equivalent of a map.
Concatenate a file multiple lines into a line http://ask.apelearn.com/question/266
A=cat file; Echo $a
awk ' {printf ("%s", + $)} ' file//%s ' Remember to have a space, otherwise come out is completely connected, there is no space in the middle
Cat file |xargs
The use of Gsub functions in awk http://ask.apelearn.com/question/200
awk ' Gsub (/www/, "abc") '/ETC/PASSWD//passwd file replace all www with ABC
Awk-f ': ' Gsub (/www/, "ABC", $ $) {print $} '/etc/passwd//replace ' www ' in ' abc '
awk intercepts specifying multiple fields as one row http://ask.apelearn.com/question/224
Specify a delimiter with awk to divide the text into segments. How do I get a line for the same piece of content?
In the case of/etc/passwd, the file is divided into 7 paragraphs with ":" As a delimiter.
For I inseq 1 7
Do
Awk-f ': '-v a= $i ' {printf $a ' "} '/etc/passwd
Echo
Done
Filter two or more keywords http://ask.apelearn.com/question/198
Grep-e ' 123|abc ' filename//Find file (filename) contains 123 or lines containing ABC
Egrep ' 123|abc ' filename//With EGREP can also be implemented
awk '/123|abc/' filename//How awk is implemented
Generate the following structure file with awk http://ask.apelearn.com/question/5494
awk Prints single quotation marks with print http://ask.apelearn.com/question/1738
Compare around, do not memorize, later use, a little more than a few times to come out.
awk ' Begin{print ' a ' "'" ' s "} '//No justification, just write a few more single quotes, double quotes
awk ' begin{print ' a ' \ ' s '} '//With a de-justification, a single quote
awk ' Begin{print ' a\ "s"} '//With a de-justification, a double quote
Merge the same rows from the two files into one line http://ask.apelearn.com/question/945
Paste filename1 filename2
If you want to connect at two file connections with a specified character, you can also use-D to specify
Paste-d ' + ' a.txt b.txt

The 9th chapter is the regular extension