grep, SED, awk real notes for the shell regex

Recently has been studying the shell script this piece, just idle down to the bottom of their own better information on some of the examples, the following is the bird in my brother's private dishes inside the regular expression inside the comparison of some of the basic syntax, suitable for novice inspection.

First copy an example:

Copy CodeThe code is as follows:
# VI Regular_express.txt
-------------------------------
"Open Source" is a good mechanism to develop programs.
Apple is my favorite food.
Football game isn't use feet only.
This dress doesn ' t fit me.
However, this dress was about $3183 dollars.
GNU is free air isn't free beer.
She hair is very beauty.
I can ' t finish the test.
Oh! The soup taste good.
Motorcycle is cheap than car.
This window is clear.
The symbol ' * ' is represented as start.
Oh! My god!
The GD software is a library for drafting programs.
You are the mean of the best is. 1.
The world <Happy> are the same with "glad".
I like dog.
Google is the best tools for search keyword.
Goooooogle yes!
Go! Go! Let ' s go.
# I AM Vbird
--------------------------------

Set Language to C

Copy CodeThe code is as follows: #export lang=c

Grep

1. Search for a specific string "the"
Note: N is the display line number

Copy CodeThe code is as follows: # grep-n ' the ' regular_express.txt

2. Reverse search for a specific string "the"

Copy CodeThe code is as follows: # grep-vn ' the ' regular_express.txt

3. Get the string "the" in any case

Copy CodeThe code is as follows: # grep-in ' the ' regular_express.txt

4. Use parentheses [] to search for set characters
When I search for the two words of test or taste, I find that they have a common ' t?st ', so we can search for them.

Copy CodeThe code is as follows: # grep-n ' t[ae]st ' regular_express.txt

This is actually looking for t[a]st and t[e]st, the two separate characters.
If you are searching for OO characters, you can use:

Copy CodeThe code is as follows: # grep-n ' oo ' regular_express.txt

If you do not want to search oo with g in front of OO, we can use reverse selection [^] to achieve:

Copy CodeThe code is as follows: # grep-n ' [^g]oo ' Regular_express.txt

If you do not want to have lowercase characters in front of the search oo:

Copy CodeThe code is as follows: # grep-n ' [^a-z]oo ' Regular_express.txt

Note: uppercase English/lowercase English/numbers can be written using [a-z]/[a-z]/[0-9], or can be written together
[A-za-z0-9] indicates that a string is required to be numeric and English
If we want to get the line with numbers, then:

Copy CodeThe code is as follows: # grep-n ' [0-9] ' regular_express.txt

Note: However, given the influence of the language family on the encoding sequence, it is possible to use the [: lower:] Instead of a-Z and [:d igit:] Instead of 0-9 in addition to continuous encoding using the minus sign [-].

Copy CodeThe code is as follows: # grep-n ' [^[:lower:]]oo ' Regular_express.txt
# grep-n ' [[:d igit:]] ' regular_express.txt

5. Display the string at the beginning of ' the '

Copy CodeThe code is as follows: # grep-n ' ^the ' regular_express.txt

Displays the beginning of the line as lowercase characters

Copy CodeThe code is as follows: # grep-n ' ^[a-z] ' regular_express.txt

6. Displays the end of the line as a point. The line

Copy CodeThe code is as follows: # grep-n ' \.$ ' regular_express.txt

7. Display 5-9 rows of data

Copy CodeThe code is as follows: # Cat-an Regular_express.txt |head-n |tail-n 6

8. Show Blank Lines

Copy CodeThe code is as follows: # grep-n ' ^$ ' regular_express.txt

9. Find out G?? D string, beginning G four string ending D

Copy CodeThe code is as follows: # Grep-n ' G.. d ' Regular_express.txt

O* represents the null character (that is, there are no characters) or one to n o characters, so grep-n ' o* ' regular_express.txt will print out all the lines,
11.oo* represents o+ empty characters or one to n o characters, so grep-n ' oo* ' regular_express.txt will print all lines of o,oo,ooo and so on
"Goo*g" stands for Gog,goog,gooog ... such as

Copy CodeThe code is as follows: # grep-n ' Goo*g ' regular_express.txt

13. Find the line with the G...G string
Note: represents any character,. * represents an empty character or one to n any character

Copy CodeThe code is as follows: # grep-n ' G.*g ' regular_express.txt

14. Find the rows that contain numbers

Copy CodeThe code is as follows: # grep-n ' [0-9][0-9]* ' Regular_express.txt

or # grep-n ' [0-9] ' regular_express.txt

15. Find a string with two O
Note: {} because there is a special meaning in the shell, you need to jump off the character \ to make it meaningless.

Copy CodeThe code is as follows: # grep-n ' o\{2\} ' regular_express.txt

Find the string after G that contains 2 to 5 O and ends with a G

Copy CodeThe code is as follows: # grep-n ' Go\{2,5\}g ' regular_express.txt

Find the string after g that contains more than 2 O and ends with G

Copy CodeThe code is as follows: # grep-n ' Go\{2,\}g ' regular_express.txt

Summarize:
^word indicates a string with a search (word) at the beginning of the line
word$ indicates a string with a search (word) at the end of a row
. Represents 1 arbitrary characters
\ denotes an escape character, preceded by a special character, will remove the meaning of the original special character
* represents repeating 0 to infinitely multiple previous re (regular expression) characters
[list] means searching for a string containing a list
[N1-N2] means searching for a specified range of strings, for example [0-9] [A-z] [a-z], etc.
[^list] represents a range of inverse strings, for example [0-9] for non-numeric characters, [A-z] for non-uppercase character ranges
\{n,m\} means finding N to m previous re characters
\{n,\} represents more than N of the previous re character
Egrep Summary:
+ represents repeating one or more of the previous re characters
Example: Egrep ' Go+d ' regular_express.txt
Expression Search (God) (good) (Goood) ... And so on string, o+ stands for [more than one o]
? Represents a repeat of 0 or one of the previous re characters
Example: Egrep ' Go?d ' regular_express.txt
Represents a search (GD) string, o represents [empty or 1 O]
Note: The result set under Egrep ' Go+d ' and ' go?d ' is equal to the ' go*d ' under grep
| To identify several strings in a way or (or)
Example: Egrep ' Gd|good|dog ' regular_express.txt
Represents a search (GD) or (God) or (God) string, | represents or
() indicates finding the group string
Example: Egrep ' G (la|oo) d ' regular_express.txt
Represents a search (glad) or (good) string
() + indicates identification of multiple repeating groups
Example: Echo ' axyzxyzxyzxyzxyzc ' |egrep ' A (xyz) +c '
Indicates that the search begins with a ending is C, with more than one ' xyz ' string in the middle

Sed:

Insert:

1. List and print line numbers for/etc/passwd, and delete 2-5 rows

Copy CodeThe code is as follows: # NL/ETC/PASSWD | Sed ' 2,5d '

Note: sed is shorthand for sed-e, followed by single quotation mark
Delete Line 2nd as ibid.

Copy CodeThe code is as follows: # NL/ETC/PASSWD | Sed ' 2d '

Delete the third row to the last row

Copy CodeThe code is as follows: # NL/ETC/PASSWD | Sed ' 3, $d '

2. Add a row after the second line test

Copy CodeThe code is as follows: # NL/ETC/PASSWD | SED ' 2a test '

Add a row of test before the second line

Copy CodeThe code is as follows: # NL/ETC/PASSWD | Sed ' 2i test '

Add two lines of test after the second line

Copy CodeThe code is as follows: # NL/ETC/PASSWD | SED ' 2a test \
> Test '

Replace line:

3. Replace 2-5 lines of content with No 2-5 number

Copy CodeThe code is as follows: # NL/ETC/PASSWD | Sed ' 2,5c No 2-5 number '

4 list 第5-7 lines in/etc/passwd

Copy CodeThe code is as follows: # nl/etc/passwd |sed-n ' 5,7p '

Replacement string:

Sed ' s/replaced by string/new string/g '

1. Get the line for the native IP

Copy CodeThe code is as follows: #/sbin/ifconfig eth0 |grep ' inet addr '

Delete the previous part of IP

Copy CodeThe code is as follows: #/sbin/ifconfig eth0 |grep ' inet addr ' | Sed ' s/^.*addr://g '

Remove the part of the IP after

Copy CodeThe code is as follows: #/sbin/ifconfig eth0 |grep ' inet addr ' | Sed ' s/^.*addr://g ' | Sed ' s/bcast:.*$//g '
-------------------
192.168.100.74
-------------------

2. Use grep to remove the keyword man row

Copy CodeThe code is as follows: # cat/etc/man.config |grep ' man '

Delete comment lines

Copy CodeThe code is as follows: # cat/etc/man.config |grep ' man ' | Sed ' s/^#.*$//g '

Delete blank Lines

Copy CodeThe code is as follows: # cat/etc/man.config |grep ' man ' | Sed ' s/^#.*$//g ' | Sed '/^$/d '

3. Use SED to replace each line in the Regular_express.txt.
Note: the-i parameter modifies the text directly, not the direct output

Copy CodeThe code is as follows: # sed-i ' s/.*\.$/\!/g ' regular_express.txt

4. Use sed to add #This is a test to the last line of text
Note: $ represents the last line a represents a row after adding

Copy CodeThe code is as follows: # sed-i ' $a #This is a test ' regular_express.txt

Change SELinux configuration file enforcing to Disabled

Copy CodeThe code is as follows: # sed-i ' 6,6c selinux=disabled '/etc/selinux/config

Extended formal notation:

Copy CodeThe code is as follows: # grep-v ' ^$ ' regular_express.txt |grep-v ' ^# '

Extended wording:

Copy CodeThe code is as follows: # egrep-v ' ^$ ' | ' ^# ' Regular_express.txt

1. + represents repeating one or more of the previous re characters

Example: Egrep-n ' Go+d ' regular_express.txt
General wording: Grep-n ' goo*d ' regular_express.txt

2. Represents repeating 0 or one of the previous re characters

Example: Egrep-n ' Go?d ' regular_express.txt

3. | means to find out several strings in a way or

Example: Egrep-n ' Gd|good ' regular_express.txt

4. () indicates finding the group string

Example: Egrep-n ' G (la|oo) d ' regular_express.txt
That is, search (glad) or good these two strings

5. () + Multiple repeating group discrimination

For example: Echo ' axyzxyzxyzxyzc ' |egrep ' A (xyz) +c '

That is, to find the beginning is a end is a C in the middle there is more than one ' xyz ' string meaning

Awk:

1. Use last to remove the log data before the five elements

Copy CodeThe code is as follows: # last-n 5

Remove account and login IP, and tab between account and IP

Copy CodeThe code is as follows: # last-n 5 |awk ' {print $ \ t ' $ $} '

Note: Represents the first field separated by a space or tab, and so on.
$ A represents the entire field of the row

Copy CodeThe code is as follows: # last-n 5 |awk ' {print $ \ t lines: "NR" \ t columes: "NF} '

Note: NF represents the total number of fields in each row
NR represents the first few rows of data that awk is currently in.
FS represents the target delimiter, which is the default space

2. In/etc/passwd: As a segmented character, we need to check the third column less than 10, and only list the account number and the third column

Copy CodeThe code is as follows: # CAT/ETC/PASSWD | awk ' {fs= ': '} $3<10 {print ' \ t \ t \ ' $ $} '

Note: The query results do not show the first row of data because we define fs= ":" But only in the second row
To read the first line, you need to begin the keyword:

Copy CodeThe code is as follows: # CAT/ETC/PASSWD | awk ' BEGIN {fs= ': '} $3<10 {print ' \ t \ t ' $ $} '

Df:
Compare the differences between two files:

Copy CodeThe code is as follows: # diff/etc/rc3.d//etc/rc5.d/
-------------------
Only in/etc/rc3.d/: K30spice-vdagentd
Only in/etc/rc5.d/: S70spice-vdagentd
-------------------

Instance:
1. Statistics TCP Connection Status

Copy CodeThe code is as follows: # Netstat-na | awk '/^tcp/{++s[$NF]} END {for (a in S) print A, s[a]} '
/^tcp/

Filter out lines that start with TCP, "^" for regular expression usage, ... First, this is to filter out lines that start with TCP.
S[]
An array named S is defined, and in awk, array subscripts usually start at 1 instead of 0.
Nf
The number of fields in the current record, separated by a space by default, as shown above, the number of NF fields equals
$NF
Represents the value of the last field of a row, such as the record shown above, $NF that is $6, which represents the value of the 6th field, that is, syn_recv or time_wait, and so on.
s[$NF]
The number of connections representing the values of the array elements, such as the record shown above, which is the s[time_wait] state
++s[$NF]
To add a number to a record, as shown above, is to add a s[time_wait] state to the number of connections
The result is a display of the final array value in the S array.
Example: s[time_wait]= final value s[testablished]= final value
END
For (key in S)
Traversal s[] Array
Print key, "\ T", S[key]
Prints the keys and values of the array, separated by the \ T tab in the middle, showing better.

grep, SED, awk real notes for the shell regex