Powerful grep Usage Details: grep and Regular Expressions

From: http://hi.baidu.com/nearlove/blog/item/11db98b6b5b8aff831add1e5.html

 

The first thing to remember is that regular expressions and wildcards have different meanings!
A regular expression is only a representation. As long as the tool supports this representation, the tool can process the string of a regular expression. Vim, grep, awk, and sed both support regular expressions, and they seem powerful because they support regular expressions. In previous companies, as the company is a web-based service-type website (nginx) with a high requirement for regular expressions, it also took some time to study regular expressions. I would like to share with you the following:

1 Basic Regular Expression
Grep tool, which was previously introduced.
Grep-[acinv] 'search content string' filename
-A search by text file
-C: calculates the number of matching rows.
-I case-insensitive
-N by the way, the output row number
-V reverse selection, that is, finding rows without a search string
The search string can be a regular expression!

1
Search for the row with the and output the row number.
$ Grep-n 'The 'regular_express.txt
Search for the rows without the and output the row number
$ Grep-NV 'the regular_express.txt

2Use [] to search for character sets
[] Indicates a character. For example, [Ade] indicates A, D, or E.
Woody @ xiaoc :~ /Tmp $ grep-n't [AE] st'regular_express.txt
8: I can't finish the test.
9: Oh! The soup taste good!

You can use the ^ symbol as the prefix in [] to indicate characters other than the characters in.
For example, you can search for the row of a string without g before oo. Use '[^ g] oo' as the search string.
Woody @ xiaoc :~ /Tmp $ grep-n' [^ g] oo 'regular_express.txt
2: Apple is my favorite food.
3: football game is not use feet only.
18: Google is the best tools for search keyword.
19: goooooogle yes!

[] Can be expressed in a range. For example, [A-Z] indicates lowercase letters, and [0-9] indicates 0 ~ 9 digits, [A-Z] are uppercase letters. [A-zA-Z0-9] represents all numbers and English characters. Of course, you can also use ^ to exclude characters.
Search for rows containing numbers
Woody @ xiaoc :~ /Tmp $ grep-n '[0-9]' regular_express.txt
5: however, this dress is about $3183 dollars.
15: you are the best is menu you are the No.1.

The beginning and end of a line character ^ $. ^ indicates the beginning of the line, $ indicates the end of the line (not a character, is a position), then '^ $' indicates a blank line, because only
The beginning and end of a row.
Here ^ is different from the ^ used in. It indicates that the string after ^ is the beginning of a row.
For example, search for the rows starting with
Woody @ xiaoc :~ /Tmp $ grep-n' ^ The 'regular_express.txt
12: The symbol '*' is represented as star.

Search for rows starting with lowercase letters
Woody @ xiaoc :~ /Tmp $ grep-n' ^ [A-Z] 'regular_express.txt
2: Apple is my favorite food.
4: This dress doesn' t fit me.
10: motorcycle is cheap than car.
12: The symbol '*' is represented as star.
18: Google is the best tools for search keyword.
19: goooooogle yes!
20: Go! Go! Let's go.
Woody @ xiaoc :~ /Tmp $

Search for rows starting with not English letters
Woody @ xiaoc :~ /Tmp $ grep-n' ^ [^ A-Za-Z] 'regular_express.txt
1: "Open Source" is a good mechanic to develop programs.
21: # I am vbird
Woody @ xiaoc :~ /Tmp $

$ Indicates the end of a row. For example, '/.' indicates the end of a row.
The row at the end of the search is.
Woody @ xiaoc :~ /Tmp $ grep-n '/. $' regular_express.txt //. It is a special symbol of a regular expression, so use/escape
1: "Open Source" is a good mechanic to develop programs.
2: Apple is my favorite food.
3: football game is not use feet only.
4: This dress doesn' t fit me.
5: however, this dress is about $3183 dollars.
6: GNU is free air not free beer.
.....

Note that for text files generated in MS, a ^ m character is added to the line feed. So the final character will be hidden ^ m.
Pay special attention to the following text!
You can use Cat dos_file | tr-D'/R'> unix_file to delete the ^ m symbol. ^ M =/R

Then '^ $' indicates that only empty rows at the end of the first line are pulled!
Search empty rows
Woody @ xiaoc :~ /Tmp $ grep-n' ^ $ 'regular_express.txt
22:
23:
Woody @ xiaoc :~ /Tmp $

Search for non-empty rows
Woody @ xiaoc :~ /Tmp $ grep-VN '^ $ 'regular_express.txt
1: "Open Source" is a good mechanic to develop programs.
2: Apple is my favorite food.
3: football game is not use feet only.
4: This dress doesn' t fit me.
..........

Any character. Repeated characters *

In bash, * represents a wildcard, which is used to represent any character. However, in a regular expression, * represents 0 or more characters with different meanings.
For example, oo * indicates that the first o must exist, and the second O can have one or more or no, so it represents at least one o.

Point. represents any character and must exist.G ?? D can be expressed in 'G .. d. Good, gxxd, gabd... all match.

Woody @ xiaoc :~ /Tmp $ grep-N 'G .. d' regular_express.txt
1: "Open Source" is a good mechanic to develop programs.
9: Oh! The soup taste good!
16: The world is the same with 'Glad '.
Woody @ xiaoc :~ /Tmp $

Search for strings with more than two o types
Woody @ xiaoc :~ /Tmp $ grep-N 'ooo * 'regular_express.txt // The first two o must exist. The third O may not exist, or multiple.
1: "Open Source" is a good mechanic to develop programs.
2: Apple is my favorite food.
3: football game is not use feet only.
9: Oh! The soup taste good!
18: Google is the best tools for search keyword.
19: goooooogle yes!

Search for the start and end of G, with at least one o string in the middle, that is, Gog, Goog... gooog...
Woody @ xiaoc :~ /Tmp $ grep-N 'Goo * G' regular_express.txt
18: Google is the best tools for search keyword.
19: goooooogle yes!

Search for the rows in which the string starting and ending with G is located.
Woody @ xiaoc :~ /Tmp $ grep-N 'G. * G' regular_express.txt //. * Indicates 0 or multiple arbitrary characters.
1: "Open Source" is a good mechanic to develop programs.
14: The GD software is a library for drafting programs.
18: Google is the best tools for search keyword.
19: goooooogle yes!
20: Go! Go! Let's go.

Limit the range of consecutive repeated characters {}
. * Only 0 or more characters are allowed. to limit the number of characters, use {range }. The range is numbers. 2 to 5 are separated to indicate 2 ~ Five,
2 indicates 2, 2 indicates 2 to more
Note that {} has special meaning in shell, so use/escape as a regular expression.

Search for rows with strings containing two O values.
Woody @ xiaoc :~ /Tmp $ grep-N 'o/{2/} 'regular_express.txt
1: "Open Source" is a good mechanic to develop programs.
2: Apple is my favorite food.
3: football game is not use feet only.
9: Oh! The soup taste good!
18: Google is the best tools for search keyword.
19: goooooogle yes!

Search for g followed by 2 ~ Five O, followed by a string of G.
Woody @ xiaoc :~ /Tmp $ grep-N 'go/{/} G' regular_express.txt
18: Google is the best tools for search keyword.

Search for rows that contain more than two o numbers after g and followed by G ..
Woody @ xiaoc :~ /Tmp $ grep-N 'go/{2,/} G' regular_express.txt
18: Google is the best tools for search keyword.
19: goooooogle yes!

Note: The ^-in [] does not have any special meaning. It can be placed behind the content in.
'[^ A-z /.! ^-] 'Indicates no lower-case letters, no. No !, No space, no-string. Note that there is a small space in.

In addition, the reverse selection in shell is [! Range], which is [^ range] in the regular expression.

2. Extended Regular Expression

The extended regular expression adds several special components to the basic regular expression.
It makes some operations more convenient.
For example, if we want to remove blank rows and rows whose first line is #, we will use the following:
Woody @ xiaoc :~ /Tmp $ grep-V '^ $ 'regular_express.txt | grep-V' ^ #'
"Open Source" is a good mechanic to develop programs.
Apple is my favorite food.
Football game is not use feet only.
This dress doesn' t fit me.
............

However, it is much easier to use egrep and extended special symbols that support extended regular expressions.
Note that grep only supports basic expressions, while egrep supports extensions. In fact, egrep is the alias of grep-e. Therefore, grep-e supports extended regular expressions.
So:
Woody @ xiaoc :~ /Tmp $ egrep-V '^ $ | ^ # 'regular_express.txt
"Open Source" is a good mechanic to develop programs.
Apple is my favorite food.
Football game is not use feet only.
This dress doesn' t fit me.
....................
Here | represents the or relationship. It is a string that meets the conditions of ^ $ or ^.

Here are several extended special symbols:
+, Similar to. *, indicates one or more repeated characters.
?, Similar to. *, it indicates 0 or one character.
|, Or relationship. For example, 'gd | good | dog' indicates a string with GD, good, or dog.
() To combine part of the content into a unit group. For example, to search for gglad or good, You Can 'G (La | OO) D' like this'
() The advantage is that it can be used for groups +? .
For example, if you want to search for at least one (XYZ) string at the beginning and end of A and C, you can: 'A (XYZ) + C'