Common commands and syntaxes for grep, awk, and sed in Shell

Common grep command syntax 1. Double quotation marks and single quotation marks

When you input a string parameter in the g r e p command, it is best to enclose it with double quotation marks. For example, "m y s t r I n g ". There are two reasons for doing this: one is to prevent being misunderstood as the s h e l command, and the other is to find strings composed of multiple words, such as "jet plane ", if double quotation marks are not used, the word p l a n e will be mistaken for a file, and the query result will return the error message "the file does not exist.

When you call a variable, double quotation marks should also be used, such as: g r e p "$ m y va R" file name. If not

No results are returned.

When matching the call mode, use single quotes. [root @ mypc] # echo 'grep 123 111.txt '(# note that it is a single quotation mark)

2. Common g r e p options include:

-C only outputs the Count of matched rows.

-I is case insensitive (only applicable to single characters ).

-H: When querying multiple files, the file name is not displayed.

-L only names containing matching characters are output when multiple files are queried.

-N: the matching row and row number are displayed.

-S does not display the error message that does not exist or does not match the text.

-V: displays all rows that do not contain matched text.

3. Special-query multiple files

$ Grep "sort" *. DOC (# search for the string "s o r t" in all. d o C files in the current directory ")

 

$ Grep "sort it" * (# or query the word "sort it" in all files ")

All the following examples are used to query a single file.

4. Row matching

$ Grep-C "48" data. f

$4 (# g r e p returns the number 4, meaning four rows contain the string "4 8 ″.)

$ Grep "48" data. F (# display four lines of text containing the "4 8" string)

5. display the number of rows that meet the matching mode:

[Root @ mypc oid2000] # grep-N 1234 111.txt

:1234

3: 1234ab

6. Exact match

[Root @ mypc oid2000] # grep "1234 \>" 111.txt

1234

7. query empty rows. query the rows that start or end with a condition.

Use ^ and $ to query empty rows. Use the-n parameter to display the actual number of rows

[Root @ mypc oid2000] # grep-n "^ $" 111.txt (result 2: # the second line is empty)

[Root @ mypc oid2000] # grep-n "^ ABC" 111.txt (# query rows starting with ABC)

[Root @ mypc oid2000] # grep-N "ABC $" 111.txt (# query rows ending with ABC)

8. Match special characters and query characters with special meanings, such as $. '"* [] ^ | \ +? , Must be preceded by a specific character \.

[Root @ mypc oid2000] # grep "\." 111.txt (#query all rows containing "." In 111.txt)

[Root @ mypc oid2000] # grep "My \. conf" 111.txt (# query rows with the file name my. c o n f)

9. directory query

[Root @ mypc oid2000] # ls-L | grep "^ d" (# If you want to query the directories in the Directory List)

[Root @ mypc oid2000] # ls-L | grep "^ d [d]" (# query all files in a directory that do not contain the Directory)

[Root @ mypc] # ls-L | GRPE "^ d ..... X. x "(# query the directory set with executable permissions of other users and user group members)

Common command syntax of awk

The awk command is good at formatting packets or extracting data packets from a large text file. The following is the basic syntax of this command.

Awk [-f Filed-separator] "commands" Input-file (s)

[-F domain separator] is optional. a w k uses space as the default domain separator. If the file to be processed is split by a colon (such as a passwd file ), during Processing, specify awk-F: command input-file (s)

1.1 domains and records

When a w k is executed, Its browsing domain is marked as $1, $2... $ n. This method is called a domain identifier. Use $1 and $3 to refer to the 1st and 3rd fields. Note that the fields are separated by commas. If you want to print all the domains of a record with five domains, you do not need to specify $1, $2, $3, $4, $5, you can use $0, all domains.

1.2 save a w k output

$ Awk '{print $0}' input-files> out-files (# redirect save output)

$ Awk '{print $0}' input-files | tee out-files (# Use the t e command to output the files to the screen at the same time)

1.3 examples of common awk commands

[Root @ mypc/] # awk '$0 ~ /User/'/etc/passwd (# print the row if a domain contains a user)

RPC: X: 32: 32: Portmapper RPC User: // sbin/nologin

Rpcuser: X: 29: 29: RPC service user:/var/lib/nfs:/sbin/nologin

[Root @ mypc/] # awk '/user/'/etc/passwd (# Same as above)

[Root @ mypc/] # awk-F: '{if ($5 ~ /User/) Print $0} '/etc/passwd (# output this row if the fifth domain has a user)

RPC: X: 32: 32: Portmapper RPC User: // sbin/nologin

[Root @ mypc/] # ifconfig | awk '/inet/{print $2}' (# extract rows containing Inet from the ifconfig output and print the second field)

[Root @ mypc/] # ifconfig | awk '/inet/{print $2}' | awk-F: '{print $2}' (# extract from the above, this command allows you to directly obtain the IP address of the local machine)

Common sed command syntax

Sed is a non-interactive text stream editor. It edits a file or copies the text exported from standard input.

1. Row matching

[Root @ mypc/] # sed-N '2p'/etc/passwd print 2nd rows

[Root @ mypc/] # sed-N '1, 3 P'/etc/passwd print 1st to 3rd rows

[Root @ mypc/] # sed-n' $ P'/etc/passwd print the last line

[Root @ mypc/] # sed-n'/user/'P/etc/passwd print the row containing the user

RPC: X: 32: 32: Portmapper RPC User: // sbin/nologin

Rpcuser: X: 29: 29: RPC service user:/var/lib/nfs:/sbin/nologin

[Root @ mypc/] # sed-n'/\ $/'p/etc/passwd print the row containing $ metacharacters, $ indicates the last row

2. Insert text and additional text (insert a new line)

[Root @ mypc/] # sed-n'/FTP/P'/etc/passwd print the FTP row

FTP: X: 14: 50: FTP user:/var/ftp:/sbin/nologin

[Root @ mypc/] # sed '/FTP/A \ 100'/etc/passwd Insert a new row after the row containing FTP, with the content 456

[Root @ mypc/] # sed '/FTP/I \ 123'/etc/passwd Insert a new row before the row containing FTP, with the content 123

[Root @ mypc/] # sed '/FTP/I \ "123"'/etc/passwd Insert a new row before the row containing FTP with the content of "123 ″

[Root @ mypc/] # sed '5 A \ 100'/etc/passwd Insert a new row after row 123 with the content of 5th

[Root @ mypc/] # sed '5 I \ "12345" '/etc/passwd Insert a new row before row 5th with the content "12345 ″

3. delete text

[Root @ mypc/] # sed '1d '/etc/passwd Delete row 1st

[Root @ mypc/] # sed '1, 3D '/etc/passwd Delete lines 1st to 3

[Root @ mypc/] # sed '/user/D'/etc/passwd Delete row with user

4. Replace the text. Replace the command with the replacement mode to replace the specified mode. The format is:

[A d r e s [, address] S/pattern-to-find/replacement-pattern/[g p w n]

[Root @ mypc/] # SED's/user/'/etc/passwd: replace 1st users with user, and G indicates global replacement.

[Root @ mypc/] # SED's/user/G'/etc/passwd replace all users with user

[Root @ mypc/] # SED's/user/# user/'/etc/passwd: replace 1st users with # user, for example, used for shielding

[Root @ mypc/] # SED's/user // '/etc/passwd: replace 1st users with null

[Root @ mypc/] # SED's/user/& 11111111111111/'/etc/passwd if you want to append or modify a long string, you can use the (&) command, & Save the discovery mode of the command to call it again, and put it in the replacement string. Here we put & above

[Root @ mypc/] # SED's/user/11111111111111 &/'/etc/passwd here will be placed behind

5. A quick command line

Below are some command lines. ([] Indicates space, [] indicates t a B key)

'S/\. $ // G' Delete the row ending with a period

'-E/ABCD/d': Delete the row that contains a B c d.

'S/[] [] [] */[]/G' Delete more than one space and replace it with one space

'S/^ [] [] * // G' Delete the first space in the row

'S/\. [] [] */[]/G' deletes a period followed by two or more spaces, replacing it with a space.

'/^ $/D' Delete empty rows

'S/^. // G' Delete the first character

'S/COL \ (... \) // G' Delete the last three letters followed by C O L

'S/^ \ // G' Delete the first \ from the path \

'S/[]/[] // G' delete all spaces and replace them with the t a B key.

'S/^ [] // G' delete all t a B keys at the beginning of the row

'S/[] * // G' delete all t a B keys

If you use s e d to filter files, it is best to divide the problem into several steps, perform the step by step, and execute the test results.

Experience tells us that this is the most effective way to execute a complex task.