Basic use of Awk and Sed

Basic use of Awk and Sed
 
You can call sed and awk in the same way. The command line is as follows:
Command [options] script filename
Like almost all unlx programs, sed and awk can get input from standard input and send output to standard output. If filename is specified, the input is taken from that file. The output contains the processed information. Standard output refers to the screen, and generally the output from these programs is output to it. The output can also be sent to a file. For example, I/O redirection in she11, but cannot be sent to the same file that provides input to the program. The options of each command are different.
 
Scept specifies the command to be executed. If the script on the command line contains spaces or any characters (such as $ and *) that can be interpreted by she11, it must be enclosed in single quotes.
 
The common option of sed and awk is-f, which allows you to specify the name of the script file. The increase in the attachment script size makes it easier to place it in a file. Therefore, you can call sed as follows:
Sed-f scriptfile inputfile
 
In sed and awk, each instruction consists of two parts: mode and process. The pattern is a regular expression separated by a slash. The process specifies one or more actions to be executed.
 
Sed:
 
-E is required only when multiple commands are provided on the command line. It tells sed to interpret the parameters as instructions. When there is only one command, sed can make its own decisions.
Sed [-e] 'insert' file
 
It is not necessary to enclose commands in single quotes in any situation, but you should develop this habit. The use of single quotes can prevent she11 from interpreting special characters or spaces in the editing instructions (she11 uses spaces to determine whether to give the program independent parameters, and the special she11 characters are expanded before the call ).
[Root @ ds-education2 ~] # Cat test
John Daggett, 341 King Road, Plymouth MA
AI ice Ford, 22 East 8 roadway, Richmond VA
Orvi 1 le Thomas, 11345 Oak 8 ridge Road, Tulsa OK
Terry Kalkas, 402 Lans Road, 8 eaver Fal Is PA
Eric Adams, 20 Post Road, Sudbury MA
Hubert Sims, 328A 8 rook Road, Roanoke VA
Amy Wi Ide, 334 8 ayshore Pkwy, Mountain View CA
Sal Carpenter, 73 6th Street, 8 oston MA
[Root @ ds-education2 ~] # Sed's/MA/MACK/'test // This command affects only three rows, but all rows are displayed.
John Daggett, 341 King Road, Plymouth uth MACK
AI ice Ford, 22 East 8 roadway, Richmond VA
Orvi 1 le Thomas, 11345 Oak 8 ridge Road, Tulsa OK
Terry Kalkas, 402 Lans Road, 8 eaver Fal Is PA
Eric Adams, 20 Post Road, Sudbury MACK
Hubert Sims, 328A 8 rook Road, Roanoke VA
Amy Wi Ide, 334 8 ayshore Pkwy, Mountain View CA
Sal Carpenter, 73 6th Street, 8 oston MACK
[Root @ ds-education2 ~] #
 
There are three ways to specify multiple commands on the command line:
1,
[Root @ ds-education2 ~] # Sed's/MA/MACK/; s/VA/VACK/'test // use; number processing
John Daggett, 341 King Road, Plymouth uth MACK
AI ice Ford, 22 East 8 roadway, Richmond VACK
Orvi 1 le Thomas, 11345 Oak 8 ridge Road, Tulsa OK
Terry Kalkas, 402 Lans Road, 8 eaver Fal Is PA
Eric Adams, 20 Post Road, Sudbury MACK
Hubert Sims, 328A 8 rook Road, Roanoke VACK
Amy Wi Ide, 334 8 ayshore Pkwy, Mountain View CA
Sal Carpenter, 73 6th Street, 8 oston MACK
[Root @ ds-education2 ~] #
2,
[Root @ ds-education2 ~] # Sed-e's/MA/MACK/'-e's/VA/VACK/'test // use the-e Option to implement
John Daggett, 341 King Road, Plymouth uth MACK
AI ice Ford, 22 East 8 roadway, Richmond VACK
Orvi 1 le Thomas, 11345 Oak 8 ridge Road, Tulsa OK
Terry Kalkas, 402 Lans Road, 8 eaver Fal Is PA
Eric Adams, 20 Post Road, Sudbury MACK
Hubert Sims, 328A 8 rook Road, Roanoke VACK
Amy Wi Ide, 334 8 ayshore Pkwy, Mountain View CA
Sal Carpenter, 73 6th Street, 8 oston MACK
[Root @ ds-education2 ~] #
3,
[Root @ ds-education2 ~] # Sed'
> S/MA/MACK/
> S/PA/PACK/
> S/CA/CACK/'test // enter sed 'and press Enter. This technology cannot be used in C she11.
John Daggett, 341 King Road, Plymouth uth MACK
AI ice Ford, 22 East 8 roadway, Richmond VA
Orvi 1 le Thomas, 11345 Oak 8 ridge Road, Tulsa OK
Terry Kalkas, 402 Lans Road, 8 eaver Fal Is PACK
Eric Adams, 20 Post Road, Sudbury MACK
Hubert Sims, 328A 8 rook Road, Roanoke VA
Amy Wi Ide, 334 8 ayshore Pkwy, Mountain View CACK
Sal Carpenter, 73 6th Street, 8 oston MACK
[Root @ ds-education2 ~] #
 
The result displayed on the screen is temporary and the input file has not changed !!!
 
Sed followed by the script file:
Sed-f script file
[Root @ ds-education2 ~] # Cat sed_test
S/MA/MACK/
S/PA/PACK/
S/CA/CACK/
[Root @ ds-education2 ~] # Sed-f sed_test test
John Daggett, 341 King Road, Plymouth uth MACK
AI ice Ford, 22 East 8 roadway, Richmond VA
Orvi 1 le Thomas, 11345 Oak 8 ridge Road, Tulsa OK
Terry Kalkas, 402 Lans Road, 8 eaver Fal Is PACK
Eric Adams, 20 Post Road, Sudbury MACK
Hubert Sims, 328A 8 rook Road, Roanoke VA
Amy Wi Ide, 334 8 ayshore Pkwy, Mountain View CACK
Sal Carpenter, 73 6th Street, 8 oston MACK
[Root @ ds-education2 ~] #
 
Save output:
[Root @ ds-education2 ~] # Sed-f sed_test test> sed_write // use redirection.
[Root @ ds-education2 ~] # Cat sed_write
John Daggett, 341 King Road, Plymouth uth MACK MA
AI ice Ford, 22 East 8 roadway, Richmond VA
Orvi 1 le Thomas, 11345 Oak 8 ridge Road, Tulsa OK
Terry Kalkas, 402 Lans Road, 8 eaver Fal Is PACK
Eric Adams, 20 Post Road, Sudbury MACK
Hubert Sims, 328A 8 rook Road, Roanoke VA
Amy Wi Ide, 334 8 ayshore Pkwy, Mountain View CACK
Sal Carpenter, 73 6th Street, 8 oston MACK
[Root @ ds-education2 ~] #
 
Disable Automatic output of input rows:
Sed outputs each input line by default. -N option can block automatic output. When this option is specified, each command to generate the output must contain the print command p
[Root @ ds-education2 ~] # Sed-n's/MA/MACK/'test // No output after-n is connected
[Root @ ds-education2 ~] # Sed-n's/MA/MACK/P' test // after p is added, all processed rows are output
John Daggett, 342 King Road, Plymouth uth MACK
Eric Adams, 20 Post Road, Sudbury MACK
Sal Carpenter, 73 6th Street, 8 oston MACK
[Root @ ds-education2 ~] #
 
Sed Command Options:
Option description
-E: Edit Subsequent commands.
-F follows the file name in the script
-N: prevents automatic output of input rows.
 
Awk
Each time a row is read from one or more files or from a standard input. Commands must be included in single quotes to separate them from shell (commands almost always contain braces and/or dollar signs, and she11 interprets them as special symbols ). You can enter multiple command lines in the same way as sed: Use semicolons to separate commands or use Boruneshell's multi-line input function.
Although the structure of the awk and sed commands is the same, they both consist of the mode and process, but the process itself is quite different. Unlike the editor, awk looks more like a programming language. Statements and functions replace a command sequence consisting of one or two characters. For example, use the print statement to print the expression value or the content of the current input line.
In general, awk interprets each input line as a record and each word on a line (separated by spaces or tabs) as each field (which can be changed by default ). One or more consecutive spaces or tabs are considered as a delimiter. Awk allows reference to these fields in mode or process. $0 indicates the entire record.
[Root @ ds-education2 ~] # Cat test
John Daggett, 342 King Road, Plymouth MA
AI ice Ford, 22 East 8 roadway, Richmond VA
Orvi 1 le Thomas, 11345 Oak 8 ridge Road, Tulsa OK
Terry Kalkas, 402 Lans Road, 8 eaver Fal Is PA
Eric Adams, 20 Post Road, Sudbury MA
Hubert Sims, 328A 8 rook Road, Roanoke VA
Amy Wi Ide, 334 8 ayshore Pkwy, Mountain View CA
Sal Carpenter, 73 6th Street, 8 oston MA
[Root @ ds-education2 ~] # Awk '{print $1}' test // print the first field of each line in the input file.
John
AI
Orvi
Terry
Eric
Hubert
Amy
Sal
[Root @ ds-education2 ~] #
 
"$1" indicates the value of the first field on each input line. Because no mode is specified here, the print statement is applied to all rows. The "/MA/" mode is specified in the next example, but no process exists. By default, this operation prints each row that matches this pattern.
[Root @ ds-education2 ~] # Awk '/MA/'test // by default, this operation prints every line matching the MA mode.
John Daggett, 342 King Road, Plymouth MA
Eric Adams, 20 Post Road, Sudbury MA
Sal Carpenter, 73 6th Street, 8 oston MA
[Root @ ds-education2 ~] #
[Root @ ds-education2 ~] # Awk '/MA/{print $1}' test // print the first field of the line containing MA (separated by spaces or tabs by default)
John
Eric
Sal
[Root @ ds-education2 ~] #
 
Use the-F option to change the field separator to comma
[Root @ ds-education2 ~] # Awk-F, '/MA/{print $1}' test
John Daggett
Eric Adams
Sal Carpenter
[Root @ ds-education2 ~] #
 
[Root @ ds-education2 ~] # Cat test1
Taobao, facebook, baidu
Google, sina, sohu
Wangyi, shengda, 51cto
[Root @ ds-education2 ~] # Awk-F, '{print $1}; {print $2}; {print $3}' test1
Taobao
Facebook
Baidu
Google
Sina
Sohu
Wangyi
Shengda
51cto
[Root @ ds-education2 ~] #
 
Command line options of awk:
-F follows the script text name
-F: Change the field separator.
-V follows var = value
 
Awk and Sed are used in combination through pipelines:
[Root @ ds-education2 ~] # Cat test1
Taobao, facebook, baidu
Google, sina, sohu
Wangyi, shengda, 51cto
[Root @ ds-education2 ~] # Cat sed_test1
S/taobao/TAOBAO/
S/google/GOOGLE/
S/wangyi/WANGYI/
[Root @ ds-education2 ~] # Sed-f sed_test1 test1 | awk-F, '{print $1 }'
TAOBAO
GOOGLE
WANGYI
[Root @ ds-education2 ~] #
 
This article is from the "dry wood Linux blog" blog