Shell text filtering programming (9): sed command

[Copyright statement: reprinted. Please retain the Source: blog.csdn.net/gentleliu. Mail: shallnew at 163 dot com]
Like awk, sed is an important text filtering tool.
Calling SED is the same as calling awk. There are three methods:
1. Type A command in the command line;
2. Insert the SED command into the script file and then call sed;
3. Insert the SED command into the script file and make the SED script executable.
The format of the SED command line is as follows:
Sed [Option] 'sed cmd' inputfile
Use the SED script file in the following format:
Sed [Option]-F sed_script_file.sed inputfile
In either case, if sed does not specify the input file, sed accepts the input from the standard input, usually the keyboard or redirection result. No matter what the command is, sed does not deal with the initialization file. It only operates on a copy, and if all the changes are not redirected to a file, the output will be to the screen.
Sed options are as follows:
-E: used when multiple sed commands are applied to the input line.
-N: cancel the default output.
-F specifies the SED script file name

Sed has the following command:
P print matching rows
= Display the file line number
A \ add new text information after locating the row number
I \ Insert new text information after locating the row number
D. Delete the row to be located.
C \ replace positioning text with new text
S replace the corresponding mode with the replacement Mode
R read text from another file
W. Write text to a file
Q. Release or release immediately after the first mode matching is completed
L display control characters equivalent to octal a s c I code
{} Command group executed on the target line
N read the next line of text from another file and append it to the next line
G paste Mode 2 to/pattern N/
Y Transfer Character
N continues to the next input line; supports cross-row pattern matching statements
The following describes the SED commands. Examples of sed in this series use the following text files:

#cat passwdroot:x:0:0:root:/root:/bin/shproxy:x:13:13:proxy:/bin:/bin/shoperator:x:37:37:Operator:/var:/bin/shftp:x:83:83:ftp:/home/ftp:/bin/shnobody:x:99:99:nobody:/home:/bin/shsshd:x:103:99:Operator:/var:/bin/sh#

1. p command
1) run the p command to print the file content.

#sed 'p' passwdroot:x:0:0:root:/root:/bin/shroot:x:0:0:root:/root:/bin/shproxy:x:13:13:proxy:/bin:/bin/shproxy:x:13:13:proxy:/bin:/bin/shoperator:x:37:37:Operator:/var:/bin/shoperator:x:37:37:Operator:/var:/bin/shftp:x:83:83:ftp:/home/ftp:/bin/shftp:x:83:83:ftp:/home/ftp:/bin/shnobody:x:99:99:nobody:/home:/bin/shnobody:x:99:99:nobody:/home:/bin/shsshd:x:103:99:Operator:/var:/bin/shsshd:x:103:99:Operator:/var:/bin/sh#

We can see that each row is printed twice. We use the-n option to disable the default output:

#sed -n 'p' passwdroot:x:0:0:root:/root:/bin/shproxy:x:13:13:proxy:/bin:/bin/shoperator:x:37:37:Operator:/var:/bin/shftp:x:83:83:ftp:/home/ftp:/bin/shnobody:x:99:99:nobody:/home:/bin/shsshd:x:103:99:Operator:/var:/bin/sh

# Awk implementation comparison:

#awk '{print}' passwdroot:x:0:0:root:/root:/bin/shproxy:x:13:13:proxy:/bin:/bin/shoperator:x:37:37:Operator:/var:/bin/shftp:x:83:83:ftp:/home/ftp:/bin/shnobody:x:99:99:nobody:/home:/bin/shsshd:x:103:99:Operator:/var:/bin/sh#

When SED is used, we do not have to print all rows. we can locate a row to operate on it, and use s e d to locate the text in the file:
X is a row number, such as 1
X, Y indicates that the row number ranges from X to Y. For example, 2nd indicates that the row number ranges from 5th.
/Pattern/query rows in the include mode. For example,/d I s k/or/[A-Z]/
/Pattern/query contains rows in two modes. For example,/d I s k/d I s K S/
Pattern/, X queries the rows in the include mode on the given row number. For example,/r I B o n/, 3
X,/pattern/query matched rows by row number and pattern. 3./V d u/
X, Y! Query rows that do not contain the specified row numbers X and Y. 1, 2!
2) print 3rd rows:

#sed -n '3p' passwdoperator:x:37:37:Operator:/var:/bin/sh#

Awk implementation comparison:

#awk '{if(NR==3)print}' passwdoperator:x:37:37:Operator:/var:/bin/sh#

3) print 2 to 4 rows:

#sed -n '2,4p' passwdproxy:x:13:13:proxy:/bin:/bin/shoperator:x:37:37:Operator:/var:/bin/shftp:x:83:83:ftp:/home/ftp:/bin/sh#

Awk implementation comparison:

#awk '{if(NR<=4 && NR>=2)print}' passwdproxy:x:13:13:proxy:/bin:/bin/shoperator:x:37:37:Operator:/var:/bin/shftp:x:83:83:ftp:/home/ftp:/bin/sh#

4) print matching rows

#sed -n '/var/p' passwdoperator:x:37:37:Operator:/var:/bin/shsshd:x:103:99:Operator:/var:/bin/sh# 

Awk implementation comparison:

#awk '/var/{print}' passwdoperator:x:37:37:Operator:/var:/bin/shsshd:x:103:99:Operator:/var:/bin/sh#

5) print the last line

#sed -n '$p' passwdsshd:x:103:99:Operator:/var:/bin/sh

2. "=" command to print the row number

#sed -n '/var/=' passwd36

Awk implementation comparison:

#awk '/var/{print NR}' passwd36

If the row number and matching line are printed, you must use two sed commands and use the e option. The first command prints the matching line. The second command uses the = option to print the line number. The format is sed-n-e/pattern/p-E/pattern/=.

#sed -n -e '/var/p' -e '/var/=' passwdoperator:x:37:37:Operator:/var:/bin/sh3sshd:x:103:99:Operator:/var:/bin/sh6#

#awk '/var/{print $0 "\n" NR}' passwdoperator:x:37:37:Operator:/var:/bin/sh3sshd:x:103:99:Operator:/var:/bin/sh6#

3. \ a command to add new text after the specified line
The A \ symbol can be used to append a specified line or multiple lines of text to the end of a specified line. When a text appending operation is performed, the result is output to the standard output. To edit the text after an additional operation, you must save the file and run another s e d command to edit it. At this time, the file content is moved to the buffer.

#sed '/root/a\this line is add by sed\haha' passwdroot:x:0:0:root:/root:/bin/shthis line is add by sedhahaproxy:x:13:13:proxy:/bin:/bin/shoperator:x:37:37:Operator:/var:/bin/shftp:x:83:83:ftp:/home/ftp:/bin/shnobody:x:99:99:nobody:/home:/bin/shsshd:x:103:99:Operator:/var:/bin/sh#

#awk '{print;if(/root/)print"this line is add by awk"}' passwdroot:x:0:0:root:/root:/bin/shthis line is add by awkproxy:x:13:13:proxy:/bin:/bin/shoperator:x:37:37:Operator:/var:/bin/shftp:x:83:83:ftp:/home/ftp:/bin/shnobody:x:99:99:nobody:/home:/bin/shsshd:x:103:99:Operator:/var:/bin/sh#

4. I \ command, similar to the \ a command, just insert a line of text before matching the line
5. Run the \ c command to modify the specified line.

#sed '/root/c\this line is modify by sed\h' passwdthis line is modify by sedhproxy:x:13:13:proxy:/bin:/bin/shoperator:x:37:37:Operator:/var:/bin/shftp:x:83:83:ftp:/home/ftp:/bin/shnobody:x:99:99:nobody:/home:/bin/shsshd:x:103:99:Operator:/var:/bin/sh

#awk '{if(/root/){print"this line is modify by awk"}else {print}}' passwdthis line is modify by awkproxy:x:13:13:proxy:/bin:/bin/shoperator:x:37:37:Operator:/var:/bin/shftp:x:83:83:ftp:/home/ftp:/bin/shnobody:x:99:99:nobody:/home:/bin/shsshd:x:103:99:Operator:/var:/bin/sh#

6. D command to delete Matching lines

#sed '1,4d' passwdnobody:x:99:99:nobody:/home:/bin/shsshd:x:103:99:Operator:/var:/bin/sh#

#awk '{if(NR>4)print}' passwdnobody:x:99:99:nobody:/home:/bin/shsshd:x:103:99:Operator:/var:/bin/sh#

7. s command to replace Line Text
Format: [Address [, address] S/pattern-to-find/replacement-pattern/[g p w n]
In the specified line ([Address [, address]), replace the pattern-to-find text with the text replacement-pattern, the P option prints the replaced rows, and the G option replaces all the global appearance modes. By default, only the first appearance mode is replaced. W file name use this option to export to a file.
Now we will replace the bin string of line 1 and 2 of the file with sedtest:

# sed  '1,2s/bin/sedtest/' passwd            root:x:0:0:root:/root:/sedtest/shproxy:x:13:13:proxy:/sedtest:/bin/shoperator:x:37:37:Operator:/var:/bin/shftp:x:83:83:ftp:/home/ftp:/bin/shnobody:x:99:99:nobody:/home:/bin/shsshd:x:103:99:Operator:/var:/bin/sh

We can use the-n option of SED and the P option of the S command to print only the replaced line:

# sed -n '1,2s/bin/sedtest/p' passwdroot:x:0:0:root:/root:/sedtest/shproxy:x:13:13:proxy:/sedtest:/bin/sh

Use the G option of the S command to replace all vertices in the current line:

# sed -n '1,2s/bin/sedtest/pg' passwdroot:x:0:0:root:/root:/sedtest/shproxy:x:13:13:proxy:/sedtest:/sedtest/sh

This command is more complicated to implement using awk.
8. R command to read text from another file
The content read from the new.txt file is appended to the specified row in the file mode.

# sed '3r new.txt' passwdroot:x:0:0:root:/root:/bin/shproxy:x:13:13:proxy:/bin:/bin/shoperator:x:37:37:Operator:/var:/bin/shthis line from new.txtftp:x:83:83:ftp:/home/ftp:/bin/shnobody:x:99:99:nobody:/home:/bin/shsshd:x:103:99:Operator:/var:/bin/sh

This operation uses awk more complex.
9. Run the Q command and exit after the compliant mode line.

# sed '/Operator/q' passwdroot:x:0:0:root:/root:/bin/shproxy:x:13:13:proxy:/bin:/bin/shoperator:x:37:37:Operator:/var:/bin/sh

# awk '{print;if (NR >= 3)exit}' passwdroot:x:0:0:root:/root:/bin/shproxy:x:13:13:proxy:/bin:/bin/shoperator:x:37:37:Operator:/var:/bin/sh

10. l command to display file control characters
Slightly modify the new.txt file (add some control characters) as follows:

# cat new.txtthis line from new.txtkkh# 

The command l is displayed as follows:

# sed  -n 'l' new.txtthis line from new.txt\033[B$kkh$\r$#

This feature is not available in awk.
11. Sed uses regular expressions for complex replacement.
Delete the first letter of a row:

# sed 's/^[a-z]*//g'  passwd

Add text after matching text:

# sed 's/[0-9][0-9]:/&8902:/g'  passwdroot:x:0:0:root:/root:/bin/shproxy:x:13:8902:13:8902:proxy:/bin:/bin/shoperator:x:37:8902:37:8902:Operator:/var:/bin/shftp:x:83:8902:83:8902:ftp:/home/ftp:/bin/shnobody:x:99:8902:99:8902:nobody:/home:/bin/shsshd:x:103:8902:99:8902:Operator:/var:/bin/sh

Perform the following operations on the password file:
A. Delete the first column separated:
B. replace all with spaces.
C. Delete the end SH of a row

# sed 's/^[a-z]*://g' passwd  |  sed 's/:/ /g' | sed 's/sh$//g'x 0 0 root /root /bin/x 13 13 proxy /bin /bin/x 37 37 Operator /var /bin/x 83 83 ftp /home/ftp /bin/x 99 99 nobody /home /bin/x 103 99 Operator /var /bin/

Delete the first two characters of a row:

# echo "hello sed." | sed 's/^..//g'llo sed.

Delete two characters at the end of a line:

# echo "hello sed." | sed 's/..$//g'hello se

Add the string "world" after the SED string ":

# echo "hello sed." | sed 's/sed/& world/g'hello sed world.

Compared with awk and SED, it seems one-sided to me that SED is more suitable for modifying and editing files, especially when it is convenient to modify a string in the line; awk is more suitable for retrieving the information separated by the specified delimiter for each row.
Awk is more complex and feels like a programming language, while SED is more like a tool, which can be easily handled with a small amount of code. These two tools have different strengths. Sometimes they can be used together to process texts.

Shell text filtering programming (9): sed command