Learn a little shell ------------------------- sed every day

Reprinted: http://blog.chinaunix.net/u/22677/showart_1076318.html

1. Introduction

Sed is a non-interactive editor. It does not modify the file unless you use shell redirection to save the results. By default, all output rows are printed to the screen. Sed editor processes files (or inputs) row by row and sends the results to the screen. The specific process is as follows: first, sed stores the currently being processed rows in a temporary cache (also known as the mode space), and then processes the rows in the temporary buffer, then, send the row to the screen. Sed deletes a row from the temporary buffer after processing, and then reads the next row for processing and display. After processing the last line of the input file, sed stops running. Sed stores each row in a temporary buffer and edits the copy. Therefore, the original file is not modified. 2. AddressingThe address is used to determine which rows to edit. The address format can be numbers, regular expressions, or a combination of the two. If no address is specified, sed processes all rows of the input file. If the address is a number, it indicates the row number. If the address is a "$" symbol, it indicates the last row. For example:

Sed-N '3' datafile Print only the third row

 

Only show the file content within the specified row range, for example:

# Only view rows 100th to 200th of the file
Sed-N '100,200 P' mysql_slow_query.log

 

The addresses are separated by commas, so the addresses to be processed are the range between the two rows (including the two rows ). The range can be represented by numbers, regular expressions, or combinations of the two. For example:

Sed '2, 5d 'datafile # Delete the second to fifth line Sed '/My/,/you/d' datafile # Delete rows containing "my" to rows containing "you" Sed '/My/, 10d' datafile # Delete the content from the row containing "my" to the row 10

 

 

 

3. commands and options

The SED Command tells sed how to process each input line specified by the address. If no address is specified, all input lines are processed.

 

3.1 sed command

Command	Function
A \	Add one or more rows after the current row. In addition to the last row, the end of each row must be continued "\".
C \	Replace the text in the current line with the new text after this symbol. In addition to the last row, you must use "\" to continue the row at the end of each line.
I \	Insert text before the current row. In addition to the last row, you must use "\" to continue the row at the end of each line.
D	Delete row
H	Copy the content in the mode space to the temporary buffer.
H	Append the content in the mode space to the temporary buffer.
G	Copy the content in the temporary buffer to the mode space to overwrite the original content.
G	Append the content of the temporary buffer to the mode space, and append the content to the end of the original content.
L	List non-printable characters
P	Print rows
N	Read the next input line and process it from the next command instead of the first command.
Q	End or exit SED
R	Read input lines from files
!	Apply commands to all rows other than the selected row
S	Replace another string
G	Global replacement within a row
W	Write the selected row to a file
X	Swap content of the temporary buffer and mode space
Y	Replace the character with another character (the y command cannot be used for the regular expression)

 

3.2 sed options

Option	Function
-E	Multiple edits, that is, when multiple sed commands are applied to the input line
-N	Cancel default output
-F	Specify the SED script file name

    4. ExitSed is not the same as grep. Whether or not the specified mode is found, its exit status is 0. The exit status of SED is not 0 only when the command has a syntax error. 5. Regular Expression metacharactersLike grep, sed also supports Special metacharacters for Pattern Search and replacement. The difference is that the regular expression used by SED is the mode between the slash line. If you want to change the regular expression separator "/" to another character, for example, O, you only need to add a backslash before the character, followed by the regular expression, and then followed by the character. Example: sed-n' \ o ^ myop 'datafile

Metacharacters	Function	Example
^	First line Separator	/^ My/match all rows starting with my
$	Line tail Locator	/My $/match all rows ending with my
.	Match a single character except line breaks	/M .. y/match with the letter M, followed by two arbitrary characters, followed by the letter y line
*	Matches zero or multiple leading characters	/My */match the row containing the letter M followed by zero or multiple y letters
[]	Match any character in the specified character group	/[Mm] Y/match the rows that contain my or my
[^]	Match any character that is not in the specified character group	/[^ Mm] Y/matches y, but the character before y is not a line of M or m
\(..\)	Save matched characters	1, 20 s/\ (you \) Self/\ 1R/mark the mode between metacharacters and save it as tag 1. Then you can reference it with \ 1. A maximum of nine tags can be defined, starting from the left and the first tag on the left. In this example, 1st to 20th rows are processed, and you are saved as tag 1. If you find youself, replace it with your.
&	Save the search string to reference it in the replacement string	S/My/** & **/Symbol & represents the search string. My will be replaced with ** my **
\ <	First-word Locator	/\ <My/match rows that contain words starting with my
\>	Suffix	/My \>/match rows that contain words ending with my
X \ {M \}	M x in a row	/9 \ {5 \}/match rows containing 5 consecutive 9
X \ {M ,\}	At least m x	/9 \ {5, \}/match rows containing at least five consecutive 9
X \ {M, N \}	At least m, but no more than N x	/9 \ {5, 7 \}/match rows containing 5 to 7 consecutive 9

6. Example  6.1 p commandCommand P is used to display the content of the mode space. By default, sed prints the input line on the screen, and option-N is used to cancel the default printing operation. When option-N and command P both appear, sed can print the selected content.

Sed '/My/P' datafile # By default, sed prints all input rows on the standard output. If a line matches the mode my, the p command prints the line again. Sed-n'/My/P' datafile # Option-N cancel SED's default printing. The p command prints the line matching the mode my.

 

6.2 D command

Command D is used to delete the input line. Sed first copies the input line from the file to the mode space, then runs the SED command on the line, and finally displays the content in the mode space on the screen. If the command D is sent, the input row in the current mode space will be deleted and not displayed.

Sed '$ d' datafile # Delete the last row, and the rest are displayed. Sed '/My/d' datafile # Delete the rows containing my, and the rest are displayed.

 

6.3 s command

 

Sed's/^ My/you/G' datafile # The g at the end of the command indicates global replacement within the row. That is to say, if multiple mys appear in a row, all my s are replaced with you. Sed-n'1, 20 s/my $/you/GP 'datafile # Cancel the default output, process the rows from 1 to 20 matching the rows ending with my, replace all the rows in the row with you, and print them to the screen.

Sed's # My # Your # G' datafile # The character that follows the S command is the separator between the search string and the replacement string. The default Delimiter is a forward slash, but it can be changed. No matter what characters (except line breaks and backlash), as long as the S command is followed, it becomes a new string separator.

 

6.4 E Option

-E is an editing command used when sed executes multiple editing tasks. Before editing the next line, all the editing actions will be applied to the row in the mode buffer.

sed -e ‘1,10d‘ -e ‘s/My/Your/g‘ datafile # Option-E is used for multiple edits. The First edits and deletes Row 1-3. The second edit replaces all my that appear with your. Because these two commands are edited row by row (both commands are executed on the current row of the mode space), the order of the commands will affect the result.

 

6.5 R command

The R command is a READ command. Sed uses this command to add the content of a text file to a specific location of the current file.

Sed '/My/R introduce.txt' datafile If a file named "datafile' is matched with a pattern", the file "introduce.txt" is read later. If the current myrow does not stop, the contents of the introduce.txt file will be read after the current myrow appears.

  6.6 million commands

sed -n ‘/hrwang/w me.txt‘ datafile

 

6.7 A \ command

The A \ command is an APPEND Command that adds new text to the current row of the file (that is, the row in the Read mode buffer. The appended text line is located at the bottom of the SED command and starts from another line. If the content to be appended exceeds one row, each row must end with a backslash, except for the last row. The last line ends with quotation marks and a file name.

Sed '/^ hrwang/\ > Hrwang and mjfan are husband \ > And wife 'datafile # If the row starting with hrwang is found in the datafile file, add hrwang and mjfan are husband wife to the row.

 

6.8 I \ command

The I \ command inserts new text before the current line.

 

6.9 C \ command

Sed uses this command to modify existing text into new text.

 

6.10 n command

Sed uses this command to get the next line of the input file and read it into the mode buffer. Any sed command will be applied to the next line followed by the matching line.

 

sed ‘/hrwang/{n;s/My/Your/;}‘ datafile

Note: If you need to use multiple commands, or you need to nest an address within a certain address range, you must enclose the commands in curly brackets and write only one command per line, or use semicolons to separate multiple commands in the same line. 6.11 y commandThis command is similar to the tr command in Unix/Linux. The characters are converted from left to right in one-to-one mode. For example, Y/ABC/converts lowercase A to A, lowercase B to B, and lowercase C to C.

Sed '1, 20y/hrwang12/hrwang ^ $/'datafile # Convert all lower-case hrwang data from 1 to 20 rows to uppercase, convert 1 to ^, and convert 2 to $. # The metacharacter of the regular expression does not work for the y command. Like the separator of the S command, the diagonal line can be replaced with other characters.

 

6.12 Q command

The Q command will cause the SED program to exit and no longer process it.

sed ‘/hrwang/{s/hrwang/HRWANG/;q;}‘ datafile

 

6.13 H commands and G commands

#cat datafile My name is hrwang. Your name is mjfan. hrwang is mjfan‘s husband. mjfan is hrwang‘s wife.    sed -e ‘/hrwang/h‘ -e ‘$G‘ datafile sed -e ‘/hrwang/H‘ -e ‘$G‘ datafile # Using the above two commands, you will find that H will clear the content of the original temporary buffer and only save the content of the mode space saved in the last execution of H. The H command appends the rows matching the hrwnag and stores them in the temporary buffer zone. sed -e ‘/hrwang/H‘ -e ‘$g‘ datafile sed -e ‘/hrwang/H‘ -e ‘$G‘ datafile # Using the above two commands, you will find that G has replaced the content in the temporary buffer with the content of the current row in the mode space. Here, the last row is replaced. The G command appends the content of the temporary buffer to the current row of the mode space. Append to the end.

 

 

7. Sed script

The SED script is a column of SED commands written in the file. In the script, no extra space or text is required at the end of the command. If multiple commands exist in one line, separate them with semicolons. When executing the script, sed first copies the first line of the input file to the mode buffer, and then executes all the commands in the script. After each row is processed, sed copies the next row of the file to the mode buffer and executes all the commands in the script. When using the SED script, no quotation marks are used to ensure that the SED command is not interpreted by shell. For example, sed script:

#handle datafile 3i\ ~~~~~~~~~~~~~~~~~~~~~ 3,$s/\(hrwang\) is \(mjfan\)/\2 is \1/ $a\ We will love eachother forever！！

 

 

# Sed-F script datafile My name is hrwang Your name is mjfan ~~~~~~~~~~~~~~~~~~~~~ Mjfan is hrwang's husband. # ~~~ Mjfan is hrwang's wife. We will love eachother forever !!