Usage of sed in Shell (mode replacement)

ZZ: http://doc.linuxpk.com/80440.html

Sed (Stream Editor) usage:

1. It is a non-interactive text stream editor. It edits a file or copies the text exported from standard input.

2. Specify the text line to be changed using the row number or regular expression.

3. Sed does not deal with the initial file, but only one copy of it. If the operation result is not redirected to a file, it will be output to the screen. Call Method: A. Command Line Method B. Insert the SED command into the script file, and then call sed C. Insert the SED command into the script file to make the script file executable. Save sed output and redirect it to a new file> SED to locate text in the form of X line number X, Y row number range/pattern/query rows in the contain mode/pattern/query rows in two modes/pattern/, X query rows in the matching mode on the specified row number X, /pattern/query matching rows X, Y through the row number and mode! The query does not contain the row number X, and the row sed editing command of Y matches the text before the metacharacters $, you must use the backslash \ = to print the row number. Use the-e option. If both the row number and the matching row are printed, you must use two sed commands and use the-E Option to append the text with the symbol \, you can specify one or more lines of text to be appended to the specified line. If no text placement position is specified, sed is placed after each row by default. Create sed script file :#! /Bin/sed-F ---- note the SED command interpretation line. The script searches for sed in this line to run the command. Locate insert text in/bin: insert the text before the specified line, and it only accepts one address. Delete text: the D replacement command replaces the specified mode with the replacement mode. An important function implemented by SED is to remove control characters from the files downloaded from another system. 1. use S/-* // g to delete the horizontal line ----- 2. delete empty rows with/^ $ S/D. use $ d to delete the last row. 4. use 1d to delete Row 5. use awk {print $1} to print the first column

In this articleArticle, We Will browse the most common commands and options and demonstrate how they work and where they are suitable for use.

Replacement command

One of the most common commands of SED utility and any other similar editor is to replace another value with one value. The command syntax used to perform this operation is as follows:

'S/{old value}/{New Value }/'

Therefore, the following demonstrates how to easily change "tiger" to "wolf ":

$ Echo the tiger cubs will meet on Tuesday after school | SED
'S/TIGER/wolf /'
The wolf cubs will meet on Tuesday after school
$

Note that if the input is from the previous command output, you do not need to specify the file name-the same principle applies to awk, sort, and most other linuxunix Command Line UtilitiesProgram.

Multiple modifications

If you need to modify the same file or row multiple times, you can implement it in three ways. The first option is to use the "-e" option, which notifies the program that multiple editing commands are used. For example:

$ Echo the tiger cubs will meet on Tuesday after school | sed-e'
S/TIGER/wolf/'-E's/after/before /'
The wolf cubs will meet on Tuesday before school
$

This is a very complex method to implement it, so the "-e" option is not often used in a wide range. A better way is to use semicolons to separate commands:

$ Echo the tiger cubs will meet on Tuesday after school | sed'
S/TIGER/wolf/; S/after/before /'
The wolf cubs will meet on Tuesday before school
$

Note that the semicolon must be the next character following the diagonal line. If there is a space between the two, the operation cannot be completed successfully and an error message is returned. Both methods are good, but many administrators prefer another method. One of the key issues to be aware of is that all content between two apostrophes ('') is interpreted as the SED command. The shell program that reads these commands does not think that you have completed the input until you enter the second marker. This means that you can enter the command on multiple lines-at the same time, Linux changes the prompt from PS1 to a continuation prompt (usually ">")-until the second marker is entered. Once the second marker is entered and the Enter key is pressed, the processing is performed and the same result is generated, as shown below:

$ Echo the tiger cubs will meet on Tuesday after school | sed'
> S/TIGER/wolf/
> S/after/before /'
The wolf cubs will meet on Tuesday before school
$

Global Modification

Let's start a seemingly simple edit. It is assumed that a project with multiple minor modifications appears in the message to be modified. By default, the results may be different from expected, as shown below:

$ Echo the tiger cubs will meet this Tuesday at the same time
As the meeting last Tuesday | SED's/Tuesday/Thursday /'
The tiger cubs will meet this Thursday at the same time
As the meeting last Tuesday
$

In contrast to changing every "Tuesday" to "Thursday", the SED editor finds a project to be modified and then processes the next row without reading the entire row. The SED command function is basically similar to the replacement command, which means they are all processed? Yan brother harmonic value Yu Xiao n song Huai Yu value to take the diligent Yu brother harmonic Yu yu nan Yu Kang that aggregation? You must specify to perform this operation globally:

$ Echo the tiger cubs will meet this Tuesday at the same time
As the meeting last Tuesday | SED's/Tuesday/Thursday/G'
The tiger cubs will meet this Thursday at the same time
As the meeting last Thursday
$

Remember that this sort requirement is required no matter whether the sequence you want to search for contains only one character or phrase.

Sed can also be used to modify the record field separator. For example, the following command will change all tabs to spaces:

Sed's // G'

Among them, the project between the first group of diagonal lines is a tab, and the project between the second group of diagonal lines is a space. As a general rule, sed can be used to change any printable character to any other printable character. If you want to change non-printable characters to printable characters-for example, modifying bell to the word "Bell"-sed is not a tool for doing this (but TR is ).

Sometimes, you do not want to modify all the specified items that appear in a file. Sometimes, you only need to modify it when certain conditions are met-for example, after matching with other data. To illustrate this, consider the following text files:

$ Cat sample_one
One 1
Two 1
Three 1
One 1
Two 1
Two 1
Three 1
$

Assume that you want to replace "1" with "2", but only after the word "two", instead of all the positions in each row. You can achieve this by specifying that a match must exist before the replacement command is given:

$ Sed '/Two/S/1/2/'sample_one
One 1
Two 2
Three 1
One 1
Two 2
Two 2
Three 1
$

Now, make it more accurate:

$ Sed'
& Gt;/Two/S/1/2/
>/Three/S/1/3/'sample_one
One 1
Two 2
Three 3
One 1
Two 2
Two 2
Three 3
$

Remember that the only change is display. If you view the source file, you will find that it remains unchanged. You must save the output to another file for permanent storage. It is worth repeating that not modifying the source file is actually a blessing-it allows you to experiment with the file without causing any actual damage, until you have the correct command to work as expected and expected.

Run the following command to save the modified output to a new file:

$ Sed'
& Gt;/Two/S/1/2/
>/Three/S/1/3/'sample_one> sample_two

The output file merges all the modifications, which are usually displayed on the screen. You can now use head, cat, or any other similar utility to view it.

Script File

SED allows you to create a script file that contains commands to process the file instead of the command line, and SED is referenced by the "-F" option. By creating a script file, you can repeat the same operation again and specify more detailed operations than the operations you want to process from the command line each time.

Consider the following script files:

$ Cat sedlist
/Two/S/1/2/
/Three/S/1/3/
$

Now we can use the script file on the data file to get the same result as we saw before:

$ Sed-F sedlist sample_one
One 1
Two 2
Three 3
One 1
Two 2
Two 2
Three 3
$

Note that when the "-F" option is called, the marker is not used in the source file or in the command line. Script files, also known as source files, are of great value for complex commands that may cause errors during repeated operations and running from command lines. It is much easier to edit the source file and modify a character than to re-enter a project with multiple lines in the command line.

Limit rows

By default, the editor displays each line entered in the stream editor. By default, each line entered in the stream editor is edited. This can be modified by specifying constraints before issuing the command. For example, replace "1" with "2" in the output lines 5th and 6th of this example file. The command will be:

$ Sed '5, 6 S/1/2/'sample_one
One 1
Two 1
Three 1
One 1
Two 2
Two 2
Three 1
$

In this case, because the row to be modified is specially specified, you do not need to replace the command. Therefore, you can flexibly select the rows to be modified based on the matching criterion (which can be a row number or a matching pattern) (fundamentally restrict the modification ).

Disable display

Sed displays each row from the source file on the screen (or redirects to a file) by default, regardless of whether the row is affected by the editing operation, the "-n" parameter overwrites this operation. "-N" overwrites all displays without displaying any rows, regardless of whether they are modified or not. For example:

$ Sed-n-F sedlist sample_one
$

$ Sed-n-F sedlist sample_one> sample_two
$ Cat sample_two
$

In the first example, nothing is displayed on the screen. In the second example, nothing is modified, so nothing is written to a new file-it is empty at last. Does this deny all the purposes of editing? Why is this useful? It is useful only because the "-n" option can be overwritten by a display command (-P. To illustrate this, it is assumed that the script file has been modified as follows:

$ Cat sedlist
/Two/S/1/2/P
/Three/S/1/3/P
$

The following is the result of running it:

$ Sed-n-F sedlist sample_one
Two 2
Three 3
Two 2
Two 2
Three 3
$

All rows that remain unchanged are not displayed. Only the rows affected by the editing operation are displayed. In this way, you can retrieve only these rows, modify them, and put them in a separate file:

$ Sed-n-F sedlist sample_one> sample_two
$

$ Cat sample_two
Two 2
Three 3
Two 2
Two 2
Three 3
$

Another way to use it is to display only a certain number of rows. For example, only 2-6 rows are displayed without any other modifications:

$ Sed-n'2, 6p' sample_one
Two 1
Three 1
One 1
Two 1
Two 1
$

All other rows are ignored and only 2-6 rows are displayed as output. This is an outstanding feature and cannot be easily implemented by any other tool. Head will display the top of a file, while tail will display the bottom of a file, but SED allows you to retrieve any desired content from any location.

Delete row

Replacing another value with one value is far from the unique function that the stream editor can perform. It also has many potential features. In my opinion, the second most commonly used feature is Delete. The deletion method is the same as the replacement method, but it only deletes the specified row (if you want to delete a word rather than a row, do not consider deleting it, instead, we should consider replacing it with an empty content-S/CAT //).

The syntax of this command is:

'{What to find} d'

Delete all rows containing "two" from the sample_one file:

$ Sed '/Two/d' sample_one
One 1
Three 1
One 1
Three 1
$

Remove the first three lines from the display, regardless of what they are:

$ Sed '1, 3 d' sample_one
One 1
Two 1
Two 1
Three 1
$

Only the remaining rows are displayed. The first three rows are not displayed on the screen. For stream editors, when they involve global expressions, especially when applied to delete operations, there are several points to remember:

The upper triangle (^) indicates the beginning of a line. Therefore, if "two" is the first three characters of the line

Sed '/^ Two/d' sample_one

Only this row will be deleted.
The dollar sign ($) indicates the end of a file or the end of a row. Therefore, if "two" is the last three characters of the row

Sed '/two $/d' sample_one

Only this row will be deleted.

Result of combining the two:

Sed '/^ $/d' {filename}

Delete all blank lines in the file. For example, the following command replaces "1" with "2", "1" with "3", and deletes all trailing empty lines in the file:

$ Sed '/Two/S/1/2/;/three/S/1/3/;/^ $/d' sample_one
One 1
Two 1
Three 1
One 1
Two 2
Two 2
Three 1
$

Its common purpose is to delete a title. The following command deletes all lines in the file from the first line to the first blank line:

Sed '1,/^ $/d' {filename}

Add and insert text

You can use the SED and "A" options to add text to the end of a file. The implementation method is as follows:

$ Sed '$
> This is where we stop
> The test 'sample_one
One 1
Two 1
Three 1
One 1
Two 1
Two 1
Three 1
This is where we stop
The test
$

In this command, the dollar sign ($) indicates that the text will be added to the end of the file. A backslash () is required, indicating that a carriage return is inserted. If they are omitted, an error is returned, indicating that the command is out of order. You must use a backslash wherever you want to press Enter.

The command changes:

$ Sed '3a
> This is where we stop
> The test 'sample_one
One 1
Two 1
Three 1
This is where we stop
The test
One 1
Two 1
Two 1
Three 1
$

Add the text to 3rd rows. Like almost all editors, you can select insert instead of add (if you want ). The difference between the two is that after the specified row is added, the insert starts from the specified row. When you use insert instead of ADD, you only need to use "I" instead of "A", as shown below:

$ Sed '3i
> This is where we stop
> The test 'sample_one
One 1
Two 1
This is where we stop
The test
Three 1
One 1
Two 1
Two 1
Three 1
$

The new text appears in the middle of the output, and the processing usually continues after the specified operation is executed.

Read/write files

The redirection output function has been demonstrated, but it should be noted that the file can be read and written synchronously during the running of the editing command. For example, execute replace and write 1-3 rows to the file named sample_three:

$ Sed'
& Gt;/Two/S/1/2/
>/Three/S/1/3/
> 1, 3 W sample_three 'sample_one
One 1
Two 2
Three 3
One 1
Two 2
Two 2
Three 3
$

$ Cat sample_three
One 1
Two 2
Three 3
$

Because "" is specified for the W (write) command, only the specified row is written to the new file. No matter which rows are written, all rows are displayed in the default output.

Modify command

In addition to replacing a project, you can also change a row from one value to another. Remember that replacement is performed on characters one by one, and the modification function is similar to deletion, which affects the entire line:

$ Sed '/Two/C
> We are no longer using two 'sample_one
One 1
We are no longer using two
Three 1
One 1
We are no longer using two
We are no longer using two
Three 1
$

The modification command works in a similar way to the replacement, but is more extensive-replace a project with another project, regardless of the character content or context. To put it bluntly, only the character "1" is replaced by the character "2" when replaced, and the original line is modified when used. In both cases, the matching conditions to be searched are only "two ".

Modify all ......

For most sed commands, describe in detail how to modify various functions. The exclamation point can be used to modify any part except the specified position-completely different from the default operation.

For example, to delete all rows containing the word "two", perform the following operations:

$ Sed '/Two/d' sample_one
One 1
Three 1
One 1
Three 1
$

To delete all rows except the row containing the word "two", the syntax is changed:

$ Sed '/Two /! D 'sample_one
Two 1
Two 1
Two 1
$

If you have a file that contains a series of projects and want to perform an operation on each project in the file, first perform a smart scan on those projects and consider what to do. To make things easier, you can combine sed with any iteration routine (for, while, until) for this purpose.

For example, assume that you have a file named "Animals", which contains the following items:

Pig
Horse
Elephant
Cow
Dog
Cat

You want to run the following routine:

# McD. ksh
For I in $ *
Do
Echo Old McDonald had a $ I
Echo E-I, E-I-O
Done

The result is that each row is displayed at the end of "Old McDonald has. Although this is true for most of these projects, for the "elephant" project, it has syntax errors because the result should be "an elephant" instead of "A elephant ". With SED, you can check this syntax error in the output from the shell file, and immediately correct them by creating a command file:

# Sublist
/A/S/A//
/A e/S/A//
/A I/S/A//
/A o/s/A//
/A u/S/A//

Then, execute the following process:

$ Sh McD. ksh 'cat animals '| sed-F sublist

Now, after running the MCD script, sed searches for a single letter A (space, "A", space) in the output followed by any position of a vowel. If this position exists, it will change the sequence to space, "an", space. In this way, the problem is corrected before it is displayed on the screen, and the editors everywhere can easily fall asleep at night. The result is:

Old McDonald had a pig
E-I, E-I-O
Old McDonald had a horse
E-I, E-I-O
Old McDonald had an elephant
E-I, E-I-O
Old McDonald had a cow
E-I, E-I-O
Old McDonald had a dog
E-I, E-I-O
Old McDonald had a cat
E-I, E-I-O

Exit early

Sed reads the entire file by default and stops only when it reaches the end. However, you can use the exit command to stop processing in advance. Only one exit command can be specified, and the process continues until the exit command is called.

For example, only Replace the first five lines of the file and exit:

$ Sed'
& Gt;/Two/S/1/2/
>/Three/S/1/3/
> 5q 'sample_one
One 1
Two 2
Three 3
One 1
Two 2
$

The project before exiting the command can be a line number (as shown above) or a search/match command:

$ Sed'
& Gt;/Two/S/1/2/
>/Three/S/1/3/
>/Three/q'sample_one
One 1
Two 2
Three 3
$

You can also use the exit command to view more than a certain number of lines, and add more functions than the head. For example, the head command allows you to specify the number of rows before a file-the default number is 10, but any number from 1 to 99 can be used. If you want to view the first 110 rows of a file, you cannot use head to achieve this goal, but use sed:

Sed 110q filename

Handling Problems

When using SED, it is important to remember how it works. It is used to read a row, execute all the tasks that it knows to execute on the row, and then continue to process the next row. Each line is affected by the specified editing command.

If your operation sequence is not fully considered, this may be troublesome. For example, assume that you need to change all "two" projects to "three", and then change all "three" to "four ":

$ Sed'
>/Two/S/two/three/
>/Three/S/three/four/'sample_one
One 1
Four 1
Four 1
One 1
Four 1
Four 1
Four 1
$

The "two" originally read is changed to "three ". Then it meets the criteria created for the next edit and becomes "four ". The final result is not the desired result-there are no projects except "four", but there should be "three" and "four ".

When performing such an operation, you must carefully specify the operation method and arrange them in a certain order so that operations do not affect each other. For example:

$ Sed'
>/Three/S/three/four/
>/Two/S/two/three/'sample_one
One 1
Three 1
Four 1
One 1
Three 1
Three 1
Four 1
$

This is effective because the "three" value is modified before "two" is changed to "three.

Tags and comments

Tags can be placed in the SED script file, so that once the file becomes large, it is easier to describe what is happening. There are various commands related to these tags, including:
Next step

Visit and add a Linux Technical Center to favorites

Read the books sed & awk, 2nd edition (O 'Reilly & Associates Press) by Dale Dougherty and Arnold Robbins ).

: Colon indicates a tag name. For example:

: Here

Labels starting with colons can be processed by the "B" and "T" commands.

B {label} acts as the "Goto" Statement and sends the processing to the label with a colon. For example,

B here

Send processing to rows

: Here

If no label is specified after B, the process is transferred to the end of the script file.

T {label} is transferred to this label as long as the replacement operation is performed since the last input line or after the "T" command is executed. Like "B", if no calibration signature is provided, the process is transferred to the end of the script file.

# As the first character of a line, the entire line is treated as a comment. The comment line is different from the label. You cannot use the B or t command to convert it to the comment line.