Sed single-line Script Quick Reference (stream editor) 1th/2 page _linux Shell

sed (stream editor)

Function Description: Use script to process text files.
Syntax: sed [-hnv][-e<script>][-f<script file >][text file]
Additional Note: SED can process and edit text files according to script instructions.
Parameters:
-e<script> or--expression=<script> the input text file with the script specified in the option.
-f<script file > or--file=<script file > process the input text file with the script file specified in the option.
-h or--help display Help.
-N or--quiet or--silent displays only the results of the script.

-V or--version display version information.

SED working principle:

SED is a non-interactive flow editor. Non-interactive refers to the use of sed to edit the text only at the command line, and then to view the output on the screen, and the so-called flow editor, which means that sed reads only one line at a time from a file (or input) and then handles the specified row. and output the results to the screen (unless you cancel the screen output without explicitly using the Print command), and then read the next line. The entire file is processed and output line by line, like running water.

Let's look at the working process of SED.

SED is not directly processed on the original input, but first put the read line into the buffer, the contents of the buffer processing, the processing is not written back to the original file (unless with the shell output redirection to save the results), but directly output to the screen. The SED maintains two buffers during operation, one is the active "mode space" and the other is the auxiliary "staging buffer (holding spaces)". In general, whenever you run sed,sed first loads the first pack into the mode space, processes it, outputs it to the screen, and then replaces the second pack into the mode space with the original contents of the mode space, then processes it, and so on.

In general, the staging buffer is not available, but there are special commands to exchange data between the mode space and the staging buffer, which is described later in this article. Since all operations of SED on text are done in a buffer, they do not cause any damage to the original file.

SED command format

The SED command format is as follows:

sed [-options] [' Commands '] filename

Where command is a sed command, the SED command must be enclosed in a pair of single quotes to avoid being interpreted by the shell, in the following format:

[Address-range] [Sed-command] or
[Pattern-to-match] [Sed-command]

Address-range refers to the range of rows to be processed, also called address ranges; Pattern-to-match is a pattern to match, a regular expression, and Sed-command is a sed command that is used to handle the specified row. The following is a simple example:

Sed–n ' 1,3p ' students

This command prints the 1th to 3rd line in the file students to the screen. Note that there is no space between the address range and the SED command, and if you add a space, SED ignores it. Parameter-n is used to cancel the default output. By default, SED reads a row to the pattern space every time, regardless of whether it is processed or not, to output the contents of the pattern space to the screen before reading the next line. Parameter-n can be used to cancel this default output and output the specified row to the screen only when the user uses the command p. If you do not use parameter-N and then execute the p command on the specified row, the lines are printed two times.

The address range can be a number that represents a line number, or a comma-delimited range of two digits, including these two lines. A range can be a number, a regular expression, or a combination of both.

Pattern-to-match is a pattern to match, and SED will perform sed-command on all matching rows. In fact, the pattern-to-match here can also be considered an address, which is the line number of all rows that match the specified pattern. So the format of SED can be summed up as a:

sed [-options] ' [Address-range][sed-command] ' filename

Text interval:

# Add a blank line after each line
Sed G
# Remove all empty rows and add a blank line after each line.
# This will have a blank line after each line in the output text.
Sed '/^$/d; G
# Add two blank lines after each line
Sed ' G; G
# Delete all empty rows generated by the first script (that is, delete all even odd rows)
Sed ' n;d '
# Inserts a blank line before the line that matches the style ' regex '
Sed '/regex/{x;p;x;} '

# Inserts a blank line after the line that matches the style ' regex '
Sed '/regex/g '
# Inserts a blank line before and after the line that matches the style ' regex '
Sed '/regex/{x;p;x; G;} '

Number:

# number each line in the file (simple left alignment). "Tabs" are used here
# (tab, see the description of ' \ t ' usage at the end of this article) instead of the empty cells alignment edge.
sed = filename | Sed ' n;s/\n/\t/'
# All row numbers in the file (line number on left [top], text right [left] end aligned).
sed = filename | Sed ' N; s/^//; s/*\ (. \{6,\}\) \n/\1/'
# numbers all the rows in the file, but only the line numbers for the nonblank lines.
Sed '/./= ' filename | Sed '/./n; s/\n//'
# Count rows (simulate "wc-l")
Sed-n ' $= '

Text Conversion and substitution:

# UNIX Environment: a new line character (CR/LF) that converts DOS is in UNIX format.


Sed ' s/.$//' # assuming all rows end with CR/LF


Sed ' s/^m$//' # in Bash/tcsh, will press ctrl-m instead press Ctrl-v


Sed ' s/\x0d$//' # ssed, gsed 3.02.80, and later


# UNIX Environment: A new Line (LF) for converting UNIX is a DOS format.


Sed "s/$/' echo-e\\\r '/"# The commands used under the Ksh


Sed ' s/$ '/' echo\\\r '/"# The commands that are used under bash


Sed "s/$/' echo"\\\r '/"# The commands used under the ZSH


Sed ' s/$/\r/' # gsed 3.02.80 and later

# DOS Environment: convert UNIX New line character (LF) to DOS format.


Sed "s/$//" # method 1


Sed-n P # Method 2


# DOS Environment: Convert DOS New line character (CR/LF) to UNIX format.


# The following script is only valid for unxutils sed 4.0.7 and later. To identify the unxutils version of the


# sed is available through its unique "--text" option. You can use the Help option ("--help") to see


# there is no "--text" item to determine whether the Unxutils version is used. Other DOS


# version of SED is not able to do this conversion. However, you can use "TR" to implement this transformation.


Sed "s/\r//" infile &gt;outfile # unxutils sed v4.0.7 or later


tr-d \ r &lt;infile &gt;outfile # GNU TR 1.22 or later


# Delete the leading "white space characters" (spaces, tabs) in each line


# Make it left aligned


Sed ' s/^[\t]*//' # See the description of ' \ t ' usage at the end of this article


# delete "White space characters" (spaces, tabs) trailing in each line


Sed ' s/[\t]*$//' # See the description of ' \ t ' usage at the end of this article

# Delete the leading and trailing whitespace characters in each row
Sed ' s/^[\t]*//;s/[\t]*$//'
# Insert 5 spaces at the beginning of each line (position to move the full text to the right 5 characters)
Sed ' s/^//'
# 79 characters wide, align all text to the right
SED-E: A-e ' s/^.\{1,78\}$/&/;ta ' # 78 characters plus the last space
# is 79 characters wide, so that all text is centered. In Method 1, in order to center the text in front of each line
# The head and the back are filled with spaces. In Method 2, you only fill in the front of the text when you center the text
# spaces, and eventually half of these spaces will be deleted. In addition, no spaces are filled behind each line.

SED-E: A-e ' s/^.\{1,77\}$/&amp;/;ta ' # method 1


SED-E: A-e ' s/^.\{1,77\}$/&amp;/;ta '-e ' s/\ (*\) \1/\1/' # method 2


# Find the string ' foo ' in each row and replace the found ' foo ' with ' Bar '


Sed ' s/foo/bar/' only replaces the first "foo" in each row String


Sed ' S/FOO/BAR/4 ' only replaces the fourth "Foo" word in each row String


The sed ' s/foo/bar/g ' replaces all "foo" in each row with the " Bar


Sed ' s/\ (. *\) foo\ (. *foo\)/\1bar\2/' # Replace the penultimate ' foo '


Sed ' s/\ (. *\) foo/\1bar/' # replaces the last "foo"


# replace ' foo ' with ' bar ' if string ' Baz ' appears in rows only


Sed '/baz/s/foo/bar/g '


# replace "foo" with "bar" and only if the string "Baz" does not appear in the row


Sed '/baz/!s/foo/bar/g '

# whether it's "scarlet" or "Ruby" or "puce", replace it with "red."


Sed ' s/scarlet/red/g;s/ruby/red/g;s/puce/red/g ' #对多数的sed都有效


Gsed ' s/scarlet\|ruby\|puce/red/g ' only works for GNU SED


# Invert all rows, the first line becomes the last line, and so on (simulate "TAC").


# for some reason, hhsed v1.5 deletes empty rows from the file when you use the following command


Sed ' 1! G;h;$!d ' # method 1


Sed-n ' 1! G;h $p ' # method 2


# The characters in the line are sorted in reverse order, the first word becomes the last word, ... (Simulate "rev")


Sed '/\n/! G;s/\ (. \) \ (. *\n\)/&amp;\2\1/;//d;s/.//'


# Connect every two lines into one line (similar to "paste")


Sed ' $! n;s/\n//'

# If the current line ends with a backslash ' \ ', the next line is to the end of the current line


# and remove the backslash from the end of the line


SED-E: A-e '/\\$/n; s/\\\n//; Ta


# If the current line begins with an equal sign, the current line is at the end of the previous line


# and replace the original wardrobe with a single space ' = '


SED-E: A-e ' $! n;s/\n=//;ta '-e ' p;d '


# adds a comma-delimited symbol to the number string, changing "1234567" to "1,234,567"


Gsed ': A;s/\b[0-9]\{3\}\&gt;/,&amp;/;ta ' # GNU SED


SED-E: A-e ' s/\ (. *[0-9]\) \ ([0-9]\{3\}\)/\1,\2/;ta ' # other SED


# adds a comma separator to a value with a decimal point and minus sign (GNU sed)


Gsed-r ': a;s/(^|[ ^0-9.]) ([0-9]+) ([0-9]{3})/\1\2,\3/g;ta '


# Add a blank line after every 5 lines (add a blank line after the line 5,10,15,20, etc.)


Gsed ' 0~5g ' only works for GNU SED


Sed ' n;n;n;n;                                                                                  G; ' # other SED

To selectively display a specific line:

# Display the first 10 rows in the file (simulate the behavior of the "head")
Sed 10q
# Display the first line in the file (simulate the "head-1" command)
SED q
# display the last 10 lines in the file (simulate "tail")
SED-E: A-e ' $q; n;11, $D Ba '

# display the last 2 lines in the file (simulate the "tail-2" command)


Sed ' $! n;$! D


# display the last line in the file (simulate "tail-1")


Sed ' $!d ' # method 1


Sed-n ' $p ' # method 2


# Display the penultimate line in the file


Sed-e ' $! {h;d;} ' -E x # When there is only one row in the file, enter a blank line


Sed-e ' 1{$q;} '-E ' $! {h;d;} ' -E x # Displays the row when there is only one row in the file


Sed-e ' 1{$d;} '-E ' $! {h;d;} ' -E x # When there is only one row in the file, no output


# only rows matching regular expressions are displayed (simulate "grep")


Sed-n '/regexp/p ' # method 1


Sed '/regexp/!d ' # method 2


# Show only rows that do not match regular expressions (simulate "grep-v")


Sed-n '/regexp/!p ' # method 1, corresponding to the previous command


Sed '/regexp/d ' # method 2, similar to syntax

# finds ' regexp ' and displays the previous line of matching rows, but does not display matching rows
Sed-n '/regexp/{g;1!p;}; H
# find ' regexp ' and display the next line of matching rows, but do not show matching rows
Sed-n '/regexp/{n;p;} '
# Displays the line that contains ' regexp ' and its front and back lines, plus the ' regexp ' before the first line
# line number of rows (similar to "GREP-A1-B1")
Sed-n-E '/regexp/{=;x;1!p;g;$! n;p;d;} ' -E H
# Displays lines containing "AAA", "BBB" or "CCC" (in any order)

Sed '/aaa/!d/bbb/!d/ccc/!d '                         &NBS P                       # The order of the strings does not affect the result
# displays lines containing "AAA", "BBB" and "CCC" (Fixed time
sed '/aaa.*bbb.*ccc/!d '
# Displays rows containing "AAA" "BBB" or "CCC" (Analog "Egrep")
Sed-e '/aaa/b '-e '/bbb/b '-e '/ccc/b '-e d& nbsp                           # most sed
gsed '/aaa\| bbb\| Ccc/!d ' # is valid for GNU SED
# displays paragraphs with "AAA" (separated by Blank lines)
# hhsed v1.5 must be in ' X; ' Add "G;" and the next 3 scripts are like this
Sed-e '/./{h;$!d;} '-E ' x;/aaa/!d; '
# Displays paragraphs containing "AAA", "BBB" and "CCC" three strings (in any order)
Sed-e '/./{h;$!d} '-e ' x;/aaa/!d;/bbb/!d;/ccc/!d '
# display contains "AAA", "BBB", The paragraphs of any of the three "CCC" (in any order)
Sed-e '/./{h;$!d;} ' E ' x;/aaa/b '-e '/bbb/b '-e '/ccc/b '-e D
gsed '/./{h;$!d;; x;/aaa\| bbb\| Ccc/b;d '                               # only for GNU S Ed is valid

# Displays rows containing 65 or more characters
Sed-n '/^.\{65\}/p '
# Displays rows containing 65 characters
Sed-n '/^.\{65\}/!p '                                           &NB Sp                       # method 1, corresponding to the above script
sed '/^.\{65\}/d '   &N Bsp                                   &NB Sp                                 # method 2, simpler one The method
# of the point displays part of the text-from the start of the line containing the regular expression to the end of the last line
Sed-n '/regexp/, $p '
# Display part of the text--Specify line number range (from 8th to 12th lines, with 8 and 12 rows)

Sed-n ' 8,12p ' # method 1


Sed ' 8,12!d ' # method 2


# Show Line 52nd


Sed-n ' 52p ' # method 1


Sed ' 52!d ' # method 2


Sed ' 52q;d ' # Method 3, more efficient when dealing with large files


# starting at line 3rd, every 7 lines are displayed once


Gsed-n ' 3~7p ' only works for GNU SED


Sed-n ' 3,${p;n;n;n;n;n;n;} ' # other SED


# Displays text between two regular expressions (included)


Sed-n '/iowa/,/montana/p ' # case sensitive

To selectively delete a specific line:

# Displays the entire document, except for the contents of two regular expressions
Sed '/iowa/,/montana/d '
# Delete adjacent duplicate rows in a file (simulate "Uniq")
# only the first row in the duplicate row is preserved, and the other rows are deleted
Sed ' $! N /^\ (. *\) \n\1$/! P D
# deletes duplicate rows in the file, regardless of whether they are adjacent or not. Note the cache that hold space can support
# size, or use GNU sed.
Sed-n ' G; s/\n/&&/; /^\ ([-~]*\n\). *\n\1/d; s/\n//; H P
# Delete all rows except duplicate rows (simulate "uniq-d")
Sed ' $! N S/^\ (. *\) \n\1$/\1/; T D

# Delete the first 10 lines in the file


Sed ' 1,10d '


# Delete the last row in the file


Sed ' $d '


# Delete the last two lines in the file


Sed ' n;$! p;$! D; $d '


# Delete the last 10 lines in the file


SED-E: A-e ' $d; N;2,10ba '-E ' p;d ' # method 1


Sed-n-e:a-E ' 1,10! P N;d;}; N;ba ' # method 2


# Delete multiple rows of 8


Gsed ' 0~8d ' only works for GNU SED


Sed ' n;n;n;n;n;n;n;d; ' # other SED


# Delete the line that matches the style


Sed '/pattern/d ' deletes lines containing pattern. Of course pattern


# can be replaced with any valid regular expression

# Deletes all empty rows in the file (with the grep. "Same effect"
sed '/^$/d ' # method 1
sed '/./!d ' # method 2
# preserves only the first row of multiple contiguous empty rows. and delete the empty lines at the top and bottom of the file.
# (simulate "cat-s")
sed '/./,/^$/!d '                                                           &NB Sp                     #方法1, delete the empty line at the top of the file, allow the tail to keep an empty line
sed '/^$/n;/\n$/d '   ;                                   &NB Sp                                   #方法 2, allow the top to keep a blank line, the tail does not leave blank line

# keep only the first two rows of multiple contiguous empty rows.
Sed '/^$/n;/\n$/n;//d '
# Delete all blank lines at the top of the file
Sed '/./,$!d '
# Delete all empty lines at the end of the file
SED-E: A-e '/^\n*$/{$d; N;ba ' e '} ' # is valid for all SED
SED-E: A-e '/^\n*$/n;/\n$/ba ' # ditto, but only for gsed 3.02.* valid
# Delete the last line of each paragraph
Sed-n '/^$/{p;h;};/. /{x;/./p;} '

Special applications:

# removes the Nroff tag in the man page (man pages). Under the Unix System v or bash shell


# You may need to add the-e option with the ' echo ' command.


Sed "s/. ' Echo\\\b '//g"# The outer brackets are required (Unix environment)


Sed ' s/.^h//g ' # in Bash or tcsh, press ctrl-v and press Ctrl- H


Sed ' s/.\x08//g ' # sed 1.5,gnu sed,ssed used in hexadecimal tables Showing method


# extract newsgroup or e-mail headers


Sed '/^$/q ' # Delete all content after the first line of blank lines


# Extract the body part of a newsgroup or e-mail


Sed ' 1,/^$/d ' # delete everything before the first line of empty rows


# extract "Subject" from the message header (title bar field) and remove the beginning of "Subject:"


Sed '/^subject: */!d; S///;q '

# Get reply address from message header
sed '/^reply-to:/q/^from:/h;/./d;g;q '
# get mail address. Further shave the part of the non-email
# address based on the line of headers generated by the previous script. (see previous script)
sed ' s/* (. *)/; s/>.*//; s/.*[:<] *//'
# Add an angle bracket and a space (reference information) to the beginning of each line
sed ' s/^/>/'
# Remove the angle brackets and spaces at the beginning of each line (dereference)
sed ' s/^>//'
# Remove most HTML tags (including cross row tags)
sed-e: A-e ' s/<[^>]*>//g;/</n;/ /ba '
# will be decoded into uuencode files with multiple volumes. Remove file header information, leaving only the Uuencode encoding portion. The
# file must be passed to sed in a specific order. The first version of the script below can be entered directly at the command line;
# The second version can be placed in a shell script with execute permissions. (Modified by a
# script from Rahul Dhesi.) )

Sed '/^end/,/^begin/d ' file1 file2 ... filex | UUDecode # vers. 1
Sed '/^end/,/^begin/d ' "$@" | UUDecode # vers. 2
# The paragraphs in the file are sorted in alphabetical order. Paragraphs are separated by blank lines (one or more lines). GNU SED uses
# character ' \v ' to represent a vertical tab, which is used as a placeholder for line breaks--you can, of course,
# replace it with other characters not used in the file.
Sed '/./{h;d; X;s/\n/={nl}=/g ' File | Sort | Sed ' 1s/={nl}=//;s/={nl}=/\n/g '
gsed '/./{h;d};x;y/\n/\v/' file | Sort | Sed ' 1s/\v//;y/\v/\n/'
# compress each individually. TXT file, compressed after the original file is deleted and will be compressed. ZIP file
# is named the same as the original name (only the extension is different). (DOS Environment: "Dir/b"

# displays a filename without a path.


echo @echo off &gt;zipup.bat


dir/b *.txt | Sed "s/^\ (. *\) \. Txt/pkzip-mo \1 \1.txt/"&gt;&gt;zipup.bat


Use sed:sed to accept one or more edit commands, and then apply the commands sequentially after each read.


When the first line of input is read, SED applies all the commands to it and then outputs the results. Then read the second line of input and apply all the commands to it ... and repeat the process. In the previous example, SED was entered by a standard input device (that is, a command interpreter, usually in the form of a pipe input). When one or more file names are given as arguments on the command line, these files replace the standard input devices as input to sed. The output of SED will be sent to the standard output (monitor). So:


Cat FileName | Sed ' 10q ' # using pipe input


Sed ' 10q ' filename # same effect, but not using pipe input


Sed ' 10q ' filename &gt; NewFile # Transfer output (redirect) to disk

For instructions on how to use the SED command, including how to use these commands from a script file rather than from the command line, see the second edition of Sed & awk, author Dale Dougherty and Arnold Robbins (O ' reilly,1997;http:/ /www.ora.com), "UNIX Text processing", author Dale Dougherty and Tim O ' Reilly (Hayden books,1987) or a tutorial written by Mike Arst--The name of the package is " U-sedit2. ZIP "(found on many sites). To discover the potential of SED, you must have an adequate understanding of the regular expression. Regular expression data can be seen in "Mastering Regular Expressions" author Jeffrey Friedl (O ' Reilly 1997).
The man page ("Man") provided by the UNIX system will also help (try these commands "man sed", "man regexp", or look at the part about regular expressions in "Man Ed", but the information provided by the manual is more "abstract"-and it has been criticized. However, it was not intended to teach beginners how to use SED or regular expressions, but to provide some textual references to those familiar with these tools.

Parenthesis Syntax: The preceding example uses single quotes (' ... ') rather than double quotes ("...") for the SED command, which is typically used on UNIX platforms. In single quotes, the UNIX shell (command interpreter) does not interpret and execute the dollar character ($) and the post quotation mark (' ... '). In double quotes the dollar character is expanded to the value of a variable or parameter, and the command in the following quotation marks is executed and replaces the contents of the following quotation marks with the output result. The use of an exclamation point (!) in "CSH" and its derived shells is preceded by an escape backslash (like this: \!) to ensure that the example used above is working properly (including using single quotes). The DOS version of SED uses double quotes ("...") instead of quotes to circle the command.
' \ t ' usage: To keep this article simple, we use ' \ t ' in the script to represent a tab character. But now most versions of SED don't recognize the shorthand for ' \ t ', so when you enter tabs for a script on the command line, you should simply press the TAB key to enter the tab instead of ' t '. The following tool software supports ' \ t ' as a regular expression character to represent tabs: awk, Perl, hhsed, Sedmod, and GNU sed v3.02.80.

Different versions of SED: the sed will be different between different versions, and you can imagine that there will be grammatical differences between them. Specifically, most of them do not support the use of tags (: name) or branch commands (b,t) in the middle of editing commands, unless they are placed at the end of those. In this document, we try to use a more portable syntax so that most versions of SED users will have access to these scripts. But the GNU version of SED allows for more concise syntax. Imagine the mood when the reader sees a long command:
Sed-e '/aaa/b '-e '/bbb/b '-e '/ccc/b '-e D
The good news is that GNU SED can make the order more compact:
Sed '/aaa/b;/bbb/b;/ccc/b;d ' can even be written
Sed '/aaa\| bbb\| Ccc/b;d '
Also, please note that although many versions of SED accept such "/one/s/re1/re2/" with empty before ' s '
Lattice commands, but some of these versions do not accept such commands: "/one/! S/re1/re2/". Then
Just take the middle of the space to remove the line.

Speed optimization: When for some reason (such as a large input file, processor or hard drive slow) need to improve

When the command executes speed, consider adding an address expression to the substitution command ("s/.../.../") before you can


Improve speed. For example:


Sed ' s/foo/bar/g ' filename # Standard replacement command


Sed '/foo/s/foo/bar/g ' filename # faster


Sed '/foo/s//bar/g ' filename # abbreviated form


You can use "Q" in your script when you want to display only the previous section of the file or if you need to delete the following content


Command (Exit command). This can save a lot of time when working with large files. So:


Sed-n ' 45,50p ' filename # shows line 45th to 50th


Sed-n ' 51q;45,50p ' filename # like, but much faster

If you have other single-line scripts that you would like to share or you have found the error in this document, please email the author of this document (Eric pement). Please remember to provide the SED version you are using, the operating system that the SED runs, and the appropriate description of the problem. The Single-line script referred to in this article refers to a SED script with a command line length of 65 characters or less than 65 1)

1: In most cases, the SED script can be written as a single line (via the '-e ' option and '; '), no matter how long. )--As long as the command interpreter supports it, the Single-line script here is limited in length, in addition to being able to write a line. Because the meaning of these single-line scripts is not that they appear in a single line. Instead, it makes sense for users to easily use these compact scripts on the command line.

Current 1/2 page 12 Next read the full text