Linux Command analysis-sed (2)

Sed single-line script Quick Reference (UNIX stream editor)

English title: Useful one-line scripts for sed (UNIX stream editor)
Original Title: handy one-liners for sed (UNIX stream editor)

Eric pement-Email: pemente [at] northpark [Dot] Edu Version 5.5
Translator: Joe Hong-Email: hq00e [at] 126 [Dot] com

Find the latest (English) version of this document:
Http://sed.sourceforge.net/sed1line.txt
Http://www.pement.org/sed/sed1line.txt

Other language versions:
Chinese-http://sed.sourceforge.net/sed1line_zh-CN.html
Czech-http://sed.sourceforge.net/sed1line_cz.html
Lotus http://sed.sourceforge.net/sed1line_nl.html
French http://sed.sourceforge.net/sed1line_fr.html
Germany-http://sed.sourceforge.net/sed1line_de.html

Portuguese-http://sed.sourceforge.net/sed1line_pt-BR.html

Text interval:
--------

# Add an empty row after each row
Sed g

# Delete all original empty rows and add an empty row after each row.
# In this way, each row in the output text is followed by an empty row.
Sed '/^ $/d; G'

# Add two blank rows after each row
Sed 'G; G'

# Delete all empty rows generated by the first script (that is, delete all even rows)
Sed 'n'; D'

# Insert an empty row before matching the row with the style "RegEx"
Sed '/RegEx/{X; P; X ;}'

# Insert an empty row after matching the row with the style "RegEx"
Sed '/RegEx/G'

# Insert an empty row before and after the row matching the style "RegEx"
Sed '/RegEx/{X; P; X; G ;}'

No:
--------

# Number each row in the file (simple left alignment ). The "tab" is used here"
# (Tab, see the '\ t' usage description at the end of this article) instead of spaces to align edges.
Sed = filename | sed 'n'; S/\ n/\ t /'

# Number of all rows in the file (the row number is left and the right of the text is aligned ).
Sed = filename | sed 'n'; S/^ //; S/* \ (. \ {6, \} \) \ n/\ 1 /'

# Number of all rows in the file, but only the row numbers of non-blank rows are displayed.
Sed '/./= 'filename| sed'/./N; S/\ N //'

# Calculate the number of rows (simulate "WC-L ")
Sed-n' $ ='

Text conversion and substitution:
--------

# Unix environment: the new line character (CR/LF) for converting DOS is in UNIX format.
Sed's/. $ // # assume that all rows end with Cr/LF
Sed's/^ m $ // '# In bash/tcsh, change Ctrl-m to Ctrl-V
Sed's/\ x0d $ // '# ssed, gsed 3.02.80, and later

# Unix environment: Convert the Unix newline character (LF) to the DOS format.
Sed "s/$/'echo-e \ R'/" # Command Used in KSh
Sed's/$ '"/'echo \ R'/" # Command Used in bash
Sed "s/$/'echo \ R'/" # Command Used in zsh
Sed's/$/\ r/'# gsed 3.02.80 and later

# DOS environment: Convert UNIX newline character (LF) to DOS format.
Sed "s/$ //" # method 1
Sed-n p # method 2

# DOS environment: Convert the DOS newline character (CR/LF) to Unix format.
# The following script is only valid for unxutils sed 4.0.7 and later versions. To identify the unxutils version
# Sed can use its unique "-- text" option. You can use the help option ("-- Help") to view
# Whether there is a "-- text" item to determine whether the version is unxutils. Other DoS
# The SED version cannot perform this conversion. However, you can use "TR" to achieve this conversion.
Sed "s/\ r //" infile> OUTFILE # unxutils sed v4.0.7 or later
Tr-d \ r <infile> OUTFILE # GNU tr 1.22 or later

# Delete the leading blank characters (spaces and tabs) of each line
# Align left
Sed's/^ [\ t] * // '# refer to the' \ t' usage description at the end of this article.

# Delete the blank characters (spaces and tabs) at the end of each line
Sed's/[\ t] * $ // # refer to the '\ t' usage description at the end of this article.

# Delete leading and trailing blank characters in each row
Sed's/^ [\ t] * //; S/[\ t] * $ //'

# Insert 5 spaces at the beginning of each line (to move the full text to the right five characters)
Sed's/^ //'

# Align all texts right with 79 characters in width
Sed-E: A-E's/^. \ {\} $/&/; TA '#78 characters plus the last space

# Use 79 characters as the width to center all texts. In method 1, to center the text before each row
# The header and the backend header are filled with spaces. In method 2, the text is only filled before the text in the center process.
# Spaces, and half of these spaces will be deleted. In addition, no spaces are filled in the backend of each row.
Sed-E: A-E's/^. \ {, 77 \} $/&/; TA '# method 1
Sed-E: A-E's/^. \ {\} $/&/; TA '-E's/\ (* \) \ 1/\ 1/' # method 2

# Search for the string "foo" in each row and replace the "foo" with "bar"
Sed's/Foo/BAR/'# Only Replace the first "foo" string in each row
Sed's/Foo/BAR/4' # Only Replace the fourth "foo" string in each row
Sed's/Foo/BAR/G' # Replace all "foo" in each row with "bar"
Sed's/\ (. * \) Foo \ (. * Foo \)/\ 1bar \ 2/'# Replace the last and second "foo"
Sed's/\ (. * \) Foo/\ 1bar/'# Replace the last "foo"

# Replace "foo" with "bar" only when the string "Baz" appears in the row"
Sed '/Baz/S/Foo/BAR/G'

# Replace "foo" with "bar", and replace it only when "Baz" is not displayed in the row.
Sed '/Baz /! S/Foo/BAR/G'

# "Red" is used for both "Scarlet", "Ruby", and "puce"
Sed's/scarlet/red/g; S/Ruby/red/g; S/puce/red/G' # effective for most SED
Gsed's/scarlet \ | Ruby \ | puce/red/G' # only valid for GNU SED

# Invert all rows. The first line is the last line, and so on (simulate "TAC ").
# For some reason, hhsed v1.5 deletes empty lines in the file when the following command is used
Sed '1! G; h; $! D' # method 1
Sed-n' 1! G; h; $ P' # method 2

# Sort the characters in the row in reverse order. The first word becomes the last word ,...... (Simulate "Rev ")
Sed '/\ n /! G; S/\ (. \) \ (. * \ n \)/& \ 2 \ 1/; // D; S /.//'

# Concatenate each two rows into one line (similar to "Paste ")
Sed '$! N; S/\ N //'

# If the current row ends with a backslash (\), the next row is added to the end of the current row.
# Remove the backslash at the end of the original line
Sed-E: A-E '/\ $/N; S/\ N //; TA'

# If the current row starts with an equal sign, add the current row to the end of the previous row
# Replace the "=" of the original line header with a single space"
Sed-E: A-E '$! N; S/\ n = //; TA '-e' p; D'

# Add a comma separator for the numeric string and change "1234567" to "1,234,567"
Gsed ': A; S/\ B [0-9] \ {3 \} \>/, &/; TA' # GNU SED
Sed-E: A-E's /\(. * [0-9] \) \ ([0-9] \ {3 \} \)/\ 1, \ 2/; TA '# other SED

# Add a comma separator (gnu sed) for values with decimal points and negative numbers)
Gsed-R': A; S/(^ | [^ 0-9.]) ([0-9] +) ([0-9] {3})/\ 1 \ 2, \ 3/g; TA'

# Add a blank row after each 5 rows (add a blank row after rows 5, 10, 15, 20, and so on)
Gsed '0 ~ 5G '# only valid for GNU SED
Sed 'n'; n; G; '# other SED

Select to display specific rows:
--------

# Display the first 10 lines in the file (simulate the "head" behavior)
Sed 10q

# Display the first line of the file (simulate the "head-1" command)
Sed Q

# Display the last 10 lines in the file (simulate "tail ")
Sed-E: A-E '$ q; n; 11, $ D; Ba'

# Display the last two lines in the file (simulate the "tail-2" command)
Sed '$! N; $! D'

# Display the last line in the file (simulate "tail-1 ")
Sed '$! D' # method 1
Sed-n' $ P' # method 2

# Display the second and last lines in the file
Sed-e '$! {H; D;} '-e x # enter a blank line when there is only one row in the file.
Sed-e '1 {$ q;} '-e' $! {H; D;} '-e x # This row is displayed when there is only one row in the file.
Sed-e '1 {$ D;} '-e' $! {H; D;} '-e x # when there is only one row in the file, no output

# Only display rows matching Regular Expressions (simulate "grep ")
Sed-n'/Regexp/P' # method 1
Sed '/Regexp /! D' # method 2

# Show only the rows that do not match the regular expression (simulate "grep-V ")
Sed-n'/Regexp /! P' # method 1, which corresponds to the preceding command
Sed '/Regexp/d' # method 2, similar syntax

# Search for "Regexp" and display the last line of the matched row, but not the matched row
Sed-n'/Regexp/{g; 1! P;}; H'

# Search for "Regexp" and display the next row of the matching row, but not the matching row
Sed-n'/Regexp/{n; P ;}'

# Display the rows that contain "Regexp" and the front and back rows, and add "Regexp" before the first row
# Line number (similar to "grep-A1-B1 ")
Sed-n-e '/Regexp/{=; X; 1! P; G; $! N; P; D;} '-E h

# Display rows containing "AAA", "BBB", or "CCC" (in any order)
Sed '/AAA /! D;/BBB /! D;/CCC /! D' # The string order does not affect the result

# Display rows containing "AAA", "BBB", and "CCC" (fixed order)
Sed '/AAA. * BBB. * CCC /! D'

# Display rows that contain "AAA" "BBB" or "CCC" (simulate "egrep ")
Sed-e '/AAA/B'-E'/BBB/B '-E'/CCC/B'-e d # majority of SED
Gsed '/AAA \ | BBB \ | CCC /! D' # valid for GNU SED

# Display the section containing "AAA" (separated by blank lines)
# Hhsed v1.5 must add "g;" after "X;". This is the case for the next three scripts.
Sed-e '/./{H; $! D;} '-e' X;/AAA /! D ;'

# Display paragraphs containing "AAA", "BBB", and "CCC" strings (in any order)
Sed-e '/./{H; $! D;} '-e' X;/AAA /! D;/BBB /! D;/CCC /! D'

# Display the section containing any string of "AAA", "BBB", and "CCC" (in any order)
Sed-e '/./{H; $! D;} '-e' X;/AAA/B'-E'/BBB/B '-E'/CCC/B'-e d
Gsed '/./{H; $! D ;}; X;/AAA \ | BBB \ | CCC/B; D' # only valid for GNU SED

# Display rows containing 65 or more characters
Sed-n'/^. \ {65 \}/P'

# Display rows with less than 65 characters
Sed-n'/^. \ {65 \}/! P' # method 1, which corresponds to the above script
Sed '/^. \ {65 \}/d' # method 2, a simpler method

# Display part of the text-starting from the row containing the regular expression to the end of the last row
Sed-n'/Regexp/, $ P'

# Display part of text -- specify the row number range (from 8th to 12th rows, including 8 and 12 rows)
Sed-n'8, 12p' # method 1
Sed '8, 12! D' # method 2

# Display rows 52nd
Sed-N '52p' # method 1
Sed '52! D' # method 2
Sed '52q; D' # method 3, which is more efficient in processing large files

# Display each 7 rows starting from 3rd
Gsed-n' 3 ~ 7p' # only valid for GNU SED
Sed-n' 3, $ {P; n;} '# other SED

# Display the text (inclusive) between two regular expressions)
Sed-n'/Iowa/,/Montana/P' # Case Sensitive Mode

Select to delete a specific row:
--------

# Display the entire document except the content between two regular expressions
Sed '/Iowa/,/Montana/d'

# Delete adjacent duplicate rows in the file (simulate "uniq ")
# Only the first row in the duplicate row is retained, and the other rows are deleted.
Sed '$! N;/^ \ (. * \) \ n \ 1 $ /! P; D'

# Delete duplicate rows in the file, regardless of whether there are adjacent rows. Note: cache supported by hold space
# Size, or use GNU sed.
Sed-n'g; S/\ n/&/;/^ \ ([-~] * \ N \). * \ n \ 1/d; S/\ N //; h; P'

# Delete all rows except duplicate rows (simulate "uniq-d ")
Sed '$! N; S/^ \ (. * \) \ n \ 1 $/\ 1/; t; D'

# Delete the first 10 lines in the file
Sed '1, 10'

# Delete the last row in the file
Sed '$ d'

# Delete the last two lines in the file
Sed 'n'; $! P; $! D; $ d'

# Delete the last 10 lines in the file
Sed-E: A-E '$ D; n; 2, 10ba'-e 'P; D' # method 1
Sed-n-e: A-E '1, 10! {P; n; D ;}; N; Ba' # method 2

# Delete multiple rows of 8
Gsed '0 ~ 8d '# only valid for GNU SED
Sed 'n'; n; D; '# other SED

# Deleting matching rows
Sed '/pattern/d' # deletes rows containing pattern. Of course, Pattern
# You can replace it with any valid regular expression.

# Delete all empty lines in the file (same effect as "grep)
Sed '/^ $/d' # method 1
Sed '/./! D' # method 2

# Only the first row of multiple adjacent empty rows is retained. Delete the empty lines at the top and end of the file.
# (Simulate "cat-S ")
Sed '/./,/^ $ /! D' # Method 1: Delete the empty lines at the top of the file and allow the trailing lines to be retained
Sed '/^ $/N;/\ N $/d' # method 2, allowing an empty row to be retained at the top, leaving no blank lines at the end

# Only the first two rows of multiple adjacent empty rows are retained.
Sed '/^ $/N;/\ N $/N; // d'

# Delete all blank lines at the top of the file
Sed '/./, $! D'

# Delete all empty lines at the end of the file
Sed-E: A-E '/^ \ n * $/{$ D; n; BA'-e'} '# valid for all SED
Sed-E: A-E '/^ \ n * $/N;/\ N $/Ba' # Same as above, but only valid for gsed 3. 02. *

# Delete the last line of each paragraph
Sed-n'/^ $/{P; H ;};/./{X;/./P ;}'

Special applications:
--------

# Remove the nroff mark from the man page. In UNIX System V or bash shell
# You may need to add the-e option when using the 'echo 'command.
Sed "s/. 'echo \ B '// G" # The outer double brackets are required (UNIX environment)
Sed's/. ^ h // G' # In bash or tcsh, press Ctrl-V and then press Ctrl-H
Sed's/. \ x08/G' # sed 1.5, gnu sed, and ssed hexadecimal Representation

# Extract the header of a news group or email
Sed '/^ $/Q' # delete all content after the first empty line

# Extract the body of a newsgroup or email
Sed '1,/^ $/d' # delete all content before the first empty line

# Extract "subject" (title bar field) from the mail header and remove the "Subject:" at the beginning
Sed '/^ subject :*/! D; S //; Q'

# Obtain the reply address from the email header
Sed '/^ reply-to:/Q;/^ From:/h;/./d; G; Q'

# Obtain the email address. Based on the line of mail headers generated by the previous script
# Part of the address. (See the previous script)
Sed's/* (. *) //; S/>. * //; S/. * [: <] */'

# Add angle brackets and spaces (reference information) at the beginning of each line)
Sed's/^/> /'

# Delete the angle brackets and spaces at the beginning of each line (unreference)
Sed's/^> //'

# Remove most HTML tags (including cross-row tags)
Sed-E: A-E's/<[^>] *> // G;/</N; // Ba'

# Decode uencode files that are divided into multiple volumes. Removes the file header information and only keeps the uencode part.
# The file must be transmitted to sed in a specific order. The script of the first version can be directly input in the command line;
# The second version can be placed in a shell script with the execution permission. (One of Rahul dhesi
# The script is modified .)
Sed '/^ end/,/^ begin/d' file1 file2... filex | uudecode # vers. 1
Sed '/^ end/,/^ begin/d' "$ @" | udecode # vers. 2

# Sort the paragraphs in the file alphabetically. Paragraphs are separated by (one or more) blank rows. Use GNU SED
# The character "\ v" is used to represent vertical tabs. It is used as a placeholder for line breaks-you can also
# Replace it with other characters that are not used in the file.
Sed '/. /{H; D ;}; X; S/\ n/={ NL }=/ G' file | sort | sed '1s/= {NL }= //; s/= {NL} =/\ n/G'
Gsed '/. /{H; d}; X; y/\ n/\ v/'file | sort | sed '1s/\ v //; y/\ v/\ n /'

# Compress each. txt file, compress the original file, and delete the compressed. ZIP file.
# Name it the same as the original name (only with Different Extensions ). (DOS environment: "DIR/B"
# Display file names without paths ).
Echo @ echo off> zipup. bat
DIR/B *. txt | sed "s/^ \ (. * \) \. txt/PKZIP-Mo \ 1 \ 1.txt/"> zipup. bat

Use sed: sed to accept one or more editing commands, and apply these commands in sequence after each line is read.
After reading the first line of input, sed applies all the commands to it and then outputs the results. Then read the second
Line input, apply all the commands to it ...... Repeat this process. In the previous example, sed is set by the standard input.
Slave (that is, the command interpreter, usually in the form of pipeline Input) to obtain the input. One or more
When file names are used as parameters, these files Replace the standard input device as sed input. Sed output will be
Delivered to standard output (Display ). Therefore:

Cat filename | sed '10q' # input using MPs queue
Sed '10q' filename # achieves the same effect, but does not use MPs queue input.
Sed '10q' FILENAME> newfile # redirects the output to the disk.

You need to know how to use sed commands, including how to use these commands through script files rather than from command lines.
For more information, see sed & awk, Author: Dale Dougherty and Arnold Robbins.
(O 'Reilly, 1997; http://www.ora.com), UNIX text processing, author
Teaching by Dale Dougherty and Tim o'reilly (Hayden Books, 1987) or by Mike arst
The name of the -- zip package is "U-SEDIT2.ZIP" (found on many sites ). To discover SED
You must have a sufficient understanding of the "Regular Expression. For more information about regular expressions, see
Jeffrey Friedl (O 'Reilly 1997), author of mastering regular expressions ).
The man page provided by UNIX systems will also be helpful (try these commands
"Man sed", "Man Regexp", or the section about regular expressions in "man Ed"),
The information provided by the manual is "abstract", which has always been criticized. However, it would not have been
Is used to teach beginners how to use sed or regular expressions, but just for those who are familiar with these tools
Provides some text references.

Bracket Syntax: the preceding example basically uses single quotation marks ('...') instead of double quotation marks for sed commands.
("...") This is because SED is usually used on UNIX platforms. Unix shell (command
INTERPRETER) will not explain and execute the dollar sign ($) and quotation marks. Under double quotation marks
The dollar sign is expanded as the value of a variable or parameter. The command in the quotation marks is executed and replaced with the output result.
Content in quotation marks. The exclamation point (!) is used in "CSH" and its derivative shell (!) Before
Add a backslash (like this: \!) for escape :\!) To ensure that the above example can run normally
(Including the use of single quotes ). All dos sed versions use double quotation marks ("...") instead
Enclose the command in quotation marks.

'\ T' usage: To keep this article concise, we use' \ t' in the script to represent a tabulation.
. However, in most versions, sed cannot recognize '\ t' in short. Therefore
When the script inputs a tab, you should press the tab key to enter the tab rather than '\ t '. The following workers
The software supports '\ t' as a regular expression character to represent tabs: awk, Perl, hhsed,
Sedmod and GNU sed v3.02.80.

Sed of different versions: sed of different versions may be somewhat different. It can be imagined that SED of different versions is in syntax.
There will be differences. Specifically, most of them do not support using tags (: Name) or tags in the process of editing commands.
Command (B, T) unless it is placed at the end of those. In this document, we try to use a high degree of portability.
So that most sed users can use these scripts. However, the GNU sed version allows
Use more concise syntax. Imagine the mood when the reader sees a long command:

Sed-e '/AAA/B'-E'/BBB/B '-E'/CCC/B'-e d

The good news is that GNU sed makes the command more compact:

Sed '/AAA/B;/BBB/B;/CCC/B; D' # It can even be written
Sed '/AAA \ | BBB \ | CCC/B; D'

In addition, note that although many versions of SED accept such as "/One/S/RE1/re2/", it is null before 'S'
But some of these versions do not accept the command: "/One /! S/RE1/re2 /". Then
You only need to remove the spaces in the middle.

Speed Optimization: when the input file is large, the processor or hard disk is slow for some reason
When running a command, you can consider adding an address expression before the replacement command ("s /.../").
Increase speed. For example:

Sed's/Foo/BAR/G' filename # standard replacement command
Sed '/Foo/S/Foo/BAR/G' filename # Faster
Sed '/Foo/S // bar/G' filename # abbreviated form

When you only need to display the front part of the file or delete the following content, you can use "Q" in the script"
Command (exit command ). This saves a lot of time when processing large files. Therefore:

Sed-n'45, 50p' filename # display 45th to 50 rows
Sed-n'51q; 45, 50p' filename # is the same, but it is much faster.

If you have other single-line scripts that you want to share with you or find errors in this document, please generate electricity
Submail to the author of this document (Eric pement ). Please remember to provide your sed version,
The operating system that the SED runs and the appropriate description of the problem. The single-line script mentioned in this Article refers to the length of the command line
Sed script with a degree of 65 characters or less 〕. The various scripts in this document are as follows:
Written or provided:

Al AAB # created the "seders" email list
Edgar Allen # many aspects
Yiorgos adamopoulos # many aspects
Dale Dougherty # author of SED & awk
Carlos Duarte # author of do it with SED
Eric pement # author of this document
Ken pizzini # author of GNU sed v3.02
S.g. ravenhall # HTML Tag script
Greg ubben # has made many contributions and provided a lot of help
-------------------------------------------------------------------------

In most cases, the SED script can be written as a single row regardless of the length (using the '-e' option and ';'
() -- As long as the command interpreter supports it, the single-line script mentioned here can be written as one line but also has
. Because these single-line scripts do not mean they appear in a single line. But enables users
It is easy to use these compact scripts in the command line to make sense.