Sed script syntax

Sed single-line script Quick Reference

For more information about exported sed single-line scripts, see the English title useful one-line scripts for sed (UNIX stream Editor). Original Title: handy one-liners for sed (UNIX stream editor) text interval: -------- # Add an empty line after each line sed G # delete all original empty lines and add an empty line after each line. # In this way, each row in the output text is followed by an empty row. Sed '/^ $/d; G' # Add two empty rows of SED' g behind each row; g' # delete all empty rows generated by the first script (that is, delete all even rows) Sed 'n; d' # insert an empty row sed '/RegEx/{X; P; X ;} '# insert an empty row sed after matching the row with the style "RegEx"'/RegEx/G' # insert an empty row sed before and after matching the row with the style "RegEx '/ regEx/{X; p; X; G;} 'No.: -------- # numbers each row in the file (simple left alignment ). Here we use "tab" # (tab, see the description of '/t' usage at the end of this article) instead of spaces to align edges. Sed = filename | sed 'n'; S // N // t/'# Number of all rows in the file (the row number is left and the text is aligned to the right ). Sed = filename | sed 'n'; S/^ //; S /*/(. /{6, //}/)/N // 1/'# Number of all rows in the file, but only the row numbers of non-blank rows are displayed. Sed '/. /= 'filename | sed '/. /n; S // N // '# calculate the number of rows (simulate "WC-L") Sed-n' $ =' text conversion and substitution: -------- # Unix environment: the new line character (CR/LF) for converting DOS is in UNIX format. Sed's /. $ // '# assume that all rows end with Cr/lf sed's/^ m $ //' # In bash/tcsh, change Ctrl-m to Ctrl-v sed's // x0d $ // '# ssed, gsed 3.02.80, and later # Unix environment: convert the Unix newline character (LF) to the DOS format. Sed "s/$/'echo-E // R'/" # The command SED's/$ '"/'echo // R' used in KSh' /"# command sed used in Bash" s/$/'echo // R'/"# command SED's/$/In zsh // r/'# gsed 3.02.80 and later # DOS environment: convert the Unix newline character (LF) to the DOS format. Sed "s/$ //" # method 1 sed-n p # method 2 # DOS environment: Convert dos newline (CR/LF) to Unix format. # The following script is only valid for unxutils sed 4.0.7 and later versions. To identify the # sed of the unxutils version, you can use its unique "-- text" option. You can use the help option ("-- Help") to check # whether there is a "-- text" item in it to determine whether the version is unxutils. Other dos # versions of SED cannot perform this conversion. However, you can use "TR" to achieve this conversion. Sed "S // R //" infile> OUTFILE # unxutils sed v4.0.7 or a later version of tr-D/r <infile> OUTFILE # GNU tr 1.22 or a later version # leading each line (space, tab) delete # align SED's/^ [/T] * // # refer to the '/t' usage description at the end of this article # drag each line to the end of the "white space" (space, tab) delete SED's/[/T] * $ // # See the description of '/t' usage at the end of this article # Delete the leading and trailing spaces in each line of SED' s/^ [/T] * //; s/[/T] * $ // # insert 5 spaces at the beginning of each line (to move the full text to the right five characters) sed's/^ // '# Take 79 characters as the width, right-aligned all text sed-E: A-E's/^. /{/} $/&/; TA '#78 characters plus the most The following space # uses 79 characters as the width to center all texts. In method 1, spaces are filled in the front # header and the back header of each line in order to center the text. In method 2, only space # is filled before the text in the center process, and half of the space will be deleted. In addition, no spaces are filled in the backend of each row. Sed-E: A-E's/^. /{/} $/&/; TA '# method 1 sed-E: A-E's/^. /{/} $/&/; TA '-E's //(*/) /1 // 1/'# method 2 # search for the string "foo" in each row ", replace "foo" with "bar" SED's/Foo/BAR/'# Only Replace the first "foo" string SED's/Foo/bar in each row. /4' # Replace only the fourth "foo" string SED's/Foo/BAR/G' in each row # Replace all "foo" in each row with "bar" SED's //(. */) Foo /(. * Foo/) // 1bar/2/'# Replace the last "foo" SED's //(. */) foo // 1bar/'# Replace the last "foo" # Replace "foo" with "bar" sed "/Baz/s only when" Baz "is displayed in the row. /Foo/ Bar/G' # Replace "foo" with "bar", and replace sed '/Baz/only when "Baz" is not displayed in the row /! S/Foo/BAR/G' # Replace it with "red" SED's/scarlet/red/g, whether it is "Scarlet", "Ruby" or "puce; s/Ruby/red/g; s/puce/red/G' # gsed's/scarlet/| Ruby/| puce/red/G' # Only for GNU sed # invert all row, the first line is the last line, and so on (simulate "TAC "). # For some reason, when you use the following command, hhsed v1.5 will delete the empty lines in the file sed '1! G; h; $! D' # method 1 sed-n' 1! G; h; $ P' # method 2 # Sort the characters in the row in reverse order. The first word becomes the last word ,...... (Simulate "Rev") Sed '// n /! G; S //(. /)/(. */n/)/&/2/1/; // D; S /. // '# connect each two rows into one (similar to "Paste") Sed' $! N; S // N // '# If the current row ends with a backslash, then, the next row is added to the end of the current row # And the backslash sed-E: A-E '// $/n at the end of the original row is removed; S // N //; TA '# If the current row starts with an equal sign, add the current row to the end of the previous line # And use a single space to replace "=" sed-E: A-E '$! N; S // n = //; TA '-e' p; D' # Add a comma separator for the numeric string and change "1234567" to "1,234,567" gsed ': a; S // B [0-9]/{3/}/>/, &/; TA '# GNU sed-E: a-E's //(. * [0-9]/)/([0-9]/{3/}/) // 1,/2 /; ta' # other sed # Add a comma separator (gnu sed) gsed-R': A; S/(^ | [^ 0-9.]) for values with decimal points and negative numbers. ([0-9] +) ([0-9] {3}) // 1/2,/3/g; ta' # Add a blank row (add a blank row after rows 5, 15, 20, and so on) gsed '0 ~ 5G '# only apply to the GNU sed: 'n'; n; G;' # select other sed rows: -------- # display the first 10 lines of the file (simulate the "head" behavior) Sed 10q # display the first line of the file (simulate the "head-1" command) sed Q # display the last 10 rows in the file (simulate "tail") Sed-E: A-E '$ q; n; 11, $ D; ba' # display the last two lines in the file (simulate the "tail-2" command) Sed '$! N; $! D' # display the last line in the file (simulate "tail-1") Sed '$! D' # method 1 sed-n' $ P' # method 2 # display the last row of SED-E' in the file $! {H; D;} '-e x # when there is only one row in the file, enter the empty line sed-e '1 {$ q;}'-e' $! {H; D;} '-e x # when there is only one row in the file, the sed-e '1 {$ D;}'-e' $! {H; D;} '-e x # when there is only one row in the file, no output # only display the rows matching the regular expression (simulate "grep ") sed-n'/Regexp/P' # method 1 sed '/Regexp /! D' # method 2 # show only the rows that do not match the regular expression (simulate "grep-V") Sed-n'/Regexp /! P' # method 1, which corresponds to the preceding command sed '/Regexp/d' # method 2, similar syntax # search for "Regexp" and display the last line of the matched row, but not match the row sed-n'/Regexp/{g; 1! P ;}; H' # Find "Regexp" and display the next row of the matched row, but not the sed-n'/Regexp/{n; P ;} '# display the rows that contain "Regexp" and the front and back rows, and add "Regexp" before the first line # The row's row number (similar to "grep-A1-B1 ") sed-n-e '/Regexp/{=; X; 1! P; G; $! N; P; D;} '-E h # display rows containing "AAA", "BBB", or "CCC" (in any order) Sed'/AAA /! D;/BBB /! D;/CCC /! D' # The string order does not affect the result # The Rows containing "AAA", "BBB", and "CCC" are displayed (in a fixed order) Sed '/AAA. * BBB. * CCC /! D' # display rows that contain "AAA" "BBB" or "CCC" (simulate "egrep ") sed-e '/AAA/B'-E'/BBB/B '-E'/CCC/B'-e d # majority of SED gsed '/AAA/| BBB/| CCC /! D' # valid for GNU sed # display paragraphs containing "AAA" (separated by blank lines) # hhsed v1.5 must be added after "X ;", the next three scripts are like this: sed-e '/. /{H; $! D;} '-e' X;/AAA /! D; '# display paragraphs (in any order) Sed-e'/./{H; $! D;} '-e' X;/AAA /! D;/BBB /! D;/CCC /! D' # display the section (in any order) Sed-e '/./{H; $! containing any string of "AAA", "BBB", and "CCC! D;} '-e' X;/AAA/B'-E'/BBB/B '-E'/CCC/B'-e d gsed '/. /{H; $! D ;}; X;/AAA/| BBB/| CCC/B; d' # only valid for GNU sed # display rows containing 65 or more characters sed-n'/^. /{65/}/P' # displays the rows sed-n'/^ that contain less than 65 characters. /{65 /}/! P' # method 1, which corresponds to SED '/^. /{65/}/d' # method 2, easier method # display part of the text-from the beginning of the row containing the regular expression to the end of the last row sed-n'/Regexp /, $ P' # display part of the text -- specify the row number range (from 8th to 12th rows, including 8 and 12 rows) Sed-n'8, 12p' # method 1 sed '8, 12! D' # method 2 # display 52nd rows of SED-N '52p' # method 1 sed '52! D' # method 2 sed '52q; D' # method 3, which is more efficient when processing large files # gsed-n' 3 ~ 7p' # only applies to GNU sed-n' 3, $ {P; n ;} '# other sed # display the text (including) Sed-n'/Iowa/,/Montana/P' between two regular expressions # select to delete a Specific Row in case-sensitive mode: -------- # display the entire document, except for the content sed '/Iowa/,/Montana/d' between two regular expressions # Delete adjacent duplicate lines in the file (simulate "uniq ") # Only the first row in the duplicate row is retained, and the other row is deleted from sed '$! N;/^/(. */)/n/1 $ /! P; D' # Delete duplicate rows in the file, regardless of whether they are adjacent. Note the cache # size supported by hold space, or use GNU sed. Sed-n 'g; S // n/& amp;/^/([-~] */N /). */n/1/d; S // N //; h; P' # delete all rows except duplicate rows (simulate "uniq-d") Sed '$! N; S/^ /(. */)/n/1 $ // 1/; t; D' # Delete the first 10 rows of SED '1, 10d '# Delete the last row of SED' $ d' # Delete the last two rows of SED 'n'; $! P; $! D; $ d' # Delete the last 10 rows of SED-E: A-E '$ D; n; 2, 10ba'-e' P; d' # method 1 sed-n-e: A-E '1, 10! {P; n; D ;}; N; Ba' # method 2 # Delete multiple rows of 8 gsed '0 ~ 8d '# used only for GNU sed 'n'; n; D; '# other sed # Delete the row sed that matches the pattern'/pattern/d' # Delete the row containing pattern. Of course, pattern # can be replaced with any valid Regular Expression # delete all blank lines in the file (with "grep '. ') Sed'/^ $/d' # method 1 sed '/. /! D' # method 2 # retain only the first row of multiple adjacent empty rows. Delete the empty lines at the top and end of the file. # (Simulate "cat-S") Sed '/./,/^ $ /! D' # Method 1: Delete the empty row at the top of the file and allow the end to keep an empty row sed '/^ $/N; // N $/d' # method 2, allow empty rows to be retained at the top, and leave no blank rows at the end # Only keep the first two rows of multiple adjacent empty rows. Sed '/^ $/N; // N $/N; // d' # delete all empty lines at the top of the file'/./, $! D' # delete all empty lines at the end of the file sed-E: A-E '/^/N * $/{$ D; N; ba '-e'}' # used for all sed-E: A-E '/^/N * $/N; // N $/Ba' # Same as above, but only for gsed 3. 02. * Valid # Delete the last sed-n '/^ $/{P; H ;};/. /{X ;/. /P;} 'special application: -------- # Remove the nroff mark from the man page. When using the # 'echo 'command in UNIX System V or bash shell, you may need to add the-e option. Sed "s /. 'echo // B '// G "# The outer double brackets are required (UNIX environment) SED's /. ^ h // G' # In bash or tcsh, press Ctrl-V and then Ctrl-h sed's /. /x08 // G' # sed 1.5, gnu sed, the hexadecimal representation used by ssed # extract the header sed '/^ $/Q' of a newsgroup or email # delete all content after the empty line on the first line # Extract sed '1, /^ $/d' # delete all content before the empty line of the first line # extract "subject" (title bar field) from the mail header and remove the "subject: "sed '/^ subject :*/! D; S //; Q' # obtain the reply address sed from the mail header '/^ reply-to:/Q;/^ From:/h ;/. /d; G; Q' # obtain the email address. Based on the line generated by the previous script, the part of the non-Email # address is further removed. (See the previous script) SED's /*(. *) //; S/>. * //; S /. * [: <] * // '# Add angle brackets and spaces at the beginning of each line (Reference Information) sed's/^/>/'# Delete the angle brackets and spaces at the beginning of each line (unreference) sed's/^> // # remove most HTML tags (including cross-row tags) Sed-E: a-E's/<[^>] *> // G;/</N; // Ba' # decodes the uencode file that is divided into multiple volumes. Removes the file header information and only keeps the uencode part. # The file must be transmitted to sed in a specific order. The script of the first version can be directly input in the command line; # the script of the second version can be placed in a shell script with the execution permission. (It was modified by a # script of Rahul dhesi .) Sed '/^ end/,/^ begin/d' file1 file2... filex | uudecode # vers. 1 sed '/^ end/,/^ begin/d' "$ @" | udecode # vers. 2 # Sort the paragraphs in the file alphabetically. Paragraphs are separated by (one or more) blank rows. GNU sed uses the # character "/V" to represent vertical tabs, here we use it as a placeholder for line breaks-you can also # Replace it with other characters not used in the file. Sed '/. /{H; D ;}; X; S // n/= {NL }=/ G' file | sort | sed '1s/= {NL} = //; s/= {NL} = // n/g 'gsed '/. /{H; d}; X; y // N // v/'file | sort | sed '1s/V //; Y // v // n/'# compress each. TXT file. After compression, delete the original file and compress the file. ZIP file # name it the same as the original name (only with Different Extensions ). (DOS environment: "DIR/B" # display file names without paths ). Echo @ echo off> zipup. bat DIR/B *. TXT | sed "s/^ /(. */)/. TXT/PKZIP-Mo/1/1.txt/ "> zipup. bat uses sed: sed to accept one or more edit commands, and each time a row is read, these commands are applied in sequence. After reading the first line of input, sed applies all the commands to it and then outputs the results. Then read the second line of input and apply all the commands to it ...... Repeat this process. In the previous example, sed is input by the standard input device (that is, the command interpreter, usually in the form of pipeline Input. When the command line provides one or more file names as parameters, these files Replace the standard input device as sed input. Sed output will be delivered to standard output (Display ). Therefore, cat filename | sed '10q' # input sed '10q' filename # In the MPs queue, however, if you do not use a pipe to input sed '10q' FILENAME> newfile # redirect the output to the disk, you must understand the usage instructions of the SED command, including how to use these commands through a script file (rather than from the command line), see sed & awk version 2, author Dale Dougherty and Arnold Robbins (O 'Reilly, 1997; http://www.ora.com), UNIX text processing by Dale Dougherty and Tim o'reilly (Hayden Books, 1987) or a tutorial written by Mike arst-the compressed package name is "U-SEDIT2.ZIP" (found on many sites ). To explore the potential of SED, you must have a sufficient understanding of the regular expression. For more information about regular expressions, see Jeffrey Friedl (O 'Reilly 1997), author of mastering regular expressions ). The manual pages provided by UNIX systems ("man") will also be helpful (try these commands "Man sed" and "Man Regexp ", or look at the regular expression section in "man Ed", but the information provided in the Manual is "abstract"-which has always been criticized. However, it is not intended to teach beginners how to use sed or regular expressions, but to provide some text references for those who are familiar with these tools. Bracket Syntax: the preceding example basically uses single quotes ('... ') instead of double quotation marks ("... ") This is because SED is usually used on UNIX platforms. Under single quotes, the Unix shell (command interpreter) will not explain and execute the dollar sign ($) and the quotation mark. In double quotation marks, the dollar sign is expanded as a variable or parameter value. The command in the quotation marks is executed and the output result replaces the content in the quotation marks. The exclamation point (!) is used in "CSH" and its derivative shell (!) Add a backslash (like this :/!) To ensure that the preceding example can run properly (including when single quotes are used ). All dos sed versions use double quotation marks ("...") instead of quotation marks to enclose the command. '/T' usage: to make this article concise, we use'/t' in the script to represent a tab. However, in most versions, sed does not recognize the '/t' shorthand format. Therefore, when you enter a tab for the script in the command line, you should press the tab key directly to enter the tab rather than '/t '. The following tool software supports '/t' as a regular expression character to represent tabs: awk, Perl, hhsed, sedmod, and GNU sed v3.02.80. Sed of different versions: sed of different versions may have some differences. It is conceivable that there will be Syntactic differences between them. Specifically, most of them do not support using tags (: Name) or branch commands (B, T) in the editing command, unless they are placed at the end of those. In this document, we try to use highly portable syntaxes so that most sed users can use these scripts. However, the GNU sed version allows more concise syntax. Imagine the mood when the reader sees a long command: sed-e '/AAA/B'-E'/BBB/B '-E'/CCC/B'-e d good news is that GNU sed makes the command more compact: sed '/AAA/B;/BBB/B;/CCC/B; D' # You can even write it as sed'/AAA/| BBB/| CCC/B; D, note that many versions of SED accept commands with spaces before 's', such as "/One/S/RE1/re2, however, some of these versions do not accept the following command: "/One /! S/RE1/re2 /". In this case, you only need to remove the spaces in the middle. Speed Optimization: to speed up command execution for some reason (such as large input files, slow processors, or hard disks), consider replacing the command ("s /... /... /) add an address expression to increase the speed. For example: sed's/Foo/BAR/G' filename # standard replacement command sed '/Foo/S/Foo/BAR/G' filename # Faster sed'/Foo/S // bar/G' filename # short form when you only need to display the part before the file or delete the content after it, you can use the "Q" command (exit command) in the script ). This saves a lot of time when processing large files. Therefore, sed-n'45, 50p' filename # displays 45th to 50 rows of SED-n'51q; 45, 50p' filename, but much faster. If you have other single-line scripts that you want to share with you or find errors in this document, please send an email to the author of this document (Eric pement ). In the email, please remember to provide your sed version, the operating system of the SED, and the appropriate description of the problem. The single-line script mentioned in this Article refers to the SED script with a command line length of 65 characters or less 〕. Various scripts in this document are written or provided by the authors listed below: al AAB # set up the "seders" email list Edgar Allen # many aspects of yiorgos adamopoulos # many aspects of Dale Dougherty # Carlos Duarte, author of SED & awk # Do it with SED Eric pement # Ken pizzini # s.g. ravenhall # HTML Tag script Greg ubben # has contributed a lot and provided a lot of help ----------------------------------------------------------------- 1: In most cases, sed scripts can be written into a single line regardless of the length (using the '-e' option and'; ') -- as long as the command interpreter supports this In addition to writing a single line, a single line script also has a limit on the length. Because these single-line scripts do not mean they appear in a single line. It makes sense to allow users to easily use these compact scripts in the command line.