Sed, awk single-line script quick reference

Text interval:
# Add a blank line after each line
Sed G awk ' {printf ("%s\n\n", $)} '
# Remove all empty rows from the original and add a blank line after each line. # This will have a blank line after each line in the output text.
Sed '/^$/d; G ' awk '!/^$/{printf ("%s\n\n", $)} '
# Add two blank lines after each line
Sed ' G; G ' awk ' {printf ("%s\n\n\n", $)} '
# Delete all empty rows from the first script (that is, delete all even rows)
Sed ' n;d ' awk ' {f=!f;if (f) print $} '
# Insert a blank line before matching the line of the style "regex"
Sed '/regex/{x;p;x;} ' awk ' {if (/regex/) printf ("\n%s\n", $); else print $} '
# Insert a blank line after the line that matches the style "regex"
Sed '/regex/g ' awk ' {if (/regex/) printf ("%s\n\n", $ $); else print $} '
# Insert a blank line before and after the line that matches the style "regex"
Sed '/regex/{x;p;x; G;} ' awk ' {if (/regex/) printf ("\n%s\n\n", $ $); else print $} '
Number:
# number each line in the file (simple left alignment). This uses the "tab" # (tab, see the description of the usage of ' \ t ' at the end of this article) instead of the empty glyd to align the edges.
sed = filename | Sed ' n;s/\n/\t/' awk ' {i++;p rintf ("%d\t%s\n", i,$0)} '
# Number of all lines in the file (line numbers are left, text is aligned at the right end).
sed = filename | Sed ' N; s/^//; s/*\ (. \{6,\}\) \n/\1/' awk ' {i++;p rintf ("%6d%s\n", i,$0)} '
# number all lines in the file, but only the line numbers of non-blank lines.
Sed '/./= ' filename | Sed '/./n; s/\n//' awk ' {i++;if (!/^$/) printf ("%d%s\n", i,$0); else print} '
# Count rows (simulate "wc-l")
Sed-n ' $= ' awk ' {i++}end{print i} '
Text Conversion and substitution:
# UNIX Environment: The New line character (CR/LF) for converting DOS is in UNIX format.
Sed ' s/.$//' # assumes all lines with cr/lf end sed ' s/^m$//' # in Bash/tcsh, will press Ctrl-m to change to ctrl-v sed ' s/\x0d$//' # ssed, gsed 3.02.80, and higher This awk ' {sub (/\x0d$/, "");p rint} '
# UNIX Environment: convert UNIX New line character (LF) to DOS format.
Sed "s/$/' echo-e \\\r '/" # commands used under Ksh sed ' s/$ '/' echo \\\r '/' # commands used under bash sed ' s/$/' echo \\\r '/' # commands used under ZSH Sed ' s/$/\r/' # gsed 3.02.80 and later awk ' {printf ("%s\r\n", $)} '
# DOS Environment: convert Unix New line character (LF) to DOS format.
Sed "s/$//" # method 1 Sed-n P # method 2
DOS Environment is skipped
# DOS Environment: Convert DOS New line character (CR/LF) to Unix format. # The following script is only valid for unxutils sed 4.0.7 and later. To identify the unxutils version of the # SED can be achieved through its unique "–text" option. You can use the Help option ("Help") to see if there is a "–text" entry to determine whether the Unxutils version is being used. Other DOS # versions of SED are not able to perform this conversion. However, you can use "TR" to achieve this conversion.
Sed "s/\r//" infile >outfile # unxutils sed v4.0.7 or later
tr-d \ r <infile >outfile # GNU TR 1.22 or later
DOS Environment is skipped
# Remove the leading "white space characters" (spaces, tabs) from each line # to align it to the left
Sed ' s/^[\t]*//' # See the description of ' \ t ' usage at the end of this article awk ' {sub (/^[\t]+/, "");p rint $} '
# delete the trailing "whitespace" (Space, tab) of each line
Sed ' s/[\t]*$//' # See the description of ' \ t ' usage at the end of this article awk ' {sub (/[\t]+$/, "");p rint $} '
# Remove leading and trailing white space characters from each row
Sed ' s/^[\t]*//;s/[\t]*$//' awk ' {sub (/^[\t]+/, ""); Sub (/[\t]+$/, "");p rint $} '
# Insert 5 spaces at the beginning of each line (to move the full text 5-character position to the right)
Sed ' s/^//' awk ' {printf ("%s\n", $)} '
# Use 79 characters for width, align all text to the right # 78 characters plus the last space
SED-E: A-e ' s/^.\{1,78\}$/&/;ta ' awk ' {printf ("%79s\n", $)} '
# The width is 79 characters, so all text is centered. In method 1, spaces are filled in the front # header and back of each line in order for the text to center. In Method 2, when you center the text, only the # spaces are filled in front of the text, and eventually half of those spaces will be deleted. In addition, spaces are not filled in the back of each row.
SED-E: A-e ' s/^.\{1,77\}$/&/;ta ' # method 1 sed-e: a-e ' s/^.\{1,77\}$/&/;ta '-e ' s/\ (*\) \1/\1/' # method 2 awk ' {f Or (I=0;i<39-length ($)/2;i++) printf ("");p rintf ("%s\n", $ A)} ' #相当于上面的方法二
# Find the string "foo" in each line and replace the found "foo" with "Bar"
Sed ' s/foo/bar/' # replaces only the first "foo" string in each line sed ' S/FOO/BAR/4 ' # replaces only the fourth "foo" string in each row sed ' s/foo/bar/g ' # converts all "foo" in each row to "bar" sed ' S/\ (. *\) foo\ (. *foo\)/\1bar\2/' # Replace the penultimate "foo" sed ' s/\ (. *\) foo/\1bar/' # replace the last "foo"
awk ' {gsub (/foo/, "bar");p rint} ' # to replace All "foo" in each row with "bar"
# replace "foo" with "bar" only if the string "Baz" appears in the row
Sed '/baz/s/foo/bar/g ' awk ' {if (/baz/) gsub (/foo/, "bar");p rint $} '
# replace "foo" with "bar" and replace only if the string "Baz" does not appear in the row
Sed '/baz/!s/foo/bar/g ' awk ' {if (/baz$/) gsub (/foo/, "bar");p rint $} '
# whether it's "Scarlet", "Ruby" or "puce", Change to "red"
Sed ' s/scarlet/red/g;s/ruby/red/g;s/puce/red/g ' #对多数的 sed is effective gsed ' s/scarlet\|ruby\|puce/red/g ' # only for the GNU sed effective awk ' {GSU B (/scarlet|ruby|puce/, "red");p rint} '
# Turn all rows upside down, the first line becomes the last row, and so on (simulates "TAC"). # for some reason, hhsed v1.5 will delete blank lines in the file when using the following command
Sed ' 1! G;h;$!d ' # method 1 Sed-n ' 1! G;h, $p ' # method 2 awk ' {a[i++]=$0}end{for (j=i-1;j>=0;j--) print A[j]} '
# The characters in the line are sorted in reverse order, the first word becomes the last word, ... (Simulate "rev")
Sed '/\n/! G;s/\ (. \) \ (. *\n\)/&\2\1/;//d;s/.//' awk ' {for (I=length ($); i>0;i--) printf ("%s", substr ($0,i,1));p rintf ("\ n ")}‘
# Concatenate each of the two lines into one line (similar to "paste")
Sed ' $! n;s/\n//' awk ' {f=!f;if (f) printf ("%s", $ $), else printf ("%s\n", $)} '
# If the current line ends with a backslash "\", the next line will go to the end of the current line # and the trailing backslash of the original line is removed
SED-E: A-e '/\\$/n; s/\\\n//; Ta ' awk ' {if (/\\$/) printf ("%s", substr ($0,0,length ($ 1)), Else printf ("%s\n", $)} '
# If the current line starts with an equal sign, put the current line to the end of the previous line # and replace the "=" with a single space in the original outfit
SED-E: A-e ' $! n;s/\n=//;ta '-e ' p;d ' awk ' {if (/^=/) printf ("%s", substr ($0,2)), Else printf ("%s%s", a,$0), a= "\ n"}end{printf ("\ n")} '
# Add a comma separator to the number string and change "1234567" to "1,234,567"
Gsed ': A;s/\b[0-9]\{3\}\>/,&/;ta ' # GNU sed sed-e: A-e ' s/\ (. *[0-9]\) \ ([0-9]\{3\}\)/\1,\2/;ta ' # other SED
#awk regular no back to match and reference, make the comparison embarrassed, hehe.
awk ' {while (match ($0,/[0-9][0-9][0-9][0-9]+/)) {$0=sprintf ("%s,%s", substr ($0,0,rstart+rlengt H-4), substr ( rstart+rlength-3))}print} '
# Add comma delimiter for numeric values with decimal point and minus sign (GNU sed)
Gsed-r ': a;s/(^|[ ^0-9.]) ([0-9]+) ([0-9]{3})/\1\2,\3/g;ta '
#和上例差不多
awk ' {while (match ($0,/[^\.0-9][0-9][0-9][0-9][0-9]+/)) {$0=sprintf ("%s,%s", substr ($0,0,rstart+rl ENGTH-4), substr ( $0,rstart+rlength-3))}print} '
# Add a blank line after every 5 lines (add a blank line after line 5,10,15,20, etc.)
Gsed ' 0~5g ' # only for the GNU sed effective sed ' n;n;n;n; G; ' # other sed awk ' {print $0;i++;if (i==5) {printf ("\ n"); i=0}} '
To selectively display specific lines:
# shows the first 10 lines in the file (simulates the behavior of "head")
Sed 10q awk ' {print;if (nr==10) Exit} '
# Displays the first line in the file (simulates the "head-1" command)
SED q awk ' {print;exit} '
# shows the last 10 lines in the file (simulates "tail")
SED-E: A-e ' $q; N;11, $D; Ba '
#用 Awk does this a little bit, the full-text cache, for large files must be very slow
awk ' {a[nr]=$0}end{for (i=nr-9;i<=nr;i++) print A[i]} '
# show the last 2 lines in the file (simulate "tail-2" command)
Sed ' $! n;$! D ' awk ' {a[nr]=$0}end{for (i=nr-1;i<=nr;i++) print A[i]} '
# Displays the last line in the file (simulates "tail-1")
Sed ' $!d ' # method 1 Sed-n ' $p ' # method 2
#这个比较好办, only the last line is saved.
awk ' {a=$0}end{print A} '
# show the penultimate line in the file
Sed-e ' $! {h;d;} ' -E x # When there is only one row in the file, the output is blank line sed-e ' 1{$q;} '-E ' $! {h;d;} ' -E x # When there is only one row in the file, display the line sed-e ' 1{$d;} '-E ' $! {h;d;} ' -E x # does not output when there is only one row in the file
#存两行呗 (output a blank line when there is only one row in the file)
awk ' {b=a; A=$0}end{print B} '
# displays only the rows that match the regular expression (simulates "grep")
Sed-n '/regexp/p ' # method 1 sed '/regexp/!d ' # method 2 awk '/regexp/{print} '
# displays only "not" matches the line of the regular expression (simulates "grep-v")
Sed-n '/regexp/!p ' # method 1, corresponds to the previous command sed '/regexp/d ' # method 2, similar to the syntax awk '!/regexp/{print} '
# finds "RegExp" and displays the previous row of the matching row, but does not display matching rows
Sed-n '/regexp/{g;1!p;}; H ' awk '/regexp/{print a}{a=$0} '
# finds "RegExp" and displays the next line of matching rows, but does not display matching rows
Sed-n '/regexp/{n;p;} ' awk ' {if (A) print; A=0}/regexp/{a=1} '
# Show lines with "RegExp" and their front and back rows, and precede the first line with the line number of the line "RegExp" (similar to "GREP-A1-B1")
Sed-n-E '/regexp/{=;x;1!p;g;$! n;p;d;} ' -E H awk ' {if (F) print; F=0}/regexp/{print Nr;print B;print; F=1}{b=$0} '
# Displays lines that contain "AAA", "BBB", and "CCC" (in any order)
Sed '/aaa/!d; /bbb/!d; /ccc/!d ' # String order does not affect the result awk ' {if (match ($0,/aaa/) && match ($0,/bbb/) && match ($0,/ccc/)) print} '
# Displays rows that contain "AAA", "BBB", and "CCC" (Fixed order)
Sed '/aaa.*bbb.*ccc/!d ' awk ' {if (match ($0,/aaa.*bbb.*ccc/)) print} '
# Displays lines containing "AAA", "BBB" or "CCC" (Analog "Egrep")
Sed-e '/aaa/b '-e '/bbb/b '-e '/ccc/b '-e D # most sed gsed '/aaa\| bbb\| Ccc/!d ' # for GNU sed effective awk '/aaa/{print;next}/bbb/{print;next}/ccc/{print} ' awk '/aaa| bbb| Ccc/{print} '
# show the paragraph with "AAA" (separated by a blank line between the paragraphs) # hhsed v1.5 must be in the "x;" After adding "G;", the next 3 scripts are the same
Sed-e '/./{h;$!d;} '-E ' x;/aaa/!d; ' awk ' begin{rs= ' "}/aaa/{print} ' awk-vrs= '/aaa/{print} '
# Show paragraphs with "AAA" "BBB" and "CCC" three strings (in any order)
Sed-e '/./{h;$!d;} '-E ' x;/aaa/!d;/bbb/!d;/ccc/!d ' awk-vrs= ' {if (match ($0,/aaa/) && match ($0,/bbb/) && Match ($0,/ccc/)) print} '
# Displays a paragraph (in any order) that contains "AAA", "BBB", and "CCC" in any of the three strings
Sed-e '/./{h;$!d;} '-E ' x;/aaa/b '-e '/bbb/b '-e '/ccc/b '-e d gsed '/./{h;$!d;}; x;/aaa\| bbb\| Ccc/b;d ' # only valid for GNU sed awk-vrs= '/aaa| bbb| Ccc/{print "";p rint} '
# Show lines with 65 or more characters
Sed-n '/^.\{65\}/p ' cat Ll.txt | awk ' {if (length ($) >=65) print} '
# displays lines with 65 characters or less
Sed-n '/^.\{65\}/!p ' # method 1, corresponding to the above script should sed '/^.\{65\}/d ' # method 2, a little easier method awk ' {if (length ($) <=65) print} '
# Show partial text--from the line containing the regular expression to the end of the last line
Sed-n '/regexp/, $p ' awk '/regexp/{f=1}{if (F) Print} '
# Show Partial text--Specify line number range (from 8th to 12th lines with 8 and 12 rows)
Sed-n ' 8,12p ' # method 1 sed ' 8,12!d ' # method 2 awk ' {if (nr>=8 && nr<12) Print} '
# Show Line 52nd
Sed-n ' 52p ' # method 1 sed ' 52!d ' # method 2 sed ' 52q;d ' # Method 3, more efficient when handling large files awk ' {if (nr==52) {print;exit}} '
# starts at line 3rd and displays every 7 rows
Gsed-n ' 3~7p ' # only valid for GNU sed sed-n ' 3,${p;n;n;n;n;n;n;} ' # other sed awk ' {if (nr==3) f=1}{if (F) {i++;if (i%7==1) Print}} '
# Displays the text between two regular expressions (inclusive)
Sed-n '/iowa/,/montana/p ' # case-sensitive awk '/iowa/{f=1}{if (F) print}/montana/{f=0} '
To selectively delete specific rows:
# Displays the entire document, except for the contents of two regular expressions
Sed '/iowa/,/montana/d ' awk '/iowa/{f=1}{if (! F) print}/montana/{f=0} '
# Delete adjacent duplicate rows in a file (simulate "Uniq") # keep only the first row in the repeating row, delete the other rows
Sed ' $! N /^\ (. *\) \n\1$/! P D
awk ' {if ($0!=b) print; B=$0} '
# delete duplicate rows in a file, whether or not adjacent. Be aware of the size of the cache supported by hold space, or use the GNU sed.
Sed-n ' G; s/\n/&&/; /^\ ([-~]*\n\). *\n\1/d; s/\n//; H P ' #bones7456 Note: This command here does not work properly with awk ' {if (! ( (+ in B)) print; B[$0]=1} '
# Delete all rows except duplicate rows (simulate "uniq-d")
Sed ' $! N S/^\ (. *\) \n\1$/\1/; T D ' awk ' {if ($0==b && $0!=l) {print;l=$0}b=$0} '
# Delete 10 lines from the beginning of the file
Sed ' 1,10d ' awk ' {if (nr>10) print} '
# Delete the last line in a file
Sed ' $d '
#awk in the process does not know that there are a few lines of files, so only the whole cache, large files may not be suitable, the following two are the same
awk ' {b[nr]=$0}end{for (i=0;i<=nr-1;i++) print B[i]} '
# Delete the last two lines in a file
Sed ' n;$! p;$! D; $d ' awk ' {b[nr]=$0}end{for (i=0;i<=nr-2;i++) print B[i]} '
# Delete the last 10 lines in a file
SED-E: A-e ' $d; N;2,10ba '-E ' p;d ' # method 1 sed-n-e:a-E ' 1,10! P N;d;}; N;ba ' # method 2 awk ' {b[nr]=$0}end{for (i=0;i<=nr-10;i++) print B[i]} '
# Delete Multiples of 8 rows
Gsed ' 0~8d ' # only for GNU sed effective sed ' n;n;n;n;n;n;n;d; ' # other sed awk ' {if (nr%8!=0) print} ' |head
# Delete lines that match styles
Sed '/pattern/d ' # deletes the line containing the pattern. Of course, the pattern can be replaced by any valid regular expression awk ' {if (!match ($0,/pattern/)) print} '
# Delete all empty lines in the file (with "grep". "Same effect)
Sed '/^$/d ' # method 1 sed '/./!d ' # method 2 awk ' {if (!match ($0,/^$/)) print} '
# keep only the first row of multiple adjacent empty rows. and delete empty lines at the top and tail of the file. # (analog "cat-s")
Sed '/./,/^$/!d ' #方法 1, delete the empty line at the top of the file, allow trailing to keep a blank line sed '/^$/n;/\n$/d ' #方法 2, allow a blank line at the top, trailing no blank line awk ' {if (!match ($0,/^$/)) {print; F=1}else{if (F) print; F=0}} ' #同上面的方法 2
# keep only the first two rows of multiple adjacent empty rows.
Sed '/^$/n;/\n$/n;//d ' awk ' {if (!match ($0,/^$/)) {print; F=0}else{if (f<2) print; f++}} '
# Delete all empty lines at the top of the file
Sed '/./,$!d ' awk ' {if (F | |!match ($0,/^$/)) {print; F=1}} '
# Delete all empty lines at the end of a file
SED-E: A-e '/^\n*$/{$d; N;ba '-e '} ' # is valid for all sed sed-e: a-e '/^\n*$/n;/\n$/ba ' # ibid, but only for gsed 3.02.* valid awk '/^.+$/{for (i=l;i<nr-1;i++) print " ";p Rint;l=nr} '
# Delete the last line of each paragraph
Sed-n '/^$/{p;h;};/. /{x;/./p;} '
#很长, it's ugly, there should be a better way.
Awk-vrs= ' {b=$0;l=0;f=1;while (Match (b,/\n/) >0) {print substr (b,l,rstart-l-f); L=rstart;sub (/\n/, "", B); f=0}; Print ""} '
Special applications:
# Remove the Nroff tag from the man page. You may need to add the-e option when using the ' echo ' command under Unix System V or bash shell.
Sed "s/. ' Echo \\\b '//g" # The outer double brackets are mandatory (Unix environment) sed ' s/.^h//g ' # in Bash or tcsh, press ctrl-v and press ctrl-h sed ' s/.\x08//g ' # sed 1.5,gnu sed,ssed the hexadecimal notation used by awk ' {gsub (/.\x08/, "", $ A);p rint} '
# extract newsgroup or e-mail headers
Sed '/^$/q ' # Delete all contents of the first line after the empty line awk ' {print}/^$/{exit} '
# Extract the body part of a newsgroup or e-mail
Sed ' 1,/^$/d ' # Delete all content before the first line of the empty line awk ' {if (F) print}/^$/{f=1} '
# extract "Subject" (title bar field) from the message header and remove the word "Subject:" At the beginning
Sed '/^subject: */!d; S///;q ' awk '/^subject:.*/{print substr ($0,10)}/^$/{exit} '
# Get reply address from message header
Sed '/^reply-to:/q; /^from:/h; /./d;g;q '
#好像是输出第一个 reply-to: Line at the beginning? What is the from? The rules are not clear.
awk '/^reply-to:.*/{print;exit}/^$/{exit} '
# Get the email address. On the basis of the line header generated by the previous script, the part of the non-e-mail address is further shaved. (see previous script)
Sed ' s/* (. *)//; s/>.*//; s/.*[:<] *//'
#取尖括号里的东西吧?
Awk-f ' [<>]+ ' {print $} '
# Add an angle bracket and a space at the beginning of each line (reference information)
Sed ' s/^/>/' awk ' {print ' > ' $ A} '
# Remove the angle brackets and spaces at the beginning of each line (dereference)
Sed ' s/^>//' awk '/^>/{print substr ($0,3)} '
# Remove most HTML tags (including cross-line labels)
SED-E: A-e ' s/<[^>]*>//g;/</n;//ba ' awk ' {gsub (/<[^>]*>/, "", $ A);p rint} '
# decode the Uuencode file into multiple volumes. Removes the file header information, leaving only the Uuencode encoding part. # files must be passed to sed in a specific order. The first version of the script below can be entered directly at the command line; # The second version can be placed in a shell script with execute permissions. (Modified by a # script from Rahul Dhesi. ）
Sed '/^end/,/^begin/d ' file1 file2 ... filex | UUDecode # vers. 1 sed '/^end/,/^begin/d ' [email protected] ' | UUDecode # vers. 2
#我不想装个 UUDecode Verification, write it roughly.
awk '/^end/{f=0}{if (F) print}/^begin/{f=1} ' file1 file2 ... Filex
# Sort the paragraphs in the file in alphabetical order. Paragraphs are separated by a blank line (one or more lines). The GNU sed uses the # character "\v" to represent vertical tabs, which are used here as placeholders for line breaks--and you can, of course, replace them with other characters that are not used in the file.
Sed '/./{h;d;}; X;s/\n/={nl}=/g ' File | Sort | Sed ' 1s/={nl}=//;s/={nl}=/\n/g ' gsed '/./{h;d};x;y/\n/\v/' file | Sort | Sed ' 1s/\v//;y/\v/\n/' awk-vrs= ' {gsub (/\n/, "\v", $);p rint} ' ll.txt | Sort | awk ' {gsub (/\v/, "\ n", $);p rint;print ""} '
# compress each individually. TXT file, compressed after deleting the original file and will compress after. ZIP file # is named with the same name as the original (except for the extension name). (DOS Environment: "Dir/b" # shows the file name without a path).
echo @echo off >zipup.bat dir/b *.txt | Sed "s/^\ (. *\) \. Txt/pkzip-mo \1 \1.txt/">>zipup.bat

Sed, awk single-line script quick reference