Sed and AWK getting started with Sed

Sed and AWK getting started. Sed and AWK are text processing artifacts in * nix command lines, which are quite powerful. they are all row-oriented, or they process text in one row and one row. They read content from standard input or files and execute script commands in one row, then print the output to the standard output until the end of the file (EOF ). sedSed is a Stream editor. It is used to edit and process an input Stream. this is equivalent to script-based editing for an input stream. in fact, it is to edit the script of an input stream ed (a row-oriented editor. the Sed command consists of two parts: command line parameters or command execution methods, and editing commands, also known as scripts. command Execution method: sed [OPTIONS]-e 'scripts' |-f script-file [input-files] such as: [plain] [alex @ alexon: ~] $ Sed-n-e 'P' speech.txt With great power comes great responsibility. the freedom is nothing but a chance to be better. as you can see, this is equivalent to the cat command. [html] [alex @ alexon: ~] $ Cat speech.txt With great power comes great responsibility. the freedom is nothing but a chance to be better. for command line parameters, see man manual. commonly used:-n -- quiet -- silent does not automatically print the mode space. simply put, the current row to be processed is not automatically printed. sed reads a row and puts it in a Pattern space to facilitate the execution of the editing command to process it. by default, this line (Content in Pattern space) is automatically printed out. when this parameter is not specified, you can see that [html] [alex @ alexon: ~] $ Sed-e 'P' speech.txt With great power comes great responsibility. with great power comes great responsibility. the freedom is nothing but a chance to be better. the freedom is nothing but a chance to be better. the reason is that the first line is the content of the Pattern space printed by default (that is, the content of the row to be processed ). then run the edit command. Because the edit command is a simple p (print content), you will see the duplicate output. but if-n (or -- quiet -- silent) is added, it will become like this: [html] [alex @ alexon: ~] $ Sed-n-e 'P' speech.txt With great power comes great responsibility. the freedom is nothing but a chance to be better. -e 'scripts' specifies the editing command or script. is the edit command supported by sed to be executed. it mainly involves editing operations such as pattern matching, text replacement, insertion, and deletion. this option can be specified. sed will be executed one by one in the order from left to right. [html] [alex @ alexon: ~] $ Sed-e '='-e 'P'-e's/great/poor/'speech.txt 1 With great power comes great responsibility. with poor power comes great responsibility. 2 The freedom is nothing but a chance to be better. the freedom is nothing but a chance to be better. resolution: The First Command '=' is to print the row number, the second command is to print this line, and the third command is to replace. -f script-file: Execute the script in the specified file. that is, the editing command is not placed in the command line, but in a file. let sed execute the commands in the file. -I [Suffix] -- in-place [= Suffix] instantly edits the input file. if Suffix is specified Use the suffix to back up the input file. the default behavior is to read a line of text from the input file, execute the command, and then output the result to the standard output. That is to say, it has no effect on the original text and will not change the original file. but sometimes we want to change the original file, that is, to edit the original file. this option is required. to avoid data loss, you can specify a suffix to back up the original file. example: [plain] [alex @ alexon: ~] $ Cat speech.txt With great power comes great responsibility. The freedom is nothing but a chance to be better. [alex @ alexon: ~] $ Sed-I. bak-e's/great/poor/G' speech.txt [alex @ alexon: ~] $ Cat speech.txt With poor power comes poor responsibility. The freedom is nothing but a chance to be better. [alex @ alexon: ~] $ Cat speech.txt. bak With great power comes great responsibility. the freedom is nothing but a chance to be better. the command is to replace great in the file with poor and back up the original file. bak. so far, does it remind you of powerful perl commands? It also has similar features: [plain] [alex @ alexon: ~] $ Perl-p-I. bak-e's/poor/great/G' speech.txt [alex @ alexon: ~] $ Cat speech.txt With great power comes great responsibility. The freedom is nothing but a chance to be better. [alex @ alexon: ~] $ Cat speech.txt. bak With poor power comes poor responsibility. the freedom is nothing but a chance to be better. the command line parameter is only part of sed. Its main core part is its editing command or its script, which is specified through the-e option, or use the script file specified by-f. edit Command Format: [command Scope] [!] Cmd [cmd-args], for example, [plain] [alex @ alexon: ~] $ Sed-n-e '1 P' speech.txt With great power comes great responsibility. A command can also be called addressing. in general, it is to specify the scope of the edit command. There are usually several ways to specify the range: not to specify-if the specific range is not specified, all rows will be used. [plain] [alex @ alexon: ~] $ Sed-n-e 'P' speech.txt With great power comes great responsibility. the freedom is nothing but a chance to be better. use the row number to specify --- n, m to the nth row. Special $ indicates the last row 1, 3 ---- 1st rows to 3rd rows 1, $ ---- 1st rows to the last row, that is, the number of rows relative to all rows. + m, such as n, + m, can be followed by a comma to indicate a relative amount of rows from n to n + m, for example: [plain] [alex @ alexon: ~] $ Sed-n-e '2, + 3 P' speech.txt 2. the freedom is nothing but a chance to be better. 3. the tree of liberty must be refreshed from time to time with blood of patroits 4. and tyrants. 5. life is like a box of chocolates, you never know what you gonna get. skip Selection Line. ------- waves can be used ~ To make a jump, n ~ M indicates that the execution starts from line n and is performed once every line m, for example, 1 ~ 2. Starting from row 3, execute every two rows, that is, execute 1, 3, 5, 7...: [plain] [alex @ alexon: ~] $ Sed-n-e '1 ~ 2 P' speech.txt 1. with great power comes great responsibility. 3. the tree of liberty must be refreshed from time to time with blood of patroits 5. life is like a box of chocolates, you never know what you gonna get. the most powerful part of pattern matching in a specified range is that pattern matching can be used to specify the scope. the format of pattern matching is: [/pattern1/]. [/pattern2/] If only one match is specified, the edit command is executed for all matching behaviors. If two matches are specified, it is the first line matching pattern1 and the first line matching pattern2. [plain] [alex @ alexon: ~] $ Sed-n-e '/great/P' speech.txt 1. With great power comes great responsibility. [alex @ alexon: ~] $ Sed-n-e '/great/,/chocolates/P' speech.txt 1. with great power comes great responsibility. 2. the freedom is nothing but a chance to be better. 3. the tree of liberty must be refreshed from time to time with blood of patroits 4. and tyrants. 5. life is like a box of chocolates, you never know what you gonna get. regular Expressions involve regular expressions when pattern matching is involved. The regular expressions of sed are slightly different from those of standard expressions. you can also specify-r -- regexp-extended in the command line to use an extended regular expression. bit Identifier: ^ --- beginning of the line $ ---- end of the line. ---- any non-linefeed '\ n' character \ B ---- end of a word. A word is defined as a series of letters or numbers. It can be placed at either end or two ends. limit * --- 0 or one or more \ + --- one or more \? --- 0 or 1 {m} --- appear m times {m, n} --- appear m times to n times, for example, {} indicates appear 1 to 5 times, 3, 4, 5 times) Escape Character \ --- can escape special character Character Set [] --- any character operator in IT \ | ---- or operation, abc \ | 123 matches 123 or abc \(... \) ---- combination to form a group, mainly used for the index \ n ---- the first n combination,/\ (123 \) \ 1/match 123123 edit command text edit command is also very familiar with adding, inserting, replacing and deleting and other such as printing, printing line number, etc. add1 [, add2] I text insert --- Insert the text [plain] [alex @ alexon: ~] before the specified row $ Sed-e '3 I abcd' speech.txt 1. with great power comes great responsibility. 2. the freedom is nothing but a chance to be better. abcd 3. the tree of liberty must be refreshed from time to time with blood of patroits 4. and tyrants. 5. life is like a box of chocolates, you never know what you gonna get. add1 [, add2] a text add --- add the text [plain] [alex @ alexon: ~] after the specified row $ Sed-e '3 a abcd' speech.txt 1. with great power comes great responsibility. 2. the freedom is nothing but a chance to be better. 3. the tree of liberty must be refreshed from time to time with blood of patroits abcd 4. and tyrants. 5. life is like a box of chocolates, you never know what you gonna get. add1 [, add2] d Delete --- Delete the specified row [plain] [alex @ alexon: ~] $ Sed-e '3 d' speech.txt 1. with great power comes great responsibility. 2. the freedom is nothing but a chance to be better. 4. and tyrants. 5. life is like a box of chocolates, you never know what you gonna get. add1 [, add2] s/pattern/replace/[opts] replace the pattern in the specified row with replace [plain] [alex @ alexon: ~] $ Sed-e '1, 3 s/great/poor/'speech.txt 1. with poor power comes great responsibility. 2. the freedom is nothing but a chance to be better. 3. the tree of liberty must be refreshed from time to time with blood of patroits 4. and tyrants. 5. life is like a box of chocolates, you never know what you gonna get. by default, only the 1st pattern entries in the row are replaced. opts, you can specify the option to control the replaced behavior n --- replace the nth pattern in the row as replace g --- replace all the patters in the row N is replace p --- print this line. If the replacement is successful,. add1 [, add2] c text replaces the specified line with text [plain] [alex @ alexon: ~]. $ Sed-e '1, 3 c abcd' speech.txt abcd 4. and tyrants. 5. life is like a box of chocolates, you never know what you gonna get. p print = print the row number. If you know this, you can handle most of the text processing. sed also has some advanced editing commands, such as operating Pattern Space or branches, but they are complicated and usually not used. we can see that sed is a stream editor. Its strength is that it can be scripted to process text in line. its main function is to delete, query, change and Add. but it is not a programming language after all, so it cannot have variables, loops, branches, and other logic. therefore, sed is usually used with AWK. AWK is more of a programming language. They complement each other and constitute two powerful tools for text processing.