Early knowledge of sed and gawk in Linux command line and shell script programming

Linux command line and shell script programming book first recognized sed and gawk text processing sed editor can process data in data streams based on commands input to the command line or stored in the command text file. It reads a line each time, matches data with the provided editor commands, and recognizes sed and gawk in "Linux command line and shell script programming ".
The sed editor of the text processing editor can process data in a data stream based on commands that are input to the command line or stored in the command text file. It reads a row each time, matches the data with the provided editor command, modifies the data in the stream in the way specified in the command, and then outputs the generated data to STDOUT. After the stream editor matches all commands with one row of data, it will read the next row of data and repeat this process. After the stream editor processes all data rows in the stream, it terminates. Sed command format: sed options script filesed command option description-when processing input, e script adds the specified command in the script to the running command-f file when processing input, add the specified command in file to the running command-n do not generate output for each command. wait for the print command to output the command in the command line definition editor by default, sed will apply the specified command to the STDIN input stream in the editor. [Plain] $ echo "This is a test" | sed's/a/not a/'This is not a test in the command line using multiple editor commands using-e. Note: Multiple commands are written between the same pair of quotation marks and separated by semicolons. There must be no space between the end of the Command and the semicolon. [Plain] $ echo "This is a test" | sed-e's/a/not /; s/This/That/'That is not a test to read the editor command from the file [plain] $ cat sed_script s/brown/white/s/fox/elephant/use the-f option you can. [Plain] $ echo "The quick brown fox jumps over the lazy dog. "| sed-f sed_script The quick white elephant jumps over the lazy dog. the sed command in the file can use line breaks or semicolons to separate gawk programs in gawk programming language. you can do the following: 1. define variables to save data 2. use arithmetic and string operators to process data 3. use structured programming concepts, such as if-then statements and loops. extract data elements from the data file and place them in another order or format. Gawk command format: gawk option program filegawk option description-F fs specifies the field separator for the fields that separate data fields in the row-f file specifies the file name of the reader-v var = value defines a variable in the gawk program and the default value -mf N specifies the maximum number of fields in the data file to be processed-mr N specifies the maximum number of data lines in the data file-W keyword specifies the gawk compatibility mode or warning level to read the program script from the command line the gawk program is defined by a pair of curly braces. Because the gawk command line assumes that the script is a single text string, you must put the script in single quotes. [Plain] $ gawk '{print "Hello"}' after you press enter, it will not be executed immediately until you enter the text. Gawk will run all the program scripts for this line of text like sed, and gawk will execute the program script for each line of text in the data stream. When the data field variable gawk is used, the following variables are allocated to each data field found in the text line: $0 indicates the entire text line; $1 indicates the first data field in the text line, the default field separator in gawk is any blank character [plain] $ echo "a B C> D E F" | gawk '{print $1} 'a D- F option specifies the separator [plain] $ gawk-F: '{print $1}'/etc/passwd root daemon bin sys ...... Multiple commands in the program script are separated by semicolons. [Plain] $ echo "I'm a person. "| gawk '{$3 =" man. "; print $0} 'I'm a man. assign a value to $3 and then print this sentence. The printed result is the result after $3 is replaced. You can also use multiple commands to read the program [plain] $ cat gawk_test {text = "'s home directory is" print $1 text $6} from a file, you can use line breaks or semicolons to separate the command [plain] $ gawk-F: -f gawk_test/etc/passwd root's home directory is/root daemon's home directory is/usr/sbin ...... When gawk references a variable, unlike shell, it needs to use the dollar sign to run the script BEGIN keyword before processing the data, run the specified script [plain] $ gawk 'In in {print "hello world"} 'after processing the data, run the script. of course, the END keyword [plain] $ cat gawk_test BEGIN is used. {print "Before read the file" FS = ": "} {print $1} END {print" The End "} note: use FS to set the delimiter [plain] $ gawk-f gawk_test/etc/passwd Before read the file root daemon ...... Sshd Debian-exim The End sed editor basic more replacement options to replace tags by default, only one [plain] $ echo "That is a test. "| sed's // 'thatis a test. to change this method, replace the s/pattern/replacement/flags with four replacement tags: numbers, indicates the place where the pattern match will be replaced [plain] $ echo "That is a test. "| sed's // 2' That isa test. $ echo "That is a test. "| sed's // 6' That is a test. g, replace all the matched text [plain] $ echo "That is a test. "| sed's // g' Thatisatest. p indicates the place where the original row is to be typed Printed. It is usually used together with the-n option as mentioned above. the-n option indicates no output, waiting for the print command to output. [Plain] $ echo "That is a test. "| sed-n's // P' Thatis a test. w file: write the replaced result to [plain] $ echo "That is a test. "| sed-n's // w out' $ cat out Thatis a test. if we want to replace the file path in the text, the escape character [plain] $ pwd | sed's/\/home/\/myhome/'/myhome/su1216/android/source/linux_learned or, we can also replace the string delimiter "/" in the command with [plain] $ pwd | sed's @ home @ myhome @ '/myhome/suzhaoqiang/android/source/linux_learned. address used by default, Sed will use commands for all rows. if you only want to apply the command to specific rows, you need to use line addressing (line addressing) sed in two forms of line addressing: 1. the number range of the Row 2. the text mode is used to filter out a line. both methods use the same format to specify the address: [address] command can also put multiple commands together for a specific address: address {command1 command2 command3} numeric row addressing [plain] $ cat test.txt This is a test. this is a test. this is a test. this is a test. this is a test. this is a test. this is a test. $ sed '3, $ s/This/That/'test.txt This is a test. this is a test. that is a tes T. that is a test. that is a test. that is a test. that is a test. from the beginning of the third line to the last line (represented by the dollar sign in the last line), use the text pattern filter format as follows: /pattern/command this command only applies to lines that match the text pattern. Sed uses regular expressions in text mode. [Plain] sed '/su1216/s! Home! Myhome! '/Etc/passwd root: x: 0: 0: root:/bin/bash ...... Su1216: x: 1000: 1000: su1216,:/myhome/su1216:/bin/bash ...... To execute multiple commands on a single line, you can use curly braces to combine multiple commands [plain] $ sed '2 {s/This/That/s/test/good test/} 'test.txt This is a test. that is a good test. this is a test. this is a test. the command for deleting a row (delete) is d. If no row number is specified, delete all rows. [Plain] $ sed '2, 4d 'test.txt line 1 line 5 line 6 sed editor pattern matching also applies to delete commands: [plain] $ sed '/line 3/d' test.txt line 1 line 2 line 4 line 5 line 6 note: sed does not modify the original file. You can use two text modes to delete rows within a certain range. The row deletion function is enabled in the specified first text mode, and the second will disable the row deletion function. If the second one is not matched, all subsequent texts are deleted. If the second one is matched, sed will continue to try to match the first one. continue to try to enable the delete function again! [Plain] $ sed '/line 2/,/line 4/d' test.txt line 1 line 5 line 6 insert and additional text insert) command I will add a new line append (append) command a will add a new line format after the specified line: sed '[address] command \ new line' [plain] $ echo "line" | sed' I \ Line 1' Line 1 Line $ echo "Line" | sed 'a \ Line 1 'line 1 add a new Line [plain] $ sed '$ a \ new line' test.txt Line 1 line 2 line 3 line 4 line 5 line 6 new line to modify the line (change) when using the address range exactly the same as before, you must note that sed will take all the content in the range as Replace [plain] $ sed '2, 4c \ new line' test.txt line 1 new line 5 line 6 with a new line to convert (transform, y) the command is the only sed editor command that can process a single character. the command format is as follows: [addressly/inchars/outchars/conversion command maps inchars and outchars in order. If inchars and outchars have different lengths, sed will generate an error message [plain] $ sed 'Y/123/abc/'test.txt A: aabbaacc B: 44aa55bb. the conversion command is global, they replace all specified characters with the target characters. Review the command used to print the information in the data stream: 1.p, used to print the text line 2. the equal sign (=) command is used to print the line number 3.l( lower case L). it is used to list the line printing line and replace the p mark in the command, the p command can print the output line [plain] $ echo "hello" | sed 'P' hello $ echo "hello" | sed-n 'P' hello. text Mode [plain] $ cat test.txt: 11221133 B: 44115522 line 3 line 4 line 5 $ sed-n '/line/p' test.txt line 3 line 4 line 5 can be used with the replacement command. Below is a line modified, and print the line [plain] $ sed-n'/3/{p; s/Line/line/p} 'test.txt A: 11221133 line 3 Li Ne 3 contains three rows, where the next row meets the conditions and is replaced. Print the row number [plain] $ sed '= 'test.txt 1 A: 11221133 2 B: 44115522 3 line 3 4 line 4 5 line 5 the row number is determined by the line break. [Plain] $ sed-n'/line/{=; p} 'test.txt 3 line 3 4 line 4 5 line 5 listing lines this command allows printing of text in the data stream and non-printable ASCII characters. Any printable characters use their octal values plus a backslash or a standard C-style name. [Plain] $ sed-n 'L' test.txt A: 11221133 $ B: 44115522 $ line 3 $ line 4 $ line 5 $ for example, A tab is represented by \ t, the row ends with $. The format of writing data to A file using sed and the file is as follows: [address] w filename [plain] $ sed '1, 2 w sed_w_test 'test.txt A: 11221133 B: 44115522 line 3 line 4 line 5 $ cat sed_w_test A: 11221133 B: 44115522 of course, we can change it to the following form [plain] $ sed-n'1, 2 w sed_w_test 'test.txt $ cat sed_w_test A: 11221133 B: 44115522 read data from A file in the following format: [address] r filename read command allows you to insert data from an independent file into A data stream. [Plain] $ cat sed_w_test A: 11221133 B: 44115522 $ sed '4r sed_w_test 'test.txt A: 11221133 B: 44115522 line 3 line 4 A: 11221133 B: 44115522 line 5 you can use $