The gawk-sed of the initial knowledge of shell text processing tools

Shellgawkgawk Options

• -f Specifies the field delimiter for delimited data fields in a row
• -f Specifies the handler's script name
• -V Var=value defining variables in the Gawk program
• -MF N Specifies the maximum number of rows to be processed in the data file
• -MR N Specifies the maximum number of data rows for the data file

gawk data field variables

• The entire text line is expressed
• $ = The first data field in the entire line of text
• $ = The second data field in the entire line of text

A simple case

# BEGIN这是处理数据之前运行的脚本BEGIN {# 这是定义的变量,在以下其他阶段的脚本中也可以调用,变量的调用方式如同shell中调用变量,但是不加$符号separator1="           "FS=":"print "用户名"separator1"密码"separator1"用户ID"separator1"组ID"separator1"用户全名"separator1"主目录"separator1"登录shell"separator1}# 这是进行数据处理的脚本{ print $1 separator1 $2 separator1 $3 separator1 $4 separator1 $5 separator1 $6 separator1 $7 separator1}# END这是数据处理最后要运行的脚本END {print "==========================THE END======================="}

The specified method is:

[email protected]:~# gawk -f ./passwd.awk /etc/passwd

SED replacement flag

• G: Global Substitution
• P: Print the replaced line
• The number indicates where the new text will be replaced by the extrude pattern match
• W: Writes the result of the substitution to the file

Replace character

To prevent conflicts with certain characters, SED supports selecting other characters to separate

sed ‘s!/bin/bash!/bin/csh‘ /etc/passwd

Row addressing

• Number range of rows
• Filter out a line in text mode

Use format for row addressing

[address]command

You can also put multiple commands for a specific address together

address{    command1    command2    command3}

Digital way of addressing

sed ‘2s/dog/cat/‘ data1‘sed ‘2,21s/dog/cat/‘ data1‘sed ‘2,$s/dog/cat/‘ data1‘

Text-mode addressing

sed ‘/admin/s/dog/cat‘ data2

Combo command

[email protected]:~# sed -n ‘2{s/daemon/admin/g;s/nologin/login/p}‘ /etc/passwdadmin:x:1:1:admin:/usr/sbin:/usr/sbin/login

Delete Row

sed ‘d‘ data2 #删除data2中的所有记录sed ‘3d‘ data2 #删除data2中第三行sed ‘4,$d‘ data2 #删除data2中第四行到最后的数据sed ‘/line   1/d‘ data2 #删除模式匹配到的行sed ‘/1/,/2/d‘ data2 #匹配到第一个打开行删除功能，匹配到第二个执行行删除功能,并删除两个匹配到的行,如果第二个模式没有被匹配到,那么整个文件的数据都被删掉了

Inserting and attaching text

• iInsert inserts a new row before the specified line
• Append a inserting a new row after the specified line

[email protected]:~# echo "line" | sed ‘i\linedemo‘linedemoline[email protected]:~# echo "line" | sed ‘a\linedemo‘linelinedemo

However, you need to specify where you want to insert or append in the data flow

[email protected]:~# cat date2 line1line2line3[email protected]:~# sed ‘1i\demo‘ date2 demoline1line2line3[email protected]:~# sed ‘2a\demo‘ date2 line1line2demoline3# sed ‘1i\first line‘ date2 # 总是插入在第一行# sed ‘$a\end line‘ date2 # 总是追加在最后一行

Modify rows

[email protected]:~# sed ‘3c\the change line‘ date2 line1line2the change line[email protected]:~# sed ‘1,$c\the change line‘ date2 the change line[email protected]:~# sed ‘1,2c\the change line‘ date2 the change lineline3[email protected]:~# sed ‘/line1/c\the change line‘ date2 the change lineline2line3

Convert command

# 前面的数字与后面的字母一对一的进行转换,1映射到a[email protected]:~# sed ‘y/123/abc/‘ date2 linealineblinec[email protected]:~# sed ‘1,2y/123/abc/‘ date2 linealinebline3

Review print

• Lowercase p is used to print the line of text
• Equal sign = used to print line numbers
• lowercase l used to list travel

Print line of text

[email protected]:~# head -3 /etc/passwd | sed ‘p‘root:x:0:0:root:/root:/bin/bashroot:x:0:0:root:/root:/bin/bashdaemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologindaemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologinbin:x:2:2:bin:/bin:/usr/sbin/nologinbin:x:2:2:bin:/bin:/usr/sbin/nologin[email protected]:~# head -3 /etc/passwd | sed -n ‘p‘root:x:0:0:root:/root:/bin/bashdaemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologinbin:x:2:2:bin:/bin:/usr/sbin/nologin

Print line number

[email protected]:~# head -3 /etc/passwd | sed -n ‘=‘123[email protected]:~# head -3 /etc/passwd | sed ‘=‘1root:x:0:0:root:/root:/bin/bash2daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin3bin:x:2:2:bin:/bin:/usr/sbin/nologin

Row Travel

# 注意每行最后的一个$字符[email protected]:~# head -3 /etc/passwd | sed -n ‘l‘root:x:0:0:root:/root:/bin/bash$daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin$bin:x:2:2:bin:/bin:/usr/sbin/nologin$

The gawk-sed of the initial knowledge of shell text processing tools