Linux Three Musketeers-awk

1th. Where is awk? 1.1 awk built-in variables

Fs	Input field (column) separator
-F:	Equivalent to-VFS:
Nr	Number of record line (record number)
Nf	Number of filed columns per row
OFS	Output delimiter
Rs	Record separator The closing tag for each of the records delimiter is carriage return by default
IGNORECASE	Whether to ignore Case 1 to ignore

1.1.1 RS record delimiter each end tag is a carriage return by default

[Root@zeq Files] # Cat Passwd.txt root:x:0:0:root:/root:/bin/bashbin:x:1:1:bin:/bin:/sbin/nologin     Default end tag is carriage return, now the file content is two lines 

Displays line numbers and file contents AS/as record separators

[Root@zeq Files] # awk-vrs= "/" ' {print nr,$0} ' Passwd.txt   NR is display line number $ A   displays the contents of a whole row  1 root:x:0:0:root:2 Root:3 bin4 bashbin:x: 1:1: Bin:5 Bin:6 sbin7 nologin               now with/as the record delimiter becomes 7 lines here the awk-vrs= "/"     -V assigns values to variables, RS is awk built-in variable, set/for record delimiter

1.2 Represents rows and columns in awk

Nr==1	First line
$	First column
$NF	Last column
$NF-1	Second-Lowest column

1.3 awk pattern matching (conditional)

Mode-pattern help you find the line you want

1) Regular Expressions

2) comparison

>

<

==

3) Range

4) begin{} end{}

1.4 Regular Expressions

~	Contains XXX in a column
!~	XXX is not included in a column
^	To.... Characters starting with (columns)
$	To.... What's the end?
.*	All
^$	Blank Line
\	Escape character Backslash take off your vest and hit it back.
[]	Match every character of []
+	One character appears 1 or more times or more than 1 times
|	Or
()	Reverse reference, a whole, to protect the content inside
*	One character appears 0 or more times or more than 0 times
{}	0{N,M} number 0 has appeared consecutively at least n times, up to M times
?	One character appears 0 or 1 times

1.4.1 awk Regular Expression exercises

Create an environment

Mkdir-p/server/files/Cat>>/server/files/reg.txt<<Eofzhang Dandan41117397:250:100:175 The first column is the last name, Zhang Xiaoyu390320151:155:90:201 The second column is the name Meng Feixue80042789:250:60:50The  first and second columns together are the names Wu Waiwai70271111:250:80:75 The third column is the corresponding ID number Liu bingbing41117483:250:100:175 The last three columns are three donations in number Wang xiaoai3515064655:50:95:135Zi Gege1986787350:250:168:200Li Youjiu918391635:175:75:300Lao Nanhai918391635:250:100:175EOF

1.4.2 Remove the line starting with the number 4 in the 3rd column

[Root@zeq Files] # awk ' $3~/^4/' reg.txt    $ 3rd   ~ including    ^4 starts with 4  Zhang Dandan    41117397   : 250:100:175Liu   bingbing   41117483   : 250:100:175

1.4.3 Display Xiaoyu's last name and ID number

[Root@zeq Files] # awk ' $2~/xiaoyu/{print $1,$3} ' reg.txt    390320151

1.4.4 Displays the full name and ID number of all people with ID numbers beginning with 41

[Root@zeq Files] # awk ' $3~/^41/{print $1,$2,$3} ' reg.txt  4111739741117483

awk Default Action

[Root@zeq Files] # awk  ' $3~/^41/' reg.txtZhang Dandan    41117397   : 250:100:175Liu   bingbing  41117483   : 250:100:175

[Root@zeq Files] # awk  ' $3~/^41/{print} ' reg.txtZhang Dandan    41117397   : 250:100:175Liu   Bingbing   41117483   : 250:100:175

1.4.5 Show all ID numbers the last digit is 1 or 5 of the person's full name

method 1[root@zeq Files] # awk ' $3~/[15]$/{print $1,$2} ' reg.txt     [] Match the inside of 1 and 5    $ to indicate ... End  Zhang xiaoyuwu waiwaiwang xiaoaili Youjiulao Nanhai

method 2[root@zeq Files] # awk ' $3~/(1|5) $/{print $1,$2} ' reg.txt     | or   1 or 5  Zhang Xiaoyuwu waiwaiwang xiaoaili Youjiulao Nanhai

1.4.6 shows donations from Xiaoyu. Each value is preceded by $. such as $520$200$135

[Root@zeq Files] # awk-f:-vofs=$ '/xiaoyu/{print "$" $2,$3,$4} ' reg.txt $155$90$201-          F Specify: Delimiter    -vofs=$ output $ to delimiter  

1.5 awk Replacement

Gsub awk built-in functions

1.5.1 format

Gsub (/What to replace/, "What to replace", replaced part)

1.5.2 "shows donations from Xiaoyu. Each value has a $ start. such as $520$200$135" the question

[Root@zeq Files] # awk ' {gsub (/:/, "$"); print} ' Reg.txt       Replace all colons with $  Zhang Dandan     41117397   $250$100$175Zhang xiaoyu    390320151  $155$90$201Meng  feixue     80042789   $250$60$50Wu    waiwai    70271111   $250$80$75Liu   bingbing  41117483   $250$100$175Wang  xiaoai    3515064655 $50$95$135Zi    gege      1986787350 $250$168$200Li    youjiu    918391635  $175$75$300Lao   Nanhai     918391635  $250$100$175

Plus the exact part of the condition.

[Root@zeq Files] # awk ' $2~/xiaoyu/{gsub (/:/, "$"); print $NF} ' Reg.txt    $NF last column  $155$90$201

1.6 Range

1, from line 1th to line 5th content

awk ' nr==1,nr==5 '

2. From the line containing the content to the row containing the content

awk '/content/,/content/'

1.6.1 display from line 1th to line 5th

[Root@zeq Files] # awk ' nr==1,nr==5 ' reg.txt Zhang Dandan     41117397   : 250:100:175Zhang xiaoyu    390320151  : 155:90:201Meng  feixue    80042789   : 250:60:50Wu    waiwai    70271111   : 250:80:75Liu   bingbing   41117483   : 250:100:175

1.6.2 display contains Xiaoyu to rows containing Waiwai

[Root@zeq Files] # awk '/xiaoyu/,/waiwai/' reg.txt Zhang Xiaoyu     390320151  : 155:90:201Meng  feixue    80042789   : 250:60:50Wu    Waiwai     70271111   : 250:80:75

1.7 Comparison expressions

$5>500                  Fifth column is greaterthan NR>20                  line greater than 20th row, 20 lines later >  >===!=                     Not equal to

1.7.1 Viewing disk information df-h

[Root@zeq Files] # df-h Filesystem      Size  used Availuse% mountedon/dev/sda3        19G  7.8G   10G  44%/tmpfs           491M     0  491M   0/dev/shm/dev/sda1       190M   61M  120M  34%/Boot /dev/sdb1       193M  1.8M  181M   1%/data

1.7.2 shows disk partition name and mount point with disk usage greater than 20% (error example)

[Root@zeq Files] # Df-h|awk ' $5>20{print $, $NF} '   Filesystemon/dev/sda3//dev/sda1/Boot[root@zeq files]#  Df-h|awk ' $5>9{print $1,$ NF} '   here will default $5>9 is a string (letter) Instead of a comparison expression  Filesystem on

1.7.3 Workaround 1 Specifies that the delimiter retains only the number part

[Root@zeq Files] # df-h|awk-f "[%]+" ' $5>9{print $, $NF} '    Filesystemon/dev/sda3//dev/sda1/boot

Want to remove the first column filesystem on information

[Root@zeq Files] # Df-h|awk ' nr>1 && $5+0>9{print $ $NF} '   after line 1th (&& and)  /dev/sda3//dev/sda1/boot

1.7.4 Solution 2 A column +0

[Root@zeq Files] # Df-h|awk ' $5+0>9{print $, $NF} '   the 5th column makes the comparison, the 5th column plus 0  /dev/sda3//dev/sda1/boot

1.8 Special Mode begin{} end{}

1) The process of awk execution

1. Parameters for executing the command (Assignment)-f-v

2.begin{} contents (Awk has not started to read the contents of the file)

3. Read the contents of the file

Determine if the condition is met (mode)

Compliance with execution commands (actions)

does not conform to reading the next line until the last line

4. After reading the contents of the file, start executing the contents of end{}

2) The contents of begin{} are executed before awk reads the contents of the file

1. Display Title

2. Modify awk built-in variables to create variables

awk ' begin{ofs=:} ' is equivalent to awk-vofs=:

3. Test calculation

3) end{} awk executes after reading the file

Show calculation results

Calculate first, end displays the result

1.8.1 statistics Number of virtual users in Passwd.txt

[Root@zeq Files] # Cat Passwd.txt        view file virtual user as Nologin  root:x:0:0:root:/root:/bin/bashbin:x:1:1:bin:/bin:/sbin/nologindaemon:x:2:2:daemon :/sbin:/sbin/nologinadm:x:3:4:adm:/var/adm:/sbin/nologinlp:x:4:7:lp:/var/spool/lpd:/sbin/ nologinsync:x:5:0:sync:/sbin:/bin/syncshutdown:x:6:0:shutdown:/sbin:/sbin/  Shutdownhalt:x:7:0:halt:/sbin:/sbin/haltmail:x:8:12:mail:/var/spool/mail:/sbin/  NOLOGINUUCP:X:10:14:uucp:/var/spool/uucp:/sbin/nologin

Number of statistics

[Root@zeq Files] # awk  '/nologin$/{i=i+1}end{print i} ' passwd.txt      i=i+1 equivalent to i++6

1.8.2 Statistics/etc/services file number of empty lines

[Root@zeq Files] # awk '/^$/{i++}end{print i} '/etc/services16

1.9 awk Action

Print

gsub function (command)

Variable calculation

1.9.1 Calculating disk usage

[Root@zeq Files] # DF Filesystem     1K-blocks    Used Available use% mountedon/dev/sda3       19534104 8076500  10458644  44%/tmpfs              502056       0    502056   0%/dev/shm/dev/sda1         194241   62009    121992  34%/ boot/dev/sdb1         197209    1813    185003   1%/data

[Root@zeq Files] # DF |awk ' nr>1{print $3/$2} '             0.41345600.3192370.00919329

1.9.2 viewing memory information free Free-h

[Root@zeq Files] #  Free        Total used       free     shared    buffers     cachedmem:       1004112     806016     198096        236     101452     563232-/+ buffers/cache:     141332     862780Swap:       786428        420     786008

1.9.3 Calculating system Memory Utilization

[Root@zeq Files] # Free |awk '/mem/{print ($3-$6-$7)/$2} '0.140705[Root@zeq files]#  Free|awk ' nr== 3{print $3/($3+$4)} '0.140705

1.9.4 calculating the residual rate of system memory

[Root@zeq Files] # Free|awk ' Nr==3{print $4/($3+$4)} '0.859211

1.9.5 Setting the variable computing system memory usage and residual rate

 [Root@zeq files]#   Free|awk ' nr==3{sum=$3+ $4;print $3/sum,$4/sum} '