Shell Programming Awk Advanced

awk operator

Arithmetic operators: x+y, X-y, x*y, x/y, X^y, x%y-x: Convert to negative +x: convert to Numeric

String operator: unsigned operator, string connection

Assignment operators: =, + =,-=, *=,/=,%=, ^= + +,--

Comparison operators: = =,! =,;, >=, <, <=

Awk-f: '! ($3==0) {print "} '/etc/passwd

Pattern match: ~ Left and right match contains!~ does not match

Awk–f: ' $ ~/root/{print $ '/etc/passwd

Awk–f: ' $3==0 '/etc/passwd

awk regular Match expression

(1) If not specified: Empty mode, match each row

(2)/regular expression/: Only the rows that can be matched to a pattern are processed, which needs to be used//enclosed.

awk '/^uuid/{print $ '/etc/fstab

(3) Relational expression: relational expressions, the result is "true" will be processed

awk '!0 '/etc/passwd

(4) Line ranges: Range

STARTLINE,ENDLINE:/PAT1/,/PAT2/does not support the direct giving of numeric formats

Awk-f: '/^root\>/,/^nobody\>/{print $ '/etc/passwd

(5) Begin/end mode

begin{}: Executes only once before starting the processing of text in a file end{}: Only once after the text processing is complete

awk Control Statements

If-else while loop Do-while loop for loop switch statement break and continue

Next: End the processing of the bank in advance and proceed directly to the next line (awk itself loops)

Awk-f: ' {if ($3%2!=0) next; print $1,$3} '/etc/passwd

Awk brings a loop to the contents of the file line, and awk uses loops to loop through the cut data columns

AWK's basic format awk-f: ' {} ' filepath

Performance comparison

Time (awk ' begin{total=0;for (i=0;i<=10000;i++) {total+=i;}; Print total;} ')

Time (Total=0;for i in {1..10000};d o total=$ (($total +i));d One;echo $total)

Time (for (i=0;i<=10000;i++);d O-let Total+=i;done;echo $total)

Time (seq–s "+" 10000|BC)

awk Array

Default is associative array

To iterate through each element in the array, use the For loop

for (var in array) {for-body} Note: var iterates through each index of the array

awk function

Numerical Processing:

RAND (): Returns a random number between 0 and 1 awk ' Begin{srand (); for (i=1;i<=10;i++) print int (rand () *100)} '

String processing:

echo $a | Awk-f "" ' {for (i=1;i<=nf;i++) {if ($i ~/^[0-9]+$/) printf '%s ', $i}} '

Length ([s]): Returns the length of the specified string

Sub (r,s,[t]): searches the T string for the pattern match of the R representation and replaces the first match with the S

echo "2008:08:08 08:08:08" | awk ' Sub (/:/, "-", $) '

Gsub (R,s,[t]): searches the T string for the pattern-matching contents of the R representation, and replaces all of the contents represented by S

echo "2008:08:08 08:08:08" | awk ' Gsub (/:/, "-", $) '

Split (S,array,[r]) takes R as a delimiter, cuts the string s, and saves the cut result to the array represented by array, the first index value is 1, the second index value is 2

Netstat-tan | awk '/^tcp\>/{split ($5,ip, ":"); count[ip[1]]++} end{for (i in count) {print I,count[i]}} '

Custom functions

Calling the shell command in awk

System command

Spaces are string connectors in awk, and if the variables in the system need to be used in awk, they can be separated by spaces, or "" are referenced in addition to the variables of awk.

Awk begin ' {System ("hostname")} ' awk begin ' {System ("Ifconfig")} '

awk ' begin{score=100; system ("Echo your score is" score)} '

Awk passes the value to the shell loop, and the shell command cannot be executed directly in Awk's action

#!/bin/bashawk'/^[0-9]/{ip[$1]++}end{for (i in IP) {if (ip[i]>30) print i}} '  while  do      Echo $ip  Done

View Code

awk Script

Write an awk program into a script that calls or executes directly

Passing parameters to an awk script

Format: Awkfile var=value var2=value2 ... Inputfile

Note: This is not available during the begin process. The variable is not usable until the first line has been entered. You can use the-v parameter to let awk get the value of the variable before executing the begin.

Each specified variable in the command line requires a-v argument

awk instances

1 awk '{for (i=1;i<=nf;i++) {word[$i]++}}end{for (J-in Word) {print j,word[j]}}'/etc/Rc.sysinit2 3[Email protected] ~]#awk '{if ($NF = = "M") {sum_m+=$2;num_m++} else{sum_f+=$2;num_f++}}end{printf "male:%.2f\nfemale:%.2f\n", Sum_m/num_m, Sum_f/num_f}'score4Male99.505Female95.006[Email protected] ~]#awk '{pcount[$3]++;s[$3]+=$2}end{for (i in Pcount) {print I, Pcount[i],s[i]/pcount[i]}}'score7M2 99.58F2  the9[Email protected] ~]#Echo "2008:08:08:08"|awk 'Sub (/:/, "-", $)'Ten  -- ,: ,: , One[Email protected] ~]#Echo "2008:08:08:08"|awk 'gsub (/:/, "-", $)' A  -- ,- ,: , -[Email protected] ~]#Echo "2008:08:08:08"|awk 'gsub (/:/, "-", $)' -  -- ,- ,- , the[Email protected] ~]#Head-n1/etc/passwd|awk '{split ($0,arr, ":")}end{for (i in arr) {print i,arr[i]}}' - 4 0 - 5Root - 6/Root + 7/bin/Bash - 1Root + 2x A 3 0 at[Email protected] ~]# Netstat-an |awk '/^tcp\>/{split ($5,ip, ":"); Count[ip[1]]++}end{for (i in count) {print I,count[i]}}' - 192.168.33.1 2 - 0.0.0.0 5

View Code

1 awk-F:-V i=0 'I{print}'/etc/Fstab2[Email protected] ~]#awk '/^[^#]/'/etc/Fstab3Uuid=1ec9c7b3-bec1-4998-9aad-057423419431/XFS Defaults0 04Uuid=df74adc0-09cd-4a97-a1e4-9f0d4a4b5796/boot XFS Defaults0 05Uuid=bacf4f88-3be9-4e74-ba5b-90274a2099ab/data XFS Defaults0 06uuid=339ab167-206a-4210-a36d-8d575579959e Swap swap Defaults0 07[Email protected] ~]#awk '/^ *#/'/etc/Fstab8 #9#/etc/FstabTen# Created by Anaconda on Wed Mar -  on: -: the 2018 One # A[Email protected] ~]#awk-F:'/\/bin\/bash$/{print $, $NF}'/etc/passwd -root/bin/Bash -yanxianghui/bin/Bash thetomcat/bin/Bash -[Email protected] ~]#awk-F:'/^root\>/,/^mail\>/{print $ {}'/etc/passwd - Root -[Email protected] ~]#DF-H |awk-f%'$ ~/^\/dev\/sd/{print $}'|awk '$5>=10' +/DEV/SDA1 1014M 194M 821M - -[Email protected] ~]#DF-H |awk-f%'$ ~/^\/dev\/sd/{print $}' +/dev/sda2 50G4.2G 46G9 A/dev/sda3 30G 82M 30G1 at/DEV/SDA1 1014M 194M 821M -

View Code

1 awk ' /^[0-9]/{ip[$1]++}end{for (i in IP) {print i,ip[i]}} '  2172.20. 101.111 2 3 172.20. 101.238  + 4 172.20. 101.188 6

View Code

Shell Programming Awk Advanced