Shell Programming---The awk command

awk Programming
Awk is a programming language. Gawk, is the latest version, the current Linux version is used Gawk
awk is a soft link to gawk

how awk works
BEGIN #在未读取文件行之前执行
Primary input loop, which executes repeatedly until the terminating condition triggers
END #在读取文件行完毕后执行


three ways to call awk
1. Invoking awk on the shell command line Input command
#awk [-F Domain delimiter] ' awk cmd ' file

2. Insert the script file in the AWK program section and call him through the awk command
#awk-F ' awk.sh ' file

3. Call the awk script directly
#./awk file
=====================================================
awk pattern Matching
Any awk statement consists of a pattern (pattern) and an action (action)
A pattern is a set of rules that test whether an input row needs to perform an action (the pattern determines when the action is triggered and the event is triggered)
Actions are procedures that contain statements, functions, and expressions (execution of processing of input rows)

Once there are blank lines in the test file, print out "This is a empty line!!!"
#awk '/^$/{print ' This was a empty line!!! '} ' Test
=====================================================
Records and Domains

awk defines each input file row as a record, each string in the row is defined as a field, and the symbol for the delimited field is called the field delimiter

Vim Student

Li Hao Njue 025-83481010
Zhang Ju Nju 025-83466534
Wang Bin Seu 025-83494883
Zhu Lin Njupt 025-83680010


Print out the 2, 1, 4, and 3 domains in the student file sequentially
#awk ' {print $2,$1,$4,$3} ' student

Print out all the fields in the student
#awk ' {print $} ' student

List the value of the third field based on the expression of an operation
#awk ' BEGIN {one=1;two=2}{print $ (one+two)} ' student

Print out the third field in the student file with the TAB key separator
#awk ' BEGIN {fs= ' \ t '} {print $} ' student

Li hao,njue,025-83481010
Zhang ju,nju,025-83466534
Wang bin,seu,025-83494883
Zhu lin,njupt,025-83680010

Print out the contents of all fields with a comma-delimited field separator
#awk ' BEGIN {fs= ', '} {print $} ' student

Print the contents of 1, 3 fields with a comma-delimited field separator
#awk ' BEGIN {fs= ', '} {print $1,$3} ' student

Abc D#中间两个tab
Abc D #中间一个tab

Print out the first field and D (two ways)
#awk ' BEGIN {fs= ' \t\t '} {print $1,$2} ' xx
#awk ' BEGIN {fs= ' \ t '} {print $1,$3} ' xx

Print out the first field and the D above
#awk ' BEGIN {fs= ' \ t '} {print $1,$2} ' xx

===============================================================
Relational Operators
< #小于
> #大于
<= #小于等于
>= #大于等于
== #等于
!= #不等于
~ #匹配正则表达式
!~ #不匹配正则表达式

Print the first domain matches the root content in the/etc/passwd file
#awk ' BEGIN {fs= ': '} $1~/root/'/etc/passwd

Print out all the fields in the/etc/passwd file that match the root content
#awk ' BEGIN {fs= ': '} $0~/root/'/etc/passwd

Print out the contents of the mismatched nologin in all fields in the/etc/passwd file
#awk ' BEGIN {fs= ': '} $0!~/nologin/'/etc/passwd

In the/etc/passwd file, if the UID is less than the GID, then all matching values are output
#awk ' BEGIN {fs= ': '} {if ($3<$4) print $} '/etc/passwd

In the/etc/passwd file, if the UID is greater than or equal to the GID, then all matching values are output
#awk ' BEGIN {fs= ': '} {if ($3>=$4) print $} '/etc/passwd

===============================================================
Boolean operator
|| #逻辑或
&& #逻辑与
! #逻辑非

Print out rows with UID equal to 10 in the/etc/passwd file or GID equals 10
#awk ' BEGIN {fs= ': '} {if ($3==10| | $4==10) print $} '/etc/passwd

Prints out the/etc/passwd file with UID equal to GID, and the login shell has the same matching information
#awk ' BEGIN {fs= ': '} {ofs= ': '} {if ($3==$4&&$7= "/sbin/nologin") print $} '/etc/passwd

Print out matching information in the/etc/passwd file where the UID matches 10 or GID matches 10
#awk ' BEGIN {fs= ': '} {if ($3~10| | $4~10) print $} '/etc/passwd

================================================================
The expression awk expression is used to store, manipulate, and fetch data, and an awk expression can consist of numeric values, character constants, variables, operators, functions, and regular
+ #加
- #减
* #乘
/ #除
% #模
^ or * * #乘方
++x #在返回x值之前, x variable plus 1
X + + #在返回x值之后, x variable plus 1

Use X+=1 to list the number of empty rows in a file test
#awk '/^$/{print x+=1} ' test

1th behavior 0, then recursive +1 lists the number of empty lines in the test file
#awk '/^$/{print x + +} ' Test

1th behavior 1, then recursive +1 lists the number of empty lines in the test file
#awk '/^$/{print ++x} ' test

Example
Li hao,njue,025-83481010,85,92,78,94,88
Zhang Ju,nju, 025-83466534,89,90,75,90,86
Wang Bin, seu,025-83494883,84,88,80,92,84
Zhu lin,njupt,025-83680010,98,78,81,87,76

Find the average value of a file student file for middle school students
#awk ' BEGIN {fs= ', '} {total=$4+$5+$6+$7+$8} {AVG=TOTAL/5} {print $1,avg} ' student

================================================================
System Variables: awk defines a number of built-in variables for setting environment variable information, which we call system variables, which are divided into two
1. To change the default value of awk, such as the domain delimiter
2. For defining system values, these system values can be read when processing a file, such as the number of fields in the record, the current number of records, the current file name, and awk dynamically changing the value of the second system variable

awk environment variables and their significance
#n #当前记录的第n个域, inter-domain division by FS
#0 #记录的所有域
ARGC #命令行参数的数量
Argind #命令行中当前文件的位置 (starting with 0 marking)
Argv #命令行参数的数组
Convfmt #数字转换格式
ENVIRON #环境变量关联数组
Errno #最后一个系统错误的描述
Filedwidths #字段宽度列表, separated by a space key
FILENAME #当前文件名
FNR #浏览文件的记录数
Fs #字段分隔符, the default is the space key
Igonrecase #布尔变量, if true, ignores case matching
Nf #当前记录中的域数量
Nr #当前记录数
Ofmt #数字的输出格式
OFS #输出域分隔符, the default is the space key
ORS #输出记录分隔符, the default is line break
Rlength #由match函数所匹配的字符串长度
Rs #记录分隔符, the default is the space key
Rstart #由match函数所匹配的字符串的第1个位置
Subsep #数组下标分隔符, the default value is \034

Output file student, and first join the current record number, the number of fields in the current record, output file name at the bottom of the files

#awk ' BEGIN {fs= ', '} {print nr,nf,$0} END {print FILENAME} ' student

====================================================================
Formatted output awk draws on the syntax of the C language, defines the printf output statement, and specifies the format of the output
printf (format control character, parameters)

The printf statement contains two parts
1, the format control character, all starts with the% symbol, describes the format specification
2, the parameter list, such as the variable name table, corresponds to the format control, is the output object,

printf modifier and its significance
- #左对齐
Width #域的步长
. prec #小数点右边的位数

printf format characters and their meanings
%c #ASCII字符
%d #整型数
%e #浮点数, Science notation
%f #浮点数
%o #八进制
%s #字符串
%x #十六进制数

Print the string $ student and the integer number $8, and then tab,$8 after the $.
#awk ' BEGIN {fs= ', '} {printf ("%s\t%d\n", $2,$8)} ' student

Converts the number 65 to an ASCII code
#awk ' BEGIN {printf ("%c\n", 65)} '

Convert 2014 to floating-point number, default decimal six-bit
#awk ' BEGIN {printf ("%f\n", 2014)} '

Prints out the first and third fields in the student file, and the string of the first field and the third field string differ by 15 spaces
#awk ' BEGIN {fs= ', '} {printf ("%-15s\t%s\n", $1,$3)} ' student

Prints out the first and third fields in the student file, and joins the first field's comment name at the beginning of the line, and the third field's comment PhoneNumber
#awk ' BEGIN {fs= ', ';p rint ' Name\t\tphonenumber '} {printf ("%-16s%s\n", $1,$3)} ' student

Print out floating-point number control at 10, three digits after the decimal point
#awk ' BEGIN {printf ("%10.3f\n", 20141126)} ' #如果数不满10位, it will be filled by space
2011111.000 Full 10-bit
201.000 less than 10 people


===================================================================
built-in string functions awk provides powerful built-in string functions for implementing text string substitution, lookup, and separation functions

awk string functions and their meanings
Gsub (R,s) #在输入文件中用s替换r
Gsub (r,s,t) #在t中用s替换r
Index (S,T) #返回s中字符串第一个t的位置
Length (s) #返回s的长度
Match (S,t) #测试s是否包含匹配t的字符串
Split (R,s,t) #在t上将r分成序列s
Sub (r,t,s) #将t中第1次出现的r替换为s
substr (r,s) #返回字符串r中从s开始的后缀部分
SUBSTR (r,s,t) #返回字符串r中从s开始长度为t的后缀部分

Replace the root of the first domain with "My name is Root" in passwd and output as: Format print out
#awk ' BEGIN {fs= ': '; ofs= ': '} gsub (/root/, "My name is Root", $) {print $} '/etc/passwd

Replace the root of all domains with "My name is Root" in passwd and output as: Format print out
#awk ' BEGIN {fs= ': '; ofs= ': '} gsub (/root/, "My name is root") {print $} '/etc/passwd

Print out the first position of F appearing in the ABCDEFG string
#awk ' BEGIN {print index ("ABCDEFG", "F")} '

Prints the length of the string "This is a httpd server script"
#awk ' BEGIN {print length ("This is a httpd server script")} '

Test "This is a httpd server script!!!" This string of strings! The first place
#awk ' BEGIN {print match ("This is a HTTPD server Sctipt!!!", "!")} '

Test "This is a HTTPD server Sctipt!!!" The first position of C in this string of strings (ignoring case)
#awk ' BEGIN {ignorecase=1;print match ("This is a HTTPD server Sctipt!!!",/c/)} '

The string "This script is a httpd server script!!!" The first occurrence of the sctipt is replaced with a script
#awk ' BEGIN {file= ' This script is a httpd server script!!! "; Sub (/script/, "script", file);p rintf ("%s\n", File)} '

Replace the first occurrence of the first field in the student text with the 10 in the Li row with 99
#awk ' BEGIN {fs= ', '} {$1~li sub (/10/, "" $ ");p Rint $} ' student

Returns the suffix part of the file that starts with the 5th character file= "This script is a httpd server script!!!"
#awk ' BEGIN {file= ' This script is a httpd server script!!! "; Print substr (file,6)} '

Returns the 6th character from the 9 suffix part of the file
#awk ' BEGIN {file= ' This script is a httpd server script!!! "; Print substr (file,6,9)} '

Inserts a record line number and number of fields in the header of the student file record, and uses separators. Separating printing
#awk ' BEGIN {fs= ', '} {print nr,nf,$0} ' ofs= '. "Student

Shell Programming---The awk command