Awk learning experience--super detail--case study

awk: Report Generator
Display information after formatting

Grammar:
awk [Options] ' script ' file1 file2, ...
awk [Options] ' Parttern {action} ' file1 file2, ...
The most common action:print,printf

Basic features of awk:
A. Every time you take a line
B. The line is cut by the specified delimiter (not specified as a bit whitespace character), using $1,$2,$3 (whole row), ... (first column, second column,... ）
C. You can specify line number, column number, cut character, post-operation delimiter
Case:
Chkconfig--list |grep 3: Enable |awk ' {print $} '
Tail-1/etc/passwd |awk-f ': ' begin{ofs= '---'}{print $1,$6,$7} ' # #OFS指定输出分隔符
Ifconfig eth1 |awk-f ' [:]+ ' nr==2 {print $4} '
Ifconfig eth1 |awk-f ' [:]+ ' nr==2 {print ' eth1_ip= ' $4} ' # #可以加入显示内容
awk ' BEGIN {print ' line one \nline two\nline three '} '


4. Use of advanced awk:
1) awk variable:
FS: Column delimiter, default bit blank
RS: Line delimiter, default bit line break
OFS: Output Column delimiter
ORS: Output Line delimiter

2) awk built-in variables
NR: Number of rows processed
FNR: Number of rows for a single file
NF: Number of columns
Case:
Ifconfig eth1 |awk ' {print NR} '
Ifconfig eth1 |awk ' {print NF} '

3) Custom variables:
awk ' begin{test= ' www.linuxfan.cn ";p rint test} '
Awk-v test= "linuxfan.cn" ' Begin{print test} '

4) printf
Format used:
printf format, item1,item2, ...
Characteristics:
A. Format must be specified to specify the output format of the following item
b.printf statement does not automatically print line breaks: \ n
C.format format to%+ a character, as follows:
%c: ASCII code for displaying characters
%d,%i: Decimal integer
%e: Scientific notation Displays values
%f: Show floating-point numbers (decimals)
%s: Display string
%u: unsigned integer
Percent: Show%
d. Modifier: N: Display width,-: left-justified, +: Displays numeric symbols, such as%-c (left-aligned)
Case:
Chkconfig--list |grep 3: Enable |awk ' {printf '%-10s ', ' $ ' # #在统一行显示
Awk-f: ' {printf '%-15s%-10d%-10s\n ', $1,$3,$7} '/etc/passwd

5) Awk's operator
A. Arithmetic operators
-X
+x
X^y
X**y
X*y
X/Y
X+y
X-y
X%y
B. Assignment operators
=
+=
-=
*=
/=
%=
++
--
C. Boolean value
In awk, any non-0 value or non-empty string is true and the reverse is false
D. Comparison operator: >,<,>=,<=,==,!=,~,!~ (x ~ y, string can be matched by expression y)

D. Logical operators: &&,| |,!
E. Conditional expression: condition? If-true-exp:if-false-exp
F. Calling function: Function_name (PA1, PA2)

6) Common pattern types for awk:
A. Regular expression (regexp), formatted as:/regular expression/
Awk-f: '/^u/{print $ '/etc/passwd

B. expression, value bits not 0 or bit non-null are eligible, such as $ ~/foo/or $ = = "Root"
Awk-f: ' $3>=500{print $1,$3,$7} '/etc/passwd
Awk-f: ' $3+1<=100&&$3+1>=10{print $1,$3,$7} '/etc/passwd
Awk-f: ' $2== '! {print $1,$2} '/etc/shadow # #检查未初始化密码的用户
Passwd-d u01
Awk-f: ' $2== ' "{print $} '/etc/shadow # #打印密码为空的用户
Awk-f: ' $7~ ' bash$ ' {print $1,$3,$7} '/etc/passwd # #匹配 $7 end line for bash
Awk-f: ' $7! ~ "bash$" {print $1,$3,$7} '/etc/passwd

C. Match range (ranges), specified match range, formatted as Part1,part2
Awk-f: ' $3==3,$3==10{print $1,$3,$7} '/etc/passwd
Awk-f: ' $1== ' root, $1== ' adm ' {print $1,$3,$7} '/etc/passwd
Awk-f: '/^r/,/^a/{print $1,$3,$7} '/etc/passwd
D. Special mode (Begin/end)
Awk-f: ' begin{printf "%-10s%-10s%-20s\n", "UserName", "ID", "Shell"}{printf "%-10s%-10s%-20s\n", $1,$3,$7} '/etc/ passwd # #在awk处理之前打印头部BEGIN {}
Awk-f: ' begin{printf "%-10s%-10s%-20s\n", "UserName", "ID", "Shell"}$7~ "bash$" {printf "%-10s%-10s%-20s\n", $1,$3,$7} ' /ETC/PASSWD # #多个模式混合使用
Awk-f: ' begin{printf '%-10s%-10s%-20s\n ', ' UserName ', ' ID ', ' Shell '}$7~ ' bash$ ' {printf '%-10s%-10s%-20s\n ', $1,$3,$7} End{print "End of Report"} '/etc/passwd # #在awk执行完成后打印尾部END {}
Awk-v i=1 ' $5~ "yum" {i++}end{print "Yum use times:", i} '/var/log/messages
/Regular Expression/: An extended collection using wildcards.
Relational expressions: Comparisons such as strings or numbers.
Pattern-Matching expression: pattern (Specifies the range of a row, not including begin,end)

7) Common action:
Expressions: An expression
Control satatements: Controlling statements
Compound statements: Mixed statement
Input statements: Enter statement
Output statements: Export statement

8) The control statement in action:
A.if-else
Syntax: if (condition) command action1; else Command Action2
Case:
Awk-f: ' $7~ ' bash {if ($1== "root") print $, "admin", else print $, "Common User"} '/etc/passwd
Awk-f:-V sum=0 ' {if ($3>=500) sum++}end{print sum} '/etc/passwd
Awk-f:-v sum=0 ' $7~ "bash" {if ($1== "root") print $, "admin" sum++;else print $, "Common User" Sum++}end{print "can log In user num: ", sum} '/etc/passwd # #二合一
Awk-f: ' $7~ ' bash$ ' {if ($1== "root") printf "%-15s:%s\n", $, "admin", Else printf "%-15s:%s\n", $, "Common user"} '/etc/pas SWD # #格式化输出

B.while loop: Loop field, awk itself is a loop of rows
Syntax: while (condition) {statement1;statement2; ...}
Case:
Awk-f: ' {i=1;while (i<=3) {print $i; i++}} '/etc/passwd # #循环打印前3列
Awk-f: ' {i=1;while (I&LT;NF) {if (length ($i) <=4) print $i; i++}} '/etc/passwd # #循环整行, and print out a field column of less than 4 length


C.do-while
Syntax: do {statement1;statement2; ...} while (condition)
Awk-f: ' {i=1;do {print $i; I++}while (i<=3)} '/etc/passwd

D.for
Syntax: for (variable assignment;condition;iteration process) {statement1, Statement2,...}
Case:
Awk-f: ' {for (i=1;i<=3;i++) print $i} '/etc/passwd

The For loop iterates through the array elements:
Syntax: for (i in array) {statement1, Statement2,...}
Case:
Awk-f: ' {sh[$NF]++;} End{for (i in SH) {print i,sh[i]}} '/etc/passwd
Netstat-ant |awk '/^tcp/{++s[$NF]}end{for (A in S) print A, s[a]} '
awk ' {counts[$1]++}end{for (URL in counts) print URL, "Access times:", Counts[url]} '/var/log/httpd/access_log
awk ' {s[$5]++}end{for (i in S) print I,s[i]} '/var/log/messages
awk ' {ah[$1]++}end{for (i in AH) printf "%-20s:%s\n", I, Ah[i]} '/var/log/httpd/access_log
9). Arrays in awk
Array[index-expression]
Index-expression can use arbitrary strings; It is important to note that if a data group element does not exist beforehand, awk automatically creates this element and initializes it to an empty string when it is referenced, so you need to use the index in to determine whether an element exists in a data group. Array in the same way. To enumerate through each element of a group, you need to use a special structure like this:
for (var in array) {statement1, ...}
For example:
State[abc]=3
State[efg]=6
For (A in state) {print a,state[a]} # #A一定是index-expression (subscript ABC/EFG), State[a] is a specific value (3/6)
[Email protected] ~]# netstat-ant
Active Internet connections (servers and established)
Proto recv-q send-q Local address Foreign address state
TCP 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
TCP 0 0 127.0.0.1:25 0.0.0.0:* LISTEN
TCP 0 0 192.168.1.106:22 192.168.1.101:36318 established
The above command shows the result, each line of the last field of the state, to count the number of each state, only need to be the status of the array subscript, because awk itself is a row of the loop, so only need to add 1 (+ +) to the number to achieve statistics, the following:
Netstat-ant |awk ' {s[$6]++}end{for (i in S) {print i,s[i]}} '

Awk's built-in functions
Split (String,array[filedsep[,seps])
Function: Separates string representations of strings with fieldsep as delimiters, and saves the separated results to an array with an array name, and the arrays are labeled as a sequence starting from 0;

# Netstat-ant | awk '/:80\>/{split ($5,clients, ":"); Ip[clients[1]]++}end{for (i in IP) {print ip[i],i}} ' | Sort-rn | Head-50

Length ([string])

Function: Returns the number of characters in string strings;

SUBSTR (String, start [, length])

Function: Take a substring in string, start with start, take length, start counting from 1;

System (Command)

Function: Execute system command and return the result to the awk command

Systime ()

Function: Take system current time

ToLower (s)

Function: Convert all letters in s to lowercase

ToUpper (s)

Function: Convert all letters in s to uppercase

This article is from the blog, "to learn the good system to work smoothly", please keep this source http://stlong.blog.51cto.com/5144113/1680855

Awk learning experience--super detail--case study