awk command, AWK programming language detailed introduction and instance _linux Shell

One, what is awk

awk is a command under Linux, his output of other commands, the processing of the file is very powerful, in fact, he is more like a programming language, he can customize variables, conditional statements, there are loops, there are arrays, there are regular, there are functions, and so on. He reads the output, or file The way is a row, read a line, according to the conditions you give to find, and in the line to find out the operation, feel his design idea, really very simple, but combined with the actual situation, concrete operation is not so simple. He has three kinds of situation, Awk,gawk,nawk, usually said awk is actually gawk.

Ii. records, fields, and delimiters in awk

When we read the output, or read the file, reading a row is a record. The record delimiter is a carriage return by default and is saved in rs,ors.
We separate the word from the record, or the phrase, we call him the domain, the domain delimiter, the default is the Space and tab 銉, save in the built-in change
Amount of ORS. As an example:
Aaaa:bbbb:ccccccc
1111:2343:5t43343
There are two lines above, these two lines is two records, after each line of carriage return, is the record separator, inside the colon, is the domain separator, split out, aaaa,1111 this kind of thing is the domain.
Awk-f: ' {print $} ' testfile

Three, awk's built-in variables and operators

1, variable

variable	Description
$n	The nth field in the current record, separated by FS between fields.
$	The full input record.
ARGC	The number of command line arguments.
Argind	The location of the current file on the command line, starting at 0.
Argv	An array that contains command line arguments.
Convfmt	numeric conversion format (default value is%.6g)
ENVIRON	An associative array of environment variables.
Errno	Description of the last system error.
FieldWidths	A list of field widths, separated by a space key.
FILENAME	The current file name.
FNR	With NR, but relative to the current file.
Fs	The field delimiter (the default is any space).
IGNORECASE	If true, a match is ignored for case-insensitive.
Nf	The number of fields in the current record.
Nr	The current number of records.
Ofmt	The output format of the number (the default value is%.6g).
OFS	The Output field delimiter (the default is a space).
ORS	The output record delimiter (the default is a newline character).
Rlength	The length of the string that is matched by the match function.
Rs	Record delimiter (default is a newline character).
Rstart	The first position of the string that is matched by the match function.
Subsep	The array subscript delimiter (the default is \034).

2, operator

operator	description
= = = = *=/=%= ^= **=	assignment
?:	c conditional Expression
| | |	Logical OR
&&	logic and
~ ~!	matches regular expressions and mismatched regular expressions
< <= > >=!= = =	relational operator
space	connection
+-	Plus, minus
*/n	multiply, except with remainder
+-!	unary Plus, minus and logical non
^ * * *	exponentiation
+ +--	increase or decrease, as a prefix or suffix
$	field reference
in	array members

Four, the regular of awk

Match character	Description
\y	Match an empty string at the beginning or end of a word
\b	Match an empty string within a word
\<	An empty string that matches the beginning of a word, anchoring begins
\>	An empty string that matches the end of a word, anchored at the end
\w	Match a word that is not alphanumeric
\w	Match a word that is made up of alphanumeric characters
\'	Matches an empty string at the end of a string
\‘	An empty string that matches the beginning of a string

Five, the function of awk

1, String function

The

name of the function	Description
Sub	A regular expression that matches the largest, most-left substring in the record, and replaces those strings with a replacement string. The entire record is used by default if no target string is specified. The substitution occurs only when the first match is taken.
Gsub	Match across the document
Index	Returns the position at which the substring was first matched, starting at position 1.
Substr	Returns a substring starting at position 1, returning the entire string if the specified length exceeds the actual length
Split	You can divide a string into an array by the given delimiter. If the delimiter is not supplied, split by current FS value
Length	Returns the number of characters in a record
Match	Returns the index of the position of the regular expression in the string, or 0 if the specified regular expression cannot be found. The match function sets the Rstart variable to the starting position of the substring of the string, rlength to the number of characters to the end of the substring. SUBSTR can be beneficial for these variables to intercept strings
ToUpper and ToLower	Can be used for conversions between string sizes, which are only valid in Gawk

2, Mathematical functions

The

name of the function	return value
ATAN2 (X,y)	Y,x within the range of cotangent
COS (x)	cosine function
EXP (x)	exponentiation
int (x)	Take the whole
Log (x)	Natural logarithm
RAND ()	Random number
Sin (x)	Sinusoidal
sqrt (x)	Square root
Srand (x)	X is the seed of the rand () function
int (x)	Rounding, the procedure is not rounded
RAND ()	Produces a random number that is greater than or equal to 0 and less than 1

Vi.. Examples

When I was learning awk, I did an example to learn something, not just to look at it, but to remember things. Just look at the words, maybe you know what's going on, really practical, not here is a problem, that is there is a problem. So be sure to do it yourself.

1, test file testing

Copy Code code as follows:

Root:x:0:0:root:/root:/bin/bash
Bin:x:1:1:bin:/bin:/bin/false
Daemon:x:2:2:daemon:/sbin:/bin/false
Mail:x:8:12:mail:/var/spool/mail:/bin/false
Ftp:x:14:11:ftp:/home/ftp:/bin/false
&nobody:$:99:99:nobody:/:/bin/false
zhangy:x:1000:100:,,,:/home/zhangy:/bin/bash
Http:x:33:33::/srv/http:/bin/false
Dbus:x:81:81:system message Bus:/:/bin/false
Hal:x:82:82:hal Daemon:/:/bin/false
Mysql:x:89:89::/var/lib/mysql:/bin/false
Aaa:x:1001:1001::/home/aaa:/bin/bash
Ba:x:1002:1002::/home/zhangy:/bin/bash
Test:x:1003:1003::/home/test:/bin/bash
@zhangying:*: 1004:1004::/home/test:/bin/bash
Policykit:x:102:1005:po

Example 1:

Copy Code code as follows:

Cat Test | Awk-f: ' {\
if ($ = = "root") {\
Print $1;\
}else if ($ = = "Bin") {\
Print $2;\
}else{\
Print $3;\
} \
}'

Example 2:

Copy Code code as follows:

awk ' {\
for (i=0;i<nf;i++) {\
if ($i ~/^root/) {\
print $i; \
}else if ($i ~/zhangy/) {\
Print $i; continue;\
}else if ($i ~/mysql/) {\
Print $i; next;\
}else if ($i ~/^test/) {\
Print $i; break;\
} \
}\
} ' Test

Example 3:

Copy Code code as follows:

Tail Test | awk ' Begin{while (getline d) {split (d,test); for (i in test) {\
Print test[i]\
}}}'

Example 4:

Copy Code code as follows:

Ls-al/home/zhangy/mytest | awk ' Begin{while (getline d) {split (d,test); \
Print test[9];}
}'

Example 5:

Copy Code code as follows:

echo "32:34" |awk-f: ' {print ' max = ', max ($1,$2)}\
function Max (one,two) {
if (one > two) {
return one;
}else{
return two;
}
}
'

Example 6:

Copy Code code as follows:

#awk ' Begin{print ' what is your name; Getline name < "/dev/tty"}$1 ~name{print
# "found name on line" Nr}end{print "" Name} ' test
#awk ' {sub (/daemon/, ' Tankzhang ');p rint} ' test
#awk ' {{sub (/zhangy/, ' Tankzhang ');};p rint} ' test

#awk ' {{gsub (/zhangy/, ' Tankzhang ');};p rint} ' test
#awk-F: ' {print index (' Zhangy ', $}} ' test
#awk-F: ' {print substr ($1,1,2)} ' test
Awk-f: ' {mat=match ($1,/^[a-za-z]+$/);p rint mat,rstart,rlength} ' test

Example 7:

Copy Code code as follows:

Cat test |awk-f: ' \
NF!= 7{\
printf ("line%d,does not have 7 fields:%s\n", nr,$0)}\
$!~/^[a-za-z0-9]/{printf ("line%d,non Alpha and numeric user id:%s:%s\n", nr,$1,$0)}\
$ = "*" {printf ("Lind%d,no password:%s\n", nr,$0)} '