awk command, AWK programming language detailed introduction and instance _linux Shell

Source: Internet
Author: User
Tags numeric mathematical functions natural logarithm rand square root alphanumeric characters

One, what is awk

awk is a command under Linux, his output of other commands, the processing of the file is very powerful, in fact, he is more like a programming language, he can customize variables, conditional statements, there are loops, there are arrays, there are regular, there are functions, and so on. He reads the output, or file The way is a row, read a line, according to the conditions you give to find, and in the line to find out the operation, feel his design idea, really very simple, but combined with the actual situation, concrete operation is not so simple. He has three kinds of situation, Awk,gawk,nawk, usually said awk is actually gawk.

Ii. records, fields, and delimiters in awk

When we read the output, or read the file, reading a row is a record. The record delimiter is a carriage return by default and is saved in rs,ors.
We separate the word from the record, or the phrase, we call him the domain, the domain delimiter, the default is the Space and tab 銉, save in the built-in change
Amount of ORS. As an example:
There are two lines above, these two lines is two records, after each line of carriage return, is the record separator, inside the colon, is the domain separator, split out, aaaa,1111 this kind of thing is the domain.
Awk-f: ' {print $} ' testfile

Three, awk's built-in variables and operators

1, variable

variable Description
$n The nth field in the current record, separated by FS between fields.
$ The full input record.
ARGC The number of command line arguments.
Argind The location of the current file on the command line, starting at 0.
Argv An array that contains command line arguments.
Convfmt numeric conversion format (default value is%.6g)
ENVIRON An associative array of environment variables.
Errno Description of the last system error.
FieldWidths A list of field widths, separated by a space key.
FILENAME The current file name.
FNR With NR, but relative to the current file.
Fs The field delimiter (the default is any space).
IGNORECASE If true, a match is ignored for case-insensitive.
Nf The number of fields in the current record.
Nr The current number of records.
Ofmt The output format of the number (the default value is%.6g).
OFS The Output field delimiter (the default is a space).
ORS The output record delimiter (the default is a newline character).
Rlength The length of the string that is matched by the match function.
Rs Record delimiter (default is a newline character).
Rstart The first position of the string that is matched by the match function.
Subsep The array subscript delimiter (the default is \034).

2, operator

operator description
= = = = *=/=%= ^= **= assignment
?: c conditional Expression
| | | Logical OR
&& logic and
~ ~! matches regular expressions and mismatched regular expressions
< <= > >=!= = = relational operator
space connection
+- Plus, minus
*/n multiply, except with remainder
+-! unary Plus, minus and logical non
^ * * * exponentiation
+ +-- increase or decrease, as a prefix or suffix
$ field reference
in array members

Four, the regular of awk

Match character Description
\y Match an empty string at the beginning or end of a word
\b Match an empty string within a word
\< An empty string that matches the beginning of a word, anchoring begins
\> An empty string that matches the end of a word, anchored at the end
\w Match a word that is not alphanumeric
\w Match a word that is made up of alphanumeric characters
\' Matches an empty string at the end of a string
\‘ An empty string that matches the beginning of a string

Five, the function of awk

1, String function

name of the function Description
Sub A regular expression that matches the largest, most-left substring in the record, and replaces those strings with a replacement string. The entire record is used by default if no target string is specified. The substitution occurs only when the first match is taken.
Gsub Match across the document
Index Returns the position at which the substring was first matched, starting at position 1.
Substr Returns a substring starting at position 1, returning the entire string if the specified length exceeds the actual length
Split You can divide a string into an array by the given delimiter. If the delimiter is not supplied, split by current FS value
Length Returns the number of characters in a record
Match Returns the index of the position of the regular expression in the string, or 0 if the specified regular expression cannot be found. The match function sets the Rstart variable to the starting position of the substring of the string, rlength to the number of characters to the end of the substring. SUBSTR can be beneficial for these variables to intercept strings
ToUpper and ToLower Can be used for conversions between string sizes, which are only valid in Gawk

2, Mathematical functions

name of the function return value
ATAN2 (X,y) Y,x within the range of cotangent
COS (x) cosine function
EXP (x) exponentiation
int (x) Take the whole
Log (x) Natural logarithm
RAND () Random number
Sin (x) Sinusoidal
sqrt (x) Square root
Srand (x) X is the seed of the rand () function
int (x) Rounding, the procedure is not rounded
RAND () Produces a random number that is greater than or equal to 0 and less than 1

Vi.. Examples

When I was learning awk, I did an example to learn something, not just to look at it, but to remember things. Just look at the words, maybe you know what's going on, really practical, not here is a problem, that is there is a problem. So be sure to do it yourself.

1, test file testing

Copy Code code as follows:

Dbus:x:81:81:system message Bus:/:/bin/false
Hal:x:82:82:hal Daemon:/:/bin/false
@zhangying:*: 1004:1004::/home/test:/bin/bash

Example 1:

Copy Code code as follows:

Cat Test | Awk-f: ' {\
if ($ = = "root") {\
Print $1;\
}else if ($ = = "Bin") {\
Print $2;\
Print $3;\
} \

Example 2:

Copy Code code as follows:

awk ' {\
for (i=0;i<nf;i++) {\
if ($i ~/^root/) {\
print $i; \
}else if ($i ~/zhangy/) {\
Print $i; continue;\
}else if ($i ~/mysql/) {\
Print $i; next;\
}else if ($i ~/^test/) {\
Print $i; break;\
} \
} ' Test

Example 3:

Copy Code code as follows:

Tail Test | awk ' Begin{while (getline d) {split (d,test); for (i in test) {\
Print test[i]\

Example 4:

Copy Code code as follows:

Ls-al/home/zhangy/mytest | awk ' Begin{while (getline d) {split (d,test); \
Print test[9];}

Example 5:

Copy Code code as follows:

echo "32:34" |awk-f: ' {print ' max = ', max ($1,$2)}\
function Max (one,two) {
if (one > two) {
return one;
return two;

Example 6:

Copy Code code as follows:

#awk ' Begin{print ' what is your name; Getline name < "/dev/tty"}$1 ~name{print
# "found name on line" Nr}end{print "" Name} ' test
#awk ' {sub (/daemon/, ' Tankzhang ');p rint} ' test
#awk ' {{sub (/zhangy/, ' Tankzhang ');};p rint} ' test

#awk ' {{gsub (/zhangy/, ' Tankzhang ');};p rint} ' test
#awk-F: ' {print index (' Zhangy ', $}} ' test
#awk-F: ' {print substr ($1,1,2)} ' test
Awk-f: ' {mat=match ($1,/^[a-za-z]+$/);p rint mat,rstart,rlength} ' test

Example 7:

Copy Code code as follows:

Cat test |awk-f: ' \
NF!= 7{\
printf ("line%d,does not have 7 fields:%s\n", nr,$0)}\
$!~/^[a-za-z0-9]/{printf ("line%d,non Alpha and numeric user id:%s:%s\n", nr,$1,$0)}\
$ = "*" {printf ("Lind%d,no password:%s\n", nr,$0)} '

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.