Linux awk Usage

AWK

awk: Report Generator, formatted text output

Gawk: Mode scan and processing language

Basic syntax:

Options: Option

Program:awk's language

Var=value: Assigning a value to Var

File: Parameter

awk [Options] ' program ' var=value file ...

awk [Options]-F programfile var=value file ...

awk [Options] ' begin{action} pattern{action; End{action, ...} ' file:

Options:

-F Specify delimiter

-V var=value: Custom variable

awk language

Basic format: awk [options] ' program ' File ....

Program:pattern{action statements, ...}

Pattern and action:

The pattern section determines when an action statement triggers and triggers an event

BEGIN: Perform an action before the file is processed

End: Performs an action when all file processing finishes

Action statements the data and places it within {} to indicate

print,printf

separators, fields, and records

When awk executes, delimited fields (fields) are marked $1,$2 ... $n become a domain identifier;

Each line of the file is called a record, or you can use something else as a record delimiter

Omit action, default execution print $

How awk Works

The first step: if you have Begin{action, ...} Priority implementation BEGIN

Step two: Read a line from the execution result of the command or file, then execute the pattern{action ...}, row by line

Step three: Execute end{action;

Awk

Item= the characters to be printed

Print format: Print item1,item2,...

Attention:

(1) Comma delimiter

(2) Each item of the output can be a string, or it can be a numeric value; An expression of a field, variable, or awk of the current record

(3) If item is omitted, it is equivalent to print $

awk variable

Field fields, columns, column column, attributes (one meaning)

To record (a meaning).

Variables: Built-in and custom variables

When using variables, add the-v option

FS: Delimiter for field, default to white space character

Awk-v fs= ': ' {print $1,fs,$2} ' passwd (specifies the FS variable value: Prints the first and second columns of the passwd file, with commas as whitespace characters)

OFS: definition of output delimiter

Awk-v ofs= "+" ' {print $1,$2} ' passwd (Specify the OFS variable value is +, print the first and second columns of the passwd file, the output delimiter is +)

RS: Specify separators for separate records

Record: Default carriage return line count one record

Awk-v rs= ";" The value in ' {print $} ' f1.txt is AA bb cc;ee

FF OO;XX

YY ZZ (Specifies the value of the RS variable; Print the second column of the F1.txt file, the result is BB ff yy)

ORS: Specifies the delimiter for record output

Awk-v rs= ""-V ors= "# # #" ' {print} ' passwd (by default, each record is differentiated by carriage return, ORS specifies the delimiter that distinguishes each record)

NF: Number of fields

Awk-f: ' {print NF} ' passwd (row by line, all displayed is 7)

Awk-f: ' {print $NF} ' passwd (line-by-row, showing the 7th column of data as NF is 7)

Awk-f: ' {print $ (NF-1)} ' passwd (progressive, displaying 7-1 columns of data)

NR: Record number

awk ' {print nr,$0} ' passwd (progressive, each line is printed, and the line number of each row is displayed)

FNR: Files are counted separately

awk ' {print fnr,$0} '/etc/fstab/etc/issue (two files showing all rows and line numbers respectively)

FileName: Current file name

awk ' {print fnr,filename,$0} '/etc/fstab/etc/issue (two files showing all rows and file names shown above)

ARGC: Number of parameters

awk ' {print ARGC} '/etc/fstab/etc/issue (view awk parameters)

ARGV: An array that displays the command line awk is the first parameter, skips the print section, followed by the second argument, and the third parameter

awk ' {print argv[0]} '/etc/fstab/etc/issue

Variables can also be defined in print

Awk's-f option reads the file and writes a series of data such as ' {print} ' to the file.

printf command

Formatted output: printf "format", item1,item2,....

(1) format must be specified

(2) does not wrap automatically, need to display the line break control, \ n

(3) Format must be specified for each subsequent item

Format character: Corresponds to item one by one

%c: ASCII code for displaying characters

%d,%i: Displaying decimal integers

%e,%e: Display scientific counting method values

%f: Displayed as a floating-point number

%g,%g: Displaying values in scientific notation or floating-point form

%s: Display string

%u: unsigned integer

Percent: show% itself

Modifier:

#[.#]: The width of the first digital control display; The second # indicates the precision after the decimal point,%3.1f

-: Left-justified (default right-aligned)%-15s (15 width in front and left-aligned display)

+: Displays the positive and negative sign of the value%+d

Operator:

X+y, X-y, x*y, x/y, X^y, x%y

-x: Convert to negative

+x: Convert to Numeric

Assignment operators:

=, +=, -=, *=, /=, %=, ^=

++, --

Comparison operators:

= =,! =,, >=, <, <=

Pattern-matching characters:

~: Whether the left and right match contains!~: does not match

You can use extended regular expressions, "" write regular expressions, or//write regular expressions

Example: awk–f: ' $ ~/^root/'/etc/passwd matches a line in a file that starts with root

Awk-f: ' $3>=0 && $3<=1000 {print '} '/etc/passwd (print match the third field is greater than 0 and the third field is less than 1000)

Awk-f: ' $3==0 | | $3>=1000 {print '} '/etc/passwd (printing matches the third field equals 0 or the third field is greater than 1000)

Awk-f: '! ($3==0) {print '} '/etc/passwd (prints a line with a third field that is not equal to 0)

Awk-f: '! ($3>=500) {print $} '/etc/passwd (a row for a field that does not have a third field equal to 500)

In awk, the variable is "" False, 0 is false, the value is True

Conditional expression (trinocular expression: using three expressions, respectively: divided into three sections)

Example: awk-f: ' {$3>=1000?usertype= ' common user ': Usertype= "Sysuser";p rintf "%-15s:%-30s%d \ n", usertype,$1,$3} '

awk PATTERN

Pattern: The part that precedes the curly braces

(1) If not specified: Empty mode, match each row

(2)/Regular expression/

(3) Relationship expression, the result is true will be processed

True: The result is a non-0 value, non-empty string

Add: The result is an empty string or a value of 0

(4) Line ranges: Range

'/Condition/,/condition/' The first condition must match, the second condition will not match to the end

(5) Begin/end mode

begin{}: Process files before processing

end{}: Processed once file processing is complete

awk action

Common action Categories

(1) Arithmetic, expression

(2) If,while, etc.

(3) The combination statement is?: Separate

(4) Input statement

(5) Output: Print

If statement

Syntax: if (expression) {action performed; ...} [Else perform action]

While loop: can be used when the number of fields is too high and must be executed

Syntax: while (expression) {action performed; ...}

Condition "true", enter loop; condition "false" Exit loop

Do-while Cycle

Syntax: do{perform operation;} while (expression)

Meaning: True or false, at least once

Next

End the processing of the bank in advance and proceed directly to the next line (awk itself loops)

awk Array

Associative array: array[subscript Custom]

(1) You can use any string; strings are enclosed in double quotes.

awk '!arr[$0]++ ' file name (remove duplicate row operation)

Iterates through an array.

for (var in array) {For-body}

awk ' {ip[$1]}end{for (i in IP) print I,ip[i]} '/var/log/access_log count the number of connections to the IP address

awk function

Numerical Processing:

RAND (): Returns a random number between 0 and 1, but if a separate print rand () is used in conjunction with the Srand () function that cannot generate a random number, the default generated floating-point numbers are generated as follows

Example: awk ' Begin{srand ();p rint Int (rand () *100)} ': Generate random integers

String processing:

Length ("Specify string"): Returns the length of the specified string

Sub (r,s, string to process): Search for the string to be processed R represents the modified string, S is the modified string, instead of the first match to the

Gsub: Is the replacement of all matches to

Split (S,ARRY,R): Cuts the string s, uses R as a delimiter, saves the resulting cut to an array of Arry, and becomes the value of the array

Custom functions

Format:

Function "Name" (parameter 1, parameter 2,..... Equivalent to the Shell's $1,$2,$3 ...) {

Code

return "expression"

}

The parameters of the call are the actual parameters, the name is the formal parameter; the names of the actual and formal parameters can be different, but the numbers are the same

Calling bash commands with system

Example: Awk BEGIN ' {System ("hostname")} '

If it is a variable that is defined in awk, awk will add the command to double quotes with the echo Call of system

Linux awk Usage