The Linux foundation of AWK

GNU awk:

Text Processing three tools: grep, sed, awk

grep, Egrep, Fgrep: Text filtering tool; pattern

Sed: line Editor

Pattern space, hold space

awk: Report Generator, formatted text output;

Awk:aho, Weinberger, Kernighan---New AWK, Nawk

GNU awk, Gawk

Gawk-pattern scanning and processing language

Basic usage: gawk [options] ' program ' FILE ...

Program:pattern{action statements}

The statements are separated by semicolons

Print, printf

Options:

-F: Indicates the field delimiter used in the input;

-V var=value: Custom variable;

1. Print

Print item1, item2, ...

Points:

(1) comma delimiter;

(2) Each item of the output can be a string, or it can be a numeric value; The field, variable, or awk expression of the current record;

(3) If the item is omitted, it is equivalent to print $;

2. Variables

2.1 Built-in variables

Fs:input field seperator, default to white space characters;

Ofs:output field seperator, default to white space characters;

Rs:input record Seperator, enter the line break;

Ors:output record seperator, line break at output;

Nf:number of field, number of fields

{print NF}, {print $NF}

Nr:number of record, number of rows;

FNR: Each file is counted, the number of rows;

FileName: Current file name;

ARGC: The number of command-line arguments;

ARGV: An array that holds the parameters given by the command line;

2.2 Custom variables

(1)-V Var=value

Variable names are case-sensitive;

(2) directly defined in program

3. printf command

Formatted output: printf format, item1, ITEM2, ...

(1) format must be given;

(2) does not wrap automatically, you need to explicitly give the line-break control, \ n

(3) in format, you need to specify a format symbol for each item that follows.

Format characters:

%c: The ASCII code that displays the characters;

%d,%i: Displays decimal integers;

%e,%e: Numerical display of scientific counting method;

%f: Displayed as floating point number;

%g,%g: Displays values in scientific notation or floating-point form;

%s: Display string;

%u: unsigned integer;

Percent: show% itself;

Modifier:

#[.#]: The width of the first digital control display; The second # indicates the precision after the decimal point;

%3.1f

-: Align Left

+: Display symbols for numeric values

4. Operator

Arithmetic operators:

X+y, X-y, x*y, x/y, X^y, x%y

-X

+x: converted to numerical value;

String operator: unsigned operator, string connection

Assignment operators:

=, +=, -=, *=, /=, %=, ^=

++, --

Comparison operators:

>=, <, <=,! =, = =

Pattern-matching characters:

~: whether match

!~: does not match

Logical operators:

&&

||

!

Function call:

Function_name (ARGU1, ARGU2, ...)

Conditional expression:

Selector?if-true-expression:if-false-expression

# awk-f: ' {$3>=1000?usertype= ' Common User ': usertype= "Sysadmin or Sysuser";p rintf "%15s:%-s\n", $1,usertype} '/etc/ passwd

5. PATTERN

(1) Empty: null mode, matching each line;

(2)/regular expression/: Only the rows that can be matched to the pattern here are processed;

(3) Relational expression: The relationship expressions, the result is "true" has "false", the result is "true" will be processed;

True: The result is a value other than 0, not an empty string;

(4) Line ranges: range,

startline,endline:/pat1/,/pat2/

Note: Formats that give numbers directly are not supported

~]# awk-f: ' (nr>=2&&nr<=10) {print '} '/etc/passwd

(5) Begin/end mode

begin{}: Executes only once before starting to process the text in the file;

end{}: Executes only once after the text processing is complete;

6. Commonly used action

(1) Expressions

(2) Control statements:if, while and so on;

(3) Compound statements: combined statement;

(4) Input statements

(5) Output statements

7. Control statements

if (condition) {statments}

if (condition) {statments} else {statements}

while (Conditon) {statments}

Do {statements} while (condition)

for (EXPR1;EXPR2;EXPR3) {statements}

Break

Continue

Delete Array[index]

Delete array

Exit

{statements}

7.1 If-else

Syntax: if (condition) statement [Else statement]

~]# awk-f: ' {if ($3>=1000) {printf ' Common User:%s\n ', $ ' else {printf ' root or Sysuser:%s\n ', ' $ '} '/etc/passwd

~]# awk-f: ' {if ($NF = = "/bin/bash") print $ '/etc/passwd

~]# awk ' {if (nf>5) print $} '/etc/fstab

~]# Df-h | awk-f[%] '/^\/dev/{print $ ' | awk ' {if ($NF >=20) print $} '

Usage scenario: Make a conditional judgment on the entire row or field obtained by awk;

7.2 While Loop

Syntax: while (condition) statement

The condition "true", enters the circulation, the condition "false", exits the circulation;

Usage Scenario: Use when processing multiple fields in a row one at a time, using each element of an array in a single process;

~]# awk '/^[[:space:]]*linux16/{i=1;while (i<=nf) {print $i, length ($i); i++}} '/etc/grub2.cfg

~]# awk '/^[[:space:]]*linux16/{i=1;while (I<=NF) {if (length ($i) >=7) {print $i, Length ($i)}; i++}} '/etc/ Grub2.cfg

7.3 Do-while Cycle

Syntax: do statement while (condition)

Meaning: At least one loop body is executed

7.4 For Loop

Syntax: for (EXPR1;EXPR2;EXPR3) statement

For (variable assignment;condition;iteration process) {For-body}

~]# awk '/^[[:space:]]*linux16/{for (i=1;i<=nf;i++) {print $i, Length ($i)}} '/etc/grub2.cfg

Special usage:

Ability to iterate through the elements in an array;

Syntax: for (var in array) {For-body}

7.5 Switch statement

Syntax: switch (expression) {case VALUE1 or/regexp/: statement, Case VALUE2 or/regexp2/: statement; ...; default:statement}

7.6 Break and continue

Break [n]

Continue

7.7 Next

End the processing of the bank in advance and go directly to the next line;

~]# awk-f: ' {if ($3%2!=0) next; print $1,$3} '/etc/passwd

8. Array

Associative array: array[index-expression]

Index-expression:

(1) You can use any string; string to use double quotation marks;

(2) If an array element does not exist beforehand, when referenced, awk automatically creates this element and initializes its value to "empty string";

To determine if an element exists in an array, use the "index in array" format;

weekdays[mon]= "Monday"

To iterate through each element in the array, use the For loop;

for (var in array) {For-body}

~]# awk ' begin{weekdays["Mon"]= "Monday" weekdays["Tue"]= "Tuesday"; for (I in weekdays) {print Weekdays[i]}} '

Note: Var iterates through each index of the array;

state["LISTEN"]++

state["established"]++

~]# Netstat-tan | awk '/^tcp\>/{state[$NF]++}end{for (i in state) {print I,state[i]}} '

~]# awk ' {ip[$1]++}end{for (i in IP) {print i,ip[i]}} '/var/log/httpd/access_log

Exercise 1: Count the number of occurrences of each file system type in the/etc/fstab file;

~]# awk '/^uuid/{fs[$3]++}end{for (i in FS) {print I,fs[i]}} '/etc/fstab

Exercise 2: Count the occurrences of each word in the specified file;

~]# awk ' {for (i=1;i<=nf;i++) {count[$i]++}}end{for (i in count) {print I,count[i]}} '/etc/fstab

9. Functions

9.1 Built-in functions

Numerical Processing:

RAND (): Returns a random number between 0 and 1;

String processing:

Length ([s]): Returns the length of the specified string;

Sub (r,s,[t]): Finds the matched content in the character represented by T in the pattern represented by R and replaces it with the content represented by S for the first time;

Gsub (R,s,[t]): Finds the matched content in the character represented by T in the pattern represented by R and replaces all occurrences with the content represented by S;

Split (S,a[,r]): Cuts the character s with the R delimiter and saves the resulting cut to the array represented by A;

~]# Netstat-tan | awk '/^tcp\>/{split ($5,ip, ":"); Count[ip[1]]++}end{for (i in count) {print I,count[i]}} '

9.2 Custom Functions

"Sed and awk"

The Linux foundation of AWK