Linux system Text processing the Three Musketeers of awk

GNU awk:

Text Processing Three musketeers: Grep,sed,awk

Grep,egrep,fgrep: Text Filtering tool: pattern

Sed: line Editor

Pattern space, hold space

awk: Report Generator, formatted text output

Gawk-pattern scanning and processing language

Basic usage: gawk [options] ' program ' FILE ...

Program:pattern{action statements}

The statements are separated by semicolons

print,printf

Options:

-F: Indicates the field delimiter used in the input;

-V var=varlue: Custom variable;

1.print

Print item1,item2, ...

Points:

(1) comma delimiter;

(2) Each item of the output can be a string, or it can be a numeric value; A field, variable, or awk expression that is currently logged;

(3) If the item is omitted, it is equivalent to print $;

2. Variables

2.1 Built-in variables

Fs:input field seperator, default to white space characters;

Ofs:output field seperator, default to white space characters;

Rs:input record seperator, line break at output;

Ors:output record seperator, line break at output;

Nf:number of field, number of fields

{Print Nf},{print $NF}

Nr:number Ofrecord, line number;

FNR: Each file separately technology: number of rows;

FileName: Current file name;

ARGC: The number of command-line arguments;

ARGV: An array that holds the parameters given by the command line;

2.2 Custom variables

(1)-V Var=value

Variable name distinguishes character case;

(2) directly defined in program

3.printf command

Formatted output: printf format,item1,item2, ...

(1) format must be given;

(2) does not wrap automatically, need to display the line break control, \ n

(3) In format, you need to specify a format symbol for each subsequent item, respectively;

Format characters:

%c: ASCII code for displaying characters

%d: Display decimal integers

%e:,%e: Numerical display of scientific counting method;

%f: Displayed as a floating-point number

%g,%g: Displaying numerical values in the form of scientific or technical law or floating point;

%s: Display string;

%u: unsigned integer;

Percent: show% itself;

Modifier:

#[.#]: The width of the first digital control display; The second # indicates the precision after the decimal point

%3.1f

-: Do the right thing.

+: Display symbols for numeric values

4. Operators

Arithmetic operators:

X+y, X-y, x*y, x/y, X^y, x%y

-x: Convert to negative

+x: Convert to Numeric

String operator: unsigned operator, string connection

Assignment operators:

=, +=, -=, *=, /=, %=, ^=

++, --

Comparison operators:

>=, <, <=,! =, = =

Pattern-matching characters:

~: Whether the left and right matches contain

!~: does not match

Logical operators:

&&

||

!

Function call:

Function_name (ARGU1, ARGU2, ...)

Conditional expression:

Selector?if-true-expression:if-false-expression

# awk-f: ' {$3>=1000?usertype= ' Common User ': usertype= "Sysadmin or Sysuser";p rintf "%15s:%-s\n", $1,usertype} '/etc/ passwd

5.PATTERN

(1) Empty: null mode, matching each line;

(2)/regular expression/: Processing only the rows that can be matched to here;

(3) Relational expression: Relationship expressions: The result is real false, the result is true will be processed;

True: The result is a value other than 0, not an empty string;

(4) Line ranges: range;

startline,endline:/pat1/,/pat2/

Note: Formats that give numbers directly are not supported

(5) Begin/end mode

begin{}: Executes only once before the text of the file rollup is started;

end{}: Executes only once after the text processing is complete;

6. Common action

(1) Expressions

(2) Control statements:if,while and so on;

(3) Compound statments: Combined statement;

(4) Input statements

(5) Output statements

7. Control statements

if (condition) {statements; ...}

if (condition) {statements; ...} else {statements; ...}

while (Conditon) {statments; ...}

do {statements; ...} while (condition)

for (EXPR1;EXPR2;EXPR3) {statements; ...}

Break

Continue

Delete Array[index]

Delete array

Exit

7.1 If-else

Syntax:: if (condition) statement [Else statement]

Awk-f: ' {if ($3>=1000) {printf ' Common User:%s\n ', $ ' else {printf ' root or Sysuser:%s\n ', ' $ '} '/etc/passwd

Awk-f: ' {if ($NF = = "/bin/bash") print $ '/etc/passwd

awk ' {if (nf>5) print $} '/etc/fstab

df-h|awk-f% '/^\/dev/{print $ ' |awk ' $NF >=80{print $1,$5} '

Usage scenario: Make a conditional judgment on the entire row or field that awk obtains

7.2 While Loop

Syntax: while (condition) statement

The condition "true", enters the circulation, the condition "false", exits the circulation;

Usage Scenario: Use when processing multiple fields in a row one at a time, using each element of an array in a single process;

awk '/^[[:space:]]*linux16/{i=1;while (i<=nf) {print $i, length ($i); i++}} '/etc/grub2.cfg

awk '/^[[:space:]]*linux16/{i=1;while (I<=NF) {if (length ($i) >=10) {print $i, Length ($i)}; i++}} '/etc/grub2.cfg

7.3 Do-while Cycle

Syntax: do statement while (condition)

Meaning: At least one loop body is executed

7.4 For Loop

Syntax: for (EXPR1;EXPR2;EXPR3) statement

For (variable assifnment;condition;iteration process) {For-body}

awk '/^[[:space:]]*linux16/{for (i=1;i<=nf;i++) {print $i, Length ($i)}} '/etc/grub2.cfg

Special usage:

Ability to iterate through the elements in an array;

Syntax: for (var in array) {For-body}

7.5 Switch statement

Syntax:: switch (expression) {case VALUE1 or/regexp/: statement, Case VALUE2 or/regexp2/: statement; ...; default:statement}

7.6 Break and continue

Break [n]

Continue

7.7 Next

End the processing of the bank in advance and go directly to the next line;

Awk-f: ' {if ($3%2!=0) next; print $1,$3} '/etc/passwd

8. Array

Associative array: array[index-expression]

Index-expression

(1) Any string can be used;

(2) If an array element does not exist beforehand, when referenced, awk automatically creates this element and initializes its value to "empty string";

To determine if an element exists in an array, use the "index in array" format;

weekdays[mon]= "Monday"

To iterate through each element in the array, use the For loop;

for (var in array) {For-body}

awk ' begin{weekdays[' mon ']= "Monday" weekdays["Tue"]= "Tuesday"; for (iin weekdays) {print Weekdays[i]}} '

Note: Var iterates through each index of the array;

state["LISTEN"]++

state["established"]++

Netstat-tan | awk '/^tcp\>/{state[$NF]++}end{for (index in state) {print Index,state[index]}} '

awk ' {ip[$1]++}end{for (iin IP) {print i,ip[i]}} '/var/log/httpd/access_log

1. Count the number of occurrences of each file system type in the/etc/fstab file

# awk '/^uuid/{fs[$3]++}end{for (iin fs) {print I,fs[i]}} '/etc/fstab

2. Count the number of occurrences of each word in the/etc/fstab file;

# awk ' {for (i=1;i<=nf;i++) {count[$i]++}}end{for (iin count) {print I,count[i]}} '/etc/fstab

9. Functions

9.1 Built-in functions

Numerical processing

RAND (): Returns a random number between 0 and 1;

String processing:

Length ([s]): Returns the length of the specified string

Sub (r,s,[t]): Finds the matched content in the character represented by T in the pattern represented by R and replaces it with the content represented by S for the first time;

Gsub (R,s,[t]): To view the matching contents of the characters represented by T in the mode of R, and replace all occurrences with the content represented by S;

Split (S,a[,r]): Cuts the character s with the R delimiter and saves the resulting cut to the array represented by A;

Netstat-tan | awk '/^tcp\>/{split ($5,ip, ":"); Count[ip[1]]++}end{for (iin count) {print I,count[i]}} '

9.2 Custom Functions

This article is from the "11798474" blog, please be sure to keep this source http://11808474.blog.51cto.com/11798474/1854253

Linux system Text processing the Three Musketeers of awk