Linux Text Processing Three Musketeers--awk

I. Introduction of AWK

Aho, Weinberger, Kernighan co-developed by three people

Report Generator, formatted text output

Available in multiple Versions: New awk (Nawk), GNU awk (gawk)

Mainstream version on Linux: gawk– mode scanning and processing language

Ii. Basic usage of gawk

awk [Options] ' program ' var=value file ...

awk [Options]-F programfile var=value file ...

awk [Options] ' begin{action; ...} pattern{action; ...}  end{action, ...} ' File ...

The awk program usually consists of a BEGIN statement block, a universal statement block capable of using pattern matching, and an end statement block, consisting of a total of 3 parts

Program is usually placed in single or double quotation marks

Options:

-F indicates the field delimiter to use when entering

-V var=value: Custom variable

Third, the awk language

Basic format: awk [options] ' program ' File ...

program:pattern{action statements;..}

Pattern and action:

The pattern section determines when an action statement triggers and triggers an event (begin,end)

Action statements processing the data and placing it in {} indicated (print, printf)

separators, fields, and records

When Awk executes, a delimiter-delimited field (field) tag $1,$2: $n is called a domain identity. $ $ $ For all domains, note: and Shell variable $ characters have different meanings

Each line of the file is called a record

Omit action, default to print $

Iv. working principle of awk

The first step: Execute Begin{action ...} Statements in a statement block

Step two: Read a line from the file or standard input (stdin), then execute the pattern{action ...} Statement block, which scans the file line by row, repeating the process from the first line to the last line until the file is fully read

Step three: When reading to the end of the input stream, execute end{action ...} Statement block

The BEGIN statement block is executed before awk begins to read rows from the input stream, which is an optional block of statements, such as variable initialization, table-top statements for printed output tables, which can usually be written in the BEGIN statement block

The end statement block is executed after awk reads all the rows from the input stream, such as the analysis results for all rows, such as a summary of information that is done in the end statement block, which is also an optional statement block

The General command in the pattern statement block is the most important part and is optional. If the pattern statement block is not provided, the default is {print}, which prints every fetched row, and every row that awk reads executes the statement block

Print format: Print item1, item2, ...

Points:

(1) Comma delimiter

(2) Each item of the output can be a string, or it can be a numeric value; An expression of a field, variable, or awk of the current record

(3) If item is omitted, it is equivalent to print $

Example:

awk ' {print ' Hello,awk '} '

Awk–f: ' {print} '/etc/passwd

Awk–f: ' {print ' Wang '} '/etc/passwd

Awk–f: ' {print '} '/etc/passwd

Awk–f: ' {print $} '/etc/passwd

Awk–f: ' {print $ ' \ t ' $/etc/passwd} '

Tail–3/etc/fstab |awk ' {print $2,$4} '

V. awk variables (built-in variables, custom variables)

1. Built-in variables

FS: Enter field delimiter, default to white space character

Awk-v fs= ': ' {print $1,$3,$7} '/etc/passwd

Awk–f: ' {print $1,$3,$7} '/etc/passwd

OFS: Output field delimiter, default to white space character

Awk-v fs= ': '-v ofs= ': ' {print $1,$3,$7} '/etc/passwd

RS: Enter the record delimiter, specify the line break at input, and the original newline character is still valid

Awk-v rs= ' {print} '/etc/passwd

ORS: Output record delimiter, output with specified symbol instead of line break

Awk-v rs= "-v ors= ' # # # ' {print} '/etc/passwd

NF: Number of fields

Awk-f: ' {print NF} '/etc/fstab, reference built-in variable without $

Awk-f: ' {print $ (NF-1)} '/etc/passwd

NR: Line number

awk ' {print NR} '/etc/fstab; awk END ' {print NR} '/etc/fstab

FNR: Lists the line numbers of each file separately

awk ' {print FNR} '/etc/fstab/etc/inittab

FileName: Current file name (how many lines of the file are printed multiple)

awk ' {print FILENAME} '/etc/fstab

ARGC: Number of command line arguments

awk ' {print ARGC} '/etc/fstab/etc/inittab

awk ' BEGIN {print ARGC} '/etc/fstab/etc/inittab

ARGV: An array that holds the arguments given by the command line

awk ' BEGIN {print argv[0]} '/etc/fstab/etc/inittab

awk ' BEGIN {print argv[1]} '/etc/fstab/etc/inittab

2. Custom variables

(1)-V Var=value

Variable name Distinguishing character case

(2) directly defined in program

Example:

Awk-v test= ' Hello gawk ' {print test} '/etc/fstab

Awk-v test= ' Hello gawk ' begin{print test} '

awk ' begin{test= ' Hello,gawk ";p rint test} '

Vi. formatted output: printf "format", Item1, item2, ...

1. Format must be specified

2, does not wrap automatically, need to explicitly give the line-break control, \ n

3. Format must be specified for each subsequent item.

(1) Format character: corresponds to item one by one

%c: ASCII code for displaying characters

%d:%i: Display decimal integers

%e:%e: Display scientific counting method values

%f: Displayed as a floating-point number

%g:%g: Displaying values in scientific notation or floating-point form

%s: Display string

%u: unsigned integer

Percent: show% itself

(2) Modifier:

#[.#]: The width of the first digital control display; The second # indicates the precision after the decimal point,%3.1f

-: Left-justified (default right-aligned)%-15s

+: Displays the positive and negative sign of the value%+d

printf Example

Awk-f: ' {printf '%s ', ' $ '/etc/passwd

Awk-f: ' {printf '%s\n ', ' $ '/etc/passwd

Awk-f: ' {printf ' Username:%s\n ', $ '/etc/passwd

Awk-f: ' {printf ' Username:%s,uid:%d\n ', $1,$3} '/etc/passwd

Awk-f: ' {printf ' Username:%15s,uid:%d\n ', $1,$3} '/etc/passwd

Awk-f: ' {printf ' Username:%-15s,uid:%d\n ', $1,$3} '/etc/passwd

Vii. awk operator

Arithmetic operators:

X+y, X-y, x*y, x/y, X^y, x%y

-x: Convert to negative

+x: Convert to Numeric

String operator: unsigned operator, string connection

Assignment operators:

=, +=, -=, *=, /=, %=, ^=

++, --

Comparison operators:

>=, <, <=,! =, = =

Pattern-matching characters:

~: Whether the left and right matches contain

!~: does not match

Awk–f: ' $ ~/root/{print $ '/etc/passwd

awk '!~/root/'/etc/passwd

Logical operators: with &&, or | |, non!

Example:

Awk–f: ' $3>=0 && $3<=1000 {print $ '/etc/passwd

Awk-f: ' $3==0 | | $3>=1000 {print '} '/etc/passwd

Awk-f: '! ($3==0) {print "} '/etc/passwd

Awk-f: '! ($3>=500) {print $}} ' /etc/passwd

Viii. function Call: Function_name (ARGU1, ARGU2, ...)

Conditional expressions (three-mesh expressions):

Selector?if-true-expression:if-false-expression

Example:

Awk-f: ' {$3>=1000?usertype= ' Common User ': usertype= ' Sysadmin

or sysuser ";p rintf"%15s:%-s\n ", $1,usertype} '/etc/passwd

Ix. PATTERN of awk

Pattern: Filter the matching rows according to the pattern condition, then do the processing

(1) If not specified: Empty mode, match each row

(2)/regular expression/: Only the rows that can be matched to a pattern are processed, which needs to be used//enclosed.

awk '/^uuid/{print $ '/etc/fstab

awk '!/^uuid/{print $ '/etc/fstab

(3) Relational expression: The relationship expressions, the result is "true" has "false", the result is "

True "will only be handled.

True: The result is a non-0 value, non-empty string

False: The result is an empty string or a value of 0

Example:

awk '!0 '/etc/passwd; awk '! 1 '/etc/passwd

Awk–f: ' $3>=1000{print $1,$3} '/etc/passwd

Awk-f: ' $3<1000{print $1,$3} '/etc/passwd

Awk-f: ' $NF = = '/bin/bash ' {print $, $NF} '/etc/passwd

Awk-f: ' $NF ==/bash$/{print $, $NF} '/etc/passwd

(4) Line ranges: Range

STARTLINE,ENDLINE:/PAT1/,/PAT2/does not support the direct giving of numeric formats

Awk-f: '/^root/,/^nobody/{print $ '

/etc/passwd

Awk-f: ' (nr>=10&&nr<=20) {print nr,$1} '

/etc/passwd

(5) Begin/end mode

begin{}: Executes only once before beginning the processing of text in a file

end{}: Only once after text processing is complete

Example

Awk-f: ' BEGIN {print ' USER USERID '} {print $ ': ' $ $} end{print ' END file '} '/etc/passwd

Awk-f: ' {print ' USER USERID ';p rint $ ': ' $ $}end{print ' END file '} '/etc/passwd

Awk-f: ' begin{print ' USER UID \ n---------------'}{print $1,$3} '/etc/passwd

Awk-f: ' begin{print ' USER UID \ n---------------'}{print $1,$3} ' end{print ' ============== '}/etc/passwd

Seq Ten |awk ' i=0 '

Seq Ten |awk ' I=1 '

Seq 10 | awk ' I=!i '

Seq 10 | awk ' {i=!i;print i} '

Seq 10 | awk '! (i=!i) '

SEQ Ten |awk-v i=1 ' I=!i '

Ten, awk action

Common action Categories

(1) Expressions: arithmetic, comparison expression, etc.

(2) Control statements:if, while etc.

(3) Compound statements: Combined statement

(4) Input statements

(5) Output Statements:print, etc.

This article is from the "Love Firewall" blog, be sure to keep this source http://183530300.blog.51cto.com/894387/1854708

Linux Text Processing Three musketeers--awk