awk in the shell script

A Basic introduction

1.awk:

Awk is a powerful text analysis tool that is irreplaceable in the processing of text files and generating reports. Awk considers text files to be structured, which defines each input row as a single record, each string in the row is defined as a field (segment), and the domain and domain are split using a delimiter.

2. Functions: Flow control, mathematical operations, Process Control, built-in variables and functions, loops and judgments

3. Working principle:

Awk makes each row a split and handles the split "segment" with the corresponding command.

(1) Line work mode, read into each line of the file, will put a line of content, save to $

(2) Use the built-in variable FS (segment delimiter, the default is white space characters), split the line, the split out of each segment to the corresponding variable $ (1-100)

(3) Output time according to the built-in variable OFS (out FS), output

(4) Read into the next line to continue operation

Simple example

[Email protected] ~]# echo "This was a book" > Awk.txt

[[email protected] ~]# awk ' {print $2,$1,$3,$4} ' Awk.txt

Is this a book

4. Awk commonly used built-in variable tables:

1 $ $ current record (as a single variable)

2 $1~ $n the nth field of the current record, separated by FS between fields

3 FS input field delimiter default is space

4 NF The number of fields in the current record, that is, how many columns

5 NR The number of records that have been read, is the line number, starting from 1

6 RS input Record he thinks of defaults. NewLine character

7 OFS output field delimiter default is also a space

8 ORS output record delimiter, default to line break

9 ARGC number of command line arguments

ARGV command line parameter array

FileName The name of the current input file

IGNORECASE If true, case-insensitive matches are ignored

Argind the argv identifier of the file currently being processed

CONVFMT Digital Conversion Format%.6g

ENVIRON UNIX Environment variables

ERRNO UNIX System error messages

FieldWidths blank delimited string for input field width

Current record number of FNR

The output format of the OFMT number%.6g

Rstart the first string matched by the matching function

Rlength the string length matched by the matching function

Two Simple use of print

Example: printing an entire line: $

[Email protected] ~]# cp/etc/passwd p1

[[email protected] ~]# awk ' {print $} ' P1

Example: Print the last field of each line: $NF

[Email protected] ~]# awk-f: ' {print $NF} ' P1

Example: Printing a third field: $ $

[Email protected] ~]# awk-f: ' {print $} ' P1

Example: Printing the first line nr==1

[[email protected] ~]# awk ' nr==1{print} ' P1

Root:x:0:0:root:/root:/bin/bash

Example: printing the last line

[[email protected] ~]# awk ' end{print} ' P1

Tx:x:500:500:tx:/home/tx:/bin/bash

Example: Printing the last field in the first line

[Email protected] ~]# awk-f: ' Nr==1{print $NF} ' P1

/bin/bash

Example: Print Last field of last line

[[Email protected] ~] #awk-F: ' End{print $NF} ' P1

Example: Print the second-to-last field of each line and print it later hello

[[email protected] ~]# awk-f: ' {print $ (NF-1), ' Nihao '} ' P1

/root Nihao

/bin Nihao

/sbin Nihao

Example: Print line number

[[email protected] ~]# awk ' {print nr,$0} ' P1

1 Root:x:0:0:root:/root:/bin/bash

2 Bin:x:1:1:bin:/bin:/sbin/nologin

3 Daemon:x:2:2:daemon:/sbin:/sbin/nologin

Example: printing a specific value for the current system environment variable

[[email protected] ~]# awk ' begin{print environ["PATH"];} '

/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin

Example: Deleting a 2nd field with: Split

[[email protected] ~]# awk ' begin{fs= ":"; ofs= ":"}{print $1,$3,$4,$5,$6,$7} ' P1

Root:0:0:root:/root:/bin/bash

Bin:1:1:bin:/bin:/sbin/nologin

Daemon:2:2:daemon:/sbin:/sbin/nologin

Three Use of printf

Print Format Generation report

%d decimal signed integer

%u decimal unsigned integer

%f floating Point

%s string

%c Displays the ASCII code of the character

%p the value of the pointer

%E Scientific and technological law shows numerical value

%x%x unsigned integer with hexadecimal representation

%o unsigned integer represented in octal

%g%g display values in scientific notation or floating-point format

Percent of the display itself

Modifier:

-: Align Left

+: Display numeric symbols

N: Display

-f Specifies the delimiter for the segment

Example: (1) Generating a report

Example: (2) Decimal problem

When you take a reserved bit of decimals, rounding

Rounding a decimal, not rounded

[email protected] ~]# cat Awk.1

23.3456 11.234 45.67

[[email protected] ~]# awk ' {printf '%.2f\t%.2f\t%.2f\n ', $1,$2,$3} ' Awk.1

23.3511.2345.67

Four Use of awk

(1) Regular expressions

\ (\) \{\} does not support

. * ^ $ ? + [] | \< \> () can be used directly

example [[email protected] ~]# awk '/^$/{print ' This is a empty line '} '/etc/inittab

This is a empty line

This is a empty line

This is a empty line

This is a empty line

This is a empty line

This is a empty line

This is a empty line

This is a empty line

This is a empty line

example [[email protected] ~]# awk-f: '/^root/{print $, $NF} '/etc/passwd

Root/bin/bash

example [[email protected] ~]# awk-f: '!/^root/{print $, $NF} '/etc/passwd|head-3

Bin/sbin/nologin

Daemon/sbin/nologin

Adm/sbin/nologin

(2) Relational operators

> < = = = >= <=

~ (Match)!~ (mismatch)

example [[email protected] ~]# cp/etc/passwd p1

[Email protected] ~]# awk-f: ' $ = = 0 {print '} ' P1

Root

example [[email protected] ~]# awk-f: ' $! = 0{print '} ' p1 | Head-2

Bin

Daemon

example [[email protected] ~]# awk-f: ' $ < 2 {print $} ' P1

Root

Bin

(3) Logical operators

&& | | !

and or non-

example [[email protected] ~]# awk-f: ' $ > 0 && $ $ < {print $, $ ' P1 |head-2

Bin 1

Daemon 2

example [[email protected] ~]# awk-f: ' $ > 10 | | $ < 5 {print $1,$3} ' p1 |head-6

Root 0

Bin 1

Daemon 2

ADM 3

LP 4

Operator 11

(4) Arithmetic operators

+-*/% (modulo (remainder)) ^ (Power operation)

Example: Output name, total score, average result

[email protected] ~]# Cat CJ

TX 90 86 86

TX1 89 78 85

TX2 79 80 85

[[email protected] ~]# awk ' {print $1,$2+$3+$4, ($2+$3+$4)/3} ' CJ

TX 262 87.3333

TX1 252 84

TX2 244 81.3333

[[email protected] ~]# awk ' {printf "%-5s%3d%.2f\n", $1,$2+$3+$4, ($2+$3+$4)/3} ' CJ

TX 262 87.33

TX1 252 84.00

TX2 244 81.33

(5) BEGIN END

begin{Action; The action to be performed before the file is processed;

end{Action; The action to be performed after the file has been processed;

BEGIN: You can add a title to a file, define a variable, define a delimiter for a file

END: Summary of Actions

Getline can read input from pipes and standard inputs, and then pass them to variables.

Cases:

[[email protected] ~]# awk ' begin{' date ' | getline a}{print}end{print A} ' CJ

TX 90 86 86

TX1 89 78 85

TX2 79 80 85

Thu Feb 7 12:39:25 CST 2013

Five Flow control and looping in awk

(1) Simple condition judgment

Syntax: (expression? value 1: Value 2) If the expression is true, the output value is 1; otherwise the output value is 2

[email protected] ~]# Cat num

2 8 9

8 4 6

3 5 7

[[email protected] ~]# awk ' {print ($ > $)} ' num

8

8

5

(2) If judgment

Grammar:

{if (An expression

{

Action 1; Action 2;

}

}

If the expression is true, then the action is executed.

[[email protected] ~]# awk ' {if ($2>=80 && $ <=100) {print $, "great"} else {print $, "Good"}} ' CJ

TX Great

TX1 Great

TX2 Good

(2) Multi-support judgment

{

if (an expression)

{Action 1; Action 2; ...}

else if (expression)

{Action 1; Action 2; ...}

else if (expression)

{Action 1; Action 2; ...}

......

Else

{Action 1; Action 2; ...}

}

[email protected] ~]# Cat CJ

TX 90 86 86

TX1 89 78 85

TX2 79 80 85

TX3 80 70 60

TX4 75 85 65

TX5 78 62 80

Criteria for judging:

90-100 A

80-89 B

70-79 C

60-69 D

0-59 E

[[email protected] ~]# awk ' {if ($ >= && $ <=) {print $, ' A '} else if ($ >= && Lt {print $, ' B '} else if ($ >= && $ < $) {print $, ' C '} else if ($ >=, $ && $ < 70) {print $, ' D '} else {print $, ' E '}} ' CJ

TX A

TX1 B

TX2 C

TX3 B

TX4 C

TX5 C

(3) Loop while

Syntax: ' var= initial value; while (expression) {action 1; .... Update the action of the variable;} '

Cases:

[Email protected] ~]# awk-f: ' {i=1; while (I<=NF) {print $i; i++} ' p1 | Head-7

Root

X

0

0

Root

/root

/bin/bash

Example. Method One

[Email protected] ~]# awk-f: ' {i=nf; while (i>=2) {printf $i ":"; i--};p rint} ' P1

/bin/bash:/root:root:0:0:x:root

/sbin/nologin:/bin:bin:1:1:x:bin

/sbin/nologin:/sbin:daemon:2:2:x:daemon

/sbin/nologin:/var/adm:adm:4:3:x:adm

Example. Method Two

[[email protected] ~]# awk ' BEGIN {fs= ': '} {i=nf; while (i>=2) {printf $i ":"; i--} print \ n} ' P1

/bin/bash:/root:root:0:0:x:root

/sbin/nologin:/bin:bin:1:1:x:bin

/sbin/nologin:/sbin:daemon:2:2:x:daemon

(4) For loop

Grammar:

{

for (an expression)

{Action 1; ...}

}

Expression: Divided into 3 parts:

(1) Initialize expression I=1

(2) test expression i<10

(3) Update test expression i++

Statement:

Next processing the next input line of the input line

Exit exits

Continue end of this cycle

Break jumps out of the loop

Cases

[[email protected] ~]# awk ' BEGIN {fs= ': '} {for (i=nf;i>=2;i--) {printf $i ";"}; print ' P1

/bin/bash;/root;root;0;0;x;root

/sbin/nologin;/bin;bin;1;1;x;bin

/sbin/nologin;/sbin;daemon;2;2;x;daemon

/sbin/nologin;/var/adm;adm;4;3;x;adm

Cases

[email protected] ~]# Cat num

2 8 9

8 4 6

3 5 7

[[email protected] ~]# awk ' {max=0; i=1; while (I<=NF) {if (max< $i) {max= $i} i++} print max} ' num

9

8

7

(5) Awk array

Cases

Example using a variable as an array subscript

Another way of reading (this is unordered, j is a variable, a is an array)

Array order

(6) function

@1split Cutting strings

Split ("String waiting to be cut", array name, "Cut delimiter")

[[email protected] ~]# awk ' begin{split ("2012/08/23", DA, "/");p rint da[2],da[3],da[1]} '

08 23 2012

@2toupper () lowercase to uppercase

ToLower () uppercase to lowercase

[[email protected] ~]# awk ' {print ToUpper ($)} ' P1 |head-3

Root:x:0:0:root:/root:/bin/bash

Bin:x:1:1:bin:/bin:/sbin/nologin

Daemon:x:2:2:daemon:/sbin:/sbin/nologin

@3sub () Partial substitution

Gsub () Global substitution

Sub (/What to replace/, "What to replace")

Gsub (/the content to replace/, "What to replace")

Gsub (/What to replace/, "What to replace", specify fields such as $7)

Cases:

[[email protected] ~]# awk-f: ' {sub (/root/, ' r00t ');p rint} ' P1

R00t:x:0:0:root:/root:/bin/bash

Cases:

[Email protected] ~]# awk-f: ' {gsub (/root/, "r00t");p rint} ' P1

R00t:x:0:0:r00t:/r00t:/bin/bash

Operator:x:11:0:operator:/r00t:/sbin/nologin

Cases:

[[email protected] ~]# awk-f[:/] ' {gsub (/root/, "r00t", $7);p rint} ' P1

Root x 0 0 root r00t bin Bash

operator x 0 operator r00t Sbin nologin

@4.length () calculates the length of a string

[Email protected] ~]# awk-f: ' {print length ($), $ ' P1

4 root

3 bin

6 Daemon

3 ADM

@5. Mathematical calculations

[[email protected] ~]# awk ' begin{print sin (30)} '

-0.988032

[[email protected] ~]# awk ' begin{print cos (60)} '

-0.952413

[[email protected] ~]# awk ' begin{print int (22/6)} '

3

[[email protected] ~]# awk ' begin{print sqrt (3)} '

1.73205

awk in the shell script