Explanation of Shell script awk

Awk explanation of Shell script

1. Basic Introduction

1. awk:

Awk is a powerful text analysis tool. awk is irreplaceable in processing text files and generating reports. Awk considers the text file to be structured. It defines each input row as a record, and each string in the row is defined as a field (segment). The fields and fields are separated by delimiters.

2. Functions: Stream Control, mathematical operations, process control, built-in variables and functions, loops and judgments

3. Working principle:

Awk splits each row and uses corresponding commands to process the split segments.

(1) In row mode, each row of the file is read, and the content of a row is saved to $0.

(2) Use the built-in variable FS (segment separator, which defaults to blank characters) to split this line, save each segment separated to the corresponding variable $ (1-100)

(3) output data according to the built-in variable OFS (out FS ).

(4) read the next row and continue the operation

Simple instance

[Root @ tx3 ~] # Echo "this is a book"> awk.txt

[Root @ tx3 ~] # Awk '{print $2, $1, $3, $4}' awk.txt

Is this a book

4. Awk common built-in variable tables:

1 $0 current record (as a single variable)

2 $1 ~ $ N the nth field of the current record. The fields are separated by FS.

3. The default delimiter of the FS input field is space.

4. The number of fields in the current NF record is the number of columns.

5 The number of records that NR has read, that is, the row number, starting from 1

6. The record input by RS is regarded as a linefeed.

7. The delimiter of the output field of OFS is also a space by default.

8. The record delimiter output by ORS. The default value is a line break.

9. Number of ARGC command line parameters

10 ARGV command line parameter Array

11 FILENAME name of the current input file

12 if IGNORECASE is true, case-insensitive matching is performed.

13 ARGIND the ARGV flag of the currently processed file

14 CONVFMT digital conversion format %. 6g

15 environ unix environment variables

16 errno unix system error message

17 blank separator string of FIELDWIDTHS input field width

18 current FNR records

19 OFMT number output format %. 6g

20 RSTART is the first string matched by the matching function

21 RLENGTH: the length of the string matched by the matching function

Ii. simple use of print

Example: print the entire row: $0

[Root @ tx3 ~] # Cp/etc/passwd p1

[Root @ tx3 ~] # Awk '{print $0} 'p1

For example, print the last field of each row: $ NF

[Root @ tx3 ~] # Awk-F: '{print $ NF}' p1

For example, print the Third Field: $3.

[Root @ tx3 ~] # Awk-F: '{print $3}' p1

For example, print the first line NR = 1

[Root @ tx3 ~] # Awk 'nr = 1 {print $0} 'p1

Root: x: 0: 0: root:/bin/bash

Example: print the last line

[Root @ tx3 ~] # Awk 'end {print $0} 'p1

Tx: x: 500: 500: tx:/home/tx:/bin/bash

For example, print the last field of the first line.

[Root @ tx3 ~] # Awk-F: 'nr = 1 {print $ NF} 'p1

/Bin/bash

For example, print the last field of the last row.

[Root @ tx3 ~] # Awk-F: 'End {print $ NF} 'p1

For example, print the second-to-last field of each line, and print hello after it.

[Root @ tx3 ~] # Awk-F: '{print $ (NF-1), "nihao"}' p1

/Root nihao

/Bin nihao

/Sbin nihao

For example, print the row number.

[Root @ tx3 ~] # Awk '{print NR, $0} 'p1

1 root: x: 0: 0: root:/bin/bash

2 bin: x: 1: 1: bin:/sbin/nologin

3 daemon: x: 2: 2: daemon:/sbin/nologin

For example, print a specific value of the current system environment variable.

[Root @ tx3 ~] # Awk 'in in {print ENVIRON ["PATH"];}'

/Usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin: /usr/bin:/root/bin

For example, use: Split to delete 2nd fields.

[Root @ tx3 ~] # Awk 'in in {FS = ":"; OFS = ":"} {print $1, $3, $4, $5, $6, $7} 'p1

Root: 0: 0: root:/bin/bash

Bin: 1: 1: bin:/sbin/nologin

Daemon: 2: 2: daemon:/sbin/nologin

Iii. Use of printf

Print format Generate Report

% D decimal signed integer

% U decimal unsigned integer

% F floating point number

% S string

% C: ASCII code of the characters displayed

% P pointer Value

% E Scientific and Technical Method: Numerical Value

% X % X unsigned integer in hexadecimal format

% O unsigned integer in octal format

% G % G displays the value in scientific notation or floating point Number Format

% Display itself

Modifier:

-: Left alignment

+: Displays numeric symbols.

N: Display

-F: delimiter of the specified segment

Example: (1) generate a report

Example: (2) Decimals

Rounded

Round decimal places without rounding

[Root @ tx3 ~] # Cat awk.1

23.3456 11.234 45.67

[Root @ tx3 ~] # Awk '{printf "%. 2f \ t %. 2f \ t %. 2f \ n", $1, $2, $3} 'awk.1

23.3511.2345.67

Iv. Use of awk

(1) Regular Expression

\ (\) \ {\} Is not supported

. * ^ $? + [] | \ <\> () Can be used directly

Example [root @ tx3 ~] # Awk '/^ $/{print "this is an empty line"}'/etc/inittab

This is an empty line

This is an empty line

This is an empty line

This is an empty line

This is an empty line

This is an empty line

This is an empty line

This is an empty line

This is an empty line

Example [root @ tx3 ~] # Awk-F: '/^ root/{print $1, $ NF}'/etc/passwd

Root/bin/bash

Example [root @ tx3 ~] # Awk-F :'! /^ Root/{print $1, $ NF} '/etc/passwd | head-3

Bin/sbin/nologin

Daemon/sbin/nologin

Adm/sbin/nologin

(2) Relational operators

> <=! ==<=

~ (Matching )!~ (Mismatch)

Example [root @ tx3 ~] # Cp/etc/passwd p1

[Root @ tx3 ~] # Awk-F: '$3 = 0 {print $1}' p1

Root

Example [root @ tx3 ~] # Awk-F: '$3! = 0 {print $1} 'p1 | head-2

Bin

Daemon

Example [root @ tx3 ~] # Awk-F: '$3 <2 {print $1}' p1

Root

Bin

(3) logical operators

& |!

And

Example [root @ tx3 ~] # Awk-F: '$3> 0 & $3 <10 {print $1, $3} 'p1 | head-2

Bin 1

Daemon 2

Example [root @ tx3 ~] # Awk-F: '$3> 10 | $3 <5 {print $1, $3} 'p1 | head-6

Root 0

Bin 1

Daemon 2

Adm 3

Lp 4

Operator 11

(4) Arithmetic Operators

+-*/% (Modulus (remainder) ^ (Power Operation)

Example: Output name, total score, average score

[Root @ tx3 ~] # Cat cj

Tx 90 86 86

Tx1 89 78 85

Tx2 79 80 85

[Root @ tx3 ~] # Awk '{print $1, $2 + $3 + $4, ($2 + $3 + $4)/3}' cj

Tx 262 87.3333

Tx1 252 84

Tx2 244 81.3333

[Root @ tx3 ~] # Awk '{printf "%-5 s % 3d %. 2f \ n ", $1, $2 + $3 + $4, ($2 + $3 + $4)/3} 'cj

Tx 262 87.33

Tx1 252 84.00

Tx2 244 81.33

(5) BEGIN END

BEGIN {action;...} the action to be executed before processing the file; only once

END {action;...} the action to be executed after the file is processed; only once

BEGIN: You can add titles, variables, and delimiters to a file.

END: Summary operation

Getline can read input from MPs queue and standard input and pass it to the variable.

Example:

[Root @ tx3 ~] # Awk 'in in {"date" | getline a} {print} END {print a} 'cj

Tx 90 86 86

Tx1 89 78 85

Tx2 79 80 85

Thu Feb 7 12:39:25 CST 2013

V. Stream Control and loop in awk

(1) simple condition judgment

Syntax: (expression? Value 1: Value 2) if the expression is true, the output value is 1; otherwise, the output value is 2.

[Root @ tx3 ~] # Cat num

2 8 9

8 4 6

3 5 7

[Root @ tx3 ~] # Awk '{print ($1> $2? $1: $2)} 'num

8

8

5

(2) if judgment

Syntax:

{If (expression

{

Action 1; Action 2 ;...

}

}

If the expression is true, the action is executed.

[Root @ tx3 ~] # Awk '{if ($2> = 80 & $2 <= 100) {print $1, "great"} else {print $1, "good"} 'cj

Tx great

Tx1 great

Tx2 good

(2) multi-branch judgment

{

If (expression)

{Action 1; Action 2 ;...}

Else if (expression)

{Action 1; Action 2 ;...}

Else if (expression)

{Action 1; Action 2 ;...}

......

Else

{Action 1; Action 2 ;...}

}

[Root @ tx3 ~] # Cat cj

Tx 90 86 86

Tx1 89 78 85

Tx2 79 80 85

Tx3 80 70 60

Tx4 75 85 65

Tx5 78 62 80

Criteria:

90-100

80-89 B

70-79 C

60-69 D

0-59 E

[Root @ tx3 ~] # Awk '{if ($2 >=90 & $2 <= 100) {print $1, "A"} else if ($2 >=80 & $2 <90) {print $1, "B"} else if ($2 >=70 & $2 <80) {print $1, "C"} else if ($2 >=60 & $2 <70) {print $1, "D"} else {print $1, "E"} 'cj

Tx

Tx1 B

Tx2 C

Tx3 B

Tx4 C

Tx5 C

(3) loop while

Syntax: 'var = initial value; while (expression) {Action 1;... update variable action ;}'

Example:

[Root @ tx3 ~] # Awk-F: '{I = 1; while (I <= NF) {print $ I; I ++}' p1 | head-7

Root

X

0

0

Root

/Root

/Bin/bash

Example. method 1

[Root @ tx3 ~] # Awk-F: '{I = NF; while (I> = 2) {printf $ I ":"; I --}; print $1} 'p1

/Bin/bash:/root: 0: 0: x: root

/Sbin/nologin:/bin: 1: 1: x: bin

/Sbin/nologin:/sbin: daemon: 2: 2: x: daemon

/Sbin/nologin:/var/adm: 4: 3: x: adm

Example. method 2

[Root @ tx3 ~] # Awk 'in in {FS = ":"} {I = NF; while (I >=2) {printf $ I ":"; I --} print $1} 'p1

/Bin/bash:/root: 0: 0: x: root

/Sbin/nologin:/bin: 1: 1: x: bin

/Sbin/nologin:/sbin: daemon: 2: 2: x: daemon

(4) for Loop

Syntax:

{

For (expression)

{Action 1 ;...}

}

Expression: divided into three parts:

(1) initialization expression I = 1

(2) test expression I <10

(3) Update the test expression I ++

Statement:

Next processes the next input row of the input row

Exit

Continue ends this cycle

Break bounce cycle

Example

[Root @ tx3 ~] # Awk 'in in {FS = ":"} {for (I = NF; I >=2; I --) {printf $ I ";"}; print $1} 'p1

/Bin/bash;/root; 0; 0; x; root

/Sbin/nologin;/bin; 1; 1; x; bin

/Sbin/nologin;/sbin; daemon; 2; 2; x; daemon

/Sbin/nologin;/var/adm; 4; 3; x; adm

Example

[Root @ tx3 ~] # Cat num

2 8 9

8 4 6

3 5 7

[Root @ tx3 ~] # Awk '{max = 0; I = 1; while (I <= NF) {if (max <$ I) {max = $ I} I ++} print max} 'num

9

8

7

(5) awk Array

Example

For example, the variable is used as the array subscript.

Another reading method (this is unordered, where j is a variable and a is an array)

Array order

(6) Functions

@ 1 split cut string

Split ("string waiting for splitting", array name, "Separator Used for splitting ")

[Root @ tx3 ~] # Awk 'in in {split ("2012/08/23", da, "/"); print da [2], da [3], da [1]}'

08 23 2012

@ 2 toupper () converts lowercase letters to uppercase letters

Tolower () in uppercase to lowercase

[Root @ tx3 ~] # Awk '{print toupper ($0)} 'p1 | head-3

ROOT: X: 0: 0: ROOT:/BIN/BASH

BIN: X: 1: 1: BIN:/SBIN/NOLOGIN

DAEMON: X: 2: 2: DAEMON:/SBIN/NOLOGIN

@ 3sub () Local replacement

Gsub () Global replacement

Sub (/content to be replaced/, "what to replace ")

Gsub (/content to be replaced/, "what to replace ")

Gsub (/content to be replaced/, "what to replace with", specifying a field such as $7)

Example:

[Root @ tx3 ~] # Awk-F: '{sub (/root/, "r00t"); print}' p1

R00t: x: 0: 0: root:/bin/bash

Example:

[Root @ tx3 ~] # Awk-F: '{gsub (/root/, "r00t"); print}' p1

R00t: x: 0: 0: r00t:/r00t:/bin/bash

Operator: x: 11: 0: operator:/r00t:/sbin/nologin

Example:

[Root @ tx3 ~] # Awk-F [:/] '{gsub (/root/, "r00t", $7); print} 'p1

Root x 0 0 root r00t bin bash

Operator x 11 0 operator r00t sbin nologin

@ 4. length () calculates the length of a string.

[Root @ tx3 ~] # Awk-F: '{print length ($1), $1}' p1

4 root

3 bin

6 daemon

3 adm

@ 5. mathematical computation

[Root @ tx3 ~] # Awk 'in in {print sin (30 )}'

-0.988032

[Root @ tx3 ~] # Awk 'in in {print cos (60 )}'

-0.952413

[Root @ tx3 ~] # Awk 'in in {print int (22/6 )}'

3

[Root @ tx3 ~] # Awk 'in in {print sqrt (3 )}'

1.73205