AWK programming language

awk Standalone programming language (variable built-in variable flow control function array)

Use awk to get data to program processing when writing scripts

When awk is used independently, it is used primarily to count data

Command format

awk [options] ' Processing actions ' file list (spaces between multiple files)

Command | awk [options] ' handling action '

How awk handles data:

Processing units by behavior;

Handles the current row after processing the output of the current row and automatically processes the row

Until all the actions in the file have been processed

By default, the data is read row by line, and the data is processed line by row. To allow awk to process only the specified rows. Add condition

The processing action is performed only if the current row matches the condition, and the reverse is not handled

awk [option] ' Condition {processing action} ' file list

How the condition is expressed:

1/Regular Expression/

2 Numerical comparison

3 character Comparison

4 Logical Comparisons

Options

-f Specifies the split symbol when splitting the field. The default split symbol is a space and tab

awk built-in variables

Save the content of Awk's current read release

FileName to save the file name of awk currently working on files

NR Save the number of rows that awk is currently working on

FNR Save the number of rows in the file that awk is currently working on

NF Store the number of the current row, column (field) after splitting with the specified split symbol

$ ... $n represents the field after the data is split with the specified separator

Variable 1th column $2nd Column

FS save field split symbol default Space and TAB key (fs=

environ["System Environment" (array type) to invoke the value of the system environment variable

Echo $USER $HOSTNAME

awk ' {print environ[' HOSTNAME "]},environ[" USER "]} ' a.txt

environ["System environment variables"] Let awk invoke system environment variables

-V Let awk use shell custom variables

#name =jim

#awk-V mz= $name ' {print mz} ' a.txt

Put awk current system existing user name output on screen/etc/passwd

Awk-f ":" ' {print nr,$1,$6} '/etc/passwd | Head-3

User name home Directory

Root/root

Bin/bin

Daemon /sbin

3 Rows of total processing

awk ' begin{fs= ': ";p rint" line number \ t user name \ t home directory "}{print nr" \ t "," \ T ", $6}end{print" total processing "NR" line "} '/etc/passwd

awk Processing Data Order

begin{} Pre-processing: actions performed before reading

Initialization operations are written in begin{}.

{} row handling: Actions performed when a row is read into

To handle the lines of reading and dropping.

end{} line post-processing: Actions performed after the line has been read

Do a summary output operation written in end{}

Each process can be used alone or reused, or it can be used together

Head-8/etc/passwd | awk ' Begin{print nr}{print nr}end{print NR} '

How the condition is expressed:

1,/Regular expression/

awk '/Regular Expression/{processing action} ' file name

awk '/root/{print nr,$0} '/etc/passwd

awk '/[0-9]/{print $} '/etc/passwd

~ No match

！ ~ No match

Awk-f ":" ' $1~/a/{print $1,$3} '/etc/passwd

Awk-f ":" ' $1!~/a/{print $1,$3} '/etc/passwd

2. Value comparison = = = > >= < <=

Awk-f ":" ' $3==0{print $1,$3} '/etc/passwd

Awk-f ":" ' $3<10{print $1,$3} '/etc/passwd

3. Character comparison = = =

Awk-f ":" ' $1!= "Bin" {print $} '/etc/passwd

Awk-f ":" ' $1== ' root ' {print $} '/etc/passwd

4. Logical comparison (use when there are multiple judging conditions)

&& Logic and

|| Logical OR

！ Logical Non-

Awk-f ":" ' $3>=5 && $3<=10{print $1,$3} '/etc/passwd

Awk-f ":" ' $3==5 | | $3==7{print $1,$3} '/etc/passwd

awk operation symbol (same as operator in Shell)

+-*/%

+= -= *= /=

++ --

Definition and invocation of variables in awk

Variable name = value

Call Variable Name

* You can also not define a direct call

Awk-f ":" ' Nr<=3{i=$1;print i} '/etc/passwd

Count the number of built-in and out-of-build users

Awk-f ":" ' Begin{i=0;j=0}$3>=500{i++}$3<=499{j++}end{print

"Inside User" I ";p rint" outside user "J" "} '/etc/passwd

awk Process Control (Process Control execution process and shell process Control execution process, just as the syntax format changed)

One, branch structure

Single Branch

What to do if (conditional expression) 1

if (conditional expression) {action performed when the condition is established 1; Perform action 2; Execute action n}

Dual Branch

if (conditional expression) {

Action 1; Action 2; Action n

｝

else{

Action 1; Action 2; Action n

｝

Awk-f ":" ' {if ($3<=499) {i++}else{j++}}end{print i;print J} '/etc/passwd

Multi-Branch

if (conditional expression) {

Action 1; Action 2; Action n

}

else if (conditional expression) {

Action 1; Action 2; Action n

｝

......

esle{

Action 1; Action 2; Action n

｝

Awk-f ":" ' Begin{x=0;y=0;z=0}{if ($3<100) {X++}else if

($3>=100&&$3<=499) {Y++}else if ($3>=500) {z++}else{print "no user"}}

End{print x, y, z} '/etc/passwd

Invoking the awk script for file processing

Awk-f xxx.awk/etc/passwd

Vim Xxx.awk

#!/usr/bin/awk-f

begin{

Fs= ":"

X=0

Y=0

Z=0

}

{

if ($3<=499) {

X + +

}else{

y++

}

｝

end{

Print "The number of built-in users is" X "

Print "The number of external users is" Y "each"

}

#chmod +x xxx.awk/etc/passwd

./xxx.awk

Second, the cycle structure

For

for (initial value; conditional expression: Step) {

The loop body that is executed when the condition is established

}

awk "begin{for (i=1;i<=10;i++) {print i}}"

While the condition is first judged and then the loop body is executed

while (conditional expression) {

Implement loop body when condition is established

}

Awk-f ":" ' {while ($3<3) {print $1,$3;} '/etc/passwd

Awk-f ":" ' {while ($3<3) {print $1,$3;$3++} '/etc/passwd

Awk-f ":" ' {while ($3<3) {print $1,$3;$3=5}} '/etc/passwd

Do....while the condition after performing the cyclic body first

do{

Loop body

}while (conditional expression)

awk ' BEGIN ' {i=10:while (i>20) {print i;i++}} '

awk ' BEGIN (i=10;do{print i;i--}while (i>20)} '

AWK Process Control Statements

Execution control of the loop structure in awk (the loop body is executed as soon as the condition is established)

awk Array

Break ends the current loop body

Continue stop this cycle and start the next cycle

awk ' Begin{for (i=1;i<=10;i++) {if (i==4) Continue;print i}} '

awk ' Begin{for (i=1;i<=10;i++) {if (i==4) break;print i} '

Control of awk (read data row by line, process data line by row)

Exit

End awk Reading

If you have end{}, perform the action in end{}

If no end{} direct junction rate processing

awk ' Nr==4{exit}{print nr,$0} ' a.txt

awk ' Nr==4{exit}{print nr,$0}end{print NR} ' a.txt

Next

Let awk read the next line of the current row

awk ' Nr==4{next}{print nr,$0}end{print NR} ' a.txt

awk Array

Define array Format: array name [numeric element subscript]= element value

* The subscript of an array element can use numbers or characters

Use array element format: array name [array element subscript]

Output array element value: Print array name [array element subscript]

awk ' BEGIN ' {stuname[0]= "Jim" stuname[1]= "Tom";p rint stuname[0];p rint stuname[1]} '

awk ' begin{for{i=0;i<=10;i++}{num[i]=i;print num[i]} '

awk ' begin{for (i=0;i<=10;i++) {Num[i]=i;print num[2]}} '

awk ' begin{for (i=0;i<=10;i++) {num[i]=i};p rint num[2];p rint num[3]} '

Iterating through the array loop structure

for (variable name in array name) {print array name [variable name]}

for (x in num) {print x;print num[x]}

awk ' Begin{for (i=0;i<=10;i++) num[i]= "User" i};for (w in num) {print w,num[w]} '

Store the first 5 usernames of the system in the array usergrp, and then output all the elements of the array usergrp

Head-5/etc/passwd | Awk-f ":" ' {print $3,$1} '

Head-5/EETC/PASSWD | Awk-f ":" ' {print usergrp[$3]=$1}end{for (i in Usergrp}{print I,usergrp[i]}} '

Count the number of occurrences of each command in the Test.txt file

awk ' {comm[$1]++}end{for (x in comm) {print x,comm[x]}} ' Test1.txt | SORT-RNK 2

awk ' {comm[$1]++}end{for (x in comm) {print x,comm[x]}} ' ~/.bash_history | SORT-RNK 2 | Head-10

R Sort by descending order

N Sort by number

K Sort by 2nd Column

PS-E Show all processes within the system

-o Specifies that the data for that column is displayed

Comm Executable file name

cmd | Simple command

PCPU CPU Occupancy Rate

Pmem Memory Utilization

Time Cumulative CPU times

Time spent after the etime process was started

Example

Ps-eo Comm,pupu,pmem--sort-pmem | Head sorted by memory usage in descending order

Ps-eo Comm,pcpu,pmem--sort +pmem | Head in ascending order by memory utilization

PS-E-O comm,pcpu | Tail-n +2

Automatic connection of SSH server scripts

#!/usr/bin/expect

Set Login_host 192.168.1.1

Set Login_user root

Set Login_password 123456

Spawn ssh [email protected]_host

Expect "[email protected]_host ' s password:"

Send "$login _password\r"

Expect "#"

Send "Useradd jim9\r"

Expect "#"

Send "Exit"

This article is from the "Wsyht blog" blog, make sure to keep this source http://wsyht2015.blog.51cto.com/9014030/1786334

AWK programming language