Linux Shell script programming-detailed explanation of awk commands

Easy to use:

Awk: perform operations on a row in a file by itself.

Awk-F: '{print $1, $4}': Use ':' to split the row and print the first and fourth fields of the row.

 

 

Details:

Awk command Introduction

 

The most basic function of the awk language is to browse and extract information based on specified rules in a file or string. Only after awk extracts information can other text operations be performed, A complete awk script is usually used to format information in a text file.

 

1. Call awk:

 

The first command line method, such:

 

Awk [-field-separator] 'commands' input-file (s)

 

Commands is a real awk command, and the [-F domain separator] is optional. awk is separated by spaces by default. Therefore, you do not need to specify this option to browse text with spaces between domains, however, if you browse a passwd file, the fields of this file use colons as the separator, you must use the-F option: awk-F: 'commands' input-File

 

Second, insert all the awk commands into a file, make the awk program executable, and then use the awk command interpreter as the first line of the script to call it by typing the Script Name

 

Third, insert all awk commands into a separate file and then call them, for example:

 

Awk-F awk-script-file input-File

 

-F indicates the awk script in the file awk-script-file. The input-file is the file name browsed by awk.

 

2. awk script:

 

The awk script consists of various operations and modes. Based on the delimiter (-F option), the default value is space. The read content is placed in the corresponding domain in sequence, and one row of records is read until the end of the file.

 

2.1. modes and actions

 

All awk statements are composed of modes and actions. There may be many statements in an awk script. The Mode part determines when the Action Statement is triggered and the event is triggered. An action is an operation performed on data. If the mode is omitted, the action is always executed.

 

The mode can be any conditional statement, compound statement, or regular expression. The mode contains two special fields begin and end. The begin statement is used to set the count and print headers. The begin statement is used before any Text Browsing action, then, the Text Browsing action starts based on the input file. The end statement is used to print the total number of output texts and the ending status mark after the awk completes the Text Browsing action. actions must be included in {}.

 

The actual action is specified in braces {} and is often used for printing, but there are still longer Code such as if and loop looping statements and loop exit. If no action is specified, awk prints all browsing records by default.

 

2.2. domain and record:

 

During awk execution, Its browsing is marked as $1, $2... $ n. This method is called domain tag. Use $1 and $3 to refer to the 1st and 3rd fields. Note that the fields are separated by commas (,) and $0 is used to represent all fields. For example:

 

Awk '{print $0}' temp.txt> sav.txt

 

Sort prints all the fields and redirects the result to sav.txt.

 

Awk '{print $0}' temp.txt | tee sav.txt

 

Similar to the previous example, the difference is that it will be displayed on the screen.

 

Awk '{print $1, $4}' temp.txt

 

Only 1st and 4th fields are printed.

 

Awk 'in in {print "name grade \ n ----"} {print $1 "\ t" $4} 'temp.txt

 

Indicates that the information header is entered, that is, "Name grade \ n -------------" is added before the first line of the input content, and the content is separated by tab.

 

Awk 'in in {print "being"} {print $1} end {print "end"} 'temp

 

Print both the information header and end

 

2.3. Conditional operators:

 

<, <=, = ,! =,> = ,~ Match regular expressions ,!~ Do not match Regular Expression

 

Match: awk '{if ($4 ~ /Asima/) Print $0} 'temp indicates that if the fourth domain contains Asima, the entire

 

Exact match: awk '$3 = "48" {print $0}' temp only prints records whose 3rd domain is equal to "48"

 

Mismatch: awk '$0 !~ /Asima/'temp print the entire record that does not contain Asima

 

Not equal to: awk '$1! = "Asima" 'temp

 

Less than: awk '{if ($1 <$2) Print $1 "is smaller"}' temp

 

Set case sensitivity: awk '/[Gg] reen/'temp print the entire record containing green or green

 

Any character: awk '$1 ~ /^... A/'temp print the fourth character in the 1st domain is a record. '^' indicates the first line and '.' indicates any character.

 

Or link match: awk '$0 ~ /(ABC) | (EFG)/'temp used | the statement must be included.

 

And Relationship: awk '{if ($1 = "A" & $2 = "B") Print $0}' temp

 

Or link: awk '{if ($1 = "A" | $1 = "B") Print $0}' temp

 

2.4. awk built-in variables:

 

Argc	Number of command line parameters	NF	Number of browsing records
Agrv	Command line parameter arrangement	NR	Number of read records
Environ	Support the use of system environment variables in the queue	OFS	Output domain Separator
Filename	Awk browsed file name	ORS	Output record Separator
FNR	Number of browsing file records	RS	Control record Separator
FS	Set the input domain separator, which is the same as the-F option.	NF	Number of browsing records

 

Example: awk 'end {print Nr} 'temp prints the number of read records at the end

 

Awk '{print NF, NR, $0} end {print filename}' temp

 

Awk '{If (NR> 0 & $4 ~ /Brown/) Print $0} 'temp has at least one record and contains Brown

 

Another usage of NF: Echo $ PWD | awk-F/'{print $ NF}' displays the current directory name

 

2.5. awk OPERATOR:

 

Using operators in awk, basic expressions can be divided into numbers, strings, variables, fields, and array elements.

 

Set the input field to the variable name:

 

Awk '{name = $1; six = $3; if (six = "man") Print name "is" Six}' temp

 

Domain value comparison operation:

 

Awk 'in in {base = "27"} {if ($4 <base) Print $0} 'temp

 

Modify the value of a numeric field: (the original input file will not be changed)

 

Awk '{if ($1 = "Asima") $6 = $6-1; print $1, $6, $7}' temp

 

Modify text fields:

 

Awk '{if ($1 = "Asima) ($1 =" DESC "); print $1}' temp

 

Show only modification records: (only show what is needed. Differentiate the previous command. Note {})

 

Awk '{if ($1 = "Asima) {$1 =" DESC "; print $1}' temp

 

Create a new output domain:

 

Awk '{$4 = $3-$2; print $4}' temp

 

Statistical column value:

 

Awk '(TOT + = $3); end {print tot}' temp displays the content of each column

 

Awk '{(TOT + = $3)}; end {print tot}' temp only displays the final result

 

File length addition:

 

Ls-L | awk '/^ [^ d]/{print $9 "\ t" $5} {tot + = $5} end {print "totkb: "tot }'

 

Only list file names:

 

Ls-L | awk '{print $9}' in general, the file name is a 9th domain

 

2.6. awk built-in string functions:

 

Gsub (R, S) replaces R with S in $0

 

Awk 'gsub (/name/, "Xingming") {print $0} 'temp

 

Gsub (R, S, T) Replace r with S in the entire t

 

Index (S, T) returns the first position of string t in S.

 

Awk 'begin{ Print Index ("Sunny", "NY")} 'temp returns 4

 

Length (s) returns the length of S.

 

Match (S, R) test whether S contains a string matching r

 

Awk '$1 = "J. Lulu" {print match ($1, "U")}' temp returns 4

 

Split (s, A, FS) divides s into sequence a in FS

 

Awk 'in in {print split ("12 #345 #6789", myarray ,"#")"'

 

Returns 3, and myarray [1] = "12", myarray [2] = "345", myarray [3] = "6789"

 

Sprint (FMT, exp) returns the exp formatted by FMT

 

Sub (R, S) replaces R with S in the leftmost longest substring in $0 (only Replace the first matched string)

 

Substr (S, P) returns the suffix starting with P in string S.

 

Substr (S, P, n) returns the suffix of string s starting from P to n.

 

2.7. Use of the printf function:

 

Character conversion: Echo "65" | awk '{printf "% C \ n", $0}' output

 

Awk 'in in {printf "% F \ n", 999} 'output 999.000000

 

Formatted output: awk '{printf "%-15 S % s \ n", $1, $3}' temp displays the first domain in the left alignment.

 

2.8. Other awk usage:

 

Pass a value to an awk command line:

 

Awk '{if ($5 <age) Print $0}' age = 10 temp

 

Who | awk '{if ($1 = user) Print $1 "are in" $2' user = $ LOGNAME use environment variables

 

Awk script command:

 

Start! /Bin/awk-F. If this sentence does not exist, the self-contained script cannot be executed. For example:

 

! /Bin/awk-F

 

# All comment lines must start with a hash '#'

 

# Name: student_tot.awk

 

# To Call: student_tot.awk grade.txt

 

# Prints total and average of club student points

 

# Print a header first

 

Begin

 

{

 

Print "Student date Member No. grade age points Max"

 

Print "name joined gained point available"

 

Print "============================================ ============================"

 

}

 

# Let's add the scores of points gained

 

(TOT + = $6 );

 

# Finished processing now Let's print the total and average point

 

End

 

{

 

Print "club student total points:" tot

 

Print "average club student points:" TOT/n

 

}

 

2.9. awk array:

 

Basic loop structure of awk

 

For (element in array) print array [element]

 

Awk 'in in {record = "123 #456 #789"; split (record, myarray ,"#")}

 

End {for (I in myarray) {print myarray [I]}

 

 

3.0 awk custom statements

 

I. Condition judgment Statement (IF)

If (expression) # If (variable in array)
Statement 1
Else
Statement 2

In the format, "Statement 1" can be multiple statements. If you want to facilitate the Unix awk judgment and facilitate your own reading, you 'd better enclose multiple statements. The UNIX awk branch structure can be nested in the following format:

If (expression)

{Statement 1}

Else if (expression)
{Statement 2}
Else
{Statement 3}

[Xifj @ localhost nginx] # awk 'in in {
Tested = 100;
If (test> 90)
{
Print "very good ";
}
Else if (test> 60)
{
Print "good ";
}
Else
{
Print "No pass ";
}
}'

Very good

 

Each command statement can end.

 

Ii. Loop statements (while, for, do)

1. While statement

Format:

While (expression)

{Statement}

Example:

[Xifj @ localhost nginx] # awk 'in in {
Tested = 100;
Total = 0;
While (I <= test)
{
Total + = I;
I ++;
}
Print total;
}'
5050

2. For Loop

The for loop has two formats:

Format 1:

For (variable in array)

{Statement}

Example:

[Xifj @ localhost nginx] # awk 'in in {
For (K in environ)
{
Print K "=" environ [k];
}
}'

Awkpath =.:/usr/share/awk
Oldpwd =/home/web97
Ssh_askpass =/usr/libexec/OpenSSH/gnome-ssh-askpass
Selinux_level_requested =
Selinux_role_requested =
Lang = zh_cn.gb2312

......

Note: Environ is an awk constant and a typical subarray.

Format 2:

For (variable; condition; expression)

{Statement}

Example:

[Xifj @ localhost nginx] # awk 'in in {
Total = 0;
For (I = 0; I <= 100; I ++)
{
Total + = I;
}
Print total;
}'

5050

3. Do Loop

Format:

Do

{Statement} while (condition)

Example:

[Xifj @ localhost nginx] # awk 'in in {
Total = 0;
I = 0;
Do
{
Total + = I;
I ++;
} While (I <= 100)
Print total;
}'
5050

 

The above is an awk flow control statement. We can see from the syntax above that it is the same as the C language. With these statements, many Shell programs can be handed over to awk, and the performance is very fast.

Break	When a break statement is used for a while or for statement, the program exit loop occurs.
Continue	When a continue statement is used for a while or for statement, the program is moved to the next iteration cyclically.
Next	Can read the next input line and return to the top of the script. This avoids other operations on the current input row.
Exit	Statement to exit the main input loop and transfer the control to the end, if the end exists. If no end rule is defined or the exit statement is applied to the end, the script is executed.