Common CentOS Shell skills awk

Practical features of awk:

Like sed, awk also scans files row by row. From the first row to the last row, it searches for the rows matching the specific template and runs the "select" action on these rows. If no action is specified for a template, the matched rows are displayed on the screen. If an action does not have a template, all rows specified by the action are processed.

1. Basic awk format:
/> Awk 'pattern' filename
/> Awk '{action}' filename
/> Awk 'pattern {action} 'filename

For specific application methods, see the following three use cases:
/> Cat employees
Tom Jones 4424 5/12/66 543354
Mary Adams 5346 11/4/63 28765
Sally Chang 1654 7/22/54 650000
Billy Black 1683 9/23/44 336500

/> Awk '/Mary/' employees# Print all rows containing the template Mary.
Mary Adams 5346 11/4/63 28765

# Print the first field in the file. This field starts at the beginning of each line and is separated by spaces or other separators by default.
/> Awk '{print $1}' employees
Tom
Mary
Sally
Billy

/> Awk '/Sally/{print $1, $2}' employees# Print the first and second fields of the row containing the template Sally.
Sally Chang

2. awk format output:
Both print and printf print functions are provided in awk. The parameters of the print function can be variables, values, or strings. The string must be referenced in double quotation marks and the parameters must be separated by commas. If there are no commas (,), the parameters are connected together and cannot be distinguished. Here, the comma serves the same purpose as the separator of the output file, except that the latter is a space. The following describes the basic transcoding sequence columns:

Transcoding meaning \ n newline \ r carriage return \ t Tab

/> Date | awk '{print "Month:" $2 "\ nYear:", $6 }'
Month: Oct
Annual: 2011

/> Awk '/Sally/{print "\ t \ tHave a nice day," $1, $2 "\! "} 'Ployees
Have a nice day, Sally Chang!

When printing a number, you may want to control the number format. We usually use printf to complete this function. The special variable OFMT of awk can also control the print format of numbers when using the print function. Its default value is "%. 6g" ---- the six digits after the decimal point will be printed.
/> Awk 'in in {OFMT = "%. 2f"; print 1.2456789, 12E-2 }'
1.25 0.12

Now we will introduce the more powerful printf function, which is similar to the printf function in C language. The following is a list of formatting specifiers for printf in awk:

Example result of the formatting specifier function % c prints a single ASCII character. Printf ("The character is % c. \ n", x) The character is A. % d prints The decimal number. Printf ("The boy is % d years old. \ n", y) The boy is 15 years old. % e print The number in scientific notation. Printf ("z is % e. \ n", z) z is 2.3e + 01.% f print floating point number. Printf ("z is % f. \ n", z) z is 2.300000% o print Octal numbers. Printf ("y is % o. \ n", y) y is 17.% s print the string. Printf ("The name of the culprit is % s. \ n", $1); The name of the culprit is Bob Smith. % x prints The hexadecimal number. Printf ("y is % x. \ n", y) y is f.

Note: assume that the face changing value in the list is x = A, y = 15, z = 2.3, $1 = "Bob Smith"

/> Echo "Linux" | awk '{printf "| %-15s | \ n", $1 }'# %-15s indicates that the space with 15 characters is retained and left aligned.
| Linux |

/> Echo "Linux" | awk '{printf "| % 15s | \ n", $1 }'# %-15s indicates that the space of 15 characters is retained and the right alignment is also made.
| Linux |

# % 8d indicates that the number is right aligned and the space of 8 characters is retained.
/> Awk '{printf "The name is %-15 s ID is % 8d \ n", $1, $3}' employees
The name is Tom ID is 4424
The name is Mary ID is 5346
The name is Sally ID is 1654
The name is Billy ID is 1683

3. Records and domains in awk:
In awk, the default record delimiter is carriage return, which is saved in its built-in variables ORS and RS. The $0 variable indicates the entire record.
/> Awk '{print $0}' employees# This is equivalent to the default print behavior.
Tom Jones 4424 5/12/66 543354
Mary Adams 5346 11/4/63 28765
Sally Chang 1654 7/22/54 650000
Billy Black 1683 9/23/44 336500

Variable NR (Number of Record), which records the Number of each Record.
/> Awk '{print NR, $0}' employees
1 Tom Jones 4424 5/12/66 543354
2 Mary Adams 5346 11/4/63 28765
3 Sally Chang 1654 7/22/54 650000
4 Billy Black 1683 9/23/44 336500

The variable NF (Number of Field) records the Number of fields in the current record.
/> Awk '{print $0, NF}' employees
Tom Jones 4424 5/12/66 543354 5
Mary Adams 5346 11/4/63 28765 5
Sally Chang 1654 7/22/54 650000 5
Billy Black 1683 9/23/44 336500 5

# Generate employees2 Based on employees. For usage of sed, refer to the previous blog.
/> Sed's/[[: space:] \ + \ ([0-9] \)/: \ 1/g; w employees2 'employees
/> Cat employees
Tom Jones: 4424: 5/12/66: 543354
Mary Adams: 5346: 11/4/63: 28765
Sally Chang: 1654: 7/22/54: 650000
Billy Black: 1683: 9/23/44: 336500

/> Awk-F: '/Tom Jones/{print $1, $2}' employees2# Here, the characters after the-F option indicate delimiters.
Tom Jones 4424

The variable OFS (Output Field Seperator) represents the delimiter between Output fields. The default value is space.
/> Awk-F: '{OFS = "? "};/Tom/{print $1, $2} 'employees2# What is the delimiter between fields in the output? (Question mark)
Tom Jones? 4424

For awk, the mode part controls the input of this action part. Only records that meet the mode condition can be processed by the basic operation part, in addition to regular expressions (in the example above), awk also supports conditional expressions, such:
/> Awk '$3 <4000 {print}' employees
Sally Chang 1654 7/22/54 650000
Billy Black 1683 9/23/44 336500

In curly braces, a statement separated by semicolons is called an action. If the mode is before the action, the mode determines when the action is triggered. An action can be a statement or a group of statements. Separate statements with semicolons or line breaks, for example:
Pattern {action statement; etc.} or
Pattern {
Action statement
Action statement
}
Modes and actions are usually bundled together. Note that the action is a statement in curly brackets. The pattern control action starts from the first left curly braces and ends with the first right curly braces, as follows:
/> Awk '$3 <4000 &/Sally/{print}' employees
Sally Chang 1654 7/22/54 650000

4. Matching OPERATOR:
"~ "Is used to match a regular expression in a record or domain.
/> Awk '$1 ~ /[Bb] ill/'ployees# Display all rows whose first domain matches Bill or bill.
Billy Black 1683 9/23/44 336500

/> Awk '$1 !~ /[Bb] ill/'ployees# Display all rows in which the first domain does not match Bill or bill !~ Indicates the meaning of the mismatch.
Tom Jones 4424 5/12/66 543354
Mary Adams 5346 11/4/63 28765
Sally Chang 1654 7/22/54 650000

5. Basic Application Example of awk:
/> Cat testfile
Northwest NW Charles Main 3.0. 98 3 34
Western WE Sharon Gray 5.3. 97 5 23
Southwest SW Lewis Dalsass 2.7. 8 2 18
Southern SO Suan Chin 5.1. 95 4 15
Southeast SE Patricia Hemenway 4.0. 7 4 17
Eastern ea tb save age 4.4. 84 5 20
Northeast ne am Main Jr. 5.1. 94 3 13
North NO Margot Weber 4.5. 89 5 9
Central CT Ann Stephen 5.7. 94 5 13

/> Awk '/^ north/'testfile# Print all rows starting with north.
Northwest NW Charles Main 3.0. 98 3 34
Northeast ne am Main Jr. 5.1. 94 3 13
North NO Margot Weber 4.5. 89 5 9

/> Awk '/^ (no | so)/'testfile# Print all rows starting with so and no.
Northwest NW Charles Main 3.0. 98 3 34
Southwest SW Lewis Dalsass 2.7. 8 2 18
Southern SO Suan Chin 5.1. 95 4 15
Southeast SE Patricia Hemenway 4.0. 7 4 17
Northeast ne am Main Jr. 5.1. 94 3 13
North NO Margot Weber 4.5. 89 5 9

/> Awk '$5 ~ /\. [7-9] +/'testfile# The fifth field match contains a. (vertex) followed by a number of 7-9.
Southwest SW Lewis Dalsass 2.7. 8 2 18
Central CT Ann Stephen 5.7. 94 5 13

/> Awk '$8 ~ /[0-9] [0-9] $/{print $8} 'testfile# Print the eighth field ending with two digits.
34
23
18
15
17
20
13

10. awk expression functions:

1. comparison expression:
The comparison expression matches the rows that run only when the condition is true. These expressions use Relational operators to compare numbers and strings. See the following list of conditional expressions supported by awk:

Example of operator meaning <less than x <y <= less than or equal to x <= y = equal to x = y! = Not equal to x! = Y> = greater than or equal to x> = y> greater than x> y ~ Match x ~ /Y /!~ Does not match x !~ /Y/

/> Cat employees
Tom Jones 4424 5/12/66 543354
Mary Adams 5346 11/4/63 28765
Sally Chang 1654 7/22/54 650000
Billy Black 1683 9/23/44 336500

/> Awk '$3 = 5346' employees# Print the row with the third domain equal to 5346.
Mary Adams 5346 11/4/63 28765

/> Awk '$3> 5000 {print $1}' employees# Print the first field of the row whose third field is greater than 5000.
Mary

/> Awk '$2 ~ /Adam/'employess# Print the row in which the second domain matches Adam.
Mary Adams 5346 11/4/63 28765

2. conditional expressions:
The conditional expression uses two symbols-question mark and colon-to assign a value to the expression: conditional expression1? Expression2: expression_3. Its logic is equivalent to the conditional expression in C language. The corresponding if/else statement is as follows:
{
If (expression1)
Expression2
Else
Expression3
}
/> Cat testfile
Northwest NW Charles Main 3.0. 98 3 34
Western WE Sharon Gray 5.3. 97 5 23
Southwest SW Lewis Dalsass 2.7. 8 2 18
Southern SO Suan Chin 5.1. 95 4 15
Southeast SE Patricia Hemenway 4.0. 7 4 17
Eastern ea tb save age 4.4. 84 5 20
Northeast ne am Main Jr. 5.1. 94 3 13
North NO Margot Weber 4.5. 89 5 9
Central CT Ann Stephen 5.7. 94 5 13

/> Awk 'nr <= 3 {print ($7> 4? "High" $7: "low" $7)} 'testfile
Low 3
High 5
Low 2

3. Mathematical expression:
Operations can be performed in the mode. awk regards all operations as floating-point operations, as shown in the following list:

Operator Description Example + Add x + y-minus x-y * multiply x * y/divide x/y % to get the remainder x % y ^ Multiplication Side x ^ y

/> Awk '/southern/{print $5 + 10}' testfile# If the record contains the regular expression southern, add 10 to the fifth field and print it.
15.1

/> Awk '/southern/{print $8/2}' testfile# If the record contains the regular expression southern, divide the eighth field by 2 and print it.
7.5

4. logical expression:
See the following list:

Example of operator meaning & logic and a & B | logic or a | B! Non-logical! A

/> Awk '$8> 10 & $8 <17' testfile# Print the records with the eighth Domain value greater than 10 and less than 17.
Southern SO Suan Chin 5.1. 95 4 15
Central CT Ann Stephen 5.7. 94 5 13

# Print the second domain that is equal to NW, or the first domain matches the first and second fields of the row in south.
/> Awk '$2 = "NW" | $1 ~ /South/{print $1, $2} 'testfile
Northwest NW
Southwest SW
Southern SO
Southeast SE

/> Awk '! ($8> 13) {print $8} 'testfile# Print the eighth field of the row whose eighth field is not greater than 13.
3
9
13

5. Scope template:
Range template matching starts from the first appearance of the first template to the first appearance of the second template, the next appearance of the first template to the next appearance of the first template, and so on. If the first template matches and the second template does not appear, awk displays all rows at the end of the file.
/> Awk '/^ western/,/^ eastern/{print $1}' testfile# Print the first domain of the record starting with western to eastern.
Western WE
Southwest SW
Southern SO
Southeast SE
Eastern EA

6. Value assignment:
# Find the record with the third domain equal to Ann, assign the domain a new value of Christian, and then print and output the record.
/> Awk '$3 = "Ann" {$3 = "Christian"; print}' testfile
Central CT Christian Stephen 5.7. 94 5 13

/> Awk '/Ann/{$8 + = 12; print $8}' testfile# Locate the record containing Ann, set the value of the eighth field of the record to + = 12, and print the output.
25