In shell commands or programming, awk's powerful text processing capabilities can be used. If you want to format a message or
If data packets are extracted from large text files, awk can complete these tasks. Awk is an explanation. Programming Language . Awk is also the most difficult to master in Shell filtering tools. Awk is a self-explanatory compilation.
Language. Combining awk, sed, and grep makes awk programming easier. The most basic feature of the awk language is to browse and extract information in a file or string based on specified rules. Three methods are available: awk. <1>. command line. <2> insert all the awk commands into a file and use chmod to make the awk file executable. <3>. Insert all the awk commands into a separate file and call it. The following describes various methods of awk. For more information, see the relevant documentation. There are four concepts in awk. 1. extraction domain 2. match Regular Expression 3. comparison domain 4. the following is a brief introduction to passing parameters to awk: 1 ). save the awk output: # awk '{print $0}' myfile> newfile2 ). use tee to output to the screen at the same time as the file (use MPs queue '|') # awk '{print $0}' myfile | tee newfile3 ). print the report header, # awk 'in in {print "Hello, this is Title \ n ---------"} {print $0} 'newfile4 ). print the end of the message. # awk 'in in {print $0} end {"End of file. "} 'myfile5 ). match. If Brown is included in the second column of myfile, print the information # awk '{if ($2 ~ /Brown/) Print $0} 'myfile6). Exact match, same as above, just set ~ Change to = 7). mismatch. Sometimes you need to browse the information and extract records of unmatched operations ~ The opposite symbol is !, # Awk '$0 !~ /Brown/'myfile8 ). for example, if it is less than: # awk 'if ($1 <$2) Print $1 'myfile9 ). set case sensitivity # awk '[Gg] reen/'myfile10 ). any character, expression/^... a/indicates that the first three characters are any characters
# Awk '$1 ~ /^... A/'myfile
11). Or link match, # awk '$0 ~ /(Yellow | red)/'myfile
12). And: &, or: |
13). awk built-in Variables
<1>.
To quickly view the number of records, use NR, for example: # awk 'end {print Nr} 'myfile
<2>. Use the NF variable below to display the number of fields in each read record and print the input file name in the end part. # Awk '{print
NR, NF, $0} end {print filename} 'myfile
<3>. Determine and output. # awk '{If (NR> 0 & $4 ~ /Brown/) Print $0 'myfile
14). You can set the input domain to the domain variable name.
15). Compare the domain values.
@ 1. assign a value to the variable name in begin. Generally, it is helpful to assign values in the begin part, which can be greatly reduced when the awk expression is modified.
@ 2. Use the actual value in the Link operation and enclose it in parentheses when using the link operation.
# Awk '{if ($6 <27) Print $0}' myfile
# Awk 'in in {Baseline = "27"} {if ($6 <baseline) Print $6} 'myfile
17). Modify the data field value
When modifying any domain in the awk, it is important that the actual file can be modified, but the awk copy in the cache is changed, awk will reflect the Modification Trace in the NR or NF variable.
# Awk '{if ($1 = "M. tans") $6 = $6-1; print $6}' myfile
18). Modify text fields
# Awk '{if ($1 = "J. troll") ($1 = "J. L. troll"); print $1}' myfile
19). Only display modification records
# Awk '{if ($1 = "J. troll") {$1 = "J. L. troll"; print $1} 'myfile
20). Create a new output domain
When processing data in awk, it is a good habit to create a new domain based on each domain. Example:
# Awk 'in in {print "name Score \ t"} if ($6 <$7) {$8 = $7-$6; print $8} 'myfile
21). Add a column value.
# Awk 'tot + = $6; end {print "club total point." tot} 'myfile
22). Print the file name and its length in the mode and put it in the tot variable.
# Ls-L | awk '/^ [^ d] {print $9 "\ t" $5} {tot + = $5} end {print "Total KB: TOT "}'
23). awk built-in string functions
24). awk script file
As described above, the command is only placed in a file, and the file must be! /Bin/awk-F, because this will make this file self-explanatory. Otherwise, it will not work. For ease of separation, it is best to include a file extension. awk. After writing the file, use chmod U + X to make the file executable.
The above are just some of my experiments. The awk command is very powerful. You can refer to some other manuals to get a clearer explanation.
Certificate --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
1. Call awk:
Method 1: Command Line
Awk [-F field-separator] 'commands' input-file (s)
[-F domain separator] is optional, because awk uses space as the default domain separator, you do not need to specify this option to browse text with spaces between domains, if you want to browse a file such as passwd, each field of this file uses a colon as the separator, you must specify the-F option, such:
Awk-F: 'commands' input-File
The second method is to insert all the awk commands into a file and make the awkProgramExecute the script and use the awk command interpreter as the first line of the script to call it by typing the Script Name.
The third method is to insert all the awk commands into a separate file and then call:
Awk-F awk-SCR convert Pt-file input-files (s)
-F indicates the awk script in the file awk_scr 1_pt _ file. input_file (s) indicates the file name for browsing using awk.
2. awk script
2.1 modes and actions
All awk statements are composed of modes and actions. There can be many statements in an awk script. The Mode part determines when the Action Statement is triggered and the event is triggered. Processing is the operation on data. If the mode part is omitted, the action is always executed.
Module
It can be any conditional statement, compound statement, or regular expression. The mode includes two special fields.
Begin and end. Use the begin statement to set the count and print headers. The begin statement is executed based on the input text before any Text Browsing action. End
The total number of output texts and the ending status flag are printed after the awk completes the Text Browsing.
2.2 domains and records
Use $1 and $3 to refer to the 1st and 3rd fields. Note that the fields are separated by commas. If you want to print all the domains of a record with five domains, you can use $0, that is, all domains.
Use the print command to print a domain or all fields. This is an awk action.
(1) extraction domain
(2) Save the awk output
$ Awk '{print $0}' grade.txt> wow
Using this method, the output result is not displayed on the display screen. Output directly to the file.
Use the tee command to output files to the screen at the same time
$ Awk '{print $0}' grade.txt | tee delete_me_and_die
(3) Enable and standard input
Use the awk script to input the file format:
● $ Belts. awk grade_student.txt
● $ Belts. awk <grade2.txt
● $ Grade2.txt | belts. awk
(4) print all records
$ Awk '{print $0}' grade.txt
(5) print individual records
$ Awk '{print $1, $4}' grade.txt
(6) print the report Header
Use \ t to divide domains. Use \ n to force the start of a new line
$ Awk 'in in {print "name belt \ n ----------------------------------"} {print $1 "\ t" $4} 'grade.txt
(7) print the end of the information
Use the end statement
$ Awk 'in in {print "Name \ n ---------"} {print $1} end {"end-of-Report"} 'grade.txt 2.5 condition Operator
(1) Matching:
$ Awk '{if ($4 ~ /Brown/) Print $0} 'grade.txt
If you query the string "brown" in the text, use/brown/
$ Awk '$0 ~ /Brown/'grade.txt
(2) exact match
$ Awk '$3 = "48" {print $0}' grade.txt
$ Awk '{if ($3 = "48") Print $0}' grade.txt
(3) Mismatch
$ Awk '$0 !~ /Brown/'grade.txt
$ Awk '{if ($4 !~ /Brown/) Print $0 'grade.txt
Exact Mismatch
$ Awk '$4! = "Brwon-2" {print $0} 'grade.txt
(4) less
$ Awk '{if ($6 <$7) Print $0 "$1 try better at the next comp"}' grade.txt
(5) less than or equal
$ Awk '{if ($6 <= $7) Print $1} 'grade.txt
(6) greater
$ Awk '{if ($6 >7 7) Print $1} 'grade.txt
(7) set Case sensitivity
$ Awk '/[Gg] reen/'grade.txt
(8) Any character
Name extraction. The fourth character in the first field of the record is a, and the period is used .. Expression/^... A/indicates any of the first three characters of a line. The fourth character is a, and the angle symbol indicates the beginning of a line.
$ Awk '$1 ~ /^... A/'grade.txt
(9) or link matching
For records whose extraction level is yellow or brown, use a vertical line |. Meaning match | one of the two modes. Note: When using a vertical line character, the statement must be enclosed in parentheses.
$ Awk '$0 ~ /(Yellow | brown)/'grade.txt
(10) first line
$ Awk '/^ 48/'input-File
(11) and
& Indicates that both sides are true.
$ Awk '{if ($1 = "P. Bunny" & $4 = "yellow") Print $0} 'grade.txt
(12) or
$ Awk '{if ($4 = "yellow" | $4 ~ /Brown/) Print $0} 'grade.txt 2.6 awk built-in Variables
The most common variables
--------------------------------------------------------
Number of argc command line parameters
Argv command line parameter arrangement
Environ supports the use of system environment variables in the queue
Filename awk browsed file name
Number of FNR browsing file records
FS sets the input domain separator, which is equivalent to the command line-F Option
Number of NF browsing records
Number of records read by NR
OFS output domain Separator
ORS output record Separator
RS control record delimiter
-------------------------------------------------------- 2.7 NF, NR, and filename
NR indicates the number of records
$ Awk 'end {print Nr} 'grade.txt
$5
Print the records of all students, carry their record numbers, and print the input file name in the end part.
NF displays several fields in a record.
$ Awk '{print NF, NR, $0} end {print filename}' grade.txt
When extracting information from a file, check whether there are records in the file. Use and to implement
$ Awk '{If (NR> 0 & $4 ~ /Brown/) Print $0} 'grade.txt
The powerful function of NF is to pass the return value of the variable $ PWD into awk and display its directory. Here you need to specify the domain Separator
$ Pwd
/Usr/local/etc
$ Echo $ PWD | awk-F/'{print $ NF }'
Another example is to display the file name:
$ Echo "/usr/local/etc/rc. sybase" | awk-F/'{print $ NF }'
RC. Sybase 2.8 awk Operator
Awk uses operators. Basic Expressions can be divided into numbers, strings, variables, fields, and array elements,
-----------------------------------------------------
= + = * =/= % = ^ = Value assignment operator
? Conditional expression Operator
| &&! And, and, non
~ !~ Matching operators, including matching and non-matching
<<==! >>> Relational operators
+-*/% ^ Arithmetic Operator
++ -- Prefix and suffix
-----------------------------------------------------
(1) set the input domain to the domain variable name
Generally, the variable name is set to name = $ n. Here, name is the name of the called domain variable, and N is the actual domain number.
For example, set the student domain name to name, the level domain name to belt, and the operation to name = $1; belts = $4. Note that the semicolon separates awk commands.
$ Awk '{name = $1; belts = $4; If (belts ~ /Yellow/) Print name "is belt" belts} 'grade.txt
$ P. Bunny is belt yellow (2) domain Value Comparison operation
There are two ways to test whether one numeric field is smaller than the other.
1) assign a value to the variable name in begin.
2) use the actual value in the Link operation.
Query all students whose scores are at in the competition
$ Awk '{if ($6 <27) Print $0}' grade.txt
J. Lulu 06/99 48317 green 9 24 26
J. Troll 07/99 4842 brown-3 12 26 26
Use begin
$ Awk 'in in {Baseline = "27"} {if ($6 <baseline) Print $0} 'grade.txt (3) modify the value of the numeric field
When any domain is modified in the awk, only the awk copies in the cache are modified. Awk will reflect the Modification Trace in the NR and NF variables.
$1 = $1 + 5: The value of domain 1 is added to 5, but make sure that the subset of the value field is Numeric.
Modify the current level shard of M. tansley,
Awk '{if ($1 = "M. tansley ") $6 = $6-1; print $1, $6, $7} 'grade.txt (4) Modify text fields
Modify the text field to assign a value to it. You need to assign a new string.
$ Awk '{if ($1 = "J. troll ") ($1 =" J. l. troll "); print $1} 'grade.txt (5) only display modification records
Use curly brackets after the mode to print only the modified part. Obtain the mode, and then implement the operation based on the results of the mode.
$ Awk '{if ($1 = "J. troll ") {$1 =" J. l. troll "; print $1 }}'grade.txt (6) create a new output domain
If you create a new addition domain {$4 = $2 + $3} based on other domains and assume that the record contains 3 upper domains, domain 4 is the new domain.
Create a new domain 8 in the file grade.txt to save the subtraction value between the current level of the domain and the highest level of the domain. The expression is '{$8 = $7-$6}'. The syntax first tests that the current level of the domain is smaller than the highest level of the domain.
$ Awk 'in in {print "Name \ t difference"} {if ($6 <$7) {$8 = $7-$6; print $1, $8} 'grade.txt
You can create a new domain and assign it to a more meaningful variable name.
$ Awk 'in in {print "Name \ t difference"} {if ($6 <$7) {diff = $7-$6; print $1, diff} 'grade.txt (7) Add the column Value
To increase the number of columns or perform running result statistics, use the symbol + =. The added result is assigned to the variable value on the left of the symbol, and the field added to the variable is on the right of the symbol. For example, add $1 to the variable total, and the expression is total + = $1.
Add the 'current level' of all students by TOT + = $6, and the tot is the sum of the results of 6 fields of the entire file browsed by the awk.
$ Awk '{(TOT + = $6)}; end {print "club student total points:" tot}' grade.txt
Club student total points: 155 (8) file length Addition
When viewing files in a directory, you can quickly view the length and total of all files, but you must exclude subdirectories. Use LS-l, and then the MPs queue is output to awk.
$ LS-L | awk '/^ [^ d]/{print $9 "\ t" $5} {tot + = $5} end {print "Total KB: "tot }'
Data. f 193
Gradc.txt 179
Grepstring 7
Local. cshrc 136
Local. login 157
Local. Profile 174
Results.txt 2129
Wow 179
Total KB: 4690 (9) built-in string functions
Awk built-in string functions
-----------------------------------------------------------
Gsub (R, S) replaces R with S in $0
Gsub (R, S, T) Replace r with S in the entire t
Index (S, T) returns the first position of string t in S.
Length (s) returns the length of S.
Match (S, R) test whether S contains a string matching r
Split (s, A, FS) divides s into sequence a in FS
Sprint (FMT, exp) returns the exp formatted by FMT
Sub (R, S) replaces s with the leftmost longest substring in $0
Substr (S, P) returns the suffix starting with P in string S.
Substr (S, P, n) returns the suffix of string s starting from P to n.
-----------------------------------------------------------
■ Gsub: ■ index:
$ Awk 'in in {print index ("bunny", "NY")} 'grade.txt
4 ■ length:
$ Awk '$1 = "J. troll" {print length ($1) "" $1}' grade.txt
7 J. troll
$ Awk 'in in {print length ("A Few Good Men ")}'
14 ■ match: If no match exists, 0 ■ split is returned.
Returns the number of elements in the string array.
$ Awk 'in in {print split ("123 #456 #678", myarray ,"#")}'
3 ■ sub
Use sub to find and replace the first appearance position of the mode. The STR string contains 'poped Popo pill'. Run the following sub command:
Sub (/OP/, "op", STR ). The result is 'poped Pope pill'
$ Awk '$1 = "L. tansley" {print substr ($1, 1, 5)}' grade.txt
L. Tan ■ substr:
It returns a part of a string based on its starting position and length.
$ Awk '$1 = "L. tansley" {print substr ($1, 1, 5)}' grade.txt
L. Tan
Another form of substr is to return a string suffix or a character after a specified position.
$ Awk '{print substr ($1, 3)}' grade.txt
Defines the string in the begin section, and returns the substring extracted from the nth character in the end section.
$ Awk 'in in {STR = "A Few Good Men"} end {print substr (STR, 7)} 'grade.txt
Good men ■ input a string to awk from shell:
Pass the string into the awk through the MPs queue
$ Echo "Stand-by" | awk '{print length ($0 )}'
8
Set the file name to a variable and output the MPs queue to awk. The file name without the extension is returned.
$ STR = "mydoc.txt"
$ Echo $ STR | awk '{print substr ($ STR, 1, 5 )}'
Mydoc (10) string shielding Sequence
When using a string or regular expression, you sometimes need to add a new line in the output or query a one-dimensional character.
Shielding sequence used in awk
------------------------------------------------------------
\ B Return key \ t Tab key
\ F go to the paper form \ DDD octal value
\ N New Line \ c any other special characters, for example \ is the backslash symbol
\ R enter key
------------------------------------------------------------
Print May Day, with the tab key in the center, followed by two new lines, and then print May Day. This time, we use the octal number.
$ Awk 'in in {print "May \ tday \ n \ nmay \ t \ 104 \ 141 \ 171"} '(11) awk output function printf
Awk printf Modifier
----------------------------------------------------------------
-Left alignment
The step in the width field. 0 indicates the STEP 0.
. Prec maximum string length, or the number of digits on the right of the decimal point
------------------------------------------------------------------ Awk printf format
-------------------------------------------------------------
% C ascii characters
% D integer
% E floating point number, scientific notation
% F floating point number, for example (123.44)
% G awk determines which floating point number to convert E or F
% O octal values
% S string
% X hexadecimal number
--------------------------------------------------------------
■ Character conversion
Observe the value of 65 in the ASCII code. The pipeline outputs 65 to awk.
$ Echo "65" | awk '{printf "% C \ n", $0 }'
A
Of course, you can use awk in the same way to get the same result.
$ Awk 'in in {printf "% C \ n", 65 }'
A
The conversion of all characters is the same. The following example shows the output result of '123' after the floating point conversion. After the integer is passed in, the decimal point is added.
$ Awk 'in in {printf "% F \ n", 999} '■ format the output
Print the names and serial numbers of all students. The names must be left aligned and 15 characters long followed by the serial number. Note that \ n line breaks are placed behind the last indicator. The output is automatically divided into two columns.
$ Awk '{printf "%-15 S % s \ n", $1, $3} 'grade.txt
M. tansley 48311
J. Lulu 48317
P. Bunny 48
J. Troll 4842
L. tansley 4712
Add text annotations to help you understand the meaning of the message. You can embed header information before the body.
$ Awk 'in in {print "Name \ t \ ts. Number"} {printf "%-15 S % s \ n", $1, $3} 'grade.txt
Name S. Number
M. tansley 48311
J. Lulu 48317
P. Bunny 48
J. Troll 4842
L. tansley 4712 ■ pass a value to an awk command line
Before viewing the awk script, let's first check how to pass variables in the awk command line.
Before running the awk command, pass the value to the awk variable. You need to put the variable in the command line. The format is as follows:
Awk command variable = input file value
In the command line, set the variable Age to 10, and then input awk to query all students under the age of 10.
$ Awk '{if ($5 <age) Print $0}' age = 10 grade.txt
You need to quickly view the space capacity of the file system and check whether it reaches a certain level.
$ DF-k | awk '($4 ~ /^ [0-9]/) {if ($4 <trigger) Print $6 "\ t" $4} 'trigger = 56000
Use the who command. The first column of the WHO command contains the registered user name.
# Who | awk '{print $1 "is logged on "}'
Root is logged on
Root is logged on
Sun is logged on
Awk is used to input environment variables. Use the environment variable LOGNAME to support the current user name.
# Who | awk '{if ($1 = user) Print $1 "you are connected to" $2}' user = $ LOGNAME
Root you are connected toconsole
Root you are connected topts/4
