Shell learning-awk getting started

Shell learning-awk getting started

I. awk application scenarios

Format data or extract records from a large text file

Ii. Usage

Command Line
$ Awk [-F field-separator] 'commands' input-file (s)
Commands is a real awk command
[-F domain delimiter] is optional. If no-F option is available, awk uses space as the Separator by default. If the domain separator is not a space, for example, to browse ": the-F option must be specified for the password file as the separator, for example:
Awk-F: 'commands' input-file

Awk script file
Write the command line in 1 into a file, use bash or sh to explain it, and execute the command Shawkscriptfile or ./Awk_script_file. The latter must grant the execution permission to the script file.

Write all the awk commands into a file awk_script_file, and use $ awk-f awk_script_file input-file (s), for example,

#record.txtzhangsan 21 M Hubeilisi     30 F Beijingwangwu   34 M Anhuiwangmazi 50 M Sichuanglilei    20 M Hubei#awk_script_file$2 > 30 {print $0}$awk -f awk.sh record.txtwangwu   34 M Anhuiwangmazi 50 M Sichuang

Iii. Composition of awk scripts

Awk statements are composed of modes and actions.

ModeThat is, the execution condition of the action, which can be a condition statement or regular expression. The mode includes two special fields BEGIN and END. Use the BEGIN statement to set the count and print headers. The BEGIN statement is executed before the text browsing. The END statement is used to print the total number of texts and the END state after the awk completes the Text Browsing.

ActionMost of them are used for printing. They are specified in {} and can contain if and loop. if they are not specified, awk will print all the browsing records.

Domains and records
Fields are separated by delimiters. $1, $2, $3 ..., $ n indicates the first, second, and third fields, and the nth field. $0 indicates all fields.

Save awk output

$awk '$2 > 30 {print $1, $2}' record.txt > tmp.txt

Regular Expression in an action
The regular expression in the action is expressed as:/this expression/, and the domain number matches the regular expression ~ Followed by a regular expression, you can also use if (). For example, print the nameIncludeAll records of "zhangsan,

$ awk '{if ($1~/zhangsan/)print $0}' record.txt

Or

$ Awk '$1 ~ /Zhangsan/'record.txt # all fields that meet the conditions are printed by default.

Common operators and regular expressions in conditional operations

+,-, *,/, %, + =,-=, % =, ^ = ++ ,--~ Fuzzy match, that is, ** include **!~ Mismatch = exact match>, <, >=, <= ,! = | ,&&

List Regular Expressions

[Gg] reen or (Green | green) match any of the first three characters of Green or green ^... a, and the fourth character is.

Print the report header and end

awk 'BEGIN{print "Name Age\n-------------------------"} $2>30 && $3 == "M" {print $1, $2} END {print "end-of-report"}' record.txt

Awk built-in Variables

AGRC command line parameter count ARGV command line parameter list ENVIRON system environment variable FNR Browse File record count FILENAME Browse File Name FS domain name separator, equivalent to the-F option, the number of NF browsing records, the number of NR read records, the number of OFS output domain delimiters, the ORS output record delimiters, the RS control record delimiters

Awk built-in string functions

Gsub (r, s) replaces rgsub (r, s, t) with s throughout $0 to replace rindex (s, t) with s throughout t) returns the length (s) at the first position of string t in s. returns the length of s. match (s, r). Tests whether s contains the matched r string split (s, a, fs) use fs to divide s into asprintf (fmt, exp) sequences. Return the fmt formatted string substr (s, p). Return the part of the string s starting from p. substr (s, p, p, n) returns the part of string s with a length of n starting from p.

Awk variable
Variables improve the readability of the awk program. The following is a simple example of variable usage.

awk '{name=$1;age=$2;sex=$3;if(age>30 && sex=="M")print "Name:" name " Age:" age}' record.txt 

Modify the value of a numeric field

# Increase the age of "lilei" by 1 and print out all records awk '{if ($1 = "lilei") $2 + = 1; print "Name: "$1" Age: "$2} 'record.txt # Change the name" lilei "to" Lilei ", print the modified record awk '$1 = "lilei" {$1 = "Lilei"; print "Name:" $1 "Age:" $2}' record.txt

Statistics

awk '{(tot += $2)};END{print "Total age:" tot}' record.txt 

This article only briefly introduces the basic syntax of awk. After reading this article, you can use awk to complete simple text processing functions. awk is profound and profound, if you want to learn more, read more professional awk documents and related books.