Shell learning-Getting started with awk

I. AWK application Scenario

Format data or extract records from a large text file

Two. How to use

2. Command line mode
   $awk [-F field-separator] ' commands ' input-file (s)
   Commands is the real awk command.
   The [-F domain delimiter] is optional, and if the-f option is not available, awk defaults to a space as a delimiter, and if the field separator is not a space, such as the password file that you want to browse with ":" As the delimiter, you must indicate the-f option, such as:
   Awk-f: ' Commands ' input-file

3. awk script File
   Write the command line in 1 to a file, use bash or sh to interpret it, and execute the command to shaW k s CR IP tf Il eor ./awk_script_file, which needs to add execute permissions to the script file.

5. Write all awk commands to a file awk_script_file, with $awk-f awk_script_file Input-file (s), for example,

#record.txt21 M Hubeilisi     30 F Beijingwangwu   3450 M Sichuanglilei    20 M Hubei#awk_script_file$230$0}$awk-f awk.sh record.txtwangwu   3450 M Sichuang

Three. awk Script composition

awk statements are made up of patterns and actions.

The pattern is the execution condition of the action, which can be a conditional statement or a regular expression, and the pattern consists of 2 special fields begin and end. Use the BEGIN statement to set the count and print head. The BEGIN statement executes before the text is browsed. The end statement is used to print the total and end state of the text after awk completes the text-browsing action.

Most actions are used for printing, indicated within {}, can include if and loops, and if not specified, awk will print out all of the browsed records.

fields and Records
fields are delimited by delimiters, representing $1，$2，$3...,$n all fields in the first, second, third, and Nth fields $0 .

Save awk output

$awk ‘$2 > 30 {print $1, $2}‘ record.txt > tmp.txt

Regular Expressions in an action
The regular expression in the action is expressed as:/This expression/, the field number matches the regular expression with the ~ followed by the regular expression, or if (). For example, all records with "Zhangsan" in the name are printed,

$ ‘{if ($1~/zhangsan/)print $0}‘ record.txt

Or

$ ‘$1~/zhangsan/‘ record.txt#默认打印所有满足条件的字段

Operators and regular expressions commonly used in conditional operations

+, -, *, /, %, +=, -=, %=, ^=++, --~模糊匹配，即**包含**!~不匹配==精确匹配 >, <, >=, <=, !=||, &&

Regular Expression Enumeration

[Gg]reen或(Green|green) 匹配Green或green^...a 行首前三个字符任意，第四个是a

Print report header and end of report

awk"Name    Age\n-------------------------"$2$3"M"$1$2"end-of-report"}‘ record.txt

awk built-in variables

AGRC          命令行参数个数ARGV          命令行参数列表ENVIRON       系统环境变量FNR           浏览文件的记录数FILENAME      浏览文件名FS            域名分隔符，等价于-F选项NF            浏览记录域个数NR            已读记录数OFS           输出域分隔符ORS           输出记录分隔符RS            控制记录分隔符

Awk built-in string functions

gsub(R, s)In the entiresAlternativeRgsub(R, S, T)In the entireTUsedsAlternativeRIndex(S, t)ReturnsThe stringTThe first positionlength(s)ReturnsLengthMatch(S, R)TestsDoes it contain a matchingRStringSplit(S, A, FS)UseFSPutsSplit into sequencesasprintf(FMT, exp)Return to WarpFMTThe formatted stringsubstr(S, p)return stringsIn fromPThe beginning partsubstr(S, p, N)return stringsIn fromPStart length isNPart of

awk Variable
Variables improve the readability of the AWK program, the following is a simple example of the use of variables

awk ‘{name=$1;age=$2;sex=$3;if(age>30 && sex=="M""Name:"name" Age:"record

To modify the value of a numeric field

#将姓名为"lilei"的年龄增加1，然后打印出所有记录awk ‘{if ($1=="lilei"$2"Name:"$1" Age:"$2#将姓名为"lilei"的姓名修改为"Lilei",打印出修改后的记录awk ‘$1=="lilei" {$1="Lilei""Name:"$1" Age:"$2

Statistics

awk ‘{(tot += $2)};END{print "Total age:" tot}‘ record.txt 

This article simply introduces the basic syntax of awk, and after reading this article you can use awk to complete a simple text processing function, and awk is extensive, if you want to learn more about the more professional awk documents and related books.

Shell learning-Getting started with awk