awk Usage Summary

Author: Dong | Can be reproduced, but must be in the form of hyperlinks to indicate the original source of the article and author information and copyright notice
URL: http://dongxicheng.org/script/awk-usage/

1. What is awk

Awk is a style-scanning and processing tool provided by Unix/linux, very good at working with structured data and generating forms. Similar to SED and grep, but more functional than both, it can be viewed as a scripting language because awk features a variety of scripting languages. This article describes how to use awk.

2. AWK Program Design Model

The awk program consists of three parts: initialization (preparation before processing input, placing in begin blocks), data processing (processing input data), finishing processing (processing of input after completion, and putting into end blocks). In the process of "data processing", the instructions are written in a series of pattern/action procedures, which are used to test the rules of the input rows to determine whether they will be applied to the input rows.

3. awk invocation Mode

There are three main modes of invocation, respectively:

(1) awk command line

You can use awk just as you would with ordinary UNIX commands, and you can use the AWK programming language on the command line, which is typically used only to solve simple problems. Of course, you can also refer to the awk command line or even the awk program script in the shell script.

(2) invoke the AWK program with the-F option

AWK allows a section of the awk program to be written to a text file and then invoked and executed with the-f option on the awk command line.

(3) Invoking the AWK program using the command interpreter

With the command interpreter features supported by UNIX, we can write an awk program to a text file and then add #!/bin/awk–f to its first line.

4. Awk Grammar

Like other UNIX commands, AWK has its own syntax:

awk [-F re] [parameter ...] [' Prog '] [F Progfile] [In_file ...]

(1)-F Re: Allow awk to change its field separator.

(2) Parameter: This parameter helps to assign values to different variables.

(3) ' Prog ': The program Statement section of AWK. This statement must be enclosed by the Tanko number: ' and ', in case the shell interprets it. The standard form for this program statement segment is: ' Pattern {action} '

Where the pattern parameter can be any of the egrep regular expressions, it can be composed by using a syntax/re/plus some style matching techniques. Like SED, you can also use "," to separate two styles to select a range. The action argument is always surrounded by braces, which consist of a series of awk statements, each with a ";" Separated. Awk interprets them and performs their actions on records that match the style given by pattern. You can omit one of the patterns and actions, but not both, and when you omit pattern, there is no style match, which means that all rows (records) are performed and the default action is performed when the action is omitted--shown on standard output.

(4)-F Progfile: Allow Awk to invoke and execute Progfile specify a program file. Progfile is a text file that must conform to AWK's syntax.

(5) In_file:awk input file, awk allows processing of multiple input files. It is worth noting that awk does not modify the input file. If you do not specify an input file, awk accepts the standard input and displays the results on the standard output.

5. awk Script Writing

5.1 Awk's built-in variables

There are two types of built-in variables in awk, a class of users can be changed according to needs, mainly: FS: input data of the field separator, RS: Enter the data of the record delimiter, OFS: Output Data of the field divider, ORS: Output data of the record separator; the other kind is the system automatically changes, such as: NF: The number of fields currently recorded, NR: current record number, and so on.

An example is provided:

Awk-f ":" ' {print $ ' "$} '/etc/passwd #打印passwd中的第1, 3 fields

5.2 Pattern/action Mode

The AWK program part uses the Pattern/action mode, which is to use action logic to handle data matching pattern.

An example is provided:

/^$/{print "This is a blank line!"} #判断当前是不是空格

$ ~/ma/{print $ ", $} #判断第5个字段是不是含有" MA "

NF = = 3 {Print "This particular the record has three fields:" $}

5.3 Begin and end

Two special expressions in awk, begin and end, both of which can be used in pattern, provide the function of beginning and ending to give the program an initial state and perform some cleanup work after the program finishes. Any actions listed after begin (within {}) will be executed before awk starts scanning input, and the operations listed after end will be executed after the input of the full section is scanned. Therefore, you typically use begin to initialize variables, and end to output final results.

Example: Sales amount in the cumulative sales file xs (assuming the sales amount is in the third field of the record):

$awk

> ' BEGIN {fs= ': ';p rint ' statistic Sales Amount '; total=0}

>{print $3;total=total+$3;}

>end {printf "Total Sales Amount:%.2f", sum} ' SX

5.4 Circular Statements

The looping statements in awk are similar to C, including Do...while,for,continue/break,while, etc.

5.5-piece statement

Conditional statements in awk are similar to C, but they are better supported.

An example is provided:

if (x ~/[yy] (es)?/) Print x #如果x符合pattern "[YY] (es)?", Print

{if ($!~/matchme/) {print $ $}} #如果 $ does not contain ' Matchme ', the 1,3,4 field is printed

5.6 function

(1) Mathematical functions

Awk contains rich mathematical functions, including: cos (x), sin (x), log (x),....

(2) String function

Awk contains rich string functions, such as:

Length (x): To find the lengths of the string x

Index (T,S): Returns the position of the string s in the string t

Match (S,R): Regular expression R appears in string s

...

(3) Custom function

AWK allows custom functions, syntax: function name (parameter-list) {statements}

Such as:

Function Insert (STRING, POS, INS) {

Before_tmp = substr (STRING, 1, POS)

After_tmp = substr (STRING, POS + 1)

return before_tmp INS after_tmp

}

Call Method: Print Insert ($ 4, "XX")

6. Awk is mixed with the shell

Because awk can be used as a shell command, awk blends well with the shell batch program, which provides the possibility to implement a mixed programming of awk and shell programs. The key to implementing mixed programming is the conversation between awk and Shell script, in other words, the exchange of information between awk and shell script: Awk Gets the required information from the shell script (usually the value of the variable), executes the shell command line in awk, and S Hell script sends the results of the command execution to awk and shell script reads awk's execution results, and so on.

6.1. awk reads shell script variables

In awk we can read variables in the Sell Scrpit program in the form of ' $ variable name '.

For example: Read the variable name in the shell Scrpit program

Here are two ways to do this:

The first is a more common approach: (double and single quotes need to be clear)

#!/bin/sh

Name= ' John '

awk ' {print $1,$2, ' $name '} ' myfile

————————————————-

The second way:

#!/bin/sh

Name= ' John '

awk ' {print $1,$2,myname} ' myname= $name myfile

In this way, however, awk custom variable myname cannot be used in begin.

6.2. Send the execution results of the shell command to awk processing

As a way of delivering information, we can pass the result of a shell command to awk through a pipe line (|):

Example: Sample awk handles the execution result of a shell command

$who-u | awk ' {printf ('%s is executing%s\n ', $2,$1)} '

6.3. Shell script reads the execution result of awk

In the shell, you can assign the awk execution result to the shell variable. We can deposit the results of awk execution into a shell script variable in the form of a variable name = ' awk statement '. Of course, you can also use the pipe line method to pass the awk execution result to the shell Script program processing.

For example, find the line with the fail string in the MyFile and count the number of rows, and finally print out the format: there are (number of rows) lines

#!/bin/sh

temp= ' awk '/fail/{print $} ' myfile |wc-l '

echo "There are $temp lines

7. Reference materials

(1) http://fanqiang.chinaunix.net/program/other/2005-09-07/3621.shtml

(2) "Unix awk use Manual", Author: Inexplicable publication time: 2002/01/27 01:39pm

(3) book "Sed and Awk" Revised third edition

Original articles, reproduced please specify: Reprinted from Dong's Blog

This article link address: http://dongxicheng.org/script/awk-usage/