awk Usage Manual

Introduction

Awk is a powerful text analysis tool, with the search for grep and the editing of SED, which is especially powerful when it comes to analyzing data and generating reports. To put it simply, awk reads the file line-by-row, using spaces as the default delimiter to slice each row, and then perform various analytical processing of the cut.

AWK has 3 different versions: AWK, Nawk, and gawk, which are not specifically described, generally referred to as the GNU version of awk, Gawk,gawk.

Awk has its name from the first letter of its founder Alfred Aho, Peter Weinberger and Brian Kernighan's surname. In fact, Awk does have its own language: The awk programming language, a three-bit creator has formally defined it as "style scanning and processing language." It allows you to create short programs that read input files, sort data, manipulate data, perform calculations on input, and generate reports, as well as countless other features.

Grammar

awk ' pattern + {action} ' {filenames}

Although the operation can be complex, the syntax is always the same, where pattern represents what AWK looks for in the data, and the action is a series of commands that are executed when a match is found. Curly braces ({}) do not need to always appear in the program, but they are used to group a series of instructions according to a particular pattern. pattern is the regular expression to be represented, surrounded by slashes.

The most basic function of the awk language is to browse and extract information in a file or string based on the specified rules, before awk extracts the information for additional text operations. A complete awk script is typically used to format the information in a text file.

Typically, awk is treated as a unit of a file's behavior. awk processes the text by executing the corresponding command for each line that receives the file.

Call awk

There are three ways of calling Awk

1. Command line mode awk [-f  field-separator]  ' commands '  input-file (s) where commands is the true awk command, [-F domain delimiter] is optional. Input-file (s) is the file to be processed. In awk, each line in a file, separated by a domain delimiter, is called a domain. In general, the default field delimiter is a space without naming the-F domain delimiter. The 2.shell script inserts all of the awk commands into a file and makes the awk program executable, and then the awk command interpreter is invoked as the first line of the script, again by typing the script name. Equivalent to the first line of the shell script: #!/bin/sh can be replaced by: #!/bin/awk3. Inserts all awk commands into a single file and then calls: Awk-f awk-script-file input-file (s) where,- The f option loads the awk script in Awk-script-file, and Input-file (s) is the same as above.

This chapter focuses on the command-line approach.

Example

1. Display/etc/passwd account;//This is an example of awk+action, each line will execute Action{print}. -f Specifies the domain delimiter as ': '.

#cat/etc/passwd |awk-  F ': '  {print $} '  Rootdaemonbinsys

2. Display the/etc/passwd account and the corresponding shell of the account, and the TAB key between the account and the shell;

#cat/etc/passwd |awk-  F ': '  {print ' \ t ' $7} ' root    /bin/bashdaemon  /bin/shbin     /bin/shsys     /bin/sh

3. Display the/etc/passwd account and the corresponding shell of the account, and the account and the shell are separated by commas, and add the column name Name,shell to all rows, add "Blue,/bin/nosh" to the last line.

The awk workflow is done by first executing the beging, then reading the file, reading a record with the/n line break, and then dividing the record by the specified field delimiter, populating the field, and $ $ representing all fields, representing the first field, $n representing the nth field, The action action corresponding to the execution pattern is then started. Then start reading the second record ... Until all the records have been read, the end operation is performed.

CAT/ETC/PASSWD |awk-  F ': '  BEGIN {print ' Name,shell '}  {print $ ', ' $7} END {print ' Blue,/bin/nosh '} ' name, Shellroot,/bin/bashdaemon,/bin/shbin,/bin/shsys,/bin/sh....blue,/bin/nosh

4. Search all lines that have the root keyword/etc/passwd. This is an example of the use of pattern, which matches the line of pattern (this is root) to execute the action (without specifying an action, the default output of the contents of each row). Search support for the regular, for example, root start: awk-f: '/^root/'/etc/passwd

#awk-F: '/root/'/etc/passwdroot:x:0:0:root:/root:/bin/bash

5. Search for all lines that have the root keyword/etc/passwd and display the corresponding shell. Action{print $7} is specified here.

# awk-f: '/root/{print $7} '/etc/passwd             /bin/bash

6, Statistics/etc/passwd: file name, the line number of each line, the number of columns per row, corresponding to the full line content.

#awk-  F ': '  {print ' filename: ' filename ', linenumber: ' NR ', columns: ' NF ', linecontent: ' $ '/etc/ passwdfilename:/etc/passwd,linenumber:1,columns:7,linecontent:root:x:0:0:root:/root:/bin/bashfilename:/etc/ Passwd,linenumber:2,columns:7,linecontent:daemon:x:1:1:daemon:/usr/sbin:/bin/shfilename:/etc/passwd,linenumber : 3,columns:7,linecontent:bin:x:2:2:bin:/bin:/bin/shfilename:/etc/passwd,linenumber:4,columns:7,linecontent:sys : x:3:3:sys:/dev:/bin/sh

7. Use printf instead of print to make the code more concise and easy to read.

awk  -F ': '  {printf ("filename:%10s,linenumber:%s,columns:%s,linecontent:%s\n", Filename,nr,nf,$0)} '/etc/ passwd

8, Statistics/etc/passwd account number. Count is a custom variable. The previous action{} has only one print, in fact print is just a statement, and action{} can have more than one statement, separated by a number.

awk ' {count++;p rint;} End{print "User Count is", count} '/etc/passwdroot:x:0:0:root:/root:/bin/bash......user count is 40

9, this example does not initialize the count, although the default is 0, but the appropriate practice is initialized to 0:

awk ' BEGIN {count=0;print ' [Start]user count is ', count} {Count=count+1;print $;} End{print "[End]user Count is], count} '/etc/passwd[start]user count is  0root:x:0:0:root:/root:/bin/bash ... [End]user count is  40

10. Count the number of bytes occupied by a file under a folder

Ls-l |awk ' BEGIN {size=0;} {size=size+$5;} End{print "[End]size is", size} '
[End]size is 8657198

11, as above, shown in M://note, statistics do not include subdirectories of folders.

[End]size is 8.25889 M

12. Count the number of bytes in a file under a folder, and filter files of 4096 size (usually folders):

[End]size is 8.22339 M

13. Display the/ETC/PASSWD account. This uses the For loop to iterate through the array

Awk-f ': ' BEGIN {count=0;} {Name[count] = $1;count++;}; End{for (i = 0; i < NR; i++) print I, Name[i]} '/etc/passwd0 root1 daemon2 bin3 sys4 sync5 Games ...

attached:

A. General expression metacharacters for awk
\ Exchange Code Sequence
^ Start matching at the beginning of the string
$ starts matching at the end of the string
. Match with any single string
[ABC] matches any of the characters in []
[A-ca-c] matches characters in the range A-c and a-c (in alphabetical order)
[^ABC] matches any character except all the characters in []
desk| Chair matches any one of the desk and Chair
[ABC] [DEF] Association. Matches any character in a, B, C, followed by any of the characters in D, E, and F.
* matches any of 0 or more characters in a, B, or C
+ match 1 or more occurrences of a, B, or C in any of the characters
？ Match with an empty string or a, B, or C in any one character
(blue| Black) Berry Merge regular expressions that match blueberry or blackberry

B. awk arithmetic operators
operator purpose
------------------
X^y X's Y power
X**y Ibid.
x%y calculating the remainder of X/y (modulo)
X+y x plus y
X × minus Y
X*y x multiply y
X/y × divide
Minus y (Y's switch symbol), also called one eye minus
++y y plus 1 use y (front Plus)
y++ using the Y value plus 1 (suffix Plus)
--y y minus 1 uses Y (pre-minus)
y--after use y minus 1 (suffix minus)
X=y assigns the value of Y to X
X+=y assigns the value of X+y to X
X-=y assigns X-y values to X.
X*=y assigns the value of X*y to X
X/=y assigns the value of x/Y to X%=y to assign the value of X%y to X
X^=y assigns the value of X^y to X
X**=y assigns the value of X**y to X

C. Tests allowed by awk:
operator meaning
X==y x equals y
X!=y x not equal to Y
X>y x greater than Y
X>=y x is greater than or equal to Y
X<y x less than Y
X<=y x less than or equal to Y?
X~re x matches regular expression re?
X!~re X does not match regular expression re?

D. awk operators (in ascending order of precedence)
=, + =,-=, *=,/=,%=
||
&&
> >= < <= = = = ~!~
XY (string link, ' x ' y ' becomes "xy")
+ -
* / %
++ --

E. [awk built-in variables] awk has many built-in variables for setting up the environment information, these variables can be changed, and some of the most commonly used variables are given below.
ARGC number of command line arguments
ARGV Command line parameter arrangement
ENVIRON support for the use of system environment variables in queues
FileName awk browses the file name
FNR number of records to browse files
FS sets the input domain delimiter, which is equivalent to the command line-F option
NF browsing the number of fields recorded
NR number of records read
OFS output Field delimiter
ORS Output Record delimiter
RS Control record delimiter
Full record
The first field of the current row

F. Awk's built-in functions
V function purpose or return value
------------------------------------------------
N gsub (Reg,string,target) replaces string in target each time regular expression reg matches
N Index (search,string) returns the position of the search string in string
A Length (string) to find the number of characters in string string
N Match (String,reg) returns the position of the regular expression reg-matched string
N printf (format,variable) formats the output, outputting the variable variable in the format provided by format.
N split (String,store,delim) breaks down the array elements of string store according to the delimiter Delim
N sprintf (format,variable) returns a formatted data containing format, variables is the data to put in the string
G strftime (Format,timestamp) returns a date or time string based on format, TIMESTMP is the time returned by the Systime () function
N Sub (reg,string,target) for the first time when the regular expression reg matches, replace the string in the target string
A substr (String,position,len) returns a substring that starts with a Len character of position
P totower (String) returns the corresponding lowercase character in a string
P ToUpper (String) returns the corresponding uppercase character in a string
Cotangent (radians) of a atan (x, y) x
Cosine of N cos (x) x (radians)
A exp (x) e X-Power
Integer portion of a int (x) x
Natural pair value of a log (x) x
Random number between N rand () 0-1
Sine (radians) of N sin (x) x
The square root of a sqrt (x) x
A Srand (x) initializes the random number generator. If x is omitted, the system () is used
G System () returns the elapsed time since January 1, 1970 (per second)

Official manual: Http://www.gnu.org/software/gawk/manual/gawk.html

awk Usage Manual