How to use awk under Unix

1. What is awk

You may be familiar with UNIX, but you may be unfamiliar with awk, and it's not surprising that awk is far from being as well known as its superior functionality. What is awk? With most other Unix

The difference is, from the name, it is impossible to know the function of awk: It is neither an English word with an independent meaning, nor an abbreviation for several related words. In fact, AWK is an abbreviation for three names, and they

are: Aho, (Peter) Weinberg and (Brain) Kernighan. It was these three people who created awk---an excellent style scanning and processing tool.

What is the function of awk? Like SED and grep, awk is a style-scanning and processing tool. But its function is much stronger than sed and grep. AWK provides an extremely powerful feature: it can almost complete grep and SED

All the work that can be done, it can also be done with style loading, flow control, mathematical operators, Process control statements, and even built-in variables and functions. It has a complete language that should have a few

All the exquisite features. In fact, Awk does have its own language: The AWK programming language, which the three-bit creator of AWK has formally defined as: style scanning and processing languages.

2. Why Use awk

Even so, you may still ask, why do I use awk?

The first reason to use awk is that text-based style scanning and processing is something we often do, and awk does some work like a database, but unlike a database, it handles text files that don't

More Wonderful content: http://www.bianceng.cn/database/Oracle/

With specialized storage formats, ordinary people can edit, read, understand, and process them. Database files tend to have special storage formats, which makes it necessary for them to handle them with a database handler. Since

This kind of database-like processing we often encounter, we should find a simple way to deal with them, Unix has a lot of these tools, such as SED, grep, sort and found, etc.,

Awk is a very good one.

The second reason to use awk is that awk is a simple tool, which, of course, is relative to its powerful capabilities. Indeed, Unix has many good tools, such as the UNIX Natural development tool C language and its continuation C + +

is very good. But it is easier and simpler for awk to accomplish the same function as they do. This is primarily because AWK offers solutions that are adapted to a variety of needs: from the awk command line of the simple problem solution to the complex

Sophisticated awk programming language, the advantage of this is that you don't have to use complex methods to solve a simple problem. For example, you can solve simple problems with a single command line, and C can't, even if

A simple program, C language must also be written, compiled the whole process. Second, awk itself interprets execution, which makes the AWK program unnecessary to compile the process, and it also makes it and the shell

The script program can fit well. Finally, awk itself is simpler than the C language, although awk absorbs many of the best C language components, and familiarity with C can be a great help in learning awk, but awk itself does not need to use C

Language-a powerful development tool that takes a lot of time to learn to master its skills.

The third reason to use awk is that awk is an easy to get tool. In contrast to the C and C + + languages, awk has only one file (/bin/awk), and almost every version of UNIX offers its own version of awk, and you completely

Don't bother to think about how to get awk. But the C language is not the case, although the C language is a natural UNIX development tool, but this development tool is issued separately, in other words, you must for your UNIX version of the C language to open

The hair tools are paid separately (except, of course, with the D version), get and install it, and then you can use it.

For the above reasons, coupled with the powerful features of awk, we have reason to say that awk should be your first choice if you are dealing with text-style scanning-related work. Here is a general principle to follow:

If you have difficulty with a common shell tool or shell script, try awk, and if awk still doesn't solve the problem, then use C, or move to C + + if the C language still fails.

3. How to Invoke awk

As I've said before, Awk offers different solutions to meet a variety of needs:

(1) awk command line, you can use awk just like normal UNIX commands, and you can use the AWK programming language on the command line, although awk supports multiple lines of entry, but it's a long command line and guarantees

It is a headache to be correct, so this method is usually only used to solve simple problems. Of course, you can also refer to the awk command line or even the awk program script in the shell script.

(2) invoke the AWK program with the-f option. AWK allows a section of the awk program to be written to a text file and then invoked and executed with the-f option on the awk command line. The specific method we'll be at the back of the awk syntax

Mentioned in.

(3) Invoke the AWK program using the command interpreter: Using the command interpreter feature supported by UNIX, we can write an awk program to a text file, and then add the following in its first line:

#!/bin/awk-f

and give this text file permission to execute. After doing so, you can invoke and execute the AWK program in a manner similar to the following in the command line.

$awk script text name to process file

4. awk Syntax:

Like other Unix commands, awk has its own syntax:

awk [-F re] [parameter ...] [' Prog '] [F Progfile] [In_file ...]

Parameter description:

-F Re: Allows awk to change its field separator.

Parameter: This parameter helps to assign a value to a different variable.

' Prog ': The program statement segment of AWK. This statement must be enclosed by the Tanko number: ' and ', in case the shell interprets it. The standard form for this program statement segment is:

' Pattern {action} '

Where the pattern parameter can be any of the egrep regular expressions, it can be composed by using a syntax/re/plus some style matching techniques. Like SED, you can also use "," to separate two styles to select a

Range. For the details of the match, you can refer to the appendix, and if you still don't understand, find a UNIX book to learn grep and sed (I learned the matching technique while learning ed). The action argument is always surrounded by braces,

It consists of a system awk statement, with ";" between the statements. Separated. Awk interprets them and performs their actions on records that match the style given by pattern. Like the shell, you can also use "#" as an annotation character

, which makes the contents of "#" to the end of the line a comment, and they are ignored when interpreting execution. You can omit one of the patterns and the action, but not both, and when the pattern is omitted there is no style match,

Represents an action on all Rows (records) and a default action when the action is omitted--displayed on standard output.

-F Progfile: Allows awk to invoke and execute Progfile specify a program file. Progfile is a text file, and he must conform to AWK's syntax.

In_file:awk input file, awk allows processing of multiple input files. It is worth noting that awk does not modify the input file. If you do not specify an input file, awk accepts the standard input and displays the results in the standard

On the output. AWK supports input and output redirection.

5. Awk records, fields, and built-in variables:

As I said before, awk handles work in the same way as the database, and one of the similarities is that awk supports processing of records and fields, where the processing of fields is not implemented by grep and SED, which is also

One of the reasons that awk is superior to both. In awk, the default is always to treat a row in a text file as a record, and one part of a row as a field in the record. In order to manipulate these different words

section, awk borrows the Shell method, uses the $1,$2,$3 ... This way, in order to represent the different fields in the row (record). In particular, awk represents the entire row (record) with $ $. The difference between the fields is called the divide

Separated by the characters of the separator. The system default separator is a space. Awk allows you to change this separator in the form of the-F re in the command line. In fact, Awk remembers this separator with a built-in variable, FS. Awk

There are several such built-in variables, such as the record separator variable RS, the number of records currently working, NR, and so on, and the schedule later in this article lists all of the built-in variables. These built-in variables can be cited in the AWK program

Use or modify, for example, you can use the NR variable to specify the scope of work in pattern matching, or you can change the record separator RS so that a special character instead of a newline character is used as the separator for the record.

Example: Displays the first field, the third and seventh fields, separated by characters in lines seventh through 15th in the text file myfile:

Awk-f% ' nr==7,nr==15 {printf $ $} '

6. Awk's built-in functions

One of the reasons why awk has become a good programming language is that it absorbs many of the advantages of some excellent programming languages (such as C) language. One of these advantages is the use of built-in functions, which awk defines

and supports a series of built-in functions that make awk more sophisticated and powerful because of the use of these functions, for example, Awk uses a series of string-handling built-in functions that look like the C

The string-handling functions are similar in that they are used in the same way as functions in C, which makes awk more powerful in handling strings because of the use of these built-in functions. Later in this article are listed in the Appendix

The built-in functions provided by General awk may be somewhat different from your awk version, so it's a good idea to refer to the online Help in your system before you use it.

As an example of a built-in function, we'll introduce awk's printf function, which allows awk to match the output of the C language. In fact, there are many references in awk that are borrowed from the C language.

。 If you are familiar with the C language, you may remember the printf function, which provides powerful format output that once took us a lot of convenience. Luckily, we were reunited with it in awk. printf in awk

Almost identical to the C language, if you are familiar with the C language, you can use the C language in the mode of printf in awk. So here, let's just give an example, if you're not familiar with it, please feel free to

Find a C-language primer and turn it over.

Example: Displays the line number and 3rd field in the file myfile:

$awk ' {printf '%03d%s\n ', nr,$1} ' myfile

7. Use awk at the command line

In order, we should explain the contents of the AWK program, but before we explain, we will use some examples to review the previous knowledge, these examples are used in the command line, so we can know

How convenient it is to use awk at the command line. The reason for this is to pave the way for the following, on the other hand is to introduce some solutions to simple problems, we do not need to use complex methods to

Solve simple problems----Now that AWK offers a simpler approach.

Example: Displays a text file MyDoc all rows that match (contain) the string "Sun".

$awk '/sun/{print} ' mydoc

Because displaying the entire record (full line) is the default action for awk, you can omit the action item.

$awk '/sun/' mydoc

Examples: Here is a more complex example of matching:

$awk '/[ss]un/,/[mm]oon/{print} ' myfile