Introduction to linux shell script 20-Introduction to awk commands

Awk was born in 1977 and has three founders: Alfred Aho, Peter Weinberger, and Brian Kernighan. The names originate from the initials of the three founders.

Purpose:Process text files.

Awk features the ability to operate rows and columns.Man awkYou can refer to the awk Manual. The following describes how to learn the awk language using examples.

Syntax:

mawk [-W option] [-F value] [-v var=value] [--]  [-W option] [-F value] [-v var=value] [-f program-] [--] [ ...]

Parameter-instance: 1. awk first recognized

Take the topic command to upload the content to test.txt:

amosli@amosli-pc:~/learn/$  root           200m  49m  11m S         : amosli         534m  21m  12m S         : gnome- mongodb        623m  15m  S         : amosli        1005m 125m  17m S         : root                       S         : kworker/:          amosli            R         : amosli         686m   S         : hud- amosli         419m  10m  S         : telepathy- root                       S         : kworker/:             root             S         : root                       S         : root                       S         : ksoftirqd/             root      RT                 S         : migration/             root      RT                 S         : watchdog/              root      RT                 S         : migration/

Print the content in the following 2nd columns. Note that the pattern contains single quotation marks.

amosli@amosli-pc:~/learn/$   test.txt rootamoslimongodbamoslirootamosliamosliamoslirootrootrootrootrootrootroot

$0 indicates to print all data. $ n, n> 0 indicates to print column n.

2. disassemble the awk statement Block

First look at the example:

amosli@amosli-pc:~/learn/$   test.txt 

OK, isn't it easy to use? It doesn't matter if you don't know much about it. The following is an awk script which is usually divided into three parts:BEGIN statement Block,END statement BlockMatching with applicable modesGeneral statement Block. These three parts are optional, and none of them can be included in the script. Scripts are usually enclosed in single or double quotation marks. ItsGeneral mode structureAs follows:

The following are common syntax formats:

   filename  

How awk works:

(1) execute the statements in the in {commands} statement block.

(2) read a row from a file or stdin and run pattern {commands }. Repeat this process until all files are read.

(3) When reading to the END of the input stream, execute the END {commands} statement block.

Open the in statement block in awkExecuted before reading rows from the input stream. This is an optional statement block.

The END statement block is in the awkIt is executed after all rows are read from the input stream.. This is also optional.

The most important part is the general commands in the pattern statement block. This is also optional. If this statement block is not provided{Print} is executed by default}Print each read row. Awk will execute this statement block for each row read. For example:

amosli@amosli-pc:~/learn/$  -e  |  

In this example, BEGIN {print "now start"} is executed first, and then line1 \ nline2 is printed by default, after reading all the rows, run the END statement block {print "this is end "}

You can also remove both BEGIN and END, and only use the default printing method:

amosli@amosli-pc:~/learn/$  -e  |  

This method is the same as that in Example 1. After learning these two examples, we can basically understand what awk uses and what its format should be.

3. About print

The print here is basically the same as the C language, but I have forgotten about the C language. Let's see how print in awk is used?

Note the following two points:

1. When the print parameters are separated by commas, a space is used as the separator for parameter printing.

2. In the print Statement of awk, double quotation marks are treated as concatenation operator. Therefore, try to use single quotation marks for the outermost layer of pattern matching.

Example 1:

amosli@amosli-pc:~/learn/$  |  

Print var1,Var2 is separated by commas (,). Therefore, a space is used as the separator for printing. What if they are not separated by commas?

amosli@amosli-pc:~/learn/$  |  

When printing, v1v2 is attached to one piece. If you use other symbols to separate them, an error occurs when printing (I tried | :\/).

Example 2:

amosli@amosli-pc:~/learn/$  |  "+" var2 } +v2

"+" Is used here, and double quotation marks play the role of connection.

 

4. Special Variables

NR: indicates the number of records, indicating the current row number, which is the same as-n in the cat command.

NF: number of fields, indicating the number of fields in the current row

$0, $1, $2, $ n... As mentioned earlier, $0 indicates that all text content is output, and $ n indicates the nth row of data.

Examples of NR and NF:

Below is NF

amosli@amosli-pc:~/learn/$  test.txt |  

Each row has 12 fields, "1067 root 20 0 200 m 49 m 11 m S 2 1.3. 91 Xorg ", the field definition here is not the number of letters, but a column generated by space separation is a field,

It can also be understoodNumber of columns;

Below is NR:

amosli@amosli-pc:~/learn/$  test.txt |  

There are 15 rows in total.

5. Pass the external variable value to the awk,-v Parameter

amosli@amosli-pc:~/learn/$ var=-pc:~/learn/$  |  -v vr=$var 

First, define a var variable with a value of 99. Then, assign the value of the var variable to the outside and print out the vr.

In fact, it is to pass the value by value. However, it is too troublesome to pass many parameters. The following describes a simple method for passing values externally.

amosli@amosli-pc:~/learn/$  |    y1=$v1 y2= 

The-v parameter is not used here, but the value of y1 = $ v1 is put below, in fact, y1 = $ v1 y2 = $ v2 is treated as a file similar to awk '{print}' file.

6. Use awk for filtering

1) only the first four rows are displayed, and the number of rows is smaller than 5, which is the same as head test.txt-n 4.

amosli@amosli-pc:~/learn/$    test.txt   root           200m  49m  11m S         : amosli         534m  21m  12m S         : gnome- mongodb        623m  15m  S         : amosli        1005m 125m  17m S         : chrome 

2) only 3-5 rows are displayed.

amosli@amosli-pc:~/learn/$    test.txt   mongodb        623m  15m  S         : amosli        1005m 125m  17m S         : root                       S         : kworker/:   

3) only display rows containing amos

amosli@amosli-pc:~/learn/$   test.txt   amosli         534m  21m  12m S         : gnome- amosli        1005m 125m  17m S         : amosli            R         : amosli         686m   S         : hud- amosli         419m  10m  S         : telepathy-indic 

7. built-in functions of awk

Toupper (string), converts string to uppercase

amosli@amosli-pc:~/learn/$  

Length

amosli@amosli-pc:~/learn/$   test.txt 

Sqrt () takes the square root

amosli@amosli-pc:~/learn/$  

Awk features are very powerful, has a lot of knowledge points, and has a lot of online materials. Here we mainly learn about the linux shell script introduction version 1st. We need to learn more about awk content!

 

 

References: 1. awk Study Notes

2. awk English document

3. Brief awk tutorial