awk usage Examples

Source: Internet
Author: User

Analyze the text of log fetch record data
After the feedback on the information. Use examples

1: Analyze the number of ranges and corresponding occurrences in one of the text files.
Text structure:
Column 1 column 2
Key1 N
Key2 Y
Key3 Y1


Cat Mark_2016-08-08.log |awk ' {a[$2]++}end{for (i in a) {print i,a[i] | "Sort-k 1"}} '

Output results:

2: Specify a column that counts the number of times a keyword appears in the column

Cat Mark_2016-08-07.log |awk ' {if ($ = ' Y ') sum+=1} end {print ' sum = ', sum} '

The results are as follows:

The results are consistent with the above results.
3: Sort
The number of occurrences of a keyword is counted in ascending order.
The test data is as follows, the first column represents the keyword (there is no duplicates here), and the second column represents value, the number of times the key appears.

K1 1
K2 2
K3 1
K4 3
K4 4
K5 5
K6 3
K7 6
K8 3


Cat Mytest.log | awk ' {a[$2]++}end{for (i in a) {print i,a[i]} '

Run Result:

4 1
5 1
6 1
1 2
2 1
3 3

The first column of the result represents value in the test data (the second column of the test data), and the second column indicates the number of times the value value occurs, that is, the count of occurrences of the value.
Sort in ascending or descending order by the number of occurrences of the value:
Here, the parameter K2 of sort represents the operation of the second column result,-R indicates a descending order.

Cat Mytest.log | awk ' {a[$2]++}end{for (i in a) {print i,a[i] | "Sort-r-k2"}} '

The results of the operation are as follows, from which we can see that the record of the Value=3, appeared 3 times, Value=1 record, appeared 2 times:

3 3
1 2
6 1
5 1
4 1
2 1

Get descending order By value value in ascending order:

Cat Mytest.log | awk ' {a[$2]++}end{for (i in a) {print i,a[i] | "Sort-r-k1"}} '

The results of the operation are as follows:

6 1
5 1
4 1
3 3
2 1
1 2

4: Find the maximum value of a column and the data of the column
With Linecontent to store the contents of the row with the maximum value, max stores the maximum value, Linecout stores all the number of rows in the file, LineNum stores the number of rows that have the maximum value.

awk ' BEGIN {max=0;linecout=0} {linecout++;if ($2>max) {max=$2;linecontent=$0;linenum=linecout} fi} end {print ' Linecontent= ", Linecontent;print" linecout= ", Linecout;print" linenum= ", linenum} ' Hashbyscid1.txt

5: When key is duplicated, it needs to be counted
Not finished, to be continued

6: Filter the data by column orientation:
Filter out row information from column 2nd or column 3rd that appears 0
Cat krc_lrc_upload_status.txt |awk ' $2==0 | | $3==0{print $} '

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.