awk usage Examples

background

Analyze the text of log fetch record data
After the feedback on the information. Use examples

1: Analyze the number of ranges and corresponding occurrences in one of the text files.
Text structure:
Column 1 column 2
Key1 N
Key2 Y
Key3 Y1

Code:

Cat Mark_2016-08-08.log |awk ' {a[$2]++}end{for (i in a) {print i,a[i] | "Sort-k 1"}} '

Output results:

2: Specify a column that counts the number of times a keyword appears in the column
Code:

Cat Mark_2016-08-07.log |awk ' {if ($ = ' Y ') sum+=1} end {print ' sum = ', sum} '

The results are as follows:

The results are consistent with the above results.
3: Sort
The number of occurrences of a keyword is counted in ascending order.
The test data is as follows, the first column represents the keyword (there is no duplicates here), and the second column represents value, the number of times the key appears.

K1 1 K2 2 K3 1 K4 3 K4 4 K5 5 K6 3 K7 6 K8 3

Code:

Cat Mytest.log | awk ' {a[$2]++}end{for (i in a) {print i,a[i]} '

Run Result:

4 1 5 1 6 1 1 2 2 1 3 3

The first column of the result represents value in the test data (the second column of the test data), and the second column indicates the number of times the value value occurs, that is, the count of occurrences of the value.
Sort in ascending or descending order by the number of occurrences of the value:
Here, the parameter K2 of sort represents the operation of the second column result,-R indicates a descending order.
Code:

Cat Mytest.log | awk ' {a[$2]++}end{for (i in a) {print i,a[i] | "Sort-r-k2"}} '

The results of the operation are as follows, from which we can see that the record of the Value=3, appeared 3 times, Value=1 record, appeared 2 times:

3 3 1 2 6 1 5 1 4 1 2 1

Get descending order By value value in ascending order:
Code:

Cat Mytest.log | awk ' {a[$2]++}end{for (i in a) {print i,a[i] | "Sort-r-k1"}} '

The results of the operation are as follows:

6 1 5 1 4 1 3 3 2 1 1 2

4: Find the maximum value of a column and the data of the column
With Linecontent to store the contents of the row with the maximum value, max stores the maximum value, Linecout stores all the number of rows in the file, LineNum stores the number of rows that have the maximum value.

awk ' BEGIN {max=0;linecout=0} {linecout++;if ($2>max) {max=$2;linecontent=$0;linenum=linecout} fi} end {print ' Linecontent= ", Linecontent;print" linecout= ", Linecout;print" linenum= ", linenum} ' Hashbyscid1.txt

5: When key is duplicated, it needs to be counted
Not finished, to be continued

6: Filter the data by column orientation:
Filter out row information from column 2nd or column 3rd that appears 0
Cat krc_lrc_upload_status.txt |awk ' $2==0 | | $3==0{print $} '