Awk 4. Example of "Reprint Update" Linux tool

======
Basic article
======

1. An action instance without pattern
A awk ' {print NR $ $NF} ' data.txt prints line numbers, first and last columns, no separators in middle
b awk ' {print $, $NF} ' data.txt prints the first and last columns, with delimiters in the middle
C awk ' {print$0, $NF +10} ' data.txt prints the entire line and prints the last line plus 10 results

2. Action instance with pattern
A awk '/[0-9]/' data.txt prints any column in the record that contains the number 0-9 row
b awk '/01/| | /02/' data.txt prints rows that contain 01 or 02
C awk '/01/,/02/' data.txt prints a line containing both 01 and 02, equivalent to awk '/01/&&/02 '
D awk ' $1= =1001{print $ Data.txt ' Prints the second column that matches the first column equals 1001
E awk ' $2== ' Steven ' {print} ' data.txt prints those rows that match the second column equals Steven
F awk ' $3>20&&$3<30 ' data.txt prints those rows in the third column between 20 and 30
G Nawk ' $3* $NF <100 ' data.txt print the rows of the third and last columns with a product less than 100
H awk ' $6~/01/{print $ ' data.txt print matches the second column in the sixth column that contains only 01 of those rows
I awk ' Nr>3{print $1,$2} ' data.txt starts printing the first and second columns from line fourth

3. Instances with begin, end and include FS, OFS
A Awk–f "+" ' {print '} ' data.txt separates columns by the + delimiter and prints the first column
b Nawk–f ' [+\t$] ' {print $6,$7,$8,$9} ' data.txt press + or \ t or $ both to make delimiters and to print the specified column
C Nawk ' begin{fs= "[\t+$]"}{print $6,$8,$9} ' data.txt Press +,\t,$ (order doesn't matter) as a separator and print the specified column
D Nawk ' begin{fs= "[\t+$]"; ofs= "%"}{print$6,$7,$8,$9} ' data.txt press +,\t,$ as separator, print specified column with% as output delimiter
E Nawk–f ' [[]] ' {print $} ' data.txt press [or] as a delimiter and print the first column
F Nawk–f ' [] ' {print '} ' data.txt, press [or] as a delimiter, and print the first column
Special attention is given to the FS definitions in-F and begin as follows:
A The-f parameter is immediately followed by a single delimiter, using the double quotation mark "", for example –f "+"
B The-f parameter is immediately followed by multiple delimiters, and the single quotation mark "is used with [], the middle order does not matter, such as-f ' [+$] '
C The FS in begin, whether individual or multiple, is enclosed in double quotation marks "", and multiple are required to be added []
Example: Nawk ' begin{fs= "[\t+$]"}{print $6,$8,$9} ' data.txt
D If there is both FS and OFS in the begin, it is recommended to separate the middle with a semicolon ";", otherwise an error is indicated.
Example: Nawk ' begin{fs= "[\t+$]"; ofs= "%"}{print$6,$7,$8,$9} ' Data.txt
E. The semi-square brackets [] can also be used as separators, the order does not matter. For example:-F ' [[]] ' or –f ' [] '
F. When using multiple separators, it is best to use Nawk to avoid grammatical errors or to get the correct results.

4. Conditional IfElse and for-while loop instances
A Nawk ' {print ($3>25 $ "old": $ "Young")} ' Data.txt if the third column is greater than 25 to print an old;
b awk ' {if ($1>1002) print$2; else print $} ' data.txt print the second column if the second column is greater than 1002, otherwise print the third column
C awk ' Nr>1{if ($3<30) Print$2;elseif ($3<40) print$2 "M"; else exit;} END{PRINT10} ' Data.txt
From the second line to determine the value of the third column, if less than 30 to print the $, if it is greater than 30 and less than 40, print a $ and followed by M, if all do not conform to exit processing line, but the operation after end is still executed
D awk ' begin{ors= ' "}{for (i=1;i<nf-2;i++) print $i" \ t "}{print" \ n "} ' Data.txt
Print out the first few columns of each row, and the last few columns do not print. Special note: By default, print is printed on another line each time it is executed. So to avoid having to re-line each column, set ORS to NULL,
However, after each qualifying column has been printed, you need to wrap it manually (the print outside of the loop acts on this).
E awk ' begin{ors= ' "}{i=1;while (i<nf-2) {print $i" \ t "; I++}}{print" \ n "} ' Data.txt
function as above, print out the first few columns of each row, and the last few columns do not print.
5. Examples of mathematical operations and string manipulation
A Nawk ' {print 2^5+sin (2.1) +int (0.9)} ' data.txt prints arithmetic values (32.8632) on each line
b awk ' end{print Length ("How is You?")} ' data.txt print out string lengths (12)
C awk ' End{print index ("How Is It", "You")} ' Data.txt returns you start position in the string (9)
D Nawk ' {gsub (/\$/, "");p rint $} ' data.txt to replace the $ symbol for each line and print the line
E awk ' End{print substr ("Howare You", 9,3)} ' Data.txt truncate the next 3-length substring starting from the nineth position of the string
F Nawk ' {print ToUpper ($), ToLower ($)} ' data.txt print second column uppercase and lowercase respectively
G Nawk ' End{print match ("How is Youyou?",/you/), rstart,rlength} ' Data.txt
Print the position and length of the first match in the string (9,9,3)
H Nawk ' end{str= "How is you doing?"; Sub (/o/, "0", str);p rint str} ' Data.txt
Assigns a string variable to the first match containing o with the 0 replacement
Special Note: The third parameter of the sub function cannot be used directly with a string, but must be a string variable.
I awk ' End{print split ("Jan,feb,mar,apr,may", Mymonths, ","), Mymonths[2]} ' data.txt
Separates the string with the third parameter separator symbol, and places the separated elements
In the array of the second argument, the function returns the number of elements that are separated. Results:
(5,feb)
=======
Advanced Article
=======
6. arrays and associative arrays (a[1],a[$1] a[$0],a[b[i])
A awk ' {for (i=1;i<nf;i++) {a[i]= $i;p rinta[i]}} ' data.txt print each column of the file by row
b awk ' {a[$1]=$2}end{for (x in a) print a[x]} ' Data.txt array A is associated to the first column, assigns the second column value to a, and finally prints all the contents of a.
Note: But not sequential printing. The x in for is a random variable, and awk automatically determines it.
C awk ' {b[i]=$1;a[$1]=$2;i++}end{for (j=1;j<i;j++) print B[j],a[b[j]} ' data.txt
Implements the ability to print the first and second columns of each row starting from the second row.
Note: I take values starting from 0, while J starts from 1. A is an associative array, and B is a sequential array.
7. Multiple input files and examples of NR, Fnr
A awk ' begin{ofs=fs= ': '} nr==fnr{a[$1]=$2}nr>fnr{$2=a[$1];p rint} '/etc/shadow/etc/passwd
The first pattern in the statement matches to the first input file shadow, the second pattern matches to the second input file passwd, and the associated array is used to replace the other column values of the different file association columns with each other.
A. The order in which awk executes the multiple input files is that the code first acts on the first file (one line is read in), and then the duplicated code acts on the second file and then on the third file.
B. Awk has a row ordinal problem with the order in which multiple input files are executed. When the first file finishes executing, the next time you read the second file, how about the first line of the second file? If it were counted as 1, wouldn't that be two 1? (Because the first file also has the first line). This is the problem of NR and FNR.
NR: Number of global rows (the first line of the second file followed by the end of the first file count of rows)
FNR: The number of rows in the current file itself (regardless of the number of self-rows and the total number of previous input files)
For example: There are 40 lines in Data1.txt, 50 rows in Data2.txt, then awk ' {} ' data1.txt data2.txt
the values of NR are: ... 40,41,42 ... -
the values of the FNR are: ... 40,1,2 ... -
8. Redirected output and special function Getline instances
A awk ' {print filename,$0} ' data1.txt data2.txt >data_all.txt
Merge the first file and the second file into Data_all.txt, and the new file is first listed as the original file name, followed by the original content.
b awk ' $1!=fd{close (FD); fd=$1} {print substr ($0,index ($, "") +1) >$1} ' Data_all.txt
Re-data.txt the merged new file into the original two sub-files, creating a new file name according to the first column of the new file. After generating a complete sub-file, close it, and then generate the next sub-file.
　　　　 A. Getline, on the whole, should understand its usage as such:
when the left and right no redirect | or <, getline acts on the current file, reads the first line of the current file to
var or $ (no variable) followed by the variable; it should be noted that awk is handling getline
It has been read in a row before, so the return result of Getline is interlaced.
when there is a redirect | or < on its left and right, Getline is acting on the directed input file because the file is just hit
open, and is not read into a line by awk, just getline read in, then Getline returns the first line of the file, not interlaced.
B. Getline usage can be broadly divided into three broad categories (each of which is divided into two small classes), that is, a total of 6 usages. The code is as follows:
Nawk ' begin{"Cat data.txt" |getline D; Print d} ' Data2.txt
Nawk ' begin{"Cat data.txt" |getline; print $} ' data2.txt
Nawk ' Begin{getline D < "data.txt"; print d} ' Data2.txt
Nawk ' Begin{getline < "data.txt"; print $} ' data2.txt
the above four lines of code are implemented to "print only the first line of the Data.txt file" (If all lines are printed, loop)
eg. nawk ' begin{fs= ":" while (getline< "/etc/passwd" >0) {print $}} ' Data.txt
Nawk ' {getline D; print D "#" $} ' data.txt
awk first reads in the first line, then processes the Getline function, then assigns the next line to the variable d, then prints D, because D is followed by a newline character, so the following # will overwrite D, and the subsequent one will also overwrite D.
Nawk ' {getline; print $ "#" $ $} ' Data.txt
Awk first reads the first line and then processes the Getline function, then assigns the next line to $ A, and now the $ is is the next line, and the following # and $ (fetch from $) will overwrite the content of $ A.
C Nawk ' Begin{system ("echo \" input yourname:\ "); getline var; print" \nyour name is ", D," \ n "} '
You are prompted to enter a name and then print out the name you entered. Special NOTE: This awk statement does not enter a file name, but instead uses keyboard input as the file name.
=========
Classic Example
=========

1. Problem Description
Have the following Bank Bill section text,
200000000000007|shiyancityqi Pei Bu
202111320000018|hospital
200000000000007|shiyan Cityrenmen Road Qibei BU
201602520002941|middle School
200000000000007|mingfeng Roadqi PEI
201602120000113|zhuanghuangfactory
201602320000115|liangyou Factory

Requirements: Merge duplicates in the first column into one row, and the second column fills the column with the longest address
The results should be as follows:
200000000000007|shiyan Cityrenmen Road Qibei BU
202111320000018|hospital
201602520002941|middle School
201602120000113|zhuanghuangfactory
201602320000115|liangyou Factory

Code: (Assume the following code in the file Myawk. SH)
#!/usr/bin/nawk-f
Begin{fs=ofs= "|"; I=1;}
{if (a[$1]==0) {b[i]=$1;a[$1]=$2;i++}
if (length (a[$1)) <length ($)) {a[$1]=$2}
}
End{for (j=1;j<i;j++) print B[j],a[b[j]]
}
Execution: myawk.sh data.txt
Compare code: (not fully implemented)
awk ' begin{fs=ofs= ' | "}! (Length (a[$1]) >length ($)) {a[$1]=$2}
End{for (i ina) print I,a[i]} ' data.txt

Explanation: In this example, two arrays are used, a is used to associate with, and B is used for sequential recording so that it is printed exactly in sequence when it is last printed. The conditional sentence first determines whether the array element is the first write, if not,
Compares the current $ value and the previously stored value length. Functional code cannot be printed sequentially. When $ $ is duplicated, and the length of the first, second, and third time is
In a descending manner, the code is applied better. However, the code does not implement the above functions when the length is indeterminate for each time. For example, the second column of line 5th in this example is longer than the second column of row 3rd
, the function is not full code can not achieve the requirements.

2. Problem Description:
There are two files as follows:
Data1 File Contents:
1 0.5 100
10 15 36.5
Data2 File Contents:
50 10 19
3.2 1 5
Requirements: To get a new file, the content is data1 and data2 corresponding to the column numbers added and, new file content:
51 10.5 119
13.2 16 41.5

Code: (Assume the following code in the file Myawk. SH)
#!/usr/bin/nawk-f
{for (i=1;i<=nf;i++) a[i]= $i;
Getline < "Data2.txt"
for (j=1;j<nf;j++) printf $j +a[j] "\ t"
PRINTF$NF+A[NF] "\ n"
}
Execution: myawk.sh data1.txt
Explanation: The above code uses Getline to read the contents of the second file and save it to $ A, and finally to use printf to achieve the added print. Note that both the \ T and \ n are printed immediately after, to use the separation between columns and
The separation between rows. It can also be done with print, but print is wrapped every time it is executed (you can use OFS for further processing, of course).
Can actually be implemented in another way, that is, using NR, FNR comparison, and then assign two files to different arrays, and then add. Of course the code is not as concise as the code above.
3. Problem Description
There are two files as follows:
The contents of the Data2.txt file are as follows:
1111 AAA ww001$$$$1000.00
2222BBB gg001%%%%2000.00
3333CCC jj001****3000.00
4444DDD ff002&&&&4000.00
5555EEE [Email protected]@@@5000.00
666 FFF uu003jjjj6000.00
777 III ii005pppp7000.00
8888TTT tt008tttt8000.00
The contents of the Data1.txt file are as follows:
AAA001 1000.00
BBB 001 2000.00
DDD 002 4000.00
EEE 002 5000.00
FFF 003 6000.00
The 1th column, the last column, and the 3rd column and the last column of the data2 are the same data1.
The data1 2nd column and the 4th column of Data2 are the same after 3 bits.
Requirements: Data2 in the third column, the fourth column after the 3-bit, the sixth column completely and data1 the same extraction to get the new file as follows:
1111AAA ww001$$$$1000.00
2222BBB gg001%%%%2000.00
4444DDD ff002&&&&4000.00
5555EEE [Email protected]@@@5000.00
666 FFF uu003jjjj6000.00
Code:
Nawk ' Nr<=fnr {a[$1]=$1 "x" $ "x" $}
NR&GT;FNR {b=substr ($4,3);
C=$3 "x" B "x" $6;
if (c==a[$3]) print} ' Data1.txt Data2.txt
Execute: Execute the above code directly
Explanation: This example skillfully utilizes a combination of strings (in combination with X) and is then used to compare.
When NR&LT;=FNR is processed, the first file is processed when the second file is NR&GT;FNR.