AWK, the Three Musketeers of Linux

GNU AWK: The Three Musketeers tool for text processing:

grep, SED, awk

grep, Egrep, Fgrep: Text filtering tool; pattern sed: Row editor mode space, hold space awk: Report Builder, formatted text output, and when working with text files, the words in the document  Segment conditional on execution, conditional display, Awk:aho, Weinberger, Kernighan--and New awk, Nawk GNU awk, abbreviation: gawk gawk-pattern Scanning and processing language supports conditional judgments, arrays, loops, and so awk: a scripting language interpreter basic usage: gawk [options] ' progr Am ' File ... program:pattern{action statement} statements are separated by semicolons Print,prin                TF option:-F: Indicates the field delimiter used for input-v var=value: Custom variable 1, print Print ITEM1,ITEM2 key: 1, comma delimiter 2, the output of each item can be a string, can also be a numeric value; The field of the current record, Variable or awk expression: 3, if Item is omitted, it is equivalent to print $; print whole line character 2, variable 2.1 built-in variable FS                    : Input field Seperator, default is white space character; [[email protected] ~] #awk-v fs= ":" ' {print $} '/etc/passwd OFS:Output field seperator, default to white space character, [[email protected] ~] #awk-v fs= ":"-V ofs= ":" ' {print $1,$3} ' /etc/passwd rs:input record seperator, input line break; [[email protected] ~] #awk- V rs= ' {print} '/etc/passwd ors:output record seperator, line break at output; [[email&nbsp                        ;p rotected] ~] #awk-V rs= "-v ors=" # "' {print $1,$3} '/etc/passwd nf:number of field, number of fields                        {print Nf},{print $NF} [[email protected] ~] #awk ' {print NF} '/etc/passwd } Nr:number of record, rows [[email protected] ~] #awk ' {print NR} '/etc/fstab FNR: Each file counts; line number; [[email protected] ~] #awk ' {print FNR} '/E Tc/fstab/etc/issue FileName: Current file name; [[email protected] ~] #awk ' {print filen AME} '/etc/fstab/etc/isSue ARGC: Number of command line arguments [[email protected] ~] #awk ' {print ARGC} '/etc/fstab/etc/ Issue ARGV: An array that holds the parameters given by the command line; [[email protected] ~] #awk ' Begin{print argv[0 ]} '/etc/fstab/etc/issue awk [[email protected] ~] #awk ' Begin{print AR GV[1]} '/etc/fstab/etc/issue/etc/fstab [[email protected] ~] #awk ' BEG In{print argv[2]} '/etc/fstab/etc/issue/etc/issue 2.2 custom variable variable name Character case sensitivity, 1,-v Var=value [[email protected] ~] #awk-v test= ' Hello gawk ' {PR                     int test} '/etc/fstab [[email protected] ~] #awk-v test= ' Hello gawk ' begin{print test} '  2, directly defined in program [[email protected] ~] #awk ' begin{test= "Hello gawk"; print Test} ' 3, printf command format output; printf format, item1, item2,.....                        1, format must be given; 2, does not wrap, need to display the line feed control, \ n 3, format needs to specify a formatting symbol for each subsequent item;                                Format character:%c: Displays the ASCII code of the characters;%d,%i: Displays decimal integers;                            [[email protected] ~] #awk-F: ' {printf ' Username:%s, UID:%d\n ", $1,$3} '/etc/passwd                            %e,%e: Numerical display of scientific counting method;%f; display as floating-point number%g,%g; display values in scientific notation or floating-point form; %s; display string; [[email protected] ~] #awk-F: ' {printf ' Username:%s                             \ n ", $ '/etc/passwd%u; unsigned integer; Percent: shows% itself; modifier:                                #[.#]: The width of the first digital control display; The second # indicates the precision after the decimal point;%3.1f [[email protected] ~] #awk-F: ' {printf ' USername:%-15s, UID:%d\n ", $1,$3} '/etc/passwd-: Left justified [[Email&nbs                P;protected] ~] #awk-F: ' {printf ' Username:%-15s, UID:%d\n ", $1,$3} '/etc/passwd +: Symbols showing values                        4, operator arithmetic operator; x+y, X-y, x*y, x/y, X^y, x%y-x                        +x: Convert to numeric value; string operator; unsigned operator, string join assignment operator; =,+=,-=,*=,/=,%=,^= ++,--, comparison operator: > >= < < = = = = Pattern match; ~ matches whether the!~ does not match the logic                        Operator: && | |                    ！                    function call; Function_name (ARGU1, ARGU2, ...) conditional expression; selector?if-true-expression; If-false-expression                        [[email protected] ~] #awk-F: ' {$3>=1000?usertype= ' Common User ': usertpye= ' Sysadmin or Sys                    User ";p rintf"%15s:%-s\n ", $1,usertype} '/etc/passwd 5, pattern 1, empty; null mode, matching each row; 2./regular expression/: Only rows that are capable of being matched to by a pattern here are taken only by the line beginning with the UUID [[email&nbsp                        ;p rotected] ~] #awk '/^uuid/{print $ '/etc/fstab uuid=3d4ae911-238f-4436-82eb-5bb4660c38c7                     The line that starts with the UUID is reversed [[[email protected] ~] #awk '!/^uuid/{print $} '/etc/fstab 3, relational expression: the expression of the relationship, the result is "true" has "false", the result is true will be processed true: The result is not 0 value, non-empty  String: empty string representation is false; [[email protected] ~] #awk-F: ' $3>=1000{print $1,$3} ' /etc/passwd nfsnobody 65534 It [[Email pro Tected] ~] #awk-F: ' $3&LT;=1000{print $1,$3} '/etc/passwd [[email protected] ~] #awk-F: ' $NF ~/bash$/{print $ $NF} '/E TC/PASSWD Root/bin/bash It/bin/bash 4, line ranges address delimitation: Line range: startline,endline/pat1/,/pat2/[[Email pro                        Tected] ~] #awk-F: '/^rpcuser/,/^sshd/{print $ '/etc/passwd note; The direct given number format is not supported; [[email protected] ~] #awk-F: ' (nr>=2&&nr<=10) {print '} '/etc/passwd 5, Begin/end Mode begin{}: Executes only once before the text in the file is started; [[email protected] ~] #awk-f: ' Begin {print "username uid \ n-------------------------------\ n"} {print $1,$3} '/etc/passwd end{}: Executes only once after the completion of the text processing; [[Email prot Ected] ~] #awk-F: ' begin{print ' username UID \ n-------------------------------\ n "}{print $1,$3}end{print" =============================\n END "} '/etc/passwd                        6, the commonly used action 1, experssions expression: 2, Control statements controls statement; If, while etc. 3, Compound statements; 4, INPUT statements Enter statement 5, OUTP UT statements Output Statement 7, control statement if (condition) {statements} if ( Condition) {Statements} else {statements} while (condition) {statements} does {                        Statements} while (condition) for (EXPR1;EXPR2;EXPR3) {statements} break                        Continue delete Array[index] Delete array Exit {Statements} 7.1 If-else syntax: if (con dition) statements [else statements] [[email protected] ~] #awk-F: ' {if ($3>=1000) print $1,$3} '/etc/passwd Nfsnobody 65534 Luo [[E Mail protected] ~]# [[email protected] ~] #awk-F: ' {if ($3>=1000) {printf ' Comm On User:%s\n ", $ ' else {printf" root or Sysuser:%s\n ", $ {}} '/etc/passwd usage scenario: Make an entire row or a field from awk                                [[email protected] ~] #awk-F: ' {if ($NF = = "/bin/bash") print $ '/etc/passwd                                [[email protected] ~] #awk ' {if (nf>5) print $} '/etc/passwd [[email protected] ~] #df-H | Awk-f "[%]" '/^\/dev/{print $} ' | awk ' {if ($NF >=15) print ' 7.2 while loop syntax: while (condition) stateme                  NTS condition "true", enter loop; condition "false", exit loop          Use a scene, use one of several fields in a row in a similar process, and use each element of an array in a single process; length () for each word section statistics alone [[email protected] ~] #awk '/^[[:space:]]*linux16/{i=1;while (i<=nf) {print $i , Length ($i); i++} '/etc/grub2.cfg each field is counted separately and the number of characters to be >=7 given [ [email protected] ~] #awk '/^[[:space:]]*linux16/{i=1;while (I&LT;=NF) {if (length ($i) >=7) {print $i, length ($i                                )};i++}} '/etc/grub2.cfg 7.3 do-while loop Syntax:                            Do statements while (condition) meaning: perform at least one loop body 7.4 for loop Syntax: for (EXPR1;EXPR2;EXPR3) statements for (variable assignment; condition; i Teration process) {for-body} [[email protected] ~] #awk '/^[[:space:]]*linux16/{for (i=1 ; i<=nf;i++) {print $i, length($i)}} '                                    /ETC/GRUB2.CFG Special Usage: Ability to traverse elements in an array; Syntax: for (var in array) {FOR-BODY} 7.5 switch statement syntax: switch (expression                        {Case VALUE1 or/regexp/: statements, case VALUE2 or/regexp2/: statements; ...; default:statements}                        7.6 Break and continue break[n] Continue                            7.7 Next to end the processing of the bank and go directly to the next line; print lines with an even ID number                        [[email protected] ~] #awk-F: ' {if ($3%2!=0) next;print $1,$3} '/etc/passwd 8, array of arrays Associative array: array[index-expression] index-expression:1, can use any                                Meaning string, the string to use double quotation marks 2, if an array element does not exist beforehand, when referenced, awk automatically creates this element, and initializes its value to "empty string" To determine if an element exists in the array,Using the "index in array" format; weekdays [mon] = "Moday" [[E                                    Mail protected] ~] #awk ' begin{weekdays["Mon"]= "Monday" weekdays["Tue"]= "Tuesday";p rint weekdays["Tue"]} '  Tuesday to iterate through each element in the array, use the For loop for (Var in Array) {for-body} [[email protected] ~] #awk ' begin{weekdays["Mon"]= "Monday"; weekdays[                                    "Tue"]= "Tuesday"; for (I in weekdays) {print Weekdays[i]}} ' NOTE: var iterates through each index of the array;                                state["LISTEN"]++ state["established"]++ Count occurrences [[email protected] ~] #netstat-tan |                                    awk '/^tcp\>/{state[$NF]++}end{for (i in state) {print I,state[i]}} ' count the occurrences of each IP [[email protected] ~]#awk ' {ip[$1]++}end{for (i in IP) {print i,ip[i]}} '/var/log/httpd/access_log Exercise: Statistics/etc/fsta                                The number of file system occurrences in the B file awk '/^uuid/{fs[$3]++}end{for (i in FS) {print I,fs[i]}} '/etc/fstab Exercise: Count the number of occurrences of each word in the specified file; [[email protected] ~] #awk ' {for (i                            =1;i<=nf;i++) {count[$i]++}}end{for (i in count) {print I,count[i]}} '/etc/passwd 9, function                                9.1 Built-in function numeric processing: rand (): Returns a random number between 0 and 1                                    String processing: Length ([s]): Returns the lengths of the specified string; Sub (r,s,[t]): Finds the matched contents of the character represented by T in the pattern represented by R and replaces it with the content represented by s in the first occurrence; Gsub (R,s,[t]): a pattern in R                           Find the matching content in the character represented by T, and replace all occurrences with the content represented by S; split (S,a,[,r]): Cut the song character s with the R delimiter and save the result of the tangent songs to the array represented by A;             [[email protected] ~] #netstat-tan| awk '/^tcp\>/{split ($5,ip, ":");p rint ip[1]} ' [[email protected] ~] #netstat -tan| awk '/^tcp\>/{split ($5,ip, ":"); Count[ip[1]]++}end{for (i in count) {print I,count[i]}} '

Awk of the Three Musketeers of Linux