Linux Operations Phase II (15) awk

Linux Operations Phase II (15)awk

grep(global researchexpression, file filter, displays the matching lines according to the pattern,#grep [options] pattern FILE, using the method described in <shell Basic >)

Sed(stream Editor, flow editors, read each line to the memory space (mode space), default does not edit the source file, only the pattern space data processing, processing the end of the pattern space to print to the screen,#sed [options] Addresscommand ' File1 ... )

Awk(the first letter of the three names a.w.k, gawk(awk implemented in GNU awk,linux),Nawk( new awk), Report Generator, extract eligible information in a text file, and display it in a specific format

#awk options ' SCRIPTS ' FILE1 FILE2

#awk options ' pattern{action} ' FILE1 FILE2(processing of rows matched to data according to pattern, general processing is printing)

Note: Each time a row is extracted from a file according to the pattern, the line is sliced by the specified delimiter, and if no delimiter is specified, the default is to slice by whitespace character (space), specifying the delimiter to use -fseperater, such as the text content: This isa test , is cut into four fragments, and each fragment can be referenced within awk using a variable similar to the positional parameter variable in scripting,which is represented by the ,$ A says it's all (cut and cut) .

Example:

#awk-F: ' {print $1,$3} '/etc/passwd( separated by commas between the $ and $ entries)

#awk ' begin{fs= ': '}{print $1,$3} '/etc/passwd

#cut-D:-f1,3/etc/passwd

Printing: Print;printf

Print item1,item2,...

Attention:

each item is separated by commas, and the content referenced by the default entries is separated by whitespace characters when output;

the output item(can be a string, a value, a field of the current record , $1,$2, and so on,awk expression (if the value is first converted to a string before output));

Item If omitted is the output of the entire content, such as print $;

If you want to output blank lines, use print "";

printf "FORMAT", item1,item2

Attention:

Unlike print , the format is specified when printf outputs;

Format is enclosed in double quotation marks, which is used to specify the formatting of the subsequent item output without separating the format from the format;

printf does not automatically print line breaks;

the indicator for FORMAT begins with% , followed by a character, with:%c(the ASCII code that displays the characters ),%d and %i(decimal integer),%e and %E(scientific notation displays values),%f(floating point),%g and %g(scientific notation or floating-point format display values),%s(string), %u(unsigned integer),percent(show % itself)

modifiers are:-(for left-aligned),+(displays numeric symbols),NUM(indicates numbers, display width)

Example:

#awk-F: ' {printf '%-10s%-10s\n ', $1,$3} '/etc/passwd

awk Variable:

Built-in variables (record variables):

FS(field separator, the fields delimiter used when reading a text file, the default whitespace character, which is the input delimiter);

RS(record separator, the newline character used when entering text information, which is the input delimiter);

OFS(output FS);

ORS(output RS);

Built-in variables (data variables):

NR(the total number Ofinput records seen so far, the number of rows processed, if more than one file, is the row count of multiple files that have been processed, as long as all the records that have been processed are absolute counts, such as #awk ' {print NR} '/etc/passwd/etc/shadow);

FNR(the input recordnumber in the current input file, which is the number of rows that are currently processed in this file, is the numberof row of total rows processed in this document, and is the respective count of different files, for example #awk ' {print FNR} '/etc/passwd/etc/shadow);

NF(the number of Fieldsin the current input record, total fields, e.g. #awk-f: '/root/{print NF} '/etc/passwd )

Note: the difference between NF and $NF, such as #awk-F: ' {print $NF} '/etc/passwd(NF is the total number of fields,$NF is the last field)

Custom variables:

#awk-v test= "Hello World" ' begin{print Test} '(Way one)

#awk ' begin{test= ' Hello World ';p rint test} '(mode two, note that there are multiple statements in action that are separated by semicolons, and the value of the variable is not added when you reference it in awk) )

awk Operator:

arithmetic operator (+,-,*,/,%,^)

assignment operator (=,+ =,-=,*=,/=,%=,^=,+ + ) ,--,**=,| )

comparison operator (<,<=,>,>=,= =,! =,~,!~ , ELEMENT in array)

such as:x~y(x is a string,y is a pattern,x can be matched by Y to True, otherwise false)

X!~y(x cannot be matched by Y to True, otherwise false)

Element in array(the element in the data set is true, otherwise false)

the logical relationship between expressions (&&,| | )

conditional expression (selector?if-true-exp:if-false-exp, equivalent to if condition judgment)

awk 's Pattern:

Regular Expressions REGEXP, formatted as:/regular express/

Expression Expressions

Match range ranges, separated by commas, such as pattern1,pattern2

BEGIN(executed before the first line executes)

End(executes only after the last line is executed)

Empty mode (processed on each line of the file)

Example:

#awk-F: '/^r/{print $ '/etc/passwd

#awk-F: ' $3==0,$7~ ' Nologin ' {print $1,$3,$7} '/etc/passwd

#awk-F: ' $3>=500{print $1,$3} '/etc/passwd

#awk-F: ' $7~ ' bash$ ' {print $1,$7}/etc/passwd

#awk-F: ' $7!~ ' bash$ ' {print $1,$7}/etc/passwd

#awk-F: ' Begin{print ' Username ID Shell "}{print $1,$3,$7}end{print" END of File} '/etc/passwd

awk Action ( in addition to printing print,printf ):

Expression(Assignment expression, judging expression)

Control Statement( You can use statements such as If,for,while,case, and you can also define functions)

Compound Statement(compound statement)

awk 's control statement controlstatement:

If-else Syntax:if (condition) {Then-body} else {else-body}(where the curly braces can be saved when only one statement is Then-body)

While syntax:while (condition) {statement1,statement2,...} ( curly braces can be saved if statement only one statement)

Do-while Syntax:do{statement1,statement2,...} while {condition}

For syntax one:for (variableassignment;condition;interation process) {statement1,statement2,...}

For syntax two:for (variable in array) {Statement1,statement2,...} (applied in an array)

Case Syntax:switch (expression) {case VALUE Or/regexp/:statement1,statement2,... default:statement1, Statement2,...}

Break,continue

Next(end the processing of the bank's text in advance and proceed to the next line)

Example:

#awk-F: ' {if ($1== "root") print $, "Admin", else print $, "Common User"} '/etc/passwd

#awk-F:-V sum=0 ' {if ($3>=500) sum++}end{print sum} '/etc/passwd

#awk-F: ' {I=1;while (I<=NF) {if (length ($i) >=4) {print $i};i++}} '/etc/passwd(length is awk Built-in functions)

#awk-F: ' {for (i=1;i<=3;i++) print $i} '/etc/passwd

#awk-F: ' {shell[$NF]++}end{for (c inshell) {print c,shell[c]}} '/etc/passwd (counts the number of shells in thecurrent system , print outside of the curly braces can be saved)

#netstat-tan | awk '/^tcp/{state[$NF]++}end{for (CIN State) print C,state[c]} '(statistics on the different states of TCP, IMPORTANT!!!) Common)

#awk ' {count[$1]++}end{for (IP in count) print C,count[ip]} '/var/log/httpd/access_log(number of single-client IP connections in the statistics log )

Note:the drop-down in the awk array starts at 1 (while the array subscript in Bash starts at 0 ), and the awk array can be labeled as not a number, and any string can be customized , such as shell[/bin/bash]++,shell[/sbin/nologin]++,state[listen]++,state[in the above examples established]++,count[192.168.1.59]++,count[192.168.1.222]++

These are the notes that are taken in the course of Ma Yun-GE.

This article is from the "Linux operation and maintenance of difficult learning notes" blog, declined reprint!

Linux Operations Phase II (15) awk