The awk of Linux

Source: Internet
Author: User
Tags uuid

a awk self-introduction
awk, one of the Three Musketeers of text processing, has its name from the first letter of its founder Alfred Aho, Peter Weinberger and Brian Kernighan's surname, which is not only one of the most powerful data processing engines available in any environment in Linux. As a powerful language, it has almost all the beautiful features that a complete language should have.

Basic format: awk [options] ' program ' File ...

program:pattern{action statements;..}

Pattern and action:

  pattern部分决定动作语句何时触发及触发事件      BEGIN,END action statements对数据进行处理,放在{}内指明      print, printf

Two awk syntax

The first step: Execute Begin{action ...} Statements in a statement block

Step two: Read a line from the file or standard input (stdin), then execute the pattern{action ...} Statement block, which scans the file row by line, repeating the process from the first line to the last line until the file is fully read.

Step three: When reading to the end of the input stream, execute end{action ...} Statement block

          BEGIN语句块在awk开始从输入流中读取行之前被执行,这是一个 可选的语句块,比如变量初始化、打印输出表格的表头等语句通常 可以写在BEGIN语句块中          END语句块在awk从输入流中读取完所有的行之后即被执行,比如 打印所有行的分析结果这类信息汇总都是在END语句块中完成,它 也是一个可选语句块    pattern语句块中的通用命令是最重要的部分,也是可选的。如果 没有提供pattern语句块,则默认执行{ print },即打印每一个读取 到的行,awk读取的每一行都会执行该语句块

Three awk operators

Four awk control statements
Control Statement If-else
Syntax: if (condition) {statement; ...} [Else statement]
if (condition1) {statement1}else if (condition2) {Statement2}
ELSE{STATEMENT3}
This statement uses the scenario: to make a conditional judgment on an entire row or a field obtained by awk
Example:
Awk-f: ' {if ($3>=1000) print $1,$3} '/etc/passwd (less than 1000 does not do processing, so there is no else)
awk ' begin{test=100;if (test>90) {print "very good"}else if (test>60) {print "good"}else{print "No Pass"}} '

Control statement while
Syntax: while (condition) {statement; ...}
Condition "true", enter loop; condition "false", exit loop
This statement uses a scenario where multiple fields within a row are treated one at a time, and each element in an array is processed
Example:
awk '/^[[:space:]]*linux16/{i=1;while (i<nf) {print $i, length ($i); i++}} '/etc/grub2.cfg

Control Statement Do-while
Syntax: do {statement; ...} while (condition)
True or false, at least once the loop body is executed
Example:
awk ' begin{sum=0;i=0;do{sum+=i;i++;} while (i<=100);p rint sum} '

Control statement for
Syntax: for (EXPR1;EXPR2;EXPR3) {statement; ...}
Example:
awk '/^[[:space:]]*linux16/{for (i=1;i<=nf;i++) {print $i, Length ($i)}} '/etc/grub2.cfg

Control statement break and continue
Example:
awk ' Begin{sum=0;for (i=1;i<=100;i++) {if (i==66) break;sum+=i}print sum} '
awk ' Begin{sum=0;for (i=1;i<=100;i++) {if (i%2==0) continue;sum+=i}print sum} '

awk Array
Associative array: array[index-expression]
Index-expression:
(1) You can use any string; strings are enclosed in double quotes.
(2) If an array element does not exist beforehand, when referenced, awk automatically creates this element and initializes its value to "empty string"
Example:
weekdays["Mon"]= "Monday"
awk ' begin{weekdays[' mon ']= "Monday" weekdays["Tue"]= "Tuesday";p rint weekdays["Mon"]} '

awk function
Numerical Processing:
RAND (): Returns a random number between 0 and 1
Example:
awk ' Begin{srand (); for (i=1;i<=10;i++) print int (rand ()100)} '
String processing:
Ength ([s]): Returns the length of the specified string
Sub (r,s,[t]): searches the T string for the pattern match of the R representation and replaces the first match with the S
Example:
echo "2008:08:08 08:08:08" | awk ' Sub (/:/, "-", $) '
Gsub (R,s,[t]): searches the T string for the pattern-matching contents of the R representation, and replaces all of the contents represented by S
Example:
echo "2008:08:08 08:08:08" | awk ' Gsub (/:/, "-", $) '
Split (S,array,[r]): Cut the string s with the R delimiter, and save the cut result to the array represented by array, the first index value is 1, the second index value is 2 ....
Example:
Netstat-tan | awk '/^tcp\>/{split ($5,ip, ":"); count[ip[1]]++} end{for (i in count) {print I,count[i]}} '

Five Little Exercises
1. Count the number of occurrences of each file system type in the/etc/fstab file
awk '/^uuid/{fs[$3]++}end{for (i in FS) {print I,fs[i]}} '/etc/fstab
Description
/^uuid/: Pattern matches lines beginning with UUID
fs[$3]++: Definition fs[] is an associative array subscript is the 3rd field of each record, the value of the array is the number of occurrences
for (i in FS) {print I,fs[i]}: After each record has been processed, iterate through the array with a For loop, print the subscript (file type) and array element values (number of occurrences of the file type) *

2. Count the number of occurrences of each word in the/etc/fstab file
awk ' {i=1;while (i<=nf) {word[$i]++;i++}}end{for (num in Word) {print num,word[num]}} '/etc/fstab
Description
Using the while loop to process each field of each record, the main thing is to set I to-for each field, so that each record is processed, the associative array is re-assigned to each field of the next record

3. Extract the string [all numbers in email PROTECTED]%9&BDH7DQ+YVIXP3VPW
echo "[Email PROTECTED]%9&BDH7DQ+YVIXP3VPW] | awk ' Gsub (/[^[:d igit:]]/, "", $) '
Description
Gsub (R,s,[t]): searches the T string for the pattern-matching contents of the R representation and replaces all of the content represented by S, pattern matching is extended regular expression

4, to solve the Dos attack production case: According to the number of web logs or network connections, monitoring when an IP concurrent connection number or a short time PV reached 100, that is, call the firewall command to seal off the corresponding IP, monitoring frequency every 5 minutes. The firewall command is: iptables-a input-s ip-j REJECT

awk ' {access[$1]++}end{for (i in Access) {if (access[i]>=100) {print I,access[i]}}} ' Access_log

With the brain to learn to day up, refueling!

The awk of Linux

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.