Linux Regular Expression awk explanation

Linux Regular Expression awk explanation

Like sed, awk is a streaming editor. It also operates on the rows in the document and runs on one row. Awk is more powerful than sed. It can achieve what sed can do, and it can also do what sed cannot do. Awk is often used for segmentation;

Awk can be implemented without adding any parameters +? *. | These special symbols;

1. intercept a segment in the document
[Root @ bkjia ~] # Head-n2 passwd | awk-F: '{print $1 }'
Root
Bin
[Root @ bkjia ~] # Head-n2 passwd | awk-F: '{print $0 }'
Root: x: 0: 0: root:/bin/bash
Bin: x: 1: 1: bin:/sbin/nologin
[Root @ bkjia ~] # Head-n2 passwd | awk-F: '{print $1, $3, $7 }'
Root 0/bin/bash
Bin 1/sbin/nologin

The-F option specifies the delimiter. If-F is not added, the Delimiter is a space or tab. Print is a Print action to Print a field. $1 is the first field, $2 is the second field, and so on. A special one is $0, which indicates the entire line.

Multiple Fields $1, $3, and $7 can be printed in {}, separated by commas;

The default delimiter for printing segments is space. You can define custom separators, which must be enclosed by colons. You can also define output separators in OFS;
[Root @ localhost ~] # Awk-F: '{print $3, $4}' 1.txt | head-5
0 0
1 1
2 2
3 4
4 7
[Root @ localhost ~] # Awk-F: '{print $3 ":" $4}' 1.txt | head-5
0: 0
1:1
2: 2
3:4
4: 7
[Root @ localhost ~] # Awk-F: 'ofs = "#" {print $3, $4} '1.txt | head-5
0 #0
1 #1
2 #2
3 #4
4 #7

[Root @ bkjia ~] # Head-n2 passwd | awk-F: '{print $1 "#" "@" $3 "#" $7 }'
Root # @ 0 #/bin/bash
Bin # @ 1 #/sbin/nologin

Pay attention to the awk format.-F is followed by single quotes and is then used as a separator. The print action must be enclosed by {}. Otherwise, an error is reported. Print can also print custom content, but the custom content should be enclosed in double quotation marks.

2. Match characters or strings
[Root @ bkjia ~] # Awk-F: '$1 ~ /Me/'passwd
Games: x: 12: 100: games:/usr/games:/sbin/nologin
[Root @ bkjia ~] # Awk-F: '$1 ~ /User/'passwd
User1: x: 600: 501:/home/user1:/bin/bash

A specific segment can be matched ,~ Indicates the meaning of the match. The first field is separated by a colon and then the keyword in // is matched;

[Root @ bkjia ~] # Awk-F: '/root/{print $1, $3}/user/{print $1, $3}' passwd
Root 0
Operator 11
Ftp 14
Saslauth 499
User1 600

The awk can also be matched multiple times. In the above example, the full text matches the row containing the root keyword, then matches the row containing the user, and prints the matched 1st and 3 segments.

3. Conditional Operators
Determine if 3rd fields are 0
[Root @ bkjia ~] # Awk-F: '$3 = "0"' passwd
Root: x: 0: 0: root:/bin/bash
[Root @ bkjia ~] # Awk-F: '$3 = 10' passwd
Uucp: x: 10: 14: uucp:/var/spool/uucp:/sbin/nologin

Determine if 3rd fields are 10 and print the 7th fields of the row;

[Root @ bkjia ~] # Awk-F: '$3 ==10 {print $7}' passwd
/Sbin/nologin
[Root @ bkjia ~] # Awk-F: '$3 = "600" 'passwd
User1: x: 600: 501:/home/user1:/bin/bash

 

In awk, it can be determined by logical symbols. For example, '=' is equal to or can be understood as 'exact Match'. In addition, there are >,'> =, '<, '<= ,'! = And so on. It is worth noting that when comparing with a number, if we use double quotation marks to compare the number, awk will not regard it as a number but a character, it is considered a number without double quotation marks.

 

For example, double quotation marks are considered as characters, while single quotation marks and no quotation marks are considered as numbers;
[Root @ bkjia ~] # Awk-F: '$3> "500" 'passwd | sort-t:-k 3-n
Shutdown: x: 6: 0: shutdown:/sbin/shutdown
Halt: x: 7: 0: halt:/sbin/halt
Mail: x: 8: 12: mail:/var/spool/mail:/sbin/nologin
Vcsa: x: 69: 69: virtual console memory owner:/dev:/sbin/nologin
Sshd: x: 74: 74: privilege-separated ssh:/var/empty/sshd:/sbin/nologin
Messages: x: 81: 81: system message bus: // sbin/nologin
Postfix: x: 89: 89:/var/spool/postfix:/sbin/nologin
Nobody: x: 99: 99: nobody: // sbin/nologin
User1: x: 600: 501:/home/user1:/bin/bash

[Root @ bkjia ~] # Awk-F: '$3> 500' passwd | sort-t:-k 3-n
User1: x: 600: 501:/home/user1:/bin/bash
[Root @ bkjia ~] # Awk-F: '$3> '000000' passwd | sort-t:-k 3-n
User1: x: 600: 501:/home/user1:/bin/bash

 

! = Is unmatched. The 7th field is not equal to/sbin/nologin rows and must be enclosed in double quotation marks.
[Root @ bkjia ~] # Awk-F: '$7! = "/Sbin/nologin" 'passwd
Root: x: 0: 0: root:/bin/bash
Sync: x: 5: 0: sync:/sbin:/bin/sync
Shutdown: x: 6: 0: shutdown:/sbin/shutdown
Halt: x: 7: 0: halt:/sbin/halt
User1: x: 600: 501:/home/user1:/bin/bash
Mysql: x: 27: 27: MySQL Server:/var/lib/mysql:/bin/bash

 

In addition to logical comparison of characters in a segment, logical comparison can be performed between two segments.

[Root @ bkjia ~] # Awk-F: '$3> "5" & $3 <"7" 'passwd
Shutdown: x: 6: 0: shutdown:/sbin/shutdown
Vcsa: x: 69: 69: virtual console memory owner:/dev:/sbin/nologin
User1: x: 600: 501:/home/user1:/bin/bash

You can also use & "and" and | "or.
For example, print the rows whose segment 3rd is greater than segment 4th and segment 7th is/bin/bash;
[Root @ bkjia ~] # Awk-F: '$3 >4 4 & $7 = "/bin/bash" 'passwd
User1: x: 600: 501:/home/user1:/bin/bash

For example, print the rows whose segment 3rd is less than 4th or whose segment 7th is/bin/bash;
[Root @ bkjia ~] # Awk-F: '$3 <$4 | $7 = "/bin/bash"' passwd
Root: x: 0: 0: root:/bin/bash
Adm: x: 3: 4: adm:/var/adm:/sbin/nologin
Lp: x: 4: 7: lp:/var/spool/lpd:/sbin/nologin
Mail: x: 8: 12: mail:/var/spool/mail:/sbin/nologin
Uucp: x: 10: 14: uucp:/var/spool/uucp:/sbin/nologin
Games: x: 12: 100: games:/usr/games:/sbin/nologin
Gopher: x: 13: 30: gopher:/var/gopher:/sbin/nologin
Ftp: x: 14: 50: ftp user:/var/ftp:/sbin/nologin
User1: x: 600: 501:/home/user1:/bin/bash
Mysql: x: 27: 27: mysql server:/var/lib/mysql:/bin/bash

4. built-in variables of awk

Common variables of awk include:
NF: How many segments are separated by delimiters
NR: number of rows
{Print NR ":" NF} lists the row numbers separated by colons to list the total number of segments;

[Root @ bkjia ~] # Head-5 passwd | awk-F: '{print NR ":" NF }'
1: 7
2: 7
3: 7
4: 7
5: 7
[Root @ bkjia ~] # Head-5 passwd | awk-F: '{print NF }'
7
7
7
7
7
[Root @ bkjia ~] # Head-5 passwd | awk-F: '{print NR }'

[Root @ bkjia ~] # Head-5 passwd | awk-F: '{print $ NF }'
/Bin/bash
/Sbin/nologin
/Sbin/nologin
/Sbin/nologin
/Sbin/nologin

NF indicates the number of segments, while $ NF indicates the value of the last segment, while NR indicates the row number. 5. awk mathematical operations awk can also perform mathematical operations on the values of each segment: 12 [root @ bkjia ~] # Awk-F: '{(tot = tot + $3)}; END {print tot}; 'passwd
1720

Here, the END should be noted that all rows have been executed. This is a special Syntax of awk. In fact, awk and sed can both be written into a script file and have their own syntax, in awk, if and for loops are acceptable.

Example: if, for example, if the value of the first segment is root, print the entire line;
[Root @ bkjia ~] # Awk-F: '{if ($1 = "root") print $0}' passwd
Root: x: 0: 0: root:/bin/bash

Example: for Loop, defines the sum variable, and the I value is the value of the 3rd segment; calculates the sum of the 3rd segment;

[Root @ bkjia ~] # Sum = 0; for I in 'awk-F: '{print $3} 'passwd'; do sum = $ [($ sum + $ I)]; done; echo $ sum
1720

-------------------------------------- Split line --------------------------------------

Introduction and use of AWK

AWK introduction and Examples

Shell script-AWK text editor syntax

Learning and using AWK in Regular Expressions

AWK diagram of Text Data Processing

How to Use the awk command in Linux

Text Analysis Tool-awk

-------------------------------------- Split line --------------------------------------

This article permanently updates the link address: