Linux_note Command Grep,sed,awk

1. grep filters out the specified line

grep [-CINVABC] ' word ' filename

--color the match to the key words in red logo such as: # grep--color ' root '/etc/passwd

-C: Print the number of lines that meet the requirements

-I: Ignore case

-N: Output with the same number of lines as required

-V: Print lines that do not meet the requirements # cg-v ' root ' 1.txt

-A: followed by a number (with or without a space), for example, Calculator 2 means that the line that meets the requirements is printed and the following two lines

# cg-a 2-n ' root ' 1.txt

-B: followed by a number, for example, 2 to print the line that meets the requirements and the above two lines # cg-b 2-n ' root ' 1.txt

-C: followed by a number, for example, C 2 means that the printed line meets the requirements and the next two lines # cg-c 2-n ' root ' 1.txt

-R: All files under directory are traversed # cg-r ' iptables '/etc/* or # cg-rh ' iptables '/etc/*

[[email protected] ~]# alias cg= ' grep--color '

[[email protected] ~]# cg-n ' root ' 1.txt

1:root:x:0:0:root:/root:/bin/bash

11:operator:x:11:0:operator:/root:/sbin/nologin

[[email protected] ~]# cg-c ' root ' 1.txt

2

[[email protected] ~]# cg-a 2-n ' root ' 1.txt

1:root:x:0:0:root:/root:/bin/bash

grep sed is a matching rule that supports {1,3}, which means repeating the preceding characters 1 to 3 times. But awk belongs to a more complicated scripting language.

Words, in it () and {} have special meanings, so we want to use these symbols, we need to give them out of righteousness, you may think of using \ Off

Meaning, but it doesn't work. One way is to add the--posix option to OK, such as

awk--posix-f ': ' $ ~/(AB) +|o{1,3}/' 1.txt

So we can match the B option.

2-bin:x:1:1:bin:/bin:/sbin/nologin

3-daemon:x:2:2:daemon:/sbin:/sbin/nologin

--

11:operator:x:11:0:operator:/root:/sbin/nologin

12-games:x:12:100:games:/usr/games:/sbin/nologin

13-gopher:x:13:30:gopher:/var/gopher:/sbin/nologin

[Email protected] ~]# cg-b 2-n ' games ' 1.txt

10-uucp:x:10:14:uucp:/var/spool/uucp:/sbin/nologin

11-operator:x:11:0:operator:/root:/sbin/nologin

12:games:x:12:100:games:/usr/games:/sbin/nologinli

Example:

Filter out a line with a keyword and lose a travel number

# cg-n ' aming ' 1.txt or grep-n--color ' aming ' 1.txt cg= ' grep--color '

Filter out a line with a keyword and lose a travel number

# cg-v-n ' nologin ' 1.txt

Filter out all rows that contain numbers

#cg-n ' [0-9] ' 1.txt

Filter out a line containing a letter and lose the travel number

# cg-n ' [an] ' 1.txt

#cg-N-i ' [an] ' 1.txt

Filter out lines that don't contain letters and lose travel numbers

[[Email Protected]klinux ~]# grep--color-n-V ' [a-za-z] ' 1.txt

22:$%$#%@^&&* () ^%#[email protected]#%^&* (*&^%$#@!% ( (*&^%$#@@***&^%

23:

33:3.,498,838,598,340,25e,+43

Remove lines that begin with a letter

[Email protected] ~]# cg-n-V ' ^[a-za-z] ' 1.txt

22:$%$#%@^&&* () ^%#[email protected]#%^&* (*&^%$#@!% ( (*&^%$#@@***&^%

23:

33:3.,498,838,598,340,25e,+43

Filter out lines that don't start with a number

#cg-n ' ^[^0-9] ' 1.txt

Match a row that does not contain a number

#cg-n ' [^0-9] ' 1.txt

Remove empty lines

[Email protected] ~]# grep-v ' ^$ ' 1.txt

Filter any one or more characters

#cg ' R.O ' 1.txt

# CG ' R*o ' 1.txt

#cg ' R*o ' 1.txt matches the character at the end of R start O, what is not important in the middle, also called greedy match.

. Represents any one character

* Represents 0 or more preceding characters

. * Denotes 0 or more arbitrary characters, and a blank line is included

grep ' R\?o ' passwd represents a match for 0 or one character preceded by "R".

grep ' R.*o ' passwd means R begins with O end, middle 0 or more arbitrary characters

grep ' R*o ' passwd matches 0 or more of the preceding characters. The preceding character is "R", which can be 0 R or more R

grep ' ^[1-9][0-9]*$ ' 1. txt: Match begins with 1 to 9 one digit, ending with 0 or more digits

Filter out lines that start with a special symbol

[Email protected] ~]# cg-n ' ^[^0-9a-za-z] ' 1.txt

22:$%$#%@^&&* () ^%#[email protected]#%^&* (*&^%$#@!% ( (*&^%$#@@***&^%

Egrep ==grep-e egrep The expanded form of grep egrep the ability to use grep can all be used

grep--color ' r\?o ' 1.txt==grep--color-e ' r?o ' 1.txt==egrep--color ' r?o ' 1.txt

Match 1 and 1 + preceding characters

# egrep--color ' R+o ' 1.txt

Summarize:. any one character; * 0 or more * preceding characters;. * any of the characters;? 0 or 1? The preceding character; + 1 or more

A + preceding character.

Match Root or Nologin

#egrep--color ' Root|nologin ' 1.txt

Match Root and Nologin

# grep--color ' root ' 1.txt |grep--color ' Nologin '

Use parentheses to denote a whole

#egrep--color ' (RR) ' 1.txt

Match 1 or more RRs

#egrep--color ' (RR) + ' 1.txt

Specify the number of matched character count ranges

# egrep--color ' (RR) {1,3} 1.txt

? + () {} | Use Egrep to sign the symbol.

2. Sed

Print the specified line

# sed-n ' P 1.txt

Prints the specified range, such as 20 lines to the end

# sed-n ' 20,$ ' P 1.txt

To print a line containing a string

# sed-n '/root/' P 1.txt

You can use ^.　* $ +? and other special symbols

# sed-n '/r.o/' P 1.txt

# sed-n '/r*o/' P 1.txt

# sed-n '/R. *o/' P 1.txt

# sed-n '/r\?o/' P 1.txt

# sed-n '/r\+o/' P 1.txt

# sed-n '/root\|nologin/' P 1.txt

# sed-n '/\ (oo\) \+/' P 1.txt

-R no need to de-symbol

# Sed-n-R '/(OO) +/' P 1.txt

# Sed-n-R '/[^0-9]/' P 1.txt

# Sed-n-R '/[^a-za-z]/' P 1.txt

Blank Line

# Sed-n-R '/^$/' P 1.txt

Delete empty lines

# sed '/^$/' d 1.txt

Delete rows that contain numbers

# sed '/[0-9]/' d 1.txt

Delete lines that contain letters

# sed '/[a-za-z]/' d 1.txt

Deletes the specified row, such as deleting 1-19 rows

# sed '/1,19/' d 1.txt

The above deletion does not directly delete the contents of the file but does not appear on the monitor. The sed-i option will delete the contents of the file

[Email protected] ~]# wc-l 2.txt

2.txt

[[email protected] ~]# sed-i ' 1,19 ' d 2.txt

[Email protected] ~]# wc-l 2.txt

2.txt

Replace function

# sed ' 1,10s/nologin/login/g ' 1.txt

With special symbols to be taken off the meaning of: # sed ' 1,10s/\/sbin\/nologin/login/g ' 2.txt can also be replaced with

# sed ' 1,[email protected]/sbin/[email protected]@g ' 2.txt

Replace an entire line

# sed ' 1,5s#^.*$ #login #g ' 2.txt

Add a string to the tail, such as login

# sed ' 1,5s#^.*$#& login#g ' 2.txt

Remove numbers from each row

# sed ' s#[0-9]# #g ' 2.txt

Remove characters other than alpha-numeric

# sed ' s#[^0-9a-za-z]# #g ' 2.txt

Swap the first and last paragraphs of each line

# sed-r ' s#^ ([a-z]+) (:. *:) (. *$) #\3\2\1#g ' 2.txt

# sed-r ' s/([^:]+) (:. *:) ([^:]+)/\3\2\1/'/etc/passwd

Understanding: Sed-r ' s/([^:]+) (:. *:) ([^:]+)/\3\2\1/'/etc/passwd//match any character in the first paragraph except ":" matches the second paragraph

": Any character:" The third paragraph is also the ":" of any character \3\2\1 equivalent to the first paragraph and the third paragraph swapped

The first character is exchanged with the last character

Sed-r ' s/^ (.) (.*) (.) $/\3\2\1/'

[Email protected] ~]# echo "abcdef" |sed-r ' s/^ (.) (.*) (.) $/\3\2\1/'

Fbcdea

Multiple tasks at the same time;-E

# sed-n '/root/p;/aming/' P 1.txt or # sed-n-E '/root/p '-e '/aming/p ' 1.txt

If you encounter a line with both root and aming will print two lines, different from |,| or only one line, the meaning is to match to print.

[[email protected] ~]# sed-n-R '/root|aming/' P 1.txt

Root:aming X:0:0:root:/root:/bin/bash

Operatorrrrro:x:11:0:operator:/root:/sbin/nologin

Aming123:x:501:513::/home/aming123:/bin/bash

[Email protected] ~]# sed-n '/root/p; /aming/p ' 1.txt

Root:aming X:0:0:root:/root:/bin/bash

Root:aming X:0:0:root:/root:/bin/bash

Operatorrrrro:x:11:0:operator:/root:/sbin/nologin

Aming123:x:501:513::/home/aming123:/bin/bash

3. awk command to intercept a paragraph in the paragraph

-f Specifies that the separator symbol is a single quote ' If the separator symbol is a special symbol

Print a paragraph of characters

# awk-f ': ' {print $1.txt} '

# awk-f ': ' {print $3,$4} ' 1.txt

Display specified characters

# awk-f ': ' ofs= ': ' {print $3,$4} ' 1.txt

# awk-f ': ' ofs= "#" {print $3,$4} ' 1.txt

Match character or string

# awk '/r/' 1.txt

# awk '/root|user/' 1.txt

awk '/r*o/' 1.txt

# awk '/r?o/' 1.txt

# awk '/r+o/' 1.txt

# awk '/r.*o/' 1.txt

Match a whole

# awk '/(oo)/' 1.txt

# awk '/(oooo) +/' 1.txt

Match by Segment

# awk-f ': ' $1~/r*o/' 1.txt

# awk-f ': ' $1~/r*o/{print $4} ' 1.txt

Multiple matches

# awk-f ': ' $1~/r*o/{print $1,$3}; $1~/user/{print $1,$3} ' 1.txt

Print two times (awk belongs to the streaming editor) if match requirements are met at the same time

[[email protected] ~]# awk-f ': ' $1~/r*o/{print $1,$4};$1~/nobody/{print $1,$4} ' 1.txt

Rooooooooot 0

Daemon 2

Shutdown 0

Operator 0

Gopher 30

Nobody 99

Nobody 99

Postfix 89

AVAHI-AUTOIPD 170

Haldaemon 68

[[email protected] ~]# awk-f ': ' $1~/r*o|nobody/{print $1,$4} ' 1.txt

Rooooooooot 0

Daemon 2

Shutdown 0

Operator 0

Gopher 30

Nobody 99

Postfix 89

AVAHI-AUTOIPD 170

Haldaemon 68

awk conditional operator ==,>,<,>=,!=;<=

# awk-f ': ' $1== "nobody" ' 1.txt

Nobody:x:99:99:nobody:/:/sbin/nologin

# awk-f ': ' $1== ' nobody ' | | $1~/nolog/' 1.txt

# awk-f ': ' $4>=500 ' 1.txt

# awk-f ': ' $7!= "/sbin/nologin" ' 1.txt

The

# awk-f ': ' $7~/nolog/' 1.txt

# awk-f ': ' $7!~/nolog/' 1.txt

Comparison

# awk-f ': ' $3<$4 ' 1.txt

# awk-f ': ' ofs= ': "; $3=$4 ' 1.txt

# awk-f ': ' $3==$4 ' 1.txt

# awk-f ': ' ofs= ': "; {if ($4>5) {$7=$3+$4}} ' 1.txt

The first paragraph equals ' aming ' and the 3rd segment is less than 100 is filtered by a blank character delimiter. (The first thing to do is to have a blank character delimiter

's files)

[Email protected] ~]# sed-i ' s/://g ' 2.txt

[[email protected] ~]# awk ' $1== ' games ' && $3<100 ' 2.txt

Games x Games/usr/games/sbin/nologin

awk built-in variable NF (number of segments) NR (number of lines)

# awk-f ': ' nr==10 ' 1.txt print line tenth

# awk-f ': ' nr>10 ' 1.txt print 10 lines after line

# awk-f ': ' nr<10 ' 1.txt

[[email protected] ~]# awk-f ': ' nr==10 {print $1,$7} ' 1.txt

Uucp/sbin/nologin

[[email protected] ~]# awk-f ': ' ofs= ': ' {if (nr==10) print $1,$7} ' 1.txt

Uucp:/sbin/nologin

[[email protected] ~]# awk-f ': ' {print NF} ' 1.txt

[[email protected] ~]# awk-f ': ' {if (nf==7) print '} ' 1.txt

# awk-f ': ' {print $NR, $NF} '

NR is changed according to the line, and NF is the total number of paragraphs in this line.

# awk-f ': ' nr==10 {ofs= "#";p rint $1,$7} ' 1.txt

Awk can also go through mathematical operations

[Email protected] ~]# awk-f ': ' $7=$3+$4 ' 1.txt

Bin x 1 1 bin/bin 2

Daemon x 2 2 Daemon/sbin 4

ADM x 3 4 Adm/var/adm 7

LP x 4 7 LP/VAR/SPOOL/LPD 11

Sync x 5 0 Sync/sbin 5

Shutdown x 6 0 Shutdown/sbin 6

Halt x 7 0 Halt/sbin 7

The default is a separator with a blank character, and if a paragraph in a text document changes to print, it defaults to whitespace as a delimiter. Need to use OFS to refer to

Delimiter.

[[email protected] ~]# awk-f ': ' ofs= ': ' {$7=$3+$4; Print $} ' 1.txt

rooooooooot:x:0:0:root:/root:0

Bin:x:1:1:bin:/bin:2

Daemon:x:2:2:daemon:/sbin:4

Adm:x:3:4:adm:/var/adm:7

Lp:x:4:7:lp:/var/spool/lpd:11

Sync:x:5:0:sync:/sbin:5

# awk-f ': ' ofs= ': ' {$7=$3+$4; Print $1,$3} ' 1.txt

Calculate the sum of a paragraph, such as the third paragraph

[Email protected] ~]# awk-f ': ' {sum=sum+$3}; END {print sum} ' 1.txt

4807

[Email protected] ~]# awk-f ': ' {(sum=sum+$3)}; END {print sum} ' 1.txt

4807

{}

AWK structure: awk-f ': ' Begin{ofs= ":"} {if (condition) {statement 1; Statement 2; Statement 3}} end{statement} '

Reference Tutorial Http://www.cnblogs.com/emanlee/p/3327576.html

Linux_note Command Grep,sed,awk