Shell regular Expressions and text-processing tools (i)

Job One: organize regular expression Blogs

1, what is the regular
A regular is a method of describing a character or string with a combination of special meanings (called regular expressions).
Or, the regular is the rule used to describe a class of things.

Wildcard characters are interpreted by the shell. If the shell is grandpa, the wildcard character is dad.
The regular expression is interpreted by the command, and the command is interpreted by the shell. If the shell is grandpa, the regular expression is the grandson.
Wildcard characters and commands are of a level.

Wildcards are for easy command operation, and regular expressions are for manipulating text content, characters, or strings.

*: The character on the left has 0 to infinity
+: The character on the left has 1 to infinity
?: The character on the left has 0 to 1
{n}: The character on the Left has n
{n,m}: The character on the left has n to M
{N,}: The character on the left has n to infinity

[A-z]: all lowercase letters
[A-Z]: all uppercase letters
[A-za-z]: All uppercase and lowercase letters, equal to [a-z]
Note: If the character to be matched is-itself, it must be put to the last [123123\-]

Job Two: grep jobs (regular expressions and character processing)

Target file/etc/passwd, using the grep command or Egrep

1. All rows containing root are displayed:

[[email protected] ~]# egrep ' root '/etc/passwdroot:x:0:0:root:/root:/bin/bashoperator:x:11:0:operator:/root:/sbin/ Nologin

2. Output any rows that contain bash, and also output the contents of the next two lines, followed by the row:

[[email protected] ~]# egrep-a 2 ' bash '/etc/passwdroot:x:0:0:root:/root:/bin/bashbin:x:1:1:bin:/bin:/sbin/ NOLOGINDAEMON:X:2:2:DAEMON:/SBIN:/SBIN/NOLOGIN--LUCHUAN:X:1001:1001::/HOME/LUCHUAN:/BIN/BASHEGON:X:1002:1002: :/home/egon:/bin/bashtom:x:1003:1003::/home/tom:/bin/bashaa:x:1004:1004::/home/aa:/bin/bashluchuangao:x : 1005:1005::/home/luchuangao:/bin/bashalex:x:1006:1006::/home/alex:/bin/bashjack:x:1007:1008::/home/jack:/bin/ bashrose:x:1008:1009::/home/rose:/bin/bashapache:x:48:48:apache:/usr/share/httpd:/sbin/nologindhcpd:x:177:177 :D HCP Server:/:/sbin/nologin

3. Shows how many lines contain nologin.

[[email protected] ~]# egrep-c ' root '/etc/passwd 2

4. Shows that the rows contain root and the line number is output.

[[email protected] ~]# egrep-n ' root '/etc/passwd1:root:x:0:0:root:/root:/bin/bash10:operator:x:11:0:operator:/root :/sbin/nologin

5. Show the file name

[[email protected] ~]# egrep-l ' root '/etc/passwd/etc/passwd

6. New user

Abominable
Abominate
Anomie
Atomize
Write regular expressions and match them up

[Email protected] ~]# egrep ' ^a[a-z]o[a-z]+e '/etc/passwdabominable:x:1009:1010::/home/abominable:/bin/ Bashabominate:x:1010:1011::/home/abominate:/bin/bashanomie:x:1011:1012::/home/anomie:/bin/bashatomize:x : 1012:1013::/home/atomize:/bin/bash

7. Built four users

Alex213sb
wpq2222b
Yh438pig
egon666
Egon

Filter out the user name consists of a letter + number + letter Line

[[email protected] ~]# egrep ' [a-z]+[0-9]+[a-z]+ '/etc/passwd      alex213sb:x:1013:1014::/home/alex213sb:/bin/ Bashwpq2222b:x:1014:1015::/home/wpq2222b:/bin/bashyh438pig:x:1015:1016::/home/yh438pig:/bin/bash

8. Displays all filenames containing root in/etc directory

[[email protected] ~]# grep-rl ' root '/etc/* | grep ' Root '/etc/security/chroot.conf

9. Filter out all comments and all blank lines in/etc/ssh/sshd_config

[Email protected] etc]# egrep-v ' ^$|^# '/etc/ssh/sshd_config   hostkey/etc/ssh/ssh_host_rsa_keyhostkey/etc/ssh/ Ssh_host_ecdsa_keyhostkey/etc/ssh/ssh_host_ed25519_keysyslogfacility authprivauthorizedkeysfile      . ssh/ Authorized_keyspasswordauthentication yeschallengeresponseauthentication nogssapiauthentication Yesgssapicleanupcredentials nousepam yesx11forwarding yesuseprivilegeseparation Sandbox          # Default for new installations. Acceptenv LANG lc_ctype lc_numeric lc_time lc_collate lc_monetary lc_messagesacceptenv LC_PAPER LC_NAME LC_ADDRESS LC_TEL EPHONE lc_measurementacceptenv lc_identification lc_all languageacceptenv xmodifierssubsystem       sftp    /usr/ Libexec/openssh/sftp-server

Job Three: SED job:/etc/passwd file as Template

1. Delete the first character of each line of the file.

[[email protected] ~]# sed-r ' s/^ (.) (. *) $/\2/' Testoot:x:0:0:root:/root:/bin/bashin:x:1:1:bin:/bin:/sbin/nologinginx:x:991:547:nginx Web server:/var/ Lib/nginx:/sbin/nologinbominable:x:1009:1010::/home/abominable:/bin/bashbominate:x:1010:1011::/home/abominate :/bin/bashnomie:x:1011:1012::/home/anomie:/bin/bashtomize:x:1012:1013::/home/atomize:/bin/bashlex213sb:x : 1013:1014::/home/alex213sb:/bin/bashpq2222b:x:1014:1015::/home/wpq2222b:/bin/bashh438pig:x:1015:1016::/home/ Yh438pig:/bin/bashgon666:x:1016:1017::/home/egon666:/bin/bashgon:x:1017:1002::/home/egon:/bin/bash

2. Delete the second character of each line of the file.

[[email protected] ~]# sed-r ' s/^ (.) (.) (. *) $/\1\3/' Testrot:x:0:0:root:/root:/bin/bashbn:x:1:1:bin:/bin:/sbin/nologinninx:x:991:547:nginx Web server:/ var/lib/nginx:/sbin/nologinaominable:x:1009:1010::/home/abominable:/bin/bashaominate:x:1010:1011::/home/ abominate:/bin/bashaomie:x:1011:1012::/home/anomie:/bin/bashaomize:x:1012:1013::/home/atomize:/bin/ Bashaex213sb:x:1013:1014::/home/alex213sb:/bin/bashwq2222b:x:1014:1015::/home/wpq2222b:/bin/bashy438pig:x : 1015:1016::/home/yh438pig:/bin/basheon666:x:1016:1017::/home/egon666:/bin/basheon:x:1017:1002::/home/egon:/ Bin/bash

3. Delete the last character of each line of the file.

[[email protected] ~]# sed-r ' s/([^a-z]) (. *) (.) $/\1\2/' Test     Root:x:0:0:root:/root:/bin/basbin:x:1:1:bin:/bin:/sbin/nologinginx:x:991:547:nginx Web server:/ var/lib/nginx:/sbin/nologiabominable:x:1009:1010::/home/abominable:/bin/basabominate:x:1010:1011::/home/ abominate:/bin/basanomie:x:1011:1012::/home/anomie:/bin/basatomize:x:1012:1013::/home/atomize:/bin/ Basalex213sb:x:1013:1014::/home/alex213sb:/bin/baswpq2222b:x:1014:1015::/home/wpq2222b:/bin/basyh438pig:x : 1015:1016::/home/yh438pig:/bin/basegon666:x:1016:1017::/home/egon666:/bin/basegon:x:1017:1002::/home/egon:/ Bin/bas

4. Delete the second-to-last character of each line of the file.

5. Delete the second word in each line of the file.

6. Delete the second-to-last word in each line of a file.

[[email protected] ~]# sed-r ' s/([^a-z0-9]) ([a-z0-9]+] ([^a-z0-9]) ([a-z0-9]+$]/\1\3\4/g ' Test root:x:0:0:root:/root:/ /bashbin:x:1:1:bin:/bin://nologinnginx:x:991:547:nginx Web server:/var/lib/nginx://nologinabominable:x:1009:1010 ::/home/abominable://bashabominate:x:1010:1011::/home/abominate://bashanomie:x:1011:1012::/home/anomie:// bashatomize:x:1012:1013::/home/atomize://bashalex213sb:x:1013:1014::/home/alex213sb://bashwpq2222b:x:1014:1015 ::/home/wpq2222b://bashyh438pig:x:1015:1016::/home/yh438pig://bashegon666:x:1016:1017::/home/egon666:// Bashegon:x:1017:1002::/home/egon://bash

7. Delete the last word in each line of the file.

[[email protected] ~]# sed-r ' s/([^a-z0-9]) ([a-z0-9]+$]/\1/g ' testroot:x:0:0:root:/root:/bin/bin:x:1:1:bin:/bin:/ Sbin/nginx:x:991:547:nginx Web server:/var/lib/nginx:/sbin/abominable:x:1009:1010::/home/abominable:/bin/ abominate:x:1010:1011::/home/abominate:/bin/anomie:x:1011:1012::/home/anomie:/bin/atomize:x:1012:1013::/home/ Atomize:/bin/alex213sb:x:1013:1014::/home/alex213sb:/bin/wpq2222b:x:1014:1015::/home/wpq2222b:/bin/yh438pig:x : 1015:1016::/home/yh438pig:/bin/egon666:x:1016:1017::/home/egon666:/bin/egon:x:1017:1002::/home/egon:/bin/

8, swaps the first character and the second character of each line.

[[email protected] ~]# sed-r ' s/^ (.) (.) /\2\1/' Testorot:x:0:0:root:/root:/bin/bashibn:x:1:1:bin:/bin:/sbin/nologingninx:x:991:547:nginx Web server:/var/ lib/nginx:/sbin/nologinbaominable:x:1009:1010::/home/abominable:/bin/bashbaominate:x:1010:1011::/home/ abominate:/bin/bashnaomie:x:1011:1012::/home/anomie:/bin/bashtaomize:x:1012:1013::/home/atomize:/bin/ Bashlaex213sb:x:1013:1014::/home/alex213sb:/bin/bashpwq2222b:x:1014:1015::/home/wpq2222b:/bin/bashhy438pig:x : 1015:1016::/home/yh438pig:/bin/bashgeon666:x:1016:1017::/home/egon666:/bin/bashgeon:x:1017:1002::/home/egon:/ Bin/bash

9, swaps the first character and the second word of each line.

[[email protected] ~]# sed-r ' s/(^[a-z]) ([a-z0-9]+] ([^a-z0-9]) ([a-z]+]/\4\2\3\1/' Test     xoot:r:0:0:root:/root:/ Bin/bashxin:b:1:1:bin:/bin:/sbin/nologinxginx:n:991:547:nginx Web server:/var/lib/nginx:/sbin/nologinxbominable: a:1009:1010::/home/abominable:/bin/bashxbominate:a:1010:1011::/home/abominate:/bin/bashxnomie:a:1011:1012::/ home/anomie:/bin/bashxtomize:a:1012:1013::/home/atomize:/bin/bashxlex213sb:a:1013:1014::/home/alex213sb:/bin/ Bashxpq2222b:w:1014:1015::/home/wpq2222b:/bin/bashxh438pig:y:1015:1016::/home/yh438pig:/bin/bashxgon666:e : 1016:1017::/home/egon666:/bin/bashxgon:e:1017:1002::/home/egon:/bin/bash

10, swap the first word and the last word of each line.

[[email protected] ~]# sed-r ' s/(^[a-z0-9]+) (. *) ([^a-z0-9]) ([a-z]+$]/\4\2\3\1/' testbash:x:0:0:root:/root:/bin/ Rootnologin:x:1:1:bin:/bin:/sbin/binnologin:x:991:547:nginx Web Server:/var/lib/nginx:/sbin/nginxbash:x : 1009:1010::/home/abominable:/bin/abominablebash:x:1010:1011::/home/abominate:/bin/abominatebash:x:1011:1012: :/home/anomie:/bin/anomiebash:x:1012:1013::/home/atomize:/bin/atomizebash:x:1013:1014::/home/alex213sb:/bin/ Alex213sbbash:x:1014:1015::/home/wpq2222b:/bin/wpq2222bbash:x:1015:1016::/home/yh438pig:/bin/yh438pigbash:x : 1016:1017::/home/egon666:/bin/egon666bash:x:1017:1002::/home/egon:/bin/egon

11, delete all the numbers in a file.

[[email protected] ~]# sed-r ' s/([0-9]+)//g ' testroot:x:::root:/root:/bin/bashbin:x:::bin:/bin:/sbin/nologinnginx:x ::: Nginx Web server:/var/lib/nginx:/sbin/nologinabominable:x::::/home/abominable:/bin/bashabominate:x::::/home/ abominate:/bin/bashanomie:x::::/home/anomie:/bin/bashatomize:x::::/home/atomize:/bin/bashalexsb:x::::/home/ alexsb:/bin/bashwpqb:x::::/home/wpqb:/bin/bashyhpig:x::::/home/yhpig:/bin/bashegon:x::::/home/egon:/bin/ Bashegon:x::::/home/egon:/bin/bash

12, remove all spaces at the beginning of each line.

[[email protected] ~]# cat testroot:x:0:0:root:/root:/bin/bashbin:x:1:1:bin:/bin:/sbin/nologinnginx:x : 991:547:nginx Web server:/var/lib/nginx:/sbin/nologinabominable:x:1009:1010::/home/abominable:/bin/ Bashabominate:x:1010:1011::/home/abominate:/bin/bashanomie:x:1011:1012::/home/anomie:/bin/bashatomize:x : 1012:1013::/home/atomize:/bin/bashalex213sb:x:1013:1014::/home/alex213sb:/bin/bashwpq2222b:x:1014:1015::/home        /wpq2222b:/bin/bashyh438pig:x:1015:1016::/home/yh438pig:/bin/bashegon666:x:1016:1017::/home/egon666:/bin/bash Egon:x:1017:1002::/home/egon:/bin/bash[[email protected] ~]# [[email protected] ~]# sed-r ' s/(^[[:blank:] ]+)//g ' Testroot:x:0:0:root:/root:/bin/bashbin:x:1:1:bin:/bin:/sbin/nologinnginx:x:991:547:nginx Web server:/var/ lib/nginx:/sbin/nologinabominable:x:1009:1010::/home/abominable:/bin/bashabominate:x:1010:1011::/home/ abominate:/bin/bashanomie:x:1011:1012::/home/anomie:/bin/bashatomize:x:1012:1013::/home/atomize:/bin/ Bashalex213sb:x:1013:1014::/home/alex213sb:/bin/bashwpq2222b:x:1014:1015::/home/wpq2222b:/bin/bashyh438pig:x:1015:1016::/home/yh438pig:/bin/ Bashegon666:x:1016:1017::/home/egon666:/bin/bashegon:x:1017:1002::/home/egon:/bin/bash

13, replace any spaces that appear in the file with tabs.

[Email protected] ~]# sed-r ' s//\t/g ' testroot:x:0:0:root:/root:/bin/bashbin:x:1:1:bin:/bin:/sbin/nologinnginx:x : 991:547:nginx   Web     server:/var/lib/nginx:/sbin/nologinabominable:x:1009:1010::/home/abominable:/bin/ Bashabominate:x:1010:1011::/home/abominate:/bin/bashanomie:x:1011:1012::/home/anomie:/bin/bashatomize:x : 1012:1013::/home/atomize:/bin/bashalex213sb:x:1013:1014::/home/alex213sb:/bin/bashwpq2222b:x:1014:1015::/home /wpq2222b:/bin/bashyh438pig:x:1015:1016::/home/yh438pig:/bin/bashegon666:x:1016:1017::/home/egon666:/bin/bash                        Egon:x:1017:1002::/home/egon:/bin/bash

14, enclose all uppercase letters in parentheses ().

[Email protected] ~]# sed-r ' s/[a-z]/(&)/g ' testroot:x:0:0:root:/root:/bin/bashbin:x:1:1:bin:/bin:/sbin/ nologinnginx:x:991:547: (N) Ginx Web server:/var/lib/nginx:/sbin/nologinabominable:x:1009:1010::/home/abominable:/ Bin/bashabominate:x:1010:1011::/home/abominate:/bin/bashanomie:x:1011:1012::/home/anomie:/bin/bashatomize:x : 1012:1013::/home/atomize:/bin/bash (a) lex213sb:x:1013:1014::/home/(a) Lex213sb:/bin/bash (W) pq2222b:x:1014:1015: :/home/(W) pq2222b:/bin/bashy (h) 438 (p) (i) (G): x:1015:1016::/home/y (h) 438 (P) (I) (g):/bin/bashegon666:x:1016:1017: :/home/egon666:/bin/bash        Egon:x:1017:1002::/home/egon:/bin/bash

15, print 3 times per line.

[[email protected] ~]# sed-r ' p;p ' testroot:x:0:0:root:/root:/bin/bashroot:x:0:0:root:/root:/bin/bashroot:x:0 : 0:root:/root:/bin/bashbin:x:1:1:bin:/bin:/sbin/nologinbin:x:1:1:bin:/bin:/sbin/nologinbin:x:1:1:bin:/bin:/ Sbin/nologinnginx:x:991:547:nginx Web Server:/var/lib/nginx:/sbin/nologinnginx:x:991:547:nginx Web server:/var/lib /nginx:/sbin/nologinnginx:x:991:547:nginx Web server:/var/lib/nginx:/sbin/nologinabominable:x:1009:1010::/home/ Abominable:/bin/bashabominable:x:1009:1010::/home/abominable:/bin/bashabominable:x:1009:1010::/home/abominable :/bin/bashabominate:x:1010:1011::/home/abominate:/bin/bashabominate:x:1010:1011::/home/abominate:/bin/ Bashabominate:x:1010:1011::/home/abominate:/bin/bashanomie:x:1011:1012::/home/anomie:/bin/bashanomie:x : 1011:1012::/home/anomie:/bin/bashanomie:x:1011:1012::/home/anomie:/bin/bashatomize:x:1012:1013::/home/atomize :/bin/bashatomize:x:1012:1013::/home/atomize:/bin/bashatomize:x:1012:1013::/home/atomize:/bin/bashalex213sb:x : 1013:1014::/home/alex213sb:/bin/bashalex213sb:x:1013:1014::/home/alex213sb:/bin/bashalex213sb:x:1013:1014::/home/alex213sb:/bin/ Bashwpq2222b:x:1014:1015::/home/wpq2222b:/bin/bashwpq2222b:x:1014:1015::/home/wpq2222b:/bin/bashwpq2222b:x : 1014:1015::/home/wpq2222b:/bin/bashyh438pig:x:1015:1016::/home/yh438pig:/bin/bashyh438pig:x:1015:1016::/home/ yh438pig:/bin/bashyh438pig:x:1015:1016::/home/yh438pig:/bin/bashegon666:x:1016:1017::/home/egon666:/bin/ BASHEGON666:X:1016:1017::/HOME/EGON666:/BIN/BASHEGON666:X:1016:1017::/HOME/EGON666:/BIN/BASHEGON:X:1017:1002: :/home/egon:/bin/bashegon:x:1017:1002::/home/egon:/bin/bashegon:x:1017:1002::/home/egon:/bin/bash

16, only the first word of each line is displayed.

[Email protected] ~]# sed-r ' s/(^[a-z0-9]+) ([^a-z]) (. *)/\1/' Testrootbinnginxabominableabominateanomieatomizealex213sbwpq2222byh438pigegon666egon

17, print the first word and the third word of each line.

[[email protected] ~]# sed-r ' s/^ ([a-z0-9]+] ([^a-z]) ([a-z0-9]+] ([^a-z]) ([a-z0-9]+] ([^a-z]) (. *)/\1\5/' Testroot0bin1nginx991abominable1009abominate1010anomie1011atomize1012alex213sb1013wpq2222b1014yh438pig1015egon6661016egon 1017

18, use the command to get the format MM/YY/DD date format, combined with the pipeline, change it to MM;YY;DD format

Reference Link: http://www.cnblogs.com/linhaifeng/p/6596660.html

Shell regular Expressions and text-processing tools (i)