Linux regular expressions, query command instances, and regular expression instances

Linux regular expressions, query command instances, and regular expression instances

When using linux, you often need to find files. The search Commands include find and grep. The two commands are partitioned.

(1) The find command searches for Objects Based on their attributes, such as the file name, file size, owner, group, whether it is empty, access time, and modification time.

(2) grep searches based on the file content and matches each row of the file according to the specified patter.

Linux Regular Expression

Grep: the earliest text matching program. It uses the basic regular expression (BRE) to match the text.

Egrep: Extended grep, which uses an extended regular expression (ERE) to match text.

Fgrep: Quick grep. This version matches fixed strings instead of regular expressions. (Regular Expression search is not supported)

I. linux text search commands

Introduction to grep commands

Purpose: The text search tool checks the target text row by row based on the user-specified "Mode" and prints the matched rows.

Mode: filtering conditions written by regular expression characters and text characters

Grep command

Syntax: grep [Option] (parameter)

Option:

Grep -- color = auto: color the matched text (displayed by default in CentOS7)

Grep-v # reverse search

Grep-I # case insensitive

Grep-n # display the matched row number

Grep-c # count the number of matched rows without displaying search results

Grep-o # only display matched strings

Grep-q # Silent (no result is displayed)

Echo $? # Display command execution results. The correct value is 0, and the error value is not 0.

Grep-A # display the search row and Its Adjacent rows down #

Grep-B # display the search row and Its Adjacent rows up #

Grep-C # display the search row and Its Adjacent rows up and down #

Grep-e # supports multiple options

Grep-e 'root'-e 'bin'/etc/passwd # The or relationship between multiple options

Grep-w # match the entire word

Grep-E # Or egrep supports extended Regular Expressions

Grep-F # Or fgrep does not support extended Regular Expressions

II. Introduction to Regular Expressions

Regular Expression (REGEXP): A pattern written by a special character or text character. Some of these characters (metacharacters) do not represent the literal meaning of the character, but represent the control or wildcard function.

Program support: grep, sed, awk, vim, less, nginx, varnish, etc.

There are two types:

Basic Regular Expression: BRE

Extended Regular Expression: ERE

Grep-E, egrep

Regular Expression Engine:

Use different algorithms to check the PCRE (Perl Compatible Regular Expressions) software module that processes Regular Expressions)

Metacharacter classification: character matching, matching times, location anchoring, grouping

Man 7 regex

Basic Regular Expression metacharacters

Character match:

. Match any single character

(): Regular expression group enclosed by square brackets

[] Matches any single character in the specified range, where hyphens (-) indicate the range of consecutive characters.

[^] Match any single character out of the specified range

[: Alnum:] letters and numbers

[: Alpha:] represents any English case character, that is, A-Z, a-z

[: Lower:] lowercase letters

[: Upper:] uppercase letters

[: Blank:] blank characters (spaces and tabs)

[: Space:] horizontal and vertical spaces (wider than [: blank)

[: Cntrl:] unprintable control characters (undefined, deleted, alert bell ...)

[: Digit:] decimal number

[: Xdigit:] hexadecimal number

[: Graph:] printable non-blank characters

[: Print:] printable characters

[: Punct:] punctuation marks

Matching times: used after the character to be specified, used to specify the number of times the previous character will appear

* Match any character before, including 0 (Greedy mode: matching as long as possible)

. * Any character of any length

\ Special meanings of subsequent characters are usually used to enable or disable

| Or (match | regular expression before or after a symbol)

\? Match the first character 0 or 1

\ + Match the first character at least once

\ {N \} matches the previous CHARACTER n times

\ {M, n \} matches the previous character at least m times, at most n times

\ {, N \} matches the preceding characters up to n times

\ {N ,\} matches the previous character at least n times

Location positioning: locate the location that appears

^ The first line is anchored to the leftmost part of the mode.

$ Anchor at the end of the line, used to the rightmost of the Mode

^ PATTERN $ used to match the entire line

^ $ Empty rows

^ [[: Space:] * $ blank row

\ <Or \ B is first anchored for the left side of the word mode

\> Or \ B ending point; used to the right of word mode

\ Match the entire word

Group

GROUP: \ (\) binds one or more characters and processes them as a whole, for example, \ (root \) \ +

The pattern matching content in the grouping brackets is recorded in internal variables by the Regular Expression Engine. These variables are named in the following way: \ 1, \ 2, \ 3 ,...

\ 1 indicates the first left brace from the left and the character matching the pattern between the right brace.

Example:

\ (String1 \ + \ (string2 \)*\)

\ 1: string1 \ + \ (string2 \)*

\ 2: string2

Backward reference: references the pattern matching characters in the preceding grouping brackets, instead of the pattern itself.

Or: |

Example:

A | B: a or B C | cat: C or cat \ (C | c \) at: Cat or cat

Iii. Extended Regular Expressions

Egrep = grep-E

Egrep [OPTIONS] PATTERN [FILE...]

Metacharacters of the extended regular expression:

Character match:

. Any single character

[] Characters in the specified range

[^] Characters out of the specified range

Matching times:

*: Match any character before

? : 0 or 1 +: 1 or multiple times

{M}: matching m times

{M, n}: At least m, up to n times

Positioning:

^: Beginning of a row

$: End of a row

\ <, \ B: Beginning

\>, \ B: End

Group

() Back Reference: \ 1, \ 2 ,...

Or:

A | B: a or B C | cat: C or cat (C | c) at: Cat or cat

(Instance operations using CentOS7.4 may differ from 6 systems)

Instance 1 Regular Expression

1. display the rows starting with big or small s in the/proc/meminfo file (two methods)

Method 1 grep-I "^ s"/proc/meminfo

[Root @ centos7 ~] # Grep-I "^ s"/proc/meminfo #-I ignore the "s" parameter at the beginning of the case ^ line

SwapCached: 0 kB

SwapTotal: 2097148 kB

SwapFree: 2097148 kB

Shmem: 9260 kB

Slab: 74036 kB

SReclaimable: 28260 kB

SUnreclaim: 45776 kB

Method 2 grep "^ [S | s]"/proc/meminfo

[Root @ centos7 ~] # Grep "^ [S | s]"/proc/meminfo # ^ first line anchor "[S | s]" S or s

SwapCached: 0 kB

SwapTotal: 2097148 kB

SwapFree: 2097148 kB

Shmem: 9260 kB

Slab: 74036 kB

SReclaimable: 28260 kB

SUnreclaim: 45776 kB

2. Display rows that do not end with/bin/bash in the/etc/passwd file

[Root @ centos7 ~] # Grep-v "/bin/bash $"/etc/passwd #-v reverse search "\ (bin/bash \) $ "rows ending with bin/bash (because only a portion of the output results are included)

Bin: x: 1: 1: bin:/sbin/nologin

Daemon: x: 2: 2: daemon:/sbin/nologin

Adm: x: 3: 4: adm:/var/adm:/sbin/nologin

Lp: x: 4: 7: lp:/var/spool/lpd:/sbin/nologin

Sync: x: 5: 0: sync:/sbin:/bin/sync

Shutdown: x: 6: 0: shutdown:/sbin/shutdown

Halt: x: 7: 0: halt:/sbin/halt

Mail: x: 8: 12: mail:/var/spool/mail:/sbin/nologin

Operator: x: 11: 0: operator:/root:/sbin/nologin

Games: x: 12: 100: games:/usr/games:/sbin/nologin

Ftp: x: 14: 50: FTP User:/var/ftp:/sbin/nologin

Nobody: x: 99: 99: Nobody: // sbin/nologin

3. display the default shell program of user rpc

[Root @ centos7 ~] # Cat/etc/passwd | grep "^ \ (rpc \) \ B" | cut-d:-f 7 # grep "^ (rpc) \ B "can replace the row \ B starting with the user's rpc with> (do not explain it later), cut-d:-f 7 and use the separator to take the seventh column

/Sbin/nologin

4. Find the two or three numbers in/etc/passwd.

Root @ centos7 ~] # Grep-o "\ B [0-9] \ {2, 3 \} \ B"/etc/passwd # (too many output values are only part of them)

12

11

12

5. In the/etc/grub2.cfg file of CentOS7, lines that start with at least one blank character and are followed by non-blank characters are displayed.

[Root @ centos7 ~] # Grep "^ [[: space:] \ + [^ [: space:]"/etc/grub2.cfg # ^ [[: space:] \ + [^ [: space:] starting with more than one blank and the following is a non-blank line (too many output values only take part)

Load_env

Set default = "nextentry" setnextentry = Principal = truesetdefault = "" role = "presentation"> nextentry "setnextentry = Principal = truesetdefault =" nextentry "setnextentry = Principal = truesetdefault =" {saved_entry }"

Menuentry_id_option = "-id"

Menuentry_id_option = ""

Set saved_entry = "$ {prev_saved_entry }"

Save_env saved_entry

Set prev_saved_entry =

6. Find the line ending with any number of blank characters following 'listen' in the result of the "netstat-tan" command

[Root @ centos7 ~] # Netstat-tan | grep "LISTEN [[: space:] \ *" # [[: space:] \ + "role =" presentation "> "#[[: space:] \ + "# [[: space:] \ + ends with more than one blank space and the front side is 'listen'

Tcp 0 0 0.0.0.0: 111 0.0.0.0: * LISTEN

Tcp 0 0 192.168.122.1: 53 0.0.0.0: * LISTEN

Tcp 0 0 0.0.0.0: 22 0.0.0.0: * LISTEN

Tcp 0 0 127.0.0.1: 631 0.0.0.0: * LISTEN

Tcp 0 0 127.0.0.1: 25 0.0.0.0: * LISTEN

Tcp 0 0 127.0.0.1: 6010 0.0.0.0: * LISTEN

Tcp6 0 0: 111: * LISTEN

Tcp6 0 0: 22: * LISTEN

Tcp6 0 0: 1: 631: * LISTEN

Tcp6 0 0: 1: 25: * LISTEN

Tcp6 0 0: 1: 6010: * LISTEN

7. display the usernames and UIDs of all system users on CentOS7

[Root @ centos7 ~] # Cat/etc/passwd | cut-d:-f1, 3 | grep "\ B [0-9] \ {1, 3 \} \ B" # Use the cut command to split the vertical column between the user and UID, run the grep command to retrieve the UID (the UID of the System user is smaller than 1000, and the version earlier than 500 of CentOS7) (if the output value is too large, only part of the UID is obtained)

Root: 0

Bin: 1

Daemon: 2

Adm: 3

Lp: 4

Sync: 5

Shutdown: 6

Halt: 7

Mail: 8

Operator: 11

Games: 12

Ftp: 14

Nobody: 99

Systemd-network: 192

8. Add the user bash, testbash, basher, sh, and nologin (the shell is/sbin/nologin), and find the line with "/etc/passwd" username and "shell" name.

[Root @ centos7 ~] # Useradd sh # create user sh

[Root @ centos7 ~] # Useradd bash # create user bash

[Root @ centos7 ~] # Useradd testbash # create user testbash

[Root @ centos7 ~] # Useradd-s/sbin/nologin create user nologin and set shell to/shin/nologin

'[Root @ centos7 ~] # Cat/etc/passwd | grep "^ \(. * \) \ B. */\ 1 "''' # Start with a section (. * \) enclosed in parentheses to reference. * It is determined by the tail of any character \ B. * Any character in the middle,/\ 1 "role =" presentation ">" ''' # Start with a group ^ section (. * \) enclosed in parentheses to reference. * It is determined by the tail of any character \ B. * Any character in the middle,/\ 1 "''' # Start with a group ^ section (. * \) enclosed in parentheses to reference. * It is determined by the tail of any character \ B. * Any character in the middle,/\ 1/directory delimiter \ 1 $ reference the part caused by parentheses before the end of the paragraph (the Group references the command result rather than the command itself)

Sync: x: 5: 0: sync:/sbin:/bin/sync

Shutdown: x: 6: 0: shutdown:/sbin/shutdown

Halt: x: 7: 0: halt:/sbin/halt

Nologin: x: 1003: 1003:/home/nologin:/sbin/nologin

Bash: x: 1005: 1005:/home/bash:/bin/bash

9. Use df and grep to retrieve the utilization rate of each disk partition and sort the data from large to small.

[Root @ centos7 ~] # Df | grep-o "[0-9] \ {1, 3 \} %" | sort-rn # "[0-9] \ {1, 3 \} %" hard disk Utilization for a maximum of 100%, three digits are used, % is added to indicate the percentage.

70%

18%

1%

1%

1%

0%

0%

0%

Example 2: Extended Regular Expression

1. display the UID and default shell of three users: root, mage, and wang.

Method 1: You can use a standard regular expression to extend the regular expression.

[Root @ centos7 ~] # Egrep-e ^ "mage"-e ^ "wang"-e "^ root"/etc/passwd | cut-d: -f 3, 7 # method 1 uses the grep multi-option search-The epartition link option can also be used with a standard regular expression.

0:/bin/bash

1001:/bin/bash

1002:/bin/bash

Method 2: extend the Regular Expression

[Root @ centos7 ~] # Egrep "^ (mage | wang | root) \>"/etc/passwd | cut-d:-f 3, 7 # method 2 exploitation or "|" extends the regular expression.

0:/bin/bash

1001:/bin/bash

1002:/bin/bash

Method 3: Standard Regular Expression

[Root @ centos7 ~] # Grep "^ \ (mage | wang | root) \>"/etc/passwd | cut-d:-f 3, 7 # method 2 exploitation or "|" standard regular expression.

0:/bin/bash

1001:/bin/bash

1002:/bin/bash

2. Find the rows in the/etc/rc. d/init. d/functions file that are followed by a parentheses after a word (including underscores ).

[Root @ centos7 ~] # Cat/etc/rc. d/init. d/functions | egrep "^ [[: alnum:] + _ +. * \ (\) "# The Beginning of the line is anchored. The letter or number is repeated once, And the underscore (_) is greater than once, and the letter or number is later than once followed by parentheses "()"

Echo_success (){

Echo_failure (){

Echo_passed (){

Echo_warning (){

Is_true (){

Is_false (){

Apply_sysctl (){

3. Use egrep to retrieve the base name of/etc/rc. d/init. d/functions

The base name is the file name or the last string of the directory.

[Root @ centos7 ~] # Echo/etc/rc. d/init. d/functions | egrep-o "[^/] + /? "#-O only displays the matched strings." [^/] + "the first line of a line does not start with more than one character ,"/? "Role =" presentation ">" #-o only displays the matched string. "[^/] +" the first line does not start with more than one/character ,"/? "#-O only displays the matched strings." [^/] + "the first line of a line does not start with more than one character ,"/?" Zero or a character ending with "/" (the directory is separated)

Functions

4. Use egrep to retrieve the Directory Name of the above path

[Root @ centos7 ~] # Echo/etc/rc. d/init. d/functions | egrep-o "/. */\ <"#-o only displays the matched string ,"/. */\ <"starts with"/"and ends with"/"." \ <"indicates that there are strings at the end. (The Directory is separated by a slash)

/Etc/rc. d/init. d/

5. count the number of logins from the root IP address of each host in the last command.

[Root @ centos7 ~] # Last output result

Root pts/0 192.168.217.1 Sat Jan 20 23: 25 still logged in

Root pts/1 192.168.217.1 Sat Jan 20)

Root pts/0 192.168.217.1 Sat Jan 20)

Reboot system boot 3.10.0-693. el7.x Sat Jan 20)

Root pts/1 192.168.217.1 Sat Jan 20-05:35)

[Root @ centos7 ~] # Last | egrep "^ root \ B" | egrep-o "([0-9] {1, 3 }\.) {3} [0-9] {1, 3} "| uniq-c # performs two searches. The beginning of the" ^ root \ B "line starts with root) -o only displays the matched string "([0-9] {1, 3 }\.) {3} [0-9] {1, 3} "0-9 repeats 1 to 3 times to indicate three digits. the IP address delimiter () is enclosed in parentheses to indicate a whole. The first three IP addresses are repeat three times, and the last [0-9] {} indicates the last IP address.

5 192.168.217.1

1 172.18.251.157

1 192.168.217.1

1 172.18.251.157

6. Use extended regular expressions to represent 0-9, 10-99, 100-199, 200-249, and 250-255, respectively.

[0-9] | [1-9] [0-9] | 1 [0-9] [0-9] | 2 [0-4] [0-9] | 25 [0-5] # [0-9] indicates 0-9, [1-9] [0-9] 1-9 and 0-9 groups are joined as 10-, 1 [0-9] [0-9] First, 1 has been determined, 23 are 0-9 groups the sum is 100-200, 2 [0-4] [0-9]. The first is determined as 2. The second is 0-4. The third is 0-9. The first and second are 249,250-255 -. the number of digits is determined to be 25. The third digit is the 0-5 group, and the sum is 250-255.

7. Display All IPv4 addresses in the ifconfig command results

[Root @ centos7 ~] # Ifconfig | egrep-o "\ B ([0-9] | [1-9] [0-9] | 1 [0-9] [0-9] | 2 [0-4] [0-9] | 25 [0-5]) \.) {3} ([0-9] | [1-9] {2} | 1 [0-9] [0-9] | 2 [0-4] [0-9- 9] | 25 [0-5]) \ B "# combination of the above two questions, with a maximum IP address of 255.

172.18.250.135

255.255.0.0

172.18.255.255

192.168.217.131

255.255.255.0

192.168.217.255

127.0.0.1

255.0.0.0

192.168.122.1

255.255.255.0

192.168.122.255

[Root @ centos7 ~] # Ifconfig | egrep netmask | tr-s '': | cut-d:-f3 # This is to retrieve only the IP address without a mask and the network administrator. First, search for the netmask lines and separate them.

172.18.250.135

192.168.217.131

127.0.0.1

192.168.122.1

8. Remove and sort each character in the string "welcome to magedu linux", and sort the preceding characters with many repetitions.

[Root @ centos7 ~] # Echo welcome to magedu linux | grep-o. | sort | uniq-c | sort-rn # Use "grep-o. "point" indicates any single character.-o sorts the matching individual characters in vertical columns, sorts, removes duplicates, and sorts again.

3 e

3

2 u

2 o

2 m

2 l

1 x

1 w

1 Tb

1 n

1 I

1g

1 d

1 c

1

9. display the. Format architecture of the rpm package in the/run/media/root/CentOS 7 x86_64/Packages/directory (the first part of. rpm in. x86_64.rpm) and check the number of duplicates

Retrieve partial results

Zlib-devel-1.2.7-17.el7.x86_64.rpm

Zlib-static-1.2.7-17.el7.i686.rpm

Zlib-static-1.2.7-17.el7.x86_64.rpm

Method 1: Use extended Regular Expression

[Root @ centos7 Packages] # ls *. rpm | egrep-o "\. [[: alnum:] _] + \. rpm $ "| cut-d. -f2 | sort | uniq-c # ls is selected first. the rpm package ending with rpm is then displayed with egrep any character starting with a dot followed by a letter, number, or underline. string ending with rpm: Cut, sort, and deduplicate.

2141 i686

3076 noarch

4374 x86_64

Method 2: Regular Expressions are not required.

[Root @ centos7 Packages] # ls *. rpm | rev | cut-d. -f2 | rev | sort | uniq-c. for the rpm package ending with rpm, "rev" is flipped around, cut and flip. Sort and deduplicate.

2141 i686

3076 noarch

4374 x86_64