Regular expressions for Shell programming

Regular Expressions && text-processing tool

Learning Goals:

L Mastering the use of regular expressions

L MASTER the use of SED and awk text processing tools

Directory structure:

Regular expression Regular Expressions overview

L Regular Expressions: Use a single string to describe, match a series of strings that conform to a certain syntactic rule

L consists of ordinary characters and special characters

L generally used in script programming, text editor, such as PHP, Python, shell, etc., shorthand for regex, RegExp, RE

L used to retrieve and replace conforming text, with powerful text matching function.

l ability to process text quickly and efficiently in the ocean of text

L Regular Expression hierarchy

Basic Regular Expressions

Expand Regular Expressions

L Linux Chinese Text Processing tool

Grep

Sed

Awk

Regular expression meta-characters

L Basic Regular expression is a common regular expression part

L common to the following metacharacters in addition to ordinary characters

\: Escape character, so that the symbol is a symbol, there is no other meaning. \!,\n, etc.

^: match the starting position of the string

Example: ^a, ^the, ^#

$: Match where the string ends

Example: word$

.: matches any one character except \ n

Example: GO.D, G.. D

*: Match front face expression 0 or more times

Example: Goo*d,go*d

[List]: matches a character in the list

Example: Go[ola]d, [ABC], [A-z], [a-z0-9]

[^list]: matches any character not in the list

Example: [^a-z], [^0-9], [^a-z0-9]

\{n,m\}: Matches the preceding subexpression N to M times, with \{n\}, \{n,\}, \{n,m\} three formats

Example: Go\{2\}d, Go\{2,3\}d, go\{2,\}

Extended regular expression meta-characters

L Extended Regular expression is the expansion and deepening of the underlying regular expression

L Extended meta-characters

+: Match front face expression more than 1 times

Example: Go+d, will match at least one O

？ : Match front face expression 0 or 1 times

Example: Go?d, will match GD or God

(): Enclose the string in parentheses as a whole

Example: (XYZ) +, will match xyz overall more than 1 times, such as xyzxyz

|: Match the note string in a way or

Example 1:good|food, will match good or food

Example 2:g (Oo|la) d, will match good or glad

SED tools Practice SED Tools overview

L SED is a text processing tool that reads text content and processes it according to specified conditions, such as deleting, replacing, adding, etc.

L can achieve quite complex text manipulation without interaction

L is widely used in shell scripts to complete automated processing tasks

L SED relies on regular expressions

Examples of SED usage

L Replacement Example

Sed ' s/xml/xml/' bfile//replace the first XML in each row with XML

Sed ' s/xml//g ' bfile//Remove all XML from the file

Sed ' 3,5s/xml/xml/g ' bfile//Replace all XML in the 第3-5 row with XML

Sed '/xml/s/com/com/g ' bfile//replace COM in all rows containing XML with COM

L execute Edit command multiple times

Sed-e ' 3,5p '-e ' 3,5s/xml/xml/g ' bfile//can save multiple editing commands to a file and specify the file via-F to complete multiple processing operations

This is only part of the case where SED and regular expressions are used together

For specific use of SED see https://www.cnblogs.com/zwgblog/p/6013975.html

awk Tools Practice awk Tools Introduction

Awk is also a powerful editing tool that, like SED, allows for quite complex text manipulation without interaction

L Command Format

awk option ' mode or condition {edit instruction} ' file 1 file 2

Awk-f script file 1 file 2

L Working principle

Read text line by row, by default separated by spaces, save separate fields into built-in variables, and execute edit commands by pattern or condition

L awk built-in variables

FS: Specifies the delimiter for each line of text, by default a space or tab stop

NF: The number of fields in the currently processed row

NR: Line number (ordinal) of the line currently being processed

$ A: The entire line of the currently processed row

$n: Nth field (column N) of the currently processed row

Examples of awk tools

L Print Text content

awk ' Nr==1,nr==3{print} ' bfile//output first to third line contents

awk ' nr==1| Nr==3{print} ' bfile//Output first line, third line content

awk '/^root/{print} '/etc/passwd//output lines starting with root

L Output text by field

awk ' {print $1,$3} ' bfile//Output 1th, 3rd field in each line

Awk-f ":" ' {print$1,$7} '/etc/shadow//The shadow record of the user with null output password

The exact description of awk is described in http://blog.chinaunix.net/uid-23302288-id-3785105.html

At this point the basic shell programming knowledge has been completed, see Bo Master homepage http://www.cnblogs.com/tzlsj. Shell programming is profound, good at handling all kinds of files, and Linux is the document of all the characteristics of a very fit.

Now know the shell knowledge points like arithmetic law, seemingly very simple, but actually various combinations of changes. Next I will give you some fun little scripts for the actual use of the shell and strengthen the reinforcement.

Regular expression for Shell programming