Introduction to Sed Regular Expressions in Linux shell scripts, shellsed

Introduction to Sed Regular Expressions in Linux shell scripts, shellsed
Sed Introduction
Sed is a lightweight stream editor, short for stream editor. Sed is also called a row editor because it is used to edit a file in a behavior unit. It can automate editing without directly editing materials.

Sed working method:
Read the editing file from the standard input, read one line or specify rows into the mode space, edit all the editing commands one by one, and then output the results to the standard output, clear the mode space at the same time. Next, read the data of the next row into the mode space. Repeat the previous operations until the last row stops the streaming editor. Source File unchanged

Ps:
1. The mode space is the cache where the read row is located, and sed processes the text row here.
2. operate one row at a time and apply commands sequentially
3. The sed command is executed almost simultaneously with the returned data. The result is displayed when processing each row of data.
4. The file content has a mode space and has not changed. The original file will not be modified unless the redirected storage output is used.


Syntax:
Composed of editing commands and files
1. # sed [sed option] 'sed command 'file to be modified> New File
2. # sed [sed option]-f sed file to be modified by the script
3. # sed script [sed option] file to be modified


Parameters (sed option ):

-E command: -- expression = command: Multiple edits are performed, indicating that subsequent strings are parsed as sed editing commands and used when multiple sed commands are applied to the input line. # Sed-e '1, 5d '-e's/boy/girl/'dodo # sed -- expression = '1, 5d '-- expression ='s/boy/girl/'dodo-e script-file: -- expression = script-file: call the specified sed script file to process the input text file-f script-file: -- filer = script-file: Call the specified sed script file to process the input text file-h: -- help: Print help-I: directly modify the read source file (by default, the source file is not modified). You can also back up the source file and then modify it. The format is as follows: # sed-I. bak '1d 'filename-n: -- quiet, -- silent: cancel the default output. Only the matching mode rows are printed (all content is output by default)-r: sed actions support Syntax of extended formal notation. (The default is the basic regular expression syntax)-V: -- version: displays version information.

Sed command

A \: add one or more lines of text after the positioning line number. (To add multiple rows, you must add "\" to the end of each line.) B lable: jumps the executed command to the reference position created, if the reference position does not exist, jump to the end of the script c \: Replace the text of the located line with the new text. D: delete location Row D: Delete the first row g in the mode space: copy the content in the current retention buffer to the mode space, replace the content of the current row G in the mode space: append the content in the current buffer to the mode space, and append the content to the current row in the mode space. h: copy the content in the mode space to the current buffer, and clear the content in the original buffer zone, add new content H: copy the content of the mode space to the current retention buffer, append the content following the original content in the retention buffer I \: insert one or more lines of text before the row number. L: list the content in the current mode space in a visible and strict form. n: Read the next input line and use the next command to process the new line. N: append the next input row to the mode space. P: Print matching row P: Print mode space's first row q: after the first pattern match is complete, exit Sedr file: Read the input row s from the file: replace one string with another. (Replace the first line by default) t label: If the replacement operation is successful in the s command, jump to the ":" mark, even if you have read the last line of the input, T label at the end of the script if there is no flag. If the replacement operation fails in the s command, skip to the ":" mark, even if the last line of the input has been read, if there is no flag, write all the content in the current mode space to the file w: write the first row in the current mode space to the file x: swap mode space and text content y: Convert character, translate one character into another character (but not used for regular expression): label: create reference position {}: command Group with the same address parameter #: display the file line number before extending the annotation to the next line break! : Do not run the command of the selected line, only apply the command to the line other than the selected line

Metacharacters

Sed supports Special metacharacters for Pattern Search and replacement ^: the first line separator. For example,/^ doiido/matches all the rows starting with doiido $: locate the end of the row. For example,/doiido $/matches all rows ending with doiido ^ $: indicates a blank row.: matches characters other than line breaks. For example:/d... do/match d followed by three arbitrary characters, and then do. *: Matches zero or multiple leading characters. For example,/doiido */matches all rows starting with doiid followed by zero or multiple o []: matches any character in the specified character group. For example:/[Dd] oiido/match all rows containing doiido or Doiido [^]: match any character that is not in the specified character group. For example,/[^ Dd] oiido/matches all rows that do not start with D or d but end with oiido \ (.. \): stores matched characters. For example: s/\ (hello \) doiido/\ 1 baby save hello as label 1 here. If hellodoiido is found, replace it with helloboy. A maximum of nine tags can be defined &: save the search string to replace other strings. For example: s/doiido/-- & --/, Symbol & represents the search string, and doiido will be changed to -- doiido -- \ <: The first separator. For example, <doiido/matches all rows that contain words starting with doiido. \>: Suffix. For example,/doiido \>/matches all rows containing words ending with doiido. X \ {m \}: m x in a row. For example,/2 \ {8 \}/matches all rows that contain eight consecutive 2 rows. X \ {m, \}: At least m x. For example,/2 \ {8, \}/matches all rows that contain at least eight consecutive 2 rows. X \ {m, n \}: At least m x, but not more than n. For example,/2 \ {6, 8 \}/matches all rows that contain 6 to 8 consecutive 2 rows.

Ps: Before matching metacharacters $, you must use the backslash \ to block its special meaning. For example,/\ $/


Character class extension:

[]: [A-z] can be used with "-": match all lowercase letters [0-9]: Match All numbers [: space:]: match space [: alnum:]: match letter number [a-z A-Z 0-9] [: alpha:]: match letter [a-z A-Z] [: blank:]: match space or tabulation key [: cntrl:]: match any control character [: digit:]: Match Number [0-9] [: graph:]: match any visible character (no space) [: lower:]: Match lower case [a-z] [: print:]: Match uncontrolled character [: punct:]: match punctuation [: space:]: match space [: upper:]: Match uppercase [A-Z] [: xdigit:]: match hexadecimal numbers [0-9 a-f A-F]

Character class example:

^ [0-9]: indicates that the first character of the line is any number ^ [0-9] *: it indicates that the beginning of a row contains any number of numbers [0-9] [0-9] * $: It indicates that the end of a row contains at least two numbers s /\. $ // g: Delete the Period s/^ [] [] * //: delete any space s/^ at the beginning of the line. //: Delete the first character s/^ \ //: Delete the character s/SP \(.. \) // g: Delete the character "SP" and any two subsequent characters, "SPLLY" --> "Y" ^ #/: match any row starting with '#'/} ^/: match any row ending with '}' (no space)/} * ^ /: match any row that ends with zero or multiple spaces after '}'/[abc]/: match any row containing lower case 'A', 'B', or 'C'/^ [abc]/: match any row starting with 'A', 'B', or 'C'

Location parameters:
By default, the commands used in the sed editor are applied to all text data rows.
If you only want to apply a command to a specific data line or a group of text data lines, you must use line addressing.
The row addressing uses addressing to locate the row you want to edit. It is determined by data, regular expressions, or the combination of the two. The format of row addressing is as follows:

X indicates the specified row number $ the last row x, y specifies the row number range from x to y/pattern/query the rows in the contain mode/pattern/query the rows in the two modes/pattern /, the Rows x,/pattern/From the matching row of pattern to the matching row of pattern, x, y! Rows that do not contain the x and y row numbers

Number of address parameters:
When there is only one address parameter, it indicates that only the data row that meets the location parameter is edited.
When there are two address parameters, such as x and y, it indicates editing the row number range data area from x to y. (Including x and y)


Quotation Mark syntax
1. Under single quotes, $ and quotation marks are not interpreted and executed. That is, the two months are treated as common characters.
2. In double quotation marks, the dollar sign is expanded as a variable or parameter value. The command in the quotation marks is executed and the output result replaces the content in the quotation marks.
Therefore, you must use single quotes ''and double quotation marks" "when using variables ""
When using variables: # sed "/$ hello/d" dodo

Sed exit status:
1. No matter whether the specified mode is found or not, the exit status is 0.
2. When the command has a syntax error, the sed exit status is not 0.

Escape:
If you need to use the slash "/", You need to first transfer it, the escape generally has the following two types
1: [/]
2 :\/


Use multiple statements for a single row
1. Use a semicolon between commands
# Sed-n' =; P' dodo

2. Use braces
# Sed-n'3 {
> =
> P} 'dodo

3. Use the-e Parameter
# Sed-n-e' = '-e' P' dodo

4. Use the script file and then use-f to reference
# Sed-f scirpt dodo

5. You can use the prompt ''In bash shell''
# Sed-I'
> S/boy/girl/
> S/hello/byebye/'data

Other sed usage notes:
1. if the address is not used, the command will apply to all rows.
2. By default, only the text that appears for the first time in each row will be replaced. If multiple lines need to be replaced, replace the tag
3. to locate a special character, you must use "\" or "[]" to block its special meaning.


====================================== Script ====================== ==============
Script: Sed script is a column of sed commands written in the file. When Sed is started, the file name of the script is guided by the-f option.

Script Mode:
When executing the script, sed first copies the first line of the input file to the mode space and then executes all the commands in the script. After a row is processed, sed copies the next row of the file to the mode space and runs all the commands in the script. Until the last line.

Script note:
1. In the script, there cannot be any blank or text at the end of the command
2. If there are multiple commands in one line, separate them with semicolons.
3. Comments rows starting with # And do not span rows.


Script format:
#! /Bin/sed-f
Command1
Command2


Script instance:
# Cat test. sed
#! /Bin/sed-f
/Hello/\
Hi boy doiido
2i \
You are great

Add permissions and execute
# Chmod u + x test. sed
#./Test. sed dodo