-------------sed Find and replace----------------
Replace the text file from the
The work of many shell scripts starts with removing the required text from grep or egrep. The initial result of a regular expression lookup is often the "raw data" to be taken for further processing. In general, there is at least one thing to do with text substitution, which is to say that some words are replaced with another word, or that a portion of a matching line is deleted.
The correct program to perform text substitution should be the sed----flow editor.
SED is designed to be used for batch processing rather than interactive editing of files. When the medicine makes several changes, whether it is for one or several files, the simpler way is to write these changes to an editing script, and then apply this script to all the files that must be modified, the purpose of the SED exists here.
In the shell, SED is mainly used for some simple text substitution, so let's start with him first.
Basic usage: We often use sed in the middle of a pipeline to perform substitution operations by using the S command----to ask for regular expressions, replace matching text with alternate text, and optionally flags:
Sed ' s ':. *//'/etcpasswd | Delete all things after the first colon
Sort-u sort the list and remove the repeating section
The syntax of sed:
sed [-n] ' editing command ' [file ...]
sed [-n]-e ' editing command ' [file ...]
sed [-n]-f script-file ... [File ...]
Uses: sed can delete (delete), change, add (append), insert (insert), close, Exchange files in the data lines, or read other documents to the text > pieces, can also replace (substuite) their strings, or CONVERT (tranfer) One of the letters and so on. For example, the continuous blank line in a file is deleted into a line, the "local" string is replaced with "remote", "T" letter to "T", the 10th row of data and the 11th data is combined.
In sum, when the SED reads a line of data into the pattern space by the standard input, SED executes the editing of the data in the pattern space one by one in accordance with the edit instruction of sed script, and then sends the result in the pattern space to the standard output And then read the next line of data in. Perform this action until all the data has been read and done.
Summary, remember:
(1) Sed always handles input in line
(2) The SED does not deal with the original file but the copy of the original file.
Main parameters:
-E: Executes instructions on the command line, for example: Sed-e ' command ' file (s)
-F: Executes the instruction in an SED script file, for example: Sed-f scriptfile file (s)
-I: The difference between-E is that when you use-e, the sed execution instruction does not modify the contents of the original input file, it only appears in bash, and when you use the-I option, the command executed by SED directly modifies the original input file.
-N: Reads the next line to pattern space.
Behavior pattern:
Reads each line of the input file. If there is no file, then it is standard input. In each line, SED executes the esiting command for each application countdown. The results are written to standard output (by default, or explicitly using the P command and the-n option). Without the-e or-f option, SED considers the first parameter to be the editing command to use.
Find/home/tolstoy-type-d-print//Find all directories
Sed ' s;/home/tolstor;/home/lt/; '//modify name; Note: This uses semicolons as delimiters
Sed ' s/^/mkdir/'//Insert mkdir command
Sh-x '//execute in Shell trace mode
The script above is about preparing a copy of the/HOME/TOLSTOY directory structure under/home.lt (probably for backup)
Replacement case:
Sed replaces strings, data lines, and even data areas in files. Where the function parameter in the instruction that represents the replacement string is s; the function parameter in the instruction that replaces the row, or the data area > is C. The above situation is illustrated in the following three examples.
Substitution of rows
Sed-e ' 1c/#!/bin/more ' file (replace the first line with #!/bin/more)
Think: Replace the nth line with just do it
Sed-e ' nc/just do it ' file
Sed-e ' 1,10c/i can do it ' file (replace 1 to 10 lines in one line: I can do it)
Think: Change to two lines (I can do it! Let ' s start)
Sed-e ' 1,10c/i can do it!/nlet ' "/'" ' s start ' file
Substitution of characters
$ sed ' s/test/mytest/g ' example-----Replace test with mytest in the entire row range. If there is no G tag, only the first matching test of each row is replaced with mytest.
$ Sed-n ' s/^test/mytest/p ' example-----(-N) option and the P flag are used together to indicate that only those rows that have substitution occur are printed. That is, if the test at the beginning of a line is replaced with mytest, it is printed.
$ sed ' s/^192.168.0.1/&localhost/' example-----& symbol represents the part that is found in the replacement string. All lines starting with 192.168.0.1 will be replaced with a self-added localhost, which becomes 192.168.0.1localhost.
$ Sed-n ' s/loveable/\1rs/p ' example-----Love is marked as 1, all loveable will be replaced with lovers, and the replacement guild is printed.
$ sed ' s#10#100#g ' example-----no matter what character, followed by the S command is considered a new delimiter, so, "#" Here is the delimiter, instead of the default "/" delimiter. means to replace all 10 with 100.
Replace and find
In the S command, the end of G is: global, meaning "alternative text replaces every match in the regular expression." If the gsed command is not set to replace the first match.
Little-known: You can specify a number at the end, only the nth match appears to be replaced:
Sed ' S/TOM/LISY/2 ' < Test.txt matches only the second Tom
The SED can accept multiple commands by adding an-e option to sed.
Sed-e ' s/foo/bar/g '-e ' s/chicken/cow/g ' myfile.txt 1>myfile2.txt
Replace the 2nd "filter" of the 第3-5 line in the Test.log file with the "haha" with the shell command
Sed-i ' 3,5S/FILTER/HAHA/2 ' Test.log
The function parameters of all SED functions (Editing command) are described below.
= Prints the number of rows of data.
A: Add data entered by the user.
The B:label will execute the instruction to jump to the reference location established by:.
C: Replace the information with the information entered by the user.
D: Delete data.
D: Delete the data before the first newline letter in the pattern space.
G: Copy data from Hold space.
G: Add data from hold space to pattern space.
H: Copy data from pattern space to hold space.
H: Add data from pattern space to hold space.
L: The nonprinting character in the L data is printed with ASCII code.
I: Insert a row to add the user input.
N: Read in the next information.
N: Add the next piece of information to the pattern space.
P: Print out the data.
P: Prints the data before the first newline letter in the pattern space.
Q: Jump out of sed edit.
R: Reads into its contents.
S: replace String.
W: Write the data to its file.
x: Swap hold space and pattern space content.
Y: Convert (transform) characters.
Accurate control of---sed regularization----------
How much text will be changed
When we use sed we look at two questions: The first question is how many people will match the text, and the second question is where to start the match.
The answer is: the regular expression can match the longest, leftmost substring in the input text of the entire expression. In addition, a matching empty (null) string is considered to be longer than a complete mismatch.
Echo Syx is a good body | Sed ' s/syx/zsf/' uses fixed strings
SED can use complete regular expressions. But should know the importance of "from the longest leftmost" rule.
echo Tolstoy is worldly | Sed ' s/t.*y/camus/'
Camus
Obviously, we just want to match the Tolstoy, but this happens because the match expands to the maximum possible length of text.
This requires us to define precisely:
echo Tolstoy is worldly | Sed ' s/t[[:alpha:]]*y/camus/'
Camus is worldly
In text lookup, a call may match to a null string, and text substitution is also allowed when a text override is performed.
EHCO ABC | Sed ' s/b*/l/'
Labc
EHCO ABC | Sed ' s/b*/l/g '
Lalcl
Notice how B*shi matches the preceding and trailing null strings of ABC.
------Line and string-------
Line V.s. String
Most of the recommended programs are rows that process input data, in which case no inline newline characters appear in the data that will be matched, and ^ and $ represent the beginning and end of the line, respectively.
Many applications combine data as a bit record with fields. A record refers to a collection of related information, such as an enterprise in which records may contain data about customers, suppliers and employees, and, in the case of schools, student data. The field refers to the part of the record, such as name or street address.
Linux encourages the use of text-type data, so the most common type of data storage on the system is text, under which one line represents a day's record. The two conventions for splitting fields within a row are:
First: The Space or TAB key (tab).
Name Sex
Syx M
The second is to use a specific delimiter to split a field, such as a colon
Name:sex
Syx:m
There are pros and cons in both ways. The most obvious difference is when dealing with multiple successive repeats. When using whitespace separation, it is common for multiple consecutive occurrences of spaces or nonalphanumeric to be treated as a delimiter. beneficence, if you use special characters separated, each delimiter will be separated by a field.
The best example of dividing a field with delimiters is/etc/passwd, in which a line identifies a user of the system, each of which is separated by a colon.
Syx5:x:511:513::/home/syx5:/bin/bash.
The file contains seven fields with the following meanings:
1: User Name
2: Encrypted password (if the account is inactive, here is an asterisk, or if the encrypted password file is stored in another/etc/shadow, this may be another character)
3: ID number of the user
4: ID number of the user group
5: User's name, sometimes recover other relevant data (telephone number, office number)
6: Root directory
7: The Landing shell
-----------SED Case Study------------------
Use Case for SED
Using SED to operate the/etc/passwd, it is best to copy (CP/ETC/PASSWD/TMP), operate the passwd under TMP (not really, because in general, SED only modifies the output and does not change the file itself unless required).
Add/Remove as behavior unit
Requirements: List The contents of the/etc/passwd and print the line number, while deleting the 2~5 line.
Practice: CAT/ETC/PASSWD | Sed ' 2,5d '
Sed action is ' 2,5d ' (action needs to be placed between single quotes)
The NL command is used in a Linux system to calculate the line number of a file. NL can automatically add a line number to the output file content! The default result is a bit different from Cat-n, NL can make the line number more display design, including the number of digits and whether auto-completion 0 and so on.
Delete only the second line:
nl/etc/passwd | Sed ' 2d '
Delete Line 3rd to last row
Cat-n/etc/passwd | Sed ' 3, $d '
Add "I am fine" after the second line (in the third row)
Cat-n/etc/passwd | Sed ' 2a I am fine '
If you want to precede the second line
nl/etc/passwd | Sed ' 2i I am fine '
If you want to add more than two lines, add two lines after the second line, such as "Hello" and "How is You"
nl/etc/passwd | SED ' 2a hello\
>how is You '
Each row must have a backslash () to add the new row, so the above example, we can find that the last side of the first line has \ exist.
(Again, you'll need to note that single quotes don't end together.)
With the substitution and reality of the behavioral Unit
Replace the contents of the 2–5 line with "hahaha"
nl/etc/passwd | Sed ' 2,5c hahaha '
With this method, we can replace the entire row of data.
List only 5-7 lines of the/etc/passwd file
Cat-n/etc/passwd | Sed-n ' 5,7p '
It is possible to select certain line numbers within a file to display through the display function of the SED in the behavior unit.
Search and display of data
Search for lines in/etc/passwd for/root keywords
nl/etc/passwd | Sed '/root/p '
Think: Why would I output all the rows?
When you use-N, only the rows that contain the template are printed.
nl/etc/passwd | Sed-n '/root/p '
Search and deletion of data
Delete/etc/passwd all rows containing root, other rows output
nl/etc/passwd | Sed '/root/d '
Search for data and execute commands
Search for/etc/passwd, locate the root row, execute a set of commands in parentheses, and separate each command with a semicolon, where bash is replaced with Blueshell, and then the line is output:
nl/etc/passwd | Sed-n '/root/{s/bash/blueshell/;p} '
If you only replace/etc/passwd's first bash keyword as Blueshell, exit
nl/etc/passwd | Sed-n '/bash/{s/bash/blueshell/;p; q} ' 1
The last Q is the exit.
Search for and replace data
In addition to the entire row of processing modes, SED can also be used to search and replace part of the data with the behavioral units. Basically, the search and replace of SED is similar to that of VI.
Sed ' s/to be replaced by string/new string/g '
First view the IP address of this machine through/sbin/ifconfig eth0, mine is (192.168.199.5)
Remove the front portion of the IP
/sbin/ifconfig eth0 | grep ' inet addr ' |sed ' s/^.*addr://g '
Remove the back part of the IP
/sbin/ifconfig eth0 | grep ' inet addr ' |sed ' s/^.*addr://g ' | Sed ' s/bcast.*$//g '
You can get IP
Multi-point editing
An sed command that removes data from the third line to the end of/etc/passwd and replaces bash with hahaha.
nl/etc/passwd | Sed-e ' 3, $d '-e ' s/bash/hahaha/g '
Note: the-e option is added in front of the daily command
Modify file Contents directly
It is best not to use if you need to add a-i option
For example, insert hahaha on the last line
Sed use collation