Sed use collation

-------------sed Find and replace----------------

Replace the text file from the

The work of many shell scripts starts with removing the required text from grep or egrep. The initial result of a regular expression lookup is often the "raw data" to be taken for further processing. In general, there is at least one thing to do with text substitution, which is to say that some words are replaced with another word, or that a portion of a matching line is deleted.

The correct program to perform text substitution should be the sed----flow editor.

SED is designed to be used for batch processing rather than interactive editing of files. When the medicine makes several changes, whether it is for one or several files, the simpler way is to write these changes to an editing script, and then apply this script to all the files that must be modified, the purpose of the SED exists here.

In the shell, SED is mainly used for some simple text substitution, so let's start with him first.

Basic usage: We often use sed in the middle of a pipeline to perform substitution operations by using the S command----to ask for regular expressions, replace matching text with alternate text, and optionally flags:

Sed ' s ':. *//'/etcpasswd | Delete all things after the first colon

Sort-u sort the list and remove the repeating section

The syntax of sed:

sed [-n] ' editing command ' [file ...]

sed [-n]-e ' editing command ' [file ...]

sed [-n]-f script-file ... [File ...]

Uses: sed can delete (delete), change, add (append), insert (insert), close, Exchange files in the data lines, or read other documents to the text > pieces, can also replace (substuite) their strings, or CONVERT (tranfer) One of the letters and so on. For example, the continuous blank line in a file is deleted into a line, the "local" string is replaced with "remote", "T" letter to "T", the 10th row of data and the 11th data is combined.

In sum, when the SED reads a line of data into the pattern space by the standard input, SED executes the editing of the data in the pattern space one by one in accordance with the edit instruction of sed script, and then sends the result in the pattern space to the standard output And then read the next line of data in. Perform this action until all the data has been read and done.

Summary, remember:

(1) Sed always handles input in line

(2) The SED does not deal with the original file but the copy of the original file.

Main parameters:

-E: Executes instructions on the command line, for example: Sed-e ' command ' file (s)

-F: Executes the instruction in an SED script file, for example: Sed-f scriptfile file (s)

-I: The difference between-E is that when you use-e, the sed execution instruction does not modify the contents of the original input file, it only appears in bash, and when you use the-I option, the command executed by SED directly modifies the original input file.

-N: Reads the next line to pattern space.

Behavior pattern:

Reads each line of the input file. If there is no file, then it is standard input. In each line, SED executes the esiting command for each application countdown. The results are written to standard output (by default, or explicitly using the P command and the-n option). Without the-e or-f option, SED considers the first parameter to be the editing command to use.

Find/home/tolstoy-type-d-print//Find all directories

Sed ' s;/home/tolstor;/home/lt/; '//modify name; Note: This uses semicolons as delimiters

Sed ' s/^/mkdir/'//Insert mkdir command

Sh-x '//execute in Shell trace mode

The script above is about preparing a copy of the/HOME/TOLSTOY directory structure under/home.lt (probably for backup)

Replacement case:

Sed replaces strings, data lines, and even data areas in files. Where the function parameter in the instruction that represents the replacement string is s; the function parameter in the instruction that replaces the row, or the data area > is C. The above situation is illustrated in the following three examples.

Substitution of rows

Sed-e ' 1c/#!/bin/more ' file (replace the first line with #!/bin/more)

Think: Replace the nth line with just do it

Sed-e ' nc/just do it ' file

Sed-e ' 1,10c/i can do it ' file (replace 1 to 10 lines in one line: I can do it)

Think: Change to two lines (I can do it! Let ' s start)

Sed-e ' 1,10c/i can do it!/nlet ' "/'" ' s start ' file

Substitution of characters

$ sed ' s/test/mytest/g ' example-----Replace test with mytest in the entire row range. If there is no G tag, only the first matching test of each row is replaced with mytest.

$ Sed-n ' s/^test/mytest/p ' example-----(-N) option and the P flag are used together to indicate that only those rows that have substitution occur are printed. That is, if the test at the beginning of a line is replaced with mytest, it is printed.

$ sed ' s/^192.168.0.1/&localhost/' example-----& symbol represents the part that is found in the replacement string. All lines starting with 192.168.0.1 will be replaced with a self-added localhost, which becomes 192.168.0.1localhost.

$ Sed-n ' s/loveable/\1rs/p ' example-----Love is marked as 1, all loveable will be replaced with lovers, and the replacement guild is printed.

$ sed ' s#10#100#g ' example-----no matter what character, followed by the S command is considered a new delimiter, so, "#" Here is the delimiter, instead of the default "/" delimiter. means to replace all 10 with 100.

Replace and find

In the S command, the end of G is: global, meaning "alternative text replaces every match in the regular expression." If the gsed command is not set to replace the first match.

Little-known: You can specify a number at the end, only the nth match appears to be replaced:

Sed ' S/TOM/LISY/2 ' < Test.txt matches only the second Tom

The SED can accept multiple commands by adding an-e option to sed.

Sed-e ' s/foo/bar/g '-e ' s/chicken/cow/g ' myfile.txt 1>myfile2.txt

Replace the 2nd "filter" of the 第3-5 line in the Test.log file with the "haha" with the shell command

Sed-i ' 3,5S/FILTER/HAHA/2 ' Test.log

The function parameters of all SED functions (Editing command) are described below.

= Prints the number of rows of data.

A: Add data entered by the user.

The B:label will execute the instruction to jump to the reference location established by:.

C: Replace the information with the information entered by the user.

D: Delete data.

D: Delete the data before the first newline letter in the pattern space.

G: Copy data from Hold space.

G: Add data from hold space to pattern space.

H: Copy data from pattern space to hold space.

H: Add data from pattern space to hold space.

L: The nonprinting character in the L data is printed with ASCII code.

I: Insert a row to add the user input.

N: Read in the next information.

N: Add the next piece of information to the pattern space.

P: Print out the data.

P: Prints the data before the first newline letter in the pattern space.

Q: Jump out of sed edit.

R: Reads into its contents.

S: replace String.

W: Write the data to its file.

x: Swap hold space and pattern space content.

Y: Convert (transform) characters.

Accurate control of---sed regularization----------

How much text will be changed

When we use sed we look at two questions: The first question is how many people will match the text, and the second question is where to start the match.

The answer is: the regular expression can match the longest, leftmost substring in the input text of the entire expression. In addition, a matching empty (null) string is considered to be longer than a complete mismatch.

Echo Syx is a good body | Sed ' s/syx/zsf/' uses fixed strings

SED can use complete regular expressions. But should know the importance of "from the longest leftmost" rule.

echo Tolstoy is worldly | Sed ' s/t.*y/camus/'

Camus

Obviously, we just want to match the Tolstoy, but this happens because the match expands to the maximum possible length of text.

This requires us to define precisely:

echo Tolstoy is worldly | Sed ' s/t[[:alpha:]]*y/camus/'

Camus is worldly

In text lookup, a call may match to a null string, and text substitution is also allowed when a text override is performed.

EHCO ABC | Sed ' s/b*/l/'

Labc

EHCO ABC | Sed ' s/b*/l/g '

Lalcl

Notice how B*shi matches the preceding and trailing null strings of ABC.

------Line and string-------

Line V.s. String

Most of the recommended programs are rows that process input data, in which case no inline newline characters appear in the data that will be matched, and ^ and $ represent the beginning and end of the line, respectively.

Many applications combine data as a bit record with fields. A record refers to a collection of related information, such as an enterprise in which records may contain data about customers, suppliers and employees, and, in the case of schools, student data. The field refers to the part of the record, such as name or street address.

Linux encourages the use of text-type data, so the most common type of data storage on the system is text, under which one line represents a day's record. The two conventions for splitting fields within a row are:

First: The Space or TAB key (tab).

Name Sex

Syx M

The second is to use a specific delimiter to split a field, such as a colon

Name:sex

Syx:m

There are pros and cons in both ways. The most obvious difference is when dealing with multiple successive repeats. When using whitespace separation, it is common for multiple consecutive occurrences of spaces or nonalphanumeric to be treated as a delimiter. beneficence, if you use special characters separated, each delimiter will be separated by a field.

The best example of dividing a field with delimiters is/etc/passwd, in which a line identifies a user of the system, each of which is separated by a colon.

Syx5:x:511:513::/home/syx5:/bin/bash.

The file contains seven fields with the following meanings:

1: User Name

2: Encrypted password (if the account is inactive, here is an asterisk, or if the encrypted password file is stored in another/etc/shadow, this may be another character)

3: ID number of the user

4: ID number of the user group

5: User's name, sometimes recover other relevant data (telephone number, office number)

6: Root directory

7: The Landing shell

-----------SED Case Study------------------

Use Case for SED

Using SED to operate the/etc/passwd, it is best to copy (CP/ETC/PASSWD/TMP), operate the passwd under TMP (not really, because in general, SED only modifies the output and does not change the file itself unless required).

Add/Remove as behavior unit

Requirements: List The contents of the/etc/passwd and print the line number, while deleting the 2~5 line.

Practice: CAT/ETC/PASSWD | Sed ' 2,5d '

Sed action is ' 2,5d ' (action needs to be placed between single quotes)

The NL command is used in a Linux system to calculate the line number of a file. NL can automatically add a line number to the output file content! The default result is a bit different from Cat-n, NL can make the line number more display design, including the number of digits and whether auto-completion 0 and so on.

Delete only the second line:

nl/etc/passwd | Sed ' 2d '

Delete Line 3rd to last row

Cat-n/etc/passwd | Sed ' 3, $d '

Add "I am fine" after the second line (in the third row)

Cat-n/etc/passwd | Sed ' 2a I am fine '

If you want to precede the second line

nl/etc/passwd | Sed ' 2i I am fine '

If you want to add more than two lines, add two lines after the second line, such as "Hello" and "How is You"

nl/etc/passwd | SED ' 2a hello\

>how is You '

Each row must have a backslash () to add the new row, so the above example, we can find that the last side of the first line has \ exist.

(Again, you'll need to note that single quotes don't end together.)

With the substitution and reality of the behavioral Unit

Replace the contents of the 2–5 line with "hahaha"

nl/etc/passwd | Sed ' 2,5c hahaha '

With this method, we can replace the entire row of data.

List only 5-7 lines of the/etc/passwd file

Cat-n/etc/passwd | Sed-n ' 5,7p '

It is possible to select certain line numbers within a file to display through the display function of the SED in the behavior unit.

Search and display of data

Search for lines in/etc/passwd for/root keywords

nl/etc/passwd | Sed '/root/p '

Think: Why would I output all the rows?

When you use-N, only the rows that contain the template are printed.

nl/etc/passwd | Sed-n '/root/p '

Search and deletion of data

Delete/etc/passwd all rows containing root, other rows output

nl/etc/passwd | Sed '/root/d '

Search for data and execute commands

Search for/etc/passwd, locate the root row, execute a set of commands in parentheses, and separate each command with a semicolon, where bash is replaced with Blueshell, and then the line is output:

nl/etc/passwd | Sed-n '/root/{s/bash/blueshell/;p} '

If you only replace/etc/passwd's first bash keyword as Blueshell, exit

nl/etc/passwd | Sed-n '/bash/{s/bash/blueshell/;p; q} ' 1

The last Q is the exit.

Search for and replace data

In addition to the entire row of processing modes, SED can also be used to search and replace part of the data with the behavioral units. Basically, the search and replace of SED is similar to that of VI.

Sed ' s/to be replaced by string/new string/g '

First view the IP address of this machine through/sbin/ifconfig eth0, mine is (192.168.199.5)

Remove the front portion of the IP

/sbin/ifconfig eth0 | grep ' inet addr ' |sed ' s/^.*addr://g '

Remove the back part of the IP

/sbin/ifconfig eth0 | grep ' inet addr ' |sed ' s/^.*addr://g ' | Sed ' s/bcast.*$//g '

You can get IP

Multi-point editing

An sed command that removes data from the third line to the end of/etc/passwd and replaces bash with hahaha.

nl/etc/passwd | Sed-e ' 3, $d '-e ' s/bash/hahaha/g '

Note: the-e option is added in front of the daily command

Modify file Contents directly

It is best not to use if you need to add a-i option

For example, insert hahaha on the last line

Sed use collation