Details about the sed command of the text processing tool

======= The knowledge learned by the blogger comes from the hands-on Teacher Ma Ge ======

Marco education's "2014 summer camp" has begun !!! Mago education is currently the most cost-effective Linux training, ranking first in the country, and known as the "Whampoa Military Academy" in the Linux field by netizens. All the courses are explained using Centos6.5x86 _ 64, after summing up and refining several network classes, the course system has been gradually improved. The students' learning progress is monitored and the quality test system tests the students' mastery, active online Q & A sessions, accompanied by famous teachers, you can't miss it.
Details DASH: http://www.magedu.com/
Course: http://www.magedu.com/mentuqc
Network Class integration service: http://mageedu.blog.51cto.com/4265610/1379598

========================================================== ====

Details about the sed command of the text processing tool

Sed: Stream Editor text Stream editing. sed is a "non-interactive" Text Stream-oriented Editor. You can process multiple lines of content from multiple files at the same time. Instead of modifying the original file, you can input the entire file to the screen and the content that matches only the mode to the screen. You can also modify the original file, but will not return results on the screen.

Sed syntax format:

Sed command Format: sed [option] 'sed command 'filename

Sed script format: sed [option]-f'sed script' filename

Sed Command option ):

-N: print only the rows matching the pattern.

-E: directly edit the sed action in command line mode. This is the default option.

-F: Write the sed action in a file and use-f filename to execute the sed action in filename.

-R: supports extension expressions.

-I: directly modify the File Content

Sed query text in a file:

1) Use a row number, which can be a simple number or a row number range.

X	X is the row number.
X, y	Indicates that the row number ranges from x to y.
/Pattern	Query rows in the include Mode
/Pattern	Query rows that contain two modes
Pattern/, x	Query rows in the include mode on a given row number
X,/pattern/	Query matched rows by row number and mode
X, y!	Query rows that do not contain the specified row numbers x and y

2) use regular expressions and extended regular expressions (must be combined with the-r option)

^	The content at the beginning of the anchor line. The format is "^ pattern"
$	The content at the beginning of the anchor line in the format of pattern $"
^ $	Blank line
.	Match any single character
*	Match any times next to the preceding character (0, 1, multiple times)
.*	Match any character of any length
\?	Match 0 or 1 times next to the preceding character
\ {M, n \}	Match the first character at least m times, at most n times
\ {M ,\}	Match the first character at least m times
\ {M \}	Exact match of previous m times \ {0, n \}: 0 to n times
\ <	The beginning of the anchor ---- equivalent to \ B. Format: \ <pattern
\>	Anchor end, format: \> pattern
\ <Pattern \>	Word anchor
\(\)	Group, format: \ (pattern \), reference \ 1, \ 2
[]	Match any single character in the specified range
[^]	Matches any single character out of the specified range
[: Digit:]	All numbers, equivalent to 0-9, [0-9] ---> [[: digit:]
[: Lower:]	All lowercase letters
[: Upper:]	All uppercase letters
[: Alpha:]	All letters
[: Alnum:]	Equivalent to 0-9a-zA-Z
[: Space:]	White space characters
[: Punct:]	All punctuation marks

####### Regular Expressions supported by sed matching modes ###################### sed '5 Q' /etc/passwd # print the first five rows of sed-n'/r * t/P'/etc/passwd # print matching r with 0 or more, followed by a t character line sed-n '/. r. */P'/etc/passwd # print the row matching the r character and the column following the r character sed-n'/o */P'/etc/passwd # print the o character again arbitrary sed-n '/o \ {1, \}/P'/etc/passwd # print the o word to repeat the previous sed-n '/o \ {1, 3 \}/P'/etc/passwd # print the o word again more than once to three times

Sed command ):

P	Print matching rows (used with the-n option)
=	Show file row number
A \	Add new text information after locating the row number
I \	Insert new text information after locating the row number
D	Delete row
C \	Replace positioning text with new text
W filename	Write text to a file, similar to output redirection>
R filename	Read text from another file, similar to input redirection <
S	Replace the corresponding mode with the replacement mode.
Q	Exit or exit immediately after the first mode match is complete.
L	Display controllers equivalent to October Codes
{}	Separate the command groups executed on the target line with semicolons.
N	Read the next line of text from another file and process it from the next command instead of the first command.
N	Add the next row to the data stream to create a multi-row group for processing
G	Paste Mode 2 to/pattern n/
Y	Transfer Character, replacing a single character

Operations on files are nothing more than "add, delete, modify, and query". How to Use the sed command to add, delete, modify, and Query files is one of the necessary foundations for writing automated scripts.

Sed command to print file information (query ):

#### Example command for Printing file information with sed ###### sed-n'/^ #/! P'/etc/vsftpd. conf sed-n'/^ #/! {/^ $ /! P} '/etc/vsftpd. confsed-e '/^ #/d'-e'/^ $/D'/etc/vsftpd. confsed-n '1,/adm/P'/etc/passwdsed-n'/adm/, 6p'/etc/passwdsed-n'/adm /, 4p'/etc/passwdsed-n'/adm/, 2p '/etc/passwd ### explain and display the results of these sed command examples in slices

Sed command to add the file content: (if you add the source file, use the-I parameter ):

#### Add the sed command ####### 1) add the first line of the matched line to the same line #2) add a character to the row that matches the row #3) Add a character to the end of the row #4) Add a character to the front of the row that matches the row #5) add a line after the row matching the row #6) add a line to the first row of the file [root @ jie1 ~] # Sed-I '1 I \ sed command start 'myfile #7) append a line at the end of the object [root @ jie1 ~] # Sed-I '$ a \ sed command end' myfile

 

 

The sed command deletes the file content: (use the-I parameter to directly Delete the source file ):

Sed deletes a row of a file. If you want to delete a character in the row, replace it with another character (don't worry, replace it later, and replace it with sed, which is the most commonly used)

 Important:Sed command to replace the file content (replace is the most used operation in shell automation scripts)

#====================== Content in the source file ============================== ============== [root @ jie1 ~] # Cat testanonymous_enable = YESwrite_enable = YESlocal_umask = 022xferlog_enable = YESconnect_from_port_20 = YESroot: x: 0: root:/bin/bashbin: x: 1: 1: bin: /bin:/sbin/nologindaemon: x: 2: 2: daemon:/sbin/nologinadm: x: 3: 4: adm:/var/adm: /sbin/nologinlp: x: 4: 7: lp:/var/spool/lpd:/sbin/nologinDEVICE = "eth0" BOOTPROTO = "static" HWADDR = "00: 0C: 29: 90: 79: 78 "ONBOOT =" yes "IPADDR = 172.16.22.1NETMASK = 255.255.255.0.0 #============ ========================================================== ======================= [Root @ jie1 ~] # Sed-I '/DEVICE/c \ Ethernet' test # Replace the line matching the DEVICE with the line "Ethernet" [root @ jie1 ~] # Sed-I's/static/dhcp/'test # replace static with dhcp (//, @, # Are all the aforementioned address delimiters) [root @ jie1 ~] # Sed-I '/IPADDR/s @ 22 \. 1@10.12 @ 'test # match the IPADDR line, replace 22.1 with 10.12 because. [root @ jie1 ~] # Sed-I '/connect/s # YES # NO # 'test # match the connect Line and replace YES with NO [root @ jie1 ~] # Sed-I's/bin/tom/2g 'test # Replace bin with tom [root @ jie1 ~] after the second and second occurrence of all rows matching the bin # Sed-I's/daemon/jerry/2p 'test # Replace the daemon that appears for the second time in all rows matching the bin with jerry, and in the production line that matches the line [root @ jie1 ~] # Sed-I's/adm/boss/2' test # replace all the rows matching adm with the adm that only appears for the second time with the boss [root @ jie1 ~] # Sed-I '/root/{s/bash/nologin/; s/0/1/g}' test # match the root row and replace bash with nologin, replace 0 with 1 [root @ jie1 ~] # Sed-I's/root/(&)/G' test # enclose root in parentheses, & indicates referencing the matched character [root @ jie1 ~] # Sed-I's/BOOTPROTO/# BOOTPROTO/'test # Replace BOOTPROTO with # BOOTPROTO, which is generally used to comment out a line in the configuration file [root @ jie1 ~] # Sed-I's/ONBOOT/# &/'test # Add the # sign in front of the line matching the ONBOOT, and comment out a line in the configuration file [root @ jie1 ~] # Sed-I '/ONBOOT/s/# // 'test # matches the ONBOOT line and replaces # With null, that is, remove, it is also used to remove # comment #=========================the content displayed in the file after the above sed command is executed ========== ============== [root @ jie1 ~] # Cat testanonymous_enable = YESwrite_enable = YESlocal_umask = Success = YESconnect_from_port_20 = NO (root): x: 1: 1 :( root):/(root):/bin/nologinbin: x: 1: 1: tom:/stom/nologindaemon: x: 2: jerry:/sbin:/stom/nologindaemon: x: 2: jerry:/sbin: /stom/nologinadm: x: 3: 4: boss:/var/adm:/sbin/nologinlp: x: 4: 7: lp:/var/spool/lpd: /sbin/nologinEthernet # BOOTPROTO = "dhcp" HWADDR = "00: 0C: 29: 90: 79: 78" ONBOOT = "yes" IPADDR = 172.16.10.12NETMASK = 255.255.255.0.0

Sed references variables: (variables are often used in automated shell scripts)

First, you can change single quotation marks to double quotation marks if the sed command does not contain default variables;

Second, when the sed command contains default variables, the variables defined by yourself must be enclosed by single quotation marks, and the statements in sed must be quoted

[Root @ jie1 ~] # Cat> myfile <EOF> hello world> I am jie> how are you> EOF # A file [root @ jie1 ~] # Cat myfilehello worldi am jiehow are you [root @ jie1 ~] # Name = li # define a variable and assign a value to the variable [root @ jie1 ~] # Sed-I "s/jie/$ name/" myfile # Replace the jie-matched characters with the variable value [root @ jie1 ~] # Cat myfilehello worldi am lihow are you [root @ jie1 ~] # Sed-I "$ a $ name" myfile # When the sed command also has a default variable, the syntax error sed:-e expression #1 will occur when you reference your defined variable, char 3: extra characters after command [root @ jie1 ~] # Sed-I '$ a' $ name'' myfile # When referencing a custom variable, the sed statement must be quoted with a single reference, then, enclose the custom variables in single quotes [root @ jie1 ~]. # Cat myfilehello worldi am lihow are youli [root @ jie1 ~] #

Other advanced usage of sed:

1) write the content of the file being operated by sed to an exception file.

[Root @ jie1 ~] # Cat test # Ethernet # BOOTPROTO = "dhcp" HWADDR = "00: 0C: 29: 90: 79: 78 "ONBOOT =" yes "IPADDR = 172.16.10.12NETMASK = 255.255.0.0 [root @ jie1 ~] # Sed-I's/IPADDR/ip/w ip.txt 'test folder store the file operated by sed to another file, while the wdfile is saved, and the ip.txt file name is [root @ jie1 ~] # Cat ip.txt # view the content of the new file ip address = 172.16.10.12 [root @ jie1 ~] #

2) read a file to the file being operated by sed.

[Root @ jie1 ~] # Cat myfile # file content hello worldi am lihow are youli [root @ jie1 ~] # Cat test # Ethernet # BOOTPROTO = "dhcp" HWADDR = "00: 0C: 29: 90: 79: 78 "ONBOOT =" yes "IPADDR = 172.16.10.12NETMASK = 255.255.0.0 [root @ jie1 ~] # Sed-I '/Ethernet/r myfile' test # Read the content of another file in the row matching Ethernet, the content of the read file will be inserted into the line matching the Ethernet. [root @ jie1 ~] # Cat test # view the line Ethernethello worldi am lihow are youli # BOOTPROTO = "dhcp" HWADDR = "00: 0C: 29: 90: 79: 78 "ONBOOT =" yes "IPADDR = 172.16.10.12NETMASK = 255.255.0.0 [root @ jie1 ~] #

Classic example of sed:

#1) process the following file content, retrieve the domain name and sort it by count, for example, processing: Dig Domain Name 3 www. baidu. com2 post. baidu. com1 mp3.baidu.com [root @ localhost shell] # cat file | sed-e's/http: \/\ // '-e's /\/. * // '| sort | uniq-c | sort-rn3 www. baidu. com2 post. baidu. c Om1 mp3.baidu.com [root @ codfei4 shell] # awk-F/'{print $3}' file | sort-r | uniq-c | awk '{print $1 "\ t", $2} '3 www. baidu. com2 post. baidu. com1 mp3.baidu.com #2) use grep in combination with sed to retrieve the IP address of the NIC [root @ jie1 ~] # Ifconfig | grep-B1 "inet addr" | grep-v "\-" | sed-n-e 'N; s/\ (eth [0-9] \). * \ n. * addr: \ ([0-9] \ {1, 3 \}\. [0-9] \ {1, 3 \}\. [0-9] \ {1, 3 \}\. [0-9] \ {1, 3 \}\). */\ 1 \ 2/P'

Using sed is the basis for writing automated shell scripts. sed is also a very useful and important command and one of the text processing tools, the above is a simple usage of the sed command I learned and summarized by myself. sed also has more advanced usage and is still learning.