Sed information and some examples

Article Title: sed information and some examples. Linux is a technology channel of the IT lab in China. Includes basic categories such as desktop applications, Linux system management, kernel research, embedded systems, and open source.
In this article series, Daniel Robbins will show you how to use the powerful (but often forgotten) UNIX stream editor sed. Sed is an ideal tool for editing files in batches or creating shell scripts in a very effective way to modify existing files.
Select Editor
There are many text editors available in the UNIX world. Think about vi, emacs, jed, and many other tools. We all have our favorite editors (as well as our favorite key combinations) that we have developed ). With a trusted Editor, we can easily handle any number of UNIX-related management or programming tasks.
Although the interactive editor is great, it has limitations. Although its interactive feature can be a strong point, it also has its shortcomings. Consider the situation where similar changes need to be made to a group of files. You may instinctively run your favorite editor and then manually execute a group of tedious, repetitive, and time-consuming editing tasks. However, there is a better way.
Go to sed
It would be great to automate the file editing process so that files can be edited in batches or even compile scripts that can make complex changes to existing files. Fortunately, there is a better method for this situation-This better method is called "sed ".
Sed is a lightweight stream editor that covers almost all UNIX platforms (including Linux. Sed has many good features. First, it is quite small, usually much smaller than your favorite scripting language. Secondly, because sed is a stream Editor, it can edit the data received from standard input such as MPs queue. Therefore, you do not need to store the data to be edited in files on the disk. Because data pipelines can be easily output to sed, it is easy to use sed as a long and complex pipeline in a powerful shell script. Try using your favorite editor.
GNU sed
Fortunately for Linux users, one of the best sed versions is GNU sed, and its current version is 3.02. Every Linux release has (or at least) GNU sed. GNU sed is popular not only because it can freely distribute its source code, but also because it happens to have many convenient and time-saving extensions to the POSIX sed standard. In addition, GNU does not have many restrictions on earlier versions of sed, such as row length restrictions-GNU can easily process rows of any length.
The latest GNU sed
When studying this article, I noticed that several online sed fans mentioned GNU sed 3.02a. The strange thing is that sed 3.02a cannot be found on ftp.gnu.org (for these links, see references), so I have to look elsewhere. I found it in/pub/sed of alpha.gnu.org. So I was happy to download, compile, and install it. A few minutes later, I found that the latest sed version is 3.02.80. The source code can be found next to the 3.02a source code on alpha.gnu.org. After installing GNU sed 3.02.80, I am fully prepared.
Correct sed
In this series, GNU sed 3.02.80 is used. In the upcoming series of subsequent articles, some (but few) Top-level examples cannot be used in GNU sed 3.02 or 3.02a. If you are not using GNU sed, The results may be different. Now, why not spend some time installing GNU sed 3.02.80? In this way, we can not only prepare for the rest of the series, but also use sed, which may be the best at present.
Sed example
Sed performs any number of user-specified editing operations ("commands") on the input data. Sed is based on rows, so the command is executed for each row in order. Sed then writes the result to the standard output (stdout) without modifying any input file. Let's take a look at some examples. The first few will be a bit strange, because I will use them to demonstrate how sed works, rather than executing any useful task. However, if you are new to sed, it is very important to understand them. The following is the first example:
$ Sed-e 'D'/etc/services
If you enter this command, no output is obtained. So what happened? In this example, use the edit command 'D' to call sed. Sed open the/etc/services file, read a row into its mode buffer, execute the edit command ("Delete row"), and then print the mode buffer (the buffer is empty ). It then repeats these steps for each row. This will not generate output, because the "d" command removes every line in the mode buffer!
In this example, pay attention to the following.
First,/etc/services is not modified at all. This is because sed only reads the file specified in the command line and uses it as an input-it does not try to modify the file.
The second thing to note is that sed is Row-oriented. The 'd' command does not simply tell sed to delete all input data at once. On the contrary, sed reads each row of/etc/services into the internal buffer zone called the mode buffer. Once a row is read into the mode buffer, it executes the 'D' command and prints the content of the mode buffer (in this example, there is no content ). I will show you how to use the address range to control the lines to which the command is applied-but if the address is not used, the command will be applied to all lines.
The third thing to note is to enclose the usage of the 'd' command in single quotes. It is a good habit to use single quotes to enclose the sed command. In this way, shell extension can be disabled.
Another sed example
① The following is an example of removing the first line of the/etc/services file from the output stream using sed:
$ Sed-e '1d '/etc/services | more
As you can see, this command is similar to the First 'D' command except for '1. If you guess '1' is the first line, you can guess it. Different from the use of 'D' in the first example, there is an optional numerical address before the use of 'D. By using the address, you can tell sed to edit only one or some specific rows.
Address range
② Now let's take a look at how to specify the address range. In this example, sed deletes rows 1st to 10 from the output:
$ Sed-e '1, 10d '/etc/services | more
When two addresses are separated by commas (,), sed applies the following commands to the range from the first address to the end of the second address. In this example, the 'D' command is applied to lines 1st to 10 (including these lines ). All other rows are ignored.
Addresses with rule expressions
③ Here is a more useful example. Suppose you want to view the content of the/etc/services file, but you are not interested in viewing the comments contained in it. As you know, you can place comments in the/etc/services file using lines starting. To avoid comments, we want sed to delete rows starting. The following are specific practices:
$ Sed-e '/^ #/D'/etc/services | more
Try this example to see what happened. You will notice that sed successfully completed the expected task. Now let's analyze the situation.
To understand the '/^ #/d' command, you must first analyze it. First, let's remove 'D' -- this is the same Delete-line command we used earlier. The newly added part is '/^ #/', which is a new rule expression address. The rule expression address is always enclosed by a slash. They specify a mode, and the command followed by the rule expression address will only apply to rows exactly matching the specific mode. Therefore, '/^ #/' is a rule expression. But what does it do? Obviously, it is time to review rule expressions.
Rule expression Review
Rule expressions can be used to represent patterns that may be found in the text. Have you used the '*' character in the shell command line? This method is similar to the rule expression, but not the same. The following are special characters that can be used in Rule expressions:
Character Description
Match with the beginning of the line
Match with the end of the row
Match any character
Match the zero or multiple occurrences of the previous character
[] Matches all characters in []
The best way to feel rule expressions is to look at several examples. All these examples will be accepted by sed as valid addresses, which appear on the left side of the command. The following are examples:
Rules
Expression description
// Match any row containing at least one character
/../Will match any line containing at least two characters
/^ #/Match any row starting '#'
/^ $/Will match all empty rows
/} ^/Will match any row ending with '}' (no space)
/} * ^/Will match any row ended with zero or multiple spaces after '}'
/[Abc]/will match any row containing lower case 'A', 'B', or 'C'
/^ [Abc]/will match any row starting with 'A', 'B', or 'C'
In these examples, you are encouraged to try a few. Take some time to familiarize yourself with rule expressions and then try several self-created rule expressions. You can use regexp as follows:
$ Sed-e '/regexp/D'/path/to/my/test/file | more
This causes sed to delete any matched rows. However, by telling sed to print the regexp match and delete the unmatched content, rather than the opposite method, it is more helpful to be familiar with rule expressions. Run the following command:
$ Sed-n-e '/regexp/P'/path/to/my/test/file | more
Note the new '-n' option, which tells sed not to do so unless the print mode space is explicitly required. You will also notice that we replaced the 'D' command with the 'p' command, which explicitly requires the sed print mode space as you guessed. In this way, only the matching part is printed.
More information about the address
So far, we have seen the row address, row range address, and regexp address. However, there are more possibilities. We can specify two rule expressions separated by commas. sed will start with the first line that matches the first rule expression and end with the line that matches the second rule expression (including this line). For example, the following command prints text blocks starting from the line containing "BEGIN" and ending with a line containing "END:
$ Sed-n-e '/BEGIN/,/END/P'/my/test/file | more
If "BEGIN" is not found, no data is printed. If "BEGIN" is found, but "END" is not found in all rows after this, all subsequent rows will be printed. This is because sed is stream-oriented and does not know whether "END" will appear ".
C source code example
If you only need to print the main () function in the C source file, enter:
$ Sed-n-e '/main [[: space:] * (/,/^}/P' sourcefile. c | more
This command has two rule expressions '/main [[: space:] * (/' and '/^}/', and a command 'P '. The first rule expression matches the string "main" with any number of spaces or tabulation keys and parentheses. This should match the beginning of the general ansi c main () statement.
The '[[: space:]' character class appears in this special rule expression. This is just a special keyword, which tells sed to match the TAB or space. If you want to, you can enter '[[: space:]' instead of '[', followed by a space letter, followed by-V, then enter the tabulation key letter and ']' -- Control-V to tell bash to insert