How to Use GNU sed on Linux

How to Use GNU sed on Linux
GuideThe Linux Foundation announced a brand new LFCS (Linux Foundation Certified System Administrator (Linux Foundation Certified Sysadmin) certification program. This program aims to help people all over the world get their certifications for processing Linux system management tasks. These capabilities include supporting system services and first-hand fault diagnosis and analysis, as well as providing informed decisions for the engineer team during upgrade.

Process text streams in Linux

In Linux, the input and output in the program are treated as the character stream or character sequence. Before starting to understand redirection and pipelines, we must first understand the three most important I/O (Input and Output) streams. In fact, they are all special files (according to the conventions in UNIX and Linux, data streams and peripheral devices (Device Files) are also considered common files ).

The difference between> (redirection operator) and | (pipeline operator) is that the former connects the command with the file, while the latter connects the command output with another command.

# command >file# command1 | command2

Since the redirection operator silently creates or overwrites a file, we must use it with caution and never confuse it with the pipeline. The advantage of pipelines in Linux and UNIX systems is that the output of the first command is directly read by the second command instead of writing a file.

In the following operation exercises, we will use this poem, A happy child (unknown to the author)

Use sed

Sed is the abbreviation of stream editor. For those who do not understand the terminology, the stream editor is a tool used to perform basic text conversion in an input stream (input in a file or pipeline.

The most basic usage of sed is character replacement. We will rewrite each lowercase y to uppercase Y and redirect the output to ahappychild2.txt. The g sign indicates that sed should replace all instances in each row of the file. If this flag is omitted, sed will only replace the instance that appears for the first time in each row.

Basic Syntax:

#sed's/term/replacement/flag'file

Example:

#sed's/y/Y/g' ahappychild.txt > ahappychild2.txt

If you want to search for or replace special characters (such as/, \, &) in the replacement text, you need to use a backslash to escape it.

For example, we use a symbol to replace a text, and we will replace the first I that appears at the beginning of a line with You.

#sed's/and/\&/g;s/^I/You/g' ahappychild.txt

In the preceding command, we all know that ^ (insert symbol) is a symbol used in a regular expression to indicate the beginning of a line.

As you can see, we can use semicolons to separate and enclose two or more replacement commands (and use regular expressions in them.

Another use of sed is to display or delete a part of a file. In the following example, the first five lines in/var/log/messages are displayed starting from January 1, June 8.

#sed-n '/^Jun 8/ p'/var/log/messages |sed-n 1,5p

Note that by default, sed prints each row. We can use the-n option to overwrite this line and tell sed to print (expressed in p) files (or pipelines) (specify the line starting with "Jun 8" in the first command, and specify one to five rows in the second command ).

Finally, it may be useful to retain the file itself and delete comments when checking the script or configuration file. The following single-line sed command deletes (d) Empty rows OR a line starting with # (| a Boolean OR operation is performed on two regular expressions ).

#sed'/^#\|^$/d' apache2.conf

Uniq command

The uniq command allows us to return or delete repeated lines in the file, which is written to the standard output by default. We must note that the uniq command will not delete two duplicate lines unless they are adjacent. Therefore, uniq is often used with a prefix sort command (an algorithm used to sort text rows. By default, sort uses the first field (separated by spaces) as the key field. To specify a different keyword segment, we need to use the-k option.

Example

The du-sch/path/to/directory/* command returns the disk space usage of each sub-folder and file in the specified directory in human readable format (each directory is also displayed) overall ), in addition, the output is not based on the size, but by the sub-folder and file name. We can use the following command to sort it by size.

#du-sch /var/* | sort -h

You can use the following command to tell uniq to compare the first 6 characters (-w 6) of each line (this is the specified date) to count the number of log events, and the number of times (-c) appears at the beginning of each line ).

#cat/var/log/mail.log |uniq-c -w6

Grep command

Grep searches for the specified Regular Expression in the file (or command output) and outputs matched rows in the standard output.

Example

Displays gacanepa information in the/etc/passwd file, regardless of case.

#grep-i gacanepa /etc/passwd

Display the contents starting with all rc and following any number in the/etc folder.

#ls-l /etc |grep rc[0-9]

Tr command usage tips

The tr command can be used to convert (Change) or delete characters from the standard input and write the results to the standard output.

Example

Change all lowercase letters in the sortuniq.txt file to uppercase letters.

#cat sortuniq.txt |tr[:lower:][:upper:]

The delimiter In the ls-l output is compressed into a space.

#ls-l |tr-s ' '

How to Use the cut command

The cut command can extract partial input (from standard input or file) based on byte (-B option), character (-c), or field (-f) and output the result to the standard output. In the last case (based on fields), the default field separator is a tab, but different separators can be specified by the-d option.

Example

Extract the user account and the default shell allocated to them from/etc/passwd (the-d option allows us to specify the delimiter, and the-f option specifies the fields to be extracted ).

#cat/etc/passwd|cut-d:-f1,7

By combining the above commands, we will use the first and third non-empty files in the output of the last command to create a text stream. We will use grep as the first filter to check the user's gacanepa session, and then compress the separator to a space (tr-s ''). Next, we will use cut to extract the first and third fields. Finally, we will use the second field (in this example, it refers to the IP address) to sort and then use uniq to remove duplicates.

#last|grep gacanepa |tr-s ‘‘|cut-d’‘-f1,3|sort-k2 |uniq

The preceding command shows how to combine multiple commands and pipelines to obtain filtered data according to our requirements. You can also gradually use it to help you understand how output is transmitted from a command to the next command (by the way, this is a good learning experience !)

Summary

Although this example (and other examples in the current tutorial) may not seem very useful at first glance, but they are a good start to create, edit, and operate files in Linux Command lines. Please leave your questions and comments at any time-thank you very much!

From: http://www.linuxidc.com/Linux/2016-03/129543.htm

Address: http://www.linuxprobe.com/linux-gnu-sed.html