Use awk to filter strings in text or files

Use awk to filter strings in text or files

When we use a specific command in Unix/Linux to read or edit text from a string or file, we often need to filter out the output to get the desired part. In this case, the regular expression will be used.

What is a regular expression?

A regular expression can be defined as a string that represents several character sequences. One of its most important functions is that it allows you to filter the output of a command or file, edit a part of the text or configuration file, and so on.

Features of Regular ExpressionsA regular expression is composed of the following:

• Common characters, such as space, underline, A-Z, a-z, 0-9.
• Can be extended to common charactersMetacharacters, which include:
  • (.) It matches any single character except the line break.
  • (*) It matches zero or multiple characters adjacent to it.
  • [Character (s)] It matches any character specified by the character/character set. You can use hyphens (-) to represent the character range, for example, [a-f] and [1-5.
  • ^ It matches the beginning of a line in the file.
  • $ It matches the end of a row in the file.
  • /This is an escape character.

You must use a text filtering tool like awk to filter text. You can also regard awk as a programming language. However, since this Guide applies to awk, I will introduce it using a simple command line filtering tool.

The general syntax of awk is as follows:

# awk 'script' filename

'Script' is a set of commands that awk can understand and apply to filename.

It reads the given row in the file, copies the content of the row, and executes the script on the row. This process repeats on all rows in the file.

The content in the 'script' script is in the format of '/pattern/action', where pattern is a regular expression, action is the action that awk should perform when this mode is found in this row.

How to Use the awk filtering tool in Linux

In the following example, we will focus on the metacharacters discussed earlier.

A simple example of using awk:

In the following example, all rows in the file/etc/hosts are printed because no mode is specified.

# awk '//{print}' /etc/hosts

Awk prints all rows in the file

Use awk in combination

In the following example, the localhost mode is specified. Therefore, awk matches the rows in the/etc/hosts file with localhost.

# awk '/localhost/{print}' /etc/hosts 

Awk prints matching rows in the file

Use wildcard (.) In awk Mode (.)

In the following example, the symbol (.) matches strings containing loc, localhost, and localnet.

Here, the regular expression indicates matching.L a single character c.

# awk '/l.c/{print}' /etc/hosts

Use awk to print matching strings in files

Use characters in awk mode (*)

In the following example, strings containing localhost, localnet, lines, and capable are matched.

# awk '/l*c/{print}' /etc/localhost

Use awk to match strings in the file

You may also realize that (*) will try to match the longest match it may detect.

Let's take a look at this example. The regular expression t * t indicates matching strings starting with t and ending with t in the following line:

this is tecmint, where you get the best good tutorials, how to's, guides, tecmint. 

When you use the/t * t/mode, you will get the following possible results:

this is tthis is tecmintthis is tecmint, where you get tthis is tecmint, where you get the best good tthis is tecmint, where you get the best good tutorials, how tthis is tecmint, where you get the best good tutorials, how tos, guides, tthis is tecmint, where you get the best good tutorials, how tos, guides, tecmint

The wildcard (*) In/t * t/enables the awk to select the last matched item:

this is tecmint, where you get the best good tutorials, how to's, guides, tecmint

Use awk in combination with the set [character (s )]

Take the collection [al1] as an example. awk will match all strings containing characters a, l, or 1 in the/etc/hosts file.

# awk '/[al1]/{print}' /etc/hosts

Use awk to print matching characters in the file

The next example matches the header starting with K or k, followed by a T string:

# awk '/[Kk]T/{print}' /etc/hosts 

Use awk to print matching characters in the file

Specify characters in a range

Characters that awk can understand:

• [0-9] represents a separate number
• [A-z] represents a single lowercase letter.
• [A-Z] represents a separate capital letter
• [A-zA-Z] represents a separate letter
• [A-zA-Z 0-9] represents a separate letter or number

Let's take a look at the following example:

# awk '/[0-9]/{print}' /etc/hosts 

Use awk to print matching numbers in the file

In the above example, all rows in the file/etc/hosts contain at least one separate number [0-9].

Use awk with metacharacters (^)

In the following example, it matches all rows starting with the given pattern:

# awk '/^fe/{print}' /etc/hosts# awk '/^ff/{print}' /etc/hosts

Use awk to print the row matching the pattern

Use awk with metacharacters ($)

It will match all rows ending with a given pattern:

# awk '/ab$/{print}' /etc/hosts# awk '/ost$/{print}' /etc/hosts# awk '/rs$/{print}' /etc/hosts

Use awk to print the string matching the pattern

Use awk with escape characters (/)

It allows you to take the character after the escape character as the text, that is, to understand it as its literal meaning.

In the following example, the First Command prints all the lines in the file. In the second command, I want to match a line with $25.00, but I do not use escape characters, therefore, nothing is printed.

The third command is correct, because here an escape character is used to escape $ to identify it as '$' (rather than a metacharacter ).

# awk '//{print}' deals.txt# awk '/$25.00/{print}' deals.txt# awk '//$25.00/{print}' deals.txt

Use awk with escape characters

Summary

The above content is not all of the filtering tools used by the awk command. The above examples are the basic operations of the awk. In the following sections, I will further introduce how to use the advanced features of awk.

From: https://linux.cn/article-7586-1.html

Address: http://www.linuxprobe.com/awk-filter-string.html