Linux learning-advanced shell script programming (2) first recognized by sed, gawk, and sedgawk

Linux learning-advanced shell script programming (2) first recognized by sed, gawk, and sedgawk

Citation: Sometimes we need to automatically process text in text files without pulling out a fully armed interactive text editor. In this case, it is much easier to have a command line editor that can automatically format, insert, modify, or delete text elements.

Fortunately, the Linux system provides two editors, sed and gawk. Let's take a look at these two editors below.

1. sed Editor

Sed editor is called stream editor, which is the opposite of a normal interactive text editor. In a text editor (such as vim), you can use keyboard commands to interactively insert, delete, and replace text in data. The stream editor will edit the data stream based on a set of pre-provided rules before the editor processes the data.

It outputs the generated data to STDIN.

The format of the sed command is as follows:

sed options script file

Option options include:

• -E: add the specified command in the script to the running command;
• -F Add the specified command in file to the running command;
• -N: do not generate output for each command. Wait for the print command to output the output.

1.1 define the editor command in the command line

A simple command line test is as follows:

echo "this is my test " | sed 's/test/big test/

The output result is as follows:

====================
This is my big test

====================

For a text file data1, the content is as follows:
This is a lazy dog
This is a lazy dog
This is a lazy dog
This is a lazy dog
This is a lazy dog
This is a lazy dog
This is a lazy dog

Run the following command:

sed 's/dog/cat/' data1

The output is as follows:

====================
This is a lazy cat
This is a lazy cat
This is a lazy cat
......

====================

Take a closer look, we can see that the command is'Dog'Changed'Cat'Is it amazing? Yes, sed is so awesome.

1.2 use multiple editor commands on the command line

Run the following command on the command line:

sed -e 's/lazy/health/; s/dog/cat/' data1

The output is as follows:

======================
This is a health cat
This is a health cat
This is a health cat
......

====================

Let's look at another input example below.

Note: sed itself does not modify the data of text files. It only sends the modified data to STDOUT.

1.3 read the editor command from the file

Write the commandScript1File

s/lazy/health/s/dog/cat/

Use-fCommand to create a file:

sed -f script data1

The above results will be the same.

2. gawk Program

Gawk is the GNU version of the original awk program in Unix. In gawk programming language, you can do the following:

• Define variables to save data;
• Use Arithmetic and string operators to process data;
• Use structured programming concepts, such as if-then statements and loops, to add logic to data;
• Extract the data elements in the data file and place them in another order or format to generate a formatted report.

2.1gawk Command Format

gawk option program file

2.2 read the program script from the command line

gawk '{print "hello john!"}'

The above command will execute the program script in each line of text and output "hello john !".

2.3 use data character Variables

• $0 indicates the entire text line
• $1 indicates the first data field in the text line.
• $2 indicates the second data field in the text line
• $ N indicates the nth data field in the text line.

For example:

gawk '{print $1} ' data1

2.4 use multiple commands in a program

As follows:

echo "my name is none!" | gawk '{$4="null";print $0}'

2.5 read a file from a file

Like sed, use-f to specify the content of the script2 file:

{print $1 "'s home directory is " $6}

Run the following command on the command line:

gawk -F: -f script2 /etc/passwd

Shows the output result:

2.6 run before processing data

Use BEGIN, as shown below

gawk 'BEGIN{print "the program begining"} {print $0}' data2

Will execute the content specified by BEGIN.

2.7 run after processing data

Use END as follows:

gawk 'BEGIN{ print "the program begining"} {print $0} END{print "END of File"}' data2

References

[1] programming of Linux Command Line and shell scripts (second edition)

This topic is being updated on Linux...