Shell script learning notes (7)-sed and awk

Sed and awk

Background

When designing a script, you may need to modify the script, such as deleting or replacing some keywords. This method of dynamically modifying files during script execution is called Stream editing. A tool with stream editing capabilities is called stream editor ). Sed is a leader in this field and can complement bash's shortcomings. In addition, when executing a script, you may need to prepare a report to present various field information. Traditionally, it works perfectly with bash, not awk.

Regular Expression

Regular Expressions are the basic syntax for making up "styles", while "styles" are essential to use sed and awk. The same running mode of sed and awk is: as long as the data row conforms to the "style", the specified "operation" is executed on it ".

There are many articles about regular expressions. See http://www.cnblogs.com/KissKnife/archive/2008/03/23/1118423.html

Sed usage

Sed is a non-interactive stream editor that allows you to dynamically edit files. The sed operation mode is to compare each data row. If the style is met, the specified operation is executed.

Syntax

Sed 'style command 'file

"Style" uses a pair of // inclusive to indicate the meaning of search. You can also specify the range of data rows.

Sed does not change the file content. Sed is used to read the file content. After being edited by the stream, the result is displayed as a standard output. Therefore, if you want to store the sed processing result, you must use the redirection output to save the result as another file.

Usage

1. delete data rows in a certain range: sed '1, 4d 'datafl deletes data rows from 1st to 4, and the rest are displayed. D Indicates deletion.

2. Delete the data rows with "styles": sed '/La/d' datafl deletes the rows with La and the rest are displayed. // Search

Sed '/[0-9] \ {3 \}/d' datafl deletes a row with three digits, and the rest is displayed .. Application of Regular Expressions

3. delete data rows without "style": sed '/La /! D' datafl deletes the rows WITHOUT La and the rest are displayed.

4. display the data rows with "styles": sed '/La/P' datafl displays the rows with La. However, sed displays non-conforming data rows by default. Therefore, sed-n '/La/P' datafl-n is used to suppress sed display.

5 replace: sed-n's/La/Oo/P' datafl s indicates selection. The string contained in the first pair of // is the search target, the second pair // contains the replaced string. It will replace the La in the Data row with Oo, but this will only Replace the first existing La. To replace all, use: sed-n's/La/Oo/gp 'dafl

6. Use the style-compliant string sed-n's/\ (La \)/\ 1Oo/P' datafl to save the found La and use \ 1 to retrieve it and then use it. The result of execution is that if the data row contains La, the first generated La will be replaced with LaOo, and the data rows containing La will be displayed.

7. Find the data row that matches the style, and then perform the replacement operation: sed-n '/AAA/s/234/567/P' datafl find the row containing AAA, replace 234 with 567 sed-n'/AAA/,/DDD/s/B/567/P' datafl will contain AAA to those lines containing DDD, change B to 567.

Example

 

#! /Bin/bash

If [$ #-ne 1]; then
Echo "Usage: $0 on or $0 off"
Exit 1
Fi

OPT = $1

Case "$ OPT" in
[Oo] [Nn]) CMD = 'yes ';;
[Oo] [Ff] [Ff]) CMD = 'no ';;
*)
Echo 'option error: on or off'
Exit 1 ;;
Esac

VSFTP_conf = '/etc/vsftpd. conf'
TMP_file = "/tmp. $"

If [-e $ VSFTP_conf]; then
Sed s/^. anonymous_enable =. */anonymous_enable = $ CMD/$ VSFTP_conf> $ TMP_file
Mv-f $ TMP_file $ VSFTP_conf
Echo 'set successfully'
Else
Echo "file: $ VSFTP_conf does not exist"
Fi

Awk usage

Awk is a language that can process data and report when data is generated. The operation is to read the data file and treat each row of data as a record. Each record is divided into several fields by field delimiter, and then the values of each field are output.

Ps auxw | awk '{print $2 }'

Awk applies a "style {operation}" to each record. If the style is correct, the specified operation is executed. Style or operation, which can be omitted. If there is only a style, the data rows that match the style are displayed. If there is only an operation, this operation is performed on each data row.

The following are common formats of awk functions:

Awk "style" file: displays the rows that match the style

Awk '{action}' file: The operation in {} is performed on each row.

Awk 'style {operation} 'file: perform the operations in {} On the rows that match the style.

Awk usage:

1 awk '/La/' datafl displays data rows containing La

2 awk '{print $1, $2}' datafl displays 1st and 2nd fields in each row of datafl

3 awk '/La/{print $1, $2}' datafl displays 1st and 2nd fields of the Data row containing the keyword La

4 awk-F: '/^ ols3/{print $3, $4}'/etc/passwd use option-F to specify: As a separator, the uid and gid fields of the account ols3 are displayed.

5 awk-F: 'In in {OFS = "++"}/^ ols3/{print $1, $2, $3, $4, $5} '/etc/passwd uses: As the separator and ++ as the output field separator, display the 1-5 Field of the account ols3 in the begin {} region. Specify the awk to initialize and set OFS to "++ ". OFS is used to store the delimiter of output fields.