Shell Scripting Learning Guide [third to fourth] find and replace, text processing tools

Summary: Chapter Three discusses the two basic operations that are commonly used when writing shell scripts. The fourth chapter introduces about 30 useful tools for processing text files.

Chapter III Find and replace

Summary: This chapter discusses the two basic operations that are commonly used when writing shell scripts: text lookup, text substitution.

3.1 Finding text

To remove a specific line of text from the input data file, the main tool is the GREP program. POSIX employs three different grep variants: grep, Egrep, Fgrep are consolidated into a single version, with different options to provide these three behavioral patterns respectively.
W.H.O. | Grep-f Root
Use the-f option above to find the fixed string root. In fact, as long as the matching pattern does not contain the meta character of the regular expression, the grep default behavior pattern is equivalent to the use of-F.

3.2 Regular Expressions

There are two types of POSIX regular expressions, basic regular Expressions (BRE), and extended regular Expressions (ERE).
SED is the primary tool for handling simple string substitution.
In some programming languages, a single string may contain several lines, in which case the ^ and $ refer to the beginning and end of the string.
Regular expression Matching example:
Expression Matching
Tolstoy 7 letters at any position on a line: Tolstoy
^tolstoy 7 Letters Tolstoy, appearing at the beginning of a line
tolstoy$ 7 Letters Tolstoy, appearing at the end of a line
^tolstoy$ exactly includes the Tolstoy of the 7-letter line, with no other characters
[Tt]olstoy on a line of any rank, containing Tolstoy or Tolstoy
Tol.toy in any row of the top, containing the Tol 3 letters, plus any one character, and then toy the three letters
Tol.*toy on a line of any rank, containing the Tol 3 letters, plus any 0 or more characters, and then continue toy these three letters

Interval-expression matching rules:
\{n\} The result of a pre-regular expression is reproduced n times
\{n,\} Pre-regex results are reproduced at least n times
\{n,m\} The result of the pre-regex expression is reproduced n to M times

In the S command, the end of G is the global. Such as:
$ echo Tolstoy reads well. Tolstoy writes well. >example.txt Input Sample
$ sed ' s/tolstoy/camus/' < example.txt not set G
Camus reads well. Tolstoy writes well.
$ sed ' s/tolstoy/camus/g ' < example.txt set ' G '
Camus reads well. Camus writes well.
$sed ' S/TOLSTOY/CAMUS/2 ' < example.txt replace only the second match
Tolstoy reads well. Camus writes well.

Sed-n ' 10,42p ' foo.xml print only 10-42 lines

3.3 Field Handling

The Cut command is used to cut down a selected character range or field, and a join is a file that combines a field with a common key value in the record.
Awk is mostly used for simple "command-line programs," Where awk comes in handy when you want to display only the selected fields, or rearrange the order of the fields in the row.

awk ' {print '} ' file prints the first field
awk ' {print $2,$5} ' file prints 2nd and 5th fields
awk ' {print $, $NF} ' file prints first and last field
awk ' nf>0 {print $} ' file prints a non-blank line
awk ' nf>0 ' file ibid.

Fourth Chapter Text Processing tool

Summary: This chapter provides a total of about 30 useful tools for working with text files. They are very powerful tool groups that can be used to write shell scripts.

4.1 Sort Text

SORT-T:-k1,1 file is sorted by the first field separated by a colon
SORT-T:-k2nr,2 file starts at field 2, reverses sort by numeric type and ends at end of Field 3
SORT-T:-k2n-k3n file is sorted by the second column, sorted in the third column
SORT-T: Unique record matching-k2n-u file output key Value field

4.2 Delete Duplicates

Sort file |uniq displays a unique sorted record
Sort file |uniq-c count unique sorted records
Sort file |uniq-d Show only duplicate records
Sort file |uniq-u show only records that are not duplicated

4.3 Reformatting a paragraph

Fmt-s-W file longer than 10 lines formatted as short rows

4.4 Count the number of rows, words, and characters

echo Hello Wrold |WC count the number of rows, words, and bytes
echo Hello World |wc-c Count bytes
echo Hello World |wc-l count rows
echo Hello World |wc-w count words
WC file1 File2 calculates data from two files

4.5 Print 4.6 Extract start or end lines

Head-n first n rows of file output files
Tail-n n rows of file output files

Shell Scripting Learning Guide [third to fourth] find and replace, text processing tools