Unix shell Learning Series-Tools

This is the first phase of the shell Learning Series and will be constantly updated...

Specific Learning books: Unix shell programming (Third edition)

This section describes some tools that are frequently used in shell programming.

 

1. Shell tool ---- Regular Expression

For regular expressions, the previous article describes most common applications.

 

2. ShellTool ---- cut

 

The cut command is used to analyze various data fields from data files or command output. The common format is:

Cut-cchars file

Chars specifies the text to be extracted from each row of the file. It can be a number representing the first character of each row:

Cut-c5 data takes the fifth character

Cut-c5, 13th data takes the fifth, 51st, and characters

Cut-c20-50 data takes characters between 20th and 50th, including themselves

Cut-c20-data takes 20th characters to the end of the line, and outputs the result to the standard output.

In the preceding example, commas and hyphens can be used together to retrieve multiple fields.

 

If the file parameter is not specified, data is read from the standard input, which means the command can be used as a pipeline filter.

 

-D and-f options of the cut command

When data is separated by a specific character, you can use these two options to extract the specified field. The command format is:

Cat-ddchar-ffields file

Dchar is used to specify a specific delimiter. fields is used to specify the number of the field to be analyzed (starting from 1). You can use commas and font size to specify multiple fields, the usage is the same as that specified in-c.

If no Delimiter is specified in the preceding format, the cut uses a tab as the default delimiter.

 

TIPS: if you do not know whether the characters are separated by tabs or spaces, you can use the following command to observe. If the characters are separated by tabs, it will explicitly output the tab to \ t:

Sed-n l filename

 

 

3. shellTool-paste

 

Paste is used to combine multiple rows, which is similar to the cut function. The general format is:

Paste files

Files are separated by spaces. The corresponding lines in all files specified by files form a row and then write them to the standard output. You can use a hyphen (-) to obtain the input from the standard input;

 

-D option

By default, when a row is formed, the row content of different files is separated by tabs. You can use the-d option to specify the delimiter, for example, paste-d '+ 'names address.

-S option

Use the-s option to tell paste to stick all rows in the same file together, instead of from other files. You can also use the-d option to specify the delimiter.

 

 

4. Shell tool-sed

 

Sed (stream editor) refers to the stream editor, which is used to edit data. The command format is:

Sed command file

Command is the same as ed command for all lines of file. If no file is specified, the standard input is used as the processing object, after the command is executed, the processing result is written to the standard output (not modified on the original file ).

 

Example of replacement:

Sed's/Unix/UNIX/'intro // Replace the first Unix of each line in the file intro with UNIX

Sed's/Unix/UNIX/G' intro // replace all strings in a row by adding the global option g behind the s command

 

Note that the commands of the sed tool are enclosed in single quotes.

 

-NOptionTell sed that no row is displayed unless explicitly stated,PCommandSpecify the rows to display

Sed-n'1, 2p 'intro // only display the first two rows

Sed-n'/UNIX/P' intro // display UNIX-containing rows

 

DCommandDeletes the entire line of text.

Sed '1, 2d 'intro // Delete the first and second rows

Sed '/UNIX/d' intro // delete all rows containing UNIX

Sed writes all input rows to the standard output by default. Therefore, the preceding command will delete the remaining rows of the specified row.

 

Examples of other powerful sed functions:

Sed '5d '// display 5th rows

Sed '/[Tt] est/d' // display rows containing Text/text

Sed-n '20, 25p' text // display line 20-25

Sed '1, 10 s/unix/UNIX/G' intro // replace all unix with UNIX in line 1-10

Sed '/jan/s/-1/-5/' // replace-1 in all rows containing jan with-5

Sed's /... // 'Data // Delete the first three characters of each line

Sed's /... $ // 'Data // Delete the three characters ending with each line

Sed-n 'l' text // display all rows of text, display all non-printable characters as \ nn (octal //), and display the tab as \ t (Not quite familiar? Explanations)

 

ShellTool ------- tr

 

The tr tool is used to convert a single character from a standard input. The general format is tr from-chars to-chars.

Here, from-chars and to-chars are one or more characters. The conversion result is written to the standard output, and the original file remains unchanged.

 

Cut-d:-f1, 6/etc/passwd | tr :''

The preceding command First extracts the first and sixth fields separated by ":" in the passwd file, and then replaces the delimiter colon with a tab by using the tr tool.

 

-SOptionCompress the repeated characters in tochars.

-DOptionDeletes a specified character from the input stream. There is only one parameter.

Fromchars and tochars can also be multiple characters, but the number of characters must be the same, to replace the matching, for example, tr 'a-Z' converts all lowercase letters to uppercase letters, tr '[A-Z] ''[N-ZA-M]' replace A-M with N-Z replace N-Z with A-M

 

 

ShellTool ------- grep

 

The grep command can search for a specific string mode from one or more files. The general format is grep pattern files.

After the command is executed, all lines of the pattern-compliant string are output to the standard output. If multiple files are specified for grep, the file name is displayed before each row, if the specified file does not contain a string that matches this mode, nothing is output. If no input file is specified, grep obtains the input from the standard input.

 

It is best to enclose the pattern in a pair of single quotes to protect it from shell interference.

 

-IOptionSpecifies whether to ignore uppercase or lowercase letters in the matching mode. The regular expression can be used in the mode.

-VOptionOutput all rows in the specified file that do not match the pattern

-LOptionWhen this option is used, only the file names that contain the specified mode are output.

-NOptionWhen this mode is used, not only the row content that matches the mode is output, but also the relative row number of the row in the file is added before each row.

 

ShellTool ------ sort

 

By default, sort extracts each row in a specified file and sorts them in ascending order. special characters are sorted according to their internal encoding.

-UOption: Remove duplicate rows from the output.

-ROption: Reverse sort order, that is, sort by descending order

-OOption: If you want to write the sorting result to the original file, the file will be damaged if you use output redirection. The-o option completes the job.

-NOption: Indicates that the first field in the row is regarded as a numerical value and sorted by arithmetic method.

 

To specify the sorting by a specific field in the row, you can use the + dn option. d indicates the Arabic number.

Sort + 1n data // when sorting, skip the first field and sort by arithmetic method. Changing 1 to 5 indicates skipping the first five fields. fields are separated by spaces or tabs by default, if other delimiters are used, use the-t option to specify

-TOption: The characters followed by t are used as separators.

Not complete...