Shell Command learning notes

Shell Command learning notes

1. tr replacement characters

Simple Example: tr 'A' a' <file.txt

Replace A with. The tr command can only be used to replace a single character. If it is written:

Tr 'abc' <file.txt

It means to replace the characters A, B, and C with the characters a, B, and c respectively, instead of replacing the string ABC with abc. The sed command is used to replace the string, which will be introduced later.

2. grep string SEARCH

Simple Example: grep world file.txt

In the input file or the standard input, find the line containing the matching string (world in this example) and display the modified line.

The result after processing with grep is often the input for subsequent processing, so you can use a pipeline to connect:

Grep world file.txt | wc-l

Wc-lw.number of rows. The statement above indicates the number of rows containing the world string in the file.txt file.

The above command is equal to grep-c world file.txt

Match multiple conditions and the conditions are in the OR relationship:

Grep 'a \ | B 'file.txt

Find all rows containing a or B in the file.txt file.

If you want to display rows in unmatched mode, you only need to add the-v option.

 3. sed (Stream Editor) is generally used to replace text.

Sed edits a file in batches rather than interactions. sed does not modify the source file, but only sends the modification results to the standard output or write them to another file.

Simple Example: sed's/world/hadoop/'<file.txt

Replace the first world in each row with hadoop.

In the command, "S" starting with "quotation marks" indicates that the replacement is required. Sed has many purposes, but is generally used for text replacement.

Main options:

-E when multiple replicas are required, each replicas must start with-E: sed-E's/World/hadoop/'-E's/Hello/Hallo/' <file.txt

-F when the command is complex, you can write the SED command into a script file (the command in the file should not be put in single quotes). Then, when executing the SED command, you only need to use-F to point to the script file: sed-F sed.txt <file.txt

-N by default, sed will send the Modification result of the source file to the standard output (whether replaced or not). Option-N will disable this function. -N is generally used in combination with P, indicating that only the replaced rows are printed, and other rows are not printed: sed-n's/World/hadoop/P' <file.txt

The SED command can end with P and end with G: SED's/World/hadoop/G', indicating global replacement, that is, replace all the matched texts in each line, instead of simply replacing the first one.

The SED command ends with a number to replace the nth matching string of each line: SED's/World/hadoop/2'

Application:

Delete all blank lines in the text: sed '/^ $/d' file.txt

4. cut selected fields

Cut is used to cut down the specified fields in a text file. fields are strings separated by delimiters. Common separators include spaces, tabs, and colons.

Simple Example:

Cut-D:-F 1,5/etc/passwd

The 1 and 5 fields (the Field starts from 1 rather than 0) of the/etc/passwd file are cut, and the Delimiter is a colon.

5. sort sorting Tool

Sort regards each input line as a record. Each record consists of multiple fields separated by blank characters.

Simple Example:

Sort-t:-k1, 1/etc/passwd

Take: As the delimiter and use the first field as the sorting field to sort records in the file

You can specify the type of the sort key value field, for example:

Sort-t:-k1, 1nr/etc/passwd

N refers to comparing the sorting field as a value rather than a string. If n is specified, 2 is placed before 10; otherwise, 10 is placed before 2.

R refers to reverse order.

Other commonly used sort key value fields are of the following types:

B. Ignore the white space at the beginning.

F is case insensitive.

In addition, if k is followed by only one number, such as-k2, it means that the sort key value is the second field and all subsequent fields, not just the second field, if you only want to use the second field as the sort key value, use-k2, 2.

When multiple-k options are displayed, it means that the key value specified by the first k option is used for sorting, sort the records with the same key value of the first group by the second key value, and so on.

Finally, the sorting of sort is unstable.

 6. wc WordCount Tool

Wc is used for counting:

Wc-c file.txt

Wc-w file.txt count words

Wc-l file.txt

If no option is specified, the number of output rows, number of words, and number of bytes are in sequence.

7. Get the beginning or end of a file using head tail

First n rows of head-n file.txt output file

The last n rows of the output file of tail-n file.txt

The last n rows of the output file of tail-n-f file.txt are displayed when new data is added to the file (useful for viewing log files)

8. Duplicate Uniq Deletion

In the example, set the file file.txt:

a 1b 1c 2d 2e 3

The result of executing uniq-f 1 file.txt is

a 1c 2e 3

The result of executing uniq-u-f 1 file.txt is

E 3

9. split the file

Split-l 100 file.txt splits the file.txt file into several files according to the 100 behavior unit. The split file is named xaa, xab ,...

Split-B 200 k file.txt split the file file.txt in KB. The split file is named xaa, xab ,...

References:

[1] Shell script Learning Guide