Text processing commands for Shell scripting Learning notes

Text processing commands for Shell scripting Learning notes

A. Cut (truncate text characters or field fields)
Command format: Cut [option] File
Combined use of 1.-d and-F to intercept fields
Description: * denotes delimiter, such as-,:, empty characters, etc.
n Denotes nth field
Usage One: cut-d*-fn to intercept a field from the text for each line that is delimited by a particular symbol
Example: Cut-d:-f1 intercept each line with a colon (:) The first field field as a delimiter from the text

Usage Two: cut-d*-fnm-n to intercept multiple field fields adjacent to each line from the text
Example: cut-d:-f1-3 to intercept multiple field fields with a colon (:) delimiter in each line from the text

Usage Three: cut-d*-fnm,n to intercept multiple field fields that are not adjacent to each row from the text
Example: cut-d:-f1,3 to intercept the first and third characters of each line from a text

2.-c usage to intercept characters
Usage One: CUT-CN to intercept a character from the text
Example: CUT-C1 to intercept the first character of each line from a text

Usage two: Cut-cnm-n to intercept multiple characters from the text adjacent to each line
Example: Cut-c1-5 the first to the fifth character of each line from a text

Usage Three: Cut-cnm,n to intercept multiple characters from text that are not adjacent to each line
Example: cut-c1,5 to intercept the first and fifth characters in each line of characters from a text

Attention:
The 1.cut command can only intercept characters or field fields in a single file
The characters or field fields that are intercepted by the 2.cut command are displayed as columns to the screen from all rows, rather than from a row or line, that is, the cut command does not have a specific character for a particular line or a specific field, and can only be intercepted from all lines in the text. The General Cut command is a text-processing command that is used for text that has some sort of order.

___________________________________________________________________________
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Two. Uniq (remove duplicate lines in text)
Command format: uniq [option] File
1.UNIQ-C Displays the number of consecutive repetitions of the same line in the text
Example 1:[[email protected] tmp]# uniq-c Job
2 GGG
2 JJJJJJJJJJJJ

Example 2:[[email protected] tmp]# uniq-c Job
2 GGG
2 JJJJJJJJJJJJ
1 GGG
1 JJJ

2.uniq-d display repeated lines in text (only once)
Example: [[email protected] tmp]# uniq-d Job
Ggg
Jjjjjjjjjjjj

3.UNIQ-U displays lines that do not have consecutive duplicates in the text
Example: [[email protected] tmp]# uniq-u Job
Ggg
Jjj

Attention:
It is important to understand that continuous repetition and repetition represent different meanings.

_______________________________________________________________________
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Three. Paste (combine data from different texts into new text)
Command format: Paste [option] file1 file2

1.paste file1 File2 Displays the contents of the File1 text as a single text and the contents of the File2 text as the first field field, the Delimiter is tab (n spaces), and file1 is displayed as the second Field field, and vice versa
Example: [[email protected] test]# paste passwd Shadow
1.root:x:0:0:root:/root:/bin/bash Root:$1$sovzvvql$8hosttnmubbedswfa3lni/:16393:0:99999:7:::
Bin:x:1:1:bin:/bin:/sbin/nologin Bin:*:16393:0:99999:7:::
2.daemon:x:2:2:daemon:/sbin:/sbin/nologin Daemon:*:16393:0:99999:7:::
3.adm:x:3:4:adm:/var/adm:/sbin/nologin Adm:*:16393:0:99999:7:::
4.lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin Lp:*:16393:0:99999:7:::
5.sync:x:5:0:sync:/sbin:/bin/sync Sync:*:16393:0:99999:7:::
Shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown Shutdown:*:16393:0:99999:7:::
6.halt:x:7:0:halt:/sbin:/sbin/halt Halt:*:16393:0:99999:7:::

2.paste-d* file1 file2 Use * (* can be: or-or other characters) to display new text after file1 and file2 combinations
Example: [[email protected] tmp]# paste-d-a B
Jjj-kkk

3.paste-s file File2 puts file1 text and file2 text in the same text horizontally, that is, file1 the text first, and then puts the file2 text in the next line, saying that the new text has only two lines, That is, File1 is a row, File2 is another line

4.paste-d*-S file file2 distinguishes between file1 text and file2 text by using the * delimiter, that is, each line of the original text is distinguished by a * delimiter as a domain field, and the text is separated by rows.
Example: [[email protected] tmp]# cat m n
A
B
C

O
P
Q
[Email protected] tmp]# paste-d:-S M n
A:B:C:
O:p:q

5.ls | Paste-d ""---
Example: [[email protected] tmp]# ls | Paste-d ""-------the option must be used in conjunction with LS and | |
A B job m n
Test Tom

Note: The paste command needs to understand the traverse, vertical lines, and field transitions

____________________________________________________________________________________
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Four, cat (display text all at once)
Cat [option] filename
1.cat-a filename display text all content

2.cat-b filename to sort the lines in the text by numbering (blank lines are not numbered)

3.CAT-E filename Displays the $ symbol at the end of each line of text

4.cat-n filename to sort the lines in the text by numbering

5.cat-s filename Displays a contiguous number of blank lines in text as a blank line

6.cat-v filename Displays characters that are not visible in text, such as tabs, except for new lines that conform to line breaks

____________________________________________________________________________
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Five, more (text content is displayed on screen, on a per-screen basis)
More [option] filename

1.more-n filename Displays N rows of content to the screen at once

2.more-d filename is used to prompt the user how to view text content on a screen and how to exit view

3.more-f FileName calculates the number of rows, counting the actual rows instead of the auto-wrapping

4.more-p filename does not display text content on a screen-by-screen basis, but instead clears the text contents of the display and displays other content

5.more-c filename Displays content first, then clears the screen

6.more-s filename counts multiple lines of blank lines in a row

7.more +/filename starts displaying all the contents of a string by searching for a string

8.more +n filename Pass nth line start display text content

_______________________________________________________________________________
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

VI. Head
Head [option] filename
1.head-n file n Indicates the first n rows of the view text
Example
Head-8 File View text first 8 lines

++++++++++++++++++++++++++++++++++++++++++++++++++++++

Seven. Tail
tail [option] filename
1.tail-n file n means viewing the text after n rows
Example
Tail-8 File View text after 8 lines

+++++++++++++++++++++++++++++++++++++++++++++++++++++++

Eight. Less
Less file View text content by screen

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Nine. Split----cut a large file into multiple small files and automatically number them in small files
Split [option] small file with cut large file output

Options
-OR-l these two options are equivalent and are used to guide the number of lines cut into small files
-B guides the bytes cut into small files
-C is similar to the-B option, but maintains the integrity of each line as much as possible when cutting

Example
Split-2 file generates a document on every two lines

SPLIT-B50 file splits files into one file in 50 bytes

SPLIT-C50 file is based on the integrity of the text content record, the incoming quantity is guaranteed to be divided into 50 bytes for a file of files

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

10. Join----used to implement a connection operation that selects records with the same domain in two files

Options
-A1 or-A2 In addition to displaying results that are connected in a common domain,-A1 indicates that there are no records in the second file that do not have a common domain
-I ignore case differences when comparing field contents
-O Formatting results display
-T change field delimiter
-v1 or-V2 is similar to-a option, but does not display results with a common domain connection
-1 and 2-1 for setting file 1 for linked domains, 2 for setting file 2 for connected domains

Syntax: join [options] File 1 file 2

1.-a and-V options

Example
Join-t: file1 file2 shows only rows that can be connected in file1 and file2
Join-t:-a1 file1 file2 In addition to displaying rows that can be connected, it also shows all the files that are not connected to File1
Join-t:-a2 file1 file2 In addition to displaying the lines that can be connected, it also shows all files that are not two pieces of file2
Join-t:-v1 file1 file2 show only rows not connected in file1
Join-t:-v2 file1 file2 show only rows not connected in file2

2.-o Options---Format for changing the display of results

Example
JOIN-T:-o11.1 2.2. File1 file2
The format displayed is the first field in the 1th file, the 2nd field of the 2nd file, and the 2nd field of the 1th file.

3.-1 and-2 options for setting the domain to which the file is linked

Example
JOIN-T:-i-1 3-2 1 file1 file2
Connection is made to the 3rd domain of file 1 and the 1th domain of file 2, and the case is ignored

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Xi. TR (for character conversions)
TR [Options] string 1 string 2 <filename

Options
-C selects the complement of the character set in String 1, which is the character set in the inverse of string 1
-D Delete all occurrences of characters in string 1
-S deletes all occurrences of a sequence of characters, leaving only one

1.tr-d character <filename Delete characters in text
Example
Tr-d A-Z <filename delete all uppercase letters in text
Tr-d "[\ n]" <filename Delete all line breaks in text

control characters supported by TR
\a Ctrl+g Ringtones \007
\b Ctrl+h backspace \010
\f Ctrl+l Walk the line to change page \014
\ n ctrl+j line break \012
\ r Ctrl+m Enter \015
\ t ctrl+i Tab key \011

2.-s--Delete all occurrences of a sequence of characters, leaving only one
Example
Tr-s "[\ n]" <filename compresses the recurring newline character into a
Tr-s "[a-z],[a-z]" <filename to compress all repeated letters into one

3. Replace
TR "[A-Z]" "[A-z]" <filename to replace lowercase characters with size letters
TR "[: Lower:]" "[: Upper:]" < filename character Set notation

4.-c---The complement of the character set in the selected string 1
Tr-cs "[a-z][a-z]" "{\012*} <filename
Replace the characters that are not in the "[A-z][a-z]" character set with a newline character, and compress the repeated occurrences of the newline character into a

TR "[o*5]" "*" <filename the text to appear consecutively 5 times to the * character

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

12. Sort

Format: sort [options] [input file]

Options
-C test file is already sorted
-k Specifies the sorted field
-m merge two sorted files
-N Sort by more numeric size
-o[input file] writes output to the specified file, equivalent to redirecting output to the specified file
-R reverse the sorting results
-T change field delimiter
-U removes duplicate rows from the result

1.-T option--for setting separators
Example:
SORT-T:-k3 filename Sorts text based on the third field

2.-K option---Specify to sort by a field
Example
SORT-T:-k3 filename Sort by the third field (sort by string)

3.-N option---Specify Sort by number size
Example:
SORT-T:-k3n filename According to the number of the third field from small to large sort

The 4.-R option---used to reverse the sorting results
Example:
SORT-T:-K3NR filename Sorts the third field by numbers from the big to the small

5.-U option---Remove duplicate rows from the sort results

Example:
SORT-T:-u filename uses-u to remove duplicate rows from text

6.-o option--Save the file output to the screen as
Example:
SORT-T:-k3n-o input file will be sorted as required text input saved as file

The 7.-C option---used to test if the file is already sorted.
SORT-T:-C filename Tests if filename is sorted by default

SORT-T:-k3n-c filename Tests if filename is sorted by the numeric size of the third field

8.-M option---Merge two well-sequenced files into a good, orderly file

Sort-t:-M file1 file2 merge File1 and File2

This article is from the "Endmoon" blog, make sure to keep this source http://endmoon.blog.51cto.com/10533729/1681700

Text processing commands for Shell scripting Learning notes