How to Use the uniq command to repeat rows

Turn: http://www.justwinit.cn/post/3671/

 

Note:
You can use the uniq command to delete adjacent duplicate rows:
Uniq
However, if a text contains duplicate but not adjacent lines, it cannot be deleted. You need to combine the sort command:
Sort | uniq
The equivalent sort command is:
Sort-u
========================================================== =
Statistics after deduplication:

Sort needsort.txt | uniq | WC

Original number of rows:

Sort needsort.txt | uniq | WC

Number of repeated rows and times:

Sort needsort.txt | uniq-c-d

Only one row of statistics is repeated:

Sort needsort.txt | uniq-u | WC

You can also sort by column:
Sort-k2nr

How does sort by specified columns?
0000 27189 41925425
065f 15 41925425
0663 7 41925425
0675 5 41925425
0691 76 41925425
0693 2 41925425
06a7 82 41925425
06a9 58 41925425
06bf 68, 41925425
06c1 56 41925425
06d7 145 41925425
06e1 127 41925425
06e3 230 41925425
06e5 7 41925425
06ef 48 41925425
06f1 463 41925425
06ff 3847 41925425
070b 2376 41925425
070d 3408 41925425
0713 109025 41925425
0715 4 41925425
0726 134 41925425
I want to sort the values in the second column, from large to small.

Sort-k2nr sorttest.txt

0713 109025 41925425
0000 27189 41925425
06ff 3847 41925425
070d 3408 41925425
070b 2376 41925425
06f1 463 41925425
06e3 230 41925425
06d7 145 41925425
0726 134 41925425
06e1 127 41925425
06a7 82 41925425
0691 76 41925425
06bf 68, 41925425
06a9 58 41925425
06c1 56 41925425
06ef 48 41925425
065f 15 41925425
0663 7 41925425
06e5 7 41925425
0675 5 41925425
0715 4 41925425
0693 2 41925425

Sort-k2nr

Author: songs from: http://www.linuxdiyf.com
Uniq command: displays unique rows. Only one row is displayed for repeated rows!
The following example shows how to use it:
[Root @ stu100 ~] # Cat Test
Boy took bat home
Boy took bat home
Girl took bat home
Dog brought hat home
Dog brought hat home
Dog brought hat home
View the content of the test file, and you can see the repeated rows.
[Root @ stu100 ~] # Uniq Test
Boy took bat home
Girl took bat home
Dog brought hat home
The uniq command only displays duplicate rows once without adding any parameters.
[Root @ stu100 ~] # Uniq-C Test
2 boy took bat home
1 girl took bat home
3 dog brought hat home
-The C parameter shows the number of consecutive occurrences of each row in the file.
[Root @ stu100 ~] # Uniq-D Test
Boy took bat home
Dog brought hat home
-D option only displays the rows that repeatedly appear in the file.
[Root @ stu100 ~] # Uniq-U test
Girl took bat home
-U option shows that there are no consecutive rows in the file.
[Root @ stu100 ~] # Uniq-F 2-S 2 Test
Boy took bat home
Ignore the first two fields of each line, ignore the first character of the second blank character and the third field, and the result is at home
[Root @ stu100 ~] # Uniq-F 1 Test
Boy took bat home
Dog brought hat home
Ignore the first field of each line. In this way, the line starting with "boy" and "girl" appears to be a row that repeats continuously.

Uniq command
Text
Uniq is a Linux Command
Purpose
Report or delete duplicate rows in the file.
Syntax
Uniq [-c |-d |-u] [-F fields] [-S characters] [-fields] [+ characters] [infile [OUTFILE]
Description
The uniq command deletes duplicate lines in the file. The uniq command reads standard input or Files specified by the infile parameter. This command first compares adjacent lines, and then removes the subsequent copies of the second line and the row. Duplicate rows must be adjacent. (Before sending the uniq command, use the sort command to adjacent all repeated rows .) Finally, the uniq command writes the final individual line to the standard output or the file specified by the OUTFILE parameter. The infile and OUTFILE parameters must specify different files.
The input file must be a text file. A text file contains characters in one or more lines. The length of these rows cannot exceed 2048 bytes (including all line breaks) and cannot contain null characters.
By default, the uniq command compares all rows. If the-F fields or-fields flag is specified, the uniq command ignores the number of fields specified by the fields variable. Field is a string separated by one or more <spaces> characters. If the-S characters or-characters flag is specified, the uniq command ignores the number of fields specified by the characters variable. The value specified for fields and characters variables must be a positive decimal integer.
The current local language environment determines the <blank> characters used by the-F flag and how the-s flag interprets bytes as characters.
If the operation is successful, the uniq command exits and the return value is 0. Otherwise, the return value of the exit command is greater than 0.
Flag
-C adds the number of times each line appears in the input file before the output line.
-D: only duplicate rows are displayed.
-F fields ignores the number of fields specified by the fields variable. If the value of the fields variable exceeds the number of fields in the input line, the uniq command uses an empty string for comparison. This flag is equivalent to the-fields flag.
-U only displays non-duplicate rows.
-S characters ignores the number of characters specified by the characters variable. If the value of the characters variable exceeds the number of characters in the input line, uniq compares it with a null string. If both the-F and-s flag are specified, the uniq command ignores the number of characters specified by the-S characters flag, starting from the field specified by the-F fields flag. This flag is equivalent to the + characters flag.
-Fields ignores the number of fields specified by the fields variable. This flag is equivalent to the-F fields flag.
+ Characters ignores the number of characters specified by the characters variable. If both the-fields and + characters are specified, the uniq command ignores the number of characters specified by the + characters flag and starts after the field specified by the-fields flag. This flag is equivalent to the-S characters flag.
Exit status
This command returns the following exit value:
The 0 command runs successfully.
> 0 error.
Example
To delete the duplicate row in the fruit file and save it to a file named newfruit, enter:
Uniq fruit newfruit
If the fruit file contains the following lines:
Apples
Apples
Peaches
Pears
Bananas
Cherries
Cherries
The newfruit file contains the following lines after you run the uniq command:
Apples
Peaches
Pears
Bananas
Cherries
File
/Usr/bin/uniq contains the uniq command.