How to Use uniq to remove duplicate commands

Original article address

Linux Shell learning: How to Use the uniq command to introduce the function of the uniq command: display unique rows, only once for those repeated rows! Next we will explain through practical examples. [keyword] Linux Shell uniq

Check the content of the test.txt file and you can see the repeated rows in the file.

Root@hexu.org ~ #Cat test.txt
Boy took bat home
Boy took bat home
Girl took bat home
Dog brought hat home
Dog brought hat home
Dog brought hat home

The uniq command only displays duplicate rows once without adding any parameters.

Root@hexu.org ~ #Uniq test.txt
Boy took bat home
Girl took bat home
Dog brought hat home

-The C parameter shows the number of consecutive occurrences of each row in the file.

Root@hexu.org ~ #Uniq-C test.txt
2 boy took bat home
1 girl took bat home
3 dog brought hat home

-D option only displays the rows that repeatedly appear in the file.

Root@hexu.org ~ #Uniq-D test.txt
Boy took bat home
Dog brought hat home

-U option shows that there are no consecutive rows in the file.

Root@hexu.org ~ #Uniq-u test.txt
Girl took bat home

Ignore the first two fields of each line, ignore the first character of the second blank character and the third field, and the result is at home

Root@hexu.org ~ #Uniq-F 2-S 2 test.txt
Boy took bat home

Ignore the first field of each line. In this way, the line starting with "boy" and "girl" appears to be a row that repeats continuously.

Root@hexu.org ~ #Uniq-F 1 test.txt
Boy took bat home
Dog brought hat home