The Linux uniq command is used to check and delete duplicate rows in a text file.
Uniq can check for repeated rows and columns in a text file.
Grammar
Uniq [-cdu][-f< field >][-s< character position >][-w< character position >][--help][--version][input file [output file]
Parameters :
-C or--count displays the number of occurrences of the row next to each column. -D or--repeated displays only the rows that appear repeatedly. -F< fields > or--skip-fields=< fields > ignores the column specified by comparison. -S< character position > or--skip-chars=< character position > ignores comparison of specified characters. -U or--unique only show one row at a time. -W< character position > or--check-chars=< character position > Specifies the character to compare. --help display Help. --version Displays version information. [Input file] Specifies a sorted text file. [Output FILE] Specifies the output file.
Instance
In the file testfile, line 2nd, line 5th, and 9th behave identically, use the uniq command to delete duplicate rows, using the following command:
The original content in Testfile is:
Cat Testfile # Original content : 95 85
After you delete the duplicate rows using the Uniq command, you have the following output:
uniq testfile # Delete duplicate line contents 85
Checks the file and deletes the duplicate rows in the file and displays the number of occurrences of the row at the beginning of the line. Use the following command:
The resulting output is as follows:
uniq-ctestfile # Delete duplicate rows after 3 Test # # The meaning of the preceding number is 3 times for the line 4 Hello # # The meaning of the preceding number is 4 occurrences for the line 2 Linux 85
Transferred from: http://www.runoob.com/linux/linux-comm-uniq.html
Linux go to heavy command Uniq (GO)