Shell command Learning, sort,unique file simple sorting to remove duplicate records

Often encountered in the file has a duplicate record of the scene, either remove duplicate records, or count the number of duplicate records, these simple functions can be implemented through the Sort,uniq combination of shell commands,

For example, file A.txt record as follows

Test,test1,test2testtest1test2test,test1,test2abcdedfgabcdddddd

To remove duplicate rows:

Sort A.txt | Uniq

Abcddddddedfgtesttest,test1,test2test1test2

Find non-repeating rows

Sort A.txt | uniq -udddddedfgtesttest1test2

Find duplicate rows

Sort A.txt | uniq -dabcdtest,test1,test2

Statistics

Sort A.txt | Uniq -C      1      2  ABCD      1  ddddd      1  EDFG      1  test      2 test,test1,test2         1  test1      1 test2

Shell command Learning, sort,unique file simple sorting to remove duplicate records