Uniq removes duplicate rows or statistics
Main options
-U (unique) and-D (repeated)-C (Statistics)-f (number of skipped fields, default tab)-S (same as F, unit character)
1. Print the same record once (it must be sorted)
Sort a.txt | uniq or
Sort-u a.txt-O B .txt [root @ m165 root] # Cat a.txt
A B 2
A B 4
A B 2
A d 4
A B 4
Count duplicate rows
[Root @ m165 root] # Sort a.txt | uniq-DC
2 a B 2
2 a B 4
The statistics are independent. I don't know why uniq-UC exists.
[Root @ m165 root] # Sort a.txt | uniq-UC
1 a d 4 separate statistics
[Root @ m165 root] # Sort a.txt | uniq-C
2 a B 2
2 a B 4
1 a d 4 [root @ m165 root] # tail-5 install. log. Syslog
<13> Aug 21 16:01:28 kudzu: aliased usb-controller1 as EHCI-HCD
<13> Aug 21 16:01:29 kudzu: aliased usb-controller1 as EHCI-HCD
<13> Aug 21 16:01:20 kudzu: aliased usb-controller1 as EHCI-HCD
<13> Aug 22 16:01:20 kudzu: aliased usb-controller1 as EHCI-HCD
<14> Aug 22 16:01:20 kudzu: aliased usb-controller1 as EHCI-HCD not compared to the first field <13> Aug
[Root @ m165 root] # tail-5 install. log. syslog | uniq-F 1
<13> Aug 21 16:01:28 kudzu: aliased usb-controller1 as EHCI-HCD
<13> Aug 21 16:01:29 kudzu: aliased usb-controller1 as EHCI-HCD
<13> Aug 21 16:01:20 kudzu: aliased usb-controller1 as EHCI-HCD
<13> Aug 22 16:01:20 kudzu: aliased usb-controller1 as EHCI-HCD
Do not compare second field 21
[Root @ m165 root] # tail-5 install. log. syslog | uniq-F 2
<13> Aug 21 16:01:28 kudzu: aliased usb-controller1 as EHCI-HCD
<13> Aug 21 16:01:29 kudzu: aliased usb-controller1 as EHCI-HCD
<13> Aug 21 16:01:20 kudzu: aliased usb-controller1 as EHCI-HCD
Do not compare the Second Field 16:01:28
[Root @ m165 root] # tail-5 install. log. syslog | uniq-F 3
<13> Aug 21 16:01:28 kudzu: aliased usb-controller1 as EHCI-HCD
[Root @ m165 root] # tail-5 install. log. syslog | uniq-F 4
<13> Aug 21 16:01:28 kudzu: aliased usb-controller1 as EHCI-HCD character Unit
[Root @ m165 root] # tail-5 install. log. syslog | uniq-S 9
<13> Aug 21 16:01:28 kudzu: aliased usb-controller1 as EHCI-HCD
<13> Aug 21 16:01:29 kudzu: aliased usb-controller1 as EHCI-HCD
<13> Aug 21 16:01:20 kudzu: aliased usb-controller1 as EHCI-HCD
<13> Aug 22 16:01:20 kudzu: aliased usb-controller1 as EHCI-HCD
[Root @ m165 root] # tail-5 install. log. syslog | uniq-S 10
<13> Aug 21 16:01:28 kudzu: aliased usb-controller1 as EHCI-HCD
<13> Aug 21 16:01:29 kudzu: aliased usb-controller1 as EHCI-HCD
<13> Aug 21 16:01:20 kudzu: aliased usb-controller1 as EHCI-HCD