Linux delete duplicate lines of code

Linux Delete duplicate lines of code

In text processing, you often delete duplicate rows, and here are three ways

First, with Sort+uniq, attention, simple uniq is not.

?

1	Sort-n Test.txt | Uniq

Second, with the Sort+awk command, note that simple awk is not the same, for the same reason.

Sort-n $file | awk ' {if ($0!=line) print; line=$0} '

Third, with the sort+sed command, the sort command needs to be sorted first.

Sort-n $file | Sed ' $! N /^.∗n1$/! P D

Shell Script

?

1 2 3 4 5 6

#!/bin/sh file= ' test.txt ' sort-n $file | Uniq Sort-n $file | awk ' {if ($0!=line) print; line=$0} ' Sort-n $file | Sed ' $! N /^ (. *) n1$/! P D

Test files:

yanggang@barry$ Cat Test.txt

Aaa

bbbbb

Ccccc

123

Aaaaa

123

Bbb

Aaa

Execution results:

yanggang@barry$./diffrow.sh

Aaa

Aaaaa

Bbb

bbbbb

Ccccc

123

Server Tutorials

Tablet Computer Tutorials

Video playback Tutorial