Several methods for shell script to go heavy _linux shell

Test files

Copy Code code as follows:

[Root@bogon ~]# Cat >test
Jason
Jason
Jason
Fffff
Jason

Press CTR + D to save

1, Sort-u

Copy Code code as follows:

[Root@bogon ~]# sort-u Test
Fffff
Jason

2, Uniq

Copy Code code as follows:

[Root@bogon ~]# Uniq Test
Jason
Fffff
Jason

This shows that uniq need to be sorted before they can be heavy

3, sort, pipe and uniq combined use
Remove duplicates and keep a duplicate of the data

Copy Code code as follows:

[Root@bogon ~]# Sort Test|uniq
Fffff
Jason

Removes all duplicate rows (without retaining duplicate rows) and calculates the number of rows

Copy Code code as follows:

[Root@bogon ~]# Sort Test|uniq-uc
1 fffff

Finds all duplicate rows and calculates the number of rows

Copy Code code as follows:

[Root@bogon ~]# Sort TEST|UNIQ-DC
4 Jason

To weight according to a field

Change the original data to

Copy Code code as follows:

[Root@bogon ~]# more Test
Jason 1
Jason 2
Jason 1
FFFFF 2
Jason 3
[Root@bogon ~]# sort-k 1,1-u test
FFFFF 2
Jason 1

Man sort looks at the use of sort, which is described in the-K argument:

Copy Code code as follows:

-K,--key=pos1[,pos2]
Start a key at POS1 (Origin 1), end it at POS2 (default "line")

That is, k which columns are sorted, POS1 is the beginning column, Pos2 is the end column