Tutorial on Basic use of UNIQ commands in Linux systems

uniq [Options] File

Description: This command reads the input file and compares the adjacent rows. Under normal circumstances, the second and more repeated rows will be deleted, and the row comparison is based on the sort sequence of the character set used. The result of this command is written to the output file. The input file and output file must be different. If the input file is represented by "-", it is read from standard input.

The options for this command have the following meanings:

–C displays the number of times that the line appears in the file at the beginning of each row in the output. It can replace the-U and-D options.

–d shows only duplicate rows.

–U displays only the rows that are not duplicated in the file.

–n the first n fields are ignored together with the blanks before each field. A field is a non-space, tab-delimited string that is separated by tabs and spaces (fields are numbered from 0 onwards).

+n the first n characters are ignored, before the characters are skipped (the characters are numbered starting from 0).

–f N is the same as-N, where n is the number of fields.

–s N is the same as +n, where n is the number of characters.

The next example is illustrated by practice:

The code is as follows:

[Root@stu100 ~]# Cat Test

Boy took Bat home

Boy took Bat home

Girl took Bat home

Boy took Bat home

Boy took Bat home

Dog brought hat home

Dog brought hat home

Dog brought hat home

Look at the contents of the test file

The code is as follows:

[root@stu100 ~]# Uniq Test

Boy took Bat home

Girl took Bat home

Boy took Bat home

Dog brought hat home

The Uniq command does not have any arguments, showing only consecutive duplicate rows once

The code is as follows:

[root@stu100 ~]# uniq-c Test

2 Boy took Bat home

1 Girl took Bat home

2 Boy took Bat home

3 Dog brought hat home

1

The-c parameter displays the number of consecutive occurrences of each line in the file.

The code is as follows:

[root@stu100 ~]# cat Test |sort | Uniq-c

1

4 Boy took Bat home

3 Dog brought hat home

1 Girl took Bat home

Sort and then show

The code is as follows:

[root@stu100 ~]# uniq-d Test

Boy took Bat home

Boy took Bat home

Dog brought hat home

The-D option displays only the rows that appear repeatedly in the file.

The code is as follows:

[root@stu100 ~]# uniq-u Test

Girl took Bat home

The-u option shows that there are no consecutive rows in the file.

The code is as follows:

[root@stu100 ~]# uniq-f 2-s 2 test

Boy took Bat home

Ignores the first 2 fields of each row, ignores the second white space character and the first character of the third field, and results at home

The code is as follows:

[root@stu100 ~]# uniq-f 1 Test

Boy took Bat home

Dog brought hat home

Ignores the first field in each row, so that the line at the beginning of the girl appears to be a boy row.

The code is as follows:

[root@stu100 ~]# uniq-d Test

Boy took Bat home

Boy took Bat home

Boy took Bat home

Boy took Bat home

Dog brought hat home

Dog brought hat home

Dog brought hat home

All duplicate rows are displayed, each repeating row is displayed

When you have an employee (employee) file that contains the same entry, you can delete the same entry in the following way

The code is as follows:

$ Sort Namesd.txt | Uniq

$ sort–u Namesd.txt

If you want to know how many lines are the same, you can do it like the following. The first column in the following example shows the number of duplicates for the row. In this case, the line that begins with Alex and Emma has two duplicate rows in the file.

The code is as follows:

$ Sort Namesd.txt | Uniq–c

2 Alex Jason:200:sales

2 Emma thomas:100:marketing

1 Madison randy:300:product Development

1 Nisha Singh:500:sales

1 Sanjay Gupta:400:support

3. The following order lists only the same entries

The code is as follows:

$ Sort Namesd.txt | Uniq–cd

2 Alex Jason:200:sales

2 Emma thomas:100:marketing