Remove Linux Shell with uniq to implement the distinct function similar to MySQL

Source: Internet
Author: User

The uniq command can only remove repeated items.

Test text

[shuohai.lhl@v035114 bin]$ cat test 711[10.235.170.22:8123]  0             10.232.20.178     1648042        1648042         0             2013-07-29 20:01:35   711[10.235.170.22:8123]  11            10.232.20.178     6445135        6129386         315749        2013-07-29 20:19:40   711[10.235.170.22:8123]  129           10.232.20.178     2220052        2220052         0             2013-07-29 19:33:04   711[10.235.170.22:8123]  111           10.232.20.178     4939938        4754688         185250        2013-07-29 20:19:40   711[10.235.170.22:8123]  123           10.232.20.178     284227         284227          0             2013-07-29 19:33:00   711[10.235.170.22:8123]  112           10.232.20.178     4393638        4262994         130644        2013-07-29 20:19:40   711[10.235.170.22:8123]  121           10.232.20.178     413891         413891          0             2013-07-29 19:33:00   711[10.235.170.22:8123]  130           10.232.20.178     2803552        2803552         0             2013-07-29 19:33:04   711[10.235.170.22:8123]  10            10.232.20.178     6558889        6235520         323369        2013-07-29 20:19:40   711[10.235.170.22:8123]  106           10.232.20.178     5080974        4886037         194937        2013-07-29 20:19:40   711[10.235.170.22:8123]  101           10.232.20.178     3655253        3655253         0             2013-07-29 20:19:20   711[10.235.170.22:8123]  124           10.232.20.178     424403         424403          0             2013-07-29 19:33:05   711[10.235.170.22:8123]  12            10.232.20.178     5507575        5271088         236487        2013-07-29 20:19:40   711[10.235.170.22:8123]  115           10.232.20.178     1936544        1936544         0             2013-07-29 20:04:40   711[10.235.170.22:8123]  100           10.232.20.178     2791119        2791119         0             2013-07-29 20:13:20   711[10.235.170.22:8123]  119           10.232.20.178     829978         829978          0             2013-07-29 19:48:05   711[10.235.170.22:8123]  109           10.232.20.178     5205710        4994482         211228        2013-07-29 20:19:40   711[10.235.170.22:8123]  114           10.232.20.178     2174793        2174793         0             2013-07-29 20:07:05   711[10.235.170.22:8123]  118           10.232.20.178     1119162        1119162         0             2013-07-29 19:54:20   711[10.235.170.22:8123]  113           10.232.20.178     2965206        2965206         0             2013-07-29 20:14:40   711[10.235.170.22:8123]  103           10.232.20.178     5642924        5396261         246663        2013-07-29 20:19:40   711[10.235.170.22:8123]  104           10.232.20.178     6269570        5970006         299564        2013-07-29 20:19:40   711[10.235.170.22:8123]  105           10.232.20.178     5939748        5665489         274259        2013-07-29 20:19:40   711[10.235.170.22:8123]  107           10.232.20.178     4796367        4623086         173281        2013-07-29 20:19:40   711[10.235.170.22:8123]  117           10.232.20.178     1662411        1662411         0             2013-07-29 20:01:40   711[10.235.170.22:8123]  125           10.232.20.178     283898         283898          0             2013-07-29 19:33:05   711[10.235.170.22:8123]  1             10.232.20.178     1669975        1669975         0             2013-07-29 20:01:45   711[10.235.170.22:8123]  126           10.232.20.178     1133277        1133277         0             2013-07-29 19:33:05   711[10.235.170.22:8123]  110           10.232.20.178     5399121        5176800         222321        2013-07-29 20:19:40   711[10.235.170.22:8123]  13            10.232.20.178     4599532        4599532         0             2013-07-29 19:33:04   711[10.235.170.22:8123]  116           10.232.20.178     1738524        1738524         0             2013-07-29 20:02:25   711[10.235.170.22:8123]  102           10.232.20.178     4360832        4230019         130813        2013-07-29 20:19:40   711[10.235.170.22:8123]  128           10.232.20.178     2534709        2534709         0             2013-07-29 19:33:04   711[10.235.170.22:8123]  120           10.232.20.178     715130         715130          0             2013-07-29 19:43:15   711[10.235.170.22:8123]  108           10.232.20.178     4900775        4720912         179863        2013-07-29 20:19:40   711[10.235.170.22:8123]  127           10.232.20.178     1386770        1386770         0             2013-07-29 19:33:04   711[10.235.170.22:8123]  122           10.232.20.178     424419         424419          0             2013-07-29 19:33:05   711[10.235.170.22:8123]  178           10.232.24.46      2962807        2962807         0             2013-07-29 19:33:05  

Print different values in column 6, first sort, then in uniq

[shuohai.lhl@v035114 bin]$ cat test| awk '{print $6}' | sort |uniq -c     24 0      1 130644      1 130813      1 173281      1 179863      1 185250      1 194937      1 211228      1 222321      1 236487      1 246663      1 274259      1 299564      1 315749      1 323369

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.