Turn: http://www.justwinit.cn/post/3671/
Note:You can use the uniq command to delete adjacent duplicate rows:UniqHowever, if a text contains duplicate but not adjacent lines, it cannot be deleted. You need to combine the sort command:Sort |
The-u parameter of the sort command for basic text processing-uniq command is only valid for key-value segments. sort will discard records with the same key-value segments even if the other parts are different. To put it simple, Linuxshell uses the
Reprint Address: http://blog.51yip.com/shell/1022.htmlInstance details Linux go down except duplicate line command UniqOne, what's uniq for?The duplicate lines in the text are basically not what we want, so we're going to get rid of them. There are
Seven examples of the uniq command: the uniq command in Linux can be used to process repeated lines in text files, this tutorial explains some of the most common usage of the uniq command, which may be helpful to you. The following File test will be
1. uniq-report or ignore duplicate rowsby default, only the same rows are adjacent to only one parameter the most commonly used general and sort commands are used to count the number of repeating rows. NAME Uniq-report or omit repeated linessynopsis
One, uniq what to do with the
The duplicate lines in the text are basically not what we want, so we need to get rid of them. Linux has other commands to remove duplicate rows, but I think Uniq is a more convenient one. When using Uniq, pay
Address: http://blog.51yip.com/shell/1022.htmlOne, what's uniq for?The duplicate lines in the text are basically not what we want, so we're going to get rid of them. There are other commands in Linux that can remove duplicate lines, but I think Uniq
Uniq command
Text Uniq is a duplicate line in the Linux command usage report or delete file. Syntax Uniq [-c |-D |-u] [f Fields] [-S Characters] [-fields] [+characters] [InFile [outfile]] Description uniq command Delete file Duplicate rows in the.
Uniq: Finds a single row from a given input (stdin or command-line parameter file) by eliminating duplicate content. It is also possible for the user to identify duplicate rows that appear in the input. Uniq can only be used for sequential data
Seven examples of the uniq command: The uniq command in Linux can be used to process repeated lines in text files, this tutorial explains some of the most common usage of the uniq command, which may be helpful to you. The following file test will be
Understand sort and uniq commands (including uniq's-u and-d options) [plain] cat a B | sort | uniq> c # c is a union B union cat a B | sort | uniq-d> c # c is a intersect B intersection cat a B B | sort | uniq-u> c # c is set difference a-B
Uniq can check repeated columns in text files.SyntaxUniq [-cdu] [-f ] [-s ] [-w ] [-- help] [-- version][Input file] [output file]Parameters: -C or -- count displays the number of repeated occurrences of the row next to each column. -D or --
uniq [Options] File
Description: This command reads the input file and compares the adjacent rows. Under normal circumstances, the second and more repeated rows will be deleted, and the row comparison is based on the sort sequence of the character
Uniq removes duplicate rows or statistics
Main options
-U (unique) and-D (repeated)-C (Statistics)-f (number of skipped fields, default tab)-S (same as F, unit character)
1. Print the same record once (it must be sorted)
Sort a.txt | uniq or
Sort-u
I believe that under Linux for file operations often use the sort and Uniq commands, the following system describes the use of these two commands.Sort Command is very useful in Linux, it will sort the files and output the sorting results standard.
Sort
The sort command sorts the rows in the files specified by the file parameter and writes the results to the standard output. If the file parameter specifies multiple files, the sort command connects the files and sorts them as a file.
Sort
ArticleDirectory
Parameters
Option
Example
Show unique rows
Uniq [Options] [Input-File] [Output-File]
Uniq is used to display its input. Only one row is retained for consecutive duplicate rows. If the files are sorted,
As you can see from the Help information in the Uniq command, this command filters only adjacent duplicate rows.If you want to remove all duplicate rows, you need to sort them first, or use the Uniq-u$Uniq--Husage:Uniq[OPTION] ... [INPUT [OUTPUT]]
The sortsort command sorts the rows in the File specified by the File parameter and writes the result to the standard output. If the File parameter specifies multiple files, the sort command
The sort command sorts the rows in the File specified by
The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion;
products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the
content of the page makes you feel confusing, please write us an email, we will handle the problem
within 5 days after receiving your email.
If you find any instances of plagiarism from the community, please send an email to:
info-contact@alibabacloud.com
and provide relevant evidence. A staff member will contact you within 5 working days.