First, the intersection of the two files, and the Union
Prerequisites: No duplicate rows are allowed in each file.
1. Extract the union of the two files (duplicate rows only retain one copy)
2. Extract the intersection of two files (only files that exist in both files are left)
3. Delete the intersection and leave other rows.
1. Cat file1 file2 | sort | uniq> file3
2. Cat file1 file2 | sort | uniq-D> file3
3. Cat file1 file2 | sort | uniq-u> file3
2. merge two files
A file is on, and a file is under
Cat file1 file2> file3
One file on the left and one file on the right
Paste file1 file2> file3
Third, remove duplicate rows from an object:
Sort file | uniq
Note: Repeat multiple rows is recorded as one row. That is to say, these duplicate rows are still present, but all are omitted as one row!
Sort file | uniq-u
The above command can remove all repeated lines, that is, non-repeated lines in the file!
For details, see the detailed introduction of commands such as CAT, sort, and uniq.