Merge linux Files and remove duplicates

 

(1) intersection of two files
Prerequisites: No duplicate rows are allowed in each file.
1. Extract the union of the two files (duplicate rows only retain one copy)
Cat file1 file2 | sort | uniq> file3
2. Extract the intersection of two files (only files that exist in both files are left)
Cat file1 file2 | sort | uniq-d> file3
3. Delete the intersection and leave other rows.
Cat file1 file2 | sort | uniq-u> file3
(2) merge two files
A file is on, and a file is under
Cat file1 file2> file3
One file on the left and one file on the right
Paste file1 file2> file3
(3) Remove duplicate rows from a file
Sort file | uniq
Note: Repeat multiple rows is recorded as one row. That is to say, these duplicate rows are still present, but all are omitted as one row!
Sort file | uniq-u
The above command can remove all repeated lines, that is, non-repeated lines in the file!