Shell operations used in Baidu NLP internship for one month

The columns in the text line are broken:

Awk 'in in {srand ()} {for (I = 1; I <= NF; I ++) B [rand () NF] = $ I} end {for (X in B) printf "% s", B [x]} 'data

 

The lines in the text line are broken down:

Awk 'in in {srand ()} {B [rand () Nr] = $0} end {for (X in B) print B [x]} 'Data

Shuf-n1000 data

 

Use a.txtas the patternto search for B .txt. The actual result is the intersection of the two:

Grep-F-X-F a.txt B .txt

 

Search for B .txtbased on a.txtfor patternto display the data in a.txt. The actual difference is B-:

Grep-F-v-X-F a.txt B .txt

 

Encoding conversion:

Iconv-F gb18030-T utf8 filename

 

The Delimiter is _. The second column is key-value sorting, which is stable (unstable by default ):

Sort-T _-K1, 2-s filename

 

Sort and remove large files externally, and use the temp directory as the cache:

Sort-u-t temp filename

 

Sort filename | unique sorting record of uniq-C count-d only show duplicate record-u only show no duplicate record

 

Open the man document of awk in preview:

Man-T awk | open-a preview-F

 

Simple usage of awk and cut:

Awk-F:-V 'ofs = \ t' {print "all users are:", $1, $2, $3, $4, $5, $6, $7} '/etc/passwd | cut-F, 6

 

WC-l rows-C bytes-w words