Linux shell -- md5sum, sha1sum, sort, uniq

Linux shell -- md5sum, sha1sum, sort, uniq 1. file verification 1. md5sumeg: md5sum filename Note: generate a 128-bit binary, that is, a 32-bit hexadecimal string 1. verify the correctness of the file: md5sum file1 file2> file_sum.md5 next time I want to see if the file has been changed md5sum-c file_sum.md5file1: okfile2: OK, the verification is successful. 2. verify the correctness of all files in the dir directory. find dir/-type f-print-exec md5sum {}> file_sum.md5 \; orfind dir/-type f-print0 | xargs-0 md5sum> file_sum.md5 Note: print0 is used because if there are spaces in the file name, a file will be generated without calculation: file_sum.md5md5sum-c file_sum.md5 if all are OK, it is correct 3. check whether the downloaded files are correctly stored in the dir directory (if the MD5 string is used, you can create a new file named file_sum.md5) cd dir md5sum-c file_sum.md5 if the output is OK, the verification is correct. the sha1sumsha1 algorithm is similar to the md5 algorithm, but it generates a 160-bit binary number, that is, the 40-bit hexadecimal number is used in the same way as md5sumeg sha1sum file1 file2> file. sha1 test: sha1sum-c file. sha1 2. sort the data in the file and find the unique values sort and uniq sort. You can sort the non-rows of the file by key values, such as month-M and dictionary order-d. Note: sort and uniq do not change the file content. Therefore, you should redirect to another file. sort the content of multiple files. For example, $ cat> file1345e $ cat> file2geree $ sort file1 file2> sorted.txt $ cat sorted.txt 345 eegere 2. sort files by column $ cat> data.txt 1 mac20002 winxp40003 bsd10004 linux1000 1. sort data in the first group in reverse order $ sort-nrk 1 data.txt Note: n is the number order, r is the reverse order, key 1 indicates the first column $ sort-k 2 data.txt 3 bsd10004 linux40001 mac20002 winxp4000 3. delete duplicate rows in the file $ cat> data.txt llllssss $ uniq data.txt> uniq.txt $ cat uniq.txt llss4. use uniq to calculate the number of repetitions in the string $ echo 'shebhaa' | sed's/[^ \ n]/& \ n/G' | sed sort | uniq-c | tr-d '\ n' 3a1b1e2h 3. split file 1. generate a file of the specified size $ dd if =/dev/zero bs = 100 k count = 1 of = data. file note: The 100k file data from all zeros will be generated. file2. split the file $ split-B 20 k data. filexaa xab xac xad xae $ split-B 20 k data. file-d-a 4x0000x0001x0002x0003x0004 use-d to indicate numbers, and-a to specify four lengths $ split-B 20 k data. file-d-a 4 hellohello0000 hello0001 hello0002 hello0003 hello00043. use csplit to split the file by keyword $ cat> server. logSERVER-1 [connection] 192.168.0.1 success [connection] 192.168.0.2 failed [disconnect] 192.168.0.3 pending [connection] 192.168.0.4 successSERVER-2 [connection] 192.168.0.1 failed [connection] 192.168.0.2 failed [disconnect] 192.168.0.3 success [connection] 192.168.0.4 failedSERVER-3 [connection] 192.168.0.1 pending [connection] 192.168.0.2 pending [disconnect] 192.168.0.3 pending [connection] 192.168.0.4 failed $ csplit server. log/SERVER/-n 2-s {*}-f server \-B "% 02d. log "; rm server00.log $ lsserver01.log server02.log server03.log server. log explanation: first, csplit can split the file/[regex]/According to the keyword. It is a pattern match that matches the keyword in the file. After the match is successful, extract the content from the last match or the beginning of the file before the match location, and generate two numbers as a split file-n, for example, 01,02, 03-s, and specify the number of matched rows, {num} specifies the number of matched rows. * indicates the prefix of the generated file to the end of the file-f.-B indicates the format of the specified suffix, and % 02d indicates two digits: filename = prefix + suffix is deleted because the keyword is displayed in the first line, so it is an empty file.