Linuxshell row-by-row file reading method

In Linux, there are many methods to read a file row by row using Linuxshell. The most common method is the method in the script below, which is the most efficient, the most commonly used method. To give you an intuitive feeling, we will generate a large file... the Linux shell row-by-row file reading method has many methods in Linux to read a file row-by-row. The most common method is the method in the script below, which is the most efficient, the most commonly used method. To give you an intuitive feeling, we will test the execution efficiency of various methods by generating a large file. Method 1: The most frequently used method with the highest execution efficiency in a while loop. Function while_read_LINE_bottm () {While read LINE doecho $ LINEdone <$ FILENAME} Note, it is like reading the file after execution. Method 2: redirection method; Pipeline Method: cat $ FILENAME | while read LINE Function While_read_LINE () {cat $ FILENAME | while read LINE do echo $ LINEdone} note: I only call this method the Pipeline Method. you can see it later. When a pipeline is encountered, the command output on the left of the pipeline serves as the command input on the right of the pipeline and is then input. Method 3: file descriptor Function while_read_line_fd () {Exec 3 <& 0 Exec 0 <$ FILENAMEWhile read LINEDo Echo $ LINE Exec 0 <& <3} note: this method involves two steps. first, disable file descriptor 0 by redirecting all content to file descriptor 3. therefore, we use the Exec 3 syntax <& 0. The second part is to send the input file to the file descriptor 0, that is, the standard input. Method 4 for loop. Function for_in_file () {For I in 'cat $ FILENAME 'doecho $ idone} Note: This method reads the content of a file through a for loop, which is familiar to you, I will not talk about it here. Test each method to see that the method is most efficient. First, we use a script (see the attachment for the script) to generate a 70000-line file, which is located in/scripts/bigfile. Then, use the following script to test the execution efficiency of each method. the script is very simple and will not be explained. #! /Bin/bashFILENAME = "$1" TIMEFILE = "/tmp/loopfile. out "> $ timefile script = $ (basename $0) function usage () {echo-e" \ nUSAGE: $ SCRIPT file \ n "exit 1} function while_read_bottm () {while read LINEdoecho $ LINE done <$ FILENAME} function while_read_line () {cat $ FILENAME | while read LINEdoecho $ LINEdone} function while_read_line_fd () {exec 3 <& 0 exec 0 <$ FILENAMEwhile read LINEdo echo $ LINEdone exec 0 <& 3} f Unction for_in_file () {for I in 'cat $ filename' doecho $ idone} if [$ #-lt 1]; thenusagefi echo-e "\ n starting file processing of each method \ n" echo-e "method 1: "echo-e" function while_read_bottm "time while_read_bottm >>$ TIMEFILE echo-e" \ n "echo-e" method 2: "echo-e" function while_read_line "time while_read_line >>$ TIMEFILE echo-e" \ n "echo-e" method 3: "echo" function while_read_line _ Fd "time while_read_line_fd >>$ TIMEFILE echo-e" \ n "echo-e" method 4: "echo-e" function for_in_file "time for_in_file >>$ TIMEFILE after executing the script: [root @ localhost shell] #. /while/scripts/bigfile script output content: method 1: function while_read_bottm real 0m5. 689 suser 0m3. 399 ssys 0m1. 588 s method 2: function while_read_line real 0m11. 612 suser 0m4. 031 ssys 0m4. 956 s method 3: function while_read_line_fd real 0m5. 853 suser 0m3. 536 ssys 0m1. 469 s method 4: function for_in_file real 0m5. 153 suser 0m3. 335 ssys 0m1. 593 s we sort the methods by speed. Real 0m5. 153 s method 4 (for loop method) real 0m5. 689 s method 1 (while salary base method) real 0m5. 853 s method 3 (identifier method) real 0m11. the 612 s method 2 (Pipeline method) shows that in each method, the for statement is the most efficient, while in the while loop, when reading and writing files, while read LINEdoecho $ LINE done <$ FILENAME method is the most efficient.