There are many ways to read one file at a row in Linux, the most common of which is the method in the following script, and the most efficient and most used method. In order to give you an intuitive feeling, we will test the execution efficiency of various methods by generating a large file.
Method 1:while the most efficient and commonly used method in the loop.
Copy Code code as follows:
function While_read_line_bottm () {
While Read line
Todo
Echo $LINE
Done < $FILENAME
}
Note: I'm used to calling this a read, because it requires the execution of a file at the end of it, like reading the file when it's done.
Method 2: Redirect method; Piping method: Cat $FILENAME | While Read line
Copy Code code as follows:
Function While_read_line () {
Cat $FILENAME | While Read line
Todo
Echo $LINE
Done
}
Note: All I have to do is call this a plumbing method, which we should be able to see. The output of the command on the left side of the pipe will be input to the command on the right of the pipe when the pipe is met.
Method 3: File Descriptor method
Copy Code code as follows:
Function while_read_line_fd () {
Exec 3<&0
Exec 0< $FILENAME
While Read line
Todo
Echo $LINE
Exec 0<&<3
}
Note: This method is divided into 2 steps, first, by redirecting all content to the file descriptor 3来 closes the file descriptor 0. For this we used the syntax of exec 3<&0. The second part is to broadcast the input file to the file descriptor 0, the standard input.
Method 4 for Loop.
Copy Code code as follows:
function For_in_file () {
For i in ' Cat $FILENAME '
Todo
Echo $i
Done
}
Note: This is the way to read the contents of a file through a for loop, which is not much said here. Test each method to see that the method is performing the most efficient.
First we use a script (see the attachment) to generate a 70000-line file, the file location in/scripts/bigfile. Then test the execution efficiency of each method through the following script, which is simply not explained.
Copy Code code as follows:
#!/bin/bash
Filename= "$"
timefile= "/tmp/loopfile.out" > $TIMEFILE
script=$ (basename $)
function usage () {
Echo-e "\nusage: $SCRIPT file \ n"
Exit 1
}
function While_read_bottm () {
While Read line
Todo
Echo $LINE
Done < $FILENAME
}
function While_read_line () {
Cat $FILENAME | While Read line
Todo
Echo $LINE
Done
}
function while_read_line_fd () {
EXEC 3<&0
EXEC 0< $FILENAME
While Read line
Todo
Echo $LINE
Done
EXEC 0<&3
}
function For_in_file () {
For i in ' Cat $FILENAME '
Todo
Echo $i
Done
}
If [$#-lt 1]; Then
Usage
Fi
Echo-e "\ Starting file processing of each method\n"
Echo-e "Method 1:"
Echo-e "Function While_read_bottm"
Time While_read_bottm >> $TIMEFILE
Echo-e "\ n"
Echo-e "Method 2:"
Echo-e "Function While_read_line"
Time While_read_line >> $TIMEFILE
Echo-e "\ n"
Echo-e "Method 3:"
echo "Function While_read_line_fd"
Time WHILE_READ_LINE_FD >> $TIMEFILE
Echo-e "\ n"
Echo-e "Method 4:"
Echo-e "Function For_in_file"
Time For_in_file >> $TIMEFILE
After executing the script: [Root@localhost shell]#./while/scripts/bigfile
Script output:
Copy Code code as follows:
Method 1:
function While_read_bottm
Real 0m5.689s
User 0m3.399s
SYS 0m1.588s
Method 2:
function While_read_line
Real 0m11.612s
User 0m4.031s
SYS 0m4.956s
Method 3:
function while_read_line_fd
Real 0m5.853s
User 0m3.536s
SYS 0m1.469s
Method 4:
function For_in_file
Real 0m5.153s
User 0m3.335s
SYS 0m1.593s
Here's how to sort each method by speed.
Copy Code code as follows:
Real 0m5.153s Methods 4 (For loop method)
Real 0m5.689s methods 1 (while algorithm)
Real 0m5.853s Methods 3 (identifier method)
Real 0m11.612s Methods 2 (Piping method)
This shows that in each method, the For statement is the most efficient, while in the while loop, when reading and writing files,
Copy Code code as follows:
While Read line
Todo
Echo $LINE
Done < $FILENAME
The method performs the most efficiently.
