Methods for reading file contents in Linux shell scripts (including efficiency testing) __linux

This paper mainly introduces the 4 methods of the shell reading files by line: While loop method, redirect method, pipeline method, file descriptor method.

method 1:while The most efficient and commonly used method in the loop.

The code is as follows:
function While_read_line_bottm () {
While Read line
Todo
Echo $LINE
Done < $FILENAME
}

Note: I'm used to calling this a read, because it requires the execution of a file at the end of it, like reading the file when it's done.

Method 2: Redirect method; Piping method: Cat $FILENAME | While read line

The code is as follows:
Function While_read_line () {
Cat $FILENAME | While Read line
Todo
Echo $LINE
Done
}

Note: All I have to do is call this a plumbing method, which we should be able to see. The output of the command on the left side of the pipe will be input to the command on the right of the pipe when the pipe is met.

Method 3: File description operator

The code is as follows:
Function while_read_line_fd () {
Exec 3<&0
Exec 0< $FILENAME
While Read line
Todo
Echo $LINE
Exec 0<&<3
}

Note: This method is divided into 2 steps, first, by redirecting all content to the file descriptor 3来 closes the file descriptor 0. For this we used the syntax of exec 3<&0. The second part is to broadcast the input file to the file descriptor 0, the standard input.

Method 4 for loop.

The code is as follows:
function For_in_file () {
For line in ' Cat $FILENAME '
Todo
Echo $line
Done
}

Note: This is the way to read the contents of a file through a for loop, which is not much said here. Test each method to see that the method is performing the most efficient.

While line-by-row reading is different from the for loop-by-row reading method, for example:

$ cat File
1111
2222
3333 4444 555

$ cat File | while read line; do echo $line; Done
1111
2222
3333 4444 555

$ for line in $ (<file); do echo $line; Done
1111
2222
3333
4444
555

Test Cases:

First we use a script (see the attachment) to generate a 70000-line file, the file location in/scripts/bigfile. Then test the execution efficiency of each method through the following script, which is simply not explained.
The code is as follows:
#!/bin/bash
Filename= "$"
timefile= "/tmp/loopfile.out" > $TIMEFILE
script=$ (basename $)
function usage () {
Echo-e "\nusage: $SCRIPT file \ n"
Exit 1
}
function While_read_bottm () {
While Read line
Todo
Echo $LINE
Done < $FILENAME
}
function While_read_line () {
Cat $FILENAME | While Read line
Todo
Echo $LINE
Done
}
function while_read_line_fd () {
EXEC 3<&0
EXEC 0< $FILENAME
While Read line
Todo
Echo $LINE
Done
EXEC 0<&3
}
function For_in_file () {
For i in ' Cat $FILENAME '
Todo
Echo $i
Done
}
If [$#-lt 1]; Then
Usage
Fi
Echo-e "\ Starting file processing of each method\n"
Echo-e "Method 1:"
Echo-e "Function While_read_bottm"
Time While_read_bottm >> $TIMEFILE
Echo-e "\ n"
Echo-e "Method 2:"
Echo-e "Function While_read_line"
Time While_read_line >> $TIMEFILE
Echo-e "\ n"
Echo-e "Method 3:"
echo "Function While_read_line_fd"
Time WHILE_READ_LINE_FD >> $TIMEFILE
Echo-e "\ n"
Echo-e "Method 4:"
Echo-e "Function For_in_file"
Time For_in_file >> $TIMEFILE

Execute script:./while/scripts/bigfile

The script output reads as follows:

Method 1:
function While_read_bottm
Real 0m5.689s
User 0m3.399s
SYS 0m1.588s
Method 2:
function While_read_line
Real 0m11.612s
User 0m4.031s
SYS 0m4.956s
Method 3:
function while_read_line_fd
Real 0m5.853s
User 0m3.536s
SYS 0m1.469s
Method 4:
function For_in_file
Real 0m5.153s
User 0m3.335s
SYS 0m1.593s

Here's how to sort each method by speed.
The code is as follows:
Real 0m5.153s Methods 4 (For loop method)
Real 0m5.689s methods 1 (while algorithm)
Real 0m5.853s Methods 3 (identifier method)
Real 0m11.612s Methods 2 (Piping method)

This shows that in each method, the For statement is the most efficient, while in the while loop, when reading and writing files,

While Read line
Todo
Echo $LINE
Done < $FILENAME

The method performs the most efficiently.

There is also an article can refer to: http://blog.itpub.net/22664653/viewspace-1175858/