In PHP, the quickest way to read a file is to use functions such as file and file_get_contents, and a few lines of code can be pretty good at accomplishing the functions we need. However, when the file being manipulated is a larger file, these functions may be out of order, the following will be from a requirement to explain the reading of large files, the common method of operation.
Demand
There is a 800M log file, about more than 5 million lines, with PHP to return the contents of the last few lines.
Implementation methods
1. Directly using the file function to operate
Note: Because the file function is a one-time read all content into memory, and PHP in order to prevent some poorly written programs to occupy too much memory and cause the system is out of memory, so that the server is down, so by default, limit the maximum use of memory 16M, this is through the php.ini Memory_ Limit = 16M To set, if the value is set to 1, then the amount of memory is unrestricted.
Here is a section of code that uses file to remove the last line of the document.
Complete code Execution time consuming 116.9613 (s).
Copy Code code as follows:
$fp = fopen ($file, "R");
$num = 10;
$chunk = 4096;
$fs = sprintf ("%u", FileSize ($file));
$max = (Intval ($fs) = = Php_int_max)? Php_int_max:filesize ($file);
for ($len = 0; $len < $max; $len + = $chunk) {
$seekSize = ($max-$len > $chunk)? $chunk: $max-$len;
Fseek ($fp, ($len + $seekSize) *-1, seek_end);
$readData = Fread ($fp, $seekSize). $readData;
if (Substr_count ($readData, "\ n") >= $num + 1) {
Preg_match ("! (). *?\n) {". ( $num). "} $! ", $readData, $match);
$data = $match [0];
Break
}
}
Fclose ($FP);
Echo $data;
My machine is 2 G of RAM, when pressed F5 run, the system directly dimmed, almost 20 minutes later to recover, visible will be so large files directly into the memory, the consequences of how much serious, so not million, memory_limit this thing can not be too high, otherwise only call the room, Let's reset the machine.
2. Direct call to Linux Tail command to display the last few lines
The Linux command line, you can directly use Tail-n access.log very easy to display the last few lines of log files, you can directly invoke the tail command PHP, execute PHP code as follows.
Complete code Execution time consuming 0.0034 (s)
Copy Code code as follows:
File = ' Access.log ';
$file = Escapeshellarg ($file); Secure escape of command-line arguments
$line = ' tail-n 1 $file ';
Echo $line;
3. Direct use of PHP fseek for file operation
This is the most common way, it does not need to read the contents of the file into the content, but directly through the pointer to operate, so the efficiency is very efficient. When using fseek to manipulate a file, there are several different ways in which the efficiency may vary slightly, and the following are two common methods.
Method One
First through the fseek find the last EOF of the file, then find the starting position of the last line, take this row of data, find the first line of the starting position, and then take the position of the line, and so on until the $num line is found.
The implementation code is as follows
Complete code Execution time consuming 0.0095 (s)
Copy Code code as follows:
function tail ($fp, $n, $base =5)
{
ASSERT ($n >0);
$pos = $n +1;
$lines = Array ();
while (count ($lines) < = $n) {
try{
Fseek ($FP,-$pos, seek_end);
catch (Exception $e) {
Fseek (0);
Break
}
$pos *= $base;
while (!feof ($fp)) {
Array_unshift ($lines, Fgets ($fp));
}
}
Return Array_slice ($lines, 0, $n);
}
Var_dump (Tail (fopen ("Access.log", "r+"), 10);
Method Two
Or in a fseek way from the end of the file to read, but this is not a read, but a piece of reading, each reading a piece of data, the read data in a buf, and then through the number of line breaks (\ n) to determine whether the last $num row of data has been read.
The implementation code is as follows
Complete code Execution time consuming 0.0009 (s).
Copy Code code as follows:
$fp = fopen ($file, "R");
$line = 10;
$pos =-2;
$t = "";
$data = "";
while ($line > 0) {
while ($t!= "\ n") {
Fseek ($fp, $pos, seek_end);
$t = fgetc ($FP);
$pos--;
}
$t = "";
$data. = Fgets ($FP);
$line--;
}
Fclose ($FP);
Echo $data
Method Three
Complete code Execution time consuming 0.0003 (s)
Copy Code code as follows:
Ini_set (' Memory_limit ', '-1 ');
$file = ' Access.log ';
$data = file ($file);
$line = $data [Count ($data)-1];
Echo $line;
