PHP Several quick read large file examples

Source: Internet
Author: User
Tags php code

In PHP, the quickest way to read a file is to use functions such as file and file_get_contents, and a few lines of code can be pretty good at accomplishing the functions we need. However, when the file being manipulated is a larger file, these functions may be out of order, the following will be from a requirement to explain the reading of large files, the common method of operation.

Demand

There is a 800M log file, about more than 5 million lines, with PHP to return the contents of the last few lines.

Implementation methods

1. Directly using the file function to operate

Because the file function is a one-time read all content into memory, and PHP in order to prevent some poorly written programs to occupy too much memory and cause the system is out of memory, so that the server is down, so by default, limit the maximum use of memory 16M, this is through the php.ini Memory_ Limit = 16M To set, if the value is set to 1, then the amount of memory is unrestricted.

Here is a section of code that uses file to remove the last line of the document:

The code is as follows
<?php
Ini_set (' Memory_limit ', '-1 ');
$file = ' Access.log ';
$data = file ($file);
$line = $data [Count ($data)-1];
Echo $line;
?>

Complete code Execution time consuming 116.9613 (s).

My machine is 2 G of RAM, when pressed F5 run, the system directly dimmed, almost 20 minutes later to recover, visible will be so large files directly into the memory, the consequences of how much serious, so not million, memory_limit this thing can not be too high, otherwise only call the room, Let's reset the machine.

2. Direct call to Linux Tail command to display the last few lines

The Linux command line, you can directly use Tail-n access.log very easy to display the last few lines of log files, you can directly invoke the tail command PHP, execute PHP code as follows:

The code is as follows
<?php
$file = ' Access.log ';
$file = Escapeshellarg ($file); Secure escape of command-line arguments
$line = ' tail-n 1 $file ';
Echo $line;
?>

Complete code Execution time consuming 0.0034 (s)

3. Direct use of PHP fseek for file operation

This is the most common way, it does not need to read the contents of the file into the content, but directly through the pointer to operate, so efficiency is quite efficient. There are a number of different ways in which you can use fseek to manipulate a file, and the efficiency may vary slightly, and here are two common methods:

Method One

First through the fseek find the last EOF of the file, then find the starting position of the last line, take this row of data, find the first line of the starting position, and then take the position of the line, and so on until the $num line is found.

#实现代码如下

The code is as follows
<?php
$fp = fopen ($file, "R");
$line = 10;
$pos =-2;
$t = "";
$data = "";
while ($line > 0)
{
while ($t!= "\ n")
{
Fseek ($fp, $pos, seek_end);
$t = fgetc ($FP);
$pos--;
}
$t = "";
$data. = Fgets ($FP);
$line--;
}
Fclose ($FP);
Echo $data
?>

Complete code Execution time consuming 0.0095 (s)

Method Two

Or in a fseek way from the end of the file to read, but this is not a read, but a piece of reading, each reading a piece of data, the read data in a buf, and then through the number of line breaks (\ n) to determine whether the last $num row of data has been read.

#实现代码如下

The code is as follows
<?php
$fp = fopen ($file, "R");
$num = 10;
$chunk = 4096;
$fs = sprintf ("%u", FileSize ($file));
$max = (Intval ($fs) = = Php_int_max)? Php_int_max:filesize ($file);
for ($len = 0; $len < $max; $len + = $chunk)
{
$seekSize = ($max-$len > $chunk)? $chunk: $max-$len;
Fseek ($fp, ($len + $seekSize) *-1, seek_end);
$readData = Fread ($fp, $seekSize). $readData;
if (Substr_count ($readData, "\ n") >= $num + 1)
{
Preg_match ("! (). *?\n) {". ($num). "}$!", $readData, $match);
$data = $match [0];
Break
}
}
Fclose ($FP);
Echo $data;
?>

Complete code Execution time consuming 0.0009 (s).

Method Three

The code is as follows

<?php
function tail ($fp, $n, $base = 5)
{
ASSERT ($n > 0);
$pos = $n + 1;
$lines = Array ();
while (count ($lines) <= $n)
{
Try
{
Fseek ($FP,-$pos, seek_end);
}
catch (Exception $e)
{
Fseek (0);
Break
}
$pos *= $base;
while (!feof ($FP))
{
Array_unshift ($lines, Fgets ($fp));
}
}

Return Array_slice ($lines, 0, $n);
}

Var_dump (Tail (fopen ("Access.log", "r+"), 10);
?>

Complete code Execution time consuming 0.0003 (s)

Related Article

E-Commerce Solutions

Leverage the same tools powering the Alibaba Ecosystem

Learn more >

Apsara Conference 2019

The Rise of Data Intelligence, September 25th - 27th, Hangzhou, China

Learn more >

Alibaba Cloud Free Trial

Learn and experience the power of Alibaba Cloud with a free trial worth $300-1200 USD

Learn more >

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.