PHP many ways to read large files, _php tutorial

PHP read large files in a variety of ways to introduce,

Reading large files has always been a headache, we like to use PHP development to read small files can directly use a variety of function implementation, but one to large articles will find common method is not normal use or time too long too card, next we look at the PHP read big file Problem resolution, I hope the examples will help you.

In PHP, the quickest way to read a file is to use functions such as file, file_get_contents, and just a few lines of code to do what we need. But when the file being manipulated is a large file, these functions may be out of the way, the following will start from a requirement to illustrate the common method of operation when reading large files.

Demand:
There is a 800M log file, approximately more than 5 million lines, with PHP returning the last few lines of content.

Implementation method:

1. Use the file function directly to manipulate
Because the file function reads everything into memory at once, and PHP prevents some poorly written programs from taking up too much memory and causing the server to go down, the default limit is to use only 16M of memory, which is through php.ini Memory_ Limit = 16M To set, this value if set-1, the memory makes the usage unrestricted.

Here is a piece of code that uses file to fetch the last line of the document:

<?php ini_set (' memory_limit ', '-1 ');　　$file = ' Access.log ';　　$data = file ($file);　　$line = $data [Count ($data)-1]; echo $line;? >

The entire code execution takes time 116.9613 (s).
My machine is 2 g of memory, when the press F5 run, the system directly gray, almost 20 minutes after the recovery, it can be seen so large files directly into memory, the consequences is how much serious, so not million can not, memory_limit this thing can not be adjusted too high, otherwise only call room, Let the reset machine out.

2. Directly call the Linux Tail command to display the last few lines
Under the Linux command line, you can directly use Tail-n access.log very easy to display the last few lines of the log file, you can directly use PHP to invoke the tail command, execute PHP code as follows:

<?php  $file = ' access.log ';  $file = Escapeshellarg ($file); Safe escape of command-line arguments  $line = ' tail-n 1 $file ';  echo $line;? >

Complete code Execution time 0.0034 (s)

3. Use PHP fseek directly for file operation
This is the most common way, it does not need to read all the contents of the file, but directly through the pointer to operate, so efficiency is quite efficient. There are a number of different ways to use fseek to manipulate files, and the efficiency may also be slight, and here are two common methods:

Method One
First, find the last EOF of the file by fseek, then find the beginning of the last line, take this row of data, then find the starting position of the next line, then take the position of this line, and so on, until the $num line is found.
The implementation code is as follows

<?PHP$FP = fopen ($file, "R"), $line = ten; $pos =-2; $t = ""; $data = ""; while ($line > 0) {while ($t! = "\ n") {fseek ( $FP, $pos, seek_end); $t = fgetc ($FP); $pos--; } $t = ""; $data. = Fgets ($FP); $line--;} Fclose ($FP); Echo $data?>

Complete code Execution time 0.0095 (s)

Method Two
Or use fseek way from the end of the file to read, but this is not a single read, but a piece of reading, each read a piece of data, the data will be read in a buf, and then by the number of newline characters (\ n) to determine whether the last $num line of data is read.
The implementation code is as follows

<?PHP$FP = fopen ($file, "R"), $num = ten; $chunk = 4096; $fs = sprintf ("%u", FileSize ($file)); $max = (Intval ($fs) = = Php_in T_max)? Php_int_max:filesize ($file); for ($len = 0; $len < $max; $len + = $chunk) {$seekSize = ($max-$len > $chunk)? $chu NK: $max-$len; Fseek ($fp, ($len + $seekSize) *-1, seek_end); $readData = Fread ($fp, $seekSize). $readData; if (Substr_count ($readData, "\ n") >= $num + 1) {Preg_match ("!". *?\n) {". ($num). "}$!", $readData, $match); $data = $match [0]; Break }}fclose ($FP); Echo $data;? >

The entire code execution takes time 0.0009 (s).

Method Three

<?phpfunction tail ($fp, $n, $base = 5) {assert ($n > 0); $pos = $n + 1; $lines = Array (); while (count ($lines) <= $n) {try {  fseek ($fp,-$pos, Seek_end),} catch (Exception $e) {  fseek (0);  Break } $pos *= $base; while (!feof ($fp)) {  array_unshift ($lines, Fgets ($FP)),} return Array_slice ($lines, 0, $n);} Var_dump (Tail (fopen ("Access.log", "r+"), 10));? >

Complete code Execution time 0.0003 (s)

Method Four, PHP stream_get_line function, read fast, read 500,000 data large files, about 20 seconds about the time! The example code is as follows

$fp = fopen ('./iis.log ', ' R '); File while (!feof ($fp)) {  //for ($j =1; $j <=1000; $j + +) {     //reads the following 1000 rows and stores them in the array  $logarray [] = stream_get_ Line ($FP, 65535, "\ n");     Break // }   }

The above is PHP read large files Four ways, I hope that everyone's learning is helpful.

Articles you may be interested in:

• The correct way for PHP to read files
• PHP Read File content code (TXT,JS, etc.)
• PHP Read File garbled problem
• PHP reads the contents of the file into a string, while removing line breaks, blank lines, line endings (zjmainstay original)
• PHP to read files and support code sharing for remote files
• A few ways to read the contents of PHP file
• PHP Read file header to determine the file type implementation code
• PHP empty File Sample code after reading the contents of the file
• 3 Ways to remove newline characters when PHP reads a file by line
• How PHP reads the contents of a file into an array

http://www.bkjia.com/PHPjc/1117025.html www.bkjia.com true http://www.bkjia.com/PHPjc/1117025.html techarticle PHP read large files in a variety of ways introduced, read large files has been a headache problem, we like to use PHP development to read small files can be directly using a variety of functions implemented, but one to the big text ...

