Have done a lot of crawling other site content products, accustomed to the use of quick and easy file_get_contents function, but always encounter the problem of getting failed, although the example set by the manual time-out, can not be effective most of the time:
Copy CodeThe code is as follows:
$config [' context '] = stream_context_create (Array (' http ' = = Array (' method ' = ' "GET",
' Timeout ' = 5//This time-out is unstable and often does not work
)
));
At this point, look at the server connection pool, you will find a bunch of similar errors, you have a headache:
File_get_contents (http://***): failed to open stream ...
Last resort, I installed the Curl library and wrote a function substitution:
Copy CodeThe code is as follows:
functionCurl_file_get_contents ( $durl ) {
$ch = Curl_init ( ) ;
curl_setopt ( $ch ,Curlopt_url , $durl ) ;
curl_setopt ( $ch ,Curlopt_timeout , 5 ) ;
curl_setopt ( $ch ,Curlopt_useragent ,_useragent_ ) ;
curl_setopt ( $ch ,Curlopt_referer ,_referer_ ) ;
curl_setopt ( $ch ,Curlopt_returntransfer , 1 ) ;
$r = curl_exec ( $ch ) ;
Curl_close ( $ch ) ;
return $r ;
}
In this way, there are no more problems than real network problems.
This is the test that someone else has done about curl and file_get_contents:
file_get_contents crawl google.com required number of seconds:
2.31319094
2.30374217
2.21512604
3.30553889
2.30124092
time used by Curl:
0.68719101
0.64675593
0.64326
0.81983113
0.63956594
It's a big gap, isn't it? Oh, from the experience I use, these two tools are not only the speed of the difference, the stability is also very big difference. It is suggested that the network data crawl stability requirement is higher friend uses above curl_file_get_contents function, not only stable speed fast, but also can fake the browser to deceive the target address Oh!
http://www.bkjia.com/PHPjc/327449.html www.bkjia.com true http://www.bkjia.com/PHPjc/327449.html techarticle have done a lot of crawling other site content products, accustomed to the use of convenient and quick file_get_contents function, but always encounter the problem of obtaining failure, although according to the example in the manual ...
