Discussion on the efficiency and stability of file_get_contents and Curl _php tutorial

Source: Internet
Author: User
Have done a lot of crawling other site content products, accustomed to the use of quick and easy file_get_contents function, but always encounter the problem of getting failed, although the example set by the manual time-out, can not be effective most of the time:
Copy CodeThe code is as follows:
$config [' context '] = stream_context_create (Array (' http ' = = Array (' method ' = ' "GET",
' Timeout ' = 5//This time-out is unstable and often does not work
)
));

At this point, look at the server connection pool, you will find a bunch of similar errors, you have a headache:
File_get_contents (http://***): failed to open stream ...
Last resort, I installed the Curl library and wrote a function substitution:
Copy CodeThe code is as follows:
functionCurl_file_get_contents ( $durl ) {
$ch = Curl_init ( ) ;
curl_setopt ( $ch ,Curlopt_url , $durl ) ;
curl_setopt ( $ch ,Curlopt_timeout , 5 ) ;
curl_setopt ( $ch ,Curlopt_useragent ,_useragent_ ) ;
curl_setopt ( $ch ,Curlopt_referer ,_referer_ ) ;
curl_setopt ( $ch ,Curlopt_returntransfer , 1 ) ;
$r = curl_exec ( $ch ) ;
Curl_close ( $ch ) ;
return $r ;
}

In this way, there are no more problems than real network problems.
This is the test that someone else has done about curl and file_get_contents:
file_get_contents crawl google.com required number of seconds:
2.31319094
2.30374217
2.21512604
3.30553889
2.30124092
time used by Curl:
0.68719101
0.64675593
0.64326
0.81983113
0.63956594
It's a big gap, isn't it? Oh, from the experience I use, these two tools are not only the speed of the difference, the stability is also very big difference. It is suggested that the network data crawl stability requirement is higher friend uses above curl_file_get_contents function, not only stable speed fast, but also can fake the browser to deceive the target address Oh!

http://www.bkjia.com/PHPjc/327449.html www.bkjia.com true http://www.bkjia.com/PHPjc/327449.html techarticle have done a lot of crawling other site content products, accustomed to the use of convenient and quick file_get_contents function, but always encounter the problem of obtaining failure, although according to the example in the manual ...

  • Related Article

    Contact Us

    The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

    If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

    A Free Trial That Lets You Build Big!

    Start building with 50+ products and up to 12 months usage for Elastic Compute Service

    • Sales Support

      1 on 1 presale consultation

    • After-Sales Support

      24/7 Technical Support 6 Free Tickets per Quarter Faster Response

    • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.