PHP Request remote URL content method

PHP requests a remote URL content with two methods fopen/file_get_contents and curl.

The difference between 1,fopen/file_get_contents and curl

(1) Fopen/file_get_contents each request will be re-made DNS query, does not cache the DNS information. However, Curl automatically caches the DNS information. Requests for Web pages or images under the same domain name require only one DNS query. This greatly reduces the number of DNS queries. So curl has a much better performance than fopen/file_get_contents.
(2) Fopen/file_get_contents is using Http_fopen_wrapper when requesting HTTP, and will not keeplive. and curl can. This makes curl more efficient when multiple links are requested more than once.
(3) Curl can simulate a variety of requests, such as post data, form submission, etc., users can customize the request according to their own needs. Instead, fopen/file_get_contents can only get data using the Get method.

2, if the remote server shutdown, file_get_contents processing method, you can refer to this article,http://www.cnblogs.com/scofi/articles/3607529.html

Companies often have such a business, need to call the third-party company to provide the HTTP interface, the interface provides information to the Web page, the code is written: file_get_contents ("http://example.com/").

One day suddenly received the report of the operation and maintenance colleague, said that the server was hung, the reason is because the file_get_contents function is caused, then why a function will be the server to hang it out? After a detailed query found that the third-party provider interface has been broken, because the interface is broken, only to cause the server to hang out. The problem is analyzed as follows: Our code is "file_get_contents (" http://example.com/")" Get the return content of a URL, if the URL provided by a third-party company responds slowly, or there is a problem, Our server PHP program will be executed to get this URL, we know that in php.ini, there is a parameter max_execution_time You can set the maximum execution time for PHP scripts, but in php-cgi (PHP-FPM), this parameter is not effective. The real ability to control the maximum execution time for PHP scripts is the following parameters in the php-fpm.conf configuration file: <value name= "Request_terminate_timeout" >0s</value>The default value is 0 seconds, that is, the PHP script will continue to execute, when the request more and more cases will cause the php-cgi process to be stuck in the file_get_contents () function, this nginx+php WebServer can no longer process the new PHP request , Nginx will return "502 bad Gateway" to the user. CPU utilization reaches 100%, and the server hangs up for a long time.     Problem solved: We have found the problem, so how can we solve it? At that time, the solution to the problem is to set the PHP time-out, with Set_time_limit; Set the time-out so that it doesn't get stuck. The code went online and found that the server would still hang, as if it didn't work at all. Later checked the data to know, Set_time_limit set is the PHP program time-out, not the file_get_contents function read the URL time-out. Set_time_limit and modify the php.ini file Max_execution_time effect is the same. To set the time-out for the file_get_contents function, you can use the timeout parameter of the resource $context, with the following code:

$opts = Array (' http ' =>array (' method ' = ' = ' GET ', ' timeout ' =>10,)); $context = Stream_context_create ($opts); $html =file_get_contents (' http://www.example.com ', false, $context); Echo $html;

The timeout in the code is the time-out of the file_get_contents read URL.

There is also a saying can change the time-out of the read URL, is to modify the php.ini in the Default_socket_timeoutValue, or Ini_set (' default_socket_timeout ', 10); But I didn't test it, I don't know. With the workaround, the server will not be hung out. Http://www.cnblogs.com/scofi/articles/3607533.html

In the previous article, we said that the Stream_context_create method was used to set the file_get_contents time-out, so what is this method?

We've got the information, stream_context_create. Create and return a text stream and apply various options for a special process such as fopen (), file_get_contents (), the time-out setting of the process, the proxy server, the request method, and the header information settings. This seems to be powerful, not only can set the timeout time, but also set the proxy server, request method and header information, below we test it: the request.php request page is responsible for initiating the request:

<?php $data = Array ("name" = ' test_name ', "content" = ' test_con '); $data = Http_build_query ($data); $opts = Array ('   http ' =>array (     ' method ' = ' "POST",     ' header ' = ') ' content-type:application/ x-www-form-urlencoded\r\n ".               " Content-length: ". strlen ($data)." \ r \ n ".               " cookie:foo=bar\r\n ".               " \ r \ n ",     ' content ' = $data,   )); $cxContext = Stream_context_create ($opts); $sFile = file_get_contents ("http://127.0.0.1/reponse.php", false, $cxContext);    Echo $sFile;? >

reponse.php The requested page:

<?phpvar_dump ($_post); Var_dump ($_cookie);? >

The result after the run is:

String "Array (2) {[" Content "]=> string (8)" Test_con "[" Name "]=> string (9)" Test_name "} Array (1) {[" foo "]=&gt ; String (3) "Bar"} "description file_get_contents can post data and cookie data to the destination URL and obtain the content. Summary of the usage of 3,curl, http://www.cnblogs.com/scofi/articles/3607538.html(1) using Curl,get to obtain data

<?php$url = ' http://www.example.com ';//Initialize a CURL object $ch  = Curl_init ();//Set the urlcurl_setopt you need to crawl ($ch, Curlopt_ URL, $url);//sets the curl parameter, which requires the result to be saved to a string or output to the screen. curl_setopt ($ch, Curlopt_returntransfer, 1);//whether to get the page after the jump curl_setopt ($ch, curlopt_followlocation, 1); $data = Curl_ EXEC ($ch); Curl_close ($ch); Echo $data; >

(2) Use curl. Post get Data

<?phpfunction curl_post ($url, $arr _data) {   $post _data = http_build_query ($url _data);   $ch = Curl_init ();    curl_setopt ($ch, Curlopt_url, $url);    curl_setopt ($ch, Curlopt_returntransfer, 1);    curl_setopt ($ch, Curlopt_post, 1);    curl_setopt ($ch,  curlopt_postflelds, $post _data);    $data = curl_exec ($ch);    Curl_close ($ch);    echo $data;} $arr _post = Array (    ' name ' = ' test_name ',    ' age ' =   1); Curl_post ("http://www.explame.com/", $arr _post);? >

(3) Use the agent to crawl the page, what to use the agent to crawl it? Take Google, for example, if you catch Google's data, catch it very often in a short time, you will not crawl. Google restrictions on your IP address this time, you can change agent re-capture.

<?php$ch = Curl_init (); curl_setopt ($ch, Curlopt_url, "http://google.com"); curl_setopt ($ch, Curlopt_header, false);  curl_setopt ($ch, Curlopt_returntransfer, 1);//whether to transmit curl_setopt via HTTP proxy ($ch, Curlopt_httpproxytunnel, TRUE); curl_ Setopt ($ch, Curlopt_proxy, 125.21.23.6:8080);  Url_setopt ($ch, curlopt_proxyuserpwd, ' User:password '); If you want a password, add this  $result =curl_exec ($ch); Curl_close ($ch) ;? >

(4) Continue to maintain the session call, in the implementation of the user synchronization login needs to share the session, if you want to continue to maintain the session of the site, then put the SessionID into the HTTP request.

<?php$session_str = Session_name (). ' = '. session_id (). '; path=/; Domain=.explame.com '; Session_write_close (); Writes data to the file and ends Session$ch = Curl_init (); curl_setopt ($ch, Curlopt_url, $url); curl_setopt ($ch, Curlopt_header, false); curl_setopt ($ch, Curlopt_returntransfer, 1); curl_setopt ($ch, Curlopt_cookie, $session _str); $ret = curl_exec ($ch); Curl_close ($ch);? >

PHP Request remote URL content method