PHP Crawl HTTPS Web content method and error handling
Recently encountered an HTTPS problem while studying the hacker News API. Because all hacker News APIs are accessed through an encrypted HTTPS protocol, unlike the normal HTTP protocol, when using PHP's function file_get_contents () to get the data provided in the API, an error occurs, using the code:
When you run the above code, you encounter the following error message:
PHP Warning: file_get_contents (): Unable to find the wrapper "https"-do you forget to enable it when you configured Php?
Why is there such a mistake?
After a search on the internet, found that there are many people who encountered such errors, the problem is very straightforward, because in the PHP configuration file does not open a parameter, on my machine is/apache/bin/php.ini; Extension=php_openssl.dll this one, You need to remove the preceding semicolon. You can use the following script to check the configuration of your PHP environment:
$w = Stream_get_wrappers (); Echo ' OpenSSL: ', extension_loaded (' OpenSSL ')? ' Yes ': ' no ', ' \ n '; Echo ' http wrapper: ', In_array (' http ', $w)? ' Yes ': ' no ', ' \ n '; Echo ' https wrapper: ', In_array (' https ', $w)? ' Yes ': ' no ', ' \ n '; Echo ' wrappers: ', Var_dump ($w);
Running the above script fragment, the result on my machine is:
Openssl:nohttp Wrapper:yeshttps Wrapper:nowrappers:array (Ten) { [0]=> string (3) "PHP" [1]=> String (4) "File" [2]=> string (4) "Glob" [3]=> string (4) "Data" [4]=> string (4) "http" [5]=> string (3) "FTP" [6]=> string (3) "Zip" [7]=> string (13) " Compress.zlib " [8]=> string" compress.bzip2 " [9]=> string (4)" Phar "}
Alternative Solutions
It is easy to find errors and correct errors, but it is difficult to find errors after they are found. I originally wanted to put this script method on the remote host, but I could not modify the remote host's PHP configuration, as a result, I could not use this scheme, but we can not be hanged in a tree, this road does not go through, see if there is no other way.
Another function I often use to crawl content in PHP is curl, which is more powerful than file_get_contents () and provides a lot of optional parameters. For the problem of accessing HTTPS content, the CURL configuration parameters we need to use are:
curl_setopt ($ch, Curlopt_ssl_verifypeer, FALSE);
As you can see semantically, it is ignoring/skipping SSL security authentication. It may not be a good idea, but it's enough for a normal scenario.
Here is a function that uses the Curl package to access HTTPS content:
function Gethttps ($url) { $ch = Curl_init (); curl_setopt ($ch, Curlopt_ssl_verifypeer, FALSE); curl_setopt ($ch, Curlopt_header, false); curl_setopt ($ch, curlopt_followlocation, true); curl_setopt ($ch, Curlopt_url, $url); curl_setopt ($ch, Curlopt_referer, $url); curl_setopt ($ch, Curlopt_returntransfer, TRUE); $result = curl_exec ($ch); Curl_close ($ch); return $result;}