Use PHP's curl extension to simulate browser access to Web pages

We generally want to crawl a static page of the site to complete our needs, in fact, there are many ways to complete,

PHP built-in function file_get_contents (); file (); ReadFile (); You can crawl Web pages, but this way is a big

limitations, such as the need to access a website that needs to be logged in, requires login verification information, which is after PHP built-in functions appear weak

So we found a PHP extension class curl to help us do this kind of thing, curl is one of the extensions of PHP we just need to install it

And in the php.ini file open his extension can be used, here we do not repeat, in fact, curl use is quite simple below

We give a simple demo:

<?php
$url = "www.baidu.com";//Take Baidu as an example
$data = Array ();
$curl = Curl_init ();//Initialize a curl session;

curl_setopt ($curl, Curlopt_url, $url);//Specify the URL of the access

curl_setopt ($curl, Curlopt_post, $data);//post the requested parameter,

curl_setopt ($curl, curlopt_returntransfer,1);//The information obtained is returned as a file stream

$data = curl_exec ($curl);//Perform curl;
Var_dump ($data);
Curl_close ($curl); Turn off the Curl session

?>

This completes a curl session, does everyone feel that there is nothing special about it? , don't worry, let's see his essence is to pass parameters, of course, what cookies Ah! This is what you can do with this verification.

Curl parameter Many here do not introduce;

It is also recommended that you take a look at Snoopy crawl class is also a good tool, Simple_html_dom document parsing is very good.

Use PHP's curl extension to simulate browser access to Web pages