PHP Curl Library: Crawl Web, post data and other, HTTP authentication fetch data

From:http://developer.51cto.com/art/200904/121739.htm

Here is a small routine:

﹤?php Initialize a CURL object $curl = Curl_init (); Set the URL you need to crawl curl_setopt ($curl, Curlopt_url, ' http://cocre.com '); Set Header curl_setopt ($curl, Curlopt_header, 1); Sets the curl parameter, which requires the result to be saved to a string or output to the screen. curl_setopt ($curl, Curlopt_returntransfer, 1); Run Curl, request a Web page $data = curl_exec ($curl); Close URL Request Curl_close ($curl); Show the data obtained Var_dump ($data);

How to post data

The above is the code to crawl the Web page, the following is the post data to a page. Suppose we have a URL http://www.example.com/sendSMS.php that handles the form, which can accept two form fields, one is the phone number, and the other is the message content.

﹤?php $phoneNumber = ' 13912345678 '; $message = ' This message is generated by curl and PHP '; $curlPost = ' pnumber= '. UrlEncode ($phoneNumber). ' &message= '. UrlEncode ($message). ' &submit=send '; $ch = Curl_init (); curl_setopt ($ch, Curlopt_url, ' http://www.example.com/sendSMS.php '); curl_setopt ($ch, Curlopt_header, 1); curl_setopt ($ch, Curlopt_returntransfer, 1); curl_setopt ($ch, Curlopt_post, 1); curl_setopt ($ch, Curlopt_postfields, $curlPost); $data = Curl_exec (); Curl_close ($ch); ? ﹥

From the above program we can see that using Curlopt_post to set the HTTP protocol post method instead of the Get method, and then set the post data to Curlopt_postfields.

About proxy servers

The following is an example of how to use a proxy server. Please note that the highlighted code, the code is very simple, I will not have to say more.

﹤?php $ch = Curl_init (); curl_setopt ($ch, Curlopt_url, ' http://www.example.com '); curl_setopt ($ch, Curlopt_header, 1); curl_setopt ($ch, Curlopt_returntransfer, 1); curl_setopt ($ch, Curlopt_httpproxytunnel, 1); curl_setopt ($ch, Curlopt_proxy, ' fakeproxy.com:1080 '); curl_setopt ($ch, curlopt_proxyuserpwd, ' User:password '); $data = Curl_exec (); Curl_close ($ch); ? ﹥

About SSL and cookies

About SSL is the HTTPS protocol, you just need to curlopt_url the connection of http://Into https://. Of course, there is also a parameter called Curlopt_ssl_verifyhost that can be set to verify the site.

For cookies, you need to understand the following three parameters:

Curlopt_cookie, set a COOKIE in the face of the conversation

Curlopt_cookiejar, save a cookie when the session ends

The Curlopt_cookiefile,cookie file.

HTTP server authentication remote fetch data $url

Finally, let's look at the HTTP server Authentication scenario.

﹤?php $ch = Curl_init (); curl_setopt ($ch, Curlopt_url, $url); curl_setopt ($ch, Curlopt_returntransfer, 1); curl_setopt ($ch, Curlopt_httpauth, Curlauth_basic); curl_setopt (Curlopt_userpwd, ' [Username]:[password] ') $data = Curl_exec (); Curl_close ($ch); ? ﹥

For additional information, please refer to the relevant Curl manual.

PHP Curl Library: Crawl Web, post data and other, HTTP authentication fetch data