PHP's Curl Library features: Crawl pages, post data, and other _php tutorials

Source: Internet
Author: User
Using the PHP Curl Library, you can easily and effectively grab pages. You just need to run a script, then analyze the page you crawled, and then you can get the data you want in a program. Whether you want to take part of the data from a link, or take an XML file and import it into the database, the fear is simply to get the Web content, CURL is a powerful PHP library. This article mainly describes if you use this PHP library.

Enable CURL settings

First, we have to make sure that our PHP opens the library, and you can get this information by using the Php_info () function.

﹤?php
Phpinfo ();
? ﹥

If you can see the following output on a webpage, it means that the Curl Library is turned on.

If you see, then you need to set up your PHP and open the library. If you are under the Windows platform, then very simple, you need to change the settings of your php.ini file, find Php_curl.dll, and cancel the previous semicolon comment on the line. As shown below:

Cancel the comment under the
Extension=php_curl.dll

If you are under Linux, then you need to recompile your PHP, and when editing, you need to open the compile parameters--add "–with-curl" parameter to the Configure command.

A small example

If everything is ready, here is a small routine:

﹤?php
Initialize a CURL object
$curl = Curl_init ();

Set the URL you need to crawl
curl_setopt ($curl, Curlopt_url, ' http://cocre.com ');

Set Header
curl_setopt ($curl, Curlopt_header, 1);

Sets the curl parameter, which requires the result to be saved to a string or output to the screen.
curl_setopt ($curl, Curlopt_returntransfer, 1);

Run Curl, request a Web page
$data = curl_exec ($curl);

Close URL Request
Curl_close ($curl);

Show the data obtained
Var_dump ($data);

How to post data

The above is the code to crawl the Web page, the following is the post data to a page. Suppose we have a URL http://www.example.com/sendSMS.php that handles the form, which can accept two form fields, one is the phone number, and the other is the message content.

﹤?php
$phoneNumber = ' 13912345678 ';
$message = ' This message is generated by curl and PHP ';
$curlPost = ' pnumber= ' . UrlEncode ($phoneNumber). ' &message= '. UrlEncode ($message). ' &submit=send ';
$ch = Curl_init ();
curl_setopt ($ch, Curlopt_url, ' http://www.example.com/sendSMS.php ');
curl_setopt ($ch, Curlopt_header, 1);
curl_setopt ($ch, Curlopt_returntransfer, 1);
curl_setopt ($ch, Curlopt_post, 1);
curl_setopt ($ch, Curlopt_postfields, $curlPost);
$data = Curl_exec ();
Curl_close ($ch);
? ﹥

From the above program we can see that using Curlopt_post to set the HTTP protocol post method instead of the Get method, and then set the post data to Curlopt_postfields.

About proxy servers

The following is an example of how to use a proxy server. Please note that the highlighted code, the code is very simple, I will not have to say more.


$ch = Curl_init ();
curl_setopt ($ch, Curlopt_url, ' http://www.example.com ');
curl_setopt ($ch, Curlopt_header, 1);
curl_setopt ($ch, Curlopt_returntransfer, 1);
curl_setopt ($ch, Curlopt_httpproxytunnel, 1);
curl_setopt ($ch, Curlopt_proxy, ' fakeproxy.com:1080 ');
curl_setopt ($ch, curlopt_proxyuserpwd, ' User:password ');
$data = Curl_exec ();
Curl_close ($ch);
? ﹥

About SSL and cookies

About SSL is the HTTPS protocol, you just need to curlopt_url the connection of http://Into https://. Of course, there is also a parameter called Curlopt_ssl_verifyhost that can be set to verify the site.

For cookies, you need to understand the following three parameters:

Curlopt_cookie, set a COOKIE in the face of the conversation

Curlopt_cookiejar, save a cookie when the session ends

The Curlopt_cookiefile,cookie file.

HTTP Server Authentication

Finally, let's look at the HTTP server Authentication scenario.

﹤?php
$ch = Curl_init ();
curl_setopt ($ch, Curlopt_url, ' http://www.example.com ');
curl_setopt ($ch, Curlopt_returntransfer, 1);
curl_setopt ($ch, Curlopt_httpauth, Curlauth_basic);
curl_setopt (Curlopt_userpwd, ' [Username]:[password] ')

$data = Curl_exec ();
Curl_close ($ch);
? ﹥

For additional information, please refer to the relevant Curl manual.


http://www.bkjia.com/PHPjc/446638.html www.bkjia.com true http://www.bkjia.com/PHPjc/446638.html techarticle using the PHP Curl Library, you can easily and effectively grab pages. You just need to run a script, then analyze the page you crawled, then you can get what you want in a program ...

  • Contact Us

    The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

    If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

    A Free Trial That Lets You Build Big!

    Start building with 50+ products and up to 12 months usage for Elastic Compute Service

    • Sales Support

      1 on 1 presale consultation

    • After-Sales Support

      24/7 Technical Support 6 Free Tickets per Quarter Faster Response

    • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.