PHP's cURL Library easily and effectively captures webpage _ PHP Tutorial

PHP cURL Library easily and effectively captures webpages. Using the PHP cURL library, you can easily and effectively capture webpages. You only need to run a script and analyze the web page you crawled, then, you can get the desired PHP cURL library in the form of a program, which can easily and effectively capture webpages. You only need to run a script, analyze the web page you crawled, and then you can get the data you want as a program. Whether you want to retrieve part of the data from a link or an XML file and import it to the database, you may simply get the webpage content, cURL is a powerful PHP library. This article describes how to use this PHP library.

Enable cURL settings

First, we must first determine whether PHP has enabled this library. you can use the php_info () function to obtain this information.

Phpinfo ();
?>

If you can see the following output on the webpage, it indicates that the cURL Library has been enabled.

If you see this, you need to set your PHP and enable this library. If you are on Windows, it is very simple. you need to modify the settings of your php. ini file, find php_curl.dll, and cancel the semicolon comment. As follows:

// Cancel the comment
Extension = php_curl.dll

If you are in Linux, you need to recompile your PHP. during editing, you need to open the compilation parameter -- add the "-with-curl" parameter to the configure command.

A small example

If everything is ready, the following is a small routine:

// Initialize a cURL object
$ Curl = curl_init ();

// Set the URL you want to capture
Curl_setopt ($ curl, CURLOPT_URL, 'http: // cocre.com ');

// Set the header
Curl_setopt ($ curl, CURLOPT_HEADER, 1 );

// Set the cURL parameter to save the result to the string or output to the screen.
Curl_setopt ($ curl, CURLOPT_RETURNTRANSFER, 1 );

// Run cURL to request the webpage
$ Data = curl_exec ($ curl );

// Close the URL request
Curl_close ($ curl );

// Display the obtained data
Var_dump ($ data );
?>

How to POST data

The above is the code for capturing web pages, and the following is to POST data to a web page. Suppose we have a form processing URL http://www.example.com/sendsms.php, which can accept two single table domain, one is the phone number, and the other is the SMS content.

$ PhoneNumber = '000000 ';
$ Message = 'This message was generated by curl and php ';
$ CurlPost = 'pnumber = '. urlencode ($ phoneNumber).' & MESSAGE = '. urlencode ($ message).' & SUBMIT = send ';
$ Ch = curl_init ();
Curl_setopt ($ ch, CURLOPT_URL, 'http: // www.example.com/sendSMS.php ');
Curl_setopt ($ ch, CURLOPT_HEADER, 1 );
Curl_setopt ($ ch, CURLOPT_RETURNTRANSFER, 1 );
Curl_setopt ($ ch, CURLOPT_POST, 1 );
Curl_setopt ($ ch, CURLOPT_POSTFIELDS, $ curlPost );
$ Data = curl_exec (); curl_close ($ ch );
?>

From the above procedure, we can see that using CURLOPT_POST to set the POST method of the HTTP protocol, instead of the GET method, and then setting the POST data with CURLOPT_POSTFIELDS.

About proxy servers

The following is an example of how to use the proxy server. Pay attention to the highlighted code. the code is very simple and I don't need to talk about it.

$ Ch = curl_init ();
Curl_setopt ($ ch, CURLOPT_URL, 'http: // www.example.com ');
Curl_setopt ($ ch, CURLOPT_HEADER, 1 );
Curl_setopt ($ ch, CURLOPT_RETURNTRANSFER, 1 );
Curl_setopt ($ ch, CURLOPT_HTTPPROXYTUNNEL, 1 );
Curl_setopt ($ ch, CURLOPT_PROXY, 'fakeproxy. com: 1080 ');
Curl_setopt ($ ch, CURLOPT_PROXYUSERPWD, 'User: password ');
$ Data = curl_exec (); curl_close ($ ch );
?>

About SSL and Cookie

For SSL, that is, the HTTPS protocol, you only need to change the http: // in the CURLOPT_URL connection to https. Of course, the parameter CURLOPT_SSL_VERIFYHOST can be set as a verification site.

For Cookie, you need to understand the following three parameters:

CURLOPT_COOKIE: set a cookie in the face-to-face session.

CURLOPT_COOKIEJAR saves a Cookie when the session ends.

CURLOPT_COOKIEFILE: the Cookie file.

HTTP server authentication

Finally, let's take a look at the situation of HTTP server authentication.

$ Ch = curl_init ();
Curl_setopt ($ ch, CURLOPT_URL, 'http: // www.example.com ');
Curl_setopt ($ ch, CURLOPT_RETURNTRANSFER, 1 );
Curl_setopt ($ ch, CURLOPT_HTTPAUTH, CURLAUTH_BASIC );
Curl_setopt (CURLOPT_USERPWD, '[username]: [password]')

$ Data = curl_exec ();
Curl_close ($ ch );
?>

For more information, see the cURL manual.

Bytes. You only need to run a script, analyze the web page you crawled, and then you can get what you want in the form of a program...