Curl is a library dedicated for network interaction. It provides a bunch of custom options to deal with different environments. the stability is naturally greater than file_get_contents curl, which is a library dedicated for network interaction, A bunch of custom options are provided to cope with different environments. the stability is naturally higher than file_get_contents.
Reason for curl selection
For curl and file_get_contents, the following is an easy-to-understand comparison:
File_get_contents is actually a combined version of a bunch of built-in file operation functions, such as file_exists, fopen, fread, and fclose. it is designed for lazy users and is mainly used to deal with local files, but it is also because of laziness, and added support for network files;
Curl is a library dedicated for network interaction. It provides a bunch of custom options to deal with different environments. the stability is naturally greater than file_get_contents.
Usage
1. enable curl support
After the php environment is installed, curl is not enabled by default. you need to modify the php. ini file and find it. extension = php_curl.dll, remove the colon, and restart the service;
2. use curl to capture data
The code is as follows:
// Initialize a cURL object $ curl = curl_init (); // Set the URL curl_setopt ($ curl, CURLOPT_URL, 'http: // www.cmx8.cn') to be crawled '); // Set the header curl_setopt ($ curl, CURLOPT_HEADER, 1); // sets the cURL parameter and requires that the result be saved to the string or output to the screen. Curl_setopt ($ curl, CURLOPT_RETURNTRANSFER, 1); // run cURL, request webpage $ data = curl_exec ($ curl); // close URL request curl_close ($ curl );
3. find key data through regular expression matching
The code is as follows:
// $ Data is the value returned by curl_exec, that is, the collected target content preg_match_all ("/(.*?) <\/Li>/", $ data, $ out, PREG_SET_ORDER); foreach ($ out as $ key => $ value) {// Here $ value is an array, at the same time, the record finds the entire sentence with matching characters and the individually matched character echo 'matched entire sentence :'. $ value [0]. ''; echo 'matched separately :'. $ value [1]. '';}
Tips
1. timeout settings
You can use curl_setopt ($ ch, opt) to set some timeout settings, including:
CURLOPT_TIMEOUT sets the maximum number of seconds that cURL can be executed.
CURLOPT_TIMEOUT_MS sets the maximum number of milliseconds that cURL can be executed. (Added in cURL 7.16.2. It can be used from PHP 5.2.3. )
CURLOPT_CONNECTTIMEOUT is the waiting time before the connection is initiated. if it is set to 0, it will wait infinitely.
CURLOPT_CONNECTTIMEOUT_MS indicates the waiting time for the connection attempt, in milliseconds. If it is set to 0, the system waits for no limit. Added to cURL 7.16.2. Available from PHP 5.2.3.
CURLOPT_DNS_CACHE_TIMEOUT sets the time for saving DNS information in the memory. the default value is 120 seconds.
The code is as follows:
Curl_setopt ($ ch, CURLOPT_TIMEOUT, 60); // you only need to set the number of seconds to curl_setopt ($ ch, CURLOPT_NOSIGNAL, 1); // note, you must set this curl_setopt ($ ch, CURLOPT_TIMEOUT_MS, 200) for millisecond timeout; // timeout in milliseconds, added to cURL 7.16.2. Available from PHP 5.2.3
2. submit data through post to retain cookies
The code is as follows:
// Here is an example to learn from: // Curl simulated login discuz program, suitable for DZ7.0! Extension_loaded ('curl') & die ('The curl extension is not loaded. '); $ discuz_url = 'http: // www.lxvoip.com'; // forum address $ login_url = $ discuz_url. '/logging. php? Action = login '; // logon page address $ get_url = $ discuz_url.'/my. php? Item = threads '; // my post $ post_fields = array (); // You do not need to modify the following two items: $ post_fields ['loginfield'] = 'username '; $ post_fields ['loginsubmit '] = 'true'; // username and password, which must be set to $ post_fields ['username'] = 'lxvoip '; $ post_fields ['password'] = '000000'; // security question $ post_fields ['questionid'] = 0; $ post_fields ['ancer'] = ''; // @ todo verification code $ post_fields ['seccodeverify '] = ''; // Obtain the form FORMHASH $ ch = curl_init ($ login_url); curl_setopt ($ ch, CURLOPT_HEADER, 0 ); curl_setopt ($ ch, CURLOPT_RETURNTRANSFER, 1); $ contents = curl_exec ($ ch); curl_close ($ ch); preg_match ('/
/I ', $ contents, $ matches); if (! Empty ($ matches) {$ formhash = $ matches [1];} else {die ('Not found the forumhash. ');} // POST the data to obtain the COOKIE $ cookie_file = dirname (_ FILE __). '/cookie.txt'; // $ cookie_file = tempnam ('/tmp'); $ ch = curl_init ($ login_url); curl_setopt ($ ch, CURLOPT_HEADER, 0 ); curl_setopt ($ ch, batch, 1); curl_setopt ($ ch, CURLOPT_POST, 1); curl_setopt ($ ch, CURLOPT_POSTFIELDS, $ post_fields); curl_setopt ($ ch, CURLOPT_COOKIEJAR, $ cookie_file); curl_exec ($ ch); curl_close ($ ch ); // Obtain the page content that requires logon with the COOKIE obtained above $ ch = curl_init ($ get_url); curl_setopt ($ ch, CURLOPT_HEADER, 0); curl_setopt ($ ch, CURLOPT_RETURNTRANSFER, 0); curl_setopt ($ ch, CURLOPT_COOKIEFILE, $ cookie_file); $ contents = curl_exec ($ ch); curl_close ($ ch); var_dump ($ contents );
The above is the details of the php curl implementation of out-of-site collection (recommended). For more information, see other related articles in the first PHP community!