CURL learning and application (with multi-thread implementation)

Curl installation:

Installation under windows
: Modify the settings of the php. ini file and find php_curl.dll.
// Cancel the comment extension = php_curl.dll
Install it in linux:
Copy codeThe Code is as follows:
# Wget http://curl.haxx.se/download/curl-7.17.1.tar.gz
# Tar zxvf curl-7.17.1.tar.gz // unzip
# Cdcurl-7.17.1
#./Configure-prefix =/usr/local/curl
# Make
# Make install


This is the method installed before installing php.
* ************************** Phpinf to check whether the load is successful!
Use the POST data fetion interface of curl
I used curl to write the Feixin interface. There are many online interfaces. Here I just want to test it.
Copy codeThe Code is as follows:
$ Username = 13800138000;
$ Password = 123456;
$ Sendto = 13912345678;
$ Message = "Try it out! ";
$ CurlPost = 'username = '. urlencode ($ username ). '& password = '. urlencode ($ password ). '& sendto = '. urlencode ($ sendto ). '& message = '. urlencode ($ message ). '';
$ Ch = curl_init (); // initialize curl
Curl_setopt ($ ch, CURLOPT_URL, 'HTTP: // sms. api. bz/fetion. php'); // capture the specified webpage
Curl_setopt ($ ch, CURLOPT_HEADER, 0); // set the header
Curl_setopt ($ ch, CURLOPT_RETURNTRANSFER, 1); // The result is a string and output to the screen.
Curl_setopt ($ ch, CURLOPT_POST, 1); // post submission method
Curl_setopt ($ ch, CURLOPT_POSTFIELDS, $ curlPost );
$ Data = curl_exec ($ ch); // run curl
Curl_close ($ ch );
Print_r ($ data); // output result


The message is submitted to the sending queue!
The address of the fetion interface is http://sms.api.bz/
Fetion interface mode:
Http://sms.api.bz/fetion.php? Username = your mobile phone number
& Password = your mobile Feixin logon password
& Sendto = mobile phone number of a friend of the flying mail who receives the SMS
& Message = text message content
The format: http://sms.api.bz/fetion.php? Username = 13800138000 & password = 123456 & sendto = 13912345678 & message = SMS content
Be sure to keep the UTF-8 format. I made a mistake.

To sum up the curl method:

Initialize curl

Use curl_setopt to set the destination url, and other options. For more information about these options, see: http://cn2.php.net/manual/zh/ref.curl.php

Curl_exec: Execute curl

Disable curl
The last step is output.
The most important curl function: curl_getinfo
Curl_getinfo (resource $ ch [, int $ opt = 0])

Copy codeThe Code is as follows:
<? Php
/* Curl instance
*/
$ Curl = curl_init ();
// Set the URL you want to capture
Curl_setopt ($ curl, CURLOPT_URL, 'HTTP: // www.baidu.com ');
// Set the header
Curl_setopt ($ curl, CURLOPT_HEADER, 0 );
// Set the cURL parameter to save the result to the string or output to the screen.
Curl_setopt ($ curl, CURLOPT_RETURNTRANSFER, 1 );
// Run cURL to request the webpage
$ Data = curl_exec ($ curl );
If ($ data = false ){
Echo curl_error ($ curl); exit;
}
$ Info = curl_getinfo ($ curl );
// Close the URL request
Curl_close ($ curl );

// Display the obtained data
Var_dump ($ info );
Var_dump ($ data );


Return Value:

URLINFO_EFFECTIVE_URL-the last valid URL address
CURLINFO_HTTP_CODE-the last HTTP Code received
CURLINFO_FILETIME-the time when the document is obtained remotely. If the document cannot be obtained, the returned value is "-1"
CURLINFO_TOTAL_TIME-the time consumed by the last transmission
CURLINFO_NAMELOOKUP_TIME-time consumed by name resolution
CURLINFO_CONNECT_TIME-time consumed by establishing a connection
CURLINFO_PRETRANSFER_TIME-the time used to prepare the transmission from the established connection
CURLINFO_STARTTRANSFER_TIME-the time used to start the connection
CURLINFO_REDIRECT_TIME-the time used for redirection before the transaction transfer starts
CURLINFO_SIZE_UPLOAD-total uploaded data volume
CURLINFO_SIZE_DOWNLOAD-total volume of downloaded data
CURLINFO_SPEED_DOWNLOAD-average download speed
CURLINFO_SPEED_UPLOAD-average upload speed
CURLINFO_HEADER_SIZE-header Part Size
CURLINFO_HEADER_OUT-request string
CURLINFO_REQUEST_SIZE-the size of the request with errors in the HTTP request
CURLINFO_SSL_VERIFYRESULT-verify the request result by setting the SSL Certificate returned by CURLOPT_SSL_VERIFYPEER
CURLINFO_CONTENT_LENGTH_DOWNLOAD-Length of the downloaded Content read from Content-Length: field
CURLINFO_CONTENT_LENGTH_UPLOAD-Description of the size of the uploaded content
CURLINFO_CONTENT_TYPE-Content-Type: value of the downloaded Content. NULL indicates that the server has not sent a valid Content-Type: header.

Use curl to implement Multithreading

Curl is generally used to capture web pages, the second is get or post data, and the third is to implement PHP multi-thread tasks.
The following describes how to implement multithreading.

Copy codeThe Code is as follows:
<? Php
/*
Curl multi-thread crawling
*/
/**
* Curl Multithreading
*
* @ Param array $ array parallel URL
* @ Param int $ timeout
* @ Return array
*/
Function Curl_http ($ array, $ timeout ){
$ Res = array ();
$ Mh = curl_multi_init (); // create multiple curl handles
$ Startime = getmicrotime ();
Foreach ($ array as $ k => $ url ){
$ Conn [$ k] = curl_init ($ url );

Curl_setopt ($ conn [$ k], CURLOPT_TIMEOUT, $ timeout); // set the timeout time
Curl_setopt ($ conn [$ k], CURLOPT_USERAGENT, 'mozilla/5.0 (compatible; MSIE 5.01; Windows NT 5.0 )');
Curl_setopt ($ conn [$ k], CURLOPT_MAXREDIRS, 7); // HTTp Targeting level
Curl_setopt ($ conn [$ k], CURLOPT_HEADER, 0); // No header here, block Efficiency
Curl_setopt ($ conn [$ k], CURLOPT_FOLLOWLOCATION, 1); // 302 redirect
Curl_setopt ($ conn [$ k], CURLOPT_RETURNTRANSFER, 1 );
Curl_multi_add_handle ($ mh, $ conn [$ k]);
}
// Prevent endless loop consumption of cpu. This section is based on the online statement.
Do {
$ Mrc = curl_multi_exec ($ mh, $ active); // when no data exists, active = true
} While ($ mrc = CURLM_CALL_MULTI_PERFORM); // when receiving data
While ($ active and $ mrc = CURLM_ OK) {// when there is no data or the request is paused, active = true
If (curl_multi_select ($ mh )! =-1 ){
Do {
$ Mrc = curl_multi_exec ($ mh, $ active );
} While ($ mrc = CURLM_CALL_MULTI_PERFORM );
}
}

Foreach ($ array as $ k => $ url ){
Curl_error ($ conn [$ k]);
$ Res [$ k] = curl_multi_getcontent ($ conn [$ k]); // obtain the returned information
$ Header [$ k] = curl_getinfo ($ conn [$ k]); // return header information
Curl_close ($ conn [$ k]); // close the slogan.
Curl_multi_remove_handle ($ mh, $ conn [$ k]); // release resources
}

Curl_multi_close ($ mh );
$ Endtime = getmicrotime ();
$ Diff_time = $ endtime-$ startime;

Return array ('diff _ time' => $ diff_time,
'Return '=> $ res,
'Header' => $ header
);

}
// Calculate the current time
Function getmicrotime (){
List ($ usec, $ sec) = explode ("", microtime ());
Return (float) $ usec + (float) $ sec );
}

// Test the curl URL.
$ Array = array (
"Http://www.weibo.com /",
"Http://www.renren.com /",
"Http://www.qq.com /"
);
$ Data = Curl_http ($ array, '10'); // call
Var_dump ($ data); // output

?>


The explanation of do while:

Because $ active is converted to false only after all url data is accepted, the returned value of curl_multi_exec is used to determine whether data exists,
When data is identified, curl_multi_exec is continuously called. If no data is available for the moment, the select stage is started, and the new data can be awakened to continue execution.
The advantage here is that there is no unnecessary CPU consumption. More detailed description: http://hi.baidu.com/%D4%C2%D2%B9%C4%FD%ED%F8/blog/item/9dfcf4fbe6b84374024f563d.html

The multi-thread writing steps:
Step 1: Call curl_multi_init
Step 2: Call curl_multi_add_handle cyclically
Note that the second parameter of curl_multi_add_handle is the sub-handle from curl_init.
Step 3: continuously call curl_multi_exec
Step 4: Call curl_multi_getcontent cyclically to obtain the result as needed
Step 5: Call curl_multi_remove_handle and call curl_close for each handle.
Step 6: Call curl_multi_close
Multi-threaded testing:

Summary: The execution of 36 http requests is chronological. The ip addresses of the three websites are overlapping, indicating that the requests are concurrent!
-------------------------
Curl under linux commands
Several common usage methods:
Download function:
Direct download is equivalent to wget
Curl-o 1.jpg http://cgi2.tky.3web.ne.jp /~ Zzh/screen1.JPG
Batch download screen1.JPG-screen10.JPG
Curl-O http://cgi2.tky.3web.ne.jp /~ Zzh/screen [1-10]. JPG
Breakpoint download
Curl-c-O http://cgi2.tky.3wb.ne.jp /~ Zzh/screen1.JPG

Reverse Proxy
Curl-x 123.45.67.89: 1080-o page.html http://www.yahoo.com
Show header files
Curl-I www.sina.com