Curl Learning and application (with multithread implementation) _php skills

Source: Internet
Author: User
Tags curl diff urlencode ssl certificate

Curl Installation:

Installation under Windows
: Modify the settings of the php.ini file to find Php_curl.dll
Cancel the comment Extension=php_curl.dll
Linux Install below:

Copy Code code as follows:

# wget http://curl.haxx.se/download/curl-7.17.1.tar.gz
# tar ZXVF curl-7.17.1.tar.gz/Decompression
#cd curl-7.17.1
#./configure–prefix=/usr/local/curl
# make
# make Install


This is the method that was installed before installing PHP.
Phpinf to see if the load succeeded!
Use Curl Post data to fly the interface
With Curl wrote the Flying Letter interface bar, there are many online, here just to do a test
Copy Code code as follows:

$username = 13800138000;
$password = 123456;
$sendto = 13912345678;
$message = "Test one try!" ";
$curlPost = ' username= '. UrlEncode ($username). ' &password= '. UrlEncode ($password). ' &sendto= '. UrlEncode ($sendto). ' &message= '. UrlEncode ($message). ';
$ch = Curl_init ();//Initialize Curl
curl_setopt ($ch, Curlopt_url, ' http://sms.api.bz/fetion.php '); crawl the specified page
curl_setopt ($ch, Curlopt_header, 0);/Set HEADER
curl_setopt ($ch, Curlopt_returntransfer, 1);/request result is string and output to screen
curl_setopt ($ch, Curlopt_post, 1);//post Submission method
curl_setopt ($ch, Curlopt_postfields, $curlPost);
$data = curl_exec ($ch);//Run Curl
Curl_close ($ch);
Print_r ($data);//output result


The result returned is: SMS has been submitted to the Send queue!
The address of the Flying letter interface is http://sms.api.bz/
Flying Letter Interface Mode:
Http://sms.api.bz/fetion.php?username= your mobile Letter login mobile phone number
&password= your mobile flying letter login Password
&sendto= receive the message of the Flying friend mobile phone number
&message= SMS Content
Format: http://sms.api.bz/fetion.php?username=13800138000&password=123456&sendto=13912345678&message= SMS Content
Note that to keep the utf-8 format, I made a mistake.

To summarize the use of the Curl method:

Initialize Curl

Use curl_setopt to set the target URL, and other options for detailed reference: http://cn2.php.net/manual/zh/ref.curl.php

Curl_exec, Execute Curl

After execution, close curl
The final step is the output
One of the most wanted curl functions: Curl_getinfo
Curl_getinfo (Resource $ch [, int $opt = 0])

Copy Code code as follows:

<?php
/*curl instance
*/
$curl = Curl_init ();
Set the URL you want to crawl
curl_setopt ($curl, Curlopt_url, ' http://www.baidu.com ');
Set Header
curl_setopt ($curl, Curlopt_header, 0);
Sets the curl parameter to require that the results be saved to the string or to the screen.
curl_setopt ($curl, Curlopt_returntransfer, 1);
Run Curl, request Web page
$data = curl_exec ($curl);
if ($data = = False) {
echo Curl_error ($curl); exit;
}
$info = Curl_getinfo ($curl);
Close URL Request
Curl_close ($curl);

Show the data you get
Var_dump ($info);
Var_dump ($data);



can return:

urlinfo_effective_url– last valid URL address
curlinfo_http_code– the last HTTP code received
curlinfo_filetime– the time the document was remotely fetched, and if it cannot be obtained, the return value is "1"
curlinfo_total_time– time spent on the last transfer
curlinfo_namelookup_time– the time consumed by name resolution
curlinfo_connect_time– time spent establishing a connection
curlinfo_pretransfer_time– the time used to establish a connection to the ready transport
curlinfo_starttransfer_time– the time used to start the connection to the transport
curlinfo_redirect_time– the time used to redirect before the transaction transfer begins
curlinfo_size_upload– The total amount of uploaded data
Total value of curlinfo_size_download– download data volume
curlinfo_speed_download– Average Download speed
curlinfo_speed_upload– Average upload speed
The size of the Curlinfo_header_size–header part
curlinfo_header_out– send the requested string
curlinfo_request_size– the size of the request with the problem in the HTTP request
curlinfo_ssl_verifyresult– the results of authentication requests by setting the SSL certificate returned by Curlopt_ssl_verifypeer
curlinfo_content_length_download– length of download content read from Content-length:field
Description of curlinfo_content_length_upload– upload content size
curlinfo_content_type– download content content-type: null indicates that the server did not send a valid Content-type:header

Using Curl to implement multithreading

Curl is generally used to crawl Web pages, the second is GET or post data, the third application is to implement the multithreaded task of PHP
Here's how to implement multithreading

Copy Code code as follows:

<?php
/*
Curl Multi-threaded crawl
*/
/**
* Curl Multithreading
*
* @param array $array parallel URLs
* @param int $timeout timeout
* @return Array
*/
function Curl_http ($array, $timeout) {
$res = Array ();
$MH = Curl_multi_init ();//Create Multiple Curl handles
$startime = Getmicrotime ();
foreach ($array as $k => $url) {
$conn [$k]=curl_init ($url);

curl_setopt ($conn [$k], curlopt_timeout, $timeout);/Set timeout time
curl_setopt ($conn [$k], Curlopt_useragent, ' mozilla/5.0 (compatible; MSIE 5.01; Windows NT 5.0);
curl_setopt ($conn [$k], Curlopt_maxredirs, 7);//http Orientation level
curl_setopt ($conn [$k], Curlopt_header, 0);/no HEADER here, add block efficiency
curl_setopt ($conn [$k], curlopt_followlocation, 1); 302 redirect
curl_setopt ($conn [$k],curlopt_returntransfer,1);
Curl_multi_add_handle ($MH, $conn [$k]);
}
Preventing dead loops from dying the CPU this paragraph is based on the online wording
do {
$MRC = Curl_multi_exec ($MH, $active);//When there is no data, active=true
while ($MRC = = Curlm_call_multi_perform);////When data is being accepted
while ($active and $MRC = = CURLM_OK) {//when there is no data or when the request is paused, active=true
if (Curl_multi_select ($MH)!=-1) {
do {
$MRC = Curl_multi_exec ($MH, $active);
while ($MRC = = Curlm_call_multi_perform);
}
}

foreach ($array as $k => $url) {
Curl_error ($conn [$k]);
$res [$k]=curl_multi_getcontent ($conn [$k]);//Get return information
$header [$k]=curl_getinfo ($conn [$k]);//Return header information
Curl_close ($conn [$k]);//close handle
Curl_multi_remove_handle ($MH, $conn [$k]); Releasing resources
}

Curl_multi_close ($MH);
$endtime = Getmicrotime ();
$diff _time = $endtime-$startime;

Return Array (' Diff_time ' => $diff _time,
' Return ' => $res,
' Header ' => $header
);

}
Calculate Current time
function Getmicrotime () {
List ($usec, $sec) = Explode ("", Microtime ());
Return ((float) $usec + (float) $sec);
}

Test it, curl three URLs
$array = Array (
"http://www.weibo.com/",
"http://www.renren.com/",
"http://www.qq.com/"
);
$data = Curl_http ($array, ' 10 ');
Var_dump ($data);//Output

?>


For the explanation of Do while:

Because $active to wait for all the URL data to be completed to false, so here we use the return value of curl_multi_exec to determine whether there is data,
When there is data, the call curl_multi_exec, temporarily no data into the Select phase, the new data can be awakened to continue execution.
The advantage here is that the unnecessary consumption of the CPU is gone. More detailed Description: http://hi.baidu.com/%D4%C2%D2%B9%C4%FD%ED%F8/blog/item/9dfcf4fbe6b84374024f563d.html

This multithreaded approach to writing:
First step: Call Curl_multi_init
Step two: Loop call Curl_multi_add_handle
In this step, it should be noted that the second parameter of Curl_multi_add_handle is a curl_init handle.
Step three: Keep calling Curl_multi_exec
Step Fourth: Loop call curl_multi_getcontent to get results as needed
Step Fifth: Call Curl_multi_remove_handle and call curl_close for each word handle
Step Sixth: Call Curl_multi_close
Multi-threaded Test effect diagram:

Summary: 36 HTTP requests, from the execution of the time sequence to look at, three sites of IP crossover, the description is concurrent!
—————————————————————————
Curl under the Linux command
Several common ways to use:
Download function:
Direct download equivalent to wget
Curl-o 1.jpg Http://cgi2.tky.3web.ne.jp/~zzh/screen1.JPG
Bulk Download Screen1. Jpg–screen10. Jpg
Curl-o http://cgi2.tky.3web.ne.jp/~zzh/screen[1-10]. Jpg
Breakpoint Download
Curl-c-O Http://cgi2.tky.3wb.ne.jp/~zzh/screen1.JPG


Reverse proxy function
Curl-x 123.45.67.89:1080-o page.html http://www.yahoo.com
Display header File
Curl-i www.sina.com

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.