Curl can download 115 of the resources

Source: Internet
Author: User
Tags http request header parameters
Can curl download 115 of resources
1 300MB of resources, with thunder Download (non-115VIP), has set the original thread 1, but the average speed of only 10 KB, down to 60% when the prompt error. Analyzed the URL address law, found that the URL of the &t=1368261625 is a variable, estimated 115 based on this time to handle the download expired transactions.

Open with Ultraediter. Td.cfg, modified the inside of the &t=1368261625 for the current time, restart the Thunderbolt, found that the file was re-downloaded (before the 60% Wood has). Do not know how to modify the Thunderbolt, can be made to modify the download URL can continue the previous download. (Thunder should also have a file parameter comparison method, but not found)

So I think of php curl Download 115 of resources (PHP can arbitrarily modify the URL address), but PHP default timeout is 30 seconds, if Set_time_limit (0), then, how to determine the resource address of 115 has been invalidated, that is, the PHP file has been run?
You want to run the PHP script with a. bat file, or directly at command prompt dos.

Problem:
1. Save the file with a Curl breakpoint, will it start again from 0% on the second download?
2. If problem 1 is feasible, how can I write the code and know that the PHP file has been run after a long connection? (This re-runs. bat or dos for breakpoint download)

PS: On-line only 115 upload class code, Wood download class code, prawns to help challenge, if possible, is the original + essence, you can get CSDN blog.

Share to:


------Solution--------------------
Ignore the rest of your content and ask only one sentence:
Why do you want PHP to load curl If you're going to use the command line? Is it okay to use Curl.exe directly?
------Solution--------------------
I'm going to tell you about the principle of continuation, and you think you can do it.

HTTP continuation, with HTTP request header parameters
Range:bytes=aaaa-bbbb/cccc
The response head must be content-length, too.
AAAA is the start position, BBBB is the end position, CCCC is the file length

Multithreading is actually the file slices, send n requests, each AAAA and bbbb points to different segments, and then splicing
The continuation is the location of the aaaa in the file that has been written to the hard drive, continue to download
Multithreading + continuation, generally need a record, note the completion of each slice, and so on
Therefore, the continuation must meet the three-party: the client can send Range,proxy is the connection mechanism rather than the forwarding mechanism, the server can accept the range request

If you use tools to download, generally, the tool does not consider too much, the record file must correspond to a URL and a local file
When the URL changes, the tool is difficult to "intelligently" select the previous record file to confirm the downloaded section
On the other hand, if multithreading is not a pre-written hard drive, it is more troublesome

Write a socket yourself or you can do it.

In addition,/m00/07/9c/d5njh05tqrmaaaaaf2wejgzscdk3421731/such a path, based on experience can be confirmed as a temporary path, the server to its processing method is difficult to anticipate, delete or prohibit access to or ...
------Solution--------------------
Your URL does not support the continuation of the breakpoint!
I put a piece of test code for you to study. But it's going to take a bit of work to be practical.
$url = ' Http://blog.51edm.org/content/uploadfile/201303/dc7f1364286796.zip ';
function Curl_get ($durl) {
$cookiejar = Realpath (' cookie.txt ');
$t = Parse_url ($durl);
$ch = Curl_init ();
curl_setopt ($ch, Curlopt_url, $durl);
curl_setopt ($ch, Curlopt_header, 1);
curl_setopt ($ch, Curlopt_range, ' 0-300 '); Take 301 bytes from 0 offset
curl_setopt ($ch, curlopt_nobody, 1);

curl_setopt ($ch, curlopt_timeout,5);
curl_setopt ($ch, curlopt_useragent, $_server[' http_user_agent ');
curl_setopt ($ch, Curlopt_referer, "http://$t [host]/");
curl_setopt ($ch, Curlopt_cookiejar, $cookiejar);
curl_setopt ($ch, curlopt_returntransfer,1);
curl_setopt ($ch, curlopt_followlocation, true);
curl_setopt ($ch, curlopt_headerfunction, ' head_func ');
curl_setopt ($ch, Curlopt_writefunction, "progress_function");
$r = curl_exec ($ch);
Curl_close ($ch);
return $r;
}
Echo ' <xmp>'; <br/>curl_get ($url); <br/><br/>//callback function $str Read content <br/>function progress_function ($ch , $str) {<br/>//print_r (Curl_getinfo ($ch)), <br/> Echo php_eol, strlen ($STR);//, ': ', substr ($str, 0,20); <br/> Return strlen ($str) <br/>}<br/>//callback function for obtaining header information <br/>function head_func ($ch, $STR) {< BR/>echo $str; <br/> Return strlen ($STR); <br/>}<br/></pre><br/> Header information similar to <BR/>c ontent-length:301 <li ><i class= "Layui-icon" >& #xe63a; </xmp>
Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.