Can curl download 115 of the resources?

Source: Internet
Author: User
Tags http request header parameters
Whether curl can download 115 of resources 1 MB of resources, with thunder (non-115VIP), the original thread 1 has been set, but the average speed is only 10 KB, an error is prompted at the bottom of 60%. After analyzing the URL address rules, we found that the & nbsp; & amp; t = 1368261625 & nbsp; curl in the URL can download 115 of resources.
1 60% MB resource, with thunder download (not 115VIP), the original thread 1 has been set, but the average speed is only 10 KB, and an error is prompted at the bottom. After analyzing the URL address rules, it is found that & t = 1368261625 in the URL is a variable. it is estimated that 115 will process the download expiration transaction based on this time.
 
Use Ultraediter to open. td. cfg, modify & t = 1368261625 in it to the current time, restart thunder, and find that the file has been downloaded again (the previous 60% has been installed ). I don't know how to modify Thunder so that the previous download can be continued after the download URL is modified. (Thunder should have another method for comparing file parameters, but it cannot be found)

So I thought of using php curl to download 115 of resources (PHP can arbitrarily modify the URL address), but PHP's default timeout is 30 seconds, if set_time_limit (0, how does one determine that 115 of the resource address has expired, that is, the PHP file has been run?
Use the. bat file, or directly run DOS to run the php script.

Problem:
1. if you use curl to save a file, will it start from 0% again during the second download?
2. if Problem 1 is feasible, how can I write code? after a persistent connection, can I know that the php file has been running successfully? (Re-run. bat or DOS for breakpoint download)

PS: only 115 of the uploaded class code is available on the Internet, and the download class code is available on the wooden pages. let's take a look at the challenge. if possible, the original + excellent posts can be obtained from the CSDN blog. Share:
------ Solution --------------------
Ignore your other content and just ask:
Why does php need to load curl? Can I simply use curl.exe?
------ Solution --------------------
Let me talk about the principle of resumable Upload. can you do it yourself?

Http request header parameters are indispensable for http resume
Range: bytes = aaaa-bbbb/cccc
The response header must also contain Content-Length.
Aaaa indicates the start position, bbbb indicates the end position, and cccc indicates the file length.

Multithreading is actually to slice the file, send N requests, each aaaa and bbbb point to different segments, and then splice
Resume is to set aaaa to the location of the file that has been written to the hard disk and continue downloading.
Multi-thread + resume, generally requires a record, recording the complete position of each slice, etc.
Therefore, the resume must meet the requirements of the three parties: the client can send range, and the proxy is a connection mechanism instead of a forwarding mechanism. the server can accept range requests.

If you use a tool to download the file, generally, the tool does not take too much into account, and the record file must correspond to a url and a local file.
When the url changes, it is difficult for the tool to intelligently select the previous record file to confirm the downloaded part.
On the other hand, it would be more difficult to use multithreading instead of pre-writing hard disks.

Write a socket by yourself.

In addition, the path/M00/07/9 C/d5NjH05TqRMAAAAAF2WEJGZScDk3421731/can be confirmed as a temporary path based on experience, and the server can hardly predict how it is processed, delete or prohibit access or ......?
------ Solution --------------------
Your url does not support resumable Upload!
I will paste the test code for your research. However, it takes some time to make it practical.
$ Url = 'http: // blog.51edm.org/content/uploadfile/201303/dc7f1364286796.zip ';
Function curl_get ($ durl ){
$ Cookiejar = realpath('cookie.txt ');
$ T = parse_url ($ durl );
$ Ch = curl_init ();
Curl_setopt ($ ch, CURLOPT_URL, $ durl );
// Curl_setopt ($ ch, CURLOPT_HEADER, 1 );
Curl_setopt ($ ch, CURLOPT_RANGE, '0-300 '); // 301 bytes from 0 offset
// Curl_setopt ($ ch, CURLOPT_NOBODY, 1 );

Curl_setopt ($ ch, CURLOPT_TIMEOUT, 5 );
Curl_setopt ($ ch, CURLOPT_USERAGENT, $ _ SERVER ['http _ USER_AGENT ']);
Curl_setopt ($ ch, CURLOPT_REFERER, "http: // $ t [host]/");
Curl_setopt ($ ch, CURLOPT_COOKIEJAR, $ cookiejar );
Curl_setopt ($ ch, CURLOPT_RETURNTRANSFER, 1 );
Curl_setopt ($ ch, CURLOPT_FOLLOWLOCATION, true );
Curl_setopt ($ ch, CURLOPT_HEADERFUNCTION, 'head _ func ');
Curl_setopt ($ ch, CURLOPT_WRITEFUNCTION, "progress_function ");
$ R = curl_exec ($ ch );
Curl_close ($ ch );
Return $ r;
}
Echo' '; <Br/> curl_get ($ url); <br/> // callback function & nbsp; $ str & nbsp; <br/> function & nbsp; progress_function ($ ch, $ str) & nbsp ;{< br/> // print_r (curl_getinfo ($ ch )); <br/> & nbsp; echo & nbsp; PHP_EOL, & nbsp; strlen ($ str );//,':', substr ($ str, 0,20); <br/> & nbsp; return & nbsp; strlen ($ str ); <br/>}< br/> // callback function, used to obtain header information <br/> function & nbsp; head_func ($ ch, & nbsp; $ str) & nbsp; {<br/> echo & nbsp; $ str; <br/> & nbsp; return & nbsp; strlen ($ str ); <br/>}< br/> </pre> <br/> The header information is similar to <br/> Content-Length: & nbsp; 301

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.