Whether curl can download 115 of resources

Whether curl can download 115 of resources 1 MB of resources, with thunder (non-115VIP), the original thread 1 has been set, but the average speed is only 10 KB, an error is prompted at the bottom of 60%. After analyzing the URL address rules, it is found that & t = 1368261625 in the URL is a variable. it is estimated that 115 will process the download expiration transaction based on this time.

Use Ultraediter to open. td. cfg, modify & t = 1368261625 in it to the current time, restart thunder, and find that the file has been downloaded again (the previous 60% has been installed ). I don't know how to modify Thunder so that the previous download can be continued after the download URL is modified. (Thunder should have another method for comparing file parameters, but it cannot be found)

So I thought of using php curl to download 115 of resources (PHP can arbitrarily modify the URL address), but PHP's default timeout is 30 seconds, if set_time_limit (0, how does one determine that 115 of the resource address has expired, that is, the PHP file has been run?
Use the. bat file, or directly run DOS to run the php script.

Problem:
1. if you use curl to save a file, will it start from 0% again during the second download?
2. if Problem 1 is feasible, how can I write code? after a persistent connection, can I know that the php file has been running successfully? (Re-run. bat or DOS for breakpoint download)

PS: only 115 of the uploaded class code is available on the Internet, and the download class code is available on the wooden pages. let's take a look at the challenge. if possible, the original + excellent posts can be obtained from the CSDN blog.

Reply to discussion (solution)

Ignore your other content and just ask:
Why does php need to load curl? Can I simply use curl.exe?

You give it first to identify it.

You give it first to identify it.
Thank you, boss!
Http: // 119.147.99.31/gdown_group121/M00/07/9 C/d5NjH05TqRMAAAAAF2WEJGZScDk3421731/2009_-_No_Line_On_The_Horizon.rar? Bytes

& T = 1368286814, which seems to be the verification time. Information.

Let me talk about the principle of resumable Upload. can you do it yourself?

Http request header parameters are indispensable for http resume
Range: bytes = aaaa-bbbb/cccc
The response header must also contain Content-Length.
Aaaa indicates the start position, bbbb indicates the end position, and cccc indicates the file length.

Multithreading is actually to slice the file, send N requests, each aaaa and bbbb point to different segments, and then splice
Resume is to set aaaa to the location of the file that has been written to the hard disk and continue downloading.
Multi-thread + resume, generally requires a record, recording the complete position of each slice, etc.
Therefore, the resume must meet the requirements of the three parties: the client can send range, and the proxy is a connection mechanism instead of a forwarding mechanism. the server can accept range requests.

If you use a tool to download the file, generally, the tool does not take too much into account, and the record file must correspond to a url and a local file.
When the url changes, it is difficult for the tool to intelligently select the previous record file to confirm the downloaded part.
On the other hand, it would be more difficult to use multithreading instead of pre-writing hard disks.

Write a socket by yourself.

In addition, the path/M00/07/9 C/d5NjH05TqRMAAAAAF2WEJGZScDk3421731/can be confirmed as a temporary path based on experience, and the server can hardly predict how it is processed, delete or prohibit access or ......?

Curl has the CURLOPT_RANGE attribute, allowing resumable data transfer.

But you can still return it last night.
[0] => HTTP/1.1 200 OK
[Server] => nginx
[Date] => Sat, 11 May 2013 15:23:43 GMT
[Content-Type] => application/octet-stream
[Content-Length] = & gt; 392528932
[Last-Modified] => Tue, 23 Aug 2011 13:20:19 GMT
[Connection] => close
[Content-Disposition] => attachment
[Accept-Ranges] => bytes
Returns only today
[0] => HTTP/1.1 403 Forbidden
[Server] => nginx
[Date] => Sun, 12 May 2013 03:20:32 GMT
[Content-Type] => text/html
[Content-Length] = & gt; 4923
[Connection] => close

So I can give up.
Wait until you find other testable URLs.

Curl has the CURLOPT_RANGE attribute, allowing resumable data transfer.
So I can give up.
Wait until you find other testable URLs.

Day, the URL is changed in a few hours. It is too troublesome to post one time.

Http: // 119.147.99.31/gdown_group121/M00/07/9 C/d5NjH05TqRMAAAAAF2WEJGZScDk3421731/2009_-_No_Line_On_The_Horizon.rar? Bytes

Boss, based on your experience, write a rough run code. let me test it myself. thank you.
After the test, I will post the results again.

Your url does not support resumable Upload!
I will paste the test code for your research. However, it takes some time to make it practical.

$ Url = 'http: // curl_get ($ durl) {$ cookiejar = realpath('cookie.txt '); $ t = parse_url ($ durl); $ ch = curl_init (); curl_setopt ($ ch, CURLOPT_URL, $ durl); // curl_setopt ($ ch, CURLOPT_HEADER, 1); curl_setopt ($ ch, CURLOPT_RANGE, '0-300 '); // get 301 bytes from 0 offset // curl_setopt ($ ch, CURLOPT_NOBODY, 1); curl_setopt ($ ch, CURLOPT_TIMEOUT, 5); curl_setopt ($ ch, CURLOPT_USERAGENT, $ _ SERVER ['http _ USER_AGENT ']); curl_setopt ($ ch, CURLOPT_REFERER, "HTTP: // $ t [host]/"); curl_setopt ($ ch, CURLOPT_COOKIEJAR, $ cookiejar); curl_setopt ($ ch, batch, 1); curl_setopt ($ ch, CURLOPT_FOLLOWLOCATION, true); curl_setopt ($ ch, CURLOPT_HEADERFUNCTION, 'head _ func '); curl_setopt ($ ch, CURLOPT_WRITEFUNCTION, "progress_function"); $ r = curl_exec ($ ch); curl_close ($ ch); return $ r;} echo'
 
  
'; Curl_get ($ url); // The callback function $ str is the read content function progress_function ($ ch, $ str) {// print_r (curl_getinfo ($ ch )); echo PHP_EOL, strlen ($ str); //, ':', substr ($ str,); return strlen ($ str);} // callback function, function head_func ($ ch, $ str) {echo $ str; return strlen ($ str );} </pre> <br/> The header information is similar to <br/> Content-Length: 301 <br/> Content-Range: bytes 0-300/1806285 <br/> is useful (this is not available if resumable data transfer is not supported, for example http://bbs.csdn.net/topics/390455233 ) </P> <p class = "sougouAnswer"> your url does not support resumable Upload! <Br/> I am posting test code for your research. However, it may take some time to reach practical results. <br/> thanks to the boss for his guidance, the basic principle is clear, and I wrote a program myself. <Br/> <pre class = "sycode" name = "code"> function curl_115 ($ durl, $ saveto, $ range) {$ ch = curl_init (); curl_setopt ($ ch, CURLOPT_URL, $ durl); curl_setopt ($ ch, CURLOPT_RANGE ,''. ($ range-1) x 1024*1024 ). '-'. ($ range x 1024*1024-1 ). ''); // save 10MBcurl_setopt ($ ch, CURLOPT_USERAGENT, $ _ SERVER ['http _ USER_AGENT ']); curl_setopt ($ ch, CURLOPT_RETURNTRANSFER, 1 ); curl_setopt ($ ch, CURLOPT_HEADER, 0); $ out = curl_exec ($ Ch); curl_close ($ ch); file_put_contents ($ saveto, $ out, FILE_APPEND);} function check_url_115 ($ durl) {$ handle = curl_init ($ durl ); curl_setopt ($ handle, handle, TRUE); curl_setopt ($ handle, CURLOPT_TIMEOUT, 5); $ response = curl_exec ($ handle); $ httpCode = curl_getinfo ($ handle, CURLINFO_HTTP_CODE ); curl_close ($ handle); return response httpcode=if(check_url_115({url}}'200'{{curl_115({url,'1.rar ', 1);} els E {echo 'manually change url ';} </pre> <br/> use file_put_contents's FILE_APPEND to save files at breakpoints <br/> A little unconfident, you can manually change the $ range value, that is, CURLOPT_RANGE ''. ($ range-1) x 1024*1024 ). '-'. ($ range x 1024*1024-1 ). '', is this calculation correct? <Br/> According to your note, <br/> if $ range = 1, 0-1048575 // take 1048576 bytes from 0 offset <br/> if $ range = 2, then 1048576-2097151 // Get the byte 1048576 from the offset of 2097152 <br/> In this case, there are 2 1048576 in the comment, will it cause the last rarfile to be damaged if it is retained? <Br/> In addition, the if (check_url_115 ($ url) = '000000') judgment is basically useless. There is also a terrible problem, that is, CURLINFO_HTTP_CODE = 200, but the URL is invalid when the download is half done. The next CURLOPT_RANGE is not a multiple of 10 m. <Br/> Solution 1: keep it a little conservative. Change $ range every 10 times and manually follow the new URL address. <Br/> Solution 2: simply do not limit 10 M. use the filesize () of PHP to read the rar size saved to the hard disk after the URL expires, and then manually set CURLOPT_RANGE, but are you afraid that the value retrieved by filesize () is actually the value of CURLOPT_RANGE starting again? This is accurate to the byte number. in case of any offset, the entire rar is ruined. <Br/> I hope to give you further advice. thank you. <Br/> </p> <p class = "sougouAnswer"> as snmr_com mentioned earlier, it is important to save the file location. <br/>! <Br/> do not simply use existing functions. calculating the offset is a prerequisite for success. <br/> overwriting is irrelevant, misplacement is important. <br/> You can use a manual or tool to download one program. Comparing the md5 values of the two files, the program is correct. <br/> </p> <p class = "sougouAnswer"> Oh, continue the test ...... </P> <p class = "sougouAnswer"> overnight test: the code is feasible, but it is very difficult to save in 10 MB and captured over 30 times, after each capture, you are afraid of making an error and backing up separately... the key is that this item does not support curl multi. if it is supported, Thunder can handle it. In addition, a 115 browser prompts that non-mobile phone registration is not supported. <Br/> It took two days to create an U2 album ...... <Br/> Conclusion: PHP is still a server-side transaction that can be done within 60 seconds, such as 115 downloads, the software that must be compiled in C language is the right path.