Resumable File Download (implementation of the client and server) and resumable File Download

[Transfer] resumable upload of object downloads (implementation of the client and server), resumable upload of object downloads

[Switch] resumable upload of file downloads (implementation of the client and server)

I talked about File Upload. Today I will talk about file download.

Old rules are the simplest and most crude. So many simple calculations? How rough is it? I told you I could not write a piece of code. Do you believe it? Directly throwing a file onto the IIS server supports downloading. You can also use TM for resumable data transfer (supported by the IIS server by default ).

Before you paste the code, first understand what is resumable data transfer (Here we are talking about downloading resumable upload.)? How to Implement resumable data transfer?
Resumable download means that half of the downloads are disconnected or paused, and then you can download them. You do not need to download it from scratch.

It's amazing. It's actually very simple. We can think of it as well.
First, the client sends a request (download file) to the server ). The server then responds to the request, including the total file size, file stream start and end location, and content size. How is it implemented?
HTTP/1.1 has a header attribute Range. For example, when you send a requestRange:0-199, Which means that you are requesting data between 0 and 199. Then the server responds to the requestContent-Range: bytes 0-199/250It indicates that you have obtained data from 0 to 199. The total size is 250. (That is, to tell you that there is still data that has not been downloaded ).

Let's draw a picture.

Is it easy? Such a magical thing is just a "Convention", that is, the so-called HTTP protocol.
However, agreement does not exist if you keep it. Just as everyone in the Republic of China believed it, it was useful. If most people do not believe it, it will be useless.
This is also true for resumable upload. Your server supports resumable data transfer if it does not comply with the requirements. Therefore, when writing a download tool, we need to determine whether the response message containsContent-RangeTo determine whether resumable data transfer is supported.
There is enough nonsense, and the sleeves can be opened below.

File Download-the server uses tag a to download files.

Use the tag to download an object, that is, the object can be downloaded without writing code. Directly put the file on the iis server, and then paste the link to the tag.

<A href = "/new folder 2.rar"> download </a>

It's simple and rude. If it is so good, then everyone will not bother writing other download logic. There is a fatal drawback here. The download provided in this method is not secure enough. Anyone can download the file without permission control, and may be scanned by another file (it seems that csdn has had such a problem ).

Use Response. TransmitFile to download files

As mentioned above, it is not safe to directly download the tag. Then how can we provide secure download. Asp.net defaultApp_DataFolder cannot be accessed directly, so we can put the downloaded file here. Then, when downloading, we read the file and return it to the response stream.

// File Download public void FileDownload5 () {// you can perform user logon verification and user permission verification before. String filename = ".rar"; // file name saved by the client string filePath = Server. mapPath ("/App_Data/ .rar"); // the file path to be downloaded: Response. contentType = "application/octet-stream"; // binary stream Response. addHeader ("Content-Disposition", "attachment; filename =" + filename); Response. transmitFile (filePath); // write the specified file to the HTTP Response output stream}

Other methods for File Download

When searching for C # files on the Internet, the so-called "four methods" are usually found ". In fact, those codes cannot be used directly.
First: (Response. BinaryWrite)

Public void FileDownload2 () {string fileName = "new folder 2.rar"; // the file name saved by the Client: string filePath = Server. mapPath ("/App_Data/new folder 2.rar"); // the file path to be downloaded: Response. contentType = "application/octet-stream"; // binary stream // notify the browser to download the file instead of opening Response. addHeader ("Content-Disposition", "attachment; filename =" + HttpUtility. urlEncode (fileName, System. text. encoding. UTF8); // download the file using (FileStream fs = new FileStream (filePath, FileMode. open, FileAccess. read) {Response. addHeader ("Content-Length", fs. length. toString (); // memory overflow is easy here // theoretically the maximum length of the array int. maxValue 2147483647 // (in reality, different programs can be assigned different values. My machine, winfrom (2147483591 difference 56), iis (almost 2 GB), iis Express (only more than 100 MB) byte [] bytes = new byte [(int) fs. length]; fs. read (bytes, 0, bytes. length); Response. binaryWrite (bytes);} Response. flush (); Response. end ();}

First, the maximum length of the array is int. MaxValue, and then the normal program will not split into such a large memory, it is easyMount servers. (That is, the maximum number of files that can be downloaded is 2 GB.) [Not recommended]

Second: (Response. WriteFile)

Public void FileDownload3 () {string fileName = "new folder 2.rar"; // the file name saved by the Client: string filePath = Server. mapPath ("/App_Data/new FOLDER 2.rar"); // path of the file to be downloaded: FileInfo fileInfo = new FileInfo (filePath); Response. clear (); Response. clearContent (); Response. clearHeaders (); Response. addHeader ("Content-Disposition", "attachment; filename = \" "+ HttpUtility. urlEncode (fileName, System. text. encoding. UTF8) + "\" "); Response. addHeader ("Content-Length", fileInfo. length. toString (); // file size Response. addHeader ("Content-Transfer-Encoding", "binary"); Response. contentType = "application/octet-stream"; Response. writeFile (fileInfo. fullName); // The size parameter must be between zero and the maximum Int32 value (that is, the maximum 2G, but this operation is very memory-consuming) // The memory overflow Response is easy here. flush (); Response. end ();}

The problem is similar to the first one, that is, files larger than 2 GB cannot be downloaded. Then, when downloading files of about 2 GB, the machine is on the verge of being hung, which is quite scary. [Not recommended]

Third: (Response. OutputStream. Write)

Public void FileDownload4 () {string fileName = ".rar"; // the file name saved by the Client: string filePath = Server. mapPath ("/App_Data/ .rar"); // path of the file to be downloaded if (System. IO. file. exists (filePath) {const long ChunkSize = 102400; // 100 K read only 100 K each time, this can relieve the pressure on the server. byte [] buffer = new byte [ChunkSize]; Response. clear (); using (FileStream fileStream = System. IO. file. openRead (filePath) {long fileSize = fileStream. length; // text Response. contentType = "application/octet-stream"; // binary stream Response. addHeader ("Content-Disposition", "attachment; filename =" + HttpUtility. urlEncode (fileName, System. text. encoding. UTF8); Response. addHeader ("Content-Length", fileStream. length. toString (); // total file size while (fileSize> 0 & Response. isClientConnected) // determines whether the client is still connected to the server {// The actual read size int readSize = fileStream. read (buffer, 0, Convert. toInt 32 (ChunkSize); Response. OutputStream. Write (buffer, 0, readSize); Response. Flush (); // if the client suspends the download, it will be blocked. FileSize = fileSize-readSize; // remaining file size} Response. Close ();}}

Here we can see that the output is read cyclically. There is no pressure to download large files. [Recommended]

Fourth: (Response. TransmitFile)
As mentioned above, there is no pressure to download large files. [Recommended]

Public void FileDownload5 () {// you can perform user logon verification and user permission verification before. String filename = ".rar"; // file name saved by the client string filePath = Server. mapPath ("/App_Data/ .rar"); // the file path to be downloaded: Response. contentType = "application/octet-stream"; // binary stream Response. addHeader ("Content-Disposition", "attachment; filename =" + filename); Response. transmitFile (filePath); // write the specified file to the HTTP Response output stream}

File Download-Client

The above implements the implementation of the file download server. Next we implement the implementation of the File Download client. The client download can be provided by a browser, or a download program written by thunder or ourselves. For better analysis, we use the winfrom program to write a download client.

Direct download

Private async void button#clickasync (object sender, EventArgs e) {using (HttpClient http = new HttpClient () {var httpResponseMessage = await http. GetAsync (" http://localhost:813/ New filename folder 2.rar "); // send the request (the link is provided by tag a) var contentLength = httpResponseMessage. content. headers. contentLength; // read the file size using (var stream = await httpResponseMessage. content. readAsStreamAsync () // read the file stream {var readLength = 1024000; // 1000 K the size of each read byte [] bytes = new byte [readLength]; int writeLength; while (writeLength = stream. read (bytes, 0, readLength)> 0) // Read the file stream in multiple parts {using (FileStream fs = new FileStream (Application. startupPath + "/temp.rar", FileMode. append, FileAccess. write) // use the append method to open a file stream {fs. write (bytes, 0, writeLength); // append the Write File contentLength-= writeLength; if (contentLength = 0) // if the Write is complete, the message MessageBox is displayed. show ("download completed ");}}}}}

Looking at such a beautiful code, it seems that there is no problem. The reality is often counterproductive.

We see an exception "System. Net. Http. HttpRequestException:" You cannot write more bytes to the buffer than the configured maximum buffer size of 2147483647 .", What is the ghost? It's the number 2147483647. Because the size of the downloaded file exceeds 2 GB, the download cannot be buffered.
But what is the ghost in "buffer download. I don't know. So can we turn this stuff off? The answer is yes.

Var httpResponseMessage = await http. GetAsync ("http: // localhost: 813/new folder 2.rar"); // send a request

Change it to the following.

Var httpResponseMessage = await http. GetAsync ("http: // localhost: 813/new folder 2.rar", HttpCompletionOption. ResponseHeadersRead); // the operation that should be completed when the title is usable and readable. (Unread content .)

We can see two enumerated values of HttpCompletionOption. One is the response read content, and the other is the response read title (that is, the content in Headers ).

Asynchronous download

We found that when downloading large files, the interface may be suspended. This is a common problem in the UI single-threaded program. Of course, we cannot tolerate such a poor user experience. Next we will open a thread for download to avoid blocking the UI thread.

/// <Summary> /// asynchronous download /// </summary> /// <param name = "sender"> </param> /// <param name = "e"> </param> private async void button2_ClickAsync (object sender, eventArgs e) {// enable an asynchronous thread await Task. run (async () =>{ // Asynchronous Operation UI element label1.Invoke (Action) () =>{ label1.Text = "prepare to download... ";})); long downloadSize = 0; // downloaded long downloadSpeed = 0; // download speed using (HttpClient http = new HttpClient ()) {var httpResponseMessage = await http. getAsync (" http://localhost:813/ New filename folder 2.rar ", HttpCompletionOption. responseHeadersRead); // send the request var contentLength = httpResponseMessage. content. headers. contentLength; // File Size using (var stream = await httpResponseMessage. content. readAsStreamAsync () {var readLength = 1024000; // 1000 K byte [] bytes = new byte [readLength]; int writeLength; var beginSecond = DateTime. now. second; // current time Second while (writeLength = stream. read (bytes, 0, readLen )> 0) {// use the append method to open a file stream using (FileStream fs = new FileStream (Application. startupPath + "/temp.rar", FileMode. append, FileAccess. write) {fs. write (bytes, 0, writeLength);} downloadSize + = writeLength; downloadSpeed + = writeLength; progressBar1.Invoke (Action) () => {var endSecond = DateTime. now. second; if (beginSecond! = EndSecond) // Calculation Speed {downloadSpeed = downloadSpeed/(endSecond-beginSecond); label1.Text = "download speed" + downloadSpeed/1024 + "KB/S"; beginSecond = DateTime. now. second; downloadSpeed = 0; // clear} progressBar1.Value = Math. max (int) (downloadSize * 100/contentLength), 1) ;});} label1.Invoke (Action )(() ==>{ label1.Text = "download completed ";}));}}});}

:

Resumable upload

In the above method, we found that if a half-disconnected network is downloaded, the next download will start again. This is obviously different from our theme today. Next, we will officially enter the topic.Resumable upload for object downloads. Use the header property Range we mentioned above.

Var request = new HttpRequestMessage {RequestUri = new Uri (url)}; request. headers. range = new RangeHeaderValue (rangeBegin, null); // [key point] The number of global variables that have been downloaded, and then the next download starts from this location. Var httpResponseMessage = await http. SendAsync (request, HttpCompletionOption. ResponseHeadersRead );

Complete code:

/// <Summary> /// whether to pause /// </summary> static bool isPause = true; /// <summary> /// download start location (that is, the location where the download has been made) /// </summary> static long rangeBegin = 0; // (of course, this value can also be stored as persistent. Private async void button3_ClickAsync (object sender, EventArgs e) {isPause =! IsPause; if (! IsPause) // click to download {button3.Text = "pause"; await Task. run (async () =>{ // Asynchronous Operation UI element label1.Invoke (Action) () =>{ label1.Text = "prepare to download... ";})); long downloadSpeed = 0; // download speed using (HttpClient http = new HttpClient () {var url =" http://localhost:813/ New file folder 2.rar "; var request = new HttpRequestMessage {RequestUri = new Uri (url)}; request. headers. range = new RangeHeaderValue (rangeBegin, null); // [key point] The number of global variables that have been downloaded, and then the next download starts from this location. Var httpResponseMessage = await http. sendAsync (request, HttpCompletionOption. responseHeadersRead); var contentLength = httpResponseMessage. content. headers. contentLength; // The content size of this request if (httpResponseMessage. content. headers. contentRange! = Null) // if it is null, the server does not support resumable data transfer {contentLength = httpResponseMessage. content. headers. contentRange. length; // file size on the server} using (var stream = await httpResponseMessage. content. readAsStreamAsync () {var readLength = 1024000; // 1000 K byte [] bytes = new byte [readLength]; int writeLength; var beginSecond = DateTime. now. second; // current time Second while (writeLength = stream. read (bytes, 0, readLength)> 0 &&! IsPause) {// use the append method to open a file stream using (FileStream fs = new FileStream (Application. startupPath + "/temp.rar", FileMode. append, FileAccess. write) {fs. write (bytes, 0, writeLength);} downloadSpeed + = writeLength; rangeBegin + = writeLength; progressBar1.Invoke (Action) () => {var endSecond = DateTime. now. second; if (beginSecond! = EndSecond) // Calculation Speed {downloadSpeed = downloadSpeed/(endSecond-beginSecond); label1.Text = "download speed" + downloadSpeed/1024 + "KB/S"; beginSecond = DateTime. now. second; downloadSpeed = 0; // clear} progressBar1.Value = Math. max (int) (rangeBegin) * 100/contentLength), 1) ;}) ;}if (rangeBegin = contentLength) {label1.Invoke (Action )(() ==>{ label1.Text = "download completed" ;}) ;}}}) ;}else // click pause {button3.Text = "Continue download "; label1.Text = "Pause download ";}}

:

So far, do you think our resumable upload is complete?
If you find that the download link we use is a tag. That is, can we write the download link provided by the server to support resumable data transfer? Next I will try another download link.

Resumable data transfer (supported by the server)

The test results are as follows:

Resumable upload is not supported. Why is the tag link directly supported, but we do not support downloading.
The link to the tag is directly directed to the file on iis (which is supported by iis by default), but we did not process the response packet header Range. (Not as intelligent as you think> _ <)

As we mentioned earlier, resumable data transfer is an HTTP protocol. If we observe it, it will exist, and if we do not observe it, it will not exist.
Next, modify the previous File Download Code (server ):

Public void FileDownload5 () {// you can perform user logon verification and user permission verification before. String filename = ".rar"; // file name saved by the client string filePath = Server. mapPath ("/App_Data/ .rar"); // file path to be downloaded var range = Request. headers ["Range"]; if (! String. isNullOrWhiteSpace (range) // if the protocol is followed, supports resumable upload {var fileLength = new FileInfo (filePath ). length; // The total file size long begin; // the start position long end of the file; // The end position long of the file. tryParse (range. split ('=') [1]. split ('-') [0], out begin); long. tryParse (range. split ('-') [1], out end); end = end-begin> 0? End: (fileLength-1); // if no end position exists, read all the remaining records. // the header indicates the start, end, and total file size of the downloaded file. addHeader ("Content-Range", "bytes" + begin + "-" + end + "/" + fileLength); Response. contentType = "application/octet-stream"; Response. addHeader ("Content-Disposition", "attachment; filename =" + filename); Response. transmitFile (filePath, begin, (end-begin); // size of the file to be read at the starting position} else {Response. contentType = "application/octet-stream"; Response. addHeader ("Content-Disposition", "attachment; filename =" + filename); Response. transmitFile (filePath );}}

Then we can test resumable data transfer, which is perfectly supported.

Multi-thread simultaneous download (multipart download)

The resumable upload of files has been analyzed. However, you can complete some details based on your actual needs. For example, the file name, whether the resumable file has changed, and whether the file is consistent with the file on the server after the download is completed.
In addition, we can implement multi-threaded download Based on the Range attribute of the header, but here we will not post the code, paste it. This is the same as multi‑thread upload in the previous file upload. You can also download and view the provided demo code, which includes the complete implementation.

 

References

• Http://www.cnblogs.com/yank/p/HTTP-Range.html
• Http://www.cnblogs.com/sparkdev/p/6141539.html

Demo

• Https://github.com/zhaopeiym/blogdemocode/tree/master/upload and download