Summary ASP. NET large file upload

Some time ago, a colleague and a buddy asked me about File Upload. Although my work was not very busy, it was really not easy to restore my work if I had to relax, some time ago, I was about to write a topic for control development. During this time, I was under pressure and studied this problem.
All Web developers know thatProgramUploading files is a common requirement. The method for uploading files using HTTP protocol is very limited. Generally, the <input type = "file"/> label is used for uploading. This upload method uses "multipart/form-Data" to encode the content (the original multipart/form-data specification), post the content to the server, and then process it. Compared with the default "application/X-URL-encoded", "multipart/form-Data" has a much higher submission timeliness rate in large data volumes. The biggest advantage of using the <input type = "file"/> label to upload files is that various server technologies have the best encapsulation and can process uploaded files intuitively after development. However, in general, this protocol is not suitable for file transmission, and the cost of parsing data streams is relatively high, and there is no such mechanism as resumable data transfer, as a result, uploading large files often fails.

Principles and Implementation of Web file upload 

This article introduces the use of the <input type = "file"/> label in Java to parse multiple files submitted from the client to the server. The. Net version is roughly similar. In this way, you can upload files by yourself.

ASP. NET Encapsulation

ASP. net 1.x provides an htmlinputfile control, while fileupload is a new control in 2.0, but the HTML generated by it is still the same. It needs to be defined as follows when used in pages:

[ASP. NET 1.x]

< Input ID = "Myfile" Type = "File" Runat = "Server"   />

<G id = "1"> [ASP. NET 2.0]

< ASP: fileupload ID = "Fileupload1" Runat = "Server"   />

In ASP. NET, file upload is synonymous with low efficiency. To be fair, IIS is behind the scenes. When you select a file and press the submit button, IIS needs to parse all the content of the file before you can read the attributes of the uploaded file. iis5.x and IIS6 both do this, the good news is that iis7.0 will use Apache. Unless you change to 7.0 now, you only have to wait for a long time until the upload is complete, and the others have no other method. You cannot display the progress bar because you have no idea how much data has been uploaded during this time.
Anyone who has used the fileupload control knows that it is a double-edged sword-either a savior or an enemy ), one of the most common problems is how to upload large files larger than 4 MB. However, we should understand that the default file size limit is 4 MB, not because the designers take it for granted, but to avoid potential DoS attacks.
To avoid this restriction, You need to modify the maxrequestlength attribute of the httpruntime section of the configuration file. The larger the file, the longer the processing time. This means that you usually have to modify the executiontimeout attribute. This attribute is 1. X is 90 s by default, and 2.0 is 110 S by default. the Config File also has the shutdowntimeout attribute, but I don't know its usefulness.

IIS Process

When a large file is uploaded to the server, no matter how large your maxrequestlength is, IIS extracts the file and then ASP. net according to system. when the size is determined by the Web/httpruntime section, an exception is thrown. This process is determined by the getentirerawcontent () method of httprequest. You can see:

Some code
Httpruntimesection httpruntime = Runtimeconfig. getconfig ( This . _ Context). httpruntime;
Int Maxrequestlengthbytes = Httpruntime. maxrequestlengthbytes;
If ( This . Contentlength > Maxrequestlengthbytes)
{
If ( ! ( This . _ WR Is Iis7workerrequest ))
{
This. Response. closeconnectionaftererror ();
}
Throw   New Httpexception (Sr. getstring ( " Max_request_length_exceeded " ), Null , 0 xbbc );
}

Theoretically, not all files can be loaded, but can I configure IIS to allow it to read files in blocks? At least I don't know!

Disadvantages of ASP. NET

ASP. net processes file uploads. The biggest problem is that the memory usage is too high. Because the entire file is loaded into the memory for processing, if the user uploads too many files or uploads too many users at the same time, this will cause the server to run out of memory. This is actually one-sided, for the early ASP. net 1.x, in order to be processed by the program, the user-uploaded content will be fully loaded into the memory, which indeed brings problems, but in ASP. NET 2.0 already exists in a temporary file on the hard disk after the user uploads more data. This is completely transparent to developers, that is, developers can process data streams as before. In httpruntime, they can set the threshold value (threshold) through the requestlengthdiskthreshold attribute. The default value is 256, that is, when the content of a request exceeds kb, the hard disk is enabled as the cache. This threshold value is irrelevant to whether the client is uploading the content. It only cares that the request sent by the client is greater than this value. Therefore, in ASP. NET 2.0, the server's memory will not be exhausted due to abnormal client requests.
Another drawback is that when the request exceeds maxrequestlength (4 MB by default), ASP. NET will not process the request. This is different from ASP. net throws an exception completely different, which is why if the user uploads a file too large, it does not see ASP. the error page specified in the. NET application (or the default one), because ASP. net has not processed this request. Another problem is processing timeout. In fact, you can read the httpruntime section in Web. config at runtime and convert it to the httpruntimesection object or rewrite it.Page. onerror() To detect the HTTP code (correspondingCode) Is it 400 for processing? I will not go into details here. The Code is as follows:

Runtime code
System. configuration. Configuration config = Webconfigurationmanager. openwebconfiguration ( " ~ " );
Httpruntimesection Section = Config. getsection ( " System. Web/httpruntime " ) As Httpruntimesection;
Double Maxfilesize = Math. Round (section. maxrequestlength /   1024.0 , 1 );
String Errorstring =   String . Format ( " Make sure your file is under {0: 0. #} MB. " , Maxfilesize );

 

Onerror code
Protected   Override   Void Onerror (eventargs E)
{
Httpcontext CTX = Httpcontext. Current;
Exception exception = CTX. server. getlasterror ();

String Errorstring =  
" <Br> offending URL: "   + CTX. Request. url. tostring () +
" <Br> Source: "   + Exception. Source +  
" <Br> message: "   + Exception. Message +
" <Br> stack trace: "   + Exception. stacktrace;

CTX. response. Write (errorstring );

CTX. server. clearerror ();

Base . Onerror (E );
}

Special requirements are required for file upload functions, such as progress bar prompts. ASP. NET encapsulated controls <asp: fileupload/> are powerless.

Good Solution

Robert bazinet suggested that the best solution is to use Ria. In most cases, it is recommended to use the Silverlight or flash Upload Component to replace the traditional fileupload component, this type of component not only provides a better upload experience, but also is more beautiful than the text box and button of the <input type = "file"> tag on the page, this <input type = "file"> tag cannot be added to a style through CSS, but some people try to solve it. So far, no commercial Upload Component has used Silverlight, but here we have demonstrated a sample program for multi-File Upload using Silverlight.
Of course, with Silverlight, you can easily implement multi-threaded upload and resumable upload. These are not the details that I want to discuss in detail. If you need them, you can check them yourself.

Available solutions

The support provided by the <input type = "file"/> label is very limited. We cannot implement some special requirements-or simply cannot implement them easily or directly. Therefore, every time we implement such a function, we have to go around a big bend. To avoid time-consuming detours every time you implement the same functions, various Upload components are available in the market or open-source world. The Upload components provide encapsulated functions, this makes it easier to implement the file upload function. For example, almost all Upload components provide progress prompts directly or indirectly, some provide the current percentage value, and some directly provide a set of UI; some components only provide a simple UI, while others provide a complete set of upload and deletion management interfaces. In addition, some components provide the ability to prevent malicious client uploads.
I think the best way is to read files in parts in httpmodule and maintain the page activation status, so that it will not time out, and you can also track the progress or cancel the upload, or implement it through httphandler, when the progress bar is displayed to the user, developers can better control the file size and possible exceptions during the upload process. These methods are used to upload components. Our options include:

Upload Component
Fileuploader. Net (mediachase, $310 or above)
Radupload (telerik, $249)
Neatupload (free of charge, compliant with lgpl Protocol)
······

Neatupload intercepts the current httpworkerrequest object in the beginrequest event of ASP. NET pipeline, and then directly calls its readentitybody and other methods to obtain the data stream passed by the client for analysis and processing. The new request is used for polling to obtain the current upload status. For more information about neatupload and other open-source components, see jeffreyzhao's file uploading in ASP. NET applications. Of course, he also mentioned memba velodoc XP edition and swfupload, which are great!

Httpworkerrequest implementation

Using the implicit httpworkerrequest, you can use its getpreloadedentitybody and readentitybody methods to read data in blocks from the pipe created by IIS for ASP. NET to upload files. The implementation method is as follows:

Httpworkerrequest code
Iserviceprovider provider = (Iserviceprovider) httpcontext. Current;
Httpworkerrequest WR = (Httpworkerrequest) provider. getservice ( Typeof (Httpworkerrequest ));
Byte [] BS = Wr. getpreloadedentitybody ();

If ( ! Wr. isentireentitybodyispreloaded ())
{
Int N = 1024 ;
Byte [] Bs2 = New   Byte [N];
While (WR. readentitybody (bs2, n) > 0 )
{

}
}

Conclusion

ASP. net File Upload is an incomplete and flawed field. I believe it will be improved and developed soon. If you have already solved this problem, it means you are in a good company, otherwise, you can consider using a third-party product. We can find many different ways to solve the File Upload problem. The challenge is to find out the advantages and disadvantages of different practices and then find a solution suitable for your project, this is not just about file upload!

References:
1. upload files in ASP. NET applications.
2. handling large file uploads in ASP. NET
3. Uploading files in ASP. NET 2.0
4. The Dark Side of file uploads
5. ASP. NET custom error pages

PS: It is too troublesome to write a blog. It has been published n times!