Implement Asynchronous upload of large files in ASP. net mvc (1)

BKJIA exclusive Article] in ASP. uploading large files via HTTP in. NET is a long-standing challenge, and it is a lot of active ASP. in addition to processing large files, users are often asked to display the File Upload progress. When you need to directly control the upload of data streams from a browser, you will hit the wall. Bkjia.com has previously reported articles such as "lift ASP. NET file upload size limit" and "ASP. NET large file upload Development Summary.

Most people think that uploading large files in ASP. NET has the following solutions:

◆ Do not do this. You 'd better embed a Silverlight or Flash process in the page to upload files.

◆ Do not do this. Because HTTP is not designed to upload large files, rethink the features you want.

◆ Do not do this. ASP. NET is designed to process files up to 2 GB.

◆ Purchase commercial products, such as SlickUpload, which uses an HttpModule to implement file stream segmentation.

◆ Use open-source products, such as NeatUpload, which uses an HttpModule to implement file stream segmentation.

Recently, I have received a task that requires an upload tool to implement the following functions:

◆ Must work on the HTTP protocol

◆ Large files must be allowed to be uploaded larger than 2 GB)

◆ Resumable upload is required.

◆ Parallel upload is required

Therefore, the first three solutions are not suitable for my needs, and other solutions are too cumbersome for me, so I started to solve the problem in ASP. net mvc. If you have a development background in this aspect, you must understand that most of the problems ultimately come due to ASP. NET input stream and chain request process control, the online information is generally described in this way, as long as your code accesses the InputStream attribute of HttpRequest, before you access the stream, ASP. NET will cache the entire uploaded file, which means that when I upload a file to cloud service, I have to wait until the entire large file arrives on the server before it can be transmitted to the destination, this means it takes two times.

First of all, we recommend you read Scott Hanselman's article about ASP. net mvc file upload, address http://www.hanselman.com/blog/CommentView.aspx? Guid = bc137b6b-d8d0-47d1-9795-f8814f7d1903, first have a rough understanding of file upload, but Scott Hanselman's method is not capable of uploading large files, according to Scott Hanselman's method, you just need to modify the web. config file to ensure ASP.. NET allows file uploads up to 2 GB. Don't worry, this setting won't eat your memory, because all data larger than KB is cached to the disk.

 
 

  
  
﹤system.web﹥  
  
  
﹤httpruntime requestlengthdiskthreshold="256" maxrequestlength="2097151"﹥  
  
  
﹤/httpruntime﹥﹤/system.web﹥ 
 
 

This is a simple solution suitable for most applications, but I cannot use this method in my tasks, even if the data is cached to the disk, however, this method similar to "save as" also uses a large amount of memory.

 
Figure 1: memory consumption suddenly increases by caching the entire file and saving it as a result

In ASP. net mvc, how does one upload large files without triggering any caching mechanism by directly accessing the stream? The solution is to try to stay away from ASP. NET, let's take a look at UploadController. It has three behavior Methods: one is to index the files we uploaded, the other is the cache logic discussed earlier, and the other is based on the real-time stream method.

 
 

  
  
public class UploadController : Controller  
  
  
{  
  
  
    [AcceptVerbs(HttpVerbs.Get)]  
  
  
    [Authorize]  
  
  
    public ActionResult Index()  
  
  
    {  
  
  
        return View();  
  
  
    }  
  
  
 
  
  
    [AcceptVerbs(HttpVerbs.Post)]  
  
  
    public ActionResult BufferToDisk()  
  
  
    {  
  
  
        var path = Server.MapPath("~/Uploads");  
  
  
 
  
  
        foreach (string file in Request.Files)  
  
  
        {  
  
  
            var fileBase = Request.Files[file];  
  
  
 
  
  
            try 
  
  
            {  
  
  
                if (fileBase.ContentLength > 0)  
  
  
                {  
  
  
                    fileBase.SaveAs(Path.Combine(path, fileBase.FileName));  
  
  
                }  
  
  
            }  
  
  
            catch (IOException)  
  
  
            {  
  
  
 
  
  
            }  
  
  
        }  
  
  
 
  
  
        return RedirectToAction("Index", "Upload");  
  
  
    }  
  
  
 
  
  
    //[AcceptVerbs(HttpVerbs.Post)]  
  
  
    //[Authorize]  
  
  
    public void LiveStream()  
  
  
    {  
  
  
        var path = Server.MapPath("~/Uploads");  
  
  
 
  
  
        var context = ControllerContext.HttpContext;  
  
  
 
  
  
        var provider = (IServiceProvider)context;  
  
  
 
  
  
        var workerRequest = (HttpWorkerRequest)provider.GetService(typeof(HttpWorkerRequest));  
  
  
 
  
  
        //[AcceptVerbs(HttpVerbs.Post)]  
  
  
        var verb = workerRequest.GetHttpVerbName();  
  
  
        if(!verb.Equals("POST"))  
  
  
        {  
  
  
            Response.StatusCode = (int)HttpStatusCode.NotFound;  
  
  
            Response.SuppressContent = true;  
  
  
            return;  
  
  
        }  
  
  
 
  
  
        //[Authorize]  
  
  
        if(!context.User.Identity.IsAuthenticated)  
  
  
        {  
  
  
            Response.StatusCode = (int)HttpStatusCode.Unauthorized;  
  
  
            Response.SuppressContent = true;  
  
  
            return;  
  
  
        }  
  
  
 
  
  
        var encoding = context.Request.ContentEncoding;  
  
  
 
  
  
        var processor = new UploadProcessor(workerRequest);  
  
  
 
  
  
        processor.StreamToDisk(context, encoding, path);  
  
  
 
  
  
        //return RedirectToAction("Index", "Upload");  
  
  
        Response.Redirect(Url.Action("Index", "Upload"));  
  
  
    }  
  
  
}  
 
 

Although one or two classes are obviously missing here, the basic method is clear. It seems that there is not much difference with the cache logic. We still cache the stream to the disk, however, the specific processing method is somewhat different. First, there is no attribute associated with the method, the predicates and authorization restrictions are removed, and the Manual equivalence is replaced, the reason why the manual response operation is not required by the ActionFilterAttribute Declaration is that these attributes involve some important ASP. NET Pipeline Code, in fact, in my code, I also specifically intercepted the original HttpWorkerRequest, because it cannot do two things at the same time.