Resumable upload of Html5 large files,

Resumable upload of Html5 large files,
Large file chunks

Generally, common web servers have limits on the size of data submitted to the server. If the file exceeds a certain size, the server returns a denial message. Of course, all web servers provide the size limit that may be modified by the configuration file. For large files uploaded to iis, you can modify the web server to limit the file size. However, this poses a security issue to the web server. Attackers can easily send a large packet and drag your web server to death.

Currently, major implementation methods for uploading large files are segmented. For example, for a M file, split it into 50 blocks at 2 m. Then upload each file to the server in sequence, and then merge the files on the server after the upload is complete.

To upload large files on the web, the core is mainly to implement file segmentation. Before the emergence of the Html5 File API, you must implement multipart File transfer on the web. Only flash or Activex is used to implement file segmentation.

In Html5, we can directly use the file slice method to implement file segmentation. For example:

file.slice(0,1000); file.slice(1000,2000); file.slice(2000,3000);

Then, it is asynchronously uploaded to the server through XMLHttpRequest.

Html5 Upload File Library

If you are interested and have time, you can use the html5 File API. I found the following two html5 class libraries on the Internet.
Resumable. js with git address: https://github.com/23/resumable.js
Pludload http://plupload.com/

Resumable is a pure html5 upload class library.
While Pludload supports html5, flash, silverlight, and html4, it automatically determines whether browsing supports html5 and does not support other upload methods.
After testing, both resumable and Pludload support multipart upload of html5 files. The resumable is suitable for use. The following describes how to use resumable.

Introduction to the use of resumable. js resumable upload

Main configurations:

        var r = new Resumable({            target:'/test/upload',            chunkSize:1*1024*1024,            simultaneousUploads:4,            testChunks: true,            throttleProgressCallbacks:1,            method: "octet"          });

ChunkSize, in bytes
SimultaneousUploads: Number of processes that upload file blocks at the same time. Multiple File blocks can be uploaded at the same time.
Whether the file block before testChunks sends the file information in get mode to check whether the file has been uploaded.

Resumable upload is implemented through the testChunks configuration node. If it is set to true. Resumable will first send a get request, if the http status returns 200. The current block has been uploaded and the next get request is made. If the http status does not return 200, the current block data packet will be sent in post mode for file block upload.

When testChunks is set to true, a get request is added for each upload. If we know the number of files before the last interrupted upload. Next time, you can upload the interrupted parts directly. In this way, an http get request can be reduced for each block.
In response to this requirement, I modified the resumable source code and added a startchunkindex attribute to the file object in resumable. The default value is 0. It is used to set which part of the current file to start uploading. In this way, we only need to perform a query from the server before the file is uploaded (query which part of the current file is uploaded) and return the index of the last uploaded file block. Then, you can set the index value to the startchunkindex attribute of file to start uploading from the last disconnected file block.
Call method:

// Handle file add event r. on ('fileadded', function (file) {file. startchunkindex = 0; // you can specify the number of parts that the current file starts to upload.

For more information, see the demo in the attachment.

Finishing work

After uploading all file blocks, the final task is to merge and save the files. The attachment is a server-side example of resumable multipart upload. net, including simple file merging. Demos in other languages can also be downloaded from resumable git.
For the sake of simplicity, the demo only stores files on the local machine. In a real production environment. Generally, it should be placed on a separate file server (the front-end web is uploaded to the file server through ftp or folder sharing), and then the uploaded files are distributed to the image or processed (such as video compression ). Of course, it is best to have a distributed file system. At present, it is a good solution to put it in the Hadoop Distributed File System (HDFS.

Demo

Download Html5 Upload demo in Vs2012