Large file multipart upload

Vue Project encountered large file multipart upload problem, before using Webuploader, simply put Vue2.0 and webuploader together to use, encapsulated a Vue upload components, use is also relatively comfortable.

Upload it, why make it so troublesome, upload with a shard?

With the combination of sharding and concurrency, a large file is divided into multiple blocks, and concurrent uploads, greatly improving the upload speed of large files.
When a network problem causes a transmission error, you only need to re-transmit the wrong shard, not the entire file. In addition, the Shard transfer can track the upload progress more in real time.

After the implementation of the interface:

The main is two files, the package upload component and the specific UI page, the upload component code is listed below. The code on these two pages is on GitHub: Https://github.com/shady-xia/Blog/tree/master/vue-webuploader.

Introducing Webuploader in the project

1. First introduces jquery to the system (plug-in based on JQ, Pit Daddy!) If you don't know where to put it, put it in index.html .
2. Download uploader.swf and webuploader.min.js on the official website, which can be placed under the project static directory static ; The webuploader.min.js is introduced in index.html .
   (there is no need to introduce WEBUPLOADER.CSS separately because there are few lines of CSS that we can copy into the Vue component.) )

<script src="/static/lib/jquery-2.2.3.min.js"></script><script src="/static/lib/webuploader/webuploader.min.js"></script>

Points to note:

1. In the Vue component, import ‘./webuploader‘; the way to introduce Webuploader, will be reported "caller", ' callee ', and ' arguments ' properties may is accessed on strict mode ... ' Wrong, this is because your Babel use strict mode, and caller these are forbidden in strict mode. So you can introduce webuploader.js directly in index.html, or manually solve the problem of ' use strict ' in Babel.

Vue components based on the Webuploader package

The packaged components are Upload.vue as follows, and the interfaces can be scaled to the specific business.

Note: Functionality and UI separation, this build encapsulates the basic functionality, does not provide ui,ui on the specific page to implement.

<template>    <div class="upload"> </div></template>

<script>ExportDefault {name:' Vue-upload ', props: {accept: {TypeObject,DefaultNULL,},Upload address URL: {TypeString,Default‘‘, },Upload Maximum number defaults to Filenumlimit: {TypeNumber,Default100,},Size limit default 2M filesinglesizelimit: {TypeNumber,Default2048000,},Upload to the backend parameters, generally token,key and other formData: {TypeObject,DefaultNULL},To generate the key for the file in Formdata, here is just an example of which form and backend to negotiate Keygenerator: {TypeFunction,Default (file) {Const CURRENTTIME =NewDate (). GetTime ();Const KEY =`${currenttime}.${file.name} ';Return key; },}, multiple: {TypeBoolean,DefaultFalse,},Upload Button ID Uploadbutton: {TypeString,Default',},}, data () {return {uploader:NULL}; }, mounted () {This.initwebupload (); }, Methods: {Initwebupload () {This.uploader = Webuploader.create ({auto:TrueWhether the SWF is automatically uploaded after you finish selecting the file:'/static/lib/webuploader/uploader.swf ',SWF file path server:This.url,File Receive Server pick: {ID:This.uploadbutton,Select the file button multiple:This.multiple,Whether to upload multiple files by default false label:',}, accept:This.getaccept (This.accept),Allows you to select a file format. Threads3, Filenumlimit:This.filenumlimit,Limit the number of uploadsFileSingleSizeLimit:this.fileSingleSizeLimit,//Limit the size of a single upload image formData:This.formdata,Upload the required parameters chunked:TrueMultipart upload chunkSize:2048000,Shard Size Duplicate:TrueRepeat upload});Add to page preview when a file is added to the queueThis.uploader.on (' filequeued ', (file) = {this. $emit (' Filechange ', file); });This.uploader.on (' Uploadstart ', (file) = {Formdata data can be prepared here.This.uploader.options.formData.key = this.keygenerator (file); });Create a progress bar in real-time display during file upload.This.uploader.on (' Uploadprogress ', (file, percentage) = {this. $emit (' Progress ', file, percentage); });This.uploader.on (' Uploadsuccess ', (file, response) = {this. $emit (' Success ', file, response); });This.uploader.on (' Uploaderror ', (file, reason) = {Console.error (reason);this. $emit (' Uploaderror ', file, reason); });This.uploader.on (' Error ', (Type) = = {Let errormessage =‘‘;if (Type = = =' F_exceed_size ') {errormessage =' File size cannot exceed${this.filesinglesizelimit/(1024x768 * +)}m '; }Elseif (Type = = =' Q_exceed_num_limit ') {errormessage =' File upload has reached the maximum limit '; }else {errormessage =' Upload Error! Please check and re-upload! Error code${type} '; }Console.error (errormessage);this. $emit (' Error ', errormessage); });This.uploader.on (' Uploadcomplete ', (file, response) = {this. $emit (' Complete ', file, response); }); }, upload (file) {This.uploader.upload (file); }, stop (file) {This.uploader.stop (file); },Cancels and interrupts file uploads Cancelfile (file) {This.uploader.cancelFile (file); },Remove a file in the queue removefile (file, bool) {This.uploader.removeFile (file, bool); }, Getaccept (accept) {Switch (accept) {Case' Text ':return {title:' Texts ', exteensions:' Doc,docx,xls,xlsx,ppt,pptx,pdf,txt ', mimetypes: Doc,docx,.xls,.xlsx,.ppt,.pptx,.pdf,.txt '}; break; case  ' video ': return {title:  ' Videos ', exteensions:  ' mp4 ', mimetypes:  . mp4 '}; break; case  ' image ': return {title:  ' Images ', exteensions:  Gif,.jpg,.jpeg,.bmp,.png '}; break; default: return Accept}},};< /SCRIPT>             

<style lang="scss">// 直接把官方的css粘过来就行了</style>

Using the encapsulated Upload component

To create a new page, use the following example:

The UI needs to be implemented on its own. The approximate code can be ordered here.

<vue-upload        ref="uploader"        url="xxxxxx"        uploadButton="#filePicker"        multiple        @fileChange="fileChange" @progress="onProgress" @success="onSuccess"></vue-upload>

The principle and process of sharding

When we upload a large file, it is fragmented by the plugin, and the AJAX request is as follows:

1. Multiple upload requests are shard requests, which divide large files into multiple small parts once to the server
2. After the Shard is complete, that is, when upload is complete, a merge request needs to be passed to the server to have the server synthesize multiple shard files into one file

Sharding

You can see the number upload of requests that have been initiated, and let's look at upload the specific parameters of the send:

The first configuration ( content-disposition ) and the guid second configuration are the parameters that access_token we pass to the server through the Webuploader configuration. formData
The following configuration is the file contents, ID, name, type, size, etc.
Which chunks is the total number of shards, which chunk is the current first shard. The images are 12 and 9, respectively. When you see chunk is a 11 upload request, it represents the last upload request.

Merge

After sharding, the files are not consolidated, and the data is probably the following:

After the Shard, in fact, the work is not finished, we also send an AJAX request to the server, tell him to put our uploaded several shards into a complete file.

How do I know when the shards are uploaded and when do I merge?

The Webuploader plugin has an event that uploadSuccess contains two parameters, file and is returned in the background response ; When all shards are uploaded, the event is triggered.
We can use the fields returned by the server to determine if we want to merge.
For example, back in the background needMerge , we true can send a merge request when we see it is.

Known issues that exist

When the single file paused and continued uploading, the bug of the plugin was found:

1, when the set threads>1 , using the single file upload function, that is, the Stop method when the file, will be errorUncaught TypeError: Cannot read property ‘file‘ of undefined

The source code for the error is as follows: This is because in order for the next file to continue to be transferred, the current pool pops out of the paused file stream. Here's the loop, and the last cycle, V is undefined.

2, set the threads is 1, can normal pause, but the pause and then continue uploading is a failure.

The principle is the same as the previous one, the current file stream is all in the pause, pool pop When the file starts upload , it will check the time period pool , and there is no previously paused file stream.

Large file multipart upload