Java http large file upload, breakpoint continuation project research, GitHub upload source code

1, Project Research

Because you need to study the problem of uploading breakpoints. Looking for a long time finally found a better project.

In the googlecode above, the code to get down Super inconvenient, or configure the hosts just good, put the code back to GitHub above.

Https://github.com/freewebsys/java-large-file-uploader-demo

Effect:

Upload, show progress, time, percentage.

Click "Pause" to pause and click "Resume" to continue.

2, Code Analysis

Original project:

https://code.google.com/p/java-large-file-uploader/

The last time this project was updated was 2012, the project was packaged using the simplest method to implement HTTP breakpoint uploads.

Because HTML5 has a class library that reads file segmentation files, it can support breakpoint uploads, so this can only be displayed on HTML5 supported browsers.

At the same time, JS and Java use Cr32 for File Block verification, ensure that the data upload is correct.

The code uses the latest Servlet 3.0 APIs, using asynchronous execution, monitoring, and so on.

Upload class Uploadservlet

@Component ("Javalargefileuploaderservlet") @WebServlet (name = "Javalargefileuploaderservlet", Urlpatterns = {"/ Javalargefileuploaderservlet "}) public class Uploadservlet extends httprequesthandlerservletimplements Httprequesthandler {private static final Logger log = Loggerfactory.getlogger (Uploadservlet.class); Autowireduploadprocessor uploadprocessor; @AutowiredFileUploaderHelper fileuploaderhelper;@ Autowiredexceptioncodemappinghelper exceptioncodemappinghelper; @AutowiredAuthorizer authorizer;@ Autowiredstaticstateidentifiermanager Staticstateidentifiermanager; @Overridepublic void HandleRequest ( HttpServletRequest request, HttpServletResponse response) throws IOException {log.trace ("Handling Request"); Serializable Jsonobject = null;try {//Extract the action from the Requestuploadservletaction actionbyparametername =uploa Dservletaction.valueof (Fileuploaderhelper.getparametervalue (Request, uploadservletparameter.action));//Check Authorizationcheckauthorization (Request, ActionbyparameternAME);/Then process the asked Actionjsonobject = Processaction (actionbyparametername, request);//If something have to IS Written to the Responseif (jsonobject! = null) {Fileuploaderhelper.writetoresponse (jsonobject, Response);}} If exception, write Itcatch (Exception e) {exceptioncodemappinghelper.processexception (E, Response);}} private void Checkauthorization (HttpServletRequest request, Uploadservletaction Actionbyparametername) throws Missingparameterexception, authorizationexception {//check authorization//if its not get progress (because we does not rea Lly care about Authorization for get//progress and it uses an array of file IDs) if (!actionbyparametername.equals (uploads ervletaction.getprogress)) {//extract uuidfinal String fileidfieldvalue = Fileuploaderhelper.getparametervalue ( Request, Uploadservletparameter.fileid, false);/If this is the init, the identifier are the one in Parameteruuid Clientorjobi D String parameter = fileuploaderhelper.getparametervalue (Request, UploadservleTparameter.clientid, False); if (Actionbyparametername.equals (uploadservletaction.getconfig) && parameter! = NULL) {clientorjobid = uuid.fromstring (parameter);} If not, get it from managerelse {clientorjobid = Staticstateidentifiermanager.getidentifier ();} Call Authorizerauthorizer.getauthorization (Request,actionbyparametername,clientorjobid,fileidfieldvalue! = null ? Getfileidsfromstring (Fileidfieldvalue). ToArray (New uuid[] {}): null);}} Private Serializable processaction (uploadservletaction actionbyparametername, httpservletrequest request) throws Exception {log.debug ("processing action" + actionbyparametername.name ()); Serializable returnobject = Null;switch (actionbyparametername) {case getconfig:string ParameterValue = Fileuploaderhelper.getparametervalue (Request, Uploadservletparameter.clientid, false); Returnobject = Uploadprocessor.getconfig (ParameterValue! = null?) Uuid.fromstring (ParameterValue): null); Break;case verifycrcofuncheckedpart:returnobject = VerifyCrcOfUncHeckedpart (Request), break;case Prepareupload:returnobject = prepareupload (request); Break;case Clearfile: Uploadprocessor.clearfile (uuid.fromstring (Fileuploaderhelper.getparametervalue (Request, Uploadservletparameter.fileid)); break;case clearAll:uploadProcessor.clearAll (); Break;case pausefile:list< Uuid> UUIDs = getfileidsfromstring (Fileuploaderhelper.getparametervalue (Request, Uploadservletparameter.fileid) ); Uploadprocessor.pausefile (UUIDs); Break;case resumefile:returnobject =uploadprocessor.resumefile ( Uuid.fromstring (Fileuploaderhelper.getparametervalue (Request, Uploadservletparameter.fileid)); break;case SetRate:uploadProcessor.setUploadRate (uuid.fromstring (Fileuploaderhelper.getparametervalue (Request, Uploadservletparameter.fileid)), long.valueof (Fileuploaderhelper.getparametervalue (Request, uploadservletparameter.rate)); Break;case getprogress:returnobject = getprogress (request); return returnobject;} List<uuid> getfileidsfromstring (String fileids) {string[] splittedFileids = Fileids.split (","); List<uuid> UUIDs = Lists.newarraylist (); for (int i = 0; i < splittedfileids.length; i++) {Uuids.add (Uuid.fromstri Ng (Splittedfileids[i]));} return UUIDs;} Private Serializable getprogress (HttpServletRequest request) throws Missingparameterexception {Serializable Returnobject; string[] IDs =new Gson (). Fromjson (Fileuploaderhelper.getparametervalue (Request, Uploadservletparameter.fileid), String[].class); Collection<uuid> UUIDs = Collections2.transform (arrays.aslist (IDS), new function<string, UUID> () {@ Overridepublic UUID Apply (String input) {return uuid.fromstring (input);}}); Returnobject = Maps.newhashmap (); for (UUID Fileid:uuids) {try {Progressjson progress = uploadprocessor.getprogress (file ID);((hashmap<string, progressjson>) returnobject). Put (Fileid.tostring (), progress);} catch (FileNotFoundException e) {log.debug ("No progress'll be a retrieved for" + FileId + "because" + e.getmessage ());} }return Returnobject;} Private SERIALIZABLe prepareupload (HttpServletRequest request) throws Missingparameterexception, IOException {//extract File Informationprepareuploadjson[] Fromjson =new Gson (). Fromjson (Fileuploaderhelper.getparametervalue (Request, Uploadservletparameter.newfiles), prepareuploadjson[].class);//Prepare themfinal hashmap<string, UUID> Prepareupload = Uploadprocessor.prepareupload (Fromjson);//Return Themreturn Maps.newhashmap (Maps.transformValues ( Prepareupload, New Function<uuid, string> () {public String apply (UUID input) {return input.tostring ();};}));} Private Boolean Verifycrcofuncheckedpart (HttpServletRequest request) throws IOException, Missingparameterexception, Filecorruptedexception, filestillprocessingexception {UUID fileId = uuid.fromstring ( Fileuploaderhelper.getparametervalue (Request, Uploadservletparameter.fileid)); try { Uploadprocessor.verifycrcofuncheckedpart (Fileid,fileuploaderhelper.getparametervalue (Request, UPLOADSERVLETPARAMETER.CRC));} catch (Invalidcrcexception e) {//No needTo log this exception, a fallback behaviour are defined in the//throwing method.//but we need to return Something!return Boolean.false;} return boolean.true;}}

Asynchronous upload Uploadservletasync

@Component ("Javalargefileuploaderasyncservlet") @WebServlet (name = "Javalargefileuploaderasyncservlet", Urlpatterns = {"/javalargefileuploaderasyncservlet"}, asyncsupported = True) public class Uploadservletasync extends Htt Prequesthandlerservletimplements Httprequesthandler {private static final Logger log = Loggerfactory.getlogger ( Uploadservletasync.class); @AutowiredExceptionCodeMappingHelper exceptioncodemappinghelper;@ Autowireduploadservletasyncprocessor Uploadservletasyncprocessor; @AutowiredStaticStateIdentifierManager Staticstateidentifiermanager, @AutowiredStaticStateManager <StaticStatePersistedOnFileSystemEntity> Staticstatemanager, @AutowiredFileUploaderHelper fileuploaderhelper, @AutowiredAuthorizer authorizer;/** * Maximum Time that a streaming request can take.<br> */private long tasktimeout = dateutils.millis_per_hour; @Overridepublic V OID HandleRequest (final httpservletrequest request, final HttpServletResponse response) throws Servletexception, IOException {//ProCess the requesttry {//check if uploads is Allowedif (!uploadservletasyncprocessor.isenabled ()) {throw new UploadIsCurre Ntlydisabled ();} Extract stuff from requestfinal fileuploadconfiguration process = fileuploaderhelper.extractfileuploadconfiguration (request); Log.debug ("received upload request with config:" +process);//verify authorizationfinal UUID clientId = StaticS Tateidentifiermanager.getidentifier (); Authorizer.getauthorization (Request, Uploadservletaction.upload, ClientId, Process.getfileid ());//check If that file was not Pausedif (uploadservletasyncprocessor.isfilepaused (Process.getfileid ())) {log.debug ("file" +process.getfileid () + "is paused, ignoring async request."); return;} Get the Modelstaticfilestate filestate = Staticstatemanager.getentityifpresent (). Getfilestates (). Get (  Process.getfileid ()), if (filestate = = null) {throw new FileNotFoundException ("File with id" + process.getfileid () + "not Found ");} Process the request asynchronouslyfinal Asynccontext Asynccontext = Request.startasync (); Asynccontext.settimeout (tasktimeout);//Add a listener to clear buckets and close inputstream when Process is complete or//with//errorasynccontext.addlistener (New Uploadservletasynclisteneradapter ( Process.getfileid ()) {@Overridevoid clean () {log.debug ("request" + Request + "completed."); /We don't need to clear the InputStream here.//and tell processor to clean its shit!uploadservletasyncprocessor.clean (c Lientid, Process.getfileid ());}); /Then Processuploadservletasyncprocessor.process (Filestate, Process.getfileid (), PROCESS.GETCRC (), Process.getinputstream (), new Writechunkcompletionlistener () {@Overridepublic void success () {Asynccontext.complete ( );} @Overridepublic void error (Exception Exception) {//handles a stream ended unexpectedly, it just means the user has//sto pped the//streamif (exception.getmessage ()! = null) {if (Exception.getmessage (). Equals ("Stream ended unexpectedly")) { Log.warn ("User have stopped streaming for file" + Process.getfileid());} else if (Exception.getmessage (). Equals ("User cancellation")) {Log.warn ("user has cancelled streaming for file id" + proce Ss.getfileid ());//Do Nothing}else {exceptioncodemappinghelper.processexception (exception, response);}} else {exceptioncodemappinghelper.processexception (exception, response);} Asynccontext.complete ();}});} catch (Exception e) {exceptioncodemappinghelper.processexception (E, Response);}}

3, Request Flowchart:
The main idea is to split the files and then upload them in chunks.

Java http large file upload, breakpoint continuation project research, GitHub upload source code