Java--Processing of large file multipart upload background

Problem:

JS upload file When the file is too large to upload the failure, in the end more than the assembly upload failed test (because I test uploaded a 10G file failed). There is a lot of reason to guess, probably related to a network protocol such as Post, which may be related to some background settings.

Ideas:

The idea is very simple, because the front-desk upload using the Web Uploader, a Baidu plug-in, the document clearly says there is a multiple attribute, set to True to open the Shard upload, listen to the name on the know can be used to upload large files, I think the breakpoint should be the continuation of such things should be done with this bar. Well, my needs are not so complicated, I can upload them on the line. Then the focus is on the backstage.

Multipart upload background receive, the practice of many online posts is to use Randomaccessfile to get the file, and then the file subscript (do not know how to describe the good) moved to the end of the file, and then the Shard uploaded data written to the back.

Then I met the problem there are two, one met. 1. Multipart upload he is fast sending the data, and Tomcat should be assigning a different thread to each request, so there are multiple threading issues (such as the first request is writing the data before the second request is finished and then the file is written). 2. Multipart upload because it is a request, so the order is not guaranteed, each time the file at the end of the write will appear file corruption. (eg: a file is called "123" and now splits the file into "1", "2", "3".) Three requests are sent to the server, and the server may receive a 321,132 order. This can cause file corruption).

Solve:

Question 1: Initially find information on the Internet found a file lock can be used, but reported a mistake, online tube This error is because the thread acquired a file lock, online Cheng not released when the line Cheng to get this lock will throw an exception. 囧 I understand the lock should not be the thread two did not get to the lock should be waiting for the thread to be released to get it, directly throw the exception is what ghost. Well, do not want so much, directly to the stitching method there add a synchronized keyword is good. This size at least looks normal (though it doesn't have any eggs).

Question 2: This is the most egg pain, when the test found that the compressed package file in such upload will appear file corruption, think for a long time or after a colleague to recognize this problem. Know the problem, the idea can have a lot of, location is not lil bit location is good. My idea is to see more uploader this plug-in, the Shard upload to the file size, the total number of shards, the current number of shards are passed to you. Then according to these parameters in the set file size, eliminate the current number of shards to move to the corresponding location to write data. It is also possible to remove the synchronized keyword that is added to the above question 1.

Because the code is copied directly, there is no format. So simply say the questions and ideas. See if there will be time to post the code again.

Welcome to join the Learning Exchange Group 569772982, we learn to communicate together.

Java--Processing of large file multipart upload background