SSH integration-online storage upload/Download System (problem accumulation)

I have been working on an SSH integrated online storage upload and download system today. I 'd like to write my thoughts!

First, I saved the uploaded files to the database. However, if I upload files larger than a few MB, I will report

java.lang.OutOfMemoryError: Java heap space 

This exception is followed by Baidu. It turns out that the database cannot store too long binary files, so the code is changed immediately because the entire project hierarchy is clear, so the code is rarely modified, once again, we have experienced the benefits of layering.

As a result, I immediately thought of saving the uploaded file under a folder of the project (any name of the folder). In the database, I only need to save the name of the uploaded file to continue and run it, at this time, it was okay to upload a large file, and the speed was very fast. As a result, I responded to my friend's call to find a bigger video and immediately put it on. However, the result was tragic and I reported it again.

java.lang.OutOfMemoryError: Java heap space 

This exception continues Baidu. The vast majority of Baidu's answers are: In JVM, if 98% of the time is used for GC and the available Heap size is less than 2%, this exception information will be thrown. JVM heap setting refers to the setting of memory space that JVM can allocate during java program running. the JVM automatically sets the Heap size value at startup. The initial space (-Xms) is 1/64 of the physical memory, and the maximum space (-Xmx) is 1/4 of the physical memory. You can use options such as-Xmn-Xms-Xmx provided by JVM to set the configuration. The root cause of this problem is that the default Heap size of the jvm virtual machine is 64 MB, which can be achieved by setting the maximum and minimum values.

According to Baidu's answer, the JVM virtual memory size was modified in tomcat.

You can set the parameters as follows:

1. You can add JAVA_OPTS =-Xms64m-Xmx512m to change the system environment variable in windows.2. If tomcat is used, in windows, it can be stored in C: \ tomcat5.5.9 \ bin \ catalina. add: set JAVA_OPTS =-Xms64m-Xmx256m to the position of: rem Guess CATALINA_HOME if not defined. (For windows, follow this method)3. in linux, set JAVA_OPTS = '-Xms64-xmx512' before {tomcat_home}/bin/catalina. sh'

So after the modification, restart tomcat and continue uploading the large video. Then the error was reported again. At that time, I was a little discouraged and did not know how to solve the problem. Baidu basically used the solution described above, so I calmed down and looked at it.
Which line of code reports an exception error? I don't know. I was shocked when I checked it .... I found the corresponding code. I first posted a warning:

// This is the previous Code. Get the input stream and write it to the output stream InputStream is = new FileInputStream (new File (..)); outputStream OS = new FileOutputStream (new File (...)); int length = is. available (); byte [] buffer = new byte [length]; OS. write (buffer); // This is the code I modified: InputStream is = new FileInputStream (new File (..)); outputStream OS = new FileOutputStream (new File (...)); int length = 0; byte [] buffer = new byte [1000]; while (-1! = (Length = is. read (buffer, 0, 1000) {OS. write (buffer );}

At this time, I tried to run it again. After about half a minute, the file was successfully uploaded (the file was relatively large). At this time, my heart was full of joy. Maybe it was because the file was too large, if you get the file size directly, then define

Byte array, When I/O operations are performed inside, the server returns a long response, so the final report

java.lang.OutOfMemoryError: Java heap space 

If this exception occurs, a byte array of the general size is defined, and a while loop is used to read some data each time, the memory usage of the JVM virtual machine should be less.

In conclusion, the problem of abnormal uploading of large files by the online storage system is finally solved.

(LAN speed is still very fast, 10 M/s, O (∩ _ ∩) O Haha ~, You don't need to upload a USB flash drive to them in the future. Open the server and upload the file, and then download it)