OutOfMemory: Java heap space Solution case

OutOfMemory: Java heap space Solution case
Scenario: read data from a large file row by row and import it to the mysql database. 3g plain text. semi-structured data requires a program to analyze the problem: An exception occurs when the file is analyzed to a certain number of rows (outOfMemory java heap space)
Guess:
1. Is the program adding text to the memory line by line, linearly increasing the StringBuffer, leading to memory overflow;
2. A new statement may exist in the while LOOP, resulting in continuous creation of an instance of a class (in fact, I did not think of it, and finally solved the problem ).
Key idea: speculation 1 is the most likely, but no problem is found after the program is checked. You need to know which data occupies the JVM memory. solution:
1. look for JVM memory detection tools (one is the built-in JDK memory analysis tools, including Jmap \ Jhat \ Jstat, but not easy to use; the other is the JProfiler, independent parameter detection tools for JVM runtime );
2. Use the JProfiler tool to detect the class instances loaded in the memory when the program is running and the occupied memory space;
3. it is found that JDBC4ResultSet and StatementImpl generate a large number of instances, the number has been increasing, not destroyed, and occupies nearly MB of memory. Other class instances also occupy a certain amount of space, however, the quantity is increased or decreased, that is, the object will be destroyed.
4. I guess it may be that the Statement object is constantly created in the while loop. Check the program carefully. The Statement object creation Statement is removed from the loop and the test is OK.
Note:
1. disassembling programs encapsulated into multiple methods can increase the logic of the program. However, you must note whether the calling method layer-by-layer in the while loop contains the new statement (Object creation ), if the new statement is included, the object will be created cyclically until the heap space overflows when the data volume is large. In this case, you need to create an object outside the loop and pass it as a parameter.
2. When a large amount of data is imported into the database, if the database log is enabled, the database log files will increase constantly, and data cannot be imported when the disk space is exhausted.
3. If the JVM debugging program needs to be tested in some scenarios, we recommend that you use the JProfiler tool, which is very useful.