Troubleshooting Java Runtime

Recently alone in charge of an application on-line, due to lack of experience, trampled a lot of pits, record, convenient to view later.

At first, my try,catch was written like this:

try {    mediaType = detector.detect(inputStream, metadata);    catch (TikaException e) {    catch (SAXException e) {    thrownew BadRequestException("文件不符合SAX标准"catch (IOException e) {    returncatch (RuntimeException e) {    thrownew BadRequestException(e.getMessage());}

The results showed that the application ran and ran, and there was no alarm ...
A face is forced, in fact I do not mind you hang, at least you have to give me an alarm or something ...

Ran to the server, to check the log, really nothing ... Well, reproduce the fault again, and found the following:

Java.lang.OutOfMemoryError:Java Heap spaceat Java. Util. Arrays. CopyOf(Arrays. Java:2367) ~[na:1.8. 0_45]at Java. Lang. Abstractstringbuilder. Expandcapacity(Abstractstringbuilder. Java: the) ~[na:1.8. 0_45]at Java. Lang. Abstractstringbuilder. Ensurecapacityinternal(Abstractstringbuilder. Java: the) ~[na:1.8. 0_45]at Java. Lang. Abstractstringbuilder. Append(Abstractstringbuilder. Java:415) ~[na:1.8. 0_45]at Java. Lang. StringBuilder. Append(StringBuilder. Java: the) ~[na:1.8. 0_45]

The virtual machine collapsed, but I didn't catch the error, so it silently left me ...

So I changed the code, like this (take care not to be bad for me, this place is reasonable for business to ignore):

try {    mediaType = detector.detect(inputStream, metadata);    catch (TikaException e) {    catch (SAXException e) {    thrownew BadRequestException("文件不符合SAX标准"catch (IOException e) {    returncatch (RuntimeException e) {    thrownewcatch (Throwable ignored) {}

After a long time, found that the virtual machine did not respond, yes, it is.

But this time is very strange, the virtual machine is not hanging, just like blocking, is it a deadlock?
So with Jstack-pid to see what it is doing, unexpectedly found that it is really blocking.
Just like this:

"Com.package.FullIndex.main ()" #30 prio=5 os_prio=31 tid=0x00007f89ca2e7000 nid=0x6713 runnable in object.wait () [0x0000000107c68000]Atcom. Package. Fullindex. Main(Fullindex. Java: A)"Main" #1 prio=5 os_prio=31 tid=0x00007f89c2029000 nid=0x1303 in object.wait () [0x0000000107c68000]at org. Apache. Maven. CLI. MAVENCLI. Main(MAVENCLI. Java:141) at Org. Codehaus. Plexus. Classworlds. Launcher. Launcher. Mainwithexitcode(Launcher. Java:409) at Org. Codehaus. Plexus. Classworlds. Launcher. Launcher. Main(Launcher. Java:352) at Org. Codehaus. Classworlds. Launcher. Main(Launcher. Java: -)

Chasing the call stack, found the reason for blocking: a inputstream from the network was passed in with no read timeout set.
So for another interface, their own socket to take inputstream, finally, the world is quiet.

But I still too yong,too simple. The program again because of Oom hung, but this time the oom changed another place .... I'm just here. Index Rich Text documents, 3G memory is not enough?

Found the corresponding record of the hanging off, it looks like this:
Path= '/bio/soapsnp/kasalath_genome/all.fasta/', size=409164635
400M of files, swallowed in memory enough to save 7 copies, why not enough? And what the hell is this suffix ...
To Google a bit, this document is the human genome to describe the human genome file ...
Will humans be able to bully the program ape casually? Why do I read memory to Oom?

So, with Jmap and Jhat look at the memory of the virtual machine, incredibly have a 2.6G char array.
The original iso-8859-1 encoding converted to Unicode encoding, so much larger ...
Finally, the loop is added again Try-catch, to circumvent the extracted content of the oom, and added the truncation function.
It looks like this after the addition:

try {    catchcatch (RetryException e) {    1);    returncatcherror// ignore it and retry later    LOG.errorerror);}

Finally, the world has been pure ~

Troubleshooting Java Runtime