Troubleshoot Scala exception handling Java.lang.OutOfMemoryError:Java heap space error

Requirements: million, tens of thousands, 40 million-level log to the device to remove the weight
Environment: Device Memory 64g,scala run shell file on stand-alone version
Log:
20G48000000.log
4.0g10000000.log
396M1000000.log

The code is as follows

Help

01020304050607080910111213141516171819202122

import scala.io.Sourceimport scala.collection.mutable.ArrayBuffervar text = Source.fromFile("/Users/shuhai/1000000.log").getLines;//imei is requiredvar log = ArrayBuffer[String]();while(text.hasNext) {    var row = text.next    if(row.contains("imei"))        log += row}//println(log.head)//println(log.length)var act = log.filter{ (row) => row.contains("active") }var imei = act.map(_.split(",").filter(_.contains("imei"))).map(_.mkString(","))var clean= imei.map(_.replaceAll("\"","")).map(_.replaceAll(" ",""))println("total:" + clean.length)var count = clean.toList.distinct.lengthprintln("distinct:" + count)

Abnormal

? Scala Scala Distinct_imei.scala
Java.lang.OutOfMemoryError:Java Heap Space
At Java.util.Arrays.copyOfRange (arrays.java:3664)
At java.lang.String. (string.java:201)
At Java.io.BufferedReader.readLine (bufferedreader.java:356)
At Java.io.BufferedReader.readLine (bufferedreader.java:389)
At Scala.io.bufferedsource$bufferedlineiterator.hasnext (bufferedsource.scala:72)
At main$ $anon $. (Distinct_imei.scala:7)
At Main$.main (distinct_imei.scala:1)
At Main.main (Distinct_imei.scala)
At Sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
At Sun.reflect.NativeMethodAccessorImpl.invoke (nativemethodaccessorimpl.java:62)
At Sun.reflect.DelegatingMethodAccessorImpl.invoke (delegatingmethodaccessorimpl.java:43)
At Java.lang.reflect.Method.invoke (method.java:483)
At scala.reflect.internal.util.scalaclassloader$ $anonfun $run$1.apply (scalaclassloader.scala:70)
At Scala.reflect.internal.util.scalaclassloader$class.ascontext (scalaclassloader.scala:31)
At Scala.reflect.internal.util.scalaclassloader$urlclassloader.ascontext (scalaclassloader.scala:101)
At Scala.reflect.internal.util.scalaclassloader$class.run (scalaclassloader.scala:70)
At Scala.reflect.internal.util.scalaclassloader$urlclassloader.run (scalaclassloader.scala:101)
At Scala.tools.nsc.commonrunner$class.run (objectrunner.scala:22)
At Scala.tools.nsc.objectrunner$.run (objectrunner.scala:39)
At Scala.tools.nsc.commonrunner$class.runandcatch (objectrunner.scala:29)
At Scala.tools.nsc.objectrunner$.runandcatch (objectrunner.scala:39)
At scala.tools.nsc.scriptrunner.scala$tools$nsc$scriptrunner$ $runCompiled (scriptrunner.scala:170)
At scala.tools.nsc.scriptrunner$ $anonfun $runscript$1.apply (scriptrunner.scala:187)
At scala.tools.nsc.scriptrunner$ $anonfun $runscript$1.apply (scriptrunner.scala:187)
At scala.tools.nsc.scriptrunner$ $anonfun $withcompiledscript$1$ $anonfun $apply$mcz$sp$1.apply (Scriptrunner.scala : 156)
At scala.tools.nsc.scriptrunner$ $anonfun $withcompiledscript$1.apply$mcz$sp (scriptrunner.scala:156)
At scala.tools.nsc.scriptrunner$ $anonfun $withcompiledscript$1.apply (scriptrunner.scala:124)
At scala.tools.nsc.scriptrunner$ $anonfun $withcompiledscript$1.apply (scriptrunner.scala:124)
At Scala.tools.nsc.util.package$.trackingthreads (package.scala:43)
At Scala.tools.nsc.util.package$.waitingforthreads (package.scala:27)
At Scala.tools.nsc.ScriptRunner.withCompiledScript (scriptrunner.scala:123)
At Scala.tools.nsc.ScriptRunner.runScript (scriptrunner.scala:187)

Workaround:

Help

1	envJAVA_OPTS="-Xms256m -Xmx2048m"scala distinct_imei.scala

Using Scala-help, you can see that there is a-j parameter, and you can achieve the same effect

Help

1	scala -J-Xms256m -J-Xmx2048m distinct_imei.scala

Via:http://alvinalexander.com/scala/scala-repl-java.lang.outofmemoryerror-java-heap-space-error

http://www.4wei.cn/archives/1002410

Troubleshoot Scala exception handling Java.lang.OutOfMemoryError:Java heap space error