Uploaded image for project: 'Scala Programming Language'
  1. Scala Programming Language
  2. SI-9737

Get OutOfMemoryError in REPL when using parallel collections

    Details

      Description

      I put a simple example for using scan method on collections in the REPL and I observe that I got OutOfMemoryError when using parallel collections. There is not problem, nor wait time when using standard collections.

      Here is a listing of what I did:

      $ scala -J-Xms1G -J-Xmx2G 
      Welcome to Scala 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_77).
      Type in expressions for evaluation. Or try :help.
       
      scala> Range(1, 10000).scanLeft(BigInt(1))(_ * _)
      res0: scala.collection.immutable.IndexedSeq[scala.math.BigInt] = Vector(1, 1, 2, 6, 24, 120, 720, 5040, 40320, 362880, 3628800, 39916800, 479001600, 6227020800, 87178291200, 1307674368000, 20922789888000, 355687428096000, 6402373705728000, 121645100408832000, 2432902008176640000, 51090942171709440000, 1124000727777607680000, 25852016738884976640000, 620448401733239439360000, 15511210043330985984000000, 403291461126605635584000000, 10888869450418352160768000000, 304888344611713860501504000000, 8841761993739701954543616000000, 265252859812191058636308480000000, 8222838654177922817725562880000000, 263130836933693530167218012160000000, 8683317618811886495518194401280000000, 295232799039604140847618609643520000000, 10333147966386144929666651337523200000000, 3719933267899012174679994481508352...
       
      scala> Range(1, 10000).par.scanLeft(BigInt(1))(_ * _)
      java.lang.OutOfMemoryError: Java heap space
      	at java.util.Arrays.copyOf(Arrays.java:3332)
      	at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:137)
      	at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:121)
      	at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:622)
      	at java.lang.StringBuilder.append(StringBuilder.java:202)
      	at scala.collection.mutable.StringBuilder.append(StringBuilder.scala:267)
      	at scala.collection.mutable.StringBuilder.$plus$eq(StringBuilder.scala:175)
      	at scala.collection.mutable.StringBuilder.$plus$eq(StringBuilder.scala:31)
      	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
      	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
      	at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
      	at scala.collection.immutable.StringOps.foreach(StringOps.scala:29)
      	at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
      	at scala.collection.immutable.StringOps.map(StringOps.scala:29)
      	at scala.tools.nsc.interpreter.Naming$class.unmangle(Naming.scala:35)
      	at scala.tools.nsc.interpreter.IMain$naming$.unmangle(IMain.scala:188)
      	at scala.tools.nsc.interpreter.IMain$ReplStrippingWriter.stripImpl(IMain.scala:1372)
      	at scala.tools.nsc.interpreter.IMain$StrippingWriter$class.strip(IMain.scala:1346)
      	at scala.tools.nsc.interpreter.IMain$StrippingTruncatingWriter.strip(IMain.scala:1357)
      	at scala.tools.nsc.interpreter.IMain$StrippingTruncatingWriter.clean(IMain.scala:1363)
      	at scala.tools.nsc.interpreter.IMain$StrippingTruncatingWriter.write(IMain.scala:1364)
      	at java.io.PrintWriter.print(PrintWriter.java:603)
      	at scala.tools.nsc.reporters.ConsoleReporter.printMessage(ConsoleReporter.scala:44)
      	at scala.tools.nsc.interpreter.ReplReporter.printMessage(ReplReporter.scala:61)
      	at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:646)
      	at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
      	at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
      	at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
      	at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
      	at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
      	at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
      	at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
       
      That entry seems to have slain the compiler.  Shall I replay
      your session? I can re-run each line except the last one.
      [y/n]
      Abandoning crashed session.
      

      I've boost the memory of JVM the example run, as seen in the run arguments and during execution I don't see any spike in memory consumption.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                apm A. P. Marki
                Reporter:
                antoniy Antoniy Chonkov
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: