Scala Programming Language
  1. Scala Programming Language
  2. SI-5789

crasher in mixins under -optimize in a REPL setting

    Details

      Description

      Given:

      class ScalaColls(n: Int) {  
        Array.fill(n)(null)
      }
      

      This compiles under:

      • Scala 2.9.2, with and without -optimize, with and without the REPL
      • Scala 2.10.0-SNAPSHOT, without the REPL, or with the REPL but without -optimize

      The glorious crash dump runs like this:

      
      ~/code/scala ./build/pack/bin/scala -optimize -Xprint:typer
      Welcome to Scala version 2.10.0-20120511-110655-3511e5960d (Java HotSpot(TM) 64-Bit Server VM, Java 1.6.0_29).
      Type in expressions to have them evaluated.
      Type :help for more information.
      
      ...
      
      scala> :reset
      Resetting interpreter state.
      Forgetting all expression results and named terms: $intp
      
      scala> :load /Users/jason/code/scratch/t3463a.scala
      Loading /Users/jason/code/scratch/t3463a.scala...
      [[syntax trees at end of typer]]// Scala source: <console>
      package $line3 {
        object $read extends scala.AnyRef {
          def <init>(): $line3.$read.type = {
            $read.super.<init>();
            ()
          };
          object $iw extends scala.AnyRef {
            def <init>(): $line3.$read.$iw.type = {
              $iw.super.<init>();
              ()
            };
            object $iw extends scala.AnyRef {
              def <init>(): $line3.$read.$iw.$iw.type = {
                $iw.super.<init>();
                ()
              };
              class ScalaColls extends scala.AnyRef {
                <paramaccessor> private[this] val n: Int = _;
                def <init>(n: Int): ScalaColls = {
                  ScalaColls.super.<init>();
                  ()
                };
                scala.Array.fill[Null](ScalaColls.this.n)(null)(ClassTag.Null)
              }
            }
          }
        }
      }
      
      error: 
           while compiling: <console>
              during phase: mixin
           library version: version 2.10.0-20120511-110655-3511e5960d
          compiler version: version 2.10.0-20120511-110655-3511e5960d
        reconstructed args: -Ydead-code -optimise -Yinline -Yclosure-elim -Yinline-handlers -Xprint:typer
      
        last tree to typer: term n
                    symbol: value n (flags: <paramaccessor> <triedcooking> private[this])
         symbol definition: private[this] val n: Int
                       tpe: <notype>
             symbol owners: value n -> class $read$ScalaColls -> package $line3
            context owners: anonymous class $read$ScalaColls$$anonfun$1 -> package $line3
      
      == Enclosing template or block ==
      
      Template( // val <local $anonfun>: <notype>, tree.tpe=ScalaColls$$anonfun$1
        "scala.runtime.AbstractFunction0", "scala.Serializable" // parents
        ValDef(
          private
          "_"
          <tpt>
          <empty>
        )
        // 3 statements
        DefDef( // final def apply(): Null
          final <method> <triedcooking>
          "apply"
          []
          List(Nil)
          <tpt> // tree.tpe=Null
          null
        )
        DefDef( // final def apply(): Object
          final <method> <bridge>
          "apply"
          []
          List(Nil)
          <tpt> // tree.tpe=Object
          Apply( // final def apply(): Null, tree.tpe=Null
            $read$ScalaColls$$anonfun$1.this."apply" // final def apply(): Null, tree.tpe=()Null
            Nil
          )
        )
        DefDef( // def <init>(arg$outer: ScalaColls): ScalaColls$$anonfun$1
          <method> <triedcooking>
          "<init>"
          []
          // 1 parameter list
          ValDef( // $outer: ScalaColls
            <param>
            "$outer"
            <tpt> // tree.tpe=ScalaColls
            <empty>
          )
          <tpt> // tree.tpe=ScalaColls$$anonfun$1
          Block( // tree.tpe=Unit
            Apply( // def <init>(): scala.runtime.AbstractFunction0 in class AbstractFunction0, tree.tpe=scala.runtime.AbstractFunction0
              $read$ScalaColls$$anonfun$1.super."<init>" // def <init>(): scala.runtime.AbstractFunction0 in class AbstractFunction0, tree.tpe=()scala.runtime.AbstractFunction0
              Nil
            )
            ()
          )
        )
      )
      
      == Expanded type of tree ==
      
      <notype>
      
      uncaught exception during compilation: java.lang.AssertionError
      java.lang.AssertionError: assertion failed: 
           while compiling: <console>
              during phase: mixin
           library version: version 2.10.0-20120511-110655-3511e5960d
          compiler version: version 2.10.0-20120511-110655-3511e5960d
        reconstructed args: -Ydead-code -optimise -Yinline -Yclosure-elim -Yinline-handlers -Xprint:typer
      
        last tree to typer: term n
                    symbol: value n (flags: <paramaccessor> <triedcooking> private[this])
         symbol definition: private[this] val n: Int
                       tpe: <notype>
             symbol owners: value n -> class $read$$iw$$iw$ScalaColls -> package $line3
            context owners: anonymous class $read$$iw$$iw$ScalaColls$$anonfun$1 -> package $line3
      
      == Enclosing template or block ==
      
      Template( // val <local $anonfun>: <notype>, tree.tpe=$line3.$read$$iw$$iw$ScalaColls$$anonfun$1
        "scala.runtime.AbstractFunction0", "scala.Serializable" // parents
        ValDef(
          private
          "_"
          <tpt>
          <empty>
        )
        // 3 statements
        DefDef( // final def apply(): Null
          final <method> <triedcooking>
          "apply"
          []
          List(Nil)
          <tpt> // tree.tpe=Null
          null
        )
        DefDef( // final def apply(): Object
          final <method> <bridge>
          "apply"
          []
          List(Nil)
          <tpt> // tree.tpe=Object
          Apply( // final def apply(): Null, tree.tpe=Null
            $read$$iw$$iw$ScalaColls$$anonfun$1.this."apply" // final def apply(): Null, tree.tpe=()Null
            Nil
          )
        )
        DefDef( // def <init>(arg$outer: $line3.$read$$iw$$iw$ScalaColls): $line3.$read$$iw$$iw$ScalaColls$$anonfun$1
          <method> <triedcooking>
          "<init>"
          []
          // 1 parameter list
          ValDef( // $outer: $line3.$read$$iw$$iw$ScalaColls
            <param>
            "$outer"
            <tpt> // tree.tpe=$line3.$read$$iw$$iw$ScalaColls
            <empty>
          )
          <tpt> // tree.tpe=$line3.$read$$iw$$iw$ScalaColls$$anonfun$1
          Block( // tree.tpe=Unit
            Apply( // def <init>(): scala.runtime.AbstractFunction0 in class AbstractFunction0, tree.tpe=scala.runtime.AbstractFunction0
              $read$$iw$$iw$ScalaColls$$anonfun$1.super."<init>" // def <init>(): scala.runtime.AbstractFunction0 in class AbstractFunction0, tree.tpe=()scala.runtime.AbstractFunction0
              Nil
            )
            ()
          )
        )
      )
      
      == Expanded type of tree ==
      
      <notype>
      
      Not an impl class:(class Function0$class in package scala (flags: abstract <java>),trait Function0 in package scala (flags: abstract <interface> <trait> <lateinterface>))
              at scala.Predef$.assert(Predef.scala:182)
              at scala.tools.nsc.Global.assert(Global.scala:189)
              at scala.tools.nsc.transform.Mixin.mixinImplClassMembers$1(Mixin.scala:272)
              at scala.tools.nsc.transform.Mixin$$anonfun$addMixedinMembers$2.apply(Mixin.scala:369)
              at scala.tools.nsc.transform.Mixin$$anonfun$addMixedinMembers$2.apply(Mixin.scala:364)
              at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:759)
              at scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59)
              at scala.collection.immutable.List.foreach(List.scala:77)
              at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:758)
              at scala.tools.nsc.transform.Mixin.addMixedinMembers(Mixin.scala:364)
              at scala.tools.nsc.transform.Mixin.addMixedinMembers(Mixin.scala:361)
              at scala.tools.nsc.transform.Mixin$MixinTransformer.preTransform(Mixin.scala:531)
              at scala.tools.nsc.transform.Mixin$MixinTransformer.transform(Mixin.scala:1245)
              at scala.reflect.api.Trees$Transformer.transformTemplate(Trees.scala:1551)
              at scala.reflect.api.Trees$Transformer$$anonfun$transform$2.apply(Trees.scala:1445)
              at scala.reflect.api.Trees$Transformer$$anonfun$transform$2.apply(Trees.scala:1444)
              at scala.reflect.api.Trees$Transformer.atOwner(Trees.scala:1574)
              at scala.reflect.api.Trees$Transformer.transform(Trees.scala:1443)
              at scala.tools.nsc.transform.Mixin$MixinTransformer.transform(Mixin.scala:1245)
              at scala.reflect.api.Trees$Transformer$$anonfun$transformStats$2.apply(Trees.scala:1567)
              at scala.reflect.api.Trees$Transformer$$anonfun$transformStats$2.apply(Trees.scala:1565)
              at scala.collection.immutable.List.loop$1(List.scala:163)
              at scala.collection.immutable.List.mapConserve(List.scala:179)
              at scala.reflect.api.Trees$Transformer.transformStats(Trees.scala:1565)
              at scala.reflect.api.Trees$Transformer$$anonfun$transform$1.apply(Trees.scala:1439)
              at scala.reflect.api.Trees$Transformer$$anonfun$transform$1.apply(Trees.scala:1439)
              at scala.reflect.api.Trees$Transformer.atOwner(Trees.scala:1574)
              at scala.reflect.api.Trees$Transformer.transform(Trees.scala:1438)
              at scala.tools.nsc.transform.Mixin$MixinTransformer.transform(Mixin.scala:1245)
              at scala.tools.nsc.ast.Trees$Transformer.transformUnit(Trees.scala:228)
              at scala.tools.nsc.transform.Transform$Phase.apply(Transform.scala:30)
              at scala.tools.nsc.Global$GlobalPhase.applyPhase(Global.scala:432)
              at scala.tools.nsc.Global$GlobalPhase$$anonfun$run$1.apply(Global.scala:398)
              at scala.tools.nsc.Global$GlobalPhase$$anonfun$run$1.apply(Global.scala:392)
              at scala.collection.Iterator$class.foreach(Iterator.scala:724)
              at scala.collection.AbstractIterator.foreach(Iterator.scala:1152)
              at scala.tools.nsc.Global$GlobalPhase.run(Global.scala:392)
              at scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1400)
              at scala.tools.nsc.Global$Run.compileUnits(Global.scala:1373)
              at scala.tools.nsc.Global$Run.compileSources(Global.scala:1367)
              at scala.tools.nsc.interpreter.IMain.compileSourcesKeepingRun(IMain.scala:461)
              at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.compileAndSaveRun(IMain.scala:845)
              at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.compile(IMain.scala:803)
              at scala.tools.nsc.interpreter.IMain$Request.compile$lzycompute(IMain.scala:979)
              at scala.tools.nsc.interpreter.IMain$Request.compile(IMain.scala:974)
              at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:629)
              at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:595)
              at scala.tools.nsc.interpreter.ILoop.reallyInterpret$1(ILoop.scala:826)
              at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:883)
              at scala.tools.nsc.interpreter.ILoop.reallyInterpret$1(ILoop.scala:844)
              at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:883)
              at scala.tools.nsc.interpreter.ILoop.reallyInterpret$1(ILoop.scala:844)
              at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:883)
              at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:783)
              at scala.tools.nsc.interpreter.ILoop.processLine$1(ILoop.scala:647)
              at scala.tools.nsc.interpreter.ILoop.innerLoop$1(ILoop.scala:654)
              at scala.tools.nsc.interpreter.ILoop.loop(ILoop.scala:657)
              at scala.tools.nsc.interpreter.ILoop$$anonfun$interpretAllFrom$1$$anonfun$apply$mcV$sp$1$$anonfun$apply$mcV$sp$2.apply(ILoop.scala:667)
              at scala.tools.nsc.interpreter.ILoop$$anonfun$interpretAllFrom$1$$anonfun$apply$mcV$sp$1$$anonfun$apply$mcV$sp$2.apply(ILoop.scala:664)
              at scala.tools.nsc.io.Streamable$Chars$class.applyReader(Streamable.scala:99)
              at scala.tools.nsc.io.File.applyReader(File.scala:77)
              at scala.tools.nsc.interpreter.ILoop$$anonfun$interpretAllFrom$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ILoop.scala:664)
              at scala.tools.nsc.interpreter.ILoop$$anonfun$interpretAllFrom$1$$anonfun$apply$mcV$sp$1.apply(ILoop.scala:664)
              at scala.tools.nsc.interpreter.ILoop$$anonfun$interpretAllFrom$1$$anonfun$apply$mcV$sp$1.apply(ILoop.scala:664)
              at scala.tools.nsc.interpreter.ILoop.savingReplayStack(ILoop.scala:141)
              at scala.tools.nsc.interpreter.ILoop$$anonfun$interpretAllFrom$1.apply$mcV$sp(ILoop.scala:663)
              at scala.tools.nsc.interpreter.ILoop$$anonfun$interpretAllFrom$1.apply(ILoop.scala:663)
              at scala.tools.nsc.interpreter.ILoop$$anonfun$interpretAllFrom$1.apply(ILoop.scala:663)
              at scala.tools.nsc.interpreter.ILoop.savingReader(ILoop.scala:146)
              at scala.tools.nsc.interpreter.ILoop.interpretAllFrom(ILoop.scala:662)
              at scala.tools.nsc.interpreter.ILoop$$anonfun$loadCommand$1.apply(ILoop.scala:726)
              at scala.tools.nsc.interpreter.ILoop$$anonfun$loadCommand$1.apply(ILoop.scala:725)
              at scala.Function1$class.apply$mcVL$sp(Function1.scala:39)
              at scala.runtime.AbstractFunction1.apply$mcVL$sp(AbstractFunction1.scala:12)
              at scala.tools.nsc.interpreter.ILoop.withFile(ILoop.scala:719)
              at scala.tools.nsc.interpreter.ILoop.loadCommand(ILoop.scala:725)
              at scala.tools.nsc.interpreter.ILoop$$anonfun$standardCommands$8.apply(ILoop.scala:291)
              at scala.tools.nsc.interpreter.ILoop$$anonfun$standardCommands$8.apply(ILoop.scala:291)
              at scala.tools.nsc.interpreter.LoopCommands$LineCmd.apply(LoopCommands.scala:81)
              at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:778)
              at scala.tools.nsc.interpreter.ILoop.processLine$1(ILoop.scala:647)
              at scala.tools.nsc.interpreter.ILoop.innerLoop$1(ILoop.scala:654)
              at scala.tools.nsc.interpreter.ILoop.loop(ILoop.scala:657)
              at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:962)
              at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:924)
              at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:924)
              at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:157)
              at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:924)
              at scala.tools.nsc.MainGenericRunner.runTarget$1(MainGenericRunner.scala:79)
              at scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:92)
              at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:101)
              at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)
      
      
      Abandoning crashed session.
      

      My attempts to reproduce the issue outside of the REPL by mimicing its wrapper objects failed:

      // compiles under scalac -optimize
      package $line3 {
        object $read extends scala.AnyRef {
          object $iw extends scala.AnyRef {
            object $iw extends scala.AnyRef {
              class ScalaColls(n: Int) extends scala.AnyRef {          
                scala.Array.fill[Null](ScalaColls.this.n)(null)(ClassTag.Null)
              }
            }
          }
        }
      }
      

        Issue Links

          Activity

          Hide
          James Iry added a comment -

          Here are some in-progress notes.

          The assert that's failing is doing a sanity check - we should only be putting mixin forwarders to a mixin impl class. Reasonable, no? The problem is that in this case the flag is lying. And it's lying because of our old friend lazy initialization. Paul's innocuous refactoring of Mixin.scala removed an initialization that was there before.

          It turns out that the easy fix is to get rid of the assert. Everything works without.

          But the interesting story here, IMO, is why this particular lie only happens in the REPL under optimize (actually REPL with -Yinline is sufficient). The answer is that the REPL does a separate compilation for each line but keeps some state lying around between compilations. In fact, it can be replicated by using fsc -Yinline to do two separate compilations because fsc also leaves some stuff lying around. Exactly what is being left around is a bit fuzzy to me. We're ending up with two symbols representing the same thing, one with the flag set and one that's not been initialized.

          Why only -Yinline? That's a bit fuzzier. And I'm investigating. It's possible it's something to do with class parsing being slightly different under optimization, but I doubt it. My stronger suspicion is that the inliner is responsible for leaving the uninitialized symbol lying around.

          Show
          James Iry added a comment - Here are some in-progress notes. The assert that's failing is doing a sanity check - we should only be putting mixin forwarders to a mixin impl class. Reasonable, no? The problem is that in this case the flag is lying. And it's lying because of our old friend lazy initialization. Paul's innocuous refactoring of Mixin.scala removed an initialization that was there before. It turns out that the easy fix is to get rid of the assert. Everything works without. But the interesting story here, IMO, is why this particular lie only happens in the REPL under optimize (actually REPL with -Yinline is sufficient). The answer is that the REPL does a separate compilation for each line but keeps some state lying around between compilations. In fact, it can be replicated by using fsc -Yinline to do two separate compilations because fsc also leaves some stuff lying around. Exactly what is being left around is a bit fuzzy to me. We're ending up with two symbols representing the same thing, one with the flag set and one that's not been initialized. Why only -Yinline? That's a bit fuzzier. And I'm investigating. It's possible it's something to do with class parsing being slightly different under optimization, but I doubt it. My stronger suspicion is that the inliner is responsible for leaving the uninitialized symbol lying around.
          Hide
          Grzegorz Kossakowski added a comment -

          Great findings James!

          Getting rid of the assert is not real fix, though. It's just hiding the fact that we get into inconsistent state and the rest of the code can recover from it by accident.

          However, that kind of issues are killing us every time we want to enable resident compiler in sbt/ide which we really want for fast incremental compilation cycles. Therefore, we really need to understand the root cause for this issue and fix it.

          Show
          Grzegorz Kossakowski added a comment - Great findings James! Getting rid of the assert is not real fix, though. It's just hiding the fact that we get into inconsistent state and the rest of the code can recover from it by accident. However, that kind of issues are killing us every time we want to enable resident compiler in sbt/ide which we really want for fast incremental compilation cycles. Therefore, we really need to understand the root cause for this issue and fix it.
          Hide
          Jason Zaugg added a comment -

          In case you haven't already, poke around here under the different scenarios: https://github.com/scala/scala/blob/master/src/compiler/scala/tools/nsc/transform/AddInterfaces.scala#L75

          Show
          Jason Zaugg added a comment - In case you haven't already, poke around here under the different scenarios: https://github.com/scala/scala/blob/master/src/compiler/scala/tools/nsc/transform/AddInterfaces.scala#L75
          Hide
          James Iry added a comment -

          I'm going to remove the assert to solve the immediate pain of this and related bugs. I've created #SI-6782 to track my investigation into the underlying cause.

          Show
          James Iry added a comment - I'm going to remove the assert to solve the immediate pain of this and related bugs. I've created # SI-6782 to track my investigation into the underlying cause.
          Show
          Adriaan Moors added a comment - https://github.com/scala/scala/pull/1729

            People

            • Assignee:
              James Iry
              Reporter:
              Jason Zaugg
            • Votes:
              3 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development