Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Abort in icode when compiling macro that reifies an anonymous function #5797

Closed
scabug opened this issue May 13, 2012 · 8 comments
Closed

Abort in icode when compiling macro that reifies an anonymous function #5797

scabug opened this issue May 13, 2012 · 8 comments

Comments

@scabug
Copy link

scabug commented May 13, 2012

The compiler aborts in icode when compiling a macro that reifies an anonymous function whose body is the macro argument. The crash happens when the macro argument contains a local variable use.

The following REPL dump shows the crash.

$ scala-2.10.0-M3/bin/scala -Xmacros
Welcome to Scala version 2.10.0-M3 (OpenJDK 64-Bit Server VM, Java 1.7.0-internal).
Type in expressions to have them evaluated.
Type :help for more information.

scala> import scala.reflect.makro._
import scala.reflect.makro._

scala> import language.experimental.macros
import language.experimental.macros

scala> object Fun { def fun[A: c.TypeTag](c: Context)(x: c.Expr[A]) = c.reify( () => x.eval ) }
defined module Fun

scala> def m[A](x: A) = macro Fun.fun[A]
m: [A](x: A)() => A

scala> m(1)
res0: () => Int = <function0>

scala> m { val x = 1; 2 }
res1: () => Int = <function0>

scala> m { val x = 1; x }
ReplGlobal.abort: symbol value x$1 does not exist in $line15.$read$$iw$$iw$$iw$$iw.<init>
error: symbol value x$1 does not exist in <init>
error: 
     while compiling:  <console>
       current phase:  icode
     library version:  version 2.10.0-M3
    compiler version:  version 2.10.0-M3
  reconstructed args:  -Xmacros

uncaught exception during compilation: scala.reflect.internal.FatalError
scala.reflect.internal.FatalError: 
     while compiling:  <console>
       current phase:  icode
     library version:  version 2.10.0-M3
    compiler version:  version 2.10.0-M3
  reconstructed args:  -Xmacros

symbol value x$1 does not exist in $line15.$read$$iw$$iw$$iw$$iw.<init>
	at scala.reflect.internal.SymbolTable.abort(SymbolTable.scala:45)
	at scala.tools.nsc.Global.abort(Global.scala:202)
	at scala.tools.nsc.interpreter.IMain$$anon$1.scala$tools$nsc$interpreter$ReplGlobal$$super$abort(IMain.scala:288)
	at scala.tools.nsc.interpreter.ReplGlobal$class.abort(ReplGlobal.scala:21)
	at scala.tools.nsc.interpreter.IMain$$anon$1.abort(IMain.scala:288)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase.liftedTree2$1(GenICode.scala:998)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase.scala$tools$nsc$backend$icode$GenICode$ICodePhase$$genLoad(GenICode.scala:992)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase$$anonfun$genLoadArguments$1.apply(GenICode.scala:1201)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase$$anonfun$genLoadArguments$1.apply(GenICode.scala:1199)
	at scala.collection.LinearSeqOptimized$class.foldLeft(LinearSeqOptimized.scala:111)
	at scala.collection.immutable.List.foldLeft(List.scala:77)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase.genLoadArguments(GenICode.scala:1199)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase.scala$tools$nsc$backend$icode$GenICode$ICodePhase$$genLoad(GenICode.scala:814)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase.scala$tools$nsc$backend$icode$GenICode$ICodePhase$$genLoad(GenICode.scala:1032)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase.scala$tools$nsc$backend$icode$GenICode$ICodePhase$$genLoad(GenICode.scala:1024)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase.scala$tools$nsc$backend$icode$GenICode$ICodePhase$$genStat(GenICode.scala:168)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase$$anonfun$genStat$1.apply(GenICode.scala:148)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase$$anonfun$genStat$1.apply(GenICode.scala:148)
	at scala.collection.LinearSeqOptimized$class.foldLeft(LinearSeqOptimized.scala:111)
	at scala.collection.immutable.List.foldLeft(List.scala:77)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase.scala$tools$nsc$backend$icode$GenICode$ICodePhase$$genLoad(GenICode.scala:1023)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase.gen(GenICode.scala:116)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase$$anonfun$gen$1.apply(GenICode.scala:72)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase$$anonfun$gen$1.apply(GenICode.scala:72)
	at scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59)
	at scala.collection.immutable.List.foreach(List.scala:77)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase.gen(GenICode.scala:72)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase.gen(GenICode.scala:141)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase.gen(GenICode.scala:91)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase$$anonfun$gen$1.apply(GenICode.scala:72)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase$$anonfun$gen$1.apply(GenICode.scala:72)
	at scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59)
	at scala.collection.immutable.List.foreach(List.scala:77)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase.gen(GenICode.scala:72)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase.gen(GenICode.scala:82)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase.gen(GenICode.scala:68)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase.apply(GenICode.scala:64)
	at scala.tools.nsc.Global$GlobalPhase.applyPhase(Global.scala:432)
	at scala.tools.nsc.Global$GlobalPhase$$anonfun$run$1.apply(Global.scala:398)
	at scala.tools.nsc.Global$GlobalPhase$$anonfun$run$1.apply(Global.scala:392)
	at scala.collection.Iterator$class.foreach(Iterator.scala:697)
	at scala.collection.AbstractIterator.foreach(Iterator.scala:1124)
	at scala.tools.nsc.Global$GlobalPhase.run(Global.scala:392)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase.run(GenICode.scala:57)
	at scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1336)
	at scala.tools.nsc.Global$Run.compileUnits(Global.scala:1309)
	at scala.tools.nsc.Global$Run.compileSources(Global.scala:1303)
	at scala.tools.nsc.interpreter.IMain.compileSourcesKeepingRun(IMain.scala:461)
	at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.compileAndSaveRun(IMain.scala:845)
	at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.compile(IMain.scala:803)
	at scala.tools.nsc.interpreter.IMain$Request.compile(IMain.scala:979)
	at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:629)
	at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:595)
	at scala.tools.nsc.interpreter.ILoop.reallyInterpret$1(ILoop.scala:826)
	at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:883)
	at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:783)
	at scala.tools.nsc.interpreter.ILoop.processLine$1(ILoop.scala:647)
	at scala.tools.nsc.interpreter.ILoop.innerLoop$1(ILoop.scala:654)
	at scala.tools.nsc.interpreter.ILoop.loop(ILoop.scala:657)
	at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:962)
	at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:924)
	at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:924)
	at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:157)
	at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:924)
	at scala.tools.nsc.MainGenericRunner.runTarget$1(MainGenericRunner.scala:79)
	at scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:92)
	at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:101)
	at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)

That entry seems to have slain the compiler.  Shall I replay
your session? I can re-run each line except the last one.
[y/n]

This does not happen when using scala.reflect.mirror directly:

scala> scala.reflect.mirror.reify { val x = 1; x }
res0: reflect.mirror.Expr[Int] = 
Expr[Int]({
  val x = 1;
  x
})

scala> scala.reflect.mirror.reify { () => res0.eval }
res1: reflect.mirror.Expr[() => Int] = 
Expr[() => Int]((() => {
  val x = 1;
  x
}))

scala> res1.eval()
res2: Int = 1
@scabug
Copy link
Author

scabug commented May 13, 2012

Imported From: https://issues.scala-lang.org/browse/SI-5797?orig=1
Reporter: Nate Nystrom (njnystrom)
Affected Versions: 2.10.0-M3

@scabug
Copy link
Author

scabug commented Feb 10, 2013

@retronym said:
This is a great example of the 'ResetAttrs' problem that has Eugene and I so flummoxed.

The "safe" way to write the macro is:

c.universe.reify( () => c.Expr[A](c.resetLocalAttrs(x.tree)).splice )

Splicing code into the body of the closure means that any symbols defined in the splicee will now have the wrong owner, the hacky way around this is to 'reset' the symbols/types to null and let the definitions pick up branch new symbols when the expanded macro is typechecked.

I'll leave this ticket open as admission that this situation is untenable, it is far to easy to write a macro that works in some circumstances but breaks in others.

retronym/scala@scala:2.10.x...retronym:ticket/5797

@scabug
Copy link
Author

scabug commented May 5, 2013

@retronym said:
I think that we have a means here for a macro to patch the owners.

class OwnerRepair[C <: reflect.macros.Context with Singleton](val c: C) {
  /**
   * If macro arguments are spliced into underneath DefTree that introduces
   * an entry into the symbol ownership chain, any symbols defined in the
   * spliced tree will be ill-owned.
   *
   * This method detects this situation, and corrects the owners.
   */
  def repairOwners[A](expr: c.Expr[A]): c.Expr[A] = {
    val symtab = c.universe.asInstanceOf[reflect.internal.SymbolTable]
    val utils = new Utils[symtab.type](symtab)

    // Proactively typecheck the tree. This will assign symbols to
    // DefTrees introduced by the macro.
    val typed = c.typeCheck(expr.tree).asInstanceOf[symtab.Tree]

    // The current owner at the call site. Symbols owned by this may need
    // to be transplanted.
    import scala.reflect.macros.runtime.{Context => MRContext}
    val callsiteOwner =
      c.asInstanceOf[MRContext]
       .callsiteTyper.context.owner
       .asInstanceOf[symtab.Symbol]

    val repairedTree = utils.repairOwners(typed, callsiteOwner)
    c.Expr[A](repairedTree.asInstanceOf[c.universe.Tree])
  }

  private class Utils[U <: reflect.internal.SymbolTable](val u: U) {
    import u._

    class ChangeOwnerAndModuleClassTraverser(oldowner: Symbol, newowner: Symbol)
      extends ChangeOwnerTraverser(oldowner, newowner) {

      override def traverse(tree: Tree) {
        tree match {
          case _: DefTree => change(tree.symbol.moduleClass)
          case _          =>
        }
        super.traverse(tree)
      }
    }

    def repairOwners(t: Tree, macroCallSiteOwner: Symbol): Tree = {
      object repairer extends Transformer {
        override def transform(t: Tree): Tree = {
          // TODO see `fixerUpper` in the pattern matcher for a slightly simpler way to do this.
          if (currentOwner.hasTransOwner(macroCallSiteOwner) && currentOwner.owner != macroCallSiteOwner)
            new ChangeOwnerAndModuleClassTraverser(macroCallSiteOwner, currentOwner)(t)
          else super.transform(t)
        }
      }
      repairer transform t
    }
  }
}
// end of the macro impl
val ownerRepair = new OwnerRepair[c.type](c)
val checked = ownerRepair.repairOwners(code).tree

@scabug
Copy link
Author

scabug commented May 6, 2013

@xeno-by said:
Do you think this can be called automatically by the expansion engine? (in the same fashion as duplicate is now called for every macro expansion)

@scabug
Copy link
Author

scabug commented May 6, 2013

@retronym said:
Yep, I think so. But we can try it out in some real world macros in 2.10.1 already with the macro-initiated repair.

I'm working towards this for async, but its not trivial, as if we remove the resetAttrs calls, the rest of the macro actually breaks, because we splice untyped trees underneath typed trees, as well as the reverse. So we really need to drop down to lower-level mechanisms, like TreeGen.mkAttributedStableRef, and manual symbol creation. I think we really need to do this anyway so as not to mess up singleton types.

@scabug
Copy link
Author

scabug commented Apr 2, 2014

@retronym said:
Here's a closer-to-working version of the symbol repair utility:

  // This is needed to repair owner chain as encountered in the following issue:
  // https://github.com/scalatest/scalatest/issues/276
  class OwnerRepair[C <: reflect.macros.Context with Singleton](val c: C) {
    /**
     * If macro arguments are spliced into underneath DefTree that introduces
     * an entry into the symbol ownership chain, any symbols defined in the
     * spliced tree will be ill-owned.
     *
     * This method detects this situation, and corrects the owners.
     */
    def repairOwners[A](expr: c.Expr[A]): c.Expr[A] = {
      val symtab = c.universe.asInstanceOf[reflect.internal.SymbolTable]
      val utils = new Utils[symtab.type](symtab)

      // Proactively typecheck the tree. This will assign symbols to
      // DefTrees introduced by the macro.
      val typed = c.typeCheck(expr.tree).asInstanceOf[symtab.Tree]

      // The current owner at the call site. Symbols owned by this may need
      // to be transplanted.
      import scala.reflect.macros.runtime.{Context => MRContext}
      val callsiteOwner =
        c.asInstanceOf[MRContext]
          .callsiteTyper.context.owner
          .asInstanceOf[symtab.Symbol]

      val repairedTree = utils.repairOwners(typed, callsiteOwner)
      c.Expr[A](repairedTree.asInstanceOf[c.universe.Tree])
    }

    private class Utils[U <: reflect.internal.SymbolTable](val u: U) {
      import u._

      class ChangeOwnerAndModuleClassTraverser(oldowner: Symbol, newowner: Symbol)
        extends ChangeOwnerTraverser(oldowner, newowner) {

        override def traverse(tree: Tree) {
          tree match {
            case _: DefTree => change(tree.symbol.moduleClass)
            case _          =>
          }
          super.traverse(tree)
        }
      }

      def repairOwners(t: Tree, macroCallSiteOwner: Symbol): Tree = {
        object repairer extends Transformer {
          override def transform(t: Tree): Tree = {
            t match {
              case (_: DefTree | _: Function | _: Import) if t.symbol.owner == macroCallSiteOwner && macroCallSiteOwner != currentOwner =>
                new ChangeOwnerAndModuleClassTraverser(macroCallSiteOwner, currentOwner)(t)
              case _ =>
                super.transform(t)
            }
          }
        }
        repairer.atOwner(macroCallSiteOwner) {
          repairer transform t
        }
      }
    }
  }

@SethTisue
Copy link
Member

closing stale backend tickets; comment/reopen if you have evidence this is still applicable

@retronym
Copy link
Member

retronym commented Mar 1, 2018

Removing the backend label (that's just where the symptom appears, not the problem.) But I'll leave this close as the need to use resetAttrs is just part of the job of the macro author under the macro system we have today.

@retronym retronym removed the backend label Mar 1, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants