Imported From: https://issues.scala-lang.org/browse/SI-7678?orig=1
Reporter: @dotta
Affected Versions: 2.10.1, 2.10.2, 2.11.0-M4

@dotta said:
I could reproduce the above issue also with the latest 2.11.0-SNAPSHOT, version number: 2.11.0-20130718-095202-1e0ae7623c

@dotta said:
I was hoping it would have been easy to create a test for this using the existing Presentation Compiler testing infrastructure, but it isn't the case (I'm not saying that it is not possible, but it requires more time than I'm currently willing to invest). On the other hand, it should be fairly easy to create a regression test for this on the Scala IDE side.

@retronym said:
After the hyperlink request, we end up with two symbols for TypeTag floating around. A new symbol is used in the tree, and the implicit search is for TypeTag#19273. Materialization of type tags is predicated on an implicit search for the original symbol, TypeTag#11117.

This is based on a slightly reduced test case:

package test

object Test {
import scala.reflect.runtime.{ universe => ru }
def getTypeTag(implicit tag: ru.TypeTag[Int]) = ()
locally {
getTypeTag
}
}

I haven't traced the point of the symbol divergence yet. Does this sound similar to any previous problems?

@retronym said:
It was a bit tricky to craft the test, but I think the fix itself is pretty trivial:

https://github.com/retronym/scala/compare/ticket/7678-2.10.x

@retronym said:
Here's another manifestation:

package test

import language.implicitConversions

object Test {
  type T = scala.languageFeature.implicitConversions
  implicit def foo(a: Int) = 0 
}

Hyperlink through the type implicitConversions, then edit this file to trigger a typechecking pass, and the feature warning is emitted.

Using this test:

import scala.tools.partest._

object Test extends CompilerTest {
  override def code = ""

  def check(source: String, unit: global.CompilationUnit) = ()

  val compiler = newCompiler()
  new compiler.Run()
  import compiler._

  val cm = reflect.runtime.currentMirror
  val instanceMirror = cm.reflect(definitions)
  val members = instanceMirror.symbol.typeSignature.members
  def resultType(tp: cm.universe.Type) = tp match {
    case cm.universe.NullaryMethodType(res) => res
    case t => t
  }

  val symFields = for {
    m <- members.toList
    if m.isMethod
    if m.asTerm.isGetter
    sig = m.typeSignature
    res = resultType(sig)
    base = res.baseType(cm.universe.typeOf[Symbol].typeSymbol)
    if base != cm.universe.NoType
    methMirror = instanceMirror.reflectMethod(m.asMethod)
    definedSym <- scala.util.Try(methMirror().asInstanceOf[Symbol]).toOption.toList
    if (!definedSym.owner.isPackageClass)
  } yield (m.name, definedSym)
  println(symFields.mkString("\n"))

}

I see these non-top-level symbols cached in Definitions:

Testing individual files
testing: [...]/files/run/t7678.scala                                  [FAILED]
(ExistentialsFeature,trait existentials)
(HigherKindsFeature,trait higherKinds)
(ImplicitConversionsFeature,trait implicitConversions)
(ReflectiveCallsFeature,trait reflectiveCalls)
(PostfixOpsFeature,trait postfixOps)
(DynamicsFeature,trait dynamics)
(MacrosFeature,trait macros)
(experimentalModule,object experimental)
(Boxes_isNumber,method isBoxedNumber)
(Boxes_isNumberOrBool,method isBoxedNumberOrBoolean)
(String_$plus,method +)
(Object_synchronized,method synchronized)
(Object_asInstanceOf,method $asInstanceOf)
(Object_isInstanceOf,method $isInstanceOf)
(Object_ne,method ne)
(Object_eq,method eq)
(Object_$bang$eq,method !=)
(Object_$eq$eq,method ==)
(Object_$hash$hash,method ##)
(Any_asInstanceOf,method asInstanceOf)
(Any_isInstanceOf,method isInstanceOf)
(Any_getClass,method getClass)
(Any_$hash$hash,method ##)
(Any_toString,method toString)
(Any_hashCode,method hashCode)
(Any_equals,method equals)
(Any_$bang$eq,method !=)
(Any_$eq$eq,method ==)
(Option_apply,method apply)
(TypeTagModule,object TypeTag)
(TypeTagClass,trait TypeTag)
(WeakTypeTagModule,object WeakTypeTag)
(WeakTypeTagClass,trait WeakTypeTag)
(ExprModule,object Expr)
(ExprClass,trait Expr)
(PartialManifestClass,type ClassManifest)
(Array_clone,method clone)
(Array_length,method length)
(Array_update,method update)
(Array_apply,method apply)
(ArrayModule_overloadedApply,value apply)
(Iterator_apply,method apply)
(List_apply,method apply)
(StringAdd_$plus,method +)
(ArrowAssocClass,class ArrowAssoc)
(Symbol_apply,method apply)
(GroupOfSpecializable,class Group)
(RootClass,package <root>)
(RootPackage,package _root_)
(Boolean_not,method unary_!)
(Boolean_or,method ||)
(Boolean_and,method &&)
(ExistentialsFeature,trait existentials)
(HigherKindsFeature,trait higherKinds)
(ImplicitConversionsFeature,trait implicitConversions)
(ReflectiveCallsFeature,trait reflectiveCalls)
(PostfixOpsFeature,trait postfixOps)
(DynamicsFeature,trait dynamics)
(MacrosFeature,trait macros)
(experimentalModule,object experimental)
(Boxes_isNumber,method isBoxedNumber)
(Boxes_isNumberOrBool,method isBoxedNumberOrBoolean)
(String_$plus,method +)
(Object_synchronized,method synchronized)
(Object_asInstanceOf,method $asInstanceOf)
(Object_isInstanceOf,method $isInstanceOf)
(Object_ne,method ne)
(Object_eq,method eq)
(Object_$bang$eq,method !=)
(Object_$eq$eq,method ==)
(Object_$hash$hash,method ##)
(Any_asInstanceOf,method asInstanceOf)
(Any_isInstanceOf,method isInstanceOf)
(Any_getClass,method getClass)
(Any_$hash$hash,method ##)
(Any_toString,method toString)
(Any_hashCode,method hashCode)
(Any_equals,method equals)
(Any_$bang$eq,method !=)
(Any_$eq$eq,method ==)
(Option_apply,method apply)
(TypeTagModule,object TypeTag)
(TypeTagClass,trait TypeTag)
(WeakTypeTagModule,object WeakTypeTag)
(WeakTypeTagClass,trait WeakTypeTag)
(ExprModule,object Expr)
(ExprClass,trait Expr)
(PartialManifestClass,type ClassManifest)
(Array_clone,method clone)
(Array_length,method length)
(Array_update,method update)
(Array_apply,method apply)
(ArrayModule_overloadedApply,value apply)
(Iterator_apply,method apply)
(List_apply,method apply)
(StringAdd_$plus,method +)
(ArrowAssocClass,class ArrowAssoc)
(Symbol_apply,method apply)
(GroupOfSpecializable,class Group)
(RootClass,package <root>)
(RootPackage,package _root_)
(Boolean_not,method unary_!)
(Boolean_or,method ||)
(Boolean_and,method &&)

I wonder whether the fix for this should be in Definitions (changing vals to defs), or in findMirrorSymbol.

@adriaanm said:
I wonder how the new symbol gets divorced from the original one, and if this happens to other symbols.
Are they in different universes?

@retronym said (edited on Jul 22, 2013 5:49:43 AM UTC):
I'm guessing that the mirror symbol is actually supposed to be a distinct symbol.

/** Find a 'mirror' of symbol `sym` in unit `unit`. Pre: `unit is loaded. */
  private def findMirrorSymbol(sym: Symbol, unit: RichCompilationUnit): Symbol = {

The problem is that it ends up in the decls of its owner's (TypeTags) info.

@retronym said:
I just chatted with Martin about this problem.

The symptoms could be fixed by avoiding caching non-top-level symbols in Definitions.

But we should also consider the broader problem: Should we let the results of type-checking sources of third-party JARs infiltrate the compiler used to type check at all?

I can think of a few reasons why this is a bad idea:

• exposure to problems like this one
• we might not be able to typecheck the source. Our classpath might be missing JARs which are required for typechecking but optional at runtime. Or that library might require different compiler options or compiler plugins to compile.
• the classfiles represent the ultimate truth; if the wrong sources are attached we will end up with different behaviour between the presentation and batch compilers.

But how could we go about firewalling our symbol table from the vaguaries of third party source JARs?

The rough idea we had:

• the presentation compiler would typecheck third-party sources with a custom type checker mode
• unlike the regular compiler, which would enter new symbols for defintions, this mode would:
  • lookup and reference the existing symbol for types, vals, vars, and nullary defs
  • create new symbols for other defs (which are potentially overloaded), but not enter these new symbols into enclosing scope.

Hyperlinking from a call-site to a non-nullary method would have to find the corresponding overload based on type signatures.

No doubt the details would be somewhat devilish. But it seems like a workable solution. Mirko suggests that this problem might become more prevalent with ScalaSearch.

@dragos said:
Another thing that might be worth considering is that "askLinkPos", the presentation compiler service for hyperlinking, does not create a new compiler Run, even though it enters new Symbols. I am not sure, but I think that's what we're seeing here: hyperlinking parses and enters a new Symbol for TypeTag, and both the old one and the new one are "valid" in the current Run, therefore the old one is not adapted.

@xeno-by said:
What's ScalaSearch?

@dotta said:

• https://github.com/scala-ide/scala-search
• http://scala-ide.org/blog/release-notes-3.0.1-RC01.html#scala_search

@retronym said:
The patch I submitted to fix this is pretty big, not sure if we'll be willing to apply it to 2.10.4.

scala/scala#3092

@dotta said:
@jason I think it's no problem to target 2.11 for this. And thanks!

@vigdorchik said:
Is it the same issue that is affecting interactive scaladoc in the IDE for 2.10? If so, I think it would be nice if the fix or a lighter version of it could be back-ported to 2.10.x.