java annotation and reflection: "illegal cyclic reference involving object InterfaceAudience" #10129
Comments
|
Imported From: https://issues.scala-lang.org/browse/SI-10129?orig=1
|
|
I just wanted to ping this issue so see what it's status was. Any idea why this error occurs? |
|
Is there any update? i met with same error |
|
a good next step would be for someone to construct a self-contained reproduction (preferably not involving any external libraries); nobody's likely to attempt a fix if we don't have one of those |
|
I met with the same error when using scala reflection to parse and execute multiple case classes. Any update on the issue? |
|
@badrinathpatchikolla are you able to construct a self-contained reproduction? |
|
I am using Spark Standalone Spark Version: 2.4.7 and Scala 2.11.12 Steps to reproduce : import scala.reflect.runtime.currentMirror
import scala.tools.reflect.ToolBox
def compile[A](codeStr: String): A = {
val toolbox = currentMirror.mkToolBox()
val tree = toolbox.parse(codeStr)
toolbox.eval(tree).asInstanceOf[A]
}
val codeStr1=
"""
import org.apache.spark.SparkConf
import org.apache.spark.sql.SparkSession
val sparkConf = new SparkConf
val spark = SparkSession.builder().config(sparkConf).getOrCreate
spark.sparkContext.hadoopConfiguration.set("fs.s3a.access.key", "tes")
""".stripMargin
println(compile[Any](codeStr1))
Error : This is coming when i add spark.sparkContext.hadoopConfiguration.set("fs.s3a.access.key", "test") hadoopConfiguration line to the code. |
|
what you've provided is a step in the right direction, but: a self-contained reproduction would need to not involve Spark it would also need to be on Scala 2.13 (or at least 2.12), not 2.11; 2.11 is no longer maintained |
|
Hey @SethTisue 👋 This is Nima from engineering team of Hootsuite. I spend a few days and created a minimal test case to reproduce this issue. I also did a tentative investigation on why this occurs. I am very much willing to contribute to my lovely programming language 😍 though I need a bit of guidance before implementing any solution. Kindly please take a look at my draft PR here and share your thoughts with me. |
|
@nimatrueway Thanks for the repro. I tried them out as partest tests at scala/scala#10078 but once they work they can run as in-process junit tests. I'll take a look at your diagnosis for fun; I don't know much about reflection; I know work has been done to support Java inner classes. |
|
@lrytz maybe you could look at Nima's code...? |
|
@SethTisue This issue present in scala 2.12.x also, is this fixed in scala 2.13.x? |
|
@badrinathpatchikolla my test linked in my previous comment still fails. That is Nima's test. I had added some debug locally, but I don't remember whether I understood Nima's helpful comments on nimatrueway. My comment "for fun" is a code phrase for "I'll run a test while doing something else [usually preparing food]." |
|
It's fairly easy to reproduce. Just copy the annotation in a file. package foo;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
@InterfaceAudience.Public
public class InterfaceAudience {
@Retention(RetentionPolicy.RUNTIME)
public @interface Public {}
}import foo.InterfaceAudience
import scala.reflect.runtime.universe._
object Annotated extends App {
@InterfaceAudience.Public class Foo
println(typeOf[Foo])
}scala.reflect.internal.Symbols$CyclicReference: illegal cyclic reference involving class InterfaceAudience
Exception in thread "main" java.lang.RuntimeException: error reading Scala signature of Annotated: illegal cyclic reference involving class InterfaceAudience
at scala.reflect.internal.pickling.UnPickler.unpickle(UnPickler.scala:48)
at scala.reflect.runtime.JavaMirrors$JavaMirror.unpickleClass(JavaMirrors.scala:676)
at scala.reflect.runtime.SymbolLoaders$TopClassCompleter.$anonfun$complete$3(SymbolLoaders.scala:37)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
at scala.reflect.internal.SymbolTable.slowButSafeEnteringPhaseNotLaterThan(SymbolTable.scala:320)
at scala.reflect.runtime.SymbolLoaders$TopClassCompleter.complete(SymbolLoaders.scala:34)
at scala.reflect.internal.Symbols$Symbol.completeInfo(Symbols.scala:1572)
at scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1535)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$8.scala$reflect$runtime$SynchronizedSymbols$SynchronizedSymbol$$super$info(SynchronizedSymbols.scala:206)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol.info(SynchronizedSymbols.scala:158)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol.info$(SynchronizedSymbols.scala:158)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$8.info(SynchronizedSymbols.scala:206)
at scala.reflect.internal.ReificationSupport$ReificationSupportImpl.select(ReificationSupport.scala:34)
at scala.reflect.internal.ReificationSupport$ReificationSupportImpl.selectType(ReificationSupport.scala:25)
at scala.reflect.internal.ReificationSupport$ReificationSupportImpl.selectType(ReificationSupport.scala:23)
at Annotated$$typecreator1$1.apply(Annotated.scala:6)
at scala.reflect.api.TypeTags$WeakTypeTagImpl.tpe$lzycompute(TypeTags.scala:238)
at scala.reflect.api.TypeTags$WeakTypeTagImpl.tpe(TypeTags.scala:238)
at scala.reflect.api.TypeTags.typeOf(TypeTags.scala:359)
at scala.reflect.api.TypeTags.typeOf$(TypeTags.scala:359)
at scala.reflect.api.Universe.typeOf(Universe.scala:73)
at Annotated$.delayedEndpoint$Annotated$1(Annotated.scala:6)
at Annotated$delayedInit$body.apply(Annotated.scala:4)
at scala.Function0.apply$mcV$sp(Function0.scala:42)
at scala.Function0.apply$mcV$sp$(Function0.scala:42)
at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
at scala.App.$anonfun$main$1(App.scala:98)
at scala.App.$anonfun$main$1$adapted(App.scala:98)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:575)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:573)
at scala.collection.AbstractIterable.foreach(Iterable.scala:933)
at scala.App.main(App.scala:98)
at scala.App.main$(App.scala:96)
at Annotated$.main(Annotated.scala:4)
at Annotated.main(Annotated.scala) |
See attached exception traces.
With Scala 2.11.8 and Hadoop CDH v5.4, calling
typeOfon a type annotated with the HadoopInterfaceAudienceannotation causes ascala.reflect.internal.Symbols$CyclicReference: illegal cyclic reference involving object InterfaceAudienceexception in the REPL. (This is a problem since I use the Ammonite REPL, and it always callstypeOfon types it wants to print to the console. But the issue exists independent of which REPL I am using, given the attached trace is in the regular REPL.)The interesting thing is that calling
typeOfwith the fully-qualified reference causes a different exception:java.lang.AssertionError: assertion failed: no symbol could be loaded from interface org.apache.hadoop.classification.InterfaceAudience$Public in object InterfaceAudience with name Public and classloader scala.reflect.internal.util.ScalaClassLoader$URLClassLoader@536aaa8dThe text was updated successfully, but these errors were encountered: