Uploaded image for project: 'Scala Programming Language'
  1. Scala Programming Language
  2. SI-9587

ScalaClassLoader.asContext highjacks the thread context classloader and causes classloader branching that destroy proper parent relationship

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: Scala 2.11.7
    • Fix Version/s: None
    • Component/s: Repl / Interpreter
    • Labels:
      None
    • Environment:
      • Windows 7; Oracle JDK build 1.8.0_31-b13; sbt 0.13.8
      • project\build.properties -> sbt.version=0.13.1

      Description

      ScalaClassLoader.asContext highjacks the thread context classloader and causes classloader branching that destroy proper parent relationship

      The thing I tried solving was running some runtime compilation that would allow some parts of my configuration be written in scala code (essentially the config is a function that maps stuff).

      To get that working this is what I did.

      val factory = new IMain.Factory()
      val engine = factory.getScriptEngine().asInstanceOf[IMain]
      val cl = java.lang.Thread.currentThread.getContextClassLoader
      engine.settings.embeddedDefaults(cl)
      engine.settings.usejavacp.value = true
       
      val sf = new BatchSourceFile(new PlainFile(Path(s"local.conf/${script}")))
      engine.compileSources(sf)
       
      val fun = engine.eval(customConf.getString(GENERATOR_FUNCTION))
         .asInstanceOf[(java.util.Map[String, Object] => java.util.Map[String, Object])]
      

      This code worked in an isolated test program, but once I started combining the thing with real code everything broke down. Example issue:

      Caused by: org.apache.velocity.exception.VelocityException: Failed to initialize an instance of org.apache.velocity.runtime.log.NullLogChute with the current runtime configuration.
          at org.apache.velocity.runtime.log.LogManager.createLogChute(LogManager.java:220)
          at org.apache.velocity.runtime.log.LogManager.updateLog(LogManager.java:269)
          at org.apache.velocity.runtime.RuntimeInstance.initializeLog(RuntimeInstance.java:871)
          ... 18 more
      Caused by: org.apache.velocity.exception.VelocityException: The specified logger class org.apache.velocity.runtime.log.NullLogChute does not implement the org.apache.velocity.runtime.log.LogChute interface.
          at org.apache.velocity.runtime.log.LogManager.createLogChute(LogManager.java:181)
          ... 20 more
      

      Those two classes come from the same jar, so typing between them should not ever be a problem. To me it looked like a Classloader issue, and it was.

      Turns out the org.apache.velocity.runtime.log.LogChute interface was loaded by:
      sun.misc.Launcher$AppClassLoader (1 instance in heap)
      org.apache.velocity.runtime.log.NullLogChute and second copy of LogChute where loaded by:
      scala.reflect.internal.util.ScalaClassLoader$URLClassLoader (1 instance in heap)
      Both classloaders had a common parent:
      sun.misc.Launcher$ExtClassLoader
      which makes them classloader siblings explaining the incompatible types.

      I thought it was going wrong in the compile part, but as it turns out, it breaks in the eval() part.
      At some point this sequence of things is called (not full stack listed, top down order).
      scala.tools.nsc.interpreter.IMain.WrappedRequest.loadAndRunReq
      scala.reflect.internal.util.ScalaClassLoader.asContext
      scala.reflect.internal.util.ScalaClassLoader.apply(cl: ClassLoader)

      These are the instresting bits
      scala.reflect.internal.util.ScalaClassLoader.asContext

        def asContext[T](action: => T): T = {
          val saved = contextLoader
          try { setContext(this) ; action }
          finally setContext(saved)
        }
      

      scala.reflect.internal.util.ScalaClassLoader.contextLoader

      def contextLoader = apply(Thread.currentThread.getContextClassLoader)
      

      scala.reflect.internal.util.ScalaClassLoader.apply(cl: ClassLoader)

        implicit def apply(cl: JClassLoader): ScalaClassLoader = cl match {
          case cl: ScalaClassLoader => cl
          case cl: JURLClassLoader  => new URLClassLoader(cl.getURLs.toSeq, cl.getParent)
          case _                    => new JClassLoader(cl) with ScalaClassLoader
        }
      

      Now what happens here is:

      1. When the context class loader happens to be a java URLClassloader, it gets stripped and repackaged in a scala URLClassloader, with reuse of the parent of the original loader
      2. This destroys a meaningful parent relationship that should have been there, allowing commons stuff to be loaded by a common Classloader
      3. It also replaces what is 'saved' in the asContext utility
      4. Hence it is what is restored as thread context class loader once asContext is done

      Which led to the breakage described above.

      With this insight I was able to build this workarround (it prepackages the context class loader as the fallback case does in the above code):

        val t = java.lang.Thread.currentThread
        val cl = t.getContextClassLoader
        try {
          val clplus = new java.lang.ClassLoader(cl) with ScalaClassLoader
          t.setContextClassLoader(clplus)
       
          val factory = new IMain.Factory()
          val engine = factory.getScriptEngine().asInstanceOf[IMain]
          engine.settings.embeddedDefaults(clplus)
          engine.settings.usejavacp.value = true
       
          val sf = new BatchSourceFile(new PlainFile(Path(s"local.conf/${script}")))
          engine.compileSources(sf)
       
          val fun = engine.eval(customConf.getString(GENERATOR_FUNCTION))
            .asInstanceOf[(java.util.Map[String, Object] => java.util.Map[String, Object])]
       
          engine.close()
       
          Some(fun) // returning the function here
        } finally {
          t.setContextClassLoader(cl)
        }
      

      Though I'm glad I could work around it, I can't help at feel this smells like a bug.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                spangaer Jean-Luc Deprez
              • Votes:
                2 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated: