Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory leak in SynchronizedTypes.uniques #9537

Open
scabug opened this issue Oct 27, 2015 · 4 comments
Open

Memory leak in SynchronizedTypes.uniques #9537

scabug opened this issue Oct 27, 2015 · 4 comments
Assignees
Milestone

Comments

@scabug
Copy link

scabug commented Oct 27, 2015

calling

scala.reflect.runtime.universe.typeOf[Option[_]]

creates new instances for each call in

scala.reflect.runtime.SynchronizedTypes.uniques

How to reproduce:

object TypeLeak extends App {

  def getUniques:scala.collection.mutable.WeakHashMap[_,_] = {
    classOf[scala.reflect.runtime.JavaUniverse].getMethods.find(_.getName.endsWith("SynchronizedTypes$$uniques"))
      .map(_.invoke(scala.reflect.runtime.universe).asInstanceOf[scala.collection.mutable.WeakHashMap[_,_]])
      .getOrElse(scala.collection.mutable.WeakHashMap())
  }

  scala.reflect.runtime.universe.typeOf[Option[Nothing]] <:< scala.reflect.runtime.universe.typeOf[Option[Any]]
  val uniques = getUniques.toMap[Any, Any]
  println("Leaking...")
  1 to 2 foreach{ _ 
    if(scala.reflect.runtime.universe.typeOf[Option[_]] <:< scala.reflect.runtime.universe.typeOf[Option[Any]])
      println(getUniques.size)
  }
  1 to 2 foreach{ _ 
    if(scala.reflect.runtime.universe.typeOf[Option[Any]] <:< scala.reflect.runtime.universe.typeOf[Option[_]])
      println(getUniques.size)
  }

  println("Not leaking...")
  1 to 2 foreach{ _ 
    if(scala.reflect.runtime.universe.typeOf[Option[Nothing]] <:< scala.reflect.runtime.universe.typeOf[Option[Any]])
      println(getUniques.size)
  }

  println("Leaked references:")
  (getUniques.toMap[Any, Any] -- uniques.keys).foreach(println)

}
@scabug
Copy link
Author

scabug commented Oct 27, 2015

Imported From: https://issues.scala-lang.org/browse/SI-9537?orig=1
Reporter: Stefan Jurco (stefan.jurco-at-gmail.com)
Affected Versions: 2.11.7

@scabug
Copy link
Author

scabug commented Oct 27, 2015

@retronym said:
My reading of the code is that the entries are free to be collected under a GC.

Here's the implementation:

  // we can keep this lock fine-grained, because super.unique just updates the cache
  // and, in particular, doesn't call any reflection APIs which makes deadlocks impossible
  private lazy val uniqueLock = new Object
  private val uniques = mutable.WeakHashMap[Type, jWeakRef[Type]]()
  override def unique[T <: Type](tp: T): T = uniqueLock.synchronized {
    // we need to have weak uniques for runtime reflection
    // because unlike the normal compiler universe, reflective universe isn't organized in runs
    // therefore perRunCaches can grow infinitely large
    //
    // despite that toolbox universes are decorated, toolboxes are compilers,
    // i.e. they have their caches cleaned up automatically on per-run basis,
    // therefore they should use vanilla uniques, which are faster
    if (!isCompilerUniverse) {
      val inCache = uniques get tp
      val result = if (inCache.isDefined) inCache.get.get else null
      if (result ne null) result.asInstanceOf[T]
      else {
        uniques(tp) = new jWeakRef(tp)
        tp
      }
    } else {
      super.unique(tp)
    }
  }

Can you demonstrate the leak in a stronger way than observing the size of this weak map?

@scabug
Copy link
Author

scabug commented Oct 27, 2015

Stefan Jurco (stefan.jurco-at-gmail.com) said:
Sorry, maybe leak is not a good description for that. It will be garbage collected in some point of time. But it can grow enormously just because the generated key is unique for each run.
I cannot provide any stronger proof just the numbers from running application..
The size of uniques was few milions while using Option[ _ ] compared with 800 while using Option[ Any ]

@scabug
Copy link
Author

scabug commented Oct 27, 2015

@retronym said:
Existential subtype calculations create temporary type variables for the quantified parameters:

https://github.com/scala/scala/blob/v2.11.7/src/reflect/scala/reflect/internal/tpe/TypeComparers.scala#L482-L483

This is a pretty fundamental part of the compiler and I can't see a way to change this.

These will indeed end up in the uniques cache until GC evicts them.

One idea you might be able to try is use this way to find whether a type is a subclass of Option or not:

scala> val optionClass = symbolOf[Option[_]] // just do this once
optionClass: reflect.runtime.universe.TypeSymbol = class Option

scala> typeOf[Option[Any]].baseType(optionClass) != NoType
res3: Boolean = true

scala> typeOf[Tuple1[Any]].baseType(optionClass) != NoType
res4: Boolean = false

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants