New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
scala.reflect.runtime.ThreadLocalStorage$MyThreadLocalStorage is forever retaining references to threads #8946
Comments
Imported From: https://issues.scala-lang.org/browse/SI-8946?orig=1 |
@timcharper said: |
@timcharper said (edited on Oct 29, 2014 7:05:39 AM UTC): [echo] Checking backward binary compatibility for scala-reflect (against 2.11.0)
[mima] Found 1 binary incompatibiities (27 were filtered out)
[mima] ======================================================
[mima] * method values()java.util.concurrent.ConcurrentHashMap in class
[mima] scala.reflect.runtime.ThreadLocalStorage#MyThreadLocalStorage has now a
[mima] different result type; was: java.util.concurrent.ConcurrentHashMap, is now:
[mima] java.util.Map
[mima] Generated filter config definition
[mima] ==================================
[mima]
[mima] filter {
[mima] problems=[
[mima] {
[mima] matchName="scala.reflect.runtime.ThreadLocalStorage#MyThreadLocalStorage.values"
[mima] problemName=IncompatibleResultTypeProblem
[mima] }
[mima] ]
[mima] }
[mima] I'm not sure why we care about binary compatibility for a private class. I will look to see if there is an efficient way to use ConcurrentHashMap with weak keys. |
@timcharper said: Also, I think the value of MyThreadLocalStorage should be marked private, too. What are your thoughts? |
@timcharper said: |
@retronym said: |
@retronym said: object Test {
def main(args: Array[String]): Unit = {
var i = 0
while (i < 16) {
i += 1
val t = new Thread {
val ballast = Array.ofDim[Byte](256 * 1024 * 1024)
override def run(): Unit = {
import reflect.runtime.universe._
typeOf[List[String]] <:< typeOf[Seq[_]]
} |
@timcharper said: |
@retronym said (edited on Nov 3, 2014 11:07:26 AM UTC): |
@timcharper said (edited on Nov 3, 2014 11:56:09 AM UTC): {code:title=MemTest.scala} import scala.ref.WeakReference object Main extends App { val threads = for (i <- (1 to 16)) yield { |
@timcharper said: |
@retronym said: |
@timcharper said: Are we okay to whitelist the binary incompatible change? That's the final item, I think, before this pull request is ready to go. |
@timcharper said: |
@timcharper said: Thanks for your help with this! |
@retronym said (edited on Nov 3, 2014 12:40:21 PM UTC): Could you please submit a pull request against the 2.11.x branch of scala/scala and @-mention me (retronym) and Eugene (xeno-by) as a reviewers? |
I'm having an issue in one of my applications where it is crashing about once every week or so due to a slow, steady memory leak. For brief overview of a recent heap dump in VisualVM, please watch this brief video I've recorded: http://screencast.com/t/3qiNuzwa
After doing my best to reason about the problem, and still admittedly rather confused about the issue, this is the best hypothesis I can form:
There are several process-lifetime long instances of
scala.reflect.runtime.ThreadLocalStorage$MyThreadLocalStorage
allocated here: https://github.com/scala/scala/blob/v2.11.2/src/reflect/scala/reflect/runtime/SynchronizedTypes.scala#L53Running any scala reflection code in a thread causes that thread to be permanently referenced as a key in the
MyThreadLocalStorage
ConcurrentLinkedHashMap
: https://github.com/scala/scala/blob/v2.11.2/src/reflect/scala/reflect/runtime/ThreadLocalStorage.scala#L24After old threads worker threads die and new ones are spawned, references to the old are retained as keys in
ConcurrentHashMap
residing inThreadLocalStorage$MyThreadLocalStorage
; eventually, all memory is consumed and the JVM crashes.I did my best to capture a stack-trace of when these thread references are being inserted into the
ThreadLocalStorage
:(Line numbers for
ThreadLocalStorage.scala
are off because I'd added some code to help with debugging)It seems that
ThreadLocalStorage
should use a storage mechanism that allows for garbage collection of the keys, such asWeakHashMap
.I have pushed a suggested fix to github, here:
https://github.com/spingo/scala/tree/issue/SI-8946
I am open to suggestions on how to write an effective test for this.
The text was updated successfully, but these errors were encountered: