You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Future's flatMap implementation holds on to a little to much inside it's inner closure. This can be seen when just using a future, but is really easy to show when using Akka's dataflow and CPS: https://gist.github.com/amirshim/5602245
defmainLoop() :Unit@cps[Future[Any]] = {
while (true) {
valfut=Promise.successful(newArray[Byte](1000000*100)).future // 100MB// This shift is the same as Akka DataFlow's DataflowFuture.apply()valresult= shift { onComplete: (Array[Byte] =>Future[Any]) =>
fut flatMap onComplete
}
println(result)
}
}
Although this is a real memory leak in an infinite loop, it doesn't need to be so bad, especially when we have long iterators that allocate lots of memory.
If we limit what the inside closure holds on to, by introducing tempP (see below), we can mitigate most of the memory use in this case: https://gist.github.com/amirshim/5602077
Future's flatMap implementation holds on to a little to much inside it's inner closure. This can be seen when just using a future, but is really easy to show when using Akka's dataflow and CPS:
https://gist.github.com/amirshim/5602245
When this code is run, it leaks because the continuation is waiting for the loop to finish. (at https://github.com/scala/scala/blob/1f4a52b4ed9457863e00fe16d18705b6c6cd5db9/src/library/scala/concurrent/Future.scala#L280) and holding onto all the previous closures.
Although this is a real memory leak in an infinite loop, it doesn't need to be so bad, especially when we have long iterators that allocate lots of memory.
If we limit what the inside closure holds on to, by introducing tempP (see below), we can mitigate most of the memory use in this case:
https://gist.github.com/amirshim/5602077
The text was updated successfully, but these errors were encountered: