New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
scala.collection.immutable.StreamIterator[+A] has O(n) memory consumption #4835
Comments
Imported From: https://issues.scala-lang.org/browse/SI-4835?orig=1
|
@dcsobral said: I'm surprised, however, that "val stream" inside the definition of "var these" doesn't suffer from the same problem -- even though it is in a small scope, it ought to be saved so that it is preserved when the by-name LazyCell gets invoked. It might even be a bug here. |
@kmizu said:
I think that it isn't big problem, too. However, I think that Iterator#next is not to expected consume O(n) memory generally.
Because primary constructor's arguments Thanks again. Kota Mizushima |
@dcsobral said: As for self, it * shouldn't* continue to exist. It only did so because it was used in a by-name parameter. |
@kmizu said:
I see that because argument self: Stream itself is passed by value, then the first element wasn't lazy
OK. I created another version of patch and will attach it as StreamIterator.patch.2. By applying this patch, |
huynhjl said: |
@kmizu said: commit 39a837c8357e78c949613db881bd060d176f3279
I'm not a commiter. However, because the fix was merged into master branch, I've thought that it should be noted. |
Jed Wesley-Smith (jedws) said: |
@kmizu said: |
In the following program using Stream#iterator, OutOfMemoryError should not occur
Because no references to Stream's head are not hold and Stream was converted to Iterator
using Stream[+A]#iterator.
object StreamIteratorProblem {
def main(args: Array[String]) {
val infiniteIterator = Stream.from(1).iterator
for(i <- 1 to 10000000) {
val e = infiniteIterator.next
if(e % 1000000 == 0) println(e)
}
}
}
However, When this program runs, OutOfMemoryError occurs as the following:
I found that there exists the reason of this problem in
StreamIterator
in "Stream.scala`.In the definition of StreamIterator as the followings,
class StreamIterator[+A](self: Stream[A]) extends Iterator[A] {
// A call-by-need cell.
class LazyCell(st: => Stream[A]) {
lazy val v = st
}
}
an argument
self
of primary constructor continues to hold the reference to a Stream.Then, whenever StreamIterator#next is called, memory consumption increases.
I created a patch to this problem and will attach it. It seems works well.
The text was updated successfully, but these errors were encountered: