You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The standard library method Iterator#++ creates a JoinIterator. The hasNext method on JoinIterator always calls the left iterator's hasNext method, rather than remembering that it has been exhausted.
In cases where the left iterator's hasNext method is expensive, such as any kind of IO, this was at the root of a baffling performance problem.
This behavior is fixed in ConcatIterator, which never calls the underlying hasNext a second time for an iterator which already returned false. So that provides an easy, if dirty, workaround:
valit1=newIteratorWithExpensiveHasNext()
valit2=newIteratorWithExpensiveHasNext()
// SLOW due to calling it1.hasNext every time
(it1 ++ it2).foreach(_ => ())
// FASTER due to NOT calling it1.hasNext every time
(Iterator.empty ++ it1 ++ it2).foreach(_ => ())
NOTE: you can use #memoizeExhaustion from my iterata project as a workaround, as well.
The text was updated successfully, but these errors were encountered:
@szeiger said:
Looks easily fixable but the ConcatIterator version is not much better. Iterating over 2 iterators of 3 elements each with JoinIterator calls hasNext (on either side) a total of 17 times, and it's still 14 times with ConcatIterator.
See demonstration of the issue.
The standard library method
Iterator#++
creates aJoinIterator
. ThehasNext
method on JoinIterator always calls the left iterator'shasNext
method, rather than remembering that it has been exhausted.In cases where the left iterator's hasNext method is expensive, such as any kind of IO, this was at the root of a baffling performance problem.
This behavior is fixed in
ConcatIterator
, which never calls the underlying hasNext a second time for an iterator which already returned false. So that provides an easy, if dirty, workaround:NOTE: you can use
#memoizeExhaustion
from my iterata project as a workaround, as well.The text was updated successfully, but these errors were encountered: