Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Source.mkString performs painfully slow compared to Source.getLines.mkString #7356

Closed
scabug opened this issue Apr 11, 2013 · 5 comments
Closed

Source.mkString performs painfully slow compared to Source.getLines.mkString #7356

scabug opened this issue Apr 11, 2013 · 5 comments
Assignees
@Ichoran
Labels
library:collections performance
Milestone
2.11.0-M6

Comments

@scabug
Copy link

scabug commented Apr 11, 2013

When reading about consuming a text file, or a stream with scala.io.Source, most literature recommends using Source.fromInputStream(...).mkString.

This, however, incurs a terrible performance penalty, presumably because of how Iterator.mkString is implemented. This performance issue is prevented by calling Source.getLines.mkString.

The issue is reproducible by the attached source file.
@scabug
Copy link
Author

scabug commented Apr 11, 2013

Imported From: https://issues.scala-lang.org/browse/SI-7356?orig=1
Reporter: Shai Yallin (electricmonk)
Attachments:

@scabug
Copy link
Author

scabug commented Apr 11, 2013

@vigdorchik said (edited on Apr 11, 2013 12:48:26 PM UTC):
scala.io.Source extends Iterator[Char], so mkString is naturally slow, has nothing to do with Iterator performance. Surely this is a bad advise to get the content of the file this way.

@scabug
Copy link
Author

scabug commented Apr 11, 2013

Shai Yallin (electricmonk) said:
My point is that the fact that it extends an Iterator[Char] is kind of an implementation detail; ideally it represent an abstraction over some source of input.
Bad advice as it is, community-speaking this seems to be the common knowledge of how to consume a Source, which - as I said - is bad.

@scabug
Copy link
Author

scabug commented Sep 9, 2013

@Ichoran said:
Overloading mkString is an easy fix. Retrofitting a Char-customized iterator into Source would improve performance in other cases also. Will do one or the other; being able to conveniently get the entire contents of a file is a fundamental feature of an i/o library.

@scabug
Copy link
Author

scabug commented Sep 10, 2013

@Ichoran said:
Overloaded mkString. Ignored issues of onException handling since getLines() already ignored those. Issue resolved by scala/scala#2929

@scabug scabug closed this as completed Sep 11, 2013
@scabug scabug added this to the 2.11.0-M6 milestone Apr 7, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Ichoran Ichoran

Labels
library:collections performance
Projects
None yet
Milestone
2.11.0-M6
Development

No branches or pull requests
2 participants
@Ichoran @scabug