Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

memory and jar file handler leak #9682

Open
scabug opened this issue Mar 2, 2016 · 5 comments
Open

memory and jar file handler leak #9682

scabug opened this issue Mar 2, 2016 · 5 comments
Labels
backend
Milestone
Backlog

Comments

@scabug
Copy link

scabug commented Mar 2, 2016

I have spotted this bug in the context of the presentation compiler but it also affects the compiler: https://github.com/ensime/ensime-server/issues/1322

The compiler's classloader is not releasing references to jars and they get kind of lost. It's actually a well known JVM bug and also affects sbt as I noted here sbt/sbt#2496

The impact is that jar files (which could be targets of the build tool, or scalac itself) on Windows become immutable and cannot be deleted, changed, or moved. This is really very bad. In the best case scenario (Linux), it's just a memory leak.

Some more reading

with the http://www.docjar.com/docs/api/com/sun/tools/javac/file/CloseableURLClassLoader.html being proposed as a workaround (I've not personally used it, but it sounds sensible). I got IllegalArgumentExceptions when I tried to reflectively access the internals of the classloader, so I hope the openjdk class is somehow able to workaround that.

Note that this is independent of #9632 but they have a similar effect. But this one is far more serious as when the classloader is thrown away, the JVM seems to be unable to ever run the finalizers of the JarFiles.
@scabug
Copy link
Author

scabug commented Mar 2, 2016

Imported From: https://issues.scala-lang.org/browse/SI-9682?orig=1
Reporter: Samuel Halliday (fommil)

@scabug
Copy link
Author

scabug commented Mar 3, 2016

@retronym said:
More specifically, Global has a lazy val macroClassLoader which is used to load macro implementations. This class loader is neither closed, nor is a reference to it released, as the end of a run.

Here's a sketch of how we could improve the situation:

https://github.com/scala/scala/compare/2.12.x...retronym:ticket/9682?expand=1

The diff gets large as I started applying the same idea to the compilation classpath. That is tricker as we need to propagate the close() calls all the way down the line to the places to all the places that opened file handles.

@scabug
Copy link
Author

scabug commented Mar 14, 2016

Samuel Halliday (fommil) said:
Sbt does something similar, I have a failing appveyor reproduction and I'm hunting for a workaround no matter how evil sbt/sbt#2496

@scabug
Copy link
Author

scabug commented Apr 21, 2016

Samuel Halliday (fommil) said:
I have a workaround for this, see https://github.com/fommil/class-monkey

It would be much better if any use of URLClassLoader was replaced with a custom implementation, aimed at cleaning up afterwards and only handling local files. Happy to contribute my code if it could form the basis of a better solution, but the tests must remain GPLv2 as they are from OpenJDK.

@scabug
Copy link
Author

scabug commented Jan 10, 2017

@adriaanm said:
Pushed to 2.12, as 2.11 development has wound down.

@scabug scabug added this to the Backlog milestone Apr 7, 2017
@scabug scabug mentioned this issue Apr 7, 2017
Open
@avakhrenev avakhrenev mentioned this issue Apr 25, 2017
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
backend
Projects
None yet
Milestone
Backlog
Development

No branches or pull requests
2 participants
@SethTisue @scabug