Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Builds relying on Scala 2.10.2 artifact in Maven are failing #8772

Closed
scabug opened this issue Aug 1, 2014 · 25 comments
Closed

Builds relying on Scala 2.10.2 artifact in Maven are failing #8772

scabug opened this issue Aug 1, 2014 · 25 comments
Assignees
Labels

Comments

@scabug
Copy link

scabug commented Aug 1, 2014

Several Spark users reported this today and it's breaking our build. I narrowed this down to the following: a trivial project build on Scala 2.10.2 can not resolve artifacts correctly:

$ echo "scalaVersion := \"2.10.2\"" > build.sbt
$ sbt compile
...
[warn] 	module not found: org.scala-lang#scala-library;2.10.2
[warn] ==== local: tried
[warn]   /home/patrick/.ivy2/local/org.scala-lang/scala-library/2.10.2/ivys/ivy.xml
[warn] ==== public: tried
[warn]   http://repo1.maven.org/maven2/org/scala-lang/scala-library/2.10.2/scala-library-2.10.2.pom
[info] Resolving org.scala-lang#scala-compiler;2.10.2 ...
[warn] 	module not found: org.scala-lang#scala-compiler;2.10.2
[warn] ==== local: tried
[warn]   /home/patrick/.ivy2/local/org.scala-lang/scala-compiler/2.10.2/ivys/ivy.xml
[warn] ==== public: tried
[warn]   http://repo1.maven.org/maven2/org/scala-lang/scala-compiler/2.10.2/scala-compiler-2.10.2.pom
[warn] 	::::::::::::::::::::::::::::::::::::::::::::::
[warn] 	::          UNRESOLVED DEPENDENCIES         ::
[warn] 	::::::::::::::::::::::::::::::::::::::::::::::
[warn] 	:: org.scala-lang#scala-library;2.10.2: not found
[warn] 	:: org.scala-lang#scala-compiler;2.10.2: not found
[warn] 	::::::::::::::::::::::::::::::::::::::::::::::
sbt.ResolveException: unresolved dependency: org.scala-lang#scala-library;2.10.2: not found
unresolved dependency: org.scala-lang#scala-compiler;2.10.2: not found

The same code works for 2.10.1 and 2.10.3. The issue seems to be that maven is returning 404 errors for the entire 2.10.2 directory.

# returns 404
http://repo1.maven.org/maven2/org/scala-lang/scala-library/2.10.2/
# works
http://repo1.maven.org/maven2/org/scala-lang/scala-library/2.10.3/

The error message from sbt seems to imply the artifact or POM can not be downloaded, but from what I can tell, the most likely issue is that the parent directory containing them cannot be listed. I can download the pom and jar easily without any issue.

As a workaround, if you have access to the machine where the build is running, you can manually populate your ivy cache with the needed jars:

$ cd ~/.ivy2/cache/org.scala-lang/
$ mkdir -p scala-library && cd scala-library
$ wget https://raw.githubusercontent.com/peterklipfel/scala_koans/master/ivyrepo/cache/org.scala-lang/scala-library/ivy-2.10.2.xml
$ wget https://raw.githubusercontent.com/peterklipfel/scala_koans/master/ivyrepo/cache/org.scala-lang/scala-library/ivydata-2.10.2.properties
$ mkdir -p jars && cd jars
$ wget https://github.com/peterklipfel/scala_koans/raw/master/ivyrepo/cache/org.scala-lang/scala-library/jars/scala-library-2.10.2.jar
@scabug
Copy link
Author

scabug commented Aug 1, 2014

Imported From: https://issues.scala-lang.org/browse/SI-8772?orig=1
Reporter: Patrick Wendell (pwendell)
Affected Versions: 2.10.2

@scabug
Copy link
Author

scabug commented Aug 1, 2014

@gourlaysama said (edited on Aug 4, 2014 12:01:31 PM UTC):
Yes, this very weird, and we don't all get the same results: from my location, I can't even download the jar directly, while apparently you can (different mirrors...).

As a workaround, you can have it download from Sonatype with (in build.sbt):

resolvers += Resolver.sonatypeRepo("releases")

@scabug
Copy link
Author

scabug commented Aug 2, 2014

@milessabin said:
That workaround isn't working for me ... I'm still seeing this Travis build failure, following this commit.

@scabug
Copy link
Author

scabug commented Aug 2, 2014

@gourlaysama said (edited on Aug 2, 2014 12:22:16 PM UTC):
Looking at the Travis build, it seems to be your /project/ project definition that fails to build, not shapeless itself (sbt-pgp transitively depends on 2.10.2), so you need to add that line to /project/plugins.sbt, not /project/Build.scala.

@scabug
Copy link
Author

scabug commented Aug 2, 2014

@milessabin said:
Yes, that's done the trick ... thanks.

@scabug
Copy link
Author

scabug commented Aug 2, 2014

fred aalbue (faalbue) said:
I have a play framework application I am deploying to heroku that still fails....and the issue of the 2.10.2 directory on the maven repo still returns 404......when will the 2.10.2 directory be populated again? Do I need to wait for replication to the mirrors?

Thanks,
Fred

@scabug
Copy link
Author

scabug commented Aug 3, 2014

Pradeep Jain (pradeep.jain-at-gmail.com) said:
My Build is also failing for the same reason.
Kindly resolve this asap.
Thanks.

@scabug
Copy link
Author

scabug commented Aug 4, 2014

Tobias Svensson (svensson.s.t) said:
If you are building a play app (or if you are building with sbt) you might be able to get around the problem by changing the sbt version to i.e. 0.13.5. In the play framework (at least in 2.2.x) you do this in build.properties.

@scabug
Copy link
Author

scabug commented Aug 4, 2014

fred aalbue (faalbue) said:
I started down the path of rebuilding on play 2.3 BUT some of my dependencies build with play 2.2 and therefore 2.10.2 scala libraries......This is a production application that has some tight deadlines and unfortunately, I do not have the bandwidth to recompile and retest all my dependent libraries. I suppose that the real answer is to bring all modules local and do local publish/build...which I may have to do in the future so this doesn't happen again, but for now, I just need the 2.10.2 to be available in the repo

@scabug
Copy link
Author

scabug commented Aug 4, 2014

fred aalbue (faalbue) said:
I already had sonatype as a resolver, which was failing. Investigating their repos, it seemed to have the same 404 error the maven repos had...again, only for 2.10.2 scala compiler...

However, I just ran my build and got passed the previous 2.10.2 issue and got it deployed to heroku. I still get a 404 at maven repo, so I don't really think it is fixed, but I just got lucky this time and found a mirror where the repo is available.

@scabug
Copy link
Author

scabug commented Aug 4, 2014

@gourlaysama said (edited on Aug 4, 2014 6:17:01 PM UTC):
Things are going back to normal: Travis is finally able to download scala-library and scala-compiler 2.10.2, see this build for example.

@scabug
Copy link
Author

scabug commented Aug 5, 2014

Pradeep Jain (pradeep.jain-at-gmail.com) said:
My build is running fine now. Thanks for resolving the issue. I hope this kind of disruptions are avoided.

@scabug
Copy link
Author

scabug commented Aug 5, 2014

Patrick Wendell (pwendell) said:
I believe this is working now. I tested my reproduction and it worked.

@scabug scabug closed this as completed Aug 5, 2014
@scabug
Copy link
Author

scabug commented Aug 6, 2014

@jsuereth said:
We've worked with Sonatype to help resolve the initial 404. However, Scala 2.10.2 is now hosted on a slower ibiblio CDN.

I'd recommend a few things:

  1. Use a repository proxy (artifactory/nexus) to avoid hitting maven central repeatedly
  2. If you're using Travis CI, look at trying to cache your ivy cache across builds. Also, vote up and highly encourage: Cache downloaded Maven artifacts travis-ci/travis-ci#1441
  3. If the slowdown is causing too many timeouts, use the solution posted above, or try hitting other alternative proxy of maven central, like jcenter.

Let us know if you're still seeing the original issue or if we can do anything else to help alleviate problems.

@scabug
Copy link
Author

scabug commented Aug 7, 2014

Mark Hibberd (markhibberd) said (edited on Aug 7, 2014 1:39:05 AM UTC):
As requested, I am adding some notes here, if anyone is looking to do more investigation. Not sure that it helps much given above.

We still see this on all our internal intermittently (approx 1 in 4 fail). I can't share internal builds details easily but as examples of what happens here are some of my argonaut builds that fail intermittently:

The exact scenario that we have is that the projects do not depend on 2.10.2 directly, but are using sbt versions/plugins that do. On travis, reducing the number of platforms helped a fair bit (i.e. just the parallel builds clashing with themselves seems to be enough to trigger it more regularly).

If there is anything we can do to help test changes / fixes let me know. This is particularly painful for open source projects who are largely at the mercy of maven central in terms of making it easy for people to clone and build.

@scabug
Copy link
Author

scabug commented Aug 7, 2014

@milessabin said:
+1 to Mark's comment from me. shapeless doesn't have any direct dependency on 2.10.2 (in fact it explicitly excludes 2.10.x versions prior to 2.10.3 for other reasons), but SBT appears to be pulling in 2.10.2 to satisfy plugins. This open SBT issue looks relevant: sbt/sbt#1439.

@scabug
Copy link
Author

scabug commented Aug 12, 2014

@paulp said:
None of the workarounds I found here or elsewhere was preventing build failures for me, which motivated me to figure out how to suppress the transitive dependency which led to the downloading of redundant 2.10.2 (not to mention 2.10.0 and probably others depending on the plugin selection) artifacts. In project/plugins.sbt:

 def plugin(m: ModuleID) = Defaults.sbtPluginExtra(m, "0.13", "2.10") excludeAll ExclusionRule("org.scala-lang")

 libraryDependencies ++= Seq(
   plugin("me.lessis"    % "bintray-sbt"     % "0.1.2"),
   plugin("com.typesafe" % "sbt-mima-plugin" % "0.1.6")
 )

@scabug
Copy link
Author

scabug commented Aug 12, 2014

Mark Hibberd (markhibberd) said:
Thanks Paul, I will give this a go as well. I can confirm that this is still causing us massive issues as well.

@scabug
Copy link
Author

scabug commented Aug 12, 2014

@paulp said:

If you're using Travis CI, look at trying to cache your ivy cache across builds.

Josh - is that a purely theoretical suggestion? I don't see any sign this is possible.

@scabug
Copy link
Author

scabug commented Aug 12, 2014

@jsuereth said:
Paul -

(a) can you post that code to sbt? I'll talk with Eugene at possibly just having that be the default for all sbt plugin dependencies in 0.13.6.
(b) As far as caching, Travis provides the ability, for paying customers, to cache a directory across build invocations. You could try to preserve the ivy cache directory across builds by zipping it up and synching it back down before starting the build. I've never tried it, as I'm not a paying customer.

Details here: http://docs.travis-ci.com/user/caching/

@scabug
Copy link
Author

scabug commented Aug 12, 2014

@paulp said:
Thanks, I saw that but misread it to be specific to the enumerated package managers and not a persist-any-directory sort of thing. In that case I'll probably sign up for that alone.

I did post that code in the comments to sbt/sbt#1439 , not sure what you mean by "to sbt" but that's in the neighborhood.

@scabug
Copy link
Author

scabug commented Aug 12, 2014

@jsuereth said:
Ah thanks, that's perfect. It was in an unexpected location, but good nonetheless. haven't gotten through all my ticket email from the past few hours.

@scabug
Copy link
Author

scabug commented Aug 12, 2014

@paulp said:
Geez, the lowest price point at travis is $129/month. I'm not scraping the bottom of the barrel cashwise but that still seems like a lot of money to mitigate the absence of caching.

@scabug
Copy link
Author

scabug commented Aug 12, 2014

@milessabin said:
Thanks Paul ... this works for me.

@scabug
Copy link
Author

scabug commented Aug 12, 2014

Samuel Halliday (fommil) said:
For anyone wanting caching on travis, check out how we do it in github.com/ensime/ensime-server

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants