Imported From: https://issues.scala-lang.org/browse/SI-3623?orig=1
Reporter: @paulp
Attachments:

• bad.txt (created on Jun 29, 2010 7:03:59 PM UTC, 3415 bytes)
• good.txt (created on Jun 29, 2010 7:04:18 PM UTC, 6369 bytes)

Bram (bneijt) said:
I'm the person who was complaining about it on IRC. The build from my home directory failed with the attached output (bad.txt) while compiling from /tmp/s worked (good.txt).

I've notified the debian scala package maintainers as I fear this may be a problem on the debian package build hosts.

@milessabin said:
Path and path element limits on Linux are filesystem dependent. To the best of my knowledge ext3 and ext4 have a 255 byte limit for path elements, but there is no overall limit for path length.

@paulp said:
Replying to [comment:2 milessabin]:

Path and path element limits on Linux are filesystem dependent. To the best of my knowledge ext3 and ext4 have a 255 byte limit for path elements, but there is no overall limit for path length.

Can you reconcile this with the fact that he was able to build in "/tmp/s" but not in a longer named directory? Obviously the longest path element is the same in both cases. Only the path is shorter.

@paulp said:
Some googling finds this: http://code.google.com/p/flies/issues/detail?id=94

...which suggests the failure is due to using an encrypted filesystem. (Reporter? Confirm or deny?)

@paulp said:
The other interesting factoid is that the filename limit is 255 bytes - not chars - so multibyte encodings will hit it faster. Regardless of what is causing bneijt's issue, we have 164 character classnames right now, and that is close enough to the wall that this ought to be on the radar.

@milessabin said:
Replying to [comment:3 extempore]:

Can you reconcile this with the fact that he was able to build in "/tmp/s" but not in a longer named directory? Obviously the longest path element is the same in both cases. Only the path is shorter.

I'm afraid I can't reconcile it. But I know for sure that on the ext3 machine I'm typing this on I have plenty of paths which are longer than 255 bytes.

You're right to flag this up ... it's a problem on all platforms. The following has a useful table summarizing the various limits,

http://en.wikipedia.org/wiki/Comparison_of_file_systems

Bram (bneijt) said:
Hi everybody,

I've done some more digging and seems that this bug is invalid :D

The problem was my encrypted home directory (ecryptfs) has a problem with the filenames, not my system in itself.

I was able to succesfully build scala 2.8.0 RC7 from
/tmp/check_for_a_long_path_when_building_scala_or_is_it_my_cryptfs_home_dir_creating_the_problems?/scala-2.8.0.RC7-sources
and I had no problem there.

Output was comparable to the ones posted earlier. I think this bug can be considered invalid from my point of view, I have no problem with it anymore and neither should packaging scala 2.8.0 for Debian (which was my main concern here).

I'd be fine with this bug being considered invalid or an enhancement.

@paulp said:
Replying to [comment:7 bneijt]:

I've done some more digging and seems that this bug is invalid :D

The problem was my encrypted home directory (ecryptfs) has a problem with the filenames, not my system in itself.

I hope you didn't have to dig too far for that given that I identified that as the likely issue pretty clearly [http://lampsvn.epfl.ch/trac/scala/ticket/3623#comment:4 on this very page of comments].

I'm keeping it open because given the tendency of mangling to go places nobody wants or expects, it's an issue regardless.

@odersky said:
Paul, do you want to have this bug, or should we close it?

@paulp said:
Replying to [comment:11 odersky]:

Paul, do you want to have this bug, or should we close it?

I'll take it.

Thomas Lochmatter (thomas.lochmatter) said:
Replying to [comment:2 milessabin]:

Path and path element limits on Linux are filesystem dependent. To the best of my knowledge ext3 and ext4 have a 255 byte limit for path elements, but there is no overall limit for path length.

On Windows, the situation is worse: Though NTFS is able to handle file names up to 255 chars, the Windows API commonly uses a maximum path length of 260 chars. There are ways get around that limit, but even Windows 7 Explorer displays error messages when trying to delete, rename or copy files with very long file names, and so do many other Windows programs.

It would help us very much to have an option in the Scala compiler to generate short file names (i.e. < 40 chars) only.

Jim Balter (jibal) said:
Per http://en.wikipedia.org/wiki/Comparison_of_file_systems#cite_note-note-12-10 "Linux has a pathname limit of 4,096." This can be confirmed by examining /usr/src/linux/include/linux/limits.h: "#define PATH_MAX 4096 /* # chars in a path name including nul */"

@paulp said:
This is addressed to the extent we can do so in ticket scope in r24504.

Scott Emmons (scotte) said:
It's time to reopen this bug... As of ubuntu 12.04 (and it's derivatives), ecryptfs now properly returns max file name lengths (see https://bugs.launchpad.net/ecryptfs/+bug/344878/comments/83). Now that this advice is available to the scala compiler, it should use it to determine a valid mangled name for the fs.

@retronym said:
@scott: You could try this, which uses hashes rather than mangled FQN's above the provided limit:

$ scala -X | grep classfile
  -Xmax-classfile-name <n>       Maximum filename length for generated classes

Scott Emmons (scotte) said:
Thank you Jason! It's too bad your comment is the first on this particular bug documenting that option - I was not aware of it, even though "scalac -X" clearly lists it. Still, I think the compiler should default this from the fs, although for now this seems a decent workaround which I'll give a try and add to our build scripts. Thanks again!

Dimitar Georgiev (dgeorgiev) said:
Is this the correct bug to track the issue, that scalac fails on ecryptfs -stored projects?
Are there any plans to fix this?
Quite some developers (not on Mac) use Ubuntu these days, so I guess more people will stumble into this.
If it cannot be fixed, maybe the error message should be more friendly.

Regards, Dimitar

@szeiger said:
There's no way to query the fs for the maximum file or path length from Java NIO but you could check if getPath fails up to a sensible limit like 255 (and always cap it at no more than that):

import java.nio.file._

val fs = FileSystems.getDefault

def ok(i: Int) =
  try {
    fs.getPath("x" * i)
    true
  } catch {
    case _: InvalidPathException => false
  }

val max = (255 to 1 by -1).iterator.find(ok)
max match {
  case Some(n) => println("Use up to "+n)
  case None => println("Some problem")  
}

This may fail if a path is too long but I don't know if the default fs providers implement this feature. On OSX it will happily report that I can create a file name of length 10 million.

Erik LaBianca (easel) said (edited on Apr 7, 2016 3:20:25 PM UTC):
Jumping on this from #9742. This is a pain point for docker, and my understanding is that changing the value of -Xmax-classfile-name isn't a usable solution because it requires everything on the class path to have been built with the same setting. I'd like to propose changing the default value to something that's safe everywhere. A quick google tells me that for ecryptfs you need source filenames to stay below 140 to be safe under encryption, so unless theres a reason not to, I'd propose going even lower than that for the default, like 128. That ensures that some bizarre ecryptfs under docker on aufs combination would still work fine.

Merlin Göttlinger (mgttlinger) said (edited on Apr 7, 2016 3:23:04 PM UTC):
As @easel already said you hit this limitation fairly soon when using container virtualization. Especially when including shapeless in your dependencies as the generated names tend to get very long there.
In a modern CI environment it would be greatly beneficial if the build nodes could be container virtualized.

Erik LaBianca (easel) said:
I went ahead and created a PR with the change to 128 characters, assuming changing the default is all that's needed. scala/scala#5088

Mario Pastorelli (melrief) said:
Any progress on this?

For people interested in a workaround, if you have a directory that is not encrypted, say /tmp, you can put target <<= baseDirectory { file("/tmp/sbt/") / _.toString } in ~/.sbt/0.13/global.sbt and move all the target output to that directory.

This is what I needed to add to ~/.sbt/0.13/global.sbt as a workaround:

target := baseDirectory { file("/tmp/sbt/") / _.toString }.value

changing the value of -Xmax-classfile-name isn't a usable solution

agree, and therefore scala/scala#7497 proposes to drop support for that flag entirely

closing the ticket since I don't think there's anything else actionable here