Scala Programming Language
  1. Scala Programming Language
  2. SI-4560

2.9.0+: Runtime exception with structural-type reflection cache and self-types

    Details

      Description

      === What steps will reproduce the problem? ===

      object Pimper {
       implicit def pimp(i: Int) = new {
          def test: String = i.toString
        }
      }
      
      class A
      
      trait B {
        self: A =>
      
        def test {
          import Pimper.pimp
      
          println(5.test)
        }
      }
      
      object Test extends A with B {
        def main(args: Array[String]) {
          test
        }
      }
      

      === What is the expected behavior? ===
      5

      === What do you see instead? ===

      A runtime error:

      java.lang.NoSuchFieldError: reflPoly$$Cache1
      	at T1$$class.reflMethod$$Method1(Test.scala:16)
      	at T1$$class.test(Test.scala:16)
      	at Test$$.test(Test.scala:20)
      	at Test$$.main(Test.scala:22)
      	at Test.main(Test.scala)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      	at java.lang.reflect.Method.invoke(Method.java:597)
      	at sbt.Run.invokeMain(Run.scala:69)
      	at sbt.Run.run0(Run.scala:60)
      	at sbt.Run.execute$$1(Run.scala:48)
      	at sbt.Run$$$$anonfun$$run$$2.apply(Run.scala:51)
      	at sbt.Run$$$$anonfun$$run$$2.apply(Run.scala:51)
      	at sbt.TrapExit$$.executeMain$$1(TrapExit.scala:33)
      	at sbt.TrapExit$$$$anon$$1.run(TrapExit.scala:42)
      

      === Additional information ===

      This happens since 2.9.0.RC2 (RC1 was fine) and happens only with the self-type in place. The problem seems to be that the reflection cache inside of `B$$class` looks for the cache field inside of class `B`:

      public static java.lang.reflect.Method reflMethod$$Method1(java.lang.Class);
        Code:
         Stack=5, Locals=2, Args_size=1
         0:   getstatic       SI-33; //Field B.reflPoly$$Cache1:Ljava/lang/ref/SoftReference;
      

      In another http://groups.google.com/group/spray-user/msg/59eb57b6cb2ce512 case for some reason the cache was looked for in a class declared as the self-type.

        Issue Links

          Activity

          Hide
          Johannes Rudolph added a comment -

          Here's a summary:

          we've got now 4 more or less different failure scenarios: the original three in this report (which are in the test file in the pull request) plus the one from the previous comment (I'm putting both previous ones into one category). Archaeological research showed the following:

          • In 2.8.0.Beta1-RC5 (that's the release before the tentative fix) none of the tests failed
          • In 2.9.0.RC1 still none of the tests failed
          • Between RC1 and 2.9.0.RC2 the three cases reported here originally started to fail
          • With the two workarounds (r24922 and r24968, which I figured are probably 124cf3f9c and 7127d829) only one test is failing which is the one where the self-type is the singleton type
          • With my proposed "fix" which reverts cb4fd6582, 124cf3f9c, and 7127d829, only the test from the previous comment fails (which is a scenario not observed before)

          My previous explanation that the tentative fix was the original commit introducing the regression is definitely wrong simply because it was already in the codebase long before the regression (actually I can't figure out any more from where I dug this commit out). It is a curious thing that the commit still is related otherwise reverting wouldn't have such an effect.

          I'm not so sure where to go from here: I wouldn't say the best solution is reverting a bunch of stuff if the outcome is at least one new previously unknown regression. In contrast, it isn't probably too hard to extend the current workaround to handle the last known remaining issue (self-type is singleton-type). The price is that we still have a workaround in the codebase for an issue for that no one seems to understand exactly where it comes from and which other variations may exist.

          Maybe I'll start a bisect between 2.9.0.RC1 and 2.9.0.RC2 to find out where it really showed up first.

          (In the meantime while scalac is compiling I read one of Alan Turing's articles, here's an excerpt about the "Learning Machine":

          An important feature of a learning machine is that its teacher will often be very largely ignorant of quite what is going on inside, although he may still be able to some extent to predict his pupil's behavior. This should apply most strongly to the later education of a machine arising from a child machine of well-tried design (or programme). This is in clear contrast with normal procedure when using a machine to do computations one's object is then to have a clear mental picture of the state of the machine at each moment in the computation. This object can only be achieved with a struggle. The view that "the machine can only do what we know how to order it to do,"' appears strange in face of this. Most of the programmes which we can put into the machine will result in its doing something that we cannot make sense.

          Sometimes I worry we are getting more and more into the area of trying to educate the compiler what to do instead of having "a clear mental picture of the state of the machine at each moment" but maybe that's just my ignorance)

          Show
          Johannes Rudolph added a comment - Here's a summary: we've got now 4 more or less different failure scenarios: the original three in this report (which are in the test file in the pull request) plus the one from the previous comment (I'm putting both previous ones into one category). Archaeological research showed the following: In 2.8.0.Beta1-RC5 (that's the release before the tentative fix) none of the tests failed In 2.9.0.RC1 still none of the tests failed Between RC1 and 2.9.0.RC2 the three cases reported here originally started to fail With the two workarounds (r24922 and r24968, which I figured are probably 124cf3f9c and 7127d829) only one test is failing which is the one where the self-type is the singleton type With my proposed "fix" which reverts cb4fd6582, 124cf3f9c, and 7127d829, only the test from the previous comment fails (which is a scenario not observed before) My previous explanation that the tentative fix was the original commit introducing the regression is definitely wrong simply because it was already in the codebase long before the regression (actually I can't figure out any more from where I dug this commit out). It is a curious thing that the commit still is related otherwise reverting wouldn't have such an effect. I'm not so sure where to go from here: I wouldn't say the best solution is reverting a bunch of stuff if the outcome is at least one new previously unknown regression. In contrast, it isn't probably too hard to extend the current workaround to handle the last known remaining issue (self-type is singleton-type). The price is that we still have a workaround in the codebase for an issue for that no one seems to understand exactly where it comes from and which other variations may exist. Maybe I'll start a bisect between 2.9.0.RC1 and 2.9.0.RC2 to find out where it really showed up first. (In the meantime while scalac is compiling I read one of Alan Turing's articles , here's an excerpt about the "Learning Machine": An important feature of a learning machine is that its teacher will often be very largely ignorant of quite what is going on inside, although he may still be able to some extent to predict his pupil's behavior. This should apply most strongly to the later education of a machine arising from a child machine of well-tried design (or programme). This is in clear contrast with normal procedure when using a machine to do computations one's object is then to have a clear mental picture of the state of the machine at each moment in the computation. This object can only be achieved with a struggle. The view that "the machine can only do what we know how to order it to do,"' appears strange in face of this. Most of the programmes which we can put into the machine will result in its doing something that we cannot make sense. Sometimes I worry we are getting more and more into the area of trying to educate the compiler what to do instead of having "a clear mental picture of the state of the machine at each moment" but maybe that's just my ignorance)
          Hide
          Johannes Rudolph added a comment -

          Seems like 8707c9ec introduced the regression between 2.9.0.RC1 and 2.9.0.RC2 (which btw I could only accurately identify because I had the old git repository lying around, in the new one it would have been this commit)

          I think the solution would then be to fix it properly at the place where the error was introduced (val hostClass = qualifier.tpe.typeSymbol.orElse(sym.owner) seems to naive to take self-types properly into account) and then try reverting the former two workaround commits.

          Show
          Johannes Rudolph added a comment - Seems like 8707c9ec introduced the regression between 2.9.0.RC1 and 2.9.0.RC2 (which btw I could only accurately identify because I had the old git repository lying around, in the new one it would have been this commit ) I think the solution would then be to fix it properly at the place where the error was introduced ( val hostClass = qualifier.tpe.typeSymbol.orElse(sym.owner) seems to naive to take self-types properly into account) and then try reverting the former two workaround commits.
          Hide
          Paul Phillips added a comment -

          After a few minutes of "educating the compiler" your test case outputs this for me:

          'Test
          Success 1
          'Test
          Success 2
          'Test
          Success 3
          

          Since you've been so helpful in this ticket already, if there are more cases not well covered by that one test it would be awesome if you could save me dredging around for more. I think I have this nailed but I'd feel a lot better with more tests.

          Show
          Paul Phillips added a comment - After a few minutes of "educating the compiler" your test case outputs this for me: 'Test Success 1 'Test Success 2 'Test Success 3 Since you've been so helpful in this ticket already, if there are more cases not well covered by that one test it would be awesome if you could save me dredging around for more. I think I have this nailed but I'd feel a lot better with more tests.
          Hide
          Paul Phillips added a comment -

          I also cobbled together this one

          
          object Outer {
            class Tester
            private[Outer] trait B4 { _: Tester =>
              protected val FREQ = 23
              def fail() = {
                println(FREQ)
              }
            }
            object C4 extends Tester with B4
          }
          
          object Outer2 {
            abstract class A5
            private[Outer2] trait C5 {
              def impl() { println("SUCCESS") }
            }
            trait B5 extends C5 { self: A5 =>
              def fail() { impl() }
            }
            object Test5 extends A5 with B5 with C5
          }
          
          object Test {
            def main(args: Array[String]): Unit = {
              Outer.C4.fail()
              Outer2.Test5.fail()
            }
          }
          

          Which prints 23 and SUCCESS as expected.

          Show
          Paul Phillips added a comment - I also cobbled together this one object Outer { class Tester private[Outer] trait B4 { _: Tester => protected val FREQ = 23 def fail() = { println(FREQ) } } object C4 extends Tester with B4 } object Outer2 { abstract class A5 private[Outer2] trait C5 { def impl() { println("SUCCESS") } } trait B5 extends C5 { self: A5 => def fail() { impl() } } object Test5 extends A5 with B5 with C5 } object Test { def main(args: Array[String]): Unit = { Outer.C4.fail() Outer2.Test5.fail() } } Which prints 23 and SUCCESS as expected.
          Show
          Paul Phillips added a comment - https://github.com/scala/scala/pull/988

            People

            • Assignee:
              Martin Odersky
              Reporter:
              Johannes Rudolph
              TracCC:
              Daniel Sobral, Ismael Juma, Paul Phillips, Seth Tisue
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development