Scala Programming Language
  1. Scala Programming Language
  2. SI-7362

Presentation compiler / typer crash on Scala-IDE unit test

    Details

      Description

      As shown by bisection, commit 9094822 introduced a regression in a presentation compiler test on the IDE side : T1207 checking for completion method plumbing (original issue : #1001207).

      Edit : there's a much simpler test case available below as a Scala PC test.

      • Context:

      The test tries to generate a completion on a Scala source containing a java package name (at the point containing an exclamation mark).

      • Problem:

      The failure manifests by returning an empty completion list (instead of the expected java package name):

       <testcase time="0.05" classname="scala.tools.eclipse.completion.CompletionTests" name="t1001207">
          <failure message="The completion should return java.util" type="java.lang.AssertionError">java.lang.AssertionError: The completion should return java.util
              at org.junit.Assert.fail(Assert.java:91)
              at org.junit.Assert.assertTrue(Assert.java:43)
              at scala.tools.eclipse.completion.CompletionTests$$anonfun$t1001207$1.apply(CompletionTests.scala:229)
              at scala.tools.eclipse.completion.CompletionTests$$anonfun$t1001207$1.apply(CompletionTests.scala:227)
              at scala.tools.eclipse.completion.CompletionTests$$anonfun$1$$anonfun$apply$1.apply$mcVI$sp(CompletionTests.scala:72)
      ...
      </testcase>
      

      The detailed output a problem in typers that makes the completion details appear coincidental :

         <system-out>
      !ENTRY org.scala-ide.sdt.core 4 0 2013-04-11 14:55:46.169
      !MESSAGE AT: RangePosition(/home/huitseeker/Scala/scala-ide/org.scala-ide.sdt.core.tests/target/work/data/completion/src/ticket_1001207/T1207.scala, 24, 31, 39)
      !STACK 0
      scala.reflect.internal.FatalError: unexpected tree: class scala.reflect.internal.Trees$Import
      import java.uti
              at scala.reflect.internal.SymbolTable.abort(SymbolTable.scala:51)
              at scala.tools.nsc.Global.abort(Global.scala:251)
              at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5261)
              at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5292)
              at scala.tools.nsc.typechecker.Typers$Typer.typedQualifier(Typers.scala:5370)
              at scala.tools.nsc.typechecker.Typers$Typer.typedQualifier(Typers.scala:5378)
              at scala.tools.nsc.interactive.Global.scala$tools$nsc$interactive$Global$$typeMembers(Global.scala:961)
              at scala.tools.nsc.interactive.Global$$anonfun$getTypeCompletion$2.apply(Global.scala:943)
              at scala.tools.nsc.interactive.Global$$anonfun$getTypeCompletion$2.apply(Global.scala:943)
              at scala.tools.nsc.interactive.Global.respondGradually(Global.scala:584)
              at scala.tools.nsc.interactive.Global.getTypeCompletion(Global.scala:943)
              at scala.tools.nsc.interactive.CompilerControl$AskTypeCompletionItem.apply$mcV$sp(CompilerControl.scala:359)
              at scala.tools.nsc.interactive.Global.pollForWork(Global.scala:384)
              at scala.tools.nsc.interactive.Global.checkForMoreWork(Global.scala:396)
              at scala.tools.nsc.interactive.Global.signalDone(Global.scala:249)
              at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5319)
              at scala.tools.nsc.typechecker.Typers$Typer.typedTypeConstructor(Typers.scala:5424)
              at scala.tools.nsc.typechecker.Typers$Typer.typedTypeConstructor(Typers.scala:5441)
              at scala.tools.nsc.typechecker.Typers$Typer.scala$tools$nsc$typechecker$Typers$Typer$$typedParentType(Typers.scala:1514)
              at scala.tools.nsc.typechecker.Typers$Typer.typedParentTypes(Typers.scala:1663)
              at scala.tools.nsc.typechecker.Typers$Typer$$anonfun$27.apply(Typers.scala:1791)
              at scala.tools.nsc.typechecker.Typers$Typer$$anonfun$27.apply(Typers.scala:1791)
              at scala.tools.nsc.typechecker.Typers$Typer.typerReportAnyContextErrors(Typers.scala:491)
              at scala.tools.nsc.typechecker.Typers$Typer.typedClassDef(Typers.scala:1790)
              at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5240)
              at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5292)
              at scala.tools.nsc.typechecker.Typers$Typer.scala$tools$nsc$typechecker$Typers$Typer$$typedStat$1(Typers.scala:2849)
              at scala.tools.nsc.typechecker.Typers$Typer$$anonfun$60.apply(Typers.scala:2953)
              at scala.tools.nsc.typechecker.Typers$Typer$$anonfun$60.apply(Typers.scala:2953)
              at scala.collection.immutable.List.loop$1(List.scala:170)
              at scala.collection.immutable.List.mapConserve(List.scala:186)
              at scala.tools.nsc.typechecker.Typers$Typer.typedStats(Typers.scala:2953)
              at scala.tools.nsc.typechecker.Typers$Typer.typedPackageDef$1(Typers.scala:4978)
              at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5244)
              at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5292)
              at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5351)
              at scala.tools.nsc.typechecker.Analyzer$typerFactory$$anon$3.apply(Analyzer.scala:99)
              at scala.tools.nsc.Global$GlobalPhase.applyPhase(Global.scala:413)
              at scala.tools.nsc.interactive.Global$TyperRun$$anonfun$applyPhase$1.apply$mcV$sp(Global.scala:1102)
              at scala.tools.nsc.interactive.Global$TyperRun$$anonfun$applyPhase$1.apply(Global.scala:1102)
              at scala.tools.nsc.interactive.Global$TyperRun$$anonfun$applyPhase$1.apply(Global.scala:1102)
              at scala.reflect.internal.SymbolTable.enteringPhase(SymbolTable.scala:214)
              at scala.tools.nsc.interactive.Global$TyperRun.applyPhase(Global.scala:1102)
              at scala.tools.nsc.interactive.Global$TyperRun.typeCheck(Global.scala:1095)
              at scala.tools.nsc.interactive.Global.scala$tools$nsc$interactive$Global$$typeCheck(Global.scala:546)
              at scala.tools.nsc.interactive.Global$$anonfun$backgroundCompile$5$$anonfun$apply$7.apply(Global.scala:462)
              at scala.tools.nsc.interactive.Global$$anonfun$backgroundCompile$5$$anonfun$apply$7.apply(Global.scala:458)
              at scala.Option.foreach(Option.scala:245)
              at scala.tools.nsc.interactive.Global$$anonfun$backgroundCompile$5.apply(Global.scala:458)
              at scala.tools.nsc.interactive.Global$$anonfun$backgroundCompile$5.apply(Global.scala:458)
              at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:775)
              at scala.collection.immutable.List.foreach(List.scala:300)
              at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:774)
              at scala.tools.nsc.interactive.Global.backgroundCompile(Global.scala:458)
              at scala.tools.nsc.interactive.PresentationCompilerThread.run(PresentationCompilerThread.scala:25)
      </system-out>
      
      • How to repdroduce:

      Compile the scala commit (9094822 or posterior) on which you have cherry-picked an sbt version update. Compile sbt 0.13 off of that, and a 2.11-compatible version of Scala-IDE on top of the previous two.

      As far as 2-11-compatible means : I'll keep the minimal tweaks needed for Scala-IDE compilation off of 2.11 on my branch build-script-tweaks.

      If this whole process sounds too complicated, you may want to try my bash script to do that (don't mind the bisect name). It just assumes scala,sbinary,sbt,scala-refactoring, scala-ide checkouts present on your system (directory names configured at the beginning of the script), and does the whole scala -> sbinary -> sbt -> scala-refactoring -> scala-ide compile based on a non-polluting use of your maven local repo and whatever you have present in said directories. Ping me if you need help using it.

      1. compilation-2013-04-10-213820-64cfaf7.log
        4.77 MB
        François Garillot
      2. TEST-scala.tools.eclipse.TestsSuite.xml
        107 kB
        François Garillot

        Activity

        Hide
        François Garillot added a comment - - edited

        @extempore : I have bisections log aplenty, if you're interested.
        I'm of course still working on that one, and will minimize/test-case/update as I go along.

        Show
        François Garillot added a comment - - edited @extempore : I have bisections log aplenty, if you're interested. I'm of course still working on that one, and will minimize/test-case/update as I go along.
        Hide
        Iulian Dragos added a comment -

        I wonder if this failure can be reproduced in a self-contained test that only calls askTypeCompletion at the right place, similar to test/files/presentation/t5708/src/Completions.scala

        Show
        Iulian Dragos added a comment - I wonder if this failure can be reproduced in a self-contained test that only calls askTypeCompletion at the right place, similar to test/files/presentation/t5708/src/Completions.scala
        Hide
        François Garillot added a comment -

        @Iulian : Yes it does !
        I've added a PC test that exercises exactly the problematic behavior in files/test/presentation/t1207(.check|/), in the branch investigation/SI-7362 here. I'ts created on top of 9094822181c398b945b7f30ac1e2b05da9796f53, with an update of sbt on top.

        Show
        François Garillot added a comment - @Iulian : Yes it does ! I've added a PC test that exercises exactly the problematic behavior in files/test/presentation/t1207(.check|/), in the branch investigation/ SI-7362 here . I'ts created on top of 9094822181c398b945b7f30ac1e2b05da9796f53, with an update of sbt on top.
        Hide
        Paul Phillips added a comment -

        According to me, this was caused by a commit which far predates 9094822. Still me of course.

        https://github.com/scala/scala/commit/a2c870569c124

        I freely admit I don't have a clue how the setError etc. code is supposed to work, which is a big problem. That patch was to fix SI-6340, which was also a big problem.

        If I revert that one-line change, your files/test/presentation/t1207 test goes from crashing to producing this:

        reload: Completions.scala
        
        askTypeCompletion at Completions.scala(3,16)
        tree = import java.uti
        ================================================================================
        [response] aksTypeCompletion at (3,16)
        retrieved 0 members
        
        ================================================================================
        

        which I assume is the correct output, the checkfile is empty.

        Show
        Paul Phillips added a comment - According to me, this was caused by a commit which far predates 9094822. Still me of course. https://github.com/scala/scala/commit/a2c870569c124 I freely admit I don't have a clue how the setError etc. code is supposed to work, which is a big problem. That patch was to fix SI-6340 , which was also a big problem. If I revert that one-line change, your files/test/presentation/t1207 test goes from crashing to producing this: reload: Completions.scala askTypeCompletion at Completions.scala(3,16) tree = import java.uti ================================================================================ [response] aksTypeCompletion at (3,16) retrieved 0 members ================================================================================ which I assume is the correct output, the checkfile is empty.
        Hide
        Paul Phillips added a comment -

        As another view on it, here is the diff for

        % scalac -Xprint-pos -Xprint:all -Yshow-trees -d /tmp test/files/presentation/t1207/src/Completions.scala
        

        before and after reverting a2c870569c124c.

         @@ -130,7 +130,12 @@ import java.uti /*!*/
          [[syntax trees at end of typer]]// Scala source: Completions.scala
          PackageDef(
            [8]"ticket_1001207" // final package ticket_1001207, tree.tpe=ticket_1001207.type
         -  <empty>
         +  [31]Import( // val <import>: ImportType([31]java)
         +    [31]"java" // final package java, tree.tpe=java.type
         +    List(
         +      ImportSelector(uti,36,uti,36)
         +    )
         +  )
            [53]ClassDef( // class T1207 extends AnyRef in package ticket_1001207
              0
              "T1207"
        
        Show
        Paul Phillips added a comment - As another view on it, here is the diff for % scalac -Xprint-pos -Xprint:all -Yshow-trees -d /tmp test/files/presentation/t1207/src/Completions.scala before and after reverting a2c870569c124c. @@ -130,7 +130,12 @@ import java.uti /*!*/ [[syntax trees at end of typer]]// Scala source: Completions.scala PackageDef( [8]"ticket_1001207" // final package ticket_1001207, tree.tpe=ticket_1001207.type - <empty> + [31]Import( // val <import>: ImportType([31]java) + [31]"java" // final package java, tree.tpe=java.type + List( + ImportSelector(uti,36,uti,36) + ) + ) [53]ClassDef( // class T1207 extends AnyRef in package ticket_1001207 0 "T1207"
        Hide
        Jason Zaugg added a comment -

        I think a2c870569c124c is something of a red herring. The test also crashes with {{import java.util /!/ }}.

        I believe something like the following is needed.

        diff --git a/src/interactive/scala/tools/nsc/interactive/Global.scala b/src/interactive/scala/tools/nsc/interactive/Global.scala
        index dbdb2d0..a5e6d47 100644
        --- a/src/interactive/scala/tools/nsc/interactive/Global.scala
        +++ b/src/interactive/scala/tools/nsc/interactive/Global.scala
        @@ -1036,10 +1036,12 @@ class Global(settings: Settings, _reporter: Reporter, projectName: String = "")
           private def typeMembers(pos: Position): Stream[List[TypeMember]] = {
             var tree = typedTreeAt(pos)
        
        -    // if tree consists of just x. or x.fo where fo is not yet a full member name
        -    // ignore the selection and look in just x.
             tree match {
        +      // if tree consists of just x. or x.fo where fo is not yet a full member name
        +      // ignore the selection and look in just x.
               case Select(qual, name) if tree.tpe == ErrorType => tree = qual
        +      // Similarly, for import x. or x.fo
        +      case Import(expr, sels) => tree = expr
               case _ =>
             }
        

        What are the expected results from these cases?

        import java./*!*/
        import java.u/*!*/
        import java.util/*!*/
        import java.{lang, u/*!*/}
        import j/*!*/
        

        With the patch, for the first four, we get:

        askTypeCompletion at Completions.scala(3,17)
        ================================================================================
        [response] aksTypeCompletion at (3,17)
        retrieved 13 members
        [accessible:  true] `package appletjava.applet.type`
        [accessible:  true] `package awtjava.awt.type`
        [accessible:  true] `package beansjava.beans.type`
        [accessible:  true] `package iojava.io.type`
        [accessible:  true] `package langtype`
        [accessible:  true] `package mathjava.math.type`
        [accessible:  true] `package netjava.net.type`
        [accessible:  true] `package niojava.nio.type`
        [accessible:  true] `package rmijava.rmi.type`
        [accessible:  true] `package securityjava.security.type`
        [accessible:  true] `package sqljava.sql.type`
        [accessible:  true] `package textjava.text.type`
        [accessible:  true] `package utiljava.util.type`
        ================================================================================
        

        The last one is perhaps a separate problem.

        Show
        Jason Zaugg added a comment - I think a2c870569c124c is something of a red herring. The test also crashes with {{import java.util / ! / }}. I believe something like the following is needed. diff --git a/src/interactive/scala/tools/nsc/interactive/Global.scala b/src/interactive/scala/tools/nsc/interactive/Global.scala index dbdb2d0..a5e6d47 100644 --- a/src/interactive/scala/tools/nsc/interactive/Global.scala +++ b/src/interactive/scala/tools/nsc/interactive/Global.scala @@ -1036,10 +1036,12 @@ class Global(settings: Settings, _reporter: Reporter, projectName: String = "") private def typeMembers(pos: Position): Stream[List[TypeMember]] = { var tree = typedTreeAt(pos) - // if tree consists of just x. or x.fo where fo is not yet a full member name - // ignore the selection and look in just x. tree match { + // if tree consists of just x. or x.fo where fo is not yet a full member name + // ignore the selection and look in just x. case Select(qual, name) if tree.tpe == ErrorType => tree = qual + // Similarly, for import x. or x.fo + case Import(expr, sels) => tree = expr case _ => } What are the expected results from these cases? import java./*!*/ import java.u/*!*/ import java.util/*!*/ import java.{lang, u/*!*/} import j/*!*/ With the patch, for the first four, we get: askTypeCompletion at Completions.scala(3,17) ================================================================================ [response] aksTypeCompletion at (3,17) retrieved 13 members [accessible: true] `package appletjava.applet.type` [accessible: true] `package awtjava.awt.type` [accessible: true] `package beansjava.beans.type` [accessible: true] `package iojava.io.type` [accessible: true] `package langtype` [accessible: true] `package mathjava.math.type` [accessible: true] `package netjava.net.type` [accessible: true] `package niojava.nio.type` [accessible: true] `package rmijava.rmi.type` [accessible: true] `package securityjava.security.type` [accessible: true] `package sqljava.sql.type` [accessible: true] `package textjava.text.type` [accessible: true] `package utiljava.util.type` ================================================================================ The last one is perhaps a separate problem.
        Hide
        Paul Phillips added a comment -

        You also get "ERROR: Right(scala.tools.nsc.typechecker.DivergentImplicit$)" I take it.

        Show
        Paul Phillips added a comment - You also get "ERROR: Right(scala.tools.nsc.typechecker.DivergentImplicit$)" I take it.
        Show
        Paul Phillips added a comment - https://github.com/scala/scala/pull/2406
        Hide
        Jason Zaugg added a comment -

        Closing as Paul's PR was merged.

        Show
        Jason Zaugg added a comment - Closing as Paul's PR was merged.

          People

          • Assignee:
            Paul Phillips
            Reporter:
            François Garillot
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development