Scala Programming Language
  1. Scala Programming Language
  2. SI-5580

Compiler Hangs and Runs Out of Memory for very small file

    Details

      Description

      When I try to compile the following program, the compiler runs out of memory after 2 minutes on my machine. Reproduced with all of the above versions. (2.9.0, 2.9.1, 2.10.0-M2). I attached a small test project, including a Gradle build script.

      import scala.collection.mutable.WeakHashMap
      class bar{ }
      class foo{
        val map = WeakHashMap[AnyRef, collection.mutable.Map[bar, collection.mutable.Set[bar]]]()
      
        def test={
          val tmp:bar=null
          if (map.get(tmp).isEmpty) map.put(tmp,collection.mutable.Set())
        }
      }
      
      1. gargantuan-type.txt
        278 kB
        Jason Zaugg

        Activity

        Hide
        Jason Zaugg added a comment - - edited

        There is a cavalcade of interesting issues illuminated by this snippet.

        Your code contains a type error – this change will make it correct: map.put(tmp,collection.mutable.Map()).

        When the compiler encounters this error, it triggers an implicit search to convert `map` to a type with a suitable `put` method. Alarmingly, this search includes `JavaConversions.asJavaDictionary`, despite the absense of an import! Why, you may ask? Well, WeakHashMap extends JavaConversions.JMapWrapper, and the implicits of a nested type include those defined in its enclosing scopes (object or package object). This behaviour isn't specified, see: SI-4427. I suspect that this consequence of putting JMapWrapper in JavaConversions was unintended.

        So, the compiler tries to use:

        collection.JavaConversions.asJavaDictionary(map).put(tmp, collection.mutable.Set())
        

        This line of code, when compiled verbatim fails as one expects. The compiler infers the type argument as:

        scala.collection.JavaConversions.asJavaDictionary[AnyRef, scala.collection.mutable.Map[Bar,scala.collection.mutable.Set[Bar]]]
        

        and fails with:

        error: polymorphic expression cannot be instantiated to expected type;
         found   : [A]scala.collection.mutable.Set[A]
         required: scala.collection.mutable.Map[Bar,scala.collection.mutable.Set[Bar]]
                        collection.JavaConversions.asJavaDictionary(map).put(tmp, collection.mutable.Set())
                                                
        

        But for implicit views, things proceed along a different path. The inferred type arguments are:

        collection.this.JavaConversions.asJavaDictionary[AnyRef, <284574 characters of inferred type elided>]
        

        No, really. I've attached said gargantuan-type.txt for posterity. Unsurprisingly, the second argument to put doesn't conform. That's when TypeDiagnostics takes over and brings down the compiler.

        TypeDiagnostics has the admirable intention to add just enough qualification of identifiers in type errors to make them unambiguous, while still maintaining readability. Processes each pair of types referenced in the type error, and adding qualification to those with the same name that refer to distinct types. This cartesian product is not processed lazily, and it exhausts the heap.

        It seems that this work is all for nought anyway – we just want to rule out ineligible implicit views, the carefully crafted message wouldn't make it to the user anyway.

        Show
        Jason Zaugg added a comment - - edited There is a cavalcade of interesting issues illuminated by this snippet. Your code contains a type error – this change will make it correct: map.put(tmp,collection.mutable.Map()) . When the compiler encounters this error, it triggers an implicit search to convert `map` to a type with a suitable `put` method. Alarmingly, this search includes `JavaConversions.asJavaDictionary`, despite the absense of an import! Why, you may ask? Well, WeakHashMap extends JavaConversions.JMapWrapper , and the implicits of a nested type include those defined in its enclosing scopes (object or package object). This behaviour isn't specified, see: SI-4427 . I suspect that this consequence of putting JMapWrapper in JavaConversions was unintended. So, the compiler tries to use: collection.JavaConversions.asJavaDictionary(map).put(tmp, collection.mutable.Set()) This line of code, when compiled verbatim fails as one expects. The compiler infers the type argument as: scala.collection.JavaConversions.asJavaDictionary[AnyRef, scala.collection.mutable.Map[Bar,scala.collection.mutable.Set[Bar]]] and fails with: error: polymorphic expression cannot be instantiated to expected type; found : [A]scala.collection.mutable.Set[A] required: scala.collection.mutable.Map[Bar,scala.collection.mutable.Set[Bar]] collection.JavaConversions.asJavaDictionary(map).put(tmp, collection.mutable.Set()) But for implicit views, things proceed along a different path. The inferred type arguments are: collection.this.JavaConversions.asJavaDictionary[AnyRef, <284574 characters of inferred type elided>] No, really. I've attached said gargantuan-type.txt for posterity. Unsurprisingly, the second argument to put doesn't conform. That's when TypeDiagnostics takes over and brings down the compiler. TypeDiagnostics has the admirable intention to add just enough qualification of identifiers in type errors to make them unambiguous, while still maintaining readability. Processes each pair of types referenced in the type error, and adding qualification to those with the same name that refer to distinct types. This cartesian product is not processed lazily, and it exhausts the heap. It seems that this work is all for nought anyway – we just want to rule out ineligible implicit views, the carefully crafted message wouldn't make it to the user anyway.
        Hide
        Paul Phillips added a comment -

        Oh, oh, aiya.

        Impressively, the code actually compiles as-is if I stop TypeDiagnostics from meddling.

        It's hard for "cavalcade" to be an understatement, but in this case...

        Show
        Paul Phillips added a comment - Oh, oh, aiya. Impressively, the code actually compiles as-is if I stop TypeDiagnostics from meddling. It's hard for "cavalcade" to be an understatement, but in this case...
        Hide
        Paul Phillips added a comment -

        Hey, you totally must have picked up "cavalcade" from my comment in JavaConversions. (Not that you learned the word, but that it was on your mind.)

          // Note to implementors: the cavalcade of deprecated methods herein should
          // serve as a warning to any who follow: don't overload implicit methods.
        
        Show
        Paul Phillips added a comment - Hey, you totally must have picked up "cavalcade" from my comment in JavaConversions. (Not that you learned the word, but that it was on your mind.) // Note to implementors: the cavalcade of deprecated methods herein should // serve as a warning to any who follow: don't overload implicit methods.
        Hide
        Jason Zaugg added a comment -

        I hadn't spotted that sage commentary – it seems that JavaConversions itself has the capacity to inspire specific language from those it crosses. Neat.

        I'm surprised it compiles once you removed the hurdle – which implicit saves the day?

        Show
        Jason Zaugg added a comment - I hadn't spotted that sage commentary – it seems that JavaConversions itself has the capacity to inspire specific language from those it crosses. Neat. I'm surprised it compiles once you removed the hurdle – which implicit saves the day?
        Hide
        Paul Phillips added a comment -

        I didn't note it; not sure. It did preserve the 285K type all the way until erasure.

        Show
        Paul Phillips added a comment - I didn't note it; not sure. It did preserve the 285K type all the way until erasure.
        Hide
        Paul Phillips added a comment -

        39938bcc29

        Show
        Paul Phillips added a comment - 39938bcc29
        Hide
        Paul Phillips added a comment -

        You can check for yourself now, as the version which compiles (after importing JavaConversions._) is test/files/pos/t5580b.scala .

        Show
        Paul Phillips added a comment - You can check for yourself now, as the version which compiles (after importing JavaConversions._) is test/files/pos/t5580b.scala .
        Hide
        Paul Phillips added a comment -

        Nice moment of the day - after rewriting a core typer method and having a test failure, I sighed and took a look to find the test referenced above, with my comment:

        /** It's a pos test because it does indeed compile,
         *  not so much because I'm glad it does.  Testing
         *  that error messages created and discarded during
         *  implicit search don't blow it up.
         */
        

        Fixed as side effect of beautiful rewrite of isSameType.

        test/files/pos/t5580b.scala:17: error: polymorphic expression cannot be instantiated to expected type;
         found   : [A]scala.collection.mutable.Set[A]
         required: scala.collection.mutable.Map[bar,scala.collection.mutable.Set[bar]]
            if (map.get(tmp).isEmpty) map.put(tmp,collection.mutable.Set())
                                                                        ^
        one error found
        
        Show
        Paul Phillips added a comment - Nice moment of the day - after rewriting a core typer method and having a test failure, I sighed and took a look to find the test referenced above, with my comment: /** It's a pos test because it does indeed compile, * not so much because I'm glad it does. Testing * that error messages created and discarded during * implicit search don't blow it up. */ Fixed as side effect of beautiful rewrite of isSameType. test/files/pos/t5580b.scala:17: error: polymorphic expression cannot be instantiated to expected type; found : [A]scala.collection.mutable.Set[A] required: scala.collection.mutable.Map[bar,scala.collection.mutable.Set[bar]] if (map.get(tmp).isEmpty) map.put(tmp,collection.mutable.Set()) ^ one error found

          People

          • Assignee:
            Paul Phillips
            Reporter:
            Ruedi Steinmann
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development