Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

slow compilation on chained calls to polymorphic method with type alias in siganature (akka-streams Flow.map) #9157

Closed
scabug opened this issue Feb 17, 2015 · 19 comments
Assignees
Milestone

Comments

@scabug
Copy link

scabug commented Feb 17, 2015

trait FlowOps[+Out] {
  type Repr[+O] <: FlowOps[O]
}

trait Flow[-In, +Out] extends FlowOps[Out] {
  override type Repr[+O] <: Flow[In, O]
  def map[T](f: Out => T): Repr[T] /* workaround: expand alias Flow[In, T] */
}

class Test {
  def slowFlow {
    (null: Flow[String, String])
      .map(b => b)
      .map(b => b)
      .map(b => b)
      .map(b => b)
      .map(b => b)
      .map(b => b)
      .map(b => b)
      .map(b => b)
      .map(b => b)
      .map(b => b)
      .map(b => b)
      .map(b => b) // takes an age to compile
  }
}

As reported https://groups.google.com/d/msg/scala-internals/wdQVas4jNdc/vzLp2uOdITsJ

@scabug
Copy link
Author

scabug commented Feb 17, 2015

Imported From: https://issues.scala-lang.org/browse/SI-9157?orig=1
Reporter: @retronym
Affected Versions: 2.11.5

@scabug
Copy link
Author

scabug commented Feb 17, 2015

@retronym said (edited on Feb 17, 2015 2:31:09 PM UTC):
/cc [~viktorklang] @rkuhn [~bjorn.antonsson] It might be worth applying the workaround (alias expanion) to the Flow DSL until we can find a fix for this in the compiler.

@scabug
Copy link
Author

scabug commented Feb 17, 2015

@rkuhn said:
Wow, thanks for letting us know. I take it that the cause has not yet been identified and as such no estimates are available, right?

@scabug
Copy link
Author

scabug commented Feb 18, 2015

@retronym said:
Still digging.

Here's a smaller repro:

trait Flow[-In, +Out] {
  type Repr[+O] <: Flow[In, O]
  def map: Repr[String]
}

class Test {
  def slowFlow(f: Flow[String,String]#Repr[String]#Repr[String]#Repr[String]#Repr[String]#Repr[String]#Repr[String]#Repr[String]#Repr[String]#Repr[String]) = {
    f.map
  }
}

As @odersky noted on the mailing list, this typechecks promptly in dotty. We've must have some exponential performance bug in scalac.

@scabug
Copy link
Author

scabug commented Feb 18, 2015

@retronym said (edited on Feb 18, 2015 2:34:37 AM UTC):
Counting the calls to TypeRef#relativeInfo with an increasing number of type projections:

% qscalac sandbox/test.scala  | grep Repr | wc -l
      26
     326
    3336
   33446
  334556

@scabug
Copy link
Author

scabug commented Feb 18, 2015

Alexey Romanchuk (13h3r) said:
@retronym, I am glad you can built short example to reproduce the issue. If you need full sources please fell free to contact

@scabug
Copy link
Author

scabug commented Feb 18, 2015

@retronym said:
I think I have a fix. Let's see how it fares with our test suite: scala/scala#4341

@scabug
Copy link
Author

scabug commented Feb 18, 2015

@adriaanm said:
The Zaugg phenomenon strikes again. Fixed before estimated timeframe for fix could be established.

@scabug scabug closed this as completed Feb 18, 2015
@scabug
Copy link
Author

scabug commented Feb 18, 2015

@rkuhn said:
Indeed, chapeau!

@scabug
Copy link
Author

scabug commented Feb 19, 2015

@paulp said:
If it's the same bug I reported six months ago you can't collect the immediate fix bounty. Still no comment on that one: #8801

The preference of just-reported bugs over long neglected bugs creates all the wrong incentives. I should probably close my 142 open tickets and re-open them one per day over the next five months.

@scabug
Copy link
Author

scabug commented Feb 19, 2015

@adriaanm said:
I don't think it's fair to expect we unmask every single duplicate. Thanks for reporting first as well as pointing out our oversight, though.
This new report came from a project that we knew was blocked by this, so it got fixed.

What are these wrong incentives you speak of? Not to look for existing bugs when reporting a new one? We're just going to have to accept we won't be able to fix all the open bugs with the resources we have, let alone keep all the tickets in our heads to spot duplicates as readily as you do.

In the end, this is like a "jira chapter 11", a soft version of the bankruptcy idea that was floated on twitter not too long ago, except that declaring such "defeat" explicitly (which we have discussed internally multiple times) would probably be met with "ha! they imploded!" cheers if it came from us.

I do understand your frustration, and wish we could address it, but I really don't see the point of all this negativity given that we are already doing the best we can.

@scabug
Copy link
Author

scabug commented Feb 19, 2015

@paulp said:
If there were at least one responsive comment on each of my open tickets - signifying that six years of my life bought me at least a single trip through triage - I would be left with a lot less ammunition.

@scabug
Copy link
Author

scabug commented Feb 19, 2015

@paulp said:
And I don't know about "every single duplicate" but that one has as its subject "exponential compilation time" and the world's clearest minimization. Not to mention such an esteemed reporter. You'd think someone somewhere somehow could spot it before going through the entire analysis and minimization from a cold start on a parallel track, six months later. Or things are somewhere past bankrupt. And there are worse kinds of bankruptcy than issue bankruptcy.

@scabug
Copy link
Author

scabug commented Feb 19, 2015

@adriaanm said (edited on Feb 19, 2015 6:40:33 PM UTC):
Any thoughts on a jira query for tickets without comments other than from the reporter? I found only 5 open tickets for you where created == updated.

@scabug
Copy link
Author

scabug commented Feb 19, 2015

@paulp said:
My JIRA expertise remains about where it was, but I can tell you how many there are with brute force after dumping the list to html.

% ack --literal '&nbsp;<font color="#ffffff"><b>Description</b></font>&nbsp;' me.html  |wc
     141     282   10152
% ack --literal '&nbsp;<font color="#ffffff"><b>Comments</b></font>&nbsp;' me.html  |wc
     114     228    7866

So 27. Okay, with a variation on that I get the coarse look:

% ack -o '^\s+\[SI-\d+\]|<b>Comments</b>|<b>Description</b>' me.html

And I can never remember how to accomplish multi-line stuff via sed and friends so I just manually deleted it down to the ones without comments.

[SI-2936] [SI-3459] [SI-5172] [SI-5505] [SI-6533] [SI-6598] [SI-6717]
[SI-6786] [SI-6880] [SI-6929] [SI-6970] [SI-7152] [SI-7156] [SI-7175]
[SI-7243] [SI-7272] [SI-7423] [SI-7522] [SI-7697] [SI-7698] [SI-7730]
[SI-7735] [SI-7768] [SI-7884] [SI-8247] [SI-8801] [SI-8812] [SI-9094]

Must have one straggler in there since that's 28.

@scabug
Copy link
Author

scabug commented Feb 19, 2015

@paulp said:
This undercounts by the way because I have a habit of commenting on my own tickets not too long after opening them. Unfortunately those are likely to be the most interesting ones. Maybe I'll do another sweep for those.

@scabug
Copy link
Author

scabug commented Feb 19, 2015

@paulp said:
BTW this is also an opportunity to close a bunch of tickets, since that is the inevitable response to many of these. It's possible things are bankrupt enough that this has no value, but I mention it in case.

@scabug
Copy link
Author

scabug commented Feb 19, 2015

@retronym said (edited on Feb 19, 2015 10:18:19 PM UTC):
I've added your test from #8801: scala/scala#4347

Always happy to kill N birds with M stones where N > M, but I agree it would be better to avoid duplicate birds from taking flight in the first place. I do close a lot of new tickets as duplicates, but don't find the old ones, and sometimes forget to look. Thankfully in this case the minimization and fix were both surprisingly forthcoming.

I'll take a look through that list, too.

@scabug
Copy link
Author

scabug commented Feb 19, 2015

@paulp said:
In case anyone is actually considering commenting on my orphan tickets, I have completed an extremely depressing tour of them and here are the results:

In addition to the 28 entirely uncommented tickets noted above, these are "effectively uncommented" as their only comments are from me, from various third parties, and/or from Joe Jirabot with "Unassigning and rescheduling to M6 as previous deadline was missed" or some variation thereof.

#3197 #4323 #4381 #4401 #4663 #4721 #4739 #4867
#6282 #6304 #6529 #6593 #6617 #6767 #6883 #7411
#7538 #7721 #7765 #8529

Here are a couple where I feel the conversation was left unreasonably hanging.

#5962 #8039

Seeing all 140 of those tickets one after another, I had a glimpse of what it must have been like to be a civil war battlefield medic. Fighting for the South.

@scabug scabug added this to the 2.11.6 milestone Apr 7, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants