New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Abstracting over sequential and parallel sequences broken #4843
Comments
Imported From: https://issues.scala-lang.org/browse/SI-4843?orig=1 |
@paulp said:
|
@hseeberger said: scala> def numbers(isPar: Boolean) = if (isPar) (1 to 1000).par else 1 to 1000 Second: How can the collections know whether the load is high or not? Even for a small number like 8 or even 3 it could make sense to perform operations in parallel. Example using a collection of functions making web service requests: object Weather { def main(args: Array[String]) { def currentTemperature(location: String): Option[Int] = { |
@hseeberger said: |
@paulp said: b) "How can it know whether the load is high or not" is a question about the design, not a bug. Please give me a chance to have something else in my life besides this bug database by reserving the bug database for bugs. |
@jrudolph said: Welcome to Scala version 2.9.1.RC1 (Java HotSpot(TM) 64-Bit Server VM, Java 1.6.0_26).
Type in expressions to have them evaluated.
Type :help for more information.
scala> def numbers(isPar: Boolean): scala.collection.GenSeq[Int] = if (isPar) (1 to 256).par else 1 to 256
numbers: (isPar: Boolean)scala.collection.GenSeq[Int]
scala> numbers(true) map { _ => Thread.currentThread.getName } groupBy (x => x) map (x => (x._1, x._2.length)) foreach println
(Thread-9,256) Aleksandar fixed that in r25235 but it was apparently not yet chosen for 2.9.1 probably also because I didn't report the bug earlier. |
@hseeberger said: (a) Sorry, but the code you posted doesn't even compile under 2.9.1.RC1. And as Johannes showed, making it compile still doesn't make it work. (b) From a user perspective, if I call someCollection.par I want it to be parallel. Therefore I consider it a bug, but I am willing to take it to the scala-internal list. Thanks, Heiko |
@paulp said: |
@jrudolph said: Put a definition like at a place where it can be seen from the call-sites of implicit def possiblyParallelCanBuildFrom[Elem]: CanBuildFrom[GenSeq.Coll, Elem, GenSeq[Elem]] =
if (isPar)
ParSeq.canBuildFrom[Elem].asInstanceOf[CanBuildFrom[GenSeq.Coll, Elem, GenSeq[Elem]]]
else
GenSeq.canBuildFrom[Elem] You can, of course, only use it if you manage to pass isPar around, as well. Overall not very usable in the end but it did work for us in one place because we configured isPar globally anyways. |
@hseeberger said: |
@jrudolph said: |
Omid Aladini (omid) said: |
@jrudolph said: In 2.10 the behavior changed to not rely on static type information. I don't know the exact commit of the fix but here's a former discussion on the mailing list from the time the issue was fixed: https://groups.google.com/d/topic/scala-language/5KZAZXnWS1I/discussion |
Omid Aladini (omid) said: |
The below REPL examples shows the observed behavior under Scala 2.9.0.1 and 2.9.1.RC1. Under 2.9.0.1 there seem to be two bugs, one regarding type inference and the other regarding abstracting over sequential and parallel Seqs (that might also apply to other collections). Under 2.9.1.RC1 only the later occurs, yet the type inferencer seems to use a structural type instead of GenSeq.
The real issue from a user's perspective is, that even if a parallel Seq is returned by the numbers-method, map won't be executed in parallel, i.e. abstracting over sequential and parallel collections is broken.
===============================================================================
Scala 2.9.0.1
Welcome to Scala version 2.9.0.1 (Java HotSpot(TM) 64-Bit Server VM, Java 1.6.0_26).
Type in expressions to have them evaluated.
Type :help for more information.
scala> def numbers(isPar: Boolean) =
| if (isPar) (1 to 8).par else 1 to 8
:7: error: kinds of the type arguments (scala.collection.GenSeq[Any]{def seq: scala.collection.immutable.Seq[Any]}) do not conform to the expected kinds of the type parameters (type CC) in class GenericCompanion.
scala.collection.GenSeq[Any]{def seq: scala.collection.immutable.Seq[Any]}'s type parameters do not match type CC's expected parameters: has no type parameters, but type CC has one
def numbers(isPar: Boolean) =
^
scala> def numbers(isPar: Boolean): scala.collection.GenSeq[Int] =
| if (isPar) (1 to 8).par else 1 to 8
numbers: (isPar: Boolean)scala.collection.GenSeq[Int]
scala> numbers(true) map { _ => Thread.currentThread.getName }
res0: scala.collection.GenSeq[java.lang.String] = ParVector(Thread-5, Thread-5, Thread-5, Thread-5, Thread-5, Thread-5, Thread-5, Thread-5)
===============================================================================
Scala 2.9.1.RC1
Welcome to Scala version 2.9.1.RC1 (Java HotSpot(TM) 64-Bit Server VM, Java 1.6.0_26).
Type in expressions to have them evaluated.
Type :help for more information.
scala> def numbers(isPar: Boolean) =
| if (isPar) (1 to 8).par else 1 to 8
numbers: (isPar: Boolean)scala.collection.GenSeq[Int] with Serializable{def companion: scala.collection.generic.GenericCompanion[scala.collection.GenSeq{def seq: scala.collection.immutable.Seq[Any]}]; def par: scala.collection.parallel.immutable.ParSeq[Int]; def seq: scala.collection.immutable.Seq[Int]; def distinct: scala.collection.GenSeq[Int]{def seq: scala.collection.immutable.Seq[Int]; def distinct: scala.collection.GenSeq[Int]{def seq: scala.collection.immutable.Seq[Int]; protected def parCombiner: scala.collection.parallel.Combiner[Int,scala.collection.parallel.immutable.ParSeq[Int]]}; def intersect[U >: Int](that: scala.collection.GenSeq[U]): scala.collection.GenSeq[Int]{def seq: scala.collection.immutable.Seq[Int]; protected def parCombiner: scala.collection.parallel.Combiner[I...
scala> numbers(true) map { _ => Thread.currentThread.getName }
res0: scala.collection.GenSeq[java.lang.String] = ParVector(Thread-5, Thread-5, Thread-5, Thread-5, Thread-5, Thread-5, Thread-5, Thread-5)
scala> def numbers(isPar: Boolean): scala.collection.GenSeq[Int] =
| if (isPar) (1 to 8).par else 1 to 8
numbers: (isPar: Boolean)scala.collection.GenSeq[Int]
scala> numbers(true) map { _ => Thread.currentThread.getName }
res1: scala.collection.GenSeq[java.lang.String] = ParVector(Thread-7, Thread-7, Thread-7, Thread-7, Thread-7, Thread-7, Thread-7, Thread-7)
The text was updated successfully, but these errors were encountered: