New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Option to failFast on zip and sequence #8994
Comments
Imported From: https://issues.scala-lang.org/browse/SI-8994?orig=1 |
@jedesah said: |
Oscar Boykin (oscar) said: https://github.com/twitter/scalding/pull/1412/files I'll try to make a PR to the scala repo to discuss mainlining this. |
I think this is an important issue. In addition to this, I’ve noticed that See this snippet: import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.{ Await, Future, blocking }
import scala.concurrent.duration.Duration
def failing() = Future.failed(new Exception("KO"))
def succeeding() = Future(blocking(Thread.sleep(5000)))
System.currentTimeMillis() / 1000 // : Long = 1603205724
Await.ready(failing().zip(succeeding()), Duration.Inf)
System.currentTimeMillis() / 1000 // : Long = 1603205724
Await.ready(succeeding().zip(failing()), Duration.Inf)
System.currentTimeMillis() / 1000 // : Long = 1603205729 Note that executing I think we should fix |
@johnynek do you want to contribute your fix? |
perhaps @viktorklang has a take on this |
@SethTisue @julienrf I definitely don't mind fail-fast behavior added—assuming that it can be done in a resource-efficient way which does not lead to performance degradations for existing code. |
In the code linked above, we use futures to wait on hadoop jobs finishing. In such cases, the real cost is running a job you know you don't need. This is not the usual case with futures in scala. My solution is fairly straight forward. I think you can improve the performance but my assumption is the linked code above will reduce performance of zip. |
I think this is out of the scope of
OK, then we need to run some benchmarks to see if the performance reduction would be significant or not. |
Note that we're talking about changing the implementation of |
Give the users of the library a choice; those who want 'the performance' can choose the current implementation. For other use cases that require a fail fast behaviour, use a new implementation ( I noticed |
Closed by scala/scala#9655 |
It should be possible to supply a flag to fail fast on both zip and sequence operations.
val a = Future{Thread.sleep(5000);1}
val b = Future{Thread.sleep(1000); throw new Exception("test")}
val c = a zip b
c will fail in 5 seconds with the exception from B.
Most clients would probably prefer it to fail in 1 second even if that means that they might get an exception from Future b instead of a possible exception from Future a.
The text was updated successfully, but these errors were encountered: