The behavior of Either.filterOrElse is confusing when the predicate doesn't match; it converts Either[A, B] to Either[AA, B] where AA is the "zero" element provided to filterOrElse. This is unexpected because for everything else, a filter operation results in a possible reduction of the number of elements in the container, and doesn't actually change the value of the elements. What I'd expect is to get an Option[Right] = None if the Either is a right and the predicate doesn't match and a Option[Right] = Some(a) otherwise. If it's a Left, the existing behavior of no-op is expected.
This causes subtle bugs in the code like the following:

def brokers(throw1: () => List[Int], throw2: List[Int] => List[String]) = {
  println("brokers ===> " +
    (Try(throw1())
      .toEither
      .filterOrElse(!_.isEmpty, Nil)
      .flatMap(xs => Try(throw2(xs)).toEither) match {
        case Right(s) => s
        case Left(f) => throw f.asInstanceOf[Throwable]
      })
    )
}

Call:
brokers(() => Nil, (xs: List[Int]) => throw new RuntimeException("throw2"))

Exception in thread "main" java.lang.ClassCastException: scala.collection.immutable.Nil$ cannot be cast to java.lang.Throwable at Practice$.brokers(Practice.scala:57) at Practice$.delayedEndpoint$Practice$1(Practice.scala:63)

First of all, the Throwable is lost causing a compile error (shown by the cast).
It converted the Either[Throwable,List[Int]] = Right(List()) to an Either[java.io.Serializable,List[Int]] = Left(List()). Thus the compile error, and the match with case Left and eventual exception. The Serializable must have come from the contravariant type parameter AA >: A, because the first common supertype for Throwable and List is a Serializable.