invalid bytecode when using range inside the map function of an Option #6720

scabug · 2012-11-26T15:34:25Z

When the following function is called:

object ScalaBug {
  def bug() = {
    val optA: Option[String] = Some("A")
    val optB: Option[Int] = Some(2)

    optA.map(concept => {
      var foo = "A"

      optB match {
        case Some(max) => (max to 1 by -1).foreach { _ => foo = "B" }
        case None => foo = "C"
      }

      foo
    })
  }
}

It result in this runtime exception:
java.lang.VerifyError: Uninitialized object exists on backward branch 92 in method com.timeout.service.test.ScalaBug$.bug()Lscala/Option; at offset 114

scabug · 2012-11-26T15:34:25Z

Imported From: https://issues.scala-lang.org/browse/SI-6720?orig=1
Reporter: Alois Cochard (aloiscochard)
Affected Versions: 2.10.0-RC2

scabug · 2012-11-26T15:40:56Z

Alois Cochard (aloiscochard) said:
It's a regression, working on 2.9.2

scabug · 2012-11-29T08:48:24Z

@paulp said:
Cannot reproduce with RC2 or current trunk. Most likely either you are omitting one or more command line options, or this is somehow a consequence of your environment (e.g. you are mixing bytecode from different scala versions.)

scabug · 2012-11-29T17:13:03Z

Alois Cochard (aloiscochard) said:
After more investigation it happen only if there is BOTH "-optimize" and "-target:jvm-1.7" flags.

FYI, I just tried with RC3 and the bug is still there.

scabug · 2012-11-29T21:50:33Z

@paulp said:
Those little details matter! I did try -optimise before closing it; trying -optimise and -target:jvm-1.7 is a little beyond my wild stab radius.

Interestingly the bytecode generated under -target:jvm-1.6 and -target:jvm-1.7 is bit-for-bit identical except for the minor version number under 1.7 being 51. This suggests the vm in java7 is being less forgiving about something we were getting away with in the past.

scabug · 2012-11-30T08:39:09Z

@magarciaEPFL said:
I'll look into it today. Main suspect: the verification type uninitializedThis used for stack-map-frames (this time not in a constructor, apparently).

scabug · 2012-11-30T19:42:40Z

@magarciaEPFL said:
Minimized:

object Test {

  def main(args: Array[String]) {  }
  
  def problematic(r: Range) {
    Some( r foreach null )
  }

}

scabug · 2012-11-30T19:59:25Z

@magarciaEPFL said (edited on Dec 1, 2012 2:52:17 PM UTC):
Pretty-fying somewhat the ICode (right after inliner) for the above:

  def problematic(r: collection.immutable.Range (REF(class Range))): Unit {
  locals: value r, variable loc41, variable loc31, variable loc21
  startBlock: 1
  blocks: [1,4,5,6,7]
  
  1: 
     	NEW REF(class Some)
     	DUP(REF(class Some))
     	(r.validateRangeBoundaries(null) == 0) ? JUMP 4 : JUMP 7
    
  7: 
     	loc21 = r.start
     	loc31 = r.terminalElement
     	loc41 = r.step
    
  5: 
     	(loc21 == loc31) ? JUMP 4 : JUMP 6
    
  4: 
     	LOAD_FIELD scala.runtime.BoxedUnit.UNIT
     	CALL_METHOD scala.Some.<init> (static-instance)
     	DROP REF(class Some)
     	RETURN(UNIT)
    
  6: 
     	CONSTANT(null)
     	LOAD_LOCAL(variable loc21)
     	BOX INT
     	CALL_METHOD scala.Function1.apply (dynamic)
     	DROP INT
        loc21 = loc21 + loc41
     	JUMP 5
    
  }

scabug · 2012-12-01T14:51:41Z

@magarciaEPFL said:
Minimized further, showing that -optimise isn't required:

object Test {

  def main(args: Array[String]) { 

    Some( while(this != null) {} )

  }

}

Error message:

java.lang.VerifyError: Uninitialized object exists on backward branch 4 in method Test$.main([Ljava/lang/String;)V at offset 5

Output produced by ASM's Textifier -debug

  public main([Ljava/lang/String;)V
   L0
    NEW scala/Some
    DUP
   L1
   FRAME FULL [Test$ [Ljava/lang/String;] [L0 L0]
    ALOAD 0
    IFNONNULL L1
    GETSTATIC scala/runtime/BoxedUnit.UNIT : Lscala/runtime/BoxedUnit;
    INVOKESPECIAL scala/Some.<init> (Ljava/lang/Object;)V
    POP
    RETURN
   L2
    LOCALVARIABLE this LTest$; L0 L2 0
    LOCALVARIABLE args [Ljava/lang/String; L0 L2 1
    MAXSTACK = 3
    MAXLOCALS = 2

Verbose output by javap

  public void main(java.lang.String[]);
    flags: ACC_PUBLIC
    Code:
      stack=3, locals=2, args_size=2
         0: new           #16                 // class scala/Some
         3: dup           
         4: aload_0       
         5: ifnonnull     4
         8: getstatic     #22                 // Field scala/runtime/BoxedUnit.UNIT:Lscala/runtime/BoxedUnit;
        11: invokespecial #25                 // Method scala/Some."<init>":(Ljava/lang/Object;)V
        14: pop           
        15: return        
      LocalVariableTable:
        Start  Length  Slot  Name   Signature
               0      16     0  this   LTest$;
               0      16     1  args   [Ljava/lang/String;
      LineNumberTable:
        line 6: 0
      StackMapTable: number_of_entries = 1
           frame_type = 255 /* full_frame */
          offset_delta = 4
          locals = [ class Test$, class "[Ljava/lang/String;" ]
          stack = [ uninitialized 0, uninitialized 0 ]

scabug · 2012-12-01T17:49:30Z

@magarciaEPFL said:
Quoting from the JVM Spec, 4.9.2 Structural Constraints , http://docs.oracle.com/javase/specs/jvms/se7/html/jvms-4.html

- There must never be an uninitialized class instance on the operand stack or in a local variable at the target of a backwards branch unless the special type of the uninitialized class instance at the branch instruction is merged with itself at the target of the branch (§4.10.2.4).

- There must never be an uninitialized class instance in a local variable in code protected by an exception handler (§4.10.2.4).

- There must never be an uninitialized class instance on the operand stack or in a local variable when a jsr or jsr_w instruction is executed.

The first constraint applies in our case, and in fact appears the bytecode should be accepted. There's been recent work on HotSpot to make VerifyErrors more informative, perhaps an unintended result of that work was checking a more stringent condition than required by the spec.

> java -version
java version "1.7.0_03"
Java(TM) SE Runtime Environment (build 1.7.0_03-b04)
Java HotSpot(TM) 64-Bit Server VM (build 22.1-b02, mixed mode)

scabug · 2012-12-02T13:04:15Z

@magarciaEPFL said:
The verifier in two other VMs also complains: in the Graal VM (same error as in HotSpot JDK7) and also in HotSpot JDK8:

java version "1.8.0-ea"
Java(TM) SE Runtime Environment (build 1.8.0-ea-b65)
Java HotSpot(TM) 64-Bit Server VM (build 25.0-b09, mixed mode)

Exception in thread "main" java.lang.VerifyError: Uninitialized object exists on backward branch 4
Exception Details:
  Location:
    Test$.main([Ljava/lang/String;)V @5: ifnonnull
  Reason:
    Error exists in the bytecode
  Bytecode:
    0000000: bb00 1059 2ac7 ffff b200 16b7 0019 57b1
    0000010:                                        
  Stackmap Table:
    full_frame(@4,{Object[#2],Object[#29]},{Uninitialized[#0],Uninitialized[#0]})

scabug · 2013-01-18T10:03:42Z

@magarciaEPFL said:

The Oracle JVM as of JDK 7 has started rejecting bytecode of the form:

NEW x
DUP
... instructions loading ctor-args, involving a backedge
INVOKESPECIAL <init>

The above can be detected bytecode-level, and solved bytecode-level by reformulating into:

... instructions loading ctor-args, involving a backedge
STORE nth-arg
...
STORE 1st-arg
NEW x
DUP
LOAD 1st-arg
...
LOAD nth-arg
INVOKESPECIAL <init>

After rewriting NEW x runs after the code to compute arguments, but it's that behavioral change or VerifyError

To implement the above, an ASM utility computing usage-definition and definition-usage webs is needed. That's already available for the experimental optimizer I've been working on (that utility is called ProdConsAnalyzer. The rewriting described above is implemented by:

private def avoidBackedgesInConstructorArgs(cnode: ClassNode) {

at https://github.com/magarciaEPFL/scala/blob/24db961fb7f1ce982a1d4a750433fe5a2c84b142/src/compiler/scala/tools/nsc/backend/jvm/BCodeOptIntra.scala#L427

scabug · 2013-02-20T20:44:42Z

@magarciaEPFL said:
@JamesIry, now that I think of it, this bug is related to the try/catch lifting discussion.

scabug · 2013-05-01T21:02:06Z

@magarciaEPFL said:
It's weird but on

java version "1.7.0_15"
Java(TM) SE Runtime Environment (build 1.7.0_15-b03)
Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode)

I can't reproduce anymore.

scabug closed this as completed May 16, 2014

scabug added the needinfo label Apr 7, 2017

scabug mentioned this issue Apr 7, 2017

Broken bytecode with "-optimize", "-target:jvm-1.7" #8645

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

invalid bytecode when using range inside the map function of an Option #6720

invalid bytecode when using range inside the map function of an Option #6720

scabug commented Nov 26, 2012

scabug commented Nov 26, 2012

scabug commented Nov 26, 2012

scabug commented Nov 29, 2012

scabug commented Nov 29, 2012

scabug commented Nov 29, 2012

scabug commented Nov 30, 2012

scabug commented Nov 30, 2012

scabug commented Nov 30, 2012

scabug commented Dec 1, 2012

scabug commented Dec 1, 2012

scabug commented Dec 2, 2012

scabug commented Jan 18, 2013

scabug commented Feb 20, 2013

scabug commented May 1, 2013

invalid bytecode when using range inside the map function of an Option #6720

invalid bytecode when using range inside the map function of an Option #6720

Comments

scabug commented Nov 26, 2012

scabug commented Nov 26, 2012

scabug commented Nov 26, 2012

scabug commented Nov 29, 2012

scabug commented Nov 29, 2012

scabug commented Nov 29, 2012

scabug commented Nov 30, 2012

scabug commented Nov 30, 2012

scabug commented Nov 30, 2012

scabug commented Dec 1, 2012

scabug commented Dec 1, 2012

scabug commented Dec 2, 2012

scabug commented Jan 18, 2013

scabug commented Feb 20, 2013

scabug commented May 1, 2013