Uploaded image for project: 'Scala Programming Language'
  1. Scala Programming Language
  2. SI-9828

Scala.Tuple incorrect work after JIT optimization

    Details

    • Type: Bug
    • Status: CLOSED
    • Priority: Major
    • Resolution: Out of Scope
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Environment:

      Java 1.8.0_60-b27
      Scala: scala-library v 2.10.4

      Description

      Wile constructing Tuple6 inside lambda I'm observing null value passing into the Tuple instead on not null value. This issue occurs only on "warmed-up" JVM after hot spots compilations, which is resulting in hardly trackable, not always reproducible bugs.

      Code snippet which causes issue is below:

      private Set<Tuple6<Long, Long, Byte, String, Byte, String>> doStuff(List<String> demoIds,
                      long programId, long date, TimeShiftingViewingPeriod viewingPeriod, Contribution contribution, String marketBreak) {
                  Set<Tuple6<Long, Long, Byte, String, Byte, String>> res = new HashSet<>();
                  demoIds.forEach(demoId -> res.add(new Tuple6<>(date, programId, viewingPeriod.getId(), demoId, contribution.getId(), marketBreak)));
                  return res;
              }
      

      The issue is that in resulting Set we may get Tuples with null values on third position, which is not possible since viewingPeriod is enum, where id is always not null value specified in constructor, and not modifiable.
      Note: Issue is not reproduced if replace Collection.forEach with old style for expression.
      Note: Issue is not reproduced if we replace Tuple with any Java Parametrized collection (Map for example)
      Note: Issue is not reproduced if we disable JIT
      Note: Issue is reproduced in random manner, and we noticed it is more often reproduced in case computer is running under load caused by some other tasks. We started build process for our project in parallel.
      Test project: both sources and executable jar is in attachment.

      How issue could be observed:
      Just run attached executable jar using java -jar <jar name> and check output.
      At some point it starts showing message like: Start reproducing: Thread <thread name> at step <step> res: <Tuple content, on which issue was detected>.
      Here step - is step in loop on which issue starts reproducing.
      Sometimes it is necessary to run application several times (up to 10-20 to get this error), we only notice it appears faster in case computer is working under load.

        Attachments

          Issue Links

            Activity

            Hide
            alexey.tsiunchik@gmail.com Alexey Tsiunchik added a comment -

            Example of output.

            Show
            alexey.tsiunchik@gmail.com Alexey Tsiunchik added a comment - Example of output.
            Hide
            soc Simon Ochsenreither added a comment -

            I think this bug should be filed in the JIT vendor's bugtracker. I think there is not much we can do here if scalac emits the right bytecode.

            Show
            soc Simon Ochsenreither added a comment - I think this bug should be filed in the JIT vendor's bugtracker. I think there is not much we can do here if scalac emits the right bytecode.
            Hide
            alexey.tsiunchik@gmail.com Alexey Tsiunchik added a comment -

            Well I was going to do this, but while investigating this issue using Java native collections, or custom java classes I cannot reproduce this issue. I think this issue is somewhere at intersection between scala and JVM JIT. I suspect that scala lazy implementation in conjunction with java lambda + JIT optimization gives this effect. Not sure what exactly causing this issue, either byte code produced by scala compiler or JIT itself. But I'm sure it is quite serious since may cause issues which would be very difficult to track in high loaded systems (with many precompiled hot spots) which occurs randomly (depending on load). So I think scala team should be interested in fixing such kind of issues since they affects reliability of solutions build with scala.

            Show
            alexey.tsiunchik@gmail.com Alexey Tsiunchik added a comment - Well I was going to do this, but while investigating this issue using Java native collections, or custom java classes I cannot reproduce this issue. I think this issue is somewhere at intersection between scala and JVM JIT. I suspect that scala lazy implementation in conjunction with java lambda + JIT optimization gives this effect. Not sure what exactly causing this issue, either byte code produced by scala compiler or JIT itself. But I'm sure it is quite serious since may cause issues which would be very difficult to track in high loaded systems (with many precompiled hot spots) which occurs randomly (depending on load). So I think scala team should be interested in fixing such kind of issues since they affects reliability of solutions build with scala.
            Hide
            rytz Lukas Rytz added a comment -

            This indeed looks like a serious issue. I managed to reproduce the issue from your source code.

            I simplified the example quite a lot and managed to make it independent of the Scala library. I pushed the code on this github repository: https://github.com/lrytz/jit-bug/tree/master.

            Now we just need to find a way to report that bug to Oracle.. Usually Twitter or StackOverflow work better than trying to go through the contact forms on their website.

            Show
            rytz Lukas Rytz added a comment - This indeed looks like a serious issue. I managed to reproduce the issue from your source code. I simplified the example quite a lot and managed to make it independent of the Scala library. I pushed the code on this github repository: https://github.com/lrytz/jit-bug/tree/master . Now we just need to find a way to report that bug to Oracle.. Usually Twitter or StackOverflow work better than trying to go through the contact forms on their website.
            Hide
            rytz Lukas Rytz added a comment -
            Show
            rytz Lukas Rytz added a comment - Trying my luck: https://twitter.com/lrytz/status/747361572138397696
            Hide
            alexey.tsiunchik@gmail.com Alexey Tsiunchik added a comment - - edited

            Thank you Lukas for further investigation of this issue and simplified example. I'm going to submit issue to Oracle bug tracker. Will post link here.

            Show
            alexey.tsiunchik@gmail.com Alexey Tsiunchik added a comment - - edited Thank you Lukas for further investigation of this issue and simplified example. I'm going to submit issue to Oracle bug tracker. Will post link here.
            Show
            rytz Lukas Rytz added a comment - It's a known issue that will be fixed in 8u112, see https://twitter.com/shipilev/status/747394501803655168 , https://bugs.openjdk.java.net/browse/JDK-8148752 , https://bugs.openjdk.java.net/browse/JDK-8153980

              People

              • Assignee:
                rytz Lukas Rytz
                Reporter:
                alexey.tsiunchik@gmail.com Alexey Tsiunchik
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: