"Deadlock" in initialization with package objects and companion objects #7646

scabug · 2013-07-10T11:55:50Z

I seem to be able to create some kind of "deadlock" by running a test (declared in Main.scala). The deadlock is caused by separate threads which are accessing a.b.c.Oxbow$class.$init$ - the first because of a call to a method brought into scope via an import a.b.c.Oxbow._ and the second by import a.b.c._. If you run the test a few times, eventually you'll see the "deadlock" - I'm left with two threads which are doing this:

"ForkJoinPool-1-worker-9" daemon prio=6 tid=0x0000000014630800 nid=0x1940 in Object.wait() [0x000000001543e000]
   java.lang.Thread.State: RUNNABLE
                at a.b.c.Oxbow$class.$init$(Oxbow.scala:4)
                at a.b.c.package$.<init>(package.scala:3)
                at a.b.c.package$.<clinit>(package.scala)
                at test.Test1$$anonfun$bar$1$$anonfun$apply$1.apply$mcV$sp(Test1.scala:12)
                at test.Test1$$anonfun$bar$1$$anonfun$apply$1.apply(Test1.scala:9)
                at test.Test1$$anonfun$bar$1$$anonfun$apply$1.apply(Test1.scala:9)
                at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
                at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
                at scala.concurrent.impl.ExecutionContextImpl$$anon$3.exec(ExecutionContextImpl.scala:107)
                at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
                at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
                at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
                at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)


"ForkJoinPool-1-worker-11" daemon prio=6 tid=0x000000001462c000 nid=0x2a1c in Object.wait() [0x0000000014fcd000]
   java.lang.Thread.State: RUNNABLE
                at a.b.c.d.Bippy$.<init>(Bippy.scala:8)
                at a.b.c.d.Bippy$.<clinit>(Bippy.scala)
                at a.b.c.Oxbow$class.$init$(Oxbow.scala:4)
                at a.b.c.Oxbow$.<init>(Oxbow.scala:13)
                at a.b.c.Oxbow$.<clinit>(Oxbow.scala)
                at test.Test2$$anonfun$foo$1$$anonfun$apply$1.apply$mcV$sp(Test2.scala:13)
                at test.Test2$$anonfun$foo$1$$anonfun$apply$1.apply(Test2.scala:10)
                at test.Test2$$anonfun$foo$1$$anonfun$apply$1.apply(Test2.scala:10)
                at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
                at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
                at scala.concurrent.impl.ExecutionContextImpl$$anon$3.exec(ExecutionContextImpl.scala:107)
                at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
                at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
                at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
                at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

I'm calling it a "deadlock" (i.e. in moderately ironic quotation marks) because, according to JConsole, there is no deadlock and according to the stacks both threads are in RUNNABLE state and are not WAITING at all (could they have been woken up by a call to notify?) - but they don't appear to be doing anything. You can leave it for 10 minutes and the threads will be in the same state

If I change the code so that both Test1 and Test2 do import a.b.c._ then the issue is not observed

The files are organized as follows:

in package a.b.c

  File Oxbow.scala
  File package.scala

In package a.b.c.d

  File Bippy.scala

in package test

  File Main.scala

You need to run Main.scala a few times to reveal the deadlock. I'm fairly confident that this must be a bug in scalac, rather than a trait initialization order problem on my side (although I could be wrong)

The text was updated successfully, but these errors were encountered:

scabug · 2013-07-10T11:55:50Z

Imported From: https://issues.scala-lang.org/browse/SI-7646?orig=1
Reporter: @oxbowlakes
Affected Versions: 2.10.2
See #7814
Attachments:

Bippy.scala (created on Jul 10, 2013 11:55:50 AM UTC, 121 bytes)
Main.scala (created on Jul 10, 2013 11:55:50 AM UTC, 1089 bytes)
Oxbow.scala (created on Jul 10, 2013 11:55:50 AM UTC, 172 bytes)
package.scala (created on Jul 10, 2013 11:55:50 AM UTC, 47 bytes)

scabug · 2013-09-04T21:19:38Z

@retronym said (edited on Sep 4, 2013 9:20:36 PM UTC):
I just took a look at this test case. It looks like a deadlock, where the locks in play are the intrinsic locks on the corresponding Class objects that occur during class initialization [1].

A Scala module Foo has a corresponding module class Foo$, which contains a static field MODULE$, which is initialized to new Foo$ in the static initializer of the class.

% cat sandbox/test.scala
object O {
  println(Thread.currentThread.getStackTrace.toList.take(5).mkString("\n"))
}

object Test {
  def main(args: Array[String]) {
    O
  }
}
% qbin/scalac sandbox/test.scala && qbin/scala Test
java.lang.Thread.getStackTrace(Thread.java:1503)
O$.<init>(test.scala:2)
O$.<clinit>(test.scala)
Test$.main(test.scala:7)
Test.main(test.scala)

Generates:

scala> :javap -v -p O$
Compiled from "test.scala"
public final class O$ extends java.lang.Object
  SourceFile: "test.scala"
  Scala: length = 0x
...
{
public static final O$ MODULE$;

public static {};
  Code:
   Stack=1, Locals=0, Args_size=0
   0:	new	#2; //class O$
   3:	invokespecial	#12; //Method "<init>":()V
   6:	return

private O$();
  Code:
   Stack=3, Locals=1, Args_size=1

If objects form a cycle, those class initialization locks can deadlock:

object O1 {
  val x = O2.x
  def y = 0
}
object O2 {
  val x = O1.y
}

object Test extends App {
  Await.result(Future.sequence(List(future(O1.y), future(O2.x)), Duration.Inf)
}

Here's a stack overflow post about this topic [2].

Similar problems are possible with lazy vals; we have a bit more of an idea how to reduce those, which we've outlined in the Lazy Val SIP [3], which is planned for 2.12. But I don't know of any proposals to change the encoding of modules.

A workaround is to eagerly force objects during single threaded application startup to break the cycle. Or avoid cycles between objects.

I should mention that this is not specific to package objects; as far as the generated bytecode is concerned, they are no different from other Scala objects.

So I have to close this one as not-a-bug. I wish we could do more.

[1] http://docs.oracle.com/javase/specs/jvms/se7/html/jvms-5.html
[2] http://stackoverflow.com/questions/15176199/scala-parallel-collection-in-object-initializer-causes-a-program-to-hang
[3] http://docs.scala-lang.org/sips/pending/improved-lazy-val-initialization.html

scabug · 2013-09-04T21:43:26Z

@gkossakowski said:
The potential deadlock in case of cyclic dependencies is speced:

Note that the value deﬁned by an object deﬁnition is instantiated lazily. The
new m$cls constructor is evaluated not at the point of the object deﬁnition, but is
instead evaluated the ﬁrst time m is dereferenced during execution of the program
(which might be never at all). An attempt to dereference m again in the course
of evaluation of the constructor leads to a inﬁnite loop or run-time error. Other
threads trying to dereference m while the constructor is being evaluated block until
evaluation is complete.

See Scala Specification (5.4).

One could wish we would just throw an exception at runtime in case of cyclic dependencies between objects or lazy vals. I believe we could do that but without invokedynamic cycle detection would be probably rather expensive.

scabug · 2013-09-05T14:24:22Z

@retronym said:
See also #7814, in which the standard library exhibits a cyclic dependencies between objects.

scabug · 2014-12-19T10:13:56Z

Richard Bradley (richard.bradley) said (edited on Dec 19, 2014 10:14:09 AM UTC):
I have a similar problem. One thing which exacerbates the issue for me is that the static initializer lock does not appear in any of the Java diagnostic tools.
These deadlocks appear to be invisible to all normal Java debugging tools.
The deadlocked threads are even marked "RUNNABLE" by the JVM.

More on that here: http://stackoverflow.com/questions/27549671/how-to-diagnose-or-detect-deadlocks-in-java-static-initializers

scabug closed this as completed Sep 4, 2013

scabug assigned retronym Apr 7, 2017

ghostdogpr mentioned this issue Jun 26, 2020

Layer composition seems to hang on some cases zio/zio#3871

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

"Deadlock" in initialization with package objects and companion objects #7646

"Deadlock" in initialization with package objects and companion objects #7646

scabug commented Jul 10, 2013

scabug commented Jul 10, 2013

scabug commented Sep 4, 2013

scabug commented Sep 4, 2013

scabug commented Sep 5, 2013

scabug commented Dec 19, 2014

"Deadlock" in initialization with package objects and companion objects #7646

"Deadlock" in initialization with package objects and companion objects #7646

Comments

scabug commented Jul 10, 2013

scabug commented Jul 10, 2013

scabug commented Sep 4, 2013

scabug commented Sep 4, 2013

scabug commented Sep 5, 2013

scabug commented Dec 19, 2014