Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

getDeclaredClasses seems to miss some (REPL) or all (scalac) classes (objects) declared #4023

Closed
scabug opened this issue Nov 22, 2010 · 38 comments
Assignees
Milestone

Comments

@scabug
Copy link

scabug commented Nov 22, 2010

This Java code works:

class A {
  static class A1{}
  private static class A2{}
  class A3{}
  private class A4{}

  public static void main(String[] args){
    A dow = new A();
    Class[] classes = dow.getClass().getDeclaredClasses();
    System.out.println("Amount: " + classes.length + " classes");
    for(Class cls : classes){
      System.out.println(cls);
    }
  } 
}
/* 
> java A
Amount: 4 classes
class A$$A4
class A$$A3
class A$$A2
class A$$A1

But this Scala code doesn't:

object B {
  class B1
  private class B2
  object B3
  private object B4
  object B5 extends B1
  private object B6 extends B2 
 
  val valuesTry1 = this.getClass.getDeclaredClasses
  val valuesTry2 = B.getClass.getDeclaredClasses
  val valuesTry3 = getClass.getDeclaredClasses
        
  def main(args: Array[String]) {
    println("Try 1: (" + valuesTry1.length + " classes)")
    valuesTry1.foreach(println)
    println("Try 2: (" + valuesTry2.length + " classes)")
    valuesTry2.foreach(println)
    println("Try 3: (" + valuesTry3.length + " classes)")
    valuesTry3.foreach(println)
  }
}

In fact the result depends on how you run the file.

If the file is compiled it prints:

> scalac B.scala
> scala B
Try 1: (0 classes)
Try 2: (0 classes)
Try 3: (0 classes)

If the code is run on the REPL I get at least classes B1 and B2, but the rest is still missing:

scala> F.main(Array(""))
Try 1: (5 classes)
class line5$$object$$$$iw$$$$iw$$F$$B2
class line5$$object$$$$iw$$$$iw$$F$$B1
class line5$$object$$$$iw$$$$iw$$F$$$$anonfun$$main$$3
class line5$$object$$$$iw$$$$iw$$F$$$$anonfun$$main$$2
class line5$$object$$$$iw$$$$iw$$F$$$$anonfun$$main$$1
Try 2: (5 classes)
class line5$$object$$$$iw$$$$iw$$F$$B2
class line5$$object$$$$iw$$$$iw$$F$$B1
class line5$$object$$$$iw$$$$iw$$F$$$$anonfun$$main$$3
class line5$$object$$$$iw$$$$iw$$F$$$$anonfun$$main$$2
class line5$$object$$$$iw$$$$iw$$F$$$$anonfun$$main$$1
Try 3: (5 classes)
class line5$$object$$$$iw$$$$iw$$F$$B2
class line5$$object$$$$iw$$$$iw$$F$$B1
class line5$$object$$$$iw$$$$iw$$F$$$$anonfun$$main$$3
class line5$$object$$$$iw$$$$iw$$F$$$$anonfun$$main$$2
class line5$$object$$$$iw$$$$iw$$F$$$$anonfun$$main$$1

In my opinion it should be investigated why Java reflection doesn't work as it is supposed to be.

Considering that the object declaration does also create a class (/class file), it should also show up in getDeclaredClasses.

@scabug
Copy link
Author

scabug commented Nov 22, 2010

Imported From: https://issues.scala-lang.org/browse/SI-4023?orig=1
Reporter: @soc
Other Milestones: 2.11.0-M1
See #4912

@scabug
Copy link
Author

scabug commented Nov 22, 2010

@paulp said:
See #2749 for more background info.

@scabug
Copy link
Author

scabug commented Jan 3, 2011

@dragos said:
As Paul says, ticket #2749 has more information about this issue. In short, inner classes defined inside an object are generated as part of the companion class, so that Java programs can access them with the usual syntax (B.Inner instead of B$$.Inner). We really need a Scala specific reflection library, which works on the correct types. Until then, I really don't know what we can do.

@scabug
Copy link
Author

scabug commented Jan 10, 2011

@soc said:
Hi dragos!

Wouldn't it be possible to do something comparable to the "static forwarders" which Scala generates in some cases?

I'm working on an Enum trait and I can't think of any way how it could possibly work without getting this information from reflection.

Or can you think of any workaround on how to see which objects are defined in an object?

Thanks!

@scabug
Copy link
Author

scabug commented Feb 23, 2011

@dragos said:
I didn't try this, but if you add a companion class to B, you could see them as part of that class. Something like

class B {
  val valuesTry1 = this.getClass.getDeclaredClasses
}

Maybe even classOf[B].getDeclaredClasses works (but you still need a class B)

@scabug
Copy link
Author

scabug commented Feb 23, 2011

@soc said:
Thanks dragos, but it didn't really work :-/ I saw the classes sometimes, but the objects are still missing:

object B {
  class B1
  private class B2
  object B3
  private object B4
  object B5 extends B1
  private object B6 extends B2 
 
  val valuesTry1 = this.getClass.getDeclaredClasses
  val valuesTry2 = B.getClass.getDeclaredClasses
  val valuesTry3 = getClass.getDeclaredClasses
        
  def main(args: Array[String]) {
    println("Try 1: (" + valuesTry1.length + " classes)")
    valuesTry1.foreach(println)
    println("Try 2: (" + valuesTry2.length + " classes)")
    valuesTry2.foreach(println)
    println("Try 3: (" + valuesTry3.length + " classes)")
    valuesTry3.foreach(println)

    val b = new B
    println("Try 4: (" + b.valuesTry4.length + " classes)")
    b.valuesTry4.foreach(println)
    println("Try 5: (" + b.valuesTry5.length + " classes)")
    b.valuesTry5.foreach(println)
    println("Try 6: (" + b.valuesTry6.length + " classes)")
    b.valuesTry6.foreach(println)
    println("Try 7: (" + b.valuesTry7.length + " classes)")
    b.valuesTry7.foreach(println)

  }
}

class B {
  val valuesTry4 = this.getClass.getDeclaredClasses
  val valuesTry5 = B.getClass.getDeclaredClasses
  val valuesTry6 = getClass.getDeclaredClasses
  val valuesTry7 = classOf[B].getDeclaredClasses
}
 scala B
Try 1: (0 classes)
Try 2: (0 classes)
Try 3: (0 classes)
Try 4: (2 classes)
class B$$B1
class B$$B2
Try 5: (0 classes)
Try 6: (2 classes)
class B$$B1
class B$$B2
Try 7: (2 classes)
class B$$B1
class B$$B2

I really wonder what the difference between B.getClass and classOf[B] is though ...

@scabug
Copy link
Author

scabug commented Feb 24, 2011

@dragos said:
So inner objects are not reported anywhere, that indeed looks wrong. I'll have a look in the coming days.

About the different ways to get a Class for B: you have two entities in your program that are called B:

  • one is a type (class B)
  • the other is a value (object B).

classOf takes a type parameter, therefore there is no ambiguity when you say classOf[B]: it will always resolve to the type B.

B.getClass is again non-ambiguous, because it is an expression, therefore the receiver must be a value (in this case, object B). Each of these have their own Class object, since they are different (different methods and fields), and different classfiles on disk (the object will be in B$$).

@scabug
Copy link
Author

scabug commented Sep 10, 2011

@gkossakowski said:
Simon,

Could you check latest nightly? It contains some changes related to inner objects.

@scabug
Copy link
Author

scabug commented Sep 10, 2011

@paulp said:
Where latest will have to be one more day, because it didn't build last night (for reasons I don't know, grek's patch led to the continuations plugin requiring fjbg on the classpath.)

@scabug
Copy link
Author

scabug commented Sep 11, 2011

@soc said:
It works in the REPL now:

scala> B.main(null)
Try 1: (6 classes)
class $line1.$read$$iw$$iw$B$B1
class $line1.$read$$iw$$iw$B$B2
class $line1.$read$$iw$$iw$B$B3$
class $line1.$read$$iw$$iw$B$B4$
class $line1.$read$$iw$$iw$B$B5$
class $line1.$read$$iw$$iw$B$B6$
Try 2: (6 classes)
class $line1.$read$$iw$$iw$B$B1
class $line1.$read$$iw$$iw$B$B2
class $line1.$read$$iw$$iw$B$B3$
class $line1.$read$$iw$$iw$B$B4$
class $line1.$read$$iw$$iw$B$B5$
class $line1.$read$$iw$$iw$B$B6$
Try 3: (6 classes)
class $line1.$read$$iw$$iw$B$B1
class $line1.$read$$iw$$iw$B$B2
class $line1.$read$$iw$$iw$B$B3$
class $line1.$read$$iw$$iw$B$B4$
class $line1.$read$$iw$$iw$B$B5$
class $line1.$read$$iw$$iw$B$B6$

but compiling it with scalac and executing it still returns:

% scala B
Try 1: (0 classes)
Try 2: (0 classes)
Try 3: (0 classes)

I belive this has something to do with the additional nesting the REPL does, because if I modify the code, so that the object are in another nested object, it starts working too:

object B {
  object C {
    class B1
    private class B2
    object B3
    private object B4
    object B5 extends B1
    private object B6 extends B2 
 
    val valuesTry1 = this.getClass.getDeclaredClasses
    val valuesTry2 = C.getClass.getDeclaredClasses
    val valuesTry3 = getClass.getDeclaredClasses
  }
        
  def main(args: Array[String]) {
    println("Try 1: (" + C.valuesTry1.length + " classes)")
    C.valuesTry1.foreach(println)
    println("Try 2: (" + C.valuesTry2.length + " classes)")
    C.valuesTry2.foreach(println)
    println("Try 3: (" + C.valuesTry3.length + " classes)")
    C.valuesTry3.foreach(println)
  }
}
% scala B
Try 1: (6 classes)
class B$C$B1
class B$C$B2
class B$C$B3$
class B$C$B4$
class B$C$B5$
class B$C$B6$
Try 2: (6 classes)
class B$C$B1
class B$C$B2
class B$C$B3$
class B$C$B4$
class B$C$B5$
class B$C$B6$
Try 3: (6 classes)
class B$C$B1
class B$C$B2
class B$C$B3$
class B$C$B4$
class B$C$B5$
class B$C$B6$

It feels like we are really close to a solution to this ...

@scabug
Copy link
Author

scabug commented Sep 11, 2011

@gkossakowski said:
Hi Simon,

Thanks for testing this. This is expected in the light of how we encode inner classes inside of top-level objects. See commit message for my change for details: https://codereview.scala-lang.org/fisheye/changelog/scala-svn?cs=25639

I tried hard to make encoding regular and not introduce special case for top-level objects but found it impossible if we want both Sun's javac compiler and Eclipse compiler happy.

There's an easy work-around for top-level objects:

object B {
  class B1
  private class B2
  object B3
  private object B4
  object B5 extends B1
  private object B6 extends B2 
  
  def mirrorClassForTopLevelObject(x: Class[_]): Class[_] = Class.forName(x.getName stripSuffix "$")
 
  val valuesTry1 = mirrorClassForTopLevelObject(this.getClass).getDeclaredClasses
  val valuesTry2 = mirrorClassForTopLevelObject(B.getClass).getDeclaredClasses
  val valuesTry3 = mirrorClassForTopLevelObject(getClass).getDeclaredClasses
        
  def main(args: Array[String]) {
    println("Try 1: (" + valuesTry1.length + " classes)")
    valuesTry1.foreach(println)
    println("Try 2: (" + valuesTry2.length + " classes)")
    valuesTry2.foreach(println)
    println("Try 3: (" + valuesTry3.length + " classes)")
    valuesTry3.foreach(println)
  }
}

this code prints:

Try 1: (6 classes)
class B$B1
class B$B2
class B$B3$
class B$B4$
class B$B5$
class B$B6$
Try 2: (6 classes)
class B$B1
class B$B2
class B$B3$
class B$B4$
class B$B5$
class B$B6$
Try 3: (6 classes)
class B$B1
class B$B2
class B$B3$
class B$B4$
class B$B5$
class B$B6$

How do you feel about it?

At the end of the day, all those problems will disappear once we get native Scala reflection which is in the works.

@scabug
Copy link
Author

scabug commented Sep 11, 2011

@paulp said:
I think classOf[B] needs to compile, if we're stealthily creating a class B and putting all the inner classes in there.

@scabug
Copy link
Author

scabug commented Sep 11, 2011

@gkossakowski said:
But there are two classes for B, which one we should return?

@scabug
Copy link
Author

scabug commented Sep 11, 2011

@paulp said:
There's only one B - it's B, not B$. classOf[B] returns B already, that's pretty well fixed. The problem is that now compilation fails because class B is unknown, because the earlier stage of the compiler isn't aware that genjvm is going to synthesize it.

As it stands, the same source will compile if you compile against bytecode (since the class will have been generated) but not if you compile with pure source. That's no good.

@scabug
Copy link
Author

scabug commented Sep 11, 2011

@gkossakowski said:
Ah, I get it now. However, if we wanted to get it to work we would need to plug the logic somewhere between namer and typer, right?

@scabug
Copy link
Author

scabug commented Sep 11, 2011

@paulp said:
Interesting, it's a little fuzzier than I'd thought. You can't compile against the B.class file; scalac seems to be aware there was never a declared B class.

Think we need martin to opine here.

@scabug
Copy link
Author

scabug commented Sep 11, 2011

@gkossakowski said:
Agreed, let's discuss this during meeting. However, I have a feeling that folks would like to get rid of B.class altogether. It's there only for Java interoperability and we wouldn't like to pollute compiler with that stuff.

Anyway, feel free to add it to the meeting list (I don't know how to do it).

@scabug
Copy link
Author

scabug commented Sep 11, 2011

@soc said:
@grzegorz:
If javac and ecj behave differently, shouldn't we file a bug report then, to get both to behave as defined in the spec(?)/the same?
Are there any other Java compilers out there, which should probably be tested, too?

@scabug
Copy link
Author

scabug commented Sep 11, 2011

@gkossakowski said:
@simon: I don't have any good experience with reporting either to javac or ecj bug tracker (none of bugs I or my colleagues found were ever fixed). Also, spec is a bit vague on those matters:

http://stackoverflow.com/questions/6167326/java-class-name-containing-dollar-sign

@scabug
Copy link
Author

scabug commented Sep 11, 2011

@paulp said:
I like how you pretty much let them think you're just a guy who likes funny class names.

@scabug
Copy link
Author

scabug commented Sep 12, 2011

@soc said:
@grzegorz: I guess then it would make more sense to have those things better documented first, before trying to get the compilers to behave the same.

@scabug
Copy link
Author

scabug commented Sep 12, 2011

@gkossakowski said:
@simon: I'm busy enough with fixing Scala compiler bugs and won't be spending my time fixing Java compilers. If you ask me, I'm happy with the situation we have now. It's an improvement over what we had before when it comes to inner objects. It's not perfect but I don't believe those problems can be solved completely.

Personally, I'd vote for closing this issue.

@scabug
Copy link
Author

scabug commented Sep 12, 2011

@soc said:
@grzegorz: I didn't want to throw that work at you. If you think it makes sense tell me and I'll file a bug in the OpenJDK/Oracle bug tracker about it.

@scabug
Copy link
Author

scabug commented Jul 11, 2012

@xeno-by said (edited on Jul 11, 2012 11:23:18 AM UTC):
Temporarily raising the priority of this bug until the review of all bugs due 2.10.0-RC1, since it shuts down an important family of reflection use cases.

@scabug
Copy link
Author

scabug commented Jul 11, 2012

@gkossakowski said:
Eugene: Examples? I thought Scala reflection will make this issue irrelevant.

@scabug
Copy link
Author

scabug commented Jul 11, 2012

@xeno-by said:
#5947 and (from the limited understanding of our codegen that I have) any scenario that involves nested objects. Scala reflection needs Java reflection to enumerate inner classes: https://github.com/scalamacros/kepler/blob/026a70d55591c3b5ee157e22998b62168afee686/src/reflect/scala/reflect/runtime/JavaMirrors.scala#L915.

@scabug
Copy link
Author

scabug commented Jul 11, 2012

@gkossakowski said:
Well, I'd go with special case for top-level objects (as presented in my comment above: https://issues.scala-lang.org/browse/SI-4023?focusedCommentId=54759&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-54759) and be done with it because everything else should just work. Right?

@scabug
Copy link
Author

scabug commented Jul 11, 2012

@xeno-by said:
That's neat! Seems to fix the immediate blocker. Let me dig deeper...

@scabug
Copy link
Author

scabug commented Jul 11, 2012

@xeno-by said:
Allrighty it worked: scala/scala#888. Thanks for the idea!

@scabug
Copy link
Author

scabug commented Jul 12, 2012

@dragos said:
I think we should find a way not to rely on Java reflection for everything that's not top-level. If Scala reflection inherits the most annoying limitation of Java reflection, well, why bother?

@scabug
Copy link
Author

scabug commented Jul 12, 2012

@gkossakowski said:
Eugene, I'm still not sure why we use getDeclaredClasses in Scala reflection? Shouldn't we traverse Scala symbols (created by unpickling) to figure out binary name of nested class and only then load it?

@scabug
Copy link
Author

scabug commented Jul 12, 2012

@xeno-by said:
Oh so you can access nested classes without going through parents?! Wow that's big news!

@scabug
Copy link
Author

scabug commented Jul 12, 2012

@xeno-by said:
Greg, is there a way to quickly find out binary name of a class/module without having to construct it manually?

@scabug
Copy link
Author

scabug commented Jul 12, 2012

@gkossakowski said:
Yup:

object A {
  object B {
    class C
  }
}

object Test {
  def main(args: Array[String]) {
    val c = Class.forName(classOf[A.B.C].getName)
    println(c)
  }
}

prints

class A$B$C

I would have suggested that before if I understood your reasoning. That's exactly why we ask for more information here and there ;-)

@scabug
Copy link
Author

scabug commented Jul 12, 2012

@gkossakowski said:
Let's move this discussion to internals mailing list. We abused this ticket for too long. Describe what you need exactly and I (and probably others) might be able to help.

@scabug
Copy link
Author

scabug commented Jul 12, 2012

@scabug
Copy link
Author

scabug commented Jul 28, 2012

@soc said:
Weird ... just saw the mail that somehow I assigned this ticket to me. No idea how that happened.

@scabug
Copy link
Author

scabug commented Jul 29, 2012

@soc said:
Testcase in 07824e5.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants