I assume JavaTokenParser.ident is meant to parse all valid Java identifiers? If so, it doesn't. For example, MODULE$ is a valid Java identifier (that Scala itself uses extensively), and this won't parse. Also, Java identifiers can contain unicode characters, so ☃ is a valid identifier. Fortunately, Java provides a very simple way to parse Java identifiers, using the following regular expression:
It uses the Character.isJavaIdentifierStart and Character.isJavaIdentifierPart methods for the first and then subsequent letters, and is guaranteed to match all valid Java identifiers, as long as they don't clash with keywords.
For language spec nuts: