Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

StackOverflow during parsing of multi line comments by combinator parser #4824

Closed
scabug opened this issue Jul 21, 2011 · 9 comments
Closed
Assignees
Milestone

Comments

@scabug
Copy link

scabug commented Jul 21, 2011

I try to parse simple string that contains a few hundreds of commented lines.

val str = (1 to 300).mkString("/* ", "\n", " */\n")

During parsing of this string by scala.util.parsing.combinator.syntactical.StandardTokenParsers there is an exception:

java.lang.StackOverflowError
at scala.util.parsing.input.CharSequenceReader.first(CharSequenceReader.scala:28)
at scala.util.parsing.combinator.Parsers$$anonfun$acceptIf$1.apply(Parsers.scala:488)
at scala.util.parsing.combinator.Parsers$$anonfun$acceptIf$1.apply(Parsers.scala:487)
at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:182)
at scala.util.parsing.combinator.Parsers$Parser$$anonfun$flatMap$1.apply(Parsers.scala:200)
at scala.util.parsing.combinator.Parsers$Parser$$anonfun$flatMap$1.apply(Parsers.scala:200)
at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:182)
at scala.util.parsing.combinator.Parsers$Parser$$anonfun$map$1.apply(Parsers.scala:203)
at scala.util.parsing.combinator.Parsers$Parser$$anonfun$map$1.apply(Parsers.scala:203)
at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:182)
at scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1.apply(Parsers.scala:208)
at scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1.apply(Parsers.scala:208)
at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:182)

@scabug
Copy link
Author

scabug commented Jul 21, 2011

Imported From: https://issues.scala-lang.org/browse/SI-4824?orig=1
Reporter: Wojciech Durczyński (wojciech.durczynski)
Affected Versions: 2.8.1

@scabug
Copy link
Author

scabug commented Jul 21, 2011

@dcsobral said:
Please add the parser to the issue, so that it can be reproduced. Make it as small as possible while still displaying the problem.

@scabug
Copy link
Author

scabug commented Jul 22, 2011

Wojciech Durczyński (wojciech.durczynski) said:
val str = (1 to 300).mkString("/* ", "\n", " */\n")

val parser = new StandardTokenParsers
val scanner = new parser.lexical.Scanner(new CharSequenceReader(str))
val result = parser.numericLit(scanner)

@scabug
Copy link
Author

scabug commented Jul 31, 2011

Trond Olsen (tolsen77) said:
{noformat}Welcome to Scala version 2.9.1.RC1 (Java HotSpot(TM) Client VM, Java 1.6.0_24).

val str = (1 to 200).mkString("/* ", "\n", " */\n")
val parser = new scala.util.parsing.combinator.syntactical.StandardTokenParsers
val reader = new scala.util.parsing.input.CharSequenceReader(str)
val scanner = new parser.lexical.Scanner(reader)


Leads to stack overflow for the following calls (repeated twice):

at scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1$$anonfun$apply$1.apply(Parsers.scala:210)
at scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1$$anonfun$apply$1.apply(Parsers.scala:210)
at scala.util.parsing.combinator.Parsers$Failure.append(Parsers.scala:163)
at scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1.apply(Parsers.scala:210)
at scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1.apply(Parsers.scala:210)
at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:183)
at scala.util.parsing.combinator.Parsers$Parser$$anonfun$map$1.apply(Parsers.scala:204)
at scala.util.parsing.combinator.Parsers$Parser$$anonfun$map$1.apply(Parsers.scala:204)
at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:183)
at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:183)
at scala.util.parsing.combinator.Parsers$Success.flatMapWithNext(Parsers.scala:114)
at scala.util.parsing.combinator.Parsers$Parser$$anonfun$flatMap$1.apply(Parsers.scala:201)
at scala.util.parsing.combinator.Parsers$Parser$$anonfun$flatMap$1.apply(Parsers.scala:201)
at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:183)

at scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1$$anonfun$apply$1.apply(Parsers.scala:210)
at scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1$$anonfun$apply$1.apply(Parsers.scala:210)
at scala.util.parsing.combinator.Parsers$Failure.append(Parsers.scala:163)
at scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1.apply(Parsers.scala:210)
at scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1.apply(Parsers.scala:210)
at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:183)
at scala.util.parsing.combinator.Parsers$Parser$$anonfun$map$1.apply(Parsers.scala:204)
at scala.util.parsing.combinator.Parsers$Parser$$anonfun$map$1.apply(Parsers.scala:204)
at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:183)
at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:183)
at scala.util.parsing.combinator.Parsers$Success.flatMapWithNext(Parsers.scala:114)
at scala.util.parsing.combinator.Parsers$Parser$$anonfun$flatMap$1.apply(Parsers.scala:201)
at scala.util.parsing.combinator.Parsers$Parser$$anonfun$flatMap$1.apply(Parsers.scala:201)
at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:183)
{noformat}

@scabug
Copy link
Author

scabug commented Aug 15, 2011

@paulp said:
Not sure how much can be done about this. Patches welcome.

@scabug
Copy link
Author

scabug commented Oct 28, 2011

Wojciech Durczyński (wojciech.durczynski) said (edited on Oct 28, 2011 8:41:42 AM UTC):
Problem is in class scala.util.parsing.combinator.lexical.StdLexical
method:

  protected def comment: Parser[Any] = (
      '*' ~ '/'  ^^ { case _ => ' '  }
    | chrExcept(EofCh) ~ comment
    )

causes recursion for every character in multiline comments.
Should be rather something like:

  protected def comment: Parser[Any] = (
      rep('*') ~ '/'  ^^ { case _ => ' '  }
    | rep('*') ~ comment //to avoid recursion for multiline comments filled with '*'
    | rep(chrExcept(EofCh, '*')) ~ comment
    )

@scabug
Copy link
Author

scabug commented Sep 9, 2013

Bodo Junglas (bjunglas) said (edited on Sep 9, 2013 9:55:15 PM UTC):
I've encountered the same problem and solved it this way:

protected def comment: Parser[Any] = 
  rep(chrExcept(EofCh, '*') | '*' ~ not('/')) ~ '*' ~ '/' 
    ^^^ ' '

Probably not the most performant solution, but without recursion whatsoever.

@scabug
Copy link
Author

scabug commented Jul 7, 2014

@scabug
Copy link
Author

scabug commented Jul 9, 2014

@scabug scabug closed this as completed Jul 9, 2014
@scabug scabug added this to the 2.11.2 milestone Apr 7, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants