Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

partest includes VM Warnings in compared output #7198

Closed
scabug opened this issue Feb 28, 2013 · 5 comments
Closed

partest includes VM Warnings in compared output #7198

scabug opened this issue Feb 28, 2013 · 5 comments
Assignees
Labels
Milestone

Comments

@scabug
Copy link

scabug commented Feb 28, 2013

partest compares raw output from Runner invocation, including warnings from the VM potentially unrelated to the test being run. This can result in false positives.

For example, if my system-wide _JAVA_OPTIONS includes -XX:+UseLargePages, it is to be expected that partest's parallel invocations will exhaust the reserved space for large pages on my system, triggering an ENOSPC. Here is one output of partest --show-diff --presentation

testing: [...]/files/presentation/ide-bug-1000349                     [  OK  ]
testing: [...]/files/presentation/callcc-interpreter                  [  OK  ]
testing: [...]/files/presentation/hyperlinks                          [FAILED]
1d0
< Java HotSpot(TM) 64-Bit Server VM warning: Failed to reserve shared memory (errno = 28).

Problems:

  • this is a head scratcher until run with --show-diff
  • while +UseLargePages is of limited use with partest, it has its use cases as a system-wide setting, as do many other edgy options.
  • in general, we may not want to let VM warnings fail tests by themselves.
@scabug
Copy link
Author

scabug commented Feb 28, 2013

Imported From: https://issues.scala-lang.org/browse/SI-7198?orig=1
Reporter: @huitseeker
Affected Versions: 2.10.1-RC2, 2.10.1

@scabug
Copy link
Author

scabug commented Mar 6, 2013

@huitseeker said (edited on Mar 6, 2013 6:10:36 PM UTC):
One fix Idea from Martin : could partest accept a filter file, or a regular expression filter as a configuration parameter ?

That way it would remove lines that match that particular filter from the program's output before comparing it to the .check output. This also solves the issue of not being able to predict all possible VM routine warnings (and it would probably allow us to remove some garbage in tests' check files).

@scabug
Copy link
Author

scabug commented Mar 6, 2013

@som-snytt said:
This is related to the problem (which I haven't solved) of system-dependent file paths in output. Paul patched that by normalizing backslashes, but that affects other output like \u0022.

I have an experimental "modeline"-style feature for configs. It was motivated by scalacheck options, e.g.:

/*
 * scalac: -deprecation
 * scalacheck: -workers 1 -minSize 0 -maxSize 4000 -minSuccessfulTests 5
 */

For config specific to a test, it avoids extra files; my other idea was to process "enclosing" .flags files, e.g., presentation.flags.

Either in a .flags or header, filter: Java HotSpot or similar as a line-oriented filter.

Or map: _ replaceAll ('\\','/') or similar would be fun.

The other thing I noticed it that we compare stdout + stderr. It would be nice to be able to diff only stdout. I see my code uses a ProcessLogger (I forget that Process API as soon as I use it).

@scabug
Copy link
Author

scabug commented May 25, 2013

@som-snytt said:
scala/scala#2591

The commit uses a filters file for all tests, or all of a kind.

The other use case would be to turn off the inliner warning for everything; right now, a handful of tests specify that filter in a source header.

@scabug
Copy link
Author

scabug commented May 26, 2013

@som-snytt said:
The commit includes a global filters file; it's not obvious that it's desirable as a default configuration.

@scabug scabug closed this as completed May 26, 2013
@scabug scabug added the build label Apr 7, 2017
@scabug scabug added this to the 2.11.0-M4 milestone Apr 7, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants