Commit Graph

4 Commits

Author SHA1 Message Date
Julian Lettner 99d6e05e71 [lit] Improve naming of test result categories
Improve consistency when printing test results:
Previously we were using different labels for group names (the header
for the list of, e.g., failing tests) and summary count lines.  For
example, "Failing Tests"/"Unexpected Failures".  This commit changes lit
to label things consistently.

Improve wording of labels:
When talking about individual test results, the first word in
"Unexpected Failures", "Expected Passes", and "Individual Timeouts" is
superfluous.  Some labels contain the word "Tests" and some don't.
Let's simplify the names.

Before:
```
Failing Tests (1):
  ...

Expected Passes    : 3
Unexpected Failures: 1
```

After:
```
Failed Tests (1):
  ...

Passed: 3
Failed: 1
```

Reviewed By: ldionne

Differential Revision: https://reviews.llvm.org/D77708
2020-06-05 08:14:42 -07:00
Julian Lettner c610807afe [lit] Temporarily disable failing tests on Windows 2020-04-10 20:03:44 -07:00
Julian Lettner 5925c4a0ff [lit] Increase sleep time in timeout test
Fixup for cbe42a9d5f.  Increase values for testing the overall lit
timeout (--max-time) which wasn't enough for the test to complete on
very slow build bots.
2020-04-10 16:22:00 -07:00
Julian Lettner cbe42a9d5f [lit] Add SKIPPED test result category
Track and print the number of skipped tests.  Skipped tests are tests
that should have been executed but weren't due to:
  * user interrupt [Ctrl+C]
  * --max-time (overall lit timeout)
  * --max-failures

This is part of a larger effort to ensure that all discovered tests are
properly accounted for.

Add test for overall lit timeout feature (`--max-time` option) to
observe skipped tests.  Extend test for `--max-failures` option.

Reviewed By: jdenny

Differential Revision: https://reviews.llvm.org/D77819
2020-04-10 15:13:30 -07:00