Improve consistency when printing test results:
Previously we were using different labels for group names (the header
for the list of, e.g., failing tests) and summary count lines. For
example, "Failing Tests"/"Unexpected Failures". This commit changes lit
to label things consistently.
Improve wording of labels:
When talking about individual test results, the first word in
"Unexpected Failures", "Expected Passes", and "Individual Timeouts" is
superfluous. Some labels contain the word "Tests" and some don't.
Let's simplify the names.
Before:
```
Failing Tests (1):
...
Expected Passes : 3
Unexpected Failures: 1
```
After:
```
Failed Tests (1):
...
Passed: 3
Failed: 1
```
Reviewed By: ldionne
Differential Revision: https://reviews.llvm.org/D77708
Without this patch, when using lit's internal shell, if `not` on a lit
RUN line calls `env`, `diff`, or any of the other in-process shell
builtins that lit implements, lit accidentally searches for the latter
as an external executable. What's worse is that works fine when a
developer is testing on a platform where those executables are
available and behave as expected, but it then breaks on other
platforms.
`not` seems useful for some builtins, such as `diff`, so this patch
supports such uses. `not --crash` does not seem useful for builtins,
so this patch diagnoses such uses. In all cases, this patch ensures
shell builtins are found behind any sequence of `env` and `not`
commands.
`not` calling `env` calling an external command appears useful when
the `env` and external command are part of a lit substitution, as in
D65156. This patch supports that by looking through any sequence of
`env` and `not` commands, building the environment from the `env`s,
and storing the `not`s. The `not`s are then added back to the command
line without the `env`s to execute externally. This avoids the need
to replicate the `not` implementation, in particular the `--crash`
option, in lit.
Reviewed By: rnk
Differential Revision: https://reviews.llvm.org/D66531
Without this patch, when using lit's internal shell, if `env` on a lit
RUN line calls `env`, lit accidentally searches for the latter as an
external executable. What's worse is that works fine when a developer
is testing on a platform where `env` is available and behaves as
expected, but it then breaks on other platforms.
`env` calling `env` can make sense if one such `env` is within a lit
substitution, as in D65156 and D65121. This patch ensures that lit
executes both as internal commands.
Reviewed By: probinson, mgorny, rnk
Differential Revision: https://reviews.llvm.org/D65697
Without this patch, when using lit's internal shell, if `env` on a lit
RUN line calls `cd`, `mkdir`, or any of the other in-process shell
builtins that lit implements, lit accidentally searches for the latter
as an external executable.
This patch puts such builtins in a map so that boilerplate for them
need be implemented only once. This patch moves that handling after
processing of `env` so that `env` calling such a builtin can be
detected. Finally, because such calls appear to be useless, this
patch takes the safe approach of diagnosing them rather than
supporting them.
Reviewed By: probinson, mgorny, rnk
Differential Revision: https://reviews.llvm.org/D66506
Without this patch, failing to provide a subcommand to lit's internal
`env` results in either a python `IndexError` or an attempt to execute
the final `env` argument, such as `FOO=1`, as a command. This patch
diagnoses those cases with a more helpful message.
Reviewed By: stella.stamenova
Differential Revision: https://reviews.llvm.org/D66482
llvm-svn: 369620
Without this patch, the internal `env` command removes `env` and its
args from the command line while parsing it. This patch modifies a
copy instead so that the original command line is printed.
Reviewed By: stella.stamenova, rnk
Differential Revision: https://reviews.llvm.org/D65624
llvm-svn: 367752
Put the main test script in the right directory, and fix a python bug
in a local script.
Reviewed By: stella.stamenova
Differential Revision: https://reviews.llvm.org/D65623
llvm-svn: 367751
Julian Lettner