Jenkins sometimes starts a new working directory by appending @2 (or
incrementing the number if the @n suffix is already there). This causes
several clang tests to fail as:
s@INPUT_DIR@%/S/Inputs@g
gets expanded to the invalid:
s@INPUT_DIR@/path/to/workdir@2/Inputs@g
~~~~~~~~~~
where the part marked with ~'s is interpreted as the flags. These are
invalid and the test fails.
Previous fixes simply exchanged the @ character for another like | but
that's just moving the problem. Address it by adding an expansion that
escapes the @ character we're using as a delimiter as well as other magic
characters in the replacement of sed's s@@@.
There's still room for expansions to cause trouble though. One I ran into
while testing this was that having a directory called foo@bar causes lots
of `CHECK-NOT: foo` directives to match. There's also things like
directories containing `\1`
Tests go through the following stages:
*) discovered
*) filtered
*) executed
Only executed tests have a result (e.g., PASS, FAIL, XFAIL, etc.). See
"result codes" in Test.py.
Reviewed By: rnk
Differential Revision: https://reviews.llvm.org/D70612
Fail early, when we discover no tests at all, or filter out all of them.
There is also `--allow-empty-runs` to disable test to allow workflows
like `LIT_FILTER=abc ninja check-all`. Apparently `check-all` invokes
lit multiple times if certain projects are enabled, which would produce
unwanted "empty runs". Specify via `LIT_OPTS=--allow-empty-runs`.
There are 3 causes for empty runs:
1) No tests discovered. This is always an error. Fix test suite config
or command line.
2) All tests filtered out. This is an error by default, but can be
suppressed via `--alow-empty-runs`. Should prevent accidentally
passing empty runs, but allow the workflow above.
3) The number of shards is greater than the number of tests. Currently,
this is never an error. Personally, I think we should consider
making this an error by default; if this happens, you are doing
something wrong. I added a warning but did not change the behavior,
since this warrants more discussion.
Reviewed By: atrick, jdenny
Differential Revision: https://reviews.llvm.org/D70105
lit's test suite calls lit multiple times for various sample test
suites. `FILECHECK_OPTS` is safe for FileCheck calls in lit's test
suite. It's not safe for FileCheck calls in the sample test suites,
whose output affects the results of lit's test suite.
Without this patch, only one such sample test suite is protected from
`FILECHECK_OPTS`, and currently `shtest-shell.py` breaks with
`FILECHECK_OPTS=-vv`. Moreover, it's hard to predict the future,
especially false passes. Thus, this patch protects all existing and
future sample test suites from `FILECHECK_OPTS` (and the deprecated
`FILECHECK_DUMP_INPUT_ON_FAILURE`).
Reviewed By: probinson
Differential Revision: https://reviews.llvm.org/D65156
Without this patch, when using lit's internal shell, if `not` on a lit
RUN line calls `env`, `diff`, or any of the other in-process shell
builtins that lit implements, lit accidentally searches for the latter
as an external executable. What's worse is that works fine when a
developer is testing on a platform where those executables are
available and behave as expected, but it then breaks on other
platforms.
`not` seems useful for some builtins, such as `diff`, so this patch
supports such uses. `not --crash` does not seem useful for builtins,
so this patch diagnoses such uses. In all cases, this patch ensures
shell builtins are found behind any sequence of `env` and `not`
commands.
`not` calling `env` calling an external command appears useful when
the `env` and external command are part of a lit substitution, as in
D65156. This patch supports that by looking through any sequence of
`env` and `not` commands, building the environment from the `env`s,
and storing the `not`s. The `not`s are then added back to the command
line without the `env`s to execute externally. This avoids the need
to replicate the `not` implementation, in particular the `--crash`
option, in lit.
Reviewed By: rnk
Differential Revision: https://reviews.llvm.org/D66531
The static analyzer's scan-build script is critical infrastructure but
is not well tested. To start to address this, add a new test directory under
tests/Analysis for scan-build lit tests and seed it with several tests. The
goal is that future scan-build changes will be accompanied by corresponding
tests.
Differential Revision: https://reviews.llvm.org/D69781
Without this patch, when using lit's internal shell, if `env` on a lit
RUN line calls `env`, lit accidentally searches for the latter as an
external executable. What's worse is that works fine when a developer
is testing on a platform where `env` is available and behaves as
expected, but it then breaks on other platforms.
`env` calling `env` can make sense if one such `env` is within a lit
substitution, as in D65156 and D65121. This patch ensures that lit
executes both as internal commands.
Reviewed By: probinson, mgorny, rnk
Differential Revision: https://reviews.llvm.org/D65697
Without this patch, when using lit's internal shell, if `env` on a lit
RUN line calls `cd`, `mkdir`, or any of the other in-process shell
builtins that lit implements, lit accidentally searches for the latter
as an external executable.
This patch puts such builtins in a map so that boilerplate for them
need be implemented only once. This patch moves that handling after
processing of `env` so that `env` calling such a builtin can be
detected. Finally, because such calls appear to be useless, this
patch takes the safe approach of diagnosing them rather than
supporting them.
Reviewed By: probinson, mgorny, rnk
Differential Revision: https://reviews.llvm.org/D66506
When using lit's internal shell, RUN lines like the following
accidentally execute an external `diff` instead of lit's internal
`diff`:
```
# RUN: program | diff file -
```
Such cases exist now, in `clang/test/Analysis` for example. We are
preparing patches to ensure lit's internal `diff` is called in such
cases, which will then fail because lit's internal `diff` doesn't
recognize `-` as a command-line option. This patch adds support for
`-` to mean stdin.
Reviewed By: probinson, rnk
Differential Revision: https://reviews.llvm.org/D67643
When using lit's internal shell, RUN lines like the following
accidentally execute an external `diff` instead of lit's internal
`diff`:
```
# RUN: program | diff file -
# RUN: not diff file1 file2 | FileCheck %s
```
Such cases exist now, in `clang/test/Analysis` for example. We are
preparing patches to ensure lit's internal `diff` is called in such
cases, which will then fail because lit's internal `diff` cannot
currently be used in pipelines and doesn't recognize `-` as a
command-line option.
To enable pipelines, this patch moves lit's `diff` implementation into
an out-of-process script, similar to lit's `cat` implementation. A
follow-up patch will implement `-` to mean stdin.
Also, when lit's `diff` prints differences to stdout in Windows, this
patch ensures it always terminate lines with `\n` not `\r\n`. That
way, strict FileCheck directives checking the `diff` output succeed in
both Linux and Windows. This wasn't an issue when `diff` was internal
to lit because `diff` didn't then write to the true stdout, which is
where the `\n` -> `\r\n` conversion happened in Python.
Reviewed By: probinson, stella.stamenova
Differential Revision: https://reviews.llvm.org/D66574
Do not add user-site packages directory to the python search path.
This avoids test failures if there's an incompatible lit module installed
inside the user-site packages directory, as it gets prioritized over the lit
from the PYTHONPATH.
Large timeout values (one year, positive infinity) trip up Python on
Windows with "OverflowError: timeout value is too large". One week
seems to work and is still large enough in practice.
Thanks to Simon Pilgrim for helping me test this.
https://reviews.llvm.org/rL375171
llvm-svn: 375264
Avoid sending back the whole run.Test object (which needs to be pickled)
from the worker process when we are only interested in the test result.
llvm-svn: 375262
This will allow us to serialize just the result object instead of the
whole lit.Test object back from the worker to the main lit process.
llvm-svn: 375195
We always want to use a deadline when calling `result.await`. Let's
synthesize an artificial deadline (now plus one year) to simplify code
and do less busy waiting.
Thanks to Reid Kleckner for diagnosing that a deadline for of "positive
infinity" does not work with Python 3 anymore. See commit:
4ff1e34b60
I tested this patch with Python 2 and Python 3.
llvm-svn: 375165
Python on Windows raises this OverflowError:
gotit = waiter.acquire(True, timeout)
OverflowError: timestamp too large to convert to C _PyTime_t
So it seems this API behave the same way on every OS.
Also reverts the dependent commit a660dc590a.
llvm-svn: 375143
We always want to use a deadline when calling `result.await`. Let's
synthesize an artificial deadline (positive infinity) to simplify code
and do less busy waiting.
llvm-svn: 375129
When using lit's internal shell, RUN lines like the following
accidentally execute an external `diff` instead of lit's internal
`diff`:
```
# RUN: program | diff file -
```
Such cases exist now, in `clang/test/Analysis` for example. We are
preparing patches to ensure lit's internal `diff` is called in such
cases, which will then fail because lit's internal `diff` doesn't
recognize `-` as a command-line option. This patch adds support for
`-` to mean stdin.
Reviewed By: probinson, rnk
Differential Revision: https://reviews.llvm.org/D67643
llvm-svn: 375116
When using lit's internal shell, RUN lines like the following
accidentally execute an external `diff` instead of lit's internal
`diff`:
```
# RUN: program | diff file -
# RUN: not diff file1 file2 | FileCheck %s
```
Such cases exist now, in `clang/test/Analysis` for example. We are
preparing patches to ensure lit's internal `diff` is called in such
cases, which will then fail because lit's internal `diff` cannot
currently be used in pipelines and doesn't recognize `-` as a
command-line option.
To enable pipelines, this patch moves lit's `diff` implementation into
an out-of-process script, similar to lit's `cat` implementation. A
follow-up patch will implement `-` to mean stdin.
Reviewed By: probinson, stella.stamenova
Differential Revision: https://reviews.llvm.org/D66574
llvm-svn: 375114
* Remove outdated precautions for Python versions < 2.7
* Remove dead code related to `maxIndividualTestTime` option
* Move printing of test and result summary out of main into its own
function
Reviewed By: rnk
Differential Revision: https://reviews.llvm.org/D68847
llvm-svn: 375046
Using GNU diff, `--strip-trailing-cr` removes a `\r` appearing before
a `\n` at the end of a line. Without this patch, lit's internal diff
only removes `\r` if it appears as the last character. That seems
useless. This patch fixes that.
This patch also adds `--strip-trailing-cr` to some tests that fail on
Windows bots when D68664 is applied. Based on what I see in the bot
logs, I think the following is happening. In each test there, lit
diff is comparing a file with `\r\n` line endings to a file with `\n`
line endings. Without D68664, lit diff reads those files in text
mode, which in Windows causes `\r\n` to be replaced with `\n`.
However, with D68664, lit diff reads the files in binary mode instead
and thus reports that every line is different, just as GNU diff does
(at least under Ubuntu). Adding `--strip-trailing-cr` to those tests
restores the previous behavior while permitting the behavior of lit
diff to be more like GNU diff.
Reviewed By: rnk
Differential Revision: https://reviews.llvm.org/D68839
llvm-svn: 375020
As suggested by rnk at D67643#1673043, instead of reading files
multiple times until an appropriate encoding is found, read them once
as binary, and then try to decode what was read.
For Python >= 3.5, don't fail when attempting to decode the
`diff_bytes` output in order to print it.
Avoid failures for Python 2.7 used on some Windows bots by
transforming diff output with `lit.util.to_string` before writing it
to stdout.
Finally, add some tests for encoding handling.
Reviewed By: rnk
Differential Revision: https://reviews.llvm.org/D68664
llvm-svn: 375018
This essentially reverts a commit [1] that removed the adaptor for
Python unittests. The code has been slightly refactored to make it more
additive: all code is contained in LitTestCase.py.
Usage sites will require a small adaption:
```
[old]
import lit.discovery
...
test_suite = lit.discovery.load_test_suite(...)
[new]
import lit.LitTestCase
...
test_suite = lit.LitTestCase.load_test_suite(...)
```
This was put back on request by Daniel Dunbar, since I wrongly assumed
that the functionality is unused. At least llbuild still uses this [2].
[1] 70ca752ccf
[2] https://github.com/apple/swift-llbuild/blob/master/utils/Xcode/LitXCTestAdaptor/LitTests.py#L16
Reviewed By: ddunbar
Differential Revision: https://reviews.llvm.org/D69002
llvm-svn: 374947
The goal is to have 100% fidelity in clang-scan-deps behavior when
--analyze is present in compilation command.
At the same time I don't want to break clang-tidy which expects
__static_analyzer__ macro defined as built-in.
I introduce new cc1 options (-setup-static-analyzer) that controls
the macro definition and is conditionally set in driver.
Differential Revision: https://reviews.llvm.org/D68093
llvm-svn: 374815
When using lit's internal shell, RUN lines like the following
accidentally execute an external `diff` instead of lit's internal
`diff`:
```
# RUN: program | diff -U1 file -
```
Such cases exist now, in `clang/test/Analysis` for example. We are
preparing patches to ensure lit's internal `diff` is called in such
cases, which will then fail because lit's internal `diff` doesn't
recognize `-U` as a command-line option. This patch adds `-U`
support.
Reviewed By: rnk
Differential Revision: https://reviews.llvm.org/D68668
llvm-svn: 374814
On that decode, Windows bots fail with:
```
UnicodeEncodeError: 'ascii' codec can't encode characters in position 7-8: ordinal not in range(128)
```
That's the same error as before r374665 except it's now at the decode
before the write to stdout.
llvm-svn: 374666
I seem to have misread the bot logs on my last attempt. When lit's
internal diff runs on Windows under Python 2.7, it's text diffs not
binary diffs that need decoding to avoid this error when writing the
diff to stdout:
```
UnicodeEncodeError: 'ascii' codec can't encode characters in position 7-8: ordinal not in range(128)
```
There is no `decode` attribute in this case under Python 3.6.8 under
Ubuntu, so this patch checks for the `decode` attribute before using
it here. Hopefully nothing else is needed when `decode` isn't
available.
It might take a couple more attempts to figure out what error
handling, if any, is needed for this decoding.
llvm-svn: 374665
Based on the bot logs, when lit's internal diff runs on Windows, it
looks like binary diffs must be decoded also for Python 2.7.
Otherwise, writing the diff to stdout fails with:
```
UnicodeEncodeError: 'ascii' codec can't encode characters in position 7-8: ordinal not in range(128)
```
I did not need to decode using Python 2.7.15 under Ubuntu. When I do
it anyway in that case, `errors="backslashreplace"` fails for me:
```
TypeError: don't know how to handle UnicodeDecodeError in error callback
```
However, `errors="ignore"` works, so this patch uses that, hoping
it'll work on Windows as well.
This patch leaves `errors="backslashreplace"` for Python >= 3.5 as
there's no evidence yet that doesn't work and it produces more
informative binary diffs. This patch also adjusts some lit tests to
succeed for either error handler.
This patch adjusts changes introduced by D68664.
llvm-svn: 374657
Using GNU diff, `--strip-trailing-cr` removes a `\r` appearing before
a `\n` at the end of a line. Without this patch, lit's internal diff
only removes `\r` if it appears as the last character. That seems
useless. This patch fixes that.
This patch also adds `--strip-trailing-cr` to some tests that fail on
Windows bots when D68664 is applied. Based on what I see in the bot
logs, I think the following is happening. In each test there, lit
diff is comparing a file with `\r\n` line endings to a file with `\n`
line endings. Without D68664, lit diff reads those files with
Python's universal newlines support activated, causing `\r` to be
dropped. However, with D68664, lit diff reads the files in binary
mode instead and thus reports that every line is different, just as
GNU diff does (at least under Ubuntu). Adding `--strip-trailing-cr`
to those tests restores the previous behavior while permitting the
behavior of lit diff to be more like GNU diff.
Reviewed By: rnk
Differential Revision: https://reviews.llvm.org/D68839
llvm-svn: 374652
To avoid breaking some tests, D66574, D68664, D67643, and D68668
landed together. However, D68664 introduced an issue now addressed by
D68839, with which these are now all relanding.
Differential Revision: https://reviews.llvm.org/D68668
llvm-svn: 374651
To avoid breaking some tests, D66574, D68664, D67643, and D68668
landed together. However, D68664 introduced an issue now addressed by
D68839, with which these are now all relanding.
Differential Revision: https://reviews.llvm.org/D67643
llvm-svn: 374650
To avoid breaking some tests, D66574, D68664, D67643, and D68668
landed together. However, D68664 introduced an issue now addressed by
D68839, with which these are now all relanding.
Differential Revision: https://reviews.llvm.org/D68664
llvm-svn: 374649
To avoid breaking some tests, D66574, D68664, D67643, and D68668
landed together. However, D68664 introduced an issue now addressed by
D68839, with which these are now all relanding.
Differential Revision: https://reviews.llvm.org/D66574
llvm-svn: 374648
* Extract separate function for running tests from main
* Push single-usage imports to point of usage
* Remove unnecessary sys.exit(0) calls
Reviewed By: rnk
Differential Revision: https://reviews.llvm.org/D68836
llvm-svn: 374602
This change is purely mechanical. I will do further cleanups of
parameter usages.
Reviewed By: rnk
Differential Revision: https://reviews.llvm.org/D68830
llvm-svn: 374452
Bring back `--threads` option which was lost in the move of the
command line argument parsing code to cl_arguments.py. Update docs
since `--workers` is preferred.
llvm-svn: 374432
When using lit's internal shell, RUN lines like the following
accidentally execute an external `diff` instead of lit's internal
`diff`:
```
# RUN: program | diff -U1 file -
```
Such cases exist now, in `clang/test/Analysis` for example. We are
preparing patches to ensure lit's internal `diff` is called in such
cases, which will then fail because lit's internal `diff` doesn't
recognize `-U` as a command-line option. This patch adds `-U`
support.
Reviewed By: rnk
Differential Revision: https://reviews.llvm.org/D68668
llvm-svn: 374392
When using lit's internal shell, RUN lines like the following
accidentally execute an external `diff` instead of lit's internal
`diff`:
```
# RUN: program | diff file -
```
Such cases exist now, in `clang/test/Analysis` for example. We are
preparing patches to ensure lit's internal `diff` is called in such
cases, which will then fail because lit's internal `diff` doesn't
recognize `-` as a command-line option. This patch adds support for
`-` to mean stdin.
Reviewed By: probinson, rnk
Differential Revision: https://reviews.llvm.org/D67643
llvm-svn: 374390
As suggested by rnk at D67643#1673043, instead of reading files
multiple times until an appropriate encoding is found, read them once
as binary, and then try to decode what was read.
For python >= 3.5, don't fail when attempting to decode the
`diff_bytes` output in order to print it.
Finally, add some tests for encoding handling.
Reviewed By: rnk
Differential Revision: https://reviews.llvm.org/D68664
llvm-svn: 374389
When using lit's internal shell, RUN lines like the following
accidentally execute an external `diff` instead of lit's internal
`diff`:
```
# RUN: program | diff file -
# RUN: not diff file1 file2 | FileCheck %s
```
Such cases exist now, in `clang/test/Analysis` for example. We are
preparing patches to ensure lit's internal `diff` is called in such
cases, which will then fail because lit's internal `diff` cannot
currently be used in pipelines and doesn't recognize `-` as a
command-line option.
To enable pipelines, this patch moves lit's `diff` implementation into
an out-of-process script, similar to lit's `cat` implementation. A
follow-up patch will implement `-` to mean stdin.
Reviewed By: probinson, stella.stamenova
Differential Revision: https://reviews.llvm.org/D66574
llvm-svn: 374388
Move progress display to separate file. Simplify some code paths.
Decouple from other components via progress callback. Remove unused
`_Display` class.
Reviewed By: serge-sans-paille
Differential Revision: https://reviews.llvm.org/D68525
llvm-svn: 374194
Lit has a "quiet" option, -q, which is documented to "suppress no
error output". Previously, LitConfig displayed notes and warnings when
the quiet option was specified. The result was that it was not
possible to get only pertinent file/line information to be used by an
editor to jump to the location where checks were failing without
passing a number of unhelpful locations first. Here, the
implementations of LitConfig.note and LitConfig.warning are modified
to account for the quiet flag and avoid displaying if the flag has
indeed been set.
Patch by Nate Chandler
Reviewed by yln
Differential Revision: https://reviews.llvm.org/D68044
llvm-svn: 374009
In the past, lit used threads to run tests in parallel. Today we use
`multiprocessing.Pool`, which uses processes. Let's stay more abstract
and use "worker" everywhere.
Reviewed By: rnk
Differential Revision: https://reviews.llvm.org/D68475
llvm-svn: 373794
When using lit's internal shell, RUN lines like the following
accidentally execute an external `diff` instead of lit's internal
`diff`:
```
# RUN: program | diff file -
# RUN: not diff file1 file2 | FileCheck %s
```
Such cases exist now, in `clang/test/Analysis` for example. We are
preparing patches to ensure lit's internal `diff` is called in such
cases, which will then fail because lit's internal `diff` cannot
currently be used in pipelines and doesn't recognize `-` as a
command-line option.
To enable pipelines, this patch moves lit's `diff` implementation into
an out-of-process script, similar to lit's `cat` implementation. A
follow-up patch will implement `-` to mean stdin.
Reviewed By: probinson, stella.stamenova
Differential Revision: https://reviews.llvm.org/D66574
llvm-svn: 372035
Without this patch, failing to provide a subcommand to lit's internal
`env` results in either a python `IndexError` or an attempt to execute
the final `env` argument, such as `FOO=1`, as a command. This patch
diagnoses those cases with a more helpful message.
Reviewed By: stella.stamenova
Differential Revision: https://reviews.llvm.org/D66482
llvm-svn: 369620
This patch extends lit's test suite to check that lit's internal shell
doesn't accidentally execute internal commands as external commands.
It does so by putting fake failing versions of those commands in
`PATH` while the entire lit test suite is running. Without the fixes
in D65697 but with its tests, this approach catches accidental
external `env` calls.
Reviewed By: probinson
Differential Revision: https://reviews.llvm.org/D66293
llvm-svn: 369309
Summary:
'OSI Approved :: Apache-2.0 with LLVM exception' is not a valid
classifier. 'OSI Approved :: Apache Software License' is the closest
fit for the new license, so we've decided to use this one.
The classifiers seem to only be used for searching on the pypi website,
so this does not actually change the license of the code.
We still pass 'Apache-2.0 with LLVM exception' as the license to setup(),
and this appears alongside the classifier on the pypi webpage for lit.
Reviewers: chandlerc, ddunbar, joerg
Reviewed By: joerg
Subscribers: delcypher, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D65762
llvm-svn: 368315
Without this patch, the internal `env` command removes `env` and its
args from the command line while parsing it. This patch modifies a
copy instead so that the original command line is printed.
Reviewed By: stella.stamenova, rnk
Differential Revision: https://reviews.llvm.org/D65624
llvm-svn: 367752
Put the main test script in the right directory, and fix a python bug
in a local script.
Reviewed By: stella.stamenova
Differential Revision: https://reviews.llvm.org/D65623
llvm-svn: 367751
Summary:
This change updates the lit.cfg file to use llvm_config when it is available, but when it is not, it directly modifies the config object. This makes it possible to run the lit tests standalone without having built llvm (as long as the correct binaries are present in the path such as FileCheck and not).
Because the lit tests don't take a hard dependency on llvm_config, some features such as system-windows have to have definitions in lit's cfg file as well. This is a potential issue as the os features sometimes change names (for example, we went from windows to system-windows, etc.). This can cause drift between lit's tests and the rest of the llvm tests.
Reviewers: probinson, mgorny
Reviewed By: mgorny
Subscribers: delcypher, llvm-commits, asmith
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D65674
llvm-svn: 367730
Set environment variables to empty values rather than attempting
to unset them via 'env -u', in order to fix NetBSD test regression
caused by r366980. POSIX does not guarantee that env(1) supports '-u'
option, and indeed NetBSD env(1) does not support it.
Differential Revision: https://reviews.llvm.org/D65335
llvm-svn: 367123
lit's test suite calls lit multiple times for various sample test
suites. `FILECHECK_OPTS` is safe for FileCheck calls in lit's test
suite. It's not safe for FileCheck calls in the sample test suites,
whose output affects the results of lit's test suite.
Without this patch, only one such sample test suite is protected from
`FILECHECK_OPTS`, and I admit I haven't discovered other cases for
which I can produce false failures using `FILECHECK_OPTS`. However,
it's hard to predict the future, especially false passes. Thus, this
patch protects all existing and future sample test suites from
`FILECHECK_OPTS` (and the deprecated
`FILECHECK_DUMP_INPUT_ON_FAILURE`).
Reviewed By: probinson
Differential Revision: https://reviews.llvm.org/D65156
llvm-svn: 366980
Summary:
On AIX psutil can run into problems with permissions to read the process
tree, which causes problems for python timeout tests which need to kill off
a test and it's children.
This patch adds a workaround by invoking shell via subprocess and using a
platform specific option to ps to list all the descendant processes so we can
kill them. We add some checks so lit can tell whether timeout tests are
supported with out exposing whether we are utilizing the psutil
implementation or the alternative.
Reviewers: hubert.reinterpretcast, andusy, davide, delcypher
Reviewed By: delcypher
Subscribers: davide, delcypher, christof, lldb-commits, libcxx-commits, llvm-commits
Tags: #lldb, #libc, #llvm
Differential Revision: https://reviews.llvm.org/D64251
llvm-svn: 366912
Summary:
This improves readability of LIT output: previously
error messages gets emitted that say that there was no error:
error: command reached timeout: False
Patch by Alexey Sachkov.
Reviewers: ddunbar, mgorny, modocache
Reviewed By: mgorny
Subscribers: delcypher, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D64240
llvm-svn: 365895
Similar to `FILECHECK_OPTS` for FileCheck, `LIT_OPTS` makes it easy to
adjust lit behavior when running the test suite via ninja. For
example:
```
$ LIT_OPTS='--time-tests -vv --filter=threadprivate' \
ninja check-clang-openmp
```
Reviewed By: probinson
Differential Revision: https://reviews.llvm.org/D64135
llvm-svn: 365313
When running LLDB lit tests on Windows, the system selects a debug version
of Python, which was issuing lots of ResourceWarnings about files that
weren't closed. There are two kinds of them, and each test triggered one
of each.
This patch fixes one kind by ensuring TestRunner explicitly close the
temporary files created for routing stderr. This is important on Windows
but has no net effect on Posix systems.
The remaining ResourceWarnings are more elusive; the bug may lie in
the Python library subprocess.py, and it may be Windows-specific.
Differential Revision: https://reviews.llvm.org/D63102
llvm-svn: 363700
Ensure that the bash script written by lit TestRunner is open with UTF-8
encoding when using Python 3. Otherwise, attempt to write non-ASCII
characters causes UnicodeEncodeError. This happened e.g. with
the following LLD test:
UNRESOLVED: lld :: ELF/format-binary-non-ascii.s (657 of 2119)
******************** TEST 'lld :: ELF/format-binary-non-ascii.s' FAILED ********************
Exception during script execution:
Traceback (most recent call last):
File "/home/mgorny/llvm-project/llvm/utils/lit/lit/worker.py", line 63, in _execute_test
result = test.config.test_format.execute(test, lit_config)
File "/home/mgorny/llvm-project/llvm/utils/lit/lit/formats/shtest.py", line 25, in execute
self.execute_external)
File "/home/mgorny/llvm-project/llvm/utils/lit/lit/TestRunner.py", line 1644, in executeShTest
res = _runShTest(test, litConfig, useExternalSh, script, tmpBase)
File "/home/mgorny/llvm-project/llvm/utils/lit/lit/TestRunner.py", line 1590, in _runShTest
res = executeScript(test, litConfig, tmpBase, script, execdir)
File "/home/mgorny/llvm-project/llvm/utils/lit/lit/TestRunner.py", line 1157, in executeScript
f.write('{ ' + '; } &&\n{ '.join(commands) + '; }')
UnicodeEncodeError: 'ascii' codec can't encode character '\xa3' in position 274: ordinal not in range(128)
Differential Revision: https://reviews.llvm.org/D63254
llvm-svn: 363388
Summary:
This test fails to link shared libraries because tries to run
a copied version of clang-check to see if the mock version of libcxx
in the same directory can be loaded dynamically. Since the test is
specifically designed not to look in the default just-built lib
directory, it must be disabled when building with
BUILD_SHARED_LIBS=ON.
Currently only disabling it on Darwin and basing it on the
enable_shared flag.
Reviewed By: ilya-biryukov
Tags: #clang, #llvm
Differential Revision: https://reviews.llvm.org/D61697
llvm-svn: 363298
In LLDB, where tests run with the debug version of Python, we get a
series of deprecation warnings because escape sequences like `\(` are
being treated as part of the string literal rather than an escape for
the regexp pattern.
NFC intended.
Differential Revision: https://reviews.llvm.org/D62882
llvm-svn: 362846
Summary: This also normalizes the config feature that represents the windows platform to "system-windows" as opposed to having both "windows" and "system-windows"
Reviewers: asmith, probinson
Subscribers: delcypher, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D61798
llvm-svn: 361998
zlib/nozlib, asan/not_asan, msan/not_msan, ubsan/not_ubsan.
We still have two other ways to express the absence of a feature.
First, we have the '!' operator to invert the sense of a keyword. For
example, given a feature that depends on zlib being unavailable, its
test can say:
REQUIRES: !zlib
Second, if a test doesn't play well with some features, such as
sanitizers, that test can say:
UNSUPPORTED: asan, msan
The different ways of writing these exclusions both have the same
technical effect, but have different implications to the reader.
llvm-svn: 360603
Summary:
Various tests in the `lit` testing suite expect specific return codes
and forms of diagnostic message from utility programs. As per
POSIX.1-2017 XCU Section 1.4, Utility Description Defaults, "[the]
format of diagnostic messages for most utilities is unspecified".
The STDERR subsections of the `cat` and `wc` utilities merely indicate
that "[the] standard error shall be used only for diagnostic messages".
The corresponding EXIT STATUS subsections merely indicate, with regard
to errors, an exit value of >0.
The affected tests are updated to accept the applicable diagnostic
message as produced by the utilities on AIX. The exit value is
normalized using `not` as necessary.
Reviewers: xingxue, sfertile, jasonliu
Reviewed By: xingxue
Subscribers: delcypher, jsji, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D60553
llvm-svn: 359690
Add a 'target-x86' and 'target-x86_64' feature sthat indicates that
the default target is 32-bit or 64-bit x86, appropriately. Combined
with 'native' feature, we're going to use this to control x86-specific
LLDB native process tests.
Differential Revision: https://reviews.llvm.org/D60474
llvm-svn: 358177
Use ctypes to call into SHFileOperationW with the extended NT path to allow us
to remove paths which exceed 261 characters on Windows. This functionality is
exercised by swift's test suite.
llvm-svn: 357778
This enables lit to work with unicode file names via mkdir, rm, and redirection.
Lit still uses utf-8 internally, but converts to utf-16 on Windows, or just utf-8
bytes on everything else.
Committed on behalf of Jason Mittertreiner
Differential Revision: https://reviews.llvm.org/D56754
llvm-svn: 355122
Julian Lettner