Update the CHECK lines in the shtest-timeout.py lit test to account for
the current output. The output has been changed in r271610 without
adjusting the tests.
Differential Revision: https://reviews.llvm.org/D25236
llvm-svn: 284057
Summary:
The Python file `utils/lit/lit/ShUtil.py` contains:
1. Logic used by lit itself
2. A set of unit tests for that logic, which can be run by invoking
`python utils/lit/lit/ShUtil.py`
Move these unit tests to a `tests/unit` subdirectory of lit, and run
the tests as part of lit's test suite. This ensures that, should the
lit test suite be included in LLVM's own regression test suite, these
unit tests will also be run.
(Instructions on how to run lit's test suite can be found in
`utils/lit/README.txt`.)
Reviewers: ddunbar, echristo, delcypher, beanz
Subscribers: mehdi_amini, llvm-commits
Differential Revision: https://reviews.llvm.org/D25411
llvm-svn: 283968
Summary:
optparse is deprecated in Python 2.7, which is the minimum version of
Python required to run the LLVM test suite. Replace its usage in lit
with argparse, optparse's 2.7 replacement module.
argparse has several benefits over optparse, but this commit does not
make use of those benefits yet. Instead, it simply uses the new API,
and attempts to keep the number of changes to a minimum.
Confirmed that lit's test suite, as well as LLVM's regression test suite,
still pass with these changes.
Patch By Brian Gesiak!
Reviewers: ddunbar, echristo, beanz, delcypher
Subscribers: llvm-commits, mehdi_amini
Differential Revision: https://reviews.llvm.org/D25173
llvm-svn: 283152
- This is primarily useful as a "fail fast" mode for lit, where it will stop
running tests after the first failure.
- Patch by Max Moiseev.
llvm-svn: 282452
- This will cause lit to automatically include the first 1K of data in
redirected output files when a command fails (previously if the command
failed, but the main point of the test was, say, a `FileCheck` later on, then
the log wasn't helpful in showing why the command failed).
llvm-svn: 272021
- This only applies to scripts executed by the _internal_ shell script
interpreter.
- This patch reworks the log to look more like a shell transcript, and be less
verbose (but in the interest of calling attention to the important parts).
Here is an example of the new format, for commands with/without failures and
with/without output:
```
$ true
$ echo hi
hi
$ false
note: command had no output on stdout or stderr
error: command failed with exit status 1
```
llvm-svn: 271610
Summary:
This patch adds a "REQUIRES-ANY" feature test that is disjunctive. This marks a test as `UNSUPPORTED` if none of the specified features are available.
Libc++ has the need to write feature test such as `// REQUIRES-ANY: c++98, c++03` when testing of behavior that is specific to older dialects but has since changed.
Reviewers: rnk, ddunbar
Subscribers: ddunbar, probinson, llvm-commits, cfe-commits
Differential Revision: http://reviews.llvm.org/D20757
llvm-svn: 271468
Summary:
Upstream googletest prints "Running main() from gtest_main.cc" to stdout prior
to running tests. LLVM removed that print statement in r61540. If a user were
to use lit to run tests that use upstream googletest, however, lit
reports "Running main()" as an invalid test name.
To avoid such a failure, add an extra conditional to `formats/googletest.py`.
Also add tests to demonstrate the modified behavior.
Reviewers: abdulras, ddunbar
Subscribers: ddunbar, llvm-commits, kastiglione
Differential Revision: http://reviews.llvm.org/D18606
llvm-svn: 265034
This reverts r257221.
This caused several build bot failures
* It looks like some of the tests don't work correctly under Windows
* It looks like the lit per test timeout tests fail
So I'm reverting for now. Once the above failures are fixed running
lit's tests can be enabled again.
llvm-svn: 257268
directy with ``make check-lit`` and are run as part of
``make check-all``.
In principle we should run lit's testsuite before testing LLVM using lit
so that any problems with lit get discovered before testing LLVM so we
can bail out early. However this implementation (``check-all`` runs all
tests together) seemed simpler and will still report failing lit tests.
Note that the tests and the configured ``lit.site.cfg`` have to be
copied into the build directory to avoid polluting the source tree.
llvm-svn: 257221
This should work with ShTest (executed externally or internally) and GTest
test formats.
To set the timeout a new option ``--timeout=`` has
been added which specifies the maximum run time of an individual test
in seconds. By default this 0 which causes no timeout to be enforced.
The timeout can also be set from a lit configuration file by modifying
the ``lit_config.maxIndividualTestTime`` property.
To implement a timeout we now require the psutil Python module if a
timeout is requested. This dependency is confined to the newly added
``lit.util.killProcessAndChildren()``. A note has been added into the
TODO document describing how we can remove the dependency on the
``pustil`` module in the future. It would be nice to remove this
immediately but that is a lot more work and Daniel Dunbar believes it is
better that we get a working implementation first and then improve it.
To avoid breaking the existing behaviour the psutil module will not be
imported if no timeout is requested.
The included testcases are derived from test cases provided by
Jonathan Roelofs which were in an previous attempt to add a per test
timeout to lit (http://reviews.llvm.org/D6584). Thanks Jonathan!
Reviewers: ddunbar, jroelofs, cmatthews, MatzeB
Subscribers: cmatthews, llvm-commits
Differential Revision: http://reviews.llvm.org/D14706
llvm-svn: 256471
Summary:
I spend some time trying to get the LIT test suite passing. Here are the changes that I needed to make on my machine.
I made the following changes for the following reasons.
1. google-test.py: The Google test format now checks for "[ PASSED ] 1 test." to check if a test passes.
2. discovery.py: The output appears in a different order on my machine than it did in the test.
3. unittest-adaptor.py: The output appears in a different order on my machine than it did in the test.
4. The classname is now formed differently in `getJUnitXML(...)`.
I'm not sure what is causing the output order to differ in discovery.py and unittest-adaptor.py. Does anybody have any thoughts?
Reviewers: ddunbar, danalbert, jroelofs
Reviewed By: jroelofs
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D9864
llvm-svn: 239663
We were already requiring 2.5, which meant that people on old linux distros
had to upgrade anyway.
Requiring python 2.6 will make supporting 3.X easier as we can use the 3.X
exception syntax.
According to the discussion on llvmdev, there is not much value is requiring
just 2.6, we may as well just require 2.7.
llvm-svn: 224129
- This is a work-in-progress and all details are subject to change, but I am
trying to build up support for allowing lit to be used as a driver for
performance tests (or other tests which might want to record information
beyond simple PASS/FAIL).
llvm-svn: 190535
- This aligns with how existing test suites end up wanting to use the local
config files, conceptually it makes sense to consider them to be inherited.
llvm-svn: 189885
- At least on OS X, it is important for correct behavior of /bin/[ that argv[0]
is passed as written, and not as the full executable path.
llvm-svn: 189559
- For whatever reason, we have a lot of test files with bogus unicode
characters. This patch allows those scripts to still be parsed on Python3 by
changing the parsing logic to work on binary files, and only require the
actual script commands to be convertible to ascii.
- This patch has been tweaked to now ensure that the command strings are not of
unicode type on Python 2.6-7.
llvm-svn: 188398
- For whatever reason, we have a lot of test files with bogus unicode
characters. This patch allows those scripts to still be parsed on Python3 by
changing the parsing logic to work on binary files, and only require the
actual script commands to be convertible to ascii.
llvm-svn: 188376
Michal Gorny