Commit Graph

50 Commits

Author SHA1 Message Date
Nico Weber ecbf2b3496 fix doc typo to cycle bots 2020-08-06 21:02:41 -04:00
Varun Gandhi 417d3d495f [docs] [lit] Add a more helpful description for lit.py's -s flag.
Reviewed By: yln

Differential Revision: https://reviews.llvm.org/D82808
2020-07-28 14:36:03 -07:00
Mike Edwards 8cd117c24f [LIT] Correcting max-failures option in lit documentation. 2020-06-27 14:57:04 -07:00
Mike Edwards 972a73a347 [LIT] NFC adding max-failures option to lit documentation.
Differential Revision: https://reviews.llvm.org/D81337
2020-06-06 18:26:45 -07:00
Jon Roelofs 42bf0756d4 [docs] Fix :option: links 2020-04-25 16:19:02 -06:00
Jon Roelofs dc5c1fa882 [docs] Fix :option: links 2020-04-22 14:00:30 -06:00
Jon Roelofs b3f168274d [docs] Document lit's --timeout=N flag 2020-04-22 12:57:25 -06:00
Louis Dionne 8a42bf24ae [lit] Move the recursiveExpansionLimit setting to TestingConfig
The LitConfig is shared across the whole test suite. However, since
enabling recursive expansion can be a breaking change for some test
suites, it's important to confine the setting to test suites that
enable it explicitly.

Note that other issues were raised with the way recursiveExpansionLimit
operates. However, this commit simply moves the setting to the right
place -- the mechanism by which it works can be improved independently.

Differential Revision: https://reviews.llvm.org/D77415
2020-04-06 13:58:00 -04:00
Louis Dionne faf415a1de [lit] Recursively expand substitutions
This allows defining substitutions in terms of other substitutions. For
example, a %build substitution could be defined in terms of a %cxx
substitution as '%cxx %s -o %t.exe' and the script would be properly
expanded.

Differential Revision: https://reviews.llvm.org/D76178
2020-03-27 09:25:26 -04:00
Louis Dionne 83346a4077 [lit] NFC: Document missing result codes
These result codes already exist, but they were not documented. I assume
this is an oversight when adding these result codes.
2020-03-24 14:46:54 -04:00
Kazuaki Ishizaki f65d4aa960 [llvm] NFC: fix trivial typos in documents
Reviewers: hans, Jim

Reviewed By: Jim

Subscribers: jvesely, nhaehnle, mgorny, arphaman, bmahjour, kerbowa, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D73017
2020-01-22 11:32:51 +08:00
Daniel Sanders 82f3c5d4a6 [lit] Document the undocumented pre-defined substitutions 2019-12-04 14:25:12 -08:00
Julian Lettner b858895c85 [lit] Bring back `--threads` option alias
Bring back `--threads` option which was lost in the move of the
command line argument parsing code to cl_arguments.py.  Update docs
since `--workers` is preferred.

llvm-svn: 374432
2019-10-10 19:43:57 +00:00
Joel E. Denny 1602058c04 [lit] Parse command-line options from LIT_OPTS
Similar to `FILECHECK_OPTS` for FileCheck, `LIT_OPTS` makes it easy to
adjust lit behavior when running the test suite via ninja.  For
example:

```
$ LIT_OPTS='--time-tests -vv --filter=threadprivate' \
  ninja check-clang-openmp
```

Reviewed By: probinson

Differential Revision: https://reviews.llvm.org/D64135

llvm-svn: 365313
2019-07-08 12:18:40 +00:00
James Henderson a056684c33 [docs][tools] Add missing "program" tags to rst files
Sphinx allows for definitions of command-line options using
`.. option <name>` and references to those options via `:option:<name>`.
However, it looks like there is no scoping of these options by default,
meaning that links can end up pointing to incorrect documents. See for
example the llvm-mca document, which contains references to -o that,
prior to this patch, pointed to a different document. What's worse is
that these links appear to be non-deterministic in which one is picked
(on my machine, some references end up pointing to opt, whereas on the
live docs, they point to llvm-dwarfdump, for example).

The fix is to add the .. program <name> tag. This essentially namespaces
the options (definitions and references) to the named program, ensuring
that the links are kept correct.

Reviwed by: andreadb

Differential Revision: https://reviews.llvm.org/D63873

llvm-svn: 364538
2019-06-27 13:24:46 +00:00
Kuba Mracek a2e0c2462a [llvm] Document "%T" as deprecated in CommandGuide/lit.rst
Differential Revision: https://reviews.llvm.org/D48842

llvm-svn: 340677
2018-08-25 01:27:48 +00:00
Joel E. Denny 31b373963f [lit] Report line number for failed RUN command
(Relands r333584, reverted in 333592.)

When debugging test failures with -vv (or -v in the case of the
internal shell), this makes it easier to locate the RUN line that
failed.  For example, clang's test/Driver/linux-ld.c has 892 total RUN
lines, and clang's test/Driver/arm-cortex-cpus.c has 424 RUN lines
after concatenation for line continuations.

When reading the generated shell script, this also makes it easier to
locate the RUN line that produced each command.

To support reporting RUN line numbers in the case of the internal
shell, this patch extends the internal shell to support the null
command, ":", except pipelines are not supported.

To support reporting RUN line numbers in the case of windows cmd.exe
as the external shell, this patch extends -vv to set "echo on" instead
of "echo off" in bat files.  (Support for windows cmd.exe as a lit
external shell will likely be dropped later, but I found out too
late.)

Reviewed By: delcypher,	asmith, stella.stamenova, jmorse, lebedev.ri, rnk

Differential Revision: https://reviews.llvm.org/D44598

llvm-svn: 333614
2018-05-31 00:55:32 +00:00
Joel E. Denny 71792c741e Revert r333584: [lit] Report line number for failed RUN command
It breaks test-suite.

llvm-svn: 333592
2018-05-30 21:07:27 +00:00
Joel E. Denny b6423479a1 [lit] Report line number for failed RUN command
(Relands r330755 (reverted in r330848) with fix for PR37239.)

When debugging test failures with -vv (or -v in the case of the
internal shell), this makes it easier to locate the RUN line that
failed.  For example, clang's test/Driver/linux-ld.c has 892 total RUN
lines, and clang's test/Driver/arm-cortex-cpus.c has 424 RUN lines
after concatenation for line continuations.

When reading the generated shell script, this also makes it easier to
locate the RUN line that produced each command.

To support reporting RUN line numbers in the case of the internal
shell, this patch extends the internal shell to support the null
command, ":", except pipelines are not supported.

To support reporting RUN line numbers in the case of windows cmd.exe
as the external shell, this patch extends -vv to set "echo on" instead
of "echo off" in bat files.  (Support for windows cmd.exe as a lit
external shell will likely be dropped later, but I found out too
late.)

Reviewed By: delcypher,	asmith, stella.stamenova, jmorse, lebedev.ri, rnk

Differential Revision: https://reviews.llvm.org/D44598

llvm-svn: 333584
2018-05-30 19:42:27 +00:00
Reid Kleckner 1ca666886f Revert r330755 "[lit] Report line number for failed RUN command"
It is causing many tests to fail on Windows buildbots:
http://lab.llvm.org:8011/builders/clang-x64-ninja-win7/builds/10211

llvm-svn: 330848
2018-04-25 17:30:00 +00:00
Joel E. Denny 8a475307bd [lit] Report line number for failed RUN command
When debugging test failures with -vv (or -v in the case of the
internal shell), this makes it easier to locate the RUN line that
failed.  For example, clang's test/Driver/linux-ld.c has 892 total RUN
lines, and clang's test/Driver/arm-cortex-cpus.c has 424 RUN lines
after concatenation for line continuations.

When reading the generated shell script, this also makes it easier to
locate the RUN line that produced each command.

To support reporting RUN line numbers in the case of the internal
shell, this patch extends the internal shell to support the null
command, ":", except pipelines are not supported.

Reviewed By: asmith, delcypher

Differential Revision: https://reviews.llvm.org/D44598

llvm-svn: 330755
2018-04-24 18:43:25 +00:00
Jordan Rose e55d9a1403 Revert "lit.py: Add new %{shared_output(LABEL)} substitution"
This reverts r315697 and my ill-fated attempts to fix it on Windows.
I'll try again when I get access to a Windows machine.

llvm-svn: 315793
2017-10-14 04:01:27 +00:00
Jordan Rose 53be3224b2 lit.py: Add new %{shared_output(LABEL)} substitution
This refers to a temporary path that can be shared across all tests,
identified by a particular label. This can be used for things like
caches.

At the moment, the character set for the LABEL is limited to C
identifier characters, plus '-', '+', '=', and '.'. This is the same
set of characters currently allowed in REQUIRES clause identifiers.

llvm-svn: 315697
2017-10-13 16:12:23 +00:00
George Karpenkov dfa6c20f4b [lit] add a -vv option to echo all executed commands.
Debugging LIT scripts can be rather painful, as LIT directly does not
specify which line has failed.
Rather, FileCheck is expected to report the failing location, but it can
be often ambiguous if multiple commands are tested against the same
prefix.  This change adds a -vv option, which echoes all output.
Then detecting the error becomes straightforward: last printed line is
the failing one.

Of course, it could be desired to try to get failing line number
directly from bash, but it involves excessive hacks on older bash
versions (cf.
https://stackoverflow.com/questions/24398691/how-to-get-the-real-line-number-of-a-failing-bash-command)

Differential Revision: https://reviews.llvm.org/D35330

llvm-svn: 307938
2017-07-13 19:26:27 +00:00
George Karpenkov 22a402fb66 [lit] Modify LIT to accept environment variable LIT_FILTER to select tests.
This is especially useful when lit is invoked indirectly by the build
system, and additional arguments can not be easily specified.

Differential Revision: https://reviews.llvm.org/D35091

llvm-svn: 307339
2017-07-07 00:22:11 +00:00
Matthias Braun 6717a0ba03 lit.rst: Fix sphinx complaint about multiple option definitions
llvm-svn: 294646
2017-02-09 23:03:22 +00:00
David Bozier 9126f54285 [docs] cleanup documentation on lit substitutions
1. Added missing substitutions to the documentation in docs/TestingGuide.rst
2. Modified docs/CommandGuide/lit.rst to only document the "base" set of substitutions and to refer the reader to docs/TestingGuide.rst for more detailed info on substitutions.

Patch by bd1976llvm

Differential Revision: https://reviews.llvm.org/D29281

llvm-svn: 294586
2017-02-09 14:12:30 +00:00
Graydon Hoare ae5d7bb4f5 [lit] Support sharding testsuites, for parallel execution.
Summary:
This change equips lit.py with two new options, --num-shards=M and
--run-shard=N (set by default from env vars LIT_NUM_SHARDS and LIT_RUN_SHARD).

The options must be used together, and N must be in 1..M.

Together these options effect only test selection: they partition the testsuite
into M equal-sized "shards", then select only the Nth shard. They can be used
in a cluster of test machines to achieve a very crude (static) form of
parallelism, with minimal configuration work.

Reviewers: modocache, ddunbar

Reviewed By: ddunbar

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D28789

llvm-svn: 292417
2017-01-18 18:12:20 +00:00
Daniel Sanders 6151699b08 [lit] Document the 'available_features' member of the config object.
llvm-svn: 276744
2016-07-26 13:23:27 +00:00
Paul Robinson f88cc148b6 Document standard substitutions defined by lit.
Patch by Guilherme Bufolo!

Differential Revision: http://reviews.llvm.org/D18752

llvm-svn: 265314
2016-04-04 17:14:45 +00:00
Matthias Braun f5e2b8fdac CommandGuide/lit.rst: Document the new commandline option -a
llvm-svn: 252257
2015-11-06 01:13:40 +00:00
Eric Christopher 572e03a396 Fix "the the" in comments.
llvm-svn: 240112
2015-06-19 01:53:21 +00:00
Matthias Braun 219144e6a7 Lit: Allow overriding llvm tool paths+arguments, make -D an alias for --param
These changes allow usages where you want to pass an additional
commandline option to all invocations of a specific llvm tool. Example:

> llvm-lit -Dllc=llc -enable-misched -verify-machineinstrs

Differential Revision: http://reviews.llvm.org/D9487

llvm-svn: 236461
2015-05-04 21:36:36 +00:00
Jonathan Roelofs c990459b07 Fix docs typo regarding lit.local.cfg files
llvm-svn: 228499
2015-02-07 17:18:26 +00:00
Eric Fiselier 5b0e0e9436 [LIT] Remove documentation for method since it does not exist
llvm-svn: 216204
2014-08-21 18:52:58 +00:00
Eric Fiselier 6c53c869d0 [LIT]Correct name of global lit configuration object to be lit_config (not lit).
llvm-svn: 215695
2014-08-15 05:54:19 +00:00
Eric Fiselier c85f00a062 [lit] Add --show-xfail flag to LIT.
Summary:
This patch add a --show-xfail flag. If this flag is specified then each xfail test will be printed to output.
When it is not given xfail tests are ignored. Ignoring xfail tests is the current behavior.

This flag is meant to mirror the --show-unsupported flag that was recently added.

Reviewers: ddunbar, EricWF

Reviewed By: EricWF

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D4750

llvm-svn: 214609
2014-08-02 01:29:52 +00:00
Eric Fiselier 18fab4684d Add documentation for lit's --show-unsupported flag
llvm-svn: 214431
2014-07-31 20:11:13 +00:00
Daniel Dunbar c4477dfd92 [lit] Change --show-{tests,suites} to exit after printing.
- This is a more sensible behavior than printing and also running tests.

llvm-svn: 188009
2013-08-08 20:59:25 +00:00
Daniel Dunbar b647d5d21d [lit] Remove --repeat option, which wasn't that useful.
llvm-svn: 188008
2013-08-08 20:59:20 +00:00
Rafael Espindola 2a2a0973b8 Use pipefail when available.
This change makes test with RUN lines like
RUN: opt ... | FileCheck

fail if opt fails, even if it prints what FileCheck wants. Enabling this
found some interesting cases of broken tests that were not being noticed
because opt (or some other tool) was crashing late.

Pipefail is used when the shell supports it or when using the internal
python based tester.

llvm-svn: 187261
2013-07-26 22:32:58 +00:00
Sergey Matveev 8dbcb356f7 Fix incorrect parameter name in LIT docs.
llvm-svn: 182926
2013-05-30 12:37:52 +00:00
Dmitri Gribenko 9e4fc280f9 Documentation: remove more mentions of Tcl
Followup for r172836

llvm-svn: 172918
2013-01-19 20:35:18 +00:00
Dmitri Gribenko e260d3af52 Documentation for lit: more formatting: use 'option' and 'program' directives.
This enables cross-referencing and now '--' in option names are no more turned into en dashes.

llvm-svn: 168906
2012-11-29 18:03:08 +00:00
Dmitri Gribenko 6e303f7615 Documentation for lit: formatting improvements.
llvm-svn: 168902
2012-11-29 17:05:34 +00:00
Eli Bendersky 9d07f94863 ReST-ify some markup in the LIT document
llvm-svn: 168345
2012-11-20 00:26:08 +00:00
Daniel Dunbar a3d9cabf8f lit: Rename the valgrind leaks feature to match what is currently used
(vg_leak).

llvm-svn: 166306
2012-10-19 20:29:30 +00:00
Daniel Dunbar 519a349c8a lit: Add 'valgrind' and 'valgrind-leaks' features when valgrind is used.
- These can be used with the XFAIL options.

llvm-svn: 166303
2012-10-19 20:12:00 +00:00
Benjamin Kramer bde9176663 Fix typos found by http://github.com/lyda/misspell-check
llvm-svn: 157885
2012-06-02 10:20:22 +00:00
Daniel Dunbar 8f4a8a636c [docs] Add ReST version of all the man pages.
- The POD versions are slated for execution, but are still around until
   llvm.org machinery is in place.

llvm-svn: 156384
2012-05-08 16:50:35 +00:00