Commit Graph

309 Commits

Author SHA1 Message Date
Rafael Espindola 169758b50e Handle ASAN_OPTIONS and UBSAN_OPTIONS in TestingConfig.py
Currently they are passed to tests of llvm itself, but not, for example, lld.

With this patch the options are visible in every test.

llvm-svn: 221198
2014-11-03 23:04:56 +00:00
Rafael Espindola 6ecaf237ef refactor duplicated code. NFC.
llvm-svn: 221191
2014-11-03 22:17:49 +00:00
Rafael Espindola f0c7dbd1c8 [lit] Forward LD_PRELOAD to tests.
With this patch I can use asan to test the gold plugin without having
to build gold itself with asan.

llvm-svn: 221183
2014-11-03 21:24:43 +00:00
Daniel Dunbar 02306e1cdb [lit] Fix Python-3 compatibility, patch by Dan Liew.
llvm-svn: 220357
2014-10-22 01:26:06 +00:00
Daniel Dunbar 21297a532e [lit] Bump version number.
llvm-svn: 220355
2014-10-22 00:48:23 +00:00
Daniel Dunbar dfbb5c1b76 Fix ShTest parsing error when a keyword line doesn't end with a newline.
llvm-svn: 220354
2014-10-22 00:34:31 +00:00
Dan Liew 2f9e28c84f Teach Lit to catch OSError exceptions when creating a process during the
execution of a shell command. This can happen for example if the
``RUN:`` line calls a python script which can work correctly under
Linux/OSX but will not work under Windows. A more useful error message
is now shown rather than an unhelpful backtrace.

llvm-svn: 220227
2014-10-20 20:14:28 +00:00
Peter Collingbourne 82e3e373b3 Initial version of Go bindings.
This code is based on the existing LLVM Go bindings project hosted at:
https://github.com/go-llvm/llvm

Note that all contributors to the gollvm project have agreed to relicense
their changes under the LLVM license and submit them to the LLVM project.

Differential Revision: http://reviews.llvm.org/D5684

llvm-svn: 219976
2014-10-16 22:48:02 +00:00
Jordan Rose ef78038775 [lit] Parse all strings as UTF-8 rather than ASCII.
As far as I can tell UTF-8 has been supported since the beginning of Python's
codec support, and it's the de facto standard for text these days, at least
for primarily-English text. This allows us to put Unicode into lit RUN lines.

rdar://problem/18311663

llvm-svn: 217688
2014-09-12 16:46:05 +00:00
Eric Fiselier 5ccd9e029f [LIT] Move display of unsupported and xfail tests to summary.
Summary:
This patch changes the way xfail and unsupported tests are displayed. 
This output is only displayed when the --show-unsupported/--show-xfail flags are passed to lit.

Currently xfail/unsupported tests are printed during the run of the test-suite. I think its better to display this information during the summary instead.
This patch removes the printing of these tests from when they are run to the summary.


Reviewers: ddunbar, EricWF

Reviewed By: EricWF

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D4842

llvm-svn: 215809
2014-08-16 02:16:25 +00:00
Eric Fiselier c85f00a062 [lit] Add --show-xfail flag to LIT.
Summary:
This patch add a --show-xfail flag. If this flag is specified then each xfail test will be printed to output.
When it is not given xfail tests are ignored. Ignoring xfail tests is the current behavior.

This flag is meant to mirror the --show-unsupported flag that was recently added.

Reviewers: ddunbar, EricWF

Reviewed By: EricWF

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D4750

llvm-svn: 214609
2014-08-02 01:29:52 +00:00
Eric Fiselier 5cfa2e4620 [lit] Add --show-unsupported flag to LIT
llvm-svn: 213227
2014-07-17 05:53:00 +00:00
Hans Wennborg a3d77e7348 lit: simplify population of the actual_inputs array
Add all inputs to the array, except those starting with @, which
are treated as response files and expanded.

llvm-svn: 211119
2014-06-17 18:17:46 +00:00
Hans Wennborg f9484b24b3 Revert "lit: warn when passed invalid pathname" (r210597)
It was pointed out that this breaks the "virtual test discovery"
mechanism, which allows for narming tests in the test exec root.

Reverting until I can figure out how to fix this.

llvm-svn: 211048
2014-06-16 20:18:41 +00:00
Hans Wennborg a55d30a223 lit: warn when passed invalid pathname
It would previously say things like

  warning: input 'test/Frontend/foo.c' contained no tests

and have the user pull their hair trying to figure out what's wrong with that
file. This patch changes the message to the much clearer:

  warning: no such file or directory: 'test/Frontend/foo.c'

Differential Revision: http://reviews.llvm.org/D4097

llvm-svn: 210597
2014-06-10 22:51:58 +00:00
Alexander Potapenko 14f8ac04e3 Add detection of OS X relocatable SDK to compiler-rt as a lit.util function
Clang's lit cfg already detects the currently selected SDK via
"xcrun --show-sdk-path". The same thing should be done for compiler-rt tests,
to make them work on recent OS X versions. Instead of duplicating the detection
code, this patch extracts the detection function into a lit.util method.

Patch by Kuba Brecka (kuba.brecka@gmail.com),
reviewed at http://reviews.llvm.org/D4072

llvm-svn: 210534
2014-06-10 14:22:00 +00:00
Joerg Sonnenberger c1e13e2fe5 Fix test syntax to work with non-bash /bin/sh.
llvm-svn: 206119
2014-04-12 21:13:41 +00:00
Daniel Dunbar 3db2acb71a [lit] Fix non-function style print statement.
llvm-svn: 203573
2014-03-11 14:05:49 +00:00
Daniel Dunbar aa344d4f92 [lit] Bump dev version number.
llvm-svn: 203498
2014-03-10 21:58:16 +00:00
Daniel Dunbar 91b4298309 [lit] Add a README.txt.
- Also, update MANIFEST.in and utils/check-sdist.

llvm-svn: 203497
2014-03-10 21:58:12 +00:00
Daniel Dunbar b8a82c295d [lit] Add --version option.
llvm-svn: 203496
2014-03-10 21:57:48 +00:00
Alp Toker 749971901a lit: Provide source locations in cfg files with older Python versions
This commit prospectively brings the benefits of r198766 to older supported
Python versions (2.5+).

Tested with Python 2.6, 2.7, 3.1 and 3.3 (!)

llvm-svn: 199009
2014-01-11 14:34:18 +00:00
Alp Toker f0a245944e lit: execfile() isn't present in Python 3.3
On the other hand, exec(compile()) doesn't work in older Python versions in the
2.x series.

This commit introduces exec(compile()) with a fallback to plain exec(). That'll
hopefully hit the sweet spot in terms of version support.

Followup to r198766 which added enhanced source locations for lit cfg parsing.

llvm-svn: 199006
2014-01-11 13:27:28 +00:00
Alp Toker 9e628916f6 lit: Provide file location in cfg error messages
Python doesn't do a good job at diagnosing string exec() so use execfile()
where available.

This should be a timesaver when trying to get to the bottom of build bot
failures.

Before:

    File "llvm/utils/lit/lit/TestingConfig.py", line 93, in load_from_path
      exec("exec data in cfg_globals")
    File "<string>", line 1, in <module>
    File "<string>", line 194, in <module>
  NameError: name 'typo' is not defined

After:

    File "llvm/utils/lit/lit/TestingConfig.py", line 95, in load_from_path
    execfile(path, cfg_globals)
    File "clang/test/lit.cfg", line 194, in <module>
      typo
      ^~~~
  NameError: name 'typo' is not defined

llvm-svn: 198766
2014-01-08 14:20:59 +00:00
Alp Toker e395023eab Python compatibility fix for r198150
Remove the stat call error reporting for now.

It wasn't essential so silent fallback should be fine here.

llvm-svn: 198155
2013-12-29 06:51:10 +00:00
Alp Toker 0324ee0a33 Prospective Python 3 fix for r198150
llvm-svn: 198152
2013-12-29 05:51:07 +00:00
Alp Toker fdef0e0d94 lit: Incremental test scheduling
Add option -i to prioritize test runs by source file modification time and
previous failure state.

This optimal scheduling reduces typical test-and-fix iteration times to a
matter of seconds by rapidly answering the questions:

  1) Did my recent change fix tests that were previously failing?
  2) Do the tests I just wrote / modified still work?

The current implementation requires write permissions to the source tree
because it uses mtimes to track failures.

llvm-svn: 198150
2013-12-29 05:09:05 +00:00
NAKAMURA Takumi b7bf04c02e Tweak lit/TestingConfig.py Py3-compatible. has_key is dead.
llvm-svn: 197462
2013-12-17 04:14:50 +00:00
Alp Toker d0cdc67caa lit: multiprocessing platform fix attempt
The error raised by Python varies by platform(!), so let's just catch any
exception and fall back.

Thanks to Sylvestre Ledru for noticing this on a Debian / Python 2.7 system
running code coverage.

llvm-svn: 193516
2013-10-28 10:26:13 +00:00
Alp Toker 31bd72fb22 Clarify the comment about BSD versions in r193465
llvm-svn: 193508
2013-10-27 20:49:19 +00:00
Joerg Sonnenberger 853b460e4f self.path may be empty or otherwise miss the normal system directories,
so try PATH next. Assume it is sane enough to cover the usual system
bash locations too, but the old list is not good enough for NetBSD.

llvm-svn: 193471
2013-10-26 13:25:45 +00:00
Alp Toker 54d210b205 lit: Issue a note when multiprocessing fails to load
If multiprocessing was requested, detected as available and subsequently failed
to initialize it's worth letting the user know about it before falling back to
threads.

This condition can arise in certain OpenBSD / FreeBSD Python versions.

llvm-svn: 193465
2013-10-26 09:29:58 +00:00
Alp Toker 6c5dbd7a0a Fix a referenced before assignment in r193463
Some versions of Python on the builders seem strict about this.

llvm-svn: 193464
2013-10-26 08:46:05 +00:00
Alp Toker 9ade45482a lit: handle late multiprocessing errors gracefully
This should be a better fix for lit multiprocessing failures, replacing the
OpenBSD and FreeBSD workarounds in r193413 and r193457.

Reference: http://bugs.python.org/issue3770
llvm-svn: 193463
2013-10-26 08:22:44 +00:00
Alp Toker 5853534b03 Attempt to fix the FreeBSD build, disable multiprocessing
Speculative quick fix based on clang-X86_64-freebsd output:

  File "/usr/local/lib/python2.6/multiprocessing/synchronize.py", line 33, in <module>
    " function, see issue 3770.")
ImportError: This platform lacks a functioning sem_open implementation, therefore, the required synchronization primitives needed will not function, see issue 3770.

llvm-svn: 193457
2013-10-26 02:43:08 +00:00
Rafael Espindola 5e82540d11 Try to fix the openbsd bot.
llvm-svn: 193413
2013-10-25 15:07:59 +00:00
Rafael Espindola f059064d95 Actually switch the default to use multiprocessing
llvm-svn: 193282
2013-10-23 22:32:53 +00:00
Daniel Dunbar cdca362e65 [lit] Stop hacking the GIL check interval.
- This was never a big win, and is irrelevant now that we commonly use
   multiprocessing based parallelism.

llvm-svn: 193280
2013-10-23 22:19:32 +00:00
Daniel Dunbar 1fc709cd3e [lit] Use multiprocessing based parallelism by default, on Unix.
llvm-svn: 193279
2013-10-23 22:19:07 +00:00
NAKAMURA Takumi 1c19a0f4a8 Lit: LIT_PRESERVES_TMP should be aware of TMPDIR, too.
TMPDIR is preferred in Unix.

llvm-svn: 192891
2013-10-17 13:11:13 +00:00
NAKAMURA Takumi 199f138a07 Lit: Introduce an environment variable, $LIT_PRESERVES_TMP, to preserve TMP (and TEMP).
This is intended to check how many temporary files would be generated in automated builders.

llvm-svn: 192887
2013-10-17 12:10:12 +00:00
Daniel Dunbar cd625f4e54 [lit] Add an --output option, for writing results in a machine readable form.
llvm-svn: 190738
2013-09-14 01:19:17 +00:00
Daniel Dunbar 9aeba4963a [lit] Add support for attach arbitrary metrics to test results.
- This is a work-in-progress and all details are subject to change, but I am
   trying to build up support for allowing lit to be used as a driver for
   performance tests (or other tests which might want to record information
   beyond simple PASS/FAIL).

llvm-svn: 190535
2013-09-11 17:45:11 +00:00
Daniel Dunbar dac381a323 [lit] Allow config files to pass arbitrary values to child configs.
- This aligns with how existing test suites end up wanting to use the local
   config files, conceptually it makes sense to consider them to be inherited.

llvm-svn: 189885
2013-09-03 23:32:55 +00:00
Daniel Dunbar 9d8271b351 [lit] [tests] Add missing test input file.
llvm-svn: 189561
2013-08-29 03:02:34 +00:00
Daniel Dunbar b8da61fd1b [lit] Fix internal shell's argv[0] handling.
- At least on OS X, it is important for correct behavior of /bin/[ that argv[0]
   is passed as written, and not as the full executable path.

llvm-svn: 189559
2013-08-29 02:52:10 +00:00
Daniel Dunbar 9a83a76b1b [lit] Add support for multiprocessing, under --use-processes for now.
llvm-svn: 189556
2013-08-29 00:54:23 +00:00
Daniel Dunbar b3954fa889 [lit] Implement --max-time support by using provider cancel method.
llvm-svn: 189555
2013-08-29 00:54:19 +00:00
Daniel Dunbar 57ab2df070 [lit] Refactor test execution logic into lit.run.Run.
llvm-svn: 189554
2013-08-29 00:54:15 +00:00
Daniel Dunbar eecb1c1d25 [lit] Put display lock inside the ThreadResultsConsumer.
llvm-svn: 189553
2013-08-29 00:54:09 +00:00