maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
313,520 Commits 23 Branches 0 Tags 2.2 GiB
Commit Graph

657 Commits

Author SHA1 Message Date
Dan Liew 8b4d36a530 [lit] Don't check output of commands used in `shtest-timeout.py` test. 
If the system is under heavy load 1 second might not be long enough
for it to produce output which could lead to spurious test failures.
What matters is that the right test cases reach a timeout.

rdar://problem/40221572

llvm-svn: 332985
 2018-05-22 15:06:20 +00:00
Douglas Yung 2dd62a3d99 Mark test with "REQUIRES: shell" since it directly invokes "sh" and was failing on Windows. 
llvm-svn: 332563
 2018-05-17 01:36:25 +00:00
Alexander Richardson 1bffd0f741 Escape ]]> in xunit xml output 
Summary:
This sequence ends the CDATA block so any characters after that are no
longer escaped. This can be fixed by replacing "]]>" with "]]]]><![CDATA[>".

Reviewers: cmatthews

Reviewed By: cmatthews

Differential Revision: https://reviews.llvm.org/D46886

llvm-svn: 332440
 2018-05-16 09:00:28 +00:00
Chris Matthews 48357c43d5 remove output xml incase it is leftover from another run 
llvm-svn: 332424
 2018-05-16 00:37:00 +00:00
Chris Matthews 5d1014bff5 Use not to catch unexpected pass as well as remove old test results 
As per review feedback, make sure we rm temp files, and make the return
code checking for lit more specific.

llvm-svn: 332423
 2018-05-16 00:33:29 +00:00
Chris Matthews f616804903 Requirements can have & in them! 
Lets escape those so the XML is valid!

llvm-svn: 332161
 2018-05-12 00:13:54 +00:00
Chris Matthews 811a83d036 Add the message attribute to skipped 
JUnit xml allows for a message attribute to be displayed on skips. Lets
populate that with an analysis of why we skipped the test.

llvm-svn: 332156
 2018-05-11 23:15:11 +00:00
Chris Matthews 6b3db8519d Overhaul unicode handling in xunit output 
I have seen a lot of errors where the xunit does not encode unicode
test output correctly.  Handle that explicitly now.

llvm-svn: 332148
 2018-05-11 22:18:22 +00:00
Chris Matthews d1386a88dd [LIT] replace output escapes wit a cdata block 
CDATA blocks don't need to have XML stuff escaped. Makes sense to wrap
output in them instead of escaping.

llvm-svn: 332116
 2018-05-11 18:38:02 +00:00
Chris Matthews 7d6224d2fa Support Unsupported Tests in xunit output 
We were reporting  "Unsupported" tests in xunit as passes, however since
they are not run, it make more sense to mark them as skipped. The Junit
xml standard has support for that, so lets use it.

llvm-svn: 332065
 2018-05-11 00:25:43 +00:00
Chris Matthews 5f74591847 Refactor xunit test case builder to not use as much str addition 
String concatenation in python is slow.  Refactor to not concatenate the
possibly large strings of test output and instead write them directly
to the output file.

llvm-svn: 332064
 2018-05-11 00:25:42 +00:00
Chris Matthews 4855c5f717 [LIT] Move xunit tests tests into their own location, and and add failures 
Failures will increase coverage.

llvm-svn: 332056
 2018-05-10 22:51:28 +00:00
Chris Matthews c3f259cb6c [LIT] Add the missing file 
I forgot to commit this file.

llvm-svn: 331946
 2018-05-10 00:08:39 +00:00
Chris Matthews bd9ffc6243 Refactor test incase results are backwards 
Looks like results can come in either way in this file.  Loosen the ordering constraints.

llvm-svn: 331945
 2018-05-10 00:06:17 +00:00
Chris Matthews 069a1eb3bb [LIT] Handle xml characters in test names 
Lit creates malformed xml when the test case has an & in the name.

Escape those correctly.

This also adds a test case which I will add other nasty encoding issues to in some followup commits.

llvm-svn: 331942
 2018-05-09 23:48:32 +00:00
Nico Weber 3934dba059 Remove 'abi-breaking-checks' lit feature. 
Its only two uses were removed in r311730.
Effectively reverts r304851 (but that code has removed around a bit since then).
https://reviews.llvm.org/D46619

clang side done in r331871.

llvm-svn: 331872
 2018-05-09 12:39:39 +00:00
Saleem Abdulrasool cc2da59f69 lit: flesh out `SubsituteCaptures` further 
Add overloads for `__len__` and `__getitem__` to allow use of this class
on Linux as well as Windows.  With these overloads, lit can be used on
both hosts for the swift testsuite.

llvm-svn: 331431
 2018-05-03 04:45:43 +00:00
Reid Kleckner 1ca666886f Revert r330755 "[lit] Report line number for failed RUN command" 
It is causing many tests to fail on Windows buildbots:
http://lab.llvm.org:8011/builders/clang-x64-ninja-win7/builds/10211

llvm-svn: 330848
 2018-04-25 17:30:00 +00:00
Joel E. Denny 8a475307bd [lit] Report line number for failed RUN command 
When debugging test failures with -vv (or -v in the case of the
internal shell), this makes it easier to locate the RUN line that
failed.  For example, clang's test/Driver/linux-ld.c has 892 total RUN
lines, and clang's test/Driver/arm-cortex-cpus.c has 424 RUN lines
after concatenation for line continuations.

When reading the generated shell script, this also makes it easier to
locate the RUN line that produced each command.

To support reporting RUN line numbers in the case of the internal
shell, this patch extends the internal shell to support the null
command, ":", except pipelines are not supported.

Reviewed By: asmith, delcypher

Differential Revision: https://reviews.llvm.org/D44598

llvm-svn: 330755
 2018-04-24 18:43:25 +00:00
Dan Liew 97a60c21d4 [lit] Remove spurious `-` in invocation of lit in 
`shtest-xunit-output.py` test.

Although there is no `-` file Jeremy Morse has reported to me that it
causes problems in their setup because lit tries to find it and ends up
loading an out of tree lit configuration file.

llvm-svn: 330728
 2018-04-24 15:42:00 +00:00
Nico Weber 1c2c321b47 Mostly revert r330672. 
The test is apparently needed e.g. for check-cfi on Windows where we get
  'C:/b/slave/sanitizer-windows/build/./bin/clang.exe': command not found
without it.  Try to fix the problem that was fixed by r330672 by also checking
for isabs() instead.

llvm-svn: 330673
 2018-04-24 01:24:42 +00:00
Nico Weber a425db5a1e Remove code that's almost always dead, and harmful if not. 
lit's util.which() would check if the passed-in path existed directly,
and if so return it as-is.  This is never the case when running llvm's, clang's,
or lld's tests normally.  But when running `./llvm-lit path/to/clang/test`
with a cwd of llvm-build/bin, this if would detect that clang exists at path
'clang' and return 'clang' as the discovered clang binary -- and then lit would
use the " clang " -> "*** Do not use 'clang' in tests, use '%clang'. ***"
substitution to replace that with a broken test.  By removing this early
return, lit ends up with the usual absolute path and everything works even
in this uncommon case.

llvm-svn: 330672
 2018-04-24 01:05:04 +00:00
Dan Liew 872b8ea596 [lit] Fix a bug where UNRESOLVED tests were not handled in the XUnit 
XML printer.

A test has been added that tries to comprehensively test emitting
XUnit XML output for shell tests.

Differential Revision: https://reviews.llvm.org/D45567

llvm-svn: 330409
 2018-04-20 10:11:41 +00:00
Aaron Smith 0c8ee891a7 [lit] Remove duplicate to_string method 
There are two versions of to_string used by TestRunner.py. The one defined 
in TestRunner.py and the one defined in utils/lit/lit/util.py. The util.py
version is superior to the TestRunner.py version.

This change removes the duplicate to_string in TestRunner.py in favor of
always using the version from util.py. Beside removing duplicate code, this
makes it easier to debug TestRunner.py since only one version of to_string
is used.

Patch by Stella Stamenova!

llvm-svn: 329972
 2018-04-12 23:45:15 +00:00
Aaron Smith e5ee89c0d3 [lit] Fix several Python 2/3 compatibility issues and tests 
- In Python 2.x, basestring is the base string type, but in 
  Python 3.x basestring is not defined and instead str includes 
  unicode strings.

- When Python is in a path that includes spaces, it needs to 
  be specified with quotes in the test files for it to run.

- The cache.ll test relies on files of a specific size being 
  created by Python, but on some versions of Windows the 
  files that are created by the current code are one byte 
  larger than expected. To fix the test, update file creation 
  to always make files of the expected size.

Patch by Stella Stamenova!

llvm-svn: 329466
 2018-04-07 00:21:28 +00:00
Reid Kleckner bdbb712b6d 'cat' command for internal shell - Support Python 3 
LLVM Bug Id : 36449

Revision 328563 caused tests to fail under python 3.

This patch modified cat.py file to support both python 2 and 3.
This patch also fixes CRLF issues on Windows.

Patch by Chamal de Silva

Differential Revision: https://reviews.llvm.org/D45077

llvm-svn: 329123
 2018-04-03 22:38:25 +00:00
Aaron Smith 061354b586 [lit] Prefer opening files with open (Python 2) rather than io.open which requires io. 
Only rely on Python 3 (io.open) when necessary. This puts TestRunnyer.py closer to how it behaved 
before the changes introduced in D43165 and silences a few Windows build bot failures.

Thanks to Stella Stamenova for the patch!

llvm-svn: 329037
 2018-04-03 00:22:12 +00:00
Aaron Smith 637fd446d9 [lit] One more try at fixing TestRunner.py for D43165 
llvm-svn: 329026
 2018-04-02 22:34:35 +00:00
Reid Kleckner 30b215e8f7 [lit] Attempt to fix builtin diff code for Python 2 
llvm-svn: 329024
 2018-04-02 22:19:42 +00:00
Aaron Smith 98d31d6880 [lit] Fix problem in how Python versions open files with different encodings 
Reapply D43165 which was reverted because of different versions of python failing. 
The one line fix for the different python versions was commited at the same time
that D43165 was reverted. If this change is giving you issues then get in touch
with your python version and we will fix it. 

llvm-svn: 329022
 2018-04-02 22:08:56 +00:00
Aaron Smith b50baa1c4b [lit] Use io.open to compare two files since it supports different encodings while older versions of open do not 
llvm-svn: 329020
 2018-04-02 21:44:51 +00:00
Reid Kleckner 2aae49d3b2 Revert r329012 "[lit] Fix problem in how Python versions open files with different encodings" 
This doesn't work with Python 2. See this build-in-progress:
http://lab.llvm.org:8011/builders/clang-x86-windows-msvc2015/builds/11105

llvm-svn: 329017
 2018-04-02 21:33:46 +00:00
Aaron Smith 321c2487d7 [lit] Fix problem in how Python versions open files with different encodings 
Summary:
This issue was found when running the clang unit test on Windows. Python 3.x cannot open some of the files that the tests are using with a simple open because of their encoding. Python 2.7+ and Python 3.x both support io.open which allows for an encoding to be specified. 

This change will determine whether two files being compared should be opened (and then compared) as text or binary and whether to use utf-8 or the default encoding before proceeding with a line-by-line comparison.

Patch by Stella Stamenova!

Reviewers: zturner, llvm-commits, rnk, MaggieYi

Reviewed By: zturner

Subscribers: asmith, MatzeB, stella.stamenova, delcypher, llvm-commits

Differential Revision: https://reviews.llvm.org/D43165

llvm-svn: 329012
 2018-04-02 20:57:06 +00:00
Dan Liew 8ade9e75b0 Revert "[lit] Temporarily disable shtest-timeout.py on darwin" 
This reverts commit 771829b640a5494ab65c810dd6b4330522bf3a33 (rr328598)

Hopefully the test will now pass on the bots.

rdar://problem/38774530

llvm-svn: 328703
 2018-03-28 13:55:13 +00:00
Dan Liew 7efde3c440 [lit] Remove a timing senstive part of `shtest-timeout.py` 
The `shtest-timeout.py` test was failing intermittently. It looks like
the issue is that on a resource constrained system lit is unable to run
`quick_then_slow.py` twice and print out the messages the tests expects
within the one second timeout.

The underlying issue is that the test is dependent on the performance of
the host machine is a rather fragile way. This is due to hardcoding
timeout values and having assumptions that the host machine is able to
perform a certain amount of work within the hardcoded timeout values.

We could increase the timeout values but that doesn't really fix the
underlying issue. Instead this patch removes one of fragile assumptions
in the hope that this will be enough to fix the bots.
There are other fragile assumptions in this test (e.g. `quick.py` can be
executed in less than 1 second). If the bots continue to fail we'll have
to revisit this.

rdar://problem/38774530

llvm-svn: 328702
 2018-03-28 13:55:08 +00:00
Mircea Trofin 56ba71b2a7 Revert "Revert "[lit] Generalized /dev/null support on Windows."" 
Summary:
This reverts commit r328596.

Checking if the arguments are strings before testing if they contain "/dev/null".

Reviewers: rnk

Reviewed By: rnk

Subscribers: delcypher, llvm-commits

Differential Revision: https://reviews.llvm.org/D44914

llvm-svn: 328603
 2018-03-27 01:39:17 +00:00
Jan Korous 1e0e0b077d [lit] Temporarily disable shtest-timeout.py on darwin 
Disabled until fixed in order to avoid random failures on green dragon.

rdar://problem/38774530

llvm-svn: 328598
 2018-03-27 00:16:28 +00:00
Mircea Trofin 373c445c24 Revert "[lit] Generalized /dev/null support on Windows." 
This reverts commit ca7fdbb974384ce5a05528b22a41d46b1cc13e92.

llvm-svn: 328596
 2018-03-26 23:59:39 +00:00
Mircea Trofin 88911686c8 [lit] Generalized /dev/null support on Windows. 
Generalized /dev/null remapping on Windows, and added test.

Reviewers: rnk

Reviewed By: rnk

Subscribers: amccarth, zturner, delcypher, llvm-commits

Differential Revision: https://reviews.llvm.org/D44771

llvm-svn: 328589
 2018-03-26 22:41:06 +00:00
Reid Kleckner 8252892951 [lit] Implement 'cat' command for internal shell 
Fixes PR36449

Patch by Chamal de Silva

Differential Revision: https://reviews.llvm.org/D43501

llvm-svn: 328563
 2018-03-26 18:05:12 +00:00
Brian Homerding d5c558ff21 [lit] - Allow 1 test to report multiple micro-test results to provide support for microbenchmarks. 
Summary:
These changes are to allow to a Result object to have nested Result objects in
order to support microbenchmarks. Currently lit is restricted to reporting one
result object for one test, this change provides support tests that want to
report individual timings for individual kernels.

This revision is the result of the discussions in
https://reviews.llvm.org/D32272#794759,
https://reviews.llvm.org/D37421#f8003b27 and https://reviews.llvm.org/D38496.
It is a separation of the changes purposed in https://reviews.llvm.org/D40077.

This change will enable adding LCALS (Livermore Compiler Analysis Loop Suite)
collection of loop kernels to the llvm test suite using the google benchmark
library (https://reviews.llvm.org/D43319) with tracking of individual kernel
timings.

Previously microbenchmarks had been handled by using macros to section groups
of microbenchmarks together and build many executables while still getting a
grouped timing (MultiSource/TSVC). Recently the google benchmark library was
added to the test suite and utilized with a litsupport plugin. However the
limitation of 1 test 1 result limited its use to passing a runtime option to
run only 1 microbenchmark with several hand written tests
(MicroBenchmarks/XRay). This runs the same executable many times with different
hand-written tests. I will update the litsupport plugin to utilize the new
functionality (https://reviews.llvm.org/D43316).

These changes allow lit to report micro test results if desired in order to get
many precise timing results from 1 run of 1 test executable.


Reviewers: MatzeB, hfinkel, rengolin, delcypher

Differential Revision: https://reviews.llvm.org/D43314

llvm-svn: 327422
 2018-03-13 16:37:59 +00:00
Saleem Abdulrasool d5453b2c3f utils: add a helper class to lit for captured substitutions 
On Windows, if the substitution contains a back reference, it would
removed due to the replacement of the escape character in lit. Create a
helper class to avoid this which will simply ignore the replacement and
mark the substitution as having capture groups being referenced.

llvm-svn: 327082
 2018-03-09 00:06:10 +00:00
Lei Huang 9db7bd5cab Losen time contraint to accommodate system loads 
llvm-svn: 326359
 2018-02-28 20:05:24 +00:00
Rui Ueyama 7c9ad29304 Remove "--full-shutdown" and instead use an environment variable LLD_IN_TEST. 
We are running lld tests with "--full-shutdown" option because we don't
want to call _exit() in lld if it is running tests. Regular shutdown
is needed for leak sanitizer.

This patch changes the way how we tell lld that it is running tests.
Now "--full-shutdown" is removed, and LLD_IN_TEST environment variable
is used instead.

This patch enables full shutdown on all ports, e.g. ELF, COFF and wasm.
Previously, we enabled it only for ELF.

Differential Revision: https://reviews.llvm.org/D43410

llvm-svn: 325413
 2018-02-16 23:41:48 +00:00
Rui Ueyama 14ee1140c1 Make sure we invoke ld64.lld and ld-wasm in the build directory. 
llvm-svn: 325397
 2018-02-16 22:19:38 +00:00
Victor Leschuk 33b9fa7caa [lit] Pass CLANG env var to testing configuration 
Allow CLANG environment variable be copied into the testing configuration
and proper support testing with a custom path to the clang executable.

Signed-off-by: Vladimir Vereschaka <vvereschaka@accesssoftek.com>
llvm-svn: 324706
 2018-02-09 05:28:38 +00:00
Max Moroz 975eacada5 [lit] Implement "-r" option for builtin "diff" command + a test using that. 
Summary:
That would allow to recursively compare directories in tests using
"diff -r" on Windows in a similar way as it can be done on Linux or Mac.

Reviewers: zturner, morehouse, vsk

Reviewed By: zturner

Subscribers: kcc, llvm-commits

Differential Revision: https://reviews.llvm.org/D41776

llvm-svn: 322102
 2018-01-09 18:23:34 +00:00
Tom Stellard d703b198f2 [lit] Bump version number 
llvm-svn: 321735
 2018-01-03 16:35:51 +00:00
Kuba Mracek 04013b704c Disable detect_leaks in the ASanified build of LLVM when using Apple LLVM. The released Apple LLVM versions don't support LSan. 
llvm-svn: 319738
 2017-12-05 05:22:02 +00:00
Jonas Devlieghere 64774bafff [NFC][lit] Use proper semantic versioning names for variables 
The variable named `minor` was actually pointing to the patch part of
the version. While I was changing this I also made the check for Apple
clang more robust by checking both patch and minor rather than just
minor.

llvm-svn: 319656
 2017-12-04 14:01:34 +00:00
Jonas Devlieghere 42b9a603da [lit] Don't enable LSan on Darwin for Apple clang 9.0.0 
The latest clang that ships with Xcode (clang 900 or 9.0.0) does not
support LSan. This fixes the lit configuration to reflect that.

Differential revision: https://reviews.llvm.org/D40672

llvm-svn: 319530
 2017-12-01 10:49:47 +00:00
Ying Yi cfb08e9e6b [lit] Implement non-pipelined ‘mkdir’, ‘diff’ and ‘rm’ commands internally 
Summary:
The internal shell already supports 'cd', ‘export’ and ‘echo’ commands.
This patch adds implementation of non-pipelined ‘mkdir’, ‘diff’ and ‘rm’
commands as the internal shell builtins.

Reviewed by: Zachary Turner, Reid Kleckner

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39567

llvm-svn: 319528
 2017-12-01 09:54:27 +00:00
Greg Bedwell 34a83f0faf Make check-lit tests respect LLVM_LIT_TOOLS_DIR 
Differential Revision: https://reviews.llvm.org/D40520

llvm-svn: 319329
 2017-11-29 18:05:26 +00:00
Matthias Braun adf7582d14 lit: Bring back -Dtool=xxx feature lost in r313928 
llvm-svn: 319139
 2017-11-28 03:23:07 +00:00
Ying Yi a0903c6e5d Reverted rL318911 since it broke the sanitizer-windows. 
llvm-svn: 318914
 2017-11-23 13:23:21 +00:00
Ying Yi 989c9e75a6 [lit] Implement non-pipelined ‘mkdir’, ‘diff’ and ‘rm’ commands internally 
Summary:
The internal shell already supports 'cd', ‘export’ and ‘echo’ commands. 
This patch adds implementation of non-pipelined ‘mkdir’, ‘diff’ and ‘rm’ 
commands as the internal shell builtins.

Reviewers: Zachary Turner, Reid Kleckner

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39567

llvm-svn: 318911
 2017-11-23 12:48:41 +00:00
Zachary Turner 79708b54f2 Re-revert "Refactor debuginfo-tests." 
This is still breaking greendragon.

At this point I give up until someone can fix the greendragon
bots, and I will probably abandon this effort in favor of using
a private github repository.

llvm-svn: 318722
 2017-11-21 01:20:28 +00:00
Zachary Turner 8b6ef88e7e Resubmit "Refactor debuginfo-tests" again. 
This was reverted due to the tests being run twice on some
build bots.  Each run had a slightly different configuration
due to the way in which it was being invoked.  This fixes
the problem (albeit in a somewhat hacky way).  Hopefully in
the future we can get rid of the workflow of running
debuginfo-tests as part of clang, and then this hack can
go away.

llvm-svn: 318697
 2017-11-20 21:41:36 +00:00
Eric Fiselier c812560c14 [LIT] Fix testing out-of-tree Clang builds 
Summary:
Currently, LIT configures the LLVM binary path before the Clang binary path. However this breaks testing out-of-tree Clang builds (where the LLVM binary path includes a copy of Clang).

This patch reverses the order of the paths when looking for Clang, putting the Clang binary directory first.

Reviewers: zturner, beanz, chapuni, modocache, EricWF

Reviewed By: EricWF

Subscribers: mgorny, cfe-commits, llvm-commits

Differential Revision: https://reviews.llvm.org/D40217

llvm-svn: 318607
 2017-11-19 00:00:49 +00:00
Reid Kleckner 6810c20c1b [lit] Try to improve Ctrl-C behavior on Windows 
This functionality was broken during a refactor a while back because
'pool' is no longer in scope.

llvm-svn: 318572
 2017-11-17 23:52:33 +00:00
Zachary Turner dfeb170011 Fix an issue with llvm lit tool substitutions. 
When using an installed clang with an in-tree llvm, we were not
searching in the right paths for the tools.

llvm-svn: 318564
 2017-11-17 22:51:43 +00:00
Zachary Turner b5c237ec3d Re-revert "Refactor debuginfo-tests" 
This is still broken because it causes certain tests to be
run twice with slightly different configurations, which is
wrong in some cases.

You can observe this by running:

  ninja -nv check-all | grep debuginfo-tests

And seeing that it passes clang/test and clang/test/debuginfo-tests
to lit, which causes it to run debuginfo-tests twice.  The fix is
going to involve either:

  a) figuring out that we're running in this "deprecated" configuration,
     and then deleting the clang/test/debuginfo-tests path, which should
     cause it to behave identically to before, or:
  b) make lit smart enough that it doesn't descend into a sub-suite if
     that sub-suite already has a lit.cfg file.

llvm-svn: 318486
 2017-11-17 00:41:18 +00:00
Zachary Turner 5e420717a1 Resubmit "Refactor debuginfo-tests" 
This was reverted due to some failures on specific darwin buildbots,
the issue being that the new lit configuration was not setting the
SDKROOT environment variable.  We've tested a fix locally and confirmed
that it works, so this patch resubmits everything with the fix
applied.

llvm-svn: 318435
 2017-11-16 18:26:20 +00:00
Zachary Turner faf04a09f6 Revert "Update test_debuginfo.pl script to point to new tree location." 
This reverts the aforementioned patch and 2 subsequent follow-ups,
as some buildbots are still failing 2 tests because of it.
Investigation is ongoing into the cause of the failures.

llvm-svn: 318112
 2017-11-13 23:33:29 +00:00
Zachary Turner 0f2ce11df7 [debuginfo-tests] Make debuginfo-tests work in a standard configuration. 
Previously, debuginfo-tests was expected to be checked out into
clang/test and then the tests would automatically run as part of
check-clang.  This is not a standard workflow for handling
external projects, and it brings with it some serious drawbacks
such as the inability to depend on things other than clang, which
we will need going forward.

The goal of this patch is to migrate towards a more standard
workflow.  To ease the transition for build bot maintainers,
this patch tries not to break the existing workflow, but instead
simply deprecate it to give maintainers a chance to update
the build infrastructure.

Differential Revision: https://reviews.llvm.org/D39605

llvm-svn: 317925
 2017-11-10 20:57:57 +00:00
Jake Ehrlich cde781015f Add system-linux to allow tests run with llvm-lit to restrict themselves to linux 
I need a test that only runs in a reasonable amount of time on systems
that have sparse files. The broadest class of systems that support
sparse files are linux systems. So restricting my test to linux systems
should suffice. This change adds the system-linux feature to llvm-lit so
that it can be required.

Differential Revision: https://reviews.llvm.org/D39482

llvm-svn: 317055
 2017-11-01 00:18:51 +00:00
Peter Collingbourne c03ceb92ca lit: Improve %: normalization. 
The new scheme should match the normalization of embedded paths in
linkrepro tar files.

Differential Revision: https://reviews.llvm.org/D39023

llvm-svn: 316044
 2017-10-17 23:46:34 +00:00
Jordan Rose e55d9a1403 Revert "lit.py: Add new %{shared_output(LABEL)} substitution" 
This reverts r315697 and my ill-fated attempts to fix it on Windows.
I'll try again when I get access to a Windows machine.

llvm-svn: 315793
 2017-10-14 04:01:27 +00:00
Jordan Rose a33a17605d lit.py: Give up and disable the new shared-output.py test on Windows 
"No such file or directory: C:\\...\\tests\\Output\\shared-output.py.tmp/Output/Shared/SHARED.tmp"

And yet other forward-slashes don't seem to be causing the same
problem. I'll see if I can get ahold of a Windows machine to poke at
this directly later.

llvm-svn: 315792
 2017-10-14 03:33:46 +00:00
Jordan Rose fddfb5ad4c lit.py: Previous test fix was a red herring; backslashes are escapes 
I don't have access to a Windows machine at the moment, so if this
doesn't fix it I'll just revert for now.

llvm-svn: 315782
 2017-10-14 00:49:19 +00:00
Jordan Rose aafbc1c4db lit.py: Fix new test harder for systems that don't use / as os.path.sep 
I didn't think about '%{inputs}' having the same problem. This one
should be a fully Windows path name.

llvm-svn: 315779
 2017-10-14 00:26:48 +00:00
Jordan Rose f4fd5938e7 lit.py: Fix new test for systems that don't use / as os.path.sep 
llvm-svn: 315773
 2017-10-13 22:46:40 +00:00
Jordan Rose 53be3224b2 lit.py: Add new %{shared_output(LABEL)} substitution 
This refers to a temporary path that can be shared across all tests,
identified by a particular label. This can be used for things like
caches.

At the moment, the character set for the LABEL is limited to C
identifier characters, plus '-', '+', '=', and '.'. This is the same
set of characters currently allowed in REQUIRES clause identifiers.

llvm-svn: 315697
 2017-10-13 16:12:23 +00:00
Zachary Turner ad5997e87a [lit] Raise the logic for enabling clang & lld substitutions to llvm. 
This paves the way for other projects which might /use/ clang or
lld but not necessarily need to the full set of functionality
available to clang and lld tests to be able to have a basic set
of substitutions that allow a project to run the clang or lld
executables.

llvm-svn: 315627
 2017-10-12 21:56:05 +00:00
Francis Ricci 072eff0930 [lit] Only enable LSan on darwin when clang supports it 
Summary:
LSan on darwin doesn't exist on older versions of clang,
causing non-boostrapped sanitized buildbots to fail

Reviewers: kubamracek, qcolombet, sqlbyme, zturner, modocache

Reviewed By: zturner

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D38703

llvm-svn: 315333
 2017-10-10 17:21:32 +00:00
Zachary Turner 96b04b68ed [lit] Improve tool substitution in lit. 
This addresses two sources of inconsistency in test configuration
files.

1. Substitution boundaries.  Previously you would specify a
   substitution, such as 'lli', and then additionally a set
   of characters that should fail to match before and after
   the tool.  This was used, for example, so that matches that
   are parts of full paths would not be replaced.  But not all
   tools did this, and those that did would often re-invent
   the set of characters themselves, leading to inconsistency.
   Now, every tool substitution defaults to using a sane set
   of reasonable defaults and you have to explicitly opt out
   of it.  This actually fixed a few latent bugs that were
   never being surfaced, but only on accident.

2. There was no standard way for the system to decide how to
   locate a tool.  Sometimes you have an explicit path, sometimes
   we would search for it and build up a path ourselves, and
   sometimes we would build up a full command line.  Furthermore,
   there was no standardized way to handle missing tools.  Do we
   warn, fail, ignore, etc?  All of this is now encapsulated in
   the ToolSubst class.  You either specify an exact command to
   run, or an instance of FindTool('<tool-name>') and everything
   else just works.  Furthermore, you can specify an action to
   take if the tool cannot be resolved.

Differential Revision: https://reviews.llvm.org/D38565

llvm-svn: 315085
 2017-10-06 17:54:46 +00:00
Zachary Turner c981448063 Run pyformat on lit code. 
llvm-svn: 315084
 2017-10-06 17:54:27 +00:00
Francis Ricci 954b94f5df [test] Fix append_path in the empty case 
Summary:
normpath() was being called on an empty string and appended to
the environment variable in the case where the environment variable
was unset. This led to ":." being appended to the path, since
normpath() of an empty string is '.', presumably to represent cwd.

Reviewers: zturner, sqlbyme, modocache

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D38542

llvm-svn: 314915
 2017-10-04 17:30:28 +00:00
Michal Gorny d6a4c79b14 [lit] Fix running lit tests in unconfigured source dir 
Fix llvm_tools_dir attribute access not to fail when the variable is not
present. This directory is not really necessary to run lit tests,
and the code already accounts for it being None.

The reference was added in r313407, and it breaks the stand-alone lit
package in Gentoo.

Differential Revision: https://reviews.llvm.org/D38442

llvm-svn: 314620
 2017-10-01 07:13:25 +00:00
Francis Ricci a7bf226529 [test] Enable LeakSanitizer on 64-bit Darwin ASan llvm builds 
Summary:
Also disables leak checking on lto tests, due to many leaks reported
in the system's ld64.

Reviewers: kcc, pcc, bogner, kubamracek

Subscribers: mehdi_amini, llvm-commits

Differential Revision: https://reviews.llvm.org/D37781

llvm-svn: 314535
 2017-09-29 16:51:50 +00:00
Zachary Turner 0aa02c08a7 Resubmit "[lit] Refactor out some more common lit configuration code." 
There were two issues, one Python 3 specific related to Unicode,
and another which is that the tool substitution for lld no longer
rejected matches where a / preceded the tool name.

llvm-svn: 313928
 2017-09-21 22:16:40 +00:00
Zachary Turner 5f2fd9b783 Revert "[lit] Refactor out some more common lit configuration code." 
This is breaking several bots.  I have enough information to
investigate, so I'm reverting to green until I get it figured
out.

llvm-svn: 313922
 2017-09-21 21:45:45 +00:00
Zachary Turner 0d36b657b9 [lit] Refactor out some more common lit configuration code. 
debuginfo-tests has need to reuse a lot of common configuration
from clang and lld, and in general it seems like all of the
projects which are tightly coupled (e.g. lld, clang, llvm, lldb,
etc) can benefit from knowing about one other.  For example,
lldb needs to know various things about how to run clang in its
test suite.  Since there's a lot of common substitutions and
operations that need to be shared among projects, sinking this
up into LLVM makes sense.

In addition, this patch introduces a function add_tool_substitution
which handles all the dirty intricacies of matching tool names
which was previously copied around the various config files.  This
is now a simple straightforward interface which is hard to mess
up.

Differential Revision: https://reviews.llvm.org/D37944

llvm-svn: 313919
 2017-09-21 21:27:31 +00:00
Zachary Turner 1ca789bdba [lit] Actually do normalize the case of files in the config map. 
This has gone back and forth, but it seems this is necessary
after all.  realpath is not sufficient because if you have a
file named 'C:\foo.txt', then both realpath('c:\foo.txt') and
realpath(C:\foo.txt') return the string that was passed to them
exactly as is, meaning the case of the drive-letter won't match.

The problem before was not that we were normalizing the case of
items going into the config map, but rather that we were
normalizing the case of something we needed to print.  The value
that is used to key on the config map should never be printed.

llvm-svn: 313918
 2017-09-21 21:27:11 +00:00
Zachary Turner 43bcf226c1 [lit] Don't norm case when inserting into the config map. 
This makes all paths lowercase on Windows, which seemed like a
good idea at the time, but it means that tests can't properly
use FileCheck to match expected path names.

llvm-svn: 313889
 2017-09-21 17:02:08 +00:00
Zachary Turner 71deeee593 [lit] Add a test for the builtin config map. 
Config map is not exposed through the command line, so testing this
is somewhat tricky.  But basically we need a test that if a custom
driver builds a config map and passes it to main, it gets respected.

A config map allows config files in the source tree to be mapped
to alternate config files in the build tree.  This particular test
works by having two config files in separate directories, and
setting up a config map to have that redirects A/lit.site.cfg
to B/altconfig.  Then, we print a message in A/lit.site.cfg
and B/altconfig and check that we do see the output from B
but don't see the output from A.  Additionally we test that
the test suite specified by A's config map is properly discovered.

Differential Revision: https://reviews.llvm.org/D38105

llvm-svn: 313887
 2017-09-21 16:18:28 +00:00
David L. Jones e85a0eca21 [lit/Win] Check if a path was found before attempting to use it. 
Summary:
This appears to break some bots, when getToolsPath fails to find some or
all of the tools (for example, an incomplete GnuWin32 installation).

Reviewers: zturner, modocache

Subscribers: sanjoy, llvm-commits

Differential Revision: https://reviews.llvm.org/D38115

llvm-svn: 313854
 2017-09-21 01:26:16 +00:00
Zachary Turner 957d611575 [lit] Make lit support config files with .py extension. 
Many editors and Python-related diagnostics tools such as
debuggers break or fail in mysterious ways when python files
don't end in .py.  This is especially true on Windows, but
still exists on other platforms.  I don't want to be too heavy
handed in changing everything across the board, but I do want
to at least *allow* lit configs to have .py extensions.  This
patch makes the discovery process first look for a config file
with a .py extension, and if one is not found, then looks for
a config file using the old method.  So for existing users, there
should be no functional change.

Differential Revision: https://reviews.llvm.org/D37838

llvm-svn: 313849
 2017-09-21 00:24:52 +00:00
Zachary Turner 08fe808b3d [lit] Undo the patch to stop writing pyc files. 
The problems on the bots appear to be resolved and this was
determined to not be the culprit.  Removing this.

llvm-svn: 313807
 2017-09-20 20:31:24 +00:00
Zachary Turner 8978363735 [lit] Reverse path list when updating environment vars. 
Bug pointed out by EricWF.  This would construct a path where
items would be added in the wrong order, potentially leading
to using the wrong tools for testing.

llvm-svn: 313765
 2017-09-20 17:08:20 +00:00
Zachary Turner d3bb80a1bc Make lit stop writing pyc files. 
Many svn-based buildbots seem to be getting stuck continually
in tree conflicts due to the output of pyc files.  I'm disabling
these as a temporary measure in an attempt to get everything
stable again.

I'll try to remove this code once I understand the problem
better.

llvm-svn: 313698
 2017-09-19 23:50:28 +00:00
Zachary Turner d1e5e35660 [lit] Use realpath when adding to the config map. 
Since the path a user specifies to the llvm-lit script might be
different than the source tree they built from (since they could
be behind different symlinks), we need to use realpath to make
sure that path comparisons work as expected.

Even better would be to use a custom dictionary comparison with
actual file equivalence comparison semantics, but this is the
least friction to unbreak things for now.

llvm-svn: 313594
 2017-09-18 23:36:35 +00:00
Zachary Turner bbf8f23ed9 Fix inverted regex search. 
I was using the pattern as the source string and vice versa
causing strange regular expression errors.

llvm-svn: 313590
 2017-09-18 23:14:15 +00:00
Zachary Turner fca1a34a11 [lit] Fix a Python 3 compatibility issue. 
llvm-svn: 313580
 2017-09-18 22:30:45 +00:00
Zachary Turner d4401d354a [lit] Update clang and lld to use new config helpers. 
NFC intended here, this only updates clang and lld's lit configs
to use some helper functionality in the lit.llvm submodule.

llvm-svn: 313579
 2017-09-18 22:26:48 +00:00
Zachary Turner b9fad9fd12 Try to fix some failing bots. 
It doesn't make sense to me why these bots are failing as the
traceback does not agree with the source code.  It's possible
something is stale or there is some other mysterious error,
but in any case hopefully this fixes it.

llvm-svn: 313469
 2017-09-16 19:20:53 +00:00
Zachary Turner c3023d1bd1 Resubmit "Add a shared llvm.lit module that all test suites can use." 
There were some issues surrounding Py2 / Py3 compatibility, but
I've now tested with both Py2 and Py3 and everything seems to
work.

llvm-svn: 313467
 2017-09-16 18:46:21 +00:00
Zachary Turner d6bca1c549 [lit] Fix some Python 3 compatibility issues. 
llvm-svn: 313466
 2017-09-16 18:45:44 +00:00
Zachary Turner f734daf36d [lit] Fix the lit unit tests. 
A few tests were manually constructing a LitConfig object, since
I added a new argument to it this was triggering some failures
I didn't detect.  `ninja check-lit` passes now.

llvm-svn: 313461
 2017-09-16 15:48:30 +00:00
Zachary Turner 42b6dcbcef [lit] Add a single process mode. 
This is helpful for debugging test failures since it removes
the multiprocessing pool from the picture.  This will obviously
slow down the test suite by a few orders of magnitude, so it
should only be used for debugging specific failures.

llvm-svn: 313460
 2017-09-16 15:31:34 +00:00