maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
290,214 Commits 23 Branches 0 Tags 2.2 GiB
Commit Graph

609 Commits

Author SHA1 Message Date
Dan Liew 6dfcc78364 [lit] Try to make `shtest-timeout.py` test more reliable by using a 
larger timeout value. This really isn't very good because it will
still be susceptible to machine performance.

While we are here also fix a bug in validation of
`maxIndividualTestTime` where previously it wasn't checked if the
type was an int.

rdar://problem/40221572

llvm-svn: 332987
 2018-05-22 15:06:29 +00:00
Dan Liew 9f7786855c [lit] Don't run `slow.py` in `shtest-timeout.py` test. 
The program used to be used in `quick_then_slow.py` but that was
removed in r328702. The tests always run `slow.py` on its own but
this doesn't really test additional code so we'll just drop running
`slow.py` so the tests run faster.

rdar://problem/40221572

llvm-svn: 332986
 2018-05-22 15:06:24 +00:00
Dan Liew 8b4d36a530 [lit] Don't check output of commands used in `shtest-timeout.py` test. 
If the system is under heavy load 1 second might not be long enough
for it to produce output which could lead to spurious test failures.
What matters is that the right test cases reach a timeout.

rdar://problem/40221572

llvm-svn: 332985
 2018-05-22 15:06:20 +00:00
Douglas Yung 2dd62a3d99 Mark test with "REQUIRES: shell" since it directly invokes "sh" and was failing on Windows. 
llvm-svn: 332563
 2018-05-17 01:36:25 +00:00
Alexander Richardson 1bffd0f741 Escape ]]> in xunit xml output 
Summary:
This sequence ends the CDATA block so any characters after that are no
longer escaped. This can be fixed by replacing "]]>" with "]]]]><![CDATA[>".

Reviewers: cmatthews

Reviewed By: cmatthews

Differential Revision: https://reviews.llvm.org/D46886

llvm-svn: 332440
 2018-05-16 09:00:28 +00:00
Chris Matthews 48357c43d5 remove output xml incase it is leftover from another run 
llvm-svn: 332424
 2018-05-16 00:37:00 +00:00
Chris Matthews 5d1014bff5 Use not to catch unexpected pass as well as remove old test results 
As per review feedback, make sure we rm temp files, and make the return
code checking for lit more specific.

llvm-svn: 332423
 2018-05-16 00:33:29 +00:00
Chris Matthews f616804903 Requirements can have & in them! 
Lets escape those so the XML is valid!

llvm-svn: 332161
 2018-05-12 00:13:54 +00:00
Chris Matthews 811a83d036 Add the message attribute to skipped 
JUnit xml allows for a message attribute to be displayed on skips. Lets
populate that with an analysis of why we skipped the test.

llvm-svn: 332156
 2018-05-11 23:15:11 +00:00
Chris Matthews 6b3db8519d Overhaul unicode handling in xunit output 
I have seen a lot of errors where the xunit does not encode unicode
test output correctly.  Handle that explicitly now.

llvm-svn: 332148
 2018-05-11 22:18:22 +00:00
Chris Matthews d1386a88dd [LIT] replace output escapes wit a cdata block 
CDATA blocks don't need to have XML stuff escaped. Makes sense to wrap
output in them instead of escaping.

llvm-svn: 332116
 2018-05-11 18:38:02 +00:00
Chris Matthews 7d6224d2fa Support Unsupported Tests in xunit output 
We were reporting  "Unsupported" tests in xunit as passes, however since
they are not run, it make more sense to mark them as skipped. The Junit
xml standard has support for that, so lets use it.

llvm-svn: 332065
 2018-05-11 00:25:43 +00:00
Chris Matthews 5f74591847 Refactor xunit test case builder to not use as much str addition 
String concatenation in python is slow.  Refactor to not concatenate the
possibly large strings of test output and instead write them directly
to the output file.

llvm-svn: 332064
 2018-05-11 00:25:42 +00:00
Chris Matthews 4855c5f717 [LIT] Move xunit tests tests into their own location, and and add failures 
Failures will increase coverage.

llvm-svn: 332056
 2018-05-10 22:51:28 +00:00
Chris Matthews c3f259cb6c [LIT] Add the missing file 
I forgot to commit this file.

llvm-svn: 331946
 2018-05-10 00:08:39 +00:00
Chris Matthews bd9ffc6243 Refactor test incase results are backwards 
Looks like results can come in either way in this file.  Loosen the ordering constraints.

llvm-svn: 331945
 2018-05-10 00:06:17 +00:00
Chris Matthews 069a1eb3bb [LIT] Handle xml characters in test names 
Lit creates malformed xml when the test case has an & in the name.

Escape those correctly.

This also adds a test case which I will add other nasty encoding issues to in some followup commits.

llvm-svn: 331942
 2018-05-09 23:48:32 +00:00
Nico Weber 3934dba059 Remove 'abi-breaking-checks' lit feature. 
Its only two uses were removed in r311730.
Effectively reverts r304851 (but that code has removed around a bit since then).
https://reviews.llvm.org/D46619

clang side done in r331871.

llvm-svn: 331872
 2018-05-09 12:39:39 +00:00
Saleem Abdulrasool cc2da59f69 lit: flesh out `SubsituteCaptures` further 
Add overloads for `__len__` and `__getitem__` to allow use of this class
on Linux as well as Windows.  With these overloads, lit can be used on
both hosts for the swift testsuite.

llvm-svn: 331431
 2018-05-03 04:45:43 +00:00
Reid Kleckner 1ca666886f Revert r330755 "[lit] Report line number for failed RUN command" 
It is causing many tests to fail on Windows buildbots:
http://lab.llvm.org:8011/builders/clang-x64-ninja-win7/builds/10211

llvm-svn: 330848
 2018-04-25 17:30:00 +00:00
Joel E. Denny 8a475307bd [lit] Report line number for failed RUN command 
When debugging test failures with -vv (or -v in the case of the
internal shell), this makes it easier to locate the RUN line that
failed.  For example, clang's test/Driver/linux-ld.c has 892 total RUN
lines, and clang's test/Driver/arm-cortex-cpus.c has 424 RUN lines
after concatenation for line continuations.

When reading the generated shell script, this also makes it easier to
locate the RUN line that produced each command.

To support reporting RUN line numbers in the case of the internal
shell, this patch extends the internal shell to support the null
command, ":", except pipelines are not supported.

Reviewed By: asmith, delcypher

Differential Revision: https://reviews.llvm.org/D44598

llvm-svn: 330755
 2018-04-24 18:43:25 +00:00
Dan Liew 97a60c21d4 [lit] Remove spurious `-` in invocation of lit in 
`shtest-xunit-output.py` test.

Although there is no `-` file Jeremy Morse has reported to me that it
causes problems in their setup because lit tries to find it and ends up
loading an out of tree lit configuration file.

llvm-svn: 330728
 2018-04-24 15:42:00 +00:00
Nico Weber 1c2c321b47 Mostly revert r330672. 
The test is apparently needed e.g. for check-cfi on Windows where we get
  'C:/b/slave/sanitizer-windows/build/./bin/clang.exe': command not found
without it.  Try to fix the problem that was fixed by r330672 by also checking
for isabs() instead.

llvm-svn: 330673
 2018-04-24 01:24:42 +00:00
Nico Weber a425db5a1e Remove code that's almost always dead, and harmful if not. 
lit's util.which() would check if the passed-in path existed directly,
and if so return it as-is.  This is never the case when running llvm's, clang's,
or lld's tests normally.  But when running `./llvm-lit path/to/clang/test`
with a cwd of llvm-build/bin, this if would detect that clang exists at path
'clang' and return 'clang' as the discovered clang binary -- and then lit would
use the " clang " -> "*** Do not use 'clang' in tests, use '%clang'. ***"
substitution to replace that with a broken test.  By removing this early
return, lit ends up with the usual absolute path and everything works even
in this uncommon case.

llvm-svn: 330672
 2018-04-24 01:05:04 +00:00
Dan Liew 872b8ea596 [lit] Fix a bug where UNRESOLVED tests were not handled in the XUnit 
XML printer.

A test has been added that tries to comprehensively test emitting
XUnit XML output for shell tests.

Differential Revision: https://reviews.llvm.org/D45567

llvm-svn: 330409
 2018-04-20 10:11:41 +00:00
Aaron Smith 0c8ee891a7 [lit] Remove duplicate to_string method 
There are two versions of to_string used by TestRunner.py. The one defined 
in TestRunner.py and the one defined in utils/lit/lit/util.py. The util.py
version is superior to the TestRunner.py version.

This change removes the duplicate to_string in TestRunner.py in favor of
always using the version from util.py. Beside removing duplicate code, this
makes it easier to debug TestRunner.py since only one version of to_string
is used.

Patch by Stella Stamenova!

llvm-svn: 329972
 2018-04-12 23:45:15 +00:00
Aaron Smith e5ee89c0d3 [lit] Fix several Python 2/3 compatibility issues and tests 
- In Python 2.x, basestring is the base string type, but in 
  Python 3.x basestring is not defined and instead str includes 
  unicode strings.

- When Python is in a path that includes spaces, it needs to 
  be specified with quotes in the test files for it to run.

- The cache.ll test relies on files of a specific size being 
  created by Python, but on some versions of Windows the 
  files that are created by the current code are one byte 
  larger than expected. To fix the test, update file creation 
  to always make files of the expected size.

Patch by Stella Stamenova!

llvm-svn: 329466
 2018-04-07 00:21:28 +00:00
Reid Kleckner bdbb712b6d 'cat' command for internal shell - Support Python 3 
LLVM Bug Id : 36449

Revision 328563 caused tests to fail under python 3.

This patch modified cat.py file to support both python 2 and 3.
This patch also fixes CRLF issues on Windows.

Patch by Chamal de Silva

Differential Revision: https://reviews.llvm.org/D45077

llvm-svn: 329123
 2018-04-03 22:38:25 +00:00
Aaron Smith 061354b586 [lit] Prefer opening files with open (Python 2) rather than io.open which requires io. 
Only rely on Python 3 (io.open) when necessary. This puts TestRunnyer.py closer to how it behaved 
before the changes introduced in D43165 and silences a few Windows build bot failures.

Thanks to Stella Stamenova for the patch!

llvm-svn: 329037
 2018-04-03 00:22:12 +00:00
Aaron Smith 637fd446d9 [lit] One more try at fixing TestRunner.py for D43165 
llvm-svn: 329026
 2018-04-02 22:34:35 +00:00
Reid Kleckner 30b215e8f7 [lit] Attempt to fix builtin diff code for Python 2 
llvm-svn: 329024
 2018-04-02 22:19:42 +00:00
Aaron Smith 98d31d6880 [lit] Fix problem in how Python versions open files with different encodings 
Reapply D43165 which was reverted because of different versions of python failing. 
The one line fix for the different python versions was commited at the same time
that D43165 was reverted. If this change is giving you issues then get in touch
with your python version and we will fix it. 

llvm-svn: 329022
 2018-04-02 22:08:56 +00:00
Aaron Smith b50baa1c4b [lit] Use io.open to compare two files since it supports different encodings while older versions of open do not 
llvm-svn: 329020
 2018-04-02 21:44:51 +00:00
Reid Kleckner 2aae49d3b2 Revert r329012 "[lit] Fix problem in how Python versions open files with different encodings" 
This doesn't work with Python 2. See this build-in-progress:
http://lab.llvm.org:8011/builders/clang-x86-windows-msvc2015/builds/11105

llvm-svn: 329017
 2018-04-02 21:33:46 +00:00
Aaron Smith 321c2487d7 [lit] Fix problem in how Python versions open files with different encodings 
Summary:
This issue was found when running the clang unit test on Windows. Python 3.x cannot open some of the files that the tests are using with a simple open because of their encoding. Python 2.7+ and Python 3.x both support io.open which allows for an encoding to be specified. 

This change will determine whether two files being compared should be opened (and then compared) as text or binary and whether to use utf-8 or the default encoding before proceeding with a line-by-line comparison.

Patch by Stella Stamenova!

Reviewers: zturner, llvm-commits, rnk, MaggieYi

Reviewed By: zturner

Subscribers: asmith, MatzeB, stella.stamenova, delcypher, llvm-commits

Differential Revision: https://reviews.llvm.org/D43165

llvm-svn: 329012
 2018-04-02 20:57:06 +00:00
Dan Liew 8ade9e75b0 Revert "[lit] Temporarily disable shtest-timeout.py on darwin" 
This reverts commit 771829b640a5494ab65c810dd6b4330522bf3a33 (rr328598)

Hopefully the test will now pass on the bots.

rdar://problem/38774530

llvm-svn: 328703
 2018-03-28 13:55:13 +00:00
Dan Liew 7efde3c440 [lit] Remove a timing senstive part of `shtest-timeout.py` 
The `shtest-timeout.py` test was failing intermittently. It looks like
the issue is that on a resource constrained system lit is unable to run
`quick_then_slow.py` twice and print out the messages the tests expects
within the one second timeout.

The underlying issue is that the test is dependent on the performance of
the host machine is a rather fragile way. This is due to hardcoding
timeout values and having assumptions that the host machine is able to
perform a certain amount of work within the hardcoded timeout values.

We could increase the timeout values but that doesn't really fix the
underlying issue. Instead this patch removes one of fragile assumptions
in the hope that this will be enough to fix the bots.
There are other fragile assumptions in this test (e.g. `quick.py` can be
executed in less than 1 second). If the bots continue to fail we'll have
to revisit this.

rdar://problem/38774530

llvm-svn: 328702
 2018-03-28 13:55:08 +00:00
Mircea Trofin 56ba71b2a7 Revert "Revert "[lit] Generalized /dev/null support on Windows."" 
Summary:
This reverts commit r328596.

Checking if the arguments are strings before testing if they contain "/dev/null".

Reviewers: rnk

Reviewed By: rnk

Subscribers: delcypher, llvm-commits

Differential Revision: https://reviews.llvm.org/D44914

llvm-svn: 328603
 2018-03-27 01:39:17 +00:00
Jan Korous 1e0e0b077d [lit] Temporarily disable shtest-timeout.py on darwin 
Disabled until fixed in order to avoid random failures on green dragon.

rdar://problem/38774530

llvm-svn: 328598
 2018-03-27 00:16:28 +00:00
Mircea Trofin 373c445c24 Revert "[lit] Generalized /dev/null support on Windows." 
This reverts commit ca7fdbb974384ce5a05528b22a41d46b1cc13e92.

llvm-svn: 328596
 2018-03-26 23:59:39 +00:00
Mircea Trofin 88911686c8 [lit] Generalized /dev/null support on Windows. 
Generalized /dev/null remapping on Windows, and added test.

Reviewers: rnk

Reviewed By: rnk

Subscribers: amccarth, zturner, delcypher, llvm-commits

Differential Revision: https://reviews.llvm.org/D44771

llvm-svn: 328589
 2018-03-26 22:41:06 +00:00
Reid Kleckner 8252892951 [lit] Implement 'cat' command for internal shell 
Fixes PR36449

Patch by Chamal de Silva

Differential Revision: https://reviews.llvm.org/D43501

llvm-svn: 328563
 2018-03-26 18:05:12 +00:00
Brian Homerding d5c558ff21 [lit] - Allow 1 test to report multiple micro-test results to provide support for microbenchmarks. 
Summary:
These changes are to allow to a Result object to have nested Result objects in
order to support microbenchmarks. Currently lit is restricted to reporting one
result object for one test, this change provides support tests that want to
report individual timings for individual kernels.

This revision is the result of the discussions in
https://reviews.llvm.org/D32272#794759,
https://reviews.llvm.org/D37421#f8003b27 and https://reviews.llvm.org/D38496.
It is a separation of the changes purposed in https://reviews.llvm.org/D40077.

This change will enable adding LCALS (Livermore Compiler Analysis Loop Suite)
collection of loop kernels to the llvm test suite using the google benchmark
library (https://reviews.llvm.org/D43319) with tracking of individual kernel
timings.

Previously microbenchmarks had been handled by using macros to section groups
of microbenchmarks together and build many executables while still getting a
grouped timing (MultiSource/TSVC). Recently the google benchmark library was
added to the test suite and utilized with a litsupport plugin. However the
limitation of 1 test 1 result limited its use to passing a runtime option to
run only 1 microbenchmark with several hand written tests
(MicroBenchmarks/XRay). This runs the same executable many times with different
hand-written tests. I will update the litsupport plugin to utilize the new
functionality (https://reviews.llvm.org/D43316).

These changes allow lit to report micro test results if desired in order to get
many precise timing results from 1 run of 1 test executable.


Reviewers: MatzeB, hfinkel, rengolin, delcypher

Differential Revision: https://reviews.llvm.org/D43314

llvm-svn: 327422
 2018-03-13 16:37:59 +00:00
Saleem Abdulrasool d5453b2c3f utils: add a helper class to lit for captured substitutions 
On Windows, if the substitution contains a back reference, it would
removed due to the replacement of the escape character in lit. Create a
helper class to avoid this which will simply ignore the replacement and
mark the substitution as having capture groups being referenced.

llvm-svn: 327082
 2018-03-09 00:06:10 +00:00
Lei Huang 9db7bd5cab Losen time contraint to accommodate system loads 
llvm-svn: 326359
 2018-02-28 20:05:24 +00:00
Rui Ueyama 7c9ad29304 Remove "--full-shutdown" and instead use an environment variable LLD_IN_TEST. 
We are running lld tests with "--full-shutdown" option because we don't
want to call _exit() in lld if it is running tests. Regular shutdown
is needed for leak sanitizer.

This patch changes the way how we tell lld that it is running tests.
Now "--full-shutdown" is removed, and LLD_IN_TEST environment variable
is used instead.

This patch enables full shutdown on all ports, e.g. ELF, COFF and wasm.
Previously, we enabled it only for ELF.

Differential Revision: https://reviews.llvm.org/D43410

llvm-svn: 325413
 2018-02-16 23:41:48 +00:00
Rui Ueyama 14ee1140c1 Make sure we invoke ld64.lld and ld-wasm in the build directory. 
llvm-svn: 325397
 2018-02-16 22:19:38 +00:00
Victor Leschuk 33b9fa7caa [lit] Pass CLANG env var to testing configuration 
Allow CLANG environment variable be copied into the testing configuration
and proper support testing with a custom path to the clang executable.

Signed-off-by: Vladimir Vereschaka <vvereschaka@accesssoftek.com>
llvm-svn: 324706
 2018-02-09 05:28:38 +00:00
Max Moroz 975eacada5 [lit] Implement "-r" option for builtin "diff" command + a test using that. 
Summary:
That would allow to recursively compare directories in tests using
"diff -r" on Windows in a similar way as it can be done on Linux or Mac.

Reviewers: zturner, morehouse, vsk

Reviewed By: zturner

Subscribers: kcc, llvm-commits

Differential Revision: https://reviews.llvm.org/D41776

llvm-svn: 322102
 2018-01-09 18:23:34 +00:00
Tom Stellard d703b198f2 [lit] Bump version number 
llvm-svn: 321735
 2018-01-03 16:35:51 +00:00