Commit Graph

206 Commits

Author SHA1 Message Date
Louis Dionne dac21fd29c [lit] Add an option to print all features used in tests
Lit test suites can tend to accumulate annotations that are not necessarily
relevant as time goes by, for example XFAILS on old compilers or platforms.
To help spot old annotations that can be cleaned up, it can be useful to
look at all features used inside a test suite.

This commit adds a new Lit option '--show-used-features' that prints all
the features used in XFAIL, REQUIRES and UNSUPPORTED of all tests that
are discovered.

Differential Revision: https://reviews.llvm.org/D78589
2020-05-29 07:00:05 -04:00
Louis Dionne 8a42bf24ae [lit] Move the recursiveExpansionLimit setting to TestingConfig
The LitConfig is shared across the whole test suite. However, since
enabling recursive expansion can be a breaking change for some test
suites, it's important to confine the setting to test suites that
enable it explicitly.

Note that other issues were raised with the way recursiveExpansionLimit
operates. However, this commit simply moves the setting to the right
place -- the mechanism by which it works can be improved independently.

Differential Revision: https://reviews.llvm.org/D77415
2020-04-06 13:58:00 -04:00
Louis Dionne 08d5426981 [lit] NFC: Move the flaky test logic to _runShTest
This minor refactoring allows reducing the amount of processing that
is duplicated when we re-run a flaky test. It also has the nice
side effect that libc++'s current test format supports flaky .sh.cpp
tests, because those are built on top of _runShTest, not executeShTest.
2020-03-27 09:32:58 -04:00
Louis Dionne faf415a1de [lit] Recursively expand substitutions
This allows defining substitutions in terms of other substitutions. For
example, a %build substitution could be defined in terms of a %cxx
substitution as '%cxx %s -o %t.exe' and the script would be properly
expanded.

Differential Revision: https://reviews.llvm.org/D76178
2020-03-27 09:25:26 -04:00
Louis Dionne 0bd1276eed [lit] NFC: Remove trailing whitespace
I keep having to remove them from my diffs!
2020-03-26 11:05:18 -04:00
Louis Dionne 8f64b02d33 [lit] Allow passing extra commands to executeShTest
This allows creating custom test formats on top of `executeShTest` that
inject commands at the beginning of the file being parsed, without
requiring these commands to physically appear in the test file itself.

For example, one could define a test format that prints out additional
debug information at the beginning of each test. More realistically,
this has been used to define custom test formats like one that supports
compilation failure tests (e.g. with the extension `compile.fail.cpp`)
by injecting a command that calls the compiler on the file itself and
expects it to fail.

Without this change, the only alternative is to create a temporary file
with the same content as the original test, then prepend the desired
`// RUN:` lines to that file, and call `executeShTest` on that file
instead. This is both slow and cumbersome to do.

Differential Revision: https://reviews.llvm.org/D76290
2020-03-24 15:02:37 -04:00
Louis Dionne f951b0f82d [lit] Add builtin support for flaky tests in lit
This commit adds a new keyword in lit called ALLOW_RETRIES. This keyword
takes a single integer as an argument, and it allows the test to fail that
number of times before it first succeeds.

This work attempts to make the existing test_retry_attempts more flexible
by allowing by-test customization, as well as eliminate libc++'s FLAKY_TEST
custom logic.

Differential Revision: https://reviews.llvm.org/D76288
2020-03-18 18:04:01 -04:00
Nico Weber f6544934b9 Make check-llvm run 50% faster on macOS, 18% faster on Windows.
While looking at cycle time graphs of some of my bots, I noticed
that 327894859c made check-llvm noticeably slower on macOS and
Windows.

As it turns out, the 5 substitutions added in that change were
enough to cause lit to thrash the build-in cache in re.compile()
(re.sub() is implemented as re.compile().sub()), and apparently
applySubstitutions() is on the cricital path and slow when all
regexes need to compile all the time.

(See `_MAXCACHE = 512` in cpython/Lib/re.py)

Supporting full regexes for lit substitutions seems a bit like
overkill, but for now add a simple unbounded cache to recover
the lost performance.

No intended behavior change.
2020-01-06 12:57:42 -05:00
Thomas Preud'homme ddd0bb8dba [lit] Remove lit's REQUIRES-ANY directive
Summary:
Remove REQUIRES-ANY alias lit directive since it is hardly used and can
be easily implemented using an OR expression using REQUIRES. Fixup
remaining testcases still using REQUIRES-ANY.

Reviewers: probinson, jdenny, gparker42

Reviewed By: gparker42

Subscribers: eugenis, asb, rbar, johnrusso, simoncook, sabuasal, niosHD, delcypher, jrtc27, zzheng, edward-jones, rogfer01, MartinMosbeck, brucehoult, the_o, PkmX, jocewei, lenary, s.egerton, pzheng, sameer.abuasal, apazos, luismarques, cfe-commits, #sanitizers, llvm-commits

Tags: #llvm, #clang, #sanitizers

Differential Revision: https://reviews.llvm.org/D71408
2019-12-17 10:36:36 +00:00
Daniel Sanders 327894859c Fix `sed -e s@FOO@%/S@` and similar when there's @'s in the working directory
Jenkins sometimes starts a new working directory by appending @2 (or
incrementing the number if the @n suffix is already there). This causes
several clang tests to fail as:
  s@INPUT_DIR@%/S/Inputs@g
gets expanded to the invalid:
  s@INPUT_DIR@/path/to/workdir@2/Inputs@g
                               ~~~~~~~~~~
where the part marked with ~'s is interpreted as the flags. These are
invalid and the test fails.

Previous fixes simply exchanged the @ character for another like | but
that's just moving the problem. Address it by adding an expansion that
escapes the @ character we're using as a delimiter as well as other magic
characters in the replacement of sed's s@@@.

There's still room for expansions to cause trouble though. One I ran into
while testing this was that having a directory called foo@bar causes lots
of `CHECK-NOT: foo` directives to match. There's also things like
directories containing `\1`
2019-12-03 15:44:01 -08:00
Joel E. Denny f1b4c4bfd0 [lit] Fix `not` calling internal commands
Without this patch, when using lit's internal shell, if `not` on a lit
RUN line calls `env`, `diff`, or any of the other in-process shell
builtins that lit implements, lit accidentally searches for the latter
as an external executable.  What's worse is that works fine when a
developer is testing on a platform where those executables are
available and behave as expected, but it then breaks on other
platforms.

`not` seems useful for some builtins, such as `diff`, so this patch
supports such uses.  `not --crash` does not seem useful for builtins,
so this patch diagnoses such uses.  In all cases, this patch ensures
shell builtins are found behind any sequence of `env` and `not`
commands.

`not` calling `env` calling an external command appears useful when
the `env` and external command are part of a lit substitution, as in
D65156.  This patch supports that by looking through any sequence of
`env` and `not` commands, building the environment from the `env`s,
and storing the `not`s.  The `not`s are then added back to the command
line without the `env`s to execute externally.  This avoids the need
to replicate the `not` implementation, in particular the `--crash`
option, in lit.

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D66531
2019-11-05 14:09:21 -05:00
Joel E. Denny cb2c4bb0e0 [lit] Fix internal env calling env
Without this patch, when using lit's internal shell, if `env` on a lit
RUN line calls `env`, lit accidentally searches for the latter as an
external executable.  What's worse is that works fine when a developer
is testing on a platform where `env` is available and behaves as
expected, but it then breaks on other platforms.

`env` calling `env` can make sense if one such `env` is within a lit
substitution, as in D65156 and D65121.  This patch ensures that lit
executes both as internal commands.

Reviewed By: probinson, mgorny, rnk

Differential Revision: https://reviews.llvm.org/D65697
2019-11-01 14:08:52 -04:00
Joel E. Denny 0d4e6519c5 [lit] Fix internal env calling other internal commands
Without this patch, when using lit's internal shell, if `env` on a lit
RUN line calls `cd`, `mkdir`, or any of the other in-process shell
builtins that lit implements, lit accidentally searches for the latter
as an external executable.

This patch puts such builtins in a map so that boilerplate for them
need be implemented only once.  This patch moves that handling after
processing of `env` so that `env` calling such a builtin can be
detected.  Finally, because such calls appear to be useless, this
patch takes the safe approach of diagnosing them rather than
supporting them.

Reviewed By: probinson, mgorny, rnk

Differential Revision: https://reviews.llvm.org/D66506
2019-10-31 14:37:51 -04:00
Joel E. Denny b163806cdc [lit] Make internal diff work in pipelines
When using lit's internal shell, RUN lines like the following
accidentally execute an external `diff` instead of lit's internal
`diff`:

```
 # RUN: program | diff file -
 # RUN: not diff file1 file2 | FileCheck %s
```

Such cases exist now, in `clang/test/Analysis` for example.  We are
preparing patches to ensure lit's internal `diff` is called in such
cases, which will then fail because lit's internal `diff` cannot
currently be used in pipelines and doesn't recognize `-` as a
command-line option.

To enable pipelines, this patch moves lit's `diff` implementation into
an out-of-process script, similar to lit's `cat` implementation.  A
follow-up patch will implement `-` to mean stdin.

Also, when lit's `diff` prints differences to stdout in Windows, this
patch ensures it always terminate lines with `\n` not `\r\n`.  That
way, strict FileCheck directives checking the `diff` output succeed in
both Linux and Windows.  This wasn't an issue when `diff` was internal
to lit because `diff` didn't then write to the true stdout, which is
where the `\n` -> `\r\n` conversion happened in Python.

Reviewed By: probinson, stella.stamenova

Differential Revision: https://reviews.llvm.org/D66574
2019-10-29 15:13:52 -04:00
Joel E. Denny e96e2d3227 Revert r375114: "[lit] Make internal diff work in pipelines"
This series of patches still breaks a Windows bot.

llvm-svn: 375121
2019-10-17 14:43:42 +00:00
Joel E. Denny 221e418f0c [lit] Make internal diff work in pipelines
When using lit's internal shell, RUN lines like the following
accidentally execute an external `diff` instead of lit's internal
`diff`:

```
 # RUN: program | diff file -
 # RUN: not diff file1 file2 | FileCheck %s
```

Such cases exist now, in `clang/test/Analysis` for example.  We are
preparing patches to ensure lit's internal `diff` is called in such
cases, which will then fail because lit's internal `diff` cannot
currently be used in pipelines and doesn't recognize `-` as a
command-line option.

To enable pipelines, this patch moves lit's `diff` implementation into
an out-of-process script, similar to lit's `cat` implementation.  A
follow-up patch will implement `-` to mean stdin.

Reviewed By: probinson, stella.stamenova

Differential Revision: https://reviews.llvm.org/D66574

llvm-svn: 375114
2019-10-17 14:02:42 +00:00
Joel E. Denny 2622419c78 [lit] Fix internal diff's --strip-trailing-cr and use it
Using GNU diff, `--strip-trailing-cr` removes a `\r` appearing before
a `\n` at the end of a line.  Without this patch, lit's internal diff
only removes `\r` if it appears as the last character.  That seems
useless.  This patch fixes that.

This patch also adds `--strip-trailing-cr` to some tests that fail on
Windows bots when D68664 is applied.  Based on what I see in the bot
logs, I think the following is happening.  In each test there, lit
diff is comparing a file with `\r\n` line endings to a file with `\n`
line endings.  Without D68664, lit diff reads those files in text
mode, which in Windows causes `\r\n` to be replaced with `\n`.
However, with D68664, lit diff reads the files in binary mode instead
and thus reports that every line is different, just as GNU diff does
(at least under Ubuntu).  Adding `--strip-trailing-cr` to those tests
restores the previous behavior while permitting the behavior of lit
diff to be more like GNU diff.

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D68839

llvm-svn: 375020
2019-10-16 17:21:57 +00:00
Joel E. Denny f095b8c425 [lit] Clean up internal diff's encoding handling
As suggested by rnk at D67643#1673043, instead of reading files
multiple times until an appropriate encoding is found, read them once
as binary, and then try to decode what was read.

For Python >= 3.5, don't fail when attempting to decode the
`diff_bytes` output in order to print it.

Avoid failures for Python 2.7 used on some Windows bots by
transforming diff output with `lit.util.to_string` before writing it
to stdout.

Finally, add some tests for encoding handling.

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D68664

llvm-svn: 375018
2019-10-16 17:21:24 +00:00
Joel E. Denny 7e385bd2f5 [lit] Extend internal diff to support -U
When using lit's internal shell, RUN lines like the following
accidentally execute an external `diff` instead of lit's internal
`diff`:

```
 # RUN: program | diff -U1 file -
```

Such cases exist now, in `clang/test/Analysis` for example.  We are
preparing patches to ensure lit's internal `diff` is called in such
cases, which will then fail because lit's internal `diff` doesn't
recognize `-U` as a command-line option.  This patch adds `-U`
support.

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D68668

llvm-svn: 374814
2019-10-14 19:59:30 +00:00
Joel E. Denny 3049748e15 Revert r374648: "Reland r374388: [lit] Make internal diff work in pipelines"
This series of patches still breaks a Windows bot.

llvm-svn: 374683
2019-10-12 18:52:46 +00:00
Joel E. Denny daf42dc36d Reland r374388: [lit] Make internal diff work in pipelines
To avoid breaking some tests, D66574, D68664, D67643, and D68668
landed together.  However, D68664 introduced an issue now addressed by
D68839, with which these are now all relanding.

Differential Revision: https://reviews.llvm.org/D66574

llvm-svn: 374648
2019-10-12 11:56:57 +00:00
Joel E. Denny a71511feb5 Revert r374388: "[lit] Make internal diff work in pipelines"
This breaks a Windows bot.

llvm-svn: 374429
2019-10-10 19:25:39 +00:00
Joel E. Denny df35ec8289 [lit] Make internal diff work in pipelines
When using lit's internal shell, RUN lines like the following
accidentally execute an external `diff` instead of lit's internal
`diff`:

```
 # RUN: program | diff file -
 # RUN: not diff file1 file2 | FileCheck %s
```

Such cases exist now, in `clang/test/Analysis` for example.  We are
preparing patches to ensure lit's internal `diff` is called in such
cases, which will then fail because lit's internal `diff` cannot
currently be used in pipelines and doesn't recognize `-` as a
command-line option.

To enable pipelines, this patch moves lit's `diff` implementation into
an out-of-process script, similar to lit's `cat` implementation.  A
follow-up patch will implement `-` to mean stdin.

Reviewed By: probinson, stella.stamenova

Differential Revision: https://reviews.llvm.org/D66574

llvm-svn: 374388
2019-10-10 17:39:24 +00:00
Joel E. Denny 0a0ea7ec99 Revert r372035: "[lit] Make internal diff work in pipelines"
This breaks a Windows bot.

llvm-svn: 372051
2019-09-16 23:47:46 +00:00
Joel E. Denny 2152ae985c [lit] Make internal diff work in pipelines
When using lit's internal shell, RUN lines like the following
accidentally execute an external `diff` instead of lit's internal
`diff`:

```
 # RUN: program | diff file -
 # RUN: not diff file1 file2 | FileCheck %s
```

Such cases exist now, in `clang/test/Analysis` for example.  We are
preparing patches to ensure lit's internal `diff` is called in such
cases, which will then fail because lit's internal `diff` cannot
currently be used in pipelines and doesn't recognize `-` as a
command-line option.

To enable pipelines, this patch moves lit's `diff` implementation into
an out-of-process script, similar to lit's `cat` implementation.  A
follow-up patch will implement `-` to mean stdin.

Reviewed By: probinson, stella.stamenova

Differential Revision: https://reviews.llvm.org/D66574

llvm-svn: 372035
2019-09-16 21:22:29 +00:00
Joel E. Denny 3c577bb415 [lit] Diagnose insufficient args to internal env
Without this patch, failing to provide a subcommand to lit's internal
`env` results in either a python `IndexError` or an attempt to execute
the final `env` argument, such as `FOO=1`, as a command.  This patch
diagnoses those cases with a more helpful message.

Reviewed By: stella.stamenova

Differential Revision: https://reviews.llvm.org/D66482

llvm-svn: 369620
2019-08-22 03:42:01 +00:00
Joel E. Denny a67d81e4e2 [lit] Print internal env commands
Without this patch, the internal `env` command removes `env` and its
args from the command line while parsing it.  This patch modifies a
copy instead so that the original command line is printed.

Reviewed By: stella.stamenova, rnk

Differential Revision: https://reviews.llvm.org/D65624

llvm-svn: 367752
2019-08-03 06:08:19 +00:00
Alexey Bader 07cbeaa118 [LIT] Emit timeout error message only if timeout was reached
Summary:
This improves readability of LIT output: previously
error messages gets emitted that say that there was no error:

error: command reached timeout: False

Patch by Alexey Sachkov.

Reviewers: ddunbar, mgorny, modocache

Reviewed By: mgorny

Subscribers: delcypher, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64240

llvm-svn: 365895
2019-07-12 12:48:43 +00:00
Adrian McCarthy 46e6e13298 Fix some lit test ResourceWarnings on Windows
When running LLDB lit tests on Windows, the system selects a debug version
of Python, which was issuing lots of ResourceWarnings about files that
weren't closed.  There are two kinds of them, and each test triggered one
of each.

This patch fixes one kind by ensuring TestRunner explicitly close the
temporary files created for routing stderr.  This is important on Windows
but has no net effect on Posix systems.

The remaining ResourceWarnings are more elusive; the bug may lie in
the Python library subprocess.py, and it may be Windows-specific.

Differential Revision: https://reviews.llvm.org/D63102

llvm-svn: 363700
2019-06-18 16:36:57 +00:00
Michal Gorny 0c28a8f628 [lit] Fix UnicodeEncodeError when test commands contain non-ASCII chars
Ensure that the bash script written by lit TestRunner is open with UTF-8
encoding when using Python 3.  Otherwise, attempt to write non-ASCII
characters causes UnicodeEncodeError.  This happened e.g. with
the following LLD test:

UNRESOLVED: lld :: ELF/format-binary-non-ascii.s (657 of 2119)
******************** TEST 'lld :: ELF/format-binary-non-ascii.s' FAILED ********************
Exception during script execution:
Traceback (most recent call last):
  File "/home/mgorny/llvm-project/llvm/utils/lit/lit/worker.py", line 63, in _execute_test
    result = test.config.test_format.execute(test, lit_config)
  File "/home/mgorny/llvm-project/llvm/utils/lit/lit/formats/shtest.py", line 25, in execute
    self.execute_external)
  File "/home/mgorny/llvm-project/llvm/utils/lit/lit/TestRunner.py", line 1644, in executeShTest
    res = _runShTest(test, litConfig, useExternalSh, script, tmpBase)
  File "/home/mgorny/llvm-project/llvm/utils/lit/lit/TestRunner.py", line 1590, in _runShTest
    res = executeScript(test, litConfig, tmpBase, script, execdir)
  File "/home/mgorny/llvm-project/llvm/utils/lit/lit/TestRunner.py", line 1157, in executeScript
    f.write('{ ' + '; } &&\n{ '.join(commands) + '; }')
UnicodeEncodeError: 'ascii' codec can't encode character '\xa3' in position 274: ordinal not in range(128)

Differential Revision: https://reviews.llvm.org/D63254

llvm-svn: 363388
2019-06-14 13:31:48 +00:00
Adrian McCarthy 4ca8435528 Fix string literals to avoid deprecation warnings in regexp patterns
In LLDB, where tests run with the debug version of Python, we get a
series of deprecation warnings because escape sequences like `\(` are
being treated as part of the string literal rather than an escape for
the regexp pattern.

NFC intended.

Differential Revision: https://reviews.llvm.org/D62882

llvm-svn: 362846
2019-06-07 21:14:33 +00:00
Saleem Abdulrasool 50e609611c lit: make rm python 3 friendly (NFC)
Add some alterations for python 3 compatibility.

llvm-svn: 357789
2019-04-05 18:00:49 +00:00
Saleem Abdulrasool d3a85a26b6 lit: support long paths on Windows
Use ctypes to call into SHFileOperationW with the extended NT path to allow us
to remove paths which exceed 261 characters on Windows. This functionality is
exercised by swift's test suite.

llvm-svn: 357778
2019-04-05 16:48:00 +00:00
Serge Guelton 54be909aa0 Add Support for Creating and Deleting Unicode Files and Directories in Lit
This enables lit to work with unicode file names via mkdir, rm, and redirection.
Lit still uses utf-8 internally, but converts to utf-16 on Windows, or just utf-8
bytes on everything else.

Committed on behalf of Jason Mittertreiner
Differential Revision: https://reviews.llvm.org/D56754

llvm-svn: 355122
2019-02-28 19:16:17 +00:00
Eric Fiselier 1989f7e04f make XFAIL, REQUIRES, and UNSUPPORTED support multi-line expressions
llvm-svn: 351668
2019-01-20 00:51:02 +00:00
Nico Weber 93c5e2abb6 Revert 349677, it contained a whole bunch of stuff I did not mean to commit
llvm-svn: 349678
2018-12-19 20:19:58 +00:00
Nico Weber 2589927307 [gn build] Add build file for clang/lib/Basic and dependencies
Adds a build file for clang-tblgen and an action for running it, and uses that
to process all the .td files in include/clang/Basic.

Also adds an action to write include/clang/Config/config.h and
include/clang/Basic/Version.inc.

Differential Revision: https://reviews.llvm.org/D55847

llvm-svn: 349677
2018-12-19 20:18:59 +00:00
Tom Stellard 04cbe721da lit: Use sys.executable for executing builtin commands
Summary:
The python executable may not exist on all systems so use sys.executable
instead.

Reviewers: ddunbar, stella.stamenova

Subscribers: delcypher, llvm-commits

Differential Revision: https://reviews.llvm.org/D51511

llvm-svn: 341244
2018-08-31 20:15:31 +00:00
Joel E. Denny fc01dd281d [lit] Terminate ": RUN at line N" with ";" not "&&"
This fixes projects/compiler-rt/test/fuzzer/sigusr.test, which was
broken by r333614.  The trouble was that "&&" changes the command for
which "$!" gives the pid.

llvm-svn: 333620
2018-05-31 03:40:37 +00:00
Joel E. Denny 31b373963f [lit] Report line number for failed RUN command
(Relands r333584, reverted in 333592.)

When debugging test failures with -vv (or -v in the case of the
internal shell), this makes it easier to locate the RUN line that
failed.  For example, clang's test/Driver/linux-ld.c has 892 total RUN
lines, and clang's test/Driver/arm-cortex-cpus.c has 424 RUN lines
after concatenation for line continuations.

When reading the generated shell script, this also makes it easier to
locate the RUN line that produced each command.

To support reporting RUN line numbers in the case of the internal
shell, this patch extends the internal shell to support the null
command, ":", except pipelines are not supported.

To support reporting RUN line numbers in the case of windows cmd.exe
as the external shell, this patch extends -vv to set "echo on" instead
of "echo off" in bat files.  (Support for windows cmd.exe as a lit
external shell will likely be dropped later, but I found out too
late.)

Reviewed By: delcypher,	asmith, stella.stamenova, jmorse, lebedev.ri, rnk

Differential Revision: https://reviews.llvm.org/D44598

llvm-svn: 333614
2018-05-31 00:55:32 +00:00
Joel E. Denny 71792c741e Revert r333584: [lit] Report line number for failed RUN command
It breaks test-suite.

llvm-svn: 333592
2018-05-30 21:07:27 +00:00
Joel E. Denny b6423479a1 [lit] Report line number for failed RUN command
(Relands r330755 (reverted in r330848) with fix for PR37239.)

When debugging test failures with -vv (or -v in the case of the
internal shell), this makes it easier to locate the RUN line that
failed.  For example, clang's test/Driver/linux-ld.c has 892 total RUN
lines, and clang's test/Driver/arm-cortex-cpus.c has 424 RUN lines
after concatenation for line continuations.

When reading the generated shell script, this also makes it easier to
locate the RUN line that produced each command.

To support reporting RUN line numbers in the case of the internal
shell, this patch extends the internal shell to support the null
command, ":", except pipelines are not supported.

To support reporting RUN line numbers in the case of windows cmd.exe
as the external shell, this patch extends -vv to set "echo on" instead
of "echo off" in bat files.  (Support for windows cmd.exe as a lit
external shell will likely be dropped later, but I found out too
late.)

Reviewed By: delcypher,	asmith, stella.stamenova, jmorse, lebedev.ri, rnk

Differential Revision: https://reviews.llvm.org/D44598

llvm-svn: 333584
2018-05-30 19:42:27 +00:00
Reid Kleckner 1ca666886f Revert r330755 "[lit] Report line number for failed RUN command"
It is causing many tests to fail on Windows buildbots:
http://lab.llvm.org:8011/builders/clang-x64-ninja-win7/builds/10211

llvm-svn: 330848
2018-04-25 17:30:00 +00:00
Joel E. Denny 8a475307bd [lit] Report line number for failed RUN command
When debugging test failures with -vv (or -v in the case of the
internal shell), this makes it easier to locate the RUN line that
failed.  For example, clang's test/Driver/linux-ld.c has 892 total RUN
lines, and clang's test/Driver/arm-cortex-cpus.c has 424 RUN lines
after concatenation for line continuations.

When reading the generated shell script, this also makes it easier to
locate the RUN line that produced each command.

To support reporting RUN line numbers in the case of the internal
shell, this patch extends the internal shell to support the null
command, ":", except pipelines are not supported.

Reviewed By: asmith, delcypher

Differential Revision: https://reviews.llvm.org/D44598

llvm-svn: 330755
2018-04-24 18:43:25 +00:00
Aaron Smith 0c8ee891a7 [lit] Remove duplicate to_string method
There are two versions of to_string used by TestRunner.py. The one defined 
in TestRunner.py and the one defined in utils/lit/lit/util.py. The util.py
version is superior to the TestRunner.py version.

This change removes the duplicate to_string in TestRunner.py in favor of
always using the version from util.py. Beside removing duplicate code, this
makes it easier to debug TestRunner.py since only one version of to_string
is used.

Patch by Stella Stamenova!

llvm-svn: 329972
2018-04-12 23:45:15 +00:00
Aaron Smith e5ee89c0d3 [lit] Fix several Python 2/3 compatibility issues and tests
- In Python 2.x, basestring is the base string type, but in 
  Python 3.x basestring is not defined and instead str includes 
  unicode strings.

- When Python is in a path that includes spaces, it needs to 
  be specified with quotes in the test files for it to run.

- The cache.ll test relies on files of a specific size being 
  created by Python, but on some versions of Windows the 
  files that are created by the current code are one byte 
  larger than expected. To fix the test, update file creation 
  to always make files of the expected size.

Patch by Stella Stamenova!

llvm-svn: 329466
2018-04-07 00:21:28 +00:00
Aaron Smith 061354b586 [lit] Prefer opening files with open (Python 2) rather than io.open which requires io.
Only rely on Python 3 (io.open) when necessary. This puts TestRunnyer.py closer to how it behaved 
before the changes introduced in D43165 and silences a few Windows build bot failures.

Thanks to Stella Stamenova for the patch!

llvm-svn: 329037
2018-04-03 00:22:12 +00:00
Aaron Smith 637fd446d9 [lit] One more try at fixing TestRunner.py for D43165
llvm-svn: 329026
2018-04-02 22:34:35 +00:00
Reid Kleckner 30b215e8f7 [lit] Attempt to fix builtin diff code for Python 2
llvm-svn: 329024
2018-04-02 22:19:42 +00:00
Aaron Smith 98d31d6880 [lit] Fix problem in how Python versions open files with different encodings
Reapply D43165 which was reverted because of different versions of python failing. 
The one line fix for the different python versions was commited at the same time
that D43165 was reverted. If this change is giving you issues then get in touch
with your python version and we will fix it. 

llvm-svn: 329022
2018-04-02 22:08:56 +00:00