Commit Graph

138 Commits

Author SHA1 Message Date
Mircea Trofin ee703eb472 [FileCheck] Address unused prefixes in tests
Added -allow-unused-prefixes as necessary.

Differential Revision: https://reviews.llvm.org/D90496
2020-10-30 14:14:02 -07:00
Mircea Trofin 871d658c9c [FileCheck] Report missing prefixes when more than one is provided.
If more than a prefix is provided - e.g. --check-prefixes=CHECK,FOO - we
don't report if (say) FOO is never used. This may lead to a gap in our
test coverage.

This patch introduces a new option, --allow-unused-prefixes. It
currently is set to true, keeping today's behavior. After we explicitly
set it in tests where this behavior was actually intentional, we will
switch it to false by default.

Differential Revision: https://reviews.llvm.org/D90281
2020-10-30 12:39:29 -07:00
Thomas Preud'homme 998709b7d5 [FileCheck] Add precision to format specifier
Add printf-style precision specifier to pad numbers to a given number of
digits when matching them if the value is smaller than the given
precision. This works on both empty numeric expression (e.g. variable
definition from input) and when matching a numeric expression. The
syntax is as follows:

[[#%.<precision><format specifier>, ...]

where <format specifier> is optional and ... can be a variable
definition or not with an empty expression or not. In the absence of a
precision specifier, a variable definition will accept leading zeros.

Reviewed By: jhenderson, grimar

Differential Revision: https://reviews.llvm.org/D81667
2020-08-30 19:40:57 +01:00
Thomas Preud'homme 2c9131665d Test all CHECK-NOT in a block even if one fails
This commit makes FileCheck print all CHECK-NOT directive failure in a
CHECK-NOT block even if one fails. Prior to that, it would stop trying
to match CHECK-NOT directive as soon as one in the block fails.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D86315
2020-08-24 15:45:05 +01:00
Joel E. Denny 9f86b8ec41 [FileCheck] Report captured variables
Report captured variables in input dumps and traces.  For example:

```
$ cat check
CHECK: hello [[WHAT:[a-z]+]]
CHECK: goodbye [[WHAT]]

$ FileCheck -dump-input=always -vv check < input |& tail -8
<<<<<<
           1: hello world
check:1'0     ^~~~~~~~~~~
check:1'1           ^~~~~ captured var "WHAT"
           2: goodbye world
check:2'0     ^~~~~~~~~~~~~
check:2'1                   with "WHAT" equal to "world"
>>>>>>

$ FileCheck -dump-input=never -vv check < input
check2:1:8: remark: CHECK: expected string found in input
CHECK: hello [[WHAT:[a-z]+]]
       ^
<stdin>:1:1: note: found here
hello world
^~~~~~~~~~~
<stdin>:1:7: note: captured var "WHAT"
hello world
      ^~~~~
check2:2:8: remark: CHECK: expected string found in input
CHECK: goodbye [[WHAT]]
       ^
<stdin>:2:1: note: found here
goodbye world
^~~~~~~~~~~~~
<stdin>:2:1: note: with "WHAT" equal to "world"
goodbye world
^
```

Reviewed By: thopre

Differential Revision: https://reviews.llvm.org/D83651
2020-07-28 19:15:18 -04:00
Joel E. Denny d680711b94 [FileCheck] Extend -dump-input with substitutions
Substitutions are already reported in the diagnostics appearing before
the input dump in the case of failed directives, and they're reported
in traces (produced by `-vv -dump-input=never`) in the case of
successful directives.  However, those reports are not always
convenient to view while investigating the input dump, so this patch
adds the substitution report to the input dump too.  For example:

```
$ cat check
CHECK: hello [[WHAT:[a-z]+]]
CHECK: [[VERB]] [[WHAT]]

$ FileCheck -vv -DVERB=goodbye check < input |& tail -8
<<<<<<
           1: hello world
check:1       ^~~~~~~~~~~
           2: goodbye word
check:2'0     X~~~~~~~~~~~ error: no match found
check:2'1                  with "VERB" equal to "goodbye"
check:2'2                  with "WHAT" equal to "world"
>>>>>>
```

Without this patch, the location reported for a substitution for a
directive match is the directive's full match range.  This location is
misleading as it implies the substitution itself matches that range.
This patch changes the reported location to just the match range start
to suggest the substitution is known at the start of the match.  (As
in the above example, input dumps don't mark any range for
substitutions.  The location info in that case simply identifies the
right line for the annotation.)

Reviewed By: mehdi_amini, thopre

Differential Revision: https://reviews.llvm.org/D83650
2020-07-28 19:15:18 -04:00
Joel E. Denny 9fd4b5faac [FileCheck] Implement -dump-input-filter
This makes the input dump filtering implemented by D82203 more
configurable.  D82203 enables filtering out everything but the initial
input lines of error diagnostics (plus some context).  This patch
enables including any line with any kind of annotation.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D83097
2020-07-10 11:02:11 -04:00
Joel E. Denny 77b6ddf1bd [FileCheck] In input dump, elide only if ellipsis is shorter
For example, given `-dump-input-context=3 -vv`, the following now
shows more leading context for the error than requested because a
leading ellipsis would occupy the same number of lines as it would
elide:

```
<<<<<<
         1: foo6
         2: foo5
         3: foo4
         4: foo3
         5: foo2
         6: foo1
         7: hello world
check:1     ^~~~~
check:2           X~~~~ error: no match found
         8: foo1
check:2     ~~~~
         9: foo2
check:2     ~~~~
        10: foo3
check:2     ~~~~
         .
         .
         .
>>>>>>
```

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D83526
2020-07-10 11:02:11 -04:00
Joel E. Denny bce8fced41 [FileCheck] Implement -dump-input-context
This patch is motivated by discussions at each of:

* <https://reviews.llvm.org/D81422>
* <http://lists.llvm.org/pipermail/llvm-dev/2020-June/142369.html>

When input is dumped as specified by `-dump-input=fail`, this patch
filters the dump to show only input lines that are the starting lines
of error diagnostics plus the number of contextual lines specified
`-dump-input-context` (defaults to 5).

When `-dump-input=always`, there might be not be any errors, so all
input lines are printed, as without this patch.

Here's some sample output with `-dump-input-context=3 -vv`:

```
<<<<<<
           .
           .
           .
          13: foo
          14: foo
          15: hello world
check:1       ^~~~~~~~~~~
          16: foo
check:2'0     X~~ error: no match found
          17: foo
check:2'0     ~~~
          18: foo
check:2'0     ~~~
          19: foo
check:2'0     ~~~
           .
           .
           .
          27: foo
check:2'0     ~~~
          28: foo
check:2'0     ~~~
          29: foo
check:2'0     ~~~
          30: goodbye word
check:2'0     ~~~~~~~~~~~~
check:2'1     ?            possible intended match
          31: foo
check:2'0     ~~~
          32: foo
check:2'0     ~~~
          33: foo
check:2'0     ~~~
           .
           .
           .
>>>>>>
```

Reviewed By: mehdi_amini, arsenm, jhenderson, rsmith, SjoerdMeijer, Meinersbur, lattner

Differential Revision: https://reviews.llvm.org/D82203
2020-07-10 11:02:10 -04:00
Joel E. Denny 839f8e4fe2 [FileCheck] Improve -dump-input documentation
Document the default of `fail` in `-help`.  Extend `-dump-input=help`
to help users find related command-line options, but let `-help`
provide their full documentation.

Reviewed By: probinson

Differential Revision: https://reviews.llvm.org/D83091
2020-07-09 18:00:30 -04:00
Joel E. Denny 782585a214 [FileCheck] Permit multiple -v or -vv
`FILECHECK_OPTS` was implemented so that a test runner, such as CI,
can specify FileCheck debugging options, such as `-v` and `-vv`.
However, if a test suite has a FileCheck call that already specifies
`-v` or `-vv`, then that call will fail if `FILECHECK_OPTS` also
specifies it.

For `-vv`, this problem already exists:

`clang/test/CodeGen/aarch64-v8.2a-fp16-intrinsics-constrained.c`

It's not yet clear if the `-vv` in that test was intentional, but this
usage shouldn't fail anyway.  It's already true that FileCheck permits
`-vv` and `-v` together even though `-vv` implies `-v`.

Compare D70784, which fixed the same problem for `-dump-input`.

Reviewed By: jhenderson, thopre

Differential Revision: https://reviews.llvm.org/D82601
2020-06-29 18:35:22 -04:00
Joel E. Denny 6324387264 [FileCheck][NFC] Clean up RUN style in verbose.txt test
Reviewed By: jhenderson, thopre

Differential Revision: https://reviews.llvm.org/D82658
2020-06-29 18:35:11 -04:00
Joel E. Denny ecb098c6de [FileCheck][NFC] Fix typo in test comment 2020-06-24 14:49:23 -04:00
Joel E. Denny 4b1b09fcc0 [FileCheck] Add missing %ProtectFileCheckOutput to FileCheck tests
This is a continuation of D65121 (committed at f471eb8e99).
2020-06-10 12:40:35 -04:00
Thomas Preud'homme 47934c7cf9 FileCheck [11/12]: Add matching constraint specification
This patch is part of a patch series to add support for FileCheck
numeric expressions. This specific patch adds support for specifying the
matching constraint for a numeric expression, ie. how the value being
matched should relate to the numeric expression.

This commit only adds the equality constraint where the numeric value
matched must be equal to the numeric expression. It is the default
matching constraint used when not specified. It is added to provision
other matching constraint (e.g. inequality relations).

Copyright:
    - Linaro (changes up to diff 183612 of revision D55940)
    - GraphCore (changes in later versions of revision D55940 and
                 in new revision created off D55940)

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D60391
2020-06-10 15:56:10 +01:00
Paul Walker 8fd2270370 [FileCheck] Add function call support to numerical expressions.
This patch extends numerical expressions to allow calls to
predefined functions. These calls can be combined with the
existing numerical operators, which includes nesting calls.

The call syntax is:

  <func>(<args>)

Where <func> is a predefined string literal, currently limited to
one of add, max, min and sub. <arg> is a comma seperated list of
numerical expressions.

Subscribers: arichardson, hiraditya, thopre, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D79936
2020-06-10 09:42:00 +00:00
Mehdi Amini d31c9e5a46 Change filecheck default to dump input on failure
Having the input dumped on failure seems like a better
default: I debugged FileCheck tests for a while without knowing
about this option, which really helps to understand failures.

Remove `-dump-input-on-failure` and the environment variable
FILECHECK_DUMP_INPUT_ON_FAILURE which are now obsolete.

Differential Revision: https://reviews.llvm.org/D81422
2020-06-09 18:57:46 +00:00
Thomas Preud'homme 23ac16cf9b FileCheck [10/12]: Add support for signed numeric values
Summary:
This patch is part of a patch series to add support for FileCheck
numeric expressions. This specific patch adds support signed numeric
values, thus allowing negative numeric values.

As such, the patch adds a new class to represent a signed or unsigned
value and add the logic for type promotion and type conversion in
numeric expression mixing signed and unsigned values. It also adds
the %d format specifier to represent signed value.

Finally, it also adds underflow and overflow detection when performing a
binary operation.

Copyright:
    - Linaro (changes up to diff 183612 of revision D55940)
    - GraphCore (changes in later versions of revision D55940 and
                 in new revision created off D55940)

Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson

Reviewed By: jhenderson, arichardson

Subscribers: MaskRay, hiraditya, llvm-commits, probinson, dblaikie, grimar, arichardson, kristina, hfinkel, rogfer01, JonChesterfield

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60390
2020-05-28 10:44:21 +01:00
Alex Richardson 3be5e53f20 [FileCheck] Allow parenthesized expressions
With this change it is be possible to write FileCheck expressions such
as [[#(VAR+1)-2]]. Currently, the only supported arithmetic operators are
plus and minus, so this is not particularly useful yet. However, it our
CHERI fork we have tests that benefit from having multiplication in
FileCheck expressions. Allowing parenthesized expressions is the simplest
way for us to work around the current lack of operator precedence in
FileCheck expressions.

Reviewed By: thopre, jhenderson
Differential Revision: https://reviews.llvm.org/D77383
2020-05-27 16:31:39 +01:00
Jon Roelofs 5a8db275f8 Revert "[llvm][test] Add COM: directives before colon-less non-CHECKs in comments. NFC"
This reverts commit 183d6af081.

Revert pending further consensus building: https://reviews.llvm.org/D79963#2050521
2020-05-22 05:36:15 -06:00
Jon Roelofs 183d6af081 [llvm][test] Add COM: directives before colon-less non-CHECKs in comments. NFC
Differential Revision: https://reviews.llvm.org/D79963
2020-05-21 09:29:27 -06:00
Thomas Preud'homme 0c2c6fce07 [test] Improve FileCheck's numeric-expression.txt
Summary:
Various improvement for FileCheck's numeric-expression.txt test:
- remove unused values in USE DEF FMT IMPL MATCH section
- replace 14 literal for 0xe and 0xE to have example of hex literals
- rename variable to be more self-descriptive
- move CHECK as comment of the values being matched to help readability
- add conversion tests
- simplify test for use of several numeric variables by using existing
  variable
- adjust position of error message check to match the alignment of the
  error message wrt. the output matched by the previous check

Reviewed By: jhenderson, jdenny

Differential Revision: https://reviews.llvm.org/D79820
2020-05-16 00:09:24 +01:00
Joel E. Denny 5df55bc7a4 [FileCheck] Fix isalpha/isalnum calls
D79276 caused the following builder to fail:

  http://lab.llvm.org:8011/builders/llvm-clang-x86_64-expensive-checks-win/builds/23489

Specifically, FileCheck dumped stack in the following tests:

  LLVM :: MC/Mips/micromips-jump-pc-region.s
  LLVM :: MC/Mips/mips-jump-pc-region.s

Those tests contained characters encoded as 160 but that render (at
least for me in vim) like a single space (32).  Those characters
appeared between the `#` and `RUN:` on several lines, and D79276
caused FileCheck to process those lines differently: `RUN:` is a
comment directive.  As a result, D79276 caused FileCheck to start
calling is `isalnum` on those characters.

The problem is that FileCheck calls `isalnum` on type `char` without
casting to `unsigned char` first, so it sign-extends 160 beyond what
`unsigned char` or `EOF` can represent.  C says that has undefined
behavior.  This problem is general to FileCheck's prefix parsing and
so exists independently of D79276.

524457edbc fixed the above tests.  This patch changes FileCheck to
use LLVM's replacements for `ctype.h` functions, and it adds tests for
cases that are representative with or without D79276.

Reviewed By: jhenderson, thopre, efriedma

Differential Revision: https://reviews.llvm.org/D79810
2020-05-14 20:24:09 -04:00
Joel E. Denny a1fd188223 [FileCheck] Support comment directives
Sometimes you want to disable a FileCheck directive without removing
it entirely, or you want to write comments that mention a directive by
name.  The `COM:` directive makes it easy to do this.  For example,
you might have:

```
; X32: pinsrd_1:
; X32:    pinsrd $1, 4(%esp), %xmm0

; COM: FIXME: X64 isn't working correctly yet for this part of codegen, but
; COM: X64 will have something similar to X32:
; COM:
; COM:   X64: pinsrd_1:
; COM:   X64:    pinsrd $1, %edi, %xmm0
```

Without this patch, you need to use some combination of rewording and
directive syntax mangling to prevent FileCheck from recognizing the
commented occurrences of `X32:` and `X64:` above as directives.
Moreover, FileCheck diagnostics have been proposed that might complain
about the occurrences of `X64` that don't have the trailing `:`
because they look like directive typos:

  <http://lists.llvm.org/pipermail/llvm-dev/2020-April/140610.html>

I think dodging all these problems can prove tedious for test authors,
and directive syntax mangling already makes the purpose of existing
test code unclear.  `COM:` can avoid all these problems.

This patch also updates the small set of existing tests that define
`COM` as a check prefix:

- clang/test/CodeGen/default-address-space.c
- clang/test/CodeGenOpenCL/addr-space-struct-arg.cl
- clang/test/Driver/hip-device-libs.hip
- llvm/test/Assembler/drop-debug-info-nonzero-alloca.ll

I think lit should support `COM:` as well.  Perhaps `clang -verify`
should too.

Reviewed By: jhenderson, thopre

Differential Revision: https://reviews.llvm.org/D79276
2020-05-13 11:29:48 -04:00
Joel E. Denny 2aa0217add [FileCheck] Make invalid prefix diagnostics more precise
This will prove especially helpful after D79276, which introduces
comment prefixes.  Specifically, identifying whether there's a
uniqueness violation will be helpful as prefixes will be required to
be unique across both check prefixes and comment prefixes.

Also, remove a related comment about `cl::list` that no longer seems
relevant now that FileCheck is also a library.

Reviewed By: jhenderson, thopre

Differential Revision: https://reviews.llvm.org/D79375
2020-05-11 21:11:58 -04:00
Joel E. Denny e1ed4d9eb5 Revert "[FileCheck] Make invalid prefix diagnostics more precise"
This reverts commit a78e13745d to try to
fix a bot:

http://lab.llvm.org:8011/builders/llvm-clang-x86_64-expensive-checks-win/builds/23489
2020-05-11 19:41:22 -04:00
Joel E. Denny d0e7fd6b62 Revert "[FileCheck] Support comment directives"
This reverts commit 9a9a5f9893 to try to
fix a bot:

http://lab.llvm.org:8011/builders/llvm-clang-x86_64-expensive-checks-win/builds/23489
2020-05-11 19:41:22 -04:00
Joel E. Denny 9a9a5f9893 [FileCheck] Support comment directives
Sometimes you want to disable a FileCheck directive without removing
it entirely, or you want to write comments that mention a directive by
name.  The `COM:` directive makes it easy to do this.  For example,
you might have:

```
; X32: pinsrd_1:
; X32:    pinsrd $1, 4(%esp), %xmm0

; COM: FIXME: X64 isn't working correctly yet for this part of codegen, but
; COM: X64 will have something similar to X32:
; COM:
; COM:   X64: pinsrd_1:
; COM:   X64:    pinsrd $1, %edi, %xmm0
```

Without this patch, you need to use some combination of rewording and
directive syntax mangling to prevent FileCheck from recognizing the
commented occurrences of `X32:` and `X64:` above as directives.
Moreover, FileCheck diagnostics have been proposed that might complain
about the occurrences of `X64` that don't have the trailing `:`
because they look like directive typos:

  <http://lists.llvm.org/pipermail/llvm-dev/2020-April/140610.html>

I think dodging all these problems can prove tedious for test authors,
and directive syntax mangling already makes the purpose of existing
test code unclear.  `COM:` can avoid all these problems.

This patch also updates the small set of existing tests that define
`COM` as a check prefix:

- clang/test/CodeGen/default-address-space.c
- clang/test/CodeGenOpenCL/addr-space-struct-arg.cl
- clang/test/Driver/hip-device-libs.hip
- llvm/test/Assembler/drop-debug-info-nonzero-alloca.ll

I think lit should support `COM:` as well.  Perhaps `clang -verify`
should too.

Reviewed By: jhenderson, thopre

Differential Revision: https://reviews.llvm.org/D79276
2020-05-11 14:53:48 -04:00
Joel E. Denny a78e13745d [FileCheck] Make invalid prefix diagnostics more precise
This will prove especially helpful after D79276, which introduces
comment prefixes.  Specifically, identifying whether there's a
uniqueness violation will be helpful as prefixes will be required to
be unique across both check prefixes and comment prefixes.

Also, remove a related comment about `cl::list` that no longer seems
relevant now that FileCheck is also a library.

Reviewed By: jhenderson, thopre

Differential Revision: https://reviews.llvm.org/D79375
2020-05-11 14:53:48 -04:00
Joel E. Denny ce685455e4 [FileCheck] Fix --dump-input annotation sort per input line
Without this patch, `--dump-input` annotations on a single input line
are sorted by the associated directive's check-file line.  That seemed
fine because that's often identical to the order in which FileCheck
looks for matches for those directives.

The first problem is that an `--implicit-check-not` pattern has no
check-file line.  The logical equivalent is sorting in command-line
order, but that's not implemented.

The second problem is that, unlike a directive, an
`--implicit-check-not` pattern applies at many points, between many
different pairs of directives.  However, sorting in command-line order
gathers all its associated diagnostics together at one point in an
input line's list of annotations.

In general, it seems to be easier to understand FileCheck's logic when
annotations on a single input line are sorted in the order FileCheck
produced the associated diagnostics, so this patch makes that change.
As documented in the patch, the annotation sort order is also
especially relevant to `CHECK-LABEL`, `CHECK-NOT`, and `CHECK-DAG`, so
this patch updates or extends tests to check the sort makes sense for
them.  (However, the sort for `CHECK-DAG` annotations should not
actually be altered by this patch.)

Reviewed By: thopre

Differential Revision: https://reviews.llvm.org/D77607
2020-04-16 15:39:35 -04:00
Joel E. Denny b5a24610fa [FileCheck] Fix --dump-input implicit pattern location
Currently, `--dump-input` implies that all `--implicit-check-not`
patterns appear on line 1 by printing annotations like:

```
       1: foo bar baz
not:1         !~~     error: no match expected
```

This patch changes that to:

```
          1: foo bar baz
not:imp1         !~~     error: no match expected
```

`imp1` indicates the first `--implicit-check-not` pattern.

Reviewed By: thopre

Differential Revision: https://reviews.llvm.org/D77605
2020-04-16 15:39:35 -04:00
Georgii Rymar 65a2de7e6c [FileCheck] - Fix the false positive when -implicit-check-not is used with an unknown -check-prefix.
Imagine we have the following invocation:

`FileCheck -check-prefix=UNKNOWN-PREFIX -implicit-check-not=something`

When the check prefix does not exist it does not fail.
This patch fixes the issue.

Differential revision: https://reviews.llvm.org/D78024
2020-04-16 15:00:50 +03:00
Thomas Preud'homme 9743123af8 [FileCheck] Better diagnostic for format conflict
Summary:
Improve error message in case of conflict between several implicit
format to mention the operand that conflict.

Reviewers: jhenderson, jdenny, probinson, grimar, arichardson, rnk

Reviewed By: jdenny

Subscribers: hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D77741
2020-04-15 13:46:45 +01:00
Joel E. Denny 8f8c4950fe [FileCheck] Add missing %ProtectFileCheckOutput to FileCheck tests
I'm committing this fixup without review because it's an obvious
continuation of D65121 (committed at f471eb8e99).
2020-03-31 17:29:11 -04:00
David Bozier 6e01cd6795 Improve error message of FileCheck when stdin is empty
Summary: Replace '-' in the error message with <stdin>. This is also consistent with another error message in the code.

Reviewers: jhenderson, probinson, jdenny, grimar, arichardson

Reviewed By: jhenderson

Subscribers: thopre, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D73793
2020-02-04 11:14:55 +00:00
Thomas Preud'homme 8e96697c7d FileCheck [9/12]: Add support for matching formats
Summary:
This patch is part of a patch series to add support for FileCheck
numeric expressions. This specific patch adds support for selecting a
matching format to match a numeric value against (ie. decimal, hex lower
case letters or hex upper case letters).

This commit allows to select what format a numeric value should be
matched against. The following formats are supported: decimal value,
lower case hex value and upper case hex value. Matching formats impact
both the format of numeric value to be matched as well as the format of
accepted numbers in a definition with empty numeric expression
constraint.

Default for absence of format is decimal value unless the numeric
expression constraint is non null and use a variable in which case the
format is the one used to define that variable. Conclict of format in
case of several variable being used is diagnosed and forces the user to
select a matching format explicitely.

This commit also enables immediates in numeric expressions to be in any
radix known to StringRef's GetAsInteger method, except for legacy
numeric expressions (ie [[@LINE+<offset>]] which only support decimal
immediates.

Copyright:
    - Linaro (changes up to diff 183612 of revision D55940)
    - GraphCore (changes in later versions of revision D55940 and
                 in new revision created off D55940)

Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson

Reviewed By: jhenderson, arichardson

Subscribers: daltenty, MaskRay, hiraditya, llvm-commits, probinson, dblaikie, grimar, arichardson, kristina, hfinkel, rogfer01, JonChesterfield

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60389
2020-01-24 14:15:28 +00:00
Joel E. Denny fdde18a7c3 [FileCheck] Given multiple -dump-input, prefer most verbose
Problem: `FILECHECK_OPTS` was implemented so that a test runner, such
as a bot, can specify FileCheck debugging options, such as
`-dump-input=fail`.  However, some existing test suites have FileCheck
calls that already specify `-dump-input=fail` or `-dump-input=always`.
Without this patch, such tests fail under such a test runner because
FileCheck doesn't accept multiple occurrences of `-dump-input`.

Solution: This patch permits multiple occurrences of `-dump-input` by
assigning precedence to its values in the following descending order:
`help`, `always`, `fail`, and `never`.  That is, any occurrence of
`help` always obtains help, and otherwise the behavior is similar to
`-v` vs. `-vv` in that the option specifying the greatest verbosity
has precedence.

Rationale: My justification for the new behavior is as follows.  I
have not experienced use cases where, either as a test runner or as a
test author, I want to **limit** the permitted debugging verbosity
(except as a test author in FileCheck's or lit's test suites where the
FileCheck debugging output itself is under test, but the solution
there is `env FILECHECK_OPTS=`, and I imagine we should use the same
solution anywhere else this need might occur).  Of course, as either a
test runner or test author, it is useful to **increase** debugging
verbosity.

Reviewed By: probinson

Differential Revision: https://reviews.llvm.org/D70784
2019-12-03 14:21:13 -05:00
Joel E. Denny f471eb8e99 [FileCheck] Make FILECHECK_OPTS useful for its test suite
Without this patch, `FILECHECK_OPTS` isn't propagated to FileCheck's
test suite so that `FILECHECK_OPTS` doesn't inadvertently affect test
results by affecting the output of FileCheck calls under test.  As a
result, `FILECHECK_OPTS` is useless for debugging FileCheck's test
suite.

In `llvm/test/FileCheck/lit.local.cfg`, this patch provides a new
subsitution, `%ProtectFileCheckOutput`, to address this problem for
both `FILECHECK_OPTS` and the deprecated
`FILECHECK_DUMP_INPUT_ON_FAILURE`.  The rest of the patch uses
`%ProtectFileCheckOutput` throughout the test suite

Fixes PR40284.

Reviewed By: probinson, thopre

Differential Revision: https://reviews.llvm.org/D65121
2019-11-21 18:01:12 -05:00
Joel E. Denny 6e418decd9 [FileCheck] Use lit's internal shell for the test suite
An advantage is that there are less portability concerns when writing
tests.  For example, `-u` is not supported by all implementations of
`env`, but lit's internal shell provides its own `env` that supports
`-u`.

A disadvantage is that some shell constructs, such as parentheses, are
not supported, but FileCheck's test suite currently doesn't require
such constructs.

For comparison, lit configures its test suite in the same manner.  See
`llvm/utils/lit/tests/lit.cfg`.

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D70278
2019-11-19 12:08:54 -05:00
Kai Nacke 5b5b2fd2b8 [FileCheck] Implement --ignore-case option.
The FileCheck utility is enhanced to support a `--ignore-case`
option. This is useful in cases where the output of Unix tools
differs in case (e.g. case not specified by Posix).

Reviewers: Bigcheese, jakehehrlich, rupprecht, espindola, alexshap, jhenderson, MaskRay

Differential Revision: https://reviews.llvm.org/D68146

llvm-svn: 374538
2019-10-11 11:59:14 +00:00
Dmitri Gribenko d3aed7fc79 Revert "[FileCheck] Implement --ignore-case option."
This reverts commit r374339. It broke tests:
http://lab.llvm.org:8011/builders/clang-x86_64-debian-fast/builds/19066

llvm-svn: 374359
2019-10-10 14:27:14 +00:00
Kai Nacke dfd2b6f07f [FileCheck] Implement --ignore-case option.
The FileCheck utility is enhanced to support a `--ignore-case`
option. This is useful in cases where the output of Unix tools
differs in case (e.g. case not specified by Posix).

Reviewers: Bigcheese, jakehehrlich, rupprecht, espindola, alexshap, jhenderson, MaskRay

Differential Revision: https://reviews.llvm.org/D68146

llvm-svn: 374339
2019-10-10 13:15:41 +00:00
Thomas Preud'homme a291b950db [FileCheck] Forbid using var defined on same line
Summary:
Commit r366897 introduced the possibility to set a variable from an
expression, such as [[#VAR2:VAR1+3]]. While introducing this feature, it
introduced extra logic to allow using such a variable on the same line
later on. Unfortunately that extra logic is flawed as it relies on a
mapping from variable to expression defining it when the mapping is from
variable definition to expression. This flaw causes among other issues
PR42896.

This commit avoids the problem by forbidding all use of a variable
defined on the same line, and removes the now useless logic. Redesign
will be done in a later commit because it will require some amount of
refactoring first for the solution to be clean. One example is the need
for some sort of transaction mechanism to set a variable temporarily and
from an expression and rollback if the CHECK pattern does not match so
that diagnostics show the right variable values.

Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk

Subscribers: JonChesterfield, rogfer01, hfinkel, kristina, rnk, tra, arichardson, grimar, dblaikie, probinson, llvm-commits, hiraditya

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D66141

llvm-svn: 370663
2019-09-02 14:04:00 +00:00
Joel E. Denny 608f2bfd65 [FileCheck] Move -dump-input diagnostic to first line
Without this patch, `-dump-input` prints a diagnostic at the end of
its marker range.  For example:

```
         1: Start.
check:1     ^~~~~~
         2: Bad.
next:2      X~~~
         3: Many lines
next:2      ~~~~~~~~~~
         4: of input.
next:2      ~~~~~~~~~
         5: End.
next:2      ~~~~ error: no match found
```

This patch moves it to the beginning like this:

```
         1: Start.
check:1     ^~~~~~
         2: Bad.
next:2      X~~~ error: no match found
         3: Many lines
next:2      ~~~~~~~~~~
         4: of input.
next:2      ~~~~~~~~~
         5: End.
next:2      ~~~~
```

The former somehow looks nicer because the diagnostic doesn't appear
to be somewhere within the marker range.  However, the latter is more
practical, especially when the marker range includes the remainder of
a very long dump.  First, in the case of an error, this patch enables
me to search the dump for `error:` and usually immediately land where
the detected error began.  Second, when trying to follow FileCheck's
logic, it's best to read top down, so this patch enables me to see
each diagnostic as soon as I encounter its marker.

Reviewed By: thopre

Differential Revision: https://reviews.llvm.org/D65702

llvm-svn: 368786
2019-08-14 02:56:09 +00:00
Thomas Preud'homme a83dfd92be [FileCheck]] Canonicalize caret location testing
Summary:
Testing of caret location in diagnostic message is currently made with
CHECK directive with the following general format:
CHECK: {{^         \^$}}

James Henderson suggested the following would be more readable:
CHECK: {{^}}         ^{{$}}

and when whole lines can be matched (as is the case for command-line
testing where error messages do not include path):
CHECK:         ^
using the option --match-full-lines.

This commit implements these 2 changes on all existing caret position
tests. It also aligns the caret to the character it is trying to match
in the above line.

Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk

Subscribers: JonChesterfield, rogfer01, hfinkel, kristina, rnk, tra, arichardson, grimar, dblaikie, probinson, llvm-commits, hiraditya

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64923

> llvm-svn: 366861

llvm-svn: 366898
2019-07-24 12:38:29 +00:00
Thomas Preud'homme 4cd9b853b5 FileCheck [8/12]: Define numeric var from expr
Summary:
This patch is part of a patch series to add support for FileCheck
numeric expressions. This specific patch lift the restriction for a
numeric expression to either be a variable definition or a numeric
expression to try to match.

This commit allows a numeric variable to be set to the result of the
evaluation of a numeric expression after it has been matched
successfully. When it happens, the variable is allowed to be used on
the same line since its value is known at match time.

It also makes use of this possibility to reuse the parsing code to
parse a command-line definition by crafting a mirror string of the
-D option with the equal sign replaced by a colon sign, e.g. for option
'-D#NUMVAL=10' it creates the string
'-D#NUMVAL=10 (parsed as [[#NUMVAL:10]])' where the numeric expression
is parsed to define NUMVAL. This result in a few tests needing updating
for the location diagnostics on top of the tests for the new feature.

It also enables empty numeric expression which match any number without
defining a variable. This is done here rather than in commit #5 of the
patch series because it requires to dissociate automatic regex insertion
in RegExStr from variable definition which would make commit #5 even
bigger than it already is.

Copyright:
    - Linaro (changes up to diff 183612 of revision D55940)
    - GraphCore (changes in later versions of revision D55940 and
                 in new revision created off D55940)

Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk

Subscribers: hiraditya, llvm-commits, probinson, dblaikie, grimar, arichardson, tra, rnk, kristina, hfinkel, rogfer01, JonChesterfield

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60388

> llvm-svn: 366860

llvm-svn: 366897
2019-07-24 12:38:22 +00:00
Thomas Preud'homme 5ecb880241 Revert "FileCheck [8/12]: Define numeric var from expr"
This reverts commit 1b05977538.

llvm-svn: 366872
2019-07-24 07:32:34 +00:00
Thomas Preud'homme 75299de191 Revert "[FileCheck]] Canonicalize caret location testing"
This reverts commit ba4e8d7913.

llvm-svn: 366871
2019-07-24 07:32:18 +00:00
Thomas Preud'homme ba4e8d7913 [FileCheck]] Canonicalize caret location testing
Summary:
Testing of caret location in diagnostic message is currently made with
CHECK directive with the following general format:
CHECK: {{^         \^$}}

James Henderson suggested the following would be more readable:
CHECK: {{^}}         ^{{$}}

and when whole lines can be matched (as is the case for command-line
testing where error messages do not include path):
CHECK:         ^
using the option --match-full-lines.

This commit implements these 2 changes on all existing caret position
tests. It also aligns the caret to the character it is trying to match
in the above line.

Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk

Subscribers: JonChesterfield, rogfer01, hfinkel, kristina, rnk, tra, arichardson, grimar, dblaikie, probinson, llvm-commits, hiraditya

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64923

llvm-svn: 366861
2019-07-23 22:41:46 +00:00
Thomas Preud'homme 1b05977538 FileCheck [8/12]: Define numeric var from expr
Summary:
This patch is part of a patch series to add support for FileCheck
numeric expressions. This specific patch lift the restriction for a
numeric expression to either be a variable definition or a numeric
expression to try to match.

This commit allows a numeric variable to be set to the result of the
evaluation of a numeric expression after it has been matched
successfully. When it happens, the variable is allowed to be used on
the same line since its value is known at match time.

It also makes use of this possibility to reuse the parsing code to
parse a command-line definition by crafting a mirror string of the
-D option with the equal sign replaced by a colon sign, e.g. for option
'-D#NUMVAL=10' it creates the string
'-D#NUMVAL=10 (parsed as [[#NUMVAL:10]])' where the numeric expression
is parsed to define NUMVAL. This result in a few tests needing updating
for the location diagnostics on top of the tests for the new feature.

It also enables empty numeric expression which match any number without
defining a variable. This is done here rather than in commit #5 of the
patch series because it requires to dissociate automatic regex insertion
in RegExStr from variable definition which would make commit #5 even
bigger than it already is.

Copyright:
    - Linaro (changes up to diff 183612 of revision D55940)
    - GraphCore (changes in later versions of revision D55940 and
                 in new revision created off D55940)

Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk

Subscribers: hiraditya, llvm-commits, probinson, dblaikie, grimar, arichardson, tra, rnk, kristina, hfinkel, rogfer01, JonChesterfield

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60388

llvm-svn: 366860
2019-07-23 22:41:38 +00:00