Commit Graph

208 Commits

Author SHA1 Message Date
Joel E. Denny 43279d1df9 [FileCheck] Try to fix buildbot failures caused by c7c542e8f3
For example,

<https://lab.llvm.org/buildbot/#/builders/132/builds/3929>

has this diagnostic:

```
/opt/gcc/9.3.0/snos/include/g++/bits/stl_tree.h:780:8: error: static assertion failed: comparison object must be invocable as const
  780 |        is_invocable_v<const _Compare&, const _Key&, const _Key&>,
      |        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
2021-03-27 11:03:10 -04:00
Joel E. Denny c7c542e8f3 [FileCheck] Fix -dump-input per-pattern diagnostic indexing
In input dump annotations, `check:2'1` indicates diagnostic 1 for the
`CHECK` directive on check file line 2.  Without this patch,
`-dump-input` computes the diagnostic index with the assumption that
FileCheck *consecutively* produces all diagnostics for the same
pattern.  Already, that can be a false assumption, as in the examples
below.  Moreover, it seems like a brittle assumption as FileCheck
evolves.  Finally, it actually complicates the implementation even if
it makes it slightly more efficient.

This patch avoids that assumption.  Examples below show results after
applying this patch.  Before applying this patch, `'N` is omitted
throughout these examples because the implementation doesn't notice
there's more than one diagnostic per pattern.

First, `CHECK-LABEL` violates the assumption because `CHECK-LABEL`
tries to match twice, and other directives can match in between:

```
$ cat check
CHECK: foobar
CHECK-LABEL: foobar

$ FileCheck -vv check < input |& tail -8
<<<<<<
           1: text
           2: foobar
label:2'0     ^~~~~~
check:1       ^~~~~~
label:2'1           X error: no match found
           3: text
>>>>>>
```

Second, `--implicit-check-not` is obviously processed many times among
other directives:

```
$ cat check
CHECK: foo
CHECK: foo

$ FileCheck -vv -dump-input=always -implicit-check-not=foo \
            check < input |& tail -16
<<<<<<
            1: text
not:imp1'0     X~~~~
            2: foo
check:1        ^~~
not:imp1'1        X
            3: text
not:imp1'1     ~~~~~
            4: foo
check:2        ^~~
not:imp1'2        X
            5: text
not:imp1'2     ~~~~~
            6:
eof:2          ^
>>>>>>
```

Reviewed By: thopre, jhenderson

Differential Revision: https://reviews.llvm.org/D97813
2021-03-27 10:36:21 -04:00
Abhina Sreeskantharajan c83cd8feef [NFC] Reordering parameters in getFile and getFileOrSTDIN
In future patches I will be setting the IsText parameter frequently so I will refactor the args to be in the following order. I have removed the FileSize parameter because it is never used.

```
  static ErrorOr<std::unique_ptr<MemoryBuffer>>
  getFile(const Twine &Filename, bool IsText = false,
          bool RequiresNullTerminator = true, bool IsVolatile = false);

  static ErrorOr<std::unique_ptr<MemoryBuffer>>
  getFileOrSTDIN(const Twine &Filename, bool IsText = false,
                 bool RequiresNullTerminator = true);

 static ErrorOr<std::unique_ptr<MB>>
 getFileAux(const Twine &Filename, uint64_t MapSize, uint64_t Offset,
            bool IsText, bool RequiresNullTerminator, bool IsVolatile);

  static ErrorOr<std::unique_ptr<WritableMemoryBuffer>>
  getFile(const Twine &Filename, bool IsVolatile = false);
```

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D99182
2021-03-25 09:47:49 -04:00
Abhina Sreeskantharajan a234d03198 [NFC] Formatting changes
This patch addresses some formatting changes from the comments in https://reviews.llvm.org/D97785.

Reviewed By: anirudhp

Differential Revision: https://reviews.llvm.org/D99072
2021-03-23 07:17:54 -04:00
Abhina Sreeskantharajan 4f750f6ebc [SystemZ][z/OS] Distinguish between text and binary files on z/OS
This patch consists of the initial changes to help distinguish between text and binary content correctly on z/OS. I would like to get feedback from Windows users on setting OF_None for all ToolOutputFiles. This seems to have been done as an optimization to prevent CRLF translation on Windows in the past.

Reviewed By: zibi

Differential Revision: https://reviews.llvm.org/D97785
2021-03-19 08:09:57 -04:00
Joel E. Denny dd59c1324d [FileCheck] Fix numeric error propagation
A more general name might be match-time error propagation.  That is,
it's conceivable we'll one day have non-numeric errors that require
the handling fixed by this patch.

Without this patch, FileCheck behaves as follows:

```
$ cat check
CHECK-NOT: [[#0x8000000000000000+0x8000000000000000]]

$ FileCheck -vv -dump-input=never check < input
check:1:54: remark: implicit EOF: expected string found in input
CHECK-NOT: [[#0x8000000000000000+0x8000000000000000]]
                                                     ^
<stdin>:2:1: note: found here

^
check:1:15: error: unable to substitute variable or numeric expression: overflow error
CHECK-NOT: [[#0x8000000000000000+0x8000000000000000]]
              ^
$ echo $?
0
```

Notice that the exit status is 0 even though there's an error.
Moreover, FileCheck doesn't print the error diagnostic unless both
`-dump-input=never` and `-vv` are specified.

The same problem occurs when `CHECK-NOT` does have a match but a
capture fails due to overflow: exit status is 0, and no diagnostic is
printed unless both `-dump-input=never` and `-vv` are specified.  The
usefulness of capturing from `CHECK-NOT` is questionable, but this
case should certainly produce an error.

With this patch, FileCheck always includes the error diagnostic and
has non-zero exit status for the above examples.  It's conceivable
that this change will cause some existing tests to fail, but my
assumption is that they should fail.  Moreover, with nearly every
project enabled, this patch didn't produce additional `check-all`
failures for me.

This patch also extends input dumps to include such numeric error
diagnostics for both expected and excluded patterns.

As noted in fixmes in some of the tests added by this patch, this
patch worsens an existing issue with redundant diagnostics.  I'll fix
that bug in a subsequent patch.

Reviewed By: thopre, jhenderson

Differential Revision: https://reviews.llvm.org/D98086
2021-03-17 19:25:41 -04:00
Thomas Preud'homme 09c3573903 [FileCheck] Do not skip end of line in diagnostics
When commit da108b4ed4 introduced
the CHECK-NEXT directive, it added logic to skip to the next line when
printing a diagnostic if the current matching position is at the end of
a line. This was fine while FileCheck did not support regular expression
but since it does now it can be confusing when the pattern to match
starts with the expectation of a newline (e.g. CHECK-NEXT: {{\n}}foo).
It is also inconsistent with the column information in the diagnostic
which does point to the end of line.

This commit removes this logic altogether, such that failure to match
diagnostic for such cases would show the end of line and be consistent
with the column information. The commit also adapts all existing
testcases accordingly.

Note to reviewers: An alternative approach would be to restrict the code
to only skip to the next line if the first character of the pattern is
known not to match a whitespace-like character. This would respect the
original intent but keep the inconsistency in terms of column info and
requires more code. I've only chosen this current approach by laziness
and would be happy to restrict the logic instead.

Reviewed By: jdenny, jhenderson

Differential Revision: https://reviews.llvm.org/D93341
2021-03-03 08:20:39 +00:00
Fangrui Song 87dbdd2e3b [FileCheck] Default --allow-unused-prefixes to false
Link: https://lists.llvm.org/pipermail/llvm-dev/2020-October/146162.html "[RFC] FileCheck: (dis)allowing unused prefixes"

If a downstream project using lit needs time for transition,
add the following to `lit.local.cfg`:

```
from lit.llvm.subst import ToolSubst

fc = ToolSubst('FileCheck', unresolved='fatal')
config.substitutions.insert(0, (fc.regex, 'FileCheck --allow-unused-prefixes'))
```

Differential Revision: https://reviews.llvm.org/D95849
2021-02-08 13:37:04 -08:00
Fangrui Song eec87db2a6 [FileCheck] Make --allow-unused-prefixes cl::ZeroOrMore
cl::ZeroOrMore allows the option to be specified multiple times, which makes
downstream projects possible to specify a default value in lit configuration
while some tests can override the value.
2021-02-02 15:18:00 -08:00
Kazu Hirata 7728cc003a [llvm] Use append_range (NFC) 2021-01-29 23:23:34 -08:00
Kazu Hirata a396e2e088 [utils] Use llvm::sort (NFC) 2021-01-15 21:00:52 -08:00
Mircea Trofin 871d658c9c [FileCheck] Report missing prefixes when more than one is provided.
If more than a prefix is provided - e.g. --check-prefixes=CHECK,FOO - we
don't report if (say) FOO is never used. This may lead to a gap in our
test coverage.

This patch introduces a new option, --allow-unused-prefixes. It
currently is set to true, keeping today's behavior. After we explicitly
set it in tests where this behavior was actually intentional, we will
switch it to false by default.

Differential Revision: https://reviews.llvm.org/D90281
2020-10-30 12:39:29 -07:00
Fangrui Song 20e9c36c01 Internalize functions from various tools. NFC
And internalize some classes if I noticed them:)
2020-09-26 15:57:13 -07:00
Raphael Isemann 5ffd940ac0 Reland [FileCheck] Move FileCheck implementation out of LLVMSupport into its own library
This relands e9a3d1a401 which was originally
missing linking LLVMSupport into LLMVFileCheck which broke the SHARED_LIBS build.

Original summary:

The actual FileCheck logic seems to be implemented in LLVMSupport. I don't see a
good reason for having FileCheck implemented there as it has a very specific use
while LLVMSupport is a dependency of pretty much every LLVM tool there is. In
fact, the only use of FileCheck I could find (outside the FileCheck tool and the
FileCheck unit test) is a single call in GISelMITest.h.

This moves the FileCheck logic to its own LLVMFileCheck library. This way only
FileCheck and the GlobalISelTests now have a dependency on this code.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D86344
2020-09-01 14:59:28 +02:00
Raphael Isemann ed89eb3571 Revert "[FileCheck] Move FileCheck implementation out of LLVMSupport into its own library"
This reverts commit e9a3d1a401. Seems the new
FileCheck library doesn't link on some bots. Reverting for now.
2020-08-31 11:38:40 +02:00
Raphael Isemann e9a3d1a401 [FileCheck] Move FileCheck implementation out of LLVMSupport into its own library
The actual FileCheck logic seems to be implemented in LLVMSupport. I don't see a
good reason for having FileCheck implemented there as it has a very specific use
while LLVMSupport is a dependency of pretty much every LLVM tool there is. In
fact, the only use of FileCheck I could find (outside the FileCheck tool and the
FileCheck unit test) is a single call in GISelMITest.h.

This moves the FileCheck logic to its own LLVMFileCheck library. This way only
FileCheck and the GlobalISelTests now have a dependency on this code.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D86344
2020-08-31 11:24:41 +02:00
Joel E. Denny d680711b94 [FileCheck] Extend -dump-input with substitutions
Substitutions are already reported in the diagnostics appearing before
the input dump in the case of failed directives, and they're reported
in traces (produced by `-vv -dump-input=never`) in the case of
successful directives.  However, those reports are not always
convenient to view while investigating the input dump, so this patch
adds the substitution report to the input dump too.  For example:

```
$ cat check
CHECK: hello [[WHAT:[a-z]+]]
CHECK: [[VERB]] [[WHAT]]

$ FileCheck -vv -DVERB=goodbye check < input |& tail -8
<<<<<<
           1: hello world
check:1       ^~~~~~~~~~~
           2: goodbye word
check:2'0     X~~~~~~~~~~~ error: no match found
check:2'1                  with "VERB" equal to "goodbye"
check:2'2                  with "WHAT" equal to "world"
>>>>>>
```

Without this patch, the location reported for a substitution for a
directive match is the directive's full match range.  This location is
misleading as it implies the substitution itself matches that range.
This patch changes the reported location to just the match range start
to suggest the substitution is known at the start of the match.  (As
in the above example, input dumps don't mark any range for
substitutions.  The location info in that case simply identifies the
right line for the annotation.)

Reviewed By: mehdi_amini, thopre

Differential Revision: https://reviews.llvm.org/D83650
2020-07-28 19:15:18 -04:00
Joel E. Denny 6dda6ff0e0 [FileCheck] Fix up -dump-input* docs
In FileCheck.rst, add `-dump-input-context` and `-dump-input-filter`,
and fix some `-dump-input` documentation.

In `FileCheck -help`, `cl::value_desc("kind")` is being ignored for
`-dump-input-filter`, so just drop it.

Extend `-dump-input=help` to mention FILECHECK_OPTS.
2020-07-10 17:21:01 -04:00
Joel E. Denny 9fd4b5faac [FileCheck] Implement -dump-input-filter
This makes the input dump filtering implemented by D82203 more
configurable.  D82203 enables filtering out everything but the initial
input lines of error diagnostics (plus some context).  This patch
enables including any line with any kind of annotation.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D83097
2020-07-10 11:02:11 -04:00
Joel E. Denny 77b6ddf1bd [FileCheck] In input dump, elide only if ellipsis is shorter
For example, given `-dump-input-context=3 -vv`, the following now
shows more leading context for the error than requested because a
leading ellipsis would occupy the same number of lines as it would
elide:

```
<<<<<<
         1: foo6
         2: foo5
         3: foo4
         4: foo3
         5: foo2
         6: foo1
         7: hello world
check:1     ^~~~~
check:2           X~~~~ error: no match found
         8: foo1
check:2     ~~~~
         9: foo2
check:2     ~~~~
        10: foo3
check:2     ~~~~
         .
         .
         .
>>>>>>
```

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D83526
2020-07-10 11:02:11 -04:00
Joel E. Denny bce8fced41 [FileCheck] Implement -dump-input-context
This patch is motivated by discussions at each of:

* <https://reviews.llvm.org/D81422>
* <http://lists.llvm.org/pipermail/llvm-dev/2020-June/142369.html>

When input is dumped as specified by `-dump-input=fail`, this patch
filters the dump to show only input lines that are the starting lines
of error diagnostics plus the number of contextual lines specified
`-dump-input-context` (defaults to 5).

When `-dump-input=always`, there might be not be any errors, so all
input lines are printed, as without this patch.

Here's some sample output with `-dump-input-context=3 -vv`:

```
<<<<<<
           .
           .
           .
          13: foo
          14: foo
          15: hello world
check:1       ^~~~~~~~~~~
          16: foo
check:2'0     X~~ error: no match found
          17: foo
check:2'0     ~~~
          18: foo
check:2'0     ~~~
          19: foo
check:2'0     ~~~
           .
           .
           .
          27: foo
check:2'0     ~~~
          28: foo
check:2'0     ~~~
          29: foo
check:2'0     ~~~
          30: goodbye word
check:2'0     ~~~~~~~~~~~~
check:2'1     ?            possible intended match
          31: foo
check:2'0     ~~~
          32: foo
check:2'0     ~~~
          33: foo
check:2'0     ~~~
           .
           .
           .
>>>>>>
```

Reviewed By: mehdi_amini, arsenm, jhenderson, rsmith, SjoerdMeijer, Meinersbur, lattner

Differential Revision: https://reviews.llvm.org/D82203
2020-07-10 11:02:10 -04:00
Joel E. Denny 839f8e4fe2 [FileCheck] Improve -dump-input documentation
Document the default of `fail` in `-help`.  Extend `-dump-input=help`
to help users find related command-line options, but let `-help`
provide their full documentation.

Reviewed By: probinson

Differential Revision: https://reviews.llvm.org/D83091
2020-07-09 18:00:30 -04:00
Joel E. Denny 782585a214 [FileCheck] Permit multiple -v or -vv
`FILECHECK_OPTS` was implemented so that a test runner, such as CI,
can specify FileCheck debugging options, such as `-v` and `-vv`.
However, if a test suite has a FileCheck call that already specifies
`-v` or `-vv`, then that call will fail if `FILECHECK_OPTS` also
specifies it.

For `-vv`, this problem already exists:

`clang/test/CodeGen/aarch64-v8.2a-fp16-intrinsics-constrained.c`

It's not yet clear if the `-vv` in that test was intentional, but this
usage shouldn't fail anyway.  It's already true that FileCheck permits
`-vv` and `-v` together even though `-vv` implies `-v`.

Compare D70784, which fixed the same problem for `-dump-input`.

Reviewed By: jhenderson, thopre

Differential Revision: https://reviews.llvm.org/D82601
2020-06-29 18:35:22 -04:00
Joel E. Denny 3b83501c29 [FileCheck][NFC] Remove redundant DumpInputDefault
Reviewed By: mehdi_amini, jhenderson

Differential Revision: https://reviews.llvm.org/D82480
2020-06-25 12:35:03 -04:00
Mehdi Amini d31c9e5a46 Change filecheck default to dump input on failure
Having the input dumped on failure seems like a better
default: I debugged FileCheck tests for a while without knowing
about this option, which really helps to understand failures.

Remove `-dump-input-on-failure` and the environment variable
FILECHECK_DUMP_INPUT_ON_FAILURE which are now obsolete.

Differential Revision: https://reviews.llvm.org/D81422
2020-06-09 18:57:46 +00:00
Joel E. Denny a1fd188223 [FileCheck] Support comment directives
Sometimes you want to disable a FileCheck directive without removing
it entirely, or you want to write comments that mention a directive by
name.  The `COM:` directive makes it easy to do this.  For example,
you might have:

```
; X32: pinsrd_1:
; X32:    pinsrd $1, 4(%esp), %xmm0

; COM: FIXME: X64 isn't working correctly yet for this part of codegen, but
; COM: X64 will have something similar to X32:
; COM:
; COM:   X64: pinsrd_1:
; COM:   X64:    pinsrd $1, %edi, %xmm0
```

Without this patch, you need to use some combination of rewording and
directive syntax mangling to prevent FileCheck from recognizing the
commented occurrences of `X32:` and `X64:` above as directives.
Moreover, FileCheck diagnostics have been proposed that might complain
about the occurrences of `X64` that don't have the trailing `:`
because they look like directive typos:

  <http://lists.llvm.org/pipermail/llvm-dev/2020-April/140610.html>

I think dodging all these problems can prove tedious for test authors,
and directive syntax mangling already makes the purpose of existing
test code unclear.  `COM:` can avoid all these problems.

This patch also updates the small set of existing tests that define
`COM` as a check prefix:

- clang/test/CodeGen/default-address-space.c
- clang/test/CodeGenOpenCL/addr-space-struct-arg.cl
- clang/test/Driver/hip-device-libs.hip
- llvm/test/Assembler/drop-debug-info-nonzero-alloca.ll

I think lit should support `COM:` as well.  Perhaps `clang -verify`
should too.

Reviewed By: jhenderson, thopre

Differential Revision: https://reviews.llvm.org/D79276
2020-05-13 11:29:48 -04:00
Joel E. Denny 2aa0217add [FileCheck] Make invalid prefix diagnostics more precise
This will prove especially helpful after D79276, which introduces
comment prefixes.  Specifically, identifying whether there's a
uniqueness violation will be helpful as prefixes will be required to
be unique across both check prefixes and comment prefixes.

Also, remove a related comment about `cl::list` that no longer seems
relevant now that FileCheck is also a library.

Reviewed By: jhenderson, thopre

Differential Revision: https://reviews.llvm.org/D79375
2020-05-11 21:11:58 -04:00
Joel E. Denny e1ed4d9eb5 Revert "[FileCheck] Make invalid prefix diagnostics more precise"
This reverts commit a78e13745d to try to
fix a bot:

http://lab.llvm.org:8011/builders/llvm-clang-x86_64-expensive-checks-win/builds/23489
2020-05-11 19:41:22 -04:00
Joel E. Denny d0e7fd6b62 Revert "[FileCheck] Support comment directives"
This reverts commit 9a9a5f9893 to try to
fix a bot:

http://lab.llvm.org:8011/builders/llvm-clang-x86_64-expensive-checks-win/builds/23489
2020-05-11 19:41:22 -04:00
Joel E. Denny 9a9a5f9893 [FileCheck] Support comment directives
Sometimes you want to disable a FileCheck directive without removing
it entirely, or you want to write comments that mention a directive by
name.  The `COM:` directive makes it easy to do this.  For example,
you might have:

```
; X32: pinsrd_1:
; X32:    pinsrd $1, 4(%esp), %xmm0

; COM: FIXME: X64 isn't working correctly yet for this part of codegen, but
; COM: X64 will have something similar to X32:
; COM:
; COM:   X64: pinsrd_1:
; COM:   X64:    pinsrd $1, %edi, %xmm0
```

Without this patch, you need to use some combination of rewording and
directive syntax mangling to prevent FileCheck from recognizing the
commented occurrences of `X32:` and `X64:` above as directives.
Moreover, FileCheck diagnostics have been proposed that might complain
about the occurrences of `X64` that don't have the trailing `:`
because they look like directive typos:

  <http://lists.llvm.org/pipermail/llvm-dev/2020-April/140610.html>

I think dodging all these problems can prove tedious for test authors,
and directive syntax mangling already makes the purpose of existing
test code unclear.  `COM:` can avoid all these problems.

This patch also updates the small set of existing tests that define
`COM` as a check prefix:

- clang/test/CodeGen/default-address-space.c
- clang/test/CodeGenOpenCL/addr-space-struct-arg.cl
- clang/test/Driver/hip-device-libs.hip
- llvm/test/Assembler/drop-debug-info-nonzero-alloca.ll

I think lit should support `COM:` as well.  Perhaps `clang -verify`
should too.

Reviewed By: jhenderson, thopre

Differential Revision: https://reviews.llvm.org/D79276
2020-05-11 14:53:48 -04:00
Joel E. Denny a78e13745d [FileCheck] Make invalid prefix diagnostics more precise
This will prove especially helpful after D79276, which introduces
comment prefixes.  Specifically, identifying whether there's a
uniqueness violation will be helpful as prefixes will be required to
be unique across both check prefixes and comment prefixes.

Also, remove a related comment about `cl::list` that no longer seems
relevant now that FileCheck is also a library.

Reviewed By: jhenderson, thopre

Differential Revision: https://reviews.llvm.org/D79375
2020-05-11 14:53:48 -04:00
Georgii Rymar 76e0ab23f6 [FileCheck] - Refactor the code related to string arrays. NFCI.
There are few `std::vector<std::string>` members in
`FileCheckRequest`. This patch changes these arrays to `std::vector<StringRef>`
and refactors the code related to cleanup/improve/simplify it.

Differential revision: https://reviews.llvm.org/D78202
2020-04-20 14:54:49 +03:00
Joel E. Denny ce685455e4 [FileCheck] Fix --dump-input annotation sort per input line
Without this patch, `--dump-input` annotations on a single input line
are sorted by the associated directive's check-file line.  That seemed
fine because that's often identical to the order in which FileCheck
looks for matches for those directives.

The first problem is that an `--implicit-check-not` pattern has no
check-file line.  The logical equivalent is sorting in command-line
order, but that's not implemented.

The second problem is that, unlike a directive, an
`--implicit-check-not` pattern applies at many points, between many
different pairs of directives.  However, sorting in command-line order
gathers all its associated diagnostics together at one point in an
input line's list of annotations.

In general, it seems to be easier to understand FileCheck's logic when
annotations on a single input line are sorted in the order FileCheck
produced the associated diagnostics, so this patch makes that change.
As documented in the patch, the annotation sort order is also
especially relevant to `CHECK-LABEL`, `CHECK-NOT`, and `CHECK-DAG`, so
this patch updates or extends tests to check the sort makes sense for
them.  (However, the sort for `CHECK-DAG` annotations should not
actually be altered by this patch.)

Reviewed By: thopre

Differential Revision: https://reviews.llvm.org/D77607
2020-04-16 15:39:35 -04:00
Joel E. Denny b5a24610fa [FileCheck] Fix --dump-input implicit pattern location
Currently, `--dump-input` implies that all `--implicit-check-not`
patterns appear on line 1 by printing annotations like:

```
       1: foo bar baz
not:1         !~~     error: no match expected
```

This patch changes that to:

```
          1: foo bar baz
not:imp1         !~~     error: no match expected
```

`imp1` indicates the first `--implicit-check-not` pattern.

Reviewed By: thopre

Differential Revision: https://reviews.llvm.org/D77605
2020-04-16 15:39:35 -04:00
David Bozier 6e01cd6795 Improve error message of FileCheck when stdin is empty
Summary: Replace '-' in the error message with <stdin>. This is also consistent with another error message in the code.

Reviewers: jhenderson, probinson, jdenny, grimar, arichardson

Reviewed By: jhenderson

Subscribers: thopre, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D73793
2020-02-04 11:14:55 +00:00
Joel E. Denny fdde18a7c3 [FileCheck] Given multiple -dump-input, prefer most verbose
Problem: `FILECHECK_OPTS` was implemented so that a test runner, such
as a bot, can specify FileCheck debugging options, such as
`-dump-input=fail`.  However, some existing test suites have FileCheck
calls that already specify `-dump-input=fail` or `-dump-input=always`.
Without this patch, such tests fail under such a test runner because
FileCheck doesn't accept multiple occurrences of `-dump-input`.

Solution: This patch permits multiple occurrences of `-dump-input` by
assigning precedence to its values in the following descending order:
`help`, `always`, `fail`, and `never`.  That is, any occurrence of
`help` always obtains help, and otherwise the behavior is similar to
`-v` vs. `-vv` in that the option specifying the greatest verbosity
has precedence.

Rationale: My justification for the new behavior is as follows.  I
have not experienced use cases where, either as a test runner or as a
test author, I want to **limit** the permitted debugging verbosity
(except as a test author in FileCheck's or lit's test suites where the
FileCheck debugging output itself is under test, but the solution
there is `env FILECHECK_OPTS=`, and I imagine we should use the same
solution anywhere else this need might occur).  Of course, as either a
test runner or test author, it is useful to **increase** debugging
verbosity.

Reviewed By: probinson

Differential Revision: https://reviews.llvm.org/D70784
2019-12-03 14:21:13 -05:00
Kai Nacke 5b5b2fd2b8 [FileCheck] Implement --ignore-case option.
The FileCheck utility is enhanced to support a `--ignore-case`
option. This is useful in cases where the output of Unix tools
differs in case (e.g. case not specified by Posix).

Reviewers: Bigcheese, jakehehrlich, rupprecht, espindola, alexshap, jhenderson, MaskRay

Differential Revision: https://reviews.llvm.org/D68146

llvm-svn: 374538
2019-10-11 11:59:14 +00:00
Dmitri Gribenko d3aed7fc79 Revert "[FileCheck] Implement --ignore-case option."
This reverts commit r374339. It broke tests:
http://lab.llvm.org:8011/builders/clang-x86_64-debian-fast/builds/19066

llvm-svn: 374359
2019-10-10 14:27:14 +00:00
Kai Nacke dfd2b6f07f [FileCheck] Implement --ignore-case option.
The FileCheck utility is enhanced to support a `--ignore-case`
option. This is useful in cases where the output of Unix tools
differs in case (e.g. case not specified by Posix).

Reviewers: Bigcheese, jakehehrlich, rupprecht, espindola, alexshap, jhenderson, MaskRay

Differential Revision: https://reviews.llvm.org/D68146

llvm-svn: 374339
2019-10-10 13:15:41 +00:00
Thomas Preud'homme 02ada9bd2b [FileCheck] Remove implementation types from API
Summary:
Remove use of FileCheckPatternContext and FileCheckString concrete types
from FileCheck API to allow moving it and the other implementation only
only declarations into a private header file.

Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk

Subscribers: hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D68186

llvm-svn: 373211
2019-09-30 14:12:03 +00:00
Joel E. Denny dbb757f462 [FileCheck] Document FILECHECK_OPTS in -help
Reviewed By: thopre

Differential Revision: https://reviews.llvm.org/D65707

llvm-svn: 368787
2019-08-14 02:56:20 +00:00
Joel E. Denny 608f2bfd65 [FileCheck] Move -dump-input diagnostic to first line
Without this patch, `-dump-input` prints a diagnostic at the end of
its marker range.  For example:

```
         1: Start.
check:1     ^~~~~~
         2: Bad.
next:2      X~~~
         3: Many lines
next:2      ~~~~~~~~~~
         4: of input.
next:2      ~~~~~~~~~
         5: End.
next:2      ~~~~ error: no match found
```

This patch moves it to the beginning like this:

```
         1: Start.
check:1     ^~~~~~
         2: Bad.
next:2      X~~~ error: no match found
         3: Many lines
next:2      ~~~~~~~~~~
         4: of input.
next:2      ~~~~~~~~~
         5: End.
next:2      ~~~~
```

The former somehow looks nicer because the diagnostic doesn't appear
to be somewhere within the marker range.  However, the latter is more
practical, especially when the marker range includes the remainder of
a very long dump.  First, in the case of an error, this patch enables
me to search the dump for `error:` and usually immediately land where
the detected error began.  Second, when trying to follow FileCheck's
logic, it's best to read top down, so this patch enables me to see
each diagnostic as soon as I encounter its marker.

Reviewed By: thopre

Differential Revision: https://reviews.llvm.org/D65702

llvm-svn: 368786
2019-08-14 02:56:09 +00:00
Rui Ueyama 4d41c332ef Revert r367649: Improve raw_ostream so that you can "write" colors using operator<<
This reverts commit r367649 in an attempt to unbreak Windows bots.

llvm-svn: 367658
2019-08-02 07:22:34 +00:00
Rui Ueyama a52f982f1c Improve raw_ostream so that you can "write" colors using operator<<
1. raw_ostream supports ANSI colors so that you can write messages to
the termina with colors. Previously, in order to change and reset
color, you had to call `changeColor` and `resetColor` functions,
respectively.

So, if you print out "error: " in red, for example, you had to do
something like this:

  OS.changeColor(raw_ostream::RED);
  OS << "error: ";
  OS.resetColor();

With this patch, you can write the same code as follows:

  OS << raw_ostream::RED << "error: " << raw_ostream::RESET;

2. Add a boolean flag to raw_ostream so that you can disable colored
output. If you disable colors, changeColor, operator<<(Color),
resetColor and other color-related functions have no effect.

Most LLVM tools automatically prints out messages using colors, and
you can disable it by passing a flag such as `--disable-colors`.
This new flag makes it easy to write code that works that way.

Differential Revision: https://reviews.llvm.org/D65564

llvm-svn: 367649
2019-08-02 04:48:30 +00:00
Michal Gorny ffc722a358 [llvm] [FileCheck] Use FILECHECK_DUMP_INPUT_ON_FAILURE only when non-empty
Enable dumping output only if FILECHECK_DUMP_INPUT_ON_FAILURE is set to
a non-empty value.  This is necessary to support disabling it via
POSIX-compliant env(1) that does not support '-u' argument,
and therefore fix regression caused by r366980.

Differential Revision: https://reviews.llvm.org/D65334

llvm-svn: 367122
2019-07-26 15:38:57 +00:00
Thomas Preud'homme 1a944d27b2 FileCheck: Improve FileCheck variable terminology
Summary:
Terminology introduced by [[#]] blocks is confusing and does not
integrate well with existing terminology.

First, variables referred by [[]] blocks are called "pattern variables"
while the text a CHECK directive needs to match is called a "CHECK
pattern". This is inconsistent with variables in [[#]] blocks since
[[#]] blocks are also found in CHECK pattern yet those variables are
called "numeric variable".

Second, the replacing of both [[]] and [[#]] blocks by the value of the
variable or expression they contain is represented by a
FileCheckPatternSubstitution class. The naming refers to being a
substitution in a CHECK pattern but could be wrongly understood as being
a substitution of a pattern variable.

Third and lastly, comments use "numeric expression" to refer both to the
[[#]] blocks as well as to the numeric expressions these blocks contain
which get evaluated at match time.

This patch solves these confusions by
- calling variables in [[]] and [[#]] blocks as string and numeric
  variables respectively;
- referring to [[]] and [[#]] as substitution *blocks*, with the former
  being a string substitution block and the latter a numeric
  substitution block;
- calling [[]] and [[#]] blocks to be replaced by the value of a
  variable or expression they contain a substitution (as opposed to
  definition when these blocks are used to defined a variable), with the
  former being a string substitution and the latter a numeric
  substitution;
- renaming the FileCheckPatternSubstitution as a FileCheckSubstitution
  class with FileCheckStringSubstitution and
  FileCheckNumericSubstitution subclasses;
- restricting the use of "numeric expression" to refer to the expression
  that is evaluated in a numeric substitution.

While numeric substitution blocks only support numeric substitutions of
numeric expressions at the moment there are plans to augment numeric
substitution blocks to support numeric definitions as well as both a
numeric definition and numeric substitution in the same numeric
substitution block.

Reviewers: jhenderson, jdenny, probinson, arichardson

Subscribers: hiraditya, arichardson, probinson, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D62146

llvm-svn: 361445
2019-05-23 00:10:14 +00:00
Rainer Orth 010982f750 [FileCheck] Fix FileCheck.cpp compilation on Solaris
Both LLVM 8.0.0 and current trunk fail to compile on Solaris with GCC 8.1.0:

  /vol/llvm/src/llvm/dist/utils/FileCheck/FileCheck.cpp: In function ‘void DumpAnnotatedInput(llvm::raw_ostream&, const llvm::FileCheckRequest&, llvm::StringRef, std::vector<InputAnnotation>&, unsigned int)’:
  /vol/llvm/src/llvm/dist/utils/FileCheck/FileCheck.cpp:408:41: error: call of overloaded ‘log10(unsigned int&)’ is ambiguous
     unsigned LineNoWidth = log10(LineCount) + 1;
                                           ^
  In file included from /vol/gcc-8/lib/gcc/i386-pc-solaris2.11/8.1.0/include-fixed/math.h:24,
                   from /vol/gcc-8/include/c++/8.1.0/cmath:45,
                   from /vol/llvm/src/llvm/dist/include/llvm-c/DataTypes.h:28,
                   from /vol/llvm/src/llvm/dist/include/llvm/Support/DataTypes.h:16,
                   from /vol/llvm/src/llvm/dist/include/llvm/ADT/Hashing.h:47,
                   from /vol/llvm/src/llvm/dist/include/llvm/ADT/ArrayRef.h:12,
                   from /vol/llvm/src/llvm/dist/include/llvm/Support/CommandLine.h:22,
                   from /vol/llvm/src/llvm/dist/utils/FileCheck/FileCheck.cpp:18:
  /vol/gcc-8/lib/gcc/i386-pc-solaris2.11/8.1.0/include-fixed/iso/math_iso.h:209:21: note: candidate: ‘long double std::log10(long double)’
    inline long double log10(long double __X) { return __log10l(__X); }
                       ^~~~~
  /vol/gcc-8/lib/gcc/i386-pc-solaris2.11/8.1.0/include-fixed/iso/math_iso.h:170:15: note: candidate: ‘float std::log10(float)’
    inline float log10(float __X) { return __log10f(__X); }
                 ^~~~~
  /vol/gcc-8/lib/gcc/i386-pc-solaris2.11/8.1.0/include-fixed/iso/math_iso.h:70:15: note: candidate: ‘double std::log10(double)’
   extern double log10 __P((double));
                 ^~~~~

Fixed by using std::log10 instead, which allowed the compilation on i386-pc-solaris2.11
and sparc-sun-solaris2.11 to continue.

Differential Revision: https://reviews.llvm.org/D60043

llvm-svn: 357509
2019-04-02 18:38:23 +00:00
Thomas Preud'homme a5e233bf79 Recommit: Detect incorrect FileCheck variable CLI definition
Summary:
While the backend code of FileCheck relies on definition of variable
from the command-line to have an equal sign '=' and a variable name
before that, the frontend does not actually enforce it. This leads to
FileCheck crashing when invoked with invalid syntax for the -D option.

This patch adds the missing validation in the frontend. It also makes
the -D option an AlwaysPrefix option to be able to detect -D=FOO as
being a define without variable and -D as missing its value.

Copyright:
- Linaro (changes in version 2 of revision D55940)
- GraphCore (changes in later versions)

Reviewers: jdenny

Subscribers: JonChesterfield, hiraditya, kristina, probinson,
llvm-commits

Differential Revision: https://reviews.llvm.org/D55940

llvm-svn: 353173
2019-02-05 14:17:28 +00:00
Thomas Preud'homme 447abc57c5 Revert "Detect incorrect FileCheck variable CLI definition"
This reverts commit r351039.

llvm-svn: 352309
2019-01-27 09:02:19 +00:00
Joel E. Denny 352695c336 [FileCheck] Suppress old -v/-vv diags if dumping input
The old diagnostic form of the trace produced by -v and -vv looks
like:

```
check1:1:8: remark: CHECK: expected string found in input
CHECK: abc
       ^
<stdin>:1:3: note: found here
; abc def
  ^~~
```

When dumping annotated input is requested (via -dump-input), I find
that this old trace is not useful and is sometimes harmful:

1. The old trace is mostly redundant because the same basic
   information also appears in the input dump's annotations.

2. The old trace buries any error diagnostic between it and the input
   dump, but I find it useful to see any error diagnostic up front.

3. FILECHECK_OPTS=-dump-input=fail requests annotated input dumps only
   for failed FileCheck calls.  However, I have to also add -v or -vv
   to get a full set of annotations, and that can produce massive
   output from all FileCheck calls in all tests.  That's a real
   problem when I run this in the IDE I use, which grinds to a halt as
   it tries to capture all that output.

When -dump-input=fail|always, this patch suppresses the old trace from
-v or -vv.  Error diagnostics still print as usual.  If you want the
old trace, perhaps to see variable expansions, you can set
-dump-input=none (the default).

Reviewed By: probinson

Differential Revision: https://reviews.llvm.org/D55825

llvm-svn: 351881
2019-01-22 21:41:42 +00:00