Summary:
With now a clear distinction between string and numeric substitutions,
this patch introduces separate classes to represent them with a parent
class implementing the common interface. Diagnostics in
printSubstitutions() are also adapted to not require knowing which
substitution is being looked at since it does not hinder clarity and
makes the implementation simpler.
Reviewers: jhenderson, jdenny, probinson, arichardson
Subscribers: llvm-commits, probinson, arichardson, hiraditya
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D62241
llvm-svn: 361446
Summary:
Terminology introduced by [[#]] blocks is confusing and does not
integrate well with existing terminology.
First, variables referred by [[]] blocks are called "pattern variables"
while the text a CHECK directive needs to match is called a "CHECK
pattern". This is inconsistent with variables in [[#]] blocks since
[[#]] blocks are also found in CHECK pattern yet those variables are
called "numeric variable".
Second, the replacing of both [[]] and [[#]] blocks by the value of the
variable or expression they contain is represented by a
FileCheckPatternSubstitution class. The naming refers to being a
substitution in a CHECK pattern but could be wrongly understood as being
a substitution of a pattern variable.
Third and lastly, comments use "numeric expression" to refer both to the
[[#]] blocks as well as to the numeric expressions these blocks contain
which get evaluated at match time.
This patch solves these confusions by
- calling variables in [[]] and [[#]] blocks as string and numeric
variables respectively;
- referring to [[]] and [[#]] as substitution *blocks*, with the former
being a string substitution block and the latter a numeric
substitution block;
- calling [[]] and [[#]] blocks to be replaced by the value of a
variable or expression they contain a substitution (as opposed to
definition when these blocks are used to defined a variable), with the
former being a string substitution and the latter a numeric
substitution;
- renaming the FileCheckPatternSubstitution as a FileCheckSubstitution
class with FileCheckStringSubstitution and
FileCheckNumericSubstitution subclasses;
- restricting the use of "numeric expression" to refer to the expression
that is evaluated in a numeric substitution.
While numeric substitution blocks only support numeric substitutions of
numeric expressions at the moment there are plans to augment numeric
substitution blocks to support numeric definitions as well as both a
numeric definition and numeric substitution in the same numeric
substitution block.
Reviewers: jhenderson, jdenny, probinson, arichardson
Subscribers: hiraditya, arichardson, probinson, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D62146
llvm-svn: 361445
This reinstates r360578 (git e47362c1ec),
reverted in r360653 (git 004393681c),
with a fix for the list added in FileCheck.rst to build without error.
Copyright:
- Linaro (changes up to diff 183612 of revision D55940)
- GraphCore (changes in later versions of revision D55940 and
in new revision created off D55940)
Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar,
arichardson, rnk
Subscribers: hiraditya, llvm-commits, probinson, dblaikie, grimar,
arichardson, tra, rnk, kristina, hfinkel, rogfer01, JonChesterfield
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D60385
llvm-svn: 360665
Summary:
This patch is part of a patch series to add support for FileCheck
numeric expressions. This specific patch introduces regular numeric
variables which can be set on the command-line.
This commit introduces regular numeric variable that can be set on the
command-line with the -D option to a numeric value. They can then be
used in CHECK patterns in numeric expression with the same shape as
@LINE numeric expression, ie. VAR, VAR+offset or VAR-offset where offset
is an integer literal.
The commit also enable strict whitespace in the verbose.txt testcase to
check that the position or the location diagnostics. It fixes one of the
existing CHECK in the process which was not accurately testing a
location diagnostic (ie. the diagnostic was correct, not the CHECK).
Copyright:
- Linaro (changes up to diff 183612 of revision D55940)
- GraphCore (changes in later versions of revision D55940 and
in new revision created off D55940)
Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk
Subscribers: hiraditya, llvm-commits, probinson, dblaikie, grimar, arichardson, tra, rnk, kristina, hfinkel, rogfer01, JonChesterfield
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D60385
llvm-svn: 360578
Summary:
Split defines.txt into diagnostics test and functionality test. Also add
comments, remove the semicolon prefix and group RUN lines with their
CHECK directives.
Reviewers: jhenderson, probinson, arichardson
Subscribers: llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D61679
llvm-svn: 360289
Summary:
By default, `parseCommandLineOptions()` will accept either a
`-` or `--` prefix for long options -- options with names longer than
a single character.
While this change does not affect behavior, it will be helpful with a
subsequent change that requires long options use the `--` prefix.
Reviewers: rnk, thopre
Reviewed By: thopre
Subscribers: thopre, cfe-commits, hiraditya, llvm-commits
Tags: #llvm, #clang
Differential Revision: https://reviews.llvm.org/D61269
llvm-svn: 359909
Summary:
This patch is part of a patch series to add support for FileCheck
numeric expressions. This specific patch introduces the @LINE numeric
expressions.
This commit introduces a new syntax to express a relation a numeric
value in the input text must have with the line number of a given CHECK
pattern: [[#<@LINE numeric expression>]]. Further commits build on that
to express relations between several numeric values in the input text.
To help with naming, regular variables are renamed into pattern
variables and old @LINE expression syntax is referred to as legacy
numeric expression.
Compared to existing @LINE expressions, this new syntax allow arbitrary
spacing between the component of the expression. It offers otherwise the
same functionality but the commit serves to introduce some of the data
structure needed to support more general numeric expressions.
Copyright:
- Linaro (changes up to diff 183612 of revision D55940)
- GraphCore (changes in later versions of revision D55940 and
in new revision created off D55940)
Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk
Subscribers: hiraditya, llvm-commits, probinson, dblaikie, grimar, arichardson, tra, rnk, kristina, hfinkel, rogfer01, JonChesterfield
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D60384
llvm-svn: 359741
Summary:
This patch is part of a patch series to add support for FileCheck
numeric expressions. This specific patch gives earlier and better
diagnostics for the @LINE expressions.
Rather than detect parsing errors at matching time, this commit adds
enhance parsing to detect issues with @LINE expressions at parse time
and diagnose them more accurately.
Copyright:
- Linaro (changes up to diff 183612 of revision D55940)
- GraphCore (changes in later versions of revision D55940 and
in new revision created off D55940)
Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk
Subscribers: hiraditya, llvm-commits, probinson, dblaikie, grimar, arichardson, tra, rnk, kristina, hfinkel, rogfer01, JonChesterfield
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D60383
llvm-svn: 359475
Summary:
This patch is part of a patch series to add support for FileCheck
numeric expressions. This specific patch gives earlier and better
diagnostics for the -D option.
Prior to this change, parsing of -D option was very loose: it assumed
that there is an equal sign (which to be fair is now checked by the
FileCheck executable) and that the part on the left of the equal sign
was a valid variable name. This commit adds logic to ensure that this
is the case and gives diagnostic when it is not, making it clear that
the issue came from a command-line option error. This is achieved by
sharing the variable parsing code into a new function ParseVariable.
Copyright:
- Linaro (changes up to diff 183612 of revision D55940)
- GraphCore (changes in later versions of revision D55940 and
in new revision created off D55940)
Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk
Subscribers: hiraditya, llvm-commits, probinson, dblaikie, grimar, arichardson, tra, rnk, kristina, hfinkel, rogfer01, JonChesterfield
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D60382
llvm-svn: 359447
This patch removes two assertions that were preventing writing of a test
that checked an empty line followed by some text. For example:
CHECK: {{^$}}
CHECK-NEXT: foo()
The assertion was because the current location the CHECK-NEXT was
scanning from was the start of the buffer. A similar issue occurred with
CHECK-SAME. These assertions don't protect against anything, as there is
already an error check that checks that CHECK-NEXT/EMPTY/SAME don't
appear first in the checks, and the following code works fine if the
pointer is at the start of the input.
Reviewed by: probinson, thopre, jdenny
Differential Revision: https://reviews.llvm.org/D58784
llvm-svn: 355928
Summary:
While the backend code of FileCheck relies on definition of variable
from the command-line to have an equal sign '=' and a variable name
before that, the frontend does not actually enforce it. This leads to
FileCheck crashing when invoked with invalid syntax for the -D option.
This patch adds the missing validation in the frontend. It also makes
the -D option an AlwaysPrefix option to be able to detect -D=FOO as
being a define without variable and -D as missing its value.
Copyright:
- Linaro (changes in version 2 of revision D55940)
- GraphCore (changes in later versions)
Reviewers: jdenny
Subscribers: JonChesterfield, hiraditya, kristina, probinson,
llvm-commits
Differential Revision: https://reviews.llvm.org/D55940
llvm-svn: 353173
On Unix/Mac OS X, normpath() returns the path unchanged (FileCheck), but
on case-insensitive filesystems (like NTFS on Windows), it converts the
path to lowercase (filecheck) which was causing the test to fail.
llvm-svn: 352735
The old diagnostic form of the trace produced by -v and -vv looks
like:
```
check1:1:8: remark: CHECK: expected string found in input
CHECK: abc
^
<stdin>:1:3: note: found here
; abc def
^~~
```
When dumping annotated input is requested (via -dump-input), I find
that this old trace is not useful and is sometimes harmful:
1. The old trace is mostly redundant because the same basic
information also appears in the input dump's annotations.
2. The old trace buries any error diagnostic between it and the input
dump, but I find it useful to see any error diagnostic up front.
3. FILECHECK_OPTS=-dump-input=fail requests annotated input dumps only
for failed FileCheck calls. However, I have to also add -v or -vv
to get a full set of annotations, and that can produce massive
output from all FileCheck calls in all tests. That's a real
problem when I run this in the IDE I use, which grinds to a halt as
it tries to capture all that output.
When -dump-input=fail|always, this patch suppresses the old trace from
-v or -vv. Error diagnostics still print as usual. If you want the
old trace, perhaps to see variable expansions, you can set
-dump-input=none (the default).
Reviewed By: probinson
Differential Revision: https://reviews.llvm.org/D55825
llvm-svn: 351881
Support arbitrary suffix when matching FileCheck executable name in
defines.txt to successfully match FileCheck.EXE on Microsoft Windows.
llvm-svn: 351042
Summary:
While the backend code of FileCheck relies on definition of variable
from the command-line to have an equal sign '=' and a variable name
before that, the frontend does not actually enforce it. This leads to
FileCheck crashing when invoked with invalid syntax for the -D option.
This patch adds the missing validation in the frontend. It also makes
the -D option an AlwaysPrefix option to be able to detect -D=FOO as
being a define without variable and -D as missing its value.
Copyright:
- Linaro (changes in version 2 of revision D55940)
- GraphCore (changes in later versions)
Reviewers: jdenny
Subscribers: JonChesterfield, hiraditya, kristina, probinson,
llvm-commits
Differential Revision: https://reviews.llvm.org/D55940
llvm-svn: 351039
`FILECHECK_OPTS` into environment for FileCheck tests.
Summary:
This fixes the following FileCheck tests:
* FileCheck/dump-input-enable.txt
* FileCheck/match-full-lines.txt
when `FILECHECK_DUMP_INPUT_ON_FAILURE` is set in the environment.
By default llvm-lit propagates `FILECHECK_DUMP_INPUT_ON_FAILURE` and
`FILECHECK_OPTS` from llvm-lit's environment into the test environment.
Unfortunately this can break FileCheck's tests because they expect that
these environment variables not to be set.
rdar://problem/47176262
Reviewers: jdenny, probinson, george.karpenkov
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D56541
llvm-svn: 350850
Apply final suggestions from probinson for this patch series plus a
few more tweaks:
* Improve various docs, for MatchType in particular.
* Rename some members of MatchType. The main problem was that the
term "final match" became a misnomer when CHECK-COUNT-<N> was
created.
* Split InputStartLine, etc. declarations into multiple lines.
Differential Revision: https://reviews.llvm.org/D55738
Reviewed By: probinson
llvm-svn: 349425
This patch implements annotations for diagnostics reporting CHECK-NOT
failed matches. These diagnostics are enabled by -vv. As for
diagnostics reporting failed matches for other directives, these
annotations mark the search ranges using `X~~`. The difference here
is that failed matches for CHECK-NOT are successes not errors, so they
are green not red when colors are enabled.
For example:
```
$ FileCheck -dump-input=help
The following description was requested by -dump-input=help to
explain the input annotations printed by -dump-input=always and
-dump-input=fail:
- L: labels line number L of the input file
- T:L labels the only match result for a pattern of type T from line L of
the check file
- T:L'N labels the Nth match result for a pattern of type T from line L of
the check file
- ^~~ marks good match (reported if -v)
- !~~ marks bad match, such as:
- CHECK-NEXT on same line as previous match (error)
- CHECK-NOT found (error)
- CHECK-DAG overlapping match (discarded, reported if -vv)
- X~~ marks search range when no match is found, such as:
- CHECK-NEXT not found (error)
- CHECK-NOT not found (success, reported if -vv)
- CHECK-DAG not found after discarded matches (error)
- ? marks fuzzy match when no match is found
- colors success, error, fuzzy match, discarded match, unmatched input
If you are not seeing color above or in input dumps, try: -color
$ FileCheck -vv -dump-input=always check5 < input5 |& sed -n '/^<<<</,$p'
<<<<<<
1: abcdef
check:1 ^~~
not:2 X~~
2: ghijkl
not:2 ~~~
check:3 ^~~
3: mnopqr
not:4 X~~~~~
4: stuvwx
not:4 ~~~~~~
5:
eof:4 ^
>>>>>>
$ cat check5
CHECK: abc
CHECK-NOT: foobar
CHECK: jkl
CHECK-NOT: foobar
$ cat input5
abcdef
ghijkl
mnopqr
stuvwx
```
Reviewed By: george.karpenkov, probinson
Differential Revision: https://reviews.llvm.org/D53899
llvm-svn: 349424
This patch implements input annotations for diagnostics reporting
CHECK-DAG discarded matches. These diagnostics are enabled by -vv.
These annotations mark discarded match ranges using `!~~` because they
are bad matches even though they are not errors.
CHECK-DAG discarded matches create another case where there can be
multiple match results for the same directive.
For example:
```
$ FileCheck -dump-input=help
The following description was requested by -dump-input=help to
explain the input annotations printed by -dump-input=always and
-dump-input=fail:
- L: labels line number L of the input file
- T:L labels the only match result for a pattern of type T from line L of
the check file
- T:L'N labels the Nth match result for a pattern of type T from line L of
the check file
- ^~~ marks good match (reported if -v)
- !~~ marks bad match, such as:
- CHECK-NEXT on same line as previous match (error)
- CHECK-NOT found (error)
- CHECK-DAG overlapping match (discarded, reported if -vv)
- X~~ marks search range when no match is found, such as:
- CHECK-NEXT not found (error)
- CHECK-DAG not found after discarded matches (error)
- ? marks fuzzy match when no match is found
- colors success, error, fuzzy match, discarded match, unmatched input
If you are not seeing color above or in input dumps, try: -color
$ FileCheck -vv -dump-input=always check4 < input4 |& sed -n '/^<<<</,$p'
<<<<<<
1: abcdef
dag:1 ^~~~
dag:2'0 !~~~ discard: overlaps earlier match
2: cdefgh
dag:2'1 ^~~~
check:3 X~ error: no match found
>>>>>>
$ cat check4
CHECK-DAG: abcd
CHECK-DAG: cdef
CHECK: efgh
$ cat input4
abcdef
cdefgh
```
This shows that the line 3 CHECK fails to match even though its
pattern appears in the input because its search range starts after the
line 2 CHECK-DAG's match range. The trouble might be that the line 2
CHECK-DAG's match range is later than expected because its first match
range overlaps with the line 1 CHECK-DAG match range and thus is
discarded.
Because `!~~` for CHECK-DAG does not indicate an error, it is not
colored red. Instead, when colors are enabled, it is colored cyan,
which suggests a match that went cold.
Reviewed By: george.karpenkov, probinson
Differential Revision: https://reviews.llvm.org/D53898
llvm-svn: 349423
This patch implements input annotations for diagnostics enabled by -v,
which report good matches for directives. These annotations mark
match ranges using `^~~`.
For example:
```
$ FileCheck -dump-input=help
The following description was requested by -dump-input=help to
explain the input annotations printed by -dump-input=always and
-dump-input=fail:
- L: labels line number L of the input file
- T:L labels the only match result for a pattern of type T from line L of
the check file
- T:L'N labels the Nth match result for a pattern of type T from line L of
the check file
- ^~~ marks good match (reported if -v)
- !~~ marks bad match, such as:
- CHECK-NEXT on same line as previous match (error)
- CHECK-NOT found (error)
- X~~ marks search range when no match is found, such as:
- CHECK-NEXT not found (error)
- ? marks fuzzy match when no match is found
- colors success, error, fuzzy match, unmatched input
If you are not seeing color above or in input dumps, try: -color
$ FileCheck -v -dump-input=always check3 < input3 |& sed -n '/^<<<</,$p'
<<<<<<
1: abc foobar def
check:1 ^~~
not:2 !~~~~~ error: no match expected
check:3 ^~~
>>>>>>
$ cat check3
CHECK: abc
CHECK-NOT: foobar
CHECK: def
$ cat input3
abc foobar def
```
-vv enables these annotations for FileCheck's implicit EOF patterns as
well. For an example where EOF patterns become relevant, see patch 7
in this series.
If colors are enabled, `^~~` is green to suggest success.
-v plus color enables highlighting of input text that has no final
match for any expected pattern. The highlight uses a cyan background
to suggest a cold section. This highlighting can make it easier to
spot text that was intended to be matched but that failed to be
matched in a long series of good matches.
CHECK-COUNT-<num> good matches are another case where there can be
multiple match results for the same directive.
Reviewed By: george.karpenkov, probinson
Differential Revision: https://reviews.llvm.org/D53897
llvm-svn: 349422
This patch implements input annotations for diagnostics that report
unexpected matches for CHECK-NOT. Like wrong-line matches for
CHECK-NEXT, CHECK-SAME, and CHECK-EMPTY, these annotations mark match
ranges using red `!~~` to indicate bad matches that are errors.
For example:
```
$ FileCheck -dump-input=help
The following description was requested by -dump-input=help to
explain the input annotations printed by -dump-input=always and
-dump-input=fail:
- L: labels line number L of the input file
- T:L labels the only match result for a pattern of type T from line L of
the check file
- T:L'N labels the Nth match result for a pattern of type T from line L of
the check file
- !~~ marks bad match, such as:
- CHECK-NEXT on same line as previous match (error)
- CHECK-NOT found (error)
- X~~ marks search range when no match is found, such as:
- CHECK-NEXT not found (error)
- ? marks fuzzy match when no match is found
- colors error, fuzzy match
If you are not seeing color above or in input dumps, try: -color
$ FileCheck -v -dump-input=always check3 < input3 |& sed -n '/^<<<</,$p'
<<<<<<
1: abc foobar def
not:2 !~~~~~ error: no match expected
>>>>>>
$ cat check3
CHECK: abc
CHECK-NOT: foobar
CHECK: def
$ cat input3
abc foobar def
```
Reviewed By: george.karpenkov, probinson
Differential Revision: https://reviews.llvm.org/D53896
llvm-svn: 349421
This patch implements input annotations for diagnostics that report
wrong-line matches for the directives CHECK-NEXT, CHECK-SAME, and
CHECK-EMPTY. Instead of the usual `^~~`, which is used by later
patches for good matches, these annotations use `!~~` to mark the bad
match ranges so that this category of errors is visually distinct.
Because such matches are errors, these annotates are red when colors
are enabled.
For example:
```
$ FileCheck -dump-input=help
The following description was requested by -dump-input=help to
explain the input annotations printed by -dump-input=always and
-dump-input=fail:
- L: labels line number L of the input file
- T:L labels the only match result for a pattern of type T from line L of
the check file
- T:L'N labels the Nth match result for a pattern of type T from line L of
the check file
- !~~ marks bad match, such as:
- CHECK-NEXT on same line as previous match (error)
- X~~ marks search range when no match is found, such as:
- CHECK-NEXT not found (error)
- ? marks fuzzy match when no match is found
- colors error, fuzzy match
If you are not seeing color above or in input dumps, try: -color
$ FileCheck -v -dump-input=always check2 < input2 |& sed -n '/^<<<</,$p'
<<<<<<
1: foo bar
next:2 !~~ error: match on wrong line
>>>>>>
$ cat check2
CHECK: foo
CHECK-NEXT: bar
$ cat input2
foo bar
```
Reviewed By: george.karpenkov, probinson
Differential Revision: https://reviews.llvm.org/D53894
llvm-svn: 349420
This patch implements input annotations for diagnostics that suggest
fuzzy matches for directives for which no matches were found. Instead
of using the usual `^~~`, which is used by later patches for good
matches, these annotations use `?` so that fuzzy matches are visually
distinct. No tildes are included as these diagnostics (independently
of this patch) currently identify only the start of the match.
For example:
```
$ FileCheck -dump-input=help
The following description was requested by -dump-input=help to
explain the input annotations printed by -dump-input=always and
-dump-input=fail:
- L: labels line number L of the input file
- T:L labels the only match result for a pattern of type T from line L of
the check file
- T:L'N labels the Nth match result for a pattern of type T from line L of
the check file
- X~~ marks search range when no match is found
- ? marks fuzzy match when no match is found
- colors error, fuzzy match
If you are not seeing color above or in input dumps, try: -color
$ FileCheck -v -dump-input=always check1 < input1 |& sed -n '/^<<<</,$p'
<<<<<<
1: ; abc def
2: ; ghI jkl
next:3'0 X~~~~~~~~ error: no match found
next:3'1 ? possible intended match
>>>>>>
$ cat check1
CHECK: abc
CHECK-SAME: def
CHECK-NEXT: ghi
CHECK-SAME: jkl
$ cat input1
; abc def
; ghI jkl
```
This patch introduces the concept of multiple "match results" per
directive. In the above example, the first match result for the
CHECK-NEXT directive is the failed match, for which the annotation
shows the search range. The second match result is the fuzzy match.
Later patches will introduce other cases of multiple match results per
directive.
When colors are enabled, `?` is colored magenta. That is, it doesn't
indicate the actual error, which a red `X~~` marker indicates, but its
color suggests it's closely related.
Reviewed By: george.karpenkov, probinson
Differential Revision: https://reviews.llvm.org/D53893
llvm-svn: 349419
Extend FileCheck to dump its input annotated with FileCheck's
diagnostics: errors, good matches if -v, and additional information if
-vv. The goal is to make it easier to visualize FileCheck's matching
behavior when debugging.
Each patch in this series implements input annotations for a
particular category of FileCheck diagnostics. While the first few
patches alone are somewhat useful, the annotations become much more
useful as later patches implement annotations for -v and -vv
diagnostics, which show the matching behavior leading up to the error.
This first patch implements boilerplate plus input annotations for
error diagnostics reporting that no matches were found for a
directive. These annotations mark the search ranges of the failed
directives. Instead of using the usual `^~~`, which is used by later
patches for good matches, these annotations use `X~~` so that this
category of errors is visually distinct.
For example:
```
$ FileCheck -dump-input=help
The following description was requested by -dump-input=help to
explain the input annotations printed by -dump-input=always and
-dump-input=fail:
- L: labels line number L of the input file
- T:L labels the match result for a pattern of type T from line L of
the check file
- X~~ marks search range when no match is found
- colors error
If you are not seeing color above or in input dumps, try: -color
$ FileCheck -v -dump-input=always check1 < input1 |& sed -n '/^Input file/,$p'
Input file: <stdin>
Check file: check1
-dump-input=help describes the format of the following dump.
Full input was:
<<<<<<
1: ; abc def
2: ; ghI jkl
next:3 X~~~~~~~~ error: no match found
>>>>>>
$ cat check1
CHECK: abc
CHECK-SAME: def
CHECK-NEXT: ghi
CHECK-SAME: jkl
$ cat input1
; abc def
; ghI jkl
```
Some additional details related to the boilerplate:
* Enabling: The annotated input dump is enabled by `-dump-input`,
which can also be set via the `FILECHECK_OPTS` environment variable.
Accepted values are `help`, `always`, `fail`, or `never`. As shown
above, `help` describes the format of the dump. `always` is helpful
when you want to investigate a successful FileCheck run, perhaps for
an unexpected pass. `-dump-input-on-failure` and
`FILECHECK_DUMP_INPUT_ON_FAILURE` remain as a deprecated alias for
`-dump-input=fail`.
* Diagnostics: The usual diagnostics are not suppressed in this mode
and are printed first. For brevity in the example above, I've
omitted them using a sed command. Sometimes they're perfectly
sufficient, and then they make debugging quicker than if you were
forced to hunt through a dump of long input looking for the error.
If you think they'll get in the way sometimes, keep in mind that
it's pretty easy to grep for the start of the input dump, which is
`<<<`.
* Colored Annotations: The annotated input is colored if colors are
enabled (enabling colors can be forced using -color). For example,
errors are red. However, as in the above example, colors are not
vital to reading the annotations.
I don't know how to test color in the output, so any hints here would
be appreciated.
Reviewed By: george.karpenkov, zturner, probinson
Differential Revision: https://reviews.llvm.org/D52999
llvm-svn: 349418
In some cases it is desirable to match the same pattern repeatedly
many times. Currently the only way to do it is to copy the same
check pattern as many times as needed. And that gets pretty unwieldy
when its more than count is big.
Introducing CHECK-COUNT-<num> directive which acts like a plain CHECK
directive yet matches the same pattern exactly <num> times.
Extended FileCheckType to a struct to add Count there.
Changed some parsing routines to handle non-fixed length of directive
(all currently existing directives were fixed-length).
The code is generic enough to allow future support for COUNT in more
than just PlainCheck directives.
See motivating example for this feature in reviews.llvm.org/D54223.
Reviewed By: chandlerc, dblaikie
Differential Revision: https://reviews.llvm.org/D54336
llvm-svn: 346722
This feature makes it easy to tune FileCheck diagnostic output when
running the test suite via ninja, a bot, or an IDE. For example:
```
$ FILECHECK_OPTS='-color -v -dump-input-on-failure' \
LIT_FILTER='OpenMP/for_codegen.cpp' ninja check-clang \
| less -R
```
Reviewed By: probinson
Differential Revision: https://reviews.llvm.org/D53517
llvm-svn: 346272
(Relands r344930, reverted in r344935, and now hopefully fixed for
Windows.)
While this change specifically targets FileCheck, it affects any tool
using the same SourceMgr facilities.
Previously, -color was documented in FileCheck's -help output, but
-color had no effect. Now, -color obeys its documentation: it forces
colors to be used in FileCheck diagnostics even when stderr is not a
terminal.
-color is especially helpful when combined with FileCheck's -v, which
can produce a long series of diagnostics that you might wish to pipe
to a pager, such as less -R. The WithColor extensions here will also
help to clean up color usage in FileCheck's annotated dump of input,
which is proposed in D52999.
Reviewed By: JDevlieghere, zturner
Differential Revision: https://reviews.llvm.org/D53419
llvm-svn: 345202
While this change specifically targets FileCheck, it affects any tool
using the same SourceMgr facilities.
Previously, -color was documented in FileCheck's -help output, but
-color had no effect. Now, -color obeys its documentation: it forces
colors to be used in FileCheck diagnostics even when stderr is not a
terminal.
-color is especially helpful when combined with FileCheck's -v, which
can produce a long series of diagnostics that you might wish to pipe
to a pager, such as less -R. The WithColor extensions here will also
help to clean up color usage in FileCheck's annotated dump of input,
which is proposed in D52999.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D53419
llvm-svn: 344930
A DAG-NOT-DAG is a CHECK-DAG group, X, followed by a CHECK-NOT group,
N, followed by a CHECK-DAG group, Y. Let y be the initial directive
of Y. This patch makes the following changes to the behavior:
1. Directives in N can no longer match within part of Y's match
range just because y happens not to be the earliest match from
Y. Specifically, this patch withdraws N's search range end
from y's match range start to Y's match range start.
2. y can no longer match within X's match range, where a y match
produced a reordering complaint, which is thus no longer
possible. Specifically, this patch withdraws y's search range
start from X's permitted range start to X's match range end,
which was already the search range start for other members of
Y.
Both of these changes can only increase the number of test passes: #1
constrains the ability of CHECK-NOTs to match, and #2 expands the
ability of CHECK-DAGs to match without complaints.
These changes are based on discussions at:
<http://lists.llvm.org/pipermail/llvm-dev/2018-May/123550.html>
<https://reviews.llvm.org/D47106>
which conclude that:
1. These changes simplify the FileCheck conceptual model. First,
it makes search ranges for DAG-NOT-DAG more consistent with
other cases. Second, it was confusing that y was treated
differently from the rest of Y.
2. These changes add theoretical use cases for DAG-NOT-DAG that
had no obvious means to be expressed otherwise. We can justify
the first half of this assertion with the observation that
these changes can only increase the number of test passes.
3. Reordering detection for DAG-NOT-DAG had no obvious real
benefit.
We don't have evidence from real uses cases to help us debate
conclusions #2 and #3, but #1 at least seems intuitive.
Reviewed By: probinson
Differential Revision: https://reviews.llvm.org/D48986
llvm-svn: 337605
-v prints all directive pattern matches.
-vv additionally prints info that might be noise to users but that can
be helpful to FileCheck developers.
To maximize code reuse and to make diagnostics more consistent, this
patch also adjusts and extends some of the existing diagnostics.
CHECK-NOT failures now report variables uses. Many more diagnostics
now report the check prefix and kind of directive.
Reviewed By: probinson
Differential Revision: https://reviews.llvm.org/D47114
llvm-svn: 336967
That is, make CHECK-DAG skip matches that overlap the matches of any
preceding consecutive CHECK-DAG directives. This change makes
CHECK-DAG more consistent with other directives, and there is evidence
it makes CHECK-DAG more intuitive and less error-prone. See the RFC
discussion starting at:
http://lists.llvm.org/pipermail/llvm-dev/2018-May/123010.html
Moreover, this behavior enables CHECK-DAG groups for unordered,
non-unique strings or patterns. For example, it is useful for
verifying output or logs from a parallel program, such as the OpenMP
runtime.
This patch also implements the command-line option
-allow-deprecated-dag-overlap, which reverts CHECK-DAG to the old
overlapping behavior. This option should not be used in new tests.
It is meant only for the existing tests that are broken by this change
and that need time to update.
See the following bugzilla issue for tracking of such tests:
https://bugs.llvm.org/show_bug.cgi?id=37532
Patches to add -allow-deprecated-dag-overlap to those tests will
follow immediately.
Reviewed By: probinson
Differential Revision: https://reviews.llvm.org/D47106
llvm-svn: 336847
That is, make CHECK-DAG skip matches that overlap the matches of any
preceding consecutive CHECK-DAG directives. This change makes
CHECK-DAG more consistent with other directives, and there is evidence
it makes CHECK-DAG more intuitive and less error-prone. See the RFC
discussion starting at:
http://lists.llvm.org/pipermail/llvm-dev/2018-May/123010.html
Moreover, this behavior enables CHECK-DAG groups for unordered,
non-unique strings or patterns. For example, it is useful for
verifying output or logs from a parallel program, such as the OpenMP
runtime.
This patch also implements the command-line option
-allow-deprecated-dag-overlap, which reverts CHECK-DAG to the old
overlapping behavior. This option should not be used in new tests.
It is meant only for the existing tests that are broken by this change
and that need time to update.
See the following bugzilla issue for tracking of such tests:
https://bugs.llvm.org/show_bug.cgi?id=37532
Patches to add -allow-deprecated-dag-overlap to those tests will
follow immediately.
Reviewed By: probinson
Differential Revision: https://reviews.llvm.org/D47106
llvm-svn: 336830
Prior to this change, there was no clean way of getting FileCheck to
check that a line is completely empty. The expected way of using
"CHECK: {{^$}}" does not work because the '^' matches the end of the
previous match (this behaviour may be desirable in certain instances).
For the same reason, "CHECK-NEXT: {{^$}}" will fail when the previous
match was at the end of the line, as the pattern will match there.
Using the recommended [[:space:]] to match an explicit new line could
also match a space, and thus is not always desired. Literal '\n'
matches also do not work. A workaround was suggested in the review, but
it is a little clunky.
This change adds a new directive that behaves the same as CHECK-NEXT,
except that it only matches against empty lines (nothing, not even
whitespace, is allowed). As with CHECK-NEXT, it will fail if more than
one newline occurs before the next blank line. Example usage:
; test.txt
foo
bar
; CHECK: foo
; CHECK-EMPTY:
; CHECK-NEXT: bar
Differential Revision: https://reviews.llvm.org/D28896
Reviewed by: probinson
llvm-svn: 335613
FileCheck tool crashes when trying to parse --check-prefix argument if there is no any
data after it.
For example test like following would crash if there are no symbols and no EOL mark after `boom`:
# REQUIRES: x86
# RUN: <skipped few lines>
# RUN: llvm-readobj -t %t | FileCheck %s --check-prefix=boom
Patch fixes the issue.
Differential revision: https://reviews.llvm.org/D42057
llvm-svn: 322536
Summary:
This makes it very easy to test files that only differ in a constant
value somewhere in the test case.
Reviewers: jlebar, hfinkel, chandlerc, probinson
Reviewed By: probinson
Subscribers: probinson, llvm-commits
Differential Revision: https://reviews.llvm.org/D39629
llvm-svn: 317572
If there's enough data in fron of it the skipped region would just
become arbitrarily large, and we scan for the CHECK-NOT everywhere.
llvm-svn: 304900
If `--enable-var-scope` is in effect, variables with names that
start with `$` are considered to be global. All other variables are
local. All local variables get undefined at the beginning of each
CHECK-LABEL block. Global variables are not affected by CHECK-LABEL.
This makes it easier to ensure that individual tests are not affected
by variables set in preceding tests.
Differential Revision: https://reviews.llvm.org/D30749
llvm-svn: 297396
I unfortunately neglected to add it in r260540, but it has been
sitting in my working dir ever since. D'oh.
Modified to work with r290069, which made the CHECK patterns
themselves whitespace-sensitive as well, and remove the test added
then, as this tests both strict and non-strict modes.
llvm-svn: 291499
Thomas Preud'homme