Commit Graph

88 Commits

Author SHA1 Message Date
Roman Lebedev 2b5a16420f
UpdateTestChecks: fix handling of UTC with spaces
We can't just split by space, that's not going to give us the same
argv we'd have gotten from the shell, it could be in a string,
we must actually parse that as argv.
2022-03-07 20:25:23 +03:00
Roman Lebedev df6c26fd34
update_analyze_test_checks.py: fix --filter handling
In particular, after filtering the check lines can't necessarily
use `-NEXT`, they may not be one directly after another.
2022-03-07 15:41:13 +03:00
Sebastian Neubauer 473efae3a1 [UpdateTestChecks] Don't skip first line with --filter
body_start was never used, resulting in the first filtered line to be
skipped.
Fixes the --filter option introduced in D117694.

Differential Revision: https://reviews.llvm.org/D119704
2022-03-03 17:27:14 +01:00
Yatao Wang 8565b6f9f2 [UpdateLLCTestChecks] Add support for isel debug output in update_llc_test_checks.py
Add a check on run lines to pick up isel options in llc commands and allow
generating check lines of isel final output other than assembly. If llc command
line contains -debug-only=isel, update_llc_test_checks.py will try to scrub isel
output, otherwise, the script will fall back on default behaviour, which is try to
scrub assembly output instead.

The motivation of this change is to allow usage of update_llc_test_checks.py to
autogenerate checks of instruction selection results. In this way, we can detect
errors at an earlier stage before the compilation goes all the way to assembly.
It is an example of having some transparency for the stages between IR and
assembly. These generated tests are almost like "unit tests" of isel stage.

This patch only implements the initial change to differentiate isel output from
assembly output for Lanai. Other targets will not be supported for isel check
generation at the moment. Although adding support for it will only require
implementing the function regex and scrubber for corresponding targets.

The Lanai implementation was chosen mainly for the simplicity of demonstrating
the difference between isel checks and asm checks.

This patch also do not include the implementation of function prefix, which is
required for the generated isel checks to pass. I will put up a follow up revision
for the function prefix change to complete isel support.

Reviewed By: Flakebi

Differential Revision: https://reviews.llvm.org/D119368
2022-03-01 10:55:53 +01:00
David Greene ecd46edd61 [UpdateTestChecks] Re-add --filter and --filter-out options
Re-add filtering options with fixes for failed tests.  We were not passing the
is_filtered argument in all check generator calls in update_cc_test_checks.py

Enhance the various update_*_test_checks.py tools to allow filtering the tool
output with regular expressions.  The --filter option will emit only tool output
lines matching the given regular expression while the --filter-out option will
emit only tools output lines not matching the given regular expression.  Filters
are applied in order of appearance on the command line (or in UTC_ARGS) and the
first matching filter terminates the search.

This allows test authors to create more focused tests by removing irrelevant
tool output and checking only the pieces of output necessary to test the desired
functionality.

Differential Revision: https://reviews.llvm.org/D117694
2022-01-31 13:11:40 -08:00
David Greene 7e32d2b21a Revert "[UpdateTestChecks] Add --filter and --filter-out options"
Broke some update-test-checks tests.  Reverting while developing a fix.

This reverts commit 030f71698d.
2022-01-28 17:06:51 -08:00
David Greene 030f71698d [UpdateTestChecks] Add --filter and --filter-out options
Enhance the various update_*_test_checks.py tools to allow filtering the tool
output with regular expressions.  The --filter option will emit only tool output
lines matching the given regular expression while the --filter-out option will
emit only tools output lines not matching the given regular expression.  Filters
are applied in order of appearance on the command line (or in UTC_ARGS) and the
first matching filter terminates the search.

This allows test authors to create more focused tests by removing irrelevant
tool output and checking only the pieces of output necessary to test the desired
functionality.

Differential Revision: https://reviews.llvm.org/D117694
2022-01-28 14:08:07 -08:00
Jay Foad de37912f00 [update_test_checks] Fix option name in warning message 2022-01-28 14:45:47 +00:00
Philip Reames 180625fcb4 Add a breadcrumb comment to make debugging a user error when using ./utils/update_analyze_test_checks.py easier.
We won't talk about how long it took me to figure this out.  The difference in batch vs interactive on the "did you mean to print bitcode" message made this confusing.
2021-11-18 09:07:19 -08:00
Alex Richardson a28177035b [UpdateTestChecks][NFC] Drop a python2 workaround 2021-09-28 17:57:36 +01:00
Sebastian Neubauer ecd5145c27 [Utils] Replace llc with cat for tests
Make the update_llc_test_checks script test independant of llc behavior
by using cat with static files to simulate llc output.

This allows changing llc without breaking the script test case.

The update script is executed in a temporary directory, so the
llc-generated assembly files are copied there. %T is deprecated, but it
allows copying a file with a predictable filename.

Differential Revision: https://reviews.llvm.org/D110143
2021-09-22 10:10:35 +02:00
Sebastian Neubauer 4864893127 [Utils] Do not remove comments in llc test script
When checking if two prefixes can be merged for a function,
update_llc_test_checks.py removed IR comments before comparing
llc outputs of different RUN lines.
This means, if one RUN line emited lines starting with ';' and another
RUN line emited the same lines except the ones starting with ';', both
RUNs would be merged (if they share a prefix).

However, CHECK-NEXT lines check the comments, otherwise they fail, so
the script should not merge RUNs if they contain different comments.

Differential Revision: https://reviews.llvm.org/D101312
2021-07-29 13:03:05 +02:00
Joel E. Denny 5b0a948a81 [UpdateCCTestChecks] Implement --global-hex-value-regex
For example, in OpenMP offload codegen tests, global variables like
`.offload_maptypes*` are much easier to read in hex.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D104743
2021-07-20 11:23:20 -04:00
Joel E. Denny 2f5b2ea6cd [UpdateCCTestChecks] Implement --global-value-regex
`--check-globals` activates checks for all global values, and
`--global-value-regex` filters them.  For example, I'd like to use it
in OpenMP offload codegen tests to check only global variables like
`.offload_maptypes*`.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D104742
2021-07-20 11:23:20 -04:00
Joel E. Denny 9eaf0d120d [UpdateCCTestChecks] Support --check-globals
This option is already supported by update_test_checks.py, but it can
also be useful in update_cc_test_checks.py.  For example, I'd like to
use it in OpenMP offload codegen tests to check global variables like
`.offload_maptypes*`.

Reviewed By: jdoerfert, arichardson, ggeorgakoudis

Differential Revision: https://reviews.llvm.org/D104714
2021-06-25 13:17:56 -04:00
Joel E. Denny 38b7b1d4a2 [UpdateCCTestChecks][NFC] Permit other comments in common.py
Some parts of common.py already permit comment styles besides `;`.
Handle the remaining cases.  Specifically, a future patch will extend
update_cc_test_checks.py to call add_global_checks.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D104713
2021-06-23 11:12:42 -04:00
Joel E. Denny 2bfe0536e5 [UpdateCCTestChecks] Fix --replace-value-regex across RUN lines
Without this patch, llvm/utils/update_cc_test_checks.py fails to
perform `--replace-value-regex` replacements when two RUN lines
produce the same output and use the same single FileCheck prefix.  The
problem is that replacements in a RUN line's output are not performed
until after comparing against previous RUN lines' output, where
replacements have already been performed.  This patch fixes that.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D104566
2021-06-21 17:01:17 -04:00
Roman Lebedev b1f55c33d4
[UpdateTestUtils] Print test filename when complaining about conflicting prefix
Now that FileCheck eagerly complains when prefixes are unused,
the update script does the same, and  is becoming very common
to need to drop some prefixes, yet figuring out the file
it complains about isn't obvious unless it actually tells us.
2021-06-20 14:12:39 +03:00
Giorgis Georgakoudis 78a7d8c4dd [Utils][NFC] Rename replace-function-regex in update_cc_test_checks
This patch renames the replace-function-regex to replace-value-regex to indicate that the existing regex replacement functionality can replace any IR value besides functions.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D101934
2021-05-05 14:19:30 -07:00
Giorgis Georgakoudis 404fa9a6cf [Utils] Add prof metadata to matched unnamed values
Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D101742
2021-05-03 15:15:34 -07:00
Alex Richardson 9692811b26 [update_(llc_)test_checks.py] Support pre-processing commands
This has been rather useful in our downstream CHERI target where we want
to run tests both with addrspace(0) and addrspace(200) pointers.
With this patch we can prefix the opt command with
`sed -e 's/addrspace(200)/addrspace(0)/g' -e 's/-A200-P200-G200//g'` to
test both cases using the same IR input.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D95137
2021-04-28 12:19:19 +01:00
Giorgis Georgakoudis 8bc2c662d9 [Utils] Add prefix parameter in update test checks to avoid FileCheck conflicts
IR values convert to check prefix FileCheck variables for IR checks. For example, nameless values, e.g., %0, convert to check prefix TMP FileCheck variables, e.g., [[TMP0:%.*]]. This check prefix may clash with named values that have the same name and that causes auto-generated tests to fail. Currently a warning is emitted to change the names of the IR values but this is not always possible, if for example they are generated by clang. Manual intervention to fix the FileCheck variable names is too tedious. This patch add a parameter to prefix conflicting FileCheck variable names with a user-provided string to automate the process.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D99415
2021-03-26 11:49:42 -07:00
Giorgis Georgakoudis 7ad55a3df5 [Utils][NFC] Fix regex substitution for update test checks
Relates to: https://reviews.llvm.org/D97107
2021-03-24 17:47:33 -07:00
Thomas Preud'homme 7dd90da79f [UpdateTestChecks] Fix typo & copy/paste in comments 2021-03-24 16:11:36 +00:00
Giorgis Georgakoudis 1ce846be04 Replace func name with regex for update test scripts
The patch adds an argument to update test scripts, such as update_cc_test_checks, for replacing a function name matching a regex. This functionality is needed to match generated function signatures that include file hashes. Example:

The function signature for the following function:

`__omp_offloading_50_b84c41e__Z9ftemplateIiET_i_l30_worker`

with `--replace-function-regex "__omp_offloading_[0-9]+_[a-z0-9]+_(.*)"` will become:

`CHECK-LABEL: @{{__omp_offloading_[0-9]+_[a-z0-9]+__Z9ftemplateIiET_i_l30_worker}}(`

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D97107
2021-03-12 17:37:09 -08:00
Giorgis Georgakoudis 9f9a4dfda7 Revert "Replace func name with regex for update test scripts"
This reverts commit 5eaf70afb5.
2021-03-12 17:20:00 -08:00
Giorgis Georgakoudis 5eaf70afb5 Replace func name with regex for update test scripts
The patch adds an argument to update test scripts, such as update_cc_test_checks, for replacing a function name matching a regex. This functionality is needed to match generated function signatures that include file hashes. Example:

The function signature for the following function:

`__omp_offloading_50_b84c41e__Z9ftemplateIiET_i_l30_worker`

with `--replace-function-regex "__omp_offloading_[0-9]+_[a-z0-9]+_(.*)"` will become:

`CHECK-LABEL: @{{__omp_offloading_[0-9]+_[a-z0-9]+__Z9ftemplateIiET_i_l30_worker}}(`

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D97107
2021-03-12 17:00:42 -08:00
Johannes Doerfert cd1bd6e587 [Utils] Check for more global information in update_test_checks
This allows to check for various globals (metadata/attributes/...) and
also resolves problems with globals (metadata/attributes/...) being
reused across different prefixes.

Reviewed By: sstefan1

Differential Revision: https://reviews.llvm.org/D94741
2021-03-11 23:31:16 -06:00
Giorgis Georgakoudis ecf68972fd Revert "Replace func name with regex in update_cc_test_checks"
This reverts commit bf58d6a1f9.

Breaks tests, fix
2021-03-10 15:05:35 -08:00
Giorgis Georgakoudis bf58d6a1f9 Replace func name with regex in update_cc_test_checks
The patch adds an argument to update_cc_test_checks for replacing a function name matching a regex. This functionality is needed to match generated function signatures that include file hashes. Example:

The function signature for the following function:

`__omp_offloading_50_b84c41e__Z9ftemplateIiET_i_l30_worker`

with `--replace-function-regex "__omp_offloading_[0-9]+_[a-z0-9]+_(.*)"` will become:

`CHECK-LABEL: @{{__omp_offloading_[0-9]+_[a-z0-9]+__Z9ftemplateIiET_i_l30_worker}}(`

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D97107
2021-03-10 12:57:35 -08:00
Mircea Trofin 7549524ac5 [NFC] Remove spurious ';' on return line in python code 2021-02-15 08:42:02 -08:00
Mircea Trofin f31ea86c80 Revert "[Utils] Add a switch controlling prefix warnings in UpdateTestChecks"
This reverts commit 87f8a08ce3.
2021-02-08 11:21:56 -08:00
Mircea Trofin 87f8a08ce3 [Utils] Add a switch controlling prefix warnings in UpdateTestChecks
The switch controls both unused prefix warnings, and warnings about
functions which differ under different runs for a prefix, and, thus, end
up not having asserts for that prefix.

(If the latter case spans to all functions, then the former case kicks
in)

The switch is on by default, and can be disabled.

Differential Revision: https://reviews.llvm.org/D95829
2021-02-01 18:04:18 -08:00
Mircea Trofin 93fd52329f [NFC][utils] Factor remaining APIs under FunctionTestBuilder
Finishing the refactoring started in D93413.

Differential Revision: https://reviews.llvm.org/D93506
2020-12-17 22:13:14 -08:00
Mircea Trofin ed1e565aaf [NFC] factor update test function test builder as a class
This allows us to have shared logic over multiple test runs, e.g. do we
have unused prefixes, or which function bodies have conflicting outputs
for a prefix appearing in different RUN lines.

This patch is just wrapping existing functionality, and replacing its uses.
A subsequent patch would then fold the current functionality into the newly
introduced class.

Differential Revision: https://reviews.llvm.org/D93413
2020-12-16 21:12:06 -08:00
Mircea Trofin 380e1d918c [utils] The func_dict for a prefix may just be empty
Follow up from D92965 - since we try to find failed prefixes
after each RUN line, it's possible the whole list of functions for a
prefix be non-existent, which is fine - this happens when none of the
RUN lines seen so far used the prefix.
2020-12-15 08:48:37 -08:00
Mircea Trofin e2dc306b1a [utils] Fix UpdateTestChecks case where 2 runs differ for last label
Two RUN lines produce outputs that, each, have some common parts and
some different parts. The common parts are checked under label A. The
differing parts are associated to a function and checked under labels B
and C, respectivelly.
When build_function_body_dictionary is called for the first RUN line, it
will attribute the function body to labels A and C. When the second RUN
is passed to build_function_body_dictionary, it sees that the function
body under A is different from what it has. If in this second RUN line,
A were at the end of the prefixes list, A's body is still kept
associated with the first run's function.

When we output the function body (i.e. add_checks), we stop after
emitting for the first prefix matching that function. So we end up with
the wrong function body (first RUN's A-association).

There is no reason to special-case the last label in the prefixes list,
and the fix is to always clear a label association if we find a RUN line
where the body is different.

Differential Revision: https://reviews.llvm.org/D93078
2020-12-15 07:16:54 -08:00
Nico Weber 7502040ed2 clang: Make changes in 7c8bb409f3 py2.7-compatible 2020-09-18 16:15:37 -04:00
David Greene 7c8bb409f3 [UpdateCCTestChecks] Include generated functions if asked
Add the --include-generated-funcs option to update_cc_test_checks.py so that any
functions created by the compiler that don't exist in the source will also be
checked.

We need to maintain the output order of generated function checks so that
CHECK-LABEL works properly.  To do so, maintain a list of functions output for
each prefix in the order they are output.  Use this list to output checks for
generated functions in the proper order.

Differential Revision: https://reviews.llvm.org/D83004
2020-09-18 06:34:59 -05:00
David Greene ce0eb81c72 [UpdateTestChecks] Allow $ in function names
Some compilers generation functions with '$' in their names, so recognize those
functions.

This also requires recognizing function names inside quotes in some contexts in
order to escape certain characters.

Differential Revision: https://reviews.llvm.org/D82995
2020-09-16 14:34:18 -05:00
Johannes Doerfert dbde3969ba [UpdateTestChecks][NFC] Fix spelling 2020-09-16 13:37:08 -05:00
sstefan1 b5dd2cd27f [UpdateTestChecks] Don't skip attributes when comparing functions 2020-08-29 15:37:36 +02:00
sstefan1 7128e64706 [UpdateTestChecks] include { in function signature check line
After D85099, if we have attribute group in the function signature that hasn't
been seen before, and later a callsite with the same attribute group, filecheck will evaluate
the first attribute group to for example '#0 {'. We now include { in the args_and_sig group to avoid this.

Differential Revision: https://reviews.llvm.org/D86769
2020-08-28 23:44:03 +02:00
Sam Parker af6140fe8a [UpdatesTestChecks] Fix typo in common.py
global_vars_see_dict -> global_vars_seen_dict
2020-08-25 11:13:33 +01:00
Johannes Doerfert 07448c5504 [UpdateTestChecks][FIX] Python 2.7 compatibility and use right prefix 2020-08-12 11:58:08 -05:00
Johannes Doerfert 97ce7fd89f [UpdateTestChecks] Match unnamed values like "@[0-9]+" and "![0-9]+"
With this patch we will match most *uses* of "temporary" named things in
the IR via regular expressions, not their name at creation time. The new
"values" we match are:
  - "unnamed" globals: `@[0-9]+`
  - debug metadata: `!dbg ![0-9]+`
  - loop metadata: `!loop ![0-9]+`
  - tbaa metadata: `!tbaa ![0-9]+`
  - range metadata: `!range ![0-9]+`
  - generic metadata: `metadata ![0-9]+`
  - attributes groups: `#[0-9]`

We still don't match the declarations but that can be done later. This
patch can introduce churn when existing check lines contain the old
hardcoded versions of the above "values". We can add a flag to opt-out,
or opt-in, if necessary.

Reviewed By: arichardson, MaskRay

Differential Revision: https://reviews.llvm.org/D85099
2020-08-12 01:04:16 -05:00
Alex Richardson d1007478f1 Fix update_cc_test_checks.py --llvm-bin after D78478
Not passing --clang would result in a python exception after this change:
(TypeError: expected str, bytes or os.PathLike object, not NoneType)
because the --clang argument default was only being populated in the
initial argument parsing pass but not later on.
Fix this by adding an argparse callback to set the default values.

Reviewed By: vitalybuka, MaskRay

Differential Revision: https://reviews.llvm.org/D84511
2020-08-03 11:18:01 +01:00
sstefan1 cf11050696 [Utils][Fix] remove unnecessary ; at the end 2020-07-19 20:48:28 +02:00
sstefan1 937bad3594 [Utils] Check function attributes in update_test_checks
Summary:
This introduces new flag to the update_test_checks and
update_cc_test_checks that allows for function attributes
to be checked in a check-line. If the flag is not set,
the behavior should remain the same.

Reviewers: jdoerfert

Subscribers: arichardson, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D83629
2020-07-19 20:07:24 +02:00
Alex Richardson aae413462f [UpdateTestChecks] Move more update_test_checks.py logic to common.py
I intend to reuse this to add UTC_ARGS support for update_llc_test_checks.py
and update_cc_test_checks.py in D78478.

Reviewed By: jdoerfert
Differential Revision: https://reviews.llvm.org/D78618
2020-07-08 10:59:28 +01:00