Commit Graph

153 Commits

Author SHA1 Message Date
Fangrui Song a36ddf0aa9 Migrate function attribute "no-frame-pointer-elim"="false" to "frame-pointer"="none" as cleanups after D56351 2019-12-24 16:27:51 -08:00
Michael Kruse 7be6ec5fa2 [GPGPU] Fix regression test after 395124.
Commit 395124 "NVPTX: Don't insert an extra empty line at the end of the last section"
changed the length of the kernel payload. Update the regression test to the new binary size.
2019-11-13 06:20:17 +00:00
Michael Kruse cc0f0582c8 [Polly-ACC] Fix test after IR-printer change.
After r367755, even unnamed parameters are printed in IR dumps. Change
the test to expect te additional %0 in the line.

llvm-svn: 368763
2019-08-13 22:42:08 +00:00
James Y Knight 693d39dd12 Remove irrelevant references to legacy git repositories from
compiler identification lines in test-cases.

(Doing so only because it's then easier to search for references which
are actually important and need fixing.)

llvm-svn: 351200
2019-01-15 16:18:52 +00:00
Philip Pfaffe cb8a82929c Fix for r336080: Missing colon in REQUIRES line
llvm-svn: 336083
2018-07-02 08:36:49 +00:00
Philip Pfaffe d71493cb06 [polly-acc] change cl_get_* return types to 32/64bit
Summary:
This patch changes the return types for ocl_get_* functions during SPIR code generation. Because these functions return size_t types, the return type needs to be changed to the actual size of size_t on the device.

Based on work by Michal Babej and Pekka Jääskeläinen

Patch by: Alain Denzler

Reviewers: grosser, philip.pfaffe, bollu

Reviewed By: grosser, philip.pfaffe

Subscribers: nemanjai, kbarton, llvm-commits

Differential Revision: https://reviews.llvm.org/D48774

llvm-svn: 336080
2018-07-02 07:40:47 +00:00
Philip Pfaffe 7cc4300dac [Acc] Followup for r333105: Fix one additional testcase
llvm-svn: 333168
2018-05-24 10:18:09 +00:00
Philip Pfaffe 356d60683b [Acc] Enable legacy stmt granularity in remaining failing testcases
The default statement granularity changed in a recent change by Micheal. To
avoid forwad-porting the testcases, enable the legacy behaviour again in these tests.

llvm-svn: 333105
2018-05-23 17:46:10 +00:00
Philip Pfaffe 2e171b52ee [Acc] Update testcases for minor changes in the PPCG mapper and
statement naming

- A recent ppcg/isl update caused the grid/block size upper bounds to
deviate by one from the oracle. This is not an effect that's visible at
runtime.
- Statement naming changed in polly. Update the testcases.

llvm-svn: 333090
2018-05-23 14:56:57 +00:00
Tobias Grosser 6bbca36414 Adjust to debug info metadata format change.
Rename variable to retainedNodes. This unbreaks the Polly builds.

llvm-svn: 331960
2018-05-10 07:09:10 +00:00
Philip Pfaffe f6f8b25e58 [NFC] In GPGPU testcases, replace numeric registers in CHECK directives.
Using numeric registers is flaky, since as soon as one additional
instruction is generated by us, all the tests need to be adapted.

llvm-svn: 319544
2017-12-01 14:16:39 +00:00
Philip Pfaffe 9b1d1e6ae7 Fix two testcases. NFC intended.
Add missing %loadPolly directive to support out of tree builds. One of
the changes is somewhat bigger, because the directive turns on LLVM
names, and the testcase deosn't use those.

llvm-svn: 316870
2017-10-29 21:00:48 +00:00
Tobias Grosser c52b71db15 [GPGPU] Make sure escaping invariant load hoisted scalars are preserved
We make sure that the final reload of an invariant scalar memory access uses the
same stack slot into which the invariant memory access was stored originally.
Earlier, this was broken as we introduce a new stack slot aside of the preload
stack slot, which remained uninitialized and caused our escaping loads to
contain garbage. This happened due to us clearing the pre-populated values
in EscapeMap after kernel code generation. We address this issue by preserving
the original host values and restoring them after kernel code generation.
EscapeMap is not expected to be used during kernel code generation, hence we
clear it during kernel generation to make sure that any unintended uses are
noticed.

llvm-svn: 314894
2017-10-04 10:24:23 +00:00
Tobias Grosser d215e684b3 Add missing REQUIRES line
llvm-svn: 314625
2017-10-01 13:14:40 +00:00
Tobias Grosser 2fb847fbf6 [GPGPU] Set Polly's RTC to false in case invariant load hoisting fails
This matches the behavior we already have in lib/Codegen/CodeGeneration.cpp and
makes sure that we fall back to the original code. It seems when invariant load
hoisting was introduced to the GPGPU backend we missed to reset the RTC flag,
such that kernels where invariant load hoisting failed executed the 'optimized'
SCoP, which however is set to a simple 'unreachable'. Unsurprisingly, this
results in hard to debug issues that are a lot of fun to debug.

llvm-svn: 314624
2017-10-01 12:39:14 +00:00
Siddharth Bhat 3928e3f50a [ISLNodeBuilder] Materialize Fortran array sizes of arrays without memory accesses.
In Polly, we specifically add a paramter to represent the outermost dimension
 size of fortran arrays. We do this because this information is statically
 available from the fortran metadata generated by dragonegg.
 However, we were only materializing these parameters (meaning, creating an
 llvm::Value to back the isl_id) from *memory accesses*. This is wrong,
 we should materialize parameters from *scop array info*.

 It is wrong because if there is a case where we detect 2 fortran arrays,
 but only one of them is accessed, we may not materialize the other array's
 dimensions at all.

 This is incorrect. We fix this by looping over all
 `polly::ScopArrayInfo` in a scop, rather that just all `polly::MemoryAccess`.

 Differential Revision: https://reviews.llvm.org/D37379

llvm-svn: 312350
2017-09-01 18:55:43 +00:00
Siddharth Bhat 56572c6a5e [PPCGCodeGen] Convert intrinsics to libdevice functions whenever possible.
This is useful when we face certain intrinsics such as `llvm.exp.*`
which cannot be lowered by the NVPTX backend while other intrinsics can.

So, we would need to keep blacklists of intrinsics that cannot be
handled by the NVPTX backend. It is much simpler to try and promote
all intrinsics to libdevice versions.

This patch makes function/intrinsic very uniform, and will always try to use
a libdevice version if it exists.

Differential Revision: https://reviews.llvm.org/D37056

llvm-svn: 312239
2017-08-31 13:03:37 +00:00
Tobias Grosser 6683c81af8 test/GPGPU/invalid-kernel-assert-verifymodule.ll also requires assertions
llvm-svn: 311423
2017-08-22 03:12:29 +00:00
Tobias Grosser b09bd74da8 [GPGPU] Add llvm.powi to the libdevice supported functions
These intrinsics are used in COSMO.

llvm-svn: 311324
2017-08-21 09:52:08 +00:00
Tobias Grosser 5170b6627a [GPGPU] Add log / logf to the libdevice supported functions
These two functions are used in COSMO

llvm-svn: 311322
2017-08-21 09:00:31 +00:00
Tobias Grosser e32498c9c3 Revert "[GPGPU] Simplify PPCGSCop to reduce compile time [NFC]"
We still see some issues with parameter space mismatches. Revert this to get
a clean baseline. We will recommit after these issues have been resolved.

This reverts commit 0e360a14194f722ded7aa2bc9d4be2ed2efeeb49.

llvm-svn: 311268
2017-08-19 23:49:26 +00:00
Tobias Grosser ecb94a0392 [GPGPU] Correctly initialize array order and fixed_element information
Summary:
This information is necessary for PPCG to perform correct life range reordering.
With these changes applied we can live-range reorder some of the important
kernels in COSMO.

We also update and rename one test case, which previously could not be optimized
and now is optimized thanks to live-range reordering. To preserve test coverage
we add a new test case scalar-writes-in-scop-requires-abort.ll, which exercises
our automatic abort in case of scalar writes in the kernel.

Reviewers: Meinersbur, bollu, singam-sanjay

Subscribers: nemanjai, pollydev, llvm-commits, kbarton

Tags: #polly

Differential Revision: https://reviews.llvm.org/D36929

llvm-svn: 311259
2017-08-19 20:21:22 +00:00
Philipp Schaad 50139f0f38 [PPCG] Only add Kernel argument sizes for OpenCL, not CUDA runtime
Kernel argument sizes now only get appended to the kernel launch parameter list if the OpenCL runtime is selected, not if CUDA runtime is chosen.

Differential revision: D36925

llvm-svn: 311248
2017-08-19 17:04:57 +00:00
Tobias Grosser 43df2020e7 [GPGPU] Collect parameter dimension used in MemoryAccesses
When using -polly-ignore-integer-wrapping and -polly-acc-codegen-managed-memory
we add parameter dimensions lazily to the domains, which results in PPCG not
including parameter dimensions that are only used in memory accesses in the
kernel space. To make sure these parameters are still passed to the kernel, we
collect these parameter dimensions and align the kernel's parameter space
before code-generating it.

llvm-svn: 311239
2017-08-19 12:58:28 +00:00
Tobias Grosser ec02acfb98 [GPGPU] Simplify PPCGSCop to reduce compile time [NFC]
Summary:
Drop unused parameter dimensions to reduce the size of the sets we are working
with. Especially the computed dependences tend to accumulate a lot of parameters
that are present in the input memory accesses, but often not necessary to
express the actual dependences. As isl represents maps and sets with dense
matrices, reducing the dimensionality of isl sets commonly reduces code
generation performance.

This reduces compile time from 17 to 11 seconds for our test case. While this is
not impressive, this patch helped me to identify the previous two performance
improvements and additionally also increases readability of the isl data
structures we use.

Reviewers: Meinersbur, bollu, singam-sanjay

Reviewed By: bollu

Subscribers: nemanjai, pollydev, llvm-commits, kbarton

Tags: #polly

Differential Revision: https://reviews.llvm.org/D36869

llvm-svn: 311161
2017-08-18 13:38:12 +00:00
Siddharth Bhat a2c4112791 [ManagedMemoryRewrite] Rewrite malloc, free correctly inside `Constant`s.
Reuse the machinery built for replacing global arrays to replace malloc/free as
well. Example replacement that was missed earlier:

```
call void \
    bitcast (void (i8*)* @free to void (%custom_type*)*) (%custom_type* %13)
```

- Since the `bitcast` is a `ConstantExpr`, `replaceAllUsesWith` would miss
this. We don't miss this anymore.

Differential Revision: https://reviews.llvm.org/D36825

llvm-svn: 311121
2017-08-17 20:26:38 +00:00
Tobias Grosser abc5416be1 [GPGPU] Make test case independent of LLVM names
In release builds LLVM may not pass along LLVM names consistently. We make the
test cases independent of the LLVM-IR names to avoid spurious test case
failures.

llvm-svn: 311118
2017-08-17 20:09:02 +00:00
Siddharth Bhat 8a2c07f6d4 [ManagedMemoryRewrite] Learn how to rewrite global arrays, allocas.
- If we have global arrays, we would like to rewrite them to global
  pointers which are allocated using `cudaMallocManaged`.

- If we have allocas in a function, we would like to rewrite them to
  heap-allocations with `cudaMallocManaged` and `cudaFree`.

- With these rewrite mechanisms, we can offload _any_ function to the
  GPU with no code rewrite whatsover.

Differential Revision: https://reviews.llvm.org/D36516

llvm-svn: 311080
2017-08-17 11:22:52 +00:00
Tobias Grosser 5502eb0986 Add missing 'REQUIRES' line
llvm-svn: 311046
2017-08-16 22:02:03 +00:00
Tobias Grosser e2a45f32dc [GPGPU] Also record invariant loads as kernel subtree values
Before this change kernels that used invariant loads would have resulted in
invalid PTX code.

llvm-svn: 311042
2017-08-16 21:37:53 +00:00
Philip Pfaffe e18f3f6708 Fix 310555: Require pollyacc instead of asserts
llvm-svn: 310595
2017-08-10 14:21:04 +00:00
Tobias Grosser 4db39c4829 Add missing 'REQUIRES' line
llvm-svn: 310555
2017-08-10 08:11:47 +00:00
Tobias Grosser cff9696e11 [GPGPU] Make the ast_build available to block generator
This is necessary for partial writes (as used by delicm) to work.

llvm-svn: 310553
2017-08-10 08:00:56 +00:00
Siddharth Bhat 9298ff2dee [ManagedMemoryRewrite] [Polly] Erase original malloc and free. [NFC]
We do not need to keep `malloc` and `free` around since they are
replaced by `polly_{malloc,free}Managed.`

llvm-svn: 310504
2017-08-09 18:19:46 +00:00
Siddharth Bhat 5a1f872623 [ManagedMemoryRewrite] Remove test case that was submitted by mistake. [NFC]
llvm-svn: 310473
2017-08-09 13:34:54 +00:00
Siddharth Bhat c4a4af47f3 [ManagedMemoryRewrite] Introduce a new pass to rewrite modules to use managed memory.
This pass is useful to automatically convert a codebase that uses malloc/free
to use their managed memory counterparts.

Currently, rewrite malloc and free to the `polly_{malloc,free}Managed` variants.

A future patch will teach ManagedMemoryRewrite to rewrite global arrays
as pointers to globally allocated managed memory.

Differential Revision: https://reviews.llvm.org/D36513

llvm-svn: 310471
2017-08-09 12:59:23 +00:00
Siddharth Bhat 34eeabbca3 [PPCGCodeGeneration] Compute element size in bytes for arrays correctly.
Previously, we used to compute this with `elementSizeInBits / 8`. This
would yield an element size of 0 when the array had element size < 8 in
bits.

To fix this, ask data layout what the size in bytes should be.

Differential Revision: https://reviews.llvm.org/D36459

llvm-svn: 310448
2017-08-09 08:29:16 +00:00
Siddharth Bhat 9aca1cb519 [NFC] [PPCGCodeGen] Add missing REQUIRES: pollyacc line.
llvm-svn: 310354
2017-08-08 12:26:37 +00:00
Siddharth Bhat 71dfb3eb07 [Polly] [PPCGCodeGeneration] Handle failing of invariant load hoisting gracefully.
To do this, we replicate what `CodeGeneration` does. We expose
`markNodeUnreachable` from `CodeGeneration` to `PPCGCodeGeneration`.

Differential Revision: https://reviews.llvm.org/D36457

llvm-svn: 310350
2017-08-08 12:00:59 +00:00
Tobias Grosser aabfbfa5fc Add missing 'REQUIRES: pollyacc' line
llvm-svn: 310197
2017-08-06 11:21:09 +00:00
Tobias Grosser b99c11710c [GPGPU] Make sure managed arrays are prepared at the beginning of the scop
Summary:
This resolves some "instruction does not dominate use" errors, as we used to
prepare the arrays at the location of the first kernel, which not necessarily
dominated all other kernel calls.

Reviewers: Meinersbur, bollu, singam-sanjay

Subscribers: nemanjai, pollydev, llvm-commits, kbarton

Differential Revision: https://reviews.llvm.org/D36372

llvm-svn: 310196
2017-08-06 11:10:38 +00:00
Tobias Grosser 5b307cdb8a [GPGPU] Rename all, not only the first libdevice function
llvm-svn: 310194
2017-08-06 03:04:15 +00:00
Siddharth Bhat e53c924b0f [Polly] [PPCGCodeGeneration] Deal with loops outside the Scop correctly in PPCGCodeGeneration.
A Scop with a loop outside it is not handled currently by
PPCGCodeGeneration. The test case is such that the Scop has only one inner loop
that is detected. This currently breaks codegen.

The fix is to reuse the existing mechanism in `IslNodeBuilder` within
`GPUNodeBuilder.

Differential Revision: https://reviews.llvm.org/D36290

llvm-svn: 310193
2017-08-06 02:39:05 +00:00
Tobias Grosser c1cfe0a828 Add missing REQUIRES line
llvm-svn: 309943
2017-08-03 14:46:53 +00:00
Tobias Grosser b5563c6817 Make sure that all parameter dimensions are set in schedule
Summary:
In case the option -polly-ignore-parameter-bounds is set, not all parameters
will be added to context and domains. This is useful to keep the size of the
sets and maps we work with small. Unfortunately, for AST generation it is
necessary to ensure all parameters are part of the schedule tree. Hence,
we modify the GPGPU code generation to make sure this is the case.

To obtain the necessary information we expose a new function
Scop::getFullParamSpace(). We also make a couple of functions const to be
able to make SCoP::getFullParamSpace() const.

Reviewers: Meinersbur, bollu, gareevroman, efriedma, huihuiz, sebpop, simbuerg

Subscribers: nemanjai, kbarton, pollydev, llvm-commits

Tags: #polly

Differential Revision: https://reviews.llvm.org/D36243

llvm-svn: 309939
2017-08-03 13:51:15 +00:00
Michael Kruse 291fd8074e [test] Fix test case without Polly-ACC.
llvm-svn: 309938
2017-08-03 13:44:31 +00:00
Siddharth Bhat eadf76d34a [PPCGCodeGeneration] Construct `isl_multi_pw_aff` of PPCGArray.bounds even when polly-ignore-parameter-bounds is turned on.
When we have `-polly-ignore-parameter-bounds`, `Scop::Context` does not contain
all the paramters present in the program.

The construction of the `isl_multi_pw_aff` requires all the indivisual `pw_aff`
to have the same parameter dimensions. To achieve this, we used to realign
every `pw_aff` with `Scop::Context`. However, in conjunction with
`-polly-ignore-parameter-bounds`, this is now incorrect, since `Scop::Context`
does not contain all parameters.

We set this up correctly by creating a space that has all the parameters
used by all the `isl_pw_aff`. Then, we realign all `isl_pw_aff` to this space.

llvm-svn: 309934
2017-08-03 12:09:33 +00:00
Singapuram Sanjay Srivallabh 188053af5e Remove debug metadata from copied instruction to prevent GPUModule verification failure
Summary:
**Remove debug metadata from instruction to be copied to prevent the source file's debug metadata being copied into GPUModule and eventually failing Module verification and ASM string codegeneration.**

When copying the instruction onto the Module meant for the GPU, debug metadata attached to an instruction causes all related metadata to be pulled into the Module, including the DICompileUnit, which is not listed in llvm.dbg.cu of the Module. This fails the verification of the Module and generation of the ASM string.

The only debug metadata of the instruction, the DebugLoc, is unset by this patch.

This patch reattempts https://reviews.llvm.org/D35630 by targeting only those instructions that are to end up in a Module meant for the GPU.

Reviewers: grosser, bollu

Reviewed By: grosser

Subscribers: pollydev

Tags: #polly

Differential Revision: https://reviews.llvm.org/D36161

llvm-svn: 309822
2017-08-02 15:20:07 +00:00
Siddharth Bhat 1ec9cba4e3 [NFC] Add 'REQUIRES: pollyacc' on 'test/GPGPU/invariant-load-hoisting-of-array.ll'
- Should fix broken build due to `r309681`.

llvm-svn: 309686
2017-08-01 14:52:18 +00:00
Siddharth Bhat edf9581e4c [PPCGCodeGeneration] Correct usage of llvm::Value with getLatestValue.
It is possible that the `HostPtr` that coresponds to an array could be
invariant load hoisted. Make sure we use the invariant load hoisted
value by using `IslNodeBuilder::getLatestValue`.

Differential Revision: https://reviews.llvm.org/D36001

llvm-svn: 309681
2017-08-01 14:26:39 +00:00