Commit Graph

798 Commits

Author SHA1 Message Date
Marek Olsak d2af89df10 R600/SI: Add an intrinsic for S_FLBIT_I32 / V_FFBH_I32
Required by OpenGL (ARB_gpu_shader5).

llvm-svn: 231259
2015-03-04 17:33:45 +00:00
David Blaikie a79ac14fa6 [opaque pointer type] Add textual IR support for explicit type parameter to load instruction
Essentially the same as the GEP change in r230786.

A similar migration script can be used to update test cases, though a few more
test case improvements/changes were required this time around: (r229269-r229278)

import fileinput
import sys
import re

pat = re.compile(r"((?:=|:|^)\s*load (?:atomic )?(?:volatile )?(.*?))(| addrspace\(\d+\) *)\*($| *(?:%|@|null|undef|blockaddress|getelementptr|addrspacecast|bitcast|inttoptr|\[\[[a-zA-Z]|\{\{).*$)")

for line in sys.stdin:
  sys.stdout.write(re.sub(pat, r"\1, \2\3*\4", line))

Reviewers: rafael, dexonsmith, grosser

Differential Revision: http://reviews.llvm.org/D7649

llvm-svn: 230794
2015-02-27 21:17:42 +00:00
David Blaikie 79e6c74981 [opaque pointer type] Add textual IR support for explicit type parameter to getelementptr instruction
One of several parallel first steps to remove the target type of pointers,
replacing them with a single opaque pointer type.

This adds an explicit type parameter to the gep instruction so that when the
first parameter becomes an opaque pointer type, the type to gep through is
still available to the instructions.

* This doesn't modify gep operators, only instructions (operators will be
  handled separately)

* Textual IR changes only. Bitcode (including upgrade) and changing the
  in-memory representation will be in separate changes.

* geps of vectors are transformed as:
    getelementptr <4 x float*> %x, ...
  ->getelementptr float, <4 x float*> %x, ...
  Then, once the opaque pointer type is introduced, this will ultimately look
  like:
    getelementptr float, <4 x ptr> %x
  with the unambiguous interpretation that it is a vector of pointers to float.

* address spaces remain on the pointer, not the type:
    getelementptr float addrspace(1)* %x
  ->getelementptr float, float addrspace(1)* %x
  Then, eventually:
    getelementptr float, ptr addrspace(1) %x

Importantly, the massive amount of test case churn has been automated by
same crappy python code. I had to manually update a few test cases that
wouldn't fit the script's model (r228970,r229196,r229197,r229198). The
python script just massages stdin and writes the result to stdout, I
then wrapped that in a shell script to handle replacing files, then
using the usual find+xargs to migrate all the files.

update.py:
import fileinput
import sys
import re

ibrep = re.compile(r"(^.*?[^%\w]getelementptr inbounds )(((?:<\d* x )?)(.*?)(| addrspace\(\d\)) *\*(|>)(?:$| *(?:%|@|null|undef|blockaddress|getelementptr|addrspacecast|bitcast|inttoptr|\[\[[a-zA-Z]|\{\{).*$))")
normrep = re.compile(       r"(^.*?[^%\w]getelementptr )(((?:<\d* x )?)(.*?)(| addrspace\(\d\)) *\*(|>)(?:$| *(?:%|@|null|undef|blockaddress|getelementptr|addrspacecast|bitcast|inttoptr|\[\[[a-zA-Z]|\{\{).*$))")

def conv(match, line):
  if not match:
    return line
  line = match.groups()[0]
  if len(match.groups()[5]) == 0:
    line += match.groups()[2]
  line += match.groups()[3]
  line += ", "
  line += match.groups()[1]
  line += "\n"
  return line

for line in sys.stdin:
  if line.find("getelementptr ") == line.find("getelementptr inbounds"):
    if line.find("getelementptr inbounds") != line.find("getelementptr inbounds ("):
      line = conv(re.match(ibrep, line), line)
  elif line.find("getelementptr ") != line.find("getelementptr ("):
    line = conv(re.match(normrep, line), line)
  sys.stdout.write(line)

apply.sh:
for name in "$@"
do
  python3 `dirname "$0"`/update.py < "$name" > "$name.tmp" && mv "$name.tmp" "$name"
  rm -f "$name.tmp"
done

The actual commands:
From llvm/src:
find test/ -name *.ll | xargs ./apply.sh
From llvm/src/tools/clang:
find test/ -name *.mm -o -name *.m -o -name *.cpp -o -name *.c | xargs -I '{}' ../../apply.sh "{}"
From llvm/src/tools/polly:
find test/ -name *.ll | xargs ./apply.sh

After that, check-all (with llvm, clang, clang-tools-extra, lld,
compiler-rt, and polly all checked out).

The extra 'rm' in the apply.sh script is due to a few files in clang's test
suite using interesting unicode stuff that my python script was throwing
exceptions on. None of those files needed to be migrated, so it seemed
sufficient to ignore those cases.

Reviewers: rafael, dexonsmith, grosser

Differential Revision: http://reviews.llvm.org/D7636

llvm-svn: 230786
2015-02-27 19:29:02 +00:00
Tom Stellard eb05c610b4 R600/SI: Remove M0 from DS assembly strings
This matches the assembly syntax for the proprietary compiler.

llvm-svn: 230645
2015-02-26 17:08:43 +00:00
Tom Stellard ecc419c31d R600/SI: Remove isel mubuf legalization
We legalize mubuf instructions post-instruction selection, so this
code is no longer needed.

llvm-svn: 230352
2015-02-24 17:59:19 +00:00
Matt Arsenault f07833057c R600/SI: Use v_madmk_f32
llvm-svn: 230149
2015-02-21 21:29:10 +00:00
Matt Arsenault 0325d3d27f R600/SI: Try to use v_madak_f32
This is a code size optimization when the constant
only has one use.

llvm-svn: 230148
2015-02-21 21:29:07 +00:00
Matt Arsenault 20711b7bae R600/SI: Remove v_sub_f64 pseudo
The expansion code does the same thing. Since
the operands were not defined with the correct
types, this has the side effect of fixing operand
folding since the expanded pseudo would never use
SGPRs or inline immediates.

llvm-svn: 230072
2015-02-20 22:10:45 +00:00
Matt Arsenault 8d6300346f R600: Use new fmad node.
This enables a few useful combines that used to only
use fma.

Also since v_mad_f32 apparently does not support denormals,
disable the existing cases that are custom handled if they are
requested.

llvm-svn: 230071
2015-02-20 22:10:41 +00:00
Matt Arsenault caa1288fff R600/SI: Add missing offset operand to buffer bothen
llvm-svn: 229605
2015-02-18 02:04:38 +00:00
Matt Arsenault 2ad8bab7ee R600/SI: Add missing soffset operand to global atomics
llvm-svn: 229604
2015-02-18 02:04:35 +00:00
Tom Stellard 7b3aa88ac1 R600/SI: Fix asam errors in SIFoldOperands
We were trying to fold into implicit uses, which led to out of bounds
access of the MCInstrDesc::OpInfo arrray.

llvm-svn: 229533
2015-02-17 20:11:54 +00:00
Tom Stellard bc3776803b R600/SI: Extend private extload pattern to include zext loads
llvm-svn: 229507
2015-02-17 16:36:00 +00:00
Matt Arsenault 0bbcd8ba2f R600/SI: Implement correct f64 fdiv
This version passes the OpenCL conformance test.

llvm-svn: 229239
2015-02-14 04:30:08 +00:00
Matt Arsenault 044f1d19cf R600/SI: Use complex operand folding for div_scale
llvm-svn: 229238
2015-02-14 04:24:28 +00:00
Matt Arsenault 7eaee80675 R600/SI: Add tests for div_fmas with inline immediate operands
llvm-svn: 229237
2015-02-14 04:22:02 +00:00
Matt Arsenault 1bc9d95047 R600/SI: Fix implicit vcc operand to v_div_fmas_*
This should allow finally fixing the f64 fdiv implementation.

Test is disabled for VI since there seems to be a problem with one
of the buffer load instructions on it.

llvm-svn: 229236
2015-02-14 04:22:00 +00:00
Reid Kleckner 65842b461a Fix R600 test deadlock on Windows by giving FileCheck an argument
llc would hang trying to write output to a full pipe that FileCheck
wasn't reading. FileCheck wasn't reading from stdin because it needs a
file as a positional argument.

llvm-svn: 229157
2015-02-13 21:27:28 +00:00
Matt Arsenault 11a4d6774b R600/SI: Allow f64 inline immediates in i64 operands
This requires considering the size of the operand when
checking immediate legality.

llvm-svn: 229135
2015-02-13 19:05:03 +00:00
Matt Arsenault 8a9e404c0e R600/SI: Minor test scheduling fixes
This prevents these from failing in a later commit.

llvm-svn: 229134
2015-02-13 19:04:56 +00:00
Andrea Di Biagio b14ae8692d [CodeGenPrepare] Removed duplicate logic. SimplifyCFG already knows how to speculate calls to cttz/ctlz.
SimplifyCFG now knows how to speculate calls to intrinsic cttz/ctlz that are
'cheap' for the target. Therefore, some of the logic in CodeGenPrepare
that was originally added at revision 224899 can now be removed.

This patch is basically a no functional change. It removes the duplicated
logic in CodeGenPrepare and converts all the existing target specific tests
for cttz/ctlz into SimplifyCFG tests.

Differential Revision: http://reviews.llvm.org/D7608

llvm-svn: 229105
2015-02-13 14:15:48 +00:00
Tom Stellard 0648588e7d R600/SI: Disable subreg liveness
This is temporary while we try to fix a crash in the register coalescer.

llvm-svn: 228861
2015-02-11 18:24:53 +00:00
Tom Stellard 502ef4e791 R600/SI: Fix -march in test
llvm-svn: 228848
2015-02-11 17:11:48 +00:00
Marek Olsak fa6607d0b6 R600/SI: Enable a lot of existing tests for VI (squashed commits)
This is a union of these commits:

* R600/SI: Enable more tests for VI which need no changes

* R600/SI: Enable V_BCNT tests for VI
    Differences:
    - v_bcnt_..._e32 -> _e64
    - s_load_dword* inline offset is in bytes instead of dwords

* R600/SI: Enable all tests for VI which use S_LOAD_DWORD
    The inline offset is changed from dwords to bytes.

* R600/SI: Enable LDS tests for VI
    Differences:
    - the s_load_dword inline offset changed from dwords to bytes
    - the tests checked very little on CI, so they have been fixed to check all
      instructions that "SI" checked

* R600/SI: Enable lshr tests for VI

* R600/SI: Fix divrem64 tests
    - "v_lshl_64" was missing "b" before "64"
    - added VI-NOT checks

* R600/SI: Enable the SI.tid test for VI

* R600/SI: Enable the frem test for VI
    Also, the frem_f64 checking is added for CI-VI.

* R600/SI: Add VI tests for rsq.clamped

llvm-svn: 228830
2015-02-11 14:26:46 +00:00
Tom Stellard 94b7231740 R600/SI: Store immediate offsets > 12-bits in soffset
This will save us from having to extend these offsets to 64-bits
and storing them in a pair of vgprs.

llvm-svn: 228776
2015-02-11 00:34:35 +00:00
Michel Danzer a786077253 R600/SI: Amend a test to ensure WQM is enabled for LDS in pixel shaders
Reviewed-by: Tom Stellard <tom@stellard.net>
llvm-svn: 228374
2015-02-06 02:51:29 +00:00
Michel Danzer d89a557f33 R600/SI: Don't enable WQM for V_INTERP_* instructions v2
Doesn't seem necessary anymore. I think this was mostly compensating for
not enabling WQM for texture sampling instructions.

v2: Add test coverage
Reviewed-by: Tom Stellard <tom@stellard.net>
llvm-svn: 228373
2015-02-06 02:51:25 +00:00
Michel Danzer 494391ba47 R600/SI: Also enable WQM for image opcodes which calculate LOD v3
If whole quad mode isn't enabled for these, the level of detail is
calculated incorrectly for pixels along diagonal triangle edges, causing
artifacts.

v2: Use a TSFlag instead of lots of switch cases
v3: Add test coverage

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=88642
Reviewed-by: Tom Stellard <tom@stellard.net>
llvm-svn: 228372
2015-02-06 02:51:20 +00:00
Tom Stellard eea3f70432 R600/SI: Fix bug in TTI loop unrolling preferences
We should be setting UnrollingPreferences::MaxCount to MAX_UINT instead
of UnrollingPreferences::Count.

Count is a 'forced unrolling factor', while MaxCount sets an upper
limit to the unrolling factor.

Setting Count to MAX_UINT was causing the loop in the testcase to be
unrolled 15 times, when it only had a maximum of 4 iterations.

llvm-svn: 228303
2015-02-05 15:32:18 +00:00
Tom Stellard 0f29de78e6 R600/SI: Fix bug from insertion of llvm.SI.end.cf into loop headers
The llvm.SI.end.cf intrinsic is used to mark the end of if-then blocks,
if-then-else blocks, and loops.  It is responsible for updating the
exec mask to re-enable threads that had been masked during the preceding
control flow block.  For example:

s_mov_b64 exec, 0x3                 ; Initial exec mask
s_mov_b64 s[0:1], exec              ; Saved exec mask
v_cmpx_gt_u32 exec, s[2:3], v0, 0   ; llvm.SI.if
do_stuff()
s_or_b64 exec, exec, s[0:1]         ; llvm.SI.end.cf

The bug fixed by this patch was one where the llvm.SI.end.cf intrinsic
was being inserted into the header of loops.  This would happen when
an if block terminated in a loop header and we would end up with
code like this:

s_mov_b64 exec, 0x3                 ; Initial exec mask
s_mov_b64 s[0:1], exec              ; Saved exec mask
v_cmpx_gt_u32 exec, s[2:3], v0, 0   ; llvm.SI.if
do_stuff()

LOOP:                       ; Start of loop header
s_or_b64 exec, exec, s[0:1] ; llvm.SI.end.cf <-BUG: The exec mask has the
                              same value at the beginning of each loop
			      iteration.
do_stuff();
s_cbranch_execnz LOOP

The fix is to create a new basic block before the loop and insert the
llvm.SI.end.cf there.  This way the exec mask is restored before the
start of the loop instead of at the beginning of each iteration.

llvm-svn: 228302
2015-02-05 15:32:15 +00:00
Matt Arsenault abd271b4e8 R600/SI: Fix i64 truncate to i1
llvm-svn: 228273
2015-02-05 06:05:13 +00:00
Tom Stellard f6afc80cc0 R600/SI: Enable subreg liveness by default
llvm-svn: 228228
2015-02-04 23:14:18 +00:00
Tom Stellard 33e64c66ac R600/SI: Expand misaligned 16-bit memory accesses
llvm-svn: 228190
2015-02-04 20:49:52 +00:00
Tom Stellard c7e448c92e R600/SI: Make more store operations legal
v2i32, i32, trunc i32 to i16, and truc i32 to i8 stores are legal for
all address spaces.  We had marked them as custom in order to lower
them for the private address space, but this is no longer necessary.

This enables lowering of misaligned stores of these types in the
DAGLegalizer.

llvm-svn: 228189
2015-02-04 20:49:51 +00:00
Tom Stellard 096b8c1e6d R600: Don't promote i64 stores to v2i32 during DAG legalization
We take care of this during instruction selection now.  This
fixes a potential infinite loop when lowering misaligned stores.

llvm-svn: 228188
2015-02-04 20:49:49 +00:00
Marek Olsak 37cd4d0f42 R600/SI: Remove the -CHECK suffix from all FileCheck prefixes in LIT tests
llvm-svn: 228040
2015-02-03 21:53:27 +00:00
Marek Olsak 707a6d0c20 R600/SI: Fix B64 VALU shifts on VI
SI only has standard versions. VI only has REV versions.

Tested-by: Michel Dänzer <michel.daenzer@amd.com>
llvm-svn: 228037
2015-02-03 21:53:01 +00:00
Marek Olsak 191507e0b7 R600/SI: Don't generate non-existent LSHL, LSHR, ASHR B32 variants on VI
This can happen when a REV instruction is commuted.

The trick is not to define the _vi versions of instructions, which has these
consequences:
- code generation will always fail if a pseudo cannot be lowered
  (very useful to catch bugs where an unsupported instruction somehow makes
   it to the printer)
- ability to query if a pseudo can be lowered, which is done in commuteOpcode
  to prevent REV from commuting to non-REV on VI

Tested-by: Michel Dänzer <michel.daenzer@amd.com>
llvm-svn: 227990
2015-02-03 17:38:12 +00:00
Marek Olsak 1bd2463548 R600/SI: Fix dependency between instruction writing M0 and S_SENDMSG on VI (v2)
This fixes a hang when using an empty geometry shader.

v2: - don't add s_nop when followed by s_waitcnt
    - comestic changes

Tested-by: Michel Dänzer <michel.daenzer@amd.com>
llvm-svn: 227986
2015-02-03 17:37:52 +00:00
Tom Stellard c6b299c8c4 R600/SI: 64-bit and larger memory access must be at least 4-byte aligned
This is true for SI only. CI+ supports unaligned memory accesses,
but this requires driver support, so for now we disallow unaligned
accesses for all GCN targets.

llvm-svn: 227822
2015-02-02 18:02:28 +00:00
Tom Stellard 4cd6dcd6c4 R600/SI: Merge two test files
llvm-svn: 227821
2015-02-02 18:02:23 +00:00
Matt Arsenault 08ad328ae2 R600/SI: Only select cvt_flr/cvt_rpi with no NaNs.
These have different behavior from cvt_i32_f32 on NaN.

llvm-svn: 227693
2015-01-31 21:28:13 +00:00
Matt Arsenault 423bf3f64a R600/SI: Implement enableAggressiveFMAFusion
Add tests for the various combines. This should
always be at least cycle neutral on all subtargets for f64,
and faster on some. For f32 we should prefer selecting
v_mad_f32 over v_fma_f32.

llvm-svn: 227484
2015-01-29 19:34:32 +00:00
Tom Stellard 83f0bcef7a R600/SI: Define a schedule model and enable the generic machine scheduler
The schedule model is not complete yet, and could be improved.

llvm-svn: 227461
2015-01-29 16:55:25 +00:00
Tom Stellard 40ce8af4a5 R600: Move DataLayout to AMDGPUTargetMachine
This is a follow up to r227113.

It is now required to use the amdgcn target for SI and newer GPUs.

llvm-svn: 227316
2015-01-28 16:04:26 +00:00
Marek Olsak 75170778ec R600/SI: Enable all tests that pass on VI without changes
llvm-svn: 227214
2015-01-27 17:27:15 +00:00
Matt Arsenault 7d734f44a3 R600: Cleanup or test
Fix broken check lines, use multiple check prefixes,
add an additional test for i1 or.

llvm-svn: 227137
2015-01-26 21:16:10 +00:00
Tom Stellard edd188c459 R600/SI: Emit .hsa.version section for amdhsa OS
llvm-svn: 226970
2015-01-23 23:59:08 +00:00
Tom Stellard 20f6c0732f R600/SI: Move i64 -> v2i32 load promotion into AMDGPUDAGToDAGISel::Select()
We used to do this promotion during DAG legalization, but this
caused an infinite loop in ExpandUnalignedLoad() because it assumed
that i64 loads were legal if i64 was a legal type.

It also seems better to report i64 loads as legal, since they actually
are and we were just promoting them to simplify our tablegen files.

llvm-svn: 226945
2015-01-23 22:05:45 +00:00
Jan Vesely 5f715d36a7 R600: Try to use lower types for 64bit division if possible
v2: add and enable tests for SI

Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>
Reviewed-by: Matt Arsenault <Matthew.Arsenault@amd.com>
llvm-svn: 226881
2015-01-22 23:42:43 +00:00