Commit Graph

240 Commits

Author SHA1 Message Date
Matt Arsenault 22ca3f8860 R600/SI: Allow using f32 rcp / rsq when denormals not handled.
These are precise enough to use for OpenCL unless denormals
are handled.

llvm-svn: 213107
2014-07-15 23:50:10 +00:00
Matt Arsenault e9fa3b8e6b R600/SI: Implement less wrong f32 fdiv
Assuming single precision denormals and accurate sqrt/div are not
reported, this passes the OpenCL conformance test.

llvm-svn: 213089
2014-07-15 20:18:31 +00:00
Marek Olsak eac5062cc0 R600/SI: Use i32 vectors for resources and samplers
This affects new intrinsics only.

What surprises me is that v32i8 still works.

llvm-svn: 212831
2014-07-11 17:11:52 +00:00
Marek Olsak d8ecaeec02 R600/SI: add sample and image intrinsics exposing all instruction fields
We need the intrinsics with offsets, so why not just add them all.
The R128 parameter will also be useful for reducing SGPR usage.
GL_ARB_image_load_store also adds some image GLSL modifiers like "coherent",
so Mesa will probably translate those to slc, glc, etc.

When LLVM 3.5 is released, I'll switch Mesa to these new intrinsics.

llvm-svn: 212830
2014-07-11 17:11:46 +00:00
Matt Arsenault b0df92577d R600/SI: Add support for llvm.convert.{to|from}.fp16
llvm-svn: 212676
2014-07-10 03:22:20 +00:00
Tom Stellard 7c1838d797 R600/SI: Use a ComplexPattern for ADDR64 addressing of MUBUF loads
llvm-svn: 212217
2014-07-02 20:53:56 +00:00
Tom Stellard 10ae6a0e6a R600: Promote i64 loads to v2i32
llvm-svn: 212216
2014-07-02 20:53:54 +00:00
Tom Stellard a305f93d81 R600/SI: Add verifier check for immediates in register operands.
llvm-svn: 212214
2014-07-02 20:53:44 +00:00
Tom Stellard b02c268cbd R600/SI: Use a ComplexPattern for MUBUF stores
Now that non-leaf ComplexPatterns are allowed we can fold all the MUBUF
store patterns into the instruction definition.  We will also be able to
reuse this new ComplexPattern for MUBUF loads and atomic operations.

llvm-svn: 211644
2014-06-24 23:33:07 +00:00
Tom Stellard 9b3816b5ee R600: Promote i64 stores to v2i32
Now we need only one 64-bit pattern for stores.

llvm-svn: 211643
2014-06-24 23:33:04 +00:00
Matt Arsenault 257d48d22c R600: Fix inconsistency in rsq instructions.
R600 was using a clamped version of rsq, but SI was not. Add a
new rsq_clamped intrinsic and use them consistently.

It's unclear to me from the documentation what behavior
the R600 instructions have, so I assume they have the legacy behavior
described by the SI documents. For R600, use RECIPSQRT_IEEE
for both llvm.AMDGPU.rsq.legacy and llvm.AMDGPU.rsq. R600 also
has RECIPSQRT_FF, which I'm not sure how it fits in here.

llvm-svn: 211637
2014-06-24 22:13:39 +00:00
Matt Arsenault bd469d5e67 R600/SI: Move pattern to instruction definition
llvm-svn: 211614
2014-06-24 17:17:06 +00:00
Matt Arsenault f2b0aebb8a R600/SI: Fix div_scale intrinsic.
The operand that must match one of the others does matter,
and implement selecting for it.

llvm-svn: 211523
2014-06-23 18:28:28 +00:00
Tom Stellard ae4c9e7bc3 R600/SI: Add patterns for ctpop inside a branch
llvm-svn: 211378
2014-06-20 17:06:11 +00:00
Tom Stellard 9c603ebca4 R600/SI: Add a pattern for f32 ftrunc
llvm-svn: 211377
2014-06-20 17:06:09 +00:00
Tom Stellard c9dedb8e29 R600/SI: Add a VALU pattern for i64 xor
llvm-svn: 211373
2014-06-20 17:05:57 +00:00
Matt Arsenault a0050b0961 R600/SI: Add intrinsics for various math instructions.
These will be used for custom lowering and for library
implementations of various math functions, so it's useful
to expose these as builtins.

llvm-svn: 211247
2014-06-19 01:19:19 +00:00
Marek Olsak 51b8e7b2e7 R600/SI: add gather4 and getlod intrinsics (v3)
This contains all the previous patches + getlod support on top of it.
It doesn't use SDNodes anymore, so it's quite small.
It also adds v16i8 to SReg_128, which is used for the sampler descriptor.

Reviewed-by: Tom Stellard
llvm-svn: 211228
2014-06-18 22:00:29 +00:00
Matt Arsenault 43160e7af2 R600/SI: Add intrinsics for brev instructions
llvm-svn: 211187
2014-06-18 17:13:57 +00:00
Matt Arsenault 520e7c44c1 R600/SI: Comparisons set vcc.
llvm-svn: 211178
2014-06-18 16:53:48 +00:00
Matt Arsenault 295b86e81d R600/SI: Match cttz_zero_undef
llvm-svn: 211116
2014-06-17 17:36:27 +00:00
Matt Arsenault 8579601050 R600/SI: Match ctlz_zero_undef
llvm-svn: 211115
2014-06-17 17:36:24 +00:00
Tom Stellard 880a80ad07 R600: Use LDS and vectors for private memory
llvm-svn: 211110
2014-06-17 16:53:14 +00:00
Tom Stellard 85ad429f1f R600/SI: Add a pattern for llvm.AMDGPU.barrier.global
llvm-svn: 211109
2014-06-17 16:53:09 +00:00
Tom Stellard bc5b5370de R600: Remove AMDIL instruction and register definitions
Most of these are no longer used any more.

llvm-svn: 210915
2014-06-13 16:38:59 +00:00
Matt Arsenault 5d47d4ac7e R600: Mostly remove remaining AMDIL intrinsics.
Delete all unused ones, and add new AMDGPU named intrinsics for
the ones that are. Handle the old AMDIL names for comptability (although
remove their GCCBuiltin names) and add tests since there weren't any
for these before.

llvm-svn: 210827
2014-06-12 21:15:44 +00:00
Matt Arsenault 2c81994f92 R600/SI: Use a register set to -1 for data0 on ds_inc*/ds_dec*
There is not such thing as a 0-data ds instruction, and the data
operand needs to be a vgpr set to something meaningful.

llvm-svn: 210756
2014-06-12 08:21:54 +00:00
Matt Arsenault 2acc7a4570 R600/SI: Fix bitcast between v2i32 and f64
This is the same problem fixed in r210664 for more types.

The test passes without this fix. For some reason
I'm only hitting this when creating selects lowered
to v2i32 selects.

llvm-svn: 210692
2014-06-11 19:31:13 +00:00
Matt Arsenault 845438204f R600/SI: Update place using old subtarget predicate
llvm-svn: 210683
2014-06-11 18:11:34 +00:00
Matt Arsenault caa0ec2851 R600/SI: Add common 64-bit LDS atomics
llvm-svn: 210680
2014-06-11 18:08:54 +00:00
Matt Arsenault 1f10c5e2c9 R600/SI: Add instruction definitions for 64-bit LDS atomics
llvm-svn: 210679
2014-06-11 18:08:50 +00:00
Matt Arsenault c793e1d9dc R600/SI: Add 32-bit LDS atomic cmpxchg
llvm-svn: 210678
2014-06-11 18:08:48 +00:00
Matt Arsenault 9e874541ac R600/SI: Use LDS atomic inc / dec
llvm-svn: 210677
2014-06-11 18:08:45 +00:00
Matt Arsenault 0e69e8128c R600/SI: Add other LDS atomic operations
llvm-svn: 210676
2014-06-11 18:08:42 +00:00
Matt Arsenault 8c6613d2bf R600/SI: Add instruction definitions for more LDS ops
llvm-svn: 210675
2014-06-11 18:08:39 +00:00
Matt Arsenault 7ddcd83d49 R600/SI: Fix backwards names for local atomic instructions.
The manual lists them as *_RTN_U32, not *_U32_RTN, which is more
consistent with how every other sized instruction is named.

llvm-svn: 210674
2014-06-11 18:08:37 +00:00
Matt Arsenault 725741004c R600/SI: Refactor local atomics.
Use patterns that will also match the immediate offset to
match the normal read / writes.

llvm-svn: 210673
2014-06-11 18:08:34 +00:00
Matt Arsenault 364a6747aa R600/SI: Use v_cvt_f32_ubyte* instructions
This eliminates extra extract instructions when loading an i8 vector to
a float vector.

llvm-svn: 210666
2014-06-11 17:50:44 +00:00
Matt Arsenault 064c206d23 R600/SI: Fix selection failure on scalar_to_vector
There seem to be only 2 places that produce these,
and it's kind of tricky to hit them.

Also fixes failure to bitcast between i64 and v2f32,
although this for some reason wasn't actually broken in the
simple bitcast testcase, but did in the scalar_to_vector one.

llvm-svn: 210664
2014-06-11 17:40:32 +00:00
Tom Stellard 060ae39022 R600/SI: Fix a crash when spilling SGPRs
We need to make sure only one new instruction is added when spilling
otherwise the register allocator may crash.

This fixes a crash in the game Antichamber.

https://bugs.freedesktop.org/show_bug.cgi?id=75276

llvm-svn: 210587
2014-06-10 21:20:38 +00:00
Matt Arsenault 8333e4378e R600/SI: Implement i64 ctpop
llvm-svn: 210568
2014-06-10 19:18:24 +00:00
Matt Arsenault b5b5110b5c R600/SI: Use bcnt instruction for ctpop
llvm-svn: 210567
2014-06-10 19:18:21 +00:00
Matt Arsenault 6e43965fbc R600: Handle fcopysign
llvm-svn: 210564
2014-06-10 19:00:20 +00:00
Matt Arsenault b2cbf799d1 R600/SI: Handle sign_extend and zero_extend to i64 with patterns.
llvm-svn: 210563
2014-06-10 18:54:59 +00:00
Matt Arsenault 93840c095a R600/SI: Rename VOP3 helper class to be more general
It has other uses besides shift instructions.

llvm-svn: 210478
2014-06-09 17:00:46 +00:00
Matt Arsenault 689f325099 R600/SI: Keep 64-bit not on SALU
llvm-svn: 210476
2014-06-09 16:36:31 +00:00
Matt Arsenault 151304691c R600/SI: Match rsq instructions
llvm-svn: 210226
2014-06-05 00:15:55 +00:00
Matt Arsenault b9e1eec363 R600/SI: Remove redundant patterns
These patterns are already handled in the instruction definition.

llvm-svn: 209979
2014-05-31 19:25:17 +00:00
Matt Arsenault aeca2fa9f7 R600/SI: Fix [s|u]int_to_fp for i1
llvm-svn: 209971
2014-05-31 06:47:42 +00:00
Matt Arsenault b5c4835502 R600/SI: Fix pattern variable names.
These are confusing enough since the order swaps,
so give them more useful names.

llvm-svn: 209787
2014-05-29 01:18:01 +00:00