7783b0adf4
Revert "SelectionDAG: Enable (and (setcc x), (setcc y)) -> (setcc (and x, y)) for vectors"
...
This reverts commit r210540, adds a testcase for the regression it
caused, and marks the R600 test it was supposed to fix as XFAIL.
llvm-svn: 210792
2014-06-12 16:04:47 +00:00
2c81994f92
R600/SI: Use a register set to -1 for data0 on ds_inc*/ds_dec*
...
There is not such thing as a 0-data ds instruction, and the data
operand needs to be a vgpr set to something meaningful.
llvm-svn: 210756
2014-06-12 08:21:54 +00:00
2acc7a4570
R600/SI: Fix bitcast between v2i32 and f64
...
This is the same problem fixed in r210664 for more types.
The test passes without this fix. For some reason
I'm only hitting this when creating selects lowered
to v2i32 selects.
llvm-svn: 210692
2014-06-11 19:31:13 +00:00
caa0ec2851
R600/SI: Add common 64-bit LDS atomics
...
llvm-svn: 210680
2014-06-11 18:08:54 +00:00
c793e1d9dc
R600/SI: Add 32-bit LDS atomic cmpxchg
...
llvm-svn: 210678
2014-06-11 18:08:48 +00:00
9e874541ac
R600/SI: Use LDS atomic inc / dec
...
llvm-svn: 210677
2014-06-11 18:08:45 +00:00
0e69e8128c
R600/SI: Add other LDS atomic operations
...
llvm-svn: 210676
2014-06-11 18:08:42 +00:00
7ddcd83d49
R600/SI: Fix backwards names for local atomic instructions.
...
The manual lists them as *_RTN_U32, not *_U32_RTN, which is more
consistent with how every other sized instruction is named.
llvm-svn: 210674
2014-06-11 18:08:37 +00:00
725741004c
R600/SI: Refactor local atomics.
...
Use patterns that will also match the immediate offset to
match the normal read / writes.
llvm-svn: 210673
2014-06-11 18:08:34 +00:00
364a6747aa
R600/SI: Use v_cvt_f32_ubyte* instructions
...
This eliminates extra extract instructions when loading an i8 vector to
a float vector.
llvm-svn: 210666
2014-06-11 17:50:44 +00:00
064c206d23
R600/SI: Fix selection failure on scalar_to_vector
...
There seem to be only 2 places that produce these,
and it's kind of tricky to hit them.
Also fixes failure to bitcast between i64 and v2f32,
although this for some reason wasn't actually broken in the
simple bitcast testcase, but did in the scalar_to_vector one.
llvm-svn: 210664
2014-06-11 17:40:32 +00:00
6042506b5c
R600: Use BCNT_INT for evergreen
...
llvm-svn: 210569
2014-06-10 19:18:28 +00:00
8333e4378e
R600/SI: Implement i64 ctpop
...
llvm-svn: 210568
2014-06-10 19:18:24 +00:00
b5b5110b5c
R600/SI: Use bcnt instruction for ctpop
...
llvm-svn: 210567
2014-06-10 19:18:21 +00:00
6e43965fbc
R600: Handle fcopysign
...
llvm-svn: 210564
2014-06-10 19:00:20 +00:00
b2cbf799d1
R600/SI: Handle sign_extend and zero_extend to i64 with patterns.
...
llvm-svn: 210563
2014-06-10 18:54:59 +00:00
3ca1bfc728
SelectionDAG: Expand SELECT_CC to SELECT + SETCC
...
This consolidates code from the Hexagon, R600, and XCore targets.
No functionality change intended.
llvm-svn: 210539
2014-06-10 16:01:22 +00:00
d3d017cf00
Reduce verbiage of lit.local.cfg files
...
We can just split targets_to_build in one place and make it immutable.
llvm-svn: 210496
2014-06-09 22:42:55 +00:00
689f325099
R600/SI: Keep 64-bit not on SALU
...
llvm-svn: 210476
2014-06-09 16:36:31 +00:00
13ccc8f1bc
R600: Fix selection failure for vector bswap
...
llvm-svn: 210475
2014-06-09 16:20:25 +00:00
284ae0899b
R600: Add more and testcases
...
llvm-svn: 210453
2014-06-09 08:36:53 +00:00
42a4c9f9e0
Allow aliases to be unnamed_addr.
...
Alias with unnamed_addr were in a strange state. It is stored in GlobalValue,
the language reference talks about "unnamed_addr aliases" but the verifier
was rejecting them.
It seems natural to allow unnamed_addr in aliases:
* It is a property of how it is accessed, not of the data itself.
* It is perfectly possible to write code that depends on the address
of an alias.
This patch then makes unname_addr legal for aliases. One side effect is that
the syntax changes for a corner case: In globals, unnamed_addr is now printed
before the address space.
llvm-svn: 210302
2014-06-06 01:20:28 +00:00
3e332a4862
R600: Fix test. Using wrong check prefix.
...
llvm-svn: 210244
2014-06-05 08:00:36 +00:00
151304691c
R600/SI: Match rsq instructions
...
llvm-svn: 210226
2014-06-05 00:15:55 +00:00
aeca2fa9f7
R600/SI: Fix [s|u]int_to_fp for i1
...
llvm-svn: 209971
2014-05-31 06:47:42 +00:00
05e96f4444
R600: Try to convert BFE back to standard bit ops when possible.
...
This allows existing DAG combines to work on them, and then
we can re-match to BFE if necessary during instruction selection.
llvm-svn: 209462
2014-05-22 18:09:12 +00:00
5565f65e13
R600: Add dag combine for BFE
...
llvm-svn: 209461
2014-05-22 18:09:07 +00:00
bf8694d36d
R600: Implement ComputeNumSignBitsForTargetNode for BFE
...
llvm-svn: 209460
2014-05-22 18:09:03 +00:00
493c5f1bc4
R600: Expand mul24 for GPUs without it
...
llvm-svn: 209458
2014-05-22 18:00:24 +00:00
f15a05623e
R600: Expand mad24 for GPUs without it
...
llvm-svn: 209457
2014-05-22 18:00:20 +00:00
eb260206c3
R600: Add intrinsics for mad24
...
llvm-svn: 209456
2014-05-22 18:00:15 +00:00
c3a73c3087
R600/SI: Match fp_to_uint / uint_to_fp for f64
...
llvm-svn: 209388
2014-05-22 03:20:30 +00:00
6a57fd8b47
R600: Partially fix constant initializers for structs and vectors.
...
This should extend the current workaround to work with structs
that only contain legal, scalar types.
llvm-svn: 209331
2014-05-21 22:42:42 +00:00
310f761a42
R600: Add failing testcases for constant initializers.
...
Constant initializers involving illegal types hit an assertion.
Patch by: Jan Vesely <jan.vesely@rutgers.edu>
llvm-svn: 209330
2014-05-21 22:42:38 +00:00
f719ee9e76
R600/SI: Promote f32 SELECT to i32
...
llvm-svn: 209024
2014-05-16 20:56:41 +00:00
73b98ed8cf
R600/SI: Only use SALU instructions for 64-bit add in a block of CF depth 0
...
llvm-svn: 208886
2014-05-15 14:41:54 +00:00
365a2b49f2
R600/SI: Use VALU instructions for i1 ops
...
llvm-svn: 208885
2014-05-15 14:41:50 +00:00
a0653a3e6c
Rename ComputeMaskedBits to computeKnownBits. "Masked" has been
...
inappropriate since it lost its Mask parameter in r154011.
llvm-svn: 208811
2014-05-14 21:14:37 +00:00
4b0402e317
R600/SI: Try to fix BFE operands when moving to VALU
...
This was broken by r208479
llvm-svn: 208740
2014-05-13 23:45:50 +00:00
62b1737081
R600: Add mul24 intrinsics
...
llvm-svn: 208604
2014-05-12 17:49:57 +00:00
2adca6090f
Make SimplifyDemandedBits understand BUILD_PAIR
...
llvm-svn: 208598
2014-05-12 17:14:48 +00:00
29c0c210fc
R600/SI: Fold fabs/fneg into src input modifier
...
llvm-svn: 208480
2014-05-10 19:18:39 +00:00
94af31fbe8
R600/SI: Prettier display of input modifiers
...
llvm-svn: 208479
2014-05-10 19:18:33 +00:00
4c00b52e1a
R600/SI: Teach SIInstrInfo::moveToVALU() how to move S_LOAD_*_IMM instructions
...
llvm-svn: 208432
2014-05-09 16:42:22 +00:00
d6cb8e8efd
R600/SI: Fix SMRD pattern for offsets > 32 bits
...
We were dropping the high bits of 64-bit immediate offsets.
llvm-svn: 208431
2014-05-09 16:42:21 +00:00
a2acad785a
R600: Expand i64 SELECT_CC
...
llvm-svn: 208430
2014-05-09 16:42:19 +00:00
afa8b532b1
R600: Move MIN/MAX matching from LowerOperation() to PerformDAGCombine()
...
llvm-svn: 208429
2014-05-09 16:42:16 +00:00
45b3dcd35b
R600: Expand i64 ISD:SUB
...
llvm-svn: 208005
2014-05-05 21:47:15 +00:00
3dbf1f8df0
R600: Expand vector sin and cos.
...
v2: move code to AMDGPUISelLowering.cpp
squash with tests (both EG and SI)
Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>
llvm-svn: 207845
2014-05-02 15:41:47 +00:00
605e116e8e
R600: Expand TruncStore i64 -> {i16,i8}
...
llvm-svn: 207844
2014-05-02 15:41:46 +00:00
06028dd7be
R600/SI: Fix verifier error with pseudo store instructions.
...
Use i32 instead of specifying SReg_32. When this is
the pseudo INDIRECT_BASE_ADDR, this would give a bogus
verifier error.
llvm-svn: 207770
2014-05-01 16:37:52 +00:00
1bd80725b3
R600/SI: Use VALU instructions for copying i1 values
...
We can't use SALU instructions for this since they ignore the EXEC mask
and are always executed.
This fixes several OpenCV tests.
llvm-svn: 207661
2014-04-30 15:31:33 +00:00
0c354f25c9
R600/SI: Teach moveToVALU how to handle some SMRD instructions
...
llvm-svn: 207660
2014-04-30 15:31:29 +00:00
919bb6b83f
R600/SI: Custom lower SI_IF and SI_ELSE to avoid machine verifier errors
...
SI_IF and SI_ELSE are terminators which also produce a value. For
these instructions ISel always inserts a COPY to move their value
to another basic block. This COPY ends up between SI_(IF|ELSE)
and the S_BRANCH* instruction at the end of the block.
This breaks MachineBasicBlock::getFirstTerminator() and also the
machine verifier which assumes that terminators are grouped together at
the end of blocks.
To solve this we coalesce the copy away right after ISel to make sure
there are no instructions in between terminators at the end of blocks.
llvm-svn: 207591
2014-04-29 23:12:53 +00:00
58ac7440e6
R600/SI: Only select SALU instructions in the entry or exit block
...
SALU instructions ignore control flow, so it is not always safe to use
them within branches. This is a partial solution to this problem
until we can come up with something better.
llvm-svn: 207590
2014-04-29 23:12:48 +00:00
676f571999
R600: optimize the UDIVREM 64 algorithm
...
This is a squash of several optimization commits:
- calculate DIV_Lo and DIV_Hi separately
- use BFE_U32 if we are operating on 32bit values
- use precomputed constants instead of shifting in UDVIREM
- skip the first 32 iterations of udivrem
v2: Check whether BFE is supported before using it
Patch by: Jan Vesely
Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>
Reviewed-by: Tom Stellard <thomas.stellard@amd.com>
llvm-svn: 207589
2014-04-29 23:12:46 +00:00
4c6ab696e2
R600: Add a test that used to be broken that I forgot to add
...
llvm-svn: 207017
2014-04-23 19:45:05 +00:00
16353871c3
R600: Emit error instead of unreachable on function call
...
llvm-svn: 206904
2014-04-22 16:42:00 +00:00
5dbd5db518
R600: Make sign_extend_inreg legal.
...
Don't know why I didn't just do this in the first place.
llvm-svn: 206862
2014-04-22 03:49:30 +00:00
78b8670aac
R600/SI: Try to use scalar BFE.
...
Use scalar BFE with constant shift and offset when possible.
This is complicated by the fact that the scalar version packs
the two operands of the vector version into one.
llvm-svn: 206558
2014-04-18 05:19:26 +00:00
27cc958dff
R600/SI: Match sign_extend_inreg to s_sext_i32_i8 and s_sext_i32_i16
...
llvm-svn: 206547
2014-04-18 01:53:18 +00:00
1aa6cb4d88
R600/SI: Use SReg_64 instead of VSrc_64 when selecting BUILD_PAIR
...
llvm-svn: 206541
2014-04-18 00:36:21 +00:00
868fd92e54
R600/SI: Stop using i128 as the resource descriptor type
...
Having i128 as a legal type complicates the legalization phase. v4i32
is already a legal type, so we will use that instead.
This fixes several piglit tests.
llvm-svn: 206500
2014-04-17 21:00:11 +00:00
a90d22fad5
R600/SI: f64 frint is legal on CI
...
llvm-svn: 206475
2014-04-17 17:06:37 +00:00
51df0c1965
R600/SI: Fix zext from i1 to i64
...
llvm-svn: 206437
2014-04-17 02:03:08 +00:00
4ef2588b65
R600: Extend r600 sign_extend_inreg tests for EG
...
Patch by: Jan Vesely <jan.vesely@rutgers.edu>
llvm-svn: 206349
2014-04-16 01:41:34 +00:00
4d7d38333b
R600/SI: Print more immediates in hex format
...
Print in decimal for inline immediates, and hex otherwise. Use hex
always for offsets in addressing offsets.
This approximately matches what the shader compiler does.
llvm-svn: 206335
2014-04-15 22:32:49 +00:00
470acd81a8
R600/SI: Fix loads of i1
...
llvm-svn: 206330
2014-04-15 22:28:39 +00:00
a1a5d9aa2e
SelectionDAG: Use helper function to improve legalization of ISD::MUL
...
The TargetLowering::expandMUL() helper contains lowering code extracted
from the DAGTypeLegalizer and allows the SelectionDAGLegalizer to expand more
ISD::MUL patterns without having to use a library call.
llvm-svn: 206037
2014-04-11 16:12:01 +00:00
2c33562cd6
R600/SI: Match not instruction.
...
llvm-svn: 205837
2014-04-09 07:16:16 +00:00
204e61bbdf
R600/SI: Handle INSERT_SUBREG in SIFixSGPRCopies
...
llvm-svn: 205732
2014-04-07 19:45:45 +00:00
50122a5890
R600: Match 24-bit arithmetic patterns in a Target DAGCombine
...
Moving these patterns from TableGen files to PerformDAGCombine()
should allow us to generate better code by eliminating unnecessary
shifts and extensions earlier.
This also fixes a bug where the MAD pattern was calling
SimplifyDemandedBits with a 24-bit mask on the first operand
even when the full pattern wasn't being matched. This occasionally
resulted in some instructions being incorrectly deleted from the
program.
v2:
- Fix bug with 64-bit mul
llvm-svn: 205731
2014-04-07 19:45:41 +00:00
a0150cb6a9
R600: Correct opcode for BFE_INT
...
Acording to AMD documentation, the correct opcode for
BFE_INT is 0x5, not 0x4
Fixes Arithm/Absdiff.Mat/3 OpenCV test
Patch by: Bruno Jiménez
llvm-svn: 205562
2014-04-03 20:19:29 +00:00
7ed0b5235a
R600/SI: Lower 64-bit immediates using REG_SEQUENCE
...
llvm-svn: 205561
2014-04-03 20:19:27 +00:00
36a031870b
TargetLibraryInfo: Disable memcpy and memset on R600
...
There are no implementations of these for R600.
llvm-svn: 205455
2014-04-02 19:53:29 +00:00
553751b9bc
Fix missing RUN line in test
...
llvm-svn: 205341
2014-04-01 18:34:13 +00:00
e407ae9846
Make isSetCCEquivalent respect the TargetBooleanContents
...
llvm-svn: 205336
2014-04-01 18:13:26 +00:00
378bf9c68b
R600: Compute masked bits for min and max
...
llvm-svn: 205242
2014-03-31 19:35:33 +00:00
4c53717787
R600: Add BFE, BFI, and BFM intrinsics to help with writing tests.
...
llvm-svn: 205236
2014-03-31 18:21:18 +00:00
7ea3d6d420
R600/SI: Lower i64 SELECT by bitcasting to a vector type
...
This allows allows us to replace ISD::EXTRACT_ELEMENT, which is lowered
using shifts, with ISD::EXTRACT_VECTOR_ELT, which is a no-op.
llvm-svn: 205187
2014-03-31 14:01:55 +00:00
b517c8128e
R600: Implement isZExtFree.
...
This allows 64-bit operations that are truncated to be reduced
to 32-bit ones.
llvm-svn: 204946
2014-03-27 17:23:31 +00:00
d125d74a73
R600/SI: Fix unreachable with a sext_in_reg to an illegal type.
...
llvm-svn: 204945
2014-03-27 17:23:24 +00:00
90b733a3cf
R600: Add a testcase for sext_in_reg I missed.
...
This sext_inreg i32 in i64 case was already handled, but not enabled.
llvm-svn: 204840
2014-03-26 18:31:06 +00:00
86673ba836
R600: Add failing testcase for <3 x i32> stores.
...
This is supposed to have the same store size and alignment as <4 x i32>,
but currently is split into a 64-bit and 32-bit store.
llvm-svn: 204729
2014-03-25 16:50:55 +00:00
684dc80b6d
R600/SI: Fix extra mov from legalizing 64-bit SALU ops.
...
Check the register class of each operand individually
to avoid an extra copy to a vgpr.
llvm-svn: 204662
2014-03-24 20:08:13 +00:00
248b7b6ba1
R600/SI: Sub-optimial fix for 64-bit immediates with SALU ops.
...
No longer asserts, but now you get moves loading legal immediates
into the split 32-bit operations.
llvm-svn: 204661
2014-03-24 20:08:09 +00:00
f35182c783
R600/SI: Fix 64-bit bit ops that require the VALU.
...
Try to match scalar and first like the other instructions.
Expand 64-bit ands to a pair of 32-bit ands since that is not
available on the VALU.
llvm-svn: 204660
2014-03-24 20:08:05 +00:00
a7f1e0c44f
R600: Implement isNarrowingProfitable.
...
llvm-svn: 204658
2014-03-24 19:43:31 +00:00
ad41d7b531
R600/SI: Fix 64-bit private loads.
...
llvm-svn: 204630
2014-03-24 17:50:46 +00:00
8e2581b11e
R600/SI: Move instruction patterns to scalar versions.
...
Some of them also had the pattern on both, so this removes the
duplication.
llvm-svn: 204492
2014-03-21 18:01:18 +00:00
1583409e33
R600/SI: Handle MUBUF instructions in SIInstrInfo::moveToVALU()
...
llvm-svn: 204476
2014-03-21 15:51:57 +00:00
e038720702
R600/SI: Handle S_MOV_B64 in SIInstrInfo::moveToVALU()
...
llvm-svn: 204475
2014-03-21 15:51:54 +00:00
d06ebd93e6
R600/SI: Add support for 64-bit LDS writes
...
llvm-svn: 204274
2014-03-19 22:19:54 +00:00
b943348cb9
R600/SI: Add support for 64-bit LDS loads.
...
v2:
-Use correct opcode for DS_READ_64
llvm-svn: 204273
2014-03-19 22:19:52 +00:00
99ed78926b
R600/SI: Match i16 immediate offset of LDS instructions.
...
llvm-svn: 204272
2014-03-19 22:19:49 +00:00
43eeee182a
R600/SI: Fix test checking wrong instruction operand.
...
The source and destination happen to be the same register.
llvm-svn: 204271
2014-03-19 22:19:45 +00:00
547aff20f5
R600/SI: Don't display the GDS bit.
...
It isn't actually used now, and probably never will be, plus it makes
tests less annoying. I also think SC prints GDS instructions as a
separate instruction name.
llvm-svn: 204270
2014-03-19 22:19:43 +00:00
7a1ac3b89b
CodeGen/R600/v_cndmask.ll: Relax an expression to unbreak msvcrt.
...
V_CNDMASK_B32_e64 v0, v0, -1.#QNAN0e+00, s[2:3], 0, 0, 0, 0
FIXME: We really need to implement our formatter...
llvm-svn: 204118
2014-03-18 06:17:22 +00:00
8d761cc56d
Making a guess to fix the test case with r204056 to get the build bot working.
...
llvm-svn: 204073
2014-03-17 19:00:03 +00:00
fae02989b7
R600: Match sign_extend_inreg to BFE instructions
...
llvm-svn: 204072
2014-03-17 18:58:11 +00:00
Tom Stellard