Commit Graph

9 Commits

Author SHA1 Message Date
Jay Foad 27ec195f39 [AMDGPU] Fix DPP combiner check for exec modification
Summary:
r363675 changed the exec modification helper function, now called
execMayBeModifiedBeforeUse, so that if no UseMI is specified it checks
all instructions in the basic block, even beyond the last use. That
meant that the DPP combiner no longer worked in any basic block that
ended with a control flow instruction, and in particular it didn't work
on code sequences generated by the atomic optimizer.

Fix it by reinstating the old behaviour but in a new helper function
execMayBeModifiedBeforeAnyUse, and limiting the number of instructions
scanned.

Reviewers: arsenm, vpykhtin

Subscribers: kzhuravl, nemanjai, jvesely, wdng, nhaehnle, yaxunl, dstuttard, tpr, t-tye, hiraditya, kbarton, MaskRay, jfb, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64393

llvm-svn: 365910
2019-07-12 15:59:40 +00:00
Jay Foad 7e0c10b55f [AMDGPU] DPP combiner: recognize identities for more opcodes
Summary:
This allows the DPP combiner to kick in more often. For example the
exclusive scan generated by the atomic optimizer for a divergent atomic
add used to look like this:

        v_mov_b32_e32 v3, v1
        v_mov_b32_e32 v5, v1
        v_mov_b32_e32 v6, v1
        v_mov_b32_dpp v3, v2  wave_shr:1 row_mask:0xf bank_mask:0xf
        s_nop 1
        v_add_u32_dpp v4, v3, v3  row_shr:1 row_mask:0xf bank_mask:0xf bound_ctrl:0
        v_mov_b32_dpp v5, v3  row_shr:2 row_mask:0xf bank_mask:0xf
        v_mov_b32_dpp v6, v3  row_shr:3 row_mask:0xf bank_mask:0xf
        v_add3_u32 v3, v4, v5, v6
        v_mov_b32_e32 v4, v1
        s_nop 1
        v_mov_b32_dpp v4, v3  row_shr:4 row_mask:0xf bank_mask:0xe
        v_add_u32_e32 v3, v3, v4
        v_mov_b32_e32 v4, v1
        s_nop 1
        v_mov_b32_dpp v4, v3  row_shr:8 row_mask:0xf bank_mask:0xc
        v_add_u32_e32 v3, v3, v4
        v_mov_b32_e32 v4, v1
        s_nop 1
        v_mov_b32_dpp v4, v3  row_bcast:15 row_mask:0xa bank_mask:0xf
        v_add_u32_e32 v3, v3, v4
        s_nop 1
        v_mov_b32_dpp v1, v3  row_bcast:31 row_mask:0xc bank_mask:0xf
        v_add_u32_e32 v1, v3, v1
        v_add_u32_e32 v1, v2, v1
        v_readlane_b32 s0, v1, 63

But now most of the dpp movs are combined into adds:

        v_mov_b32_e32 v3, v1
        v_mov_b32_e32 v5, v1
        s_nop 0
        v_mov_b32_dpp v3, v2  wave_shr:1 row_mask:0xf bank_mask:0xf
        s_nop 1
        v_add_u32_dpp v4, v3, v3  row_shr:1 row_mask:0xf bank_mask:0xf bound_ctrl:0
        v_mov_b32_dpp v5, v3  row_shr:2 row_mask:0xf bank_mask:0xf
        v_mov_b32_dpp v1, v3  row_shr:3 row_mask:0xf bank_mask:0xf
        v_add3_u32 v1, v4, v5, v1
        s_nop 1
        v_add_u32_dpp v1, v1, v1  row_shr:4 row_mask:0xf bank_mask:0xe
        s_nop 1
        v_add_u32_dpp v1, v1, v1  row_shr:8 row_mask:0xf bank_mask:0xc
        s_nop 1
        v_add_u32_dpp v1, v1, v1  row_bcast:15 row_mask:0xa bank_mask:0xf
        s_nop 1
        v_add_u32_dpp v1, v1, v1  row_bcast:31 row_mask:0xc bank_mask:0xf
        v_add_u32_e32 v1, v2, v1
        v_readlane_b32 s0, v1, 63

Reviewers: arsenm, vpykhtin

Subscribers: kzhuravl, nemanjai, jvesely, wdng, nhaehnle, yaxunl, dstuttard, tpr, t-tye, hiraditya, kbarton, MaskRay, jfb, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64207

llvm-svn: 365211
2019-07-05 14:52:48 +00:00
Jay Foad 0cd50b2a95 Fix typos in comments and debug output.
llvm-svn: 365146
2019-07-04 15:04:29 +00:00
Matt Arsenault f39f3bd056 AMDGPU: Change API for checking for exec modification
Invert the name and return value to better reflect the imprecise
nature.

Force passing in the DefMI, since it's known in the 2 users and could
possibly fail for an arbitrary vreg.

Allow specifying a specific user instruction. Scan through use
instructions, instead of use operands. Add scan thresholds instead of
searching infinitely.

Stop using a set to track seen uses. I didn't understand this usage,
or why it would not check the last use. I don't think the use list has
any particular order.

llvm-svn: 363675
2019-06-18 12:48:36 +00:00
Valery Pykhtin 7fe97f8c7c [AMDGPU] Fix DPP combiner
Differential revision: https://reviews.llvm.org/D55444

dpp move with uses and old reg initializer should be in the same BB.
bound_ctrl:0 is only considered when bank_mask and row_mask are fully enabled (0xF). Otherwise the old register value is checked for identity.
Added add, subrev, and, or instructions to the old folding function.
Kill flag is cleared for the src0 (DPP register) as it may be copied into more than one user.

The pass is still disabled by default.

llvm-svn: 353513
2019-02-08 11:59:48 +00:00
Chandler Carruth 2946cd7010 Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636
2019-01-19 08:50:56 +00:00
Valery Pykhtin b7a459547d Revert "[AMDGPU] Fix DPP combiner"
This reverts commit e3e2923a39cbec3b3bc3a7d3f0e9a77a4115080e, svn revision rL350721

llvm-svn: 350730
2019-01-09 15:21:53 +00:00
Valery Pykhtin 1e0b5c719b [AMDGPU] Fix DPP combiner
Fixed issue with identity values and other cases, f32/f16 identity values to be added later. fma/mac instructions is disabled for now.
Test is fully reworked, added comments. Other fixes:

1. dpp move with uses and old reg initializer should be in the same BB.
2. bound_ctrl:0 is only considered when bank_mask and row_mask are fully enabled (0xF). Othervise the old register value is checked for identity.
3. Added add, subrev, and, or instructions to the old folding function.
4. Kill flag is cleared for the src0 (DPP register) as it may be copied into more than one user.

Differential revision: https://reviews.llvm.org/D55444

llvm-svn: 350721
2019-01-09 13:43:32 +00:00
Valery Pykhtin 3d9afa273f [AMDGPU] Combine DPP mov with use instructions (VOP1/2/3)
Introduces DPP pseudo instructions and the pass that combines DPP mov with subsequent uses.

Differential revision: https://reviews.llvm.org/D53762

llvm-svn: 347993
2018-11-30 14:21:56 +00:00