llvm-project/llvm/lib/Target/AArch64
Geoff Berry 5d534b6a11 [CodeGenPrepare] Sink and duplicate more 'and' instructions.
Summary:
Rework the code that was sinking/duplicating (icmp and, 0) sequences
into blocks where they were being used by conditional branches to form
more tbz instructions on AArch64.  The new code is more general in that
it just looks for 'and's that have all icmp 0's as users, with a target
hook used to select which subset of 'and' instructions to consider.
This change also enables 'and' sinking for X86, where it is more widely
beneficial than on AArch64.

The 'and' sinking/duplicating code is moved into the optimizeInst phase
of CodeGenPrepare, where it can take advantage of the fact the
OptimizeCmpExpression has already sunk/duplicated any icmps into the
blocks where they are used.  One minor complication from this change is
that optimizeLoadExt needed to be updated to always mark 'and's it has
determined should be in the same block as their feeding load in the
InsertedInsts set to avoid an infinite loop of hoisting and sinking the
same 'and'.

This change fixes a regression on X86 in the tsan runtime caused by
moving GVNHoist to a later place in the optimization pipeline (see
PR31382).

Reviewers: t.p.northover, qcolombet, MatzeB

Subscribers: aemerson, mcrosier, sebpop, llvm-commits

Differential Revision: https://reviews.llvm.org/D28813

llvm-svn: 295746
2017-02-21 18:53:14 +00:00
..
AsmParser AArch64AsmParser: tablegen the isBranchTarget helper functions 2017-02-20 10:57:54 +00:00
Disassembler [AArch64, Lanai] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). 2017-01-06 00:30:53 +00:00
InstPrinter [AArch64] Fix some Include What You Use warnings; other minor fixes (NFC). 2017-02-03 21:49:13 +00:00
MCTargetDesc [AArch64] Fix some Include What You Use warnings; other minor fixes (NFC). 2017-02-03 21:49:13 +00:00
TargetInfo Move the global variables representing each Target behind accessor function 2016-10-09 23:00:34 +00:00
Utils AArch64: try to fix optimized build failure. 2016-07-05 23:15:58 +00:00
AArch64.h [AArch64] Avoid generating indexed vector instructions for Exynos 2016-10-08 12:30:07 +00:00
AArch64.td [AArch64] Add Cavium ThunderX support 2017-02-17 18:34:24 +00:00
AArch64A53Fix835769.cpp Use StringRef in Pass/PassManager APIs (NFC) 2016-10-01 02:56:57 +00:00
AArch64A57FPLoadBalancing.cpp LiveRegUnits: Add accumulateBackward() function 2017-01-21 02:21:04 +00:00
AArch64AddressTypePromotion.cpp [AArch64] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). 2017-01-25 00:29:26 +00:00
AArch64AdvSIMDScalarPass.cpp Use StringRef in Pass/PassManager APIs (NFC) 2016-10-01 02:56:57 +00:00
AArch64AsmPrinter.cpp [XRay] Merge instrumentation point table emission code into AsmPrinter. 2017-01-03 04:30:21 +00:00
AArch64CallLowering.cpp GlobalISel: select G_VASTART on iOS AArch64. 2017-02-08 17:57:27 +00:00
AArch64CallLowering.h [AArch64] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). 2017-02-01 22:56:06 +00:00
AArch64CallingConvention.h
AArch64CallingConvention.td SwiftCC: swifterror register cannot be as the base register 2017-02-09 01:52:17 +00:00
AArch64CleanupLocalDynamicTLSPass.cpp Use StringRef in Pass/PassManager APIs (NFC) 2016-10-01 02:56:57 +00:00
AArch64CollectLOH.cpp AArch64CollectLOH: Rewrite as block-local analysis. 2017-01-06 19:22:01 +00:00
AArch64ConditionOptimizer.cpp [CodeGen] Rename MachineInstrBuilder::addOperand. NFC 2017-01-13 09:58:52 +00:00
AArch64ConditionalCompares.cpp [CodeGen] Rename MachineInstrBuilder::addOperand. NFC 2017-01-13 09:58:52 +00:00
AArch64DeadRegisterDefinitionsPass.cpp AArch64: Use DeadRegisterDefinitionsPass before regalloc. 2016-11-16 03:38:27 +00:00
AArch64ExpandPseudoInsts.cpp [AArch64] Generate literals by the little end 2017-01-18 18:57:08 +00:00
AArch64FastISel.cpp [AArch64] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). 2017-01-25 00:29:26 +00:00
AArch64FrameLowering.cpp [AArch64] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). 2017-01-25 00:29:26 +00:00
AArch64FrameLowering.h [PEI, AArch64] Use empty spaces in stack area for local stack slot allocation. 2016-06-02 16:22:07 +00:00
AArch64GenRegisterBankInfo.def GlobalISel: fall back gracefully when we can't map an operand's size. 2017-02-06 21:57:06 +00:00
AArch64ISelDAGToDAG.cpp [AArch64] Fix useful bits detection for BFM instructions 2016-11-30 17:04:22 +00:00
AArch64ISelLowering.cpp [CodeGenPrepare] Sink and duplicate more 'and' instructions. 2017-02-21 18:53:14 +00:00
AArch64ISelLowering.h [CodeGenPrepare] Sink and duplicate more 'and' instructions. 2017-02-21 18:53:14 +00:00
AArch64InstrAtomics.td AArch64: properly calculate cmpxchg status in FastISel. 2016-08-02 20:22:36 +00:00
AArch64InstrFormats.td AArch64AsmParser: tablegen the isBranchTarget helper functions 2017-02-20 10:57:54 +00:00
AArch64InstrInfo.cpp [CodeGen] Move MacroFusion to the target 2017-02-01 02:54:34 +00:00
AArch64InstrInfo.h [CodeGen] Move MacroFusion to the target 2017-02-01 02:54:34 +00:00
AArch64InstrInfo.td Revert r294437 as it broke an asan buildbot. 2017-02-08 21:41:16 +00:00
AArch64InstructionSelector.cpp GlobalISel: deal with new G_PTR_MASK instruction on AArch64. 2017-02-14 20:56:29 +00:00
AArch64InstructionSelector.h GlobalISel: select G_VASTART on iOS AArch64. 2017-02-08 17:57:27 +00:00
AArch64LegalizerInfo.cpp GlobalISel: legalize va_arg on AArch64. 2017-02-15 23:22:50 +00:00
AArch64LegalizerInfo.h GlobalISel: legalize va_arg on AArch64. 2017-02-15 23:22:50 +00:00
AArch64LoadStoreOptimizer.cpp AArch64LoadStoreOptimizer: Correctly clear kill flags 2017-02-17 23:15:03 +00:00
AArch64MCInstLower.cpp Remove TargetTriple from AArch64MCInstLower as it's used in few places 2016-10-01 01:50:25 +00:00
AArch64MCInstLower.h
AArch64MachineFunctionInfo.h [AArch64, Lanai] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). 2017-01-06 00:30:53 +00:00
AArch64MacroFusion.cpp [AArch64] Prefer static_cast<> to C-style cast. NFCI. 2017-02-19 21:31:14 +00:00
AArch64MacroFusion.h [CodeGen] Move MacroFusion to the target 2017-02-01 02:54:34 +00:00
AArch64PBQPRegAlloc.cpp
AArch64PBQPRegAlloc.h
AArch64PerfectShuffle.h
AArch64PromoteConstant.cpp Use StringRef in Pass/PassManager APIs (NFC) 2016-10-01 02:56:57 +00:00
AArch64RedundantCopyElimination.cpp [AArch64] Minor code refactoring. NFC. 2017-01-25 15:56:59 +00:00
AArch64RegisterBankInfo.cpp GlobalISel: fall back gracefully when we can't map an operand's size. 2017-02-06 21:57:06 +00:00
AArch64RegisterBankInfo.h GlobalISel: fall back gracefully when we can't map an operand's size. 2017-02-06 21:57:06 +00:00
AArch64RegisterBanks.td Re-commit: [globalisel] Tablegen-erate current Register Bank Information 2017-01-19 11:15:55 +00:00
AArch64RegisterInfo.cpp AArch64RegisterInfo: Simplify getReservedReg(); NFC 2017-02-02 02:23:25 +00:00
AArch64RegisterInfo.h AArch64: Enable post-ra liveness updates 2016-12-16 23:55:43 +00:00
AArch64RegisterInfo.td [AArch64] Corrected spill size for DDD register class. NFCI 2016-10-21 09:53:42 +00:00
AArch64SchedA53.td
AArch64SchedA57.td [AArch64] Add new subtarget feature to fuse AES crypto operations 2017-02-01 02:54:39 +00:00
AArch64SchedA57WriteRes.td [AArch64] Cortex-A57 FDIV/FSQRT scheduling fix (W-unit) 2016-12-23 12:51:41 +00:00
AArch64SchedCyclone.td CodeGen: Use MachineInstr& in TargetInstrInfo, NFC 2016-06-30 00:01:54 +00:00
AArch64SchedFalkor.td [AArch64] Add a basic SchedMachineModel for Falkor. 2016-11-29 20:00:27 +00:00
AArch64SchedKryo.td
AArch64SchedKryoDetails.td [AArch64] Refine Kryo Machine Model 2017-01-26 20:10:41 +00:00
AArch64SchedM1.td [AArch64] Add new subtarget feature to fuse AES crypto operations 2017-02-01 02:54:39 +00:00
AArch64SchedThunderX.td [AArch64] Add Cavium ThunderX support 2017-02-17 18:34:24 +00:00
AArch64SchedVulcan.td [AArch64] Maximize 80-column. NFC. 2016-11-22 14:12:09 +00:00
AArch64Schedule.td CodeGen: Use MachineInstr& in TargetInstrInfo, NFC 2016-06-30 00:01:54 +00:00
AArch64SelectionDAGInfo.cpp [SDAG] Remove FixedArgs parameter from CallLoweringInfo::setCallee 2016-06-22 12:54:25 +00:00
AArch64SelectionDAGInfo.h Pass DebugLoc and SDLoc by const ref. 2016-06-12 15:39:02 +00:00
AArch64StorePairSuppress.cpp Use StringRef in Pass/PassManager APIs (NFC) 2016-10-01 02:56:57 +00:00
AArch64Subtarget.cpp [AArch64] Add Cavium ThunderX support 2017-02-17 18:34:24 +00:00
AArch64Subtarget.h [AArch64] Add Cavium ThunderX support 2017-02-17 18:34:24 +00:00
AArch64SystemOperands.td AArch64: TableGenerate system instruction operands. 2016-07-05 21:23:04 +00:00
AArch64TargetMachine.cpp [AArch64] Add new target feature to fuse literal generation 2017-02-01 02:54:42 +00:00
AArch64TargetMachine.h Revert "[AArch64] Use the reciprocal estimation machinery" 2016-09-20 19:02:06 +00:00
AArch64TargetObjectFile.cpp CodeGen: simplify TargetMachine::getSymbol interface. NFC. 2016-11-22 16:17:20 +00:00
AArch64TargetObjectFile.h Move the Mangler from the AsmPrinter down to TLOF and clean up the 2016-09-16 07:33:15 +00:00
AArch64TargetTransformInfo.cpp [X86] updating TTI costs for arithmetic instructions on X86\SLM arch. 2017-01-11 08:23:37 +00:00
AArch64TargetTransformInfo.h [TargetTransformInfo] Refactor and improve getScalarizationOverhead() 2017-01-26 07:03:25 +00:00
AArch64VectorByElementOpt.cpp [AArch64] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). 2017-01-25 00:29:26 +00:00
CMakeLists.txt [CodeGen] Move MacroFusion to the target 2017-02-01 02:54:34 +00:00
LLVMBuild.txt