maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/llvm/test/CodeGen/X86/opt-pipeline.ll

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

209 lines
9.9 KiB
LLVM
Raw Normal View History

[X86] Fix pipeline tests when enabling MIR verification, NFC llvm-svn: 345226
2018-10-25 07:52:22 +08:00
; When EXPENSIVE_CHECKS are enabled, the machine verifier appears between each
; pass. Ignore it with 'grep -v'.
[X86] Rename O3-pipeline.ll to opt-pipeline.ll and add O1/O2 command lines Eric Cristopher asked me about possibly disabling some passes at -O1/Og. Figured a good first step was to test all the pipelines. They all appear to be the same for now. Hoping we can use FileCheck prefixes for differences to avoid repeating the contents 3 times.
2020-06-25 01:56:20 +08:00
; RUN: llc -mtriple=x86_64-- -O1 -debug-pass=Structure < %s -o /dev/null 2>&1 \
; RUN: | grep -v 'Verify generated machine code' | FileCheck %s
; RUN: llc -mtriple=x86_64-- -O2 -debug-pass=Structure < %s -o /dev/null 2>&1 \
; RUN: | grep -v 'Verify generated machine code' | FileCheck %s
[X86] Fix pipeline tests when enabling MIR verification, NFC llvm-svn: 345226
2018-10-25 07:52:22 +08:00
; RUN: llc -mtriple=x86_64-- -O3 -debug-pass=Structure < %s -o /dev/null 2>&1 \
; RUN: | grep -v 'Verify generated machine code' | FileCheck %s
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; REQUIRES: asserts
; CHECK-LABEL: Pass Arguments:
; CHECK-NEXT: Target Library Information
; CHECK-NEXT: Target Pass Configuration
; CHECK-NEXT: Machine Module Information
; CHECK-NEXT: Target Transform Information
; CHECK-NEXT: Type-Based Alias Analysis
; CHECK-NEXT: Scoped NoAlias Alias Analysis
; CHECK-NEXT: Assumption Cache Tracker
Revert "[PGO][PGSO] Instrument the code gen / target passes." This reverts commit 9a0b5e14075a1f42a72eedb66fd4fde7985d37ac. This seems to break buildbots.
2019-12-07 04:17:32 +08:00
; CHECK-NEXT: Profile summary info
[PGO][PGSO] Instrument the code gen / target passes. Summary: Split off of D67120. Add the profile guided size optimization instrumentation / queries in the code gen or target passes. This doesn't enable the size optimizations in those passes yet as they are currently disabled in shouldOptimizeForSize (for non-IR pass queries). A second try after reverted D71072. Reviewers: davidxl Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D71149
2019-12-06 01:39:37 +08:00
; CHECK-NEXT: Create Garbage Collector Module Metadata
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; CHECK-NEXT: Machine Branch Probability Analysis
; CHECK-NEXT: ModulePass Manager
; CHECK-NEXT: Pre-ISel Intrinsic Lowering
; CHECK-NEXT: FunctionPass Manager
; CHECK-NEXT: Expand Atomic instructions
[X86] Pass to transform amx intrinsics to scalar operation. This pass runs in any situations but we skip it when it is not O0 and the function doesn't have optnone attribute. With -O0, the def of shape to amx intrinsics is near the amx intrinsics code. We are not able to find a point which post-dominate all the shape and dominate all amx intrinsics. To decouple the dependency of the shape, we transform amx intrinsics to scalar operation, so that compiling doesn't fail. In long term, we should improve fast register allocation to allocate amx register. Reviewed By: pengfei Differential Revision: https://reviews.llvm.org/D93594
2021-03-16 09:44:42 +08:00
; CHECK-NEXT: Lower AMX intrinsics
[X86] AMX programming model. This patch implements amx programming model that discussed in llvm-dev (http://lists.llvm.org/pipermail/llvm-dev/2020-August/144302.html). Thank Hal for the good suggestion in the RA. The fast RA is not in the patch yet. This patch implemeted 7 components. 1. The c interface to end user. 2. The AMX intrinsics in LLVM IR. 3. Transform load/store <256 x i32> to AMX intrinsics or split the type into two <128 x i32>. 4. The Lowering from AMX intrinsics to AMX pseudo instruction. 5. Insert psuedo ldtilecfg and build the def-use between ldtilecfg to amx intruction. 6. The register allocation for tile register. 7. Morph AMX pseudo instruction to AMX real instruction. Change-Id: I935e1080916ffcb72af54c2c83faa8b2e97d5cb0 Differential Revision: https://reviews.llvm.org/D87981
2020-09-06 10:17:22 +08:00
; CHECK-NEXT: Lower AMX type for load/store
[TargetPassConfig] Don't add alias analysis at optnone When performing codegen at optnone, don't add alias analysis to the pipeline. We don't need it, but it causes an unnecessary dominator tree calculation. I've also moved the module verifier call to the top so that a bunch of disabled-at-optnone passes group more nicely. Differential Revision: https://reviews.llvm.org/D80378
2020-05-20 04:37:37 +08:00
; CHECK-NEXT: Module Verifier
[X86] Fix compile time regression of D93594. D93594 depend on the dominate tree and loop information. It increased the compile time when build with -O0. However this is just to amend the dominate tree and loop information, so that it is unnecessary to re-analyze them again. Given the dominate tree of loop information are absent in this pass, we can avoid amending them. Differential Revision: https://reviews.llvm.org/D98773
2021-03-17 19:17:18 +08:00
; CHECK-NEXT: Dominator Tree Construction
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; CHECK-NEXT: Basic Alias Analysis (stateless AA impl)
[X86] Fix compile time regression of D93594. D93594 depend on the dominate tree and loop information. It increased the compile time when build with -O0. However this is just to amend the dominate tree and loop information, so that it is unnecessary to re-analyze them again. Given the dominate tree of loop information are absent in this pass, we can avoid amending them. Differential Revision: https://reviews.llvm.org/D98773
2021-03-17 19:17:18 +08:00
; CHECK-NEXT: Natural Loop Information
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; CHECK-NEXT: Canonicalize natural loops
; CHECK-NEXT: Scalar Evolution Analysis
; CHECK-NEXT: Loop Pass Manager
[TargetPassConfig] Add CanonicalizeFreezeInLoops before LSR Summary: This patch adds CanonicalizeFreezeInLoops before LSR. Relevant patch: https://reviews.llvm.org/D77523 Reviewers: spatel, efriedma, jdoerfert, fhahn, nikic, reames, xbolva00 Reviewed By: nikic Subscribers: xbolva00, nikic, lebedev.ri, hiraditya, llvm-commits, sanwou01, nlopes Tags: #llvm Differential Revision: https://reviews.llvm.org/D77524
2020-05-28 00:48:37 +08:00
; CHECK-NEXT: Canonicalize Freeze Instructions in Loops
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; CHECK-NEXT: Induction Variable Users
; CHECK-NEXT: Loop Strength Reduction
Revert "Reland "r364412 [ExpandMemCmp][MergeICmps] Move passes out of CodeGen into opt pipeline."" This reverts commit r371502, it broke tests (clang/test/CodeGenCXX/auto-var-init.cpp). llvm-svn: 371507
2019-09-10 18:39:09 +08:00
; CHECK-NEXT: Basic Alias Analysis (stateless AA impl)
; CHECK-NEXT: Function Alias Analysis Results
; CHECK-NEXT: Merge contiguous icmps into a memcmp
[PGO][PGSO] Instrument the code gen / target passes. Summary: Split off of D67120. Add the profile guided size optimization instrumentation / queries in the code gen or target passes. This doesn't enable the size optimizations in those passes yet as they are currently disabled in shouldOptimizeForSize (for non-IR pass queries). A second try after reverted D71072. Reviewers: davidxl Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D71149
2019-12-06 01:39:37 +08:00
; CHECK-NEXT: Natural Loop Information
; CHECK-NEXT: Lazy Branch Probability Analysis
; CHECK-NEXT: Lazy Block Frequency Analysis
Revert "Reland "r364412 [ExpandMemCmp][MergeICmps] Move passes out of CodeGen into opt pipeline."" This reverts commit r371502, it broke tests (clang/test/CodeGenCXX/auto-var-init.cpp). llvm-svn: 371507
2019-09-10 18:39:09 +08:00
; CHECK-NEXT: Expand memcmp() to load/stores
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; CHECK-NEXT: Lower Garbage Collection Instructions
; CHECK-NEXT: Shadow Stack GC Lowering
Reapply r374743 with a fix for the ocaml binding Add a pass to lower is.constant and objectsize intrinsics This pass lowers is.constant and objectsize intrinsics not simplified by earlier constant folding, i.e. if the object given is not constant or if not using the optimized pass chain. The result is recursively simplified and constant conditionals are pruned, so that dead blocks are removed even for -O0. This allows inline asm blocks with operand constraints to work all the time. The new pass replaces the existing lowering in the codegen-prepare pass and fallbacks in SDAG/GlobalISEL and FastISel. The latter now assert on the intrinsics. Differential Revision: https://reviews.llvm.org/D65280 llvm-svn: 374784
2019-10-15 00:15:14 +08:00
; CHECK-NEXT: Lower constant intrinsics
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; CHECK-NEXT: Remove unreachable blocks from the CFG
; CHECK-NEXT: Natural Loop Information
[BPI][NFC] Reuse post dominantor tree from analysis manager when available Summary: Currenlty BPI unconditionally creates post dominator tree each time. While this is not incorrect we can save compile time by reusing existing post dominator tree (when it's valid) provided by analysis manager. Reviewers: skatkov, taewookoh, yrouban Reviewed By: skatkov Subscribers: hiraditya, steven_wu, dexonsmith, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D78987
2020-04-28 17:31:20 +08:00
; CHECK-NEXT: Post-Dominator Tree Construction
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; CHECK-NEXT: Branch Probability Analysis
; CHECK-NEXT: Block Frequency Analysis
; CHECK-NEXT: Constant Hoisting
[CodeGen] New pass: Replace vector intrinsics with call to vector library This patch adds a pass to replace calls to vector intrinsics (i.e., LLVM intrinsics operating on vector operands) with calls to a vector library. Currently, calls to LLVM intrinsics are only replaced with calls to vector libraries when scalar calls to intrinsics are vectorized by the Loop- or SLP-Vectorizer. With this pass, it is now possible to replace calls to LLVM intrinsics already operating on vector operands, e.g., if such code was generated by MLIR. For the replacement, information from the TargetLibraryInfo, e.g., as specified via -vector-library is used. This is a re-try of the original commit 2303e93e66 that was reverted due to pass manager problems. Other minor changes have also been made. Differential Revision: https://reviews.llvm.org/D95373
2021-02-13 01:22:28 +08:00
; CHECK-NEXT: Replace intrinsics with calls to vector library
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; CHECK-NEXT: Partially inline calls to library functions
; CHECK-NEXT: Scalarize Masked Memory Intrinsics
; CHECK-NEXT: Expand reduction intrinsics
; CHECK-NEXT: Interleaved Access Pass
[X86] Move combineLoopMAddPattern and combineLoopSADPattern to an IR pass before SelecitonDAG. These transforms rely on a vector reduction flag on the SDNode set by SelectionDAGBuilder. This flag exists because SelectionDAG can't see across basic blocks so SelectionDAGBuilder is looking across and saving the info. X86 is the only target that uses this flag currently. By removing the X86 code we can remove the flag and the SelectionDAGBuilder code. This pass adds a dedicated IR pass for X86 that looks across the blocks and transforms the IR into a form that the X86 SelectionDAG can finish. An advantage of this new approach is that we can enhance it to shrink the phi nodes and final reduction tree based on the zeroes that we need to concatenate to bring the partially reduced reduction back up to the original width. Differential Revision: https://reviews.llvm.org/D76649
2020-03-27 02:09:08 +08:00
; CHECK-NEXT: X86 Partial Reduction
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; CHECK-NEXT: Expand indirectbr instructions
; CHECK-NEXT: Natural Loop Information
; CHECK-NEXT: CodeGen Prepare
Revert "Remove "Rewrite Symbols" from codegen pipeline" This reverts commit 6210261ecb21c84c9a440a76c0ccbc8ad211bed3. addr-label.ll crashes on armv7.
2021-04-11 14:28:16 +08:00
; CHECK-NEXT: Rewrite Symbols
; CHECK-NEXT: FunctionPass Manager
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; CHECK-NEXT: Dominator Tree Construction
; CHECK-NEXT: Exception handling preparation
; CHECK-NEXT: Safe Stack instrumentation pass
; CHECK-NEXT: Insert stack protectors
; CHECK-NEXT: Module Verifier
; CHECK-NEXT: Basic Alias Analysis (stateless AA impl)
; CHECK-NEXT: Function Alias Analysis Results
; CHECK-NEXT: Natural Loop Information
[BPI][NFC] Reuse post dominantor tree from analysis manager when available Summary: Currenlty BPI unconditionally creates post dominator tree each time. While this is not incorrect we can save compile time by reusing existing post dominator tree (when it's valid) provided by analysis manager. Reviewers: skatkov, taewookoh, yrouban Reviewed By: skatkov Subscribers: hiraditya, steven_wu, dexonsmith, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D78987
2020-04-28 17:31:20 +08:00
; CHECK-NEXT: Post-Dominator Tree Construction
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; CHECK-NEXT: Branch Probability Analysis
[PGO][PGSO] Instrument the code gen / target passes. Summary: Split off of D67120. Add the profile guided size optimization instrumentation / queries in the code gen or target passes. This doesn't enable the size optimizations in those passes yet as they are currently disabled in shouldOptimizeForSize (for non-IR pass queries). A second try after reverted D71072. Reviewers: davidxl Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D71149
2019-12-06 01:39:37 +08:00
; CHECK-NEXT: Lazy Branch Probability Analysis
; CHECK-NEXT: Lazy Block Frequency Analysis
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; CHECK-NEXT: X86 DAG->DAG Instruction Selection
; CHECK-NEXT: MachineDominator Tree Construction
; CHECK-NEXT: Local Dynamic TLS Access Clean-up
; CHECK-NEXT: X86 PIC Global Base Reg Initialization
Rename ExpandISelPseudo->FinalizeISel, delay register reservation This allows targets to make more decisions about reserved registers after isel. For example, now it should be certain there are calls or stack objects in the frame or not, which could have been introduced by legalization. Patch by Matthias Braun llvm-svn: 363757
2019-06-19 08:25:39 +08:00
; CHECK-NEXT: Finalize ISel and expand pseudo-instructions
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; CHECK-NEXT: X86 Domain Reassignment Pass
[PGO][PGSO] Instrument the code gen / target passes. Summary: Split off of D67120. Add the profile guided size optimization instrumentation / queries in the code gen or target passes. This doesn't enable the size optimizations in those passes yet as they are currently disabled in shouldOptimizeForSize (for non-IR pass queries). A second try after reverted D71072. Reviewers: davidxl Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D71149
2019-12-06 01:39:37 +08:00
; CHECK-NEXT: Lazy Machine Block Frequency Analysis
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; CHECK-NEXT: Early Tail Duplication
; CHECK-NEXT: Optimize machine instruction PHIs
; CHECK-NEXT: Slot index numbering
; CHECK-NEXT: Merge disjoint stack slots
; CHECK-NEXT: Local Stack Slot Allocation
; CHECK-NEXT: Remove dead machine instructions
; CHECK-NEXT: MachineDominator Tree Construction
; CHECK-NEXT: Machine Natural Loop Construction
; CHECK-NEXT: Machine Trace Metrics
; CHECK-NEXT: Early If-Conversion
[PGO][PGSO] Instrument the code gen / target passes. Summary: Split off of D67120. Add the profile guided size optimization instrumentation / queries in the code gen or target passes. This doesn't enable the size optimizations in those passes yet as they are currently disabled in shouldOptimizeForSize (for non-IR pass queries). A second try after reverted D71072. Reviewers: davidxl Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D71149
2019-12-06 01:39:37 +08:00
; CHECK-NEXT: Lazy Machine Block Frequency Analysis
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; CHECK-NEXT: Machine InstCombiner
; CHECK-NEXT: X86 cmov Conversion
; CHECK-NEXT: MachineDominator Tree Construction
; CHECK-NEXT: Machine Natural Loop Construction
Disable hoisting MI to hotter basic blocks when using pgo This is a follow up patch for https://reviews.llvm.org/D63676 to enable the feature when using pgo. Differential Revision: https://reviews.llvm.org/D85240
2020-09-18 03:13:29 +08:00
; CHECK-NEXT: Machine Block Frequency Analysis
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; CHECK-NEXT: Early Machine Loop Invariant Code Motion
[Dominators][CodeGen] Don't mark MachineDominatorTree as preserved in MachineLICM llvm-svn: 373378
2019-10-02 02:27:44 +08:00
; CHECK-NEXT: MachineDominator Tree Construction
[MachineCSE][MachinePRE] Avoid hoisting code from code regions into hot BBs. Summary: Current PRE hoists common computations into CMBB = DT->findNearestCommonDominator(MBB, MBB1). However, if CMBB is in a hot loop body, we might get performance degradation. Differential Revision: https://reviews.llvm.org/D64394 llvm-svn: 366570
2019-07-19 20:58:16 +08:00
; CHECK-NEXT: Machine Block Frequency Analysis
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; CHECK-NEXT: Machine Common Subexpression Elimination
; CHECK-NEXT: MachinePostDominator Tree Construction
; CHECK-NEXT: Machine code sinking
; CHECK-NEXT: Peephole Optimizations
; CHECK-NEXT: Remove dead machine instructions
; CHECK-NEXT: Live Range Shrink
; CHECK-NEXT: X86 Fixup SetCC
[PGO][PGSO] Instrument the code gen / target passes. Summary: Split off of D67120. Add the profile guided size optimization instrumentation / queries in the code gen or target passes. This doesn't enable the size optimizations in those passes yet as they are currently disabled in shouldOptimizeForSize (for non-IR pass queries). A second try after reverted D71072. Reviewers: davidxl Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D71149
2019-12-06 01:39:37 +08:00
; CHECK-NEXT: Lazy Machine Block Frequency Analysis
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; CHECK-NEXT: X86 LEA Optimize
; CHECK-NEXT: X86 Optimize Call Frame
[X86] Reduce Store Forward Block issues in HW - Recommit after fixing Bug 36346 If a load follows a store and reloads data that the store has written to memory, Intel microarchitectures can in many cases forward the data directly from the store to the load, This "store forwarding" saves cycles by enabling the load to directly obtain the data instead of accessing the data from cache or memory. A "store forward block" occurs in cases that a store cannot be forwarded to the load. The most typical case of store forward block on Intel Core microarchiticutre that a small store cannot be forwarded to a large load. The estimated penalty for a store forward block is ~13 cycles. This pass tries to recognize and handle cases where "store forward block" is created by the compiler when lowering memcpy calls to a sequence of a load and a store. The pass currently only handles cases where memcpy is lowered to XMM/YMM registers, it tries to break the memcpy into smaller copies. breaking the memcpy should be possible since there is no atomicity guarantee for loads and stores to XMM/YMM. Differential revision: https://reviews.llvm.org/D41330 Change-Id: Ib48836ccdf6005989f7d4466fa2035b7b04415d9 llvm-svn: 328973
2018-04-02 21:48:28 +08:00
; CHECK-NEXT: X86 Avoid Store Forwarding Block
[x86/SLH] Add a real Clang flag and LLVM IR attribute for Speculative Load Hardening. Wires up the existing pass to work with a proper IR attribute rather than just a hidden/internal flag. The internal flag continues to work for now, but I'll likely remove it soon. Most of the churn here is adding the IR attribute. I talked about this Kristof Beyls and he seemed at least initially OK with this direction. The idea of using a full attribute here is that we *do* expect at least some forms of this for other architectures. There isn't anything *inherently* x86-specific about this technique, just that we only have an implementation for x86 at the moment. While we could potentially expose this as a Clang-level attribute as well, that seems like a good question to defer for the moment as it isn't 100% clear whether that or some other programmer interface (or both?) would be best. We'll defer the programmer interface side of this for now, but at least get to the point where the feature can be enabled without relying on implementation details. This also allows us to do something that was really hard before: we can enable *just* the indirect call retpolines when using SLH. For x86, we don't have any other way to mitigate indirect calls. Other architectures may take a different approach of course, and none of this is surfaced to user-level flags. Differential Revision: https://reviews.llvm.org/D51157 llvm-svn: 341363
2018-09-04 20:38:00 +08:00
; CHECK-NEXT: X86 speculative load hardening
[x86] Fix PR37100 by teaching the EFLAGS copy lowering to rewrite uses across basic blocks in the limited cases where it is very straight forward to do so. This will also be useful for other places where we do some limited EFLAGS propagation across CFG edges and need to handle copy rewrites afterward. I think this is rapidly approaching the maximum we can and should be doing here. Everything else begins to require either heroic analysis to prove how to do PHI insertion manually, or somehow managing arbitrary PHI-ing of EFLAGS with general PHI insertion. Neither of these seem at all promising so if those cases come up, we'll almost certainly need to rewrite the parts of LLVM that produce those patterns. We do now require dominator trees in order to reliably diagnose patterns that would require PHI nodes. This is a bit unfortunate but it seems better than the completely mysterious crash we would get otherwise. Differential Revision: https://reviews.llvm.org/D45673 llvm-svn: 330264
2018-04-18 23:13:16 +08:00
; CHECK-NEXT: MachineDominator Tree Construction
[x86] Introduce a pass to begin more systematically fixing PR36028 and similar issues. The key idea is to lower COPY nodes populating EFLAGS by scanning the uses of EFLAGS and introducing dedicated code to preserve the necessary state in a GPR. In the vast majority of cases, these uses are cmovCC and jCC instructions. For such cases, we can very easily save and restore the necessary information by simply inserting a setCC into a GPR where the original flags are live, and then testing that GPR directly to feed the cmov or conditional branch. However, things are a bit more tricky if arithmetic is using the flags. This patch handles the vast majority of cases that seem to come up in practice: adc, adcx, adox, rcl, and rcr; all without taking advantage of partially preserved EFLAGS as LLVM doesn't currently model that at all. There are a large number of operations that techinaclly observe EFLAGS currently but shouldn't in this case -- they typically are using DF. Currently, they will not be handled by this approach. However, I have never seen this issue come up in practice. It is already pretty rare to have these patterns come up in practical code with LLVM. I had to resort to writing MIR tests to cover most of the logic in this pass already. I suspect even with its current amount of coverage of arithmetic users of EFLAGS it will be a significant improvement over the current use of pushf/popf. It will also produce substantially faster code in most of the common patterns. This patch also removes all of the old lowering for EFLAGS copies, and the hack that forced us to use a frame pointer when EFLAGS copies were found anywhere in a function so that the dynamic stack adjustment wasn't a problem. None of this is needed as we now lower all of these copies directly in MI and without require stack adjustments. Lots of thanks to Reid who came up with several aspects of this approach, and Craig who helped me work out a couple of things tripping me up while working on this. Differential Revision: https://reviews.llvm.org/D45146 llvm-svn: 329657
2018-04-10 09:41:17 +08:00
; CHECK-NEXT: X86 EFLAGS copy lowering
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; CHECK-NEXT: X86 WinAlloca Expander
[X86] AMX programming model. This patch implements amx programming model that discussed in llvm-dev (http://lists.llvm.org/pipermail/llvm-dev/2020-August/144302.html). Thank Hal for the good suggestion in the RA. The fast RA is not in the patch yet. This patch implemeted 7 components. 1. The c interface to end user. 2. The AMX intrinsics in LLVM IR. 3. Transform load/store <256 x i32> to AMX intrinsics or split the type into two <128 x i32>. 4. The Lowering from AMX intrinsics to AMX pseudo instruction. 5. Insert psuedo ldtilecfg and build the def-use between ldtilecfg to amx intruction. 6. The register allocation for tile register. 7. Morph AMX pseudo instruction to AMX real instruction. Change-Id: I935e1080916ffcb72af54c2c83faa8b2e97d5cb0 Differential Revision: https://reviews.llvm.org/D87981
2020-09-06 10:17:22 +08:00
; CHECK-NEXT: MachineDominator Tree Construction
[X86][AMX] Hoist ldtilecfg The previous code calculated the first ldtilecfg by dominating all AMX registers' def. This may result in the ldtilecfg being inserted into a loop. This patch try to calculate the nearest point where all shapes of AMX registers are reachable. Reviewed By: LuoYuanke Differential Revision: https://reviews.llvm.org/D99010
2021-04-12 22:08:27 +08:00
; CHECK-NEXT: Machine Natural Loop Construction
[X86] AMX programming model. This patch implements amx programming model that discussed in llvm-dev (http://lists.llvm.org/pipermail/llvm-dev/2020-August/144302.html). Thank Hal for the good suggestion in the RA. The fast RA is not in the patch yet. This patch implemeted 7 components. 1. The c interface to end user. 2. The AMX intrinsics in LLVM IR. 3. Transform load/store <256 x i32> to AMX intrinsics or split the type into two <128 x i32>. 4. The Lowering from AMX intrinsics to AMX pseudo instruction. 5. Insert psuedo ldtilecfg and build the def-use between ldtilecfg to amx intruction. 6. The register allocation for tile register. 7. Morph AMX pseudo instruction to AMX real instruction. Change-Id: I935e1080916ffcb72af54c2c83faa8b2e97d5cb0 Differential Revision: https://reviews.llvm.org/D87981
2020-09-06 10:17:22 +08:00
; CHECK-NEXT: Tile Register Pre-configure
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; CHECK-NEXT: Detect Dead Lanes
; CHECK-NEXT: Process Implicit Definitions
; CHECK-NEXT: Remove unreachable machine basic blocks
; CHECK-NEXT: Live Variable Analysis
; CHECK-NEXT: Eliminate PHI nodes for register allocation
; CHECK-NEXT: Two-Address instruction pass
; CHECK-NEXT: Slot index numbering
; CHECK-NEXT: Live Interval Analysis
; CHECK-NEXT: Simple Register Coalescing
; CHECK-NEXT: Rename Disconnected Subregister Components
; CHECK-NEXT: Machine Instruction Scheduler
; CHECK-NEXT: Machine Block Frequency Analysis
; CHECK-NEXT: Debug Variable Analysis
; CHECK-NEXT: Live Stack Slot Analysis
; CHECK-NEXT: Virtual Register Map
; CHECK-NEXT: Live Register Matrix
; CHECK-NEXT: Bundle Machine CFG Edges
; CHECK-NEXT: Spill Code Placement Analysis
; CHECK-NEXT: Lazy Machine Block Frequency Analysis
; CHECK-NEXT: Machine Optimization Remark Emitter
; CHECK-NEXT: Greedy Register Allocator
[X86] AMX programming model. This patch implements amx programming model that discussed in llvm-dev (http://lists.llvm.org/pipermail/llvm-dev/2020-August/144302.html). Thank Hal for the good suggestion in the RA. The fast RA is not in the patch yet. This patch implemeted 7 components. 1. The c interface to end user. 2. The AMX intrinsics in LLVM IR. 3. Transform load/store <256 x i32> to AMX intrinsics or split the type into two <128 x i32>. 4. The Lowering from AMX intrinsics to AMX pseudo instruction. 5. Insert psuedo ldtilecfg and build the def-use between ldtilecfg to amx intruction. 6. The register allocation for tile register. 7. Morph AMX pseudo instruction to AMX real instruction. Change-Id: I935e1080916ffcb72af54c2c83faa8b2e97d5cb0 Differential Revision: https://reviews.llvm.org/D87981
2020-09-06 10:17:22 +08:00
; CHECK-NEXT: Tile Register Configure
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; CHECK-NEXT: Virtual Register Rewriter
; CHECK-NEXT: Stack Slot Coloring
; CHECK-NEXT: Machine Copy Propagation Pass
; CHECK-NEXT: Machine Loop Invariant Code Motion
[X86][AMX] Lower tile copy instruction. Since there is no tile copy instruction, we need to store tile register to stack and load from stack to another tile register. We need extra GR to hold the stride, and we need stack slot to hold the tile data register. We would run this pass after copy propagation, so that we don't miss copy optimization. And we would run this pass before prolog/epilog insertion, so that we can allocate stack slot. Differential Revision: https://reviews.llvm.org/D97112
2021-02-20 15:05:07 +08:00
; CHECK-NEXT: X86 Lower Tile Copy
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; CHECK-NEXT: Bundle Machine CFG Edges
; CHECK-NEXT: X86 FP Stackifier
[X86] Add a Pass that builds a Condensed CFG for Load Value Injection (LVI) Gadgets Adds a new data structure, ImmutableGraph, and uses RDF to find LVI gadgets and add them to a MachineGadgetGraph. More specifically, a new X86 machine pass finds Load Value Injection (LVI) gadgets consisting of a load from memory (i.e., SOURCE), and any operation that may transmit the value loaded from memory over a covert channel, or use the value loaded from memory to determine a branch/call target (i.e., SINK). Also adds a new target feature to X86: +lvi-load-hardening The feature can be added via the clang CLI using -mlvi-hardening. Differential Revision: https://reviews.llvm.org/D75936
2020-05-12 01:25:35 +08:00
; CHECK-NEXT: MachineDominator Tree Construction
; CHECK-NEXT: Machine Dominance Frontier Construction
; CHECK-NEXT: X86 Load Value Injection (LVI) Load Hardening
Re-land [Codegen/Statepoint] Allow usage of registers for non gc deopt values. The change introduces the usage of physical registers for non-gc deopt values. This require runtime support to know how to take a value from register. By default usage is off and can be switched on by option. The change also introduces additional fix-up patch which forces the spilling of caller saved registers (clobbered after the call) and re-writes statepoint to use spill slots instead of caller saved registers. Reviewers: reames, danstrushin Reviewed By: dantrushin Subscribers: mgorny, hiraditya, mgrang, llvm-commits Differential Revision: https://reviews.llvm.org/D77797
2020-04-09 19:40:53 +08:00
; CHECK-NEXT: Fixup Statepoint Caller Saved
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; CHECK-NEXT: PostRA Machine Sink
Re-land [Codegen/Statepoint] Allow usage of registers for non gc deopt values. The change introduces the usage of physical registers for non-gc deopt values. This require runtime support to know how to take a value from register. By default usage is off and can be switched on by option. The change also introduces additional fix-up patch which forces the spilling of caller saved registers (clobbered after the call) and re-writes statepoint to use spill slots instead of caller saved registers. Reviewers: reames, danstrushin Reviewed By: dantrushin Subscribers: mgorny, hiraditya, mgrang, llvm-commits Differential Revision: https://reviews.llvm.org/D77797
2020-04-09 19:40:53 +08:00
; CHECK-NEXT: Machine Block Frequency Analysis
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; CHECK-NEXT: MachinePostDominator Tree Construction
; CHECK-NEXT: Lazy Machine Block Frequency Analysis
; CHECK-NEXT: Machine Optimization Remark Emitter
[ShrinkWrap] Add optimization remarks to the shrink-wrapping pass Start by emitting remarks for very basic unsupported cases such as irreducible CFGs and EHFunclets. The end goal is to be able to cover all the cases where we give up with an explanation. llvm-svn: 333972
2018-06-05 08:27:24 +08:00
; CHECK-NEXT: Shrink Wrapping analysis
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; CHECK-NEXT: Prologue/Epilogue Insertion & Frame Finalization
; CHECK-NEXT: Control Flow Optimizer
[PGO][PGSO] Instrument the code gen / target passes. Summary: Split off of D67120. Add the profile guided size optimization instrumentation / queries in the code gen or target passes. This doesn't enable the size optimizations in those passes yet as they are currently disabled in shouldOptimizeForSize (for non-IR pass queries). A second try after reverted D71072. Reviewers: davidxl Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D71149
2019-12-06 01:39:37 +08:00
; CHECK-NEXT: Lazy Machine Block Frequency Analysis
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; CHECK-NEXT: Tail Duplication
; CHECK-NEXT: Machine Copy Propagation Pass
; CHECK-NEXT: Post-RA pseudo instruction expansion pass
; CHECK-NEXT: X86 pseudo instruction expansion pass
; CHECK-NEXT: MachineDominator Tree Construction
; CHECK-NEXT: Machine Natural Loop Construction
; CHECK-NEXT: Post RA top-down list latency scheduler
; CHECK-NEXT: Analyze Machine Code For Garbage Collection
; CHECK-NEXT: Machine Block Frequency Analysis
; CHECK-NEXT: MachinePostDominator Tree Construction
; CHECK-NEXT: Branch Probability Basic Block Placement
[CodeGen] Move fentry-insert, xray-instrumentation and patchable-function before addPreEmitPass() This intention is to move patchable-function before aarch64-branch-targets (configured in AArch64PassConfig::addPreEmitPass) so that we emit BTI before NOPs (see https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92424). This also allows addPreEmitPass() passes to know the precise instruction sizes if they want. Tried x86-64 Debug/Release builds of ccls with -fxray-instrument -fxray-instruction-threshold=1. No output difference with this commit and the previous commit.
2020-01-19 13:44:06 +08:00
; CHECK-NEXT: Insert fentry calls
; CHECK-NEXT: Insert XRay ops
; CHECK-NEXT: Implement the 'patchable-function' attribute
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; CHECK-NEXT: ReachingDefAnalysis
; CHECK-NEXT: X86 Execution Dependency Fix
; CHECK-NEXT: BreakFalseDeps
; CHECK-NEXT: X86 Indirect Branch Tracking
; CHECK-NEXT: X86 vzeroupper inserter
; CHECK-NEXT: MachineDominator Tree Construction
; CHECK-NEXT: Machine Natural Loop Construction
[PGO][PGSO] Instrument the code gen / target passes. Summary: Split off of D67120. Add the profile guided size optimization instrumentation / queries in the code gen or target passes. This doesn't enable the size optimizations in those passes yet as they are currently disabled in shouldOptimizeForSize (for non-IR pass queries). A second try after reverted D71072. Reviewers: davidxl Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D71149
2019-12-06 01:39:37 +08:00
; CHECK-NEXT: Lazy Machine Block Frequency Analysis
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; CHECK-NEXT: X86 Byte/Word Instruction Fixup
[PGO][PGSO] Instrument the code gen / target passes. Summary: Split off of D67120. Add the profile guided size optimization instrumentation / queries in the code gen or target passes. This doesn't enable the size optimizations in those passes yet as they are currently disabled in shouldOptimizeForSize (for non-IR pass queries). A second try after reverted D71072. Reviewers: davidxl Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D71149
2019-12-06 01:39:37 +08:00
; CHECK-NEXT: Lazy Machine Block Frequency Analysis
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; CHECK-NEXT: X86 Atom pad short functions
; CHECK-NEXT: X86 LEA Fixup
; CHECK-NEXT: Compressing EVEX instrs to VEX encoding when possible
Revert "Revert r347596 "Support for inserting profile-directed cache prefetches"" Summary: This reverts commit d8517b96dfbd42e6a8db33c50d1fa1e58e63fbb9. Fix: correct the use of DenseMap. Reviewers: davidxl, hans, wmi Reviewed By: wmi Subscribers: mgorny, eraman, llvm-commits Differential Revision: https://reviews.llvm.org/D55088 llvm-svn: 347938
2018-11-30 09:01:52 +08:00
; CHECK-NEXT: X86 Discriminate Memory Operands
; CHECK-NEXT: X86 Insert Cache Prefetches
Insert wait instruction after X87 instructions which could raise float-point exception. This patch also modify some mayRaiseFPException flag which set in D68854. Differential Revision: https://reviews.llvm.org/D72750
2020-01-16 10:49:59 +08:00
; CHECK-NEXT: X86 insert wait instruction
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; CHECK-NEXT: Contiguously Lay Out Funclets
; CHECK-NEXT: StackMap Liveness Analysis
; CHECK-NEXT: Live DEBUG_VALUE analysis
[x86][seses] Introduce SESES pass for LVI This is an implementation of Speculative Execution Side Effect Suppression which is intended as a last resort mitigation against Load Value Injection, LVI, a newly disclosed speculative execution side channel vulnerability. One pager: https://software.intel.com/security-software-guidance/software-guidance/load-value-injection Deep dive: https://software.intel.com/security-software-guidance/insights/deep-dive-load-value-injection The mitigation consists of a compiler pass that inserts an LFENCE before each memory read instruction, memory write instruction, and the first branch instruction in a group of terminators at the end of a basic block. The goal is to prevent speculative execution, potentially based on misspeculated conditions and/or containing secret data, from leaking that data via side channels embedded in such instructions. This is something of a last-resort mitigation: it is expected to have extreme performance implications and it may not be a complete mitigation due to trying to enumerate side channels. In addition to the full version of the mitigation, this patch implements three flags to turn off part of the mitigation. These flags are disabled by default. The flags are not intended to result in a secure variant of the mitigation. The flags are intended to be used by users who would like to experiment with improving the performance of the mitigation. I ran benchmarks with each of these flags enabled in order to find if there was any room for further optimization of LFENCE placement with respect to LVI. Performance Testing Results When applying this mitigation to BoringSSL, we see the following results. These are a summary/aggregation of the performance changes when this mitigation is applied versus when no mitigation is applied. Fully Mitigated vs Baseline Geometric mean 0.071 (Note: This can be read as the ops/s of the mitigated program was 7.1% of the ops/s of the unmitigated program.) Minimum 0.041 Quartile 1 0.060 Median 0.063 Quartile 3 0.077 Maximum 0.230 Reviewed By: george.burgess.iv Differential Revision: https://reviews.llvm.org/D75939
2020-05-12 00:33:55 +08:00
; CHECK-NEXT: X86 Speculative Execution Side Effect Suppression
[X86][NFC] Generalize the naming of "Retpoline Thunks" and related code to "Indirect Thunks" There are applications for indirect call/branch thunks other than retpoline for Spectre v2, e.g., https://software.intel.com/security-software-guidance/software-guidance/load-value-injection Therefore it makes sense to refactor X86RetpolineThunks as a more general capability. Differential Revision: https://reviews.llvm.org/D76810
2020-04-03 12:00:44 +08:00
; CHECK-NEXT: X86 Indirect Thunks
Correct dwarf unwind information in function epilogue This patch aims to provide correct dwarf unwind information in function epilogue for X86. It consists of two parts. The first part inserts CFI instructions that set appropriate cfa offset and cfa register in emitEpilogue() in X86FrameLowering. This part is X86 specific. The second part is platform independent and ensures that: * CFI instructions do not affect code generation (they are not counted as instructions when tail duplicating or tail merging) * Unwind information remains correct when a function is modified by different passes. This is done in a late pass by analyzing information about cfa offset and cfa register in BBs and inserting additional CFI directives where necessary. Added CFIInstrInserter pass: * analyzes each basic block to determine cfa offset and register are valid at its entry and exit * verifies that outgoing cfa offset and register of predecessor blocks match incoming values of their successors * inserts additional CFI directives at basic block beginning to correct the rule for calculating CFA Having CFI instructions in function epilogue can cause incorrect CFA calculation rule for some basic blocks. This can happen if, due to basic block reordering, or the existence of multiple epilogue blocks, some of the blocks have wrong cfa offset and register values set by the epilogue block above them. CFIInstrInserter is currently run only on X86, but can be used by any target that implements support for adding CFI instructions in epilogue. Patch by Violeta Vukobrat. Differential Revision: https://reviews.llvm.org/D42848 llvm-svn: 330706
2018-04-24 18:32:08 +08:00
; CHECK-NEXT: Check CFA info and insert CFI instructions if needed
[X86] Add RET-hardening Support to mitigate Load Value Injection (LVI) Adding a pass that replaces every ret instruction with the sequence: pop <scratch-reg> lfence jmp *<scratch-reg> where <scratch-reg> is some available scratch register, according to the calling convention of the function being mitigated. Differential Revision: https://reviews.llvm.org/D75935
2020-04-04 01:58:38 +08:00
; CHECK-NEXT: X86 Load Value Injection (LVI) Ret-Hardening
Reapply "[test] Add tests for llc passes pipelines." with a fix for bots with expensive checks on. llvm-svn: 328267
2018-03-23 07:02:48 +08:00
; CHECK-NEXT: Lazy Machine Block Frequency Analysis
; CHECK-NEXT: Machine Optimization Remark Emitter
; CHECK-NEXT: X86 Assembly Printer
; CHECK-NEXT: Free MachineFunction
define void @f() {
ret void
}