maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/llvm/lib/Target/X86/X86MCInstLower.cpp

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

2632 lines
97 KiB
C++
Raw Normal View History

split mcinst lowering stuff out to its own file. llvm-svn: 80806
2009-09-03 01:35:12 +08:00
//===-- X86MCInstLower.cpp - Convert X86 MachineInstr to an MCInst --------===//
//
Update the file headers across all of the LLVM projects in the monorepo to reflect the new license. We understand that people may be surprised that we're moving the header entirely to discuss the new license. We checked this carefully with the Foundation's lawyer and we believe this is the correct approach. Essentially, all code in the project is now made available by the LLVM project under our new license, so you will see that the license headers include that license only. Some of our contributors have contributed code under our old license, and accordingly, we have retained a copy of our old license notice in the top-level files in each project and repository. llvm-svn: 351636
2019-01-19 16:50:56 +08:00
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
split mcinst lowering stuff out to its own file. llvm-svn: 80806
2009-09-03 01:35:12 +08:00
//
//===----------------------------------------------------------------------===//
//
// This file contains code to lower X86 MachineInstrs to their corresponding
// MCInst records.
//
//===----------------------------------------------------------------------===//
[X86] Move InstPrinter files to MCTargetDesc. NFC For some targets, there is a circular dependency between InstPrinter and MCTargetDesc. Merging them together will fix this. For the other targets, the merging is to maintain consistency so all targets will have the same structure. llvm-svn: 360484
2019-05-11 07:24:38 +08:00
#include "MCTargetDesc/X86ATTInstPrinter.h"
Prune includes in X86 target. llvm-svn: 204216
2014-03-19 14:53:25 +08:00
#include "MCTargetDesc/X86BaseInfo.h"
[X86] Move InstPrinter files to MCTargetDesc. NFC For some targets, there is a circular dependency between InstPrinter and MCTargetDesc. Merging them together will fix this. For the other targets, the merging is to maintain consistency so all targets will have the same structure. llvm-svn: 360484
2019-05-11 07:24:38 +08:00
#include "MCTargetDesc/X86InstComments.h"
[X86] Move X86ShuffleDecode.cpp/h into MCTargetDesc and remove X86Utils library. NFC The shuffle decoding is used by X86ISelLowering and MCTargetDesc/X86InstComments. The latter used to be in a separate InstPrinter library. The Utils library existed to allow InstPrinter and CodeGen to share the shuffle decoding. Since X86InstComments now lives in the MCTargetDesc, which CodeGen already depends on, we can sink the shuffle decoding there as well. Differential Revision: https://reviews.llvm.org/D77980
2020-04-14 00:33:05 +08:00
#include "MCTargetDesc/X86ShuffleDecode.h"
[codeview] Implement FPO data assembler directives Summary: This adds a set of new directives that describe 32-bit x86 prologues. The directives are limited and do not expose the full complexity of codeview FPO data. They are merely a convenience for the compiler to generate more readable assembly so we don't need to generate tons of labels in CodeGen. If our prologue emission changes in the future, we can change the set of available directives to suit our needs. These are modelled after the .seh_ directives, which use a different format that interacts with exception handling. The directives are: .cv_fpo_proc _foo .cv_fpo_pushreg ebp/ebx/etc .cv_fpo_setframe ebp/esi/etc .cv_fpo_stackalloc 200 .cv_fpo_endprologue .cv_fpo_endproc .cv_fpo_data _foo I tried to follow the implementation of ARM EHABI CFI directives by sinking most directives out of MCStreamer and into X86TargetStreamer. This helps avoid polluting non-X86 code with WinCOFF specific logic. I used cdb to confirm that this can show locals in parent CSRs in a few cases, most importantly the one where we use ESI as a frame pointer, i.e. the one in http://crbug.com/756153#c28 Once we have cdb integration in debuginfo-tests, we can add integration tests there. Reviewers: majnemer, hans Subscribers: aemerson, mgorny, kristof.beyls, llvm-commits, hiraditya Differential Revision: https://reviews.llvm.org/D38776 llvm-svn: 315513
2017-10-12 05:24:33 +08:00
#include "MCTargetDesc/X86TargetStreamer.h"
Sort the remaining #include lines in include/... and lib/.... I did this a long time ago with a janky python script, but now clang-format has built-in support for this. I fed clang-format every line with a #include and let it re-sort things according to the precise LLVM rules for include ordering baked into clang-format these days. I've reverted a number of files where the results of sorting includes isn't healthy. Either places where we have legacy code relying on particular include ordering (where possible, I'll fix these separately) or where we have particular formatting around #include lines that I didn't want to disturb in this patch. This patch is *entirely* mechanical. If you get merge conflicts or anything, just ignore the changes in this patch and run clang-format over your #include lines in the files. Sorry for any noise here, but it is important to keep these things stable. I was seeing an increasing number of patches with irrelevant re-ordering of #include lines because clang-format was used. This patch at least isolates that churn, makes it easy to skip when resolving conflicts, and gets us to a clean baseline (again). llvm-svn: 304787
2017-06-06 19:49:48 +08:00
#include "X86AsmPrinter.h"
#include "X86RegisterInfo.h"
#include "X86ShuffleDecodeConstantPool.h"
X86AsmPrinter.h - cleanup includes and forward declarations. NFC. Reduce X86Subtarget.h/MCCodeEmitter.h/TargetMachine.h includes to forward declarations Add explicit X86Subtarget.h/TargetMachine.h includes to X86AsmPrinter.cpp/X86MCInstLower.cpp Remove unused MCSymbol forward declaration
2020-04-19 18:38:50 +08:00
#include "X86Subtarget.h"
[NFC] Extract X86MCInstLower::LowerMachineOperand. Summary: Refactoring-only change that will be used later. Reviewers: reames, atrick Reviewed By: atrick Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D10196 llvm-svn: 239739
2015-06-16 02:44:01 +08:00
#include "llvm/ADT/Optional.h"
Use the new script to sort the includes of every file under lib. Sooooo many of these had incorrect or strange main module includes. I have manually inspected all of these, and fixed the main module include to be the nearest plausible thing I could find. If you own or care about any of these source files, I encourage you to take some time and check that these edits were sensible. I can't have broken anything (I strictly added headers, and reordered them, never removed), but they may not be the headers you'd really like to identify as containing the API being implemented. Many forward declarations and missing includes were added to a header files to allow them to parse cleanly when included first. The main module rule does in fact have its merits. =] llvm-svn: 169131
2012-12-04 00:50:05 +08:00
#include "llvm/ADT/SmallString.h"
Introduce a "patchable-function" function attribute Summary: The `"patchable-function"` attribute can be used by an LLVM client to influence LLVM's code generation in ways that makes the generated code easily patchable at runtime (for instance, to redirect control). Right now only one patchability scheme is supported, `"prologue-short-redirect"`, but this can be expanded in the future. Reviewers: joker.eph, rnk, echristo, dberris Subscribers: joker.eph, echristo, mcrosier, llvm-commits Differential Revision: http://reviews.llvm.org/D19046 llvm-svn: 266715
2016-04-19 13:24:47 +08:00
#include "llvm/ADT/iterator_range.h"
[x86] Teach the X86 backend to print shuffle comments for PSHUFB instructions which happen to have a constant mask. Currently, this only handles a very narrow set of cases, but those happen to be the cases that I care about for testing shuffles sanely. This is a bit trickier than other shuffle instructions because we're decoding constants out of the constant pool. The current MC layer makes it completely impossible to inspect a constant pool entry, so we have to do it at the MI level and attach the comment to the streamer on its way out. So no joy for disassembling, but it does make test cases and asm dumps *much* nicer. Sorry for no test cases, but it didn't really seem that valuable to go trolling through existing old test cases and updating them. I'll have lots of testing of this in the upcoming patch for SSSE3 emission in the new vector shuffle lowering code paths. llvm-svn: 213986
2014-07-26 07:47:11 +08:00
#include "llvm/CodeGen/MachineConstantPool.h"
Sort the remaining #include lines in include/... and lib/.... I did this a long time ago with a janky python script, but now clang-format has built-in support for this. I fed clang-format every line with a #include and let it re-sort things according to the precise LLVM rules for include ordering baked into clang-format these days. I've reverted a number of files where the results of sorting includes isn't healthy. Either places where we have legacy code relying on particular include ordering (where possible, I'll fix these separately) or where we have particular formatting around #include lines that I didn't want to disturb in this patch. This patch is *entirely* mechanical. If you get merge conflicts or anything, just ignore the changes in this patch and run clang-format over your #include lines in the files. Sorry for any noise here, but it is important to keep these things stable. I was seeing an increasing number of patches with irrelevant re-ordering of #include lines because clang-format was used. This patch at least isolates that churn, makes it easy to skip when resolving conflicts, and gets us to a clean baseline (again). llvm-svn: 304787
2017-06-06 19:49:48 +08:00
#include "llvm/CodeGen/MachineFunction.h"
move FnStubs/GVSTubs/HiddenGVStub handling out of the X86 asmprinter and use MachineModuleInfoMachO instead. llvm-svn: 82022
2009-09-16 14:25:03 +08:00
#include "llvm/CodeGen/MachineModuleInfoImpls.h"
Sort the remaining #include lines in include/... and lib/.... I did this a long time ago with a janky python script, but now clang-format has built-in support for this. I fed clang-format every line with a #include and let it re-sort things according to the precise LLVM rules for include ordering baked into clang-format these days. I've reverted a number of files where the results of sorting includes isn't healthy. Either places where we have legacy code relying on particular include ordering (where possible, I'll fix these separately) or where we have particular formatting around #include lines that I didn't want to disturb in this patch. This patch is *entirely* mechanical. If you get merge conflicts or anything, just ignore the changes in this patch and run clang-format over your #include lines in the files. Sorry for any noise here, but it is important to keep these things stable. I was seeing an increasing number of patches with irrelevant re-ordering of #include lines because clang-format was used. This patch at least isolates that churn, makes it easy to skip when resolving conflicts, and gets us to a clean baseline (again). llvm-svn: 304787
2017-06-06 19:49:48 +08:00
#include "llvm/CodeGen/MachineOperand.h"
Add support for stack map generation in the X86 backend. Originally implemented by Lang Hames. llvm-svn: 193811
2013-11-01 06:11:56 +08:00
#include "llvm/CodeGen/StackMaps.h"
Prune includes in X86 target. llvm-svn: 204216
2014-03-19 14:53:25 +08:00
#include "llvm/IR/DataLayout.h"
#include "llvm/IR/GlobalValue.h"
Move the llvm mangler to lib/IR. This makes it available to tools that don't link with target (like llvm-ar). llvm-svn: 198708
2014-01-08 05:19:40 +08:00
#include "llvm/IR/Mangler.h"
Rename createAsmInfo to createMCAsmInfo and move registration code to MCTargetDesc to prepare for next round of changes. llvm-svn: 135219
2011-07-15 07:50:31 +08:00
#include "llvm/MC/MCAsmInfo.h"
[X86] Optimize stackmap shadows on X86. This patch minimizes the number of nops that must be emitted on X86 to satisfy stackmap shadow constraints. To minimize the number of nops inserted, the X86AsmPrinter now records the size of the most recent stackmap's shadow in the StackMapShadowTracker class, and tracks the number of instruction bytes emitted since the that stackmap instruction was encountered. Padding is emitted (if it is required at all) immediately before the next stackmap/patchpoint instruction, or at the end of the basic block. This optimization should reduce code-size and improve performance for people using the llvm stackmap intrinsic on X86. <rdar://problem/14959522> llvm-svn: 213892
2014-07-25 04:40:55 +08:00
#include "llvm/MC/MCCodeEmitter.h"
split mcinst lowering stuff out to its own file. llvm-svn: 80806
2009-09-03 01:35:12 +08:00
#include "llvm/MC/MCContext.h"
#include "llvm/MC/MCExpr.h"
Remove MCAssembler.h include from MCStreamer.h and fix users of MCStreamer.h llvm-svn: 237483
2015-05-16 06:19:42 +08:00
#include "llvm/MC/MCFixup.h"
split mcinst lowering stuff out to its own file. llvm-svn: 80806
2009-09-03 01:35:12 +08:00
#include "llvm/MC/MCInst.h"
Add MCInstBuilder, a utility class to simplify MCInst creation similar to MachineInstrBuilder. Simplify some repetitive code with it. No functionality change. llvm-svn: 168587
2012-11-26 21:34:22 +08:00
#include "llvm/MC/MCInstBuilder.h"
XRay: Add entry and exit sleds Summary: In this patch we implement the following parts of XRay: - Supporting a function attribute named 'function-instrument' which currently only supports 'xray-always'. We should be able to use this attribute for other instrumentation approaches. - Supporting a function attribute named 'xray-instruction-threshold' used to determine whether a function is instrumented with a minimum number of instructions (IR instruction counts). - X86-specific nop sleds as described in the white paper. - A machine function pass that adds the different instrumentation marker instructions at a very late stage. - A way of identifying which return opcode is considered "normal" for each architecture. There are some caveats here: 1) We don't handle PATCHABLE_RET in platforms other than x86_64 yet -- this means if IR used PATCHABLE_RET directly instead of a normal ret, instruction lowering for that platform might do the wrong thing. We think this should be handled at instruction selection time to by default be unpacked for platforms where XRay is not availble yet. 2) The generated section for X86 is different from what is described from the white paper for the sole reason that LLVM allows us to do this neatly. We're taking the opportunity to deviate from the white paper from this perspective to allow us to get richer information from the runtime library. Reviewers: sanjoy, eugenis, kcc, pcc, echristo, rnk Subscribers: niravd, majnemer, atrick, rnk, emaste, bmakam, mcrosier, mehdi_amini, llvm-commits Differential Revision: http://reviews.llvm.org/D19904 llvm-svn: 275367
2016-07-14 12:06:33 +08:00
#include "llvm/MC/MCSection.h"
Sort the remaining #include lines in include/... and lib/.... I did this a long time ago with a janky python script, but now clang-format has built-in support for this. I fed clang-format every line with a #include and let it re-sort things according to the precise LLVM rules for include ordering baked into clang-format these days. I've reverted a number of files where the results of sorting includes isn't healthy. Either places where we have legacy code relying on particular include ordering (where possible, I'll fix these separately) or where we have particular formatting around #include lines that I didn't want to disturb in this patch. This patch is *entirely* mechanical. If you get merge conflicts or anything, just ignore the changes in this patch and run clang-format over your #include lines in the files. Sorry for any noise here, but it is important to keep these things stable. I was seeing an increasing number of patches with irrelevant re-ordering of #include lines because clang-format was used. This patch at least isolates that churn, makes it easy to skip when resolving conflicts, and gets us to a clean baseline (again). llvm-svn: 304787
2017-06-06 19:49:48 +08:00
#include "llvm/MC/MCSectionELF.h"
split mcinst lowering stuff out to its own file. llvm-svn: 80806
2009-09-03 01:35:12 +08:00
#include "llvm/MC/MCStreamer.h"
eliminate the string form of DecorateCygMingName llvm-svn: 98373
2010-03-13 03:42:40 +08:00
#include "llvm/MC/MCSymbol.h"
XRay: Add entry and exit sleds Summary: In this patch we implement the following parts of XRay: - Supporting a function attribute named 'function-instrument' which currently only supports 'xray-always'. We should be able to use this attribute for other instrumentation approaches. - Supporting a function attribute named 'xray-instruction-threshold' used to determine whether a function is instrumented with a minimum number of instructions (IR instruction counts). - X86-specific nop sleds as described in the white paper. - A machine function pass that adds the different instrumentation marker instructions at a very late stage. - A way of identifying which return opcode is considered "normal" for each architecture. There are some caveats here: 1) We don't handle PATCHABLE_RET in platforms other than x86_64 yet -- this means if IR used PATCHABLE_RET directly instead of a normal ret, instruction lowering for that platform might do the wrong thing. We think this should be handled at instruction selection time to by default be unpacked for platforms where XRay is not availble yet. 2) The generated section for X86 is different from what is described from the white paper for the sole reason that LLVM allows us to do this neatly. We're taking the opportunity to deviate from the white paper from this perspective to allow us to get richer information from the runtime library. Reviewers: sanjoy, eugenis, kcc, pcc, echristo, rnk Subscribers: niravd, majnemer, atrick, rnk, emaste, bmakam, mcrosier, mehdi_amini, llvm-commits Differential Revision: http://reviews.llvm.org/D19904 llvm-svn: 275367
2016-07-14 12:06:33 +08:00
#include "llvm/MC/MCSymbolELF.h"
Move TargetLoweringObjectFile from CodeGen to Target to fix layering It's implemented in Target & include from other Target headers, so the header should be in Target. llvm-svn: 328392
2018-03-24 07:58:19 +08:00
#include "llvm/Target/TargetLoweringObjectFile.h"
X86Subtarget.h - remove unnecessary TargetMachine.h include. NFC. By moving X86Subtarget::isPositionIndependent() into X86Subtarget.cpp we can remove the header dependency and move the few uses into source files.
2020-05-24 19:30:22 +08:00
#include "llvm/Target/TargetMachine.h"
XRay: Add entry and exit sleds Summary: In this patch we implement the following parts of XRay: - Supporting a function attribute named 'function-instrument' which currently only supports 'xray-always'. We should be able to use this attribute for other instrumentation approaches. - Supporting a function attribute named 'xray-instruction-threshold' used to determine whether a function is instrumented with a minimum number of instructions (IR instruction counts). - X86-specific nop sleds as described in the white paper. - A machine function pass that adds the different instrumentation marker instructions at a very late stage. - A way of identifying which return opcode is considered "normal" for each architecture. There are some caveats here: 1) We don't handle PATCHABLE_RET in platforms other than x86_64 yet -- this means if IR used PATCHABLE_RET directly instead of a normal ret, instruction lowering for that platform might do the wrong thing. We think this should be handled at instruction selection time to by default be unpacked for platforms where XRay is not availble yet. 2) The generated section for X86 is different from what is described from the white paper for the sole reason that LLVM allows us to do this neatly. We're taking the opportunity to deviate from the white paper from this perspective to allow us to get richer information from the runtime library. Reviewers: sanjoy, eugenis, kcc, pcc, echristo, rnk Subscribers: niravd, majnemer, atrick, rnk, emaste, bmakam, mcrosier, mehdi_amini, llvm-commits Differential Revision: http://reviews.llvm.org/D19904 llvm-svn: 275367
2016-07-14 12:06:33 +08:00
split mcinst lowering stuff out to its own file. llvm-svn: 80806
2009-09-03 01:35:12 +08:00
using namespace llvm;
Move X86MCInstLower class definition into implementation file. It's not needed outside. llvm-svn: 166014
2012-10-16 14:01:50 +08:00
namespace {
/// X86MCInstLower - This class is used to lower an MachineInstr into an MCInst.
class X86MCInstLower {
MCContext &Ctx;
const MachineFunction &MF;
const TargetMachine &TM;
const MCAsmInfo &MAI;
X86AsmPrinter &AsmPrinter;
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
Move X86MCInstLower class definition into implementation file. It's not needed outside. llvm-svn: 166014
2012-10-16 14:01:50 +08:00
public:
The asm printer has a mangler. Don't keep a second pointer to it. llvm-svn: 193616
2013-10-30 00:11:22 +08:00
X86MCInstLower(const MachineFunction &MF, X86AsmPrinter &asmprinter);
Move X86MCInstLower class definition into implementation file. It's not needed outside. llvm-svn: 166014
2012-10-16 14:01:50 +08:00
[NFC] Extract X86MCInstLower::LowerMachineOperand. Summary: Refactoring-only change that will be used later. Reviewers: reames, atrick Reviewed By: atrick Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D10196 llvm-svn: 239739
2015-06-16 02:44:01 +08:00
Optional<MCOperand> LowerMachineOperand(const MachineInstr *MI,
const MachineOperand &MO) const;
Move X86MCInstLower class definition into implementation file. It's not needed outside. llvm-svn: 166014
2012-10-16 14:01:50 +08:00
void Lower(const MachineInstr *MI, MCInst &OutMI) const;
MCSymbol *GetSymbolFromOperand(const MachineOperand &MO) const;
MCOperand LowerSymbolOperand(const MachineOperand &MO, MCSymbol *Sym) const;
private:
MachineModuleInfoMachO &getMachOMMI() const;
};
} // end anonymous namespace
[X86][BranchAlign] Suppress branch alignment for {,_}__tls_get_addr The x86-64 General Dynamic TLS code sequence uses prefixes to allow linker relaxation. Adding segment override prefix or NOPs can break linker relaxation (ld -pie/-no-pie). i386 General Dynamic and x86-64 Local Dynamic do not use prefixes, but for simplicity, just disable auto padding consistently. Reviewed By: skan, LuoYuanke Differential Revision: https://reviews.llvm.org/D72878
2020-01-17 06:08:31 +08:00
/// A RAII helper which defines a region of instructions which can't have
/// padding added between them for correctness.
struct NoAutoPaddingScope {
MCStreamer &OS;
const bool OldAllowAutoPadding;
NoAutoPaddingScope(MCStreamer &OS)
: OS(OS), OldAllowAutoPadding(OS.getAllowAutoPadding()) {
changeAndComment(false);
}
~NoAutoPaddingScope() { changeAndComment(OldAllowAutoPadding); }
void changeAndComment(bool b) {
if (b == OS.getAllowAutoPadding())
return;
OS.setAllowAutoPadding(b);
if (b)
OS.emitRawComment("autopadding");
else
OS.emitRawComment("noautopadding");
}
};
[X86] Optimize stackmap shadows on X86. This patch minimizes the number of nops that must be emitted on X86 to satisfy stackmap shadow constraints. To minimize the number of nops inserted, the X86AsmPrinter now records the size of the most recent stackmap's shadow in the StackMapShadowTracker class, and tracks the number of instruction bytes emitted since the that stackmap instruction was encountered. Padding is emitted (if it is required at all) immediately before the next stackmap/patchpoint instruction, or at the end of the basic block. This optimization should reduce code-size and improve performance for people using the llvm stackmap intrinsic on X86. <rdar://problem/14959522> llvm-svn: 213892
2014-07-25 04:40:55 +08:00
// Emit a minimal sequence of nops spanning NumBytes bytes.
[X86] Change signature of EmitNops. NFC. This is to support https://reviews.llvm.org/D81301.
2020-06-18 00:08:12 +08:00
static void emitX86Nops(MCStreamer &OS, unsigned NumBytes,
const X86Subtarget *Subtarget);
[X86] Optimize stackmap shadows on X86. This patch minimizes the number of nops that must be emitted on X86 to satisfy stackmap shadow constraints. To minimize the number of nops inserted, the X86AsmPrinter now records the size of the most recent stackmap's shadow in the StackMapShadowTracker class, and tracks the number of instruction bytes emitted since the that stackmap instruction was encountered. Padding is emitted (if it is required at all) immediately before the next stackmap/patchpoint instruction, or at the end of the basic block. This optimization should reduce code-size and improve performance for people using the llvm stackmap intrinsic on X86. <rdar://problem/14959522> llvm-svn: 213892
2014-07-25 04:40:55 +08:00
[X86] Simplify StackMapShadowTracker; NFC - Elide trivial contructor and desctructor - Move implementation out of an unnecessary explicit llvm namespace scope llvm-svn: 266794
2016-04-20 02:48:16 +08:00
void X86AsmPrinter::StackMapShadowTracker::count(MCInst &Inst,
const MCSubtargetInfo &STI,
MCCodeEmitter *CodeEmitter) {
if (InShadow) {
SmallString<256> Code;
SmallVector<MCFixup, 4> Fixups;
raw_svector_ostream VecOS(Code);
CodeEmitter->encodeInstruction(Inst, VecOS, Fixups, STI);
CurrentShadowSize += Code.size();
if (CurrentShadowSize >= RequiredShadowSize)
InShadow = false; // The shadow is big enough. Stop counting.
[X86] Optimize stackmap shadows on X86. This patch minimizes the number of nops that must be emitted on X86 to satisfy stackmap shadow constraints. To minimize the number of nops inserted, the X86AsmPrinter now records the size of the most recent stackmap's shadow in the StackMapShadowTracker class, and tracks the number of instruction bytes emitted since the that stackmap instruction was encountered. Padding is emitted (if it is required at all) immediately before the next stackmap/patchpoint instruction, or at the end of the basic block. This optimization should reduce code-size and improve performance for people using the llvm stackmap intrinsic on X86. <rdar://problem/14959522> llvm-svn: 213892
2014-07-25 04:40:55 +08:00
}
[X86] Simplify StackMapShadowTracker; NFC - Elide trivial contructor and desctructor - Move implementation out of an unnecessary explicit llvm namespace scope llvm-svn: 266794
2016-04-20 02:48:16 +08:00
}
[X86] Optimize stackmap shadows on X86. This patch minimizes the number of nops that must be emitted on X86 to satisfy stackmap shadow constraints. To minimize the number of nops inserted, the X86AsmPrinter now records the size of the most recent stackmap's shadow in the StackMapShadowTracker class, and tracks the number of instruction bytes emitted since the that stackmap instruction was encountered. Padding is emitted (if it is required at all) immediately before the next stackmap/patchpoint instruction, or at the end of the basic block. This optimization should reduce code-size and improve performance for people using the llvm stackmap intrinsic on X86. <rdar://problem/14959522> llvm-svn: 213892
2014-07-25 04:40:55 +08:00
[X86] Simplify StackMapShadowTracker; NFC - Elide trivial contructor and desctructor - Move implementation out of an unnecessary explicit llvm namespace scope llvm-svn: 266794
2016-04-20 02:48:16 +08:00
void X86AsmPrinter::StackMapShadowTracker::emitShadowPadding(
[X86] Optimize stackmap shadows on X86. This patch minimizes the number of nops that must be emitted on X86 to satisfy stackmap shadow constraints. To minimize the number of nops inserted, the X86AsmPrinter now records the size of the most recent stackmap's shadow in the StackMapShadowTracker class, and tracks the number of instruction bytes emitted since the that stackmap instruction was encountered. Padding is emitted (if it is required at all) immediately before the next stackmap/patchpoint instruction, or at the end of the basic block. This optimization should reduce code-size and improve performance for people using the llvm stackmap intrinsic on X86. <rdar://problem/14959522> llvm-svn: 213892
2014-07-25 04:40:55 +08:00
MCStreamer &OutStreamer, const MCSubtargetInfo &STI) {
[X86] Simplify StackMapShadowTracker; NFC - Elide trivial contructor and desctructor - Move implementation out of an unnecessary explicit llvm namespace scope llvm-svn: 266794
2016-04-20 02:48:16 +08:00
if (InShadow && CurrentShadowSize < RequiredShadowSize) {
InShadow = false;
[X86] Change signature of EmitNops. NFC. This is to support https://reviews.llvm.org/D81301.
2020-06-18 00:08:12 +08:00
emitX86Nops(OutStreamer, RequiredShadowSize - CurrentShadowSize,
&MF->getSubtarget<X86Subtarget>());
[X86] Optimize stackmap shadows on X86. This patch minimizes the number of nops that must be emitted on X86 to satisfy stackmap shadow constraints. To minimize the number of nops inserted, the X86AsmPrinter now records the size of the most recent stackmap's shadow in the StackMapShadowTracker class, and tracks the number of instruction bytes emitted since the that stackmap instruction was encountered. Padding is emitted (if it is required at all) immediately before the next stackmap/patchpoint instruction, or at the end of the basic block. This optimization should reduce code-size and improve performance for people using the llvm stackmap intrinsic on X86. <rdar://problem/14959522> llvm-svn: 213892
2014-07-25 04:40:55 +08:00
}
[X86] Simplify StackMapShadowTracker; NFC - Elide trivial contructor and desctructor - Move implementation out of an unnecessary explicit llvm namespace scope llvm-svn: 266794
2016-04-20 02:48:16 +08:00
}
[X86] Optimize stackmap shadows on X86. This patch minimizes the number of nops that must be emitted on X86 to satisfy stackmap shadow constraints. To minimize the number of nops inserted, the X86AsmPrinter now records the size of the most recent stackmap's shadow in the StackMapShadowTracker class, and tracks the number of instruction bytes emitted since the that stackmap instruction was encountered. Padding is emitted (if it is required at all) immediately before the next stackmap/patchpoint instruction, or at the end of the basic block. This optimization should reduce code-size and improve performance for people using the llvm stackmap intrinsic on X86. <rdar://problem/14959522> llvm-svn: 213892
2014-07-25 04:40:55 +08:00
[X86] Simplify StackMapShadowTracker; NFC - Elide trivial contructor and desctructor - Move implementation out of an unnecessary explicit llvm namespace scope llvm-svn: 266794
2016-04-20 02:48:16 +08:00
void X86AsmPrinter::EmitAndCountInstruction(MCInst &Inst) {
[AsmPrinter][MCStreamer] De-capitalize EmitInstruction and EmitCFI*
2020-02-14 13:58:16 +08:00
OutStreamer->emitInstruction(Inst, getSubtargetInfo());
[X86] Simplify StackMapShadowTracker; NFC - Elide trivial contructor and desctructor - Move implementation out of an unnecessary explicit llvm namespace scope llvm-svn: 266794
2016-04-20 02:48:16 +08:00
SMShadowTracker.count(Inst, getSubtargetInfo(), CodeEmitter.get());
}
[X86] Optimize stackmap shadows on X86. This patch minimizes the number of nops that must be emitted on X86 to satisfy stackmap shadow constraints. To minimize the number of nops inserted, the X86AsmPrinter now records the size of the most recent stackmap's shadow in the StackMapShadowTracker class, and tracks the number of instruction bytes emitted since the that stackmap instruction was encountered. Padding is emitted (if it is required at all) immediately before the next stackmap/patchpoint instruction, or at the end of the basic block. This optimization should reduce code-size and improve performance for people using the llvm stackmap intrinsic on X86. <rdar://problem/14959522> llvm-svn: 213892
2014-07-25 04:40:55 +08:00
The asm printer has a mangler. Don't keep a second pointer to it. llvm-svn: 193616
2013-10-30 00:11:22 +08:00
X86MCInstLower::X86MCInstLower(const MachineFunction &mf,
X86MCInstLower now depends on AsmPrinter being around. llvm-svn: 109154
2010-07-23 05:10:04 +08:00
X86AsmPrinter &asmprinter)
Reuse a bunch of cached subtargets and remove getSubtarget calls without a Function argument. llvm-svn: 227814
2015-02-03 01:38:43 +08:00
: Ctx(mf.getContext()), MF(mf), TM(mf.getTarget()), MAI(*TM.getMCAsmInfo()),
AsmPrinter(asmprinter) {}
split MachineInstr -> MCInst lowering into its own class (not being embedded into X86ATTAsmPrinter). This still depends heavily on X86ATTAsmPrinter, but this is a step in the right direction. llvm-svn: 81627
2009-09-13 04:34:57 +08:00
move FnStubs/GVSTubs/HiddenGVStub handling out of the X86 asmprinter and use MachineModuleInfoMachO instead. llvm-svn: 82022
2009-09-16 14:25:03 +08:00
MachineModuleInfoMachO &X86MCInstLower::getMachOMMI() const {
pass around MF, not MMI. llvm-svn: 108949
2010-07-21 06:26:07 +08:00
return MF.getMMI().getObjFileInfo<MachineModuleInfoMachO>();
move FnStubs/GVSTubs/HiddenGVStub handling out of the X86 asmprinter and use MachineModuleInfoMachO instead. llvm-svn: 82022
2009-09-16 14:25:03 +08:00
}
unify the paths for external symbols and global variables: 2 files changed, 48 insertions(+), 83 deletions(-) llvm-svn: 95599
2010-02-09 07:03:41 +08:00
/// GetSymbolFromOperand - Lower an MO_GlobalAddress or MO_ExternalSymbol
/// operand to an MCSymbol.
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
MCSymbol *X86MCInstLower::GetSymbolFromOperand(const MachineOperand &MO) const {
[X86][ELF] Prefer to lower MC_GlobalAddress operands to .Lfoo$local For a MC_GlobalAddress reference to a dso_local external GlobalValue with a definition, emit .Lfoo$local to avoid a relocation. -fno-pic and -fpie can infer dso_local but -fpic cannot. In the future, we can explore the possibility of inferring dso_local with -fpic. As the description of D73228 says, LLVM's existing IPO optimization behaviors (like -fno-semantic-interposition) and a previous assembly behavior give us enough license to be aggressive here. Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D73230
2020-01-23 04:27:17 +08:00
const Triple &TT = TM.getTargetTriple();
if (MO.isGlobal() && TT.isOSBinFormatELF())
return AsmPrinter.getSymbolPreferLocal(*MO.getGlobal());
Move most user of TargetMachine::getDataLayout to the Module one Summary: This change is part of a series of commits dedicated to have a single DataLayout during compilation by using always the one owned by the module. This patch is quite boring overall, except for some uglyness in ASMPrinter which has a getDataLayout function but has some clients that use it without a Module (llmv-dsymutil, llvm-dwarfdump), so some methods are taking a DataLayout as parameter. Reviewers: echristo Subscribers: yaron.keren, rafael, llvm-commits, jholewinski Differential Revision: http://reviews.llvm.org/D11090 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 242386
2015-07-16 14:11:10 +08:00
const DataLayout &DL = MF.getDataLayout();
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
assert((MO.isGlobal() || MO.isSymbol() || MO.isMBB()) &&
"Isn't a symbol reference");
unify the paths for external symbols and global variables: 2 files changed, 48 insertions(+), 83 deletions(-) llvm-svn: 95599
2010-02-09 07:03:41 +08:00
Avoid a call to getOrCreateSymbol when we already have the symbol. llvm-svn: 238890
2015-06-03 08:02:40 +08:00
MCSymbol *Sym = nullptr;
reimplement X86ATTAsmPrinter::GetGlobalAddressSymbol in terms of Mangler::getNameWithPrefix. In addition to avoiding some over quoting, this also is more efficient because it uses smallvector instead of std::string thrashing. llvm-svn: 81508
2009-09-11 13:58:44 +08:00
SmallString<128> Name;
Refactor to remove a bit of duplication. No functionality change. llvm-svn: 195933
2013-11-29 04:12:44 +08:00
StringRef Suffix;
switch (MO.getTargetFlags()) {
Revert "Move dllimport name mangling to IR mangler." This reverts commit r239437. This broke clang-cl self-hosts. We'd end up calling the __imp_ symbol directly instead of using it to do an indirect function call. llvm-svn: 239502
2015-06-11 09:31:48 +08:00
case X86II::MO_DLLIMPORT:
// Handle dllimport linkage.
Name += "__imp_";
break;
[MinGW] [X86] Add stubs for references to data variables that might end up imported from a dll Variables declared with the dllimport attribute are accessed via a stub variable named __imp_<var>. In MinGW configurations, variables that aren't declared with a dllimport attribute might still end up imported from another DLL with runtime pseudo relocs. For x86_64, this avoids the risk that the target is out of range for a 32 bit PC relative reference, in case the target DLL is loaded further than 4 GB from the reference. It also avoids having to make the text section writable at runtime when doing the runtime fixups, which makes it worthwhile to do for i386 as well. Add stub variables for all dso local data references where a definition of the variable isn't visible within the module, since the DLL data autoimporting might make them imported even though they are marked as dso local within LLVM. Don't do this for variables that actually are defined within the same module, since we then know for sure that it actually is dso local. Don't do this for references to functions, since there's no need for runtime pseudo relocations for autoimporting them; if a function from a different DLL is called without the appropriate dllimport attribute, the call just gets routed via a thunk instead. GCC does something similar since 4.9 (when compiling with -mcmodel=medium or large; from that version, medium is the default code model for x86_64 mingw), but only for x86_64. Differential Revision: https://reviews.llvm.org/D51288 llvm-svn: 340942
2018-08-30 01:28:34 +08:00
case X86II::MO_COFFSTUB:
Name += ".refptr.";
break;
Refactor to remove a bit of duplication. No functionality change. llvm-svn: 195933
2013-11-29 04:12:44 +08:00
case X86II::MO_DARWIN_NONLAZY:
case X86II::MO_DARWIN_NONLAZY_PIC_BASE:
Suffix = "$non_lazy_ptr";
break;
}
Whitespace. llvm-svn: 161122
2012-08-02 02:39:17 +08:00
Hide the stub created for MO_ExternalSymbol too. given declare void @llvm.memset.p0i8.i32(i8* nocapture, i8, i32, i32, i1) declare void @foo() define void @bar() { call void @foo() call void @llvm.memset.p0i8.i32(i8* null, i8 0, i32 188, i32 1, i1 false) ret void } We used to produce L_foo$stub: .indirect_symbol _foo .ascii "\364\364\364\364\364" _memset$stub: .indirect_symbol _memset .ascii "\364\364\364\364\364" We not produce a private stub for memset too. Stubs are not needed with recent linkers, but we still produce them for darwin8. Thanks to David Fang for confirming that gcc used to do this too. llvm-svn: 196468
2013-12-05 13:19:12 +08:00
if (!Suffix.empty())
Move most user of TargetMachine::getDataLayout to the Module one Summary: This change is part of a series of commits dedicated to have a single DataLayout during compilation by using always the one owned by the module. This patch is quite boring overall, except for some uglyness in ASMPrinter which has a getDataLayout function but has some clients that use it without a Module (llmv-dsymutil, llvm-dwarfdump), so some methods are taking a DataLayout as parameter. Reviewers: echristo Subscribers: yaron.keren, rafael, llvm-commits, jholewinski Differential Revision: http://reviews.llvm.org/D11090 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 242386
2015-07-16 14:11:10 +08:00
Name += DL.getPrivateGlobalPrefix();
Hide the stub created for MO_ExternalSymbol too. given declare void @llvm.memset.p0i8.i32(i8* nocapture, i8, i32, i32, i1) declare void @foo() define void @bar() { call void @foo() call void @llvm.memset.p0i8.i32(i8* null, i8 0, i32 188, i32 1, i1 false) ret void } We used to produce L_foo$stub: .indirect_symbol _foo .ascii "\364\364\364\364\364" _memset$stub: .indirect_symbol _memset .ascii "\364\364\364\364\364" We not produce a private stub for memset too. Stubs are not needed with recent linkers, but we still produce them for darwin8. Thanks to David Fang for confirming that gcc used to do this too. llvm-svn: 196468
2013-12-05 13:19:12 +08:00
Fix setjmp on models with non-Small code model nor non-Static relocation model - MBB address is only valid as an immediate value in Small & Static code/relocation models. On other models, LEA is needed to load IP address of the restore MBB. - A minor fix of MBB in MC lowering is added as well to enable target relocation flag being propagated into MC. llvm-svn: 166084
2012-10-17 10:22:27 +08:00
if (MO.isGlobal()) {
eliminate the string form of DecorateCygMingName llvm-svn: 98373
2010-03-13 03:42:40 +08:00
const GlobalValue *GV = MO.getGlobal();
Add back r201608, r201622, r201624 and r201625 r201608 made llvm corretly handle private globals with MachO. r201622 fixed a bug in it and r201624 and r201625 were changes for using private linkage, assuming that llvm would do the right thing. They all got reverted because r201608 introduced a crash in LTO. This patch includes a fix for that. The issue was that TargetLoweringObjectFile now has to be initialized before we can mangle names of private globals. This is trivially true during the normal codegen pipeline (the asm printer does it), but LTO has to do it manually. llvm-svn: 201700
2014-02-20 01:23:20 +08:00
AsmPrinter.getNameWithPrefix(Name, GV);
Fix setjmp on models with non-Small code model nor non-Static relocation model - MBB address is only valid as an immediate value in Small & Static code/relocation models. On other models, LEA is needed to load IP address of the restore MBB. - A minor fix of MBB in MC lowering is added as well to enable target relocation flag being propagated into MC. llvm-svn: 166084
2012-10-17 10:22:27 +08:00
} else if (MO.isSymbol()) {
Move most user of TargetMachine::getDataLayout to the Module one Summary: This change is part of a series of commits dedicated to have a single DataLayout during compilation by using always the one owned by the module. This patch is quite boring overall, except for some uglyness in ASMPrinter which has a getDataLayout function but has some clients that use it without a Module (llmv-dsymutil, llvm-dwarfdump), so some methods are taking a DataLayout as parameter. Reviewers: echristo Subscribers: yaron.keren, rafael, llvm-commits, jholewinski Differential Revision: http://reviews.llvm.org/D11090 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 242386
2015-07-16 14:11:10 +08:00
Mangler::getNameWithPrefix(Name, MO.getSymbolName(), DL);
Fix setjmp on models with non-Small code model nor non-Static relocation model - MBB address is only valid as an immediate value in Small & Static code/relocation models. On other models, LEA is needed to load IP address of the restore MBB. - A minor fix of MBB in MC lowering is added as well to enable target relocation flag being propagated into MC. llvm-svn: 166084
2012-10-17 10:22:27 +08:00
} else if (MO.isMBB()) {
Avoid a call to getOrCreateSymbol when we already have the symbol. llvm-svn: 238890
2015-06-03 08:02:40 +08:00
assert(Suffix.empty());
Sym = MO.getMBB()->getSymbol();
unify the paths for external symbols and global variables: 2 files changed, 48 insertions(+), 83 deletions(-) llvm-svn: 95599
2010-02-09 07:03:41 +08:00
}
Refactor to remove a bit of duplication. No functionality change. llvm-svn: 195933
2013-11-29 04:12:44 +08:00
Name += Suffix;
Avoid a call to getOrCreateSymbol when we already have the symbol. llvm-svn: 238890
2015-06-03 08:02:40 +08:00
if (!Sym)
Sym = Ctx.getOrCreateSymbol(Name);
Hide the stub created for MO_ExternalSymbol too. given declare void @llvm.memset.p0i8.i32(i8* nocapture, i8, i32, i32, i1) declare void @foo() define void @bar() { call void @foo() call void @llvm.memset.p0i8.i32(i8* null, i8 0, i32 188, i32 1, i1 false) ret void } We used to produce L_foo$stub: .indirect_symbol _foo .ascii "\364\364\364\364\364" _memset$stub: .indirect_symbol _memset .ascii "\364\364\364\364\364" We not produce a private stub for memset too. Stubs are not needed with recent linkers, but we still produce them for darwin8. Thanks to David Fang for confirming that gcc used to do this too. llvm-svn: 196468
2013-12-05 13:19:12 +08:00
unify the paths for external symbols and global variables: 2 files changed, 48 insertions(+), 83 deletions(-) llvm-svn: 95599
2010-02-09 07:03:41 +08:00
// If the target flags on the operand changes the name of the symbol, do that
// before we return the symbol.
merge globaladdress symbol processing stuff into other stuff. Now all global variable operand flag processing stuff is shared between different operand types. llvm-svn: 80886
2009-09-03 13:06:07 +08:00
switch (MO.getTargetFlags()) {
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
default:
break;
[MinGW] [X86] Add stubs for references to data variables that might end up imported from a dll Variables declared with the dllimport attribute are accessed via a stub variable named __imp_<var>. In MinGW configurations, variables that aren't declared with a dllimport attribute might still end up imported from another DLL with runtime pseudo relocs. For x86_64, this avoids the risk that the target is out of range for a 32 bit PC relative reference, in case the target DLL is loaded further than 4 GB from the reference. It also avoids having to make the text section writable at runtime when doing the runtime fixups, which makes it worthwhile to do for i386 as well. Add stub variables for all dso local data references where a definition of the variable isn't visible within the module, since the DLL data autoimporting might make them imported even though they are marked as dso local within LLVM. Don't do this for variables that actually are defined within the same module, since we then know for sure that it actually is dso local. Don't do this for references to functions, since there's no need for runtime pseudo relocations for autoimporting them; if a function from a different DLL is called without the appropriate dllimport attribute, the call just gets routed via a thunk instead. GCC does something similar since 4.9 (when compiling with -mcmodel=medium or large; from that version, medium is the default code model for x86_64 mingw), but only for x86_64. Differential Revision: https://reviews.llvm.org/D51288 llvm-svn: 340942
2018-08-30 01:28:34 +08:00
case X86II::MO_COFFSTUB: {
MachineModuleInfoCOFF &MMICOFF =
MF.getMMI().getObjFileInfo<MachineModuleInfoCOFF>();
MachineModuleInfoImpl::StubValueTy &StubSym = MMICOFF.getGVStubEntry(Sym);
if (!StubSym.getPointer()) {
assert(MO.isGlobal() && "Extern symbol not handled yet");
StubSym = MachineModuleInfoImpl::StubValueTy(
[MinGW] [X86] Pass true for the second parameter to StubValueTy for MO_COFFSTUB. NFC. These stubs should never be emitted for internal symbols, and nothing in AsmPrinter ever actually use this value when producing the stubs for COFF anyway. llvm-svn: 341177
2018-08-31 16:00:31 +08:00
AsmPrinter.getSymbol(MO.getGlobal()), true);
[MinGW] [X86] Add stubs for references to data variables that might end up imported from a dll Variables declared with the dllimport attribute are accessed via a stub variable named __imp_<var>. In MinGW configurations, variables that aren't declared with a dllimport attribute might still end up imported from another DLL with runtime pseudo relocs. For x86_64, this avoids the risk that the target is out of range for a 32 bit PC relative reference, in case the target DLL is loaded further than 4 GB from the reference. It also avoids having to make the text section writable at runtime when doing the runtime fixups, which makes it worthwhile to do for i386 as well. Add stub variables for all dso local data references where a definition of the variable isn't visible within the module, since the DLL data autoimporting might make them imported even though they are marked as dso local within LLVM. Don't do this for variables that actually are defined within the same module, since we then know for sure that it actually is dso local. Don't do this for references to functions, since there's no need for runtime pseudo relocations for autoimporting them; if a function from a different DLL is called without the appropriate dllimport attribute, the call just gets routed via a thunk instead. GCC does something similar since 4.9 (when compiling with -mcmodel=medium or large; from that version, medium is the default code model for x86_64 mingw), but only for x86_64. Differential Revision: https://reviews.llvm.org/D51288 llvm-svn: 340942
2018-08-30 01:28:34 +08:00
}
break;
}
merge globaladdress symbol processing stuff into other stuff. Now all global variable operand flag processing stuff is shared between different operand types. llvm-svn: 80886
2009-09-03 13:06:07 +08:00
case X86II::MO_DARWIN_NONLAZY:
Fix a bug I introduced in FnStubs generation, switch GVStubs to be a densemap instead of StringMap to match FnStubs. llvm-svn: 81513
2009-09-11 14:59:18 +08:00
case X86II::MO_DARWIN_NONLAZY_PIC_BASE: {
Add a bit along with the MCSymbols stored in the MachineModuleInfo maps that indicates that an MCSymbol is external or not. (It's true if it's external.) This will be used to specify the correct information to add to non-lazy pointers. That will be explained further when this bit is used. llvm-svn: 98199
2010-03-11 06:34:10 +08:00
MachineModuleInfoImpl::StubValueTy &StubSym =
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
getMachOMMI().getGVStubEntry(Sym);
[C++] Use 'nullptr'. Target edition. llvm-svn: 207197
2014-04-25 13:30:21 +08:00
if (!StubSym.getPointer()) {
unify the paths for external symbols and global variables: 2 files changed, 48 insertions(+), 83 deletions(-) llvm-svn: 95599
2010-02-09 07:03:41 +08:00
assert(MO.isGlobal() && "Extern symbol not handled yet");
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
StubSym = MachineModuleInfoImpl::StubValueTy(
AsmPrinter.getSymbol(MO.getGlobal()),
!MO.getGlobal()->hasInternalLinkage());
unify the paths for external symbols and global variables: 2 files changed, 48 insertions(+), 83 deletions(-) llvm-svn: 95599
2010-02-09 07:03:41 +08:00
}
Refactor to remove a bit of duplication. No functionality change. llvm-svn: 195933
2013-11-29 04:12:44 +08:00
break;
Fix a bug I introduced in FnStubs generation, switch GVStubs to be a densemap instead of StringMap to match FnStubs. llvm-svn: 81513
2009-09-11 14:59:18 +08:00
}
implement lowering support for constant pool index operands, this gets a bunch more olden programs working. llvm-svn: 80881
2009-09-03 12:44:53 +08:00
}
unify the paths for external symbols and global variables: 2 files changed, 48 insertions(+), 83 deletions(-) llvm-svn: 95599
2010-02-09 07:03:41 +08:00
Refactor to remove a bit of duplication. No functionality change. llvm-svn: 195933
2013-11-29 04:12:44 +08:00
return Sym;
Split the "operand -> symbol" logic from the "get offset and other munging from operand" logic. GlobalAddress still todo. llvm-svn: 80884
2009-09-03 12:56:20 +08:00
}
split MachineInstr -> MCInst lowering into its own class (not being embedded into X86ATTAsmPrinter). This still depends heavily on X86ATTAsmPrinter, but this is a step in the right direction. llvm-svn: 81627
2009-09-13 04:34:57 +08:00
MCOperand X86MCInstLower::LowerSymbolOperand(const MachineOperand &MO,
MCSymbol *Sym) const {
Implement support for X86II::MO_GOT_ABSOLUTE_ADDRESS. We get very different formatting from the old asmprinter, but it should be semantically the same. We used to get: popl %eax addl $_GLOBAL_OFFSET_TABLE_ + [.-.Lllvm$6.$piclabel], %eax ... Now we get: popl %eax .Lpicbaseref6: addl $(_GLOBAL_OFFSET_TABLE_ + (.Lpicbaseref6 - .Lllvm$6.$piclabel)), %eax ... llvm-svn: 80905
2009-09-03 15:30:56 +08:00
// FIXME: We would like an efficient form for this, so we don't have to do a
// lot of extra uniquing.
[C++] Use 'nullptr'. Target edition. llvm-svn: 207197
2014-04-25 13:30:21 +08:00
const MCExpr *Expr = nullptr;
MC: Allow modifiers in MCSymbolRefExpr, and eliminate X86MCTargetExpr. - Although it would be nice to allow this decoupling, the assembler needs to be able to reason about MCSymbolRefExprs in too many places to make this viable. We can use a target specific encoding of the variant if this becomes an issue. - This patch also extends llvm-mc to support parsing of the modifiers, as opposed to lumping them in with the symbol. llvm-svn: 98592
2010-03-16 07:51:06 +08:00
MCSymbolRefExpr::VariantKind RefKind = MCSymbolRefExpr::VK_None;
Whitespace. llvm-svn: 161122
2012-08-02 02:39:17 +08:00
Split the "operand -> symbol" logic from the "get offset and other munging from operand" logic. GlobalAddress still todo. llvm-svn: 80884
2009-09-03 12:56:20 +08:00
switch (MO.getTargetFlags()) {
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
default:
llvm_unreachable("Unknown target flag on GV operand");
case X86II::MO_NO_FLAG: // No flag.
merge globaladdress symbol processing stuff into other stuff. Now all global variable operand flag processing stuff is shared between different operand types. llvm-svn: 80886
2009-09-03 13:06:07 +08:00
// These affect the name of the symbol, not any suffix.
case X86II::MO_DARWIN_NONLAZY:
case X86II::MO_DLLIMPORT:
[MinGW] [X86] Add stubs for references to data variables that might end up imported from a dll Variables declared with the dllimport attribute are accessed via a stub variable named __imp_<var>. In MinGW configurations, variables that aren't declared with a dllimport attribute might still end up imported from another DLL with runtime pseudo relocs. For x86_64, this avoids the risk that the target is out of range for a 32 bit PC relative reference, in case the target DLL is loaded further than 4 GB from the reference. It also avoids having to make the text section writable at runtime when doing the runtime fixups, which makes it worthwhile to do for i386 as well. Add stub variables for all dso local data references where a definition of the variable isn't visible within the module, since the DLL data autoimporting might make them imported even though they are marked as dso local within LLVM. Don't do this for variables that actually are defined within the same module, since we then know for sure that it actually is dso local. Don't do this for references to functions, since there's no need for runtime pseudo relocations for autoimporting them; if a function from a different DLL is called without the appropriate dllimport attribute, the call just gets routed via a thunk instead. GCC does something similar since 4.9 (when compiling with -mcmodel=medium or large; from that version, medium is the default code model for x86_64 mingw), but only for x86_64. Differential Revision: https://reviews.llvm.org/D51288 llvm-svn: 340942
2018-08-30 01:28:34 +08:00
case X86II::MO_COFFSTUB:
merge globaladdress symbol processing stuff into other stuff. Now all global variable operand flag processing stuff is shared between different operand types. llvm-svn: 80886
2009-09-03 13:06:07 +08:00
break;
Whitespace. llvm-svn: 161122
2012-08-02 02:39:17 +08:00
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
case X86II::MO_TLVP:
RefKind = MCSymbolRefExpr::VK_TLVP;
break;
Add first pass at darwin tls compiler support. llvm-svn: 105381
2010-06-03 12:07:48 +08:00
case X86II::MO_TLVP_PIC_BASE:
MC: Clean up MCExpr naming. NFC. llvm-svn: 238634
2015-05-30 09:25:56 +08:00
Expr = MCSymbolRefExpr::create(Sym, MCSymbolRefExpr::VK_TLVP, Ctx);
fix indentation llvm-svn: 108368
2010-07-15 07:04:59 +08:00
// Subtract the pic base.
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
Expr = MCBinaryExpr::createSub(
Expr, MCSymbolRefExpr::create(MF.getPICBaseSymbol(), Ctx), Ctx);
break;
case X86II::MO_SECREL:
RefKind = MCSymbolRefExpr::VK_SECREL;
break;
case X86II::MO_TLSGD:
RefKind = MCSymbolRefExpr::VK_TLSGD;
break;
case X86II::MO_TLSLD:
RefKind = MCSymbolRefExpr::VK_TLSLD;
break;
case X86II::MO_TLSLDM:
RefKind = MCSymbolRefExpr::VK_TLSLDM;
break;
case X86II::MO_GOTTPOFF:
RefKind = MCSymbolRefExpr::VK_GOTTPOFF;
break;
case X86II::MO_INDNTPOFF:
RefKind = MCSymbolRefExpr::VK_INDNTPOFF;
break;
case X86II::MO_TPOFF:
RefKind = MCSymbolRefExpr::VK_TPOFF;
break;
case X86II::MO_DTPOFF:
RefKind = MCSymbolRefExpr::VK_DTPOFF;
break;
case X86II::MO_NTPOFF:
RefKind = MCSymbolRefExpr::VK_NTPOFF;
break;
case X86II::MO_GOTNTPOFF:
RefKind = MCSymbolRefExpr::VK_GOTNTPOFF;
break;
case X86II::MO_GOTPCREL:
RefKind = MCSymbolRefExpr::VK_GOTPCREL;
break;
case X86II::MO_GOT:
RefKind = MCSymbolRefExpr::VK_GOT;
break;
case X86II::MO_GOTOFF:
RefKind = MCSymbolRefExpr::VK_GOTOFF;
break;
case X86II::MO_PLT:
RefKind = MCSymbolRefExpr::VK_PLT;
break;
case X86II::MO_ABS8:
RefKind = MCSymbolRefExpr::VK_X86_ABS8;
break;
merge globaladdress symbol processing stuff into other stuff. Now all global variable operand flag processing stuff is shared between different operand types. llvm-svn: 80886
2009-09-03 13:06:07 +08:00
case X86II::MO_PIC_BASE_OFFSET:
case X86II::MO_DARWIN_NONLAZY_PIC_BASE:
MC: Clean up MCExpr naming. NFC. llvm-svn: 238634
2015-05-30 09:25:56 +08:00
Expr = MCSymbolRefExpr::create(Sym, Ctx);
merge globaladdress symbol processing stuff into other stuff. Now all global variable operand flag processing stuff is shared between different operand types. llvm-svn: 80886
2009-09-03 13:06:07 +08:00
// Subtract the pic base.
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
Expr = MCBinaryExpr::createSub(
Expr, MCSymbolRefExpr::create(MF.getPICBaseSymbol(), Ctx), Ctx);
Fix a bit of confusion about .set and produce more readable assembly. Every target we support has support for assembly that looks like a = b - c .long a What is special about MachO is that the above combination suppresses the production of a relocation. With this change we avoid producing the intermediary labels when they don't add any value. llvm-svn: 220256
2014-10-21 09:17:30 +08:00
if (MO.isJTI()) {
doesSetDirectiveSuppressesReloc -> doesSetDirectiveSuppressReloc, the former is grammatically incorrect. llvm-svn: 273100
2016-06-19 07:25:37 +08:00
assert(MAI.doesSetDirectiveSuppressReloc());
Use .set expression for x86 pic jump table reference to reduce assembly relocation. rdar://7738756 llvm-svn: 101085
2010-04-13 07:07:17 +08:00
// If .set directive is supported, use it to reduce the number of
// relocations the assembler will generate for differences between
// local labels. This is only safe when the symbols are in the same
// section so we are restricting it to jumptable references.
MC: Clean up method names in MCContext. The naming was a mish-mash of old and new style. Update to be consistent with the new. NFC. llvm-svn: 237594
2015-05-19 02:43:14 +08:00
MCSymbol *Label = Ctx.createTempSymbol();
[MC] De-capitalize some MCStreamer::Emit* functions
2020-02-15 10:16:24 +08:00
AsmPrinter.OutStreamer->emitAssignment(Label, Expr);
MC: Clean up MCExpr naming. NFC. llvm-svn: 238634
2015-05-30 09:25:56 +08:00
Expr = MCSymbolRefExpr::create(Label, Ctx);
Use .set expression for x86 pic jump table reference to reduce assembly relocation. rdar://7738756 llvm-svn: 101085
2010-04-13 07:07:17 +08:00
}
merge globaladdress symbol processing stuff into other stuff. Now all global variable operand flag processing stuff is shared between different operand types. llvm-svn: 80886
2009-09-03 13:06:07 +08:00
break;
Implement support for X86II::MO_GOT_ABSOLUTE_ADDRESS. We get very different formatting from the old asmprinter, but it should be semantically the same. We used to get: popl %eax addl $_GLOBAL_OFFSET_TABLE_ + [.-.Lllvm$6.$piclabel], %eax ... Now we get: popl %eax .Lpicbaseref6: addl $(_GLOBAL_OFFSET_TABLE_ + (.Lpicbaseref6 - .Lllvm$6.$piclabel)), %eax ... llvm-svn: 80905
2009-09-03 15:30:56 +08:00
}
Whitespace. llvm-svn: 161122
2012-08-02 02:39:17 +08:00
[C++] Use 'nullptr'. Target edition. llvm-svn: 207197
2014-04-25 13:30:21 +08:00
if (!Expr)
MC: Clean up MCExpr naming. NFC. llvm-svn: 238634
2015-05-30 09:25:56 +08:00
Expr = MCSymbolRefExpr::create(Sym, RefKind, Ctx);
Whitespace. llvm-svn: 161122
2012-08-02 02:39:17 +08:00
Fix setjmp on models with non-Small code model nor non-Static relocation model - MBB address is only valid as an immediate value in Small & Static code/relocation models. On other models, LEA is needed to load IP address of the restore MBB. - A minor fix of MBB in MC lowering is added as well to enable target relocation flag being propagated into MC. llvm-svn: 166084
2012-10-17 10:22:27 +08:00
if (!MO.isJTI() && !MO.isMBB() && MO.getOffset())
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
Expr = MCBinaryExpr::createAdd(
Expr, MCConstantExpr::create(MO.getOffset(), Ctx), Ctx);
MC: Modernize MCOperand API naming. NFC. MCOperand::Create*() methods renamed to MCOperand::create*(). llvm-svn: 237275
2015-05-14 02:37:00 +08:00
return MCOperand::createExpr(Expr);
implement lowering support for constant pool index operands, this gets a bunch more olden programs working. llvm-svn: 80881
2009-09-03 12:44:53 +08:00
}
Remove \brief commands from doxygen comments. We've been running doxygen with the autobrief option for a couple of years now. This makes the \brief markers into our comments redundant. Since they are a visual distraction and we don't want to encourage more \brief markers in new code either, this patch removes them all. Patch produced by for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done Differential Revision: https://reviews.llvm.org/D46290 llvm-svn: 331272
2018-05-01 23:54:18 +08:00
/// Simplify FOO $imm, %{al,ax,eax,rax} to FOO $imm, for instruction with
MC/X86: Implement custom lowering to make sure we match things like X86::ADC32ri $0, %eax to X86::ADC32i32 $0 llvm-svn: 104030
2010-05-19 01:22:24 +08:00
/// a short fixed-register form.
static void SimplifyShortImmForm(MCInst &Inst, unsigned Opcode) {
unsigned ImmOp = Inst.getNumOperands() - 1;
Add support for implicit TLS model used with MS VC runtime. Patch by Kai Nacke! llvm-svn: 150307
2012-02-12 01:26:53 +08:00
assert(Inst.getOperand(0).isReg() &&
(Inst.getOperand(ImmOp).isImm() || Inst.getOperand(ImmOp).isExpr()) &&
MC/X86: Implement custom lowering to make sure we match things like X86::ADC32ri $0, %eax to X86::ADC32i32 $0 llvm-svn: 104030
2010-05-19 01:22:24 +08:00
((Inst.getNumOperands() == 3 && Inst.getOperand(1).isReg() &&
Inst.getOperand(0).getReg() == Inst.getOperand(1).getReg()) ||
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
Inst.getNumOperands() == 2) &&
"Unexpected instruction!");
MC/X86: Implement custom lowering to make sure we match things like X86::ADC32ri $0, %eax to X86::ADC32i32 $0 llvm-svn: 104030
2010-05-19 01:22:24 +08:00
// Check whether the destination register can be fixed.
unsigned Reg = Inst.getOperand(0).getReg();
if (Reg != X86::AL && Reg != X86::AX && Reg != X86::EAX && Reg != X86::RAX)
return;
// If so, rewrite the instruction.
MC/X86: Lower MOV{8,16,32,64}{rm,mr} to fixed-register forms, as appropriate. llvm-svn: 104112
2010-05-19 14:20:44 +08:00
MCOperand Saved = Inst.getOperand(ImmOp);
Inst = MCInst();
Inst.setOpcode(Opcode);
Inst.addOperand(Saved);
}
Remove \brief commands from doxygen comments. We've been running doxygen with the autobrief option for a couple of years now. This makes the \brief markers into our comments redundant. Since they are a visual distraction and we don't want to encourage more \brief markers in new code either, this patch removes them all. Patch produced by for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done Differential Revision: https://reviews.llvm.org/D46290 llvm-svn: 331272
2018-05-01 23:54:18 +08:00
/// If a movsx instruction has a shorter encoding for the used register
X86: Shrink certain forms of movsx. In particular: movsbw %al, %ax --> cbtw movswl %ax, %eax --> cwtl movslq %eax, %rax --> cltq According to Intel's manual those have the same performance characteristics but come with a smaller encoding. llvm-svn: 186174
2013-07-13 02:06:44 +08:00
/// simplify the instruction to use it instead.
static void SimplifyMOVSX(MCInst &Inst) {
unsigned NewOpcode = 0;
unsigned Op0 = Inst.getOperand(0).getReg(), Op1 = Inst.getOperand(1).getReg();
switch (Inst.getOpcode()) {
default:
llvm_unreachable("Unexpected instruction!");
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
case X86::MOVSX16rr8: // movsbw %al, %ax --> cbtw
X86: Shrink certain forms of movsx. In particular: movsbw %al, %ax --> cbtw movswl %ax, %eax --> cwtl movslq %eax, %rax --> cltq According to Intel's manual those have the same performance characteristics but come with a smaller encoding. llvm-svn: 186174
2013-07-13 02:06:44 +08:00
if (Op0 == X86::AX && Op1 == X86::AL)
NewOpcode = X86::CBW;
break;
case X86::MOVSX32rr16: // movswl %ax, %eax --> cwtl
if (Op0 == X86::EAX && Op1 == X86::AX)
NewOpcode = X86::CWDE;
break;
case X86::MOVSX64rr32: // movslq %eax, %rax --> cltq
if (Op0 == X86::RAX && Op1 == X86::EAX)
NewOpcode = X86::CDQE;
break;
}
if (NewOpcode != 0) {
Inst = MCInst();
Inst.setOpcode(NewOpcode);
}
}
Remove \brief commands from doxygen comments. We've been running doxygen with the autobrief option for a couple of years now. This makes the \brief markers into our comments redundant. Since they are a visual distraction and we don't want to encourage more \brief markers in new code either, this patch removes them all. Patch produced by for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done Differential Revision: https://reviews.llvm.org/D46290 llvm-svn: 331272
2018-05-01 23:54:18 +08:00
/// Simplify things like MOV32rm to MOV32o32a.
Don't attempt to SimplifyShortMoveForm in 64-bit mode. llvm-svn: 111182
2010-08-17 05:03:32 +08:00
static void SimplifyShortMoveForm(X86AsmPrinter &Printer, MCInst &Inst,
unsigned Opcode) {
// Don't make these simplifications in 64-bit mode; other assemblers don't
// perform them because they make the code larger.
if (Printer.getSubtarget().is64Bit())
return;
MC/X86: Lower MOV{8,16,32,64}{rm,mr} to fixed-register forms, as appropriate. llvm-svn: 104112
2010-05-19 14:20:44 +08:00
bool IsStore = Inst.getOperand(0).isReg() && Inst.getOperand(1).isReg();
unsigned AddrBase = IsStore;
unsigned RegOp = IsStore ? 0 : 5;
unsigned AddrOp = AddrBase + 3;
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
assert(
Inst.getNumOperands() == 6 && Inst.getOperand(RegOp).isReg() &&
Inst.getOperand(AddrBase + X86::AddrBaseReg).isReg() &&
Inst.getOperand(AddrBase + X86::AddrScaleAmt).isImm() &&
Inst.getOperand(AddrBase + X86::AddrIndexReg).isReg() &&
Inst.getOperand(AddrBase + X86::AddrSegmentReg).isReg() &&
(Inst.getOperand(AddrOp).isExpr() || Inst.getOperand(AddrOp).isImm()) &&
"Unexpected instruction!");
MC/X86: Lower MOV{8,16,32,64}{rm,mr} to fixed-register forms, as appropriate. llvm-svn: 104112
2010-05-19 14:20:44 +08:00
// Check whether the destination register can be fixed.
unsigned Reg = Inst.getOperand(RegOp).getReg();
if (Reg != X86::AL && Reg != X86::AX && Reg != X86::EAX && Reg != X86::RAX)
return;
// Check whether this is an absolute address.
Whitespace. llvm-svn: 161122
2012-08-02 02:39:17 +08:00
// FIXME: We know TLVP symbol refs aren't, but there should be a better way
Hack to let the move lowering handle dynamic-no-pic absolute moves of TLVP: movl _a@TLVP, %eax Daniel: Please review if you get a chance. llvm-svn: 106194
2010-06-17 08:51:48 +08:00
// to do this here.
bool Absolute = true;
if (Inst.getOperand(AddrOp).isExpr()) {
const MCExpr *MCE = Inst.getOperand(AddrOp).getExpr();
if (const MCSymbolRefExpr *SRE = dyn_cast<MCSymbolRefExpr>(MCE))
if (SRE->getKind() == MCSymbolRefExpr::VK_TLVP)
Absolute = false;
}
Whitespace. llvm-svn: 161122
2012-08-02 02:39:17 +08:00
Hack to let the move lowering handle dynamic-no-pic absolute moves of TLVP: movl _a@TLVP, %eax Daniel: Please review if you get a chance. llvm-svn: 106194
2010-06-17 08:51:48 +08:00
if (Absolute &&
X86: Use enums for memory operand decoding instead of integer literals. Summary: X86BaseInfo.h defines an enum for the offset of each operand in a memory operand sequence. Some code uses it and some does not. This patch replaces (hopefully) all remaining locations where an integer literal was used instead of this enum. No functionality change intended. Reviewers: nadav CC: llvm-commits, t.p.northover Differential Revision: http://llvm-reviews.chandlerc.com/D3108 llvm-svn: 204158
2014-03-19 00:14:11 +08:00
(Inst.getOperand(AddrBase + X86::AddrBaseReg).getReg() != 0 ||
Inst.getOperand(AddrBase + X86::AddrScaleAmt).getImm() != 1 ||
Inst.getOperand(AddrBase + X86::AddrIndexReg).getReg() != 0))
MC/X86: Lower MOV{8,16,32,64}{rm,mr} to fixed-register forms, as appropriate. llvm-svn: 104112
2010-05-19 14:20:44 +08:00
return;
// If so, rewrite the instruction.
MCOperand Saved = Inst.getOperand(AddrOp);
X86: Use enums for memory operand decoding instead of integer literals. Summary: X86BaseInfo.h defines an enum for the offset of each operand in a memory operand sequence. Some code uses it and some does not. This patch replaces (hopefully) all remaining locations where an integer literal was used instead of this enum. No functionality change intended. Reviewers: nadav CC: llvm-commits, t.p.northover Differential Revision: http://llvm-reviews.chandlerc.com/D3108 llvm-svn: 204158
2014-03-19 00:14:11 +08:00
MCOperand Seg = Inst.getOperand(AddrBase + X86::AddrSegmentReg);
MC/X86: Lower MOV{8,16,32,64}{rm,mr} to fixed-register forms, as appropriate. llvm-svn: 104112
2010-05-19 14:20:44 +08:00
Inst = MCInst();
Inst.setOpcode(Opcode);
Inst.addOperand(Saved);
Copy segment register when optimizing to MOV8ao8/MOV16ao16/MOV32ao32. llvm-svn: 199365
2014-01-16 15:57:45 +08:00
Inst.addOperand(Seg);
MC/X86: Implement custom lowering to make sure we match things like X86::ADC32ri $0, %eax to X86::ADC32i32 $0 llvm-svn: 104030
2010-05-19 01:22:24 +08:00
}
split MachineInstr -> MCInst lowering into its own class (not being embedded into X86ATTAsmPrinter). This still depends heavily on X86ATTAsmPrinter, but this is a step in the right direction. llvm-svn: 81627
2009-09-13 04:34:57 +08:00
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
static unsigned getRetOpcode(const X86Subtarget &Subtarget) {
return Subtarget.is64Bit() ? X86::RETQ : X86::RETL;
[x86] Disambiguate RET[QL] and fix aliases for 16-bit mode I couldn't see how to do this sanely without splitting RETQ from RETL. Eric says: "sad about the inability to roundtrip them now, but...". I have no idea what that means, but perhaps it wants preserving in the commit comment. llvm-svn: 198756
2014-01-08 20:58:07 +08:00
}
[NFC] Extract X86MCInstLower::LowerMachineOperand. Summary: Refactoring-only change that will be used later. Reviewers: reames, atrick Reviewed By: atrick Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D10196 llvm-svn: 239739
2015-06-16 02:44:01 +08:00
Optional<MCOperand>
X86MCInstLower::LowerMachineOperand(const MachineInstr *MI,
const MachineOperand &MO) const {
switch (MO.getType()) {
default:
Cleanup dump() functions. We had various variants of defining dump() functions in LLVM. Normalize them (this should just consistently implement the things discussed in http://lists.llvm.org/pipermail/cfe-dev/2014-January/034323.html For reference: - Public headers should just declare the dump() method but not use LLVM_DUMP_METHOD or #if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP) - The definition of a dump method should look like this: #if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP) LLVM_DUMP_METHOD void MyClass::dump() { // print stuff to dbgs()... } #endif llvm-svn: 293359
2017-01-28 10:02:38 +08:00
MI->print(errs());
[NFC] Extract X86MCInstLower::LowerMachineOperand. Summary: Refactoring-only change that will be used later. Reviewers: reames, atrick Reviewed By: atrick Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D10196 llvm-svn: 239739
2015-06-16 02:44:01 +08:00
llvm_unreachable("unknown operand type");
case MachineOperand::MO_Register:
// Ignore all implicit register operands.
if (MO.isImplicit())
return None;
return MCOperand::createReg(MO.getReg());
case MachineOperand::MO_Immediate:
return MCOperand::createImm(MO.getImm());
case MachineOperand::MO_MachineBasicBlock:
case MachineOperand::MO_GlobalAddress:
case MachineOperand::MO_ExternalSymbol:
return LowerSymbolOperand(MO, GetSymbolFromOperand(MO));
Avoid a Symbol -> Name -> Symbol conversion. Before this we were producing a TargetExternalSymbol from a MCSymbol. That meant extracting the symbol name and fetching the symbol again down the pipeline. This patch adds a DAG.getMCSymbol that lets the MCSymbol pass unchanged on the DAG. Doing so removes the need for MO_NOPREFIX and fixes the root cause of pr23900, allowing r240130 to be committed again. llvm-svn: 240300
2015-06-23 01:46:53 +08:00
case MachineOperand::MO_MCSymbol:
return LowerSymbolOperand(MO, MO.getMCSymbol());
[NFC] Extract X86MCInstLower::LowerMachineOperand. Summary: Refactoring-only change that will be used later. Reviewers: reames, atrick Reviewed By: atrick Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D10196 llvm-svn: 239739
2015-06-16 02:44:01 +08:00
case MachineOperand::MO_JumpTableIndex:
return LowerSymbolOperand(MO, AsmPrinter.GetJTISymbol(MO.getIndex()));
case MachineOperand::MO_ConstantPoolIndex:
return LowerSymbolOperand(MO, AsmPrinter.GetCPISymbol(MO.getIndex()));
case MachineOperand::MO_BlockAddress:
return LowerSymbolOperand(
MO, AsmPrinter.GetBlockAddressSymbol(MO.getBlockAddress()));
case MachineOperand::MO_RegisterMask:
// Ignore call clobbers.
return None;
}
}
[X86] Remove encoding information from the TAILJMP instructions that are lowered by MCInstLowering. Fix LowerPATCHABLE_TAIL_CALL to also convert them to regular JMP/JCC instructions There are 5 instructions here that are converted from TAILJMP opcodes to regular JMP/JCC opcodes during MCInstLowering. So normally there encoding information isn't used. The exception being when XRay wraps them in PATCHABLE_TAIL_CALL. For the ones that weren't already handled in MCInstLowering, add handling for those and remove their encoding information. This patch fixes PATCHABLE_TAIL_CALL to do the same opcode conversion as the regular lowering patch. Then removes the encoding information. Differential Revision: https://reviews.llvm.org/D66561 llvm-svn: 370079
2019-08-28 01:24:23 +08:00
// Replace TAILJMP opcodes with their equivalent opcodes that have encoding
// information.
static unsigned convertTailJumpOpcode(unsigned Opcode) {
switch (Opcode) {
case X86::TAILJMPr:
Opcode = X86::JMP32r;
break;
case X86::TAILJMPm:
Opcode = X86::JMP32m;
break;
case X86::TAILJMPr64:
Opcode = X86::JMP64r;
break;
case X86::TAILJMPm64:
Opcode = X86::JMP64m;
break;
case X86::TAILJMPr64_REX:
Opcode = X86::JMP64r_REX;
break;
case X86::TAILJMPm64_REX:
Opcode = X86::JMP64m_REX;
break;
case X86::TAILJMPd:
case X86::TAILJMPd64:
Opcode = X86::JMP_1;
break;
case X86::TAILJMPd_CC:
case X86::TAILJMPd64_CC:
Opcode = X86::JCC_1;
break;
}
return Opcode;
}
split MachineInstr -> MCInst lowering into its own class (not being embedded into X86ATTAsmPrinter). This still depends heavily on X86ATTAsmPrinter, but this is a step in the right direction. llvm-svn: 81627
2009-09-13 04:34:57 +08:00
void X86MCInstLower::Lower(const MachineInstr *MI, MCInst &OutMI) const {
OutMI.setOpcode(MI->getOpcode());
Whitespace. llvm-svn: 161122
2012-08-02 02:39:17 +08:00
[NFC] Extract X86MCInstLower::LowerMachineOperand. Summary: Refactoring-only change that will be used later. Reviewers: reames, atrick Reviewed By: atrick Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D10196 llvm-svn: 239739
2015-06-16 02:44:01 +08:00
for (const MachineOperand &MO : MI->operands())
if (auto MaybeMCOp = LowerMachineOperand(MI, MO))
OutMI.addOperand(MaybeMCOp.getValue());
Whitespace. llvm-svn: 161122
2012-08-02 02:39:17 +08:00
rearrange some code, export a SmallString version of DecorateCygMingName. llvm-svn: 81502
2009-09-11 12:28:13 +08:00
// Handle a few special cases to eliminate operand modifiers.
split MachineInstr -> MCInst lowering into its own class (not being embedded into X86ATTAsmPrinter). This still depends heavily on X86ATTAsmPrinter, but this is a step in the right direction. llvm-svn: 81627
2009-09-13 04:34:57 +08:00
switch (OutMI.getOpcode()) {
X86: Stop LEA64_32r doing unspeakable things to its arguments. Previously LEA64_32r went through virtually the entire backend thinking it was using 32-bit registers until its blissful illusions were cruelly snatched away by MCInstLower and 64-bit equivalents were substituted at the last minute. This patch makes it behave normally, and take 64-bit registers as sources all the way through. Previous uses (for 32-bit arithmetic) are accommodated via SUBREG_TO_REG instructions which make the types and classes agree properly. llvm-svn: 183693
2013-06-11 04:43:49 +08:00
case X86::LEA64_32r:
Change LEA to have 5 operands for its memory operand, just like all other instructions, even though a segment is not allowed. This resolves a bunch of gross hacks in the encoder and makes LEA more consistent with the rest of the instruction set. No functionality change. llvm-svn: 107934
2010-07-09 07:46:44 +08:00
case X86::LEA64r:
case X86::LEA16r:
case X86::LEA32r:
// LEA should have a segment register, but it must be empty.
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
assert(OutMI.getNumOperands() == 1 + X86::AddrNumOperands &&
Change LEA to have 5 operands for its memory operand, just like all other instructions, even though a segment is not allowed. This resolves a bunch of gross hacks in the encoder and makes LEA more consistent with the rest of the instruction set. No functionality change. llvm-svn: 107934
2010-07-09 07:46:44 +08:00
"Unexpected # of LEA operands");
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
assert(OutMI.getOperand(1 + X86::AddrSegmentReg).getReg() == 0 &&
Change LEA to have 5 operands for its memory operand, just like all other instructions, even though a segment is not allowed. This resolves a bunch of gross hacks in the encoder and makes LEA more consistent with the rest of the instruction set. No functionality change. llvm-svn: 107934
2010-07-09 07:46:44 +08:00
"LEA has segment specified!");
split mcinst lowering stuff out to its own file. llvm-svn: 80806
2009-09-03 01:35:12 +08:00
break;
lower the last of the MRMInitReg instructions in MCInstLower. llvm-svn: 95435
2010-02-06 05:30:49 +08:00
[X86] Add pseudo instructions to use MULX with a single destination when the low result isn't used. The instruction is defined to only produce high result if both destinations are the same. We can exploit this to avoid unnecessarily clobbering a register. In order to hide this from register allocation we use a pseudo instruction and expand the result during MCInst creation. Differential Revision: https://reviews.llvm.org/D80500
2020-05-31 06:51:56 +08:00
case X86::MULX32Hrr:
case X86::MULX32Hrm:
case X86::MULX64Hrr:
case X86::MULX64Hrm: {
// Turn into regular MULX by duplicating the destination.
unsigned NewOpc;
switch (OutMI.getOpcode()) {
default: llvm_unreachable("Invalid opcode");
case X86::MULX32Hrr: NewOpc = X86::MULX32rr; break;
[X86] Fix a typo error. Summary: This will result opcode MULX32Hrm been emitted to MULX32Hrr. Reviewed by: craig.topper Differential Revision: https://reviews.llvm.org/D82472
2020-06-24 23:37:06 +08:00
case X86::MULX32Hrm: NewOpc = X86::MULX32rm; break;
[X86] Add pseudo instructions to use MULX with a single destination when the low result isn't used. The instruction is defined to only produce high result if both destinations are the same. We can exploit this to avoid unnecessarily clobbering a register. In order to hide this from register allocation we use a pseudo instruction and expand the result during MCInst creation. Differential Revision: https://reviews.llvm.org/D80500
2020-05-31 06:51:56 +08:00
case X86::MULX64Hrr: NewOpc = X86::MULX64rr; break;
case X86::MULX64Hrm: NewOpc = X86::MULX64rm; break;
}
OutMI.setOpcode(NewOpc);
// Duplicate the destination.
unsigned DestReg = OutMI.getOperand(0).getReg();
OutMI.insert(OutMI.begin(), MCOperand::createReg(DestReg));
break;
}
Teach X86 MC instruction lowering that VMOVAPSrr and other VEX-encoded register to register moves should be switched from using the MRMSrcReg form to the MRMDestReg form if the source register is a 64-bit extended register and the destination register is not. This allows the instruction to be encoded using the 2-byte VEX form instead of the 3-byte VEX form. The GNU assembler has similar behavior. llvm-svn: 177011
2013-03-14 15:09:57 +08:00
// Commute operands to get a smaller encoding by using VEX.R instead of VEX.B
// if one of the registers is extended, but other isn't.
[X86] Add support to assembler and MCInst lowering to use the other vmovq %xmmX, %xmmX encoding if it would be a shorter VEX encoding. llvm-svn: 250014
2015-10-12 12:57:59 +08:00
case X86::VMOVZPQILo2PQIrr:
Teach X86 MC instruction lowering that VMOVAPSrr and other VEX-encoded register to register moves should be switched from using the MRMSrcReg form to the MRMDestReg form if the source register is a 64-bit extended register and the destination register is not. This allows the instruction to be encoded using the 2-byte VEX form instead of the 3-byte VEX form. The GNU assembler has similar behavior. llvm-svn: 177011
2013-03-14 15:09:57 +08:00
case X86::VMOVAPDrr:
case X86::VMOVAPDYrr:
case X86::VMOVAPSrr:
case X86::VMOVAPSYrr:
case X86::VMOVDQArr:
case X86::VMOVDQAYrr:
case X86::VMOVDQUrr:
case X86::VMOVDQUYrr:
case X86::VMOVUPDrr:
case X86::VMOVUPDYrr:
case X86::VMOVUPSrr:
case X86::VMOVUPSYrr: {
Add X86 code emitter support AVX encoded MRMDestReg instructions. Previously we weren't skipping the VVVV encoded register. Based on patch by Michael Liao. llvm-svn: 177221
2013-03-16 11:44:31 +08:00
if (!X86II::isX86_64ExtendedReg(OutMI.getOperand(0).getReg()) &&
X86II::isX86_64ExtendedReg(OutMI.getOperand(1).getReg())) {
unsigned NewOpc;
switch (OutMI.getOpcode()) {
[X86] Put some cases in switch statements back on one line to be more compact and make it easier to see the similarities. NFC It looks like someone ran clang-format over this entire file which reformatted these switches into a multiline form. But I think the single line form is more useful here. llvm-svn: 336077
2018-07-02 14:42:42 +08:00
default: llvm_unreachable("Invalid opcode");
case X86::VMOVZPQILo2PQIrr: NewOpc = X86::VMOVPQI2QIrr; break;
case X86::VMOVAPDrr: NewOpc = X86::VMOVAPDrr_REV; break;
case X86::VMOVAPDYrr: NewOpc = X86::VMOVAPDYrr_REV; break;
case X86::VMOVAPSrr: NewOpc = X86::VMOVAPSrr_REV; break;
case X86::VMOVAPSYrr: NewOpc = X86::VMOVAPSYrr_REV; break;
case X86::VMOVDQArr: NewOpc = X86::VMOVDQArr_REV; break;
case X86::VMOVDQAYrr: NewOpc = X86::VMOVDQAYrr_REV; break;
case X86::VMOVDQUrr: NewOpc = X86::VMOVDQUrr_REV; break;
case X86::VMOVDQUYrr: NewOpc = X86::VMOVDQUYrr_REV; break;
case X86::VMOVUPDrr: NewOpc = X86::VMOVUPDrr_REV; break;
case X86::VMOVUPDYrr: NewOpc = X86::VMOVUPDYrr_REV; break;
case X86::VMOVUPSrr: NewOpc = X86::VMOVUPSrr_REV; break;
case X86::VMOVUPSYrr: NewOpc = X86::VMOVUPSYrr_REV; break;
Add X86 code emitter support AVX encoded MRMDestReg instructions. Previously we weren't skipping the VVVV encoded register. Based on patch by Michael Liao. llvm-svn: 177221
2013-03-16 11:44:31 +08:00
}
OutMI.setOpcode(NewOpc);
}
break;
}
case X86::VMOVSDrr:
case X86::VMOVSSrr: {
if (!X86II::isX86_64ExtendedReg(OutMI.getOperand(0).getReg()) &&
X86II::isX86_64ExtendedReg(OutMI.getOperand(2).getReg())) {
unsigned NewOpc;
switch (OutMI.getOpcode()) {
[X86] Put some cases in switch statements back on one line to be more compact and make it easier to see the similarities. NFC It looks like someone ran clang-format over this entire file which reformatted these switches into a multiline form. But I think the single line form is more useful here. llvm-svn: 336077
2018-07-02 14:42:42 +08:00
default: llvm_unreachable("Invalid opcode");
case X86::VMOVSDrr: NewOpc = X86::VMOVSDrr_REV; break;
case X86::VMOVSSrr: NewOpc = X86::VMOVSSrr_REV; break;
Add X86 code emitter support AVX encoded MRMDestReg instructions. Previously we weren't skipping the VVVV encoded register. Based on patch by Michael Liao. llvm-svn: 177221
2013-03-16 11:44:31 +08:00
}
OutMI.setOpcode(NewOpc);
Teach X86 MC instruction lowering that VMOVAPSrr and other VEX-encoded register to register moves should be switched from using the MRMSrcReg form to the MRMDestReg form if the source register is a 64-bit extended register and the destination register is not. This allows the instruction to be encoded using the 2-byte VEX form instead of the 3-byte VEX form. The GNU assembler has similar behavior. llvm-svn: 177011
2013-03-14 15:09:57 +08:00
}
break;
}
[X86] Remove isel patterns for mask vpcmpgt/vpcmpeq. Switch vpcmp to these based on the immediate in MCInstLower The immediate form of VPCMP can represent these completely. The vpcmpgt/eq are just shorter encodings. This patch removes the isel patterns and just swaps the opcodes and removes the immediate in MCInstLower. This matches where we do some other encodings tricks. Removes over 10K bytes from the isel table. Differential Revision: https://reviews.llvm.org/D68446 llvm-svn: 373766
2019-10-05 02:02:46 +08:00
case X86::VPCMPBZ128rmi: case X86::VPCMPBZ128rmik:
case X86::VPCMPBZ128rri: case X86::VPCMPBZ128rrik:
case X86::VPCMPBZ256rmi: case X86::VPCMPBZ256rmik:
case X86::VPCMPBZ256rri: case X86::VPCMPBZ256rrik:
case X86::VPCMPBZrmi: case X86::VPCMPBZrmik:
case X86::VPCMPBZrri: case X86::VPCMPBZrrik:
case X86::VPCMPDZ128rmi: case X86::VPCMPDZ128rmik:
case X86::VPCMPDZ128rmib: case X86::VPCMPDZ128rmibk:
case X86::VPCMPDZ128rri: case X86::VPCMPDZ128rrik:
case X86::VPCMPDZ256rmi: case X86::VPCMPDZ256rmik:
case X86::VPCMPDZ256rmib: case X86::VPCMPDZ256rmibk:
case X86::VPCMPDZ256rri: case X86::VPCMPDZ256rrik:
case X86::VPCMPDZrmi: case X86::VPCMPDZrmik:
case X86::VPCMPDZrmib: case X86::VPCMPDZrmibk:
case X86::VPCMPDZrri: case X86::VPCMPDZrrik:
case X86::VPCMPQZ128rmi: case X86::VPCMPQZ128rmik:
case X86::VPCMPQZ128rmib: case X86::VPCMPQZ128rmibk:
case X86::VPCMPQZ128rri: case X86::VPCMPQZ128rrik:
case X86::VPCMPQZ256rmi: case X86::VPCMPQZ256rmik:
case X86::VPCMPQZ256rmib: case X86::VPCMPQZ256rmibk:
case X86::VPCMPQZ256rri: case X86::VPCMPQZ256rrik:
case X86::VPCMPQZrmi: case X86::VPCMPQZrmik:
case X86::VPCMPQZrmib: case X86::VPCMPQZrmibk:
case X86::VPCMPQZrri: case X86::VPCMPQZrrik:
case X86::VPCMPWZ128rmi: case X86::VPCMPWZ128rmik:
case X86::VPCMPWZ128rri: case X86::VPCMPWZ128rrik:
case X86::VPCMPWZ256rmi: case X86::VPCMPWZ256rmik:
case X86::VPCMPWZ256rri: case X86::VPCMPWZ256rrik:
case X86::VPCMPWZrmi: case X86::VPCMPWZrmik:
case X86::VPCMPWZrri: case X86::VPCMPWZrrik: {
// Turn immediate 0 into the VPCMPEQ instruction.
if (OutMI.getOperand(OutMI.getNumOperands() - 1).getImm() == 0) {
unsigned NewOpc;
switch (OutMI.getOpcode()) {
Fix "use of uninitialized variable" static analyzer warnings. NFCI. Add "unreachable" default cases like we do for the other switch()s in X86MCInstLower::Lower
2020-01-07 00:17:05 +08:00
default: llvm_unreachable("Invalid opcode");
[X86] Remove isel patterns for mask vpcmpgt/vpcmpeq. Switch vpcmp to these based on the immediate in MCInstLower The immediate form of VPCMP can represent these completely. The vpcmpgt/eq are just shorter encodings. This patch removes the isel patterns and just swaps the opcodes and removes the immediate in MCInstLower. This matches where we do some other encodings tricks. Removes over 10K bytes from the isel table. Differential Revision: https://reviews.llvm.org/D68446 llvm-svn: 373766
2019-10-05 02:02:46 +08:00
case X86::VPCMPBZ128rmi: NewOpc = X86::VPCMPEQBZ128rm; break;
case X86::VPCMPBZ128rmik: NewOpc = X86::VPCMPEQBZ128rmk; break;
case X86::VPCMPBZ128rri: NewOpc = X86::VPCMPEQBZ128rr; break;
case X86::VPCMPBZ128rrik: NewOpc = X86::VPCMPEQBZ128rrk; break;
case X86::VPCMPBZ256rmi: NewOpc = X86::VPCMPEQBZ256rm; break;
case X86::VPCMPBZ256rmik: NewOpc = X86::VPCMPEQBZ256rmk; break;
case X86::VPCMPBZ256rri: NewOpc = X86::VPCMPEQBZ256rr; break;
case X86::VPCMPBZ256rrik: NewOpc = X86::VPCMPEQBZ256rrk; break;
case X86::VPCMPBZrmi: NewOpc = X86::VPCMPEQBZrm; break;
case X86::VPCMPBZrmik: NewOpc = X86::VPCMPEQBZrmk; break;
case X86::VPCMPBZrri: NewOpc = X86::VPCMPEQBZrr; break;
case X86::VPCMPBZrrik: NewOpc = X86::VPCMPEQBZrrk; break;
case X86::VPCMPDZ128rmi: NewOpc = X86::VPCMPEQDZ128rm; break;
case X86::VPCMPDZ128rmib: NewOpc = X86::VPCMPEQDZ128rmb; break;
case X86::VPCMPDZ128rmibk: NewOpc = X86::VPCMPEQDZ128rmbk; break;
case X86::VPCMPDZ128rmik: NewOpc = X86::VPCMPEQDZ128rmk; break;
case X86::VPCMPDZ128rri: NewOpc = X86::VPCMPEQDZ128rr; break;
case X86::VPCMPDZ128rrik: NewOpc = X86::VPCMPEQDZ128rrk; break;
case X86::VPCMPDZ256rmi: NewOpc = X86::VPCMPEQDZ256rm; break;
case X86::VPCMPDZ256rmib: NewOpc = X86::VPCMPEQDZ256rmb; break;
case X86::VPCMPDZ256rmibk: NewOpc = X86::VPCMPEQDZ256rmbk; break;
case X86::VPCMPDZ256rmik: NewOpc = X86::VPCMPEQDZ256rmk; break;
case X86::VPCMPDZ256rri: NewOpc = X86::VPCMPEQDZ256rr; break;
case X86::VPCMPDZ256rrik: NewOpc = X86::VPCMPEQDZ256rrk; break;
case X86::VPCMPDZrmi: NewOpc = X86::VPCMPEQDZrm; break;
case X86::VPCMPDZrmib: NewOpc = X86::VPCMPEQDZrmb; break;
case X86::VPCMPDZrmibk: NewOpc = X86::VPCMPEQDZrmbk; break;
case X86::VPCMPDZrmik: NewOpc = X86::VPCMPEQDZrmk; break;
case X86::VPCMPDZrri: NewOpc = X86::VPCMPEQDZrr; break;
case X86::VPCMPDZrrik: NewOpc = X86::VPCMPEQDZrrk; break;
case X86::VPCMPQZ128rmi: NewOpc = X86::VPCMPEQQZ128rm; break;
case X86::VPCMPQZ128rmib: NewOpc = X86::VPCMPEQQZ128rmb; break;
case X86::VPCMPQZ128rmibk: NewOpc = X86::VPCMPEQQZ128rmbk; break;
case X86::VPCMPQZ128rmik: NewOpc = X86::VPCMPEQQZ128rmk; break;
case X86::VPCMPQZ128rri: NewOpc = X86::VPCMPEQQZ128rr; break;
case X86::VPCMPQZ128rrik: NewOpc = X86::VPCMPEQQZ128rrk; break;
case X86::VPCMPQZ256rmi: NewOpc = X86::VPCMPEQQZ256rm; break;
case X86::VPCMPQZ256rmib: NewOpc = X86::VPCMPEQQZ256rmb; break;
case X86::VPCMPQZ256rmibk: NewOpc = X86::VPCMPEQQZ256rmbk; break;
case X86::VPCMPQZ256rmik: NewOpc = X86::VPCMPEQQZ256rmk; break;
case X86::VPCMPQZ256rri: NewOpc = X86::VPCMPEQQZ256rr; break;
case X86::VPCMPQZ256rrik: NewOpc = X86::VPCMPEQQZ256rrk; break;
case X86::VPCMPQZrmi: NewOpc = X86::VPCMPEQQZrm; break;
case X86::VPCMPQZrmib: NewOpc = X86::VPCMPEQQZrmb; break;
case X86::VPCMPQZrmibk: NewOpc = X86::VPCMPEQQZrmbk; break;
case X86::VPCMPQZrmik: NewOpc = X86::VPCMPEQQZrmk; break;
case X86::VPCMPQZrri: NewOpc = X86::VPCMPEQQZrr; break;
case X86::VPCMPQZrrik: NewOpc = X86::VPCMPEQQZrrk; break;
case X86::VPCMPWZ128rmi: NewOpc = X86::VPCMPEQWZ128rm; break;
case X86::VPCMPWZ128rmik: NewOpc = X86::VPCMPEQWZ128rmk; break;
case X86::VPCMPWZ128rri: NewOpc = X86::VPCMPEQWZ128rr; break;
case X86::VPCMPWZ128rrik: NewOpc = X86::VPCMPEQWZ128rrk; break;
case X86::VPCMPWZ256rmi: NewOpc = X86::VPCMPEQWZ256rm; break;
case X86::VPCMPWZ256rmik: NewOpc = X86::VPCMPEQWZ256rmk; break;
case X86::VPCMPWZ256rri: NewOpc = X86::VPCMPEQWZ256rr; break;
case X86::VPCMPWZ256rrik: NewOpc = X86::VPCMPEQWZ256rrk; break;
case X86::VPCMPWZrmi: NewOpc = X86::VPCMPEQWZrm; break;
case X86::VPCMPWZrmik: NewOpc = X86::VPCMPEQWZrmk; break;
case X86::VPCMPWZrri: NewOpc = X86::VPCMPEQWZrr; break;
case X86::VPCMPWZrrik: NewOpc = X86::VPCMPEQWZrrk; break;
}
OutMI.setOpcode(NewOpc);
OutMI.erase(&OutMI.getOperand(OutMI.getNumOperands() - 1));
break;
}
// Turn immediate 6 into the VPCMPGT instruction.
if (OutMI.getOperand(OutMI.getNumOperands() - 1).getImm() == 6) {
unsigned NewOpc;
switch (OutMI.getOpcode()) {
Fix "use of uninitialized variable" static analyzer warnings. NFCI. Add "unreachable" default cases like we do for the other switch()s in X86MCInstLower::Lower
2020-01-07 00:17:05 +08:00
default: llvm_unreachable("Invalid opcode");
[X86] Remove isel patterns for mask vpcmpgt/vpcmpeq. Switch vpcmp to these based on the immediate in MCInstLower The immediate form of VPCMP can represent these completely. The vpcmpgt/eq are just shorter encodings. This patch removes the isel patterns and just swaps the opcodes and removes the immediate in MCInstLower. This matches where we do some other encodings tricks. Removes over 10K bytes from the isel table. Differential Revision: https://reviews.llvm.org/D68446 llvm-svn: 373766
2019-10-05 02:02:46 +08:00
case X86::VPCMPBZ128rmi: NewOpc = X86::VPCMPGTBZ128rm; break;
case X86::VPCMPBZ128rmik: NewOpc = X86::VPCMPGTBZ128rmk; break;
case X86::VPCMPBZ128rri: NewOpc = X86::VPCMPGTBZ128rr; break;
case X86::VPCMPBZ128rrik: NewOpc = X86::VPCMPGTBZ128rrk; break;
case X86::VPCMPBZ256rmi: NewOpc = X86::VPCMPGTBZ256rm; break;
case X86::VPCMPBZ256rmik: NewOpc = X86::VPCMPGTBZ256rmk; break;
case X86::VPCMPBZ256rri: NewOpc = X86::VPCMPGTBZ256rr; break;
case X86::VPCMPBZ256rrik: NewOpc = X86::VPCMPGTBZ256rrk; break;
case X86::VPCMPBZrmi: NewOpc = X86::VPCMPGTBZrm; break;
case X86::VPCMPBZrmik: NewOpc = X86::VPCMPGTBZrmk; break;
case X86::VPCMPBZrri: NewOpc = X86::VPCMPGTBZrr; break;
case X86::VPCMPBZrrik: NewOpc = X86::VPCMPGTBZrrk; break;
case X86::VPCMPDZ128rmi: NewOpc = X86::VPCMPGTDZ128rm; break;
case X86::VPCMPDZ128rmib: NewOpc = X86::VPCMPGTDZ128rmb; break;
case X86::VPCMPDZ128rmibk: NewOpc = X86::VPCMPGTDZ128rmbk; break;
case X86::VPCMPDZ128rmik: NewOpc = X86::VPCMPGTDZ128rmk; break;
case X86::VPCMPDZ128rri: NewOpc = X86::VPCMPGTDZ128rr; break;
case X86::VPCMPDZ128rrik: NewOpc = X86::VPCMPGTDZ128rrk; break;
case X86::VPCMPDZ256rmi: NewOpc = X86::VPCMPGTDZ256rm; break;
case X86::VPCMPDZ256rmib: NewOpc = X86::VPCMPGTDZ256rmb; break;
case X86::VPCMPDZ256rmibk: NewOpc = X86::VPCMPGTDZ256rmbk; break;
case X86::VPCMPDZ256rmik: NewOpc = X86::VPCMPGTDZ256rmk; break;
case X86::VPCMPDZ256rri: NewOpc = X86::VPCMPGTDZ256rr; break;
case X86::VPCMPDZ256rrik: NewOpc = X86::VPCMPGTDZ256rrk; break;
case X86::VPCMPDZrmi: NewOpc = X86::VPCMPGTDZrm; break;
case X86::VPCMPDZrmib: NewOpc = X86::VPCMPGTDZrmb; break;
case X86::VPCMPDZrmibk: NewOpc = X86::VPCMPGTDZrmbk; break;
case X86::VPCMPDZrmik: NewOpc = X86::VPCMPGTDZrmk; break;
case X86::VPCMPDZrri: NewOpc = X86::VPCMPGTDZrr; break;
case X86::VPCMPDZrrik: NewOpc = X86::VPCMPGTDZrrk; break;
case X86::VPCMPQZ128rmi: NewOpc = X86::VPCMPGTQZ128rm; break;
case X86::VPCMPQZ128rmib: NewOpc = X86::VPCMPGTQZ128rmb; break;
case X86::VPCMPQZ128rmibk: NewOpc = X86::VPCMPGTQZ128rmbk; break;
case X86::VPCMPQZ128rmik: NewOpc = X86::VPCMPGTQZ128rmk; break;
case X86::VPCMPQZ128rri: NewOpc = X86::VPCMPGTQZ128rr; break;
case X86::VPCMPQZ128rrik: NewOpc = X86::VPCMPGTQZ128rrk; break;
case X86::VPCMPQZ256rmi: NewOpc = X86::VPCMPGTQZ256rm; break;
case X86::VPCMPQZ256rmib: NewOpc = X86::VPCMPGTQZ256rmb; break;
case X86::VPCMPQZ256rmibk: NewOpc = X86::VPCMPGTQZ256rmbk; break;
case X86::VPCMPQZ256rmik: NewOpc = X86::VPCMPGTQZ256rmk; break;
case X86::VPCMPQZ256rri: NewOpc = X86::VPCMPGTQZ256rr; break;
case X86::VPCMPQZ256rrik: NewOpc = X86::VPCMPGTQZ256rrk; break;
case X86::VPCMPQZrmi: NewOpc = X86::VPCMPGTQZrm; break;
case X86::VPCMPQZrmib: NewOpc = X86::VPCMPGTQZrmb; break;
case X86::VPCMPQZrmibk: NewOpc = X86::VPCMPGTQZrmbk; break;
case X86::VPCMPQZrmik: NewOpc = X86::VPCMPGTQZrmk; break;
case X86::VPCMPQZrri: NewOpc = X86::VPCMPGTQZrr; break;
case X86::VPCMPQZrrik: NewOpc = X86::VPCMPGTQZrrk; break;
case X86::VPCMPWZ128rmi: NewOpc = X86::VPCMPGTWZ128rm; break;
case X86::VPCMPWZ128rmik: NewOpc = X86::VPCMPGTWZ128rmk; break;
case X86::VPCMPWZ128rri: NewOpc = X86::VPCMPGTWZ128rr; break;
case X86::VPCMPWZ128rrik: NewOpc = X86::VPCMPGTWZ128rrk; break;
case X86::VPCMPWZ256rmi: NewOpc = X86::VPCMPGTWZ256rm; break;
case X86::VPCMPWZ256rmik: NewOpc = X86::VPCMPGTWZ256rmk; break;
case X86::VPCMPWZ256rri: NewOpc = X86::VPCMPGTWZ256rr; break;
case X86::VPCMPWZ256rrik: NewOpc = X86::VPCMPGTWZ256rrk; break;
case X86::VPCMPWZrmi: NewOpc = X86::VPCMPGTWZrm; break;
case X86::VPCMPWZrmik: NewOpc = X86::VPCMPGTWZrmk; break;
case X86::VPCMPWZrri: NewOpc = X86::VPCMPGTWZrr; break;
case X86::VPCMPWZrrik: NewOpc = X86::VPCMPGTWZrrk; break;
}
OutMI.setOpcode(NewOpc);
OutMI.erase(&OutMI.getOperand(OutMI.getNumOperands() - 1));
break;
}
break;
}
[X86] Remove encoding information from the TAILJMP instructions that are lowered by MCInstLowering. Fix LowerPATCHABLE_TAIL_CALL to also convert them to regular JMP/JCC instructions There are 5 instructions here that are converted from TAILJMP opcodes to regular JMP/JCC opcodes during MCInstLowering. So normally there encoding information isn't used. The exception being when XRay wraps them in PATCHABLE_TAIL_CALL. For the ones that weren't already handled in MCInstLowering, add handling for those and remove their encoding information. This patch fixes PATCHABLE_TAIL_CALL to do the same opcode conversion as the regular lowering patch. Then removes the encoding information. Differential Revision: https://reviews.llvm.org/D66561 llvm-svn: 370079
2019-08-28 01:24:23 +08:00
// CALL64r, CALL64pcrel32 - These instructions used to have
[X86] Remove MCInstLower code that drops operands from some CALL and TAILJMP instructions. Add asserts to verify operand count It appears the FIXME here was handled at some point. r159728 from 2012 seems to be at least aportion of fixing it. Differential Revision: https://reviews.llvm.org/D66570 llvm-svn: 369665
2019-08-23 00:23:35 +08:00
// register inputs modeled as normal uses instead of implicit uses. As such,
// they we used to truncate off all but the first operand (the callee). This
// issue seems to have been fixed at some point. This assert verifies that.
MC/X86: Strip spurious operands from CALL64r as we do for CALL64pcrel32, to avoid same prefix byte problem as in r104062. llvm-svn: 104108
2010-05-19 12:31:36 +08:00
case X86::CALL64r:
[X86] Remove MCInstLower code that drops operands from some CALL and TAILJMP instructions. Add asserts to verify operand count It appears the FIXME here was handled at some point. r159728 from 2012 seems to be at least aportion of fixing it. Differential Revision: https://reviews.llvm.org/D66570 llvm-svn: 369665
2019-08-23 00:23:35 +08:00
case X86::CALL64pcrel32:
assert(OutMI.getNumOperands() == 1 && "Unexpected number of operands!");
make mcinstlower remove all but the first operand to CALL64pcrel32. The register use operands (e.g. the first argument is passed in a register) is currently being modeled as a normal register use, instead of correctly being an implicit use. This causes the operand to get propagated onto the mcinst, which was causing the encoder to emit a rex prefix byte, which generates an invalid call. This fixes rdar://7998435 llvm-svn: 104062
2010-05-19 05:40:18 +08:00
break;
MC/X86: Strip spurious operands from CALL64r as we do for CALL64pcrel32, to avoid same prefix byte problem as in r104062. llvm-svn: 104108
2010-05-19 12:31:36 +08:00
handle X86::EH_RETURN64 and X86::EH_RETURN. llvm-svn: 117378
2010-10-27 02:09:55 +08:00
case X86::EH_RETURN:
case X86::EH_RETURN64: {
OutMI = MCInst();
[x86] Disambiguate RET[QL] and fix aliases for 16-bit mode I couldn't see how to do this sanely without splitting RETQ from RETL. Eric says: "sad about the inability to roundtrip them now, but...". I have no idea what that means, but perhaps it wants preserving in the commit comment. llvm-svn: 198756
2014-01-08 20:58:07 +08:00
OutMI.setOpcode(getRetOpcode(AsmPrinter.getSubtarget()));
handle X86::EH_RETURN64 and X86::EH_RETURN. llvm-svn: 117378
2010-10-27 02:09:55 +08:00
break;
}
[WinEH] Make FuncletLayout more robust against catchret Catchret transfers control from a catch funclet to an earlier funclet. However, it is not completely clear which funclet the catchret target is part of. Make this clear by stapling the catchret target's funclet membership onto the CATCHRET SDAG node. llvm-svn: 249052
2015-10-02 02:44:59 +08:00
case X86::CLEANUPRET: {
X86: fix a comment copy-paste issue (NFC) The comment was copy-pasted but not updated. NFC. llvm-svn: 344973
2018-10-23 07:34:24 +08:00
// Replace CLEANUPRET with the appropriate RET.
[WinEH] Make FuncletLayout more robust against catchret Catchret transfers control from a catch funclet to an earlier funclet. However, it is not completely clear which funclet the catchret target is part of. Make this clear by stapling the catchret target's funclet membership onto the CATCHRET SDAG node. llvm-svn: 249052
2015-10-02 02:44:59 +08:00
OutMI = MCInst();
OutMI.setOpcode(getRetOpcode(AsmPrinter.getSubtarget()));
break;
}
case X86::CATCHRET: {
// Replace CATCHRET with the appropriate RET.
const X86Subtarget &Subtarget = AsmPrinter.getSubtarget();
unsigned ReturnReg = Subtarget.is64Bit() ? X86::RAX : X86::EAX;
OutMI = MCInst();
OutMI.setOpcode(getRetOpcode(Subtarget));
OutMI.addOperand(MCOperand::createReg(ReturnReg));
break;
}
[X86] Remove encoding information from the TAILJMP instructions that are lowered by MCInstLowering. Fix LowerPATCHABLE_TAIL_CALL to also convert them to regular JMP/JCC instructions There are 5 instructions here that are converted from TAILJMP opcodes to regular JMP/JCC opcodes during MCInstLowering. So normally there encoding information isn't used. The exception being when XRay wraps them in PATCHABLE_TAIL_CALL. For the ones that weren't already handled in MCInstLowering, add handling for those and remove their encoding information. This patch fixes PATCHABLE_TAIL_CALL to do the same opcode conversion as the regular lowering patch. Then removes the encoding information. Differential Revision: https://reviews.llvm.org/D66561 llvm-svn: 370079
2019-08-28 01:24:23 +08:00
// TAILJMPd, TAILJMPd64, TailJMPd_cc - Lower to the correct jump
// instruction.
case X86::TAILJMPr:
case X86::TAILJMPr64:
case X86::TAILJMPr64_REX:
case X86::TAILJMPd:
case X86::TAILJMPd64:
assert(OutMI.getNumOperands() == 1 && "Unexpected number of operands!");
OutMI.setOpcode(convertTailJumpOpcode(OutMI.getOpcode()));
break;
MC/X86: Lower TAILCALLd[64] to JMP_1, to allow relaxation and to avoid same prefix byte problem as in r104062. - As a total hack to keep the TAILCALL markers in the output, which some tests depend on, this invents a new TAILJMP_1 instruction. llvm-svn: 104120
2010-05-19 23:26:43 +08:00
[X86] Merge the different Jcc instructions for each condition code into single instructions that store the condition code as an operand. Summary: This avoids needing an isel pattern for each condition code. And it removes translation switches for converting between Jcc instructions and condition codes. Now the printer, encoder and disassembler take care of converting the immediate. We use InstAliases to handle the assembly matching. But we print using the asm string in the instruction definition. The instruction itself is marked IsCodeGenOnly=1 to hide it from the assembly parser. Reviewers: spatel, lebedev.ri, courbet, gchatelet, RKSimon Reviewed By: RKSimon Subscribers: MatzeB, qcolombet, eraman, hiraditya, arphaman, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D60228 llvm-svn: 357802
2019-04-06 03:28:09 +08:00
case X86::TAILJMPd_CC:
[X86] Remove encoding information from the TAILJMP instructions that are lowered by MCInstLowering. Fix LowerPATCHABLE_TAIL_CALL to also convert them to regular JMP/JCC instructions There are 5 instructions here that are converted from TAILJMP opcodes to regular JMP/JCC opcodes during MCInstLowering. So normally there encoding information isn't used. The exception being when XRay wraps them in PATCHABLE_TAIL_CALL. For the ones that weren't already handled in MCInstLowering, add handling for those and remove their encoding information. This patch fixes PATCHABLE_TAIL_CALL to do the same opcode conversion as the regular lowering patch. Then removes the encoding information. Differential Revision: https://reviews.llvm.org/D66561 llvm-svn: 370079
2019-08-28 01:24:23 +08:00
case X86::TAILJMPd64_CC:
[X86] Remove MCInstLower code that drops operands from some CALL and TAILJMP instructions. Add asserts to verify operand count It appears the FIXME here was handled at some point. r159728 from 2012 seems to be at least aportion of fixing it. Differential Revision: https://reviews.llvm.org/D66570 llvm-svn: 369665
2019-08-23 00:23:35 +08:00
assert(OutMI.getNumOperands() == 2 && "Unexpected number of operands!");
[X86] Remove encoding information from the TAILJMP instructions that are lowered by MCInstLowering. Fix LowerPATCHABLE_TAIL_CALL to also convert them to regular JMP/JCC instructions There are 5 instructions here that are converted from TAILJMP opcodes to regular JMP/JCC opcodes during MCInstLowering. So normally there encoding information isn't used. The exception being when XRay wraps them in PATCHABLE_TAIL_CALL. For the ones that weren't already handled in MCInstLowering, add handling for those and remove their encoding information. This patch fixes PATCHABLE_TAIL_CALL to do the same opcode conversion as the regular lowering patch. Then removes the encoding information. Differential Revision: https://reviews.llvm.org/D66561 llvm-svn: 370079
2019-08-28 01:24:23 +08:00
OutMI.setOpcode(convertTailJumpOpcode(OutMI.getOpcode()));
break;
case X86::TAILJMPm:
case X86::TAILJMPm64:
case X86::TAILJMPm64_REX:
assert(OutMI.getNumOperands() == X86::AddrNumOperands &&
"Unexpected number of operands!");
OutMI.setOpcode(convertTailJumpOpcode(OutMI.getOpcode()));
[X86] Merge the different Jcc instructions for each condition code into single instructions that store the condition code as an operand. Summary: This avoids needing an isel pattern for each condition code. And it removes translation switches for converting between Jcc instructions and condition codes. Now the printer, encoder and disassembler take care of converting the immediate. We use InstAliases to handle the assembly matching. But we print using the asm string in the instruction definition. The instruction itself is marked IsCodeGenOnly=1 to hide it from the assembly parser. Reviewers: spatel, lebedev.ri, courbet, gchatelet, RKSimon Reviewed By: RKSimon Subscribers: MatzeB, qcolombet, eraman, hiraditya, arphaman, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D60228 llvm-svn: 357802
2019-04-06 03:28:09 +08:00
break;
[X86] Make isel select the 2-byte register form of INC/DEC even in non-64-bit mode. Convert to the 1-byte form in non-64-bit mode as part of MCInst lowering. Overall this seems simpler. It reduces duplication of patterns between both modes and it simplifies the memory folding/unfolding tables as they don't need to create fake instructions just to keep track of 64-bitness. llvm-svn: 225252
2015-01-06 15:35:50 +08:00
case X86::DEC16r:
case X86::DEC32r:
case X86::INC16r:
case X86::INC32r:
// If we aren't in 64-bit mode we can use the 1-byte inc/dec instructions.
if (!AsmPrinter.getSubtarget().is64Bit()) {
unsigned Opcode;
switch (OutMI.getOpcode()) {
[X86] Put some cases in switch statements back on one line to be more compact and make it easier to see the similarities. NFC It looks like someone ran clang-format over this entire file which reformatted these switches into a multiline form. But I think the single line form is more useful here. llvm-svn: 336077
2018-07-02 14:42:42 +08:00
default: llvm_unreachable("Invalid opcode");
case X86::DEC16r: Opcode = X86::DEC16r_alt; break;
case X86::DEC32r: Opcode = X86::DEC32r_alt; break;
case X86::INC16r: Opcode = X86::INC16r_alt; break;
case X86::INC32r: Opcode = X86::INC32r_alt; break;
[X86] Make isel select the 2-byte register form of INC/DEC even in non-64-bit mode. Convert to the 1-byte form in non-64-bit mode as part of MCInst lowering. Overall this seems simpler. It reduces duplication of patterns between both modes and it simplifies the memory folding/unfolding tables as they don't need to create fake instructions just to keep track of 64-bitness. llvm-svn: 225252
2015-01-06 15:35:50 +08:00
}
OutMI.setOpcode(Opcode);
}
break;
MC/X86: Implement custom lowering to make sure we match things like X86::ADC32ri $0, %eax to X86::ADC32i32 $0 llvm-svn: 104030
2010-05-19 01:22:24 +08:00
// We don't currently select the correct instruction form for instructions
// which have a short %eax, etc. form. Handle this by custom lowering, for
// now.
//
// Note, we are currently not handling the following instructions:
MC/X86: Lower MOV{8,16,32,64}{rm,mr} to fixed-register forms, as appropriate. llvm-svn: 104112
2010-05-19 14:20:44 +08:00
// MOV64ao8, MOV64o8a
MC/X86: Implement custom lowering to make sure we match things like X86::ADC32ri $0, %eax to X86::ADC32i32 $0 llvm-svn: 104030
2010-05-19 01:22:24 +08:00
// XCHG16ar, XCHG32ar, XCHG64ar
Revert r314249 "Recommit r314151 "[X86] Make all the NOREX CodeGenOnly instructions into postRA pseudos like the NOREX version of TEST.""" This caused PR34751 llvm-svn: 314339
2017-09-28 04:34:17 +08:00
case X86::MOV8mr_NOREX:
[X86] Use nested switches to vary the operand to helper functions that were previously called in multiple cases. This seems to help the inliner reduce code. NFC llvm-svn: 267964
2016-04-29 08:51:30 +08:00
case X86::MOV8mr:
Revert r314249 "Recommit r314151 "[X86] Make all the NOREX CodeGenOnly instructions into postRA pseudos like the NOREX version of TEST.""" This caused PR34751 llvm-svn: 314339
2017-09-28 04:34:17 +08:00
case X86::MOV8rm_NOREX:
[X86] Use nested switches to vary the operand to helper functions that were previously called in multiple cases. This seems to help the inliner reduce code. NFC llvm-svn: 267964
2016-04-29 08:51:30 +08:00
case X86::MOV8rm:
case X86::MOV16mr:
case X86::MOV16rm:
case X86::MOV32mr:
case X86::MOV32rm: {
unsigned NewOpc;
switch (OutMI.getOpcode()) {
[X86] Put some cases in switch statements back on one line to be more compact and make it easier to see the similarities. NFC It looks like someone ran clang-format over this entire file which reformatted these switches into a multiline form. But I think the single line form is more useful here. llvm-svn: 336077
2018-07-02 14:42:42 +08:00
default: llvm_unreachable("Invalid opcode");
Revert r314249 "Recommit r314151 "[X86] Make all the NOREX CodeGenOnly instructions into postRA pseudos like the NOREX version of TEST.""" This caused PR34751 llvm-svn: 314339
2017-09-28 04:34:17 +08:00
case X86::MOV8mr_NOREX:
[X86] Put some cases in switch statements back on one line to be more compact and make it easier to see the similarities. NFC It looks like someone ran clang-format over this entire file which reformatted these switches into a multiline form. But I think the single line form is more useful here. llvm-svn: 336077
2018-07-02 14:42:42 +08:00
case X86::MOV8mr: NewOpc = X86::MOV8o32a; break;
Revert r314249 "Recommit r314151 "[X86] Make all the NOREX CodeGenOnly instructions into postRA pseudos like the NOREX version of TEST.""" This caused PR34751 llvm-svn: 314339
2017-09-28 04:34:17 +08:00
case X86::MOV8rm_NOREX:
[X86] Put some cases in switch statements back on one line to be more compact and make it easier to see the similarities. NFC It looks like someone ran clang-format over this entire file which reformatted these switches into a multiline form. But I think the single line form is more useful here. llvm-svn: 336077
2018-07-02 14:42:42 +08:00
case X86::MOV8rm: NewOpc = X86::MOV8ao32; break;
case X86::MOV16mr: NewOpc = X86::MOV16o32a; break;
case X86::MOV16rm: NewOpc = X86::MOV16ao32; break;
case X86::MOV32mr: NewOpc = X86::MOV32o32a; break;
case X86::MOV32rm: NewOpc = X86::MOV32ao32; break;
[X86] Use nested switches to vary the operand to helper functions that were previously called in multiple cases. This seems to help the inliner reduce code. NFC llvm-svn: 267964
2016-04-29 08:51:30 +08:00
}
SimplifyShortMoveForm(AsmPrinter, OutMI, NewOpc);
break;
}
[X86] Put some cases in switch statements back on one line to be more compact and make it easier to see the similarities. NFC It looks like someone ran clang-format over this entire file which reformatted these switches into a multiline form. But I think the single line form is more useful here. llvm-svn: 336077
2018-07-02 14:42:42 +08:00
case X86::ADC8ri: case X86::ADC16ri: case X86::ADC32ri: case X86::ADC64ri32:
case X86::ADD8ri: case X86::ADD16ri: case X86::ADD32ri: case X86::ADD64ri32:
case X86::AND8ri: case X86::AND16ri: case X86::AND32ri: case X86::AND64ri32:
case X86::CMP8ri: case X86::CMP16ri: case X86::CMP32ri: case X86::CMP64ri32:
case X86::OR8ri: case X86::OR16ri: case X86::OR32ri: case X86::OR64ri32:
case X86::SBB8ri: case X86::SBB16ri: case X86::SBB32ri: case X86::SBB64ri32:
case X86::SUB8ri: case X86::SUB16ri: case X86::SUB32ri: case X86::SUB64ri32:
case X86::TEST8ri:case X86::TEST16ri:case X86::TEST32ri:case X86::TEST64ri32:
case X86::XOR8ri: case X86::XOR16ri: case X86::XOR32ri: case X86::XOR64ri32: {
[X86] Use nested switches to vary the operand to helper functions that were previously called in multiple cases. This seems to help the inliner reduce code. NFC llvm-svn: 267964
2016-04-29 08:51:30 +08:00
unsigned NewOpc;
switch (OutMI.getOpcode()) {
[X86] Put some cases in switch statements back on one line to be more compact and make it easier to see the similarities. NFC It looks like someone ran clang-format over this entire file which reformatted these switches into a multiline form. But I think the single line form is more useful here. llvm-svn: 336077
2018-07-02 14:42:42 +08:00
default: llvm_unreachable("Invalid opcode");
case X86::ADC8ri: NewOpc = X86::ADC8i8; break;
case X86::ADC16ri: NewOpc = X86::ADC16i16; break;
case X86::ADC32ri: NewOpc = X86::ADC32i32; break;
case X86::ADC64ri32: NewOpc = X86::ADC64i32; break;
case X86::ADD8ri: NewOpc = X86::ADD8i8; break;
case X86::ADD16ri: NewOpc = X86::ADD16i16; break;
case X86::ADD32ri: NewOpc = X86::ADD32i32; break;
case X86::ADD64ri32: NewOpc = X86::ADD64i32; break;
case X86::AND8ri: NewOpc = X86::AND8i8; break;
case X86::AND16ri: NewOpc = X86::AND16i16; break;
case X86::AND32ri: NewOpc = X86::AND32i32; break;
case X86::AND64ri32: NewOpc = X86::AND64i32; break;
case X86::CMP8ri: NewOpc = X86::CMP8i8; break;
case X86::CMP16ri: NewOpc = X86::CMP16i16; break;
case X86::CMP32ri: NewOpc = X86::CMP32i32; break;
case X86::CMP64ri32: NewOpc = X86::CMP64i32; break;
case X86::OR8ri: NewOpc = X86::OR8i8; break;
case X86::OR16ri: NewOpc = X86::OR16i16; break;
case X86::OR32ri: NewOpc = X86::OR32i32; break;
case X86::OR64ri32: NewOpc = X86::OR64i32; break;
case X86::SBB8ri: NewOpc = X86::SBB8i8; break;
case X86::SBB16ri: NewOpc = X86::SBB16i16; break;
case X86::SBB32ri: NewOpc = X86::SBB32i32; break;
case X86::SBB64ri32: NewOpc = X86::SBB64i32; break;
case X86::SUB8ri: NewOpc = X86::SUB8i8; break;
case X86::SUB16ri: NewOpc = X86::SUB16i16; break;
case X86::SUB32ri: NewOpc = X86::SUB32i32; break;
case X86::SUB64ri32: NewOpc = X86::SUB64i32; break;
case X86::TEST8ri: NewOpc = X86::TEST8i8; break;
case X86::TEST16ri: NewOpc = X86::TEST16i16; break;
case X86::TEST32ri: NewOpc = X86::TEST32i32; break;
case X86::TEST64ri32: NewOpc = X86::TEST64i32; break;
case X86::XOR8ri: NewOpc = X86::XOR8i8; break;
case X86::XOR16ri: NewOpc = X86::XOR16i16; break;
case X86::XOR32ri: NewOpc = X86::XOR32i32; break;
case X86::XOR64ri32: NewOpc = X86::XOR64i32; break;
[X86] Use nested switches to vary the operand to helper functions that were previously called in multiple cases. This seems to help the inliner reduce code. NFC llvm-svn: 267964
2016-04-29 08:51:30 +08:00
}
SimplifyShortImmForm(OutMI, NewOpc);
break;
}
This commit introduces two fake instructions MORESTACK_RET and MORESTACK_RET_RESTORE_R10; which are lowered to a RET and a RET followed by a MOV respectively. Having a fake instruction prevents the verifier from seeing a MachineBasicBlock end with a non-terminator (MOV). It also prevents the rather eccentric case of a MachineBasicBlock ending with RET but having successors nevertheless. Patch by Sanjoy Das. llvm-svn: 143062
2011-10-27 05:12:27 +08:00
X86: Shrink certain forms of movsx. In particular: movsbw %al, %ax --> cbtw movswl %ax, %eax --> cwtl movslq %eax, %rax --> cltq According to Intel's manual those have the same performance characteristics but come with a smaller encoding. llvm-svn: 186174
2013-07-13 02:06:44 +08:00
// Try to shrink some forms of movsx.
case X86::MOVSX16rr8:
case X86::MOVSX32rr16:
case X86::MOVSX64rr32:
SimplifyMOVSX(OutMI);
break;
[X86] Teach X86MCInstLower to swap operands of commutable instructions to enable 2-byte VEX encoding. Summary: The 2 source operands commutable instructions are encoded in the VEX.VVVV field and the r/m field of the MODRM byte plus the VEX.B field. The VEX.B field is missing from the 2-byte VEX encoding. If the VEX.VVVV source is 0-7 and the other register is 8-15 we can swap them to avoid needing the VEX.B field. This works as long as the VEX.W, VEX.mmmmm, and VEX.X fields are also not needed. Fixes PR36706. Reviewers: RKSimon, spatel Reviewed By: RKSimon Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68550
2019-11-05 11:58:53 +08:00
case X86::VCMPPDrri:
case X86::VCMPPDYrri:
case X86::VCMPPSrri:
case X86::VCMPPSYrri:
case X86::VCMPSDrr:
case X86::VCMPSSrr: {
// Swap the operands if it will enable a 2 byte VEX encoding.
// FIXME: Change the immediate to improve opportunities?
if (!X86II::isX86_64ExtendedReg(OutMI.getOperand(1).getReg()) &&
X86II::isX86_64ExtendedReg(OutMI.getOperand(2).getReg())) {
unsigned Imm = MI->getOperand(3).getImm() & 0x7;
switch (Imm) {
default: break;
case 0x00: // EQUAL
case 0x03: // UNORDERED
case 0x04: // NOT EQUAL
case 0x07: // ORDERED
std::swap(OutMI.getOperand(1), OutMI.getOperand(2));
break;
}
}
break;
}
case X86::VMOVHLPSrr:
case X86::VUNPCKHPDrr:
// These are not truly commutable so hide them from the default case.
break;
default: {
// If the instruction is a commutable arithmetic instruction we might be
// able to commute the operands to get a 2 byte VEX prefix.
uint64_t TSFlags = MI->getDesc().TSFlags;
if (MI->getDesc().isCommutable() &&
(TSFlags & X86II::EncodingMask) == X86II::VEX &&
(TSFlags & X86II::OpMapMask) == X86II::TB &&
(TSFlags & X86II::FormMask) == X86II::MRMSrcReg &&
!(TSFlags & X86II::VEX_W) && (TSFlags & X86II::VEX_4V) &&
OutMI.getNumOperands() == 3) {
if (!X86II::isX86_64ExtendedReg(OutMI.getOperand(1).getReg()) &&
X86II::isX86_64ExtendedReg(OutMI.getOperand(2).getReg()))
std::swap(OutMI.getOperand(1), OutMI.getOperand(2));
}
break;
}
This commit introduces two fake instructions MORESTACK_RET and MORESTACK_RET_RESTORE_R10; which are lowered to a RET and a RET followed by a MOV respectively. Having a fake instruction prevents the verifier from seeing a MachineBasicBlock end with a non-terminator (MOV). It also prevents the rather eccentric case of a MachineBasicBlock ending with RET but having successors nevertheless. Patch by Sanjoy Das. llvm-svn: 143062
2011-10-27 05:12:27 +08:00
}
split MachineInstr -> MCInst lowering into its own class (not being embedded into X86ATTAsmPrinter). This still depends heavily on X86ATTAsmPrinter, but this is a step in the right direction. llvm-svn: 81627
2009-09-13 04:34:57 +08:00
}
[X86] Optimize stackmap shadows on X86. This patch minimizes the number of nops that must be emitted on X86 to satisfy stackmap shadow constraints. To minimize the number of nops inserted, the X86AsmPrinter now records the size of the most recent stackmap's shadow in the StackMapShadowTracker class, and tracks the number of instruction bytes emitted since the that stackmap instruction was encountered. Padding is emitted (if it is required at all) immediately before the next stackmap/patchpoint instruction, or at the end of the basic block. This optimization should reduce code-size and improve performance for people using the llvm stackmap intrinsic on X86. <rdar://problem/14959522> llvm-svn: 213892
2014-07-25 04:40:55 +08:00
void X86AsmPrinter::LowerTlsAddr(X86MCInstLower &MCInstLowering,
const MachineInstr &MI) {
[X86][BranchAlign] Suppress branch alignment for {,_}__tls_get_addr The x86-64 General Dynamic TLS code sequence uses prefixes to allow linker relaxation. Adding segment override prefix or NOPs can break linker relaxation (ld -pie/-no-pie). i386 General Dynamic and x86-64 Local Dynamic do not use prefixes, but for simplicity, just disable auto padding consistently. Reviewed By: skan, LuoYuanke Differential Revision: https://reviews.llvm.org/D72878
2020-01-17 06:08:31 +08:00
NoAutoPaddingScope NoPadScope(*OutStreamer);
[X86] Add TLS_(base_)addrX32 for X32 mode LLVM has TLS_(base_)addr32 for 32-bit TLS addresses in 32-bit mode, and TLS_(base_)addr64 for 64-bit TLS addresses in 64-bit mode. x32 mode wants 32-bit TLS addresses in 64-bit mode, which were not yet handled. This adds TLS_(base_)addrX32 as copies of TLS_(base_)addr64, except that they use tls32(base)addr rather than tls64(base)addr, and then restricts TLS_(base_)addr64 to 64-bit LP64 mode, TLS_(base_)addrX32 to 64-bit ILP32 mode. Reviewed By: RKSimon Differential Revision: https://reviews.llvm.org/D92346
2020-12-03 06:20:36 +08:00
bool Is64Bits = MI.getOpcode() != X86::TLS_addr32 &&
MI.getOpcode() != X86::TLS_base_addr32;
[X86] Avoid data16 prefix for lea in x32 mode The ABI demands a data16 prefix for lea in 64-bit LP64 mode, but not in 64-bit ILP32 mode. In both modes this prefix would ordinarily be ignored, but the instructions may be changed by the linker to instructions that are affected by the prefix. Reviewed By: RKSimon Differential Revision: https://reviews.llvm.org/D93157
2020-12-13 01:05:24 +08:00
bool Is64BitsLP64 = MI.getOpcode() == X86::TLS_addr64 ||
MI.getOpcode() == X86::TLS_base_addr64;
[X86] Support -fno-plt __tls_get_addr calls In general dynamic/local dynamic TLS models, with -fno-plt, * x86: emit `calll *___tls_get_addr@GOT(%ebx)` instead of `calll ___tls_get_addr@PLT` Note, on x86, if we can get rid of %ebx as the PIC register, it may be better to use a register not preserved across function calls. * x86_64: emit `callq *__tls_get_addr@GOTPCREL(%rip)` instead of `callq __tls_get_addr@PLT` Reorganize the code by separating 32-bit and 64-bit. Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D62106 llvm-svn: 361453
2019-05-23 09:05:13 +08:00
MCContext &Ctx = OutStreamer->getContext();
Implement the local-dynamic TLS model for x86 (PR3985) This implements codegen support for accesses to thread-local variables using the local-dynamic model, and adds a clean-up pass so that the base address for the TLS block can be re-used between local-dynamic access on an execution path. llvm-svn: 157818
2012-06-02 00:27:21 +08:00
MCSymbolRefExpr::VariantKind SRVK;
switch (MI.getOpcode()) {
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
case X86::TLS_addr32:
case X86::TLS_addr64:
[X86] Add TLS_(base_)addrX32 for X32 mode LLVM has TLS_(base_)addr32 for 32-bit TLS addresses in 32-bit mode, and TLS_(base_)addr64 for 64-bit TLS addresses in 64-bit mode. x32 mode wants 32-bit TLS addresses in 64-bit mode, which were not yet handled. This adds TLS_(base_)addrX32 as copies of TLS_(base_)addr64, except that they use tls32(base)addr rather than tls64(base)addr, and then restricts TLS_(base_)addr64 to 64-bit LP64 mode, TLS_(base_)addrX32 to 64-bit ILP32 mode. Reviewed By: RKSimon Differential Revision: https://reviews.llvm.org/D92346
2020-12-03 06:20:36 +08:00
case X86::TLS_addrX32:
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
SRVK = MCSymbolRefExpr::VK_TLSGD;
break;
case X86::TLS_base_addr32:
SRVK = MCSymbolRefExpr::VK_TLSLDM;
break;
case X86::TLS_base_addr64:
[X86] Add TLS_(base_)addrX32 for X32 mode LLVM has TLS_(base_)addr32 for 32-bit TLS addresses in 32-bit mode, and TLS_(base_)addr64 for 64-bit TLS addresses in 64-bit mode. x32 mode wants 32-bit TLS addresses in 64-bit mode, which were not yet handled. This adds TLS_(base_)addrX32 as copies of TLS_(base_)addr64, except that they use tls32(base)addr rather than tls64(base)addr, and then restricts TLS_(base_)addr64 to 64-bit LP64 mode, TLS_(base_)addrX32 to 64-bit ILP32 mode. Reviewed By: RKSimon Differential Revision: https://reviews.llvm.org/D92346
2020-12-03 06:20:36 +08:00
case X86::TLS_base_addrX32:
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
SRVK = MCSymbolRefExpr::VK_TLSLD;
break;
default:
llvm_unreachable("unexpected opcode");
Implement the local-dynamic TLS model for x86 (PR3985) This implements codegen support for accesses to thread-local variables using the local-dynamic model, and adds a clean-up pass so that the base address for the TLS block can be re-used between local-dynamic access on an execution path. llvm-svn: 157818
2012-06-02 00:27:21 +08:00
}
[X86] Support -fno-plt __tls_get_addr calls In general dynamic/local dynamic TLS models, with -fno-plt, * x86: emit `calll *___tls_get_addr@GOT(%ebx)` instead of `calll ___tls_get_addr@PLT` Note, on x86, if we can get rid of %ebx as the PIC register, it may be better to use a register not preserved across function calls. * x86_64: emit `callq *__tls_get_addr@GOTPCREL(%rip)` instead of `callq __tls_get_addr@PLT` Reorganize the code by separating 32-bit and 64-bit. Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D62106 llvm-svn: 361453
2019-05-23 09:05:13 +08:00
const MCSymbolRefExpr *Sym = MCSymbolRefExpr::create(
MCInstLowering.GetSymbolFromOperand(MI.getOperand(3)), SRVK, Ctx);
[X86] -fno-plt: use GOT __tls_get_addr only if GOTPCRELX is enabled Summary: As of binutils 2.32, ld has a bogus TLS relaxation error when the GD/LD code sequence using R_X86_64_GOTPCREL (instead of R_X86_64_GOTPCRELX) is attempted to be relaxed to IE/LE (binutils PR24784). gold and lld are good. In gcc/config/i386/i386.md, there is a configure-time check of as/ld support and the GOT relaxation will not be used if as/ld doesn't support it: if (flag_plt || !HAVE_AS_IX86_TLS_GET_ADDR_GOT) return "call\t%P2"; return "call\t{*%p2@GOT(%1)|[DWORD PTR %p2@GOT[%1]]}"; In clang, -DENABLE_X86_RELAX_RELOCATIONS=OFF is the default. The ld.bfd bogus error can be reproduced with: thread_local int a; int main() { return a; } clang -fno-plt -fpic a.cc -fuse-ld=bfd GOTPCRELX gained relative good support in 2016, which is considered relatively new. It is even difficult to conditionally default to -DENABLE_X86_RELAX_RELOCATIONS=ON due to cross compilation reasons. So work around the ld.bfd bug by only using GOT when GOTPCRELX is enabled. Reviewers: dalias, hjl.tools, nikic, rnk Reviewed By: nikic Subscribers: llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D64304 llvm-svn: 365752
2019-07-11 18:10:09 +08:00
// As of binutils 2.32, ld has a bogus TLS relaxation error when the GD/LD
// code sequence using R_X86_64_GOTPCREL (instead of R_X86_64_GOTPCRELX) is
// attempted to be relaxed to IE/LE (binutils PR24784). Work around the bug by
// only using GOT when GOTPCRELX is enabled.
// TODO Delete the workaround when GOTPCRELX becomes commonplace.
bool UseGot = MMI->getModule()->getRtLibUseGOT() &&
Ctx.getAsmInfo()->canRelaxRelocations();
[X86] Support -fno-plt __tls_get_addr calls In general dynamic/local dynamic TLS models, with -fno-plt, * x86: emit `calll *___tls_get_addr@GOT(%ebx)` instead of `calll ___tls_get_addr@PLT` Note, on x86, if we can get rid of %ebx as the PIC register, it may be better to use a register not preserved across function calls. * x86_64: emit `callq *__tls_get_addr@GOTPCREL(%rip)` instead of `callq __tls_get_addr@PLT` Reorganize the code by separating 32-bit and 64-bit. Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D62106 llvm-svn: 361453
2019-05-23 09:05:13 +08:00
if (Is64Bits) {
bool NeedsPadding = SRVK == MCSymbolRefExpr::VK_TLSGD;
[X86] Avoid data16 prefix for lea in x32 mode The ABI demands a data16 prefix for lea in 64-bit LP64 mode, but not in 64-bit ILP32 mode. In both modes this prefix would ordinarily be ignored, but the instructions may be changed by the linker to instructions that are affected by the prefix. Reviewed By: RKSimon Differential Revision: https://reviews.llvm.org/D93157
2020-12-13 01:05:24 +08:00
if (NeedsPadding && Is64BitsLP64)
[X86] Support -fno-plt __tls_get_addr calls In general dynamic/local dynamic TLS models, with -fno-plt, * x86: emit `calll *___tls_get_addr@GOT(%ebx)` instead of `calll ___tls_get_addr@PLT` Note, on x86, if we can get rid of %ebx as the PIC register, it may be better to use a register not preserved across function calls. * x86_64: emit `callq *__tls_get_addr@GOTPCREL(%rip)` instead of `callq __tls_get_addr@PLT` Reorganize the code by separating 32-bit and 64-bit. Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D62106 llvm-svn: 361453
2019-05-23 09:05:13 +08:00
EmitAndCountInstruction(MCInstBuilder(X86::DATA16_PREFIX));
EmitAndCountInstruction(MCInstBuilder(X86::LEA64r)
.addReg(X86::RDI)
.addReg(X86::RIP)
.addImm(1)
.addReg(0)
.addExpr(Sym)
.addReg(0));
const MCSymbol *TlsGetAddr = Ctx.getOrCreateSymbol("__tls_get_addr");
if (NeedsPadding) {
if (!UseGot)
EmitAndCountInstruction(MCInstBuilder(X86::DATA16_PREFIX));
EmitAndCountInstruction(MCInstBuilder(X86::DATA16_PREFIX));
EmitAndCountInstruction(MCInstBuilder(X86::REX64_PREFIX));
}
if (UseGot) {
const MCExpr *Expr = MCSymbolRefExpr::create(
TlsGetAddr, MCSymbolRefExpr::VK_GOTPCREL, Ctx);
EmitAndCountInstruction(MCInstBuilder(X86::CALL64m)
.addReg(X86::RIP)
.addImm(1)
.addReg(0)
.addExpr(Expr)
.addReg(0));
} else {
EmitAndCountInstruction(
MCInstBuilder(X86::CALL64pcrel32)
.addExpr(MCSymbolRefExpr::create(TlsGetAddr,
MCSymbolRefExpr::VK_PLT, Ctx)));
}
Move lowering of TLS_addr32 and TLS_addr64 to X86MCInstLower. llvm-svn: 120263
2010-11-29 05:16:39 +08:00
} else {
[X86] Support -fno-plt __tls_get_addr calls In general dynamic/local dynamic TLS models, with -fno-plt, * x86: emit `calll *___tls_get_addr@GOT(%ebx)` instead of `calll ___tls_get_addr@PLT` Note, on x86, if we can get rid of %ebx as the PIC register, it may be better to use a register not preserved across function calls. * x86_64: emit `callq *__tls_get_addr@GOTPCREL(%rip)` instead of `callq __tls_get_addr@PLT` Reorganize the code by separating 32-bit and 64-bit. Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D62106 llvm-svn: 361453
2019-05-23 09:05:13 +08:00
if (SRVK == MCSymbolRefExpr::VK_TLSGD && !UseGot) {
EmitAndCountInstruction(MCInstBuilder(X86::LEA32r)
.addReg(X86::EAX)
.addReg(0)
.addImm(1)
.addReg(X86::EBX)
.addExpr(Sym)
.addReg(0));
} else {
EmitAndCountInstruction(MCInstBuilder(X86::LEA32r)
.addReg(X86::EAX)
.addReg(X86::EBX)
.addImm(1)
.addReg(0)
.addExpr(Sym)
.addReg(0));
}
Move lowering of TLS_addr32 and TLS_addr64 to X86MCInstLower. llvm-svn: 120263
2010-11-29 05:16:39 +08:00
[X86] Support -fno-plt __tls_get_addr calls In general dynamic/local dynamic TLS models, with -fno-plt, * x86: emit `calll *___tls_get_addr@GOT(%ebx)` instead of `calll ___tls_get_addr@PLT` Note, on x86, if we can get rid of %ebx as the PIC register, it may be better to use a register not preserved across function calls. * x86_64: emit `callq *__tls_get_addr@GOTPCREL(%rip)` instead of `callq __tls_get_addr@PLT` Reorganize the code by separating 32-bit and 64-bit. Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D62106 llvm-svn: 361453
2019-05-23 09:05:13 +08:00
const MCSymbol *TlsGetAddr = Ctx.getOrCreateSymbol("___tls_get_addr");
if (UseGot) {
const MCExpr *Expr =
MCSymbolRefExpr::create(TlsGetAddr, MCSymbolRefExpr::VK_GOT, Ctx);
EmitAndCountInstruction(MCInstBuilder(X86::CALL32m)
.addReg(X86::EBX)
.addImm(1)
.addReg(0)
.addExpr(Expr)
.addReg(0));
} else {
EmitAndCountInstruction(
MCInstBuilder(X86::CALLpcrel32)
.addExpr(MCSymbolRefExpr::create(TlsGetAddr,
MCSymbolRefExpr::VK_PLT, Ctx)));
}
Move lowering of TLS_addr32 and TLS_addr64 to X86MCInstLower. llvm-svn: 120263
2010-11-29 05:16:39 +08:00
}
}
Emit debug info for byval parameters. llvm-svn: 102486
2010-04-28 09:39:28 +08:00
Remove \brief commands from doxygen comments. We've been running doxygen with the autobrief option for a couple of years now. This makes the \brief markers into our comments redundant. Since they are a visual distraction and we don't want to encourage more \brief markers in new code either, this patch removes them all. Patch produced by for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done Differential Revision: https://reviews.llvm.org/D46290 llvm-svn: 331272
2018-05-01 23:54:18 +08:00
/// Emit the largest nop instruction smaller than or equal to \p NumBytes
[X86MCInstLower] Clean up EmitNops; NFC Instead of having a conditional assert inside EmitNops, refactor so that the caller can have the assert instead. llvm-svn: 266793
2016-04-20 02:48:13 +08:00
/// bytes. Return the size of nop emitted.
[X86] Change signature of EmitNops. NFC. This is to support https://reviews.llvm.org/D81301.
2020-06-18 00:08:12 +08:00
static unsigned emitNop(MCStreamer &OS, unsigned NumBytes,
const X86Subtarget *Subtarget) {
[X86] Add back comment inadvertently lost in 1a1448e6568d9b11f198e510fa9c4cb6b1f4216a.
2020-07-28 00:40:26 +08:00
// Determine the longest nop which can be efficiently decoded for the given
// target cpu. 15-bytes is the longest single NOP instruction, but some
// platforms can't decode the longest forms efficiently.
[X86] Merge X86MCInstLowering's maxLongNopLength into emitNop and remove check for FeatureNOPL. The switch in emitNop uses 64-bit registers for nops exceeding 2 bytes. This isn't valid outside 64-bit mode. We could fix this easily enough, but there are no users that ask for more than 2 bytes outside 64-bit mode. Inlining the method to make the coupling between the two methods more explicit.
2020-07-26 13:05:46 +08:00
unsigned MaxNopLength = 1;
if (Subtarget->is64Bit()) {
// FIXME: We can use NOOPL on 32-bit targets with FeatureNOPL, but the
// IndexReg/BaseReg below need to be updated.
if (Subtarget->hasFeature(X86::FeatureFast7ByteNOP))
MaxNopLength = 7;
else if (Subtarget->hasFeature(X86::FeatureFast15ByteNOP))
MaxNopLength = 15;
else if (Subtarget->hasFeature(X86::FeatureFast11ByteNOP))
MaxNopLength = 11;
else
MaxNopLength = 10;
} if (Subtarget->is32Bit())
MaxNopLength = 2;
[X86] Adjust nop emission by compiler to consider target decode limitations The primary motivation of this change is to bring the code more closely in sync behavior wise with the assembler's version of nop emission. I'd like to eventually factor them into one, but that's hard to do when one has features the other doesn't. The longest encodeable nop on x86 is 15 bytes, but many processors - for instance all intel chips - can't decode the 15 byte form efficiently. On those processors, it's better to use either a 10 byte or 11 byte sequence depending.
2020-01-12 00:41:35 +08:00
// Cap a single nop emission at the profitable value for the target
[X86] Merge X86MCInstLowering's maxLongNopLength into emitNop and remove check for FeatureNOPL. The switch in emitNop uses 64-bit registers for nops exceeding 2 bytes. This isn't valid outside 64-bit mode. We could fix this easily enough, but there are no users that ask for more than 2 bytes outside 64-bit mode. Inlining the method to make the coupling between the two methods more explicit.
2020-07-26 13:05:46 +08:00
NumBytes = std::min(NumBytes, MaxNopLength);
[X86] Adjust nop emission by compiler to consider target decode limitations The primary motivation of this change is to bring the code more closely in sync behavior wise with the assembler's version of nop emission. I'd like to eventually factor them into one, but that's hard to do when one has features the other doesn't. The longest encodeable nop on x86 is 15 bytes, but many processors - for instance all intel chips - can't decode the 15 byte form efficiently. On those processors, it's better to use either a 10 byte or 11 byte sequence depending.
2020-01-12 00:41:35 +08:00
[X86MCInstLower] Clean up EmitNops; NFC Instead of having a conditional assert inside EmitNops, refactor so that the caller can have the assert instead. llvm-svn: 266793
2016-04-20 02:48:13 +08:00
unsigned NopSize;
unsigned Opc, BaseReg, ScaleVal, IndexReg, Displacement, SegmentReg;
Fixed "Value stored to 'Opc' is never read" warning. NFCI. llvm-svn: 360133
2019-05-07 19:09:16 +08:00
IndexReg = Displacement = SegmentReg = 0;
[X86MCInstLower] Clean up EmitNops; NFC Instead of having a conditional assert inside EmitNops, refactor so that the caller can have the assert instead. llvm-svn: 266793
2016-04-20 02:48:13 +08:00
BaseReg = X86::RAX;
ScaleVal = 1;
switch (NumBytes) {
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
case 0:
llvm_unreachable("Zero nops?");
break;
case 1:
NopSize = 1;
Opc = X86::NOOP;
break;
case 2:
NopSize = 2;
Opc = X86::XCHG16ar;
break;
case 3:
NopSize = 3;
Opc = X86::NOOPL;
break;
case 4:
NopSize = 4;
Opc = X86::NOOPL;
Displacement = 8;
break;
case 5:
NopSize = 5;
Opc = X86::NOOPL;
Displacement = 8;
IndexReg = X86::RAX;
break;
case 6:
NopSize = 6;
Opc = X86::NOOPW;
Displacement = 8;
IndexReg = X86::RAX;
break;
case 7:
NopSize = 7;
Opc = X86::NOOPL;
Displacement = 512;
break;
case 8:
NopSize = 8;
Opc = X86::NOOPL;
Displacement = 512;
IndexReg = X86::RAX;
break;
case 9:
NopSize = 9;
Opc = X86::NOOPW;
Displacement = 512;
IndexReg = X86::RAX;
break;
default:
NopSize = 10;
Opc = X86::NOOPW;
Displacement = 512;
IndexReg = X86::RAX;
SegmentReg = X86::CS;
break;
[X86MCInstLower] Clean up EmitNops; NFC Instead of having a conditional assert inside EmitNops, refactor so that the caller can have the assert instead. llvm-svn: 266793
2016-04-20 02:48:13 +08:00
}
[Stackmap] Emit multi-byte nops for X86. llvm-svn: 196334
2013-12-04 08:39:08 +08:00
[X86MCInstLower] Clean up EmitNops; NFC Instead of having a conditional assert inside EmitNops, refactor so that the caller can have the assert instead. llvm-svn: 266793
2016-04-20 02:48:13 +08:00
unsigned NumPrefixes = std::min(NumBytes - NopSize, 5U);
NopSize += NumPrefixes;
for (unsigned i = 0; i != NumPrefixes; ++i)
[MC] De-capitalize some MCStreamer::Emit* functions
2020-02-15 10:16:24 +08:00
OS.emitBytes("\x66");
Introduce a "patchable-function" function attribute Summary: The `"patchable-function"` attribute can be used by an LLVM client to influence LLVM's code generation in ways that makes the generated code easily patchable at runtime (for instance, to redirect control). Right now only one patchability scheme is supported, `"prologue-short-redirect"`, but this can be expanded in the future. Reviewers: joker.eph, rnk, echristo, dberris Subscribers: joker.eph, echristo, mcrosier, llvm-commits Differential Revision: http://reviews.llvm.org/D19046 llvm-svn: 266715
2016-04-19 13:24:47 +08:00
[X86MCInstLower] Clean up EmitNops; NFC Instead of having a conditional assert inside EmitNops, refactor so that the caller can have the assert instead. llvm-svn: 266793
2016-04-20 02:48:13 +08:00
switch (Opc) {
[X86] Put some cases in switch statements back on one line to be more compact and make it easier to see the similarities. NFC It looks like someone ran clang-format over this entire file which reformatted these switches into a multiline form. But I think the single line form is more useful here. llvm-svn: 336077
2018-07-02 14:42:42 +08:00
default: llvm_unreachable("Unexpected opcode");
[X86MCInstLower] Clean up EmitNops; NFC Instead of having a conditional assert inside EmitNops, refactor so that the caller can have the assert instead. llvm-svn: 266793
2016-04-20 02:48:13 +08:00
case X86::NOOP:
[X86] Change signature of EmitNops. NFC. This is to support https://reviews.llvm.org/D81301.
2020-06-18 00:08:12 +08:00
OS.emitInstruction(MCInstBuilder(Opc), *Subtarget);
[X86MCInstLower] Clean up EmitNops; NFC Instead of having a conditional assert inside EmitNops, refactor so that the caller can have the assert instead. llvm-svn: 266793
2016-04-20 02:48:13 +08:00
break;
case X86::XCHG16ar:
[X86] Change signature of EmitNops. NFC. This is to support https://reviews.llvm.org/D81301.
2020-06-18 00:08:12 +08:00
OS.emitInstruction(MCInstBuilder(Opc).addReg(X86::AX).addReg(X86::AX),
*Subtarget);
[X86MCInstLower] Clean up EmitNops; NFC Instead of having a conditional assert inside EmitNops, refactor so that the caller can have the assert instead. llvm-svn: 266793
2016-04-20 02:48:13 +08:00
break;
case X86::NOOPL:
case X86::NOOPW:
[AsmPrinter][MCStreamer] De-capitalize EmitInstruction and EmitCFI*
2020-02-14 13:58:16 +08:00
OS.emitInstruction(MCInstBuilder(Opc)
[X86MCInstLower] Clean up EmitNops; NFC Instead of having a conditional assert inside EmitNops, refactor so that the caller can have the assert instead. llvm-svn: 266793
2016-04-20 02:48:13 +08:00
.addReg(BaseReg)
.addImm(ScaleVal)
.addReg(IndexReg)
.addImm(Displacement)
.addReg(SegmentReg),
[X86] Change signature of EmitNops. NFC. This is to support https://reviews.llvm.org/D81301.
2020-06-18 00:08:12 +08:00
*Subtarget);
[X86MCInstLower] Clean up EmitNops; NFC Instead of having a conditional assert inside EmitNops, refactor so that the caller can have the assert instead. llvm-svn: 266793
2016-04-20 02:48:13 +08:00
break;
}
assert(NopSize <= NumBytes && "We overemitted?");
return NopSize;
}
Remove \brief commands from doxygen comments. We've been running doxygen with the autobrief option for a couple of years now. This makes the \brief markers into our comments redundant. Since they are a visual distraction and we don't want to encourage more \brief markers in new code either, this patch removes them all. Patch produced by for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done Differential Revision: https://reviews.llvm.org/D46290 llvm-svn: 331272
2018-05-01 23:54:18 +08:00
/// Emit the optimal amount of multi-byte nops on X86.
[X86] Change signature of EmitNops. NFC. This is to support https://reviews.llvm.org/D81301.
2020-06-18 00:08:12 +08:00
static void emitX86Nops(MCStreamer &OS, unsigned NumBytes,
const X86Subtarget *Subtarget) {
[MC] EmitNop: Make an assertion more useful. Differential Revision: http://reviews.llvm.org/D19334 llvm-svn: 266895
2016-04-21 01:53:21 +08:00
unsigned NopsToEmit = NumBytes;
[MC] Silence warning due to unused variable in !Debug builds. llvm-svn: 266901
2016-04-21 02:45:31 +08:00
(void)NopsToEmit;
[X86MCInstLower] Clean up EmitNops; NFC Instead of having a conditional assert inside EmitNops, refactor so that the caller can have the assert instead. llvm-svn: 266793
2016-04-20 02:48:13 +08:00
while (NumBytes) {
[X86] Change signature of EmitNops. NFC. This is to support https://reviews.llvm.org/D81301.
2020-06-18 00:08:12 +08:00
NumBytes -= emitNop(OS, NumBytes, Subtarget);
[MC] EmitNop: Make an assertion more useful. Differential Revision: http://reviews.llvm.org/D19334 llvm-svn: 266895
2016-04-21 01:53:21 +08:00
assert(NopsToEmit >= NumBytes && "Emitted more than I asked for!");
[X86MCInstLower] Clean up EmitNops; NFC Instead of having a conditional assert inside EmitNops, refactor so that the caller can have the assert instead. llvm-svn: 266793
2016-04-20 02:48:13 +08:00
}
[Stackmap] Emit multi-byte nops for X86. llvm-svn: 196334
2013-12-04 08:39:08 +08:00
}
[X86MCInst] Move LowerSTATEPOINT to inside X86AsmPrinter. NFC. llvm-svn: 236676
2015-05-07 07:53:26 +08:00
void X86AsmPrinter::LowerSTATEPOINT(const MachineInstr &MI,
X86MCInstLower &MCIL) {
assert(Subtarget->is64Bit() && "Statepoint currently only supports X86-64");
[Statepoints 2/4] Statepoint infrastructure for garbage collection: MI & x86-64 Backend This is the second patch in a small series. This patch contains the MachineInstruction and x86-64 backend pieces required to lower Statepoints. It does not include the code to actually generate the STATEPOINT machine instruction and as a result, the entire patch is currently dead code. I will be submitting the SelectionDAG parts within the next 24-48 hours. Since those pieces are by far the most complicated, I wanted to minimize the size of that patch. That patch will include the tests which exercise the functionality in this patch. The entire series can be seen as one combined whole in http://reviews.llvm.org/D5683. The STATEPOINT psuedo node is generated after all gc values are explicitly spilled to stack slots. The purpose of this node is to wrap an actual call instruction while recording the spill locations of the meta arguments used for garbage collection and other purposes. The STATEPOINT is modeled as modifing all of those locations to prevent backend optimizations from forwarding the value from before the STATEPOINT to after the STATEPOINT. (Doing so would break relocation semantics for collectors which wish to relocate roots.) The implementation of STATEPOINT is closely modeled on PATCHPOINT. Eventually, much of the code in this patch will be removed. The long term plan is to merge the functionality provided by statepoints and patchpoints. Merging their implementations in the backend is likely to be a good starting point. Reviewed by: atrick, ributzka llvm-svn: 223085
2014-12-02 06:52:56 +08:00
[BranchAlign] Compiler support for suppressing branch align As discussed heavily in the original review (D70157), there's a need for the compiler to be able to selective suppress padding (either nop or prefix) to respect assumptions about the meaning of labels and instructions in generated code. Rather than wait for syntax to be finalized - which appears to be a very slow process - this patch focuses on the compiler use case and *only* worries about the integrated assembler. To my knowledge, this covers all cases mentioned to date for clang/JIT support. For testing purposes, I wired it up so that if the integrated assembler was using autopadding for branch alignment (e.g. enabled at command line) then the textual assembly output would contain a comment for each location where padding was enabled or disabled. This seemed like the least painful choice overall. Note that the result of this patch effective disables the jcc errata mitigation for many constructs (statepoints, implicit null checks, xray, etc...) which is non ideal. It is at least *correct* and should allow us to enable the mitigation for the compiler. Once that's done, and a few other items are worked through, we probably want to come back to this an explore a bundling based approach instead so that we can pad instructions while keeping labels in the right place. Differential Revision: https://reviews.llvm.org/D72303
2020-01-09 01:58:42 +08:00
NoAutoPaddingScope NoPadScope(*OutStreamer);
[Statepoints] Support for "patchable" statepoints. Summary: This change adds two new parameters to the statepoint intrinsic, `i64 id` and `i32 num_patch_bytes`. `id` gets propagated to the ID field in the generated StackMap section. If the `num_patch_bytes` is non-zero then the statepoint is lowered to `num_patch_bytes` bytes of nops instead of a call (the spill and reload code remains unchanged). A non-zero `num_patch_bytes` is useful in situations where a language runtime requires complete control over how a call is lowered. This change brings statepoints one step closer to patchpoints. With some additional work (that is not part of this patch) it should be possible to get rid of `TargetOpcode::STATEPOINT` altogether. PlaceSafepoints generates `statepoint` wrappers with `id` set to `0xABCDEF00` (the old default value for the ID reported in the stackmap) and `num_patch_bytes` set to `0`. This can be made more sophisticated later. Reviewers: reames, pgavlin, swaroop.sridhar, AndyAyers Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D9546 llvm-svn: 237214
2015-05-13 07:52:24 +08:00
StatepointOpers SOpers(&MI);
if (unsigned PatchBytes = SOpers.getNumPatchBytes()) {
[X86] Change signature of EmitNops. NFC. This is to support https://reviews.llvm.org/D81301.
2020-06-18 00:08:12 +08:00
emitX86Nops(*OutStreamer, PatchBytes, Subtarget);
[Statepoints] Support for "patchable" statepoints. Summary: This change adds two new parameters to the statepoint intrinsic, `i64 id` and `i32 num_patch_bytes`. `id` gets propagated to the ID field in the generated StackMap section. If the `num_patch_bytes` is non-zero then the statepoint is lowered to `num_patch_bytes` bytes of nops instead of a call (the spill and reload code remains unchanged). A non-zero `num_patch_bytes` is useful in situations where a language runtime requires complete control over how a call is lowered. This change brings statepoints one step closer to patchpoints. With some additional work (that is not part of this patch) it should be possible to get rid of `TargetOpcode::STATEPOINT` altogether. PlaceSafepoints generates `statepoint` wrappers with `id` set to `0xABCDEF00` (the old default value for the ID reported in the stackmap) and `num_patch_bytes` set to `0`. This can be made more sophisticated later. Reviewers: reames, pgavlin, swaroop.sridhar, AndyAyers Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D9546 llvm-svn: 237214
2015-05-13 07:52:24 +08:00
} else {
// Lower call target and choose correct opcode
const MachineOperand &CallTarget = SOpers.getCallTarget();
MCOperand CallTargetMCOp;
unsigned CallOpcode;
switch (CallTarget.getType()) {
case MachineOperand::MO_GlobalAddress:
case MachineOperand::MO_ExternalSymbol:
CallTargetMCOp = MCIL.LowerSymbolOperand(
CallTarget, MCIL.GetSymbolFromOperand(CallTarget));
CallOpcode = X86::CALL64pcrel32;
// Currently, we only support relative addressing with statepoints.
// Otherwise, we'll need a scratch register to hold the target
// address. You'll fail asserts during load & relocation if this
// symbol is to far away. (TODO: support non-relative addressing)
break;
case MachineOperand::MO_Immediate:
MC: Modernize MCOperand API naming. NFC. MCOperand::Create*() methods renamed to MCOperand::create*(). llvm-svn: 237275
2015-05-14 02:37:00 +08:00
CallTargetMCOp = MCOperand::createImm(CallTarget.getImm());
[Statepoints] Support for "patchable" statepoints. Summary: This change adds two new parameters to the statepoint intrinsic, `i64 id` and `i32 num_patch_bytes`. `id` gets propagated to the ID field in the generated StackMap section. If the `num_patch_bytes` is non-zero then the statepoint is lowered to `num_patch_bytes` bytes of nops instead of a call (the spill and reload code remains unchanged). A non-zero `num_patch_bytes` is useful in situations where a language runtime requires complete control over how a call is lowered. This change brings statepoints one step closer to patchpoints. With some additional work (that is not part of this patch) it should be possible to get rid of `TargetOpcode::STATEPOINT` altogether. PlaceSafepoints generates `statepoint` wrappers with `id` set to `0xABCDEF00` (the old default value for the ID reported in the stackmap) and `num_patch_bytes` set to `0`. This can be made more sophisticated later. Reviewers: reames, pgavlin, swaroop.sridhar, AndyAyers Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D9546 llvm-svn: 237214
2015-05-13 07:52:24 +08:00
CallOpcode = X86::CALL64pcrel32;
// Currently, we only support relative addressing with statepoints.
// Otherwise, we'll need a scratch register to hold the target
// immediate. You'll fail asserts during load & relocation if this
// address is to far away. (TODO: support non-relative addressing)
break;
case MachineOperand::MO_Register:
Introduce the "retpoline" x86 mitigation technique for variant #2 of the speculative execution vulnerabilities disclosed today, specifically identified by CVE-2017-5715, "Branch Target Injection", and is one of the two halves to Spectre.. Summary: First, we need to explain the core of the vulnerability. Note that this is a very incomplete description, please see the Project Zero blog post for details: https://googleprojectzero.blogspot.com/2018/01/reading-privileged-memory-with-side.html The basis for branch target injection is to direct speculative execution of the processor to some "gadget" of executable code by poisoning the prediction of indirect branches with the address of that gadget. The gadget in turn contains an operation that provides a side channel for reading data. Most commonly, this will look like a load of secret data followed by a branch on the loaded value and then a load of some predictable cache line. The attacker then uses timing of the processors cache to determine which direction the branch took *in the speculative execution*, and in turn what one bit of the loaded value was. Due to the nature of these timing side channels and the branch predictor on Intel processors, this allows an attacker to leak data only accessible to a privileged domain (like the kernel) back into an unprivileged domain. The goal is simple: avoid generating code which contains an indirect branch that could have its prediction poisoned by an attacker. In many cases, the compiler can simply use directed conditional branches and a small search tree. LLVM already has support for lowering switches in this way and the first step of this patch is to disable jump-table lowering of switches and introduce a pass to rewrite explicit indirectbr sequences into a switch over integers. However, there is no fully general alternative to indirect calls. We introduce a new construct we call a "retpoline" to implement indirect calls in a non-speculatable way. It can be thought of loosely as a trampoline for indirect calls which uses the RET instruction on x86. Further, we arrange for a specific call->ret sequence which ensures the processor predicts the return to go to a controlled, known location. The retpoline then "smashes" the return address pushed onto the stack by the call with the desired target of the original indirect call. The result is a predicted return to the next instruction after a call (which can be used to trap speculative execution within an infinite loop) and an actual indirect branch to an arbitrary address. On 64-bit x86 ABIs, this is especially easily done in the compiler by using a guaranteed scratch register to pass the target into this device. For 32-bit ABIs there isn't a guaranteed scratch register and so several different retpoline variants are introduced to use a scratch register if one is available in the calling convention and to otherwise use direct stack push/pop sequences to pass the target address. This "retpoline" mitigation is fully described in the following blog post: https://support.google.com/faqs/answer/7625886 We also support a target feature that disables emission of the retpoline thunk by the compiler to allow for custom thunks if users want them. These are particularly useful in environments like kernels that routinely do hot-patching on boot and want to hot-patch their thunk to different code sequences. They can write this custom thunk and use `-mretpoline-external-thunk` *in addition* to `-mretpoline`. In this case, on x86-64 thu thunk names must be: ``` __llvm_external_retpoline_r11 ``` or on 32-bit: ``` __llvm_external_retpoline_eax __llvm_external_retpoline_ecx __llvm_external_retpoline_edx __llvm_external_retpoline_push ``` And the target of the retpoline is passed in the named register, or in the case of the `push` suffix on the top of the stack via a `pushl` instruction. There is one other important source of indirect branches in x86 ELF binaries: the PLT. These patches also include support for LLD to generate PLT entries that perform a retpoline-style indirection. The only other indirect branches remaining that we are aware of are from precompiled runtimes (such as crt0.o and similar). The ones we have found are not really attackable, and so we have not focused on them here, but eventually these runtimes should also be replicated for retpoline-ed configurations for completeness. For kernels or other freestanding or fully static executables, the compiler switch `-mretpoline` is sufficient to fully mitigate this particular attack. For dynamic executables, you must compile *all* libraries with `-mretpoline` and additionally link the dynamic executable and all shared libraries with LLD and pass `-z retpolineplt` (or use similar functionality from some other linker). We strongly recommend also using `-z now` as non-lazy binding allows the retpoline-mitigated PLT to be substantially smaller. When manually apply similar transformations to `-mretpoline` to the Linux kernel we observed very small performance hits to applications running typical workloads, and relatively minor hits (approximately 2%) even for extremely syscall-heavy applications. This is largely due to the small number of indirect branches that occur in performance sensitive paths of the kernel. When using these patches on statically linked applications, especially C++ applications, you should expect to see a much more dramatic performance hit. For microbenchmarks that are switch, indirect-, or virtual-call heavy we have seen overheads ranging from 10% to 50%. However, real-world workloads exhibit substantially lower performance impact. Notably, techniques such as PGO and ThinLTO dramatically reduce the impact of hot indirect calls (by speculatively promoting them to direct calls) and allow optimized search trees to be used to lower switches. If you need to deploy these techniques in C++ applications, we *strongly* recommend that you ensure all hot call targets are statically linked (avoiding PLT indirection) and use both PGO and ThinLTO. Well tuned servers using all of these techniques saw 5% - 10% overhead from the use of retpoline. We will add detailed documentation covering these components in subsequent patches, but wanted to make the core functionality available as soon as possible. Happy for more code review, but we'd really like to get these patches landed and backported ASAP for obvious reasons. We're planning to backport this to both 6.0 and 5.0 release streams and get a 5.0 release with just this cherry picked ASAP for distros and vendors. This patch is the work of a number of people over the past month: Eric, Reid, Rui, and myself. I'm mailing it out as a single commit due to the time sensitive nature of landing this and the need to backport it. Huge thanks to everyone who helped out here, and everyone at Intel who helped out in discussions about how to craft this. Also, credit goes to Paul Turner (at Google, but not an LLVM contributor) for much of the underlying retpoline design. Reviewers: echristo, rnk, ruiu, craig.topper, DavidKreitzer Subscribers: sanjoy, emaste, mcrosier, mgorny, mehdi_amini, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D41723 llvm-svn: 323155
2018-01-23 06:05:25 +08:00
// FIXME: Add retpoline support and remove this.
[X86][NFC] Generalize the naming of "Retpoline Thunks" and related code to "Indirect Thunks" There are applications for indirect call/branch thunks other than retpoline for Spectre v2, e.g., https://software.intel.com/security-software-guidance/software-guidance/load-value-injection Therefore it makes sense to refactor X86RetpolineThunks as a more general capability. Differential Revision: https://reviews.llvm.org/D76810
2020-04-03 12:00:44 +08:00
if (Subtarget->useIndirectThunkCalls())
report_fatal_error("Lowering register statepoints with thunks not "
Introduce the "retpoline" x86 mitigation technique for variant #2 of the speculative execution vulnerabilities disclosed today, specifically identified by CVE-2017-5715, "Branch Target Injection", and is one of the two halves to Spectre.. Summary: First, we need to explain the core of the vulnerability. Note that this is a very incomplete description, please see the Project Zero blog post for details: https://googleprojectzero.blogspot.com/2018/01/reading-privileged-memory-with-side.html The basis for branch target injection is to direct speculative execution of the processor to some "gadget" of executable code by poisoning the prediction of indirect branches with the address of that gadget. The gadget in turn contains an operation that provides a side channel for reading data. Most commonly, this will look like a load of secret data followed by a branch on the loaded value and then a load of some predictable cache line. The attacker then uses timing of the processors cache to determine which direction the branch took *in the speculative execution*, and in turn what one bit of the loaded value was. Due to the nature of these timing side channels and the branch predictor on Intel processors, this allows an attacker to leak data only accessible to a privileged domain (like the kernel) back into an unprivileged domain. The goal is simple: avoid generating code which contains an indirect branch that could have its prediction poisoned by an attacker. In many cases, the compiler can simply use directed conditional branches and a small search tree. LLVM already has support for lowering switches in this way and the first step of this patch is to disable jump-table lowering of switches and introduce a pass to rewrite explicit indirectbr sequences into a switch over integers. However, there is no fully general alternative to indirect calls. We introduce a new construct we call a "retpoline" to implement indirect calls in a non-speculatable way. It can be thought of loosely as a trampoline for indirect calls which uses the RET instruction on x86. Further, we arrange for a specific call->ret sequence which ensures the processor predicts the return to go to a controlled, known location. The retpoline then "smashes" the return address pushed onto the stack by the call with the desired target of the original indirect call. The result is a predicted return to the next instruction after a call (which can be used to trap speculative execution within an infinite loop) and an actual indirect branch to an arbitrary address. On 64-bit x86 ABIs, this is especially easily done in the compiler by using a guaranteed scratch register to pass the target into this device. For 32-bit ABIs there isn't a guaranteed scratch register and so several different retpoline variants are introduced to use a scratch register if one is available in the calling convention and to otherwise use direct stack push/pop sequences to pass the target address. This "retpoline" mitigation is fully described in the following blog post: https://support.google.com/faqs/answer/7625886 We also support a target feature that disables emission of the retpoline thunk by the compiler to allow for custom thunks if users want them. These are particularly useful in environments like kernels that routinely do hot-patching on boot and want to hot-patch their thunk to different code sequences. They can write this custom thunk and use `-mretpoline-external-thunk` *in addition* to `-mretpoline`. In this case, on x86-64 thu thunk names must be: ``` __llvm_external_retpoline_r11 ``` or on 32-bit: ``` __llvm_external_retpoline_eax __llvm_external_retpoline_ecx __llvm_external_retpoline_edx __llvm_external_retpoline_push ``` And the target of the retpoline is passed in the named register, or in the case of the `push` suffix on the top of the stack via a `pushl` instruction. There is one other important source of indirect branches in x86 ELF binaries: the PLT. These patches also include support for LLD to generate PLT entries that perform a retpoline-style indirection. The only other indirect branches remaining that we are aware of are from precompiled runtimes (such as crt0.o and similar). The ones we have found are not really attackable, and so we have not focused on them here, but eventually these runtimes should also be replicated for retpoline-ed configurations for completeness. For kernels or other freestanding or fully static executables, the compiler switch `-mretpoline` is sufficient to fully mitigate this particular attack. For dynamic executables, you must compile *all* libraries with `-mretpoline` and additionally link the dynamic executable and all shared libraries with LLD and pass `-z retpolineplt` (or use similar functionality from some other linker). We strongly recommend also using `-z now` as non-lazy binding allows the retpoline-mitigated PLT to be substantially smaller. When manually apply similar transformations to `-mretpoline` to the Linux kernel we observed very small performance hits to applications running typical workloads, and relatively minor hits (approximately 2%) even for extremely syscall-heavy applications. This is largely due to the small number of indirect branches that occur in performance sensitive paths of the kernel. When using these patches on statically linked applications, especially C++ applications, you should expect to see a much more dramatic performance hit. For microbenchmarks that are switch, indirect-, or virtual-call heavy we have seen overheads ranging from 10% to 50%. However, real-world workloads exhibit substantially lower performance impact. Notably, techniques such as PGO and ThinLTO dramatically reduce the impact of hot indirect calls (by speculatively promoting them to direct calls) and allow optimized search trees to be used to lower switches. If you need to deploy these techniques in C++ applications, we *strongly* recommend that you ensure all hot call targets are statically linked (avoiding PLT indirection) and use both PGO and ThinLTO. Well tuned servers using all of these techniques saw 5% - 10% overhead from the use of retpoline. We will add detailed documentation covering these components in subsequent patches, but wanted to make the core functionality available as soon as possible. Happy for more code review, but we'd really like to get these patches landed and backported ASAP for obvious reasons. We're planning to backport this to both 6.0 and 5.0 release streams and get a 5.0 release with just this cherry picked ASAP for distros and vendors. This patch is the work of a number of people over the past month: Eric, Reid, Rui, and myself. I'm mailing it out as a single commit due to the time sensitive nature of landing this and the need to backport it. Huge thanks to everyone who helped out here, and everyone at Intel who helped out in discussions about how to craft this. Also, credit goes to Paul Turner (at Google, but not an LLVM contributor) for much of the underlying retpoline design. Reviewers: echristo, rnk, ruiu, craig.topper, DavidKreitzer Subscribers: sanjoy, emaste, mcrosier, mgorny, mehdi_amini, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D41723 llvm-svn: 323155
2018-01-23 06:05:25 +08:00
"yet implemented.");
MC: Modernize MCOperand API naming. NFC. MCOperand::Create*() methods renamed to MCOperand::create*(). llvm-svn: 237275
2015-05-14 02:37:00 +08:00
CallTargetMCOp = MCOperand::createReg(CallTarget.getReg());
[Statepoints] Support for "patchable" statepoints. Summary: This change adds two new parameters to the statepoint intrinsic, `i64 id` and `i32 num_patch_bytes`. `id` gets propagated to the ID field in the generated StackMap section. If the `num_patch_bytes` is non-zero then the statepoint is lowered to `num_patch_bytes` bytes of nops instead of a call (the spill and reload code remains unchanged). A non-zero `num_patch_bytes` is useful in situations where a language runtime requires complete control over how a call is lowered. This change brings statepoints one step closer to patchpoints. With some additional work (that is not part of this patch) it should be possible to get rid of `TargetOpcode::STATEPOINT` altogether. PlaceSafepoints generates `statepoint` wrappers with `id` set to `0xABCDEF00` (the old default value for the ID reported in the stackmap) and `num_patch_bytes` set to `0`. This can be made more sophisticated later. Reviewers: reames, pgavlin, swaroop.sridhar, AndyAyers Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D9546 llvm-svn: 237214
2015-05-13 07:52:24 +08:00
CallOpcode = X86::CALL64r;
break;
default:
llvm_unreachable("Unsupported operand type in statepoint call target");
break;
}
// Emit call
MCInst CallInst;
CallInst.setOpcode(CallOpcode);
CallInst.addOperand(CallTargetMCOp);
[AsmPrinter][MCStreamer] De-capitalize EmitInstruction and EmitCFI*
2020-02-14 13:58:16 +08:00
OutStreamer->emitInstruction(CallInst, getSubtargetInfo());
[Statepoints] Support for "patchable" statepoints. Summary: This change adds two new parameters to the statepoint intrinsic, `i64 id` and `i32 num_patch_bytes`. `id` gets propagated to the ID field in the generated StackMap section. If the `num_patch_bytes` is non-zero then the statepoint is lowered to `num_patch_bytes` bytes of nops instead of a call (the spill and reload code remains unchanged). A non-zero `num_patch_bytes` is useful in situations where a language runtime requires complete control over how a call is lowered. This change brings statepoints one step closer to patchpoints. With some additional work (that is not part of this patch) it should be possible to get rid of `TargetOpcode::STATEPOINT` altogether. PlaceSafepoints generates `statepoint` wrappers with `id` set to `0xABCDEF00` (the old default value for the ID reported in the stackmap) and `num_patch_bytes` set to `0`. This can be made more sophisticated later. Reviewers: reames, pgavlin, swaroop.sridhar, AndyAyers Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D9546 llvm-svn: 237214
2015-05-13 07:52:24 +08:00
}
[Statepoints 2/4] Statepoint infrastructure for garbage collection: MI & x86-64 Backend This is the second patch in a small series. This patch contains the MachineInstruction and x86-64 backend pieces required to lower Statepoints. It does not include the code to actually generate the STATEPOINT machine instruction and as a result, the entire patch is currently dead code. I will be submitting the SelectionDAG parts within the next 24-48 hours. Since those pieces are by far the most complicated, I wanted to minimize the size of that patch. That patch will include the tests which exercise the functionality in this patch. The entire series can be seen as one combined whole in http://reviews.llvm.org/D5683. The STATEPOINT psuedo node is generated after all gc values are explicitly spilled to stack slots. The purpose of this node is to wrap an actual call instruction while recording the spill locations of the meta arguments used for garbage collection and other purposes. The STATEPOINT is modeled as modifing all of those locations to prevent backend optimizations from forwarding the value from before the STATEPOINT to after the STATEPOINT. (Doing so would break relocation semantics for collectors which wish to relocate roots.) The implementation of STATEPOINT is closely modeled on PATCHPOINT. Eventually, much of the code in this patch will be removed. The long term plan is to merge the functionality provided by statepoints and patchpoints. Merging their implementations in the backend is likely to be a good starting point. Reviewed by: atrick, ributzka llvm-svn: 223085
2014-12-02 06:52:56 +08:00
// Record our statepoint node in the same section used by STACKMAP
// and PATCHPOINT
[StackMaps] Be explicit about label formation [NFC] (try 2) Recommit after making the same API change in non-x86 targets. This has been build for all targets, and tested for effected ones. Why the difference? Because my disk filled up when I tried make check for all. For auto-padding assembler support, we'll need to bundle the label with the instructions (nops or call sequences) so that they don't get separated. This just rearranges the code to make the upcoming change more obvious.
2019-12-20 06:03:19 +08:00
auto &Ctx = OutStreamer->getContext();
MCSymbol *MILabel = Ctx.createTempSymbol();
[MC] De-capitalize another set of MCStreamer::Emit* functions Emit{ValueTo,Code}Alignment Emit{DTP,TP,GP}* EmitSymbolValue etc
2020-02-15 11:21:58 +08:00
OutStreamer->emitLabel(MILabel);
[StackMaps] Be explicit about label formation [NFC] (try 2) Recommit after making the same API change in non-x86 targets. This has been build for all targets, and tested for effected ones. Why the difference? Because my disk filled up when I tried make check for all. For auto-padding assembler support, we'll need to bundle the label with the instructions (nops or call sequences) so that they don't get separated. This just rearranges the code to make the upcoming change more obvious.
2019-12-20 06:03:19 +08:00
SM.recordStatepoint(*MILabel, MI);
[Statepoints 2/4] Statepoint infrastructure for garbage collection: MI & x86-64 Backend This is the second patch in a small series. This patch contains the MachineInstruction and x86-64 backend pieces required to lower Statepoints. It does not include the code to actually generate the STATEPOINT machine instruction and as a result, the entire patch is currently dead code. I will be submitting the SelectionDAG parts within the next 24-48 hours. Since those pieces are by far the most complicated, I wanted to minimize the size of that patch. That patch will include the tests which exercise the functionality in this patch. The entire series can be seen as one combined whole in http://reviews.llvm.org/D5683. The STATEPOINT psuedo node is generated after all gc values are explicitly spilled to stack slots. The purpose of this node is to wrap an actual call instruction while recording the spill locations of the meta arguments used for garbage collection and other purposes. The STATEPOINT is modeled as modifing all of those locations to prevent backend optimizations from forwarding the value from before the STATEPOINT to after the STATEPOINT. (Doing so would break relocation semantics for collectors which wish to relocate roots.) The implementation of STATEPOINT is closely modeled on PATCHPOINT. Eventually, much of the code in this patch will be removed. The long term plan is to merge the functionality provided by statepoints and patchpoints. Merging their implementations in the backend is likely to be a good starting point. Reviewed by: atrick, ributzka llvm-svn: 223085
2014-12-02 06:52:56 +08:00
}
[ImplicitNullCheck] Extend Implicit Null Check scope by using stores Summary: This change allows usage of store instruction for implicit null check. Memory Aliasing Analisys is not used and change conservatively supposes that any store and load may access the same memory. As a result re-ordering of store-store, store-load and load-store is prohibited. Patch by Serguei Katkov! Reviewers: reames, sanjoy Reviewed By: sanjoy Subscribers: atrick, llvm-commits Differential Revision: https://reviews.llvm.org/D29400 llvm-svn: 294338
2017-02-08 03:19:49 +08:00
void X86AsmPrinter::LowerFAULTING_OP(const MachineInstr &FaultingMI,
X86MCInstLower &MCIL) {
// FAULTING_LOAD_OP <def>, <faltinf type>, <MBB handler>,
// <opcode>, <operands>
[CodeGen] Introduce a FAULTING_LOAD_OP pseudo-op. Summary: This instruction encodes a loading operation that may fault, and a label to branch to if the load page-faults. The locations of potentially faulting loads and their "handler" destinations are recorded in a FaultMap section, meant to be consumed by LLVM's clients. Nothing generates FAULTING_LOAD_OP instructions yet, but they will be used in a future change. The documentation (FaultMaps.rst) needs improvement and I will update this diff with a more expanded version shortly. Depends on D10196 Reviewers: rnk, reames, AndyAyers, ab, atrick, pgavlin Reviewed By: atrick, pgavlin Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D10197 llvm-svn: 239740
2015-06-16 02:44:08 +08:00
[BranchAlign] Compiler support for suppressing branch align As discussed heavily in the original review (D70157), there's a need for the compiler to be able to selective suppress padding (either nop or prefix) to respect assumptions about the meaning of labels and instructions in generated code. Rather than wait for syntax to be finalized - which appears to be a very slow process - this patch focuses on the compiler use case and *only* worries about the integrated assembler. To my knowledge, this covers all cases mentioned to date for clang/JIT support. For testing purposes, I wired it up so that if the integrated assembler was using autopadding for branch alignment (e.g. enabled at command line) then the textual assembly output would contain a comment for each location where padding was enabled or disabled. This seemed like the least painful choice overall. Note that the result of this patch effective disables the jcc errata mitigation for many constructs (statepoints, implicit null checks, xray, etc...) which is non ideal. It is at least *correct* and should allow us to enable the mitigation for the compiler. Once that's done, and a few other items are worked through, we probably want to come back to this an explore a bundling based approach instead so that we can pad instructions while keeping labels in the right place. Differential Revision: https://reviews.llvm.org/D72303
2020-01-09 01:58:42 +08:00
NoAutoPaddingScope NoPadScope(*OutStreamer);
Apply llvm-prefer-register-over-unsigned from clang-tidy to LLVM Summary: This clang-tidy check is looking for unsigned integer variables whose initializer starts with an implicit cast from llvm::Register and changes the type of the variable to llvm::Register (dropping the llvm:: where possible). Partial reverts in: X86FrameLowering.cpp - Some functions return unsigned and arguably should be MCRegister X86FixupLEAs.cpp - Some functions return unsigned and arguably should be MCRegister X86FrameLowering.cpp - Some functions return unsigned and arguably should be MCRegister HexagonBitSimplify.cpp - Function takes BitTracker::RegisterRef which appears to be unsigned& MachineVerifier.cpp - Ambiguous operator==() given MCRegister and const Register PPCFastISel.cpp - No Register::operator-=() PeepholeOptimizer.cpp - TargetInstrInfo::optimizeLoadInstr() takes an unsigned& MachineTraceMetrics.cpp - MachineTraceMetrics lacks a suitable constructor Manual fixups in: ARMFastISel.cpp - ARMEmitLoad() now takes a Register& instead of unsigned& HexagonSplitDouble.cpp - Ternary operator was ambiguous between unsigned/Register HexagonConstExtenders.cpp - Has a local class named Register, used llvm::Register instead of Register. PPCFastISel.cpp - PPCEmitLoad() now takes a Register& instead of unsigned& Depends on D65919 Reviewers: arsenm, bogner, craig.topper, RKSimon Reviewed By: arsenm Subscribers: RKSimon, craig.topper, lenary, aemerson, wuzish, jholewinski, MatzeB, qcolombet, dschuff, jyknight, dylanmckay, sdardis, nemanjai, jvesely, wdng, nhaehnle, sbc100, jgravelle-google, kristof.beyls, hiraditya, aheejin, kbarton, fedor.sergeev, javed.absar, asb, rbar, johnrusso, simoncook, apazos, sabuasal, niosHD, jrtc27, MaskRay, zzheng, edward-jones, atanasyan, rogfer01, MartinMosbeck, brucehoult, the_o, tpr, PkmX, jocewei, jsji, Petar.Avramovic, asbirlea, Jim, s.egerton, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D65962 llvm-svn: 369041
2019-08-16 03:22:08 +08:00
Register DefRegister = FaultingMI.getOperand(0).getReg();
[ImplicitNullCheck] Extend Implicit Null Check scope by using stores Summary: This change allows usage of store instruction for implicit null check. Memory Aliasing Analisys is not used and change conservatively supposes that any store and load may access the same memory. As a result re-ordering of store-store, store-load and load-store is prohibited. Patch by Serguei Katkov! Reviewers: reames, sanjoy Reviewed By: sanjoy Subscribers: atrick, llvm-commits Differential Revision: https://reviews.llvm.org/D29400 llvm-svn: 294338
2017-02-08 03:19:49 +08:00
FaultMaps::FaultKind FK =
static_cast<FaultMaps::FaultKind>(FaultingMI.getOperand(1).getImm());
MCSymbol *HandlerLabel = FaultingMI.getOperand(2).getMBB()->getSymbol();
unsigned Opcode = FaultingMI.getOperand(3).getImm();
unsigned OperandsBeginIdx = 4;
[CodeGen] Introduce a FAULTING_LOAD_OP pseudo-op. Summary: This instruction encodes a loading operation that may fault, and a label to branch to if the load page-faults. The locations of potentially faulting loads and their "handler" destinations are recorded in a FaultMap section, meant to be consumed by LLVM's clients. Nothing generates FAULTING_LOAD_OP instructions yet, but they will be used in a future change. The documentation (FaultMaps.rst) needs improvement and I will update this diff with a more expanded version shortly. Depends on D10196 Reviewers: rnk, reames, AndyAyers, ab, atrick, pgavlin Reviewed By: atrick, pgavlin Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D10197 llvm-svn: 239740
2015-06-16 02:44:08 +08:00
[FaultMaps] Make label formation a bit more explicit [NFC] This is in advance of assembler padding directives support where we'll need to bundle the label w/the corresponding faulting instruction to avoid padding being inserted between.
2019-12-20 04:01:51 +08:00
auto &Ctx = OutStreamer->getContext();
MCSymbol *FaultingLabel = Ctx.createTempSymbol();
[MC] De-capitalize another set of MCStreamer::Emit* functions Emit{ValueTo,Code}Alignment Emit{DTP,TP,GP}* EmitSymbolValue etc
2020-02-15 11:21:58 +08:00
OutStreamer->emitLabel(FaultingLabel);
[FaultMaps] Make label formation a bit more explicit [NFC] This is in advance of assembler padding directives support where we'll need to bundle the label w/the corresponding faulting instruction to avoid padding being inserted between.
2019-12-20 04:01:51 +08:00
[ImplicitNullCheck] Extend Implicit Null Check scope by using stores Summary: This change allows usage of store instruction for implicit null check. Memory Aliasing Analisys is not used and change conservatively supposes that any store and load may access the same memory. As a result re-ordering of store-store, store-load and load-store is prohibited. Patch by Serguei Katkov! Reviewers: reames, sanjoy Reviewed By: sanjoy Subscribers: atrick, llvm-commits Differential Revision: https://reviews.llvm.org/D29400 llvm-svn: 294338
2017-02-08 03:19:49 +08:00
assert(FK < FaultMaps::FaultKindMax && "Invalid Faulting Kind!");
[FaultMaps] Make label formation a bit more explicit [NFC] This is in advance of assembler padding directives support where we'll need to bundle the label w/the corresponding faulting instruction to avoid padding being inserted between.
2019-12-20 04:01:51 +08:00
FM.recordFaultingOp(FK, FaultingLabel, HandlerLabel);
[CodeGen] Introduce a FAULTING_LOAD_OP pseudo-op. Summary: This instruction encodes a loading operation that may fault, and a label to branch to if the load page-faults. The locations of potentially faulting loads and their "handler" destinations are recorded in a FaultMap section, meant to be consumed by LLVM's clients. Nothing generates FAULTING_LOAD_OP instructions yet, but they will be used in a future change. The documentation (FaultMaps.rst) needs improvement and I will update this diff with a more expanded version shortly. Depends on D10196 Reviewers: rnk, reames, AndyAyers, ab, atrick, pgavlin Reviewed By: atrick, pgavlin Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D10197 llvm-svn: 239740
2015-06-16 02:44:08 +08:00
[ImplicitNullCheck] Extend Implicit Null Check scope by using stores Summary: This change allows usage of store instruction for implicit null check. Memory Aliasing Analisys is not used and change conservatively supposes that any store and load may access the same memory. As a result re-ordering of store-store, store-load and load-store is prohibited. Patch by Serguei Katkov! Reviewers: reames, sanjoy Reviewed By: sanjoy Subscribers: atrick, llvm-commits Differential Revision: https://reviews.llvm.org/D29400 llvm-svn: 294338
2017-02-08 03:19:49 +08:00
MCInst MI;
MI.setOpcode(Opcode);
[ImplicitNullChecks] Work with implicit defs. Summary: This change generalizes the implicit null checks pass to work with instructions that don't have any explicit register defs. This lets us use X86's `cmp` against memory as faulting load instructions. Reviewers: reames, JosephTremoulet Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D11286 llvm-svn: 242703
2015-07-21 04:31:39 +08:00
[ImplicitNullCheck] Extend Implicit Null Check scope by using stores Summary: This change allows usage of store instruction for implicit null check. Memory Aliasing Analisys is not used and change conservatively supposes that any store and load may access the same memory. As a result re-ordering of store-store, store-load and load-store is prohibited. Patch by Serguei Katkov! Reviewers: reames, sanjoy Reviewed By: sanjoy Subscribers: atrick, llvm-commits Differential Revision: https://reviews.llvm.org/D29400 llvm-svn: 294338
2017-02-08 03:19:49 +08:00
if (DefRegister != X86::NoRegister)
MI.addOperand(MCOperand::createReg(DefRegister));
[ImplicitNullChecks] Work with implicit defs. Summary: This change generalizes the implicit null checks pass to work with instructions that don't have any explicit register defs. This lets us use X86's `cmp` against memory as faulting load instructions. Reviewers: reames, JosephTremoulet Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D11286 llvm-svn: 242703
2015-07-21 04:31:39 +08:00
[ImplicitNullCheck] Extend Implicit Null Check scope by using stores Summary: This change allows usage of store instruction for implicit null check. Memory Aliasing Analisys is not used and change conservatively supposes that any store and load may access the same memory. As a result re-ordering of store-store, store-load and load-store is prohibited. Patch by Serguei Katkov! Reviewers: reames, sanjoy Reviewed By: sanjoy Subscribers: atrick, llvm-commits Differential Revision: https://reviews.llvm.org/D29400 llvm-svn: 294338
2017-02-08 03:19:49 +08:00
for (auto I = FaultingMI.operands_begin() + OperandsBeginIdx,
E = FaultingMI.operands_end();
[CodeGen] Introduce a FAULTING_LOAD_OP pseudo-op. Summary: This instruction encodes a loading operation that may fault, and a label to branch to if the load page-faults. The locations of potentially faulting loads and their "handler" destinations are recorded in a FaultMap section, meant to be consumed by LLVM's clients. Nothing generates FAULTING_LOAD_OP instructions yet, but they will be used in a future change. The documentation (FaultMaps.rst) needs improvement and I will update this diff with a more expanded version shortly. Depends on D10196 Reviewers: rnk, reames, AndyAyers, ab, atrick, pgavlin Reviewed By: atrick, pgavlin Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D10197 llvm-svn: 239740
2015-06-16 02:44:08 +08:00
I != E; ++I)
[ImplicitNullCheck] Extend Implicit Null Check scope by using stores Summary: This change allows usage of store instruction for implicit null check. Memory Aliasing Analisys is not used and change conservatively supposes that any store and load may access the same memory. As a result re-ordering of store-store, store-load and load-store is prohibited. Patch by Serguei Katkov! Reviewers: reames, sanjoy Reviewed By: sanjoy Subscribers: atrick, llvm-commits Differential Revision: https://reviews.llvm.org/D29400 llvm-svn: 294338
2017-02-08 03:19:49 +08:00
if (auto MaybeOperand = MCIL.LowerMachineOperand(&FaultingMI, *I))
MI.addOperand(MaybeOperand.getValue());
[CodeGen] Introduce a FAULTING_LOAD_OP pseudo-op. Summary: This instruction encodes a loading operation that may fault, and a label to branch to if the load page-faults. The locations of potentially faulting loads and their "handler" destinations are recorded in a FaultMap section, meant to be consumed by LLVM's clients. Nothing generates FAULTING_LOAD_OP instructions yet, but they will be used in a future change. The documentation (FaultMaps.rst) needs improvement and I will update this diff with a more expanded version shortly. Depends on D10196 Reviewers: rnk, reames, AndyAyers, ab, atrick, pgavlin Reviewed By: atrick, pgavlin Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D10197 llvm-svn: 239740
2015-06-16 02:44:08 +08:00
For faulting ops, include a comment w/the fault destination A faulting_op is one that has specified behavior when a fault occurs, generally redirecting control flow to another location. This change just adds a comment to the assembly output which makes it both human readable, and machine checkable w/o having to parse the FaultMap section. This is used to split a test file into two parts, so that I can (in a near future commit) easily extend the test file to demonstrate another case. llvm-svn: 355982
2019-03-13 05:05:31 +08:00
OutStreamer->AddComment("on-fault: " + HandlerLabel->getName());
[AsmPrinter][MCStreamer] De-capitalize EmitInstruction and EmitCFI*
2020-02-14 13:58:16 +08:00
OutStreamer->emitInstruction(MI, getSubtargetInfo());
[CodeGen] Introduce a FAULTING_LOAD_OP pseudo-op. Summary: This instruction encodes a loading operation that may fault, and a label to branch to if the load page-faults. The locations of potentially faulting loads and their "handler" destinations are recorded in a FaultMap section, meant to be consumed by LLVM's clients. Nothing generates FAULTING_LOAD_OP instructions yet, but they will be used in a future change. The documentation (FaultMaps.rst) needs improvement and I will update this diff with a more expanded version shortly. Depends on D10196 Reviewers: rnk, reames, AndyAyers, ab, atrick, pgavlin Reviewed By: atrick, pgavlin Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D10197 llvm-svn: 239740
2015-06-16 02:44:08 +08:00
}
[Statepoints 2/4] Statepoint infrastructure for garbage collection: MI & x86-64 Backend This is the second patch in a small series. This patch contains the MachineInstruction and x86-64 backend pieces required to lower Statepoints. It does not include the code to actually generate the STATEPOINT machine instruction and as a result, the entire patch is currently dead code. I will be submitting the SelectionDAG parts within the next 24-48 hours. Since those pieces are by far the most complicated, I wanted to minimize the size of that patch. That patch will include the tests which exercise the functionality in this patch. The entire series can be seen as one combined whole in http://reviews.llvm.org/D5683. The STATEPOINT psuedo node is generated after all gc values are explicitly spilled to stack slots. The purpose of this node is to wrap an actual call instruction while recording the spill locations of the meta arguments used for garbage collection and other purposes. The STATEPOINT is modeled as modifing all of those locations to prevent backend optimizations from forwarding the value from before the STATEPOINT to after the STATEPOINT. (Doing so would break relocation semantics for collectors which wish to relocate roots.) The implementation of STATEPOINT is closely modeled on PATCHPOINT. Eventually, much of the code in this patch will be removed. The long term plan is to merge the functionality provided by statepoints and patchpoints. Merging their implementations in the backend is likely to be a good starting point. Reviewed by: atrick, ributzka llvm-svn: 223085
2014-12-02 06:52:56 +08:00
[X86] Implement -mfentry Summary: Insert calls to __fentry__ at function entry. Reviewers: hfinkel, craig.topper Subscribers: mgorny, llvm-commits Differential Revision: https://reviews.llvm.org/D28000 llvm-svn: 293648
2017-02-01 01:00:27 +08:00
void X86AsmPrinter::LowerFENTRY_CALL(const MachineInstr &MI,
X86MCInstLower &MCIL) {
bool Is64Bits = Subtarget->is64Bit();
MCContext &Ctx = OutStreamer->getContext();
MCSymbol *fentry = Ctx.getOrCreateSymbol("__fentry__");
const MCSymbolRefExpr *Op =
MCSymbolRefExpr::create(fentry, MCSymbolRefExpr::VK_None, Ctx);
EmitAndCountInstruction(
MCInstBuilder(Is64Bits ? X86::CALL64pcrel32 : X86::CALLpcrel32)
.addExpr(Op));
}
Introduce a "patchable-function" function attribute Summary: The `"patchable-function"` attribute can be used by an LLVM client to influence LLVM's code generation in ways that makes the generated code easily patchable at runtime (for instance, to redirect control). Right now only one patchability scheme is supported, `"prologue-short-redirect"`, but this can be expanded in the future. Reviewers: joker.eph, rnk, echristo, dberris Subscribers: joker.eph, echristo, mcrosier, llvm-commits Differential Revision: http://reviews.llvm.org/D19046 llvm-svn: 266715
2016-04-19 13:24:47 +08:00
void X86AsmPrinter::LowerPATCHABLE_OP(const MachineInstr &MI,
X86MCInstLower &MCIL) {
// PATCHABLE_OP minsize, opcode, operands
[BranchAlign] Compiler support for suppressing branch align As discussed heavily in the original review (D70157), there's a need for the compiler to be able to selective suppress padding (either nop or prefix) to respect assumptions about the meaning of labels and instructions in generated code. Rather than wait for syntax to be finalized - which appears to be a very slow process - this patch focuses on the compiler use case and *only* worries about the integrated assembler. To my knowledge, this covers all cases mentioned to date for clang/JIT support. For testing purposes, I wired it up so that if the integrated assembler was using autopadding for branch alignment (e.g. enabled at command line) then the textual assembly output would contain a comment for each location where padding was enabled or disabled. This seemed like the least painful choice overall. Note that the result of this patch effective disables the jcc errata mitigation for many constructs (statepoints, implicit null checks, xray, etc...) which is non ideal. It is at least *correct* and should allow us to enable the mitigation for the compiler. Once that's done, and a few other items are worked through, we probably want to come back to this an explore a bundling based approach instead so that we can pad instructions while keeping labels in the right place. Differential Revision: https://reviews.llvm.org/D72303
2020-01-09 01:58:42 +08:00
NoAutoPaddingScope NoPadScope(*OutStreamer);
Introduce a "patchable-function" function attribute Summary: The `"patchable-function"` attribute can be used by an LLVM client to influence LLVM's code generation in ways that makes the generated code easily patchable at runtime (for instance, to redirect control). Right now only one patchability scheme is supported, `"prologue-short-redirect"`, but this can be expanded in the future. Reviewers: joker.eph, rnk, echristo, dberris Subscribers: joker.eph, echristo, mcrosier, llvm-commits Differential Revision: http://reviews.llvm.org/D19046 llvm-svn: 266715
2016-04-19 13:24:47 +08:00
unsigned MinSize = MI.getOperand(0).getImm();
unsigned Opcode = MI.getOperand(1).getImm();
MCInst MCI;
MCI.setOpcode(Opcode);
[llvm] Use llvm::drop_begin (NFC)
2021-01-15 12:30:32 +08:00
for (auto &MO : drop_begin(MI.operands(), 2))
Introduce a "patchable-function" function attribute Summary: The `"patchable-function"` attribute can be used by an LLVM client to influence LLVM's code generation in ways that makes the generated code easily patchable at runtime (for instance, to redirect control). Right now only one patchability scheme is supported, `"prologue-short-redirect"`, but this can be expanded in the future. Reviewers: joker.eph, rnk, echristo, dberris Subscribers: joker.eph, echristo, mcrosier, llvm-commits Differential Revision: http://reviews.llvm.org/D19046 llvm-svn: 266715
2016-04-19 13:24:47 +08:00
if (auto MaybeOperand = MCIL.LowerMachineOperand(&MI, MO))
MCI.addOperand(MaybeOperand.getValue());
SmallString<256> Code;
SmallVector<MCFixup, 4> Fixups;
raw_svector_ostream VecOS(Code);
CodeEmitter->encodeInstruction(MCI, VecOS, Fixups, getSubtargetInfo());
if (Code.size() < MinSize) {
[X86] For 32-bit targets, emit two-byte NOP when possible In order to support hot-patching, we need to make sure the first emitted instruction in a function is a two-byte+ op. This is already the case on x86_64, which seems to always emit two-byte+ ops. However on 32-bit targets this wasn't the case. PATCHABLE_OP now lowers to a XCHG AX, AX, (66 90) like MSVC does. However when targetting pentium3 (/arch:SSE) or i386 (/arch:IA32) targets, we generate MOV EDI,EDI (8B FF) like MSVC does. This is for compatiblity reasons with older tools that rely on this two byte pattern. Differential Revision: https://reviews.llvm.org/D81301
2020-06-18 01:26:10 +08:00
if (MinSize == 2 && Subtarget->is32Bit() &&
Subtarget->isTargetWindowsMSVC() &&
(Subtarget->getCPU().empty() || Subtarget->getCPU() == "pentium3")) {
// For compatibilty reasons, when targetting MSVC, is is important to
// generate a 'legacy' NOP in the form of a 8B FF MOV EDI, EDI. Some tools
// rely specifically on this pattern to be able to patch a function.
// This is only for 32-bit targets, when using /arch:IA32 or /arch:SSE.
OutStreamer->emitInstruction(
MCInstBuilder(X86::MOV32rr_REV).addReg(X86::EDI).addReg(X86::EDI),
*Subtarget);
} else if (MinSize == 2 && Opcode == X86::PUSH64r) {
Introduce a "patchable-function" function attribute Summary: The `"patchable-function"` attribute can be used by an LLVM client to influence LLVM's code generation in ways that makes the generated code easily patchable at runtime (for instance, to redirect control). Right now only one patchability scheme is supported, `"prologue-short-redirect"`, but this can be expanded in the future. Reviewers: joker.eph, rnk, echristo, dberris Subscribers: joker.eph, echristo, mcrosier, llvm-commits Differential Revision: http://reviews.llvm.org/D19046 llvm-svn: 266715
2016-04-19 13:24:47 +08:00
// This is an optimization that lets us get away without emitting a nop in
// many cases.
//
[CodeGen] Print register names in lowercase in both MIR and debug output As part of the unification of the debug format and the MIR format, always print registers as lowercase. * Only debug printing is affected. It now follows MIR. Differential Revision: https://reviews.llvm.org/D40417 llvm-svn: 319187
2017-11-29 01:15:09 +08:00
// NB! In some cases the encoding for PUSH64r (e.g. PUSH64r %r9) takes two
Introduce a "patchable-function" function attribute Summary: The `"patchable-function"` attribute can be used by an LLVM client to influence LLVM's code generation in ways that makes the generated code easily patchable at runtime (for instance, to redirect control). Right now only one patchability scheme is supported, `"prologue-short-redirect"`, but this can be expanded in the future. Reviewers: joker.eph, rnk, echristo, dberris Subscribers: joker.eph, echristo, mcrosier, llvm-commits Differential Revision: http://reviews.llvm.org/D19046 llvm-svn: 266715
2016-04-19 13:24:47 +08:00
// bytes too, so the check on MinSize is important.
MCI.setOpcode(X86::PUSH64rmr);
} else {
[X86] Change signature of EmitNops. NFC. This is to support https://reviews.llvm.org/D81301.
2020-06-18 00:08:12 +08:00
unsigned NopSize = emitNop(*OutStreamer, MinSize, Subtarget);
[X86MCInstLower] Clean up EmitNops; NFC Instead of having a conditional assert inside EmitNops, refactor so that the caller can have the assert instead. llvm-svn: 266793
2016-04-20 02:48:13 +08:00
assert(NopSize == MinSize && "Could not implement MinSize!");
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
(void)NopSize;
Introduce a "patchable-function" function attribute Summary: The `"patchable-function"` attribute can be used by an LLVM client to influence LLVM's code generation in ways that makes the generated code easily patchable at runtime (for instance, to redirect control). Right now only one patchability scheme is supported, `"prologue-short-redirect"`, but this can be expanded in the future. Reviewers: joker.eph, rnk, echristo, dberris Subscribers: joker.eph, echristo, mcrosier, llvm-commits Differential Revision: http://reviews.llvm.org/D19046 llvm-svn: 266715
2016-04-19 13:24:47 +08:00
}
}
[AsmPrinter][MCStreamer] De-capitalize EmitInstruction and EmitCFI*
2020-02-14 13:58:16 +08:00
OutStreamer->emitInstruction(MCI, getSubtargetInfo());
Introduce a "patchable-function" function attribute Summary: The `"patchable-function"` attribute can be used by an LLVM client to influence LLVM's code generation in ways that makes the generated code easily patchable at runtime (for instance, to redirect control). Right now only one patchability scheme is supported, `"prologue-short-redirect"`, but this can be expanded in the future. Reviewers: joker.eph, rnk, echristo, dberris Subscribers: joker.eph, echristo, mcrosier, llvm-commits Differential Revision: http://reviews.llvm.org/D19046 llvm-svn: 266715
2016-04-19 13:24:47 +08:00
}
Add an abstraction to handle patchpoint operands. Hard-coded operand indices were scattered throughout lowering stages and layers. It was super bug prone. llvm-svn: 195093
2013-11-19 11:29:56 +08:00
// Lower a stackmap of the form:
// <id>, <shadowBytes>, ...
[X86] Optimize stackmap shadows on X86. This patch minimizes the number of nops that must be emitted on X86 to satisfy stackmap shadow constraints. To minimize the number of nops inserted, the X86AsmPrinter now records the size of the most recent stackmap's shadow in the StackMapShadowTracker class, and tracks the number of instruction bytes emitted since the that stackmap instruction was encountered. Padding is emitted (if it is required at all) immediately before the next stackmap/patchpoint instruction, or at the end of the basic block. This optimization should reduce code-size and improve performance for people using the llvm stackmap intrinsic on X86. <rdar://problem/14959522> llvm-svn: 213892
2014-07-25 04:40:55 +08:00
void X86AsmPrinter::LowerSTACKMAP(const MachineInstr &MI) {
[AsmPrinter] Make AsmPrinter's OutStreamer member a unique_ptr. AsmPrinter owns the OutStreamer, so an owning pointer makes sense here. Using a reference for this is crufty. llvm-svn: 235752
2015-04-25 03:11:51 +08:00
SMShadowTracker.emitShadowPadding(*OutStreamer, getSubtargetInfo());
[StackMaps] Be explicit about label formation [NFC] (try 2) Recommit after making the same API change in non-x86 targets. This has been build for all targets, and tested for effected ones. Why the difference? Because my disk filled up when I tried make check for all. For auto-padding assembler support, we'll need to bundle the label with the instructions (nops or call sequences) so that they don't get separated. This just rearranges the code to make the upcoming change more obvious.
2019-12-20 06:03:19 +08:00
auto &Ctx = OutStreamer->getContext();
MCSymbol *MILabel = Ctx.createTempSymbol();
[MC] De-capitalize another set of MCStreamer::Emit* functions Emit{ValueTo,Code}Alignment Emit{DTP,TP,GP}* EmitSymbolValue etc
2020-02-15 11:21:58 +08:00
OutStreamer->emitLabel(MILabel);
[StackMaps] Be explicit about label formation [NFC] (try 2) Recommit after making the same API change in non-x86 targets. This has been build for all targets, and tested for effected ones. Why the difference? Because my disk filled up when I tried make check for all. For auto-padding assembler support, we'll need to bundle the label with the instructions (nops or call sequences) so that they don't get separated. This just rearranges the code to make the upcoming change more obvious.
2019-12-20 06:03:19 +08:00
SM.recordStackMap(*MILabel, MI);
[X86] Optimize stackmap shadows on X86. This patch minimizes the number of nops that must be emitted on X86 to satisfy stackmap shadow constraints. To minimize the number of nops inserted, the X86AsmPrinter now records the size of the most recent stackmap's shadow in the StackMapShadowTracker class, and tracks the number of instruction bytes emitted since the that stackmap instruction was encountered. Padding is emitted (if it is required at all) immediately before the next stackmap/patchpoint instruction, or at the end of the basic block. This optimization should reduce code-size and improve performance for people using the llvm stackmap intrinsic on X86. <rdar://problem/14959522> llvm-svn: 213892
2014-07-25 04:40:55 +08:00
unsigned NumShadowBytes = MI.getOperand(1).getImm();
SMShadowTracker.reset(NumShadowBytes);
Add support for stack map generation in the X86 backend. Originally implemented by Lang Hames. llvm-svn: 193811
2013-11-01 06:11:56 +08:00
}
Minor extension to llvm.experimental.patchpoint: don't require a call. If a null call target is provided, don't emit a dummy call. This allows the runtime to reserve as little nop space as it needs without the requirement of emitting a call. llvm-svn: 194676
2013-11-14 14:54:10 +08:00
// Lower a patchpoint of the form:
Add an abstraction to handle patchpoint operands. Hard-coded operand indices were scattered throughout lowering stages and layers. It was super bug prone. llvm-svn: 195093
2013-11-19 11:29:56 +08:00
// [<def>], <id>, <numBytes>, <target>, <numArgs>, <cc>, ...
[patchpoint] Add support for symbolic patchpoint targets to SelectionDAG and the X86 backend. The code generated for symbolic targets is identical to the code generated for constant targets, except that a relocation is emitted to fix up the actual target address at link-time. This allows IR and object files containing patchpoints to be cached across JIT-invocations where the target address may change. llvm-svn: 235483
2015-04-22 14:02:31 +08:00
void X86AsmPrinter::LowerPATCHPOINT(const MachineInstr &MI,
X86MCInstLower &MCIL) {
[X86] Optimize stackmap shadows on X86. This patch minimizes the number of nops that must be emitted on X86 to satisfy stackmap shadow constraints. To minimize the number of nops inserted, the X86AsmPrinter now records the size of the most recent stackmap's shadow in the StackMapShadowTracker class, and tracks the number of instruction bytes emitted since the that stackmap instruction was encountered. Padding is emitted (if it is required at all) immediately before the next stackmap/patchpoint instruction, or at the end of the basic block. This optimization should reduce code-size and improve performance for people using the llvm stackmap intrinsic on X86. <rdar://problem/14959522> llvm-svn: 213892
2014-07-25 04:40:55 +08:00
assert(Subtarget->is64Bit() && "Patchpoint currently only supports X86-64");
[AsmPrinter] Make AsmPrinter's OutStreamer member a unique_ptr. AsmPrinter owns the OutStreamer, so an owning pointer makes sense here. Using a reference for this is crufty. llvm-svn: 235752
2015-04-25 03:11:51 +08:00
SMShadowTracker.emitShadowPadding(*OutStreamer, getSubtargetInfo());
[X86] Optimize stackmap shadows on X86. This patch minimizes the number of nops that must be emitted on X86 to satisfy stackmap shadow constraints. To minimize the number of nops inserted, the X86AsmPrinter now records the size of the most recent stackmap's shadow in the StackMapShadowTracker class, and tracks the number of instruction bytes emitted since the that stackmap instruction was encountered. Padding is emitted (if it is required at all) immediately before the next stackmap/patchpoint instruction, or at the end of the basic block. This optimization should reduce code-size and improve performance for people using the llvm stackmap intrinsic on X86. <rdar://problem/14959522> llvm-svn: 213892
2014-07-25 04:40:55 +08:00
[BranchAlign] Compiler support for suppressing branch align As discussed heavily in the original review (D70157), there's a need for the compiler to be able to selective suppress padding (either nop or prefix) to respect assumptions about the meaning of labels and instructions in generated code. Rather than wait for syntax to be finalized - which appears to be a very slow process - this patch focuses on the compiler use case and *only* worries about the integrated assembler. To my knowledge, this covers all cases mentioned to date for clang/JIT support. For testing purposes, I wired it up so that if the integrated assembler was using autopadding for branch alignment (e.g. enabled at command line) then the textual assembly output would contain a comment for each location where padding was enabled or disabled. This seemed like the least painful choice overall. Note that the result of this patch effective disables the jcc errata mitigation for many constructs (statepoints, implicit null checks, xray, etc...) which is non ideal. It is at least *correct* and should allow us to enable the mitigation for the compiler. Once that's done, and a few other items are worked through, we probably want to come back to this an explore a bundling based approach instead so that we can pad instructions while keeping labels in the right place. Differential Revision: https://reviews.llvm.org/D72303
2020-01-09 01:58:42 +08:00
NoAutoPaddingScope NoPadScope(*OutStreamer);
[StackMaps] Be explicit about label formation [NFC] (try 2) Recommit after making the same API change in non-x86 targets. This has been build for all targets, and tested for effected ones. Why the difference? Because my disk filled up when I tried make check for all. For auto-padding assembler support, we'll need to bundle the label with the instructions (nops or call sequences) so that they don't get separated. This just rearranges the code to make the upcoming change more obvious.
2019-12-20 06:03:19 +08:00
auto &Ctx = OutStreamer->getContext();
MCSymbol *MILabel = Ctx.createTempSymbol();
[MC] De-capitalize another set of MCStreamer::Emit* functions Emit{ValueTo,Code}Alignment Emit{DTP,TP,GP}* EmitSymbolValue etc
2020-02-15 11:21:58 +08:00
OutStreamer->emitLabel(MILabel);
[StackMaps] Be explicit about label formation [NFC] (try 2) Recommit after making the same API change in non-x86 targets. This has been build for all targets, and tested for effected ones. Why the difference? Because my disk filled up when I tried make check for all. For auto-padding assembler support, we'll need to bundle the label with the instructions (nops or call sequences) so that they don't get separated. This just rearranges the code to make the upcoming change more obvious.
2019-12-20 06:03:19 +08:00
SM.recordPatchPoint(*MILabel, MI);
Add support for stack map generation in the X86 backend. Originally implemented by Lang Hames. llvm-svn: 193811
2013-11-01 06:11:56 +08:00
Add an abstraction to handle patchpoint operands. Hard-coded operand indices were scattered throughout lowering stages and layers. It was super bug prone. llvm-svn: 195093
2013-11-19 11:29:56 +08:00
PatchPointOpers opers(&MI);
unsigned ScratchIdx = opers.getNextScratchIdx();
Minor extension to llvm.experimental.patchpoint: don't require a call. If a null call target is provided, don't emit a dummy call. This allows the runtime to reserve as little nop space as it needs without the requirement of emitting a call. llvm-svn: 194676
2013-11-14 14:54:10 +08:00
unsigned EncodedBytes = 0;
[stackmaps] More extraction of common code [NFCI] General cleanup before starting to work on the part I want to actually change. llvm-svn: 279586
2016-08-24 07:33:29 +08:00
const MachineOperand &CalleeMO = opers.getCallTarget();
[patchpoint] Add support for symbolic patchpoint targets to SelectionDAG and the X86 backend. The code generated for symbolic targets is identical to the code generated for constant targets, except that a relocation is emitted to fix up the actual target address at link-time. This allows IR and object files containing patchpoints to be cached across JIT-invocations where the target address may change. llvm-svn: 235483
2015-04-22 14:02:31 +08:00
// Check for null target. If target is non-null (i.e. is non-zero or is
// symbolic) then emit a call.
if (!(CalleeMO.isImm() && !CalleeMO.getImm())) {
MCOperand CalleeMCOp;
switch (CalleeMO.getType()) {
default:
/// FIXME: Add a verifier check for bad callee types.
llvm_unreachable("Unrecognized callee operand type.");
case MachineOperand::MO_Immediate:
if (CalleeMO.getImm())
MC: Modernize MCOperand API naming. NFC. MCOperand::Create*() methods renamed to MCOperand::create*(). llvm-svn: 237275
2015-05-14 02:37:00 +08:00
CalleeMCOp = MCOperand::createImm(CalleeMO.getImm());
[patchpoint] Add support for symbolic patchpoint targets to SelectionDAG and the X86 backend. The code generated for symbolic targets is identical to the code generated for constant targets, except that a relocation is emitted to fix up the actual target address at link-time. This allows IR and object files containing patchpoints to be cached across JIT-invocations where the target address may change. llvm-svn: 235483
2015-04-22 14:02:31 +08:00
break;
case MachineOperand::MO_ExternalSymbol:
case MachineOperand::MO_GlobalAddress:
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
CalleeMCOp = MCIL.LowerSymbolOperand(CalleeMO,
MCIL.GetSymbolFromOperand(CalleeMO));
[patchpoint] Add support for symbolic patchpoint targets to SelectionDAG and the X86 backend. The code generated for symbolic targets is identical to the code generated for constant targets, except that a relocation is emitted to fix up the actual target address at link-time. This allows IR and object files containing patchpoints to be cached across JIT-invocations where the target address may change. llvm-svn: 235483
2015-04-22 14:02:31 +08:00
break;
}
Minor extension to llvm.experimental.patchpoint: don't require a call. If a null call target is provided, don't emit a dummy call. This allows the runtime to reserve as little nop space as it needs without the requirement of emitting a call. llvm-svn: 194676
2013-11-14 14:54:10 +08:00
// Emit MOV to materialize the target address and the CALL to target.
// This is encoded with 12-13 bytes, depending on which register is used.
Apply llvm-prefer-register-over-unsigned from clang-tidy to LLVM Summary: This clang-tidy check is looking for unsigned integer variables whose initializer starts with an implicit cast from llvm::Register and changes the type of the variable to llvm::Register (dropping the llvm:: where possible). Partial reverts in: X86FrameLowering.cpp - Some functions return unsigned and arguably should be MCRegister X86FixupLEAs.cpp - Some functions return unsigned and arguably should be MCRegister X86FrameLowering.cpp - Some functions return unsigned and arguably should be MCRegister HexagonBitSimplify.cpp - Function takes BitTracker::RegisterRef which appears to be unsigned& MachineVerifier.cpp - Ambiguous operator==() given MCRegister and const Register PPCFastISel.cpp - No Register::operator-=() PeepholeOptimizer.cpp - TargetInstrInfo::optimizeLoadInstr() takes an unsigned& MachineTraceMetrics.cpp - MachineTraceMetrics lacks a suitable constructor Manual fixups in: ARMFastISel.cpp - ARMEmitLoad() now takes a Register& instead of unsigned& HexagonSplitDouble.cpp - Ternary operator was ambiguous between unsigned/Register HexagonConstExtenders.cpp - Has a local class named Register, used llvm::Register instead of Register. PPCFastISel.cpp - PPCEmitLoad() now takes a Register& instead of unsigned& Depends on D65919 Reviewers: arsenm, bogner, craig.topper, RKSimon Reviewed By: arsenm Subscribers: RKSimon, craig.topper, lenary, aemerson, wuzish, jholewinski, MatzeB, qcolombet, dschuff, jyknight, dylanmckay, sdardis, nemanjai, jvesely, wdng, nhaehnle, sbc100, jgravelle-google, kristof.beyls, hiraditya, aheejin, kbarton, fedor.sergeev, javed.absar, asb, rbar, johnrusso, simoncook, apazos, sabuasal, niosHD, jrtc27, MaskRay, zzheng, edward-jones, atanasyan, rogfer01, MartinMosbeck, brucehoult, the_o, tpr, PkmX, jocewei, jsji, Petar.Avramovic, asbirlea, Jim, s.egerton, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D65962 llvm-svn: 369041
2019-08-16 03:22:08 +08:00
Register ScratchReg = MI.getOperand(ScratchIdx).getReg();
[Stackmap] Emit multi-byte nops for X86. llvm-svn: 196334
2013-12-04 08:39:08 +08:00
if (X86II::isX86_64ExtendedReg(ScratchReg))
EncodedBytes = 13;
else
EncodedBytes = 12;
[patchpoint] Add support for symbolic patchpoint targets to SelectionDAG and the X86 backend. The code generated for symbolic targets is identical to the code generated for constant targets, except that a relocation is emitted to fix up the actual target address at link-time. This allows IR and object files containing patchpoints to be cached across JIT-invocations where the target address may change. llvm-svn: 235483
2015-04-22 14:02:31 +08:00
EmitAndCountInstruction(
MCInstBuilder(X86::MOV64ri).addReg(ScratchReg).addOperand(CalleeMCOp));
Introduce the "retpoline" x86 mitigation technique for variant #2 of the speculative execution vulnerabilities disclosed today, specifically identified by CVE-2017-5715, "Branch Target Injection", and is one of the two halves to Spectre.. Summary: First, we need to explain the core of the vulnerability. Note that this is a very incomplete description, please see the Project Zero blog post for details: https://googleprojectzero.blogspot.com/2018/01/reading-privileged-memory-with-side.html The basis for branch target injection is to direct speculative execution of the processor to some "gadget" of executable code by poisoning the prediction of indirect branches with the address of that gadget. The gadget in turn contains an operation that provides a side channel for reading data. Most commonly, this will look like a load of secret data followed by a branch on the loaded value and then a load of some predictable cache line. The attacker then uses timing of the processors cache to determine which direction the branch took *in the speculative execution*, and in turn what one bit of the loaded value was. Due to the nature of these timing side channels and the branch predictor on Intel processors, this allows an attacker to leak data only accessible to a privileged domain (like the kernel) back into an unprivileged domain. The goal is simple: avoid generating code which contains an indirect branch that could have its prediction poisoned by an attacker. In many cases, the compiler can simply use directed conditional branches and a small search tree. LLVM already has support for lowering switches in this way and the first step of this patch is to disable jump-table lowering of switches and introduce a pass to rewrite explicit indirectbr sequences into a switch over integers. However, there is no fully general alternative to indirect calls. We introduce a new construct we call a "retpoline" to implement indirect calls in a non-speculatable way. It can be thought of loosely as a trampoline for indirect calls which uses the RET instruction on x86. Further, we arrange for a specific call->ret sequence which ensures the processor predicts the return to go to a controlled, known location. The retpoline then "smashes" the return address pushed onto the stack by the call with the desired target of the original indirect call. The result is a predicted return to the next instruction after a call (which can be used to trap speculative execution within an infinite loop) and an actual indirect branch to an arbitrary address. On 64-bit x86 ABIs, this is especially easily done in the compiler by using a guaranteed scratch register to pass the target into this device. For 32-bit ABIs there isn't a guaranteed scratch register and so several different retpoline variants are introduced to use a scratch register if one is available in the calling convention and to otherwise use direct stack push/pop sequences to pass the target address. This "retpoline" mitigation is fully described in the following blog post: https://support.google.com/faqs/answer/7625886 We also support a target feature that disables emission of the retpoline thunk by the compiler to allow for custom thunks if users want them. These are particularly useful in environments like kernels that routinely do hot-patching on boot and want to hot-patch their thunk to different code sequences. They can write this custom thunk and use `-mretpoline-external-thunk` *in addition* to `-mretpoline`. In this case, on x86-64 thu thunk names must be: ``` __llvm_external_retpoline_r11 ``` or on 32-bit: ``` __llvm_external_retpoline_eax __llvm_external_retpoline_ecx __llvm_external_retpoline_edx __llvm_external_retpoline_push ``` And the target of the retpoline is passed in the named register, or in the case of the `push` suffix on the top of the stack via a `pushl` instruction. There is one other important source of indirect branches in x86 ELF binaries: the PLT. These patches also include support for LLD to generate PLT entries that perform a retpoline-style indirection. The only other indirect branches remaining that we are aware of are from precompiled runtimes (such as crt0.o and similar). The ones we have found are not really attackable, and so we have not focused on them here, but eventually these runtimes should also be replicated for retpoline-ed configurations for completeness. For kernels or other freestanding or fully static executables, the compiler switch `-mretpoline` is sufficient to fully mitigate this particular attack. For dynamic executables, you must compile *all* libraries with `-mretpoline` and additionally link the dynamic executable and all shared libraries with LLD and pass `-z retpolineplt` (or use similar functionality from some other linker). We strongly recommend also using `-z now` as non-lazy binding allows the retpoline-mitigated PLT to be substantially smaller. When manually apply similar transformations to `-mretpoline` to the Linux kernel we observed very small performance hits to applications running typical workloads, and relatively minor hits (approximately 2%) even for extremely syscall-heavy applications. This is largely due to the small number of indirect branches that occur in performance sensitive paths of the kernel. When using these patches on statically linked applications, especially C++ applications, you should expect to see a much more dramatic performance hit. For microbenchmarks that are switch, indirect-, or virtual-call heavy we have seen overheads ranging from 10% to 50%. However, real-world workloads exhibit substantially lower performance impact. Notably, techniques such as PGO and ThinLTO dramatically reduce the impact of hot indirect calls (by speculatively promoting them to direct calls) and allow optimized search trees to be used to lower switches. If you need to deploy these techniques in C++ applications, we *strongly* recommend that you ensure all hot call targets are statically linked (avoiding PLT indirection) and use both PGO and ThinLTO. Well tuned servers using all of these techniques saw 5% - 10% overhead from the use of retpoline. We will add detailed documentation covering these components in subsequent patches, but wanted to make the core functionality available as soon as possible. Happy for more code review, but we'd really like to get these patches landed and backported ASAP for obvious reasons. We're planning to backport this to both 6.0 and 5.0 release streams and get a 5.0 release with just this cherry picked ASAP for distros and vendors. This patch is the work of a number of people over the past month: Eric, Reid, Rui, and myself. I'm mailing it out as a single commit due to the time sensitive nature of landing this and the need to backport it. Huge thanks to everyone who helped out here, and everyone at Intel who helped out in discussions about how to craft this. Also, credit goes to Paul Turner (at Google, but not an LLVM contributor) for much of the underlying retpoline design. Reviewers: echristo, rnk, ruiu, craig.topper, DavidKreitzer Subscribers: sanjoy, emaste, mcrosier, mgorny, mehdi_amini, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D41723 llvm-svn: 323155
2018-01-23 06:05:25 +08:00
// FIXME: Add retpoline support and remove this.
[X86][NFC] Generalize the naming of "Retpoline Thunks" and related code to "Indirect Thunks" There are applications for indirect call/branch thunks other than retpoline for Spectre v2, e.g., https://software.intel.com/security-software-guidance/software-guidance/load-value-injection Therefore it makes sense to refactor X86RetpolineThunks as a more general capability. Differential Revision: https://reviews.llvm.org/D76810
2020-04-03 12:00:44 +08:00
if (Subtarget->useIndirectThunkCalls())
Introduce the "retpoline" x86 mitigation technique for variant #2 of the speculative execution vulnerabilities disclosed today, specifically identified by CVE-2017-5715, "Branch Target Injection", and is one of the two halves to Spectre.. Summary: First, we need to explain the core of the vulnerability. Note that this is a very incomplete description, please see the Project Zero blog post for details: https://googleprojectzero.blogspot.com/2018/01/reading-privileged-memory-with-side.html The basis for branch target injection is to direct speculative execution of the processor to some "gadget" of executable code by poisoning the prediction of indirect branches with the address of that gadget. The gadget in turn contains an operation that provides a side channel for reading data. Most commonly, this will look like a load of secret data followed by a branch on the loaded value and then a load of some predictable cache line. The attacker then uses timing of the processors cache to determine which direction the branch took *in the speculative execution*, and in turn what one bit of the loaded value was. Due to the nature of these timing side channels and the branch predictor on Intel processors, this allows an attacker to leak data only accessible to a privileged domain (like the kernel) back into an unprivileged domain. The goal is simple: avoid generating code which contains an indirect branch that could have its prediction poisoned by an attacker. In many cases, the compiler can simply use directed conditional branches and a small search tree. LLVM already has support for lowering switches in this way and the first step of this patch is to disable jump-table lowering of switches and introduce a pass to rewrite explicit indirectbr sequences into a switch over integers. However, there is no fully general alternative to indirect calls. We introduce a new construct we call a "retpoline" to implement indirect calls in a non-speculatable way. It can be thought of loosely as a trampoline for indirect calls which uses the RET instruction on x86. Further, we arrange for a specific call->ret sequence which ensures the processor predicts the return to go to a controlled, known location. The retpoline then "smashes" the return address pushed onto the stack by the call with the desired target of the original indirect call. The result is a predicted return to the next instruction after a call (which can be used to trap speculative execution within an infinite loop) and an actual indirect branch to an arbitrary address. On 64-bit x86 ABIs, this is especially easily done in the compiler by using a guaranteed scratch register to pass the target into this device. For 32-bit ABIs there isn't a guaranteed scratch register and so several different retpoline variants are introduced to use a scratch register if one is available in the calling convention and to otherwise use direct stack push/pop sequences to pass the target address. This "retpoline" mitigation is fully described in the following blog post: https://support.google.com/faqs/answer/7625886 We also support a target feature that disables emission of the retpoline thunk by the compiler to allow for custom thunks if users want them. These are particularly useful in environments like kernels that routinely do hot-patching on boot and want to hot-patch their thunk to different code sequences. They can write this custom thunk and use `-mretpoline-external-thunk` *in addition* to `-mretpoline`. In this case, on x86-64 thu thunk names must be: ``` __llvm_external_retpoline_r11 ``` or on 32-bit: ``` __llvm_external_retpoline_eax __llvm_external_retpoline_ecx __llvm_external_retpoline_edx __llvm_external_retpoline_push ``` And the target of the retpoline is passed in the named register, or in the case of the `push` suffix on the top of the stack via a `pushl` instruction. There is one other important source of indirect branches in x86 ELF binaries: the PLT. These patches also include support for LLD to generate PLT entries that perform a retpoline-style indirection. The only other indirect branches remaining that we are aware of are from precompiled runtimes (such as crt0.o and similar). The ones we have found are not really attackable, and so we have not focused on them here, but eventually these runtimes should also be replicated for retpoline-ed configurations for completeness. For kernels or other freestanding or fully static executables, the compiler switch `-mretpoline` is sufficient to fully mitigate this particular attack. For dynamic executables, you must compile *all* libraries with `-mretpoline` and additionally link the dynamic executable and all shared libraries with LLD and pass `-z retpolineplt` (or use similar functionality from some other linker). We strongly recommend also using `-z now` as non-lazy binding allows the retpoline-mitigated PLT to be substantially smaller. When manually apply similar transformations to `-mretpoline` to the Linux kernel we observed very small performance hits to applications running typical workloads, and relatively minor hits (approximately 2%) even for extremely syscall-heavy applications. This is largely due to the small number of indirect branches that occur in performance sensitive paths of the kernel. When using these patches on statically linked applications, especially C++ applications, you should expect to see a much more dramatic performance hit. For microbenchmarks that are switch, indirect-, or virtual-call heavy we have seen overheads ranging from 10% to 50%. However, real-world workloads exhibit substantially lower performance impact. Notably, techniques such as PGO and ThinLTO dramatically reduce the impact of hot indirect calls (by speculatively promoting them to direct calls) and allow optimized search trees to be used to lower switches. If you need to deploy these techniques in C++ applications, we *strongly* recommend that you ensure all hot call targets are statically linked (avoiding PLT indirection) and use both PGO and ThinLTO. Well tuned servers using all of these techniques saw 5% - 10% overhead from the use of retpoline. We will add detailed documentation covering these components in subsequent patches, but wanted to make the core functionality available as soon as possible. Happy for more code review, but we'd really like to get these patches landed and backported ASAP for obvious reasons. We're planning to backport this to both 6.0 and 5.0 release streams and get a 5.0 release with just this cherry picked ASAP for distros and vendors. This patch is the work of a number of people over the past month: Eric, Reid, Rui, and myself. I'm mailing it out as a single commit due to the time sensitive nature of landing this and the need to backport it. Huge thanks to everyone who helped out here, and everyone at Intel who helped out in discussions about how to craft this. Also, credit goes to Paul Turner (at Google, but not an LLVM contributor) for much of the underlying retpoline design. Reviewers: echristo, rnk, ruiu, craig.topper, DavidKreitzer Subscribers: sanjoy, emaste, mcrosier, mgorny, mehdi_amini, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D41723 llvm-svn: 323155
2018-01-23 06:05:25 +08:00
report_fatal_error(
[X86][NFC] Generalize the naming of "Retpoline Thunks" and related code to "Indirect Thunks" There are applications for indirect call/branch thunks other than retpoline for Spectre v2, e.g., https://software.intel.com/security-software-guidance/software-guidance/load-value-injection Therefore it makes sense to refactor X86RetpolineThunks as a more general capability. Differential Revision: https://reviews.llvm.org/D76810
2020-04-03 12:00:44 +08:00
"Lowering patchpoint with thunks not yet implemented.");
[X86] Optimize stackmap shadows on X86. This patch minimizes the number of nops that must be emitted on X86 to satisfy stackmap shadow constraints. To minimize the number of nops inserted, the X86AsmPrinter now records the size of the most recent stackmap's shadow in the StackMapShadowTracker class, and tracks the number of instruction bytes emitted since the that stackmap instruction was encountered. Padding is emitted (if it is required at all) immediately before the next stackmap/patchpoint instruction, or at the end of the basic block. This optimization should reduce code-size and improve performance for people using the llvm stackmap intrinsic on X86. <rdar://problem/14959522> llvm-svn: 213892
2014-07-25 04:40:55 +08:00
EmitAndCountInstruction(MCInstBuilder(X86::CALL64r).addReg(ScratchReg));
Minor extension to llvm.experimental.patchpoint: don't require a call. If a null call target is provided, don't emit a dummy call. This allows the runtime to reserve as little nop space as it needs without the requirement of emitting a call. llvm-svn: 194676
2013-11-14 14:54:10 +08:00
}
[patchpoint] Add support for symbolic patchpoint targets to SelectionDAG and the X86 backend. The code generated for symbolic targets is identical to the code generated for constant targets, except that a relocation is emitted to fix up the actual target address at link-time. This allows IR and object files containing patchpoints to be cached across JIT-invocations where the target address may change. llvm-svn: 235483
2015-04-22 14:02:31 +08:00
Add support for stack map generation in the X86 backend. Originally implemented by Lang Hames. llvm-svn: 193811
2013-11-01 06:11:56 +08:00
// Emit padding.
[stackmaps] More extraction of common code [NFCI] General cleanup before starting to work on the part I want to actually change. llvm-svn: 279586
2016-08-24 07:33:29 +08:00
unsigned NumBytes = opers.getNumPatchBytes();
Add an abstraction to handle patchpoint operands. Hard-coded operand indices were scattered throughout lowering stages and layers. It was super bug prone. llvm-svn: 195093
2013-11-19 11:29:56 +08:00
assert(NumBytes >= EncodedBytes &&
Add support for stack map generation in the X86 backend. Originally implemented by Lang Hames. llvm-svn: 193811
2013-11-01 06:11:56 +08:00
"Patchpoint can't request size less than the length of a call.");
[X86] Change signature of EmitNops. NFC. This is to support https://reviews.llvm.org/D81301.
2020-06-18 00:08:12 +08:00
emitX86Nops(*OutStreamer, NumBytes - EncodedBytes, Subtarget);
Add support for stack map generation in the X86 backend. Originally implemented by Lang Hames. llvm-svn: 193811
2013-11-01 06:11:56 +08:00
}
[XRay] Custom event logging intrinsic This patch introduces an LLVM intrinsic and a target opcode for custom event logging in XRay. Initially, its use case will be to allow users of XRay to log some type of string ("poor man's printf"). The target opcode compiles to a noop sled large enough to enable calling through to a runtime-determined relative function call. At runtime, when X-Ray is enabled, the sled is replaced by compiler-rt with a trampoline to the logic for creating the custom log entries. Future patches will implement the compiler-rt parts and clang-side support for emitting the IR corresponding to this intrinsic. Reviewers: timshen, dberris Subscribers: igorb, pelikan, rSerge, timshen, echristo, dberris, llvm-commits Differential Revision: https://reviews.llvm.org/D27503 llvm-svn: 302405
2017-05-08 13:45:21 +08:00
void X86AsmPrinter::LowerPATCHABLE_EVENT_CALL(const MachineInstr &MI,
X86MCInstLower &MCIL) {
fix trivial typos; NFC suport -> support llvm-svn: 306968
2017-07-02 11:24:54 +08:00
assert(Subtarget->is64Bit() && "XRay custom events only supports X86-64");
[XRay] Custom event logging intrinsic This patch introduces an LLVM intrinsic and a target opcode for custom event logging in XRay. Initially, its use case will be to allow users of XRay to log some type of string ("poor man's printf"). The target opcode compiles to a noop sled large enough to enable calling through to a runtime-determined relative function call. At runtime, when X-Ray is enabled, the sled is replaced by compiler-rt with a trampoline to the logic for creating the custom log entries. Future patches will implement the compiler-rt parts and clang-side support for emitting the IR corresponding to this intrinsic. Reviewers: timshen, dberris Subscribers: igorb, pelikan, rSerge, timshen, echristo, dberris, llvm-commits Differential Revision: https://reviews.llvm.org/D27503 llvm-svn: 302405
2017-05-08 13:45:21 +08:00
[BranchAlign] Compiler support for suppressing branch align As discussed heavily in the original review (D70157), there's a need for the compiler to be able to selective suppress padding (either nop or prefix) to respect assumptions about the meaning of labels and instructions in generated code. Rather than wait for syntax to be finalized - which appears to be a very slow process - this patch focuses on the compiler use case and *only* worries about the integrated assembler. To my knowledge, this covers all cases mentioned to date for clang/JIT support. For testing purposes, I wired it up so that if the integrated assembler was using autopadding for branch alignment (e.g. enabled at command line) then the textual assembly output would contain a comment for each location where padding was enabled or disabled. This seemed like the least painful choice overall. Note that the result of this patch effective disables the jcc errata mitigation for many constructs (statepoints, implicit null checks, xray, etc...) which is non ideal. It is at least *correct* and should allow us to enable the mitigation for the compiler. Once that's done, and a few other items are worked through, we probably want to come back to this an explore a bundling based approach instead so that we can pad instructions while keeping labels in the right place. Differential Revision: https://reviews.llvm.org/D72303
2020-01-09 01:58:42 +08:00
NoAutoPaddingScope NoPadScope(*OutStreamer);
[XRay] Custom event logging intrinsic This patch introduces an LLVM intrinsic and a target opcode for custom event logging in XRay. Initially, its use case will be to allow users of XRay to log some type of string ("poor man's printf"). The target opcode compiles to a noop sled large enough to enable calling through to a runtime-determined relative function call. At runtime, when X-Ray is enabled, the sled is replaced by compiler-rt with a trampoline to the logic for creating the custom log entries. Future patches will implement the compiler-rt parts and clang-side support for emitting the IR corresponding to this intrinsic. Reviewers: timshen, dberris Subscribers: igorb, pelikan, rSerge, timshen, echristo, dberris, llvm-commits Differential Revision: https://reviews.llvm.org/D27503 llvm-svn: 302405
2017-05-08 13:45:21 +08:00
// We want to emit the following pattern, which follows the x86 calling
// convention to prepare for the trampoline call to be patched in.
//
// .p2align 1, ...
// .Lxray_event_sled_N:
[XRay][CodeGen] Use PIC-friendly code in XRay sleds and remove synthetic references in .text Summary: This is a re-roll of D36615 which uses PLT relocations in the back-end to the call to __xray_CustomEvent() when building in -fPIC and -fxray-instrument mode. Reviewers: pcc, djasper, bkramer Subscribers: sdardis, javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D37373 llvm-svn: 312466
2017-09-04 13:34:58 +08:00
// jmp +N // jump across the instrumentation sled
// ... // set up arguments in register
// callq __xray_CustomEvent@plt // force dependency to symbol
// ...
// <jump here>
[XRay] Custom event logging intrinsic This patch introduces an LLVM intrinsic and a target opcode for custom event logging in XRay. Initially, its use case will be to allow users of XRay to log some type of string ("poor man's printf"). The target opcode compiles to a noop sled large enough to enable calling through to a runtime-determined relative function call. At runtime, when X-Ray is enabled, the sled is replaced by compiler-rt with a trampoline to the logic for creating the custom log entries. Future patches will implement the compiler-rt parts and clang-side support for emitting the IR corresponding to this intrinsic. Reviewers: timshen, dberris Subscribers: igorb, pelikan, rSerge, timshen, echristo, dberris, llvm-commits Differential Revision: https://reviews.llvm.org/D27503 llvm-svn: 302405
2017-05-08 13:45:21 +08:00
//
// After patching, it would look something like:
//
// nopw (2-byte nop)
[XRay][CodeGen] Use PIC-friendly code in XRay sleds and remove synthetic references in .text Summary: This is a re-roll of D36615 which uses PLT relocations in the back-end to the call to __xray_CustomEvent() when building in -fPIC and -fxray-instrument mode. Reviewers: pcc, djasper, bkramer Subscribers: sdardis, javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D37373 llvm-svn: 312466
2017-09-04 13:34:58 +08:00
// ...
[XRay] Custom event logging intrinsic This patch introduces an LLVM intrinsic and a target opcode for custom event logging in XRay. Initially, its use case will be to allow users of XRay to log some type of string ("poor man's printf"). The target opcode compiles to a noop sled large enough to enable calling through to a runtime-determined relative function call. At runtime, when X-Ray is enabled, the sled is replaced by compiler-rt with a trampoline to the logic for creating the custom log entries. Future patches will implement the compiler-rt parts and clang-side support for emitting the IR corresponding to this intrinsic. Reviewers: timshen, dberris Subscribers: igorb, pelikan, rSerge, timshen, echristo, dberris, llvm-commits Differential Revision: https://reviews.llvm.org/D27503 llvm-svn: 302405
2017-05-08 13:45:21 +08:00
// callq __xrayCustomEvent // already lowered
[XRay][CodeGen] Use PIC-friendly code in XRay sleds and remove synthetic references in .text Summary: This is a re-roll of D36615 which uses PLT relocations in the back-end to the call to __xray_CustomEvent() when building in -fPIC and -fxray-instrument mode. Reviewers: pcc, djasper, bkramer Subscribers: sdardis, javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D37373 llvm-svn: 312466
2017-09-04 13:34:58 +08:00
// ...
[XRay] Custom event logging intrinsic This patch introduces an LLVM intrinsic and a target opcode for custom event logging in XRay. Initially, its use case will be to allow users of XRay to log some type of string ("poor man's printf"). The target opcode compiles to a noop sled large enough to enable calling through to a runtime-determined relative function call. At runtime, when X-Ray is enabled, the sled is replaced by compiler-rt with a trampoline to the logic for creating the custom log entries. Future patches will implement the compiler-rt parts and clang-side support for emitting the IR corresponding to this intrinsic. Reviewers: timshen, dberris Subscribers: igorb, pelikan, rSerge, timshen, echristo, dberris, llvm-commits Differential Revision: https://reviews.llvm.org/D27503 llvm-svn: 302405
2017-05-08 13:45:21 +08:00
//
// ---
// First we emit the label and the jump.
auto CurSled = OutContext.createTempSymbol("xray_event_sled_", true);
OutStreamer->AddComment("# XRay Custom Event Log");
[MC] De-capitalize another set of MCStreamer::Emit* functions Emit{ValueTo,Code}Alignment Emit{DTP,TP,GP}* EmitSymbolValue etc
2020-02-15 11:21:58 +08:00
OutStreamer->emitCodeAlignment(2);
OutStreamer->emitLabel(CurSled);
[XRay] Custom event logging intrinsic This patch introduces an LLVM intrinsic and a target opcode for custom event logging in XRay. Initially, its use case will be to allow users of XRay to log some type of string ("poor man's printf"). The target opcode compiles to a noop sled large enough to enable calling through to a runtime-determined relative function call. At runtime, when X-Ray is enabled, the sled is replaced by compiler-rt with a trampoline to the logic for creating the custom log entries. Future patches will implement the compiler-rt parts and clang-side support for emitting the IR corresponding to this intrinsic. Reviewers: timshen, dberris Subscribers: igorb, pelikan, rSerge, timshen, echristo, dberris, llvm-commits Differential Revision: https://reviews.llvm.org/D27503 llvm-svn: 302405
2017-05-08 13:45:21 +08:00
// Use a two-byte `jmp`. This version of JMP takes an 8-bit relative offset as
// an operand (computed as an offset from the jmp instruction).
// FIXME: Find another less hacky way do force the relative jump.
[MC] De-capitalize some MCStreamer::Emit* functions
2020-02-15 10:16:24 +08:00
OutStreamer->emitBinaryData("\xeb\x0f");
[XRay] Custom event logging intrinsic This patch introduces an LLVM intrinsic and a target opcode for custom event logging in XRay. Initially, its use case will be to allow users of XRay to log some type of string ("poor man's printf"). The target opcode compiles to a noop sled large enough to enable calling through to a runtime-determined relative function call. At runtime, when X-Ray is enabled, the sled is replaced by compiler-rt with a trampoline to the logic for creating the custom log entries. Future patches will implement the compiler-rt parts and clang-side support for emitting the IR corresponding to this intrinsic. Reviewers: timshen, dberris Subscribers: igorb, pelikan, rSerge, timshen, echristo, dberris, llvm-commits Differential Revision: https://reviews.llvm.org/D27503 llvm-svn: 302405
2017-05-08 13:45:21 +08:00
// The default C calling convention will place two arguments into %rcx and
// %rdx -- so we only work with those.
[X86] Use 64-bit version of source register in LowerPATCHABLE_EVENT_CALL and LowerPATCHABLE_TYPED_EVENT_CALL Summary: The PATCHABLE_EVENT_CALL uses i32 in the intrinsic. This results in the register allocator picking a 32-bit register. We need to use the 64-bit register when forming the MOV64rr instructions. Otherwise we print illegal assembly in the text output. I think prior to this it was impossible for SrcReg to be equal to DstReg so the NOP code was not reachable. While there use Register instead of unsigned. Also add a FIXME for what looks like a bug. Reviewers: dberris Reviewed By: dberris Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D69365
2019-10-28 08:13:50 +08:00
const Register DestRegs[] = {X86::RDI, X86::RSI};
[XRay][CodeGen] Use PIC-friendly code in XRay sleds and remove synthetic references in .text Summary: This is a re-roll of D36615 which uses PLT relocations in the back-end to the call to __xray_CustomEvent() when building in -fPIC and -fxray-instrument mode. Reviewers: pcc, djasper, bkramer Subscribers: sdardis, javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D37373 llvm-svn: 312466
2017-09-04 13:34:58 +08:00
bool UsedMask[] = {false, false};
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
// Filled out in loop.
[X86] Use 64-bit version of source register in LowerPATCHABLE_EVENT_CALL and LowerPATCHABLE_TYPED_EVENT_CALL Summary: The PATCHABLE_EVENT_CALL uses i32 in the intrinsic. This results in the register allocator picking a 32-bit register. We need to use the 64-bit register when forming the MOV64rr instructions. Otherwise we print illegal assembly in the text output. I think prior to this it was impossible for SrcReg to be equal to DstReg so the NOP code was not reachable. While there use Register instead of unsigned. Also add a FIXME for what looks like a bug. Reviewers: dberris Reviewed By: dberris Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D69365
2019-10-28 08:13:50 +08:00
Register SrcRegs[] = {0, 0};
[XRay][CodeGen] Use PIC-friendly code in XRay sleds and remove synthetic references in .text Summary: This is a re-roll of D36615 which uses PLT relocations in the back-end to the call to __xray_CustomEvent() when building in -fPIC and -fxray-instrument mode. Reviewers: pcc, djasper, bkramer Subscribers: sdardis, javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D37373 llvm-svn: 312466
2017-09-04 13:34:58 +08:00
// Then we put the operands in the %rdi and %rsi registers. We spill the
// values in the register before we clobber them, and mark them as used in
// UsedMask. In case the arguments are already in the correct register, we use
// emit nops appropriately sized to keep the sled the same size in every
// situation.
[XRay] Custom event logging intrinsic This patch introduces an LLVM intrinsic and a target opcode for custom event logging in XRay. Initially, its use case will be to allow users of XRay to log some type of string ("poor man's printf"). The target opcode compiles to a noop sled large enough to enable calling through to a runtime-determined relative function call. At runtime, when X-Ray is enabled, the sled is replaced by compiler-rt with a trampoline to the logic for creating the custom log entries. Future patches will implement the compiler-rt parts and clang-side support for emitting the IR corresponding to this intrinsic. Reviewers: timshen, dberris Subscribers: igorb, pelikan, rSerge, timshen, echristo, dberris, llvm-commits Differential Revision: https://reviews.llvm.org/D27503 llvm-svn: 302405
2017-05-08 13:45:21 +08:00
for (unsigned I = 0; I < MI.getNumOperands(); ++I)
if (auto Op = MCIL.LowerMachineOperand(&MI, MI.getOperand(I))) {
[XRay][CodeGen] Use PIC-friendly code in XRay sleds and remove synthetic references in .text Summary: This is a re-roll of D36615 which uses PLT relocations in the back-end to the call to __xray_CustomEvent() when building in -fPIC and -fxray-instrument mode. Reviewers: pcc, djasper, bkramer Subscribers: sdardis, javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D37373 llvm-svn: 312466
2017-09-04 13:34:58 +08:00
assert(Op->isReg() && "Only support arguments in registers");
[X86] Use 64-bit version of source register in LowerPATCHABLE_EVENT_CALL and LowerPATCHABLE_TYPED_EVENT_CALL Summary: The PATCHABLE_EVENT_CALL uses i32 in the intrinsic. This results in the register allocator picking a 32-bit register. We need to use the 64-bit register when forming the MOV64rr instructions. Otherwise we print illegal assembly in the text output. I think prior to this it was impossible for SrcReg to be equal to DstReg so the NOP code was not reachable. While there use Register instead of unsigned. Also add a FIXME for what looks like a bug. Reviewers: dberris Reviewed By: dberris Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D69365
2019-10-28 08:13:50 +08:00
SrcRegs[I] = getX86SubSuperRegister(Op->getReg(), 64);
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
if (SrcRegs[I] != DestRegs[I]) {
[XRay][CodeGen] Use PIC-friendly code in XRay sleds and remove synthetic references in .text Summary: This is a re-roll of D36615 which uses PLT relocations in the back-end to the call to __xray_CustomEvent() when building in -fPIC and -fxray-instrument mode. Reviewers: pcc, djasper, bkramer Subscribers: sdardis, javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D37373 llvm-svn: 312466
2017-09-04 13:34:58 +08:00
UsedMask[I] = true;
EmitAndCountInstruction(
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
MCInstBuilder(X86::PUSH64r).addReg(DestRegs[I]));
[XRay][CodeGen] Use PIC-friendly code in XRay sleds and remove synthetic references in .text Summary: This is a re-roll of D36615 which uses PLT relocations in the back-end to the call to __xray_CustomEvent() when building in -fPIC and -fxray-instrument mode. Reviewers: pcc, djasper, bkramer Subscribers: sdardis, javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D37373 llvm-svn: 312466
2017-09-04 13:34:58 +08:00
} else {
[X86] Change signature of EmitNops. NFC. This is to support https://reviews.llvm.org/D81301.
2020-06-18 00:08:12 +08:00
emitX86Nops(*OutStreamer, 4, Subtarget);
[XRay] Custom event logging intrinsic This patch introduces an LLVM intrinsic and a target opcode for custom event logging in XRay. Initially, its use case will be to allow users of XRay to log some type of string ("poor man's printf"). The target opcode compiles to a noop sled large enough to enable calling through to a runtime-determined relative function call. At runtime, when X-Ray is enabled, the sled is replaced by compiler-rt with a trampoline to the logic for creating the custom log entries. Future patches will implement the compiler-rt parts and clang-side support for emitting the IR corresponding to this intrinsic. Reviewers: timshen, dberris Subscribers: igorb, pelikan, rSerge, timshen, echristo, dberris, llvm-commits Differential Revision: https://reviews.llvm.org/D27503 llvm-svn: 302405
2017-05-08 13:45:21 +08:00
}
}
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
// Now that the register values are stashed, mov arguments into place.
[X86] Use 64-bit version of source register in LowerPATCHABLE_EVENT_CALL and LowerPATCHABLE_TYPED_EVENT_CALL Summary: The PATCHABLE_EVENT_CALL uses i32 in the intrinsic. This results in the register allocator picking a 32-bit register. We need to use the 64-bit register when forming the MOV64rr instructions. Otherwise we print illegal assembly in the text output. I think prior to this it was impossible for SrcReg to be equal to DstReg so the NOP code was not reachable. While there use Register instead of unsigned. Also add a FIXME for what looks like a bug. Reviewers: dberris Reviewed By: dberris Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D69365
2019-10-28 08:13:50 +08:00
// FIXME: This doesn't work if one of the later SrcRegs is equal to an
// earlier DestReg. We will have already overwritten over the register before
// we can copy from it.
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
for (unsigned I = 0; I < MI.getNumOperands(); ++I)
if (SrcRegs[I] != DestRegs[I])
EmitAndCountInstruction(
MCInstBuilder(X86::MOV64rr).addReg(DestRegs[I]).addReg(SrcRegs[I]));
[XRay] Custom event logging intrinsic This patch introduces an LLVM intrinsic and a target opcode for custom event logging in XRay. Initially, its use case will be to allow users of XRay to log some type of string ("poor man's printf"). The target opcode compiles to a noop sled large enough to enable calling through to a runtime-determined relative function call. At runtime, when X-Ray is enabled, the sled is replaced by compiler-rt with a trampoline to the logic for creating the custom log entries. Future patches will implement the compiler-rt parts and clang-side support for emitting the IR corresponding to this intrinsic. Reviewers: timshen, dberris Subscribers: igorb, pelikan, rSerge, timshen, echristo, dberris, llvm-commits Differential Revision: https://reviews.llvm.org/D27503 llvm-svn: 302405
2017-05-08 13:45:21 +08:00
// We emit a hard dependency on the __xray_CustomEvent symbol, which is the
[XRay][CodeGen] Use PIC-friendly code in XRay sleds and remove synthetic references in .text Summary: This is a re-roll of D36615 which uses PLT relocations in the back-end to the call to __xray_CustomEvent() when building in -fPIC and -fxray-instrument mode. Reviewers: pcc, djasper, bkramer Subscribers: sdardis, javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D37373 llvm-svn: 312466
2017-09-04 13:34:58 +08:00
// name of the trampoline to be implemented by the XRay runtime.
[XRay] Custom event logging intrinsic This patch introduces an LLVM intrinsic and a target opcode for custom event logging in XRay. Initially, its use case will be to allow users of XRay to log some type of string ("poor man's printf"). The target opcode compiles to a noop sled large enough to enable calling through to a runtime-determined relative function call. At runtime, when X-Ray is enabled, the sled is replaced by compiler-rt with a trampoline to the logic for creating the custom log entries. Future patches will implement the compiler-rt parts and clang-side support for emitting the IR corresponding to this intrinsic. Reviewers: timshen, dberris Subscribers: igorb, pelikan, rSerge, timshen, echristo, dberris, llvm-commits Differential Revision: https://reviews.llvm.org/D27503 llvm-svn: 302405
2017-05-08 13:45:21 +08:00
auto TSym = OutContext.getOrCreateSymbol("__xray_CustomEvent");
MachineOperand TOp = MachineOperand::CreateMCSymbol(TSym);
[XRay][CodeGen] Use PIC-friendly code in XRay sleds and remove synthetic references in .text Summary: This is a re-roll of D36615 which uses PLT relocations in the back-end to the call to __xray_CustomEvent() when building in -fPIC and -fxray-instrument mode. Reviewers: pcc, djasper, bkramer Subscribers: sdardis, javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D37373 llvm-svn: 312466
2017-09-04 13:34:58 +08:00
if (isPositionIndependent())
TOp.setTargetFlags(X86II::MO_PLT);
[XRay] Custom event logging intrinsic This patch introduces an LLVM intrinsic and a target opcode for custom event logging in XRay. Initially, its use case will be to allow users of XRay to log some type of string ("poor man's printf"). The target opcode compiles to a noop sled large enough to enable calling through to a runtime-determined relative function call. At runtime, when X-Ray is enabled, the sled is replaced by compiler-rt with a trampoline to the logic for creating the custom log entries. Future patches will implement the compiler-rt parts and clang-side support for emitting the IR corresponding to this intrinsic. Reviewers: timshen, dberris Subscribers: igorb, pelikan, rSerge, timshen, echristo, dberris, llvm-commits Differential Revision: https://reviews.llvm.org/D27503 llvm-svn: 302405
2017-05-08 13:45:21 +08:00
// Emit the call instruction.
[XRay][CodeGen] Use PIC-friendly code in XRay sleds and remove synthetic references in .text Summary: This is a re-roll of D36615 which uses PLT relocations in the back-end to the call to __xray_CustomEvent() when building in -fPIC and -fxray-instrument mode. Reviewers: pcc, djasper, bkramer Subscribers: sdardis, javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D37373 llvm-svn: 312466
2017-09-04 13:34:58 +08:00
EmitAndCountInstruction(MCInstBuilder(X86::CALL64pcrel32)
.addOperand(MCIL.LowerSymbolOperand(TOp, TSym)));
[XRay] Custom event logging intrinsic This patch introduces an LLVM intrinsic and a target opcode for custom event logging in XRay. Initially, its use case will be to allow users of XRay to log some type of string ("poor man's printf"). The target opcode compiles to a noop sled large enough to enable calling through to a runtime-determined relative function call. At runtime, when X-Ray is enabled, the sled is replaced by compiler-rt with a trampoline to the logic for creating the custom log entries. Future patches will implement the compiler-rt parts and clang-side support for emitting the IR corresponding to this intrinsic. Reviewers: timshen, dberris Subscribers: igorb, pelikan, rSerge, timshen, echristo, dberris, llvm-commits Differential Revision: https://reviews.llvm.org/D27503 llvm-svn: 302405
2017-05-08 13:45:21 +08:00
// Restore caller-saved and used registers.
[XRay][CodeGen] Use PIC-friendly code in XRay sleds and remove synthetic references in .text Summary: This is a re-roll of D36615 which uses PLT relocations in the back-end to the call to __xray_CustomEvent() when building in -fPIC and -fxray-instrument mode. Reviewers: pcc, djasper, bkramer Subscribers: sdardis, javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D37373 llvm-svn: 312466
2017-09-04 13:34:58 +08:00
for (unsigned I = sizeof UsedMask; I-- > 0;)
if (UsedMask[I])
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
EmitAndCountInstruction(MCInstBuilder(X86::POP64r).addReg(DestRegs[I]));
[XRay][CodeGen] Use PIC-friendly code in XRay sleds and remove synthetic references in .text Summary: This is a re-roll of D36615 which uses PLT relocations in the back-end to the call to __xray_CustomEvent() when building in -fPIC and -fxray-instrument mode. Reviewers: pcc, djasper, bkramer Subscribers: sdardis, javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D37373 llvm-svn: 312466
2017-09-04 13:34:58 +08:00
else
[X86] Change signature of EmitNops. NFC. This is to support https://reviews.llvm.org/D81301.
2020-06-18 00:08:12 +08:00
emitX86Nops(*OutStreamer, 1, Subtarget);
[XRay][CodeGen] Use PIC-friendly code in XRay sleds and remove synthetic references in .text Summary: This is a re-roll of D36615 which uses PLT relocations in the back-end to the call to __xray_CustomEvent() when building in -fPIC and -fxray-instrument mode. Reviewers: pcc, djasper, bkramer Subscribers: sdardis, javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D37373 llvm-svn: 312466
2017-09-04 13:34:58 +08:00
[XRay] Custom event logging intrinsic This patch introduces an LLVM intrinsic and a target opcode for custom event logging in XRay. Initially, its use case will be to allow users of XRay to log some type of string ("poor man's printf"). The target opcode compiles to a noop sled large enough to enable calling through to a runtime-determined relative function call. At runtime, when X-Ray is enabled, the sled is replaced by compiler-rt with a trampoline to the logic for creating the custom log entries. Future patches will implement the compiler-rt parts and clang-side support for emitting the IR corresponding to this intrinsic. Reviewers: timshen, dberris Subscribers: igorb, pelikan, rSerge, timshen, echristo, dberris, llvm-commits Differential Revision: https://reviews.llvm.org/D27503 llvm-svn: 302405
2017-05-08 13:45:21 +08:00
OutStreamer->AddComment("xray custom event end.");
[XRay] Change xray_instr_map sled addresses from absolute to PC relative for x86-64 xray_instr_map contains absolute addresses of sleds, which are relocated by `R_*_RELATIVE` when linked in -pie or -shared mode. By making these addresses relative to PC, we can avoid the dynamic relocations and remove the SHF_WRITE flag from xray_instr_map. We can thus save VM pages containg xray_instr_map (because they are not modified). This patch changes x86-64 and bumps the sled version to 2. Subsequent changes will change powerpc64le and AArch64. Reviewed By: dberris, ianlevesque Differential Revision: https://reviews.llvm.org/D78082
2020-04-14 13:28:16 +08:00
// Record the sled version. Version 0 of this sled was spelled differently, so
// we let the runtime handle the different offsets we're using. Version 2
// changed the absolute address to a PC-relative address.
recordSled(CurSled, MI, SledKind::CUSTOM_EVENT, 2);
[XRay] Custom event logging intrinsic This patch introduces an LLVM intrinsic and a target opcode for custom event logging in XRay. Initially, its use case will be to allow users of XRay to log some type of string ("poor man's printf"). The target opcode compiles to a noop sled large enough to enable calling through to a runtime-determined relative function call. At runtime, when X-Ray is enabled, the sled is replaced by compiler-rt with a trampoline to the logic for creating the custom log entries. Future patches will implement the compiler-rt parts and clang-side support for emitting the IR corresponding to this intrinsic. Reviewers: timshen, dberris Subscribers: igorb, pelikan, rSerge, timshen, echristo, dberris, llvm-commits Differential Revision: https://reviews.llvm.org/D27503 llvm-svn: 302405
2017-05-08 13:45:21 +08:00
}
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
void X86AsmPrinter::LowerPATCHABLE_TYPED_EVENT_CALL(const MachineInstr &MI,
X86MCInstLower &MCIL) {
assert(Subtarget->is64Bit() && "XRay typed events only supports X86-64");
[BranchAlign] Compiler support for suppressing branch align As discussed heavily in the original review (D70157), there's a need for the compiler to be able to selective suppress padding (either nop or prefix) to respect assumptions about the meaning of labels and instructions in generated code. Rather than wait for syntax to be finalized - which appears to be a very slow process - this patch focuses on the compiler use case and *only* worries about the integrated assembler. To my knowledge, this covers all cases mentioned to date for clang/JIT support. For testing purposes, I wired it up so that if the integrated assembler was using autopadding for branch alignment (e.g. enabled at command line) then the textual assembly output would contain a comment for each location where padding was enabled or disabled. This seemed like the least painful choice overall. Note that the result of this patch effective disables the jcc errata mitigation for many constructs (statepoints, implicit null checks, xray, etc...) which is non ideal. It is at least *correct* and should allow us to enable the mitigation for the compiler. Once that's done, and a few other items are worked through, we probably want to come back to this an explore a bundling based approach instead so that we can pad instructions while keeping labels in the right place. Differential Revision: https://reviews.llvm.org/D72303
2020-01-09 01:58:42 +08:00
NoAutoPaddingScope NoPadScope(*OutStreamer);
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
// We want to emit the following pattern, which follows the x86 calling
// convention to prepare for the trampoline call to be patched in.
//
// .p2align 1, ...
// .Lxray_event_sled_N:
// jmp +N // jump across the instrumentation sled
// ... // set up arguments in register
// callq __xray_TypedEvent@plt // force dependency to symbol
// ...
// <jump here>
//
// After patching, it would look something like:
//
// nopw (2-byte nop)
// ...
// callq __xrayTypedEvent // already lowered
// ...
//
// ---
// First we emit the label and the jump.
auto CurSled = OutContext.createTempSymbol("xray_typed_event_sled_", true);
OutStreamer->AddComment("# XRay Typed Event Log");
[MC] De-capitalize another set of MCStreamer::Emit* functions Emit{ValueTo,Code}Alignment Emit{DTP,TP,GP}* EmitSymbolValue etc
2020-02-15 11:21:58 +08:00
OutStreamer->emitCodeAlignment(2);
OutStreamer->emitLabel(CurSled);
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
// Use a two-byte `jmp`. This version of JMP takes an 8-bit relative offset as
// an operand (computed as an offset from the jmp instruction).
// FIXME: Find another less hacky way do force the relative jump.
[MC] De-capitalize some MCStreamer::Emit* functions
2020-02-15 10:16:24 +08:00
OutStreamer->emitBinaryData("\xeb\x14");
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
// An x86-64 convention may place three arguments into %rcx, %rdx, and R8,
// so we'll work with those. Or we may be called via SystemV, in which case
// we don't have to do any translation.
[X86] Use 64-bit version of source register in LowerPATCHABLE_EVENT_CALL and LowerPATCHABLE_TYPED_EVENT_CALL Summary: The PATCHABLE_EVENT_CALL uses i32 in the intrinsic. This results in the register allocator picking a 32-bit register. We need to use the 64-bit register when forming the MOV64rr instructions. Otherwise we print illegal assembly in the text output. I think prior to this it was impossible for SrcReg to be equal to DstReg so the NOP code was not reachable. While there use Register instead of unsigned. Also add a FIXME for what looks like a bug. Reviewers: dberris Reviewed By: dberris Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D69365
2019-10-28 08:13:50 +08:00
const Register DestRegs[] = {X86::RDI, X86::RSI, X86::RDX};
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
bool UsedMask[] = {false, false, false};
// Will fill out src regs in the loop.
[X86] Use 64-bit version of source register in LowerPATCHABLE_EVENT_CALL and LowerPATCHABLE_TYPED_EVENT_CALL Summary: The PATCHABLE_EVENT_CALL uses i32 in the intrinsic. This results in the register allocator picking a 32-bit register. We need to use the 64-bit register when forming the MOV64rr instructions. Otherwise we print illegal assembly in the text output. I think prior to this it was impossible for SrcReg to be equal to DstReg so the NOP code was not reachable. While there use Register instead of unsigned. Also add a FIXME for what looks like a bug. Reviewers: dberris Reviewed By: dberris Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D69365
2019-10-28 08:13:50 +08:00
Register SrcRegs[] = {0, 0, 0};
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
// Then we put the operands in the SystemV registers. We spill the values in
// the registers before we clobber them, and mark them as used in UsedMask.
// In case the arguments are already in the correct register, we emit nops
// appropriately sized to keep the sled the same size in every situation.
for (unsigned I = 0; I < MI.getNumOperands(); ++I)
if (auto Op = MCIL.LowerMachineOperand(&MI, MI.getOperand(I))) {
// TODO: Is register only support adequate?
assert(Op->isReg() && "Only supports arguments in registers");
[X86] Use 64-bit version of source register in LowerPATCHABLE_EVENT_CALL and LowerPATCHABLE_TYPED_EVENT_CALL Summary: The PATCHABLE_EVENT_CALL uses i32 in the intrinsic. This results in the register allocator picking a 32-bit register. We need to use the 64-bit register when forming the MOV64rr instructions. Otherwise we print illegal assembly in the text output. I think prior to this it was impossible for SrcReg to be equal to DstReg so the NOP code was not reachable. While there use Register instead of unsigned. Also add a FIXME for what looks like a bug. Reviewers: dberris Reviewed By: dberris Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D69365
2019-10-28 08:13:50 +08:00
SrcRegs[I] = getX86SubSuperRegister(Op->getReg(), 64);
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
if (SrcRegs[I] != DestRegs[I]) {
UsedMask[I] = true;
EmitAndCountInstruction(
MCInstBuilder(X86::PUSH64r).addReg(DestRegs[I]));
} else {
[X86] Change signature of EmitNops. NFC. This is to support https://reviews.llvm.org/D81301.
2020-06-18 00:08:12 +08:00
emitX86Nops(*OutStreamer, 4, Subtarget);
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
}
}
// In the above loop we only stash all of the destination registers or emit
// nops if the arguments are already in the right place. Doing the actually
// moving is postponed until after all the registers are stashed so nothing
// is clobbers. We've already added nops to account for the size of mov and
// push if the register is in the right place, so we only have to worry about
// emitting movs.
[X86] Use 64-bit version of source register in LowerPATCHABLE_EVENT_CALL and LowerPATCHABLE_TYPED_EVENT_CALL Summary: The PATCHABLE_EVENT_CALL uses i32 in the intrinsic. This results in the register allocator picking a 32-bit register. We need to use the 64-bit register when forming the MOV64rr instructions. Otherwise we print illegal assembly in the text output. I think prior to this it was impossible for SrcReg to be equal to DstReg so the NOP code was not reachable. While there use Register instead of unsigned. Also add a FIXME for what looks like a bug. Reviewers: dberris Reviewed By: dberris Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D69365
2019-10-28 08:13:50 +08:00
// FIXME: This doesn't work if one of the later SrcRegs is equal to an
// earlier DestReg. We will have already overwritten over the register before
// we can copy from it.
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
for (unsigned I = 0; I < MI.getNumOperands(); ++I)
if (UsedMask[I])
EmitAndCountInstruction(
MCInstBuilder(X86::MOV64rr).addReg(DestRegs[I]).addReg(SrcRegs[I]));
// We emit a hard dependency on the __xray_TypedEvent symbol, which is the
// name of the trampoline to be implemented by the XRay runtime.
auto TSym = OutContext.getOrCreateSymbol("__xray_TypedEvent");
MachineOperand TOp = MachineOperand::CreateMCSymbol(TSym);
if (isPositionIndependent())
TOp.setTargetFlags(X86II::MO_PLT);
// Emit the call instruction.
EmitAndCountInstruction(MCInstBuilder(X86::CALL64pcrel32)
.addOperand(MCIL.LowerSymbolOperand(TOp, TSym)));
// Restore caller-saved and used registers.
for (unsigned I = sizeof UsedMask; I-- > 0;)
if (UsedMask[I])
EmitAndCountInstruction(MCInstBuilder(X86::POP64r).addReg(DestRegs[I]));
else
[X86] Change signature of EmitNops. NFC. This is to support https://reviews.llvm.org/D81301.
2020-06-18 00:08:12 +08:00
emitX86Nops(*OutStreamer, 1, Subtarget);
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
OutStreamer->AddComment("xray typed event end.");
// Record the sled version.
[XRay] Change xray_instr_map sled addresses from absolute to PC relative for x86-64 xray_instr_map contains absolute addresses of sleds, which are relocated by `R_*_RELATIVE` when linked in -pie or -shared mode. By making these addresses relative to PC, we can avoid the dynamic relocations and remove the SHF_WRITE flag from xray_instr_map. We can thus save VM pages containg xray_instr_map (because they are not modified). This patch changes x86-64 and bumps the sled version to 2. Subsequent changes will change powerpc64le and AArch64. Reviewed By: dberris, ianlevesque Differential Revision: https://reviews.llvm.org/D78082
2020-04-14 13:28:16 +08:00
recordSled(CurSled, MI, SledKind::TYPED_EVENT, 2);
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
}
XRay: Add entry and exit sleds Summary: In this patch we implement the following parts of XRay: - Supporting a function attribute named 'function-instrument' which currently only supports 'xray-always'. We should be able to use this attribute for other instrumentation approaches. - Supporting a function attribute named 'xray-instruction-threshold' used to determine whether a function is instrumented with a minimum number of instructions (IR instruction counts). - X86-specific nop sleds as described in the white paper. - A machine function pass that adds the different instrumentation marker instructions at a very late stage. - A way of identifying which return opcode is considered "normal" for each architecture. There are some caveats here: 1) We don't handle PATCHABLE_RET in platforms other than x86_64 yet -- this means if IR used PATCHABLE_RET directly instead of a normal ret, instruction lowering for that platform might do the wrong thing. We think this should be handled at instruction selection time to by default be unpacked for platforms where XRay is not availble yet. 2) The generated section for X86 is different from what is described from the white paper for the sole reason that LLVM allows us to do this neatly. We're taking the opportunity to deviate from the white paper from this perspective to allow us to get richer information from the runtime library. Reviewers: sanjoy, eugenis, kcc, pcc, echristo, rnk Subscribers: niravd, majnemer, atrick, rnk, emaste, bmakam, mcrosier, mehdi_amini, llvm-commits Differential Revision: http://reviews.llvm.org/D19904 llvm-svn: 275367
2016-07-14 12:06:33 +08:00
void X86AsmPrinter::LowerPATCHABLE_FUNCTION_ENTER(const MachineInstr &MI,
X86MCInstLower &MCIL) {
[BranchAlign] Compiler support for suppressing branch align As discussed heavily in the original review (D70157), there's a need for the compiler to be able to selective suppress padding (either nop or prefix) to respect assumptions about the meaning of labels and instructions in generated code. Rather than wait for syntax to be finalized - which appears to be a very slow process - this patch focuses on the compiler use case and *only* worries about the integrated assembler. To my knowledge, this covers all cases mentioned to date for clang/JIT support. For testing purposes, I wired it up so that if the integrated assembler was using autopadding for branch alignment (e.g. enabled at command line) then the textual assembly output would contain a comment for each location where padding was enabled or disabled. This seemed like the least painful choice overall. Note that the result of this patch effective disables the jcc errata mitigation for many constructs (statepoints, implicit null checks, xray, etc...) which is non ideal. It is at least *correct* and should allow us to enable the mitigation for the compiler. Once that's done, and a few other items are worked through, we probably want to come back to this an explore a bundling based approach instead so that we can pad instructions while keeping labels in the right place. Differential Revision: https://reviews.llvm.org/D72303
2020-01-09 01:58:42 +08:00
NoAutoPaddingScope NoPadScope(*OutStreamer);
[X86] Support function attribute "patchable-function-entry" For x86-64, we diverge from GCC -fpatchable-function-entry in that we emit multi-byte NOPs. Differential Revision: https://reviews.llvm.org/D72220
2020-01-05 11:52:36 +08:00
const Function &F = MF->getFunction();
if (F.hasFnAttribute("patchable-function-entry")) {
unsigned Num;
if (F.getFnAttribute("patchable-function-entry")
.getValueAsString()
.getAsInteger(10, Num))
return;
[X86] Change signature of EmitNops. NFC. This is to support https://reviews.llvm.org/D81301.
2020-06-18 00:08:12 +08:00
emitX86Nops(*OutStreamer, Num, Subtarget);
[X86] Support function attribute "patchable-function-entry" For x86-64, we diverge from GCC -fpatchable-function-entry in that we emit multi-byte NOPs. Differential Revision: https://reviews.llvm.org/D72220
2020-01-05 11:52:36 +08:00
return;
}
XRay: Add entry and exit sleds Summary: In this patch we implement the following parts of XRay: - Supporting a function attribute named 'function-instrument' which currently only supports 'xray-always'. We should be able to use this attribute for other instrumentation approaches. - Supporting a function attribute named 'xray-instruction-threshold' used to determine whether a function is instrumented with a minimum number of instructions (IR instruction counts). - X86-specific nop sleds as described in the white paper. - A machine function pass that adds the different instrumentation marker instructions at a very late stage. - A way of identifying which return opcode is considered "normal" for each architecture. There are some caveats here: 1) We don't handle PATCHABLE_RET in platforms other than x86_64 yet -- this means if IR used PATCHABLE_RET directly instead of a normal ret, instruction lowering for that platform might do the wrong thing. We think this should be handled at instruction selection time to by default be unpacked for platforms where XRay is not availble yet. 2) The generated section for X86 is different from what is described from the white paper for the sole reason that LLVM allows us to do this neatly. We're taking the opportunity to deviate from the white paper from this perspective to allow us to get richer information from the runtime library. Reviewers: sanjoy, eugenis, kcc, pcc, echristo, rnk Subscribers: niravd, majnemer, atrick, rnk, emaste, bmakam, mcrosier, mehdi_amini, llvm-commits Differential Revision: http://reviews.llvm.org/D19904 llvm-svn: 275367
2016-07-14 12:06:33 +08:00
// We want to emit the following pattern:
//
[XRay] Align entry and return sleds to 2 byte boundaries This should ensure that we can atomically write two bytes (on top of the retq and the one past it) and have those two bytes not straddle cache lines. We also move the label past the alignment instruction so that we can refer to the actual first instruction, as opposed to potential padding before the aligned instruction. Update the tests to allow us to reflect the new order of assembly. Reviewers: rSerge, echristo, majnemer Subscribers: llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D23101 llvm-svn: 277701
2016-08-04 15:37:28 +08:00
// .p2align 1, ...
XRay: Add entry and exit sleds Summary: In this patch we implement the following parts of XRay: - Supporting a function attribute named 'function-instrument' which currently only supports 'xray-always'. We should be able to use this attribute for other instrumentation approaches. - Supporting a function attribute named 'xray-instruction-threshold' used to determine whether a function is instrumented with a minimum number of instructions (IR instruction counts). - X86-specific nop sleds as described in the white paper. - A machine function pass that adds the different instrumentation marker instructions at a very late stage. - A way of identifying which return opcode is considered "normal" for each architecture. There are some caveats here: 1) We don't handle PATCHABLE_RET in platforms other than x86_64 yet -- this means if IR used PATCHABLE_RET directly instead of a normal ret, instruction lowering for that platform might do the wrong thing. We think this should be handled at instruction selection time to by default be unpacked for platforms where XRay is not availble yet. 2) The generated section for X86 is different from what is described from the white paper for the sole reason that LLVM allows us to do this neatly. We're taking the opportunity to deviate from the white paper from this perspective to allow us to get richer information from the runtime library. Reviewers: sanjoy, eugenis, kcc, pcc, echristo, rnk Subscribers: niravd, majnemer, atrick, rnk, emaste, bmakam, mcrosier, mehdi_amini, llvm-commits Differential Revision: http://reviews.llvm.org/D19904 llvm-svn: 275367
2016-07-14 12:06:33 +08:00
// .Lxray_sled_N:
// jmp .tmpN
// # 9 bytes worth of noops
//
// We need the 9 bytes because at runtime, we'd be patching over the full 11
// bytes with the following pattern:
//
// mov %r10, <function id, 32-bit> // 6 bytes
// call <relative offset, 32-bits> // 5 bytes
//
auto CurSled = OutContext.createTempSymbol("xray_sled_", true);
[MC] De-capitalize another set of MCStreamer::Emit* functions Emit{ValueTo,Code}Alignment Emit{DTP,TP,GP}* EmitSymbolValue etc
2020-02-15 11:21:58 +08:00
OutStreamer->emitCodeAlignment(2);
OutStreamer->emitLabel(CurSled);
XRay: Add entry and exit sleds Summary: In this patch we implement the following parts of XRay: - Supporting a function attribute named 'function-instrument' which currently only supports 'xray-always'. We should be able to use this attribute for other instrumentation approaches. - Supporting a function attribute named 'xray-instruction-threshold' used to determine whether a function is instrumented with a minimum number of instructions (IR instruction counts). - X86-specific nop sleds as described in the white paper. - A machine function pass that adds the different instrumentation marker instructions at a very late stage. - A way of identifying which return opcode is considered "normal" for each architecture. There are some caveats here: 1) We don't handle PATCHABLE_RET in platforms other than x86_64 yet -- this means if IR used PATCHABLE_RET directly instead of a normal ret, instruction lowering for that platform might do the wrong thing. We think this should be handled at instruction selection time to by default be unpacked for platforms where XRay is not availble yet. 2) The generated section for X86 is different from what is described from the white paper for the sole reason that LLVM allows us to do this neatly. We're taking the opportunity to deviate from the white paper from this perspective to allow us to get richer information from the runtime library. Reviewers: sanjoy, eugenis, kcc, pcc, echristo, rnk Subscribers: niravd, majnemer, atrick, rnk, emaste, bmakam, mcrosier, mehdi_amini, llvm-commits Differential Revision: http://reviews.llvm.org/D19904 llvm-svn: 275367
2016-07-14 12:06:33 +08:00
// Use a two-byte `jmp`. This version of JMP takes an 8-bit relative offset as
// an operand (computed as an offset from the jmp instruction).
// FIXME: Find another less hacky way do force the relative jump.
[MC] De-capitalize some MCStreamer::Emit* functions
2020-02-15 10:16:24 +08:00
OutStreamer->emitBytes("\xeb\x09");
[X86] Change signature of EmitNops. NFC. This is to support https://reviews.llvm.org/D81301.
2020-06-18 00:08:12 +08:00
emitX86Nops(*OutStreamer, 9, Subtarget);
[XRay] Change xray_instr_map sled addresses from absolute to PC relative for x86-64 xray_instr_map contains absolute addresses of sleds, which are relocated by `R_*_RELATIVE` when linked in -pie or -shared mode. By making these addresses relative to PC, we can avoid the dynamic relocations and remove the SHF_WRITE flag from xray_instr_map. We can thus save VM pages containg xray_instr_map (because they are not modified). This patch changes x86-64 and bumps the sled version to 2. Subsequent changes will change powerpc64le and AArch64. Reviewed By: dberris, ianlevesque Differential Revision: https://reviews.llvm.org/D78082
2020-04-14 13:28:16 +08:00
recordSled(CurSled, MI, SledKind::FUNCTION_ENTER, 2);
XRay: Add entry and exit sleds Summary: In this patch we implement the following parts of XRay: - Supporting a function attribute named 'function-instrument' which currently only supports 'xray-always'. We should be able to use this attribute for other instrumentation approaches. - Supporting a function attribute named 'xray-instruction-threshold' used to determine whether a function is instrumented with a minimum number of instructions (IR instruction counts). - X86-specific nop sleds as described in the white paper. - A machine function pass that adds the different instrumentation marker instructions at a very late stage. - A way of identifying which return opcode is considered "normal" for each architecture. There are some caveats here: 1) We don't handle PATCHABLE_RET in platforms other than x86_64 yet -- this means if IR used PATCHABLE_RET directly instead of a normal ret, instruction lowering for that platform might do the wrong thing. We think this should be handled at instruction selection time to by default be unpacked for platforms where XRay is not availble yet. 2) The generated section for X86 is different from what is described from the white paper for the sole reason that LLVM allows us to do this neatly. We're taking the opportunity to deviate from the white paper from this perspective to allow us to get richer information from the runtime library. Reviewers: sanjoy, eugenis, kcc, pcc, echristo, rnk Subscribers: niravd, majnemer, atrick, rnk, emaste, bmakam, mcrosier, mehdi_amini, llvm-commits Differential Revision: http://reviews.llvm.org/D19904 llvm-svn: 275367
2016-07-14 12:06:33 +08:00
}
void X86AsmPrinter::LowerPATCHABLE_RET(const MachineInstr &MI,
X86MCInstLower &MCIL) {
[BranchAlign] Compiler support for suppressing branch align As discussed heavily in the original review (D70157), there's a need for the compiler to be able to selective suppress padding (either nop or prefix) to respect assumptions about the meaning of labels and instructions in generated code. Rather than wait for syntax to be finalized - which appears to be a very slow process - this patch focuses on the compiler use case and *only* worries about the integrated assembler. To my knowledge, this covers all cases mentioned to date for clang/JIT support. For testing purposes, I wired it up so that if the integrated assembler was using autopadding for branch alignment (e.g. enabled at command line) then the textual assembly output would contain a comment for each location where padding was enabled or disabled. This seemed like the least painful choice overall. Note that the result of this patch effective disables the jcc errata mitigation for many constructs (statepoints, implicit null checks, xray, etc...) which is non ideal. It is at least *correct* and should allow us to enable the mitigation for the compiler. Once that's done, and a few other items are worked through, we probably want to come back to this an explore a bundling based approach instead so that we can pad instructions while keeping labels in the right place. Differential Revision: https://reviews.llvm.org/D72303
2020-01-09 01:58:42 +08:00
NoAutoPaddingScope NoPadScope(*OutStreamer);
XRay: Add entry and exit sleds Summary: In this patch we implement the following parts of XRay: - Supporting a function attribute named 'function-instrument' which currently only supports 'xray-always'. We should be able to use this attribute for other instrumentation approaches. - Supporting a function attribute named 'xray-instruction-threshold' used to determine whether a function is instrumented with a minimum number of instructions (IR instruction counts). - X86-specific nop sleds as described in the white paper. - A machine function pass that adds the different instrumentation marker instructions at a very late stage. - A way of identifying which return opcode is considered "normal" for each architecture. There are some caveats here: 1) We don't handle PATCHABLE_RET in platforms other than x86_64 yet -- this means if IR used PATCHABLE_RET directly instead of a normal ret, instruction lowering for that platform might do the wrong thing. We think this should be handled at instruction selection time to by default be unpacked for platforms where XRay is not availble yet. 2) The generated section for X86 is different from what is described from the white paper for the sole reason that LLVM allows us to do this neatly. We're taking the opportunity to deviate from the white paper from this perspective to allow us to get richer information from the runtime library. Reviewers: sanjoy, eugenis, kcc, pcc, echristo, rnk Subscribers: niravd, majnemer, atrick, rnk, emaste, bmakam, mcrosier, mehdi_amini, llvm-commits Differential Revision: http://reviews.llvm.org/D19904 llvm-svn: 275367
2016-07-14 12:06:33 +08:00
// Since PATCHABLE_RET takes the opcode of the return statement as an
// argument, we use that to emit the correct form of the RET that we want.
// i.e. when we see this:
//
// PATCHABLE_RET X86::RET ...
//
// We should emit the RET followed by sleds.
//
[XRay] Align entry and return sleds to 2 byte boundaries This should ensure that we can atomically write two bytes (on top of the retq and the one past it) and have those two bytes not straddle cache lines. We also move the label past the alignment instruction so that we can refer to the actual first instruction, as opposed to potential padding before the aligned instruction. Update the tests to allow us to reflect the new order of assembly. Reviewers: rSerge, echristo, majnemer Subscribers: llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D23101 llvm-svn: 277701
2016-08-04 15:37:28 +08:00
// .p2align 1, ...
XRay: Add entry and exit sleds Summary: In this patch we implement the following parts of XRay: - Supporting a function attribute named 'function-instrument' which currently only supports 'xray-always'. We should be able to use this attribute for other instrumentation approaches. - Supporting a function attribute named 'xray-instruction-threshold' used to determine whether a function is instrumented with a minimum number of instructions (IR instruction counts). - X86-specific nop sleds as described in the white paper. - A machine function pass that adds the different instrumentation marker instructions at a very late stage. - A way of identifying which return opcode is considered "normal" for each architecture. There are some caveats here: 1) We don't handle PATCHABLE_RET in platforms other than x86_64 yet -- this means if IR used PATCHABLE_RET directly instead of a normal ret, instruction lowering for that platform might do the wrong thing. We think this should be handled at instruction selection time to by default be unpacked for platforms where XRay is not availble yet. 2) The generated section for X86 is different from what is described from the white paper for the sole reason that LLVM allows us to do this neatly. We're taking the opportunity to deviate from the white paper from this perspective to allow us to get richer information from the runtime library. Reviewers: sanjoy, eugenis, kcc, pcc, echristo, rnk Subscribers: niravd, majnemer, atrick, rnk, emaste, bmakam, mcrosier, mehdi_amini, llvm-commits Differential Revision: http://reviews.llvm.org/D19904 llvm-svn: 275367
2016-07-14 12:06:33 +08:00
// .Lxray_sled_N:
// ret # or equivalent instruction
// # 10 bytes worth of noops
//
// This just makes sure that the alignment for the next instruction is 2.
auto CurSled = OutContext.createTempSymbol("xray_sled_", true);
[MC] De-capitalize another set of MCStreamer::Emit* functions Emit{ValueTo,Code}Alignment Emit{DTP,TP,GP}* EmitSymbolValue etc
2020-02-15 11:21:58 +08:00
OutStreamer->emitCodeAlignment(2);
OutStreamer->emitLabel(CurSled);
XRay: Add entry and exit sleds Summary: In this patch we implement the following parts of XRay: - Supporting a function attribute named 'function-instrument' which currently only supports 'xray-always'. We should be able to use this attribute for other instrumentation approaches. - Supporting a function attribute named 'xray-instruction-threshold' used to determine whether a function is instrumented with a minimum number of instructions (IR instruction counts). - X86-specific nop sleds as described in the white paper. - A machine function pass that adds the different instrumentation marker instructions at a very late stage. - A way of identifying which return opcode is considered "normal" for each architecture. There are some caveats here: 1) We don't handle PATCHABLE_RET in platforms other than x86_64 yet -- this means if IR used PATCHABLE_RET directly instead of a normal ret, instruction lowering for that platform might do the wrong thing. We think this should be handled at instruction selection time to by default be unpacked for platforms where XRay is not availble yet. 2) The generated section for X86 is different from what is described from the white paper for the sole reason that LLVM allows us to do this neatly. We're taking the opportunity to deviate from the white paper from this perspective to allow us to get richer information from the runtime library. Reviewers: sanjoy, eugenis, kcc, pcc, echristo, rnk Subscribers: niravd, majnemer, atrick, rnk, emaste, bmakam, mcrosier, mehdi_amini, llvm-commits Differential Revision: http://reviews.llvm.org/D19904 llvm-svn: 275367
2016-07-14 12:06:33 +08:00
unsigned OpCode = MI.getOperand(0).getImm();
MCInst Ret;
Ret.setOpcode(OpCode);
[llvm] Use the default value of drop_begin (NFC)
2021-01-19 02:16:36 +08:00
for (auto &MO : drop_begin(MI.operands()))
XRay: Add entry and exit sleds Summary: In this patch we implement the following parts of XRay: - Supporting a function attribute named 'function-instrument' which currently only supports 'xray-always'. We should be able to use this attribute for other instrumentation approaches. - Supporting a function attribute named 'xray-instruction-threshold' used to determine whether a function is instrumented with a minimum number of instructions (IR instruction counts). - X86-specific nop sleds as described in the white paper. - A machine function pass that adds the different instrumentation marker instructions at a very late stage. - A way of identifying which return opcode is considered "normal" for each architecture. There are some caveats here: 1) We don't handle PATCHABLE_RET in platforms other than x86_64 yet -- this means if IR used PATCHABLE_RET directly instead of a normal ret, instruction lowering for that platform might do the wrong thing. We think this should be handled at instruction selection time to by default be unpacked for platforms where XRay is not availble yet. 2) The generated section for X86 is different from what is described from the white paper for the sole reason that LLVM allows us to do this neatly. We're taking the opportunity to deviate from the white paper from this perspective to allow us to get richer information from the runtime library. Reviewers: sanjoy, eugenis, kcc, pcc, echristo, rnk Subscribers: niravd, majnemer, atrick, rnk, emaste, bmakam, mcrosier, mehdi_amini, llvm-commits Differential Revision: http://reviews.llvm.org/D19904 llvm-svn: 275367
2016-07-14 12:06:33 +08:00
if (auto MaybeOperand = MCIL.LowerMachineOperand(&MI, MO))
Ret.addOperand(MaybeOperand.getValue());
[AsmPrinter][MCStreamer] De-capitalize EmitInstruction and EmitCFI*
2020-02-14 13:58:16 +08:00
OutStreamer->emitInstruction(Ret, getSubtargetInfo());
[X86] Change signature of EmitNops. NFC. This is to support https://reviews.llvm.org/D81301.
2020-06-18 00:08:12 +08:00
emitX86Nops(*OutStreamer, 10, Subtarget);
[XRay] Change xray_instr_map sled addresses from absolute to PC relative for x86-64 xray_instr_map contains absolute addresses of sleds, which are relocated by `R_*_RELATIVE` when linked in -pie or -shared mode. By making these addresses relative to PC, we can avoid the dynamic relocations and remove the SHF_WRITE flag from xray_instr_map. We can thus save VM pages containg xray_instr_map (because they are not modified). This patch changes x86-64 and bumps the sled version to 2. Subsequent changes will change powerpc64le and AArch64. Reviewed By: dberris, ianlevesque Differential Revision: https://reviews.llvm.org/D78082
2020-04-14 13:28:16 +08:00
recordSled(CurSled, MI, SledKind::FUNCTION_EXIT, 2);
XRay: Add entry and exit sleds Summary: In this patch we implement the following parts of XRay: - Supporting a function attribute named 'function-instrument' which currently only supports 'xray-always'. We should be able to use this attribute for other instrumentation approaches. - Supporting a function attribute named 'xray-instruction-threshold' used to determine whether a function is instrumented with a minimum number of instructions (IR instruction counts). - X86-specific nop sleds as described in the white paper. - A machine function pass that adds the different instrumentation marker instructions at a very late stage. - A way of identifying which return opcode is considered "normal" for each architecture. There are some caveats here: 1) We don't handle PATCHABLE_RET in platforms other than x86_64 yet -- this means if IR used PATCHABLE_RET directly instead of a normal ret, instruction lowering for that platform might do the wrong thing. We think this should be handled at instruction selection time to by default be unpacked for platforms where XRay is not availble yet. 2) The generated section for X86 is different from what is described from the white paper for the sole reason that LLVM allows us to do this neatly. We're taking the opportunity to deviate from the white paper from this perspective to allow us to get richer information from the runtime library. Reviewers: sanjoy, eugenis, kcc, pcc, echristo, rnk Subscribers: niravd, majnemer, atrick, rnk, emaste, bmakam, mcrosier, mehdi_amini, llvm-commits Differential Revision: http://reviews.llvm.org/D19904 llvm-svn: 275367
2016-07-14 12:06:33 +08:00
}
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
void X86AsmPrinter::LowerPATCHABLE_TAIL_CALL(const MachineInstr &MI,
X86MCInstLower &MCIL) {
[BranchAlign] Compiler support for suppressing branch align As discussed heavily in the original review (D70157), there's a need for the compiler to be able to selective suppress padding (either nop or prefix) to respect assumptions about the meaning of labels and instructions in generated code. Rather than wait for syntax to be finalized - which appears to be a very slow process - this patch focuses on the compiler use case and *only* worries about the integrated assembler. To my knowledge, this covers all cases mentioned to date for clang/JIT support. For testing purposes, I wired it up so that if the integrated assembler was using autopadding for branch alignment (e.g. enabled at command line) then the textual assembly output would contain a comment for each location where padding was enabled or disabled. This seemed like the least painful choice overall. Note that the result of this patch effective disables the jcc errata mitigation for many constructs (statepoints, implicit null checks, xray, etc...) which is non ideal. It is at least *correct* and should allow us to enable the mitigation for the compiler. Once that's done, and a few other items are worked through, we probably want to come back to this an explore a bundling based approach instead so that we can pad instructions while keeping labels in the right place. Differential Revision: https://reviews.llvm.org/D72303
2020-01-09 01:58:42 +08:00
NoAutoPaddingScope NoPadScope(*OutStreamer);
[XRay] Detect and emit sleds for sibling/tail calls Summary: This change promotes the 'isTailCall(...)' member function to TargetInstrInfo as a query interface for determining on a per-target basis whether a given MachineInstr is a tail call instruction. We build upon this in the XRay instrumentation pass to emit special sleds for tail call optimisations, where we emit the correct kind of sled. The tail call sleds look like a mix between the function entry and function exit sleds. Form-wise, the sled comes before the "jmp" instruction that implements the tail call similar to how we do it for the function entry sled. Functionally, because we know this is a tail call, it behaves much like an exit sled -- i.e. at runtime we may use the exit trampolines instead of a different kind of trampoline. A follow-up change to recognise these sleds will be done in compiler-rt, so that we can start intercepting these initially as exits, but also have the option to have different log entries to more accurately reflect that this is actually a tail call. Reviewers: echristo, rSerge, majnemer Subscribers: mehdi_amini, dberris, llvm-commits Differential Revision: https://reviews.llvm.org/D23986 llvm-svn: 280334
2016-09-01 09:29:13 +08:00
// Like PATCHABLE_RET, we have the actual instruction in the operands to this
// instruction so we lower that particular instruction and its operands.
// Unlike PATCHABLE_RET though, we put the sled before the JMP, much like how
// we do it for PATCHABLE_FUNCTION_ENTER. The sled should be very similar to
// the PATCHABLE_FUNCTION_ENTER case, followed by the lowering of the actual
// tail call much like how we have it in PATCHABLE_RET.
auto CurSled = OutContext.createTempSymbol("xray_sled_", true);
[MC] De-capitalize another set of MCStreamer::Emit* functions Emit{ValueTo,Code}Alignment Emit{DTP,TP,GP}* EmitSymbolValue etc
2020-02-15 11:21:58 +08:00
OutStreamer->emitCodeAlignment(2);
OutStreamer->emitLabel(CurSled);
[XRay] Detect and emit sleds for sibling/tail calls Summary: This change promotes the 'isTailCall(...)' member function to TargetInstrInfo as a query interface for determining on a per-target basis whether a given MachineInstr is a tail call instruction. We build upon this in the XRay instrumentation pass to emit special sleds for tail call optimisations, where we emit the correct kind of sled. The tail call sleds look like a mix between the function entry and function exit sleds. Form-wise, the sled comes before the "jmp" instruction that implements the tail call similar to how we do it for the function entry sled. Functionally, because we know this is a tail call, it behaves much like an exit sled -- i.e. at runtime we may use the exit trampolines instead of a different kind of trampoline. A follow-up change to recognise these sleds will be done in compiler-rt, so that we can start intercepting these initially as exits, but also have the option to have different log entries to more accurately reflect that this is actually a tail call. Reviewers: echristo, rSerge, majnemer Subscribers: mehdi_amini, dberris, llvm-commits Differential Revision: https://reviews.llvm.org/D23986 llvm-svn: 280334
2016-09-01 09:29:13 +08:00
auto Target = OutContext.createTempSymbol();
// Use a two-byte `jmp`. This version of JMP takes an 8-bit relative offset as
// an operand (computed as an offset from the jmp instruction).
// FIXME: Find another less hacky way do force the relative jump.
[MC] De-capitalize some MCStreamer::Emit* functions
2020-02-15 10:16:24 +08:00
OutStreamer->emitBytes("\xeb\x09");
[X86] Change signature of EmitNops. NFC. This is to support https://reviews.llvm.org/D81301.
2020-06-18 00:08:12 +08:00
emitX86Nops(*OutStreamer, 9, Subtarget);
[MC] De-capitalize another set of MCStreamer::Emit* functions Emit{ValueTo,Code}Alignment Emit{DTP,TP,GP}* EmitSymbolValue etc
2020-02-15 11:21:58 +08:00
OutStreamer->emitLabel(Target);
[XRay] Change xray_instr_map sled addresses from absolute to PC relative for x86-64 xray_instr_map contains absolute addresses of sleds, which are relocated by `R_*_RELATIVE` when linked in -pie or -shared mode. By making these addresses relative to PC, we can avoid the dynamic relocations and remove the SHF_WRITE flag from xray_instr_map. We can thus save VM pages containg xray_instr_map (because they are not modified). This patch changes x86-64 and bumps the sled version to 2. Subsequent changes will change powerpc64le and AArch64. Reviewed By: dberris, ianlevesque Differential Revision: https://reviews.llvm.org/D78082
2020-04-14 13:28:16 +08:00
recordSled(CurSled, MI, SledKind::TAIL_CALL, 2);
[XRay] Detect and emit sleds for sibling/tail calls Summary: This change promotes the 'isTailCall(...)' member function to TargetInstrInfo as a query interface for determining on a per-target basis whether a given MachineInstr is a tail call instruction. We build upon this in the XRay instrumentation pass to emit special sleds for tail call optimisations, where we emit the correct kind of sled. The tail call sleds look like a mix between the function entry and function exit sleds. Form-wise, the sled comes before the "jmp" instruction that implements the tail call similar to how we do it for the function entry sled. Functionally, because we know this is a tail call, it behaves much like an exit sled -- i.e. at runtime we may use the exit trampolines instead of a different kind of trampoline. A follow-up change to recognise these sleds will be done in compiler-rt, so that we can start intercepting these initially as exits, but also have the option to have different log entries to more accurately reflect that this is actually a tail call. Reviewers: echristo, rSerge, majnemer Subscribers: mehdi_amini, dberris, llvm-commits Differential Revision: https://reviews.llvm.org/D23986 llvm-svn: 280334
2016-09-01 09:29:13 +08:00
unsigned OpCode = MI.getOperand(0).getImm();
[X86] Remove encoding information from the TAILJMP instructions that are lowered by MCInstLowering. Fix LowerPATCHABLE_TAIL_CALL to also convert them to regular JMP/JCC instructions There are 5 instructions here that are converted from TAILJMP opcodes to regular JMP/JCC opcodes during MCInstLowering. So normally there encoding information isn't used. The exception being when XRay wraps them in PATCHABLE_TAIL_CALL. For the ones that weren't already handled in MCInstLowering, add handling for those and remove their encoding information. This patch fixes PATCHABLE_TAIL_CALL to do the same opcode conversion as the regular lowering patch. Then removes the encoding information. Differential Revision: https://reviews.llvm.org/D66561 llvm-svn: 370079
2019-08-28 01:24:23 +08:00
OpCode = convertTailJumpOpcode(OpCode);
[XRay] Detect and emit sleds for sibling/tail calls Summary: This change promotes the 'isTailCall(...)' member function to TargetInstrInfo as a query interface for determining on a per-target basis whether a given MachineInstr is a tail call instruction. We build upon this in the XRay instrumentation pass to emit special sleds for tail call optimisations, where we emit the correct kind of sled. The tail call sleds look like a mix between the function entry and function exit sleds. Form-wise, the sled comes before the "jmp" instruction that implements the tail call similar to how we do it for the function entry sled. Functionally, because we know this is a tail call, it behaves much like an exit sled -- i.e. at runtime we may use the exit trampolines instead of a different kind of trampoline. A follow-up change to recognise these sleds will be done in compiler-rt, so that we can start intercepting these initially as exits, but also have the option to have different log entries to more accurately reflect that this is actually a tail call. Reviewers: echristo, rSerge, majnemer Subscribers: mehdi_amini, dberris, llvm-commits Differential Revision: https://reviews.llvm.org/D23986 llvm-svn: 280334
2016-09-01 09:29:13 +08:00
MCInst TC;
TC.setOpcode(OpCode);
// Before emitting the instruction, add a comment to indicate that this is
// indeed a tail call.
OutStreamer->AddComment("TAILCALL");
[llvm] Use the default value of drop_begin (NFC)
2021-01-19 02:16:36 +08:00
for (auto &MO : drop_begin(MI.operands()))
[XRay] Detect and emit sleds for sibling/tail calls Summary: This change promotes the 'isTailCall(...)' member function to TargetInstrInfo as a query interface for determining on a per-target basis whether a given MachineInstr is a tail call instruction. We build upon this in the XRay instrumentation pass to emit special sleds for tail call optimisations, where we emit the correct kind of sled. The tail call sleds look like a mix between the function entry and function exit sleds. Form-wise, the sled comes before the "jmp" instruction that implements the tail call similar to how we do it for the function entry sled. Functionally, because we know this is a tail call, it behaves much like an exit sled -- i.e. at runtime we may use the exit trampolines instead of a different kind of trampoline. A follow-up change to recognise these sleds will be done in compiler-rt, so that we can start intercepting these initially as exits, but also have the option to have different log entries to more accurately reflect that this is actually a tail call. Reviewers: echristo, rSerge, majnemer Subscribers: mehdi_amini, dberris, llvm-commits Differential Revision: https://reviews.llvm.org/D23986 llvm-svn: 280334
2016-09-01 09:29:13 +08:00
if (auto MaybeOperand = MCIL.LowerMachineOperand(&MI, MO))
TC.addOperand(MaybeOperand.getValue());
[AsmPrinter][MCStreamer] De-capitalize EmitInstruction and EmitCFI*
2020-02-14 13:58:16 +08:00
OutStreamer->emitInstruction(TC, getSubtargetInfo());
[XRay] Detect and emit sleds for sibling/tail calls Summary: This change promotes the 'isTailCall(...)' member function to TargetInstrInfo as a query interface for determining on a per-target basis whether a given MachineInstr is a tail call instruction. We build upon this in the XRay instrumentation pass to emit special sleds for tail call optimisations, where we emit the correct kind of sled. The tail call sleds look like a mix between the function entry and function exit sleds. Form-wise, the sled comes before the "jmp" instruction that implements the tail call similar to how we do it for the function entry sled. Functionally, because we know this is a tail call, it behaves much like an exit sled -- i.e. at runtime we may use the exit trampolines instead of a different kind of trampoline. A follow-up change to recognise these sleds will be done in compiler-rt, so that we can start intercepting these initially as exits, but also have the option to have different log entries to more accurately reflect that this is actually a tail call. Reviewers: echristo, rSerge, majnemer Subscribers: mehdi_amini, dberris, llvm-commits Differential Revision: https://reviews.llvm.org/D23986 llvm-svn: 280334
2016-09-01 09:29:13 +08:00
}
Fix failure to invoke exception handler on Win64 When the last instruction prior to a function epilogue is a call, we need to emit a nop so that the return address is not in the epilogue IP range. This is consistent with MSVC's behavior, and may be a workaround for a bug in the Win64 unwinder. Differential Revision: http://reviews.llvm.org/D4751 Patch by Vadim Chugunov! llvm-svn: 214775
2014-08-05 05:05:27 +08:00
// Returns instruction preceding MBBI in MachineFunction.
// If MBBI is the first instruction of the first basic block, returns null.
static MachineBasicBlock::const_iterator
PrevCrossBBInst(MachineBasicBlock::const_iterator MBBI) {
const MachineBasicBlock *MBB = MBBI->getParent();
while (MBBI == MBB->begin()) {
ADT: Remove == and != comparisons between ilist iterators and pointers I missed == and != when I removed implicit conversions between iterators and pointers in r252380 since they were defined outside ilist_iterator. Since they depend on getNodePtrUnchecked(), they indirectly rely on UB. This commit removes all uses of these operators. (I'll delete the operators themselves in a separate commit so that it can be easily reverted if necessary.) There should be NFC here. llvm-svn: 261498
2016-02-22 04:39:50 +08:00
if (MBB == &MBB->getParent()->front())
X86: Avoid implicit iterator conversions, NFC Avoid implicit conversions from MachineInstrBundleIterator to MachineInstr*, mainly by preferring MachineInstr& over MachineInstr* and using range-based for loops. llvm-svn: 275149
2016-07-12 11:18:50 +08:00
return MachineBasicBlock::const_iterator();
Fix failure to invoke exception handler on Win64 When the last instruction prior to a function epilogue is a call, we need to emit a nop so that the return address is not in the epilogue IP range. This is consistent with MSVC's behavior, and may be a workaround for a bug in the Win64 unwinder. Differential Revision: http://reviews.llvm.org/D4751 Patch by Vadim Chugunov! llvm-svn: 214775
2014-08-05 05:05:27 +08:00
MBB = MBB->getPrevNode();
MBBI = MBB->end();
}
Assigning to a local object in a return statement prevents copy elision. NFC. I added a diagnostic along the lines of `-Wpessimizing-move` to detect `return x = y` suppressing copy elision, but I don't know if the diagnostic is really worth it. Anyway, here are the places where my diagnostic reported that copy elision would have been possible if not for the assignment. P1155R1 in the post-San-Diego WG21 (C++ committee) mailing discusses whether WG21 should fix this pitfall by just changing the core language to permit copy elision in cases like these. (Kona update: The bulk of P1155 is proceeding to CWG review, but specifically *not* the parts that explored the notion of permitting copy-elision in these specific cases.) Reviewed By: dblaikie Author: Arthur O'Dwyer Differential Revision: https://reviews.llvm.org/D54885 llvm-svn: 359236
2019-04-26 04:09:00 +08:00
--MBBI;
return MBBI;
Fix failure to invoke exception handler on Win64 When the last instruction prior to a function epilogue is a call, we need to emit a nop so that the return address is not in the epilogue IP range. This is consistent with MSVC's behavior, and may be a workaround for a bug in the Win64 unwinder. Differential Revision: http://reviews.llvm.org/D4751 Patch by Vadim Chugunov! llvm-svn: 214775
2014-08-05 05:05:27 +08:00
}
[x86] Teach the instruction lowering to add comments describing constant pool data being loaded into a vector register. The comments take the form of: # ymm0 = [a,b,c,d,...] # xmm1 = <x,y,z...> The []s are used for generic sequential data and the <>s are used for specifically ConstantVector loads. Undef elements are printed as the letter 'u', integers in decimal, and floating point values as floating point values. Suggestions on improving the formatting or other aspects of the display are very welcome. My primary use case for this is to be able to FileCheck test masks passed to vector shuffle instructions in-register. It isn't fantastic for that (no decoding special zeroing semantics or other tricks), but it at least puts the mask onto an instruction line that could reasonably be checked. I've updated many of the new vector shuffle lowering tests to leverage this in their test cases so that we're actually checking the shuffle masks remain as expected. Before implementing this, I tried a *bunch* of different approaches. I looked into teaching the MCInstLower code to scan up the basic block and find a definition of a register used in a shuffle instruction and then decode that, but this seems incredibly brittle and complex. I talked to Hal a lot about the "right" way to do this: attach the raw shuffle mask to the instruction itself in some form of unencoded operands, and then use that to emit the comments. I still think that's the optimal solution here, but it proved to be beyond what I'm up for here. In particular, it seems likely best done by completing the plumbing of metadata through these layers and attaching the shuffle mask in metadata which could have fully automatic dropping when encoding an actual instruction. llvm-svn: 218377
2014-09-24 17:39:41 +08:00
static const Constant *getConstantFromPool(const MachineInstr &MI,
const MachineOperand &Op) {
[X86] Only extract constant pool shuffle mask data with zero offsets D53306 exposes an issue where we sometimes use constant pool data from bigger vectors than the target shuffle mask. This should be safe to do, but we have to be certain that we're using the bottom most part of the vector as the shuffle mask decoders have no way to peek into subvectors with non-zero offsets. llvm-svn: 344867
2018-10-21 19:55:56 +08:00
if (!Op.isCPI() || Op.getOffset() != 0)
[x86] More refactoring of the shuffle comment emission. The previous attempt didn't work out so well. It looks like it will be much better for introducing extra logic to find a shuffle mask if the finding logic is totally separate. This also makes it easy to sink the opcode logic completely out of the routine so we don't re-dispatch across it. Still no functionality changed. llvm-svn: 218363
2014-09-24 11:06:37 +08:00
return nullptr;
[x86] Start refactoring the comment printing logic in the MC lowering of vector shuffles. This is just the beginning by hoisting it into its own function and making use of early exit to dramatically simplify the flow of the function. I'm going to be incrementally refactoring this until it is a bit less magical how this applies to other instructions, and I can teach it how to dig a shuffle mask out of a register. Then I plan to hook it up to VPERMD so we get our mask comments for it. No functionality changed yet. llvm-svn: 218357
2014-09-24 10:16:12 +08:00
[x86] More refactoring of the shuffle comment emission. The previous attempt didn't work out so well. It looks like it will be much better for introducing extra logic to find a shuffle mask if the finding logic is totally separate. This also makes it easy to sink the opcode logic completely out of the routine so we don't re-dispatch across it. Still no functionality changed. llvm-svn: 218363
2014-09-24 11:06:37 +08:00
ArrayRef<MachineConstantPoolEntry> Constants =
MI.getParent()->getParent()->getConstantPool()->getConstants();
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
const MachineConstantPoolEntry &ConstantEntry = Constants[Op.getIndex()];
[x86] Start refactoring the comment printing logic in the MC lowering of vector shuffles. This is just the beginning by hoisting it into its own function and making use of early exit to dramatically simplify the flow of the function. I'm going to be incrementally refactoring this until it is a bit less magical how this applies to other instructions, and I can teach it how to dig a shuffle mask out of a register. Then I plan to hook it up to VPERMD so we get our mask comments for it. No functionality changed yet. llvm-svn: 218357
2014-09-24 10:16:12 +08:00
// Bail if this is a machine constant pool entry, we won't be able to dig out
// anything useful.
[x86] Teach the instruction lowering to add comments describing constant pool data being loaded into a vector register. The comments take the form of: # ymm0 = [a,b,c,d,...] # xmm1 = <x,y,z...> The []s are used for generic sequential data and the <>s are used for specifically ConstantVector loads. Undef elements are printed as the letter 'u', integers in decimal, and floating point values as floating point values. Suggestions on improving the formatting or other aspects of the display are very welcome. My primary use case for this is to be able to FileCheck test masks passed to vector shuffle instructions in-register. It isn't fantastic for that (no decoding special zeroing semantics or other tricks), but it at least puts the mask onto an instruction line that could reasonably be checked. I've updated many of the new vector shuffle lowering tests to leverage this in their test cases so that we're actually checking the shuffle masks remain as expected. Before implementing this, I tried a *bunch* of different approaches. I looked into teaching the MCInstLower code to scan up the basic block and find a definition of a register used in a shuffle instruction and then decode that, but this seems incredibly brittle and complex. I talked to Hal a lot about the "right" way to do this: attach the raw shuffle mask to the instruction itself in some form of unencoded operands, and then use that to emit the comments. I still think that's the optimal solution here, but it proved to be beyond what I'm up for here. In particular, it seems likely best done by completing the plumbing of metadata through these layers and attaching the shuffle mask in metadata which could have fully automatic dropping when encoding an actual instruction. llvm-svn: 218377
2014-09-24 17:39:41 +08:00
if (ConstantEntry.isMachineConstantPoolEntry())
[x86] More refactoring of the shuffle comment emission. The previous attempt didn't work out so well. It looks like it will be much better for introducing extra logic to find a shuffle mask if the finding logic is totally separate. This also makes it easy to sink the opcode logic completely out of the routine so we don't re-dispatch across it. Still no functionality changed. llvm-svn: 218363
2014-09-24 11:06:37 +08:00
return nullptr;
[x86] Start refactoring the comment printing logic in the MC lowering of vector shuffles. This is just the beginning by hoisting it into its own function and making use of early exit to dramatically simplify the flow of the function. I'm going to be incrementally refactoring this until it is a bit less magical how this applies to other instructions, and I can teach it how to dig a shuffle mask out of a register. Then I plan to hook it up to VPERMD so we get our mask comments for it. No functionality changed yet. llvm-svn: 218357
2014-09-24 10:16:12 +08:00
[NFC] Add the getSizeInBytes() interface for MachineConstantPoolValue Current implementation assumes that, each MachineConstantPoolValue takes up sizeof(MachineConstantPoolValue::Ty) bytes. For PowerPC, we want to lump all the constants with the same type as one MachineConstantPoolValue to save the cost that calculate the TOC entry for each const. So, we need to extend the MachineConstantPoolValue that break this assumption. Reviewed By: RKSimon Differential Revision: https://reviews.llvm.org/D89108
2021-01-05 11:22:45 +08:00
return ConstantEntry.Val.ConstVal;
[x86] More refactoring of the shuffle comment emission. The previous attempt didn't work out so well. It looks like it will be much better for introducing extra logic to find a shuffle mask if the finding logic is totally separate. This also makes it easy to sink the opcode logic completely out of the routine so we don't re-dispatch across it. Still no functionality changed. llvm-svn: 218363
2014-09-24 11:06:37 +08:00
}
[x86] Start refactoring the comment printing logic in the MC lowering of vector shuffles. This is just the beginning by hoisting it into its own function and making use of early exit to dramatically simplify the flow of the function. I'm going to be incrementally refactoring this until it is a bit less magical how this applies to other instructions, and I can teach it how to dig a shuffle mask out of a register. Then I plan to hook it up to VPERMD so we get our mask comments for it. No functionality changed yet. llvm-svn: 218357
2014-09-24 10:16:12 +08:00
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
static std::string getShuffleComment(const MachineInstr *MI, unsigned SrcOp1Idx,
unsigned SrcOp2Idx, ArrayRef<int> Mask) {
[x86] More refactoring of the shuffle comment emission. The previous attempt didn't work out so well. It looks like it will be much better for introducing extra logic to find a shuffle mask if the finding logic is totally separate. This also makes it easy to sink the opcode logic completely out of the routine so we don't re-dispatch across it. Still no functionality changed. llvm-svn: 218363
2014-09-24 11:06:37 +08:00
std::string Comment;
[x86] Start refactoring the comment printing logic in the MC lowering of vector shuffles. This is just the beginning by hoisting it into its own function and making use of early exit to dramatically simplify the flow of the function. I'm going to be incrementally refactoring this until it is a bit less magical how this applies to other instructions, and I can teach it how to dig a shuffle mask out of a register. Then I plan to hook it up to VPERMD so we get our mask comments for it. No functionality changed yet. llvm-svn: 218357
2014-09-24 10:16:12 +08:00
// Compute the name for a register. This is really goofy because we have
// multiple instruction printers that could (in theory) use different
// names. Fortunately most people use the ATT style (outside of Windows)
// and they actually agree on register naming here. Ultimately, this is
// a comment, and so its OK if it isn't perfect.
auto GetRegisterName = [](unsigned RegNum) -> StringRef {
return X86ATTInstPrinter::getRegisterName(RegNum);
};
[X86][AVX512] Add mask/maskz writemask support to constant pool shuffle decode commentx llvm-svn: 284488
2016-10-18 23:45:37 +08:00
const MachineOperand &DstOp = MI->getOperand(0);
const MachineOperand &SrcOp1 = MI->getOperand(SrcOp1Idx);
const MachineOperand &SrcOp2 = MI->getOperand(SrcOp2Idx);
[x86] Start refactoring the comment printing logic in the MC lowering of vector shuffles. This is just the beginning by hoisting it into its own function and making use of early exit to dramatically simplify the flow of the function. I'm going to be incrementally refactoring this until it is a bit less magical how this applies to other instructions, and I can teach it how to dig a shuffle mask out of a register. Then I plan to hook it up to VPERMD so we get our mask comments for it. No functionality changed yet. llvm-svn: 218357
2014-09-24 10:16:12 +08:00
StringRef DstName = DstOp.isReg() ? GetRegisterName(DstOp.getReg()) : "mem";
[X86][XOP] Support for VPPERM 2-input shuffle mask decoding This patch adds support for decoding XOP VPPERM instruction when it represents a basic shuffle. The mask decoding required the existing MCInstrLowering code to be updated to support binary shuffles - the implementation now matches what is done in X86InstrComments.cpp. Differential Revision: http://reviews.llvm.org/D18441 llvm-svn: 265874
2016-04-09 22:51:26 +08:00
StringRef Src1Name =
SrcOp1.isReg() ? GetRegisterName(SrcOp1.getReg()) : "mem";
StringRef Src2Name =
SrcOp2.isReg() ? GetRegisterName(SrcOp2.getReg()) : "mem";
// One source operand, fix the mask to print all elements in one span.
SmallVector<int, 8> ShuffleMask(Mask.begin(), Mask.end());
if (Src1Name == Src2Name)
for (int i = 0, e = ShuffleMask.size(); i != e; ++i)
if (ShuffleMask[i] >= e)
ShuffleMask[i] -= e;
[x86] Start refactoring the comment printing logic in the MC lowering of vector shuffles. This is just the beginning by hoisting it into its own function and making use of early exit to dramatically simplify the flow of the function. I'm going to be incrementally refactoring this until it is a bit less magical how this applies to other instructions, and I can teach it how to dig a shuffle mask out of a register. Then I plan to hook it up to VPERMD so we get our mask comments for it. No functionality changed yet. llvm-svn: 218357
2014-09-24 10:16:12 +08:00
raw_string_ostream CS(Comment);
[X86][AVX512] Add mask/maskz writemask support to constant pool shuffle decode commentx llvm-svn: 284488
2016-10-18 23:45:37 +08:00
CS << DstName;
// Handle AVX512 MASK/MASXZ write mask comments.
// MASK: zmmX {%kY}
// MASKZ: zmmX {%kY} {z}
if (SrcOp1Idx > 1) {
assert((SrcOp1Idx == 2 || SrcOp1Idx == 3) && "Unexpected writemask");
const MachineOperand &WriteMaskOp = MI->getOperand(SrcOp1Idx - 1);
if (WriteMaskOp.isReg()) {
CS << " {%" << GetRegisterName(WriteMaskOp.getReg()) << "}";
if (SrcOp1Idx == 2) {
CS << " {z}";
}
}
}
CS << " = ";
[X86][XOP] Support for VPPERM 2-input shuffle mask decoding This patch adds support for decoding XOP VPPERM instruction when it represents a basic shuffle. The mask decoding required the existing MCInstrLowering code to be updated to support binary shuffles - the implementation now matches what is done in X86InstrComments.cpp. Differential Revision: http://reviews.llvm.org/D18441 llvm-svn: 265874
2016-04-09 22:51:26 +08:00
for (int i = 0, e = ShuffleMask.size(); i != e; ++i) {
if (i != 0)
[x86] Start refactoring the comment printing logic in the MC lowering of vector shuffles. This is just the beginning by hoisting it into its own function and making use of early exit to dramatically simplify the flow of the function. I'm going to be incrementally refactoring this until it is a bit less magical how this applies to other instructions, and I can teach it how to dig a shuffle mask out of a register. Then I plan to hook it up to VPERMD so we get our mask comments for it. No functionality changed yet. llvm-svn: 218357
2014-09-24 10:16:12 +08:00
CS << ",";
[X86][XOP] Support for VPPERM 2-input shuffle mask decoding This patch adds support for decoding XOP VPPERM instruction when it represents a basic shuffle. The mask decoding required the existing MCInstrLowering code to be updated to support binary shuffles - the implementation now matches what is done in X86InstrComments.cpp. Differential Revision: http://reviews.llvm.org/D18441 llvm-svn: 265874
2016-04-09 22:51:26 +08:00
if (ShuffleMask[i] == SM_SentinelZero) {
[x86] Start refactoring the comment printing logic in the MC lowering of vector shuffles. This is just the beginning by hoisting it into its own function and making use of early exit to dramatically simplify the flow of the function. I'm going to be incrementally refactoring this until it is a bit less magical how this applies to other instructions, and I can teach it how to dig a shuffle mask out of a register. Then I plan to hook it up to VPERMD so we get our mask comments for it. No functionality changed yet. llvm-svn: 218357
2014-09-24 10:16:12 +08:00
CS << "zero";
[X86][XOP] Support for VPPERM 2-input shuffle mask decoding This patch adds support for decoding XOP VPPERM instruction when it represents a basic shuffle. The mask decoding required the existing MCInstrLowering code to be updated to support binary shuffles - the implementation now matches what is done in X86InstrComments.cpp. Differential Revision: http://reviews.llvm.org/D18441 llvm-svn: 265874
2016-04-09 22:51:26 +08:00
continue;
}
// Otherwise, it must come from src1 or src2. Print the span of elements
// that comes from this src.
bool isSrc1 = ShuffleMask[i] < (int)e;
CS << (isSrc1 ? Src1Name : Src2Name) << '[';
bool IsFirst = true;
while (i != e && ShuffleMask[i] != SM_SentinelZero &&
(ShuffleMask[i] < (int)e) == isSrc1) {
if (!IsFirst)
CS << ',';
else
IsFirst = false;
if (ShuffleMask[i] == SM_SentinelUndef)
[x86] Start refactoring the comment printing logic in the MC lowering of vector shuffles. This is just the beginning by hoisting it into its own function and making use of early exit to dramatically simplify the flow of the function. I'm going to be incrementally refactoring this until it is a bit less magical how this applies to other instructions, and I can teach it how to dig a shuffle mask out of a register. Then I plan to hook it up to VPERMD so we get our mask comments for it. No functionality changed yet. llvm-svn: 218357
2014-09-24 10:16:12 +08:00
CS << "u";
else
[X86][XOP] Support for VPPERM 2-input shuffle mask decoding This patch adds support for decoding XOP VPPERM instruction when it represents a basic shuffle. The mask decoding required the existing MCInstrLowering code to be updated to support binary shuffles - the implementation now matches what is done in X86InstrComments.cpp. Differential Revision: http://reviews.llvm.org/D18441 llvm-svn: 265874
2016-04-09 22:51:26 +08:00
CS << ShuffleMask[i] % (int)e;
++i;
[x86] Start refactoring the comment printing logic in the MC lowering of vector shuffles. This is just the beginning by hoisting it into its own function and making use of early exit to dramatically simplify the flow of the function. I'm going to be incrementally refactoring this until it is a bit less magical how this applies to other instructions, and I can teach it how to dig a shuffle mask out of a register. Then I plan to hook it up to VPERMD so we get our mask comments for it. No functionality changed yet. llvm-svn: 218357
2014-09-24 10:16:12 +08:00
}
[X86][XOP] Support for VPPERM 2-input shuffle mask decoding This patch adds support for decoding XOP VPPERM instruction when it represents a basic shuffle. The mask decoding required the existing MCInstrLowering code to be updated to support binary shuffles - the implementation now matches what is done in X86InstrComments.cpp. Differential Revision: http://reviews.llvm.org/D18441 llvm-svn: 265874
2016-04-09 22:51:26 +08:00
CS << ']';
--i; // For loop increments element #.
[x86] Start refactoring the comment printing logic in the MC lowering of vector shuffles. This is just the beginning by hoisting it into its own function and making use of early exit to dramatically simplify the flow of the function. I'm going to be incrementally refactoring this until it is a bit less magical how this applies to other instructions, and I can teach it how to dig a shuffle mask out of a register. Then I plan to hook it up to VPERMD so we get our mask comments for it. No functionality changed yet. llvm-svn: 218357
2014-09-24 10:16:12 +08:00
}
CS.flush();
return Comment;
}
[X86] Add APInt constant assembly printer helper llvm-svn: 343577
2018-10-02 19:32:33 +08:00
static void printConstant(const APInt &Val, raw_ostream &CS) {
if (Val.getBitWidth() <= 64) {
CS << Val.getZExtValue();
} else {
// print multi-word constant as (w0,w1)
CS << "(";
for (int i = 0, N = Val.getNumWords(); i < N; ++i) {
if (i > 0)
CS << ",";
CS << Val.getRawData()[i];
}
CS << ")";
}
}
static void printConstant(const APFloat &Flt, raw_ostream &CS) {
[X86] Standardize floating point assembly comments Consistently try to use APFloat::toString for floating point constant comments to get rid of differences between Constant / ConstantDataSequential values - it should help stop some of the linux-windows buildbot failures matching NaN/INF etc. as well. Differential Revision: https://reviews.llvm.org/D52702 llvm-svn: 343562
2018-10-02 17:08:51 +08:00
SmallString<32> Str;
[X86] Force floating point values in constant pool decoding to print in scientific notation so they can't be confused with integers. When the floating point constants are whole numbers they have no decimal point so look like integers, but mean something very different in something like an 'and' instruction. Ideally we would just print a decimal point and a 0, but I couldn't see how to make APFloat::toString do that. llvm-svn: 345488
2018-10-29 12:52:04 +08:00
// Force scientific notation to distinquish from integers.
Flt.toString(Str, 0, 0);
[X86] Standardize floating point assembly comments Consistently try to use APFloat::toString for floating point constant comments to get rid of differences between Constant / ConstantDataSequential values - it should help stop some of the linux-windows buildbot failures matching NaN/INF etc. as well. Differential Revision: https://reviews.llvm.org/D52702 llvm-svn: 343562
2018-10-02 17:08:51 +08:00
CS << Str;
}
[X86] Add comment string for broadcast loads from the constant pool. Summary: When broadcasting from the constant pool its useful to print out the final vector similar to what we do for normal moves from the constant pool. I changed only a couple tests that were broadcast focused. One of them had been previously hand tweaked after running the script so that it could check the constant pool declaration. But I think this patch makes that unnecessary now since we can check the comment instead. Reviewers: spatel, RKSimon, zvi Reviewed By: spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D34923 llvm-svn: 307062
2017-07-04 13:46:11 +08:00
static void printConstant(const Constant *COp, raw_ostream &CS) {
if (isa<UndefValue>(COp)) {
CS << "u";
} else if (auto *CI = dyn_cast<ConstantInt>(COp)) {
[X86] Add APInt constant assembly printer helper llvm-svn: 343577
2018-10-02 19:32:33 +08:00
printConstant(CI->getValue(), CS);
[X86] Add comment string for broadcast loads from the constant pool. Summary: When broadcasting from the constant pool its useful to print out the final vector similar to what we do for normal moves from the constant pool. I changed only a couple tests that were broadcast focused. One of them had been previously hand tweaked after running the script so that it could check the constant pool declaration. But I think this patch makes that unnecessary now since we can check the comment instead. Reviewers: spatel, RKSimon, zvi Reviewed By: spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D34923 llvm-svn: 307062
2017-07-04 13:46:11 +08:00
} else if (auto *CF = dyn_cast<ConstantFP>(COp)) {
[X86] Standardize floating point assembly comments Consistently try to use APFloat::toString for floating point constant comments to get rid of differences between Constant / ConstantDataSequential values - it should help stop some of the linux-windows buildbot failures matching NaN/INF etc. as well. Differential Revision: https://reviews.llvm.org/D52702 llvm-svn: 343562
2018-10-02 17:08:51 +08:00
printConstant(CF->getValueAPF(), CS);
[X86] Add comment string for broadcast loads from the constant pool. Summary: When broadcasting from the constant pool its useful to print out the final vector similar to what we do for normal moves from the constant pool. I changed only a couple tests that were broadcast focused. One of them had been previously hand tweaked after running the script so that it could check the constant pool declaration. But I think this patch makes that unnecessary now since we can check the comment instead. Reviewers: spatel, RKSimon, zvi Reviewed By: spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D34923 llvm-svn: 307062
2017-07-04 13:46:11 +08:00
} else {
CS << "?";
}
}
[codeview] Implement FPO data assembler directives Summary: This adds a set of new directives that describe 32-bit x86 prologues. The directives are limited and do not expose the full complexity of codeview FPO data. They are merely a convenience for the compiler to generate more readable assembly so we don't need to generate tons of labels in CodeGen. If our prologue emission changes in the future, we can change the set of available directives to suit our needs. These are modelled after the .seh_ directives, which use a different format that interacts with exception handling. The directives are: .cv_fpo_proc _foo .cv_fpo_pushreg ebp/ebx/etc .cv_fpo_setframe ebp/esi/etc .cv_fpo_stackalloc 200 .cv_fpo_endprologue .cv_fpo_endproc .cv_fpo_data _foo I tried to follow the implementation of ARM EHABI CFI directives by sinking most directives out of MCStreamer and into X86TargetStreamer. This helps avoid polluting non-X86 code with WinCOFF specific logic. I used cdb to confirm that this can show locals in parent CSRs in a few cases, most importantly the one where we use ESI as a frame pointer, i.e. the one in http://crbug.com/756153#c28 Once we have cdb integration in debuginfo-tests, we can add integration tests there. Reviewers: majnemer, hans Subscribers: aemerson, mgorny, kristof.beyls, llvm-commits, hiraditya Differential Revision: https://reviews.llvm.org/D38776 llvm-svn: 315513
2017-10-12 05:24:33 +08:00
void X86AsmPrinter::EmitSEHInstruction(const MachineInstr *MI) {
assert(MF->hasWinCFI() && "SEH_ instruction in function without WinCFI?");
assert(getSubtarget().isOSWindows() && "SEH_ instruction Windows only");
// Use the .cv_fpo directives if we're emitting CodeView on 32-bit x86.
if (EmitFPOData) {
X86TargetStreamer *XTS =
static_cast<X86TargetStreamer *>(OutStreamer->getTargetStreamer());
switch (MI->getOpcode()) {
case X86::SEH_PushReg:
XTS->emitFPOPushReg(MI->getOperand(0).getImm());
break;
case X86::SEH_StackAlloc:
XTS->emitFPOStackAlloc(MI->getOperand(0).getImm());
break;
[codeview] Fix 32-bit x86 variable locations in realigned stack frames Add the .cv_fpo_stackalign directive so that we can define $T0, or the VFRAME virtual register, with it. This was overlooked in the initial implementation because unlike MSVC, we push CSRs before allocating stack space, so this value is only needed to describe local variable locations. Variables that the compiler now addresses via ESP are instead described as being stored at offsets from VFRAME, which for us is ESP after alignment in the prologue. This adds tests that show that we use the VFRAME register properly in our S_DEFRANGE records, and that we emit the correct FPO data to define it. Fixes PR38857 llvm-svn: 343603
2018-10-03 00:43:52 +08:00
case X86::SEH_StackAlign:
XTS->emitFPOStackAlign(MI->getOperand(0).getImm());
break;
[codeview] Implement FPO data assembler directives Summary: This adds a set of new directives that describe 32-bit x86 prologues. The directives are limited and do not expose the full complexity of codeview FPO data. They are merely a convenience for the compiler to generate more readable assembly so we don't need to generate tons of labels in CodeGen. If our prologue emission changes in the future, we can change the set of available directives to suit our needs. These are modelled after the .seh_ directives, which use a different format that interacts with exception handling. The directives are: .cv_fpo_proc _foo .cv_fpo_pushreg ebp/ebx/etc .cv_fpo_setframe ebp/esi/etc .cv_fpo_stackalloc 200 .cv_fpo_endprologue .cv_fpo_endproc .cv_fpo_data _foo I tried to follow the implementation of ARM EHABI CFI directives by sinking most directives out of MCStreamer and into X86TargetStreamer. This helps avoid polluting non-X86 code with WinCOFF specific logic. I used cdb to confirm that this can show locals in parent CSRs in a few cases, most importantly the one where we use ESI as a frame pointer, i.e. the one in http://crbug.com/756153#c28 Once we have cdb integration in debuginfo-tests, we can add integration tests there. Reviewers: majnemer, hans Subscribers: aemerson, mgorny, kristof.beyls, llvm-commits, hiraditya Differential Revision: https://reviews.llvm.org/D38776 llvm-svn: 315513
2017-10-12 05:24:33 +08:00
case X86::SEH_SetFrame:
assert(MI->getOperand(1).getImm() == 0 &&
".cv_fpo_setframe takes no offset");
XTS->emitFPOSetFrame(MI->getOperand(0).getImm());
break;
case X86::SEH_EndPrologue:
XTS->emitFPOEndPrologue();
break;
case X86::SEH_SaveReg:
case X86::SEH_SaveXMM:
case X86::SEH_PushFrame:
llvm_unreachable("SEH_ directive incompatible with FPO");
break;
default:
llvm_unreachable("expected SEH_ instruction");
}
return;
}
// Otherwise, use the .seh_ directives for all other Windows platforms.
switch (MI->getOpcode()) {
case X86::SEH_PushReg:
[X86] Print register names in .seh_* directives Also improve assembler parser register validation for .seh_ directives. This requires moving X86-specific seh directive handling into the x86 backend, which addresses some assembler FIXMEs. Differential Revision: https://reviews.llvm.org/D66625 llvm-svn: 370533
2019-08-31 05:23:05 +08:00
OutStreamer->EmitWinCFIPushReg(MI->getOperand(0).getImm());
[codeview] Implement FPO data assembler directives Summary: This adds a set of new directives that describe 32-bit x86 prologues. The directives are limited and do not expose the full complexity of codeview FPO data. They are merely a convenience for the compiler to generate more readable assembly so we don't need to generate tons of labels in CodeGen. If our prologue emission changes in the future, we can change the set of available directives to suit our needs. These are modelled after the .seh_ directives, which use a different format that interacts with exception handling. The directives are: .cv_fpo_proc _foo .cv_fpo_pushreg ebp/ebx/etc .cv_fpo_setframe ebp/esi/etc .cv_fpo_stackalloc 200 .cv_fpo_endprologue .cv_fpo_endproc .cv_fpo_data _foo I tried to follow the implementation of ARM EHABI CFI directives by sinking most directives out of MCStreamer and into X86TargetStreamer. This helps avoid polluting non-X86 code with WinCOFF specific logic. I used cdb to confirm that this can show locals in parent CSRs in a few cases, most importantly the one where we use ESI as a frame pointer, i.e. the one in http://crbug.com/756153#c28 Once we have cdb integration in debuginfo-tests, we can add integration tests there. Reviewers: majnemer, hans Subscribers: aemerson, mgorny, kristof.beyls, llvm-commits, hiraditya Differential Revision: https://reviews.llvm.org/D38776 llvm-svn: 315513
2017-10-12 05:24:33 +08:00
break;
case X86::SEH_SaveReg:
[X86] Print register names in .seh_* directives Also improve assembler parser register validation for .seh_ directives. This requires moving X86-specific seh directive handling into the x86 backend, which addresses some assembler FIXMEs. Differential Revision: https://reviews.llvm.org/D66625 llvm-svn: 370533
2019-08-31 05:23:05 +08:00
OutStreamer->EmitWinCFISaveReg(MI->getOperand(0).getImm(),
[codeview] Implement FPO data assembler directives Summary: This adds a set of new directives that describe 32-bit x86 prologues. The directives are limited and do not expose the full complexity of codeview FPO data. They are merely a convenience for the compiler to generate more readable assembly so we don't need to generate tons of labels in CodeGen. If our prologue emission changes in the future, we can change the set of available directives to suit our needs. These are modelled after the .seh_ directives, which use a different format that interacts with exception handling. The directives are: .cv_fpo_proc _foo .cv_fpo_pushreg ebp/ebx/etc .cv_fpo_setframe ebp/esi/etc .cv_fpo_stackalloc 200 .cv_fpo_endprologue .cv_fpo_endproc .cv_fpo_data _foo I tried to follow the implementation of ARM EHABI CFI directives by sinking most directives out of MCStreamer and into X86TargetStreamer. This helps avoid polluting non-X86 code with WinCOFF specific logic. I used cdb to confirm that this can show locals in parent CSRs in a few cases, most importantly the one where we use ESI as a frame pointer, i.e. the one in http://crbug.com/756153#c28 Once we have cdb integration in debuginfo-tests, we can add integration tests there. Reviewers: majnemer, hans Subscribers: aemerson, mgorny, kristof.beyls, llvm-commits, hiraditya Differential Revision: https://reviews.llvm.org/D38776 llvm-svn: 315513
2017-10-12 05:24:33 +08:00
MI->getOperand(1).getImm());
break;
case X86::SEH_SaveXMM:
[X86] Print register names in .seh_* directives Also improve assembler parser register validation for .seh_ directives. This requires moving X86-specific seh directive handling into the x86 backend, which addresses some assembler FIXMEs. Differential Revision: https://reviews.llvm.org/D66625 llvm-svn: 370533
2019-08-31 05:23:05 +08:00
OutStreamer->EmitWinCFISaveXMM(MI->getOperand(0).getImm(),
[codeview] Implement FPO data assembler directives Summary: This adds a set of new directives that describe 32-bit x86 prologues. The directives are limited and do not expose the full complexity of codeview FPO data. They are merely a convenience for the compiler to generate more readable assembly so we don't need to generate tons of labels in CodeGen. If our prologue emission changes in the future, we can change the set of available directives to suit our needs. These are modelled after the .seh_ directives, which use a different format that interacts with exception handling. The directives are: .cv_fpo_proc _foo .cv_fpo_pushreg ebp/ebx/etc .cv_fpo_setframe ebp/esi/etc .cv_fpo_stackalloc 200 .cv_fpo_endprologue .cv_fpo_endproc .cv_fpo_data _foo I tried to follow the implementation of ARM EHABI CFI directives by sinking most directives out of MCStreamer and into X86TargetStreamer. This helps avoid polluting non-X86 code with WinCOFF specific logic. I used cdb to confirm that this can show locals in parent CSRs in a few cases, most importantly the one where we use ESI as a frame pointer, i.e. the one in http://crbug.com/756153#c28 Once we have cdb integration in debuginfo-tests, we can add integration tests there. Reviewers: majnemer, hans Subscribers: aemerson, mgorny, kristof.beyls, llvm-commits, hiraditya Differential Revision: https://reviews.llvm.org/D38776 llvm-svn: 315513
2017-10-12 05:24:33 +08:00
MI->getOperand(1).getImm());
break;
case X86::SEH_StackAlloc:
OutStreamer->EmitWinCFIAllocStack(MI->getOperand(0).getImm());
break;
case X86::SEH_SetFrame:
[X86] Print register names in .seh_* directives Also improve assembler parser register validation for .seh_ directives. This requires moving X86-specific seh directive handling into the x86 backend, which addresses some assembler FIXMEs. Differential Revision: https://reviews.llvm.org/D66625 llvm-svn: 370533
2019-08-31 05:23:05 +08:00
OutStreamer->EmitWinCFISetFrame(MI->getOperand(0).getImm(),
MI->getOperand(1).getImm());
[codeview] Implement FPO data assembler directives Summary: This adds a set of new directives that describe 32-bit x86 prologues. The directives are limited and do not expose the full complexity of codeview FPO data. They are merely a convenience for the compiler to generate more readable assembly so we don't need to generate tons of labels in CodeGen. If our prologue emission changes in the future, we can change the set of available directives to suit our needs. These are modelled after the .seh_ directives, which use a different format that interacts with exception handling. The directives are: .cv_fpo_proc _foo .cv_fpo_pushreg ebp/ebx/etc .cv_fpo_setframe ebp/esi/etc .cv_fpo_stackalloc 200 .cv_fpo_endprologue .cv_fpo_endproc .cv_fpo_data _foo I tried to follow the implementation of ARM EHABI CFI directives by sinking most directives out of MCStreamer and into X86TargetStreamer. This helps avoid polluting non-X86 code with WinCOFF specific logic. I used cdb to confirm that this can show locals in parent CSRs in a few cases, most importantly the one where we use ESI as a frame pointer, i.e. the one in http://crbug.com/756153#c28 Once we have cdb integration in debuginfo-tests, we can add integration tests there. Reviewers: majnemer, hans Subscribers: aemerson, mgorny, kristof.beyls, llvm-commits, hiraditya Differential Revision: https://reviews.llvm.org/D38776 llvm-svn: 315513
2017-10-12 05:24:33 +08:00
break;
case X86::SEH_PushFrame:
OutStreamer->EmitWinCFIPushFrame(MI->getOperand(0).getImm());
break;
case X86::SEH_EndPrologue:
OutStreamer->EmitWinCFIEndProlog();
break;
default:
llvm_unreachable("expected SEH_ instruction");
}
}
Recommit r344877 "[X86] Stop promoting integer loads to vXi64" I've included a fix to DAGCombiner::ForwardStoreValueToDirectLoad that I believe will prevent the previous miscompile. Original commit message: Theoretically this was done to simplify the amount of isel patterns that were needed. But it also meant a substantial number of our isel patterns have to match an explicit bitcast. By making the vXi32/vXi16/vXi8 types legal for loads, DAG combiner should be able to change the load type to rem I had to add some additional plain load instruction patterns and a few other special cases, but overall the isel table has reduced in size by ~12000 bytes. So it looks like this promotion was hurting us more than helping. I still have one crash in vector-trunc.ll that I'm hoping @RKSimon can help with. It seems to relate to using getTargetConstantFromNode on a load that was shrunk due to an extract_subvector combine after the constant pool entry was created. So we end up decoding more mask elements than the lo I'm hoping this patch will simplify the number of patterns needed to remove the and/or/xor promotion. Reviewers: RKSimon, spatel Reviewed By: RKSimon Subscribers: llvm-commits, RKSimon Differential Revision: https://reviews.llvm.org/D53306 llvm-svn: 344965
2018-10-23 06:14:05 +08:00
static unsigned getRegisterWidth(const MCOperandInfo &Info) {
if (Info.RegClass == X86::VR128RegClassID ||
Info.RegClass == X86::VR128XRegClassID)
return 128;
if (Info.RegClass == X86::VR256RegClassID ||
Info.RegClass == X86::VR256XRegClassID)
return 256;
if (Info.RegClass == X86::VR512RegClassID)
return 512;
llvm_unreachable("Unknown register class!");
}
[X86] Factor constant pool comment printing out of the switch in X86AsmPrinter::emitInstruction. NFC Pull the verbose asm check out of the cases and move it up to the call of the new function.
2020-05-30 15:31:21 +08:00
static void addConstantComments(const MachineInstr *MI,
MCStreamer &OutStreamer) {
split MachineInstr -> MCInst lowering into its own class (not being embedded into X86ATTAsmPrinter). This still depends heavily on X86ATTAsmPrinter, but this is a step in the right direction. llvm-svn: 81627
2009-09-13 04:34:57 +08:00
switch (MI->getOpcode()) {
[X86] Add support for printing shuffle comments for AVX512 PSHUFB instructions. llvm-svn: 256452
2015-12-27 03:48:43 +08:00
// Lower PSHUFB and VPERMILP normally but add a comment if we can find
// a constant shuffle mask. We won't be able to do this at the MC layer
// because the mask isn't an immediate.
[x86] Teach the X86 backend to print shuffle comments for PSHUFB instructions which happen to have a constant mask. Currently, this only handles a very narrow set of cases, but those happen to be the cases that I care about for testing shuffles sanely. This is a bit trickier than other shuffle instructions because we're decoding constants out of the constant pool. The current MC layer makes it completely impossible to inspect a constant pool entry, so we have to do it at the MI level and attach the comment to the streamer on its way out. So no joy for disassembling, but it does make test cases and asm dumps *much* nicer. Sorry for no test cases, but it didn't really seem that valuable to go trolling through existing old test cases and updating them. I'll have lots of testing of this in the upcoming patch for SSSE3 emission in the new vector shuffle lowering code paths. llvm-svn: 213986
2014-07-26 07:47:11 +08:00
case X86::PSHUFBrm:
[x86] Implement v16i16 support with AVX2 in the new vector shuffle lowering. This also implements the fancy blend lowering for v16i16 using AVX2 and teaches the X86 backend to print shuffle masks for 256-bit PSHUFB and PBLENDW instructions. It also makes the mask decoding correct for PBLENDW instructions. The yaks, they are legion. Tests are updated accordingly. There are some missing tests for the VBLENDVB lowering, but I'll add those in a follow-up as this commit has accumulated enough cruft already. llvm-svn: 218430
2014-09-25 08:24:19 +08:00
case X86::VPSHUFBrm:
[X86] Add support for printing shuffle comments for AVX512 PSHUFB instructions. llvm-svn: 256452
2015-12-27 03:48:43 +08:00
case X86::VPSHUFBYrm:
case X86::VPSHUFBZ128rm:
case X86::VPSHUFBZ128rmk:
case X86::VPSHUFBZ128rmkz:
case X86::VPSHUFBZ256rm:
case X86::VPSHUFBZ256rmk:
case X86::VPSHUFBZ256rmkz:
case X86::VPSHUFBZrm:
case X86::VPSHUFBZrmk:
case X86::VPSHUFBZrmkz: {
[X86] Minor cleanups to addShuffleComments in X86MCInstPrinter.cpp. NFCI -Replace some ifs that should be impossible with asserts. -Use X86::AddrDisp and X86::AddrNumOperands to make code more readable -Use X86II::isKMasked/isKMergeMasked to do some operand skipping to remove or simplify switches
2020-05-31 03:53:56 +08:00
unsigned SrcIdx = 1;
if (X86II::isKMasked(MI->getDesc().TSFlags)) {
// Skip mask operand.
++SrcIdx;
if (X86II::isKMergeMasked(MI->getDesc().TSFlags)) {
// Skip passthru operand.
++SrcIdx;
}
[X86] Add support for printing shuffle comments for AVX512 PSHUFB instructions. llvm-svn: 256452
2015-12-27 03:48:43 +08:00
}
[X86] Minor cleanups to addShuffleComments in X86MCInstPrinter.cpp. NFCI -Replace some ifs that should be impossible with asserts. -Use X86::AddrDisp and X86::AddrNumOperands to make code more readable -Use X86II::isKMasked/isKMergeMasked to do some operand skipping to remove or simplify switches
2020-05-31 03:53:56 +08:00
unsigned MaskIdx = SrcIdx + 1 + X86::AddrDisp;
[X86] Add support for printing shuffle comments for AVX512 PSHUFB instructions. llvm-svn: 256452
2015-12-27 03:48:43 +08:00
[X86] Minor cleanups to addShuffleComments in X86MCInstPrinter.cpp. NFCI -Replace some ifs that should be impossible with asserts. -Use X86::AddrDisp and X86::AddrNumOperands to make code more readable -Use X86II::isKMasked/isKMergeMasked to do some operand skipping to remove or simplify switches
2020-05-31 03:53:56 +08:00
assert(MI->getNumOperands() >= (SrcIdx + 1 + X86::AddrNumOperands) &&
"Unexpected number of operands!");
[x86] More refactoring of the shuffle comment emission. The previous attempt didn't work out so well. It looks like it will be much better for introducing extra logic to find a shuffle mask if the finding logic is totally separate. This also makes it easy to sink the opcode logic completely out of the routine so we don't re-dispatch across it. Still no functionality changed. llvm-svn: 218363
2014-09-24 11:06:37 +08:00
[X86][AVX512] Add mask/maskz writemask support to constant pool shuffle decode commentx llvm-svn: 284488
2016-10-18 23:45:37 +08:00
const MachineOperand &MaskOp = MI->getOperand(MaskIdx);
[x86] Teach the instruction lowering to add comments describing constant pool data being loaded into a vector register. The comments take the form of: # ymm0 = [a,b,c,d,...] # xmm1 = <x,y,z...> The []s are used for generic sequential data and the <>s are used for specifically ConstantVector loads. Undef elements are printed as the letter 'u', integers in decimal, and floating point values as floating point values. Suggestions on improving the formatting or other aspects of the display are very welcome. My primary use case for this is to be able to FileCheck test masks passed to vector shuffle instructions in-register. It isn't fantastic for that (no decoding special zeroing semantics or other tricks), but it at least puts the mask onto an instruction line that could reasonably be checked. I've updated many of the new vector shuffle lowering tests to leverage this in their test cases so that we're actually checking the shuffle masks remain as expected. Before implementing this, I tried a *bunch* of different approaches. I looked into teaching the MCInstLower code to scan up the basic block and find a definition of a register used in a shuffle instruction and then decode that, but this seems incredibly brittle and complex. I talked to Hal a lot about the "right" way to do this: attach the raw shuffle mask to the instruction itself in some form of unencoded operands, and then use that to emit the comments. I still think that's the optimal solution here, but it proved to be beyond what I'm up for here. In particular, it seems likely best done by completing the plumbing of metadata through these layers and attaching the shuffle mask in metadata which could have fully automatic dropping when encoding an actual instruction. llvm-svn: 218377
2014-09-24 17:39:41 +08:00
if (auto *C = getConstantFromPool(*MI, MaskOp)) {
Recommit r344877 "[X86] Stop promoting integer loads to vXi64" I've included a fix to DAGCombiner::ForwardStoreValueToDirectLoad that I believe will prevent the previous miscompile. Original commit message: Theoretically this was done to simplify the amount of isel patterns that were needed. But it also meant a substantial number of our isel patterns have to match an explicit bitcast. By making the vXi32/vXi16/vXi8 types legal for loads, DAG combiner should be able to change the load type to rem I had to add some additional plain load instruction patterns and a few other special cases, but overall the isel table has reduced in size by ~12000 bytes. So it looks like this promotion was hurting us more than helping. I still have one crash in vector-trunc.ll that I'm hoping @RKSimon can help with. It seems to relate to using getTargetConstantFromNode on a load that was shrunk due to an extract_subvector combine after the constant pool entry was created. So we end up decoding more mask elements than the lo I'm hoping this patch will simplify the number of patterns needed to remove the and/or/xor promotion. Reviewers: RKSimon, spatel Reviewed By: RKSimon Subscribers: llvm-commits, RKSimon Differential Revision: https://reviews.llvm.org/D53306 llvm-svn: 344965
2018-10-23 06:14:05 +08:00
unsigned Width = getRegisterWidth(MI->getDesc().OpInfo[0]);
[X86] Size a SmallVector to the worst case mask size for a 512-bit shuffle. NFCI llvm-svn: 287908
2016-11-25 10:29:21 +08:00
SmallVector<int, 64> Mask;
Recommit r344877 "[X86] Stop promoting integer loads to vXi64" I've included a fix to DAGCombiner::ForwardStoreValueToDirectLoad that I believe will prevent the previous miscompile. Original commit message: Theoretically this was done to simplify the amount of isel patterns that were needed. But it also meant a substantial number of our isel patterns have to match an explicit bitcast. By making the vXi32/vXi16/vXi8 types legal for loads, DAG combiner should be able to change the load type to rem I had to add some additional plain load instruction patterns and a few other special cases, but overall the isel table has reduced in size by ~12000 bytes. So it looks like this promotion was hurting us more than helping. I still have one crash in vector-trunc.ll that I'm hoping @RKSimon can help with. It seems to relate to using getTargetConstantFromNode on a load that was shrunk due to an extract_subvector combine after the constant pool entry was created. So we end up decoding more mask elements than the lo I'm hoping this patch will simplify the number of patterns needed to remove the and/or/xor promotion. Reviewers: RKSimon, spatel Reviewed By: RKSimon Subscribers: llvm-commits, RKSimon Differential Revision: https://reviews.llvm.org/D53306 llvm-svn: 344965
2018-10-23 06:14:05 +08:00
DecodePSHUFBMask(C, Width, Mask);
[x86] More refactoring of the shuffle comment emission. The previous attempt didn't work out so well. It looks like it will be much better for introducing extra logic to find a shuffle mask if the finding logic is totally separate. This also makes it easy to sink the opcode logic completely out of the routine so we don't re-dispatch across it. Still no functionality changed. llvm-svn: 218363
2014-09-24 11:06:37 +08:00
if (!Mask.empty())
[X86] Factor constant pool comment printing out of the switch in X86AsmPrinter::emitInstruction. NFC Pull the verbose asm check out of the cases and move it up to the call of the new function.
2020-05-30 15:31:21 +08:00
OutStreamer.AddComment(getShuffleComment(MI, SrcIdx, SrcIdx, Mask));
[x86] More refactoring of the shuffle comment emission. The previous attempt didn't work out so well. It looks like it will be much better for introducing extra logic to find a shuffle mask if the finding logic is totally separate. This also makes it easy to sink the opcode logic completely out of the routine so we don't re-dispatch across it. Still no functionality changed. llvm-svn: 218363
2014-09-24 11:06:37 +08:00
}
break;
}
[X86][AVX512] Add support for VPERMILPD/VPERMILPS variable shuffle mask comments llvm-svn: 275272
2016-07-13 23:45:36 +08:00
[AVX-512] Add support for decoding shuffle mask from constant pool for masked VPERMILPS/PD. llvm-svn: 284450
2016-10-18 11:36:52 +08:00
case X86::VPERMILPSrm:
case X86::VPERMILPSYrm:
case X86::VPERMILPSZ128rm:
case X86::VPERMILPSZ128rmk:
case X86::VPERMILPSZ128rmkz:
case X86::VPERMILPSZ256rm:
case X86::VPERMILPSZ256rmk:
case X86::VPERMILPSZ256rmkz:
case X86::VPERMILPSZrm:
case X86::VPERMILPSZrmk:
case X86::VPERMILPSZrmkz:
[x86] Teach the AVX1 path of the new vector shuffle lowering one more trick that I missed. VPERMILPS has a non-immediate memory operand mode that allows it to do asymetric shuffles in the two 128-bit lanes. Use this rather than two shuffles and a blend. However, it turns out the variable shuffle path to VPERMILPS (and VPERMILPD, although that one offers no functional differenc from the immediate operand other than variability) wasn't even plumbed through codegen. Do such plumbing so that we can reasonably emit a variable-masked VPERMILP instruction. Also plumb basic comment parsing and printing through so that the tests are reasonable. There are still a few tests which don't show the shuffle pattern. These are tests with undef lanes. I'll teach the shuffle decoding and printing to handle undef mask entries in a follow-up. I've looked at the masks and they seem reasonable. llvm-svn: 218300
2014-09-23 18:08:29 +08:00
case X86::VPERMILPDrm:
[X86][AVX512] Add support for VPERMILPD/VPERMILPS variable shuffle mask comments llvm-svn: 275272
2016-07-13 23:45:36 +08:00
case X86::VPERMILPDYrm:
case X86::VPERMILPDZ128rm:
[AVX-512] Add support for decoding shuffle mask from constant pool for masked VPERMILPS/PD. llvm-svn: 284450
2016-10-18 11:36:52 +08:00
case X86::VPERMILPDZ128rmk:
case X86::VPERMILPDZ128rmkz:
[X86][AVX512] Add support for VPERMILPD/VPERMILPS variable shuffle mask comments llvm-svn: 275272
2016-07-13 23:45:36 +08:00
case X86::VPERMILPDZ256rm:
[AVX-512] Add support for decoding shuffle mask from constant pool for masked VPERMILPS/PD. llvm-svn: 284450
2016-10-18 11:36:52 +08:00
case X86::VPERMILPDZ256rmk:
case X86::VPERMILPDZ256rmkz:
case X86::VPERMILPDZrm:
case X86::VPERMILPDZrmk:
case X86::VPERMILPDZrmkz: {
unsigned ElSize;
switch (MI->getOpcode()) {
[X86] Put some cases in switch statements back on one line to be more compact and make it easier to see the similarities. NFC It looks like someone ran clang-format over this entire file which reformatted these switches into a multiline form. But I think the single line form is more useful here. llvm-svn: 336077
2018-07-02 14:42:42 +08:00
default: llvm_unreachable("Invalid opcode");
[AVX-512] Add support for decoding shuffle mask from constant pool for masked VPERMILPS/PD. llvm-svn: 284450
2016-10-18 11:36:52 +08:00
case X86::VPERMILPSrm:
case X86::VPERMILPSYrm:
case X86::VPERMILPSZ128rm:
case X86::VPERMILPSZ256rm:
case X86::VPERMILPSZrm:
case X86::VPERMILPSZ128rmkz:
case X86::VPERMILPSZ256rmkz:
case X86::VPERMILPSZrmkz:
case X86::VPERMILPSZ128rmk:
case X86::VPERMILPSZ256rmk:
case X86::VPERMILPSZrmk:
[X86] Minor cleanups to addShuffleComments in X86MCInstPrinter.cpp. NFCI -Replace some ifs that should be impossible with asserts. -Use X86::AddrDisp and X86::AddrNumOperands to make code more readable -Use X86II::isKMasked/isKMergeMasked to do some operand skipping to remove or simplify switches
2020-05-31 03:53:56 +08:00
ElSize = 32;
break;
[AVX-512] Add support for decoding shuffle mask from constant pool for masked VPERMILPS/PD. llvm-svn: 284450
2016-10-18 11:36:52 +08:00
case X86::VPERMILPDrm:
case X86::VPERMILPDYrm:
case X86::VPERMILPDZ128rm:
case X86::VPERMILPDZ256rm:
case X86::VPERMILPDZrm:
case X86::VPERMILPDZ128rmkz:
case X86::VPERMILPDZ256rmkz:
case X86::VPERMILPDZrmkz:
case X86::VPERMILPDZ128rmk:
case X86::VPERMILPDZ256rmk:
case X86::VPERMILPDZrmk:
[X86] Minor cleanups to addShuffleComments in X86MCInstPrinter.cpp. NFCI -Replace some ifs that should be impossible with asserts. -Use X86::AddrDisp and X86::AddrNumOperands to make code more readable -Use X86II::isKMasked/isKMergeMasked to do some operand skipping to remove or simplify switches
2020-05-31 03:53:56 +08:00
ElSize = 64;
break;
}
unsigned SrcIdx = 1;
if (X86II::isKMasked(MI->getDesc().TSFlags)) {
// Skip mask operand.
++SrcIdx;
if (X86II::isKMergeMasked(MI->getDesc().TSFlags)) {
// Skip passthru operand.
++SrcIdx;
}
[X86] Fix shuffle decoding for variable VPERMIL to be tolerant of the Constant type not matching due to folding in the constant pool and to get VPERMILPD correct. llvm-svn: 256433
2015-12-26 12:50:07 +08:00
}
[X86] Minor cleanups to addShuffleComments in X86MCInstPrinter.cpp. NFCI -Replace some ifs that should be impossible with asserts. -Use X86::AddrDisp and X86::AddrNumOperands to make code more readable -Use X86II::isKMasked/isKMergeMasked to do some operand skipping to remove or simplify switches
2020-05-31 03:53:56 +08:00
unsigned MaskIdx = SrcIdx + 1 + X86::AddrDisp;
[X86][AVX512] Add support for VPERMILPD/VPERMILPS variable shuffle mask comments llvm-svn: 275272
2016-07-13 23:45:36 +08:00
[X86] Minor cleanups to addShuffleComments in X86MCInstPrinter.cpp. NFCI -Replace some ifs that should be impossible with asserts. -Use X86::AddrDisp and X86::AddrNumOperands to make code more readable -Use X86II::isKMasked/isKMergeMasked to do some operand skipping to remove or simplify switches
2020-05-31 03:53:56 +08:00
assert(MI->getNumOperands() >= (SrcIdx + 1 + X86::AddrNumOperands) &&
"Unexpected number of operands!");
[X86] Fix shuffle decoding for variable VPERMIL to be tolerant of the Constant type not matching due to folding in the constant pool and to get VPERMILPD correct. llvm-svn: 256433
2015-12-26 12:50:07 +08:00
[X86][AVX512] Add mask/maskz writemask support to constant pool shuffle decode commentx llvm-svn: 284488
2016-10-18 23:45:37 +08:00
const MachineOperand &MaskOp = MI->getOperand(MaskIdx);
[x86] Teach the instruction lowering to add comments describing constant pool data being loaded into a vector register. The comments take the form of: # ymm0 = [a,b,c,d,...] # xmm1 = <x,y,z...> The []s are used for generic sequential data and the <>s are used for specifically ConstantVector loads. Undef elements are printed as the letter 'u', integers in decimal, and floating point values as floating point values. Suggestions on improving the formatting or other aspects of the display are very welcome. My primary use case for this is to be able to FileCheck test masks passed to vector shuffle instructions in-register. It isn't fantastic for that (no decoding special zeroing semantics or other tricks), but it at least puts the mask onto an instruction line that could reasonably be checked. I've updated many of the new vector shuffle lowering tests to leverage this in their test cases so that we're actually checking the shuffle masks remain as expected. Before implementing this, I tried a *bunch* of different approaches. I looked into teaching the MCInstLower code to scan up the basic block and find a definition of a register used in a shuffle instruction and then decode that, but this seems incredibly brittle and complex. I talked to Hal a lot about the "right" way to do this: attach the raw shuffle mask to the instruction itself in some form of unencoded operands, and then use that to emit the comments. I still think that's the optimal solution here, but it proved to be beyond what I'm up for here. In particular, it seems likely best done by completing the plumbing of metadata through these layers and attaching the shuffle mask in metadata which could have fully automatic dropping when encoding an actual instruction. llvm-svn: 218377
2014-09-24 17:39:41 +08:00
if (auto *C = getConstantFromPool(*MI, MaskOp)) {
Recommit r344877 "[X86] Stop promoting integer loads to vXi64" I've included a fix to DAGCombiner::ForwardStoreValueToDirectLoad that I believe will prevent the previous miscompile. Original commit message: Theoretically this was done to simplify the amount of isel patterns that were needed. But it also meant a substantial number of our isel patterns have to match an explicit bitcast. By making the vXi32/vXi16/vXi8 types legal for loads, DAG combiner should be able to change the load type to rem I had to add some additional plain load instruction patterns and a few other special cases, but overall the isel table has reduced in size by ~12000 bytes. So it looks like this promotion was hurting us more than helping. I still have one crash in vector-trunc.ll that I'm hoping @RKSimon can help with. It seems to relate to using getTargetConstantFromNode on a load that was shrunk due to an extract_subvector combine after the constant pool entry was created. So we end up decoding more mask elements than the lo I'm hoping this patch will simplify the number of patterns needed to remove the and/or/xor promotion. Reviewers: RKSimon, spatel Reviewed By: RKSimon Subscribers: llvm-commits, RKSimon Differential Revision: https://reviews.llvm.org/D53306 llvm-svn: 344965
2018-10-23 06:14:05 +08:00
unsigned Width = getRegisterWidth(MI->getDesc().OpInfo[0]);
[x86] More refactoring of the shuffle comment emission. The previous attempt didn't work out so well. It looks like it will be much better for introducing extra logic to find a shuffle mask if the finding logic is totally separate. This also makes it easy to sink the opcode logic completely out of the routine so we don't re-dispatch across it. Still no functionality changed. llvm-svn: 218363
2014-09-24 11:06:37 +08:00
SmallVector<int, 16> Mask;
Recommit r344877 "[X86] Stop promoting integer loads to vXi64" I've included a fix to DAGCombiner::ForwardStoreValueToDirectLoad that I believe will prevent the previous miscompile. Original commit message: Theoretically this was done to simplify the amount of isel patterns that were needed. But it also meant a substantial number of our isel patterns have to match an explicit bitcast. By making the vXi32/vXi16/vXi8 types legal for loads, DAG combiner should be able to change the load type to rem I had to add some additional plain load instruction patterns and a few other special cases, but overall the isel table has reduced in size by ~12000 bytes. So it looks like this promotion was hurting us more than helping. I still have one crash in vector-trunc.ll that I'm hoping @RKSimon can help with. It seems to relate to using getTargetConstantFromNode on a load that was shrunk due to an extract_subvector combine after the constant pool entry was created. So we end up decoding more mask elements than the lo I'm hoping this patch will simplify the number of patterns needed to remove the and/or/xor promotion. Reviewers: RKSimon, spatel Reviewed By: RKSimon Subscribers: llvm-commits, RKSimon Differential Revision: https://reviews.llvm.org/D53306 llvm-svn: 344965
2018-10-23 06:14:05 +08:00
DecodeVPERMILPMask(C, ElSize, Width, Mask);
[x86] More refactoring of the shuffle comment emission. The previous attempt didn't work out so well. It looks like it will be much better for introducing extra logic to find a shuffle mask if the finding logic is totally separate. This also makes it easy to sink the opcode logic completely out of the routine so we don't re-dispatch across it. Still no functionality changed. llvm-svn: 218363
2014-09-24 11:06:37 +08:00
if (!Mask.empty())
[X86] Factor constant pool comment printing out of the switch in X86AsmPrinter::emitInstruction. NFC Pull the verbose asm check out of the cases and move it up to the call of the new function.
2020-05-30 15:31:21 +08:00
OutStreamer.AddComment(getShuffleComment(MI, SrcIdx, SrcIdx, Mask));
[X86][XOP] Support for VPPERM 2-input shuffle mask decoding This patch adds support for decoding XOP VPPERM instruction when it represents a basic shuffle. The mask decoding required the existing MCInstrLowering code to be updated to support binary shuffles - the implementation now matches what is done in X86InstrComments.cpp. Differential Revision: http://reviews.llvm.org/D18441 llvm-svn: 265874
2016-04-09 22:51:26 +08:00
}
break;
}
[X86][XOP] Added VPERMIL2PD/VPERMIL2PS shuffle mask comment decoding llvm-svn: 271809
2016-06-05 05:44:28 +08:00
case X86::VPERMIL2PDrm:
case X86::VPERMIL2PSrm:
[X86][XOP] Reduce the size of a multiclass by moving more stuff to parameters instead of doing 128-bit and 256-bit simultaneously. This requires some instructions to be renamed to move the Y earlier in the instruction name. The new names are more consistent with other instructions. llvm-svn: 295579
2017-02-19 06:53:43 +08:00
case X86::VPERMIL2PDYrm:
case X86::VPERMIL2PSYrm: {
[X86] Minor cleanups to addShuffleComments in X86MCInstPrinter.cpp. NFCI -Replace some ifs that should be impossible with asserts. -Use X86::AddrDisp and X86::AddrNumOperands to make code more readable -Use X86II::isKMasked/isKMergeMasked to do some operand skipping to remove or simplify switches
2020-05-31 03:53:56 +08:00
assert(MI->getNumOperands() >= (3 + X86::AddrNumOperands + 1) &&
"Unexpected number of operands!");
[X86][XOP] Added VPERMIL2PD/VPERMIL2PS shuffle mask comment decoding llvm-svn: 271809
2016-06-05 05:44:28 +08:00
[X86][AVX512] Add mask/maskz writemask support to constant pool shuffle decode commentx llvm-svn: 284488
2016-10-18 23:45:37 +08:00
const MachineOperand &CtrlOp = MI->getOperand(MI->getNumOperands() - 1);
[X86][XOP] Added VPERMIL2PD/VPERMIL2PS shuffle mask comment decoding llvm-svn: 271809
2016-06-05 05:44:28 +08:00
if (!CtrlOp.isImm())
break;
unsigned ElSize;
switch (MI->getOpcode()) {
[X86] Put some cases in switch statements back on one line to be more compact and make it easier to see the similarities. NFC It looks like someone ran clang-format over this entire file which reformatted these switches into a multiline form. But I think the single line form is more useful here. llvm-svn: 336077
2018-07-02 14:42:42 +08:00
default: llvm_unreachable("Invalid opcode");
case X86::VPERMIL2PSrm: case X86::VPERMIL2PSYrm: ElSize = 32; break;
case X86::VPERMIL2PDrm: case X86::VPERMIL2PDYrm: ElSize = 64; break;
[X86][XOP] Added VPERMIL2PD/VPERMIL2PS shuffle mask comment decoding llvm-svn: 271809
2016-06-05 05:44:28 +08:00
}
[X86] Minor cleanups to addShuffleComments in X86MCInstPrinter.cpp. NFCI -Replace some ifs that should be impossible with asserts. -Use X86::AddrDisp and X86::AddrNumOperands to make code more readable -Use X86II::isKMasked/isKMergeMasked to do some operand skipping to remove or simplify switches
2020-05-31 03:53:56 +08:00
const MachineOperand &MaskOp = MI->getOperand(3 + X86::AddrDisp);
[X86][XOP] Added VPERMIL2PD/VPERMIL2PS shuffle mask comment decoding llvm-svn: 271809
2016-06-05 05:44:28 +08:00
if (auto *C = getConstantFromPool(*MI, MaskOp)) {
Recommit r344877 "[X86] Stop promoting integer loads to vXi64" I've included a fix to DAGCombiner::ForwardStoreValueToDirectLoad that I believe will prevent the previous miscompile. Original commit message: Theoretically this was done to simplify the amount of isel patterns that were needed. But it also meant a substantial number of our isel patterns have to match an explicit bitcast. By making the vXi32/vXi16/vXi8 types legal for loads, DAG combiner should be able to change the load type to rem I had to add some additional plain load instruction patterns and a few other special cases, but overall the isel table has reduced in size by ~12000 bytes. So it looks like this promotion was hurting us more than helping. I still have one crash in vector-trunc.ll that I'm hoping @RKSimon can help with. It seems to relate to using getTargetConstantFromNode on a load that was shrunk due to an extract_subvector combine after the constant pool entry was created. So we end up decoding more mask elements than the lo I'm hoping this patch will simplify the number of patterns needed to remove the and/or/xor promotion. Reviewers: RKSimon, spatel Reviewed By: RKSimon Subscribers: llvm-commits, RKSimon Differential Revision: https://reviews.llvm.org/D53306 llvm-svn: 344965
2018-10-23 06:14:05 +08:00
unsigned Width = getRegisterWidth(MI->getDesc().OpInfo[0]);
[X86][XOP] Added VPERMIL2PD/VPERMIL2PS shuffle mask comment decoding llvm-svn: 271809
2016-06-05 05:44:28 +08:00
SmallVector<int, 16> Mask;
Recommit r344877 "[X86] Stop promoting integer loads to vXi64" I've included a fix to DAGCombiner::ForwardStoreValueToDirectLoad that I believe will prevent the previous miscompile. Original commit message: Theoretically this was done to simplify the amount of isel patterns that were needed. But it also meant a substantial number of our isel patterns have to match an explicit bitcast. By making the vXi32/vXi16/vXi8 types legal for loads, DAG combiner should be able to change the load type to rem I had to add some additional plain load instruction patterns and a few other special cases, but overall the isel table has reduced in size by ~12000 bytes. So it looks like this promotion was hurting us more than helping. I still have one crash in vector-trunc.ll that I'm hoping @RKSimon can help with. It seems to relate to using getTargetConstantFromNode on a load that was shrunk due to an extract_subvector combine after the constant pool entry was created. So we end up decoding more mask elements than the lo I'm hoping this patch will simplify the number of patterns needed to remove the and/or/xor promotion. Reviewers: RKSimon, spatel Reviewed By: RKSimon Subscribers: llvm-commits, RKSimon Differential Revision: https://reviews.llvm.org/D53306 llvm-svn: 344965
2018-10-23 06:14:05 +08:00
DecodeVPERMIL2PMask(C, (unsigned)CtrlOp.getImm(), ElSize, Width, Mask);
[X86][XOP] Added VPERMIL2PD/VPERMIL2PS shuffle mask comment decoding llvm-svn: 271809
2016-06-05 05:44:28 +08:00
if (!Mask.empty())
[X86] Factor constant pool comment printing out of the switch in X86AsmPrinter::emitInstruction. NFC Pull the verbose asm check out of the cases and move it up to the call of the new function.
2020-05-30 15:31:21 +08:00
OutStreamer.AddComment(getShuffleComment(MI, 1, 2, Mask));
[X86][XOP] Added VPERMIL2PD/VPERMIL2PS shuffle mask comment decoding llvm-svn: 271809
2016-06-05 05:44:28 +08:00
}
break;
}
[X86][XOP] Support for VPPERM 2-input shuffle mask decoding This patch adds support for decoding XOP VPPERM instruction when it represents a basic shuffle. The mask decoding required the existing MCInstrLowering code to be updated to support binary shuffles - the implementation now matches what is done in X86InstrComments.cpp. Differential Revision: http://reviews.llvm.org/D18441 llvm-svn: 265874
2016-04-09 22:51:26 +08:00
case X86::VPPERMrrm: {
[X86] Minor cleanups to addShuffleComments in X86MCInstPrinter.cpp. NFCI -Replace some ifs that should be impossible with asserts. -Use X86::AddrDisp and X86::AddrNumOperands to make code more readable -Use X86II::isKMasked/isKMergeMasked to do some operand skipping to remove or simplify switches
2020-05-31 03:53:56 +08:00
assert(MI->getNumOperands() >= (3 + X86::AddrNumOperands) &&
"Unexpected number of operands!");
[X86][XOP] Support for VPPERM 2-input shuffle mask decoding This patch adds support for decoding XOP VPPERM instruction when it represents a basic shuffle. The mask decoding required the existing MCInstrLowering code to be updated to support binary shuffles - the implementation now matches what is done in X86InstrComments.cpp. Differential Revision: http://reviews.llvm.org/D18441 llvm-svn: 265874
2016-04-09 22:51:26 +08:00
[X86] Minor cleanups to addShuffleComments in X86MCInstPrinter.cpp. NFCI -Replace some ifs that should be impossible with asserts. -Use X86::AddrDisp and X86::AddrNumOperands to make code more readable -Use X86II::isKMasked/isKMergeMasked to do some operand skipping to remove or simplify switches
2020-05-31 03:53:56 +08:00
const MachineOperand &MaskOp = MI->getOperand(3 + X86::AddrDisp);
[X86][XOP] Support for VPPERM 2-input shuffle mask decoding This patch adds support for decoding XOP VPPERM instruction when it represents a basic shuffle. The mask decoding required the existing MCInstrLowering code to be updated to support binary shuffles - the implementation now matches what is done in X86InstrComments.cpp. Differential Revision: http://reviews.llvm.org/D18441 llvm-svn: 265874
2016-04-09 22:51:26 +08:00
if (auto *C = getConstantFromPool(*MI, MaskOp)) {
Recommit r344877 "[X86] Stop promoting integer loads to vXi64" I've included a fix to DAGCombiner::ForwardStoreValueToDirectLoad that I believe will prevent the previous miscompile. Original commit message: Theoretically this was done to simplify the amount of isel patterns that were needed. But it also meant a substantial number of our isel patterns have to match an explicit bitcast. By making the vXi32/vXi16/vXi8 types legal for loads, DAG combiner should be able to change the load type to rem I had to add some additional plain load instruction patterns and a few other special cases, but overall the isel table has reduced in size by ~12000 bytes. So it looks like this promotion was hurting us more than helping. I still have one crash in vector-trunc.ll that I'm hoping @RKSimon can help with. It seems to relate to using getTargetConstantFromNode on a load that was shrunk due to an extract_subvector combine after the constant pool entry was created. So we end up decoding more mask elements than the lo I'm hoping this patch will simplify the number of patterns needed to remove the and/or/xor promotion. Reviewers: RKSimon, spatel Reviewed By: RKSimon Subscribers: llvm-commits, RKSimon Differential Revision: https://reviews.llvm.org/D53306 llvm-svn: 344965
2018-10-23 06:14:05 +08:00
unsigned Width = getRegisterWidth(MI->getDesc().OpInfo[0]);
[X86][XOP] Support for VPPERM 2-input shuffle mask decoding This patch adds support for decoding XOP VPPERM instruction when it represents a basic shuffle. The mask decoding required the existing MCInstrLowering code to be updated to support binary shuffles - the implementation now matches what is done in X86InstrComments.cpp. Differential Revision: http://reviews.llvm.org/D18441 llvm-svn: 265874
2016-04-09 22:51:26 +08:00
SmallVector<int, 16> Mask;
Recommit r344877 "[X86] Stop promoting integer loads to vXi64" I've included a fix to DAGCombiner::ForwardStoreValueToDirectLoad that I believe will prevent the previous miscompile. Original commit message: Theoretically this was done to simplify the amount of isel patterns that were needed. But it also meant a substantial number of our isel patterns have to match an explicit bitcast. By making the vXi32/vXi16/vXi8 types legal for loads, DAG combiner should be able to change the load type to rem I had to add some additional plain load instruction patterns and a few other special cases, but overall the isel table has reduced in size by ~12000 bytes. So it looks like this promotion was hurting us more than helping. I still have one crash in vector-trunc.ll that I'm hoping @RKSimon can help with. It seems to relate to using getTargetConstantFromNode on a load that was shrunk due to an extract_subvector combine after the constant pool entry was created. So we end up decoding more mask elements than the lo I'm hoping this patch will simplify the number of patterns needed to remove the and/or/xor promotion. Reviewers: RKSimon, spatel Reviewed By: RKSimon Subscribers: llvm-commits, RKSimon Differential Revision: https://reviews.llvm.org/D53306 llvm-svn: 344965
2018-10-23 06:14:05 +08:00
DecodeVPPERMMask(C, Width, Mask);
[X86][XOP] Support for VPPERM 2-input shuffle mask decoding This patch adds support for decoding XOP VPPERM instruction when it represents a basic shuffle. The mask decoding required the existing MCInstrLowering code to be updated to support binary shuffles - the implementation now matches what is done in X86InstrComments.cpp. Differential Revision: http://reviews.llvm.org/D18441 llvm-svn: 265874
2016-04-09 22:51:26 +08:00
if (!Mask.empty())
[X86] Factor constant pool comment printing out of the switch in X86AsmPrinter::emitInstruction. NFC Pull the verbose asm check out of the cases and move it up to the call of the new function.
2020-05-30 15:31:21 +08:00
OutStreamer.AddComment(getShuffleComment(MI, 1, 2, Mask));
[x86] More refactoring of the shuffle comment emission. The previous attempt didn't work out so well. It looks like it will be much better for introducing extra logic to find a shuffle mask if the finding logic is totally separate. This also makes it easy to sink the opcode logic completely out of the routine so we don't re-dispatch across it. Still no functionality changed. llvm-svn: 218363
2014-09-24 11:06:37 +08:00
}
[x86] Teach the X86 backend to print shuffle comments for PSHUFB instructions which happen to have a constant mask. Currently, this only handles a very narrow set of cases, but those happen to be the cases that I care about for testing shuffles sanely. This is a bit trickier than other shuffle instructions because we're decoding constants out of the constant pool. The current MC layer makes it completely impossible to inspect a constant pool entry, so we have to do it at the MI level and attach the comment to the streamer on its way out. So no joy for disassembling, but it does make test cases and asm dumps *much* nicer. Sorry for no test cases, but it didn't really seem that valuable to go trolling through existing old test cases and updating them. I'll have lots of testing of this in the upcoming patch for SSSE3 emission in the new vector shuffle lowering code paths. llvm-svn: 213986
2014-07-26 07:47:11 +08:00
break;
split MachineInstr -> MCInst lowering into its own class (not being embedded into X86ATTAsmPrinter). This still depends heavily on X86ATTAsmPrinter, but this is a step in the right direction. llvm-svn: 81627
2009-09-13 04:34:57 +08:00
}
[x86] Teach the instruction lowering to add comments describing constant pool data being loaded into a vector register. The comments take the form of: # ymm0 = [a,b,c,d,...] # xmm1 = <x,y,z...> The []s are used for generic sequential data and the <>s are used for specifically ConstantVector loads. Undef elements are printed as the letter 'u', integers in decimal, and floating point values as floating point values. Suggestions on improving the formatting or other aspects of the display are very welcome. My primary use case for this is to be able to FileCheck test masks passed to vector shuffle instructions in-register. It isn't fantastic for that (no decoding special zeroing semantics or other tricks), but it at least puts the mask onto an instruction line that could reasonably be checked. I've updated many of the new vector shuffle lowering tests to leverage this in their test cases so that we're actually checking the shuffle masks remain as expected. Before implementing this, I tried a *bunch* of different approaches. I looked into teaching the MCInstLower code to scan up the basic block and find a definition of a register used in a shuffle instruction and then decode that, but this seems incredibly brittle and complex. I talked to Hal a lot about the "right" way to do this: attach the raw shuffle mask to the instruction itself in some form of unencoded operands, and then use that to emit the comments. I still think that's the optimal solution here, but it proved to be beyond what I'm up for here. In particular, it seems likely best done by completing the plumbing of metadata through these layers and attaching the shuffle mask in metadata which could have fully automatic dropping when encoding an actual instruction. llvm-svn: 218377
2014-09-24 17:39:41 +08:00
[X86][MMX] Improve handling of 64-bit MMX constants 64-bit MMX constant generation usually ends up lowering into SSE instructions before being spilled/reloaded as a MMX type. This patch bitcasts the constant to a double value to allow correct loading directly to the MMX register. I've added MMX constant asm comment support to improve testing, it's better to always print the double values as hex constants as MMX is mainly an integer unit (and even with 3DNow! its just floats). Differential Revision: https://reviews.llvm.org/D43616 llvm-svn: 326497
2018-03-02 06:22:31 +08:00
case X86::MMX_MOVQ64rm: {
[X86] Minor cleanups to addShuffleComments in X86MCInstPrinter.cpp. NFCI -Replace some ifs that should be impossible with asserts. -Use X86::AddrDisp and X86::AddrNumOperands to make code more readable -Use X86II::isKMasked/isKMergeMasked to do some operand skipping to remove or simplify switches
2020-05-31 03:53:56 +08:00
assert(MI->getNumOperands() == (1 + X86::AddrNumOperands) &&
"Unexpected number of operands!");
if (auto *C = getConstantFromPool(*MI, MI->getOperand(1 + X86::AddrDisp))) {
[X86][MMX] Improve handling of 64-bit MMX constants 64-bit MMX constant generation usually ends up lowering into SSE instructions before being spilled/reloaded as a MMX type. This patch bitcasts the constant to a double value to allow correct loading directly to the MMX register. I've added MMX constant asm comment support to improve testing, it's better to always print the double values as hex constants as MMX is mainly an integer unit (and even with 3DNow! its just floats). Differential Revision: https://reviews.llvm.org/D43616 llvm-svn: 326497
2018-03-02 06:22:31 +08:00
std::string Comment;
raw_string_ostream CS(Comment);
const MachineOperand &DstOp = MI->getOperand(0);
CS << X86ATTInstPrinter::getRegisterName(DstOp.getReg()) << " = ";
if (auto *CF = dyn_cast<ConstantFP>(C)) {
CS << "0x" << CF->getValueAPF().bitcastToAPInt().toString(16, false);
[X86] Factor constant pool comment printing out of the switch in X86AsmPrinter::emitInstruction. NFC Pull the verbose asm check out of the cases and move it up to the call of the new function.
2020-05-30 15:31:21 +08:00
OutStreamer.AddComment(CS.str());
[X86][MMX] Improve handling of 64-bit MMX constants 64-bit MMX constant generation usually ends up lowering into SSE instructions before being spilled/reloaded as a MMX type. This patch bitcasts the constant to a double value to allow correct loading directly to the MMX register. I've added MMX constant asm comment support to improve testing, it's better to always print the double values as hex constants as MMX is mainly an integer unit (and even with 3DNow! its just floats). Differential Revision: https://reviews.llvm.org/D43616 llvm-svn: 326497
2018-03-02 06:22:31 +08:00
}
}
break;
}
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
#define MOV_CASE(Prefix, Suffix) \
case X86::Prefix##MOVAPD##Suffix##rm: \
case X86::Prefix##MOVAPS##Suffix##rm: \
case X86::Prefix##MOVUPD##Suffix##rm: \
case X86::Prefix##MOVUPS##Suffix##rm: \
case X86::Prefix##MOVDQA##Suffix##rm: \
AVX-512: Lowering for 512-bit vector shuffles. Vector types: <8 x 64>, <16 x 32>, <32 x 16> float and integer. Differential Revision: http://reviews.llvm.org/D10683 llvm-svn: 246981
2015-09-08 14:38:21 +08:00
case X86::Prefix##MOVDQU##Suffix##rm:
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
#define MOV_AVX512_CASE(Suffix) \
case X86::VMOVDQA64##Suffix##rm: \
case X86::VMOVDQA32##Suffix##rm: \
case X86::VMOVDQU64##Suffix##rm: \
case X86::VMOVDQU32##Suffix##rm: \
case X86::VMOVDQU16##Suffix##rm: \
case X86::VMOVDQU8##Suffix##rm: \
case X86::VMOVAPS##Suffix##rm: \
case X86::VMOVAPD##Suffix##rm: \
case X86::VMOVUPS##Suffix##rm: \
AVX-512: Lowering for 512-bit vector shuffles. Vector types: <8 x 64>, <16 x 32>, <32 x 16> float and integer. Differential Revision: http://reviews.llvm.org/D10683 llvm-svn: 246981
2015-09-08 14:38:21 +08:00
case X86::VMOVUPD##Suffix##rm:
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
#define CASE_ALL_MOV_RM() \
MOV_CASE(, ) /* SSE */ \
MOV_CASE(V, ) /* AVX-128 */ \
MOV_CASE(V, Y) /* AVX-256 */ \
MOV_AVX512_CASE(Z) \
MOV_AVX512_CASE(Z256) \
AVX-512: Lowering for 512-bit vector shuffles. Vector types: <8 x 64>, <16 x 32>, <32 x 16> float and integer. Differential Revision: http://reviews.llvm.org/D10683 llvm-svn: 246981
2015-09-08 14:38:21 +08:00
MOV_AVX512_CASE(Z128)
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
// For loads from a constant pool to a vector register, print the constant
// loaded.
CASE_ALL_MOV_RM()
[X86] Add comment string for broadcast loads from the constant pool. Summary: When broadcasting from the constant pool its useful to print out the final vector similar to what we do for normal moves from the constant pool. I changed only a couple tests that were broadcast focused. One of them had been previously hand tweaked after running the script so that it could check the constant pool declaration. But I think this patch makes that unnecessary now since we can check the comment instead. Reviewers: spatel, RKSimon, zvi Reviewed By: spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D34923 llvm-svn: 307062
2017-07-04 13:46:11 +08:00
case X86::VBROADCASTF128:
case X86::VBROADCASTI128:
case X86::VBROADCASTF32X4Z256rm:
case X86::VBROADCASTF32X4rm:
case X86::VBROADCASTF32X8rm:
case X86::VBROADCASTF64X2Z128rm:
case X86::VBROADCASTF64X2rm:
case X86::VBROADCASTF64X4rm:
case X86::VBROADCASTI32X4Z256rm:
case X86::VBROADCASTI32X4rm:
case X86::VBROADCASTI32X8rm:
case X86::VBROADCASTI64X2Z128rm:
case X86::VBROADCASTI64X2rm:
case X86::VBROADCASTI64X4rm:
[X86] Minor cleanups to addShuffleComments in X86MCInstPrinter.cpp. NFCI -Replace some ifs that should be impossible with asserts. -Use X86::AddrDisp and X86::AddrNumOperands to make code more readable -Use X86II::isKMasked/isKMergeMasked to do some operand skipping to remove or simplify switches
2020-05-31 03:53:56 +08:00
assert(MI->getNumOperands() >= (1 + X86::AddrNumOperands) &&
"Unexpected number of operands!");
if (auto *C = getConstantFromPool(*MI, MI->getOperand(1 + X86::AddrDisp))) {
[X86] Add comment string for broadcast loads from the constant pool. Summary: When broadcasting from the constant pool its useful to print out the final vector similar to what we do for normal moves from the constant pool. I changed only a couple tests that were broadcast focused. One of them had been previously hand tweaked after running the script so that it could check the constant pool declaration. But I think this patch makes that unnecessary now since we can check the comment instead. Reviewers: spatel, RKSimon, zvi Reviewed By: spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D34923 llvm-svn: 307062
2017-07-04 13:46:11 +08:00
int NumLanes = 1;
// Override NumLanes for the broadcast instructions.
switch (MI->getOpcode()) {
[X86] Put some cases in switch statements back on one line to be more compact and make it easier to see the similarities. NFC It looks like someone ran clang-format over this entire file which reformatted these switches into a multiline form. But I think the single line form is more useful here. llvm-svn: 336077
2018-07-02 14:42:42 +08:00
case X86::VBROADCASTF128: NumLanes = 2; break;
case X86::VBROADCASTI128: NumLanes = 2; break;
case X86::VBROADCASTF32X4Z256rm: NumLanes = 2; break;
case X86::VBROADCASTF32X4rm: NumLanes = 4; break;
case X86::VBROADCASTF32X8rm: NumLanes = 2; break;
case X86::VBROADCASTF64X2Z128rm: NumLanes = 2; break;
case X86::VBROADCASTF64X2rm: NumLanes = 4; break;
case X86::VBROADCASTF64X4rm: NumLanes = 2; break;
case X86::VBROADCASTI32X4Z256rm: NumLanes = 2; break;
case X86::VBROADCASTI32X4rm: NumLanes = 4; break;
case X86::VBROADCASTI32X8rm: NumLanes = 2; break;
case X86::VBROADCASTI64X2Z128rm: NumLanes = 2; break;
case X86::VBROADCASTI64X2rm: NumLanes = 4; break;
case X86::VBROADCASTI64X4rm: NumLanes = 2; break;
[X86] Add comment string for broadcast loads from the constant pool. Summary: When broadcasting from the constant pool its useful to print out the final vector similar to what we do for normal moves from the constant pool. I changed only a couple tests that were broadcast focused. One of them had been previously hand tweaked after running the script so that it could check the constant pool declaration. But I think this patch makes that unnecessary now since we can check the comment instead. Reviewers: spatel, RKSimon, zvi Reviewed By: spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D34923 llvm-svn: 307062
2017-07-04 13:46:11 +08:00
}
[x86] Teach the instruction lowering to add comments describing constant pool data being loaded into a vector register. The comments take the form of: # ymm0 = [a,b,c,d,...] # xmm1 = <x,y,z...> The []s are used for generic sequential data and the <>s are used for specifically ConstantVector loads. Undef elements are printed as the letter 'u', integers in decimal, and floating point values as floating point values. Suggestions on improving the formatting or other aspects of the display are very welcome. My primary use case for this is to be able to FileCheck test masks passed to vector shuffle instructions in-register. It isn't fantastic for that (no decoding special zeroing semantics or other tricks), but it at least puts the mask onto an instruction line that could reasonably be checked. I've updated many of the new vector shuffle lowering tests to leverage this in their test cases so that we're actually checking the shuffle masks remain as expected. Before implementing this, I tried a *bunch* of different approaches. I looked into teaching the MCInstLower code to scan up the basic block and find a definition of a register used in a shuffle instruction and then decode that, but this seems incredibly brittle and complex. I talked to Hal a lot about the "right" way to do this: attach the raw shuffle mask to the instruction itself in some form of unencoded operands, and then use that to emit the comments. I still think that's the optimal solution here, but it proved to be beyond what I'm up for here. In particular, it seems likely best done by completing the plumbing of metadata through these layers and attaching the shuffle mask in metadata which could have fully automatic dropping when encoding an actual instruction. llvm-svn: 218377
2014-09-24 17:39:41 +08:00
std::string Comment;
raw_string_ostream CS(Comment);
const MachineOperand &DstOp = MI->getOperand(0);
CS << X86ATTInstPrinter::getRegisterName(DstOp.getReg()) << " = ";
if (auto *CDS = dyn_cast<ConstantDataSequential>(C)) {
CS << "[";
[X86] Add comment string for broadcast loads from the constant pool. Summary: When broadcasting from the constant pool its useful to print out the final vector similar to what we do for normal moves from the constant pool. I changed only a couple tests that were broadcast focused. One of them had been previously hand tweaked after running the script so that it could check the constant pool declaration. But I think this patch makes that unnecessary now since we can check the comment instead. Reviewers: spatel, RKSimon, zvi Reviewed By: spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D34923 llvm-svn: 307062
2017-07-04 13:46:11 +08:00
for (int l = 0; l != NumLanes; ++l) {
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
for (int i = 0, NumElements = CDS->getNumElements(); i < NumElements;
++i) {
[X86] Add comment string for broadcast loads from the constant pool. Summary: When broadcasting from the constant pool its useful to print out the final vector similar to what we do for normal moves from the constant pool. I changed only a couple tests that were broadcast focused. One of them had been previously hand tweaked after running the script so that it could check the constant pool declaration. But I think this patch makes that unnecessary now since we can check the comment instead. Reviewers: spatel, RKSimon, zvi Reviewed By: spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D34923 llvm-svn: 307062
2017-07-04 13:46:11 +08:00
if (i != 0 || l != 0)
CS << ",";
if (CDS->getElementType()->isIntegerTy())
[X86] Add APInt constant assembly printer helper llvm-svn: 343577
2018-10-02 19:32:33 +08:00
printConstant(CDS->getElementAsAPInt(i), CS);
[X86] Standardize floating point assembly comments Consistently try to use APFloat::toString for floating point constant comments to get rid of differences between Constant / ConstantDataSequential values - it should help stop some of the linux-windows buildbot failures matching NaN/INF etc. as well. Differential Revision: https://reviews.llvm.org/D52702 llvm-svn: 343562
2018-10-02 17:08:51 +08:00
else if (CDS->getElementType()->isHalfTy() ||
CDS->getElementType()->isFloatTy() ||
CDS->getElementType()->isDoubleTy())
printConstant(CDS->getElementAsAPFloat(i), CS);
[X86] Add comment string for broadcast loads from the constant pool. Summary: When broadcasting from the constant pool its useful to print out the final vector similar to what we do for normal moves from the constant pool. I changed only a couple tests that were broadcast focused. One of them had been previously hand tweaked after running the script so that it could check the constant pool declaration. But I think this patch makes that unnecessary now since we can check the comment instead. Reviewers: spatel, RKSimon, zvi Reviewed By: spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D34923 llvm-svn: 307062
2017-07-04 13:46:11 +08:00
else
CS << "?";
}
[x86] Teach the instruction lowering to add comments describing constant pool data being loaded into a vector register. The comments take the form of: # ymm0 = [a,b,c,d,...] # xmm1 = <x,y,z...> The []s are used for generic sequential data and the <>s are used for specifically ConstantVector loads. Undef elements are printed as the letter 'u', integers in decimal, and floating point values as floating point values. Suggestions on improving the formatting or other aspects of the display are very welcome. My primary use case for this is to be able to FileCheck test masks passed to vector shuffle instructions in-register. It isn't fantastic for that (no decoding special zeroing semantics or other tricks), but it at least puts the mask onto an instruction line that could reasonably be checked. I've updated many of the new vector shuffle lowering tests to leverage this in their test cases so that we're actually checking the shuffle masks remain as expected. Before implementing this, I tried a *bunch* of different approaches. I looked into teaching the MCInstLower code to scan up the basic block and find a definition of a register used in a shuffle instruction and then decode that, but this seems incredibly brittle and complex. I talked to Hal a lot about the "right" way to do this: attach the raw shuffle mask to the instruction itself in some form of unencoded operands, and then use that to emit the comments. I still think that's the optimal solution here, but it proved to be beyond what I'm up for here. In particular, it seems likely best done by completing the plumbing of metadata through these layers and attaching the shuffle mask in metadata which could have fully automatic dropping when encoding an actual instruction. llvm-svn: 218377
2014-09-24 17:39:41 +08:00
}
CS << "]";
[X86] Factor constant pool comment printing out of the switch in X86AsmPrinter::emitInstruction. NFC Pull the verbose asm check out of the cases and move it up to the call of the new function.
2020-05-30 15:31:21 +08:00
OutStreamer.AddComment(CS.str());
[x86] Teach the instruction lowering to add comments describing constant pool data being loaded into a vector register. The comments take the form of: # ymm0 = [a,b,c,d,...] # xmm1 = <x,y,z...> The []s are used for generic sequential data and the <>s are used for specifically ConstantVector loads. Undef elements are printed as the letter 'u', integers in decimal, and floating point values as floating point values. Suggestions on improving the formatting or other aspects of the display are very welcome. My primary use case for this is to be able to FileCheck test masks passed to vector shuffle instructions in-register. It isn't fantastic for that (no decoding special zeroing semantics or other tricks), but it at least puts the mask onto an instruction line that could reasonably be checked. I've updated many of the new vector shuffle lowering tests to leverage this in their test cases so that we're actually checking the shuffle masks remain as expected. Before implementing this, I tried a *bunch* of different approaches. I looked into teaching the MCInstLower code to scan up the basic block and find a definition of a register used in a shuffle instruction and then decode that, but this seems incredibly brittle and complex. I talked to Hal a lot about the "right" way to do this: attach the raw shuffle mask to the instruction itself in some form of unencoded operands, and then use that to emit the comments. I still think that's the optimal solution here, but it proved to be beyond what I'm up for here. In particular, it seems likely best done by completing the plumbing of metadata through these layers and attaching the shuffle mask in metadata which could have fully automatic dropping when encoding an actual instruction. llvm-svn: 218377
2014-09-24 17:39:41 +08:00
} else if (auto *CV = dyn_cast<ConstantVector>(C)) {
CS << "<";
[X86] Add comment string for broadcast loads from the constant pool. Summary: When broadcasting from the constant pool its useful to print out the final vector similar to what we do for normal moves from the constant pool. I changed only a couple tests that were broadcast focused. One of them had been previously hand tweaked after running the script so that it could check the constant pool declaration. But I think this patch makes that unnecessary now since we can check the comment instead. Reviewers: spatel, RKSimon, zvi Reviewed By: spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D34923 llvm-svn: 307062
2017-07-04 13:46:11 +08:00
for (int l = 0; l != NumLanes; ++l) {
[XRay] Typed event logging intrinsic Summary: Add an LLVM intrinsic for type discriminated event logging with XRay. Similar to the existing intrinsic for custom events, but also accepts a type tag argument to allow plugins to be aware of different types and semantically interpret logged events they know about without choking on those they don't. Relies on a symbol defined in compiler-rt patch D43668. I may wait to submit before I can see demo everything working together including a still to come clang patch. Reviewers: dberris, pelikan, eizan, rSerge, timshen Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45633 llvm-svn: 330219
2018-04-18 05:30:29 +08:00
for (int i = 0, NumOperands = CV->getNumOperands(); i < NumOperands;
++i) {
[X86] Add comment string for broadcast loads from the constant pool. Summary: When broadcasting from the constant pool its useful to print out the final vector similar to what we do for normal moves from the constant pool. I changed only a couple tests that were broadcast focused. One of them had been previously hand tweaked after running the script so that it could check the constant pool declaration. But I think this patch makes that unnecessary now since we can check the comment instead. Reviewers: spatel, RKSimon, zvi Reviewed By: spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D34923 llvm-svn: 307062
2017-07-04 13:46:11 +08:00
if (i != 0 || l != 0)
CS << ",";
printConstant(CV->getOperand(i), CS);
[x86] Teach the instruction lowering to add comments describing constant pool data being loaded into a vector register. The comments take the form of: # ymm0 = [a,b,c,d,...] # xmm1 = <x,y,z...> The []s are used for generic sequential data and the <>s are used for specifically ConstantVector loads. Undef elements are printed as the letter 'u', integers in decimal, and floating point values as floating point values. Suggestions on improving the formatting or other aspects of the display are very welcome. My primary use case for this is to be able to FileCheck test masks passed to vector shuffle instructions in-register. It isn't fantastic for that (no decoding special zeroing semantics or other tricks), but it at least puts the mask onto an instruction line that could reasonably be checked. I've updated many of the new vector shuffle lowering tests to leverage this in their test cases so that we're actually checking the shuffle masks remain as expected. Before implementing this, I tried a *bunch* of different approaches. I looked into teaching the MCInstLower code to scan up the basic block and find a definition of a register used in a shuffle instruction and then decode that, but this seems incredibly brittle and complex. I talked to Hal a lot about the "right" way to do this: attach the raw shuffle mask to the instruction itself in some form of unencoded operands, and then use that to emit the comments. I still think that's the optimal solution here, but it proved to be beyond what I'm up for here. In particular, it seems likely best done by completing the plumbing of metadata through these layers and attaching the shuffle mask in metadata which could have fully automatic dropping when encoding an actual instruction. llvm-svn: 218377
2014-09-24 17:39:41 +08:00
}
}
CS << ">";
[X86] Factor constant pool comment printing out of the switch in X86AsmPrinter::emitInstruction. NFC Pull the verbose asm check out of the cases and move it up to the call of the new function.
2020-05-30 15:31:21 +08:00
OutStreamer.AddComment(CS.str());
[x86] Teach the instruction lowering to add comments describing constant pool data being loaded into a vector register. The comments take the form of: # ymm0 = [a,b,c,d,...] # xmm1 = <x,y,z...> The []s are used for generic sequential data and the <>s are used for specifically ConstantVector loads. Undef elements are printed as the letter 'u', integers in decimal, and floating point values as floating point values. Suggestions on improving the formatting or other aspects of the display are very welcome. My primary use case for this is to be able to FileCheck test masks passed to vector shuffle instructions in-register. It isn't fantastic for that (no decoding special zeroing semantics or other tricks), but it at least puts the mask onto an instruction line that could reasonably be checked. I've updated many of the new vector shuffle lowering tests to leverage this in their test cases so that we're actually checking the shuffle masks remain as expected. Before implementing this, I tried a *bunch* of different approaches. I looked into teaching the MCInstLower code to scan up the basic block and find a definition of a register used in a shuffle instruction and then decode that, but this seems incredibly brittle and complex. I talked to Hal a lot about the "right" way to do this: attach the raw shuffle mask to the instruction itself in some form of unencoded operands, and then use that to emit the comments. I still think that's the optimal solution here, but it proved to be beyond what I'm up for here. In particular, it seems likely best done by completing the plumbing of metadata through these layers and attaching the shuffle mask in metadata which could have fully automatic dropping when encoding an actual instruction. llvm-svn: 218377
2014-09-24 17:39:41 +08:00
}
}
break;
[X86] Factor constant pool comment printing out of the switch in X86AsmPrinter::emitInstruction. NFC Pull the verbose asm check out of the cases and move it up to the call of the new function.
2020-05-30 15:31:21 +08:00
[X86] Add 128 MOVDDUP to the constant pool printing in X86AsmPrinter::EmitInstruction. We use this instruction to broadcast a single 64-bit value to a v2i64/v2f64 vector. llvm-svn: 344486
2018-10-15 09:51:53 +08:00
case X86::MOVDDUPrm:
case X86::VMOVDDUPrm:
case X86::VMOVDDUPZ128rm:
[X86] Add comment string for broadcast loads from the constant pool. Summary: When broadcasting from the constant pool its useful to print out the final vector similar to what we do for normal moves from the constant pool. I changed only a couple tests that were broadcast focused. One of them had been previously hand tweaked after running the script so that it could check the constant pool declaration. But I think this patch makes that unnecessary now since we can check the comment instead. Reviewers: spatel, RKSimon, zvi Reviewed By: spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D34923 llvm-svn: 307062
2017-07-04 13:46:11 +08:00
case X86::VBROADCASTSSrm:
case X86::VBROADCASTSSYrm:
[X86] Standardize BROADCAST enum names (PR31079) Tweak EVEX implementation names so it matches the other variants by adding the 'r' prefix. Oddly some of the subvec broadcast ops already matched.
2020-02-09 00:54:46 +08:00
case X86::VBROADCASTSSZ128rm:
case X86::VBROADCASTSSZ256rm:
case X86::VBROADCASTSSZrm:
[X86] Add comment string for broadcast loads from the constant pool. Summary: When broadcasting from the constant pool its useful to print out the final vector similar to what we do for normal moves from the constant pool. I changed only a couple tests that were broadcast focused. One of them had been previously hand tweaked after running the script so that it could check the constant pool declaration. But I think this patch makes that unnecessary now since we can check the comment instead. Reviewers: spatel, RKSimon, zvi Reviewed By: spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D34923 llvm-svn: 307062
2017-07-04 13:46:11 +08:00
case X86::VBROADCASTSDYrm:
[X86] Standardize BROADCAST enum names (PR31079) Tweak EVEX implementation names so it matches the other variants by adding the 'r' prefix. Oddly some of the subvec broadcast ops already matched.
2020-02-09 00:54:46 +08:00
case X86::VBROADCASTSDZ256rm:
case X86::VBROADCASTSDZrm:
[X86] Add comment string for broadcast loads from the constant pool. Summary: When broadcasting from the constant pool its useful to print out the final vector similar to what we do for normal moves from the constant pool. I changed only a couple tests that were broadcast focused. One of them had been previously hand tweaked after running the script so that it could check the constant pool declaration. But I think this patch makes that unnecessary now since we can check the comment instead. Reviewers: spatel, RKSimon, zvi Reviewed By: spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D34923 llvm-svn: 307062
2017-07-04 13:46:11 +08:00
case X86::VPBROADCASTBrm:
case X86::VPBROADCASTBYrm:
[X86] Standardize BROADCAST enum names (PR31079) Tweak EVEX implementation names so it matches the other variants by adding the 'r' prefix. Oddly some of the subvec broadcast ops already matched.
2020-02-09 00:54:46 +08:00
case X86::VPBROADCASTBZ128rm:
case X86::VPBROADCASTBZ256rm:
case X86::VPBROADCASTBZrm:
[X86] Add comment string for broadcast loads from the constant pool. Summary: When broadcasting from the constant pool its useful to print out the final vector similar to what we do for normal moves from the constant pool. I changed only a couple tests that were broadcast focused. One of them had been previously hand tweaked after running the script so that it could check the constant pool declaration. But I think this patch makes that unnecessary now since we can check the comment instead. Reviewers: spatel, RKSimon, zvi Reviewed By: spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D34923 llvm-svn: 307062
2017-07-04 13:46:11 +08:00
case X86::VPBROADCASTDrm:
case X86::VPBROADCASTDYrm:
[X86] Standardize BROADCAST enum names (PR31079) Tweak EVEX implementation names so it matches the other variants by adding the 'r' prefix. Oddly some of the subvec broadcast ops already matched.
2020-02-09 00:54:46 +08:00
case X86::VPBROADCASTDZ128rm:
case X86::VPBROADCASTDZ256rm:
case X86::VPBROADCASTDZrm:
[X86] Add comment string for broadcast loads from the constant pool. Summary: When broadcasting from the constant pool its useful to print out the final vector similar to what we do for normal moves from the constant pool. I changed only a couple tests that were broadcast focused. One of them had been previously hand tweaked after running the script so that it could check the constant pool declaration. But I think this patch makes that unnecessary now since we can check the comment instead. Reviewers: spatel, RKSimon, zvi Reviewed By: spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D34923 llvm-svn: 307062
2017-07-04 13:46:11 +08:00
case X86::VPBROADCASTQrm:
case X86::VPBROADCASTQYrm:
[X86] Standardize BROADCAST enum names (PR31079) Tweak EVEX implementation names so it matches the other variants by adding the 'r' prefix. Oddly some of the subvec broadcast ops already matched.
2020-02-09 00:54:46 +08:00
case X86::VPBROADCASTQZ128rm:
case X86::VPBROADCASTQZ256rm:
case X86::VPBROADCASTQZrm:
[X86] Add comment string for broadcast loads from the constant pool. Summary: When broadcasting from the constant pool its useful to print out the final vector similar to what we do for normal moves from the constant pool. I changed only a couple tests that were broadcast focused. One of them had been previously hand tweaked after running the script so that it could check the constant pool declaration. But I think this patch makes that unnecessary now since we can check the comment instead. Reviewers: spatel, RKSimon, zvi Reviewed By: spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D34923 llvm-svn: 307062
2017-07-04 13:46:11 +08:00
case X86::VPBROADCASTWrm:
case X86::VPBROADCASTWYrm:
[X86] Standardize BROADCAST enum names (PR31079) Tweak EVEX implementation names so it matches the other variants by adding the 'r' prefix. Oddly some of the subvec broadcast ops already matched.
2020-02-09 00:54:46 +08:00
case X86::VPBROADCASTWZ128rm:
case X86::VPBROADCASTWZ256rm:
case X86::VPBROADCASTWZrm:
[X86] Minor cleanups to addShuffleComments in X86MCInstPrinter.cpp. NFCI -Replace some ifs that should be impossible with asserts. -Use X86::AddrDisp and X86::AddrNumOperands to make code more readable -Use X86II::isKMasked/isKMergeMasked to do some operand skipping to remove or simplify switches
2020-05-31 03:53:56 +08:00
assert(MI->getNumOperands() >= (1 + X86::AddrNumOperands) &&
"Unexpected number of operands!");
if (auto *C = getConstantFromPool(*MI, MI->getOperand(1 + X86::AddrDisp))) {
[X86] Add comment string for broadcast loads from the constant pool. Summary: When broadcasting from the constant pool its useful to print out the final vector similar to what we do for normal moves from the constant pool. I changed only a couple tests that were broadcast focused. One of them had been previously hand tweaked after running the script so that it could check the constant pool declaration. But I think this patch makes that unnecessary now since we can check the comment instead. Reviewers: spatel, RKSimon, zvi Reviewed By: spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D34923 llvm-svn: 307062
2017-07-04 13:46:11 +08:00
int NumElts;
switch (MI->getOpcode()) {
[X86] Put some cases in switch statements back on one line to be more compact and make it easier to see the similarities. NFC It looks like someone ran clang-format over this entire file which reformatted these switches into a multiline form. But I think the single line form is more useful here. llvm-svn: 336077
2018-07-02 14:42:42 +08:00
default: llvm_unreachable("Invalid opcode");
[X86] Standardize BROADCAST enum names (PR31079) Tweak EVEX implementation names so it matches the other variants by adding the 'r' prefix. Oddly some of the subvec broadcast ops already matched.
2020-02-09 00:54:46 +08:00
case X86::MOVDDUPrm: NumElts = 2; break;
case X86::VMOVDDUPrm: NumElts = 2; break;
case X86::VMOVDDUPZ128rm: NumElts = 2; break;
case X86::VBROADCASTSSrm: NumElts = 4; break;
case X86::VBROADCASTSSYrm: NumElts = 8; break;
case X86::VBROADCASTSSZ128rm: NumElts = 4; break;
case X86::VBROADCASTSSZ256rm: NumElts = 8; break;
case X86::VBROADCASTSSZrm: NumElts = 16; break;
case X86::VBROADCASTSDYrm: NumElts = 4; break;
case X86::VBROADCASTSDZ256rm: NumElts = 4; break;
case X86::VBROADCASTSDZrm: NumElts = 8; break;
case X86::VPBROADCASTBrm: NumElts = 16; break;
case X86::VPBROADCASTBYrm: NumElts = 32; break;
case X86::VPBROADCASTBZ128rm: NumElts = 16; break;
case X86::VPBROADCASTBZ256rm: NumElts = 32; break;
case X86::VPBROADCASTBZrm: NumElts = 64; break;
case X86::VPBROADCASTDrm: NumElts = 4; break;
case X86::VPBROADCASTDYrm: NumElts = 8; break;
case X86::VPBROADCASTDZ128rm: NumElts = 4; break;
case X86::VPBROADCASTDZ256rm: NumElts = 8; break;
case X86::VPBROADCASTDZrm: NumElts = 16; break;
case X86::VPBROADCASTQrm: NumElts = 2; break;
case X86::VPBROADCASTQYrm: NumElts = 4; break;
case X86::VPBROADCASTQZ128rm: NumElts = 2; break;
case X86::VPBROADCASTQZ256rm: NumElts = 4; break;
case X86::VPBROADCASTQZrm: NumElts = 8; break;
case X86::VPBROADCASTWrm: NumElts = 8; break;
case X86::VPBROADCASTWYrm: NumElts = 16; break;
case X86::VPBROADCASTWZ128rm: NumElts = 8; break;
case X86::VPBROADCASTWZ256rm: NumElts = 16; break;
case X86::VPBROADCASTWZrm: NumElts = 32; break;
[X86] Add comment string for broadcast loads from the constant pool. Summary: When broadcasting from the constant pool its useful to print out the final vector similar to what we do for normal moves from the constant pool. I changed only a couple tests that were broadcast focused. One of them had been previously hand tweaked after running the script so that it could check the constant pool declaration. But I think this patch makes that unnecessary now since we can check the comment instead. Reviewers: spatel, RKSimon, zvi Reviewed By: spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D34923 llvm-svn: 307062
2017-07-04 13:46:11 +08:00
}
std::string Comment;
raw_string_ostream CS(Comment);
const MachineOperand &DstOp = MI->getOperand(0);
CS << X86ATTInstPrinter::getRegisterName(DstOp.getReg()) << " = ";
CS << "[";
for (int i = 0; i != NumElts; ++i) {
if (i != 0)
CS << ",";
printConstant(C, CS);
}
CS << "]";
[X86] Factor constant pool comment printing out of the switch in X86AsmPrinter::emitInstruction. NFC Pull the verbose asm check out of the cases and move it up to the call of the new function.
2020-05-30 15:31:21 +08:00
OutStreamer.AddComment(CS.str());
}
}
}
void X86AsmPrinter::emitInstruction(const MachineInstr *MI) {
X86MCInstLower MCInstLowering(*MF, *this);
const X86RegisterInfo *RI =
MF->getSubtarget<X86Subtarget>().getRegisterInfo();
// Add a comment about EVEX-2-VEX compression for AVX-512 instrs that
// are compressed from EVEX encoding to VEX encoding.
if (TM.Options.MCOptions.ShowMCEncoding) {
if (MI->getAsmPrinterFlags() & X86::AC_EVEX_2_VEX)
OutStreamer->AddComment("EVEX TO VEX Compression ", false);
}
// Add comments for values loaded from constant pool.
if (OutStreamer->isVerboseAsm())
addConstantComments(MI, *OutStreamer);
switch (MI->getOpcode()) {
case TargetOpcode::DBG_VALUE:
llvm_unreachable("Should be handled target independently");
// Emit nothing here but a comment if we can.
case X86::Int_MemBarrier:
OutStreamer->emitRawComment("MEMBARRIER");
return;
case X86::EH_RETURN:
case X86::EH_RETURN64: {
// Lower these as normal, but add some comments.
Register Reg = MI->getOperand(0).getReg();
OutStreamer->AddComment(StringRef("eh_return, addr: %") +
X86ATTInstPrinter::getRegisterName(Reg));
break;
}
case X86::CLEANUPRET: {
// Lower these as normal, but add some comments.
OutStreamer->AddComment("CLEANUPRET");
break;
}
case X86::CATCHRET: {
// Lower these as normal, but add some comments.
OutStreamer->AddComment("CATCHRET");
break;
}
case X86::ENDBR32:
case X86::ENDBR64: {
// CurrentPatchableFunctionEntrySym can be CurrentFnBegin only for
// -fpatchable-function-entry=N,0. The entry MBB is guaranteed to be
// non-empty. If MI is the initial ENDBR, place the
// __patchable_function_entries label after ENDBR.
if (CurrentPatchableFunctionEntrySym &&
CurrentPatchableFunctionEntrySym == CurrentFnBegin &&
MI == &MF->front().front()) {
MCInst Inst;
MCInstLowering.Lower(MI, Inst);
EmitAndCountInstruction(Inst);
CurrentPatchableFunctionEntrySym = createTempSymbol("patch");
OutStreamer->emitLabel(CurrentPatchableFunctionEntrySym);
return;
}
break;
}
case X86::TAILJMPr:
case X86::TAILJMPm:
case X86::TAILJMPd:
case X86::TAILJMPd_CC:
case X86::TAILJMPr64:
case X86::TAILJMPm64:
case X86::TAILJMPd64:
case X86::TAILJMPd64_CC:
case X86::TAILJMPr64_REX:
case X86::TAILJMPm64_REX:
// Lower these as normal, but add some comments.
OutStreamer->AddComment("TAILCALL");
break;
case X86::TLS_addr32:
case X86::TLS_addr64:
[X86] Add TLS_(base_)addrX32 for X32 mode LLVM has TLS_(base_)addr32 for 32-bit TLS addresses in 32-bit mode, and TLS_(base_)addr64 for 64-bit TLS addresses in 64-bit mode. x32 mode wants 32-bit TLS addresses in 64-bit mode, which were not yet handled. This adds TLS_(base_)addrX32 as copies of TLS_(base_)addr64, except that they use tls32(base)addr rather than tls64(base)addr, and then restricts TLS_(base_)addr64 to 64-bit LP64 mode, TLS_(base_)addrX32 to 64-bit ILP32 mode. Reviewed By: RKSimon Differential Revision: https://reviews.llvm.org/D92346
2020-12-03 06:20:36 +08:00
case X86::TLS_addrX32:
[X86] Factor constant pool comment printing out of the switch in X86AsmPrinter::emitInstruction. NFC Pull the verbose asm check out of the cases and move it up to the call of the new function.
2020-05-30 15:31:21 +08:00
case X86::TLS_base_addr32:
case X86::TLS_base_addr64:
[X86] Add TLS_(base_)addrX32 for X32 mode LLVM has TLS_(base_)addr32 for 32-bit TLS addresses in 32-bit mode, and TLS_(base_)addr64 for 64-bit TLS addresses in 64-bit mode. x32 mode wants 32-bit TLS addresses in 64-bit mode, which were not yet handled. This adds TLS_(base_)addrX32 as copies of TLS_(base_)addr64, except that they use tls32(base)addr rather than tls64(base)addr, and then restricts TLS_(base_)addr64 to 64-bit LP64 mode, TLS_(base_)addrX32 to 64-bit ILP32 mode. Reviewed By: RKSimon Differential Revision: https://reviews.llvm.org/D92346
2020-12-03 06:20:36 +08:00
case X86::TLS_base_addrX32:
[X86] Factor constant pool comment printing out of the switch in X86AsmPrinter::emitInstruction. NFC Pull the verbose asm check out of the cases and move it up to the call of the new function.
2020-05-30 15:31:21 +08:00
return LowerTlsAddr(MCInstLowering, *MI);
case X86::MOVPC32r: {
// This is a pseudo op for a two instruction sequence with a label, which
// looks like:
// call "L1$pb"
// "L1$pb":
// popl %esi
// Emit the call.
MCSymbol *PICBase = MF->getPICBaseSymbol();
// FIXME: We would like an efficient form for this, so we don't have to do a
// lot of extra uniquing.
EmitAndCountInstruction(
MCInstBuilder(X86::CALLpcrel32)
.addExpr(MCSymbolRefExpr::create(PICBase, OutContext)));
const X86FrameLowering *FrameLowering =
MF->getSubtarget<X86Subtarget>().getFrameLowering();
bool hasFP = FrameLowering->hasFP(*MF);
// TODO: This is needed only if we require precise CFA.
bool HasActiveDwarfFrame = OutStreamer->getNumFrameInfos() &&
!OutStreamer->getDwarfFrameInfos().back().End;
int stackGrowth = -RI->getSlotSize();
if (HasActiveDwarfFrame && !hasFP) {
OutStreamer->emitCFIAdjustCfaOffset(-stackGrowth);
}
// Emit the label.
OutStreamer->emitLabel(PICBase);
// popl $reg
EmitAndCountInstruction(
MCInstBuilder(X86::POP32r).addReg(MI->getOperand(0).getReg()));
if (HasActiveDwarfFrame && !hasFP) {
OutStreamer->emitCFIAdjustCfaOffset(stackGrowth);
}
return;
}
case X86::ADD32ri: {
// Lower the MO_GOT_ABSOLUTE_ADDRESS form of ADD32ri.
if (MI->getOperand(2).getTargetFlags() != X86II::MO_GOT_ABSOLUTE_ADDRESS)
break;
// Okay, we have something like:
// EAX = ADD32ri EAX, MO_GOT_ABSOLUTE_ADDRESS(@MYGLOBAL)
// For this, we want to print something like:
// MYGLOBAL + (. - PICBASE)
// However, we can't generate a ".", so just emit a new label here and refer
// to it.
MCSymbol *DotSym = OutContext.createTempSymbol();
OutStreamer->emitLabel(DotSym);
// Now that we have emitted the label, lower the complex operand expression.
MCSymbol *OpSym = MCInstLowering.GetSymbolFromOperand(MI->getOperand(2));
const MCExpr *DotExpr = MCSymbolRefExpr::create(DotSym, OutContext);
const MCExpr *PICBase =
MCSymbolRefExpr::create(MF->getPICBaseSymbol(), OutContext);
DotExpr = MCBinaryExpr::createSub(DotExpr, PICBase, OutContext);
DotExpr = MCBinaryExpr::createAdd(
MCSymbolRefExpr::create(OpSym, OutContext), DotExpr, OutContext);
EmitAndCountInstruction(MCInstBuilder(X86::ADD32ri)
.addReg(MI->getOperand(0).getReg())
.addReg(MI->getOperand(1).getReg())
.addExpr(DotExpr));
return;
}
case TargetOpcode::STATEPOINT:
return LowerSTATEPOINT(*MI, MCInstLowering);
case TargetOpcode::FAULTING_OP:
return LowerFAULTING_OP(*MI, MCInstLowering);
case TargetOpcode::FENTRY_CALL:
return LowerFENTRY_CALL(*MI, MCInstLowering);
case TargetOpcode::PATCHABLE_OP:
return LowerPATCHABLE_OP(*MI, MCInstLowering);
case TargetOpcode::STACKMAP:
return LowerSTACKMAP(*MI);
case TargetOpcode::PATCHPOINT:
return LowerPATCHPOINT(*MI, MCInstLowering);
case TargetOpcode::PATCHABLE_FUNCTION_ENTER:
return LowerPATCHABLE_FUNCTION_ENTER(*MI, MCInstLowering);
case TargetOpcode::PATCHABLE_RET:
return LowerPATCHABLE_RET(*MI, MCInstLowering);
case TargetOpcode::PATCHABLE_TAIL_CALL:
return LowerPATCHABLE_TAIL_CALL(*MI, MCInstLowering);
case TargetOpcode::PATCHABLE_EVENT_CALL:
return LowerPATCHABLE_EVENT_CALL(*MI, MCInstLowering);
case TargetOpcode::PATCHABLE_TYPED_EVENT_CALL:
return LowerPATCHABLE_TYPED_EVENT_CALL(*MI, MCInstLowering);
case X86::MORESTACK_RET:
EmitAndCountInstruction(MCInstBuilder(getRetOpcode(*Subtarget)));
return;
case X86::MORESTACK_RET_RESTORE_R10:
// Return, then restore R10.
EmitAndCountInstruction(MCInstBuilder(getRetOpcode(*Subtarget)));
EmitAndCountInstruction(
MCInstBuilder(X86::MOV64rr).addReg(X86::R10).addReg(X86::RAX));
return;
case X86::SEH_PushReg:
case X86::SEH_SaveReg:
case X86::SEH_SaveXMM:
case X86::SEH_StackAlloc:
case X86::SEH_StackAlign:
case X86::SEH_SetFrame:
case X86::SEH_PushFrame:
case X86::SEH_EndPrologue:
EmitSEHInstruction(MI);
return;
case X86::SEH_Epilogue: {
assert(MF->hasWinCFI() && "SEH_ instruction in function without WinCFI?");
MachineBasicBlock::const_iterator MBBI(MI);
// Check if preceded by a call and emit nop if so.
for (MBBI = PrevCrossBBInst(MBBI);
MBBI != MachineBasicBlock::const_iterator();
MBBI = PrevCrossBBInst(MBBI)) {
// Conservatively assume that pseudo instructions don't emit code and keep
// looking for a call. We may emit an unnecessary nop in some cases.
if (!MBBI->isPseudo()) {
if (MBBI->isCall())
EmitAndCountInstruction(MCInstBuilder(X86::NOOP));
break;
}
[X86] Add comment string for broadcast loads from the constant pool. Summary: When broadcasting from the constant pool its useful to print out the final vector similar to what we do for normal moves from the constant pool. I changed only a couple tests that were broadcast focused. One of them had been previously hand tweaked after running the script so that it could check the constant pool declaration. But I think this patch makes that unnecessary now since we can check the comment instead. Reviewers: spatel, RKSimon, zvi Reviewed By: spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D34923 llvm-svn: 307062
2017-07-04 13:46:11 +08:00
}
[X86] Factor constant pool comment printing out of the switch in X86AsmPrinter::emitInstruction. NFC Pull the verbose asm check out of the cases and move it up to the call of the new function.
2020-05-30 15:31:21 +08:00
return;
}
UBSAN: emit distinctive traps Sometimes people get minimal crash reports after a UBSAN incident. This change tags each trap with an integer representing the kind of failure encountered, which can aid in tracking down the root cause of the problem.
2020-10-21 17:11:25 +08:00
case X86::UBSAN_UD1:
EmitAndCountInstruction(MCInstBuilder(X86::UD1Lm)
.addReg(X86::EAX)
.addReg(X86::EAX)
.addImm(1)
.addReg(X86::NoRegister)
.addImm(MI->getOperand(0).getImm())
.addReg(X86::NoRegister));
return;
[x86] Start refactoring the comment printing logic in the MC lowering of vector shuffles. This is just the beginning by hoisting it into its own function and making use of early exit to dramatically simplify the flow of the function. I'm going to be incrementally refactoring this until it is a bit less magical how this applies to other instructions, and I can teach it how to dig a shuffle mask out of a register. Then I plan to hook it up to VPERMD so we get our mask comments for it. No functionality changed yet. llvm-svn: 218357
2014-09-24 10:16:12 +08:00
}
Whitespace. llvm-svn: 161122
2012-08-02 02:39:17 +08:00
split MachineInstr -> MCInst lowering into its own class (not being embedded into X86ATTAsmPrinter). This still depends heavily on X86ATTAsmPrinter, but this is a step in the right direction. llvm-svn: 81627
2009-09-13 04:34:57 +08:00
MCInst TmpInst;
MCInstLowering.Lower(MI, TmpInst);
Stackmap shadows should consider call returns a branch target. To avoid emitting too many nops, a stackmap shadow can include emitted instructions in the shadow, but these must not include branch targets. A return from a call should count as a branch target as patching over the instructions after the call would lead to incorrect behaviour for threads currently making that call, when they return. llvm-svn: 220710
2014-10-28 03:40:35 +08:00
// Stackmap shadows cannot include branch targets, so we can count the bytes
Fix a stackmap bug introduced in r220710. For a call to not return in to the stackmap shadow, the shadow must end with the call. To do this, we must insert any required nops *before* the call, and not after it. llvm-svn: 220728
2014-10-28 06:38:45 +08:00
// in a call towards the shadow, but must ensure that the no thread returns
// in to the stackmap shadow. The only way to achieve this is if the call
// is at the end of the shadow.
if (MI->isCall()) {
// Count then size of the call towards the shadow
Introduce a "patchable-function" function attribute Summary: The `"patchable-function"` attribute can be used by an LLVM client to influence LLVM's code generation in ways that makes the generated code easily patchable at runtime (for instance, to redirect control). Right now only one patchability scheme is supported, `"prologue-short-redirect"`, but this can be expanded in the future. Reviewers: joker.eph, rnk, echristo, dberris Subscribers: joker.eph, echristo, mcrosier, llvm-commits Differential Revision: http://reviews.llvm.org/D19046 llvm-svn: 266715
2016-04-19 13:24:47 +08:00
SMShadowTracker.count(TmpInst, getSubtargetInfo(), CodeEmitter.get());
Fix a stackmap bug introduced in r220710. For a call to not return in to the stackmap shadow, the shadow must end with the call. To do this, we must insert any required nops *before* the call, and not after it. llvm-svn: 220728
2014-10-28 06:38:45 +08:00
// Then flush the shadow so that we fill with nops before the call, not
// after it.
[AsmPrinter] Make AsmPrinter's OutStreamer member a unique_ptr. AsmPrinter owns the OutStreamer, so an owning pointer makes sense here. Using a reference for this is crufty. llvm-svn: 235752
2015-04-25 03:11:51 +08:00
SMShadowTracker.emitShadowPadding(*OutStreamer, getSubtargetInfo());
Fix a stackmap bug introduced in r220710. For a call to not return in to the stackmap shadow, the shadow must end with the call. To do this, we must insert any required nops *before* the call, and not after it. llvm-svn: 220728
2014-10-28 06:38:45 +08:00
// Then emit the call
[AsmPrinter][MCStreamer] De-capitalize EmitInstruction and EmitCFI*
2020-02-14 13:58:16 +08:00
OutStreamer->emitInstruction(TmpInst, getSubtargetInfo());
Fix a stackmap bug introduced in r220710. For a call to not return in to the stackmap shadow, the shadow must end with the call. To do this, we must insert any required nops *before* the call, and not after it. llvm-svn: 220728
2014-10-28 06:38:45 +08:00
return;
}
EmitAndCountInstruction(TmpInst);
split mcinst lowering stuff out to its own file. llvm-svn: 80806
2009-09-03 01:35:12 +08:00
}