For another 1% speedup on the testcase in PR23865, push the
`ModuleSlotTracker` through to metadata-related printing in
`MachineBasicBlock::print()`.
llvm-svn: 240848
Push `ModuleSlotTracker` through `MachineOperand`s, dropping the time
for `llc -print-machineinstrs` on the testcase in PR23865 from ~13
seconds to ~9 seconds. Now `SlotTracker::processFunctionMetadata()`
accounts for only 8% of the runtime, which seems reasonable.
llvm-svn: 240845
Expose enough of the IR-level `SlotTracker` so that
`MachineFunction::print()` can use a single one for printing
`BasicBlock`s. Next step would be to lift this through a few more APIs
so that we can make other print methods faster.
Fixes PR23865, changing the runtime of `llc -print-machineinstrs` from
many minutes (killed after 3 minutes, but it wasn't very close) to
13 seconds for a 502185 line dump.
llvm-svn: 240842
The body of the loops here only contained asserts. This triggered an unused variable
warning on release builds and -Werror on the bots.
llvm-svn: 240819
This commit serializes machine basic block operands. The
machine basic block operands use the following syntax:
%bb.<id>[.<name>]
This commit also modifies the YAML representation for the
machine basic blocks - a new, required field 'id' is added
to the MBB YAML mapping.
The id is used to resolve the MBB references to the
actual MBBs. And while the name of the MBB can be
included in a MBB reference, this name isn't used to
resolve MBB references - as it's possible that multiple
MBBs will reference the same BB and thus they will have the
same name. If the name is specified, the parser will verify
that it is equal to the name of the MBB with the specified id.
Reviewers: Duncan P. N. Exon Smith
Differential Revision: http://reviews.llvm.org/D10608
llvm-svn: 240792
E.g. An interleaved load (Factor = 2):
%wide.vec = load <8 x i32>, <8 x i32>* %ptr
%v0 = shuffle <8 x i32> %wide.vec, <8 x i32> undef, <0, 2, 4, 6>
%v1 = shuffle <8 x i32> %wide.vec, <8 x i32> undef, <1, 3, 5, 7>
It can be transformed into a ld2 intrinsic in AArch64 backend or a vld2 intrinsic in ARM backend.
E.g. An interleaved store (Factor = 3):
%i.vec = shuffle <8 x i32> %v0, <8 x i32> %v1, <0, 4, 8, 1, 5, 9, 2, 6, 10, 3, 7, 11>
store <12 x i32> %i.vec, <12 x i32>* %ptr
It can be transformed into a st3 intrinsic in AArch64 backend or a vst3 intrinsic in ARM backend.
Differential Revision: http://reviews.llvm.org/D10533
llvm-svn: 240751
Replace the `std::vector<>` for `DIE::Children` with an intrusively
linked list. This is a strict memory improvement: it requires no
auxiliary storage, and reduces `sizeof(DIE)` by one pointer. It also
factors out the DIE-related malloc traffic.
This drops llc memory usage from 735 MB down to 718 MB, or ~2.3%.
(I'm looking at `llc` memory usage on `verify-uselistorder.lto.opt.bc`;
see r236629 for details.)
llvm-svn: 240736
Change `DIE::Values` to a singly linked list, where each node is
allocated on a `BumpPtrAllocator`. In order to support `push_back()`,
the list is circular, and points at the tail element instead of the
head. I abstracted the core list logic out to `IntrusiveBackList` so
that it can be reused for `DIE::Children`, which also cares about
`push_back()`.
This drops llc memory usage from 799 MB down to 735 MB, about 8%.
(I'm looking at `llc` memory usage on `verify-uselistorder.lto.opt.bc`;
see r236629 for details.)
llvm-svn: 240733
Check for symbols in MCValue before using them. Bail out early in case
they are null. This fixes PR23779.
Differential Revision: http://reviews.llvm.org/D10712
rdar://problem/21532830
llvm-svn: 240649
This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead
of recursion. Previously, this function would get called recursively a large
number of times and eventually segfault when a function with the following CFG
was compiled:
BB0:
if (condition0)
goto BB1
goto BB2
BB1:
goto BB2
BB2:
if (condition1)
goto BB3
goto BB4
BB3:
...
(repeat until BB7488)
rdar://problem/21386145
Differential Revision: http://reviews.llvm.org/D10587
llvm-svn: 240589
This commit serializes the 3 scalar boolean attributes from the
MachineRegisterInfo class: IsSSA, TracksRegLiveness, and
TracksSubRegLiveness. These attributes are serialized as part
of the machine function YAML mapping.
Reviewers: Duncan P. N. Exon Smith
Differential Revision: http://reviews.llvm.org/D10618
llvm-svn: 240579
Stop taking a `dwarf::Form` in `DIEValue::EmitValue()` and
`DIEValue::SizeOf()`, since they're always passed `DIEValue::getForm()`
anyway. This is just left over from when `DIEValue` didn't know its own
form.
llvm-svn: 240566
This commit serializes the null register machine operands.
It uses the '_' keyword to represent them, but the parser
also allows the '%noreg' named register syntax.
Reviewers: Duncan P. N. Exon Smith
Differential Revision: http://reviews.llvm.org/D10580
llvm-svn: 240558
Summary:
This patch fixes PR23405 (https://llvm.org/bugs/show_bug.cgi?id=23405).
During a node unscheduling an entry in LiveRegGens can be replaced with a new value. That corrupts the live reg tracking and LiveReg* structure is not cleared as should be during unscheduling. Problematic condition that enforces Gen replacement is `I->getSUnit()->getHeight() < LiveRegGens[I->getReg()]->getHeight()`. This condition should be checked only if LiveRegGen was set in current node unscheduling.
Test Plan: Regression test included.
Reviewers: hfinkel, atrick
Reviewed By: atrick
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D9993
llvm-svn: 240538
This commit translates the source locations for MIParser diagnostics from
the locations in the machine instruction string to the locations in the
MIR file.
Reviewers: Duncan P. N. Exon Smith
Differential Revision: http://reviews.llvm.org/D10574
llvm-svn: 240474
This commit introduces functionality that's used to serialize machine operands.
Only the physical register operands are serialized by this commit.
Reviewers: Duncan P. N. Exon Smith
Differential Revision: http://reviews.llvm.org/D10525
llvm-svn: 240425
The summary is that it moves the mangling earlier and replaces a few
calls to .addExternalSymbol with addSym.
I originally wanted to replace all the uses of addExternalSymbol with
addSym, but noticed it was a lot of work and doesn't need to be done
all at once.
llvm-svn: 240395
Summary:
That way llvm-objdump can rely on it without adding an extra dependency
on CodeGen.
This change duplicates the FaultKind enum and the code that serializes
it to a string. I could not figure out a way to get around this without
adding a new dependency to Object
Reviewers: rafael, ab
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D10619
llvm-svn: 240364
Currently ( D10321, http://reviews.llvm.org/rL239486 ), we can use the machine combiner pass
to reassociate the following sequence to reduce the critical path:
A = ? op ?
B = A op X
C = B op Y
-->
A = ? op ?
B = X op Y
C = A op B
'op' is currently limited to x86 AVX scalar FP adds (with fast-math on), but in theory, it could
be any associative math/logic op (see TODO in code comment).
This patch generalizes the pattern match to ignore the instruction that defines 'A'. So instead of
a sequence of 3 adds, we now only need to find 2 dependent adds and decide if it's worth
reassociating them.
This generalization has a compile-time cost because we can now match more instruction sequences
and we rely more heavily on the machine combiner to discard sequences where reassociation doesn't
improve the critical path.
For example, in the new test case:
A = M div N
B = A add X
C = B add Y
We'll match 2 reassociation patterns, but this transform doesn't reduce the critical path:
A = M div N
B = A add Y
C = B add X
We need the combiner to reject that pattern but select this:
A = M div N
B = X add Y
C = B add A
Differential Revision: http://reviews.llvm.org/D10460
llvm-svn: 240361
This commit adds a function that tokenizes the string containing
the machine instruction. This commit also adds a struct called
'MIToken' which is used to represent the lexer's tokens.
Reviewers: Sean Silva
Differential Revision: http://reviews.llvm.org/D10521
llvm-svn: 240323
I had some unnecessary `typename`s left in after addressing review.
This compiled successfully with clang++ but MSVC reported an error. Fix
the build error by removing the redundant `typename`s.
llvm-svn: 240307
Summary:
The parser is exercised by llvm-objdump using -print-fault-maps. As is
probably obvious, the code itself was "heavily inspired" by
http://reviews.llvm.org/D10434.
Reviewers: reames, atrick, JosephTremoulet
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D10491
llvm-svn: 240304
Before this we were producing a TargetExternalSymbol from a MCSymbol.
That meant extracting the symbol name and fetching the symbol again
down the pipeline.
This patch adds a DAG.getMCSymbol that lets the MCSymbol pass unchanged on the
DAG.
Doing so removes the need for MO_NOPREFIX and fixes the root cause of pr23900,
allowing r240130 to be committed again.
llvm-svn: 240300
This commit implements initial machine instruction serialization. It
serializes machine instruction names. The instructions are represented
using a YAML sequence of string literals and are a part of machine
basic block YAML mapping.
This commit introduces a class called 'MIParser' which will be used to
parse the machine instructions and operands.
Reviewers: Duncan P. N. Exon Smith
Differential Revision: http://reviews.llvm.org/D10481
llvm-svn: 240295
Summary: The code responsible for shl folding in the DAGCombiner was assuming incorrectly that all constants are less than 64 bits. This patch simply changes the way values are compared.
Test Plan: A regression test included.
Reviewers: andreadb
Reviewed By: andreadb
Subscribers: andreadb, test, llvm-commits
Differential Revision: http://reviews.llvm.org/D10602
llvm-svn: 240291
This will allow classes to implement the AA interface without deriving
from the class or referencing an internal enum of some other class as
their return types.
Also, to a pretty fundamental extent, concepts such as 'NoAlias',
'MayAlias', and 'MustAlias' are first class concepts in LLVM and we
aren't saving anything by scoping them heavily.
My mild preference would have been to use a scoped enum, but that
feature is essentially completely broken AFAICT. I'm extremely
disappointed. For example, we cannot through any reasonable[1] means
construct an enum class (or analog) which has scoped names but converts
to a boolean in order to test for the possibility of aliasing.
[1]: Richard Smith came up with a "solution", but it requires class
templates, and lots of boilerplate setting up the enumeration multiple
times. Something like Boost.PP could potentially bundle this up, but
even that would be quite painful and it doesn't seem realistically worth
it. The enum class solution would probably work without the need for
a bool conversion.
Differential Revision: http://reviews.llvm.org/D10495
llvm-svn: 240255
If we don't know how to represent a .debug_loc entry, skip the entry
entirely rather than emitting an empty one. Similarly, if a .debug_loc
list has no entries, don't create the list.
We still want to create the variables, just in an optimized-out form
that doesn't have a DW_AT_location.
llvm-svn: 240244
There are three types of `DbgVariable`:
- alloca variables, created based on the MMI table,
- register variables, created based on DBG_VALUE instructions, and
- optimized-out variables.
This commit reconfigures `DbgVariable` to make it easier to tell which
kind we have, and make initialization a little clearer.
For MMI/alloca variables, `FrameIndex.size()` must always equal
`Expr.size()`, and there shouldn't be an `MInsn`. For register
variables (with a `MInsn`), `FrameIndex` must be empty, and `Expr`
should have 0 or 1 element depending on whether it has a complex
expression (registers with multiple locations use `DebugLocListIndex`).
Optimized-out variables shouldn't have any of these fields.
Moreover, this separates DBG_VALUE initialization until after the
variable is created, simplifying logic in a future commit that changes
`collectVariableInfo()` to stop creating empty .debug_loc entries/lists.
llvm-svn: 240243
Sparse switches with profile info are lowered as weight-balanced BSTs. For
example, if the node weights are {1,1,1,1,1,1000}, the right-most node would
end up in a tree by itself, bringing it closer to the top.
However, a leaf in this BST can contain up to 3 cases, and having a single
case in a leaf node as in the example means the tree might become
unnecessarily high.
This patch adds a heauristic to the pivot selection algorithm that moves more
cases into leaf nodes unless that would lower their rank. It still doesn't
yield the optimal tree in every case, but I believe it's conservatibely correct.
llvm-svn: 240224
This commit implements the initial serialization of machine basic blocks in a
machine function. Only the simple, scalar MBB attributes are serialized. The
reference to LLVM IR's basic block is preserved when that basic block has a name.
Reviewers: Duncan P. N. Exon Smith
Differential Revision: http://reviews.llvm.org/D10465
llvm-svn: 240145
The patch is generated using this command:
tools/clang/tools/extra/clang-tidy/tool/run-clang-tidy.py -fix \
-checks=-*,llvm-namespace-comment -header-filter='llvm/.*|clang/.*' \
llvm/lib/
Thanks to Eugene Kosov for the original patch!
llvm-svn: 240137
To same compile time, the analysis to find dense case-clusters in switches is
not done at -O0. However, when the whole switch is dense enough, it is easy to
turn it into a jump table, resulting in much faster code with no extra effort.
llvm-svn: 240071
- zext the value to alloc size first, then check if the value repeats
with zero padding included. If so we can still emit a .space
- Do the checking with APInt.isSplat(8), which handles non-pow2 types
- Also handle large constants (bit width > 64)
- In a ConstantArray all elements have the same type, so it's sufficient
to check the first constant recursively and then just compare if all
following constants are the same by pointer compare
llvm-svn: 239977
The personality routine currently lives in the LandingPadInst.
This isn't desirable because:
- All LandingPadInsts in the same function must have the same
personality routine. This means that each LandingPadInst beyond the
first has an operand which produces no additional information.
- There is ongoing work to introduce EH IR constructs other than
LandingPadInst. Moving the personality routine off of any one
particular Instruction and onto the parent function seems a lot better
than have N different places a personality function can sneak onto an
exceptional function.
Differential Revision: http://reviews.llvm.org/D10429
llvm-svn: 239940
It's been used before to avoid infinite loops caused by separate CGP
optimizations undoing one another. We found one more such issue
caused by r238054. To avoid it, generalize the "InsertedTruncs"
set to any inst, and use it to avoid touching those again.
llvm-svn: 239938
that it is its own entity in the form of MemoryLocation, and update all
the callers.
This is an entirely mechanical change. References to "Location" within
AA subclases become "MemoryLocation", and elsewhere
"AliasAnalysis::Location" becomes "MemoryLocation". Hope that helps
out-of-tree folks update.
llvm-svn: 239885
Different object formats represent references from dwarf in different ways.
ELF uses a relocation to the referenced point (except for .dwo) and
COFF/MachO use the offset of the referenced point inside its section.
This patch renames emitSectionOffset because
* It doesn't produce an offset on ELF.
* It changes behavior depending on how DWARF is represented, so adding
dwarf to its name is probably a good thing.
The patch also adds an option to force the use of offsets.That avoids
funny looking code like
if (!UseOffsets)
Asm->emitSectionOffset....
It was correct, but read as if the ! was inverted.
llvm-svn: 239866
While completely undefined registers are easy to catch and get their
<undef> flag early in ProcessImplicitDefs/RegisterCoalescer reading from
a partially defined register where just the subreg happens to be
undefined is harder to catch so we only add the undef flag in the
virtual register rewriting step.
No testcase as I cannot reproduce the problem on any of the in-tree targets at
the moment.
This fixes rdar://21387089
Differential Revision: http://reviews.llvm.org/D10470
llvm-svn: 239838
LaneMasks as given by getSubRegIndexLaneMask() have a limited number of
of bits, so for targets with more than 31 disjunct subregister there may
be cases where:
getSubReg(Reg,A) does not overlap getSubReg(Reg,B)
but we still have
(getSubRegIndexLaneMask(A) & getSubRegIndexLaneMask(B)) != 0.
I had hoped to keep this an implementation detail of the tablegen but as
my next commit shows we can avoid unnecessary imp-defs operands if we
know that the lane masks in use are precise.
This is in preparation to http://reviews.llvm.org/D10470.
llvm-svn: 239837
This commit reports an error when a machine function from a MIR file that contains
LLVM IR can't find a function with the same name in the loaded LLVM IR module.
Reviewers: Duncan P. N. Exon Smith
Differential Revision: http://reviews.llvm.org/D10468
llvm-svn: 239831
This is an updated version of the patch that was checked in at:
http://reviews.llvm.org/rL237046
but subsequently reverted because it exposed a bug in the DAG Combiner:
http://reviews.llvm.org/D9893
This time, there's an enablement flag ("EnableFMFInDAG") around the code in
SelectionDAGBuilder where we copy the set of FP optimization flags from IR
instructions to DAG nodes. So, in theory, there should be no functional change
from this patch as-is, but it will allow testing with the added functionality
to proceed via "-enable-fmf-dag" passed to llc.
This patch adds the minimum plumbing necessary to use IR-level
fast-math-flags (FMF) in the backend without actually using
them for anything yet. This is a follow-on to:
http://reviews.llvm.org/rL235997
Differential Revision: http://reviews.llvm.org/D10403
llvm-svn: 239828
Reapply r239539. Don't assume the collected number of
stores is the same vector size. Just take the first N
stores to fill the vector.
llvm-svn: 239825
Summary:
This continues the patch series to eliminate StringRef forms of GNU triples
from the internals of LLVM that began in r239036.
Reviewers: rengolin
Reviewed By: rengolin
Subscribers: llvm-commits, rengolin
Differential Revision: http://reviews.llvm.org/D10381
llvm-svn: 239815
This commit serializes the simple, scalar attributes from the
'MachineFunction' class.
Reviewers: Duncan P. N. Exon Smith
Differential Revision: http://reviews.llvm.org/D10449
llvm-svn: 239790
This commit decouples the MIR printer and the MIR printing pass so
that it will be possible to move the MIR printer into a separate
machine IR library later on.
Reviewers: Duncan P. N. Exon Smith
llvm-svn: 239788
This commit creates a dummy LLVM IR function with one basic block and an unreachable
instruction for each parsed machine function when the MIR file doesn't have LLVM IR.
This change is required as the machine function analysis pass creates machine
functions only for the functions that are defined in the current LLVM module.
Reviewers: Duncan P. N. Exon Smith
Differential Revision: http://reviews.llvm.org/D10135
llvm-svn: 239778
This commit reports an error when the MIR parser encounters a machine
function with the name that is the same as the name of a different
machine function.
Reviewers: Duncan P. N. Exon Smith
Differential Revision: http://reviews.llvm.org/D10130
llvm-svn: 239774
This patch adds the safe stack instrumentation pass to LLVM, which separates
the program stack into a safe stack, which stores return addresses, register
spills, and local variables that are statically verified to be accessed
in a safe way, and the unsafe stack, which stores everything else. Such
separation makes it much harder for an attacker to corrupt objects on the
safe stack, including function pointers stored in spilled registers and
return addresses. You can find more information about the safe stack, as
well as other parts of or control-flow hijack protection technique in our
OSDI paper on code-pointer integrity (http://dslab.epfl.ch/pubs/cpi.pdf)
and our project website (http://levee.epfl.ch).
The overhead of our implementation of the safe stack is very close to zero
(0.01% on the Phoronix benchmarks). This is lower than the overhead of
stack cookies, which are supported by LLVM and are commonly used today,
yet the security guarantees of the safe stack are strictly stronger than
stack cookies. In some cases, the safe stack improves performance due to
better cache locality.
Our current implementation of the safe stack is stable and robust, we
used it to recompile multiple projects on Linux including Chromium, and
we also recompiled the entire FreeBSD user-space system and more than 100
packages. We ran unit tests on the FreeBSD system and many of the packages
and observed no errors caused by the safe stack. The safe stack is also fully
binary compatible with non-instrumented code and can be applied to parts of
a program selectively.
This patch is our implementation of the safe stack on top of LLVM. The
patches make the following changes:
- Add the safestack function attribute, similar to the ssp, sspstrong and
sspreq attributes.
- Add the SafeStack instrumentation pass that applies the safe stack to all
functions that have the safestack attribute. This pass moves all unsafe local
variables to the unsafe stack with a separate stack pointer, whereas all
safe variables remain on the regular stack that is managed by LLVM as usual.
- Invoke the pass as the last stage before code generation (at the same time
the existing cookie-based stack protector pass is invoked).
- Add unit tests for the safe stack.
Original patch by Volodymyr Kuznetsov and others at the Dependable Systems
Lab at EPFL; updates and upstreaming by myself.
Differential Revision: http://reviews.llvm.org/D6094
llvm-svn: 239761
This commit connects the machine function analysis pass (which creates machine
functions) to the MIR parser, which will initialize the machine functions
with the state from the MIR file and reconstruct the machine IR.
This commit introduces a new interface called 'MachineFunctionInitializer',
which can be used to provide custom initialization for the machine functions.
This commit also introduces a new diagnostic class called
'DiagnosticInfoMIRParser' which is used for MIR parsing errors.
This commit modifies the default diagnostic handling in LLVMContext - now the
the diagnostics are printed directly into llvm::errs() so that the MIR parsing
errors can be printed with colours.
Reviewers: Justin Bogner
Differential Revision: http://reviews.llvm.org/D9928
llvm-svn: 239753
Summary:
TargetInstrInfo::getLdStBaseRegImmOfs to
TargetInstrInfo::getMemOpBaseRegImmOfs and implement for x86. The
implementation only handles a few easy cases now and will be made more
sophisticated in the future.
This is NFCI: the only user of `getLdStBaseRegImmOfs` (now
`getmemOpBaseRegImmOfs`) is `LoadClusterMotion` and `LoadClusterMotion`
is disabled for x86.
Reviewers: reames, ab, MatzeB, atrick
Reviewed By: MatzeB, atrick
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D10199
llvm-svn: 239741
Summary:
This instruction encodes a loading operation that may fault, and a label
to branch to if the load page-faults. The locations of potentially
faulting loads and their "handler" destinations are recorded in a
FaultMap section, meant to be consumed by LLVM's clients.
Nothing generates FAULTING_LOAD_OP instructions yet, but they will be
used in a future change.
The documentation (FaultMaps.rst) needs improvement and I will update
this diff with a more expanded version shortly.
Depends on D10196
Reviewers: rnk, reames, AndyAyers, ab, atrick, pgavlin
Reviewed By: atrick, pgavlin
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D10197
llvm-svn: 239740
This patch fixes a compilation time issue, when MachineSink faces PHIs
with a huge number of operands. This can happen for example in goto table
based interpreters, where some basic blocks can have several of those PHIs,
each one with several hundreds operands. MachineSink was spending a
significant time re-building and re-sorting the list of successors of
the current MachineBasicBlock. The computing and sorting of the current
MachineBasicBlock successors is now cached.
llvm-svn: 239720
r213101 changed the behaviour of this method to not only affect the
PostMachineScheduler scheduler but also the PostRAScheduler scheduler,
renaming should make this fact clear. Also document that the preferred
way is to specify this in the scheduling model instead of overriding
this method.
Differential Revision: http://reviews.llvm.org/D10427
llvm-svn: 239659
This will use Itinieraries if available, but will also work if just a
MCSchedModel is available.
Differential Revision: http://reviews.llvm.org/D10428
llvm-svn: 239658
We were putting them in the filter field, which is correct for 64-bit
but wrong for 32-bit.
Also switch the order of scope table entry emission so outermost entries
are emitted first, and fix an obvious state assignment bug.
llvm-svn: 239574
This intrinsic is like framerecover plus a load. It recovers the EH
registration stack allocation from the parent frame and loads the
exception information field out of it, giving back a pointer to an
EXCEPTION_POINTERS struct. It's designed for clang to use in SEH filter
expressions instead of accessing the EXCEPTION_POINTERS parameter that
is available on x64.
This required a minor change to MC to allow defining a label variable to
another absolute framerecover label variable.
llvm-svn: 239567
Summary:
For the moment, TargetMachine::getTargetTriple() still returns a StringRef.
This continues the patch series to eliminate StringRef forms of GNU triples
from the internals of LLVM that began in r239036.
Reviewers: rengolin
Reviewed By: rengolin
Subscribers: ted, llvm-commits, rengolin, jholewinski
Differential Revision: http://reviews.llvm.org/D10362
llvm-svn: 239554
This makes emitAbsoluteSymbolDiff always succeed and moves logic from the asm
printer to it.
The object one now also works on ELF. If two symbols are in the same fragment,
we will never move them apart.
llvm-svn: 239552
Now actually stores the non-zero constant instead of 0.
I somehow forgot to include this part of r238108.
The test change was just an independent instruction order swap,
so just add another check line to satisfy CHECK-NEXT.
llvm-svn: 239539
On large goto table based interpreters, where phi nodes can have (very) large
fan-ins, isLiveOut exhibited poor performances: about 40% of the full
codegen time was spent in PHIElim, sorting MachineBasicBlock addresses.
This patch improve the performances for such cases, and does not show
compile time regressions on the LNT, at bootstrap (llvm+clang+lldb) or
any other benchmarks we have in-house.
llvm-svn: 239510
It hasn't been used since r130964.
This also removes MachineModuleInfo::isUsedFunction and
MachineModuleInfo::AnalyzeModule, both of which were only
there to support UsedFunctions.
llvm-svn: 239501
During statepoint lowering we can sometimes avoid spilling of the value if we know that it was already spilled for previous statepoint.
We were doing this by checking if incoming statepoint value was lowered into load from stack slot. This was working only in boundaries of one basic block.
But instead of looking at the lowered node we can look directly at the llvm-ir value and if it was gc.relocate (or some simple modification of it) look up stack slot for it's derived pointer and reuse stack slot from it. This allows us to look across basic block boundaries.
Differential Revision: http://reviews.llvm.org/D10251
llvm-svn: 239472
Use a "safeseh" string attribute to do this. You would think we chould
just accumulate the set of personalities like we do on dwarf, but this
fails to account for the LSDA-loading thunks we use for
__CxxFrameHandler3. Each of those needs to make it into .sxdata as well.
The string attribute seemed like the most straightforward approach.
llvm-svn: 239448
Summary:
The RegisterScavenger explicitly ignores <kill> flags on operands of
predicated instructions and therefore assumes that such registers remain
live. When it then scavenges such a register, it inserts a spill of this
(killed) register. This is invalid code and gets flagged up by the
verifier.
Nowadays kill flags are set correctly on predicated instructions. This
patch makes the Scavenger respect them.
The bug has so far only been triggered by an internal pass, so I don't
have a test case unfortunately.
Fixes PR23119.
Reviewers: hfinkel, tobiasvk_caf
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D9039
llvm-svn: 239439
This gets all the handler info through to the asm printer and we can
look at the .xdata tables now. I've convinced one small catch-all test
case to work, but other than that, it would be a stretch to say this is
functional.
The state numbering algorithm avoids doing any scope reconstruction as
we do for C++ to simplify the implementation.
llvm-svn: 239433
Summary: I noticed an object file with `DW_OP_reg4 DW_OP_breg4 0` as a DWARF expression,
which I traced to a missing break (and `++I`) in this code snippet.
While I was at it, I also added support for a few other corner cases
along the same lines that I could think of.
Test Plan: Hand-crafted test case to exercises these cases is included.
Reviewers: echristo, dblaikie, aprantl
Reviewed By: aprantl
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D10302
llvm-svn: 239380
Summary:
This was a longstanding FIXME and is a necessary precursor to cases
where foldOperandImpl may have to create more than one instruction
(e.g. to constrain a register class). This is the split out NFC changes from
D6262.
Reviewers: pete, ributzka, uweigand, mcrosier
Reviewed By: mcrosier
Subscribers: mcrosier, ted, llvm-commits
Differential Revision: http://reviews.llvm.org/D10174
llvm-svn: 239336
on a per-function basis.
Previously some of the passes were conditionally added to ARM's pass pipeline
based on the target machine's subtarget. This patch makes changes to add those
passes unconditionally and execute them conditonally based on the predicate
functor passed to the pass constructors. This enables running different sets of
passes for different functions in the module.
rdar://problem/20542263
Differential Revision: http://reviews.llvm.org/D8717
llvm-svn: 239325
The global-merge pass was crashing because it assumes that all ConstantExprs
(reached via the global variables that they use) have at least one user.
I haven't worked out a way to test this, as an unused ConstantExpr cannot be
represented by serialised IR, and global-merge can only be run in llc, which
does not run any passes which can make a ConstantExpr dead.
This (reduced to the point of silliness) C code triggers this bug when compiled
for arm-none-eabi at -O1:
static a = 7;
static volatile b[10] = {&a};
c;
main() {
c = 0;
for (; c < 10;)
printf(b[c]);
}
Differential Revision: http://reviews.llvm.org/D10314
llvm-svn: 239308
the overloaded version of addPass which takes Pass*.
This change enables inserting the machine printer pass when the overloaded
version of addPass that takes Pass* is called to add a pass, instead of the
one which takes AnalysisID. I need this to prevent make-check tests from
failing when I commit another patch later.
llvm-svn: 239192
For targets with a free fneg, this fold is always a net loss if it
ends up duplicating the multiply, so definitely avoid it.
This might be true for some targets without a free fneg too, but
I'll leave that for future investigation.
llvm-svn: 239167
Also, moved test cases from CodeGen/X86/fold-buildvector-bug.ll into
CodeGen/X86/buildvec-insertvec.ll and regenerated CHECK lines using
update_llc_test_checks.py.
llvm-svn: 239142
gc.statepoint intrinsics with a far immediate call target
were lowered incorrectly as pc-rel32 calls.
This change fixes the problem, and generates an indirect call
via a scratch register.
For example:
Intrinsic:
%safepoint_token = call i32 (i64, i32, void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(i64 0, i32 0, void ()* inttoptr (i64 140727162896504 to void ()*), i32 0, i32 0, i32 0, i32 0)
Old Incorrect Lowering:
callq 140727162896504
New Correct Lowering:
movabsq $140727162896504, %rax
callq *%rax
In lowerCallFromStatepoint(), the callee-target was modified and
represented as a "TargetConstant" node, rather than a "Constant" node.
Undoing this modification enabled LowerCall() to generate the
correct CALL instruction.
llvm-svn: 239114
The big/small ordering here is based on signed values so SmallValue will
be INT_MIN and BigValue 0. This shouldn't be a problem but the code
assumed that BigValue always had more bits set than SmallValue.
We used to just miss the transformation, but a recent refactoring of
mine turned this into an assertion failure.
llvm-svn: 239105
Basic block selection involves checking successor BBs for PHI nodes
that depend on the current BB. In case such BBs are found, the value
being selected is a constant and such constant already exists in
current BB, it's value is reused.
This might lead to wrong locations in some situations, especially if
same constant value ends up being materialized twice in two different
ways, which discards that sharing and leaves us with wrong debug
location in the successor BB.
In code this involves the following sequence of calls:
SelectionDAGBuilder::HandlePHINodesInSuccessorBlocks ->
SelectionDAGBuilder::CopyValueToVirtualRegister ->
SelectionDAGBuilder::getNonRegisterValue
llvm-svn: 239089
Now that we can look at users, we can trivially do this: when we would
have otherwise disabled GlobalMerge (currently -O<3), we can just run
it for minsize functions, as it's usually a codesize win.
Differential Revision: http://reviews.llvm.org/D10054
llvm-svn: 239087
Method 'visitBUILD_VECTOR' in the DAGCombiner knows how to combine a
build_vector of a bunch of extract_vector_elt nodes and constant zero nodes
into a shuffle blend with a zero vector.
However, method 'visitBUILD_VECTOR' forgot that a floating point
build_vector may contain negative zero as well as positive zero.
Example:
define <2 x double> @example(<2 x double> %A) {
entry:
%0 = extractelement <2 x double> %A, i32 0
%1 = insertelement <2 x double> undef, double %0, i32 0
%2 = insertelement <2 x double> %1, double -0.0, i32 1
ret <2 x double> %2
}
Before this patch, llc (with -mattr=+sse4.1) wrongly generated
movq %xmm0, %xmm0 # xmm0 = xmm0[0],zero
So, the sign bit of the negative zero was effectively lost.
This patch fixes the problem by adding explicit checks for positive zero.
With this patch, llc produces the following code for the example above:
movhpd .LCPI0_0(%rip), %xmm0
where .LCPI0_0 referes to a 'double -0'.
llvm-svn: 239070
When checking (High - Low + 1).sle(BitWidth), BitWidth would be truncated
to the size of the left-hand side. In the case of this PR, the left-hand
side was i4, so BitWidth=64 got truncated to 0 and the assert failed.
llvm-svn: 239048
If the compare in a select pattern has another use then it can't be removed, so we'd just
be creating repeated code if we created a min/max node.
Spotted by Matt Arsenault!
llvm-svn: 239037
Summary:
LLVM's MI level notion of invariant_load is different from LLVM's IR
level notion of invariant_load with respect to dereferenceability. The
IR notion of invariant_load only guarantees that all *non-faulting*
invariant loads result in the same value. The MI notion of invariant
load guarantees that the load can be legally moved to any location
within its containing function. The MI notion of invariant_load is
stronger than the IR notion of invariant_load -- an MI invariant_load is
an IR invariant_load + a guarantee that the location being loaded from
is dereferenceable throughout the function's lifetime.
Reviewers: hfinkel, reames
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D10075
llvm-svn: 238881
This create a MCSymbolELF class and moves SymbolSize since only ELF
needs a size expression.
This reduces the size of MCSymbol from 56 to 48 bytes.
llvm-svn: 238801
If a dead instruction we may not only have a last-use in the main live
range but also in a subregister range if subregisters are tracked. We
need to partially rebuild live ranges in both cases.
The testcase only broke when subregister liveness was enabled. I
commited it in the current form because there is currently no flag to
enable/disable subregister liveness.
This fixes PR23720.
llvm-svn: 238785
This is important because of different addressing modes
depending on the address space for GPU targets.
This only adds the argument, and does not update
any of the uses to provide the correct address space.
llvm-svn: 238723
r238503 fixed the problem of too-small shift types by promoting them
during legalization, but the correct solution is to promote only the
operands that actually demand promotion.
This fixes a crash on an out-of-tree target caused by trying to
promote an operand that can't be promoted.
llvm-svn: 238632
If the type isn't trivially moveable emplace can skip a potentially
expensive move. It also saves a couple of characters.
Call sites were found with the ASTMatcher + some semi-automated cleanup.
memberCallExpr(
argumentCountIs(1), callee(methodDecl(hasName("push_back"))),
on(hasType(recordDecl(has(namedDecl(hasName("emplace_back")))))),
hasArgument(0, bindTemporaryExpr(
hasType(recordDecl(hasNonTrivialDestructor())),
has(constructExpr()))),
unless(isInTemplateInstantiation()))
No functional change intended.
llvm-svn: 238602
For some history here see the commit messages of r199797 and r169060.
The original intent was to fix cases like:
%EAX<def> = COPY %ECX<kill>, %RAX<imp-def>
%RCX<def> = COPY %RAX<kill>
where simply removing the copies would have RCX undefined as in terms of
machine operands only the ECX part of it is defined. The machine
verifier would complain about this so 169060 changed such COPY
instructions into KILL instructions so some super-register imp-defs
would be preserved. In r199797 it was finally decided to always do this
regardless of super-register defs.
But this is wrong, consider:
R1 = COPY R0
...
R0 = COPY R1
getting changed to:
R1 = KILL R0
...
R0 = KILL R1
It now looks like R0 dies at the first KILL and won't be alive until the
second KILL, while in reality R0 is alive and must not change in this
part of the program.
As this only happens after register allocation there is not much code
still performing liveness queries so the issue was not noticed. In fact
I didn't manage to create a testcase for this, without unrelated changes
I am working on at the moment.
The fix is simple: As of r223896 the MachineVerifier allows reads from
partially defined registers, so the whole transforming COPY->KILL thing
is not necessary anymore. This patch also changes a similar (but more
benign case as the def and src are the same register) case in the
VirtRegRewriter.
Differential Revision: http://reviews.llvm.org/D10117
llvm-svn: 238588
This commit translates the line and column numbers for LLVM IR
errors from the numbers in the YAML block scalar to the numbers
in the MIR file so that the MIRParser users can report LLVM IR
errors with the correct line and column numbers.
Reviewers: Duncan P. N. Exon Smith
Differential Revision: http://reviews.llvm.org/D10108
llvm-svn: 238576
Small (really small!) C++ exception handling examples work on 32-bit x86
now.
This change disables the use of .seh_* directives in WinException when
CFI is not in use. It also uses absolute symbol references in the tables
instead of imagerel32 relocations.
Also fixes a cache invalidation bug in MMI personality classification.
llvm-svn: 238575
MIOperands/ConstMIOperands are classes iterating over the MachineOperand
of a MachineInstr, however MachineInstr::mop_iterator does the same
thing.
I assume these two iterators exist to have a uniform interface to
iterate over the operands of a machine instruction bundle and a single
machine instruction. However in practice I find it more confusing to have 2
different iterator classes, so this patch transforms (nearly all) the
code to use mop_iterators.
The only exception being MIOperands::anlayzePhysReg() and
MIOperands::analyzeVirtReg() still needing an equivalent, I leave that
as an exercise for the next patch.
Differential Revision: http://reviews.llvm.org/D9932
This version is slightly modified from the proposed revision in that it
introduces MachineInstr::getOperandNo to avoid the extra counting
variable in the few loops that previously used MIOperands::getOperandNo.
llvm-svn: 238539
About pristine regsiters:
Pristine registers "hold a value that is useless to the current
function, but that must be preserved - they are callee saved registers
that have not been saved." This concept saves compile time as it frees
the prologue/epilogue inserter from adding every such register to every
basic blocks live-in list.
However the current code in getPristineRegs is formulated in a
complicated way: Inside the function prologue and epilogue all callee
saves are considered pristine, while in the rest of the code only the
non-saved ones are considered pristine. This requires logic to
differentiate between prologue/epilogue and the rest and in the presence
of shrink-wrapping this even becomes complicated/expensive. It's also
unnecessary because the prologue epilogue inserters already mark
callee-save registers that are saved/restores properly in the respective
blocks in the prologue/epilogue (see updateLiveness() in
PrologueEpilogueInserter.cpp). So only declaring non-saved/restored
callee saved registers as pristine just works.
Differential Revision: http://reviews.llvm.org/D10101
llvm-svn: 238524
This commit introduces a serializable structure called
'llvm::yaml::MachineFunction' that stores the machine
function's name. This structure will mirror the machine
function's state in the future.
This commit prints machine functions as YAML documents
containing a YAML mapping that stores the state of a machine
function. This commit also parses the YAML documents
that contain the machine functions.
Reviewers: Duncan P. N. Exon Smith
Differential Revision: http://reviews.llvm.org/D9841
llvm-svn: 238519
This moves all the state numbering code for C++ EH to WinEHPrepare so
that we can call it from the X86 state numbering IR pass that runs
before isel.
Now we just call the same state numbering machinery and insert a bunch
of stores. It also populates MachineModuleInfo with information about
the current function.
llvm-svn: 238514
DIEAbbrev contains a SmallVector that can leak for overly large abbrevs. They
used to be owned by the DIE, but after the recent refactoring DWARFFile
allocates its own abbrevs.
Leak found by asan.
llvm-svn: 238418
Change `DIE::addChild()` to return a reference to the just-added node,
and update consumers to use it directly. An upcoming commit will
abstract away (and eventually change) the underlying storage of
`DIE::Children`.
llvm-svn: 238372
Stop storing a `DIEAbbrev` in `DIE`, since the data fits neatly inside
the `DIEValue` list. Besides being a cleaner data structure (avoiding
the parallel arrays), this gives us more freedom to rearrange the
`DIEValue` list.
This fixes the temporary memory regression from 845 MB up to 879 MB, and
drops it further to 829 MB for a net memory decrease of around 1.9%
(incremental decrease around 5.7%).
(I'm looking at `llc` memory usage on `verify-uselistorder.lto.opt.bc`;
see r236629 for details.)
llvm-svn: 238364
This reverts commit r238350, effectively reapplying r238349 after fixing
(all?) the problems, all somehow related to how I was using
`AlignedArrayCharUnion<>` inside `DIEValue`:
- MSVC can only handle `sizeof()` on types, not values. Change the
assert.
- GCC doesn't know the `is_trivially_copyable` type trait. Instead of
asserting it, add destructors.
- Call placement new even when constructing POD (i.e., the pointers).
- Instead of copying the char buffer, copy the casted classes.
I've left in a couple of `static_assert`s that I think both MSVC and GCC
know how to handle. If the bots disagree with me, I'll remove them.
- Check that the constructed type is either standard layout or a
pointer. This protects against a programming error: we really want
the "small" `DIEValue`s to be small and simple, so don't
accidentally change them not to be.
- Similarly, check that the size of the buffer is no bigger than a
`uint64_t` or a pointer. (I thought checking against
`sizeof(uint64_t)` would be good enough, but Chandler suggested that
pointers might sometimes be bigger than that in the context of
sanitizers.)
I've also committed r238359 in the meantime, which introduces a
DIEValue.def to simplify dispatching between the various types (thanks
to a review comment by David Blaikie). Without that, this commit would
be almost unintelligible.
Here's the original commit message:
--
Change `DIEValue` to be stored/passed/etc. by value, instead of
reference. It's now a discriminated union, with a `Val` field storing
the actual type. The classes that used to inherit from `DIEValue` no
longer do. There are two categories of these:
- Small values fit in a single pointer and are stored by value.
- Large values require auxiliary storage, and are stored by reference.
The only non-mechanical change is to tools/dsymutil/DwarfLinker.cpp. It
was relying on `DIEInteger`s being passed around by reference, so I
replaced that assumption with a `PatchLocation` type that stores a safe
reference to where the `DIEInteger` lives instead.
This commit causes a temporary regression in memory usage, since I've
left merging `DIEAbbrevData` into `DIEValue` for a follow-up commit. I
measured an increase from 845 MB to 879 MB, around 3.9%. The follow-up
drops it lower than the starting point, and I've only recently brought
the memory this low anyway, so I'm committing these changes separately
to keep them incremental. (I also considered swapping the commits, but
the other one first would cause a lot more code churn.)
(I'm looking at `llc` memory usage on `verify-uselistorder.lto.opt.bc`;
see r236629 for details.)
--
llvm-svn: 238362
This reverts commit r238349, since it caused some errors on bots:
- std::is_trivially_copyable isn't available until GCC 5.0.
- It was complaining about strict aliasing with my use of
ArrayCharUnion.
llvm-svn: 238350
Change `DIEValue` to be stored/passed/etc. by value, instead of
reference. It's now a discriminated union, with a `Val` field storing
the actual type. The classes that used to inherit from `DIEValue` no
longer do. There are two categories of these:
- Small values fit in a single pointer and are stored by value.
- Large values require auxiliary storage, and are stored by reference.
The only non-mechanical change is to tools/dsymutil/DwarfLinker.cpp. It
was relying on `DIEInteger`s being passed around by reference, so I
replaced that assumption with a `PatchLocation` type that stores a safe
reference to where the `DIEInteger` lives instead.
This commit causes a temporary regression in memory usage, since I've
left merging `DIEAbbrevData` into `DIEValue` for a follow-up commit. I
measured an increase from 845 MB to 879 MB, around 3.9%. The follow-up
drops it lower than the starting point, and I've only recently brought
the memory this low anyway, so I'm committing these changes separately
to keep them incremental. (I also considered swapping the commits, but
the other one first would cause a lot more code churn.)
(I'm looking at `llc` memory usage on `verify-uselistorder.lto.opt.bc`;
see r236629 for details.)
llvm-svn: 238349
This commit a 3rd attempt at comitting the initial MIR serialization patch.
The first commit (r237708) was reverted in 237730. Then the second commit
(r237954) was reverted in r238007, as the MIR library under CodeGen caused
a circular dependency where the CodeGen library depended on MIR and MIR
library depended on CodeGen.
This commit has fixed the dependencies between CodeGen and MIR by
reorganizing the MIR serialization code - the code that prints out
MIR has been moved to CodeGen, and the MIR library has been renamed
to MIRParser. Now the CodeGen library doesn't depend on the
MIRParser library, thus the circular dependency no longer exists.
--Original Commit Message--
MIR Serialization: print and parse LLVM IR using MIR format.
This commit is the initial commit for the MIR serialization project.
It creates a new library under CodeGen called 'MIR'. This new
library adds a new machine function pass that prints out the LLVM IR
using the MIR format. This pass is then added as a last pass when a
'stop-after' option is used in llc. The new library adds the initial
functionality for parsing of MIR files as well. This commit also
extends the llc tool so that it can recognize and parse MIR input files.
Reviewers: Duncan P. N. Exon Smith, Matthias Braun, Philip Reames
Differential Revision: http://reviews.llvm.org/D9616
llvm-svn: 238341
v2: TargetLoweringBase:: -> TargetLowering::
Use Ops array
v3: Explicitly use value 0 for ?DIV
Remove redundant newline
Differential revision: http://reviews.llvm.org/D7803
reviewer: ab
llvm-svn: 238336
- Clean documentation comment
- Change the API to accept an iterator so you can actually pass
MachineBasicBlock::end() now.
- Add more "const".
llvm-svn: 238288
remove ExecutionEngine's dependence on CodeGen. NFC.
This is a follow-up to r238080.
Differential Revision: http://reviews.llvm.org/D9830
llvm-svn: 238244
Stop creating symbols we don't need in `DwarfStringPool`. The consumers
only call `DwarfStringPoolEntryRef::getSymbol()` when DWARF is
relocatable, so this just stops creating the unused symbols when it's
not. This drops memory usage from 851 MB to 845 MB, around 0.7%.
(I'm looking at `llc` memory usage on `verify-uselistorder.lto.opt.bc`;
see r236629 for details.)
llvm-svn: 238122
Mint a new function, `AsmPrinter::emitDwarfStringOffset()`, which takes
a `DwarfStringPoolEntryRef`. When DWARF is relocatable across sections,
this defers to `emitSectionOffset()` and emits the `MCSymbol`;
otherwise, just emit the offset directly, without using any intermediate
symbols.
`EmitLabelDifference()` is already optimized to emit absolute label
differences cheaply when possible, so there aren't any major memory
savings here (853 MB down to 851 MB, or 0.2%). However, it prepares for
making the `MCSymbol`s in the `DwarfStringPool` optional.
(I'm looking at `llc` memory usage on `verify-uselistorder.lto.opt.bc`;
see r236629 for details.)
llvm-svn: 238119
Expose the `DwarfStringPool` entry in a header, and store a pointer to
it directly in `DIEString`. Instead of choosing at creation time how to
emit it, use the `dwarf::Form` to determine that at emission time.
Besides avoiding the other `DIEValue`, this shaves two pointers off of
`DIEString`; the data is now a single pointer. This is a nice cleanup
on its own -- and drops memory usage from 861 MB down to 853 MB, around
0.9% -- but it's also preparation for passing `DIEValue`s by value.
(I'm looking at `llc` memory usage on `verify-uselistorder.lto.opt.bc`;
see r236629 for details.)
llvm-svn: 238117
Extract out `DwarfStringPoolEntry` and `DwarfStringPoolRef` from
`DwarfStringPool` so that downstream users can start using
`DwarfStringPool::getEntry()` directly. This will allow users to delay
the decision between emitting a symbol or an offset until later.
llvm-svn: 238116
Duncan P. N. Exon Smith