Summary:
A desired property of the node order in Swing Modulo Scheduling is
that for nodes outside circuits the following holds: none of them is
scheduled after both a successor and a predecessor. We call
node orders that meet this property valid.
Although invalid node orders do not lead to the generation of incorrect
code, they can cause the pipeliner not being able to find a pipelined schedule
for arbitrary II. The reason is that after scheduling the successor and the
predecessor of a node, no room may be left to schedule the node itself.
For data flow graphs with 0-latency edges, the node ordering algorithm
of Swing Modulo Scheduling can generate such undesired invalid node orders.
This patch fixes that.
In the remainder of this commit message, I will give an example
demonstrating the issue, explain the fix, and explain how the the fix is tested.
Consider, as an example, the following data flow graph with all
edge latencies 0 and all edges pointing downward.
```
n0
/ \
n1 n3
\ /
n2
|
n4
```
Consider the implemented node order algorithm in top-down mode. In that mode,
the algorithm orders the nodes based on greatest Height and in case of equal
Height on lowest Movability. Finally, in case of equal Height and
Movability, given two nodes with an edge between them, the algorithm prefers
the source-node.
In the graph, for every node, the Height and Movability are equal to 0.
As will be explained below, the algorithm can generate the order n0, n1, n2, n3, n4.
So, node n3 is scheduled after its predecessor n0 and after its successor n2.
The reason that the algorithm can put node n2 in the order before node n3,
even though they have an edge between them in which node n3 is the source,
is the following: Suppose the algorithm has constructed the partial node
order n0, n1. Then, the nodes left to be ordered are nodes n2, n3, and n4. Suppose
that the while-loop in the implemented algorithm considers the nodes in
the order n4, n3, n2. The algorithm will start with node n4, and look for
more preferable nodes. First, node n4 will be compared with node n3. As the nodes
have equal Height and Movability and have no edge between them, the algorithm
will stick with node n4. Then node n4 is compared with node n2. Again the
Height and Movability are equal. But, this time, there is an edge between
the two nodes, and the algorithm will prefer the source node n2.
As there are no nodes left to compare, the algorithm will add node n2 to
the node order, yielding the partial node order n0, n1, n2. In this way node n2
arrives in the node-order before node n3.
To solve this, this patch introduces the ZeroLatencyHeight (ZLH) property
for nodes. It is defined as the maximum unweighted length of a path from the
given node to an arbitrary node in which each edge has latency 0.
So, ZLH(n0)=3, ZLH(n1)=ZLH(n3)=2, ZLH(n2)=1, and ZLH(n4)=0
In this patch, the preference for a greater ZeroLatencyHeight
is added in the top-down mode of the node ordering algorithm, after the
preference for a greater Height, and before the preference for a
lower Movability.
Therefore, the two allowed node-orders are n0, n1, n3, n2, n4 and n0, n3, n1, n2, n4.
Both of them are valid node orders.
In the same way, the bottom-up mode of the node ordering algorithm is adapted
by introducing the ZeroLatencyDepth property for nodes.
The patch is tested by adding extra checks to the following existing
lit-tests:
test/CodeGen/Hexagon/SUnit-boundary-prob.ll
test/CodeGen/Hexagon/frame-offset-overflow.ll
test/CodeGen/Hexagon/vect/vect-shuffle.ll
Before this patch, the pipeliner failed to pipeline the loops in these tests
due to invalid node-orders. After the patch, the pipeliner successfully
pipelines all these loops.
Reviewers: bcahoon
Reviewed By: bcahoon
Subscribers: Ayal, mgrang, llvm-commits
Differential Revision: https://reviews.llvm.org/D43620
llvm-svn: 326925
Fixes the bug found by asan. Also XFAIL the new test for Darwin,
which is stuck on DWARF v2, and fix up other tests so they stop
failing on Windows.
llvm-svn: 326839
Summary:
- Emit UdtSourceLine information for enums to match MSVC
- Add a method to add UDTSrcLine and call it for all Class/Struct/Union/Enum
- Update test cases to verify the changes
Reviewers: zturner, llvm-commits, rnk
Reviewed By: rnk
Differential Revision: https://reviews.llvm.org/D44116
llvm-svn: 326824
I obviously messed up arguments to MachineOperand::CreateReg
in rL326769. This should make it work as intended.
Thanks to RKSimon for spotting this.
llvm-svn: 326780
Summary:
This is a workaround for pr36417
https://bugs.llvm.org/show_bug.cgi?id=36417
LiveDebugVariables will now verify that the DBG_VALUE instructions
are sane (prior to register allocation) by asking LIS if a virtual
register used in the DBG_VALUE is live (or dead def) in the slot
index before the DBG_VALUE. If it isn't sane the DBG_VALUE is
discarded.
One pass that was identified as introducing non-sane DBG_VALUE
instructtons, when analysing pr36417, was the DAG->DAG Instruction
Selection. It sometimes inserts DBG_VALUE instructions referring to
a virtual register that is defined later in the same basic block.
So it is a use before def kind of problem. The DBG_VALUE is
typically inserted in the beginning of a basic block when this
happens. The problem can be seen in the test case
test/DebugInfo/X86/dbg-value-inlined-parameter.ll
Reviewers: aprantl, rnk, probinson
Reviewed By: aprantl
Subscribers: vsk, davide, alexcrichton, Ka-Ka, eraman, llvm-commits, JDevlieghere
Differential Revision: https://reviews.llvm.org/D43956
llvm-svn: 326769
DWARF v5 specifies that the root file (also given in the DW_AT_name
attribute of the compilation unit DIE) should be emitted explicitly to
the line table's list of files. This makes the line table more
independent of the .debug_info section.
Differential Revision: https://reviews.llvm.org/D44054
llvm-svn: 326758
Summary:
Fabs is a common floating-point operation, especially for some expansions. This patch adds
a new generic opcode for llvm.fabs.* intrinsic in order to avoid building/matching this intrinsic.
Reviewers: qcolombet, aditya_nandakumar, dsanders, rovka
Reviewed By: aditya_nandakumar
Subscribers: kristof.beyls, javed.absar, llvm-commits
Differential Revision: https://reviews.llvm.org/D43864
llvm-svn: 326749
This is a NFC simple patch that changes the DEBUG dumping in the
MachineScheduler so that the dumping of the built SUnits is done before the
SchedImpl->initialize() is called.
This is better on SystemZ, since it has a strategy that does some dumping at
the start of the region, and it is not possible to easily read it if it is
output above a long list of SU.
Review: Javed Absar
https://reviews.llvm.org/D44089
llvm-svn: 326716
Loading a constant into a k-register in AVX512 requires a bitcast from a scalar constant. In the test case here we have a k-register store that gets split into multiple parts of KNL. MergeConsecutiveStores sees each of these pieces as a consecutive store and looks through the bitcast to find the underly scalar constant. But when we went to create the combined store we didn't look through the same bitcast.
llvm-svn: 326677
X86 considers v1i1 a legal type under AVX512 and as such a truncate from a v1iX type to v1i1 can be turned into a scalar truncate plus a conversion to v1i1. We would much prefer a v1i1 SCALAR_TO_VECTOR over a one element BUILD_VECTOR.
During lowering we were detecting the v1i1 BUILD_VECTOR as a splat BUILD_VECTOR like we try to do for v2i1/v4i1/etc. In this case we create (select i1 splat_elt, v1i1 all-ones, v1i1 all-zeroes). That goes through some more legalization and we end up with a CMOV choosing between 0 and 1 in scalar and a scalar_to_vector.
Arguably we could detect the v1i1 BUILD_VECTOR and do this better in X86 target code. But just using a SCALAR_TO_VECTOR in legalization is much easier.
llvm-svn: 326637
The fast/linear DAG scheduler doesn't lower DBG_VALUEs except for
function entry nodes.
Patch by Joshua Cranmer!
Differential Revision: https://reviews.llvm.org/D43028
llvm-svn: 326631
This patch adds support for detecting outer loops with irreducible control
flow in LV. Current detection uses SCCs and only works for innermost loops.
This patch adds a utility function that works on any CFG, given its RPO
traversal and its LoopInfoBase. This function is a generalization
of isIrreducibleCFG from lib/CodeGen/ShrinkWrap.cpp. The code in
lib/CodeGen/ShrinkWrap.cpp is also updated to use the new generic utility
function.
Patch by Diego Caballero <diego.caballero@intel.com>
Differential Revision: https://reviews.llvm.org/D40874
llvm-svn: 326568
Masking first, prevents the extend from being combine with loads. Its also interfering with some vXi1 extraction code.
Differential Revision: https://reviews.llvm.org/D42679
llvm-svn: 326500
This supports things like
(setcc ugt X, 0) -> (setcc ne X, 0)
I've restricted to only make changes to vectors before legalize ops because I doubt all targets have accurate condition code legality information for vectors given how little we did before.
Differential Revision: https://reviews.llvm.org/D42948
llvm-svn: 326495
Currently it's impossible to test InstructionSelect pass with MIR which
is considered illegal by the Legalizer in Assert builds. In early stages
of porting an existing backend from SelectionDAG ISel to GlobalISel,
however, we would have very basic CallLowering, Legalizer, and
RegBankSelect implementations, but rather functional Instruction Select
with quite a few patterns selectable due to the semi-automatic porting
process borrowing them from SelectionDAG ISel.
As we are trying to define legality as a property of being selectable by
the instruction selector, it would be nice to be able to easily check
what the selector can do in its current state w/o the legality check
provided by the Legalizer getting in the way.
It also seems beneficial to have a regression testing set up that would
not allow the selector to silently regress in its support of the MIR not
supported yet by the previous passes in the GlobalISel pipeline.
This commit adds -disable-gisel-legality-check command line option to
llc that disables those legality checks in RegBankSelect and
InstructionSelect passes.
It also adds quite a few MIR test cases for AArch64's Instruction
Selector. Every one of them would fail on the legality check at the
moment, but will select just fine if the check is disabled. Every test
MachineFunction is intended to exercise a specific selection rule and
that rule only, encoded in the MachineFunction's name by the rule's
number, ID, and index of its GIM_Try opcode in TableGen'erated
MatchTable (-optimize-match-table=false).
Reviewers: ab, dsanders, qcolombet, rovka
Reviewed By: bogner
Subscribers: kristof.beyls, volkan, aditya_nandakumar, aemerson,
rengolin, t.p.northover, javed.absar, llvm-commits
Differential Revision: https://reviews.llvm.org/D42886
llvm-svn: 326396
FailedISel MachineFunction property is part of the CodeGen pipeline
state as much as every other property, notably, Legalized,
RegBankSelected, and Selected. Let's make that part of the state also
serializable / de-serializable, so if GlobalISel aborts on some of the
functions of a large module, but not the others, it could be easily seen
and the state of the pipeline could be maintained through llc's
invocations with -stop-after / -start-after.
To make MIR printable and generally to not to break it too much too
soon, this patch also defers cleaning up the vreg -> LLT map until
ResetMachineFunctionPass.
To make MIR with FailedISel: true also machine verifiable, machine
verifier is changed so it treats a MIR-module as non-regbankselected and
non-selected if there is FailedISel property set.
Reviewers: qcolombet, ab
Reviewed By: dsanders
Subscribers: javed.absar, rovka, kristof.beyls, llvm-commits
Differential Revision: https://reviews.llvm.org/D42877
llvm-svn: 326343
Emulated TLS is enabled by llc flag -emulated-tls,
which is passed by clang driver.
When llc is called explicitly or from other drivers like LTO,
missing -emulated-tls flag would generate wrong TLS code for targets
that supports only this mode.
Now use useEmulatedTLS() instead of Options.EmulatedTLS to decide whether
emulated TLS code should be generated.
Unit tests are modified to run with and without the -emulated-tls flag.
Differential Revision: https://reviews.llvm.org/D42999
llvm-svn: 326341
Summary:
Some targets does not support labels inside debug sections, but support
references in form `section +|- offset`. Patch adds initial support
for this. Also, this patch disables emission of all additional debug
sections that may have labels inside of it (like pub sections and
string tables).
Reviewers: probinson, echristo
Subscribers: JDevlieghere, llvm-commits
Differential Revision: https://reviews.llvm.org/D43627
llvm-svn: 326328
Removes verifyDomTree, using assert(verify()) everywhere instead, and
changes verify a little to always run IsSameAsFreshTree first in order
to print good output when we find errors. Also adds verifyAnalysis for
PostDomTrees, which will allow checking of PostDomTrees it the same way
we check DomTrees and MachineDomTrees.
Differential Revision: https://reviews.llvm.org/D41298
llvm-svn: 326315
Absence of memory operands is treated as "aliasing everything", so
dropping them is sufficient.
Recommit r326256 with a fixed testcase.
llvm-svn: 326262
Qualifiers on a pointer or reference type may apply to either the
pointee or the pointer itself. Consider 'const char *' and 'char *
const'. In the first example, the pointee data may not be modified
without casts, and in the second example, the pointer may not be updated
to point to new data.
In the general case, qualifiers are applied to types with LF_MODIFIER
records, which support the usual const and volatile qualifiers as well
as the __unaligned extension qualifier.
However, LF_POINTER records, which are used for pointers, references,
and member pointers, have flags for qualifiers applying to the
*pointer*. In fact, this is the only way to represent the restrict
qualifier, which can only apply to pointers, and cannot qualify regular
data types.
This patch causes LLVM to correctly fold 'const' and 'volatile' pointer
qualifiers into the pointer record, as well as adding support for
'__restrict' qualifiers in the same place.
Based on a patch from Aaron Smith
Differential Revision: https://reviews.llvm.org/D43060
llvm-svn: 326260
When attempting to compile the following Objective-C++ code with
CodeView debug info:
void (^b)(void) = []() {};
The generated debug metadata contains a structure like the following:
!43 = !DICompositeType(tag: DW_TAG_structure_type, name: "__block_literal_1", scope: !6, file: !6, line: 1, size: 168, elements: !44)
!44 = !{!45, !46, !47, !48, !49, !52}
...
!52 = !DIDerivedType(tag: DW_TAG_member, scope: !6, file: !6, line: 1, baseType: !53, size: 8, offset: 160, flags: DIFlagPublic)
!53 = !DIDerivedType(tag: DW_TAG_const_type, baseType: !54)
!54 = !DICompositeType(tag: DW_TAG_class_type, file: !6, line: 1, flags: DIFlagFwdDecl)
Note that the member node (!52) is unnamed, but rather than pointing to
a DICompositeType directly, it points to a DIDerivedType with tag
DW_TAG_const_type, which then points to the DICompositeType. However,
the CodeView assembly printer currently assumes that the base type for
an unnamed member will always be a DICompositeType, and attempts to
perform that cast, which triggers an assertion failure, since in this
case the base type is actually a DIDerivedType, not a DICompositeType
(and we would have to get the base type of the DIDerivedType to reach
the DICompositeType). I think the debug metadata being generated by the
frontend is correct (or at least plausible), and the CodeView printer
needs to handle this case.
This patch teaches the CodeView printer to unwrap any qualifier types.
The qualifiers are just dropped for now. Ideally, they would be applied
to the added indirect members instead, but this occurs infrequently
enough that adding the logic to handle the qualifiers correctly isn't
worth it for now. A FIXME is added to note this.
Additionally, Reid pointed out that the underlying assumption that an
unnamed member must be a composite type is itself incorrect and may not
hold for all frontends. Therefore, after all qualifiers have been
stripped, check if the resulting type is in fact a DICompositeType and
just return if it isn't, rather than assuming the type and crashing if
that assumption is violated.
Differential Revision: https://reviews.llvm.org/D43803
llvm-svn: 326255
Currently when abort is enabled, we get a diagnostic saying "Fallback
path used .... " and the program terminates. To actually figure out what
the reason is, we need to run again with another verbose argument
"-pass-remarks-missed=gisel". Instead, when we are going to abort,
we might as well print expensive remarks.
https://reviews.llvm.org/D43796
llvm-svn: 326215
Re-enable commit r323991 now that r325931 has been committed to make
MachineOperand::isRenamable() check more conservative w.r.t. code
changes and opt-in on a per-target basis.
llvm-svn: 326208
AVX512 used to promote v32i1 to v32i8 during legalization when BWI was disabled. So this code was added to improve legalization of v32i1 concat_vectors of v16i1 by extending the v16i1 to v16i8 to avoid scalarization.
X86 has since switched to legalizing v32i1 by splitting to v16i1 instead. This has rendered this code unnecessary and its no longer exercised.
llvm-svn: 326153
Currently we assert that only non target specific opcodes can have
missing RegisterClass constraints in the MCDesc. The backend can have
instructions with register operands but don't have RegisterClass
constraints (say using unknown_class) in which case the instruction
defining the register will constrain it.
Change the assert to only fire if a def has no regclass.
https://reviews.llvm.org/D43409
llvm-svn: 326142
In r322867, we introduced IsStandalone when printing MIR in -debug
output. The default behaviour for that was:
1) If any of MBB, MI, or MO are -debug-printed separately, don't omit any
redundant information.
2) When -debug-printing a MF entirely, don't print any redundant
information.
3) When printing MIR, don't print any redundant information.
I'd like to change 2) to:
2) When -debug-printing a MF entirely, don't omit any redundant information.
Differential Revision: https://reviews.llvm.org/D43337
llvm-svn: 326094
Summary:
In the test case, the machine scheduler moves a dead write to a subreg
up into the middle of a segment of the overall reg's live range, where
the segment had liveness only for other subregs in the reg.
handleMoveUp created an invalid live range, causing an assert a bit
later.
This commit fixes it to handle that situation. The segment is split in
two at the insertion point, and the part after the split, and any
subsequent segments up to the old position, are changed to be defined by
the moved def.
V2: Better test.
Subscribers: MatzeB, nhaehnle, llvm-commits
Differential Revision: https://reviews.llvm.org/D43478
Change-Id: Ibc42445ddca84e79ad1f616401015d22bc63832e
llvm-svn: 326087
In DWARF v5 the Line Number Program Header is extensible, allowing values with
new content types. In this extension a content type is added,
DW_LNCT_LLVM_source, which contains the embedded source code of the file.
Add new optional attribute for !DIFile IR metadata called source which contains
source text. Use this to output the source to the DWARF line table of code
objects. Analogously extend METADATA_FILE in Bitcode and .file directive in ASM
to support optional source.
Teach llvm-dwarfdump and llvm-objdump about the new values. Update the output
format of llvm-dwarfdump to make room for the new attribute on file_names
entries, and support embedded sources for the -source option in llvm-objdump.
Differential Revision: https://reviews.llvm.org/D42765
llvm-svn: 325970
Summary:
Add a target option AllowRegisterRenaming that is used to opt in to
post-register-allocation renaming of registers. This is set to 0 by
default, which causes the hasExtraSrcRegAllocReq/hasExtraDstRegAllocReq
fields of all opcodes to be set to 1, causing
MachineOperand::isRenamable to always return false.
Set the AllowRegisterRenaming flag to 1 for all in-tree targets that
have lit tests that were effected by enabling COPY forwarding in
MachineCopyPropagation (AArch64, AMDGPU, ARM, Hexagon, Mips, PowerPC,
RISCV, Sparc, SystemZ and X86).
Add some more comments describing the semantics of the
MachineOperand::isRenamable function and how it is set and maintained.
Change isRenamable to check the operand's opcode
hasExtraSrcRegAllocReq/hasExtraDstRegAllocReq bit directly instead of
relying on it being consistently reflected in the IsRenamable bit
setting.
Clear the IsRenamable bit when changing an operand's register value.
Remove target code that was clearing the IsRenamable bit when changing
registers/opcodes now that this is done conservatively by default.
Change setting of hasExtraSrcRegAllocReq in AMDGPU target to be done in
one place covering all opcodes that have constant pipe read limit
restrictions.
Reviewers: qcolombet, MatzeB
Subscribers: aemerson, arsenm, jyknight, mcrosier, sdardis, nhaehnle, javed.absar, tpr, arichardson, kristof.beyls, kbarton, fedor.sergeev, asb, rbar, johnrusso, simoncook, jordy.potman.lists, apazos, sabuasal, niosHD, escha, nemanjai, llvm-commits
Differential Revision: https://reviews.llvm.org/D43042
llvm-svn: 325931
Summary:
There are transformation that change setcc into other constructs, and transform that try to reconstruct a setcc from the brcond condition. Depending on what order these transform are done, the end result differs.
Most of the time, it is preferable to get a setcc as a brcond argument (and this is why brcond try to recreate the setcc in the first place) so we ensure this is done every time by also doing it at the setcc level when the only user is a brcond.
Reviewers: spatel, hfinkel, niravd, craig.topper
Subscribers: nhaehnle, llvm-commits
Differential Revision: https://reviews.llvm.org/D41235
llvm-svn: 325892
isCondCodeLegal internally checked Legal or Custom which is misleading. Though no targets set any cond code action to Custom today.
So I've renamed isCondCodeLegal to isCondCodeLegalOrCustom and added a real isCondCodeLegal that only checks Legal.
I've changed legalization code to use isCondCodeLegalOrCustom and left things reachable via DAG combine as isCondCodeLegal. I've also changed some places that called getCondCodeAction and compared to Legal to just use isCondCodeLegal.
I'm looking at trying to keep SETCC all the way to isel for the AVX512 integer comparisons and I suspect I'll need to make some condition codes Custom to stop DAG combine from changing things post LegalizeOps. Prior to this only Expand stopped DAG combine, but that causes LegalizeOps to try to swap operands or invert rather than calling our Custom handler.
Differential Revision: https://reviews.llvm.org/D43607
llvm-svn: 325829
Roorda, Jan-Willem