Commit Graph

43450 Commits

Author SHA1 Message Date
Michael Kruse 86abc9a119 [OpenMPIRBuilder] Fix typo. NFC. 2020-12-11 00:15:23 -06:00
Arthur Eubanks c80e193587 [NFC] Inline maxDevirtIterationsReached()
This was separated in the past because the cl::opt was in the .cpp file
but DevirtSCCRepeatedPass::run() was in the .h file. Now that
DevirtSCCRepeatedPass::run() is in the .cpp file, get rid of the tiny
maxDevirtIterationsReached(), it's bad for readability.
2020-12-10 22:12:29 -08:00
Kazu Hirata fd78012b9e [MemorySSA] Remove unused declaration optimizeUses (NFC)
The declaration was introduced on Aug 2, 2016 in commit
c43aa5a5b6 without a corresponding
definition.

Note that we do have a definition for
MmeorySSA::OptimizeUses::optimizeUses but not for
MmeorySSA::optimizeUses.
2020-12-10 20:54:37 -08:00
Hongtao Yu 705a4c149d [CSSPGO] Pseudo probe encoding and emission.
This change implements pseudo probe encoding and emission for CSSPGO. Please see RFC here for more context: https://groups.google.com/g/llvm-dev/c/1p1rdYbL93s

Pseudo probes are in the form of intrinsic calls on IR/MIR but they do not turn into any machine instructions. Instead they are emitted into the binary as a piece of data in standalone sections.  The probe-specific sections are not needed to be loaded into memory at execution time, thus they do not incur a runtime overhead. 

**ELF object emission**

The binary data to emit are organized as two ELF sections, i.e, the `.pseudo_probe_desc` section and the `.pseudo_probe` section. The `.pseudo_probe_desc` section stores a function descriptor for each function and the `.pseudo_probe` section stores the actual probes, each fo which corresponds to an IR basic block or an IR function callsite. A function descriptor is stored as a module-level metadata during the compilation and is serialized into the object file during object emission.

Both the probe descriptors and pseudo probes can be emitted into a separate ELF section per function to leverage the linker for deduplication.  A `.pseudo_probe` section shares the same COMDAT group with the function code so that when the function is dead, the probes are dead and disposed too. On the contrary, a `.pseudo_probe_desc` section has its own COMDAT group. This is because even if a function is dead, its probes may be inlined into other functions and its descriptor is still needed by the profile generation tool.

The format of `.pseudo_probe_desc` section looks like:

```
.section   .pseudo_probe_desc,"",@progbits
.quad   6309742469962978389  // Func GUID
.quad   4294967295           // Func Hash
.byte   9                    // Length of func name
.ascii  "_Z5funcAi"          // Func name
.quad   7102633082150537521
.quad   138828622701
.byte   12
.ascii  "_Z8funcLeafi"
.quad   446061515086924981
.quad   4294967295
.byte   9
.ascii  "_Z5funcBi"
.quad   -2016976694713209516
.quad   72617220756
.byte   7
.ascii  "_Z3fibi"
```

For each `.pseudoprobe` section, the encoded binary data consists of a single function record corresponding to an outlined function (i.e, a function with a code entry in the `.text` section). A function record has the following format :

```
FUNCTION BODY (one for each outlined function present in the text section)
    GUID (uint64)
        GUID of the function
    NPROBES (ULEB128)
        Number of probes originating from this function.
    NUM_INLINED_FUNCTIONS (ULEB128)
        Number of callees inlined into this function, aka number of
        first-level inlinees
    PROBE RECORDS
        A list of NPROBES entries. Each entry contains:
          INDEX (ULEB128)
          TYPE (uint4)
            0 - block probe, 1 - indirect call, 2 - direct call
          ATTRIBUTE (uint3)
            reserved
          ADDRESS_TYPE (uint1)
            0 - code address, 1 - address delta
          CODE_ADDRESS (uint64 or ULEB128)
            code address or address delta, depending on ADDRESS_TYPE
    INLINED FUNCTION RECORDS
        A list of NUM_INLINED_FUNCTIONS entries describing each of the inlined
        callees.  Each record contains:
          INLINE SITE
            GUID of the inlinee (uint64)
            ID of the callsite probe (ULEB128)
          FUNCTION BODY
            A FUNCTION BODY entry describing the inlined function.
```

To support building a context-sensitive profile, probes from inlinees are grouped by their inline contexts. An inline context is logically a call path through which a callee function lands in a caller function. The probe emitter builds an inline tree based on the debug metadata for each outlined function in the form of a trie tree. A tree root is the outlined function. Each tree edge stands for a callsite where inlining happens. Pseudo probes originating from an inlinee function are stored in a tree node and the tree path starting from the root all the way down to the tree node is the inline context of the probes. The emission happens on the whole tree top-down recursively. Probes of a tree node will be emitted altogether with their direct parent edge. Since a pseudo probe corresponds to a real code address, for size savings, the address is encoded as a delta from the previous probe except for the first probe. Variant-sized integer encoding, aka LEB128, is used for address delta and probe index.

**Assembling**

Pseudo probes can be printed as assembly directives alternatively. This allows for good assembly code readability and also provides a view of how optimizations and pseudo probes affect each other, especially helpful for diff time assembly analysis.

A pseudo probe directive has the following operands in order: function GUID, probe index, probe type, probe attributes and inline context. The directive is generated by the compiler and can be parsed by the assembler to form an encoded `.pseudoprobe` section in the object file.

A example assembly looks like:

```
foo2: # @foo2
# %bb.0: # %bb0
pushq %rax
testl %edi, %edi
.pseudoprobe 837061429793323041 1 0 0
je .LBB1_1
# %bb.2: # %bb2
.pseudoprobe 837061429793323041 6 2 0
callq foo
.pseudoprobe 837061429793323041 3 0 0
.pseudoprobe 837061429793323041 4 0 0
popq %rax
retq
.LBB1_1: # %bb1
.pseudoprobe 837061429793323041 5 1 0
callq *%rsi
.pseudoprobe 837061429793323041 2 0 0
.pseudoprobe 837061429793323041 4 0 0
popq %rax
retq
# -- End function
.section .pseudo_probe_desc,"",@progbits
.quad 6699318081062747564
.quad 72617220756
.byte 3
.ascii "foo"
.quad 837061429793323041
.quad 281547593931412
.byte 4
.ascii "foo2"
```

With inlining turned on, the assembly may look different around %bb2 with an inlined probe:

```
# %bb.2:                                # %bb2
.pseudoprobe    837061429793323041 3 0
.pseudoprobe    6699318081062747564 1 0 @ 837061429793323041:6
.pseudoprobe    837061429793323041 4 0
popq    %rax
retq
```

**Disassembling**

We have a disassembling tool (llvm-profgen) that can display disassembly alongside with pseudo probes. So far it only supports ELF executable file.

An example disassembly looks like:

```
00000000002011a0 <foo2>:
  2011a0: 50                    push   rax
  2011a1: 85 ff                 test   edi,edi
  [Probe]:  FUNC: foo2  Index: 1  Type: Block
  2011a3: 74 02                 je     2011a7 <foo2+0x7>
  [Probe]:  FUNC: foo2  Index: 3  Type: Block
  [Probe]:  FUNC: foo2  Index: 4  Type: Block
  [Probe]:  FUNC: foo   Index: 1  Type: Block  Inlined: @ foo2:6
  2011a5: 58                    pop    rax
  2011a6: c3                    ret
  [Probe]:  FUNC: foo2  Index: 2  Type: Block
  2011a7: bf 01 00 00 00        mov    edi,0x1
  [Probe]:  FUNC: foo2  Index: 5  Type: IndirectCall
  2011ac: ff d6                 call   rsi
  [Probe]:  FUNC: foo2  Index: 4  Type: Block
  2011ae: 58                    pop    rax
  2011af: c3                    ret
```

Reviewed By: wmi

Differential Revision: https://reviews.llvm.org/D91878
2020-12-10 17:29:28 -08:00
Mitch Phillips 7ead5f5aa3 Revert "[CSSPGO] Pseudo probe encoding and emission."
This reverts commit b035513c06.

Reason: Broke the ASan buildbots:
  http://lab.llvm.org:8011/#/builders/5/builds/2269
2020-12-10 15:53:39 -08:00
Xinhao Yuan 97260ab478 [llvm-cov][gcov] Optimize the cycle counting algorithm by skipping zero count cycles
This change is similar to http://gcc.gnu.org/PR90380

This reduces the complexity from exponential to polynomial of the arcs.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D93036
2020-12-10 15:22:29 -08:00
Scott Linder 10b5eaed91 [SmallVector] Copy new docs into Doxygen comment
Copy the `ProgrammersManual.rst` changes from D92522 to the Doxygen
comment for `SmallVector`, to hopefully encourage new uses migrating to
the no-explicit-`N` form.

Differential Revision: https://reviews.llvm.org/D93069
2020-12-10 22:20:37 +00:00
Arthur Eubanks ff7e1da68f [NPM] Support -fmerge-functions
I tried to put it in the same place in the pipeline as the legacy PM.

Fixes PR48399.

Reviewed By: asbirlea, nikic

Differential Revision: https://reviews.llvm.org/D93002
2020-12-10 11:45:08 -08:00
Hongtao Yu b035513c06 [CSSPGO] Pseudo probe encoding and emission.
This change implements pseudo probe encoding and emission for CSSPGO. Please see RFC here for more context: https://groups.google.com/g/llvm-dev/c/1p1rdYbL93s

Pseudo probes are in the form of intrinsic calls on IR/MIR but they do not turn into any machine instructions. Instead they are emitted into the binary as a piece of data in standalone sections.  The probe-specific sections are not needed to be loaded into memory at execution time, thus they do not incur a runtime overhead. 

**ELF object emission**

The binary data to emit are organized as two ELF sections, i.e, the `.pseudo_probe_desc` section and the `.pseudo_probe` section. The `.pseudo_probe_desc` section stores a function descriptor for each function and the `.pseudo_probe` section stores the actual probes, each fo which corresponds to an IR basic block or an IR function callsite. A function descriptor is stored as a module-level metadata during the compilation and is serialized into the object file during object emission.

Both the probe descriptors and pseudo probes can be emitted into a separate ELF section per function to leverage the linker for deduplication.  A `.pseudo_probe` section shares the same COMDAT group with the function code so that when the function is dead, the probes are dead and disposed too. On the contrary, a `.pseudo_probe_desc` section has its own COMDAT group. This is because even if a function is dead, its probes may be inlined into other functions and its descriptor is still needed by the profile generation tool.

The format of `.pseudo_probe_desc` section looks like:

```
.section   .pseudo_probe_desc,"",@progbits
.quad   6309742469962978389  // Func GUID
.quad   4294967295           // Func Hash
.byte   9                    // Length of func name
.ascii  "_Z5funcAi"          // Func name
.quad   7102633082150537521
.quad   138828622701
.byte   12
.ascii  "_Z8funcLeafi"
.quad   446061515086924981
.quad   4294967295
.byte   9
.ascii  "_Z5funcBi"
.quad   -2016976694713209516
.quad   72617220756
.byte   7
.ascii  "_Z3fibi"
```

For each `.pseudoprobe` section, the encoded binary data consists of a single function record corresponding to an outlined function (i.e, a function with a code entry in the `.text` section). A function record has the following format :

```
FUNCTION BODY (one for each outlined function present in the text section)
    GUID (uint64)
        GUID of the function
    NPROBES (ULEB128)
        Number of probes originating from this function.
    NUM_INLINED_FUNCTIONS (ULEB128)
        Number of callees inlined into this function, aka number of
        first-level inlinees
    PROBE RECORDS
        A list of NPROBES entries. Each entry contains:
          INDEX (ULEB128)
          TYPE (uint4)
            0 - block probe, 1 - indirect call, 2 - direct call
          ATTRIBUTE (uint3)
            reserved
          ADDRESS_TYPE (uint1)
            0 - code address, 1 - address delta
          CODE_ADDRESS (uint64 or ULEB128)
            code address or address delta, depending on ADDRESS_TYPE
    INLINED FUNCTION RECORDS
        A list of NUM_INLINED_FUNCTIONS entries describing each of the inlined
        callees.  Each record contains:
          INLINE SITE
            GUID of the inlinee (uint64)
            ID of the callsite probe (ULEB128)
          FUNCTION BODY
            A FUNCTION BODY entry describing the inlined function.
```

To support building a context-sensitive profile, probes from inlinees are grouped by their inline contexts. An inline context is logically a call path through which a callee function lands in a caller function. The probe emitter builds an inline tree based on the debug metadata for each outlined function in the form of a trie tree. A tree root is the outlined function. Each tree edge stands for a callsite where inlining happens. Pseudo probes originating from an inlinee function are stored in a tree node and the tree path starting from the root all the way down to the tree node is the inline context of the probes. The emission happens on the whole tree top-down recursively. Probes of a tree node will be emitted altogether with their direct parent edge. Since a pseudo probe corresponds to a real code address, for size savings, the address is encoded as a delta from the previous probe except for the first probe. Variant-sized integer encoding, aka LEB128, is used for address delta and probe index.

**Assembling**

Pseudo probes can be printed as assembly directives alternatively. This allows for good assembly code readability and also provides a view of how optimizations and pseudo probes affect each other, especially helpful for diff time assembly analysis.

A pseudo probe directive has the following operands in order: function GUID, probe index, probe type, probe attributes and inline context. The directive is generated by the compiler and can be parsed by the assembler to form an encoded `.pseudoprobe` section in the object file.

A example assembly looks like:

```
foo2: # @foo2
# %bb.0: # %bb0
pushq %rax
testl %edi, %edi
.pseudoprobe 837061429793323041 1 0 0
je .LBB1_1
# %bb.2: # %bb2
.pseudoprobe 837061429793323041 6 2 0
callq foo
.pseudoprobe 837061429793323041 3 0 0
.pseudoprobe 837061429793323041 4 0 0
popq %rax
retq
.LBB1_1: # %bb1
.pseudoprobe 837061429793323041 5 1 0
callq *%rsi
.pseudoprobe 837061429793323041 2 0 0
.pseudoprobe 837061429793323041 4 0 0
popq %rax
retq
# -- End function
.section .pseudo_probe_desc,"",@progbits
.quad 6699318081062747564
.quad 72617220756
.byte 3
.ascii "foo"
.quad 837061429793323041
.quad 281547593931412
.byte 4
.ascii "foo2"
```

With inlining turned on, the assembly may look different around %bb2 with an inlined probe:

```
# %bb.2:                                # %bb2
.pseudoprobe    837061429793323041 3 0
.pseudoprobe    6699318081062747564 1 0 @ 837061429793323041:6
.pseudoprobe    837061429793323041 4 0
popq    %rax
retq
```

**Disassembling**

We have a disassembling tool (llvm-profgen) that can display disassembly alongside with pseudo probes. So far it only supports ELF executable file.

An example disassembly looks like:

```
00000000002011a0 <foo2>:
  2011a0: 50                    push   rax
  2011a1: 85 ff                 test   edi,edi
  [Probe]:  FUNC: foo2  Index: 1  Type: Block
  2011a3: 74 02                 je     2011a7 <foo2+0x7>
  [Probe]:  FUNC: foo2  Index: 3  Type: Block
  [Probe]:  FUNC: foo2  Index: 4  Type: Block
  [Probe]:  FUNC: foo   Index: 1  Type: Block  Inlined: @ foo2:6
  2011a5: 58                    pop    rax
  2011a6: c3                    ret
  [Probe]:  FUNC: foo2  Index: 2  Type: Block
  2011a7: bf 01 00 00 00        mov    edi,0x1
  [Probe]:  FUNC: foo2  Index: 5  Type: IndirectCall
  2011ac: ff d6                 call   rsi
  [Probe]:  FUNC: foo2  Index: 4  Type: Block
  2011ae: 58                    pop    rax
  2011af: c3                    ret
```

Reviewed By: wmi

Differential Revision: https://reviews.llvm.org/D91878
2020-12-10 09:50:08 -08:00
Florian Hahn bb9cef7628
[CallBase] Add hasRetAttr version that takes StringRef.
This makes it slightly easier to deal with custom attributes and
CallBase already provides hasFnAttr versions that support both AttrKind
and StringRef arguments in a similar fashion.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D92567
2020-12-10 17:00:16 +00:00
Irina Dobrescu c9e967af3f [flang]Add Parser Support for Allocate Directive
Differential Revision: https://reviews.llvm.org/D89562
2020-12-10 16:21:19 +00:00
clementval 456c885df3 Revert "[openmp] Remove clause from OMPKinds.def and use OMP.td info"
This reverts commit a7b2847216.

failing buildbot on warnings
2020-12-10 10:34:59 -05:00
Nuno Lopes d2a7b83c5c AA: make AliasAnalysis.h compatible with C++20 (NFC)
can't mix arithmetic with different enums
2020-12-10 15:32:11 +00:00
Valentin Clement a7b2847216 [openmp] Remove clause from OMPKinds.def and use OMP.td info
Remove the OpenMP clause information from the OMPKinds.def file and use the
information from the new OMP.td file. There is now a single source of truth for the
directives and clauses.

To avoid generate lots of specific small code from tablegen, the macros previously
used in OMPKinds.def are generated almost as identical. This can be polished and
possibly removed in a further patch.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D92955
2020-12-10 10:19:09 -05:00
Kazushi (Jam) Marukawa 4b1e329255 [VE] Add vector reduce intrinsic instructions
Add vrmax, vrmin, vfrmax, vfrmin, vrand, vror, and vrxor intrinsic
instructions and regression tests.

Reviewed By: simoll

Differential Revision: https://reviews.llvm.org/D92941
2020-12-10 22:21:17 +09:00
Sjoerd Meijer 99ad078b91 [AArch64] Cortex-R82: remove crypto
Remove target features crypto for Cortex-R82, because it doesn't have any, and
add LSE which was missing while we are at it.
This also removes crypto from the v8-R architecture description because that
aligns better with GCC and so far none of the R-cores have implemented crypto,
so is probably a more sensible default.

Differential Revision: https://reviews.llvm.org/D91994
2020-12-10 12:54:51 +00:00
David Green 0447f3508f [ARM][RegAlloc] Add t2LoopEndDec
We currently have problems with the way that low overhead loops are
specified, with LR being spilled between the t2LoopDec and the t2LoopEnd
forcing the entire loop to be reverted late in the backend. As they will
eventually become a single instruction, this patch introduces a
t2LoopEndDec which is the combination of the two, combined before
registry allocation to make sure this does not fail.

Unfortunately this instruction is a terminator that produces a value
(and also branches - it only produces the value around the branching
edge). So this needs some adjustment to phi elimination and the register
allocator to make sure that we do not spill this LR def around the loop
(needing to put a spill after the terminator). We treat the loop very
carefully, making sure that there is nothing else like calls that would
break it's ability to use LR. For that, this adds a
isUnspillableTerminator to opt in the new behaviour.

There is a chance that this could cause problems, and so I have added an
escape option incase. But I have not seen any problems in the testing
that I've tried, and not reverting Low overhead loops is important for
our performance. If this does work then we can hopefully do the same for
t2WhileLoopStart and t2DoLoopStart instructions.

This patch also contains the code needed to convert or revert the
t2LoopEndDec in the backend (which just needs a subs; bne) and the code
pre-ra to create them.

Differential Revision: https://reviews.llvm.org/D91358
2020-12-10 12:14:23 +00:00
Alexey Lapshin 693da9df74 [dsymutil][DWARFLinker][NFC] Make interface of AddressMap more general.
Current interface of AddressMap assumes that relocations exist.
That is correct for not-linked object file but is not correct
for linked executable. This patch changes interface in such way
that AddressMap could be used not only with not-linked object files:

hasValidRelocationAt()

replaced with:

hasLiveMemoryLocation()
hasLiveAddressRange()

Differential Revision: https://reviews.llvm.org/D87723
2020-12-10 14:57:08 +03:00
Benjamin Kramer 60806e856a Remove Shapet assignment operator that's identical to the default. NFC. 2020-12-10 10:58:41 +01:00
Luo, Yuanke f80b29878b [X86] AMX programming model.
This patch implements amx programming model that discussed in llvm-dev
 (http://lists.llvm.org/pipermail/llvm-dev/2020-August/144302.html).
 Thank Hal for the good suggestion in the RA. The fast RA is not in the patch yet.
 This patch implemeted 7 components.

1. The c interface to end user.
2. The AMX intrinsics in LLVM IR.
3. Transform load/store <256 x i32> to AMX intrinsics or split the
   type into two <128 x i32>.
4. The Lowering from AMX intrinsics to AMX pseudo instruction.
5. Insert psuedo ldtilecfg and build the def-use between ldtilecfg to amx
   intruction.
6. The register allocation for tile register.
7. Morph AMX pseudo instruction to AMX real instruction.

Change-Id: I935e1080916ffcb72af54c2c83faa8b2e97d5cb0

Differential Revision: https://reviews.llvm.org/D87981
2020-12-10 17:01:54 +08:00
Arthur Eubanks cc15e75ce8 [LTO][NPM] Default to using NPM under ENABLE_EXPERIMENTAL_NEW_PASS_MANAGER
This affects users of LTO that don't explicitly set UseNewPM.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D92894
2020-12-09 17:48:47 -08:00
Sam Clegg 9a72d3e3e4 [WebAssembly] Add support for named data sections in wasm binaries
Followup to https://reviews.llvm.org/D91769 which added support
for names globals.

Differential Revision: https://reviews.llvm.org/D92909
2020-12-09 12:57:07 -08:00
Florian Hahn 77fd12a66e
[AArch64] Add aarch64_neon_vcmla{_rot{90,180,270}} intrinsics.
Add builtins required to implement vcmla and rotated variants from
the ACLE

Reviewed By: t.p.northover

Differential Revision: https://reviews.llvm.org/D92929
2020-12-09 19:46:49 +00:00
Justin Bogner e6a1187dd8 Limit the recursion depth of SelectionDAG::isSplatValue()
This method previously always recursively checked both the left-hand
side and right-hand side of binary operations for splatted (broadcast)
vector values to determine if the parent DAG node is a splat.

Like several other SelectionDAG methods, limit the recursion depth to
MaxRecursionDepth (6). This prevents stack overflow.
See also https://issuetracker.google.com/173785481

Patch by Nicolas Capens. Thanks!

Differential Revision: https://reviews.llvm.org/D92421
2020-12-09 10:35:07 -08:00
Kazushi (Jam) Marukawa 1a2147fead [VE] Add vsum and vfsum intrinsic instructions
Add vsum and vfsum intrinsic instructions and regression tests.

Reviewed By: simoll

Differential Revision: https://reviews.llvm.org/D92938
2020-12-10 01:11:53 +09:00
Paul C. Anagnostopoulos 6266f36226 [TableGen] Cache the vectors of records returned by getAllDerivedDefinitions().
Differential Revision: https://reviews.llvm.org/D92674
2020-12-09 10:54:04 -05:00
Kazushi (Jam) Marukawa 398f29fbb0 [VE] Add vfmk intrinsic instructions
Add vfmk intrinsic instructions, a few pseudo instructions to expand
vfmk intrinsic using VM512 correctly, and regression tests.

Reviewed By: simoll

Differential Revision: https://reviews.llvm.org/D92758
2020-12-10 00:08:20 +09:00
Sander de Smalen d568cff696 [LoopVectorizer][SVE] Vectorize a simple loop with with a scalable VF.
* Steps are scaled by `vscale`, a runtime value.
* Changes to circumvent the cost-model for now (temporary)
  so that the cost-model can be implemented separately.

This can vectorize the following loop [1]:

   void loop(int N, double *a, double *b) {
     #pragma clang loop vectorize_width(4, scalable)
     for (int i = 0; i < N; i++) {
       a[i] = b[i] + 1.0;
     }
   }

[1] This source-level example is based on the pragma proposed
separately in D89031. This patch only implements the LLVM part.

Reviewed By: dmgreen

Differential Revision: https://reviews.llvm.org/D91077
2020-12-09 11:25:21 +00:00
Kerry McLaughlin 4519ff4b6f [SVE][CodeGen] Add the ExtensionType flag to MGATHER
Adds the ExtensionType flag, which reflects the LoadExtType of a MaskedGatherSDNode.
Also updated SelectionDAGDumper::print_details so that details of the gather
load (is signed, is scaled & extension type) are printed.

Reviewed By: sdesmalen

Differential Revision: https://reviews.llvm.org/D91084
2020-12-09 11:19:08 +00:00
Joe Ellis 80c33de2d3 [SelectionDAG] Add llvm.vector.{extract,insert} intrinsics
This commit adds two new intrinsics.

- llvm.experimental.vector.insert: used to insert a vector into another
  vector starting at a given index.

- llvm.experimental.vector.extract: used to extract a subvector from a
  larger vector starting from a given index.

The codegen work for these intrinsics has already been completed; this
commit is simply exposing the existing ISD nodes to LLVM IR.

Reviewed By: cameron.mcinally

Differential Revision: https://reviews.llvm.org/D91362
2020-12-09 11:08:41 +00:00
Simon Moll 3ffbc79357 [VP] Build VP SDNodes
Translate VP intrinsics to VP_* SDNodes.  The tests check whether a
matching vp_* SDNode is emitted.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D91441
2020-12-09 11:36:51 +01:00
Roman Lebedev a2876ec74f
[NFC][Instructions] Refactor CmpInst::getFlippedStrictnessPredicate() in terms of is{,Non}StrictPredicate()/get{Non,}StrictPredicate()
In particular, this creates getStrictPredicate() method,
to be symmetrical with already-existing getNonStrictPredicate().
2020-12-09 12:43:08 +03:00
Kazu Hirata f57d4c29da [MemorySSA] Remove unused declaration determineInsertionPoint (NFC)
The declaration was introduced without a corresponding definition on
Feb 2, 2016 in commit e1100f533f.
2020-12-08 19:21:44 -08:00
Duncan P. N. Exon Smith 75cd8d756d Support: Add RedirectingFileSystem::create from simple list of redirections
Add an overload of `RedirectingFileSystem::create` that builds a
redirecting filesystem off of a simple vector of string pairs. This is
intended to be used to support `clang::arcmt::FileRemapper` and
`clang::PreprocessorOptions::RemappedFiles`.

Differential Revision: https://reviews.llvm.org/D91317
2020-12-08 17:53:30 -08:00
Duncan P. N. Exon Smith a22eda548b VFS: Return new file systems as uniquely owned when possible, almost NFC
Uniformly return uniquely-owned filesystems from VFS creation APIs. The
one exception is `getRealFileSystem`, which has a single instance and
needs to be shared.

This is almost NFC, except that it fixes a memory leak in
`vfs::collectVFSFromYAML()`.

Depends on https://reviews.llvm.org/D92888

Differential Revision: https://reviews.llvm.org/D92890
2020-12-08 17:33:46 -08:00
Duncan P. N. Exon Smith 5207f19d10 ADT: Allow IntrusiveRefCntPtr construction from std::unique_ptr, NFC
Allow a `std::unique_ptr` to be moved into the an `IntrusiveRefCntPtr`,
and remove a couple of now-unnecessary `release()` calls.

Differential Revision: https://reviews.llvm.org/D92888
2020-12-08 17:33:19 -08:00
Wei Mi 64e7685368 [SampleFDO] Store fixed length MD5 in NameTable instead of using ULEB128 if
MD5 is used.

Currently during sample profile loading, NameTable has to be loaded entirely
up front before any name string is retrieved. That is because NameTable is
stored using ULEB128 encoding and cannot be directly accessed like an array.
However, if MD5 is used to represent name in the NameTable, it has fixed
length. If MD5 names are stored in uint64_t type instead of ULEB128, NameTable
can be accessed like an array then in many cases only part of the NameTable
has to be read. This is helpful for reducing compile time especially when
small source file is compiled. We find that after this change, the elapsed
time to build a large application distributively is reduced by 5% and the
accumulative cpu time used for building is also reduced by 5%. The size of
the profile is slightly reduced with this change by ~0.2%, and that also
indicates encoding MD5 in ULEB128 doesn't save the storage space.

Differential Revision: https://reviews.llvm.org/D92621
2020-12-08 16:21:01 -08:00
Ilya Leoshkevich d58f112ce0 Prevent FENTRY_CALL reordering
FEntryInserter prepends FENTRY_CALL to the first basic block. In case
there are other instructions, PostRA Machine Instruction Scheduler can
move FENTRY_CALL call around. This actually occurs on SystemZ (see the
testcase). This is bad for the following reasons:

* FENTRY_CALL clobbers registers.
* Linux Kernel depends on whatever FENTRY_CALL expands to to be the very
  first instruction in the function.

Fix by adding isCall attribute to FENTRY_CALL, which prevents reordering
by making it a scheduling boundary for PostRA Machine Instruction
Scheduler.

Reviewed By: niravd

Differential Revision: https://reviews.llvm.org/D91218
2020-12-09 00:59:01 +01:00
Duncan P. N. Exon Smith caab41c08e ADT: Add hash_value overload for Optional
Add a `hash_value` for Optional so that other data structures with
optional fields can easily hash them. I have a use for this in an
upcoming patch.

Differential Revision: https://reviews.llvm.org/D92676
2020-12-08 15:25:03 -08:00
Anna Thomas 29356e3279 [ScalarizeMaskedMemIntrin] Add new PM support
This patch adds new PM support for the pass and the pass can be now used
during middle-end transforms. The old pass is remamed to
ScalarizeMaskedMemIntrinLegacyPass.

Reviewed-By: skatkov, aeubanks
Differential Revision: https://reviews.llvm.org/D92743
2020-12-08 17:15:22 -05:00
Saleem Abdulrasool 939c8f676c Bitcode: add some convenience helpers for streaming bitcode
This adds a set of metaprogramming helpers to help define records and
serialize them out.  This is motivated by API Notes which use the
bitcode format to serialize out a binary representation of the data.
These helpers are generically useful though and could help simplify some
of the existing bitcode consumers as well.

This is extracted from the code contributed by Apple at
https://github.com/llvm/llvm-project-staging/tree/staging/swift/apinotes.

Differential Revision: https://reviews.llvm.org/D88582
2020-12-08 21:15:41 +00:00
Valentin Clement d553243fe4 [flang][openacc] Update reference to OpenACC 3.1 specification
Update all reference from the specification to the new OpenACC 3.1
document.

Reviewed By: SouraVX

Differential Revision: https://reviews.llvm.org/D92120
2020-12-08 14:36:38 -05:00
Nuno Lopes 3c01af9aee DenseMap: fix build with clang in C++20 mode
clang was complaing about this code:
llvm/include/llvm/IR/PassManager.h:715:17: error: ISO C++20 considers use of overloaded operator '!=' to be ambiguous despite there being a unique best viable function with non-reversed arguments [-Werror,-Wambiguous-reversed-operator]
      if (IMapI != IsResultInvalidated.end())
          ~~~~~ ^  ~~~~~~~~~~~~~~~~~~~~~~~~~
llvm/include/llvm/ADT/DenseMap.h:1253:8: note: candidate function with non-reversed arguments
  bool operator!=(const ConstIterator &RHS) const {
       ^
llvm/include/llvm/ADT/DenseMap.h:1246:8: note: ambiguous candidate function with reversed arguments
  bool operator==(const ConstIterator &RHS) const {
       ^

The warning is triggered when the DenseMapIterator (lhs) is not const and so
the == operator is applied to different types on lhs/rhs.
Using a template allows the function to be available for both const/non-const
iterator types and gets rid of the warning
2020-12-08 18:39:31 +00:00
Yuanfang Chen 1821265db6 [Time-report] Add a flag -ftime-report={per-pass,per-pass-run} to control the pass timing aggregation
Currently, -ftime-report + new pass manager emits one line of report for each
pass run. This potentially causes huge output text especially with regular LTO
or large single file (Obeserved in private tests and was reported in D51276).
The behaviour of -ftime-report + legacy pass manager is
emitting one line of report for each pass object which has relatively reasonable
text output size. This patch adds a flag `-ftime-report=` to control time report
aggregation for new pass manager.

The flag is for new pass manager only. Using it with legacy pass manager gives
an error. It is a driver and cc1 flag. `per-pass` is the new default so
`-ftime-report` is aliased to `-ftime-report=per-pass`. Before this patch,
functionality-wise `-ftime-report` is aliased to `-ftime-report=per-pass-run`.

* Adds an boolean variable TimePassesHandler::PerRun to control per-pass vs per-pass-run.
* Adds a new clang CodeGen flag CodeGenOptions::TimePassesPerRun to work with the existing CodeGenOptions::TimePasses.
* Remove FrontendOptions::ShowTimers, its uses are replaced by the existing CodeGenOptions::TimePasses.
* Remove FrontendTimesIsEnabled (It was introduced in D45619 which was largely reverted.)

Differential Revision: https://reviews.llvm.org/D92436
2020-12-08 10:13:19 -08:00
Benjamin Kramer 5f18e2f31e Move createScalarizeMaskedMemIntrinPass to Scalar.h 2020-12-08 19:08:09 +01:00
Xun Li 31e60b9133 [coroutine] should disable inline before calling coro split
This is a rework of D85812, which didn't land.
When callee coroutine function is inlined into caller coroutine function before coro-split pass, llvm will emits "coroutine should have exactly one defining @llvm.coro.begin". It seems that coro-early pass can not handle this quiet well.
So we believe that unsplited coroutine function should not be inlined.
This patch fix such issue by not inlining function if it has attribute "coroutine.presplit" (it means the function has not been splited) to fix this issue
test plan: check-llvm, check-clang

In D85812, there was suggestions on moving the macros to Attributes.td to avoid circular header dependency issue.
I believe it's not worth doing just to be able to use one constant string in one place.
Today, there are already 3 possible attribute values for "coroutine.presplit": c6543cc6b8/llvm/lib/Transforms/Coroutines/CoroInternal.h (L40-L42)
If we move them into Attributes.td, we would be adding 3 new attributes to EnumAttr, just to support this, which I think is an overkill.

Instead, I think the best way to do this is to add an API in Function class that checks whether this function is a coroutine, by checking the attribute by name directly.

Differential Revision: https://reviews.llvm.org/D92706
2020-12-08 08:53:08 -08:00
Valentin Clement 1dd24e6ab7 [flang][openacc] Add clause validity tests for the update directive
Add couple of clause validity tests for the update directive and check for
the restriction where at least self, host or device clause must appear on the directive.

Reviewed By: sameeranjoshi

Differential Revision: https://reviews.llvm.org/D92447
2020-12-08 10:47:06 -05:00
Jan Svoboda 083e035c47 [clang][cli] Unify boolean marshalling
Use lambdas with captures to replace the redundant infrastructure for marshalling of two boolean flags that control the same keypath.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D92773
2020-12-08 13:47:30 +01:00
Pan, Tao 7af802994e [CodeGen] Add text section prefix for COFF object file
Text section prefix is created in CodeGenPrepare, it's file format independent implementation,  text section name is written into object file in TargetLoweringObjectFile, it's file format dependent implementation, port code of adding text section prefix to text section name from ELF to COFF.
Different with ELF that use '.' as concatenation character, COFF use '$' as concatenation character. That is, concatenation character is variable, so split concatenation character from text section prefix.
Text section prefix is existing feature of ELF, it can help to reduce icache and itlb misses, it's also make possible aggregate other compilers e.g. v8 created same prefix sections. Furthermore, the recent feature Machine Function Splitter (basic block level text prefix section) is based on text section prefix.

Reviewed By: pengfei, rnk

Differential Revision: https://reviews.llvm.org/D92073
2020-12-08 18:56:21 +08:00
Cullen Rhodes 2cfbdaf601 [IR] Remove CastInst::isCastable since it is not used
It was removed back in 2013 (f63dfbb) by Matt Arsenault but then
reverted since DragonEgg used it, but that project is no longer
maintained.

Reviewed By: ldionne, dexonsmith

Differential Revision: https://reviews.llvm.org/D92571
2020-12-08 10:31:53 +00:00
Tim Northover c5978f42ec UBSAN: emit distinctive traps
Sometimes people get minimal crash reports after a UBSAN incident. This change
tags each trap with an integer representing the kind of failure encountered,
which can aid in tracking down the root cause of the problem.
2020-12-08 10:28:26 +00:00
Qiu Chaofan 5e85a2ba16 [PowerPC] Implement intrinsic for DARN instruction
Instruction darn was introduced in ISA 3.0. It means 'Deliver A Random
Number'. The immediate number L means:

- L=0, the number is 32-bit (higher 32-bits are all-zero)
- L=1, the number is 'conditioned' (processed by hardware to reduce bias)
- L=2, the number is not conditioned, directly from noise source

GCC implements them in three separate intrinsics: __builtin_darn,
__builtin_darn_32 and __builtin_darn_raw. This patch implements the
same intrinsics. And this change also addresses Bugzilla PR39800.

Reviewed By: steven.zhang

Differential Revision: https://reviews.llvm.org/D92465
2020-12-08 14:08:52 +08:00
Kai Luo 44bd8ea167 [DAGCombine][PowerPC] Simplify nabs by using legal `smin` operation
Convert `0 - abs(x)` to `smin (x, -x)` if `smin` is a legal operation.

Verification: https://alive2.llvm.org/ce/z/vpquFR

Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D92637
2020-12-08 03:24:07 +00:00
Yaxun (Sam) Liu 40ad476a32 [clang][AMDGPU] rename sram-ecc as sramecc
As backend renamed sram-ecc to sramecc, this patch makes
corresponding change in clang.

Differential Revision: https://reviews.llvm.org/D86217
2020-12-07 18:05:47 -05:00
wlei 1f05b1a9f5 [CSSPGO][llvm-profgen] Context-sensitive profile data generation
This stack of changes introduces `llvm-profgen` utility which generates a profile data file from given perf script data files for sample-based PGO. It’s part of(not only) the CSSPGO work. Specifically to support context-sensitive with/without pseudo probe profile, it implements a series of functionalities including perf trace parsing, instruction symbolization, LBR stack/call frame stack unwinding, pseudo probe decoding, etc. Also high throughput is achieved by multiple levels of sample aggregation and compatible format with one stop is generated at the end. Please refer to: https://groups.google.com/g/llvm-dev/c/1p1rdYbL93s for the CSSPGO RFC.

This change supports context-sensitive profile data generation into llvm-profgen. With simultaneous sampling for LBR and call stack, we can identify leaf of LBR sample with calling context from stack sample . During the process of deriving fall through path from LBR entries, we unwind LBR by replaying all the calls and returns (including implicit calls/returns due to inlining) backwards on top of the sampled call stack. Then the state of call stack as we unwind through LBR always represents the calling context of current fall through path.

we have two types of virtual unwinding 1) LBR unwinding and 2) linear range unwinding.
Specifically, for each LBR entry which can be classified into call, return, regular branch, LBR unwinding will replay the operation by pushing, popping or switching leaf frame towards the call stack and since the initial call stack is most recently sampled, the replay should be in anti-execution order, i.e. for the regular case, pop the call stack when LBR is call, push frame on call stack when LBR is return. After each LBR processed, it also needs to align with the next LBR by going through instructions from previous LBR's target to current LBR's source, which we named linear unwinding. As instruction from linear range can come from different function by inlining, linear unwinding will do the range splitting and record counters through the range with same inline context.

With each fall through path from LBR unwinding, we aggregate each sample into counters by the calling context and eventually generate full context sensitive profile (without relying on inlining) to driver compiler's PGO/FDO.

A breakdown of noteworthy changes:
- Added `HybridSample` class as the abstraction perf sample including LBR stack and call stack
* Extended `PerfReader` to implement auto-detect whether input perf script output contains CS profile, then do the parsing. Multiple `HybridSample` are extracted
* Speed up by aggregating  `HybridSample` into `AggregatedSamples`
* Added VirtualUnwinder that consumes aggregated  `HybridSample` and implements unwinding of calls, returns, and linear path that contains implicit call/return from inlining. Ranges and branches counters are aggregated by the calling context.
 Here calling context is string type, each context is a pair of function name and callsite location info, the whole context is like `main:1 @ foo:2 @ bar`.
* Added PorfileGenerater that accumulates counters by ranges unfolding or branch target mapping, then generates context-sensitive function profile including function body, inferring callee's head sample, callsite target samples, eventually records into ProfileMap.

* Leveraged LLVM build-in(`SampleProfWriter`) writer to support different serialization format with no stop
- `getCanonicalFnName` for callee name and name from ELF section
- Added regression test for both unwinding and profile generation

Test Plan:
ninja & ninja check-llvm

Reviewed By: hoy, wenlei, wmi

Differential Revision: https://reviews.llvm.org/D89723
2020-12-07 13:48:58 -08:00
Alex Zinenko c102c783cd [OpenMPIRBuilder] introduce createStaticWorkshareLoop
Introduce a function that creates a statically-scheduled workshare loop
out of a canonical loop created earlier by the OpenMPIRBuilder. This
basically amounts to injecting runtime calls to the preheader and the
after block and updating the trip count. Static scheduling kind is
currently hardcoded and needs to be extracted from the runtime library
into common TableGen definitions.

Differential Revision: https://reviews.llvm.org/D92476
2020-12-07 22:30:59 +01:00
Nathan James a61d508473
[llvm][NFC] Made RefCountBase constructors protected
Matches ThreadSafeRefCountBase and forces the class to be inherited.
2020-12-07 20:23:11 +00:00
Nathan James dc361d5c2a
[llvm] Add asserts in (ThreadSafe)?RefCountedBase destructors
Added a trivial destructor in release mode and in debug mode a destructor that asserts RefCount is indeed zero.
This ensure people aren't manually (maybe accidentally) destroying these objects like in this contrived example.
```lang=c++
{
  std::unique_ptr<SomethingRefCounted> Object;
  holdIntrusiveOwnership(Object.get());
  // Object Destructor called here will assert.
}
```

Reviewed By: dblaikie

Differential Revision: https://reviews.llvm.org/D92480
2020-12-07 20:20:08 +00:00
Derek Schuff 0a391060f1 [WebAssembly] Add Object and ObjectWriter support for wasm COMDAT sections
Allow sections to be placed into COMDAT groups, in addtion to functions and data
segments.

Also make section symbols unnamed, which allows sections with identical names
(section names are independent of their section symbols, but previously we
gave the symbols the same name as their sections, which results in collisions
when sections are identically-named).

Differential Revision: https://reviews.llvm.org/D92691
2020-12-07 12:12:44 -08:00
Valentin Clement 875b4fd5ee [flang][openacc] Add clause validity tests for the kernels directive
Add some clause validity tests for the kernels directive

Reviewed By: sameeranjoshi

Differential Revision: https://reviews.llvm.org/D91873
2020-12-07 14:27:18 -05:00
Kazushi (Jam) Marukawa 9d4501e2b4 [VE] Add vcp and vex intrinsic instructions
Add vcp and vex intrinsic instructions and regression tests.

Reviewed By: simoll

Differential Revision: https://reviews.llvm.org/D92752
2020-12-07 22:56:55 +09:00
Georgii Rymar 347ea1af34 [llvm-readobj/elf] - Refine the implementation of "printFunctionStackSize".
This rewrites the logic to get rid of "ELFSymbolRef" API where possible.
This allowed to handle possible errors better, improve warnings reported and add new ones.
Also 'reportWarning' was replaced with 'reportUniqueWarning'

Differential revision: https://reviews.llvm.org/D92545
2020-12-07 14:57:44 +03:00
Kazushi (Jam) Marukawa 03898b79fb [VE] Add vrcp, vrsqrt, vcvt, vmrg, and vshf intrinsic instructions
Add vrcp, vrsqrt, vcvt, vmrg, and vshf intrinsic instructions and
regression tests.

Reviewed By: simoll

Differential Revision: https://reviews.llvm.org/D92750
2020-12-07 20:30:12 +09:00
Cullen Rhodes 9b01896555 [IR] Support scalable vectors in ShuffleVectorInst::increasesLength
Since the length of the llvm::SmallVector shufflemask is related to the
minimum number of elements in a scalable vector, it is fine to just get
the Min field of the ElementCount. This is already done for the similar
function changesLength, tests have been added for both.

Reviewed By: sdesmalen

Differential Revision: https://reviews.llvm.org/D92472
2020-12-07 10:42:48 +00:00
Kazushi (Jam) Marukawa 67dbc8195d [VE] Add vfmad, vfmsb, vfnmad, and vfnmsb intrinsic instructions
Add vfmad, vfmsb, vfnmad, and vfnmsb intrinsic instructions and
regression tests.

Reviewed By: simoll

Differential Revision: https://reviews.llvm.org/D92697
2020-12-07 19:28:17 +09:00
Esme-Yi 28fdeea952 [PowerPC] Add support for intrinsics dcbfps and dcbstps in P10.
Summary: This patch added support for the intrinsics llvm.ppc.dcbfps and llvm.ppc.dcbstps.
dcbfps and dcbstps are actually extended mnemonics of dcbf.
dcbfps RA,RB ---> dcbf RA,RB,4
dcbstps RA,RB ---> dcbf RA,RB,6

Reviewed By: amyk, steven.zhang

Differential Revision: https://reviews.llvm.org/D91323
2020-12-07 05:19:06 +00:00
Fangrui Song 9c53b2adc8 [MC] Delete unused declarations
Notes:

* llvm::createAsmStreamer: it has been moved to TargetRegistry.h
* (anon ns)::WasmObjectWriter::updateCustomSectionRelocations: remnant of D46335
* COFFAsmParser::ParseSEHRegisterNumber: remnant of D66625
* llvm::CodeViewContext::isValidCVFileNumber: accidentally added by r279847
2020-12-06 15:36:39 -08:00
Fangrui Song 6785ca0124 [llvm-c] Delete unimplemented llvm-c/LinkTimeOptimizer.h
The file was added in 2007 but the functions have never been implemented.
Having the file can only cause confusion to existing C API (llvm-c/lto.h) users.
2020-12-06 15:18:25 -08:00
Fangrui Song 2d03c8e2c8 [CodeGen] Delete 4 unused declarations 2020-12-06 15:02:18 -08:00
Fangrui Song 0e0d616fa2 [CodeGen] Delete 15 unused declarations
Notes about a few declarations:

* LiveVariables::RegisterDefIsDead: deleted by r47927
* createForwardControlFlowIntegrityPass, createJumpInstrTablesPass: deleted by r230780
* RegScavenger::setLiveInsUsed: deleted by r292543
* ScheduleDAGInstrs::{toggleKillFlag,startBlockForKills}: deleted by r304055
* Localizer::shouldLocalize: remnant of D75207
* DwarfDebug::addSectionLabel: deleted by r373273
2020-12-06 14:55:04 -08:00
Fangrui Song a2f922140f [TableGen] Delete 11 unused declarations 2020-12-06 13:21:07 -08:00
Florian Hahn f19876c536 [ConstraintElimination] Bail out if system gets too big.
For some inputs, the constraint system can grow quite large during
solving, because it replaces complex constraints with one or more
simpler constraints. This adds a cut-off to avoid compile-time explosion
on problematic inputs.
2020-12-06 20:19:15 +00:00
Wenlei He 6b989a1710 [CSSPGO] Infrastructure for context-sensitive Sample PGO and Inlining
This change adds the context-senstive sample PGO infracture described in CSSPGO RFC (https://groups.google.com/g/llvm-dev/c/1p1rdYbL93s). It introduced an abstraction between input profile and profile loader that queries input profile for functions. Specifically, there's now the notion of base profile and context profile, and they are managed by the new SampleContextTracker for adjusting and merging profiles based on inline decisions. It works with top-down profiled guided inliner in profile loader (https://reviews.llvm.org/D70655) for better inlining with specialization and better post-inline profile fidelity. In the future, we can also expose this infrastructure to CGSCC inliner in order for it to take advantage of context-sensitive profile. This change is the consumption part of context-sensitive profile (The generation part is in this stack: https://reviews.llvm.org/D89707). We've seen good results internally in conjunction with Pseudo-probe (https://reviews.llvm.org/D86193). Pacthes for integration with Pseudo-probe coming up soon.

Currently the new infrastructure kick in when input profile contains the new context-sensitive profile; otherwise it's no-op and does not affect existing AutoFDO.

**Interface**

There're two sets of interfaces for query and tracking respectively exposed from SampleContextTracker. For query, now instead of simply getting a profile from input for a function, we can explicitly query base profile or context profile for given call path of a function. For tracking, there're separate APIs for marking context profile as inlined, or promoting and merging not inlined context profile.

- Query base profile (`getBaseSamplesFor`)
Base profile is the merged synthetic profile for function's CFG profile from any outstanding (not inlined) context. We can query base profile by function.

- Query context profile (`getContextSamplesFor`)
Context profile is a function's CFG profile for a given calling context. We can query context profile by context string.

- Track inlined context profile (`markContextSamplesInlined`)
When a function is inlined for given calling context, we need to mark the context profile for that context as inlined. This is to make sure we don't include inlined context profile when synthesizing base profile for that inlined function.

- Track not-inlined context profile (`promoteMergeContextSamplesTree`)
When a function is not inlined for given calling context, we need to promote the context profile tree so the not inlined context becomes top-level context. This preserve the sub-context under that function so later inline decision for that not inlined function will still have context profile for its call tree. Note that profile will be merged if needed when promoting a context profile tree if any of the node already exists at its promoted destination.

**Implementation**

Implementation-wise, `SampleContext` is created as abstraction for context. Currently it's a string for call path, and we can later optimize it to something more efficient, e.g. context id. Each `SampleContext` also has a `ContextState` indicating whether it's raw context profile from input, whether it's inlined or merged, whether it's synthetic profile created by compiler. Each `FunctionSamples` now has a `SampleContext` that tells whether it's base profile or context profile, and for context profile what is the context and state.

On top of the above context representation, a custom trie tree is implemented to track and manager context profiles. Specifically, `SampleContextTracker` is implemented that encapsulates a trie tree with `ContextTireNode` as node. Each node of the trie tree represents a frame in calling context, thus the path from root to a node represents a valid calling context. We also track `FunctionSamples` for each node, so this trie tree can serve efficient query for context profile. Accordingly, context profile tree promotion now becomes moving a subtree to be under the root of entire tree, and merge nodes for subtree if this move encounters existing nodes.

**Integration**

`SampleContextTracker` is now also integrated with AutoFDO, `SampleProfileReader` and `SampleProfileLoader`. When we detected input profile contains context-sensitive profile, `SampleContextTracker` will be used to track profiles, and all profile query will go to `SampleContextTracker` instead of `SampleProfileReader` automatically. Tracking APIs are called automatically for each inline decision from `SampleProfileLoader`.

Differential Revision: https://reviews.llvm.org/D90125
2020-12-06 11:49:18 -08:00
Nikita Popov 5e69e2ebad [BasicAA] Migrate "same base pointer" logic to decomposed GEPs
BasicAA has some special bit of logic for "same base pointer" GEPs
that performs a structural comparison: It only looks at two GEPs
with the same base (as opposed to two GEP chains with a MustAlias
base) and compares their indexes in a limited way. I generalized
part of this code in D91027, and this patch merges the remainder
into the normal decomposed GEP logic.

What this code ultimately wants to do is to determine that
gep %base, %idx1 and gep %base, %idx2 don't alias if %idx1 != %idx2,
and the access size fits within the stride.

We can express this in terms of a decomposed GEP expression with
two indexes scale*%idx1 + -scale*%idx2 where %idx1 != %idx2, and
some appropriate checks for sizes and offsets.

This makes the reasoning slightly more powerful, and more
importantly brings all the GEP logic under a common umbrella.

Differential Revision: https://reviews.llvm.org/D92723
2020-12-06 10:27:35 +01:00
Lang Hames 5bc9c858e3 [ORC] Fix missing forward of Allow filter in TPCDynamicLibrarySearchGenerator. 2020-12-06 15:42:45 +11:00
Kazu Hirata 5121400e71 [ConstantHoisting] Remove unused declaration optimizeConstants (NFC)
The function was renamed to runImpl on Jul 2, 2016 in commit
071d8306b0, but the old declaration has
remained since.
2020-12-05 16:22:12 -08:00
Philip Reames bfda69416c [BasicAA] Fix a bug with relational reasoning across iterations
Due to the recursion through phis basicaa does, the code needs to be extremely careful not to reason about equality between values which might represent distinct iterations. I'm generally skeptical of the correctness of the whole scheme, but this particular patch fixes one particular instance which is demonstrateable incorrect.

Interestingly, this appears to be the second attempted fix for the same issue. The former fix is incomplete and doesn't address the actual issue.

Differential Revision: https://reviews.llvm.org/D92694
2020-12-05 14:10:21 -08:00
Benjamin Kramer 2a136a7a9c [X86] Autodetect znver3 2020-12-05 19:08:20 +01:00
Florian Hahn 4ceecc820b [ConstraintElimination] Handle constraints with all zero var coeffs.
Constraints where all variable coefficients are 0 do not add any useful
information. When checking, we can check if they are always true/false.
2020-12-05 12:06:53 +00:00
Nikita Popov 0003600aaf [AA] Initialize Depth member
Fix mistake introduced in f8afba5f7a25a69c12191d979d78d40fa6e5b684:
I failed to initialize the Depth member to zero.
2020-12-05 11:37:36 +01:00
Nikita Popov f8afba5f7a [AA] Add statistics for alias results (NFC)
Count how many NoAlias/MustAlias/MayAlias we get from top-level
queries.
2020-12-05 11:09:15 +01:00
Sam Clegg ab58e4cb51 [lld][WebAssembly] Add suppport for PIC + passive data initialization
This change improves our support for shared memory to include
PIC executables (and shared libraries).

To handle this case the linker-generated `__wasm_init_memory`
function (that only exists in shared memory builds) must be
capable of loading memory segements at non-const offsets based
on the runtime value of `__memory_base`.

Differential Revision: https://reviews.llvm.org/D92620
2020-12-04 17:28:23 -08:00
Kazushi (Jam) Marukawa 23034a4a63 [VE] Add vfsqrt, vfcmp, vfmax, and vfmin intrinsic instructions
Add vfsqrt, vfcmp, vfmax, and vfmin intrinsic instructions and
regression tests.

Reviewed By: simoll

Differential Revision: https://reviews.llvm.org/D92651
2020-12-05 07:52:14 +09:00
Duncan P. N. Exon Smith 4d8bf870a8 ADT: Remove AlignedCharArrayUnion, NFC
Prep commit already migrated users over to std::aligned_union_t; this
just deletes the type / header / test.

Differential Revision: https://reviews.llvm.org/D92517
2020-12-04 14:13:22 -08:00
Duncan P. N. Exon Smith d10f9863a5 ADT: Migrate users of AlignedCharArrayUnion to std::aligned_union_t, NFC
Prepare to delete `AlignedCharArrayUnion` by migrating its users over to
`std::aligned_union_t`.

I will delete `AlignedCharArrayUnion` and its tests in a follow-up
commit so that it's easier to revert in isolation in case some
downstream wants to keep using it.

Differential Revision: https://reviews.llvm.org/D92516
2020-12-04 12:34:49 -08:00
Valentin Clement a366188d80 [flang][openacc] Add clause validaty tests for data construct + fix default restriction
Add clause validity tests for the data construct. The default clause can appear only once
and this was not enforce in the ACC.td.

Reviewed By: sameeranjoshi

Differential Revision: https://reviews.llvm.org/D91888
2020-12-04 15:12:15 -05:00
Duncan P. N. Exon Smith 4b5dc150b9 ADT: Change AlignedCharArrayUnion to an alias of std::aligned_union_t, NFC
All the users of `AlignedCharArrayUnion` were changed in
5b267fb796 to stop peeking inside (to look
at `buffer`), so this finishes gutting it. It's now an alias of
`std::aligned_union_t`, with a minor difference in template parameters
(`std::aligned_union_t` takes a minimum size and 0+ types, whereas this
just takes 1+ types... maybe a bit simpler to use correctly?).

A follow up will remove `AlignedCharArrayUnion` entirely, inlining this
alias into its users.

Differential Revision: https://reviews.llvm.org/D92512
2020-12-04 11:59:35 -08:00
Duncan P. N. Exon Smith 5b267fb796 ADT: Stop peeking inside AlignedCharArrayUnion, NFC
Update all the users of `AlignedCharArrayUnion` to stop peeking inside
(to look at `buffer`) so that a follow-up patch can replace it with an
alias to `std::aligned_union_t`.

This was reviewed as part of https://reviews.llvm.org/D92512, but I'm
splitting this bit out to commit first to reduce churn in case the
change to `AlignedCharArrayUnion` needs to be reverted for some
unexpected reason.
2020-12-04 11:07:42 -08:00
Simon Pilgrim 9cf4f493a7 [DAG] Move SelectionDAG implementation to KnownBits::setInReg(). NFCI. 2020-12-04 18:09:08 +00:00
Arthur Eubanks 7f6f9f4cf9 [NewPM] Make pass adaptors less templatey
Currently PassBuilder.cpp is by far the file that takes longest to
compile. This is due to tons of templates being instantiated per pass.

Follow PassManager by using wrappers around passes to avoid making
the adaptors templated on the pass type. This allows us to move various
adaptors' run methods into .cpp files.

This reduces the compile time of PassBuilder.cpp on my machine from 66
to 39 seconds. It also reduces the size of opt from 685M to 676M.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D92616
2020-12-04 08:30:50 -08:00
Georgii Rymar ffbce65f95 [lib/Object, tools] - Make ELFObjectFile::getELFFile return reference.
We always have an object, so we don't have to return a pointer.

Differential revision: https://reviews.llvm.org/D92560
2020-12-04 16:02:29 +03:00
Kazushi (Jam) Marukawa e936d1e113 [VE] Add vfadd, vfsub, vfmul, and vfdiv intrinsic instructions
Add vfadd, vfsub, vfmul, and vfdiv intrinsic instructions and
regression tests.

Reviewed By: simoll

Differential Revision: https://reviews.llvm.org/D92649
2020-12-04 21:58:51 +09:00
Cullen Rhodes 333de690ea [IR] Disallow scalable vectors in ShuffleVectorInst::isExtractSubvectorMask
It's not possible to express an extract subvector shuffle mask for
a scalable vector.

Reviewed By: david-arm

Differential Revision: https://reviews.llvm.org/D92312
2020-12-04 11:30:51 +00:00
Georgii Rymar 7ac06444b8 [yaml2obj,obj2yaml] - Make Symbol::Section field optional.
This is similar to what we did earlier for fields of the Section class.

When a field is optional we can use the =<none> syntax in macros.

This was splitted from D92478.

Differential revision: https://reviews.llvm.org/D92565
2020-12-04 13:45:47 +03:00
Evgeniy Brevnov 061cebb46f [NFC][NARY-REASSOCIATE] Restructure code to aviod isPotentiallyReassociatable
Currently we have to duplicate the same checks in isPotentiallyReassociatable and tryReassociate. With simple pattern like add/mul this may be not a big deal. But the situation gets much worse when I try to add support for min/max. Min/Max may be represented by several instructions and can take different forms. In order reduce complexity for upcoming min/max support we need to restructure the code a bit to avoid mentioned code duplication.

Reviewed By: mkazantsev

Differential Revision: https://reviews.llvm.org/D88286
2020-12-04 16:19:43 +07:00
David Blaikie c98a7ab1b1 IntrusiveRefCntPtr: Ensure RefCount is initialized during ThreadSafeRefCountedBase copy construction
Post commit feedback on 2e83ccc2ee from Nathan James!
2020-12-03 18:43:01 -08:00
Duncan P. N. Exon Smith e763e032f8 Support: Change InMemoryFileSystem::addFileNoOwn to take a MemoryBufferRef, NFC
Found this by chance when looking at the InMemoryFileSystem API, seems
like an easy cleanup.

Differential Revision: https://reviews.llvm.org/D90893
2020-12-03 18:09:52 -08:00
David Blaikie 2e83ccc2ee ADT: Support copying of IntrusiveRefCntPtr objects
This was partially supported but untested for RefCountedBase (the
implicit copy assignment would've been problematic - so delete that) and
unsupported (would not have compiled, because std::atomic is
non-copyable) for ThreadSafeRefCountedBase (implement similar support
to RefCountedBase)

Fix the test that had a copy ctor for the derived object but called
RefCountBase's default ctor from that copy ctor - which meant it wasn't
actually testing RefCountBase's copy semantics.
2020-12-03 17:42:32 -08:00
Sean Silva ae9fd5578e [SmallVector] Allow SmallVector<T>
This patch adds a capability to SmallVector to decide a number of
inlined elements automatically. The policy is:

- A minimum of 1 inlined elements, with more as long as
sizeof(SmallVector<T>) <= 64.
- If sizeof(T) is "too big", then trigger a static_assert: this dodges
the more pathological cases

This is expected to systematically improve SmallVector use in the
LLVM codebase, which has historically been plagued by semi-arbitrary /
cargo culted N parameters, often leading to bad outcomes due to
excessive sizeof(SmallVector<T, N>). This default also makes
programming more convenient by avoiding edit/rebuild cycles due to
forgetting to type the N parameter.

Differential Revision: https://reviews.llvm.org/D92522
2020-12-03 17:21:44 -08:00
Arthur Eubanks 2f0de58294 [NewPM] Support --print-before/after in NPM
This changes --print-before/after to be a list of strings rather than
legacy passes. (this also has the effect of not showing the entire list
of passes in --help-hidden after --print-before/after, which IMO is
great for making it less verbose).

Currently PrintIRInstrumentation passes the class name rather than pass
name to llvm::shouldPrintBeforePass(), meaning
llvm::shouldPrintBeforePass() never functions as intended in the NPM.
There is no easy way of converting class names to pass names outside of
within an instance of PassBuilder.

This adds a map of pass class names to their short names in
PassRegistry.def within PassInstrumentationCallbacks. It is populated
inside the constructor of PassBuilder, which takes a
PassInstrumentationCallbacks.

Add a pointer to PassInstrumentationCallbacks inside
PrintIRInstrumentation and use the newly created map.

This is a bit hacky, but I can't think of a better way since the short
id to class name only exists within PassRegistry.def. This also doesn't
handle passes not in PassRegistry.def but rather added via
PassBuilder::registerPipelineParsingCallback().

llvm/test/CodeGen/Generic/print-after.ll doesn't seem very useful now
with this change.

Reviewed By: ychen, jamieschmeiser

Differential Revision: https://reviews.llvm.org/D87216
2020-12-03 16:52:14 -08:00
Philip Reames 55db6ec1cc [BasicAA] Move newline to dump from printer 2020-12-03 14:35:43 -08:00
Philip Reames 17b195b632 [BasicAA] Minor formatting improvements for printers 2020-12-03 13:08:56 -08:00
Philip Reames d93b8acd09 [BasicAA] Add print routines to DecomposedGEP for ease of debugging 2020-12-03 12:43:39 -08:00
Mehdi Amini 6cd9608fb3 Revert "Switch to std::is_trivially_move_constructible and std::is_trivially_copy_constructible"
This reverts commit c8d406c93c.

Builds are broken with some versions of GCC.
2020-12-03 19:58:06 +00:00
Fangrui Song 756fa8b9be [Metadata] Fix layer violation in D91576
There is a library layering issue. LLVMAnalysis provides llvm/Analysis/ScopedNoAliasAA.h and depends on LLVMCore.

LLVMCore provides llvm/IR/Metadata.cpp and it should not include a header file in LLVMAnalysis
2020-12-03 10:58:46 -08:00
modimo 1860331932 [MemCpyOpt] Correctly merge alias scopes during call slot optimization
When MemCpyOpt performs call slot optimization it will concatenate the `alias.scope` metadata between the function call and the memcpy. However, scoped AA relies on the domains in metadata to be maintained in a caller-callee relationship. Naive concatenation breaks this assumption leading to bad AA results.

The fix is to take the intersection of domains then union the scopes within those domains.

The original bug came from a case of rust bad codegen which uses this bad aliasing to perform additional memcpy optimizations. As show in the added test case `%src` got forwarded past its lifetime leading to a dereference of garbage data.

Testing
ninja check-llvm

Reviewed By: jeroen.dobbelaere

Differential Revision: https://reviews.llvm.org/D91576
2020-12-03 09:23:37 -08:00
Fangrui Song c8d406c93c Switch to std::is_trivially_move_constructible and std::is_trivially_copy_constructible
Differential Revision: https://reviews.llvm.org/D92543
2020-12-03 09:15:40 -08:00
Ahmed Bougacha f77c948d56 [Triple][MachO] Define "arm64e", an AArch64 subarch for Pointer Auth.
This also teaches MachO writers/readers about the MachO cpu subtype,
beyond the minimal subtype reader support present at the moment.

This also defines a preprocessor macro to allow users to distinguish
__arm64__ from __arm64e__.

arm64e defaults to an "apple-a12" CPU, which supports v8.3a, allowing
pointer-authentication codegen.
It also currently defaults to ios14 and macos11.

Differential Revision: https://reviews.llvm.org/D87095
2020-12-03 07:53:59 -08:00
dfukalov 2ce38b3f03 [NFC] Reduce include files dependency.
1. Removed #include "...AliasAnalysis.h" in other headers and modules.
2. Cleaned up includes in AliasAnalysis.h.

Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D92489
2020-12-03 18:25:05 +03:00
Paul C. Anagnostopoulos 415fab6f67 [TableGen] Eliminate the 'code' type
Update the documentation.

Rework various backends that relied on the code type.

Differential Revision: https://reviews.llvm.org/D92269
2020-12-03 10:19:11 -05:00
Kazushi (Jam) Marukawa 1365718778 [VE] Add vsll, vsrl, vsla, vsra, and vsfa intrinsic instructions
Add vsll, vsrl, vsla, vsra, and vsfa intrinsic instructions and
regression tests.

Reviewed By: simoll

Differential Revision: https://reviews.llvm.org/D92550
2020-12-03 23:19:58 +09:00
Jay Foad 0f32e81407 [TableGen] Remove unused class RecordValResolver. NFC.
Differential Revision: https://reviews.llvm.org/D92477
2020-12-03 13:36:58 +00:00
Tim Northover 152df3add1 arm64: count Triple::aarch64_32 as an aarch64 target and enable leaf frame pointers 2020-12-03 11:09:44 +00:00
Kazushi (Jam) Marukawa b91238173d [VE] Add veqv and vseq intrinsic instructions
Add veqv and vseq intrinsic instructions and regression tests.

Reviewed By: simoll

Differential Revision: https://reviews.llvm.org/D92527
2020-12-03 17:39:24 +09:00
Fangrui Song 1d6ebdfb66 Switch from llvm::is_trivially_copyable to std::is_trivially_copyable
GCC<5 did not support std::is_trivially_copyable. Now LLVM builds require 5.1
we can migrate to std::is_trivially_copyable.

The Optional.h change made MSVC choke
(https://buildkite.com/llvm-project/premerge-checks/builds/18587#cd1bb616-ffdc-4581-9795-b42c284196de)
so I leave it out for now.

Differential Revision: https://reviews.llvm.org/D92514
2020-12-02 22:02:48 -08:00
Kazu Hirata 4cc56d2b19 [MemorySSA] Remove unused declaration findDominatingDef (NFC)
The function definition was removed on Feb 22, 2017 in commit
17e8d0eae2.  The declaration has
remained since.
2020-12-02 17:40:20 -08:00
Hsiangkai Wang f7bc7c2981 [RISCV] Support Zfh half-precision floating-point extension.
Support "Zfh" extension according to
https://github.com/riscv/riscv-isa-manual/blob/zfh/src/zfh.tex

Differential Revision: https://reviews.llvm.org/D90738
2020-12-03 09:16:33 +08:00
Duncan P. N. Exon Smith 65c5c9f92e ADT: Rely on std::aligned_union_t for math in AlignedCharArrayUnion, NFC
Instead of computing the alignment and size of the `char` buffer in
`AlignedCharArrayUnion`, rely on the math in `std::aligned_union_t`.
Because some users of this rely on the `buffer` field existing with a
type convertible to `char *`, we can't change the field type, but we can
still avoid duplicating the logic.

A potential follow up would be to delete `AlignedCharArrayUnion` after
updating its users to use `std::aligned_union_t` directly; or if we like
our template parameters better, could update users to stop peeking
inside and then replace the definition with:
```
template <class T, class... Ts>
using AlignedCharArrayUnion = std::aligned_union_t<1, T, Ts...>;
```

Differential Revision: https://reviews.llvm.org/D92500
2020-12-02 15:56:12 -08:00
Mircea Trofin bab72dd5d5 [NFC][MC] TargetRegisterInfo::getSubReg is a MCRegister.
Typing the API appropriately.

Differential Revision: https://reviews.llvm.org/D92341
2020-12-02 15:46:38 -08:00
Duncan P. N. Exon Smith 168b803b45 ADT: Remove redundant `alignas` from IntervalMap, NFC
`AlignedArrayCharUnion` is now using `alignas`, which is properly
supported now by all the host toolchains we support. As a result, the
extra `alignas` on `IntervalMap` isn't needed anymore.

This is effectively a revert of 379daa2974.

Differential Revision: https://reviews.llvm.org/D92509
2020-12-02 14:33:20 -08:00
Reid Kleckner 91e66bfd32 Revert "Use std::is_trivially_copyable", breaks MSVC build
Revert "Delete llvm::is_trivially_copyable and CMake variable HAVE_STD_IS_TRIVIALLY_COPYABLE"

This reverts commit 4d4bd40b57.

This reverts commit 557b00e0af.
2020-12-02 14:30:46 -08:00
Hongtao Yu 24d4291ca7 [CSSPGO] Pseudo probes for function calls.
An indirect call site needs to be probed for its potential call targets. With CSSPGO a direct call also needs a probe so that a calling context can be represented by a stack of callsite probes. Unlike pseudo probes for basic blocks that are in form of standalone intrinsic call instructions, pseudo probes for callsites have to be attached to the call instruction, thus a separate instruction would not work.

One possible way of attaching a probe to a call instruction is to use a special metadata that carries information about the probe. The special metadata will have to make its way through the optimization pipeline down to object emission. This requires additional efforts to maintain the metadata in various places. Given that the `!dbg` metadata is a first-class metadata and has all essential support in place , leveraging the `!dbg` metadata as a channel to encode pseudo probe information is probably the easiest solution.

With the requirement of not inflating `!dbg` metadata that is allocated for almost every instruction, we found that the 32-bit DWARF discriminator field which mainly serves AutoFDO can be reused for pseudo probes. DWARF discriminators distinguish identical source locations between instructions and with pseudo probes such support is not required. In this change we are using the discriminator field to encode the ID and type of a callsite probe and the encoded value will be unpacked and consumed right before object emission. When a callsite is inlined, the callsite discriminator field will go with the inlined instructions. The `!dbg` metadata of an inlined instruction is in form of a scope stack. The top of the stack is the instruction's original `!dbg` metadata and the bottom of the stack is for the original callsite of the top-level inliner. Except for the top of the stack, all other elements of the stack actually refer to the nested inlined callsites whose discriminator field (which actually represents a calliste probe) can be used together to represent the inline context of an inlined PseudoProbeInst or CallInst.

To avoid collision with the baseline AutoFDO in various places that handles dwarf discriminators where a check against  the `-pseudo-probe-for-profiling` switch is not available, a special encoding scheme is used to tell apart a pseudo probe discriminator from a regular discriminator. For the regular discriminator, if all lowest 3 bits are non-zero, it means the discriminator is basically empty and all higher 29 bits can be reversed for pseudo probe use.

Callsite pseudo probes are inserted in `SampleProfileProbePass` and a target-independent MIR pass `PseudoProbeInserter` is added to unpack the probe ID/type from `!dbg`.

Note that with this work the switch -debug-info-for-profiling will not work with -pseudo-probe-for-profiling anymore. They cannot be used at the same time.

Reviewed By: wmi

Differential Revision: https://reviews.llvm.org/D91756
2020-12-02 13:45:20 -08:00
jasonliu 2c63e7604c [XCOFF][AIX] Alternative path in EHStreamer for platforms do not have uleb128 support
Summary:
Not all system assembler supports `.uleb128 label2 - label1` form.
When the target do not support this form, we have to take
alternative manual calculation to get the offsets from them.

Reviewed By: hubert.reinterpretcast

Diffierential Revision: https://reviews.llvm.org/D92058
2020-12-02 20:03:15 +00:00
Nick Desaulniers bc044a88ee [Inline] prevent inlining on stack protector mismatch
It's common for code that manipulates the stack via inline assembly or
that has to set up its own stack canary (such as the Linux kernel) would
like to avoid stack protectors in certain functions. In this case, we've
been bitten by numerous bugs where a callee with a stack protector is
inlined into an attribute((no_stack_protector)) caller, which
generally breaks the caller's assumptions about not having a stack
protector. LTO exacerbates the issue.

While developers can avoid this by putting all no_stack_protector
functions in one translation unit together and compiling those with
-fno-stack-protector, it's generally not very ergonomic or as
ergonomic as a function attribute, and still doesn't work for LTO. See also:
https://lore.kernel.org/linux-pm/20200915172658.1432732-1-rkir@google.com/
https://lore.kernel.org/lkml/20200918201436.2932360-30-samitolvanen@google.com/T/#u

SSP attributes can be ordered by strength. Weakest to strongest, they
are: ssp, sspstrong, sspreq.  Callees with differing SSP attributes may be
inlined into each other, and the strongest attribute will be applied to the
caller. (No change)

After this change:
* A callee with no SSP attributes will no longer be inlined into a
  caller with SSP attributes.
* The reverse is also true: a callee with an SSP attribute will not be
  inlined into a caller with no SSP attributes.
* The alwaysinline attribute overrides these rules.

Functions that get synthesized by the compiler may not get inlined as a
result if they are not created with the same stack protector function
attribute as their callers.

Alternative approach to https://reviews.llvm.org/D87956.

Fixes pr/47479.

Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>

Reviewed By: rnk, MaskRay

Differential Revision: https://reviews.llvm.org/D91816
2020-12-02 11:00:16 -08:00
jasonliu a65d8c5d72 [XCOFF][AIX] Generate LSDA data and compact unwind section on AIX
Summary:
AIX uses the existing EH infrastructure in clang and llvm.
The major differences would be
1. AIX do not have CFI instructions.
2. AIX uses a new personality routine, named __xlcxx_personality_v1.
   It doesn't use the GCC personality rountine, because the
   interoperability is not there yet on AIX.
3. AIX do not use eh_frame sections. Instead, it would use a eh_info
section (compat unwind section) to store the information about
personality routine and LSDA data address.

Reviewed By: daltenty, hubert.reinterpretcast

Differential Revision: https://reviews.llvm.org/D91455
2020-12-02 18:42:44 +00:00
Fangrui Song 557b00e0af Delete llvm::is_trivially_copyable and CMake variable HAVE_STD_IS_TRIVIALLY_COPYABLE
GCC<5 did not support std::is_trivially_copyable. Now LLVM builds
require 5.1 we can delete llvm::is_trivially_copyable after the users
have been migrated to std::is_trivially_copyable.
2020-12-02 09:58:08 -08:00
Fangrui Song 4d4bd40b57 Use std::is_trivially_copyable
GCC<5 did not support std::is_trivially_copyable. Now LLVM builds require 5.1
we can migrate to std::is_trivially_copyable.
2020-12-02 09:58:07 -08:00
Alex Zinenko 240dd92432 [OpenMPIRBuilder] forward arguments as pointers to outlined function
OpenMPIRBuilder::createParallel outlines the body region of the parallel
construct into a new function that accepts any value previously defined outside
the region as a function argument. This function is called back by OpenMP
runtime function __kmpc_fork_call, which expects trailing arguments to be
pointers. If the region uses a value that is not of a pointer type, e.g. a
struct, the produced code would be invalid. In such cases, make createParallel
emit IR that stores the value on stack and pass the pointer to the outlined
function instead. The outlined function then loads the value back and uses as
normal.

Reviewed By: jdoerfert, llitchev

Differential Revision: https://reviews.llvm.org/D92189
2020-12-02 14:59:41 +01:00
Kazushi (Jam) Marukawa dd0159bd81 [VE] Add vand, vor, and vxor intrinsic instructions
Add vand, vor, and vxor intrinsic instructions and regression tests.

Reviewed By: simoll

Differential Revision: https://reviews.llvm.org/D92454
2020-12-02 22:52:54 +09:00
David Sherwood 71bd59f0cb [SVE] Add support for scalable vectors with vectorize.scalable.enable loop attribute
In this patch I have added support for a new loop hint called
vectorize.scalable.enable that says whether we should enable scalable
vectorization or not. If a user wants to instruct the compiler to
vectorize a loop with scalable vectors they can now do this as
follows:

  br i1 %exitcond, label %for.end, label %for.body, !llvm.loop !2
  ...
  !2 = !{!2, !3, !4}
  !3 = !{!"llvm.loop.vectorize.width", i32 8}
  !4 = !{!"llvm.loop.vectorize.scalable.enable", i1 true}

Setting the hint to false simply reverts the behaviour back to the
default, using fixed width vectors.

Differential Revision: https://reviews.llvm.org/D88962
2020-12-02 13:23:43 +00:00
David Green 99eb0f16c3 [Intrinsics] Re-remove experimental_vector_reduce intrinsics
These were re-added by fbfb1c7909 but
should not have been. This removes the old experimental versions of the
reduction intrinsics again, leaving the new non experimental ones.

Differential Revision: https://reviews.llvm.org/D92411
2020-12-02 09:22:41 +00:00
Kazu Hirata 0cca328681 [Transforms] Remove unused declaration fillImplicitControlFlowInfo (NFC)
The definition was removed on Aug 7, 2018 in commit
640cb00365, but the declaration has
remained since.
2020-12-01 19:42:14 -08:00
Arthur O'Dwyer e181a6aedd s/instantate/instantiate/ throughout. NFCI.
The static_assert in "libcxx/include/memory" was the main offender here,
but then I figured I might as well `git grep -i instantat` and fix all
the instances I found. One was in user-facing HTML documentation;
the rest were in comments or tests.
2020-12-01 22:13:40 -05:00
Kazushi (Jam) Marukawa c1762bcf0a [VE] Add vcmp, vmax, and vmin intrinsic instructions
Add vcmp, vmax, and vmin intrinsic instructions and regression tests.

Reviewed By: simoll

Differential Revision: https://reviews.llvm.org/D92387
2020-12-02 11:16:52 +09:00
Eric Astor c64037b784 [ms] [llvm-ml] Support command-line defines
Enable command-line defines as textmacros

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D90059
2020-12-01 18:06:05 -05:00
James Park 78b0ec3d1c Avoid redundant inline with LLVM_ATTRIBUTE_ALWAYS_INLINE
Fix MSVC warning when __forceinline is paired with inline.

Reviewed By: dblaikie

Differential Revision: https://reviews.llvm.org/D85264
2020-12-01 14:43:16 -08:00
David Blaikie 615f63e149 Revert "[FastISel] Flush local value map on ever instruction" and dependent patches
This reverts commit cf1c774d6a.

This change caused several regressions in the gdb test suite - at least
a sample of which was due to line zero instructions making breakpoints
un-lined. I think they're worth investigating/understanding more (&
possibly addressing) before moving forward with this change.

Revert "[FastISel] NFC: Clean up unnecessary bookkeeping"
This reverts commit 3fd39d3694.

Revert "[FastISel] NFC: Remove obsolete -fast-isel-sink-local-values option"
This reverts commit a474657e30.

Revert "Remove static function unused after cf1c774."
This reverts commit dc35368ccf.

Revert "[lldb] Fix TestThreadStepOut.py after "Flush local value map on every instruction""
This reverts commit 53a14a47ee.
2020-12-01 14:26:23 -08:00
Arthur Eubanks aafb366210 Reland [CMake][NewPM] Move ENABLE_EXPERIMENTAL_NEW_PASS_MANAGER into llvm/
This allows us to use its value everywhere, rather than just clang. Some
other places, like opt and lld, will use its value soon.

Rename it internally to LLVM_ENABLE_NEW_PASS_MANAGER.

The #define for it is now in llvm-config.h.

The initial land accidentally set the value of
LLVM_ENABLE_NEW_PASS_MANAGER to the string
ENABLE_EXPERIMENTAL_NEW_PASS_MANAGER instead of its value.

Reviewed By: rnk, hans

Differential Revision: https://reviews.llvm.org/D92072
2020-12-01 14:00:32 -08:00
Arthur Eubanks 806a76c001 Revert "[CMake][NewPM] Move ENABLE_EXPERIMENTAL_NEW_PASS_MANAGER into llvm/"
The new pass manager was accidentally enabled by default with this change.

This reverts commit a36bd4c90d.
2020-12-01 13:12:12 -08:00
Arthur Eubanks a36bd4c90d [CMake][NewPM] Move ENABLE_EXPERIMENTAL_NEW_PASS_MANAGER into llvm/
This allows us to use its value everywhere, rather than just clang. Some
other places, like opt and lld, will use its value soon.

The #define for it is now in llvm-config.h.

Reviewed By: rnk, hans

Differential Revision: https://reviews.llvm.org/D92072
2020-12-01 11:42:17 -08:00
Nikita Popov 624af932a8 [MemCpyOpt] Port to MemorySSA
This is a straightforward port of MemCpyOpt to MemorySSA following
the approach of D26739. MemDep queries are replaced with MSSA queries
without changing the overall structure of the pass. Some care has
to be taken to account for differences between these APIs
(MemDep also returns reads, MSSA doesn't).

Differential Revision: https://reviews.llvm.org/D89207
2020-12-01 17:57:41 +01:00
Fangrui Song f0659c0673 [X86] Support modifier @PLTOFF for R_X86_64_PLTOFF64
`gcc -mcmodel=large` can emit @PLTOFF.

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D92294
2020-12-01 08:39:01 -08:00
Kazushi (Jam) Marukawa 10b164d2f7 [VE] Add vmul and vdiv intrinsic instructions
Add vmul and vdiv intrinsic instructions and regression tests.

Reviewed By: simoll

Differential Revision: https://reviews.llvm.org/D92377
2020-12-01 23:03:49 +09:00
Caroline Concatto 4b0ef2b075 [NFC][CostModel]Extend class IntrinsicCostAttributes to use ElementCount Type
This patch replaces the attribute  `unsigned VF`  in the class
IntrinsicCostAttributes by `ElementCount VF`.
This is a non-functional change to help upcoming patches to compute the cost
model for scalable vector inside this class.

Differential Revision: https://reviews.llvm.org/D91532
2020-12-01 11:12:51 +00:00
Kazushi (Jam) Marukawa c3fe6ea22e [VE] Add vadd and vsub intrinsic instructions
Add vadd and vsub intrinsic instructions and regression tests.

Reviewed By: simoll

Differential Revision: https://reviews.llvm.org/D92332
2020-12-01 19:57:22 +09:00
Jan Svoboda 8e41a688a5 [clang][cli] Port DependencyOutput option flags to new option parsing system
Depends on D91861.

Reviewed By: dexonsmith

Original patch by Daniel Grumberg.

Differential Revision: https://reviews.llvm.org/D83694
2020-12-01 10:36:12 +01:00
Jan Svoboda 2b84efa000 [clang][cli] Port Frontend option flags to new option parsing system
Depends on D91861.

Reviewed By: dexonsmith

Original patch by Daniel Grumberg.

Differential Revision: https://reviews.llvm.org/D83697
2020-12-01 10:02:08 +01:00
Jan Svoboda 88ab38449b [clang][cli] Split DefaultAnyOf into a default value and ImpliedByAnyOf
This makes the options API composable, allows boolean flags to imply non-boolean values and makes the code more logical (IMO).

Differential Revision: https://reviews.llvm.org/D91861
2020-12-01 09:50:11 +01:00
Wei Wang 93dc1b5b8c [Remarks][2/2] Expand remarks hotness threshold option support in more tools
This is the #2 of 2 changes that make remarks hotness threshold option
available in more tools. The changes also allow the threshold to sync with
hotness threshold from profile summary with special value 'auto'.

This change expands remarks hotness threshold option
-fdiagnostics-hotness-threshold in clang and *-remarks-hotness-threshold in
other tools to utilize hotness threshold from profile summary.

Remarks hotness filtering relies on several driver options. Table below lists
how different options are correlated and affect final remarks outputs:

| profile | hotness | threshold | remarks printed |
|---------|---------|-----------|-----------------|
| No      | No      | No        | All             |
| No      | No      | Yes       | None            |
| No      | Yes     | No        | All             |
| No      | Yes     | Yes       | None            |
| Yes     | No      | No        | All             |
| Yes     | No      | Yes       | None            |
| Yes     | Yes     | No        | All             |
| Yes     | Yes     | Yes       | >=threshold     |

In the presence of profile summary, it is often more desirable to directly use
the hotness threshold from profile summary. The new argument value 'auto'
indicates threshold will be synced with hotness threshold from profile summary
during compilation. The "auto" threshold relies on the availability of profile
summary. In case of missing such information, no remarks will be generated.

Differential Revision: https://reviews.llvm.org/D85808
2020-11-30 21:55:50 -08:00
Wei Wang 3acda91742 [Remarks][1/2] Expand remarks hotness threshold option support in more tools
This is the #1 of 2 changes that make remarks hotness threshold option
available in more tools. The changes also allow the threshold to sync with
hotness threshold from profile summary with special value 'auto'.

This change modifies the interface of lto::setupLLVMOptimizationRemarks() to
accept remarks hotness threshold. Update all the tools that use it with remarks
hotness threshold options:

* lld: '--opt-remarks-hotness-threshold='
* llvm-lto2: '--pass-remarks-hotness-threshold='
* llvm-lto: '--lto-pass-remarks-hotness-threshold='
* gold plugin: '-plugin-opt=opt-remarks-hotness-threshold='

Differential Revision: https://reviews.llvm.org/D85809
2020-11-30 21:55:49 -08:00
Hendrik Greving d4ba5e15f4 Add MachineModuleInfo constructor with external MCContext
Adds a constructor to MachineModuleInfo and MachineModuleInfoWapperPass that
takes an external MCContext. If provided, the external context will be used
throughout codegen instead of MMI's default one.

This enables external drivers to take ownership of data put on the MMI's context
during codegen. The internal context is used otherwise and destroyed upon
finish.

Differential Revision: https://reviews.llvm.org/D91313
2020-11-30 20:28:13 -08:00
Amy Huang efd1ec0dec Recommit "[llvm-symbolizer] Switch to using native symbolizer by default on Windows"
This reverts commit 1b63177a56.
2020-11-30 17:36:12 -08:00
Amara Emerson 87ff156414 [AArch64][GlobalISel] Fix crash during legalization of a vector G_SELECT with scalar mask.
The lowering of vector selects needs to first splat the scalar mask into a vector
first.

This was causing a crash when building oggenc in the test suite.

Differential Revision: https://reviews.llvm.org/D91655
2020-11-30 16:37:49 -08:00
Paul Robinson 3fd39d3694 [FastISel] NFC: Clean up unnecessary bookkeeping
Now that we flush the local value map for every instruction, we don't
need any extra flushes for specific cases.  Also, LastFlushPoint is
not used for anything.  Follow-ups to #dc35368 (D91734).

Differential Revision: https://reviews.llvm.org/D92338
2020-11-30 12:27:50 -08:00
Mircea Trofin 5fe10263ab [llvm][inliner] Reuse the inliner pass to implement 'always inliner'
Enable performing mandatory inlinings upfront, by reusing the same logic
as the full inliner, instead of the AlwaysInliner. This has the
following benefits:
- reduce code duplication - one inliner codebase
- open the opportunity to help the full inliner by performing additional
function passes after the mandatory inlinings, but before th full
inliner. Performing the mandatory inlinings first simplifies the problem
the full inliner needs to solve: less call sites, more contextualization, and,
depending on the additional function optimization passes run between the
2 inliners, higher accuracy of cost models / decision policies.

Note that this patch does not yet enable much in terms of post-always
inline function optimization.

Differential Revision: https://reviews.llvm.org/D91567
2020-11-30 12:03:39 -08:00
Nikita Popov b5f23189fb [DL] Inline getAlignmentInfo() implementation (NFC)
Apart from getting the entry in the table (which is already a
separate function), the remaining logic is different for all
alignment types and is better combined with getAlignment().

This is a minor efficiency improvement, and should make further
improvements like using separate storage for different alignment
types simpler.
2020-11-30 20:56:15 +01:00
Nick Lewycky fe43168348 Creating a named struct requires only a Context and a name, but looking up a struct by name requires a Module. The method on Module merely accesses the LLVMContextImpl and no data from the module itself, so this patch moves getTypeByName to a static method on StructType that takes a Context and a name.
There's a small number of users of this function, they are all updated.

This updates the C API adding a new method LLVMGetTypeByName2 that takes a context and a name.

Differential Revision: https://reviews.llvm.org/D78793
2020-11-30 11:34:12 -08:00
Eric Astor abef659a45 [ms] [llvm-ml] Implement the statement expansion operator
If prefaced with a %, expand text macros and macro functions in any statement.

Also, prevent expanding text macros in the message of an ECHO directive unless expanded explicitly by the statement expansion operator.

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D89740
2020-11-30 14:33:24 -05:00
Hongtao Yu c083fededf [CSSPGO] A Clang switch -fpseudo-probe-for-profiling for pseudo-probe instrumentation.
This change introduces a new clang switch `-fpseudo-probe-for-profiling` to enable AutoFDO with pseudo instrumentation. Please refer to https://reviews.llvm.org/D86193 for the whole story.

One implication from pseudo-probe instrumentation is that the profile is now sensitive to CFG changes. We perform the pseudo instrumentation very early in the pre-LTO pipeline, before any CFG transformation. This ensures that the CFG instrumented and annotated is stable and optimization-resilient.

The early instrumentation also allows the inliner to duplicate probes for inlined instances. When a probe along with the other instructions of a callee function are inlined into its caller function, the GUID of the callee function goes with the probe. This allows samples collected on inlined probes to be reported for the original callee function.

Reviewed By: wmi

Differential Revision: https://reviews.llvm.org/D86502
2020-11-30 10:16:54 -08:00
Hongtao Yu 64fa8cce22 [CSSPGO] Pseudo probe instrumentation pass
This change introduces a pseudo probe instrumentation pass for block instrumentation. Please refer to https://reviews.llvm.org/D86193 for the whole story.

Given the following LLVM IR:

```
define internal void @foo2(i32 %x, void (i32)* %f) !dbg !4 {
bb0:
  %cmp = icmp eq i32 %x, 0
   br i1 %cmp, label %bb1, label %bb2
bb1:
   br label %bb3
bb2:
   br label %bb3
bb3:
   ret void
}
```

The instrumented IR will look like below. Note that each llvm.pseudoprobe intrinsic call represents a pseudo probe at a block, of which the first parameter is the GUID of the probe’s owner function and the second parameter is the probe’s ID.

```
define internal void @foo2(i32 %x, void (i32)* %f) !dbg !4 {
bb0:
   %cmp = icmp eq i32 %x, 0
   call void @llvm.pseudoprobe(i64 837061429793323041, i64 1)
   br i1 %cmp, label %bb1, label %bb2
bb1:
   call void @llvm.pseudoprobe(i64 837061429793323041, i64 2)
   br label %bb3
bb2:
   call void @llvm.pseudoprobe(i64 837061429793323041, i64 3)
   br label %bb3
bb3:
   call void @llvm.pseudoprobe(i64 837061429793323041, i64 4)
   ret void
}
```

Reviewed By: wmi

Differential Revision: https://reviews.llvm.org/D86499
2020-11-30 10:16:54 -08:00
Sanjay Patel 355aee3dcd Revert "[IR][LoopRotate] avoid leaving phi with no operands (PR48296)"
This reverts commit bfd2c216ea.
This appears to be causing stage2 msan failures on buildbots:
  FAIL: LLVM :: Transforms/SimplifyCFG/X86/bug-25299.ll (65872 of 71835)
  ******************** TEST 'LLVM :: Transforms/SimplifyCFG/X86/bug-25299.ll' FAILED ********************
  Script:
  --
  : 'RUN: at line 1';   /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/opt < /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/Transforms/SimplifyCFG/X86/bug-25299.ll -simplifycfg -S | /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/FileCheck /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/Transforms/SimplifyCFG/X86/bug-25299.ll
  --
  Exit Code: 2
  Command Output (stderr):
  --
  ==87374==WARNING: MemorySanitizer: use-of-uninitialized-value
      #0 0x9de47b6 in getBasicBlockIndex /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/include/llvm/IR/Instructions.h:2749:5
      #1 0x9de47b6 in simplifyCommonResume /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/Transforms/Utils/SimplifyCFG.cpp:4112:23
      #2 0x9de47b6 in simplifyResume /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/Transforms/Utils/SimplifyCFG.cpp:4039:12
      #3 0x9de47b6 in (anonymous namespace)::SimplifyCFGOpt::simplifyOnce(llvm::BasicBlock*) /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/Transforms/Utils/SimplifyCFG.cpp:6330:16
      #4 0x9dcca13 in run /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/Transforms/Utils/SimplifyCFG.cpp:6358:16
      #5 0x9dcca13 in llvm::simplifyCFG(llvm::BasicBlock*, llvm::TargetTransformInfo const&, llvm::SimplifyCFGOptions const&, llvm::SmallPtrSetImpl<llvm::BasicBlock*>*) /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/Transforms/Utils/SimplifyCFG.cpp:6369:8
      #6 0x974643d in iterativelySimplifyCFG(
2020-11-30 10:15:42 -05:00
Sanjay Patel bfd2c216ea [IR][LoopRotate] avoid leaving phi with no operands (PR48296)
https://llvm.org/PR48296 shows an example where we delete all of the operands
of a phi without actually deleting the phi, and that is currently considered
invalid IR. The reduced test included here would crash for that reason.

A suggested follow-up is to loosen the assert to allow 0-operand phis
in unreachable blocks.

Differential Revision: https://reviews.llvm.org/D92247
2020-11-30 09:28:45 -05:00
Nikita Popov 891170e863 [DL] Optimize address space zero lookup (NFC)
Information for pointer size/alignment/etc is queried a lot, but
the binary search based implementation makes this fairly slow.

Add an explicit check for address space zero and skip the search
in that case -- we need to specially handle the zero address space
anyway, as it serves as the fallback for all address spaces that
were not explicitly defined.

I initially wanted to simply replace the binary search with a
linear search, which would handle both address space zero and the
general case efficiently, but I was not sure whether there are
any degenerate targets that use more than a handful of declared
address spaces (in-tree, even AMDGPU only declares six).
2020-11-29 22:49:55 +01:00
Paul C. Anagnostopoulos 0aeaec13e7 [Timer] Add a command option to enable/disable timer sorting.
Add one more timer to DAGISelEmitter to test the option.

Differential Revision: https://reviews.llvm.org/D92146
2020-11-28 11:43:38 -05:00
Andrew Litteken a8a43b6338 Revert "[IRSim][IROutliner] Adding the extraction basics for the IROutliner."
Reverting commit due to address sanitizer errors.

> Extracting the similar regions is the first step in the IROutliner.
> 
> Using the IRSimilarityIdentifier, we collect the SimilarityGroups and
> sort them by how many instructions will be removed.  Each
> IRSimilarityCandidate is used to define an OutlinableRegion.  Each
> region is ordered by their occurrence in the Module and the regions that
> are not compatible with previously outlined regions are discarded.
> 
> Each region is then extracted with the CodeExtractor into its own
> function.
> 
> We test that correctly extract in:
> test/Transforms/IROutliner/extraction.ll
> test/Transforms/IROutliner/address-taken.ll
> test/Transforms/IROutliner/outlining-same-globals.ll
> test/Transforms/IROutliner/outlining-same-constants.ll
> test/Transforms/IROutliner/outlining-different-structure.ll
> 
> Reviewers: paquette, jroelofs, yroux
> 
> Differential Revision: https://reviews.llvm.org/D86975

This reverts commit bf899e8913.
2020-11-27 19:55:57 -06:00
Andrew Litteken bf899e8913 [IRSim][IROutliner] Adding the extraction basics for the IROutliner.
Extracting the similar regions is the first step in the IROutliner.

Using the IRSimilarityIdentifier, we collect the SimilarityGroups and
sort them by how many instructions will be removed.  Each
IRSimilarityCandidate is used to define an OutlinableRegion.  Each
region is ordered by their occurrence in the Module and the regions that
are not compatible with previously outlined regions are discarded.

Each region is then extracted with the CodeExtractor into its own
function.

We test that correctly extract in:
test/Transforms/IROutliner/extraction.ll
test/Transforms/IROutliner/address-taken.ll
test/Transforms/IROutliner/outlining-same-globals.ll
test/Transforms/IROutliner/outlining-same-constants.ll
test/Transforms/IROutliner/outlining-different-structure.ll

Reviewers: paquette, jroelofs, yroux

Differential Revision: https://reviews.llvm.org/D86975
2020-11-27 19:08:29 -06:00
Martin Storsjö fa10383664 Revert "[BasicAA] Fix BatchAA results for phi-phi assumptions"
This reverts commit 8166ed1a7a,
as it caused some compilations to hang/loop indefinitely, see
https://reviews.llvm.org/D91936 for details.
2020-11-27 21:50:59 +02:00
diggerlin 851779652a [AIX][XCOFF][NFC] Change geNumberOfVRSaved function name to getNumberOfVRSaved.
SUMMARY:

  Change geNumberOfVRSaved function name to getNumberOfVRSaved of class TBVectorExt

Reviewers: hubert.reinterpretcast, Jason Liu
Differential Revision: https://reviews.llvm.org/D92225
2020-11-27 13:37:43 -05:00
Francesco Petrogalli 8e0148dff7 [AllocaInst] Update `getAllocationSizeInBits` to return `TypeSize`.
Reviewed By: peterwaller-arm, sdesmalen

Differential Revision: https://reviews.llvm.org/D92020
2020-11-27 16:39:10 +00:00
Jay Foad 0d9166ff79 [LegacyPM] Remove unused undocumented parameter. NFC.
The Direction parameter to AnalysisResolver::getAnalysisIfAvailable has
never been documented or used for anything.
2020-11-27 10:41:38 +00:00
Markus Lavin 808fcfe594 Revert "[DebugInfo] Improve dbg preservation in LSR."
This reverts commit 06758c6a61.

Bug: https://bugs.llvm.org/show_bug.cgi?id=48166
Additional discussion in: https://reviews.llvm.org/D91711
2020-11-27 08:52:32 +01:00
Kazu Hirata 9178c630c3 [IR] Use llvm::is_contained and pred_size (NFC) 2020-11-26 22:02:04 -08:00
QingShan Zhang 4d83aba422 [DAGCombine] Adding a hook to improve the precision of fsqrt if the input is denormal
For now, we will hardcode the result as 0.0 if the input is denormal or 0. That will
have the impact the precision. As the fsqrt added belong to the cold path of the
cmp+branch, it won't impact the performance for normal inputs for PowerPC, but improve
the precision if the input is denormal.

Reviewed By: Spatel

Differential Revision: https://reviews.llvm.org/D80974
2020-11-27 02:10:55 +00:00
Nikita Popov 8166ed1a7a [BasicAA] Fix BatchAA results for phi-phi assumptions
Add a flag that disables caching when computing aliasing results
potentially based on a phi-phi NoAlias assumption. We'll still
insert cache entries temporarily to catch infinite recursion,
but will drop them afterwards, so they won't persist in BatchAA.

Differential Revision: https://reviews.llvm.org/D91936
2020-11-26 21:43:50 +01:00
Nikita Popov 4df8efce80 [AA] Split up LocationSize::unknown()
Currently, we have some confusion in the codebase regarding the
meaning of LocationSize::unknown(): Some parts (including most of
BasicAA) assume that LocationSize::unknown() only allows accesses
after the base pointer. Some parts (various callers of AA) assume
that LocationSize::unknown() allows accesses both before and after
the base pointer (but within the underlying object).

This patch splits up LocationSize::unknown() into
LocationSize::afterPointer() and LocationSize::beforeOrAfterPointer()
to make this completely unambiguous. I tried my best to determine
which one is appropriate for all the existing uses.

The test changes in cs-cs.ll in particular illustrate a previously
clearly incorrect AA result: We were effectively assuming that
argmemonly functions were only allowed to access their arguments
after the passed pointer, but not before it. I'm pretty sure that
this was not intentional, and it's certainly not specified by
LangRef that way.

Differential Revision: https://reviews.llvm.org/D91649
2020-11-26 18:39:55 +01:00
Florian Hahn 926681b6be
[CostModel] Add basic implementation of getGatherScatterOpCost.
Add a basic implementation of getGatherScatterOpCost to BasicTTIImpl.

The implementation estimates the cost of scalarizing the loads/stores,
the cost of packing/extracting the individual lanes and the cost of
only selecting enabled lanes.

This more accurately reflects the current cost on targets like AArch64.

Reviewed By: dmgreen

Differential Revision: https://reviews.llvm.org/D91984
2020-11-26 12:02:25 +00:00
Kazu Hirata 728634d0cf [Support] Use llvm::is_contained (NFC) 2020-11-25 23:52:52 -08:00
Zhengyang Liu 75f50e15bf Adding PoisonValue for representing poison value explicitly in IR
Define ConstantData::PoisonValue.
Add support for poison value to LLLexer/LLParser/BitcodeReader/BitcodeWriter.
Add support for poison value to llvm-c interface.
Add support for poison value to OCaml binding.
Add m_Poison in PatternMatch.

Differential Revision: https://reviews.llvm.org/D71126
2020-11-25 17:33:51 -07:00
Paul Robinson cf1c774d6a [FastISel] Flush local value map on ever instruction
Local values are constants or addresses that can't be folded into
the instruction that uses them. FastISel materializes these in a
"local value" area that always dominates the current insertion
point, to try to avoid materializing these values more than once
(per block).

https://reviews.llvm.org/D43093 added code to sink these local
value instructions to their first use, which has two beneficial
effects. One, it is likely to avoid some unnecessary spills and
reloads; two, it allows us to attach the debug location of the
user to the local value instruction. The latter effect can
improve the debugging experience for debuggers with a "set next
statement" feature, such as the Visual Studio debugger and PS4
debugger, because instructions to set up constants for a given
statement will be associated with the appropriate source line.

There are also some constants (primarily addresses) that could be
produced by no-op casts or GEP instructions; the main difference
from "local value" instructions is that these are values from
separate IR instructions, and therefore could have multiple users
across multiple basic blocks. D43093 avoided sinking these, even
though they were emitted to the same "local value" area as the
other instructions. The patch comment for D43093 states:

  Local values may also be used by no-op casts, which adds the
  register to the RegFixups table. Without reversing the RegFixups
  map direction, we don't have enough information to sink these
  instructions.

This patch undoes most of D43093, and instead flushes the local
value map after(*) every IR instruction, using that instruction's
debug location. This avoids sometimes incorrect locations used
previously, and emits instructions in a more natural order.

This does mean materialized values are not re-used across IR
instruction boundaries; however, only about 5% of those values
were reused in an experimental self-build of clang.

(*) Actually, just prior to the next instruction. It seems like
it would be cleaner the other way, but I was having trouble
getting that to work.

Differential Revision: https://reviews.llvm.org/D91734
2020-11-25 13:05:00 -05:00
Joe Ellis 06654a5348 [SVE] Fix TypeSize warning in RuntimePointerChecking::insert
The TypeSize warning would occur because RuntimePointerChecking::insert
was not scalable vector aware. The fix is to use
ScalarEvolution::getSizeOfExpr to grab the size of types.

Differential Revision: https://reviews.llvm.org/D90171
2020-11-25 16:59:03 +00:00
Andy Wingo feac819e50 [MC][WebAssembly] Only emit indirect function table import if needed
The indirect function table, synthesized by the linker, is needed if and
only if there are TABLE_INDEX relocs.

Differential Revision: https://reviews.llvm.org/D91637
2020-11-25 08:38:43 -08:00
Andy Wingo 1933c9d41a [WebAssembly] Factor out WasmTableType in binary format
This commit factors out a WasmTableType definition from WasmTable, as is
the case for WasmGlobal and other data types.  Also add support for
extracting the SymbolName for a table from the linking section's symbol
table.

Differential Revision: https://reviews.llvm.org/D91849
2020-11-25 08:00:08 -08:00
Cullen Rhodes 1ba4b82f67 [LAA] NFC: Rename [get]MaxSafeRegisterWidth -> [get]MaxSafeVectorWidthInBits
MaxSafeRegisterWidth is a misnomer since it actually returns the maximum
safe vector width. Register suggests it relates directly to a physical
register where it could be a vector spanning one or more physical
registers.

Reviewed By: sdesmalen

Differential Revision: https://reviews.llvm.org/D91727
2020-11-25 13:06:26 +00:00
Mark Murray 2b6691894a [ARM][AArch64] Adding Neoverse N2 CPU support
Add support for the Neoverse N2 CPU to the ARM and AArch64 backends.

Differential Revision: https://reviews.llvm.org/D91695
2020-11-25 11:42:54 +00:00
Martin Storsjö d2b71ea03d [ORC] Cast to const void* to silence a GCC warning. NFC. 2020-11-25 12:10:53 +02:00
Georgii Rymar fee910e522 [libObject,llvm-readelf] - Stop describing a section/segment in `notes_begin()`.
`notes_begin()` is used for iterating over notes. This API in some cases might print
section type and index. At the same time during iterating, the `Elf_Note_Iterator`
might omit it as it doesn't have this info.

Because of above we might have the redundant duplication of information in warnings:
(See D92021).

```
warning: '[[FILE]]': unable to read notes from the SHT_NOTE section with index 1: SHT_NOTE section [index 1] has invalid offset (0x40) or size (0xffff0000)
```

This change stops reporting section index/type in Object/ELF.h/notes_begin().
(FTR, this was introduced by me for llvm-readobj in D64470).
Instead we can describe sections/program headers on the caller side.

Differential revision: https://reviews.llvm.org/D92081
2020-11-25 12:51:40 +03:00
Georgii Rymar 5edb90c927 [obj2yaml] - Dump section offsets in some cases.
Currently we never dump the `sh_offset` key.
Though it sometimes an important information.

To reduce the noise this patch implements the following logic:
1) The "Offset" key for the first section is always emitted.
2) If we can derive the offset for a next section naturally,
   then the "Offset" key is omitted.

By "naturally" I mean that section[X] offset is expected to be:
```
offsetOf(section[X]) == alignTo(section[X - 1].sh_offset + section[X - 1].sh_size, section[X].sh_addralign)
```

So, when it has the expected value, we omit it from the output.

Differential revision: https://reviews.llvm.org/D91152
2020-11-25 12:41:01 +03:00
QingShan Zhang 9c588f53fc [DAGCombine] Add hook to allow target specific test for sqrt input
PowerPC has instruction ftsqrt/xstsqrtdp etc to do the input test for software square root.
LLVM now tests it with smallest normalized value using abs + setcc. We should add hook to
target that has test instructions.

Reviewed By: Spatel, Chen Zheng, Qiu Chao Fang

Differential Revision: https://reviews.llvm.org/D80706
2020-11-25 05:37:15 +00:00
Arthur Eubanks 2c7870dcca [NewPM] Add pipeline EP callback after initial frontend cleanup
This matches the legacy PM's EP_ModuleOptimizerEarly. Some backends use
this extension point and adding the pass somewhere else like
PipelineStartEPCallback doesn't work.

Reviewed By: ychen

Differential Revision: https://reviews.llvm.org/D91804
2020-11-24 21:14:36 -08:00
Kai Luo 8e6d92026c [DAG][PowerPC] Fix dropped `nsw` flag in `SimplifySetCC` by adding `doesNodeExist` helper
`SimplifySetCC` invokes `getNodeIfExists` without passing `Flags` argument and `getNodeIfExists` uses a default `SDNodeFlags` to intersect the original flags, as a consequence, flags like `nsw` is dropped. Added a new helper function `doesNodeExist` to check if a node exists without modifying its flags.

Reviewed By: #powerpc, nemanjai

Differential Revision: https://reviews.llvm.org/D89938
2020-11-25 04:39:03 +00:00
Zarko Todorovski be7d425edc [PPC][AIX] Add vector callee saved registers for AIX extended vector ABI
This patch is the initial patch for support of the AIX extended vector ABI.  The extended ABI treats vector registers V20-V31 as non-volatile and we add them as callee saved registers in this patch.

Reviewed By: sfertile

Differential Revision: https://reviews.llvm.org/D88676
2020-11-24 23:01:51 -05:00
Hongtao Yu abbf4802bb [SelectionDAG] Add PseudoProbeSDNode to LargestSDNode to fix 32-bt build break. 2020-11-24 15:35:08 -08:00
Zarko Todorovski c92f29b05e [AIX] Add mabi=vec-extabi options to enable the AIX extended and default vector ABIs.
Added support for the options mabi=vec-extabi and mabi=vec-default which are analogous to qvecnvol and qnovecnvol when using XL on AIX.
The extended Altivec ABI on AIX is enabled using mabi=vec-extabi in clang and vec-extabi in llc.

Reviewed By: Xiangling_L, DiggerLin

Differential Revision: https://reviews.llvm.org/D89684
2020-11-24 18:17:53 -05:00
Janek van Oirschot 42eaf4fe0a [HardwareLoops] Change order of SCEV expression construction for InitLoopCount.
Putting the +1 before the zero-extend will allow scalar evolution to fold the expression in some cases such as the one shown in PowerPC's `shrink-wrap.ll` test.

Reviewed By: samparker

Differential Revision: https://reviews.llvm.org/D91724
2020-11-24 18:01:42 +00:00
diggerlin c80fbdf291 [NFC][AIX][XCOFF] change function name from getNumofGPRsSaved to getNumOfGPRsSaved
change function name from getNumofGPRsSaved to getNumOfGPRsSaved for class XCOFFTracebackTable

Reviewers: Jason Liu
Differential Revision: https://reviews.llvm.org/D91882
2020-11-24 10:23:57 -05:00
Max Kazantsev 02fdbc3567 Revert "[NFC][SCEV] Generalize monotonicity check for full and limited iteration space"
This reverts commit 2734a9ebf4.

This patch appeared to not be a NFC. It introduced an execution path where
monotonicity check on limited space started relying in existing nsw/nuw
flags, which is illegal. The motivating test will follow-up.
2020-11-24 17:56:59 +07:00
Arthur Eubanks aff058b1a9 Reland [CGSCC] Detect devirtualization in more cases
The devirtualization wrapper misses cases where if it wraps a pass
manager, an individual pass may devirtualize an indirect call created by
a previous pass. For example, inlining may create a new indirect call
which is devirtualized by instcombine. Currently the devirtualization
wrapper will not see that because it only checks cgscc edges at the very
beginning and end of the pass (manager) it wraps.

This fixes some tests testing this exact behavior in the legacy PM.

Instead of checking WeakTrackingVHs for CallBases at the very beginning
and end of the pass it wraps, check every time
updateCGAndAnalysisManagerForPass() is called.

check-llvm and check-clang with -abort-on-max-devirt-iterations-reached
on by default doesn't show any failures outside of tests specifically
testing it so it doesn't needlessly rerun passes more than necessary.
(The NPM -O2/3 pipeline run the inliner/function simplification pipeline
under a devirtualization repeater pass up to 4 times by default).

http://llvm-compile-time-tracker.com/?config=O3&stat=instructions&remote=aeubanks
shows that 7zip has ~1% compile time regression. I looked at it and saw
that there indeed was devirtualization happening that was not previously
caught, so now it reruns the CGSCC pipeline on some SCCs, which is WAI.

The initial land assumed CallBase WeakTrackingVHs would always be
CallBases, but they can be RAUW'd with undef.

Reviewed By: asbirlea

Differential Revision: https://reviews.llvm.org/D89587
2020-11-23 21:28:59 -08:00
Jameson Nash 8f1e0dcbb0 fix some Wundef warnings in public headers
Differential Revision: https://reviews.llvm.org/D91094
2020-11-23 20:17:46 -05:00
Amy Huang 1b63177a56 Revert "[llvm-symbolizer] Switch to using native symbolizer by default on Windows"
Breaks some asan tests on the buildbot.

This reverts commit c74b427cb2.
2020-11-23 16:29:45 -08:00
Amy Huang c74b427cb2 [llvm-symbolizer] Switch to using native symbolizer by default on Windows
llvm-symbolizer used to use the DIA SDK for symbolization on
Windows; this patch switches to using native symbolization, which was
implemented recently.

Users can still make the symbolizer use DIA by adding the `-dia` flag
in the LLVM_SYMBOLIZER_OPTS environment variable.

Differential Revision: https://reviews.llvm.org/D91814
2020-11-23 15:57:08 -08:00