Commit Graph

12656 Commits

Author SHA1 Message Date
Kazu Hirata 1a2d67fa23 [llvm] Use llvm::lower_bound and llvm::upper_bound (NFC) 2021-01-29 23:23:36 -08:00
Kazu Hirata 7728cc003a [llvm] Use append_range (NFC) 2021-01-29 23:23:34 -08:00
Greg McGary 61a5502a93 [llvm-objdump-macho] print per-second-level-page encodings for option --unwind-info
Compact unwind entries have 8 bits for the encoding-table offset:
* offsets 0..126 reference the global commmon-encodings table, while
* offsets 127..255 reference a per-second-level-page table.
This diff teaches `llvm-objdump` to print this per-page encodings table.

Differential Revision: https://reviews.llvm.org/D93265
2021-01-29 21:59:07 -07:00
Florian Hahn f3a710cade [LTO] Update splitCodeGen to take a reference to the module. (NFC)
splitCodeGen does not need to take ownership of the module, as it
currently clones the original module for each split operation.

There is an ~4 year old fixme to change that, but until this is
addressed, the function can just take a reference to the module.

This makes the transition of LTOCodeGenerator to use LTOBackend a bit
easier, because under some circumstances, LTOCodeGenerator needs to
write the original module back after codegen.

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D95222
2021-01-29 11:53:11 +00:00
Rafik Zurob cba2552bfe [llvm-jitlink] Replace use of deprecated gethostbyname by getaddrinfo.
This patch replaces use of deprecated gethostbyname by getaddrinfo.

Author: Rafik Zurob

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D95477
2021-01-29 03:11:16 -06:00
Georgii Rymar a5154ab9b0 [llvm-readobj/elf] - Report "bitcode files are not supported" warning for bitcode files.
Fixes https://bugs.llvm.org/show_bug.cgi?id=43543

Currently we report "The file was not recognized as a valid object file" for BC files.
Also, we terminate dumping.

Instead we could report a better warning and try to continue dumping other files.
This is what this patch implements.

Differential revision: https://reviews.llvm.org/D95605
2021-01-29 12:04:41 +03:00
Yang Fan d6d0c09e84
[NFC][llvm-nm] Fix unused variable warning 2021-01-29 11:42:23 +08:00
Fangrui Song b3af96d07b [llvm-nm] Display defined weak STT_GNU_IFUNC symbols as 'i'
This patch makes the behavior match GNU nm.
Note: undefined STT_GNU_IFUNC symbols use 'U'.

Differential Revision: https://reviews.llvm.org/D95461
2021-01-28 09:46:05 -08:00
Hongtao Yu 7e99bddfea [CSSPGO] Support of CS profiles in extended binary format.
This change brings up support of context-sensitive profiles in the format of extended binary. Existing sample profile reader/writer/merger code is being tweaked to reflect the fact of bracketed input contexts, like (`[...]`). The paired brackets are also needed in extbinary profiles because we don't yet have an otherwise good way to tell calling contexts apart from regular function names since the context delimiter `@` can somehow serve as a part of the C++ mangled names.

Reviewed By: wmi, wenlei

Differential Revision: https://reviews.llvm.org/D95547
2021-01-27 21:29:46 -08:00
Teresa Johnson 1487747e99 [LTO] Prevent devirtualization for symbols dynamically exported
Identify dynamically exported symbols (--export-dynamic[-symbol=],
--dynamic-list=, or definitions needed to preempt shared objects) and
prevent their LTO visibility from being upgraded.
This helps avoid use of whole program devirtualization when there may
be overrides in dynamic libraries.

Differential Revision: https://reviews.llvm.org/D91583
2021-01-27 15:54:13 -08:00
Craig Topper 0b50fa9945 [FaultsMaps][llvm-objdump] Move FaultMapParser to Object/. Remove CodeGen dependency from llvm-objdump
FaultsMapParser lived in CodeGen and was forcing llvm-objdump to
link CodeGen and everything CodeGen depends on.

This was previously attempted in r240364 to fix a link failure.
The CodeGen dependency was independently added to fix the same
link failure, and that ended up being kept.

Removing the dependency seems like the correct layering for
llvm-objdump.

Reviewed By: MaskRay, jhenderson

Differential Revision: https://reviews.llvm.org/D95414
2021-01-27 10:39:59 -08:00
Kazu Hirata 48bdd676a1 [llvm-objdump] Use append_range (NFC) 2021-01-26 20:00:19 -08:00
Fangrui Song 4d28f0a6a4 [llc] Add reportError helper and canonicalize error messages 2021-01-26 15:33:37 -08:00
Fangrui Song 34b60d8a56 Add -fbinutils-version= to gate ELF features on the specified binutils version
There are two use cases.

Assembler
We have accrued some code gated on MCAsmInfo::useIntegratedAssembler().  Some
features are supported by latest GNU as, but we have to use
MCAsmInfo::useIntegratedAs() because the newer versions have not been widely
adopted (e.g. SHF_LINK_ORDER 'o' and 'unique' linkage in 2.35, --compress-debug-sections= in 2.26).

Linker
We want to use features supported only by LLD or very new GNU ld, or don't want
to work around older GNU ld. We currently can't represent that "we don't care
about old GNU ld".  You can find such workarounds in a few other places, e.g.
Mips/MipsAsmprinter.cpp PowerPC/PPCTOCRegDeps.cpp X86/X86MCInstrLower.cpp
AArch64 TLS workaround for R_AARCH64_TLSLD_MOVW_DTPREL_* (PR ld/18276),
R_AARCH64_TLSLE_LDST8_TPREL_LO12 (https://bugs.llvm.org/show_bug.cgi?id=36727 https://sourceware.org/bugzilla/show_bug.cgi?id=22969)

Mixed SHF_LINK_ORDER and non-SHF_LINK_ORDER components (supported by LLD in D84001;
GNU ld feature request https://sourceware.org/bugzilla/show_bug.cgi?id=16833 may take a while before available).
This feature allows to garbage collect some unused sections (e.g. fragmented .gcc_except_table).

This patch adds `-fbinutils-version=` to clang and `-binutils-version` to llc.
It changes one codegen place in SHF_MERGE to demonstrate its usage.
`-fbinutils-version=2.35` means the produced object file does not care about GNU
ld<2.35 compatibility. When `-fno-integrated-as` is specified, the produced
assembly can be consumed by GNU as>=2.35, but older versions may not work.

`-fbinutils-version=none` means that we can use all ELF features, regardless of
GNU as/ld support.

Both clang and llc need `parseBinutilsVersion`. Such command line parsing is
usually implemented in `llvm/lib/CodeGen/CommandFlags.cpp` (LLVMCodeGen),
however, ClangCodeGen does not depend on LLVMCodeGen. So I add
`parseBinutilsVersion` to `llvm/lib/Target/TargetMachine.cpp` (LLVMTarget).

Differential Revision: https://reviews.llvm.org/D85474
2021-01-26 12:28:23 -08:00
Martin Storsjö 510b3d4b3e [llvm-nm] Silence a gcc warning about a stray semicolon. NFC. 2021-01-26 12:29:14 +02:00
Georgii Rymar db92d47cf7 [llvm-nm][ELF] - Use @@ prefix when printing default versions.
llvm-readelf prints default versions with `@@` prefix.
This patch does the same for llvm-nm.

Differential revision: https://reviews.llvm.org/D94912
2021-01-26 12:16:38 +03:00
Georgii Rymar e98d5c3192 [libObject,llvm-readelf/obj] - Don't use @@ when printing versions of undefined symbols.
A default version (@@) is only available for defined symbols.

Currently we use "@@" for undefined symbols too.
This patch fixes the issue and improves our test case.

Differential revision: https://reviews.llvm.org/D95219
2021-01-26 12:05:59 +03:00
Philip Pfaffe da489946a9 [llvm-dwp] Automatically set the target triple
The llvm-dwp tool hard-codes the target triple to x86. Instead, deduce the
target triple from the object files being read.

Reviewed By: dblaikie

Differential Revision: https://reviews.llvm.org/D93749
2021-01-25 11:58:54 +01:00
Georgii Rymar 9c89dcf807 [yaml2obj, obj2yaml] - Implement section header table as a special Chunk.
This was discussed in D93678 thread.
Currently we have one special chunk - Fill.

This patch re implements the "SectionHeaderTable" key to become a special chunk too.
With that we are able to place the section header table at any location,
just like we place sections.

Differential revision: https://reviews.llvm.org/D95140
2021-01-25 13:08:08 +03:00
Florian Hahn f959d8195d
[LTO] Move DisableVerify setting to LTOCodeGenerator class (NFC).
To simplify the transition to using LTOBackend, move DisableVerify to
the LTOCodeGenerator class, like most/all other options.

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D95223
2021-01-24 14:14:40 +00:00
Arthur Eubanks c37dd3b6d5 [NewPM][opt] Make -enable-new-pm default to LLVM_ENABLE_NEW_PASS_MANAGER
This is controlled by the ENABLE_EXPERIMENTAL_NEW_PASS_MANAGER CMake flag.

https://lists.llvm.org/pipermail/llvm-dev/2021-January/147993.html

Reviewed By: ychen, asbirlea

Differential Revision: https://reviews.llvm.org/D95254
2021-01-23 12:36:09 -08:00
Florian Hahn 166d40f2ed
[FuzzMutate] Add mutator to modify instruction flags.
This patch adds a new InstModificationIRStrategy to mutate flags/options
for instructions. For example, it may add or remove nuw/nsw flags from
add, mul, sub, shl instructions or change the predicate for icmp
instructions.

Subtle changes such as those mentioned above should lead to a more
interesting range of inputs. The presence or absence of overflow flags
can expose subtle bugs, for example.

Reviewed By: bogner

Differential Revision: https://reviews.llvm.org/D94905
2021-01-23 19:05:20 +00:00
Florian Hahn 08dbcc14e2
[LTO] Store target attributes as vector of strings (NFC).
The target features are obtained as a list of features/attributes.
Instead of storing them in a single string, store the vector. This
matches lto::Config's behavior and simplifies the transition to
lto::backend().

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D95224
2021-01-23 12:11:58 +00:00
Florian Hahn 2a8cbdd830
[LTO] Add support for existing Config::Freestanding option.
lto::Config has a field to control whether the build is "freestanding"
(no builtins) or not, but it is not hooked up to the code actually
running the passes.

This patch adds support for the flag to both the code that runs
optimization with the new and old pass managers, by explicitly adding a
TargetLibraryInfo instance. If Freestanding is true, all library functions
are disabled.

Reviewed By: steven_wu

Differential Revision: https://reviews.llvm.org/D94630
2021-01-22 13:45:39 +00:00
Arthur Eubanks 6699029b67 [NewPM][opt] Run the "default" AA pipeline by default
We tend to assume that the AA pipeline is by default the default AA
pipeline and it's confusing when it's empty instead.

PR48779

Initially reverted due to BasicAA running analyses in an unspecified
order (multiple function calls as parameters), fixed by fetching
analyses before the call to construct BasicAA.

Reviewed By: asbirlea

Differential Revision: https://reviews.llvm.org/D95117
2021-01-21 21:08:54 -08:00
Arthur Eubanks ba9b4ea4ee Revert "[NewPM][opt] Run the "default" AA pipeline by default"
This reverts commit be611431cd.

Other/new-pm-lto-defaults.ll failing
2021-01-21 20:16:34 -08:00
Kazu Hirata cfa241680f [llvm] Don't include StringSwitch.h where unnecessary (NFC) 2021-01-21 19:59:48 -08:00
Arthur Eubanks be611431cd [NewPM][opt] Run the "default" AA pipeline by default
We tend to assume that the AA pipeline is by default the default AA
pipeline and it's confusing when it's empty instead.

PR48779

Reviewed By: asbirlea

Differential Revision: https://reviews.llvm.org/D95117
2021-01-21 19:46:38 -08:00
Wolfgang Pieb c6e8f81410 [llvm-mca] Addressing build failures due to missing override specifiers 2021-01-21 17:32:18 -08:00
Wolfgang Pieb 04af1ca2e9 [llvm-mca] Forgot a couple of override specifiers.
Differential Revision: https://reviews.llvm.org/D86644
2021-01-21 15:44:14 -08:00
Wolfgang Pieb d38be2ba0e [llvm-mca] Initial implementation of serialization using JSON. The views
implemented at this time are Summary, Timeline, ResourcePressure and InstructionInfo.
Use --json on the command line to obtain JSON output.
2021-01-21 15:15:54 -08:00
Georgii Rymar dd5c982804 [llvm-nm][ELF] - Make -D display symbol versions.
This fixes https://bugs.llvm.org/show_bug.cgi?id=48670.

Since binutils 2.35, nm -D displays symbol versions by default.
This patch teaches llvm-nm to do the same.

Differential revision: https://reviews.llvm.org/D94907
2021-01-21 11:23:45 +03:00
Georgii Rymar 51f4958057 [yaml2obj/obj2yaml] - Improve dumping/creating of ELF versioning sections.
This makes the following improvements.

For `SHT_GNU_versym`:
 * yaml2obj: set `sh_link` to index of `.dynsym` section automatically.
For `SHT_GNU_verdef`:
 * yaml2obj: set `sh_link` to index of `.dynstr` section automatically.
 * yaml2obj: set `sh_info` field automatically.
 * obj2yaml: don't dump the `Info` field when its value matches the number of version definitions.
For `SHT_GNU_verneed`:
 * yaml2obj: set `sh_link` to index of `.dynstr` section automatically.
 * yaml2obj: set `sh_info` field automatically.
 * obj2yaml: don't dump the `Info` field when its value matches the number of version dependencies.

Also, simplifies few test cases.

Differential revision: https://reviews.llvm.org/D94956
2021-01-21 10:36:48 +03:00
Jonas Devlieghere f354b87df2 [dsymutil] Compare object modification times using second precision
The modification time in the debug map is expressed using second
precision, while the modification time returned by the filesystem could
be more precise. Avoid spurious warnings about timestamp mismatches by
truncating the modification time reported by the system to seconds.
2021-01-20 18:45:30 -08:00
Kazu Hirata 978c754076 [llvm] Use llvm::any_of (NFC) 2021-01-19 20:19:16 -08:00
wlei daeea961a6 [llvm-profgen][NFC] Fix the incorrect computation of callsite sample count
Differential Revision: https://reviews.llvm.org/D95009
2021-01-19 17:50:48 -08:00
Sergey Dmitriev 233106269d [llvm-link] Improve link time for bitcode archives [NFC]
Linking large bitcode archives currently takes a lot of time with llvm-link,
this patch adds couple improvements which reduce link time for archives
- Use one Linker instance for archive instead of recreating it for each member
- Lazy load archive members

Reviewed By: tra, jdoerfert

Differential Revision: https://reviews.llvm.org/D94643
2021-01-19 16:41:28 -08:00
Arthur Eubanks cabe1b1124 [polly][NewPM][test] Fix polly tests under -enable-new-pm
In preparation for turning on opt's -enable-new-pm by default, this pins
uses of passes via the legacy "opt -passname" with pass names beginning
with "polly-" and "polyhedral-info" to the legacy PM. Many of these
tests use -analyze, which isn't supported in the new PM.

(This doesn't affect uses of "opt -passes=passname").

rL240766 accidentally removed `-polly-prepare` in
phi_not_grouped_at_top.ll, and it also doesn't use the output of
-analyze.

Reviewed By: Meinersbur

Differential Revision: https://reviews.llvm.org/D94266
2021-01-19 12:38:58 -08:00
Kazu Hirata 23b0ab2acb [llvm] Use the default value of drop_begin (NFC) 2021-01-18 10:16:36 -08:00
Georgii Rymar b9ce772b8f [Object, llvm-readelf] - Move the API for retrieving symbol versions to ELF.h
`ELFDumper.cpp` implements the functionality that allows to get symbol versions.
It is used for dumping versioned symbols.

This helps to implement https://bugs.llvm.org/show_bug.cgi?id=48670 ("make llvm-nm -D print version names"):
we can move out and reuse the code from `ELFDumper.cpp`.
This is what this patch do: it moves the related functionality to `ELFFile<ELFT>`.

Differential revision: https://reviews.llvm.org/D94771
2021-01-18 12:50:29 +03:00
Kazu Hirata 352fcfc697 [llvm] Use llvm::sort (NFC) 2021-01-17 10:39:45 -08:00
Kazu Hirata 2082b10d10 [llvm] Use *::empty (NFC) 2021-01-16 09:40:55 -08:00
Florian Hahn bca16e2fbb
[LTO] Remove options to disable inlining, vectorization & GVNLoadPRE.
This patch removes some ancient options as a clean-up before moving
code-gen to use LTOBackend in D94487.

I think it would preferable to remove those ancient options, because

  1. There are no corresponding options in LTOBackend based tools,
  2. There are no unit tests for them,
  3. They are not passed through by Clang,
  4. At least for GNVLoadPRE, users could just use GVN's `enable-load-pre`.

Alternatively we could add support for those options to lto::Config &
co, but I think it would be better to remove them, unless they are
actually used in practice.

Reviewed By: steven_wu, tejohnson

Differential Revision: https://reviews.llvm.org/D94783
2021-01-16 16:29:15 +00:00
Georgii Rymar d9afe8588e [yaml2obj/obj2yaml] - Refine handling of SHT_GNU_verdef sections.
This patch:
1) Makes `Version`, `Flags`, `VersionNdx` and `Hash` fields to be `Optional<>`.
2) Disallows dumping version definitions that have `vd_version != 1`.
   `vd_version` identifies the version of the structure itself.
   (https://refspecs.linuxfoundation.org/LSB_5.0.0/LSB-Core-generic/LSB-Core-generic/symversion.html,
    https://docs.oracle.com/cd/E19683-01/816-7777/chapter6-80869/index.html)
3) Stops dumping default values for `Version`, `Flags`, `VersionNdx` and `Hash` fields.
4) Refines testing.

Differential revision: https://reviews.llvm.org/D94659
2021-01-15 12:40:42 +03:00
Georgii Rymar 021ea78a97 [llvm-nm] - Simplify the code in dumpSymbolNamesFromObject. NFC.
It is possible to simplify the logic that extracts symbol names.

D94667 made the `NMSymbol::Name` to be `std::string`,
what allowed this simplification.

Differential revision: https://reviews.llvm.org/D94669
2021-01-15 12:29:49 +03:00
Georgii Rymar bfb8f45ef3 [llvm-nm] - Move MachO specific logic out from the dumpSymbolNamesFromObject(). NFC.
`dumpSymbolNamesFromObject` is the method that dumps symbol names.

It has 563 lines, mostly because of huge piece of MachO specific code.
In this patch I move it to separate helper method.

The new size of `dumpSymbolNamesFromObject` is 93 lines. With it it becomes
much easier to maintain it.

I had to change the type of 2 name fields to `std::string`, because MachO logic
uses temporarily buffer strings (e.g `ExportsNameBuffer`, `BindsNameBuffer` etc):

```
  std::string ExportsNameBuffer;
  raw_string_ostream EOS(ExportsNameBuffer);
```

these buffers were moved to `dumpSymbolsFromDLInfoMachO` by this patch and
invalidated after return. Technically, before this patch we had a situation
when local pointers (symbol names) were assigned to members of global static `SymbolList`,
what is dirty by itself.

Differential revision: https://reviews.llvm.org/D94667
2021-01-15 12:18:37 +03:00
Georgii Rymar 1185d3f43d [llvm-readobj] - Fix the compilation with GCC < 7.0.
This addressed post commit comments for D93900.

GCC had an issue and requires placing a specialization of
`printUnwindInfo` to a namespace to compile:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=56480
2021-01-15 11:58:04 +03:00
Kazu Hirata 7dc3575ef2 [llvm] Remove redundant return and continue statements (NFC)
Identified with readability-redundant-control-flow.
2021-01-14 20:30:34 -08:00
Kazu Hirata 2efcbe24a7 [llvm] Use llvm::drop_begin (NFC) 2021-01-14 20:30:33 -08:00
Andy Wingo 53e3b81faa [lld][WebAssembly] Add support for handling table symbols
This commit adds table symbol support in a partial way, while still
including some special cases for the __indirect_function_table symbol.
No change in tests.

Differential Revision: https://reviews.llvm.org/D94075
2021-01-14 11:13:13 +01:00
Kazu Hirata 4c1617dac8 [llvm] Use std::any_of (NFC) 2021-01-13 19:14:44 -08:00
wlei 35debdfcac [NFC] Fix build break by a initializer list converting error 2021-01-13 14:28:02 -08:00
wlei 33a8466531 [NFC] fix missing SectionName declaration 2021-01-13 11:30:09 -08:00
wlei c681400b25 [CSSPGO][llvm-profgen] Virtual unwinding with pseudo probe
This change extends virtual unwinder to support pseudo probe in llvm-profgen. Please refer https://groups.google.com/g/llvm-dev/c/1p1rdYbL93s and https://reviews.llvm.org/D89707 for more context about CSSPGO and llvm-profgen.

**Implementation**

- Added `ProbeBasedCtxKey` derived from `ContextKey` for sample counter aggregation. As we need string splitting to infer the profile for callee function, string based context introduces more string handling overhead, here we just use probe pointer based context.
- For linear unwinding, as inline context is encoded in each pseudo probe, we don't need to go through each instruction to extract range sharing same inliner. So just record the range for the context.
- For probe based context, we should ignore the top frame probe since it will be extracted from the address range. we defer the extraction in `ProfileGeneration`.
- Added `PseudoProbeProfileGenerator` for pseudo probe based profile generation.
- Some helper function to get pseduo probe info(call probe, inline context) from profiled binary.
- Added regression test for unwinder's output

The pseudo probe based profile generation will be in the upcoming patch.

Test Plan:

ninja & ninja check-llvm

Differential Revision: https://reviews.llvm.org/D92896
2021-01-13 11:02:58 -08:00
wlei 414930b91b [CSSPGO][llvm-profgen] Refactor to unify hashable interface for trace sample and context-sensitive counter
As we plan to support both CSSPGO and AutoFDO for llvm-profgen, we will have different kinds of perf sample and different kinds of sample counter(cs/non-cs, with/without pseudo probe) which both need to do aggregation in hash map.  This change implements the hashable interface(`Hashable`) and the unified base class for them to have better extensibility and reusability.

Currently perf trace sample and sample counter with context implemented this `Hashable` and  the class hierarchy is like:

```
| Hashable
           | PerfSample
                          | HybridSample
                          | LBRSample
           | ContextKey
                          | StringBasedCtxKey
                          | ProbeBasedCtxKey
                          | CallsiteBasedCtxKey
           | ...
```

- Class specifying `Hashable` should implement `getHashCode` and `isEqual`. Here we make `getHashCode` a non-virtual function to avoid vtable overhead, so derived class should calculate and assign the base class's HashCode manually. This also provides the flexibility for calculating the hash code incrementally(like rolling hash) during frame stack unwinding
- `isEqual` is a virtual function, which will have perf overhead. In the future, if we redesign a better hash function, then we can just skip this or switch to non-virtual function.
- Added `PerfSample` and `ContextKey` as base class for perf sample and counter context key, leveraging llvm-style RTTI for this.
- Added `StringBasedCtxKey` class extending  `ContextKey` to use string as context id.
- Refactor `AggregationCounter` to take all kinds of `PerfSample` as key
- Refactor `ContextSampleCounter` to take all kinds of `ContextKey` as key
- Other refactoring work:
 - Create a wrapper class `SampleCounter` to wrap `RangeCounter` and `BranchCounter`
 - Hoist `ContextId` and `FunctionProfile` out of `populateFunctionBodySamples` and `populateFunctionBoundarySamples` to reuse them in ProfileGenerator

Differential Revision: https://reviews.llvm.org/D92584
2021-01-13 11:02:57 -08:00
wlei b3154d11bc [CSSPGO][llvm-profgen] Pseudo probe decoding and disassembling
This change implements pseudo probe decoding and disassembling for llvm-profgen/CSSPGO. Please see https://groups.google.com/g/llvm-dev/c/1p1rdYbL93s and https://reviews.llvm.org/D89707 for more context about CSSPGO and llvm-profgen.

**ELF section format**
Please see the encoding patch(https://reviews.llvm.org/D91878) for more details of the format, just copy the example here:

Two section(`.pseudo_probe_desc` and  `.pseudoprobe` ) is emitted in ELF to support pseudo probe.
The format of `.pseudo_probe_desc` section looks like:

```
.section   .pseudo_probe_desc,"",@progbits
.quad   6309742469962978389  // Func GUID
.quad   4294967295           // Func Hash
.byte   9                    // Length of func name
.ascii  "_Z5funcAi"          // Func name
.quad   7102633082150537521
.quad   138828622701
.byte   12
.ascii  "_Z8funcLeafi"
.quad   446061515086924981
.quad   4294967295
.byte   9
.ascii  "_Z5funcBi"
.quad   -2016976694713209516
.quad   72617220756
.byte   7
.ascii  "_Z3fibi"
```

For each `.pseudoprobe` section, the encoded binary data consists of a single function record corresponding to an outlined function (i.e, a function with a code entry in the `.text` section). A function record has the following format :

```
FUNCTION BODY (one for each outlined function present in the text section)
    GUID (uint64)
        GUID of the function
    NPROBES (ULEB128)
        Number of probes originating from this function.
    NUM_INLINED_FUNCTIONS (ULEB128)
        Number of callees inlined into this function, aka number of
        first-level inlinees
    PROBE RECORDS
        A list of NPROBES entries. Each entry contains:
          INDEX (ULEB128)
          TYPE (uint4)
            0 - block probe, 1 - indirect call, 2 - direct call
          ATTRIBUTE (uint3)
            reserved
          ADDRESS_TYPE (uint1)
            0 - code address, 1 - address delta
          CODE_ADDRESS (uint64 or ULEB128)
            code address or address delta, depending on ADDRESS_TYPE
    INLINED FUNCTION RECORDS
        A list of NUM_INLINED_FUNCTIONS entries describing each of the inlined
        callees.  Each record contains:
          INLINE SITE
            GUID of the inlinee (uint64)
            ID of the callsite probe (ULEB128)
          FUNCTION BODY
            A FUNCTION BODY entry describing the inlined function.
```

**Disassembling**
A switch `--show-pseudo-probe` is added to use along with `--show-disassembly` to print disassembly code with pseudo probe directives.

For example:
```
00000000002011a0 <foo2>:
  2011a0: 50                    push   rax
  2011a1: 85 ff                 test   edi,edi
  [Probe]:  FUNC: foo2  Index: 1  Type: Block
  2011a3: 74 02                 je     2011a7 <foo2+0x7>
  [Probe]:  FUNC: foo2  Index: 3  Type: Block
  [Probe]:  FUNC: foo2  Index: 4  Type: Block
  [Probe]:  FUNC: foo   Index: 1  Type: Block  Inlined: @ foo2:6
  2011a5: 58                    pop    rax
  2011a6: c3                    ret
  [Probe]:  FUNC: foo2  Index: 2  Type: Block
  2011a7: bf 01 00 00 00        mov    edi,0x1
  [Probe]:  FUNC: foo2  Index: 5  Type: IndirectCall
  2011ac: ff d6                 call   rsi
  [Probe]:  FUNC: foo2  Index: 4  Type: Block
  2011ae: 58                    pop    rax
  2011af: c3                    ret
```

**Implementation**
- `PseudoProbeDecoder` is added in ProfiledBinary as an infra for the decoding. It decoded the two section and generate two map: `GUIDProbeFunctionMap` stores all the `PseudoProbeFunction` which is the abstraction of a general function. `AddressProbesMap` stores all the pseudo probe info indexed by its address.
- All the inline info is encoded into binary as a trie(`PseudoProbeInlineTree`) and will be constructed from the decoding. Each pseudo probe can get its inline context(`getInlineContext`) by traversing its inline tree node backwards.

Test Plan:
ninja & ninja check-llvm

Differential Revision: https://reviews.llvm.org/D92334
2021-01-13 11:02:57 -08:00
Jonas Devlieghere 48d2068fb7 [dsymutil] Warn on timestmap mismatch between object file and debug map
This re-lands e5553b9a6a with two small fixes to the tests:

 - Don't touch the source directory in debug-map-parsing.test but
   instead copy everything over in a temporary directory in
   timestamp-mismatch.test.
 - Don't redirect stderr to stdout to avoid the output getting
   intertwined in extern-alias.test.
2021-01-13 09:15:30 -08:00
David Zarzycki c6e341c899 Revert "[dsymutil] Warn on timestmap mismatch between object file and debug map"
This reverts commit e5553b9a6a.

Tests are not allowed to modify the source. Please figure out a way to
use %t rather than dynamically modifying the inputs.
2021-01-13 07:23:34 -05:00
Georgii Rymar 6d3098e7ff [obj2yaml,yaml2obj] - Refine how we set/dump the sh_entsize field.
This reuses the code from yaml2obj (moves it to ELFYAML.h).
With it we can set the `sh_entsize` in a single place in `obj2yaml`.

Note that it also fixes a bug of `yaml2obj`: we do not
set the `sh_entsize` field for the `SHT_ARM_EXIDX` section properly.

Differential revision: https://reviews.llvm.org/D93858
2021-01-13 11:52:40 +03:00
Georgii Rymar 141906fa14 [llvm-readelf/obj] - Add support of multiple SHT_SYMTAB_SHNDX sections.
Currently we don't support multiple SHT_SYMTAB_SHNDX sections
and the DT_SYMTAB_SHNDX tag currently.

This patch implements it and fixes the
https://bugs.llvm.org/show_bug.cgi?id=43991.

I had to introduce the `struct DataRegion` to ELF.h,
it is used to represent a region that might have no known size.
It is needed, because we don't know the size of the extended
section indices table when it is located via DT_SYMTAB_SHNDX.
In this case we still want to validate that we don't read
past the end of the file.

Differential revision: https://reviews.llvm.org/D92923
2021-01-13 11:36:43 +03:00
Jonas Devlieghere f1d5cbbdee [dsymutil] Add preliminary support for DWARF 5.
Currently dsymutil will silently fail when processing binaries with
Dwarf 5 debug info. This patch adds rudimentary support for Dwarf 5 in
dsymutil.

 - Recognize relocations in the debug_addr section.
 - Recognize (a subset of) Dwarf 5 form values.
 - Emits valid Dwarf 5 compile unit header chains.

To simplify things (and avoid having to emit indexed sections) I decided
to emit the relocated addresses directly in the debug info section.

 - DW_FORM_strx gets relocated and rewritten to DW_FORM_strp
 - DW_FORM_addrx gets relocated and rewritten to DW_FORM_addr

Obviously there's a lot of work left, but this should be a step in the
right direction.

rdar://62345491

Differential revision: https://reviews.llvm.org/D94323
2021-01-12 21:55:41 -08:00
Kazu Hirata 12fc9ca3a4 [llvm] Remove redundant string initialization (NFC)
Identified with readability-redundant-string-init.
2021-01-12 21:43:46 -08:00
Jonas Devlieghere 8a47d875b0 [dsymutil] Copy eh_frame content into the dSYM companion file.
Copy over the __eh_frame from the binary into the dSYM. This helps
kernel developers that are working with only dSYMs (i.e. no binaries)
when debugging a core file. This only kicks in when the __eh_frame
exists in the linked binary. Most of the time ld64 will remove the
section in favor of compact unwind info. When it is emitted, it's
generally small enough and should not bloat the dSYM.

rdar://69774935

Differential revision: https://reviews.llvm.org/D94460
2021-01-12 19:50:34 -08:00
Jonas Devlieghere e5553b9a6a [dsymutil] Warn on timestmap mismatch between object file and debug map
Add a warning when the timestmap doesn't match between the object file
and the debug map entry. We were already emitting such warnings for
archive members and swift interface files. This patch also unifies the
warning across all three.

rdar://65614640

Differential revision: https://reviews.llvm.org/D94536
2021-01-12 18:58:10 -08:00
Georgii Rymar c15a57cc1a [obj2yaml] - Don't crash when an object has an empty symbol table.
Currently we crash when we have an object with SHT_SYMTAB/SHT_DYNSYM sections
of size 0.

With this patch instead of the crash we start to dump them properly.

Differential revision: https://reviews.llvm.org/D93697
2021-01-12 14:08:59 +03:00
Georgii Rymar 60df7c08b1 [obj2yaml,yaml2obj] - Fix issues with creating/dumping group sections.
We have the following issues related to group sections:
1) yaml2obj is unable to set the custom `sh_entsize` value, because the `EntSize`
   key is currently ignored.
2) obj2yaml is unable to dump the group section which `sh_entsize != 4`.
3) obj2yaml always dumps the "EntSize" for group sections, though
   usually we are trying to omit dumping default values when dumping keys.
   I.e. we should not print the "EntSize" key when `sh_entsize` == 4.

This patch fixes (1),(3) and adds the test case to document the behavior of (2).

Differential revision: https://reviews.llvm.org/D93854
2021-01-12 14:07:42 +03:00
Georgii Rymar 891b4873c1 [llvm-readobj] - One more attempt to fix BB.
Add `this->` for `W`, which is the member of `ObjDumper`

An example of error:
readobj/ELFDumper.cpp:738:13: error: use of undeclared identifier 'W'
    assert(&W.getOStream() == &llvm::fouts());
2021-01-12 13:17:59 +03:00
Georgii Rymar cc91efdabe [llvm-readobj] - An attempt to fix BB.
This adds the `template` keyword for 'getAsArrayRef' calls.

An example of error:
/b/1/openmp-gcc-x86_64-linux-debian/llvm.src/llvm/tools/llvm-readobj/ELFDumper.cpp:4491:50: error: use 'template' keyword to treat 'getAsArrayRef' as a dependent template name
    for (const Elf_Rel &Rel : this->DynRelRegion.getAsArrayRef<Elf_Rel>())
2021-01-12 13:09:49 +03:00
Georgii Rymar 1e11402aa8 [llvm-readobj] - Add 'override' to fix build bots.
This should fix bots after landing D93900.

An example of error is:

/home/worker/2.0.1/lldb-x86_64-debian/llvm-project/llvm/tools/llvm-readobj/ELFDumper.cpp:883:8: warning: 'printSectionMapping' overrides a member function but is not marked 'override' [-Winconsistent-missing-override]
  void printSectionMapping() {}
2021-01-12 13:01:15 +03:00
Georgii Rymar 9ec72cfc61 [llvm-readef/obj] - Change the design structure of ELF dumper. NFCI.
This is a refactoring for design of stuff in `ELFDumper.cpp`.
The current design of ELF dumper is far from ideal.

Currently most overridden functions (inherited from `ObjDumper`) in `ELFDumper` just forward to
the functions of `ELFDumperStyle` (which can be either `GNUStyle` or `LLVMStyle`).
A concrete implementation may be in any of `ELFDumper`/`DumperStyle`/`GNUStyle`/`LLVMStyle`.

This patch reorganizes the classes by introducing `GNUStyleELFDumper`/`LLVMStyleELFDumper`
which inherit from `ELFDumper`. The implementations are moved:

`DumperStyle` -> `ELFDumper`
`GNUStyle` -> `GNUStyleELFDumper`
`LLVMStyle` -> `LLVMStyleELFDumper`

With that we can avoid having a lot of redirection calls and helper methods.
The number of code lines changes from 7142 to 6922 (reduced by ~3%) and the
code overall looks cleaner.

Differential revision: https://reviews.llvm.org/D93900
2021-01-12 12:36:17 +03:00
Kazu Hirata e5b4dbab04 [llvm] Simplify string comparisons (NFC)
Identified with readability-string-compare.
2021-01-11 18:48:09 -08:00
Kazu Hirata 8590a3e3ad [llvm] Use *Set::contains (NFC) 2021-01-11 18:48:07 -08:00
Kazu Hirata 89e8eb946d [llvm] Use llvm::find_if (NFC) 2021-01-11 18:48:06 -08:00
Abhina Sreeskantharajan 8ad998a611 [tools] Mark output of tools as text if it is really text
This is a continuation of https://reviews.llvm.org/D67696. The following tools also need to set the OF_Text flag correctly.

  -   llvm-profdata
  -   llvm-link

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D94313
2021-01-11 15:14:03 -05:00
Georgii Rymar a6db7cf1ce [llvm-readelf/obj] - Index phdrs and relocations from 0 when reporting warnings.
As was mentioned in comments here:
https://reviews.llvm.org/D92636#inline-864967

we are not consistent and sometimes index things from 0, but sometimes
from 1 in warnings.

This patch fixes 2 places: messages reported for
program headers and messages reported for relocations.

Differential revision: https://reviews.llvm.org/D93805
2021-01-11 15:13:54 +03:00
Georgii Rymar c74751d4b5 [obj2yaml] - Fix the crash in getUniquedSectionName().
`getUniquedSectionName(const Elf_Shdr *Sec)` assumes that
`Sec` is not `nullptr`.

I've found one place in `getUniquedSymbolName` where it is
not true (because of that we crash when trying to dump
unnamed null section symbols).

Patch fixes the crash and changes the signature of the
`getUniquedSectionName` section to accept a reference.

Differential revision: https://reviews.llvm.org/D93754
2021-01-11 15:04:00 +03:00
Kazu Hirata 6a6e382161 [llvm] Drop unnecessary make_range (NFC) 2021-01-09 09:25:00 -08:00
Martin Storsjö 7a91dad9e5 [llvm-readobj] [ARMWinEH] Clearly print an invalid case of packed unwind info as such
As the actual windows unwinder doesn't support this case, don't
pretend that it is supported when dumping the generated unwind info
either, even if it would be possible to interpret it as something
sensible.

This should reduce the risk of us emitting such a case in code
(although it's unlikely as long as the unwind info is generated
through the SEH opcodes, as the opcodes can't describe this case).

Differential Revision: https://reviews.llvm.org/D91529
2021-01-08 10:04:44 +02:00
Arthur Eubanks 9ccf13c36d [NewPM][NVPTX] Port NVPTX opt passes
There are only two used in the IR optimization pipeline.
Port these and add them to the default pipeline.

Similar to https://reviews.llvm.org/D93863.

I added -mtriple to some tests since under the new PM, the passes are
only available when the TargetMachine is specified.

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D93930
2021-01-07 15:12:35 -08:00
Alexandre Ganea ce7f30b2a8 [llvm-pdbutil] Don't crash when printing unknown CodeView type records
Differential Revision: https://reviews.llvm.org/D93720
2021-01-07 15:44:55 -05:00
Roman Lebedev 8dee0b4bd6
[llvm-reduce] ReduceGlobalVarInitializers delta pass: fix handling of globals w/ comdat/non-external linkage
Much like with ReduceFunctionBodies delta pass,
we need to remove comdat and set linkage to external,
else verifier will complain, and our deltas are invalid.
2021-01-07 18:05:03 +03:00
Simon Pilgrim a9a8caf2ce [llvm-objdump] Pass Twine by const reference instead of by value. NFCI. 2021-01-07 12:53:29 +00:00
Kazu Hirata cd088ba7e6 [llvm] Use llvm::lower_bound and llvm::upper_bound (NFC) 2021-01-05 21:15:59 -08:00
Kazu Hirata 441650d589 [tools] Use llvm::append_range (NFC) 2021-01-05 21:15:56 -08:00
Christudasan Devadasan d68458bd56 [GlobalISel] Base implementation for sret demotion.
If the return values can't be lowered to registers
SelectionDAG performs the sret demotion. This patch
contains the basic implementation for the same in
the GlobalISel pipeline.

Furthermore, targets should bring relevant changes
during lowerFormalArguments, lowerReturn and
lowerCall to make use of this feature.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D92953
2021-01-06 10:30:50 +05:30
Sergey Dmitriev 761aca1e2e [llvm-link] fix linker behavior when linking archives with --only-needed option
This patch fixes linker behavior when archive is linked with other inputs
as a library (i.e. when --only-needed option is specified). In this case library
is expected to be normally linked first into a separate module and only after
that linker should import required symbols from the linked library module.

Reviewed By: tra

Differential Revision: https://reviews.llvm.org/D92535
2021-01-05 10:02:51 -08:00
Alan Phipps 9f2967bcfe [Coverage] Add support for Branch Coverage in LLVM Source-Based Code Coverage
This is an enhancement to LLVM Source-Based Code Coverage in clang to track how
many times individual branch-generating conditions are taken (evaluate to TRUE)
and not taken (evaluate to FALSE).  Individual conditions may comprise larger
boolean expressions using boolean logical operators.  This functionality is
very similar to what is supported by GCOV except that it is very closely
anchored to the ASTs.

Differential Revision: https://reviews.llvm.org/D84467
2021-01-05 09:51:51 -06:00
Arthur Eubanks 4e838ba9ea [NewPM][AMDGPU] Port amdgpu-always-inline
And add to AMDGPU opt pipeline.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D94025
2021-01-04 12:27:01 -08:00
Arthur Eubanks fd323a897c [NewPM][AMDGPU] Port amdgpu-printf-runtime-binding
And add to AMDGPU opt pipeline.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D94026
2021-01-04 12:25:50 -08:00
Arthur Eubanks e1833e7493 [NewPM][AMDGPU] Port amdgpu-unify-metadata
And add to AMDGPU opt pipeline.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D94023
2021-01-04 11:57:46 -08:00
Arthur Eubanks a5f863e076 [NewPM][AMDGPU] Port amdgpu-propagate-attributes-early/late
And add to AMDGPU opt pipeline.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D94022
2021-01-04 11:53:37 -08:00
Kazu Hirata eb198f4c3c [llvm] Use llvm::any_of (NFC) 2021-01-04 11:42:47 -08:00
Roman Lebedev 5799fc79c3
[llvm-reduce] Refactor global variable delta pass
The limitation of the current pass that it skips initializer-less GV's
seems arbitrary, in all the reduced cases i (personally) looked at,
the globals weren't needed, yet they were kept.

So let's do two things:
1. allow reducing initializer-less globals
2. before reducing globals, reduce their initializers, much like we do function bodies
2021-01-03 01:45:47 +03:00
Roman Lebedev 19ab1817b6
[llvm-reduce] Fix removal of unused llvm intrinsics declarations
ee6e25e439 changed
the delta pass to skip intrinsics, which means we may end up being
left with declarations of intrinsics, that aren't otherwise referenced
in the module. This is obviously unwanted, do drop them.
2021-01-03 01:45:47 +03:00
Hongtao Yu 01f0d162d6 Moving UniqueInternalLinkageNamesPass to the start of IR pipelines.
`UniqueInternalLinkageNamesPass` is useful to CSSPGO, especially when pseudo probe is used. It solves naming conflict for static functions which otherwise will share a merged profile and likely have a profile quality issue with mismatched CFG checksums. Since the pseudo probe instrumentation happens very early in the pipeline, I'm moving `UniqueInternalLinkageNamesPass` right before it. This is being done only to the new pass manager.

Reviewed By: dblaikie, aeubanks

Differential Revision: https://reviews.llvm.org/D93656
2021-01-02 14:26:21 -08:00
Kazu Hirata 171c5fd43e [llvm] Use llvm::erase_value and llvm::erase_if (NFC) 2021-01-02 09:24:15 -08:00
Roman Lebedev e6b1a27fb9
[NFC][CodeGen] Split DwarfEHPrepare pass into an actual transform and an legacy-PM wrapper
This is consistent with the layout of other passes,
and simplifies further refinements regarding DomTree handling.

This is indended to be a NFC commit.
2021-01-02 01:01:19 +03:00
Kazu Hirata 9a90c4ea8a [llvm] Use isa instead of dyn_cast (NFC) 2021-01-01 12:44:56 -08:00
Kazu Hirata bea8d021a3 [llvm] Use *Map::lookup (NFC) 2021-01-01 12:44:54 -08:00
Kazu Hirata f904b46b1a [llvm-objcopy] Use llvm::erase_if (NFC) 2020-12-31 09:39:09 -08:00
Bogdan Graur 2016f2c8a7 Fixes warning 'enumeration value not handled in switch'.
This was introduced in commit: 981a0bd858.

Differential Revision: https://reviews.llvm.org/D93944
2020-12-30 06:56:29 -08:00
Haowei Wu a1d0589266 [llvm-elfabi] Add flag to preserve timestamp when output is the same
This change adds '--write-if-changed' flag to llvm-elfabi tool. When
enabled, llvm-elfabi will not overwrite the existing file if the
content of the file will not be changed, which preserves the
timestamp.

Differential Revision: https://reviews.llvm.org/D92902
2020-12-29 20:27:06 -08:00
Lang Hames 5efc71e119 [ORC] Move Orc RPC code into Shared, rename some RPC types.
Moves all headers from Orc/RPC to Orc/Shared, and from the llvm::orc::rpc
namespace into llvm::orc::shared. Also renames RPCTypeName to
SerializationTypeName and Function to RPCFunction.

In addition to being a more reasonable home for this code, this will make it
easier for the upcoming Orc runtime to re-use the Serialization system for
creating and parsing wrapper-function binary blobs.
2020-12-30 12:48:20 +11:00
Haowei Wu d034a94e7b Revert "[llvm-elfabi] Add flag to preserve timestamp when output is the same"
This reverts commit fddb417449. which
causes test failures on Mac builders.
2020-12-29 17:26:22 -08:00
Haowei Wu fddb417449 [llvm-elfabi] Add flag to preserve timestamp when output is the same
This change adds '--write-if-changed' flag to llvm-elfabi tool. When
enabled, llvm-elfabi will not overwrite the existing file if the
content of the file will not be changed, which preserves the
timestamp.

Differential Revision: https://reviews.llvm.org/D92902
2020-12-29 14:43:47 -08:00
Arthur Eubanks 7ecbe0c7a0 [NewPM][AMDGPU] Port amdgpu-lower-kernel-attributes
And add it to the AMDGPU opt pipeline.

This is a function pass instead of a module pass (like the legacy pass)
because it's getting added to a CGSCCPassManager, and you can't put a
module pass in a CGSCCPassManager.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D93885
2020-12-29 10:26:06 -08:00
Arthur Eubanks c2ef06d3dd [NewPM] Port infer-address-spaces
And add it to the AMDGPU opt pipeline.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D93880
2020-12-28 19:58:12 -08:00
Arthur Eubanks 0e9abcfc19 [AMDGPU][NewPM] Port amdgpu-promote-alloca(-to-vector)
And add to AMDGPU opt pipeline.

Don't pin an opt run to the legacy PM when -enable-new-pm=1 if these
passes (or passes introduced in https://reviews.llvm.org/D93863) are in
the list of passes.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D93875
2020-12-28 17:52:31 -08:00
Kazu Hirata 079923309c [llvm-cov] Use is_contained (NFC) 2020-12-27 09:57:25 -08:00
Kazu Hirata b676f2fee1 [llvm-cov, llvm-symbolizer] Use llvm::erase_if (NFC) 2020-12-26 12:06:27 -08:00
Kazu Hirata 9c9bca45f0 [llvm-pdbutil] Use llvm::is_contained (NFC) 2020-12-26 12:06:24 -08:00
Kazu Hirata e334c52add [llvm-objcopy] Use llvm::erase_if (NFC) 2020-12-25 10:13:18 -08:00
Kazu Hirata ea39991251 [llvm-nm, llvm-objdump] Use llvm::is_contained (NFC) 2020-12-25 09:22:37 -08:00
Georgii Rymar 893c84d71c [obj2yaml] - Dump the content of a broken hash table properly.
This is similar to D93760.

When something is wrong with the hash table header we dump
its context as a raw data.

Currently we have the calculation overflow issue and it is possible to
bypass the validation we have (and crash).

The patch fixes it.

Differential revision: https://reviews.llvm.org/D93799
2020-12-25 11:51:28 +03:00
Georgii Rymar 177779e8dd [llvm-readelf/obj] - Improve the warning reported when unable to read the stack size.
It was discussed in D92545 that we might want to improve messages
reported when something is wrong with the stack size section.
This patch does it.

Differential revision: https://reviews.llvm.org/D93802
2020-12-25 11:40:35 +03:00
Georgii Rymar 438bc157a4 [libObject] - Add more ELF types to LLVM_ELF_IMPORT_TYPES_ELFT define (ELFTypes.h).
This allows to get rid of lots for typedefs/usings from many places.

Differential revision: https://reviews.llvm.org/D93801
2020-12-25 11:39:05 +03:00
Kazu Hirata d6ff5cf995 [Target] Use llvm::any_of (NFC) 2020-12-24 19:43:26 -08:00
Georgii Rymar b8cb1802a8 [obj2yaml] - Dump the content of a broken GNU hash table properly.
When something is wrong with the GNU hash table header we dump
its context as a raw data.

Currently we have the calculation overflow issue and it is possible to
bypass the validation we have (and crash).

The patch fixes it.

Differential revision: https://reviews.llvm.org/D93760
2020-12-24 11:16:31 +03:00
Georgii Rymar bdef1f87ab [llvm-readobj] - Dump the ELF file type better.
Currently llvm-readelf might print "OS Specific/Processor Specific/<unknown>"
hint when dumping the ELF file type. The patch teaches llvm-readobj to do the same.

This fixes https://bugs.llvm.org/show_bug.cgi?id=40868

I am removing `Object/elf-unknown-type.test` test because it is not in the right place,
it is outdated and very limited.
The `readobj/ELF/file-types.test` checks the functionality much better.

Differential revision: https://reviews.llvm.org/D93689
2020-12-23 11:13:19 +03:00
Arthur O'Dwyer 22cf54a7fb Replace `T(x)` with `reinterpret_cast<T>(x)` everywhere it means reinterpret_cast. NFC.
Differential Revision: https://reviews.llvm.org/D76572
2020-12-22 19:54:29 -05:00
Tom Stellard 4ad0cfd4de llvm-profgen: Parse command line arguments after initializing targets
I am experimenting with turning backends into loadable modules and in
that scenario, target specific command line arguments won't be available
until after the targets are initialized.

Also, most other tools initialize targets before parsing arguments.

Reviewed By: wlei

Differential Revision: https://reviews.llvm.org/D93348
2020-12-21 15:13:10 -08:00
Georgii Rymar 8590b5ccd5 [libObject, llvm-readobj] - Reimplement `ELFFile<ELFT>::getEntry`.
Currently, `ELFFile<ELFT>::getEntry` does not check an index of
an entry. Because of that the code might read past the end of the symbol
table silently. I've added a test to `llvm-readobj\ELF\relocations.test`
to demonstrate the possible issue. Also, I've added a unit test for
this method.

After this change, `getEntry` stops reporting the section index and
reuses the `getSectionContentsAsArray` method, which already has
all the validation needed. Our related warnings now provide
more and better context sometimes.

Differential revision: https://reviews.llvm.org/D93209
2020-12-18 16:52:27 +03:00
Adhemerval Zanella e04dc5f557 [llvm-readobj/elf] - AArch64: Handle AARCH64_VARIANT_PCS for GNUStyle
It mimics the GNU readelf where it prints a [VARIANT_PCS] for symbols
with st_other with STO_AARCH64_VARIANT_PCS.

Reviewed By: grimar, MaskRay

Differential Revision: https://reviews.llvm.org/D93044
2020-12-17 11:09:53 -03:00
dfukalov 9ed8e0caab [NFC] Reduce include files dependency and AA header cleanup (part 2).
Continuing work started in https://reviews.llvm.org/D92489:

Removed a bunch of includes from "AliasAnalysis.h" and "LoopPassManager.h".

Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D92852
2020-12-17 14:04:48 +03:00
Barry Revzin 92310454bf Make LLVM build in C++20 mode
Part of the <=> changes in C++20 make certain patterns of writing equality
operators ambiguous with themselves (sorry!).
This patch goes through and adjusts all the comparison operators such that
they should work in both C++17 and C++20 modes. It also makes two other small
C++20-specific changes (adding a constructor to a type that cases to be an
aggregate, and adding casts from u8 literals which no longer have type
const char*).

There were four categories of errors that this review fixes.
Here are canonical examples of them, ordered from most to least common:

// 1) Missing const
namespace missing_const {
    struct A {
    #ifndef FIXED
        bool operator==(A const&);
    #else
        bool operator==(A const&) const;
    #endif
    };

    bool a = A{} == A{}; // error
}

// 2) Type mismatch on CRTP
namespace crtp_mismatch {
    template <typename Derived>
    struct Base {
    #ifndef FIXED
        bool operator==(Derived const&) const;
    #else
        // in one case changed to taking Base const&
        friend bool operator==(Derived const&, Derived const&);
    #endif
    };

    struct D : Base<D> { };

    bool b = D{} == D{}; // error
}

// 3) iterator/const_iterator with only mixed comparison
namespace iter_const_iter {
    template <bool Const>
    struct iterator {
        using const_iterator = iterator<true>;

        iterator();

        template <bool B, std::enable_if_t<(Const && !B), int> = 0>
        iterator(iterator<B> const&);

    #ifndef FIXED
        bool operator==(const_iterator const&) const;
    #else
        friend bool operator==(iterator const&, iterator const&);
    #endif
    };

    bool c = iterator<false>{} == iterator<false>{} // error
          || iterator<false>{} == iterator<true>{}
          || iterator<true>{} == iterator<false>{}
          || iterator<true>{} == iterator<true>{};
}

// 4) Same-type comparison but only have mixed-type operator
namespace ambiguous_choice {
    enum Color { Red };

    struct C {
        C();
        C(Color);
        operator Color() const;
        bool operator==(Color) const;
        friend bool operator==(C, C);
    };

    bool c = C{} == C{}; // error
    bool d = C{} == Red;
}

Differential revision: https://reviews.llvm.org/D78938
2020-12-17 10:44:10 +00:00
Fangrui Song c70f36865e Use basic_string::find(char) instead of basic_string::find(const char *s, size_type pos=0)
Many (StringRef) cannot be detected by clang-tidy performance-faster-string-find.
2020-12-16 23:28:32 -08:00
Hongtao Yu ac068e014b [CSSPGO] Consume pseudo-probe-based AutoFDO profile
This change enables pseudo-probe-based sample counts to be consumed by the sample profile loader under the regular `-fprofile-sample-use` switch with minimal adjustments to the existing sample file formats. After the counts are imported, a probe helper, aka, a `PseudoProbeManager` object, is automatically launched to verify the CFG checksum of every function in the current compilation against the corresponding checksum from the profile. Mismatched checksums will cause a function profile to be slipped. A `SampleProfileProber` pass is scheduled before any of the `SampleProfileLoader` instances so that the CFG checksums as well as probe mappings are available during the profile loading time. The `PseudoProbeManager` object is set up right after the profile reading is done. In the future a CFG-based fuzzy matching could be done in `PseudoProbeManager`.

Samples will be applied only to pseudo probe instructions as well as probed callsites once the checksum verification goes through. Those instructions are processed in the same way that regular instructions would be processed in the line-number-based scenario. In other words, a function is processed in a regular way as if it was reduced to just containing pseudo probes (block probes and callsites).

**Adjustment to profile format **

A CFG checksum field is being added to the existing AutoFDO profile formats. So far only the text format and the extended binary format are supported. For the text format, a new line like
```
!CFGChecksum: 12345
```
is added to the end of the body sample lines. For the extended binary profile format, we introduce a metadata section to store the checksum map from function names to their CFG checksums.

Differential Revision: https://reviews.llvm.org/D92347
2020-12-16 15:57:18 -08:00
Georgii Rymar 8c2cf89834 [yaml2obj/obj2yaml] - Make Value/Size fields of Symbol optional.
When a field is optional we can use the `=<none>` syntax in macros.
This patch makes `Value`/`Size` fields of `Symbol` optional
and adds test cases for them.

Differential revision: https://reviews.llvm.org/D93010
2020-12-16 13:49:57 +03:00
Georgii Rymar 407d420029 [lib/Object] - Make ELFObjectFile::getSymbol() return Expected<>.
This was requested in comments for D93209:
https://reviews.llvm.org/D93209#inline-871192

D93209 fixes an issue with `ELFFile<ELFT>::getEntry`,
after what `getSymbol` starts calling `report_fatal_error` for previously
missed invalid cases.

This patch makes it return `Expected<>` and updates callers.
For few of them I had to add new `report_fatal_error` calls. But I see no
way to avoid it currently. The change would affects too many places, e.g:
`getSymbolBinding` and other methods are used from `ELFSymbolRef`
which is used in too many places across LLVM.

Differential revision: https://reviews.llvm.org/D93297
2020-12-16 13:14:23 +03:00
Georgii Rymar 78aea98308 [llvm-readelf/obj] - Handle out-of-order PT_LOADs better.
This is https://bugs.llvm.org/show_bug.cgi?id=45698.

Specification says that
"Loadable segment entries in the program header table appear
in ascending order, sorted on the p_vaddr member."

Our `toMappedAddr()` relies on this condition. This patch
adds a warning when the sorting order of loadable segments is wrong.
In this case we force segments sorting and that allows
`toMappedAddr()` to work as expected.

Differential revision: https://reviews.llvm.org/D92641
2020-12-16 12:59:32 +03:00
Amy Huang aa7ae25613 [llvm-symbolizer] Add missing include for config.h
The cmake variable LLVM_ENABLE_DIA_SDK was being used here but
was undefined because config.h wasn't included.

Differential Revision: https://reviews.llvm.org/D93309
2020-12-15 09:20:31 -08:00
Georgii Rymar 83aea14ed6 [llvm-readelf] - Don't print OS/Processor specific prefix for known ELF file types.
This is a change suggested in post commit comments for
D93096 (https://reviews.llvm.org/D93096#2451796).

Imagine we want to add a custom OS specific ELF file type.
For that we can update the `ElfObjectFileType` array:

```
static const EnumEntry<unsigned> ElfObjectFileType[] = {
...
  {"Core",         "CORE (Core file)",         ELF::ET_CORE},
  {"MyType",       "MyType (my description)",     0xfe01},
};
```

The current code then might print:
```
OS Specific: (MyType (my description))
```

Though instead we probably would like to see a nicer output, e.g:
```
Type: MyType (my description)
```

To achieve that we can reorder the code slightly.

It is impossible to add a test I think, because we have no custom values in
the `ElfObjectFileType` array in LLVM.

Differential revision: https://reviews.llvm.org/D93217
2020-12-15 10:56:25 +03:00
David Spickett aabaca3363 [llvm-objdump] Use "--" for long options in --help text
Single dash for these options is not recognised.

Changes found by running this on the --help output
and the user guide:
grep -e ' -[a-zA-Z]\{2,\}'

The user guide was updated in https://reviews.llvm.org/D92305
so no change there.

Reviewed By: jhenderson, MaskRay

Differential Revision: https://reviews.llvm.org/D92310
2020-12-14 13:11:29 +00:00
Georgii Rymar 98a4289810 [llvm-readobj] - For SHT_REL relocations, don't display an addend.
This is https://bugs.llvm.org/show_bug.cgi?id=44257.

In LLVM style we always print `0` as addend when dumping
SHT_REL relocations. It is confusing, this patch stops
printing it as the first comment on the bug page suggests.

Differential revision: https://reviews.llvm.org/D93033
2020-12-14 12:03:00 +03:00
Georgii Rymar 4e2e785ddd [llvm-readelf] - Improve ELF type field dumping.
This is related to https://bugs.llvm.org/show_bug.cgi?id=40868.

Currently we don't print `OS Specific`/``Processor Specific`/`<unknown>`
prefixes when dumping the ELF file type. This is not consistent
with GNU readelf. The patch fixes it.

Also, this patch removes the `types.test`, because we already have
`file-types.test`, which tests more cases and this patch revealed that
we have such a duplicate.

Differential revision: https://reviews.llvm.org/D93096
2020-12-14 11:24:08 +03:00
Arthur Eubanks 655011c713 [opt][NPM] Pin -lower-amx-type to legacy PM
This is part of the codegen pipeline.
2020-12-13 19:16:20 -08:00
Lang Hames 04795ab836 Re-apply 8904ee8ac7 with missing header included this time. 2020-12-14 13:39:33 +11:00
Nico Weber 5b112bcc0d Revert "[JITLink] Add JITLinkDylib type, thread through JITLinkMemoryManager APIs."
This reverts commit 8904ee8ac7.
Didn't `git add` llvm/ExecutionEngine/JITLink/JITLinkDylib.h and hence doesn't
build anywhere.
2020-12-13 21:30:38 -05:00
Lang Hames 8904ee8ac7 [JITLink] Add JITLinkDylib type, thread through JITLinkMemoryManager APIs.
JITLinkDylib represents a target dylib for a JITLink link. By representing this
explicitly we can:
  - Enable JITLinkMemoryManagers to manage allocations on a per-dylib basis
    (e.g by maintaining a seperate allocation pool for each JITLinkDylib).
  - Enable new features and diagnostics that require information about the
    target dylib (not implemented in this patch).
2020-12-14 12:29:16 +11:00
Martin Storsjö 879c15e890 [llvm-rc] Handle driveless absolute windows paths when loading external files
When llvm-rc loads an external file, it looks for it relative to
a number of include directories and the current working directory.
If the path is considered absolute, llvm-rc tries to open the
filename as such, and doesn't try to open it relative to other
paths.

On Windows, a path name like "\dir\file" isn't considered absolute
as it lacks the drive name, but by appending it on top of the search
dirs, it's not found.

LLVM's sys::path::append just appends such a path (same with a properly
absolute posix path) after the paths it's supposed to be relative to.

This fix doesn't handle the case if the resource script and the
external file are on a different drive than the current working
directory; to fix that, we'd have to make LLVM's sys::path::append
handle appending fully absolute and partially absolute paths (ones
lacking a drive prefix but containing a root directory), or switch
to C++17's std::filesystem.

Differential Revision: https://reviews.llvm.org/D92558
2020-12-10 14:11:06 +02:00
Alexey Lapshin 693da9df74 [dsymutil][DWARFLinker][NFC] Make interface of AddressMap more general.
Current interface of AddressMap assumes that relocations exist.
That is correct for not-linked object file but is not correct
for linked executable. This patch changes interface in such way
that AddressMap could be used not only with not-linked object files:

hasValidRelocationAt()

replaced with:

hasLiveMemoryLocation()
hasLiveAddressRange()

Differential Revision: https://reviews.llvm.org/D87723
2020-12-10 14:57:08 +03:00
Sergey Dmitriev 025d4faadb [llvm-link][NFC] Minor cleanup
llvm::Linker::linkModules() is a static member, so there is no need
to pass reference to llvm::Linker instance to loadArFile() function.

Reviewed By: tra

Differential Revision: https://reviews.llvm.org/D92918
2020-12-09 23:16:13 -08:00
Fangrui Song 7adcacda06 Rename -plugin-opt=no-new-pass-manager to -plugin-opt=legacy-pass-manager 2020-12-09 16:43:30 -08:00
Fangrui Song 68ff3b3376 [LLD][gold] Add -plugin-opt=no-new-pass-manager
-DENABLE_EXPERIMENTAL_NEW_PASS_MANAGER=on configured LLD and LLVMgold.so
will use the new pass manager by default. Add an option to
use the legacy pass manager. This will also be used by the Clang driver
when -fno-new-pass-manager (D92915) / -fno-experimental-new-pass-manager is set.

Reviewed By: aeubanks, tejohnson

Differential Revision: https://reviews.llvm.org/D92916
2020-12-09 13:31:03 -08:00
Sam Clegg 9a72d3e3e4 [WebAssembly] Add support for named data sections in wasm binaries
Followup to https://reviews.llvm.org/D91769 which added support
for names globals.

Differential Revision: https://reviews.llvm.org/D92909
2020-12-09 12:57:07 -08:00
Arthur Eubanks 664b187160 Reland Pin -loop-reduce to legacy PM
This was accidentally reverted by a later change.

LSR currently only runs in the codegen pass manager.
There are a couple issues with LSR and the NPM.
1) Lots of tests assume that LCSSA isn't run before LSR. This breaks a
bunch of tests' expected output. This is fixable with some time put in.
2) LSR doesn't preserve LCSSA. See
llvm/test/Analysis/MemorySSA/update-remove-deadblocks.ll. LSR's use of
SCEVExpander is the only use of SCEVExpander where the PreserveLCSSA option is
off. Turning it on causes some code sinking out of loops to fail due to
SCEVExpander's inability to handle the newly created trivial PHI nodes in the
broken critical edge (I was looking at
llvm/test/Transforms/LoopStrengthReduce/X86/2011-11-29-postincphi.ll).
I also tried simply just calling formLCSSA() at the end of LSR, but the extra
PHI nodes cause regressions in codegen tests.

We'll delay figuring these issues out until later.

This causes the number of check-llvm failures with -enable-new-pm true
by default to go from 60 to 29.

Reviewed By: asbirlea

Differential Revision: https://reviews.llvm.org/D92796
2020-12-09 09:57:57 -08:00
Georgii Rymar bdfafc4613 [llvm-readelf/obj] - Improve diagnostics when printing NT_FILE notes.
This changes the `printNotesHelper` to report warnings on its side when
there are errors when dumping notes.

With that we can provide more content when reporting warnings about broken notes.

Differential revision: https://reviews.llvm.org/D92636
2020-12-09 12:31:46 +03:00
Georgii Rymar abae3c1196 [obj2yaml] - Support dumping objects that have multiple SHT_SYMTAB_SHNDX sections.
It is allowed to have multiple `SHT_SYMTAB_SHNDX` sections, though
we currently don't implement it.

The current implementation assumes that there is a maximum of one SHT_SYMTAB_SHNDX
section and that it is always linked with .symtab section.

This patch drops this limitations.

Differential revision: https://reviews.llvm.org/D92644
2020-12-09 12:14:58 +03:00
Arthur Eubanks f0e89e69d6 [gold][NPM] Use NPM with ENABLE_EXPERIMENTAL_NEW_PASS_MANAGER
Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D92869
2020-12-08 15:13:34 -08:00
Anna Thomas 29356e3279 [ScalarizeMaskedMemIntrin] Add new PM support
This patch adds new PM support for the pass and the pass can be now used
during middle-end transforms. The old pass is remamed to
ScalarizeMaskedMemIntrinLegacyPass.

Reviewed-By: skatkov, aeubanks
Differential Revision: https://reviews.llvm.org/D92743
2020-12-08 17:15:22 -05:00
Arthur Eubanks 3900f3f18c Pin -loop-reduce to legacy PM
LSR currently only runs in the codegen pass manager.
There are a couple issues with LSR and the NPM.
1) Lots of tests assume that LCSSA isn't run before LSR. This breaks a
bunch of tests' expected output. This is fixable with some time put in.
2) LSR doesn't preserve LCSSA. See
llvm/test/Analysis/MemorySSA/update-remove-deadblocks.ll. LSR's use of
SCEVExpander is the only use of SCEVExpander where the PreserveLCSSA option is
off. Turning it on causes some code sinking out of loops to fail due to
SCEVExpander's inability to handle the newly created trivial PHI nodes in the
broken critical edge (I was looking at
llvm/test/Transforms/LoopStrengthReduce/X86/2011-11-29-postincphi.ll).
I also tried simply just calling formLCSSA() at the end of LSR, but the extra
PHI nodes cause regressions in codegen tests.

We'll delay figuring these issues out until later.

This causes the number of check-llvm failures with -enable-new-pm true
by default to go from 60 to 29.

Reviewed By: asbirlea

Differential Revision: https://reviews.llvm.org/D92796
2020-12-08 13:48:23 -08:00
Arthur Eubanks eca13e995c [NFC] Rename IsCodeGenPass to ShouldPinPassToLegacyPM
Codegen-specific passes are being ported to the NPM. Rename for better
clarity and note that ported passes that fully work with the NPM should
be removed from these lists.

Reviewed By: asbirlea

Differential Revision: https://reviews.llvm.org/D92818
2020-12-08 13:38:56 -08:00
Arthur Eubanks dee1e6ac42 [llvm-lto2] Use NPM with ENABLE_EXPERIMENTAL_NEW_PASS_MANAGER
Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D92870
2020-12-08 11:43:19 -08:00
wlei eea67baf87 [llvm-profgen][NFC] Fix test failure by making unwinder's output deterministic
Don't know why under Sanitizer build(asan/msan/ubsan), the `std::unordered_map<string, ...>`'s output order is reversed, make the regression test failed.

This change creates a workaround by using sorted container to make the output deterministic.

Reviewed By: hoy, wenlei

Differential Revision: https://reviews.llvm.org/D92816
2020-12-07 22:36:25 -08:00
wlei 1f05b1a9f5 [CSSPGO][llvm-profgen] Context-sensitive profile data generation
This stack of changes introduces `llvm-profgen` utility which generates a profile data file from given perf script data files for sample-based PGO. It’s part of(not only) the CSSPGO work. Specifically to support context-sensitive with/without pseudo probe profile, it implements a series of functionalities including perf trace parsing, instruction symbolization, LBR stack/call frame stack unwinding, pseudo probe decoding, etc. Also high throughput is achieved by multiple levels of sample aggregation and compatible format with one stop is generated at the end. Please refer to: https://groups.google.com/g/llvm-dev/c/1p1rdYbL93s for the CSSPGO RFC.

This change supports context-sensitive profile data generation into llvm-profgen. With simultaneous sampling for LBR and call stack, we can identify leaf of LBR sample with calling context from stack sample . During the process of deriving fall through path from LBR entries, we unwind LBR by replaying all the calls and returns (including implicit calls/returns due to inlining) backwards on top of the sampled call stack. Then the state of call stack as we unwind through LBR always represents the calling context of current fall through path.

we have two types of virtual unwinding 1) LBR unwinding and 2) linear range unwinding.
Specifically, for each LBR entry which can be classified into call, return, regular branch, LBR unwinding will replay the operation by pushing, popping or switching leaf frame towards the call stack and since the initial call stack is most recently sampled, the replay should be in anti-execution order, i.e. for the regular case, pop the call stack when LBR is call, push frame on call stack when LBR is return. After each LBR processed, it also needs to align with the next LBR by going through instructions from previous LBR's target to current LBR's source, which we named linear unwinding. As instruction from linear range can come from different function by inlining, linear unwinding will do the range splitting and record counters through the range with same inline context.

With each fall through path from LBR unwinding, we aggregate each sample into counters by the calling context and eventually generate full context sensitive profile (without relying on inlining) to driver compiler's PGO/FDO.

A breakdown of noteworthy changes:
- Added `HybridSample` class as the abstraction perf sample including LBR stack and call stack
* Extended `PerfReader` to implement auto-detect whether input perf script output contains CS profile, then do the parsing. Multiple `HybridSample` are extracted
* Speed up by aggregating  `HybridSample` into `AggregatedSamples`
* Added VirtualUnwinder that consumes aggregated  `HybridSample` and implements unwinding of calls, returns, and linear path that contains implicit call/return from inlining. Ranges and branches counters are aggregated by the calling context.
 Here calling context is string type, each context is a pair of function name and callsite location info, the whole context is like `main:1 @ foo:2 @ bar`.
* Added PorfileGenerater that accumulates counters by ranges unfolding or branch target mapping, then generates context-sensitive function profile including function body, inferring callee's head sample, callsite target samples, eventually records into ProfileMap.

* Leveraged LLVM build-in(`SampleProfWriter`) writer to support different serialization format with no stop
- `getCanonicalFnName` for callee name and name from ELF section
- Added regression test for both unwinding and profile generation

Test Plan:
ninja & ninja check-llvm

Reviewed By: hoy, wenlei, wmi

Differential Revision: https://reviews.llvm.org/D89723
2020-12-07 13:48:58 -08:00
Derek Schuff 0a391060f1 [WebAssembly] Add Object and ObjectWriter support for wasm COMDAT sections
Allow sections to be placed into COMDAT groups, in addtion to functions and data
segments.

Also make section symbols unnamed, which allows sections with identical names
(section names are independent of their section symbols, but previously we
gave the symbols the same name as their sections, which results in collisions
when sections are identically-named).

Differential Revision: https://reviews.llvm.org/D92691
2020-12-07 12:12:44 -08:00
Georgii Rymar 347ea1af34 [llvm-readobj/elf] - Refine the implementation of "printFunctionStackSize".
This rewrites the logic to get rid of "ELFSymbolRef" API where possible.
This allowed to handle possible errors better, improve warnings reported and add new ones.
Also 'reportWarning' was replaced with 'reportUniqueWarning'

Differential revision: https://reviews.llvm.org/D92545
2020-12-07 14:57:44 +03:00
Fangrui Song 71d43d314c [llvm-readobj] Delete unused declaration 2020-12-06 15:54:17 -08:00
Fangrui Song 6785ca0124 [llvm-c] Delete unimplemented llvm-c/LinkTimeOptimizer.h
The file was added in 2007 but the functions have never been implemented.
Having the file can only cause confusion to existing C API (llvm-c/lto.h) users.
2020-12-06 15:18:25 -08:00
Chris Sears 9737c128f1 [llvmbuildectomy] removed vestigial LLVMBuild.txt files
LLVMBuild has been removed from the build system. However, three LLVMBuild.txt
files remain in the tree. This patch simply removes them.

llvm/lib/ExecutionEngine/Orc/TargetProcess/LLVMBuild.txt
llvm/tools/llvm-jitlink/llvm-jitlink-executor/LLVMBuild.txt
llvm/tools/llvm-profgen/LLVMBuild.txt

Differential Revision: https://reviews.llvm.org/D92693
2020-12-05 22:00:22 +01:00
Francis Visoiu Mistrih 3d381a7102 [llvm-nm][MachO] Don't call getFlags on redacted symbols
Avoid calling getFlags on a non-existent symbol.

The way this is triggered is by calling strip -N on a binary, which sets
the MH_NLIST_OUTOFSYNC_WITH_DYLDINFO header flag. Then, in the
LC_FUNCTION_STARTS command, nm is trying to print the stripped symbols
and needs the proper checks.
2020-12-04 21:48:53 -08:00
Jinsong Ji c8ec685ca5 [llvm-exegesis][PowerPC] Add more register classes
This PR adds more register class support in PowerPC,
mark OperandType for imm and memory operands.

Also added more unit tests for SnippetGenerator.

Reviewed By: #powerpc, steven.zhang

Differential Revision: https://reviews.llvm.org/D88044
2020-12-04 15:02:12 +00:00
Georgii Rymar 44794cde18 [llvm-profgen] - Fix compilation issue after ELFFile<ELFT> interface update.
`D92560` changed `ELFObjectFile::getELFFile` to return reference.
2020-12-04 16:09:25 +03:00
Georgii Rymar ffbce65f95 [lib/Object, tools] - Make ELFObjectFile::getELFFile return reference.
We always have an object, so we don't have to return a pointer.

Differential revision: https://reviews.llvm.org/D92560
2020-12-04 16:02:29 +03:00
Georgii Rymar 5c650d3d9b [llvm-readobj] - Report unique warnings in printProgramHeaders.
This converts `reportWarning` -> `reportUniqueWarning`

Differential revision: https://reviews.llvm.org/D92568
2020-12-04 13:35:44 +03:00
dfukalov 2ce38b3f03 [NFC] Reduce include files dependency.
1. Removed #include "...AliasAnalysis.h" in other headers and modules.
2. Cleaned up includes in AliasAnalysis.h.

Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D92489
2020-12-03 18:25:05 +03:00
Georgii Rymar 50de7d5504 [llvm-readelf/obj] - Report unique warnings in getSymbolForReloc() helper.
Use `reportUniqueWarning` instead of `reportWarning` and refine the
interface of the helper.

Differential revision: https://reviews.llvm.org/D92556
2020-12-03 14:13:26 +03:00
Georgii Rymar 351f736368 [llvm-readelf] - Report unique warnings when dumping hash symbols/histogram.
This converts 2 more places to use `reportUniqueWarning` and adds tests.

Differential revision: https://reviews.llvm.org/D92551
2020-12-03 14:05:04 +03:00
Fangrui Song 1d6ebdfb66 Switch from llvm::is_trivially_copyable to std::is_trivially_copyable
GCC<5 did not support std::is_trivially_copyable. Now LLVM builds require 5.1
we can migrate to std::is_trivially_copyable.

The Optional.h change made MSVC choke
(https://buildkite.com/llvm-project/premerge-checks/builds/18587#cd1bb616-ffdc-4581-9795-b42c284196de)
so I leave it out for now.

Differential Revision: https://reviews.llvm.org/D92514
2020-12-02 22:02:48 -08:00
Sergey Dmitriev 715ba18d3e [llvm-link] use file magic when deciding if input should be loaded as archive
llvm-link should not rely on the '.a' file extension when deciding if input file
should be loaded as archive. Archives may have other extensions (f.e. .lib) or no
extensions at all. This patch changes llvm-link to use llvm::file_magic to check
if input file is an archive.

Reviewed By: RaviNarayanaswamy

Differential Revision: https://reviews.llvm.org/D92376
2020-12-02 17:21:34 -08:00
Sergey Dmitriev 9c955b79fb Revert "[llvm-link] use file magic when deciding if input should be loaded as archive"
This reverts commit 55f8c2fdfb.
2020-12-02 16:53:57 -08:00
Sergey Dmitriev 55f8c2fdfb [llvm-link] use file magic when deciding if input should be loaded as archive
llvm-link should not rely on the '.a' file extension when deciding if input file
should be loaded as archive. Archives may have other extensions (f.e. .lib) or no
extensions at all. This patch changes llvm-link to use llvm::file_magic to check
if input file is an archive.

Reviewed By: RaviNarayanaswamy

Differential Revision: https://reviews.llvm.org/D92376
2020-12-02 16:29:41 -08:00
Reid Kleckner 91e66bfd32 Revert "Use std::is_trivially_copyable", breaks MSVC build
Revert "Delete llvm::is_trivially_copyable and CMake variable HAVE_STD_IS_TRIVIALLY_COPYABLE"

This reverts commit 4d4bd40b57.

This reverts commit 557b00e0af.
2020-12-02 14:30:46 -08:00
Fangrui Song 4d4bd40b57 Use std::is_trivially_copyable
GCC<5 did not support std::is_trivially_copyable. Now LLVM builds require 5.1
we can migrate to std::is_trivially_copyable.
2020-12-02 09:58:07 -08:00
Georgii Rymar 137a25f04a [llvm-readobj, libSupport] - Refine the implementation of the code that dumps build attributes.
This implementation of `ELFDumper<ELFT>::printAttributes()` in llvm-readobj has issues:
1) It crashes when the content of the attribute section is empty.
2) It uses `unwrapOrError` and `reportWarning` calls, though
   ideally we want to use `reportUniqueWarning`.
3) It contains a TODO about redundant format version check.

`lib/Support/ELFAttributeParser.cpp` uses a hardcoded constant instead of the named constant.

This patch fixes all these issues.

Differential revision: https://reviews.llvm.org/D92318
2020-12-02 13:51:32 +03:00
Georgii Rymar 1daace3fbb [llvm-readelf/obj] - Lowercase the warning message reported.
Our warnings/errors reported are using lowercase normally.

This addresses one of review comments from D92382.
2020-12-02 13:09:47 +03:00
Georgii Rymar 21b6c04e3a [llvm-readelf/obj] - Report unique warnings in `parseDynamicTable`.
This makes the warnings reported to be unique and adds test cases.

Differential revision: https://reviews.llvm.org/D92382
2020-12-02 12:52:42 +03:00
Georgii Rymar 01e49204b8 [llvm-readelf/obj] - Refine the error message about the broken string table.
This:
1) Changes `reportWarning` to `reportUniqueWarning` (no-op here).
2) Adds more context to the message.
3) Merges `broken-dynsym-link.test` into `dyn-symbols.test`, adds more testing.

Differential revision: https://reviews.llvm.org/D92380
2020-12-02 12:06:16 +03:00
Eric Astor c64037b784 [ms] [llvm-ml] Support command-line defines
Enable command-line defines as textmacros

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D90059
2020-12-01 18:06:05 -05:00
Eric Astor 8fee2ee9a6 [ms] [llvm-ml] Introduce command-line compatibility for ml.exe and ml64.exe
Switch to OptParser for command-line handling

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D90058
2020-12-01 17:43:44 -05:00
Georgii Rymar ea8c8a5097 [obj2yaml] - Teach tool to emit the "SectionHeaderTable" key and sort sections by file offset.
Currently when we dump sections, we dump them in the order,
which is specified in the sections header table.

With that the order in the output might not match the order in the file.
This patch starts sorting them by by file offsets when dumping.

When the order in the section header table doesn't match the order
in the file, we should emit the "SectionHeaderTable" key. This patch does it.

Differential revision: https://reviews.llvm.org/D91249
2020-12-01 12:59:15 +03:00
Georgii Rymar 82d9fb0ac1 [llvm-readobj] - Introduce `ObjDumper::reportUniqueWarning(const Twine &Msg)`.
This introduces the overload for `reportUniqueWarning` which allows
to avoid using `createError` in many places.

Differential revision: https://reviews.llvm.org/D92371
2020-12-01 12:36:44 +03:00
Georgii Rymar 87481068fd [llvm-readelf] - Switch to using from `reportWarning` to `reportUniqueWarning` in `DynRegionInfo`.
This is a part of the plan we had previously to convert all calls to
`reportUniqueWarning` and then rename it to just `reportWarning`.

I was a bit unsure about this particular change at first, because it doesn't add a
new functionality: seems it is impossible to trigger a warning duplication currently.

At the same time I find the idea of the plan mentioned very reasonable.
And with that we will be sure that `DynRegionInfo` can't report duplicate
warnings, what looks like a nice feature for possible refactorings and further tool development.

Differential revision: https://reviews.llvm.org/D92224
2020-12-01 11:09:30 +03:00
Georgii Rymar 31eeac915a [llvm-readelf/obj] - Move unique warning handling logic to the `ObjDumper`.
This moves the `reportUniqueWarning` method to the base class.

My motivation is the following:
I've experimented with replacing `reportWarning` calls with `reportUniqueWarning`
in ELF dumper. I've found that for example for removing them from `DynRegionInfo` helper
class, it is worth to pass a dumper instance to it (to be able to call dumper()->reportUniqueWarning()).
The problem was that `ELFDumper<ELFT>` is a template class. I had to make `DynRegionInfo` to be templated
and do lots of minor changes everywhere what did not look reasonable/nice.

At the same time I guess one day other dumpers like COFF/MachO/Wasm etc might want to
start using `reportUniqueWarning` API too. Then it looks reasonable to move the logic to the
base class.

With that the problem of passing the dumper instance will be gone.

Differential revision: https://reviews.llvm.org/D92218
2020-12-01 10:53:00 +03:00
Wei Wang 3acda91742 [Remarks][1/2] Expand remarks hotness threshold option support in more tools
This is the #1 of 2 changes that make remarks hotness threshold option
available in more tools. The changes also allow the threshold to sync with
hotness threshold from profile summary with special value 'auto'.

This change modifies the interface of lto::setupLLVMOptimizationRemarks() to
accept remarks hotness threshold. Update all the tools that use it with remarks
hotness threshold options:

* lld: '--opt-remarks-hotness-threshold='
* llvm-lto2: '--pass-remarks-hotness-threshold='
* llvm-lto: '--lto-pass-remarks-hotness-threshold='
* gold plugin: '-plugin-opt=opt-remarks-hotness-threshold='

Differential Revision: https://reviews.llvm.org/D85809
2020-11-30 21:55:49 -08:00
Amy Huang efd1ec0dec Recommit "[llvm-symbolizer] Switch to using native symbolizer by default on Windows"
This reverts commit 1b63177a56.
2020-11-30 17:36:12 -08:00
Nick Lewycky fe43168348 Creating a named struct requires only a Context and a name, but looking up a struct by name requires a Module. The method on Module merely accesses the LLVMContextImpl and no data from the module itself, so this patch moves getTypeByName to a static method on StructType that takes a Context and a name.
There's a small number of users of this function, they are all updated.

This updates the C API adding a new method LLVMGetTypeByName2 that takes a context and a name.

Differential Revision: https://reviews.llvm.org/D78793
2020-11-30 11:34:12 -08:00
David Spickett c2ead57ccf [llvm-objdump] Document --mattr=help in --help output
This does the same as `--mcpu=help` but was only
documented in the user guide.

* Added a test for both options.
* Corrected the single dash in `-mcpu=help` text.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D92305
2020-11-30 12:52:54 +00:00
Georgii Rymar ee9ffc7345 [obj2yaml] - Dump the `EShNum` key in some cases.
This patch starts emitting the `EShNum` key, when the `e_shnum = 0`
and the section header table exists.

`e_shnum` might be 0, when the the number of entries in the section header
table is larger than or equal to SHN_LORESERVE (0xff00).
In this case the real number of entries
in the section header table is held in the `sh_size`
member of the initial entry in section header table.

Currently, obj2yaml crashes when an object has `e_shoff != 0` and the `sh_size`
member of the initial entry in section header table is `0`.
This patch fixes it.

Differential revision: https://reviews.llvm.org/D92098
2020-11-27 15:56:10 +03:00
Georgii Rymar c2090ff594 [obj2yaml] - Don't assert when trying to calculate the expected section offset.
The following line asserts when `sh_addralign > MAX_UINT32 && (uint32_t)sh_addralign == 0`:

```
    ExpectedOffset = alignTo(ExpectedOffset,
                             SecHdr.sh_addralign ? SecHdr.sh_addralign : 1);
```

it happens because `sh_addralign` is truncated to 32-bit value, but `alignTo`
doesn't accept `Align == 0`. We should change `1` to `1uLL`.

Differential revision: https://reviews.llvm.org/D92163
2020-11-27 15:38:22 +03:00
Georgii Rymar 3d811c57aa [llvm-readelf/obj] - Stop calling `reportError` in `printArchSpecificInfo()`.
This is related to MIPS. Currently we might report an error and exit,
though there is no problem to report a warning and try to continue dumping
an object. The code uses `MipsGOTParser<ELFT> Parser`, which is isolated
in this method.

Differential revision: https://reviews.llvm.org/D92090
2020-11-27 10:27:00 +03:00
Georgii Rymar 54ec9bb551 [llvm-readelf/obj] - Report a warning when the value of the DT_PLTREL dynamic tag is invalid.
We report an error for unknown `DT_PLTREL` values.
This switches the error to warning.

Differential revision: https://reviews.llvm.org/D92087
2020-11-26 13:15:59 +03:00
Georgii Rymar c3673ea65d [llvm-readobj] - Fix a warning.
This addresses post review comment for D92018.

The warning was:

```
error: loop variable 'Note' is always a copy because the range of type 'iterator_range<llvm::object::ELFFile<llvm::object::ELFType<llvm::support::big, true> >::Elf_Note_Iterator>' (aka 'iterator_range<Elf_Note_Iterator_Impl<ELFType<(llvm::support::endianness)0U, true> > >') does not return a reference [-Werror,-Wrange-loop-analysis]
      for (const typename ELFT::Note &Note : Obj.notes(S, Err))
```
2020-11-26 10:24:24 +03:00
Zhengyang Liu 75f50e15bf Adding PoisonValue for representing poison value explicitly in IR
Define ConstantData::PoisonValue.
Add support for poison value to LLLexer/LLParser/BitcodeReader/BitcodeWriter.
Add support for poison value to llvm-c interface.
Add support for poison value to OCaml binding.
Add m_Poison in PatternMatch.

Differential Revision: https://reviews.llvm.org/D71126
2020-11-25 17:33:51 -07:00
Andy Wingo 1933c9d41a [WebAssembly] Factor out WasmTableType in binary format
This commit factors out a WasmTableType definition from WasmTable, as is
the case for WasmGlobal and other data types.  Also add support for
extracting the SymbolName for a table from the linking section's symbol
table.

Differential Revision: https://reviews.llvm.org/D91849
2020-11-25 08:00:08 -08:00
Georgii Rymar ce322fb0b8 [llvm-readelf/obj] - Stop using `reportError` when dumping notes.
This starts using `reportUniqueWarnings` instead of `reportError`
in the code that is responsible for dumping notes.

Differential revision: https://reviews.llvm.org/D92021
2020-11-25 15:22:56 +03:00
Martin Storsjö df54f50437 [ORC] Remove a superfluous semicolon, silencing GCC warnings. NFC. 2020-11-25 12:10:53 +02:00
Georgii Rymar fee910e522 [libObject,llvm-readelf] - Stop describing a section/segment in `notes_begin()`.
`notes_begin()` is used for iterating over notes. This API in some cases might print
section type and index. At the same time during iterating, the `Elf_Note_Iterator`
might omit it as it doesn't have this info.

Because of above we might have the redundant duplication of information in warnings:
(See D92021).

```
warning: '[[FILE]]': unable to read notes from the SHT_NOTE section with index 1: SHT_NOTE section [index 1] has invalid offset (0x40) or size (0xffff0000)
```

This change stops reporting section index/type in Object/ELF.h/notes_begin().
(FTR, this was introduced by me for llvm-readobj in D64470).
Instead we can describe sections/program headers on the caller side.

Differential revision: https://reviews.llvm.org/D92081
2020-11-25 12:51:40 +03:00
Georgii Rymar 5edb90c927 [obj2yaml] - Dump section offsets in some cases.
Currently we never dump the `sh_offset` key.
Though it sometimes an important information.

To reduce the noise this patch implements the following logic:
1) The "Offset" key for the first section is always emitted.
2) If we can derive the offset for a next section naturally,
   then the "Offset" key is omitted.

By "naturally" I mean that section[X] offset is expected to be:
```
offsetOf(section[X]) == alignTo(section[X - 1].sh_offset + section[X - 1].sh_size, section[X].sh_addralign)
```

So, when it has the expected value, we omit it from the output.

Differential revision: https://reviews.llvm.org/D91152
2020-11-25 12:41:01 +03:00
Georgii Rymar ae7ac2d665 [llvm-readobj] - An attempt to fix BB after D92018.
AVR and PPC64 bots reports link errors:
(http://lab.llvm.org:8011/#/builders/112/builds/1522)
(http://lab.llvm.org:8011/#/builders/52/builds/1764)

/tmp/cclOvLx0.s: Assembler messages:
/tmp/cclOvLx0.s:9223: Error: symbol `_ZN4llvm12function_refIFvvEE11callback_fnIUlvE2_EEvl' is already defined
/tmp/cclOvLx0.s:9227: Error: symbol `.L._ZN4llvm12function_refIFvvEE11callback_fnIUlvE2_EEvl' is already defined
/tmp/cclOvLx0.s:10272: Error: symbol `_ZN4llvm12function_refIFvvEE11callback_fnIUlvE2_EEvl' is already defined
/tmp/cclOvLx0.s:10276: Error: symbol `.L._ZN4llvm12function_refIFvvEE11callback_fnIUlvE2_EEvl' is already defined
/tmp/cclOvLx0.s:10285: Error: symbol `_ZN4llvm12function_refIFvvEE11callback_fnIUlvE2_EEvl' is already defined
/tmp/cclOvLx0.s:10289: Error: symbol `.L._ZN4llvm12function_refIFvvEE11callback_fnIUlvE2_EEvl' is already defined

/tmp/ccFJYr6I.s: Assembler messages:
/tmp/ccFJYr6I.s:6284: Error: symbol `_ZN4llvm12function_refIFvvEE11callback_fnIUlvE2_EEvl' is already defined
/tmp/ccFJYr6I.s:7053: Error: symbol `_ZN4llvm12function_refIFvvEE11callback_fnIUlvE2_EEvl' is already defined
/tmp/ccFJYr6I.s:7093: Error: symbol `_ZN4llvm12function_refIFvvEE11callback_fnIUlvE2_EEvl' is already defined

I *guess* the reason might be the default lambda argument. I've removed it.
2020-11-25 11:38:58 +03:00
Georgii Rymar ec0b927e4a [llvm-readelf/obj] - Deduplicate the logic that prints notes. NFCI.
We have a similar logic for LLVM/GNU styles that can be deduplicated.
This will allow to replace `reportError` calls with `reportUniqueWarning`
calls in a single place.

Differential revision: https://reviews.llvm.org/D92018
2020-11-25 11:04:13 +03:00
Arthur Eubanks 2c7870dcca [NewPM] Add pipeline EP callback after initial frontend cleanup
This matches the legacy PM's EP_ModuleOptimizerEarly. Some backends use
this extension point and adding the pass somewhere else like
PipelineStartEPCallback doesn't work.

Reviewed By: ychen

Differential Revision: https://reviews.llvm.org/D91804
2020-11-24 21:14:36 -08:00
Georgii Rymar 6b74eabfdd [llvm-readelf/obj] - Refine the implementation of `printGNUVersionSectionProlog`
This:
1) Changes its signature.
2) Refines the name of local variable (`SymTabName`->`LinkedSecName`,
   because SHT_GNU_verneed/SHT_GNU_verdef are linked with the string table, not with the symbol table).
3) Stops using the `unwrapOrError` inside.

Differential revision: https://reviews.llvm.org/D91964
2020-11-24 11:56:22 +03:00
Georgii Rymar 245052ac30 [llvm-readelf/obj] - Improve the error reporting in printStackSize().
This stops using `RelocationRef` API in the `printStackSize` method
and starts using the "regular" API that is used in almost all other places
in ELFDumper.cpp.

This is not only makes the code to be more consistent, but helps to diagnose
issues better, because the `ELFObjectFile` API, which is used
currently to implement stack sized dumping sometimes has a behavior
that just doesn't work well for broken inputs.

E.g see how it gets the `symbol_end` iterator. It will just not work
well for a case when the `sh_size` is broken.

```
template <class ELFT>
basic_symbol_iterator ELFObjectFile<ELFT>::symbol_end() const {
...
  DataRefImpl Sym = toDRI(SymTab, SymTab->sh_size / sizeof(Elf_Sym));
  return basic_symbol_iterator(SymbolRef(Sym, this));
}
```

Differential revision: https://reviews.llvm.org/D91624
2020-11-24 11:49:00 +03:00
Georgii Rymar c3914bf28e [[lvm-readelf/obj] - Remove `tryGetSectionName` helper.
D91867 introduced the `tryGetSectionName` helper.
But we have `getPrintableSectionName` member with the similar
behavior which we can reuse. This patch does it.

Differential revision: https://reviews.llvm.org/D91954
2020-11-24 11:34:27 +03:00
Amy Huang 1b63177a56 Revert "[llvm-symbolizer] Switch to using native symbolizer by default on Windows"
Breaks some asan tests on the buildbot.

This reverts commit c74b427cb2.
2020-11-23 16:29:45 -08:00
Amy Huang c74b427cb2 [llvm-symbolizer] Switch to using native symbolizer by default on Windows
llvm-symbolizer used to use the DIA SDK for symbolization on
Windows; this patch switches to using native symbolization, which was
implemented recently.

Users can still make the symbolizer use DIA by adding the `-dia` flag
in the LLVM_SYMBOLIZER_OPTS environment variable.

Differential Revision: https://reviews.llvm.org/D91814
2020-11-23 15:57:08 -08:00
Haowei Wu cf43308718 [llvm-elfabi] Emit ELF header and string table sections
This change serves to create the initial framework for outputting ELF
files from llvm-elfabi.

Differential Revision: https://reviews.llvm.org/D61767
2020-11-23 12:18:58 -08:00
Haowei Wu ae736d2957 Revert "[llvm-elfabi] Emit ELF header and string table sections"
This reverts commit 53c5fdd59a.

Reason of revert: Some builders failed to build with ld.
2020-11-23 11:58:51 -08:00
Haowei Wu 53c5fdd59a [llvm-elfabi] Emit ELF header and string table sections
This change serves to create the initial framework for outputting ELF
files from llvm-elfabi.

Differential Revision: https://reviews.llvm.org/D61767
2020-11-23 11:31:57 -08:00
Georgii Rymar 76a626b206 [llvm-readelf/obj] - Fix the possible crash when dumping group sections.
It is possible to trigger a crash/misbehavior when the st_name field of
the signature symbol goes past the end of the string table.

This patch fixes it.

Differential revision: https://reviews.llvm.org/D91943
2020-11-23 13:05:12 +03:00
Georgii Rymar 4dcdf0df31 [llvm-readobj] - Stop using `unwrapOrError` in `DumpStyle<ELFT>::getGroups()`
With this we are able to diagnose possible issues much better and
don't exit on an error.

Differential revision: https://reviews.llvm.org/D91867
2020-11-23 12:48:33 +03:00
Georgii Rymar 2584e1e324 [llvm-readobj] - Don't crash when relocation table goes past the EOF.
It is possible to trigger reading past the EOF by breaking fields like
DT_PLTRELSZ, DT_RELSZ or DT_RELASZ

This patch adds a validation in `DynRegionInfo` helper class.

Differential revision: https://reviews.llvm.org/D91787
2020-11-23 10:31:04 +03:00
Ella Ma 1756d67934 [llvm][clang][mlir] Add checks for the return values from Target::createXXX to prevent protential null deref
All these potential null pointer dereferences are reported by my static analyzer for null smart pointer dereferences, which has a different implementation from `alpha.cplusplus.SmartPtr`.

The checked pointers in this patch are initialized by Target::createXXX functions. When the creator function pointer is not correctly set, a null pointer will be returned, or the creator function may originally return a null pointer.

Some of them may not make sense as they may be checked before entering the function, but I fixed them all in this patch. I submit this fix because 1) similar checks are found in some other places in the LLVM codebase for the same return value of the function; and, 2) some of the pointers are dereferenced before they are checked, which may definitely trigger a null pointer dereference if the return value is nullptr.

Reviewed By: tejohnson, MaskRay, jpienaar

Differential Revision: https://reviews.llvm.org/D91410
2020-11-21 21:04:12 -08:00
Michael Liao dcc06597b1 Fix shared build. 2020-11-21 17:07:42 -05:00
wlei 21c91454a8 [llvm-profgen][NFC]Fix build failure on different platform
see titile
Test Plan:
ninja & ninja check-llvm

Reviewed By: hoy

Differential Revision: https://reviews.llvm.org/D91897
2020-11-20 16:36:04 -08:00
wlei 0196b45cea [CSSPGO][llvm-profgen] Instruction symbolization
This stack of changes introduces `llvm-profgen` utility which generates a profile data file from given perf script data files for sample-based PGO. It’s part of(not only) the CSSPGO work. Specifically to support context-sensitive with/without pseudo probe profile, it implements a series of functionalities including perf trace parsing, instruction symbolization, LBR stack/call frame stack unwinding, pseudo probe decoding, etc. Also high throughput is achieved by multiple levels of sample aggregation and compatible format with one stop is generated at the end. Please refer to: https://groups.google.com/g/llvm-dev/c/1p1rdYbL93s for the CSSPGO RFC.

This change adds the support of instruction symbolization. Given the RVA on an instruction pointer, a full calling context can be printed side-by-side with the disassembly code.
E.g.
```
 Disassembly of section .text [0x0, 0x4a]:

 <funcA>:
     0:	mov	eax, edi                           funcA:0
     2:	mov	ecx, dword ptr [rip]               funcLeaf:2 @ funcA:1
     8:	lea	edx, [rcx + 3]                     fib:2 @ funcLeaf:2 @ funcA:1
     b:	cmp	ecx, 3                             fib:2 @ funcLeaf:2 @ funcA:1
     e:	cmovl	edx, ecx                           fib:2 @ funcLeaf:2 @ funcA:1
    11:	sub	eax, edx                           funcLeaf:2 @ funcA:1
    13:	ret                                        funcA:2
    14:	nop	word ptr cs:[rax + rax]
    1e:	nop

 <funcLeaf>:
    20:	mov	eax, edi                           funcLeaf:1
    22:	mov	ecx, dword ptr [rip]               funcLeaf:2
    28:	lea	edx, [rcx + 3]                     fib:2 @ funcLeaf:2
    2b:	cmp	ecx, 3                             fib:2 @ funcLeaf:2
    2e:	cmovl	edx, ecx                           fib:2 @ funcLeaf:2
    31:	sub	eax, edx                           funcLeaf:2
    33:	ret                                        funcLeaf:3
    34:	nop	word ptr cs:[rax + rax]
    3e:	nop

 <fib>:
    40:	lea	eax, [rdi + 3]                     fib:2
    43:	cmp	edi, 3                             fib:2
    46:	cmovl	eax, edi                           fib:2
    49:	ret                                        fib:8
```

Test Plan:
ninja check-llvm

Reviewed By: wenlei, wmi

Differential Revision: https://reviews.llvm.org/D89715
2020-11-20 14:26:27 -08:00
wlei 32221694cb [CSSPGO][llvm-profgen] Disassemble text sections
This stack of changes introduces `llvm-profgen` utility which generates a profile data file from given perf script data files for sample-based PGO. It’s part of(not only) the CSSPGO work. Specifically to support context-sensitive with/without pseudo probe profile, it implements a series of functionalities including perf trace parsing, instruction symbolization, LBR stack/call frame stack unwinding, pseudo probe decoding, etc. Also high throughput is achieved by multiple levels of sample aggregation and compatible format with one stop is generated at the end. Please refer to: https://groups.google.com/g/llvm-dev/c/1p1rdYbL93s for the CSSPGO RFC.

This change enables disassembling the text sections to build various address maps that are potentially used by the virtual unwinder.  A switch `--show-disassembly` is being added to print the disassembly code.

Like the llvm-objdump tool, this change leverages existing LLVM components to parse and disassemble ELF binary files. So far X86 is supported.

Test Plan:

ninja check-llvm

Reviewed By: wmi, wenlei

Differential Revision: https://reviews.llvm.org/D89712
2020-11-20 14:26:26 -08:00
wlei a94fa86229 [CSSPGO][llvm-profgen] Parse mmap events from perf script
This stack of changes introduces `llvm-profgen` utility which generates a profile data file from given perf script data files for sample-based PGO. It’s part of(not only) the CSSPGO work. Specifically to support context-sensitive with/without pseudo probe profile, it implements a series of functionalities including perf trace parsing, instruction symbolization, LBR stack/call frame stack unwinding, pseudo probe decoding, etc. Also high throughput is achieved by multiple levels of sample aggregation and compatible format with one stop is generated at the end. Please refer to: https://groups.google.com/g/llvm-dev/c/1p1rdYbL93s for the CSSPGO RFC.

As a starter, this change sets up an entry point by introducing PerfReader to load profiled binaries and perf traces(including perf events and perf samples). For the event, here it parses the mmap2 events from perf script to build the loader snaps, which is used to retrieve the image load address in the subsequent perf tracing parsing.

As described in llvm-profgen.rst, the tool being built aims to support multiple input perf data (preprocessed by perf script) as well as multiple input binary images. It should also support dynamic reload/unload shared objects by leveraging the loader snaps being built by this change

Reviewed By: wenlei, wmi

Differential Revision: https://reviews.llvm.org/D89707
2020-11-20 14:26:26 -08:00
Georgii Rymar 343dceb831 [llvm-readelf/obj] - Improve error reporting when dumping group sections.
Our code that dumps groups has 3 noticeable issues:
1) It uses `unwrapOrError` in many places.
2) It doesn't allow reporting unique warnings, because the `getGroups` helper is not
   a member of `DumpStyle<ELFT>`.
3) It might just crash. See the comment for `StrTableOrErr->data() + Sym.st_name` line.

In this patch I am starting addressing these points.
For start I've converted one of `unwrapOrError` calls to a unique warning.

Differential revision: https://reviews.llvm.org/D91798
2020-11-20 12:40:23 +03:00
Georgii Rymar aadbe20622 [llvm-readobj] - Introduce `forEachRelocationDo` helper.
Our `printStackSize` implementation currently uses
API like `RelocationRef`, `object::symbol_iterator`.
It is not ideal as it doesn't allow
to handle possible error conditions properly.

Some time ago I started rewriting it and this NFC patch is
a one more step toward to it. Here I am introducing the
`forEachRelocationDo` helper. With it it is possible to iterate
over all kinds of relocations, what is helpful for improving
the code in `printStackSize` and around.

Differential revision: https://reviews.llvm.org/D91530
2020-11-20 12:21:42 +03:00
Georgii Rymar 9a99d23a1b [lib/Object] - Generalize the RelocationResolver API.
This allows to reuse the RelocationResolver from the code
that doesn't want to deal with `RelocationRef` class.

I am going to use it in llvm-readobj. See the description
of D91530 for more details.

Differential revision: https://reviews.llvm.org/D91533
2020-11-20 10:32:49 +03:00
Sam Clegg 1827005cfc [WebAssembly] Add support for named globals in the object format.
Differential Revision: https://reviews.llvm.org/D91769
2020-11-19 00:17:22 -08:00
Scott Linder 2980933d85 [YAMLIO] Support non-null-terminated inputs
In some places the parser guards against dereferencing `End`, while in
others it relies on the presence of a trailing `'\0'` to elide checks.

Add the remaining guards needed to ensure the parser never attempts to
dereference `End`, making it safe to not require a null-terminated input
buffer.

Update the parser fuzzer harness so that it tests with buffers that are
guaranteed to be non-null-terminated, null-terminated, and 1-terminated,
additionally ensuring the result of the parse is the same in each case.

Some of the regression tests were written by inspection, and some are
cases caught by the fuzzer which required additional fixes in the
parser.

Differential Revision: https://reviews.llvm.org/D84050
2020-11-18 23:06:03 +00:00
Scott Linder 544cb649d7 [YAMLIO] Add a generic YAML fuzzer harness
This is essentially a clone of the existing fuzzer added in D50839, but
for the whole parser Streamer, and currently only testing for sanitizer
violations.

Differential Revision: https://reviews.llvm.org/D91573
2020-11-18 23:06:03 +00:00
Arthur Eubanks e597116f05 [NPM] Add implicit basic-aa before other AA
This matches the legacy AA infra and fixes
llvm/test/ANalysis/TypeBasedAliasAnalysis/precedence.ll under NPM.
2020-11-18 13:48:24 -08:00
Andrew Paverd 0139c8af8d [CFGuard] Add address-taken IAT tables and delay-load support
This patch adds support for creating Guard Address-Taken IAT Entry Tables (.giats$y sections) in object files, matching the behavior of MSVC. These contain lists of address-taken imported functions, which are used by the linker to create the final GIATS table.
Additionally, if any DLLs are delay-loaded, the linker must look through the .giats tables and add the respective load thunks of address-taken imports to the GFIDS table, as these are also valid call targets.

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D87544
2020-11-17 18:24:45 -08:00
Serge Guelton 6795984a47 [build] Fix dependencies of LLVMExegesis 2020-11-17 06:15:02 -05:00
Jonas Devlieghere ee91e2311c [dsymutil] Generalize handling of aliases external symbols
In f9fb9da36c we fixed spurious warnings
caused by aliases to private extern symbols. This patch generalizes that
to regular external symbols as well.
2020-11-16 11:04:31 -08:00
Lang Hames f62e5f4569 [MCJIT] Profile the code generated by MCJIT engine using Intel VTune profiler
Patch by Elena Kovanova. Thanks Elena!

Problem:

LLVM already has a feature to profile the JIT-compiled code with VTune. This is
done using Intel JIT Profiling API (https://github.com/intel/ittapi). Function
information is captured by VTune as soon as the function is JIT-compiled. We
tried to use the same approach to report the function information generated by
the MCJIT engine – read parsing the debug information for in-memory ELF module
and report it using JIT API. As the results, we figured out that it did not work
properly for the following cases: inline functions, the functions located in
multiple source files, the functions having several bodies (address ranges).

Solution:

To overcome limitations described above, we have introduced new APIs as a part
of Intel ITT APIs to report the entire in-memory ELF module to be further
processed as regular ELF binaries with debug information.

This patch

1. Switches LLVM to open source version of Intel ITT/JIT APIs
(https://github.com/intel/ittapi) to keep it always up to date.

2. Adds support of profiling the code generated by MCJIT engine using Intel
VTune profiler

Another separate patch will get rid of obsolete Intel ITT APIs stuff, having
LLVM already switched to https://github.com/intel/ittapi.

Differential Revision: https://reviews.llvm.org/D86435
2020-11-16 19:28:14 +11:00
Lang Hames f2980e8849 [ORC] Fix missing return value. 2020-11-14 09:31:20 +11:00
serge-sans-paille 9218ff50f9 llvmbuildectomy - replace llvm-build by plain cmake
No longer rely on an external tool to build the llvm component layout.

Instead, leverage the existing `add_llvm_componentlibrary` cmake function and
introduce `add_llvm_component_group` to accurately describe component behavior.

These function store extra properties in the created targets. These properties
are processed once all components are defined to resolve library dependencies
and produce the header expected by llvm-config.

Differential Revision: https://reviews.llvm.org/D90848
2020-11-13 10:35:24 +01:00
Lang Hames 1d0676b54c [ORC] Break up OrcJIT library, add Orc-RPC based remote TargetProcessControl
implementation.

This patch aims to improve support for out-of-process JITing using OrcV2. It
introduces two new class templates, OrcRPCTargetProcessControlBase and
OrcRPCTPCServer, which together implement the TargetProcessControl API by
forwarding operations to an execution process via an Orc-RPC Endpoint. These
utilities are used to implement out-of-process JITing from llvm-jitlink to
a new llvm-jitlink-executor tool.

This patch also breaks the OrcJIT library into three parts:
  -- OrcTargetProcess: Contains code needed by the JIT execution process.
  -- OrcShared: Contains code needed by the JIT execution and compiler
     processes
  -- OrcJIT: Everything else.

This break-up allows JIT executor processes to link against OrcTargetProcess
and OrcShared only, without having to link in all of OrcJIT. Clients executing
JIT'd code in-process should start linking against OrcTargetProcess as well as
OrcJIT.

In the near future these changes will enable:
  -- Removal of the OrcRemoteTargetClient/OrcRemoteTargetServer class templates
     which provided similar functionality in OrcV1.
  -- Restoration of Chapter 5 of the Building-A-JIT tutorial series, which will
     serve as a simple usage example for these APIs.
  -- Implementation of lazy, cross-target compilation in lli's -jit-kind=orc-lazy
     mode.
2020-11-13 17:05:13 +11:00
Haowei Wu ad0da312c0 [NFC] Reformat llvm-elfabi
Makes diff in next commit more readable.
2020-11-11 10:41:51 -08:00
Fangrui Song 16f8142b11 [llvm-objcopy][ELF] Try fixing non-determinism of Segment::firstSection 2020-11-11 10:20:30 -08:00
Fangrui Song 20de182246 [llvm-objcopy] --only-keep-debug: place zero-size segment according to its parent segment
Alternative to D74755. sectionWithinSegment() treats an empty section as having
a size of 1. Due to the rule, an empty .tdata will not be attributed to an
empty PT_TLS. (The empty p_align=64 PT_TLS is for Android Bionic's TCB
compatibility (ELF-TLS). See https://reviews.llvm.org/D62055#1507426)

Currently --only-keep-debug will not layout a segment with no section
(layoutSegmentsForOnlyKeepDebug()), thus p_offset of PT_TLS can go past the end
of the file. The strange p_offset can trigger validation errors for subsequent
tools, e.g. llvm-objcopy errors when reading back the separate debug file
(readProgramHeaders()).

This patch places such an empty segment according to its parent segment.  This
special cases works for the empty PT_TLS used in Android. For a non-empty
segment, it should have at least one non-empty section and will be handled by
the normal code. Note, p_memsz PT_LOAD is rejected by both Linux and FreeBSD.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D90897
2020-11-11 09:21:10 -08:00
Hans Wennborg 418f18c6cd Revert "Reland [CFGuard] Add address-taken IAT tables and delay-load support"
This broke both Firefox and Chromium (PR47905) due to what seems like dllimport
function not being handled correctly.

> This patch adds support for creating Guard Address-Taken IAT Entry Tables (.giats$y sections) in object files, matching the behavior of MSVC. These contain lists of address-taken imported functions, which are used by the linker to create the final GIATS table.
> Additionally, if any DLLs are delay-loaded, the linker must look through the .giats tables and add the respective load thunks of address-taken imports to the GFIDS table, as these are also valid call targets.
>
> Reviewed By: rnk
>
> Differential Revision: https://reviews.llvm.org/D87544

This reverts commit cfd8481da1.
2020-11-11 16:03:33 +01:00
Florian Hahn 250de7388b
[llvm-reduce] Add reduction for special globals like llvm.used.
This patch adds a reduction of 'special' globals that lead to further
reductions (e.g. alias or regular globals reduction) being less efficient
because there are special constraints on values referenced in those
special globals. For example, values in @llvm.used and
@llvm.compiler.used need to be named, so replacing all uses of an
alias/global with undef or a different unnamed constant results in
invalid IR.

More details:
https://llvm.org/docs/LangRef.html#intrinsic-global-variables

Reviewed By: lebedev.ri

Differential Revision: https://reviews.llvm.org/D90302
2020-11-11 11:25:05 +00:00
Keith Smiley 2319c49339 [llvm-cov][NFC] Fix typo in format comment 2020-11-09 14:47:39 -08:00
Georgii Rymar a7a447be0f [yaml2obj] - ProgramHeaders: introduce FirstSec/LastSec instead of Sections list.
Imagine we have a YAML declaration of few sections: `foo1`, `<unnamed 2>`, `foo3`, `foo4`.

To put them into segment we can do (1*):

```
Sections:
 - Section: foo1
 - Section: foo4
```

or we can use (2*):

```
Sections:
 - Section: foo1
 - Section: foo3
 - Section: foo4
```

or (3*) :

```
Sections:
 - Section: foo1
## "(index 2)" here is a name that we automatically created for a unnamed section.
 - Section: (index 2)
 - Section: foo3
 - Section: foo4
```

It looks really confusing that we don't have to list all of sections.

At first I've tried to make this rule stricter and report an error when there is a gap
(i.e. when a section is included into segment, but not listed explicitly).
This did not work perfect, because such approach conflicts with unnamed sections/fills (see (3*)).

This patch drops "Sections" key and introduces 2 keys instead: `FirstSec` and `LastSec`.
Both are optional.

Differential revision: https://reviews.llvm.org/D90458
2020-11-09 13:00:50 +03:00
Georgii Rymar 99a6401acc Recommit: [llvm-readelf/obj] - Allow dumping of ELF header even if some elements are corrupt.
This is recommit for D90903 with fixes for BB:
1) Used std::move<> when returning Expected<> (http://lab.llvm.org:8011/#/builders/112/builds/913)
2) Fixed the name of temporarily file in the file-headers.test (http://lab.llvm.org:8011/#/builders/36/builds/1269)
   (a local old temporarily file was used before)

For creating `ELFObjectFile` instances we have the factory method
`ELFObjectFile<ELFT>::create(MemoryBufferRef Object)`.

The problem of this method is that it scans the section header to locate some sections.
When a file is truncated or has broken fields in the ELF header, this approach does
not allow us to create the `ELFObjectFile` and dump the ELF header.

This is https://bugs.llvm.org/show_bug.cgi?id=40804

This patch suggests a solution - it allows to delay scaning sections in the
`ELFObjectFile<ELFT>::create`. It now allows user code to call an object
initialization (`initContent()`) later. With that it is possible,
for example, for dumpers just to dump the file header and exit.
By default initialization is still performed as before, what helps to keep
the logic of existent callers untouched.

I've experimented with different approaches when worked on this patch.
I think this approach is better than doing initialization of sections (i.e. scan of them)
on demand, because normally users of `ELFObjectFile` API expect to work with a valid object.
In most cases when a section header table can't be read (because of an error), we don't
have to continue to work with object. So we probably don't need to implement a more complex API.

Differential revision: https://reviews.llvm.org/D90903
2020-11-09 12:53:53 +03:00
Georgii Rymar f59216b58f Revert "[llvm-readelf/obj] - Allow dumping of ELF header even if some elements are corrupt."
This reverts commit ea8a0b8b29.

It broke BBots.
http://lab.llvm.org:8011/#/builders/14/builds/1439
http://lab.llvm.org:8011/#/builders/112/builds/913
2020-11-09 11:50:50 +03:00
Georgii Rymar ea8a0b8b29 [llvm-readelf/obj] - Allow dumping of ELF header even if some elements are corrupt.
For creating `ELFObjectFile` instances we have the factory method
`ELFObjectFile<ELFT>::create(MemoryBufferRef Object)`.

The problem of this method is that it scans the section header to locate some sections.
When a file is truncated or has broken fields in the ELF header, this approach does
not allow us to create the `ELFObjectFile` and dump the ELF header.

This is https://bugs.llvm.org/show_bug.cgi?id=40804

This patch suggests a solution - it allows to delay scaning sections in the
`ELFObjectFile<ELFT>::create`. It now allows user code to call an object
initialization (`initContent()`) later. With that it is possible,
for example, for dumpers just to dump the file header and exit.
By default initialization is still performed as before, what helps to keep
the logic of existent callers untouched.

I've experimented with different approaches when worked on this patch.
I think this approach is better than doing initialization of sections (i.e. scan of them)
on demand, because normally users of `ELFObjectFile` API expect to work with a valid object.
In most cases when a section header table can't be read (because of an error), we don't
have to continue to work with object. So we probably don't need to implement a more complex API.

Differential revision: https://reviews.llvm.org/D90903
2020-11-09 11:27:07 +03:00
António Afonso 32d239a758 Fix debug_abbrev emitter to only assign table id once
While generating yamls for my tests I noticed that the new debug_abbrev format (with multiple table support) was incorrectly assigning id's to the table because it was generating one per abbrev entry in the table. For instance, the first table would get id 4 when 5 abbrev entries existed in the table. By itself this is not a problem but the corresponding debug_info sections were still referencing id 0. This was introduced here: https://reviews.llvm.org/D83116.

Maybe a better fix is to actually correctly calculate the table id when emitting debug info? From a quick glance it seems to me the ID is just being calculated as the distance between the first DWARFAbbreviationDeclarationSet and the one the debug info entry points to, which means it's just its index and not the actual table id that was generated when emitting the debug_abbrev tables. With my fix I guess this is fine but on the diff that introduced this Pavel mentioned that he would like to have some sort of unique id between them but not necessarily +1 increasing, but for that to work we need to actually find the table ID, I guess by going directly to Y.DebugAbbrev but to honest I have no idea how to link the DWARFAbbreviationDeclarationSet and the Y.DebugAbbrev, so I just did this simple fix.

I also realized there's barely any tests for MachO so it might useful to invest on that if the tool is being reworked on.

Reviewed By: Higuoxing, jhenderson

Differential Revision: https://reviews.llvm.org/D87179
2020-11-08 18:11:50 -08:00
Alexander Shaposhnikov 273ba73689 [llvm-objcopy][MachO] Minor refactoring of MachOLayoutBuilder
Move the implementation of getStringTableBuilderKind into
MachOLayoutBuilder.cpp. NFC.

Test plan: make check-all
2020-11-07 20:25:49 -08:00
Cornelius Aschermann 4d54c6fc5a [llvm-objcopy][MachO] Fix adding multiple sections
This diff fixes missing fields initialization (Size, VMSize).
Previously this resulted in broken binaries when multiple sections
were added in one tool's invocatation.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D90690
2020-11-07 18:16:06 -08:00
Zequan Wu c75a0a1e9d [llvm-cov] Fix missing slash in -path-equivalence 2020-11-06 14:54:11 -08:00
Alexander Shaposhnikov f147f59cd3 [llvm-objcopy][MachO] Skip sections with zero offset
Some binaries can contain regular sections with zero offset and zero size.
This diff makes llvm-objcopy's handling of such sections consistent with
cctools's strip (which doesn't modify them),
previously the tool would allocate file space for them.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D90796
2020-11-06 13:29:43 -08:00
Rahman Lavaee 82e7c4ce45 [obj2yaml] [yaml2obj] Add yaml support for SHT_LLVM_BB_ADDR_MAP section.
YAML support allows us to better test the feature in the subsequent patches. The implementation is quite similar to the .stack_sizes section.

Reviewed By: jhenderson, grimar

Differential Revision: https://reviews.llvm.org/D88717
2020-11-06 12:44:42 -08:00
Arthur Eubanks ab0ddbc38a Reland [NewPM] Add OptimizationLevel param to registerPipelineStartEPCallback
This allows targets to skip optional optimization passes at -O0.

Reviewed By: ychen

Differential Revision: https://reviews.llvm.org/D90777
2020-11-04 13:11:40 -08:00