Commit Graph

446 Commits

Author SHA1 Message Date
Mehdi Amini a0ddb1ed46 [ThinLTO] Make a copy of buffer identifier in ThinLTOCodeGenerator
We can't assume that the `const char *` provided through libLTO has a
lifetime that expands beyond the codegenerator itself.

llvm-svn: 295018
2017-02-14 02:20:51 +00:00
Davide Italiano 20a895c4be [LTO] Make sure we flush buffers to work around linker shenanigans.
lld, at least, doesn't call global destructors by default (unless
--full-shutdown is passed) because it's, allegedly, expensive.

llvm-svn: 294953
2017-02-13 14:39:51 +00:00
Davide Italiano ebd471974a [lib/LTO] Initial support for optimization remarks in the new API.
llvm-svn: 294882
2017-02-12 03:31:30 +00:00
Davide Italiano 690ed9dec7 [LTO] Share the optimization remarks setup between Thin/Full LTO.
llvm-svn: 294807
2017-02-10 23:49:38 +00:00
Davide Italiano 46d72b1b7f [lib/LTO] Rework optimization remarkers setup.
This makes this code much more similar to what ThinLTO is
using (also API wise), so now we can probably use a single
code path instead of copying stuff around.

llvm-svn: 294792
2017-02-10 22:16:17 +00:00
Bob Haarman e2fe59fddd fix nullptr Mangler in LTOModule
Reviewers: kcc, pcc

Subscribers: mehdi_amini

Differential Revision: https://reviews.llvm.org/D29523

llvm-svn: 294079
2017-02-04 01:28:44 +00:00
Peter Collingbourne e6fd9ff96a IRMover: Merge flags LinkModuleInlineAsm and IsPerformingImport.
Currently these flags are always the inverse of each other, so there is
no need to keep them separate.

Differential Revision: https://reviews.llvm.org/D29471

llvm-svn: 294016
2017-02-03 17:01:14 +00:00
Mehdi Amini 1380edf4ef Revert "[ThinLTO] Add an auto-hide feature"
This reverts commit r293970.

After more discussion, this belongs to the linker side and
there is no added value to do it at this level.

llvm-svn: 293993
2017-02-03 07:41:43 +00:00
Mehdi Amini b0a8ff71e5 [ThinLTO] Add an auto-hide feature
When a symbol is not exported outside of the
DSO, it is can be hidden. Usually we try to internalize
as much as possible, but it is not always possible, for
instance a symbol can be referenced outside of the LTO
unit, or there can be cross-module reference in ThinLTO.

This is a recommit of r293912 after fixing build failures,
and a recommit of r293918 after fixing LLD tests.

Differential Revision: https://reviews.llvm.org/D28978

llvm-svn: 293970
2017-02-03 00:32:38 +00:00
Mehdi Amini 21c89dc920 Revert "[ThinLTO] Add an auto-hide feature"
This reverts commit r293918, one lld test does not pass.

llvm-svn: 293961
2017-02-02 23:20:36 +00:00
Bob Haarman dd4ebc1d3b [lto] add getLinkerOpts()
Summary: Some compilers, including MSVC and Clang, allow linker options to be specified in source files. In the legacy LTO API, there is a getLinkerOpts() method that returns linker options for the bitcode module being processed. This change adds that method to the new API, so that the COFF linker can get the right linker options when using the new LTO API.

Reviewers: pcc, ruiu, mehdi_amini, tejohnson

Reviewed By: pcc

Differential Revision: https://reviews.llvm.org/D29207

llvm-svn: 293950
2017-02-02 23:00:49 +00:00
Mehdi Amini 97624fb1ec [ThinLTO] Add an auto-hide feature
When a symbol is not exported outside of the
DSO, it is can be hidden. Usually we try to internalize
as much as possible, but it is not always possible, for
instance a symbol can be referenced outside of the LTO
unit, or there can be cross-module reference in ThinLTO.

This is a recommit of r293912 after fixing build failures.

Differential Revision: https://reviews.llvm.org/D28978

llvm-svn: 293918
2017-02-02 18:31:35 +00:00
Mehdi Amini 827600deaf Revert "[ThinLTO] Add an auto-hide feature"
This reverts r293912, bots are broken.

llvm-svn: 293914
2017-02-02 18:24:37 +00:00
Mehdi Amini dc5a7444f0 [ThinLTO] Add an auto-hide feature
When a symbol is not exported outside of the
DSO, it is can be hidden. Usually we try to internalize
as much as possible, but it is not always possible, for
instance a symbol can be referenced outside of the LTO
unit, or there can be cross-module reference in ThinLTO.

Differential Revision: https://reviews.llvm.org/D28978

llvm-svn: 293912
2017-02-02 18:13:46 +00:00
Peter Collingbourne 4613626d49 LTO: Link non-prevailing weak_odr or linkonce_odr globals into the combined module with available_externally linkage.
These linkages mean that the ultimately prevailing symbol will have the same
semantics as any non-prevailing copy of the symbol, so we are free to ignore
the linker's resolution.

Differential Revision: https://reviews.llvm.org/D29367

llvm-svn: 293865
2017-02-02 05:22:42 +00:00
Peter Collingbourne c387e70c69 Linker: Move special casing for available_externally in IRMover to clients. NFCI.
The goal is to simplify the semantic model for clients of IRMover.

Differential Revision: https://reviews.llvm.org/D29435

llvm-svn: 293864
2017-02-02 05:12:15 +00:00
Davide Italiano 0dd200e04d [LTO] Teach lib/LTO about the new pass manager.
Differential Revision:  https://reviews.llvm.org/D28997

llvm-svn: 292864
2017-01-24 00:58:24 +00:00
David L. Jones d21529fa0d [Analysis] Add LibFunc_ prefix to enums in TargetLibraryInfo. (NFC)
Summary:
The LibFunc::Func enum holds enumerators named for libc functions.
Unfortunately, there are real situations, including libc implementations, where
function names are actually macros (musl uses "#define fopen64 fopen", for
example; any other transitively visible macro would have similar effects).

Strictly speaking, a conforming C++ Standard Library should provide any such
macros as functions instead (via <cstdio>). However, there are some "library"
functions which are not part of the standard, and thus not subject to this
rule (fopen64, for example). So, in order to be both portable and consistent,
the enum should not use the bare function names.

The old enum naming used a namespace LibFunc and an enum Func, with bare
enumerators. This patch changes LibFunc to be an enum with enumerators prefixed
with "LibFFunc_". (Unfortunately, a scoped enum is not sufficient to override
macros.)

There are additional changes required in clang.

Reviewers: rsmith

Subscribers: mehdi_amini, mzolotukhin, nemanjai, llvm-commits

Differential Revision: https://reviews.llvm.org/D28476

llvm-svn: 292848
2017-01-23 23:16:46 +00:00
Mehdi Amini f39ce99474 [ThinLTO] Don't perform computeDeadSymbols during O0 link, as the result is never used (NFC)
llvm-svn: 292679
2017-01-20 23:34:12 +00:00
Mehdi Amini b29904667f [ThinLTO] The "codegen only" path didn't honor the recently added file-based API
llvm-svn: 292667
2017-01-20 22:45:34 +00:00
Peter Collingbourne e02b74e294 IPO, LTO: Plumb the summary from the LTO API into the pass manager.
Differential Revision: https://reviews.llvm.org/D28840

llvm-svn: 292661
2017-01-20 22:18:52 +00:00
Teresa Johnson 4566c6db87 [ThinLTO] Drop non-prevailing non-ODR weak to declarations
Summary:
Allow non-ODR weak/linkonce non-prevailing copies to be marked
as available_externally in the index. Add support for dropping these to
declarations in the backend.

Reviewers: mehdi_amini, pcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D28806

llvm-svn: 292656
2017-01-20 21:54:58 +00:00
Peter Collingbourne 58ffcfbffa LTO: Flush the resolution file after writing to it.
Without this the file could be truncated if the linker crashes.

llvm-svn: 292532
2017-01-19 23:10:14 +00:00
Benjamin Kramer 061f4a5fe6 Apply clang-tidy's performance-unnecessary-value-param to LLVM.
With some minor manual fixes for using function_ref instead of
std::function. No functional change intended.

llvm-svn: 291904
2017-01-13 14:39:03 +00:00
Chandler Carruth 3bab7e1a79 [PM] Separate the LoopAnalysisManager from the LoopPassManager and move
the latter to the Transforms library.

While the loop PM uses an analysis to form the IR units, the current
plan is to have the PM itself establish and enforce both loop simplified
form and LCSSA. This would be a layering violation in the analysis
library.

Fundamentally, the idea behind the loop PM is to *transform* loops in
addition to running passes over them, so it really seemed like the most
natural place to sink this was into the transforms library.

We can't just move *everything* because we also have loop analyses that
rely on a subset of the invariants. So this patch splits the the loop
infrastructure into the analysis management that has to be part of the
analysis library, and the transform-aware pass manager.

This also required splitting the loop analyses' printer passes out to
the transforms library, which makes sense to me as running these will
transform the code into LCSSA in theory.

I haven't split the unittest though because testing one component
without the other seems nearly intractable.

Differential Revision: https://reviews.llvm.org/D28452

llvm-svn: 291662
2017-01-11 09:43:56 +00:00
Mehdi Amini c92b612636 [ThinLTO] Hash more part of the config to build cache entries
This has been fixed in the "new" LTO API used by Gold/LLD, this is
fixing the same issue in the libLTO API used by ld64 (amongst other)

llvm-svn: 291518
2017-01-10 00:55:47 +00:00
Mehdi Amini 83a807ebae [ThinLTO] Expected<> return values need to be handled to avoid an assertion
llvm-svn: 291377
2017-01-08 00:30:27 +00:00
Teresa Johnson 6c475a7595 ThinLTO: add early "dead-stripping" on the Index
Summary:
Using the linker-supplied list of "preserved" symbols, we can compute
the list of "dead" symbols, i.e. the one that are not reachable from
a "preserved" symbol transitively on the reference graph.
Right now we are using this information to mark these functions as
non-eligible for import.

The impact is two folds:
- Reduction of compile time: we don't import these functions anywhere
  or import the function these symbols are calling.
- The limited number of import/export leads to better internalization.

Patch originally by Mehdi Amini.

Reviewers: mehdi_amini, pcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D23488

llvm-svn: 291177
2017-01-05 21:34:18 +00:00
Davide Italiano 6309895770 [lib/LTO] Simplify logic removing set but unused variable. NFCI.
Reported by David Binderman and ack'ed by Teresa on IRC.
PR: 31527

llvm-svn: 291000
2017-01-04 20:37:57 +00:00
Mehdi Amini cc7fbf718d [ThinLTO] Honor -O{0,1,2,4} passed through the libLTO interface for ThinLTO
This was hardcoded to be O3 till now, without any way to change it
without changing the code.

llvm-svn: 290682
2016-12-28 19:37:16 +00:00
Teresa Johnson a61f5e3796 [ThinLTO] Import composite types as declarations
Summary:
When reading the metadata bitcode, create a type declaration when
possible for composite types when we are importing. Doing this in
the bitcode reader saves memory. Also it works naturally in the case
when the type ODR map contains a definition for the same composite type
because it was used in the importing module (buildODRType will
automatically use the existing definition and not create a type
declaration).

For Chromium built with -g2, this reduces the aggregate size of the
generated native object files by 66% (from 31G to 10G). It reduced
the time through the ThinLTO link and backend phases by about 20% on
my machine.

Reviewers: mehdi_amini, dblaikie, aprantl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D27775

llvm-svn: 289993
2016-12-16 21:25:01 +00:00
Dehao Chen 2797800595 Pass sample pgo flags to thinlto.
Summary: ThinLTO needs to invoke SampleProfileLoader pass during link time in order to annotate profile correctly after module importing.

Reviewers: davidxl, mehdi_amini, tejohnson

Subscribers: pcc, davide, llvm-commits, mehdi_amini

Differential Revision: https://reviews.llvm.org/D27790

llvm-svn: 289957
2016-12-16 16:48:46 +00:00
Davide Italiano 2ceb628f36 [LTO] Reject modules without datalayout.
Also, udpate the ~60 failing tests in the tree which did
not contain a valid datalayout.
This fixes PR31123. lld will be updated in a following patch,
immediately after this is committed.

Differential Revision:  https://reviews.llvm.org/D27082

llvm-svn: 289719
2016-12-14 21:57:04 +00:00
Mehdi Amini 8e13bc4562 [ThinLTO] Add an API to trigger file-based API for returning objects to the linker
Summary:
The motivation is to support better the -object_path_lto option on
Darwin. The linker needs to write down the generate object files on
disk for later use by lldb or dsymutil (debug info are not present
in the final binary). We're moving this into libLTO so that we can
be smarter when a cache is enabled and hard-link when possible
instead of duplicating the files.

Reviewers: tejohnson, deadalnix, pcc

Subscribers: dexonsmith, llvm-commits

Differential Revision: https://reviews.llvm.org/D27507

llvm-svn: 289631
2016-12-14 04:56:42 +00:00
Peter Collingbourne 1a0720e8c4 LTO: Add support for multi-module bitcode files.
Differential Revision: https://reviews.llvm.org/D27313

llvm-svn: 289621
2016-12-14 01:17:59 +00:00
Peter Collingbourne 77f4c30d6f LTO: Port the legacy LTO API to ModuleSymbolTable.
Differential Revision: https://reviews.llvm.org/D27078

llvm-svn: 289576
2016-12-13 20:01:58 +00:00
Peter Collingbourne ad90369a94 LTO: Port the new LTO API to ModuleSymbolTable.
Differential Revision: https://reviews.llvm.org/D27077

llvm-svn: 289574
2016-12-13 19:43:49 +00:00
Teresa Johnson 040cc16835 [ThinLTO] Import only necessary DICompileUnit fields
Summary:
As discussed on mailing list, for ThinLTO importing we don't need
to import all the fields of the DICompileUnit. Don't import enums,
macros, retained types lists. Also only import local scoped imported
entities. Since we don't currently import any global variables,
we also don't need to import the list of global variables (added an
assert to verify none are being imported).

This is being done by pre-populating the value map entries to map
the unneeded metadata to nullptr. For the imported entities, we can
simply replace the source module's list with a new list containing
only those needed imported entities. This is done in the IRLinker
constructor so that value mapping automatically does the desired
mapping.

Reviewers: mehdi_amini, dexonsmith, dblaikie, aprantl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D27635

llvm-svn: 289441
2016-12-12 16:09:30 +00:00
Peter Collingbourne f4257528e9 LTO: Hash the parts of the LTO configuration that affect code generation.
Most importantly, we need to hash the relocation model, otherwise we can
end up trying to link non-PIC object files into PIEs or DSOs.

Differential Revision: https://reviews.llvm.org/D27556

llvm-svn: 289024
2016-12-08 05:28:30 +00:00
Adam Nemet 4c207a6a1f [LTOs] Allow generation of hotness information
The flag is passed by the clang driver.

Differential Revision: https://reviews.llvm.org/D27331

llvm-svn: 288519
2016-12-02 17:53:56 +00:00
Peter Collingbourne 863cbfbeba Object: Extract a ModuleSymbolTable class from IRObjectFile.
This class represents a symbol table built from in-memory IR. It provides
access to GlobalValues and should only be used if such access is required
(e.g. in the LTO implementation). We will eventually change IRObjectFile
to read from a bitcode symbol table rather than using ModuleSymbolTable,
so it would not be able to expose the module.

Differential Revision: https://reviews.llvm.org/D27073

llvm-svn: 288319
2016-12-01 06:51:47 +00:00
Peter Collingbourne dac43b49bd LTO: Remove ModuleLoader, make loadModuleFromBuffer static and move into its only client, ThinLTOCodeGenerator.
This is no longer the recommended way to load modules for importing, so it should not be public API.

Differential Revision: https://reviews.llvm.org/D27292

llvm-svn: 288316
2016-12-01 05:52:32 +00:00
Peter Collingbourne fb8c2a4a6b LTO: Remove Symbol::getIRName().
Its only use was in the LTO implementation. Also document
Symbol::getName().

llvm-svn: 288302
2016-12-01 02:51:12 +00:00
Adam Nemet 58951d3869 [LTO] Move finishOptimizationRemarks after codegen
This addresses the comment D26832.

llvm-svn: 288041
2016-11-28 16:51:49 +00:00
Davide Italiano 8812f28f47 [lib/LTO] Rename few instances of Lto to LTO.
llvm-svn: 287840
2016-11-24 00:23:09 +00:00
Adam Nemet de33651bd9 Rename option to -lto-pass-remarks-output
The new option -pass-remarks-output broke LLVM_LINK_LLVM_DYLIB because
of the duplicate option name with opt.

llvm-svn: 287627
2016-11-22 07:35:14 +00:00
Benjamin Kramer ffd3715d16 Give some helper classes/functions internal linkage. NFC.
llvm-svn: 287462
2016-11-19 20:44:26 +00:00
Mehdi Amini 19f176b982 [ThinLTO] Implement -pass-remarks-output in ThinLTOCodeGenerator
Summary:
This will also be added to the LTO API, right now this will
bring ThinLTO on par with Monolithic LTO on Darwin.

Reviewers: anemet

Subscribers: tejohnson, llvm-commits

Differential Revision: https://reviews.llvm.org/D26886

llvm-svn: 287450
2016-11-19 18:20:05 +00:00
Mehdi Amini 6f40836823 Change setDiagnosticsOutputFile to take a unique_ptr from a raw pointer (NFC)
Summary:
This makes it explicit that ownership is taken. Also replace all `new`
with make_unique<> at call sites.

Reviewers: anemet

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D26884

llvm-svn: 287449
2016-11-19 18:19:41 +00:00
Adam Nemet e9bd022c41 [LTO] Add option to generate optimization records
It is used to drive this from the clang driver via -mllvm.

Same option name is used as in opt.

Differential Revision: https://reviews.llvm.org/D26832

llvm-svn: 287356
2016-11-18 18:06:28 +00:00