Commit Graph

275 Commits

Author SHA1 Message Date
Peter Collingbourne edf8432480 LTO: Don't bother trying to mangle unnamed globals, as they can't be preserved with MustPreserveSymbols.
Summary: Should fix sanitizer-windows bot.

Reviewers: joker.eph

Subscribers: llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D19635

llvm-svn: 267820
2016-04-27 23:48:11 +00:00
Sjoerd Meijer 41beee6575 Clean up to avoid compiler warnings for casting away const qualifiers.
Differential Revision: http://reviews.llvm.org/D19598

llvm-svn: 267753
2016-04-27 18:35:02 +00:00
Mehdi Amini c7b950171d Revert "Support "preserving" the summary information when using setModule() API in LTOCodeGenerator"
This reverts commit r267665.
ASAN shows that there is a use of undefined value.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 267668
2016-04-27 05:11:44 +00:00
Mehdi Amini 360ed847bc Support "preserving" the summary information when using setModule() API in LTOCodeGenerator
Another attempt at r267655...

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 267665
2016-04-27 04:24:10 +00:00
Mehdi Amini a1b8b6cd56 Revert "Support "preserving" the summary information when using setModule() API in LTOCodeGenerator"
This reverts commit r267657, r267656, and r267655.
The test does not pass on multiple bots, I'm unsure why yet but let's unbreak them.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 267664
2016-04-27 03:34:28 +00:00
Mehdi Amini e2a65fe5ec Support "preserving" the summary information when using setModule() API in LTOCodeGenerator
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 267655
2016-04-27 01:46:48 +00:00
Mehdi Amini da168fbc2e LTOCodeGenerator: turns linkonce(_odr) into weak_(odr) when present "MustPreserve" set
Summary:
If the linker requested to preserve a linkonce function, we should
honor this even if we drop all uses.

Reviewers: dexonsmith

Subscribers: llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D19527

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 267644
2016-04-27 00:32:02 +00:00
Justin Bogner 4563a06cee PM: Port Internalize to the new pass manager
llvm-svn: 267596
2016-04-26 20:15:52 +00:00
Mehdi Amini aa309b1a81 ThinLTOCodeGenerator: preserve linkonce when in "MustPreserved" set
If the linker specifically requested for a linkonce to be preserved,
we need to make sure we won't drop it even if all the uses in the
current module disappear.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 267543
2016-04-26 10:35:01 +00:00
Justin Bogner 1a07501379 PM: Port GlobalOpt to the new pass manager
llvm-svn: 267499
2016-04-26 00:28:01 +00:00
Teresa Johnson c851d216e2 [ThinLTO] Introduce typedef for commonly-used map type (NFC)
Add a typedef for the std::map<GlobalValue::GUID, GlobalValueSummary *>
map that is passed around to identify summaries for values defined in a
particular module. This shortens up declarations in a variety of places.

llvm-svn: 267471
2016-04-25 21:09:51 +00:00
Teresa Johnson 28e457bccd [ThinLTO] Remove GlobalValueInfo class from index
Summary:
Remove the GlobalValueInfo and change the ModuleSummaryIndex to directly
reference summary objects. The info structure was there to support lazy
parsing of the combined index summary objects, which is no longer
needed and not supported.

Reviewers: joker.eph

Subscribers: joker.eph, llvm-commits

Differential Revision: http://reviews.llvm.org/D19462

llvm-svn: 267344
2016-04-24 14:57:11 +00:00
Mehdi Amini 059464fe36 Add an internalization step to the ThinLTOCodeGenerator
Keeping as much as possible internal/private is
known to help the optimizer. Let's try to benefit from
this in ThinLTO.
Note: this is early work, but is enough to build clang (and
all the LLVM tools). I still need to write some lit-tests...

Differential Revision: http://reviews.llvm.org/D19103

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 267317
2016-04-24 03:18:01 +00:00
Justin Bogner 395c2127ed PM: Port DCE to the new pass manager
Also add a very basic test, since apparently there aren't any tests
for DCE whatsoever to add the new pass version to.

llvm-svn: 267196
2016-04-22 19:40:41 +00:00
Teresa Johnson 6fb3f19959 [ThinLTO] Remove unused/incomplete lazy summary reading support (NFC)
This removes the interfaces added (and not yet complete) to support
lazy reading of summaries. This support is not expected to be needed
since we are moving to a model where the full index is only being
traversed in the thin link step, instead of the back ends.

(The second part of this that I plan to do next is remove the
GlobalValueInfo from the ModuleSummaryIndex - it was mostly needed to
support lazy parsing of summaries. The index can instead reference the
summary structures directly.)

llvm-svn: 267097
2016-04-22 01:52:00 +00:00
Mehdi Amini f95f77adf6 ThinLTO: add module caching handling.
Differential Revision: http://reviews.llvm.org/D18494

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266959
2016-04-21 05:54:23 +00:00
Mehdi Amini a71a5a6289 ThinLTO: Resolve linkonce_odr aliases just like functions
This help to streamline the process of handling importing since
we don't need to special case alias everywhere: just like
linkonce_odr function, make sure at least one alias is emitted
by turning it weak.

Differential Revision: http://reviews.llvm.org/D19308

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266958
2016-04-21 05:47:17 +00:00
Duncan P. N. Exon Smith b011ad7330 LTO: Verify the input even if optimize() isn't called
Clients may call writeMergedModules before calling optimize, or call
compileOptimized without calling optimize.  Make sure they don't sneak
past the verifier.  This adds LTOCodeGenerator::verifyMergedModuleOnce,
and calls it from writeMergedModule, optimize, and codegenOptimized.

I couldn't find a good way to test this.  I tried writing broken IR to
send into llvm-lto, but LTOCodeGenerator doesn't understand textual IR,
and assembler runs the verifier itself anyway.  Checking in
valid-but-doesn't-verify bitcode here doesn't seem valuable.

llvm-svn: 266894
2016-04-20 17:48:22 +00:00
Teresa Johnson 07c7e25718 Enable ODR uniquing of DITypes in more places
Summary:
This is a follow-on to apply Duncan's new DIType ODR uniquing from
r266549 and r266713 in more places.

Enable enableDebugTypeODRUniquing() for ThinLTO backends invoked via
libLTO, similar to the way r266549 enabled this for ThinLTO backend
threads launched from gold-plugin.

Also enable enableDebugTypeODRUniquing in opt, similar to the way
r266549 enabled this for llvm-link (on by default, can be disabled with
new -disable-debug-info-type-map option), since we may perform ThinLTO
importing from opt.

Reviewers: dexonsmith, joker.eph

Subscribers: joker.eph, llvm-commits

Differential Revision: http://reviews.llvm.org/D19263

llvm-svn: 266746
2016-04-19 15:48:30 +00:00
Duncan P. N. Exon Smith ed8fdb2a0e IR: Rename API for enabling ODR uniquing of DITypes, NFC
As per David's review, rename everything in the new API for ODR type
uniquing of debug info.

    ensureDITypeMap  => enableDebugTypeODRUniquing
    destroyDITypeMap => disableDebugTypeODRUniquing
    hasDITypeMap     => isODRUniquingDebugTypes

llvm-svn: 266713
2016-04-19 04:55:25 +00:00
Mehdi Amini b550cb1750 [NFC] Header cleanup
Removed some unused headers, replaced some headers with forward class declarations.

Found using simple scripts like this one:
clear && ack --cpp -l '#include "llvm/ADT/IndexedMap.h"' | xargs grep -L 'IndexedMap[<]' | xargs grep -n --color=auto 'IndexedMap'

Patch by Eugene Kosov <claprix@yandex.ru>

Differential Revision: http://reviews.llvm.org/D19219

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266595
2016-04-18 09:17:29 +00:00
Rafael Espindola 3c1c9875b9 Keep only the splitCodegen version that takes a factory.
This makes it much easier to see that all created TargetMachines are
equivalent.

llvm-svn: 266564
2016-04-17 18:42:27 +00:00
Duncan P. N. Exon Smith 5ab2be094e IR: Use an explicit map for debug info type uniquing
Rather than relying on the structural equivalence of DICompositeType to
merge type definitions, use an explicit map on the LLVMContext that
LLParser and BitcodeReader consult when constructing new nodes.
Each non-forward-declaration DICompositeType with a non-empty
'identifier:' field is stored/loaded from the type map, and the first
definiton will "win".

This map is opt-in: clients that expect ODR types from different modules
to be merged must call LLVMContext::ensureDITypeMap.

  - Clients that just happen to load more than one Module in the same
    LLVMContext won't magically merge types.

  - Clients (like LTO) that want to continue to merge types based on ODR
    identifiers should opt-in immediately.

I have updated LTOCodeGenerator.cpp, the two "linking" spots in
gold-plugin.cpp, and llvm-link (unless -disable-debug-info-type-map) to
set this.

With this in place, it will be straightforward to remove the DITypeRef
concept (i.e., referencing types by their 'identifier:' string rather
than pointing at them directly).

llvm-svn: 266549
2016-04-17 03:58:21 +00:00
Mehdi Amini 1d30fcaccf Add SVN version to libLLVMLTO
Summary: For Incremental LTO, we need to make sure that an old
cache entry is not used when incrementally re-linking with a new
libLTO.
Adding a global LLVM_REVISION in llvm-config.h would for to
rebuild/relink the world for every "git pull"/"svn update".
So instead only libLTO is made dependent on the VCS and will
be rebuilt (and the dependent binaries relinked, i.e. as of
today: libLTO.dylib and llvm-lto).

Reviewers: beanz

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D18987

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266523
2016-04-16 07:33:14 +00:00
Mehdi Amini 1aafabf752 ThinLTO: Move the ODR resolution to be based purely on the summary.
This is a requirement for the cache handling in D18494

Differential Revision: http://reviews.llvm.org/D18908

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266519
2016-04-16 07:02:16 +00:00
Mehdi Amini 8dcc8080ce ThinLTO: linkonce compile-time optimization, do not bother when there is only one input file
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266281
2016-04-14 08:46:22 +00:00
Mehdi Amini ce23e9702e Simplify LTOInternalize into UpdateLLVMCompilerUsed
It is now only doing the update to the llvm.compiler_used global.
The client has to call separately the internalization stage.
Hopefully the code is simpler to understand this way.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266174
2016-04-13 06:32:46 +00:00
Mehdi Amini 105938302a Minor cleanup in Internalize, hide helper class using anonymous namespace (NFC)
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266173
2016-04-13 06:32:29 +00:00
Mehdi Amini 16fcb418ad LTOInternalize: Use a StringSet instead of a sorted vector and a binary search query for each function
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266172
2016-04-13 06:32:04 +00:00
Mehdi Amini deee003a58 Move "ExternalSymbols" out of LTOInternalize (NFC)
This is not really related to internalization per se.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266170
2016-04-13 05:36:06 +00:00
Mehdi Amini 24d3414f06 Refactor the InternalizePass into a helper class, and expose it through a public free function (NFC)
There is really no reason to require to instanciate a pass manager to
internalize.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266167
2016-04-13 05:25:08 +00:00
Mehdi Amini 4078709957 Refactor Internalization pass to use as a callback instead of a StringSet (NFC)
This will save a bunch of copies / initialization of intermediate
datastructure, and (hopefully) simplify the code.

This also abstract the symbol preservation mechanism outside of the
Internalization pass into the client code, which is not forced
to keep a map of strings for instance (ThinLTO will prefere hashes).

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266163
2016-04-13 04:20:32 +00:00
Mehdi Amini 3dfc952e83 LTOInternalize: Fix member type, should be a reference and not a copy
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266153
2016-04-12 23:58:30 +00:00
Stephan Bergmann a6ced02a67 Avoid GCC -fpermissive error about llvm::Mangler hidden by member named Mangler
llvm-svn: 266049
2016-04-12 08:23:44 +00:00
Mehdi Amini f59f2bb1b5 Refactor the Internalize stage of libLTO in a separate file (NFC)
This is intended to be shared by the ThinLTOCodeGenerator.

Note that there is a change in the way the verifier is run, previously
it was ran as a Pass on the merged module during internalization.
While now the verifier is called explicitely on the merged module
outside of the internalize "pass pipeline".

What remains strange in the API is the fact that `DisableVerify` in
the API does not disable this initial verifier.

Differential Revision: http://reviews.llvm.org/D19000

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266047
2016-04-12 06:34:10 +00:00
Teresa Johnson 2d5487cf44 [ThinLTO] Move summary computation from BitcodeWriter to new pass
Summary:
This is the first step in also serializing the index out to LLVM
assembly.

The per-module summary written to bitcode is moved out of the bitcode
writer and to a new analysis pass (ModuleSummaryIndexWrapperPass).
The pass itself uses a new builder class to compute index, and the
builder class is used directly in places where we don't have a pass
manager (e.g. llvm-as).

Because we are computing summaries outside of the bitcode writer, we no
longer can use value ids created by the bitcode writer's
ValueEnumerator. This required changing the reference graph edge type
to use a new ValueInfo class holding a union between a GUID (combined
index) and Value* (permodule index). The Value* are converted to the
appropriate value ID during bitcode writing.

Also, this enables removal of the BitWriter library's dependence on the
Analysis library that was previously required for the summary computation.

Reviewers: joker.eph

Subscribers: joker.eph, llvm-commits

Differential Revision: http://reviews.llvm.org/D18763

llvm-svn: 265941
2016-04-11 13:58:45 +00:00
Evgeniy Stepanov 268826a287 [gold] Save bitcode for module partitions (save-temps + split codegen).
llvm-svn: 265583
2016-04-06 18:32:13 +00:00
Teresa Johnson 3c35e0999b Clean up calls to WriteBitcodeToFile (NFC)
Remove a default parameter value being passed unnecessarily, which
also reduces the changes required when this parameter is changed in
D18763.

Document the remaining non-default bool value passed for another
parameter.

llvm-svn: 265348
2016-04-04 21:19:31 +00:00
Mehdi Amini 024a79f780 Revert "ThinLTO: add module caching handling."
This reverts commit r265214, unintentionally commited.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 265216
2016-04-02 05:08:18 +00:00
Mehdi Amini 2cd609482d ThinLTO: add module caching handling.
Reviewers: tejohnson

Subscribers: llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D18494

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 265214
2016-04-02 05:07:08 +00:00
Hans Wennborg fa6e414eef Fix -Wpedantic warning about extra semi-colon
llvm-svn: 265204
2016-04-02 01:03:41 +00:00
Mehdi Amini 5a2e5d324e ThinLTO: special handling for LinkOnce functions
These function can be dropped by the compiler if they are no longer
referenced in the current module. However there is a change that
another module is still referencing them because of the import.

Multiple solutions can be used:

- Always import LinkOnce when a caller is imported. This ensure that
  every module with a call to a LinkOnce has the definition and will
  be able to emit it if it emits the call.
- Turn the LinkOnce into Weak, so that it is always emitted.
- Turn all LinkOnce into available_externally and come back after all
  modules are codegen'ed to emit only one copy of the linkonce, when
  there is still a reference to it.

This patch implement the second option, with am optimization that
only *one* module will turn the LinkOnce into Weak, while the others
will turn it into available_externally, so that there is exactly one
copy emitted for the whole compilation.

http://reviews.llvm.org/D18346

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 265190
2016-04-01 21:53:50 +00:00
Mehdi Amini 215d59e7b0 ThinLTO: move ObjCARCContractPass in the CodeGen pipeline
This is to be coherent with Full LTO.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 265118
2016-04-01 08:22:59 +00:00
Mehdi Amini 43b657b5c7 Add a libLTO API to stop/restart ThinLTO between optimizations and CodeGen
This allows the linker to instruct ThinLTO to perform only the
optimization part or only the codegen part of the process.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 265113
2016-04-01 06:47:02 +00:00
Mehdi Amini 01e321306b ThinLTO: use the callgraph from the combined index to drive the FunctionImporter
Summary:
Now that the summary contains the full reference/call graph, we can
replace the existing function importer that loads and inspect the IR
to iteratively walk the call graph by a traversal based purely on the
summary information. Decouple the actual importing decision from any
IR manipulation.

Reviewers: tejohnson

Subscribers: llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D18343

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 264503
2016-03-26 05:40:34 +00:00
Teresa Johnson 26ab5772b0 [ThinLTO] Renaming of function index to module summary index (NFC)
(Resubmitting after fixing missing file issue)

With the changes in r263275, there are now more than just functions in
the summary. Completed the renaming of data structures (started in
r263275) to reflect the wider scope. In particular, changed the
FunctionIndex* data structures to ModuleIndex*, and renamed related
variables and comments. Also renamed the files to reflect the changes.

A companion clang patch will immediately succeed this patch to reflect
this renaming.

llvm-svn: 263513
2016-03-15 00:04:37 +00:00
Teresa Johnson cec0cae313 Revert "[ThinLTO] Renaming of function index to module summary index (NFC)"
This reverts commit r263490. Missed a file.

llvm-svn: 263493
2016-03-14 21:18:10 +00:00
Teresa Johnson 892920b358 [ThinLTO] Renaming of function index to module summary index (NFC)
With the changes in r263275, there are now more than just functions in
the summary. Completed the renaming of data structures (started in
r263275) to reflect the wider scope. In particular, changed the
FunctionIndex* data structures to ModuleIndex*, and renamed related
variables and comments. Also renamed the files to reflect the changes.

A companion clang patch will immediately succeed this patch to reflect
this renaming.

llvm-svn: 263490
2016-03-14 21:05:56 +00:00
Teresa Johnson 76a1c1d0ba [ThinLTO] Support for reference graph in per-module and combined summary.
Summary:
This patch adds support for including a full reference graph including
call graph edges and other GV references in the summary.

The reference graph edges can be used to make importing decisions
without materializing any source modules, can be used in the plugin
to make file staging decisions for distributed build systems, and is
expected to have other uses.

The call graph edges are recorded in each function summary in the
bitcode via a list of <CalleeValueIds, StaticCount> tuples when no PGO
data exists, or <CalleeValueId, StaticCount, ProfileCount> pairs when
there is PGO, where the ValueId can be mapped to the function GUID via
the ValueSymbolTable. In the function index in memory, the call graph
edges reference the target via the CalleeGUID instead of the
CalleeValueId.

The reference graph edges are recorded in each summary record with a
list of referenced value IDs, which can be mapped to value GUID via the
ValueSymbolTable.

Addtionally, a new summary record type is added to record references
from global variable initializers. A number of bitcode records and data
structures have been renamed to reflect the newly expanded scope of the
summary beyond functions. More cleanup will follow.

Reviewers: joker.eph, davidxl

Subscribers: joker.eph, llvm-commits

Differential Revision: http://reviews.llvm.org/D17212

llvm-svn: 263275
2016-03-11 18:52:24 +00:00
Chandler Carruth 89c45a162f [PM] Port GVN to the new pass manager, wire it up, and teach a couple of
tests to run GVN in both modes.

This is mostly the boring refactoring just like SROA and other complex
transformation passes. There is some trickiness in that GVN's
ValueNumber class requires hand holding to get to compile cleanly. I'm
open to suggestions about a better pattern there, but I tried several
before settling on this. I was trying to balance my desire to sink as
much implementation detail into the source file as possible without
introducing overly many layers of abstraction.

Much like with SROA, the design of this system is made somewhat more
cumbersome by the need to support both pass managers without duplicating
the significant state and logic of the pass. The same compromise is
struck here.

I've also left a FIXME in a doxygen comment as the GVN pass seems to
have pretty woeful documentation within it. I'd like to submit this with
the FIXME and let those more deeply familiar backfill the information
here now that we have a nice place in an interface to put that kind of
documentaiton.

Differential Revision: http://reviews.llvm.org/D18019

llvm-svn: 263208
2016-03-11 08:50:55 +00:00