Commit Graph

510 Commits

Author SHA1 Message Date
Davide Italiano 41315f7873 [PM] Revert the port of MergeLoadStoreMotion to the new pass manager.
Daniel Berlin expressed some real concerns about the port and proposed
and alternative approach. I'll revert this for now while working on a
new patch, which I hope to put up for review shortly. Sorry for the churn.

llvm-svn: 272925
2016-06-16 17:40:53 +00:00
David Majnemer cbf614a93b Remove the ScalarReplAggregates pass
Nearly all the changes to this pass have been done while maintaining and
updating other parts of LLVM.  LLVM has had another pass, SROA, which
has superseded ScalarReplAggregates for quite some time.

Differential Revision: http://reviews.llvm.org/D21316

llvm-svn: 272737
2016-06-15 00:19:09 +00:00
Sean Silva 6347df0f81 [PM] Port MemCpyOpt to the new PM.
The need for all these Lookup* functions is just because of calls to
getAnalysis inside methods (i.e. not at the top level) of the
runOnFunction method. They should be straightforward to clean up when
the old PM is gone.

llvm-svn: 272615
2016-06-14 02:44:55 +00:00
Davide Italiano 89ab89d6cd [PM] Port MergedLoadStoreMotion to the new pass manager.
llvm-svn: 272606
2016-06-14 00:49:23 +00:00
Benjamin Kramer d3f4c05aea Move instances of std::function.
Or replace with llvm::function_ref if it's never stored. NFC intended.

llvm-svn: 272513
2016-06-12 16:13:55 +00:00
Sean Silva f5080194fd [PM] Port ReversePostOrderFunctionAttrs to the new PM
Below are my super rough notes when porting. They can probably serve as
a basic guide for porting other passes to the new PM. As I port more
passes I'll expand and generalize this and make a proper
docs/HowToPortToNewPassManager.rst document. There is also missing
documentation for general concepts and API's in the new PM which will
require some documentation.
Once there is proper documentation in place we can put up a list of
passes that have to be ported and game-ify/crowdsource the rest of the
porting (at least of the middle end; the backend is still unclear).

I will however be taking personal responsibility for ensuring that the
LLD/ELF LTO pipeline is ported in a timely fashion. The remaining passes
to be ported are (do something like
`git grep "<the string in the bullet point below>"` to find the pass):

General Scalar:
[ ] Simplify the CFG
[ ] Jump Threading
[ ] MemCpy Optimization
[ ] Promote Memory to Register
[ ] MergedLoadStoreMotion
[ ] Lazy Value Information Analysis

General IPO:
[ ] Dead Argument Elimination
[ ] Deduce function attributes in RPO

Loop stuff / vectorization stuff:
[ ] Alignment from assumptions
[ ] Canonicalize natural loops
[ ] Delete dead loops
[ ] Loop Access Analysis
[ ] Loop Invariant Code Motion
[ ] Loop Vectorization
[ ] SLP Vectorizer
[ ] Unroll loops

Devirtualization / CFI:
[ ] Cross-DSO CFI
[ ] Whole program devirtualization
[ ] Lower bitset metadata

CGSCC passes:
[ ] Function Integration/Inlining
[ ] Remove unused exception handling info
[ ] Promote 'by reference' arguments to scalars

Please let me know if you are interested in working on any of the passes
in the above list (e.g. reply to the post-commit thread for this patch).
I'll probably be tackling "General Scalar" and "General IPO" first FWIW.

Steps as I port "Deduce function attributes in RPO"
---------------------------------------------------

(note: if you are doing any work based on these notes, please leave a
note in the post-commit review thread for this commit with any
improvements / suggestions / incompleteness you ran into!)

Note: "Deduce function attributes in RPO" is a module pass.

1. Do preparatory refactoring.

Do preparatory factoring. In this case all I had to do was to pull out a static helper (r272503).
(TODO: give more advice here e.g. if pass holds state or something)

2. Rename the old pass class.

llvm/lib/Transforms/IPO/FunctionAttrs.cpp
Rename class ReversePostOrderFunctionAttrs -> ReversePostOrderFunctionAttrsLegacyPass
in preparation for adding a class ReversePostOrderFunctionAttrs as the pass in the new PM.
(edit: actually wait what? The new class name will be
ReversePostOrderFunctionAttrsPass, so it doesn't conflict. So this step is
sort of useless churn).

llvm/include/llvm/InitializePasses.h
llvm/lib/LTO/LTOCodeGenerator.cpp
llvm/lib/Transforms/IPO/IPO.cpp
llvm/lib/Transforms/IPO/FunctionAttrs.cpp
Rename initializeReversePostOrderFunctionAttrsPass -> initializeReversePostOrderFunctionAttrsLegacyPassPass
(note that the "PassPass" thing falls out of `s/ReversePostOrderFunctionAttrs/ReversePostOrderFunctionAttrsLegacyPass/`)
Note that the INITIALIZE_PASS macro is what creates this identifier name, so renaming the class requires this renaming too.

Note that createReversePostOrderFunctionAttrsPass does not need to be
renamed since its name is not generated from the class name.

3. Add the new PM pass class.

In the new PM all passes need to have their
declaration in a header somewhere, so you will often need to add a header.
In this case
llvm/include/llvm/Transforms/IPO/FunctionAttrs.h is already there because
PostOrderFunctionAttrsPass was already ported.
The file-level comment from the .cpp file can be used as the file-level
comment for the new header. You may want to tweak the wording slightly
from "this file implements" to "this file provides" or similar.

Add declaration for the new PM pass in this header:

    class ReversePostOrderFunctionAttrsPass
        : public PassInfoMixin<ReversePostOrderFunctionAttrsPass> {
    public:
      PreservedAnalyses run(Module &M, AnalysisManager<Module> &AM);
    };

Its name should end with `Pass` for consistency (note that this doesn't
collide with the names of most old PM passes). E.g. call it
`<name of the old PM pass>Pass`.

Also, move the doxygen comment from the old PM pass to the declaration of
this class in the header.
Also, include the declaration for the new PM class
`llvm/Transforms/IPO/FunctionAttrs.h` at the top of the file (in this case,
it was already done when the other pass in this file was ported).

Now define the `run` method for the new class.
The main things here are:
a) Use AM.getResult<...>(M) to get results instead of `getAnalysis<...>()`

b) If the old PM pass would have returned "false" (i.e. `Changed ==
false`), then you should return PreservedAnalyses::all();

c) In the old PM getAnalysisUsage method, observe the calls
   `AU.addPreserved<...>();`.

   In the case `Changed == true`, for each preserved analysis you should do
   call `PA.preserve<...>()` on a PreservedAnalyses object and return it.
   E.g.:

       PreservedAnalyses PA;
       PA.preserve<CallGraphAnalysis>();
       return PA;

Note that calls to skipModule/skipFunction are not supported in the new PM
currently, so optnone and optimization bisect support do not work. You can
just drop those calls for now.

4. Add the pass to the new PM pass registry to make it available in opt.

In llvm/lib/Passes/PassBuilder.cpp add a #include for your header.
`#include "llvm/Transforms/IPO/FunctionAttrs.h"`
In this case there is already an include (from when
PostOrderFunctionAttrsPass was ported).

Add your pass to llvm/lib/Passes/PassRegistry.def
In this case, I added
`MODULE_PASS("rpo-functionattrs", ReversePostOrderFunctionAttrsPass())`
The string is from the `INITIALIZE_PASS*` macros used in the old pass
manager.

Then choose a test that uses the pass and use the new PM `-passes=...` to
run it.
E.g. in this case there is a test that does:
; RUN: opt < %s -basicaa -functionattrs -rpo-functionattrs -S | FileCheck %s
I have added the line:
; RUN: opt < %s -aa-pipeline=basic-aa -passes='require<targetlibinfo>,cgscc(function-attrs),rpo-functionattrs' -S | FileCheck %s
The `-aa-pipeline=basic-aa` and
`require<targetlibinfo>,cgscc(function-attrs)` are what is needed to run
functionattrs in the new PM (note that in the new PM "functionattrs"
becomes "function-attrs" for some reason). This is just pulled from
`readattrs.ll` which contains the change from when functionattrs was ported
to the new PM.
Adding rpo-functionattrs causes the pass that was just ported to run.

llvm-svn: 272505
2016-06-12 07:48:51 +00:00
Benjamin Kramer c321e53402 Apply most suggestions of clang-tidy's performance-unnecessary-value-param
Avoids unnecessary copies. All changes audited & pass tests with asan.
No functional change intended.

llvm-svn: 272190
2016-06-08 19:09:22 +00:00
Teresa Johnson 28c03b56ec [ThinLTO] Resolve LinkOnceAny
Summary:
Ensure we keep prevailing copy of LinkOnceAny by converting it to
WeakAny.

Rename odr_resolution test to the now more appropriate weak_resolution
(weak in the linker sense includes linkonce).

Reviewers: joker.eph

Subscribers: llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D20634

llvm-svn: 270850
2016-05-26 14:16:52 +00:00
Teresa Johnson 04c9a2d63d [ThinLTO] Refactor ODR resolution and internalization (NFC)
Move the now index-based ODR resolution and internalization routines out
of ThinLTOCodeGenerator.cpp and into either LTO.cpp (index-based
analysis) or FunctionImport.cpp (index-driven optimizations).
This is to enable usage by other linkers.

llvm-svn: 270698
2016-05-25 14:03:11 +00:00
Teresa Johnson 4ae5ce7e8e [ThinLTO] Handle empty exports list for module.
We might have an empty exports list for a module and shouldn't assert
when checking if a symbol is exported.

llvm-svn: 270600
2016-05-24 19:12:48 +00:00
Teresa Johnson 141149ff48 [ThinLTO] Fix bot failures by avoiding threaded map modifications
Ensure multiple threads won't have to modify map, probable cause of
bot failures from r270584 such as:
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/13095

llvm-svn: 270595
2016-05-24 18:44:01 +00:00
Teresa Johnson 4d2613f276 [ThinLTO] Change ODR resolution and internalization to be index-based
Summary:
This patch changes the ODR resolution and internalization to be based on
updates to the Index, which are consumed by the backend portion of the
transformations.

It will be followed by an NFC change to move these out of libLTO's
ThinLTOCodeGenerator so that it can be used by other linkers
(gold and lld) and by ThinLTO distributed backends.

The global summary-based portions use callbacks so that the client can
determine the prevailing copy and other information in a client-specific
way. Eventually, with the API being developed in D20268, these may be
modified to use information such as symbol resolutions, supplied by the
clients to the API.

Reviewers: joker-eph

Subscribers: joker.eph, pcc, llvm-commits

Differential Revision: http://reviews.llvm.org/D20290

llvm-svn: 270584
2016-05-24 17:24:25 +00:00
Teresa Johnson df6edc5277 [ThinLTO] Refactor module loader handling into new LTO file (NFC)
Moved the ModuleLoader and supporting helper loadModuleFromBuffer out of
ThinLTOCodeGenerator and into new LTO.h/LTO.cpp files. This is in
preparation for a patch that will utilize these in the gold-plugin.

Note that there are some other pending patches (D20268 and D20290) that
also plan to refactor common interfaces and functionality into this same
pair of new files.

llvm-svn: 270509
2016-05-23 22:54:06 +00:00
Rafael Espindola 8c34dd8257 Delete Reloc::Default.
Having an enum member named Default is quite confusing: Is it distinct
from the others?

This patch removes that member and instead uses Optional<Reloc> in
places where we have a user input that still hasn't been maped to the
default value, which is now clear has no be one of the remaining 3
options.

llvm-svn: 269988
2016-05-18 22:04:49 +00:00
Mehdi Amini 819e9cdfb4 ThinLTO: sort inputs and schedule by decreasing size
This is a compile time optimization: keeping a large file to process
at the end hurts parallelism.
The heurisitic used right now is the input buffer size, however we
may want to consider the number of functions to import or the
different number of files to load for importing as well.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 269684
2016-05-16 19:33:07 +00:00
Mehdi Amini 001bb41556 ThinLTO caching: reload cached file with mmap and drop heap-allocated memory buffer
This is reducing pressure on the OS memory system, and is NFC
when not using a cache.
I measure a 10x memory consumption reduction when linking opt
with full debug info.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 269682
2016-05-16 19:11:59 +00:00
Mehdi Amini af52f28d9d ThinLTOCodeGenerator: fix race condition found using TSAN
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 269588
2016-05-15 05:49:47 +00:00
Mehdi Amini b02139d66f ThinLTOCodeGenerator: handle cases where temporary files can't be renamed
For instance when they're on different filesystem.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 269544
2016-05-14 05:16:35 +00:00
Mehdi Amini 2a16a5fae5 ThinLTOCodeGenerator: handle std::error_code instead of silently dropping it.
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 269541
2016-05-14 04:58:38 +00:00
Teresa Johnson 8570fe47ef [ThinLTO] Add option to emit imports files for distributed backends
Summary:
Add support for emission of plaintext lists of the imported files for
each distributed backend compilation. Used for distributed build file
staging.

Invoked with new gold-plugin thinlto-emit-imports-files option, which is
only valid with thinlto-index-only (i.e. for distributed builds), or
from llvm-lto with new -thinlto-action=emitimports value.

Depends on D19556.

Reviewers: joker.eph

Subscribers: llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D19636

llvm-svn: 269067
2016-05-10 15:54:09 +00:00
Teresa Johnson 84174c3771 Restore "[ThinLTO] Emit individual index files for distributed backends"
This restores commit r268627:
    Summary:
    When launching ThinLTO backends in a distributed build (currently
    supported in gold via the thinlto-index-only plugin option), emit
    an individual index file for each backend process as described here:
    http://lists.llvm.org/pipermail/llvm-dev/2016-April/098272.html

    ...

    Differential Revision: http://reviews.llvm.org/D19556

Address msan failures by avoiding std::prev on map.end(), the
theory is that this is causing issues due to some known UB problems
in __tree.

llvm-svn: 269059
2016-05-10 13:48:23 +00:00
Adrian Prantl fe7a382453 Allow the LTO code generator to strip invalid debug info from the input.
This patch introduces a new option -lto-strip-invalid-debug-info, which
drops malformed debug info from the input.

The problem I'm trying to solve with this sequence of patches is that
historically we've done a really bad job at verifying debug info. We want
to be able to make the verifier stricter without having to worry about
breaking bitcode compatibility with existing producers. For example, we
don't necessarily want IR produced by an older version of clang to be
rejected by an LTO link just because of malformed debug info, and rather
provide an option to strip it. Note that merely outdated (but well-formed)
debug info would continue to be auto-upgraded in this scenario.

rdar://problem/25818489
http://reviews.llvm.org/D19987

This reapplies 268936 with a test case fix for Linux (-exported-symbol foo)

llvm-svn: 268965
2016-05-09 19:57:15 +00:00
Adrian Prantl 6d80100c6a Revert "Allow the LTO code generator to strip invalid debug info from the input."
This reverts commit 268936 while investigating buildbot breakage.

llvm-svn: 268940
2016-05-09 17:43:30 +00:00
Adrian Prantl 4a9292b127 Allow the LTO code generator to strip invalid debug info from the input.
This patch introduces a new option -lto-strip-invalid-debug-info, which
drops malformed debug info from the input.

The problem I'm trying to solve with this sequence of patches is that
historically we've done a really bad job at verifying debug info. We want
to be able to make the verifier stricter without having to worry about
breaking bitcode compatibility with existing producers. For example, we
don't necessarily want IR produced by an older version of clang to be
rejected by an LTO link just because of malformed debug info, and rather
provide an option to strip it. Note that merely outdated (but well-formed)
debug info would continue to be auto-upgraded in this scenario.

rdar://problem/25818489
http://reviews.llvm.org/D19987

llvm-svn: 268936
2016-05-09 17:37:33 +00:00
Davide Italiano f54f2f0893 [PM] Port Interprocedural SCCP to the new pass manager.
llvm-svn: 268684
2016-05-05 21:05:36 +00:00
Mehdi Amini 5eba657ff3 Revert "LTOCodeGenerator: handle correctly "unnamed" symbol"
This reverts commit r268658.

I incorrectly diagnose this as the source of an assertion during an
LTO bootstrap of clang.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 268680
2016-05-05 20:44:21 +00:00
Mehdi Amini 8eb3397a73 Degrade assertions to a warning in LTOCodeGenerator for preserved linkonce
The assertions were assuming that the linker will not ask to preserve
a global that is internal or available_externally, as it does not
really make sense. In practice this break the bootstrap of clang,
I degrade to a warning for now.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 268671
2016-05-05 20:05:33 +00:00
Vitaly Buka 1df2338bb6 Revert "[ThinLTO] Emit individual index files for distributed backends"
MemorySanitizer: use-of-uninitialized-value in lib/Bitcode/Writer/BitcodeWriter.cpp:364:70
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/12544/steps/check-llvm%20msan/logs/stdio

This reverts commit 0c4a898ea550699d1b2f4fe3767251c8f9a48d52.

llvm-svn: 268660
2016-05-05 18:31:00 +00:00
Mehdi Amini 2ee1874aa8 LTOCodeGenerator: handle correctly "unnamed" symbol
This should fix the assertions in a clang LTO bootstrap we're seeing.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 268658
2016-05-05 18:14:55 +00:00
Teresa Johnson 9254ebe3c0 [ThinLTO] Emit individual index files for distributed backends
Summary:
When launching ThinLTO backends in a distributed build (currently
supported in gold via the thinlto-index-only plugin option), emit
an individual index file for each backend process as described here:
http://lists.llvm.org/pipermail/llvm-dev/2016-April/098272.html

The individual index file encodes the summary and module information
required for implementing the importing/exporting decisions made
for a given module in the thin link step.
This is in place of the current mechanism that uses the combined index
to make importing decisions in each back end independently. It is an
enabler for doing global summary based optimizations in the thin link
step (which will be recorded in the individual index files), and reduces
the size of the index that must be sent to each backend process, and
the amount of work to scan it in the backends.

Rather than create entirely new ModuleSummaryIndex structures (and all
the included unique_ptrs) for each backend index file, a map is created
to record all of the GUID and summary pointers needed for a particular
index file. The IndexBitcodeWriter walks this map instead of the full
index (hiding the details of managing the appropriate summary iteration
in a new iterator subclass). This is more efficient than walking the
entire combined index and filtering out just the needed summaries during
each backend bitcode index write.

Depends on D19481.

Reviewers: joker.eph

Subscribers: llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D19556

llvm-svn: 268627
2016-05-05 13:44:56 +00:00
Mehdi Amini 022b5bcb7a LTOCodeGenerator: add linkonce(_odr) to "llvm.compiler.used" when present in "MustPreserve" set
If the linker requested to preserve a linkonce function, we should
honor this even if we drop all uses.
We explicitely avoid turning them into weak_odr (unlike the first
version of this patch in r267644), because the codegen can be
different on Darwin: because of `llvm::canBeOmittedFromSymbolTable()`
we may emit the symbol as weak_def_can_be_hidden instead of
weak_definition.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 268607
2016-05-05 05:14:24 +00:00
Mehdi Amini 752ffe9c5f Revert "LTOCodeGenerator: turns linkonce(_odr) into weak_(odr) when present "MustPreserve" set"
This reverts commit r267644. Turning linkonce_odr into weak_odr is
a sementic change on Darwin: because of
`llvm::canBeOmittedFromSymbolTable()` we may emit the symbol as
weak_def_can_be_hidden instead of weak_definition.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 268606
2016-05-05 05:14:20 +00:00
Mehdi Amini 4b300e0ac2 ThinLTOCodeGenerator: remove useless temporary file that was emitted.
This was a remaining of a previous scheme where some IPOs were taking
place before we enter this code. This is not relevant anymore.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 268605
2016-05-05 05:14:16 +00:00
Davide Italiano 164b9bc6fe [PM] Port ConstantMerge to the new pass manager.
llvm-svn: 268582
2016-05-05 00:51:09 +00:00
Davide Italiano 66228c4cf1 [IPO/GlobalDCE] Port to the new pass manager.
Differential Revision:  http://reviews.llvm.org/D19782

llvm-svn: 268425
2016-05-03 19:39:15 +00:00
Peter Collingbourne edf8432480 LTO: Don't bother trying to mangle unnamed globals, as they can't be preserved with MustPreserveSymbols.
Summary: Should fix sanitizer-windows bot.

Reviewers: joker.eph

Subscribers: llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D19635

llvm-svn: 267820
2016-04-27 23:48:11 +00:00
Sjoerd Meijer 41beee6575 Clean up to avoid compiler warnings for casting away const qualifiers.
Differential Revision: http://reviews.llvm.org/D19598

llvm-svn: 267753
2016-04-27 18:35:02 +00:00
Mehdi Amini c7b950171d Revert "Support "preserving" the summary information when using setModule() API in LTOCodeGenerator"
This reverts commit r267665.
ASAN shows that there is a use of undefined value.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 267668
2016-04-27 05:11:44 +00:00
Mehdi Amini 360ed847bc Support "preserving" the summary information when using setModule() API in LTOCodeGenerator
Another attempt at r267655...

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 267665
2016-04-27 04:24:10 +00:00
Mehdi Amini a1b8b6cd56 Revert "Support "preserving" the summary information when using setModule() API in LTOCodeGenerator"
This reverts commit r267657, r267656, and r267655.
The test does not pass on multiple bots, I'm unsure why yet but let's unbreak them.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 267664
2016-04-27 03:34:28 +00:00
Mehdi Amini e2a65fe5ec Support "preserving" the summary information when using setModule() API in LTOCodeGenerator
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 267655
2016-04-27 01:46:48 +00:00
Mehdi Amini da168fbc2e LTOCodeGenerator: turns linkonce(_odr) into weak_(odr) when present "MustPreserve" set
Summary:
If the linker requested to preserve a linkonce function, we should
honor this even if we drop all uses.

Reviewers: dexonsmith

Subscribers: llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D19527

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 267644
2016-04-27 00:32:02 +00:00
Justin Bogner 4563a06cee PM: Port Internalize to the new pass manager
llvm-svn: 267596
2016-04-26 20:15:52 +00:00
Mehdi Amini aa309b1a81 ThinLTOCodeGenerator: preserve linkonce when in "MustPreserved" set
If the linker specifically requested for a linkonce to be preserved,
we need to make sure we won't drop it even if all the uses in the
current module disappear.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 267543
2016-04-26 10:35:01 +00:00
Justin Bogner 1a07501379 PM: Port GlobalOpt to the new pass manager
llvm-svn: 267499
2016-04-26 00:28:01 +00:00
Teresa Johnson c851d216e2 [ThinLTO] Introduce typedef for commonly-used map type (NFC)
Add a typedef for the std::map<GlobalValue::GUID, GlobalValueSummary *>
map that is passed around to identify summaries for values defined in a
particular module. This shortens up declarations in a variety of places.

llvm-svn: 267471
2016-04-25 21:09:51 +00:00
Teresa Johnson 28e457bccd [ThinLTO] Remove GlobalValueInfo class from index
Summary:
Remove the GlobalValueInfo and change the ModuleSummaryIndex to directly
reference summary objects. The info structure was there to support lazy
parsing of the combined index summary objects, which is no longer
needed and not supported.

Reviewers: joker.eph

Subscribers: joker.eph, llvm-commits

Differential Revision: http://reviews.llvm.org/D19462

llvm-svn: 267344
2016-04-24 14:57:11 +00:00
Mehdi Amini 059464fe36 Add an internalization step to the ThinLTOCodeGenerator
Keeping as much as possible internal/private is
known to help the optimizer. Let's try to benefit from
this in ThinLTO.
Note: this is early work, but is enough to build clang (and
all the LLVM tools). I still need to write some lit-tests...

Differential Revision: http://reviews.llvm.org/D19103

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 267317
2016-04-24 03:18:01 +00:00
Justin Bogner 395c2127ed PM: Port DCE to the new pass manager
Also add a very basic test, since apparently there aren't any tests
for DCE whatsoever to add the new pass version to.

llvm-svn: 267196
2016-04-22 19:40:41 +00:00
Teresa Johnson 6fb3f19959 [ThinLTO] Remove unused/incomplete lazy summary reading support (NFC)
This removes the interfaces added (and not yet complete) to support
lazy reading of summaries. This support is not expected to be needed
since we are moving to a model where the full index is only being
traversed in the thin link step, instead of the back ends.

(The second part of this that I plan to do next is remove the
GlobalValueInfo from the ModuleSummaryIndex - it was mostly needed to
support lazy parsing of summaries. The index can instead reference the
summary structures directly.)

llvm-svn: 267097
2016-04-22 01:52:00 +00:00
Mehdi Amini f95f77adf6 ThinLTO: add module caching handling.
Differential Revision: http://reviews.llvm.org/D18494

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266959
2016-04-21 05:54:23 +00:00
Mehdi Amini a71a5a6289 ThinLTO: Resolve linkonce_odr aliases just like functions
This help to streamline the process of handling importing since
we don't need to special case alias everywhere: just like
linkonce_odr function, make sure at least one alias is emitted
by turning it weak.

Differential Revision: http://reviews.llvm.org/D19308

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266958
2016-04-21 05:47:17 +00:00
Duncan P. N. Exon Smith b011ad7330 LTO: Verify the input even if optimize() isn't called
Clients may call writeMergedModules before calling optimize, or call
compileOptimized without calling optimize.  Make sure they don't sneak
past the verifier.  This adds LTOCodeGenerator::verifyMergedModuleOnce,
and calls it from writeMergedModule, optimize, and codegenOptimized.

I couldn't find a good way to test this.  I tried writing broken IR to
send into llvm-lto, but LTOCodeGenerator doesn't understand textual IR,
and assembler runs the verifier itself anyway.  Checking in
valid-but-doesn't-verify bitcode here doesn't seem valuable.

llvm-svn: 266894
2016-04-20 17:48:22 +00:00
Teresa Johnson 07c7e25718 Enable ODR uniquing of DITypes in more places
Summary:
This is a follow-on to apply Duncan's new DIType ODR uniquing from
r266549 and r266713 in more places.

Enable enableDebugTypeODRUniquing() for ThinLTO backends invoked via
libLTO, similar to the way r266549 enabled this for ThinLTO backend
threads launched from gold-plugin.

Also enable enableDebugTypeODRUniquing in opt, similar to the way
r266549 enabled this for llvm-link (on by default, can be disabled with
new -disable-debug-info-type-map option), since we may perform ThinLTO
importing from opt.

Reviewers: dexonsmith, joker.eph

Subscribers: joker.eph, llvm-commits

Differential Revision: http://reviews.llvm.org/D19263

llvm-svn: 266746
2016-04-19 15:48:30 +00:00
Duncan P. N. Exon Smith ed8fdb2a0e IR: Rename API for enabling ODR uniquing of DITypes, NFC
As per David's review, rename everything in the new API for ODR type
uniquing of debug info.

    ensureDITypeMap  => enableDebugTypeODRUniquing
    destroyDITypeMap => disableDebugTypeODRUniquing
    hasDITypeMap     => isODRUniquingDebugTypes

llvm-svn: 266713
2016-04-19 04:55:25 +00:00
Mehdi Amini b550cb1750 [NFC] Header cleanup
Removed some unused headers, replaced some headers with forward class declarations.

Found using simple scripts like this one:
clear && ack --cpp -l '#include "llvm/ADT/IndexedMap.h"' | xargs grep -L 'IndexedMap[<]' | xargs grep -n --color=auto 'IndexedMap'

Patch by Eugene Kosov <claprix@yandex.ru>

Differential Revision: http://reviews.llvm.org/D19219

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266595
2016-04-18 09:17:29 +00:00
Rafael Espindola 3c1c9875b9 Keep only the splitCodegen version that takes a factory.
This makes it much easier to see that all created TargetMachines are
equivalent.

llvm-svn: 266564
2016-04-17 18:42:27 +00:00
Duncan P. N. Exon Smith 5ab2be094e IR: Use an explicit map for debug info type uniquing
Rather than relying on the structural equivalence of DICompositeType to
merge type definitions, use an explicit map on the LLVMContext that
LLParser and BitcodeReader consult when constructing new nodes.
Each non-forward-declaration DICompositeType with a non-empty
'identifier:' field is stored/loaded from the type map, and the first
definiton will "win".

This map is opt-in: clients that expect ODR types from different modules
to be merged must call LLVMContext::ensureDITypeMap.

  - Clients that just happen to load more than one Module in the same
    LLVMContext won't magically merge types.

  - Clients (like LTO) that want to continue to merge types based on ODR
    identifiers should opt-in immediately.

I have updated LTOCodeGenerator.cpp, the two "linking" spots in
gold-plugin.cpp, and llvm-link (unless -disable-debug-info-type-map) to
set this.

With this in place, it will be straightforward to remove the DITypeRef
concept (i.e., referencing types by their 'identifier:' string rather
than pointing at them directly).

llvm-svn: 266549
2016-04-17 03:58:21 +00:00
Mehdi Amini 1d30fcaccf Add SVN version to libLLVMLTO
Summary: For Incremental LTO, we need to make sure that an old
cache entry is not used when incrementally re-linking with a new
libLTO.
Adding a global LLVM_REVISION in llvm-config.h would for to
rebuild/relink the world for every "git pull"/"svn update".
So instead only libLTO is made dependent on the VCS and will
be rebuilt (and the dependent binaries relinked, i.e. as of
today: libLTO.dylib and llvm-lto).

Reviewers: beanz

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D18987

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266523
2016-04-16 07:33:14 +00:00
Mehdi Amini 1aafabf752 ThinLTO: Move the ODR resolution to be based purely on the summary.
This is a requirement for the cache handling in D18494

Differential Revision: http://reviews.llvm.org/D18908

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266519
2016-04-16 07:02:16 +00:00
Mehdi Amini 8dcc8080ce ThinLTO: linkonce compile-time optimization, do not bother when there is only one input file
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266281
2016-04-14 08:46:22 +00:00
Mehdi Amini ce23e9702e Simplify LTOInternalize into UpdateLLVMCompilerUsed
It is now only doing the update to the llvm.compiler_used global.
The client has to call separately the internalization stage.
Hopefully the code is simpler to understand this way.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266174
2016-04-13 06:32:46 +00:00
Mehdi Amini 105938302a Minor cleanup in Internalize, hide helper class using anonymous namespace (NFC)
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266173
2016-04-13 06:32:29 +00:00
Mehdi Amini 16fcb418ad LTOInternalize: Use a StringSet instead of a sorted vector and a binary search query for each function
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266172
2016-04-13 06:32:04 +00:00
Mehdi Amini deee003a58 Move "ExternalSymbols" out of LTOInternalize (NFC)
This is not really related to internalization per se.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266170
2016-04-13 05:36:06 +00:00
Mehdi Amini 24d3414f06 Refactor the InternalizePass into a helper class, and expose it through a public free function (NFC)
There is really no reason to require to instanciate a pass manager to
internalize.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266167
2016-04-13 05:25:08 +00:00
Mehdi Amini 4078709957 Refactor Internalization pass to use as a callback instead of a StringSet (NFC)
This will save a bunch of copies / initialization of intermediate
datastructure, and (hopefully) simplify the code.

This also abstract the symbol preservation mechanism outside of the
Internalization pass into the client code, which is not forced
to keep a map of strings for instance (ThinLTO will prefere hashes).

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266163
2016-04-13 04:20:32 +00:00
Mehdi Amini 3dfc952e83 LTOInternalize: Fix member type, should be a reference and not a copy
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266153
2016-04-12 23:58:30 +00:00
Stephan Bergmann a6ced02a67 Avoid GCC -fpermissive error about llvm::Mangler hidden by member named Mangler
llvm-svn: 266049
2016-04-12 08:23:44 +00:00
Mehdi Amini f59f2bb1b5 Refactor the Internalize stage of libLTO in a separate file (NFC)
This is intended to be shared by the ThinLTOCodeGenerator.

Note that there is a change in the way the verifier is run, previously
it was ran as a Pass on the merged module during internalization.
While now the verifier is called explicitely on the merged module
outside of the internalize "pass pipeline".

What remains strange in the API is the fact that `DisableVerify` in
the API does not disable this initial verifier.

Differential Revision: http://reviews.llvm.org/D19000

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266047
2016-04-12 06:34:10 +00:00
Teresa Johnson 2d5487cf44 [ThinLTO] Move summary computation from BitcodeWriter to new pass
Summary:
This is the first step in also serializing the index out to LLVM
assembly.

The per-module summary written to bitcode is moved out of the bitcode
writer and to a new analysis pass (ModuleSummaryIndexWrapperPass).
The pass itself uses a new builder class to compute index, and the
builder class is used directly in places where we don't have a pass
manager (e.g. llvm-as).

Because we are computing summaries outside of the bitcode writer, we no
longer can use value ids created by the bitcode writer's
ValueEnumerator. This required changing the reference graph edge type
to use a new ValueInfo class holding a union between a GUID (combined
index) and Value* (permodule index). The Value* are converted to the
appropriate value ID during bitcode writing.

Also, this enables removal of the BitWriter library's dependence on the
Analysis library that was previously required for the summary computation.

Reviewers: joker.eph

Subscribers: joker.eph, llvm-commits

Differential Revision: http://reviews.llvm.org/D18763

llvm-svn: 265941
2016-04-11 13:58:45 +00:00
Evgeniy Stepanov 268826a287 [gold] Save bitcode for module partitions (save-temps + split codegen).
llvm-svn: 265583
2016-04-06 18:32:13 +00:00
Teresa Johnson 3c35e0999b Clean up calls to WriteBitcodeToFile (NFC)
Remove a default parameter value being passed unnecessarily, which
also reduces the changes required when this parameter is changed in
D18763.

Document the remaining non-default bool value passed for another
parameter.

llvm-svn: 265348
2016-04-04 21:19:31 +00:00
Mehdi Amini 024a79f780 Revert "ThinLTO: add module caching handling."
This reverts commit r265214, unintentionally commited.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 265216
2016-04-02 05:08:18 +00:00
Mehdi Amini 2cd609482d ThinLTO: add module caching handling.
Reviewers: tejohnson

Subscribers: llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D18494

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 265214
2016-04-02 05:07:08 +00:00
Hans Wennborg fa6e414eef Fix -Wpedantic warning about extra semi-colon
llvm-svn: 265204
2016-04-02 01:03:41 +00:00
Mehdi Amini 5a2e5d324e ThinLTO: special handling for LinkOnce functions
These function can be dropped by the compiler if they are no longer
referenced in the current module. However there is a change that
another module is still referencing them because of the import.

Multiple solutions can be used:

- Always import LinkOnce when a caller is imported. This ensure that
  every module with a call to a LinkOnce has the definition and will
  be able to emit it if it emits the call.
- Turn the LinkOnce into Weak, so that it is always emitted.
- Turn all LinkOnce into available_externally and come back after all
  modules are codegen'ed to emit only one copy of the linkonce, when
  there is still a reference to it.

This patch implement the second option, with am optimization that
only *one* module will turn the LinkOnce into Weak, while the others
will turn it into available_externally, so that there is exactly one
copy emitted for the whole compilation.

http://reviews.llvm.org/D18346

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 265190
2016-04-01 21:53:50 +00:00
Mehdi Amini 215d59e7b0 ThinLTO: move ObjCARCContractPass in the CodeGen pipeline
This is to be coherent with Full LTO.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 265118
2016-04-01 08:22:59 +00:00
Mehdi Amini 43b657b5c7 Add a libLTO API to stop/restart ThinLTO between optimizations and CodeGen
This allows the linker to instruct ThinLTO to perform only the
optimization part or only the codegen part of the process.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 265113
2016-04-01 06:47:02 +00:00
Mehdi Amini 01e321306b ThinLTO: use the callgraph from the combined index to drive the FunctionImporter
Summary:
Now that the summary contains the full reference/call graph, we can
replace the existing function importer that loads and inspect the IR
to iteratively walk the call graph by a traversal based purely on the
summary information. Decouple the actual importing decision from any
IR manipulation.

Reviewers: tejohnson

Subscribers: llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D18343

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 264503
2016-03-26 05:40:34 +00:00
Teresa Johnson 26ab5772b0 [ThinLTO] Renaming of function index to module summary index (NFC)
(Resubmitting after fixing missing file issue)

With the changes in r263275, there are now more than just functions in
the summary. Completed the renaming of data structures (started in
r263275) to reflect the wider scope. In particular, changed the
FunctionIndex* data structures to ModuleIndex*, and renamed related
variables and comments. Also renamed the files to reflect the changes.

A companion clang patch will immediately succeed this patch to reflect
this renaming.

llvm-svn: 263513
2016-03-15 00:04:37 +00:00
Teresa Johnson cec0cae313 Revert "[ThinLTO] Renaming of function index to module summary index (NFC)"
This reverts commit r263490. Missed a file.

llvm-svn: 263493
2016-03-14 21:18:10 +00:00
Teresa Johnson 892920b358 [ThinLTO] Renaming of function index to module summary index (NFC)
With the changes in r263275, there are now more than just functions in
the summary. Completed the renaming of data structures (started in
r263275) to reflect the wider scope. In particular, changed the
FunctionIndex* data structures to ModuleIndex*, and renamed related
variables and comments. Also renamed the files to reflect the changes.

A companion clang patch will immediately succeed this patch to reflect
this renaming.

llvm-svn: 263490
2016-03-14 21:05:56 +00:00
Teresa Johnson 76a1c1d0ba [ThinLTO] Support for reference graph in per-module and combined summary.
Summary:
This patch adds support for including a full reference graph including
call graph edges and other GV references in the summary.

The reference graph edges can be used to make importing decisions
without materializing any source modules, can be used in the plugin
to make file staging decisions for distributed build systems, and is
expected to have other uses.

The call graph edges are recorded in each function summary in the
bitcode via a list of <CalleeValueIds, StaticCount> tuples when no PGO
data exists, or <CalleeValueId, StaticCount, ProfileCount> pairs when
there is PGO, where the ValueId can be mapped to the function GUID via
the ValueSymbolTable. In the function index in memory, the call graph
edges reference the target via the CalleeGUID instead of the
CalleeValueId.

The reference graph edges are recorded in each summary record with a
list of referenced value IDs, which can be mapped to value GUID via the
ValueSymbolTable.

Addtionally, a new summary record type is added to record references
from global variable initializers. A number of bitcode records and data
structures have been renamed to reflect the newly expanded scope of the
summary beyond functions. More cleanup will follow.

Reviewers: joker.eph, davidxl

Subscribers: joker.eph, llvm-commits

Differential Revision: http://reviews.llvm.org/D17212

llvm-svn: 263275
2016-03-11 18:52:24 +00:00
Chandler Carruth 89c45a162f [PM] Port GVN to the new pass manager, wire it up, and teach a couple of
tests to run GVN in both modes.

This is mostly the boring refactoring just like SROA and other complex
transformation passes. There is some trickiness in that GVN's
ValueNumber class requires hand holding to get to compile cleanly. I'm
open to suggestions about a better pattern there, but I tried several
before settling on this. I was trying to balance my desire to sink as
much implementation detail into the source file as possible without
introducing overly many layers of abstraction.

Much like with SROA, the design of this system is made somewhat more
cumbersome by the need to support both pass managers without duplicating
the significant state and logic of the pass. The same compromise is
struck here.

I've also left a FIXME in a doxygen comment as the GVN pass seems to
have pretty woeful documentation within it. I'd like to submit this with
the FIXME and let those more deeply familiar backfill the information
here now that we have a nice place in an interface to put that kind of
documentaiton.

Differential Revision: http://reviews.llvm.org/D18019

llvm-svn: 263208
2016-03-11 08:50:55 +00:00
Mehdi Amini 1592cb9aa1 Rename -discard-value-names into -lto-discard-value-names in libLLVMLTO
This is avoiding a naming conflict with opt and llc.
While opt and llc don't link to LTO usually, users that are building a
monolithic libLLVM.dylib and linking the tools to it would have a
runtime error because of the duplicate cl::opt registration.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 263127
2016-03-10 17:06:52 +00:00
Mehdi Amini 09b4a8daa3 Add a flag to the LLVMContext to disable name for Value other than GlobalValue
Summary:
This is intended to be a performance flag, on the same level as clang
cc1 option "--disable-free". LLVM will never initialize it by default,
it will be up to the client creating the LLVMContext to request this
behavior. Clang will do it by default in Release build (just like
--disable-free).

"opt" and "llc" can opt-in using -disable-named-value command line
option.

When performing LTO on llvm-tblgen, the initial merging of IR peaks
at 92MB without this patch, and 86MB after this patch,setNameImpl()
drops from 6.5MB to 0.5MB.
The total link time goes from ~29.5s to ~27.8s.

Compared to a compile-time flag (like the IRBuilder one), it performs
very close. I profiled on SROA and obtain these results:

 420ms with IRBuilder that preserve name
 372ms with IRBuilder that strip name
 375ms with IRBuilder that preserve name, and a runtime flag to strip

Reviewers: chandlerc, dexonsmith, bogner

Subscribers: joker.eph, llvm-commits

Differential Revision: http://reviews.llvm.org/D17946

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 263086
2016-03-10 01:28:54 +00:00
Mehdi Amini 7c4a1a8d48 libLTO: add a ThinLTOCodeGenerator on the model of LTOCodeGenerator.
This is intended to provide a parallel (threaded) ThinLTO scheme
for linker plugin use through the libLTO C API.

The intent of this patch is to provide a first implementation as a
proof-of-concept and allows linker to start supporting ThinLTO by
definiing the libLTO C API. Some part of the libLTO API are left
unimplemented yet. Following patches will add support for these.

The current implementation can link all clang/llvm binaries.

Differential Revision: http://reviews.llvm.org/D17066

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 262977
2016-03-09 01:37:22 +00:00
David Blaikie bcb1c7e369 Fix some warnings a bit harder/different
This is an alternate fix to 262378 and a fix to a pessimizing-move
warning.

llvm-svn: 262390
2016-03-01 20:41:17 +00:00
Petr Pavlu 7ad9ec9fcf [LTO] Fix error reporting from lto_module_create_in_local_context()
Function lto_module_create_in_local_context() would previously
rely on the default LLVMContext being created for it by
LTOModule::makeLTOModule(). This context exits the program on
error and is not arranged to update sLastStringError in
tools/lto/lto.cpp.

Function lto_module_create_in_local_context() now creates an
LLVMContext by itself, sets it up correctly to its needs and then
passes it to LTOModule::createInLocalContext() which takes
ownership of the context and keeps it present for the lifetime of
the returned LTOModule.

Function LTOModule::makeLTOModule() is modified to take a
reference to LLVMContext (instead of a pointer) and no longer
creates a default context when nullptr is passed to it. Method
LTOModule::createInContext() that takes a pointer to LLVMContext
is removed because it allows to pass a nullptr to it. Instead
LTOModule::createFromBuffer() (that takes a reference to
LLVMContext) should be used.

Differential Revision: http://reviews.llvm.org/D17715

llvm-svn: 262330
2016-03-01 13:13:49 +00:00
Chandler Carruth 9c4ed175c2 [PM] Port the PostOrderFunctionAttrs pass to the new pass manager and
convert one test to use this.

This is a particularly significant milestone because it required
a working per-function AA framework which can be queried over each
function from within a CGSCC transform pass (and additionally a module
analysis to be accessible). This is essentially *the* point of the
entire pass manager rewrite. A CGSCC transform is able to query for
multiple different function's analysis results. It works. The whole
thing appears to actually work and accomplish the original goal. While
we were able to hack function attrs and basic-aa to "work" in the old
pass manager, this port doesn't use any of that, it directly leverages
the new fundamental functionality.

For this to work, the CGSCC framework also has to support SCC-based
behavior analysis, etc. The only part of the CGSCC pass infrastructure
not sorted out at this point are the updates in the face of inlining and
running function passes that mutate the call graph.

The changes are pretty boring and boiler-plate. Most of the work was
factored into more focused preperatory patches. But this is what wires
it all together.

llvm-svn: 261203
2016-02-18 11:03:11 +00:00
Adam Nemet 106fedab6f [LTO] Support Statistics
Summary:
I thought -Xlinker -mllvm -Xlinker -stats worked at some point but maybe
it never did.

For clang, I believe that stats are printed from cc1_main.  This patch
also prints them for LTO, specifically right after codegen happens.

I only looked at the C API for LTO briefly to see if this is a good
place.  Probably there are still cases where this wouldn't be printed
but it seems to be working for the common case.  I also experimented
putting this in the LTOCodeGenerator destructor but that didn't trigger
for me because ld64 does not destroy the LTOCodeGenerator.

Reviewers: dexonsmith, joker.eph

Subscribers: rafael, joker.eph, llvm-commits

Differential Revision: http://reviews.llvm.org/D17302

llvm-svn: 261013
2016-02-16 21:41:51 +00:00
Benjamin Kramer b3e8a6d2b8 Move MCTargetAsmParser.h to llvm/MC/MCParser where it belongs.
llvm-svn: 258917
2016-01-27 10:01:28 +00:00
Chris Bieneman e49730d4ba Remove autoconf support
Summary:
This patch is provided in preparation for removing autoconf on 1/26. The proposal to remove autoconf on 1/26 was discussed on the llvm-dev thread here: http://lists.llvm.org/pipermail/llvm-dev/2016-January/093875.html

"I felt a great disturbance in the [build system], as if millions of [makefiles] suddenly cried out in terror and were suddenly silenced. I fear something [amazing] has happened."
- Obi Wan Kenobi

Reviewers: chandlerc, grosbach, bob.wilson, tstellarAMD, echristo, whitequark

Subscribers: chfast, simoncook, emaste, jholewinski, tberghammer, jfb, danalbert, srhines, arsenm, dschuff, jyknight, dsanders, joker.eph, llvm-commits

Differential Revision: http://reviews.llvm.org/D16471

llvm-svn: 258861
2016-01-26 21:29:08 +00:00
Petr Pavlu eba3039238 [LTO] Fix error reporting when a file passed to libLTO is invalid or non-existent
This addresses PR26060 where function lto_module_create() could return nullptr
but lto_get_error_message() returned an empty string.

The error() call after LTOModule::createFromFile() in llvm-lto is then removed
because any error from this function should go through the diagnostic handler in
llvm-lto which will exit the program. The error() call was added because this
previously did not happen when the file was non-existent. This is fixed by the
patch. (The situation that llvm-lto reports an error when the input file does
not exist is tested by llvm/tools/llvm-lto/error.ll).

Differential Revision: http://reviews.llvm.org/D16106

llvm-svn: 258298
2016-01-20 09:03:42 +00:00
Tobias Edler von Koch 8ecaf69291 [LTO] Restore original linkage of externals prior to splitting
Summary:
This is a companion patch for http://reviews.llvm.org/D16124.

Internalized symbols increase the size of strongly-connected components in
SCC-based module splitting and thus reduce the amount of parallelism. This
patch records the original linkage of non-local symbols prior to
internalization and then restores it just before splitting/CodeGen. This is
also useful for cases where the linker requires symbols to remain external, for
instance, so they can be placed according to linker script rules.

It's currently under its own flag (-restore-globals) but should eventually
share a common flag with D16124.

Reviewers: joker.eph, pcc

Subscribers: slarin, llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D16229

llvm-svn: 258100
2016-01-18 23:24:54 +00:00
Chandler Carruth 1926b70e37 [attrs] Split the late-revisit pattern for deducing norecurse in
a top-down manner into a true top-down or RPO pass over the call graph.

There are specific patterns of function attributes, notably the
norecurse attribute, which are most effectively propagated top-down
because all they us caller information.

Walk in RPO over the call graph SCCs takes the form of a module pass run
immediately after the CGSCC pass managers postorder walk of the SCCs,
trying again to deduce norerucrse for each singular SCC in the call
graph.

This removes a very legacy pass manager specific trick of using a lazy
revisit list traversed during finalization of the CGSCC pass. There is
no analogous finalization step in the new pass manager, and a lazy
revisit list is just trying to produce an RPO iteration of the call
graph. We can do that more directly if more expensively. It seems
unlikely that this will be the expensive part of any compilation though
as we never examine the function bodies here. Even in an LTO run over
a very large module, this should be a reasonable fast set of operations
over a reasonably small working set -- the function call graph itself.

In the future, if this really is a compile time performance issue, we
can look at building support for both post order and RPO traversals
directly into a pass manager that builds and maintains the PO list of
SCCs.

Differential Revision: http://reviews.llvm.org/D15785

llvm-svn: 257163
2016-01-08 10:55:52 +00:00
Teresa Johnson bef543635a Rename variables to reflect linker split (NFC)
Renamed variables to be more reflective of whether they are
an instance of Linker, IRLinker or ModuleLinker. Also fix a stale
comment.

llvm-svn: 256011
2015-12-18 19:28:59 +00:00
Rafael Espindola 434e956181 Change linkInModule to take a std::unique_ptr.
Passing in a std::unique_ptr should help find errors when the module
is used after being linked into another module.

llvm-svn: 255842
2015-12-16 23:16:33 +00:00
Rafael Espindola 9d2bfc4874 Use diagnostic handler in the LLVMContext
This patch converts code that has access to a LLVMContext to not take a
diagnostic handler.

This has a few advantages

* It is easier to use a consistent diagnostic handler in a single program.
* Less clutter since we are not passing a handler around.

It does make it a bit awkward to implement some C APIs that return a
diagnostic string. I will propose new versions of these APIs and
deprecate the current ones.

llvm-svn: 255571
2015-12-14 23:17:03 +00:00
Rafael Espindola f85e9729e9 MSVC complains about this being ambiguous.
llvm-svn: 254782
2015-12-04 22:26:21 +00:00
Rafael Espindola f49a38fc08 Always pass a diagnostic handler to the linker.
Before this patch the diagnostic handler was optional. If it was not
passed, the one in the LLVMContext was used.

That is probably not a pattern we want to follow. If each area has an
optional callback, there is a sea of callbacks and it is hard to follow
which one is called.

Doing this also found cases where the callback is a nice addition, like
testing that no errors or warnings are reported.

The other option is to always use the diagnostic handler in the
LLVMContext. That has a few problems

* To implement the C API we would have to set the diag handler and then
  set it back to the original value.
* Code that creates the context might be far away from code that wants
  the diagnostics.

I do have a patch that implements the second option and will send that as
an RFC.

llvm-svn: 254777
2015-12-04 22:08:53 +00:00
Rafael Espindola a7612b4fac Modernize the C++ APIs for creating LTO modules.
This is a continuation of r253367.

These functions return is owned by the caller, so they return
std::unique_ptr now.

The call can fail, so the return is wrapped in ErrorOr.

They have a context where to report diagnostics, so they don't need to
take a string out parameter.

With this there are no call to getGlobalContext in lib/LTO.

llvm-svn: 254721
2015-12-04 16:14:31 +00:00
Rafael Espindola 7b8a24e5bb Move a call to getGlobalContext out of lib/LTO.
llvm-svn: 254696
2015-12-04 02:42:28 +00:00
Rafael Espindola a06bb16f95 Simplify since this function never fails.
llvm-svn: 254667
2015-12-03 23:56:42 +00:00
Rafael Espindola 0e309fe860 Use references now that it is natural to do so.
The linker never takes ownership of a module or changes which module it
is refering to, making it natural to use references.

llvm-svn: 254449
2015-12-01 19:50:54 +00:00
Rafael Espindola 4808c6d064 Delete the setModule method from the Linker.
It was only used from LTO for a debug feature, and LTO can just create
another linker.

It is pretty odd to have a method to reset the module in the middle of a
link. It would make IdentifiedStructTypes inconsistent with the Module
for example.

llvm-svn: 254434
2015-12-01 18:41:30 +00:00
Tobias Edler von Koch 4d45090659 [LTO] Add option to emit assembly from LTOCodeGenerator
This adds a new API, LTOCodeGenerator::setFileType, to choose the output file
format for LTO CodeGen. A corresponding change to use this new API from
llvm-lto and a test case is coming in a separate commit.

Differential Revision: http://reviews.llvm.org/D14554

llvm-svn: 253622
2015-11-19 23:59:24 +00:00
Yunzhong Gao 8e348cc732 Switch lto codegen to using diagnostic handlers.
This patch removes the std::string& argument from a number of C++ LTO API calls
and instead makes them use the installed diagnostic handler. This would also
improve consistency of diagnostic handling infrastructure: if an LTO client used
lto_codegen_set_diagnostic_handler() to install a custom error handler, we do
not want some error messages to go through the custom error handler, and some
other error messages to go into sLastErrorString.

llvm-svn: 253367
2015-11-17 19:48:12 +00:00
Yunzhong Gao ea7b3a2320 Add a libLTO diagnostic handler that supports lto_get_error_message API
This is a follow-up from the previous discussion on the thread:
http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20151019/307763.html

The LibLTO lto_get_error_message() API reads error messages from a std::string
sLastErrorString. Instead of passing this string around as an argument, this
patch creates a diagnostic handler and then sends this handler to the
constructor of LTOCodeGenerator.

Differential Revision: http://reviews.llvm.org/D14313

llvm-svn: 252791
2015-11-11 19:59:08 +00:00
Mehdi Amini 3383ccc400 Add a method to the BitcodeReader to parse only the identification block
Summary: Mimic parseTriple(); and exposes it to LTOModule.cpp

Reviewers: dexonsmith, rafael

Subscribers: llvm-commits

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 252442
2015-11-09 02:46:41 +00:00
Duncan P. N. Exon Smith cff5feff6f Reapply "LTO: Disable extra verify runs in release builds"
This reverts commit r247730, effectively reapplying r247729.  This time
I have an lld commit ready to follow.

llvm-svn: 247735
2015-09-15 23:05:59 +00:00
Duncan P. N. Exon Smith 7de73e56a4 Revert "LTO: Disable extra verify runs in release builds"
This temporarily reverts commit r247729, as it caused lld build
failures.  I'll recommit once I have an lld patch ready-to-go.

llvm-svn: 247730
2015-09-15 22:47:38 +00:00
Duncan P. N. Exon Smith 236787838c LTO: Disable extra verify runs in release builds
The verifier currently runs three times in LTO: (1) after parsing, (2)
at the beginning of the optimization pipeline, and (3) at the end of it.

The first run is important, since we're not sure where the bitcode comes
from and it's nice to validate it, but in release builds the extra runs
aren't appropriate.

This commit:
  - Allows these runs to be disabled in LTOCodeGenerator.
  - Adds command-line options to llvm-lto.
  - Adds command-line options to libLTO.dylib, and disables the verifier
    by default in release builds (based on NDEBUG).

This shaves about 3.5% off the runtime of ld64 when linking
verify-uselistorder with -flto -g.

rdar://22509081

llvm-svn: 247729
2015-09-15 22:26:11 +00:00
Chandler Carruth 29a18a4663 [PM] Port SROA to the new pass manager.
In some ways this is a very boring port to the new pass manager as there
are no interesting analyses or dependencies or other oddities.

However, this does introduce the first good example of a transformation
pass with non-trivial state porting to the new pass manager. I've tried
to carve out patterns here to replicate elsewhere, and would appreciate
comments on whether folks like these patterns:

- A common need in the new pass manager is to effectively lift the pass
  class and some of its state into a public header file. Prior to this,
  LLVM used anonymous namespaces to provide "module private" types and
  utilities, but that doesn't scale to cases where a public header file
  is needed and the new pass manager will exacerbate that. The pattern
  I've adopted here is to use the namespace-cased-name of the core pass
  (what would be a module if we had them) as a module-private namespace.
  Then utility and other code can be declared and defined in this
  namespace. At some point in the future, we could even have
  (conditionally compiled) code that used modules features when
  available to do the same basic thing.

- I've split the actual pass run method in two in order to expose
  a private method usable by the old pass manager to wrap the new class
  with a minimum of duplicated code. I actually looked at a bunch of
  ways to automate or generate these, but they are all quite terrible
  IMO. The fundamental need is to extract the set of analyses which need
  to cross this interface boundary, and that will end up being too
  unpredictable to effectively encapsulate IMO. This is also
  a relatively small amount of boiler plate that will live a relatively
  short time, so I'm not too worried about the fact that it is boiler
  plate.

The rest of the patch is totally boring but results in a massive diff
(sorry). It just moves code around and removes or adds qualifiers to
reflect the new name and nesting structure.

Differential Revision: http://reviews.llvm.org/D12773

llvm-svn: 247501
2015-09-12 09:09:14 +00:00
Yunzhong Gao 46261a74db Add a non-exiting diagnostic handler for LTO.
This is in order to give LTO clients a chance to do some clean-up before
terminating the process.

llvm-svn: 247461
2015-09-11 20:01:53 +00:00
Chandler Carruth 7b560d40bd [PM/AA] Rebuild LLVM's alias analysis infrastructure in a way compatible
with the new pass manager, and no longer relying on analysis groups.

This builds essentially a ground-up new AA infrastructure stack for
LLVM. The core ideas are the same that are used throughout the new pass
manager: type erased polymorphism and direct composition. The design is
as follows:

- FunctionAAResults is a type-erasing alias analysis results aggregation
  interface to walk a single query across a range of results from
  different alias analyses. Currently this is function-specific as we
  always assume that aliasing queries are *within* a function.

- AAResultBase is a CRTP utility providing stub implementations of
  various parts of the alias analysis result concept, notably in several
  cases in terms of other more general parts of the interface. This can
  be used to implement only a narrow part of the interface rather than
  the entire interface. This isn't really ideal, this logic should be
  hoisted into FunctionAAResults as currently it will cause
  a significant amount of redundant work, but it faithfully models the
  behavior of the prior infrastructure.

- All the alias analysis passes are ported to be wrapper passes for the
  legacy PM and new-style analysis passes for the new PM with a shared
  result object. In some cases (most notably CFL), this is an extremely
  naive approach that we should revisit when we can specialize for the
  new pass manager.

- BasicAA has been restructured to reflect that it is much more
  fundamentally a function analysis because it uses dominator trees and
  loop info that need to be constructed for each function.

All of the references to getting alias analysis results have been
updated to use the new aggregation interface. All the preservation and
other pass management code has been updated accordingly.

The way the FunctionAAResultsWrapperPass works is to detect the
available alias analyses when run, and add them to the results object.
This means that we should be able to continue to respect when various
passes are added to the pipeline, for example adding CFL or adding TBAA
passes should just cause their results to be available and to get folded
into this. The exception to this rule is BasicAA which really needs to
be a function pass due to using dominator trees and loop info. As
a consequence, the FunctionAAResultsWrapperPass directly depends on
BasicAA and always includes it in the aggregation.

This has significant implications for preserving analyses. Generally,
most passes shouldn't bother preserving FunctionAAResultsWrapperPass
because rebuilding the results just updates the set of known AA passes.
The exception to this rule are LoopPass instances which need to preserve
all the function analyses that the loop pass manager will end up
needing. This means preserving both BasicAAWrapperPass and the
aggregating FunctionAAResultsWrapperPass.

Now, when preserving an alias analysis, you do so by directly preserving
that analysis. This is only necessary for non-immutable-pass-provided
alias analyses though, and there are only three of interest: BasicAA,
GlobalsAA (formerly GlobalsModRef), and SCEVAA. Usually BasicAA is
preserved when needed because it (like DominatorTree and LoopInfo) is
marked as a CFG-only pass. I've expanded GlobalsAA into the preserved
set everywhere we previously were preserving all of AliasAnalysis, and
I've added SCEVAA in the intersection of that with where we preserve
SCEV itself.

One significant challenge to all of this is that the CGSCC passes were
actually using the alias analysis implementations by taking advantage of
a pretty amazing set of loop holes in the old pass manager's analysis
management code which allowed analysis groups to slide through in many
cases. Moving away from analysis groups makes this problem much more
obvious. To fix it, I've leveraged the flexibility the design of the new
PM components provides to just directly construct the relevant alias
analyses for the relevant functions in the IPO passes that need them.
This is a bit hacky, but should go away with the new pass manager, and
is already in many ways cleaner than the prior state.

Another significant challenge is that various facilities of the old
alias analysis infrastructure just don't fit any more. The most
significant of these is the alias analysis 'counter' pass. That pass
relied on the ability to snoop on AA queries at different points in the
analysis group chain. Instead, I'm planning to build printing
functionality directly into the aggregation layer. I've not included
that in this patch merely to keep it smaller.

Note that all of this needs a nearly complete rewrite of the AA
documentation. I'm planning to do that, but I'd like to make sure the
new design settles, and to flesh out a bit more of what it looks like in
the new pass manager first.

Differential Revision: http://reviews.llvm.org/D12080

llvm-svn: 247167
2015-09-09 17:55:00 +00:00
Yaron Keren 55f5c3d43b Fix typo.
llvm-svn: 246538
2015-09-01 10:13:49 +00:00
Duncan P. N. Exon Smith f4967754a5 LTO: Cleanup parameter names and header docs, NFC
Follow LLVM style for the parameter names (`CamelCase` not `camelCase`),
and surface the header docs in doxygen.  No functionality change
intended.

llvm-svn: 246509
2015-08-31 23:44:06 +00:00
Peter Collingbourne c269ed5115 CodeGen: Introduce splitCodeGen and teach LTOCodeGenerator to use it.
llvm::splitCodeGen is a function that implements the core of parallel LTO
code generation. It uses llvm::SplitModule to split the module into linkable
partitions and spawning one code generation thread per partition. The function
produces multiple object files which can be linked in the usual way.

This has been threaded through to LTOCodeGenerator (and llvm-lto for testing
purposes). Separate patches will add parallel LTO support to the gold plugin
and lld.

Differential Revision: http://reviews.llvm.org/D12260

llvm-svn: 246236
2015-08-27 23:37:36 +00:00
Peter Collingbourne 9c8909dbd1 LTO: Simplify merged module ownership.
This change moves LTOCodeGenerator's ownership of the merged module to a
field of type std::unique_ptr<Module>. This helps simplify parts of the code
and clears the way for the module to be consumed by LLVM CodeGen (see D12132
review comments).

Differential Revision: http://reviews.llvm.org/D12205

llvm-svn: 245891
2015-08-24 22:22:53 +00:00
Peter Collingbourne e34034c8d0 LTO: Rename mergedModule variables to MergedModule to prepare for ownership change.
Also convert a few loops to range-for loops and correct a comment.

llvm-svn: 245874
2015-08-24 21:15:35 +00:00
Peter Collingbourne c7b675f48c LTO: Maintain target triple, FeatureStr and CGOptLevel in the module or LTOCodeGenerator.
This makes it easier to create new TargetMachines on demand.

llvm-svn: 245781
2015-08-22 02:25:53 +00:00
Peter Collingbourne 44ee84eec5 LTO: Change signature of LTOCodeGenerator::setCodePICModel() to take a Reloc::Model.
This allows us to remove a bunch of code in LTOCodeGenerator and llvm-lto
and has the side effect of improving error handling in the libLTO C API.

llvm-svn: 245756
2015-08-21 22:57:17 +00:00
Peter Collingbourne ec43d0f356 LTO: Simplify ownership of LTOCodeGenerator::TargetMach.
llvm-svn: 245671
2015-08-21 04:45:57 +00:00
Peter Collingbourne 2257512f87 LTO: Simplify ownership of LTOCodeGenerator::CodegenOptions.
llvm-svn: 245670
2015-08-21 04:45:55 +00:00
Chandler Carruth 7adc3a2b0e [PM/AA] Remove the last relics of the separate IPA library from LLVM,
folding the code into the main Analysis library.

There already wasn't much of a distinction between Analysis and IPA.
A number of the passes in Analysis are actually IPA passes, and there
doesn't seem to be any advantage to separating them.

Moreover, it makes it hard to have interactions between analyses that
are both local and interprocedural. In trying to make the Alias Analysis
infrastructure work with the new pass manager, it becomes particularly
awkward to navigate this split.

I've tried to find all the places where we referenced this, but I may
have missed some. I have also adjusted the C API to continue to be
equivalently functional after this change.

Differential Revision: http://reviews.llvm.org/D12075

llvm-svn: 245318
2015-08-18 17:51:53 +00:00
Mehdi Amini 26d481311a Remove access to the DataLayout in the TargetMachine
Summary:
Replace getDataLayout() with a createDataLayout() method to make
explicit that it is intended to create a DataLayout only and not
accessing it for other purpose.

This change is the last of a series of commits dedicated to have a
single DataLayout during compilation by using always the one owned
by the module.

Reviewers: echristo

Subscribers: jholewinski, llvm-commits, rafael, yaron.keren

Differential Revision: http://reviews.llvm.org/D11103

(cherry picked from commit 5609fc56bca971e5a7efeaa6ca4676638eaec5ea)

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 243114
2015-07-24 16:04:22 +00:00
Mehdi Amini 5d8e569926 Revert "Remove access to the DataLayout in the TargetMachine"
This reverts commit 0f720d984f419c747709462f7476dff962c0bc41.

It breaks clang too badly, I need to prepare a proper patch for clang
first.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 243089
2015-07-24 03:36:55 +00:00
Mehdi Amini b4bc424c9a Remove access to the DataLayout in the TargetMachine
Summary:
Replace getDataLayout() with a createDataLayout() method to make
explicit that it is intended to create a DataLayout only and not
accessing it for other purpose.

This change is the last of a series of commits dedicated to have a
single DataLayout during compilation by using always the one owned
by the module.

Reviewers: echristo

Subscribers: jholewinski, llvm-commits, rafael, yaron.keren

Differential Revision: http://reviews.llvm.org/D11103

(cherry picked from commit 5609fc56bca971e5a7efeaa6ca4676638eaec5ea)

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 243083
2015-07-24 01:44:39 +00:00
Peter Collingbourne 17eff10f68 LTO: expose LTO_SYMBOL_ALIAS, which indicates that the symbol is an alias.
This is needed for COFF linkers to distinguish between weak external aliases
and regular symbols with LLVM weak linkage, which are represented as strong
symbols in COFF.

llvm-svn: 241389
2015-07-04 03:42:35 +00:00
Peter Collingbourne aef3659e18 Teach LTOModule to emit linker flags for dllexported symbols, plus interface cleanup.
This change unifies how LTOModule and the backend obtain linker flags
for globals: via a new TargetLoweringObjectFile member function named
emitLinkerFlagsForGlobal. A new function LTOModule::getLinkerOpts() returns
the list of linker flags as a single concatenated string.

This change affects the C libLTO API: the function lto_module_get_*deplibs now
exposes an empty list, and lto_module_get_*linkeropts exposes a single element
which combines the contents of all observed flags. libLTO should never have
tried to parse the linker flags; it is the linker's job to do so. Because
linkers will need to be able to parse flags in regular object files, it
makes little sense for libLTO to have a redundant mechanism for doing so.

The new API is compatible with the old one. It is valid for a user to specify
multiple linker flags in a single pragma directive like this:

 #pragma comment(linker, "/defaultlib:foo /defaultlib:bar")

The previous implementation would not have exposed
either flag via lto_module_get_*deplibs (as the test in
TargetLoweringObjectFileCOFF::getDepLibFromLinkerOpt was case sensitive)
and would have exposed "/defaultlib:foo /defaultlib:bar" as a single flag via
lto_module_get_*linkeropts. This may have been a bug in the implementation,
but it does give us a chance to fix the interface.

Differential Revision: http://reviews.llvm.org/D10548

llvm-svn: 241010
2015-06-29 22:04:09 +00:00
Rafael Espindola c233f74e6e Simplify the Mangler interface now that DataLayout is mandatory.
We only need to pass in a DataLayout when mangling a raw string, not when
constructing the mangler.

llvm-svn: 240405
2015-06-23 13:59:29 +00:00
Rafael Espindola dcd1dca275 Return a unique_ptr from getLazyBitcodeModule and parseBitcodeFile. NFC.
llvm-svn: 239858
2015-06-16 22:27:55 +00:00
Douglas Katzman 8f01f1cfc3 Wrap some long lines in LLVMBuild files. NFC
As suggested by jroelofs in a prior review (D9752),
it makes sense to generally prefer multi-line format.

llvm-svn: 239632
2015-06-12 18:44:57 +00:00
Peter Collingbourne 485ad4860e LTO: expose LTO_SYMBOL_COMDAT flag, which indicates that the definition is part of a comdat group.
Reviewers: rafael

Subscribers: llvm-commits, ruiu

Differential Revision: http://reviews.llvm.org/D10330

llvm-svn: 239559
2015-06-11 21:41:27 +00:00
Peter Collingbourne 3cc69d90f0 Make the C++ LTO API easier to use from C++ clients.
Start using C++ types such as StringRef and MemoryBuffer in the C++ LTO
API. In doing so, clarify the ownership of the native object file: the caller
now owns it, not the LTOCodeGenerator. The C libLTO library has been modified
to use a derived class of LTOCodeGenerator that owns the object file.

Differential Revision: http://reviews.llvm.org/D10114

llvm-svn: 238776
2015-06-01 20:08:30 +00:00
Duncan P. N. Exon Smith 5a490d0026 LTO: Add API to choose whether to embed uselists
Reverse libLTO's default behaviour for preserving use-list order in
bitcode, and add API for controlling it.  The default setting is now
`false` (don't preserve them), which is consistent with `clang`'s
default behaviour.

Users of libLTO should call `lto_codegen_should_embed_uselists(CG,true)`
prior to calling `lto_codegen_write_merged_modules()` whenever the
output file isn't part of the production workflow in order to reproduce
results with subsequent calls to `llc`.

(I haven't added tests since `llvm-lto` (the test tool for LTO) doesn't
support bitcode output, and even if it did: there isn't actually a good
way to test whether a tool has passed the flag.  If the order is already
"natural" (if the order will already round-trip) then no use-list
directives are emitted at all.  At some point I'll circle back to add
tests to `llvm-as` (etc.) that they actually respect the flag, at which
point I can somehow add a test here as well.)

llvm-svn: 235943
2015-04-27 23:38:54 +00:00
Duncan P. N. Exon Smith 7832e0a2f0 LTO: Simplify code generator initialization
Simplify `LTOCodeGenerator` initialization by initializing simple fields
at their definition.

llvm-svn: 235939
2015-04-27 23:19:26 +00:00
Manman Ren ce0a066524 [LTO API] add lto_codegen_set_should_internalize.
When debugging LTO issues with ld64, we use -save-temps to save the merged
optimized bitcode file, then invoke ld64 again on the single bitcode file.
The saved bitcode file is already internalized, so we can call
lto_codegen_set_should_internalize and skip running internalization again.

rdar://20227235

llvm-svn: 235211
2015-04-17 17:10:09 +00:00
Duncan P. N. Exon Smith 8a7b84b4d0 uselistorder: Remove the global bits
Remove all the global bits to do with preserving use-list order by
moving the `cl::opt`s to the individual tools that want them.  There's a
minor functionality change to `libLTO`, in that you can't send in
`-preserve-bc-uselistorder=false`, but making that bit settable (if it's
worth doing) should be through explicit LTO API.

As a drive-by fix, I removed some includes of `UseListOrder.h` that were
made unnecessary by recent commits.

llvm-svn: 234973
2015-04-15 03:14:06 +00:00
Duncan P. N. Exon Smith a052ed6381 uselistorder: Pull the bit through WriteToBitcodFile()
Change the callers of `WriteToBitcodeFile()` to pass `true` or
`shouldPreserveBitcodeUseListOrder()` explicitly.  I left the callers
that want to send `false` alone.

I'll keep pushing the bit higher until hopefully I can delete the global
`cl::opt` entirely.

llvm-svn: 234957
2015-04-15 00:10:50 +00:00
Rafael Espindola 5560a4cfbd Use raw_pwrite_stream in the object writer/streamer.
The ELF object writer will take advantage of that in the next commit.

llvm-svn: 234950
2015-04-14 22:14:34 +00:00
Duncan P. N. Exon Smith c55dee1c2f IR: Set -preserve-bc-uselistorder=false by default
But keep it on by default in `llvm-as`, `opt`, `bugpoint`, `llvm-link`,
`llvm-extract`, and `LTOCodeGenerator`.  Part of PR5680.

llvm-svn: 234921
2015-04-14 18:33:00 +00:00
Rafael Espindola 5682ce2ceb Simplify use of formatted_raw_ostream.
formatted_raw_ostream is a wrapper over another stream to add column and line
number tracking.

It is used only for asm printing.

This patch moves the its creation down to where we know we are printing
assembly. This has the following advantages:

* Simpler lifetime management: std::unique_ptr
* We don't compute column and line number of object files :-)

llvm-svn: 234535
2015-04-09 21:06:08 +00:00
Rafael Espindola ee0dd4d289 This reverts commit r234460 and r234461.
Revert "Add classof implementations to the raw_ostream classes."
Revert "Use the cast machinery to remove dummy uses of formatted_raw_ostream."

The underlying issue can be fixed without classof.

llvm-svn: 234495
2015-04-09 15:54:59 +00:00
Rafael Espindola 132381f981 Use the cast machinery to remove dummy uses of formatted_raw_ostream.
If we know we are producing an object, we don't need to wrap the stream
in a formatted_raw_ostream anymore.

llvm-svn: 234461
2015-04-09 02:28:12 +00:00
Manman Ren ed6b5fc4b0 [LTO] do not run internalize pass from compileOptimized.
The input to compileOptimized is already optimized and internalized, so remove
internalize pass from compileOptimized.

rdar://20227235

llvm-svn: 234446
2015-04-08 22:02:11 +00:00
Yaron Keren 075759aadd Remove more superfluous .str() and replace std::string concatenation with Twine.
Following r233392, http://llvm.org/viewvc/llvm-project?rev=233392&view=rev.

llvm-svn: 233555
2015-03-30 15:42:36 +00:00
Duncan P. N. Exon Smith ab58a568ee Verifier: Remove the separate -verify-di pass
Remove `DebugInfoVerifierLegacyPass` and the `-verify-di` pass.
Instead, call into the `DebugInfoVerifier` from inside
`VerifierLegacyPass::finalizeModule()`.  This better matches the logic
in `verifyModule()` (used by the new PassManager), avoids requiring two
separate passes to verify the IR, and makes the API for "add a pass to
verify the IR" simple.

Note: the `-verify-debug-info` flag still works (for now, at least;
eventually it might make sense to just remove it).

llvm-svn: 232772
2015-03-19 22:24:17 +00:00
Peter Collingbourne 070843d60b libLTO, llvm-lto, gold: Introduce flag for controlling optimization level.
This change also introduces a link-time optimization level of 1. This
optimization level runs only the globaldce pass as well as cleanup passes for
passes that run at -O0, specifically simplifycfg which cleans up lowerbitsets.

http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20150316/266951.html

llvm-svn: 232769
2015-03-19 22:01:00 +00:00
Manman Ren 4a9b0ebe83 Add a parameter for getLazyBitcodeModule to lazily load Metadata.
We only defer loading metadata inside ParseModule when ShouldLazyLoadMetadata
is true and we have not loaded any Metadata block yet.

This commit implements all-or-nothing loading of Metadata. If there is a
request to load any metadata block, we will load all deferred metadata blocks.

We make sure the deferred metadata blocks are loaded before we materialize any
function or a module.

The default value of the added parameter ShouldLazyLoadMetadata for
getLazyBitcodeModule is false, so the default behavior stays the same.

We only set the parameter to true when creating LTOModule in local contexts.
These can only really be used for parsing symbols, so it's unnecessary to ever
load the metadata blocks.

If we are going to enable lazy-loading of Metadata for other usages of
getLazyBitcodeModule, where deferred metadata blocks need to be loaded, we can
expose BitcodeReader::materializeMetadata to Module, similar to
Module::materialize.

rdar://19804575

llvm-svn: 232198
2015-03-13 19:24:30 +00:00
Mehdi Amini 46a43556db Make DataLayout Non-Optional in the Module
Summary:
DataLayout keeps the string used for its creation.

As a side effect it is no longer needed in the Module.
This is "almost" NFC, the string is no longer
canonicalized, you can't rely on two "equals" DataLayout
having the same string returned by getStringRepresentation().

Get rid of DataLayoutPass: the DataLayout is in the Module

The DataLayout is "per-module", let's enforce this by not
duplicating it more than necessary.
One more step toward non-optionality of the DataLayout in the
module.

Make DataLayout Non-Optional in the Module

Module->getDataLayout() will never returns nullptr anymore.

Reviewers: echristo

Subscribers: resistor, llvm-commits, jholewinski

Differential Revision: http://reviews.llvm.org/D7992

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 231270
2015-03-04 18:43:29 +00:00
Manman Ren 082a336a89 [LTO API] fix memory leakage introduced at r230290.
r230290 released the LLVM module but not the LTOModule.

rdar://19024554

llvm-svn: 230544
2015-02-25 21:20:53 +00:00
Manman Ren 6487ce955a [LTO API] add lto_codegen_set_module to set the destination module.
When debugging LTO issues with ld64, we use -save-temps to save the merged
optimized bitcode file, then invoke ld64 again on the single bitcode file to
speed up debugging code generation passes and ld64 stuff after code generation.

llvm linking a single bitcode file via lto_codegen_add_module will generate a
different bitcode file from the single input. With the newly-added
lto_codegen_set_module, we can make sure the destination module is the same as
the input.

lto_codegen_set_module will transfer the ownship of the module to code
generator.

rdar://19024554

llvm-svn: 230290
2015-02-24 00:45:56 +00:00
Chandler Carruth 30d69c2e36 [PM] Remove the old 'PassManager.h' header file at the top level of
LLVM's include tree and the use of using declarations to hide the
'legacy' namespace for the old pass manager.

This undoes the primary modules-hostile change I made to keep
out-of-tree targets building. I sent an email inquiring about whether
this would be reasonable to do at this phase and people seemed fine with
it, so making it a reality. This should allow us to start bootstrapping
with modules to a certain extent along with making it easier to mix and
match headers in general.

The updates to any code for users of LLVM are very mechanical. Switch
from including "llvm/PassManager.h" to "llvm/IR/LegacyPassManager.h".
Qualify the types which now produce compile errors with "legacy::". The
most common ones are "PassManager", "PassManagerBase", and
"FunctionPassManager".

llvm-svn: 229094
2015-02-13 10:01:29 +00:00
Zachary Turner 3bd47cee78 Use ADDITIONAL_HEADER_DIRS in all LLVM CMake projects.
This allows IDEs to recognize the entire set of header files for
each of the core LLVM projects.

Differential Revision: http://reviews.llvm.org/D7526
Reviewed By: Chris Bieneman

llvm-svn: 228798
2015-02-11 03:28:02 +00:00
Manman Ren 8121e1db91 [LTO API] split lto_codegen_compile to lto_codegen_optimize and
lto_codegen_compile_optimized. Also add lto_api_version.

Before this commit, we can only dump the optimized bitcode after running
lto_codegen_compile, but it includes some impacts of running codegen passes,
one example is StackProtector pass. We will get assertion failure when running
llc on the optimized bitcode, because StackProtector is effectively run twice.

After splitting lto_codegen_compile, the linker can choose to dump the bitcode
before running lto_codegen_compile_optimized.

lto_api_version is added so ld64 can check for runtime-availability of the new
API.

rdar://19565500

llvm-svn: 228000
2015-02-03 18:39:15 +00:00
Eric Christopher 36fe028a2a Only access TLOF via the TargetMachine, not TargetLowering.
llvm-svn: 227949
2015-02-03 07:22:52 +00:00
Chandler Carruth 5ec2b1d11a [multiversion] Implement the old pass manager's TTI wrapper pass in
terms of the new pass manager's TargetIRAnalysis.

Yep, this is one of the nicer bits of the new pass manager's design.
Passes can in many cases operate in a vacuum and so we can just nest
things when convenient. This is particularly convenient here as I can
now consolidate all of the TargetMachine logic on this analysis.

The most important change here is that this pushes the function we need
TTI for all the way into the TargetMachine, and re-creates the TTI
object for each function rather than re-using it for each function.
We're now prepared to teach the targets to produce function-specific TTI
objects with specific subtargets cached, etc.

One piece of feedback I'd love here is whether its worth renaming any of
this stuff. None of the names really seem that awesome to me at this
point, but TargetTransformInfoWrapperPass is particularly ... odd.
TargetIRAnalysisWrapper might make more sense. I would want to do that
rename separately anyways, but let me know what you think.

llvm-svn: 227731
2015-02-01 12:26:09 +00:00
NAKAMURA Takumi 7ae226dfaf [CMake] LLVMLTO requires Intrinsics.gen since r227685 introduced llvm/Analysis/TargetTransformInfo.h.
llvm-svn: 227700
2015-02-01 00:55:43 +00:00
Chandler Carruth 93dcdc47db [PM] Switch the TargetMachine interface from accepting a pass manager
base which it adds a single analysis pass to, to instead return the type
erased TargetTransformInfo object constructed for that TargetMachine.

This removes all of the pass variants for TTI. There is now a single TTI
*pass* in the Analysis layer. All of the Analysis <-> Target
communication is through the TTI's type erased interface itself. While
the diff is large here, it is nothing more that code motion to make
types available in a header file for use in a different source file
within each target.

I've tried to keep all the doxygen comments and file boilerplate in line
with this move, but let me know if I missed anything.

With this in place, the next step to making TTI work with the new pass
manager is to introduce a really simple new-style analysis that produces
a TTI object via a callback into this routine on the target machine.
Once we have that, we'll have the building blocks necessary to accept
a function argument as well.

llvm-svn: 227685
2015-01-31 11:17:59 +00:00
Chandler Carruth 1efa12d6d8 [PM] Sink the population of the pass manager with target-specific
analyses back into the LTO code generator.

The pass manager builder (and the transforms library in general)
shouldn't be referencing the target machine at all.

This makes the LTO population work like the others -- the data layout
and target transform info need to be pre-populated.

llvm-svn: 227576
2015-01-30 13:33:42 +00:00
Akira Hatanaka 8fba18e958 [LTO] Scan all per-function subtargets when collecting runtime library names.
accumulateAndSortLibcalls in LTOCodeGenerator.cpp collects names of runtime
library functions which are used to identify user-defined functions that should
be protected. Previously, this function would only scan the TargetLowering
object belonging to the "main" subtarget for the library function names. This
commit changes it to scan all per-function subtargets.

Differential Revision: http://reviews.llvm.org/D7275

llvm-svn: 227533
2015-01-30 01:16:24 +00:00
Eric Christopher 8b7706517c Move DataLayout back to the TargetMachine from TargetSubtargetInfo
derived classes.

Since global data alignment, layout, and mangling is often based on the
DataLayout, move it to the TargetMachine. This ensures that global
data is going to be layed out and mangled consistently if the subtarget
changes on a per function basis. Prior to this all targets(*) have
had subtarget dependent code moved out and onto the TargetMachine.

*One target hasn't been migrated as part of this change: R600. The
R600 port has, as a subtarget feature, the size of pointers and
this affects global data layout. I've currently hacked in a FIXME
to enable progress, but the port needs to be updated to either pass
the 64-bitness to the TargetMachine, or fix the DataLayout to
avoid subtarget dependent features.

llvm-svn: 227113
2015-01-26 19:03:15 +00:00
Chandler Carruth c0291865ed [PM] Rework how the TargetLibraryInfo pass integrates with the new pass
manager to support the actual uses of it. =]

When I ported instcombine to the new pass manager I discover that it
didn't work because TLI wasn't available in the right places. This is
a somewhat surprising and/or subtle aspect of the new pass manager
design that came up before but I think is useful to be reminded of:

While the new pass manager *allows* a function pass to query a module
analysis, it requires that the module analysis is already run and cached
prior to the function pass manager starting up, possibly with
a 'require<foo>' style utility in the pass pipeline. This is an
intentional hurdle because using a module analysis from a function pass
*requires* that the module analysis is run prior to entering the
function pass manager. Otherwise the other functions in the module could
be in who-knows-what state, etc.

A somewhat surprising consequence of this design decision (at least to
me) is that you have to design a function pass that leverages
a module analysis to do so as an optional feature. Even if that means
your function pass does no work in the absence of the module analysis,
you have to handle that possibility and remain conservatively correct.
This is a natural consequence of things being able to invalidate the
module analysis and us being unable to re-run it. And it's a generally
good thing because it lets us reorder passes arbitrarily without
breaking correctness, etc.

This ends up causing problems in one case. What if we have a module
analysis that is *definitionally* impossible to invalidate. In the
places this might come up, the analysis is usually also definitionally
trivial to run even while other transformation passes run on the module,
regardless of the state of anything. And so, it follows that it is
natural to have a hard requirement on such analyses from a function
pass.

It turns out, that TargetLibraryInfo is just such an analysis, and
InstCombine has a hard requirement on it.

The approach I've taken here is to produce an analysis that models this
flexibility by making it both a module and a function analysis. This
exposes the fact that it is in fact safe to compute at any point. We can
even make it a valid CGSCC analysis at some point if that is useful.
However, we don't want to have a copy of the actual target library info
state for each function! This state is specific to the triple. The
somewhat direct and blunt approach here is to turn TLI into a pimpl,
with the state and mutators in the implementation class and the query
routines primarily in the wrapper. Then the analysis can lazily
construct and cache the implementations, keyed on the triple, and
on-demand produce wrappers of them for each function.

One minor annoyance is that we will end up with a wrapper for each
function in the module. While this is a bit wasteful (one pointer per
function) it seems tolerable. And it has the advantage of ensuring that
we pay the absolute minimum synchronization cost to access this
information should we end up with a nice parallel function pass manager
in the future. We could look into trying to mark when analysis results
are especially cheap to recompute and more eagerly GC-ing the cached
results, or we could look at supporting a variant of analyses whose
results are specifically *not* cached and expected to just be used and
discarded by the consumer. Either way, these seem like incremental
enhancements that should happen when we start profiling the memory and
CPU usage of the new pass manager and not before.

The other minor annoyance is that if we end up using the TLI in both
a module pass and a function pass, those will be produced by two
separate analyses, and thus will point to separate copies of the
implementation state. While a minor issue, I dislike this and would like
to find a way to cleanly allow a single analysis instance to be used
across multiple IR unit managers. But I don't have a good solution to
this today, and I don't want to hold up all of the work waiting to come
up with one. This too seems like a reasonable thing to incrementally
improve later.

llvm-svn: 226981
2015-01-24 02:06:09 +00:00
Chandler Carruth 1edb9d63e9 [PM] Separate the InstCombiner from its pass.
This creates a small internal pass which runs the InstCombiner over
a function. This is the hard part of porting InstCombine to the new pass
manager, as at this point none of the code in InstCombine has access to
a Pass object any longer.

The resulting interface for the InstCombiner is pretty terrible. I'm not
planning on leaving it that way. The key thing missing is that we need
to separate the worklist from the combiner a touch more. Once that's
done, it should be possible for *any* part of LLVM to just create
a worklist with instructions, populate it, and then combine it until
empty. The pass will just be the (obvious and important) special case of
doing that for an entire function body.

For now, this is the first increment of factoring to make all of this
work.

llvm-svn: 226618
2015-01-20 22:44:35 +00:00
NAKAMURA Takumi 24ebfcb619 Update libdeps since TLI was moved from Target to Analysis in r226078.
llvm-svn: 226126
2015-01-15 05:21:00 +00:00
NAKAMURA Takumi ab7289dd0e Reorder.
llvm-svn: 226125
2015-01-15 05:20:46 +00:00
Chandler Carruth 62d4215baa [PM] Move TargetLibraryInfo into the Analysis library.
While the term "Target" is in the name, it doesn't really have to do
with the LLVM Target library -- this isn't an abstraction which LLVM
targets generally need to implement or extend. It has much more to do
with modeling the various runtime libraries on different OSes and with
different runtime environments. The "target" in this sense is the more
general sense of a target of cross compilation.

This is in preparation for porting this analysis to the new pass
manager.

No functionality changed, and updates inbound for Clang and Polly.

llvm-svn: 226078
2015-01-15 02:16:27 +00:00
Rafael Espindola d0b23bef6f Use the DiagnosticHandler to print diagnostics when reading bitcode.
The bitcode reading interface used std::error_code to report an error to the
callers and it is the callers job to print diagnostics.

This is not ideal for error handling or diagnostic reporting:

* For error handling, all that the callers care about is 3 possibilities:
  * It worked
  * The bitcode file is corrupted/invalid.
  * The file is not bitcode at all.

* For diagnostic, it is user friendly to include far more information
  about the invalid case so the user can find out what is wrong with the
  bitcode file. This comes up, for example, when a developer introduces a
  bug while extending the format.

The compromise we had was to have a lot of error codes.

With this patch we use the DiagnosticHandler to communicate with the
human and std::error_code to communicate with the caller.

This allows us to have far fewer error codes and adds the infrastructure to
print better diagnostics. This is so because the diagnostics are printed when
he issue is found. The code that detected the problem in alive in the stack and
can pass down as much context as needed. As an example the patch updates
test/Bitcode/invalid.ll.

Using a DiagnosticHandler also moves the fatal/non-fatal error decision to the
caller. A simple one like llvm-dis can just use fatal errors. The gold plugin
needs a bit more complex treatment because of being passed non-bitcode files. An
hypothetical interactive tool would make all bitcode errors non-fatal.

llvm-svn: 225562
2015-01-10 00:07:30 +00:00
Duncan P. N. Exon Smith 140d41b791 LTO: Lazy-load LTOModule in local contexts
Start lazy-loading `LTOModule`s that own their contexts.  These can only
really be used for parsing symbols, so its unnecessary to ever
materialize their functions.

I looked into using `IRObjectFile::create()` and optionally calling
`materializAllPermanently()` afterwards, but this turned out to be
awkward.

  - The default target triple and data layout logic needs to happen
    *before* the call to `IRObjectFile::IRObjectFile()`, but after
    `Module` was created.

  - I tried passing a lambda in to do the module initialization, but
    this seemed to require threading the error message from
    `TargetRegistry::lookupTarget()` through `std::error_code`.

  - I also looked at setting `errMsg` directly from within the lambda,
    but this didn't look any better.

(I guess there's a reason we weren't already using that function.)

llvm-svn: 224466
2014-12-17 22:05:42 +00:00
Duncan P. N. Exon Smith 5bf8fef580 IR: Split Metadata from Value
Split `Metadata` away from the `Value` class hierarchy, as part of
PR21532.  Assembly and bitcode changes are in the wings, but this is the
bulk of the change for the IR C++ API.

I have a follow-up patch prepared for `clang`.  If this breaks other
sub-projects, I apologize in advance :(.  Help me compile it on Darwin
I'll try to fix it.  FWIW, the errors should be easy to fix, so it may
be simpler to just fix it yourself.

This breaks the build for all metadata-related code that's out-of-tree.
Rest assured the transition is mechanical and the compiler should catch
almost all of the problems.

Here's a quick guide for updating your code:

  - `Metadata` is the root of a class hierarchy with three main classes:
    `MDNode`, `MDString`, and `ValueAsMetadata`.  It is distinct from
    the `Value` class hierarchy.  It is typeless -- i.e., instances do
    *not* have a `Type`.

  - `MDNode`'s operands are all `Metadata *` (instead of `Value *`).

  - `TrackingVH<MDNode>` and `WeakVH` referring to metadata can be
    replaced with `TrackingMDNodeRef` and `TrackingMDRef`, respectively.

    If you're referring solely to resolved `MDNode`s -- post graph
    construction -- just use `MDNode*`.

  - `MDNode` (and the rest of `Metadata`) have only limited support for
    `replaceAllUsesWith()`.

    As long as an `MDNode` is pointing at a forward declaration -- the
    result of `MDNode::getTemporary()` -- it maintains a side map of its
    uses and can RAUW itself.  Once the forward declarations are fully
    resolved RAUW support is dropped on the ground.  This means that
    uniquing collisions on changing operands cause nodes to become
    "distinct".  (This already happened fairly commonly, whenever an
    operand went to null.)

    If you're constructing complex (non self-reference) `MDNode` cycles,
    you need to call `MDNode::resolveCycles()` on each node (or on a
    top-level node that somehow references all of the nodes).  Also,
    don't do that.  Metadata cycles (and the RAUW machinery needed to
    construct them) are expensive.

  - An `MDNode` can only refer to a `Constant` through a bridge called
    `ConstantAsMetadata` (one of the subclasses of `ValueAsMetadata`).

    As a side effect, accessing an operand of an `MDNode` that is known
    to be, e.g., `ConstantInt`, takes three steps: first, cast from
    `Metadata` to `ConstantAsMetadata`; second, extract the `Constant`;
    third, cast down to `ConstantInt`.

    The eventual goal is to introduce `MDInt`/`MDFloat`/etc. and have
    metadata schema owners transition away from using `Constant`s when
    the type isn't important (and they don't care about referring to
    `GlobalValue`s).

    In the meantime, I've added transitional API to the `mdconst`
    namespace that matches semantics with the old code, in order to
    avoid adding the error-prone three-step equivalent to every call
    site.  If your old code was:

        MDNode *N = foo();
        bar(isa             <ConstantInt>(N->getOperand(0)));
        baz(cast            <ConstantInt>(N->getOperand(1)));
        bak(cast_or_null    <ConstantInt>(N->getOperand(2)));
        bat(dyn_cast        <ConstantInt>(N->getOperand(3)));
        bay(dyn_cast_or_null<ConstantInt>(N->getOperand(4)));

    you can trivially match its semantics with:

        MDNode *N = foo();
        bar(mdconst::hasa               <ConstantInt>(N->getOperand(0)));
        baz(mdconst::extract            <ConstantInt>(N->getOperand(1)));
        bak(mdconst::extract_or_null    <ConstantInt>(N->getOperand(2)));
        bat(mdconst::dyn_extract        <ConstantInt>(N->getOperand(3)));
        bay(mdconst::dyn_extract_or_null<ConstantInt>(N->getOperand(4)));

    and when you transition your metadata schema to `MDInt`:

        MDNode *N = foo();
        bar(isa             <MDInt>(N->getOperand(0)));
        baz(cast            <MDInt>(N->getOperand(1)));
        bak(cast_or_null    <MDInt>(N->getOperand(2)));
        bat(dyn_cast        <MDInt>(N->getOperand(3)));
        bay(dyn_cast_or_null<MDInt>(N->getOperand(4)));

  - A `CallInst` -- specifically, intrinsic instructions -- can refer to
    metadata through a bridge called `MetadataAsValue`.  This is a
    subclass of `Value` where `getType()->isMetadataTy()`.

    `MetadataAsValue` is the *only* class that can legally refer to a
    `LocalAsMetadata`, which is a bridged form of non-`Constant` values
    like `Argument` and `Instruction`.  It can also refer to any other
    `Metadata` subclass.

(I'll break all your testcases in a follow-up commit, when I propagate
this change to assembly.)

llvm-svn: 223802
2014-12-09 18:38:53 +00:00
David Blaikie 5106ce7897 Remove StringMap::GetOrCreateValue in favor of StringMap::insert
Having two ways to do this doesn't seem terribly helpful and
consistently using the insert version (which we already has) seems like
it'll make the code easier to understand to anyone working with standard
data structures. (I also updated many references to the Entry's
key and value to use first() and second instead of getKey{Data,Length,}
and get/setValue - for similar consistency)

Also removes the GetOrCreateValue functions so there's less surface area
to StringMap to fix/improve/change/accommodate move semantics, etc.

llvm-svn: 222319
2014-11-19 05:49:42 +00:00
Duncan P. N. Exon Smith 9419863909 libLTO: Assert if LTOCodeGenerator and LTOModule are from different contexts
llvm-svn: 221730
2014-11-11 23:13:10 +00:00
Duncan P. N. Exon Smith 97b45874bf libLTO: Allow LTOModule to own a context
llvm-svn: 221728
2014-11-11 23:08:05 +00:00
Duncan P. N. Exon Smith de5e32b5b4 libLTO: Allow LTOCodeGenerator to own a context
llvm-svn: 221726
2014-11-11 23:03:29 +00:00
Arnold Schwaighofer eb1a38fa73 Add an option to the LTO code generator to disable vectorization during LTO
We used to always vectorize (slp and loop vectorize) in the LTO pass pipeline.

r220345 changed it so that we used the PassManager's fields 'LoopVectorize' and
'SLPVectorize' out of the desire to be able to disable vectorization using the
cl::opt flags 'vectorize-loops'/'slp-vectorize' which the before mentioned
fields default to.
Unfortunately, this turns off vectorization because those fields
default to false.
This commit adds flags to the LTO library to disable lto vectorization which
reconciles the desire to optionally disable vectorization during LTO and
the desired behavior of defaulting to enabled vectorization.

We really want tools to set PassManager flags directly to enable/disable
vectorization and not go the route via cl::opt flags *in*
PassManagerBuilder.cpp.

llvm-svn: 220652
2014-10-26 21:50:58 +00:00
Rafael Espindola d12b4a334b Update the error handling of lib/Linker.
Instead of passing a std::string&, use the new diagnostic infrastructure.

llvm-svn: 220608
2014-10-25 04:06:10 +00:00
Duncan P. N. Exon Smith f02fe70805 LTO: Document the Boolean argument from r218784
llvm-svn: 218907
2014-10-02 21:11:04 +00:00
Duncan P. N. Exon Smith 30c9242caa LTO: Ignore disabled diagnostic remarks
r206400 and r209442 added remarks that are disabled by default.
However, if a diagnostic handler is registered, the remarks are sent
unfiltered to the handler.  This is the right behaviour for clang, since
it has its own filters.

However, the diagnostic handler exposed in the LTO API receives only the
severity and message.  It doesn't have the information to filter by pass
name.  For LTO, disabled remarks should be filtered by the producer.

I've changed `LLVMContext::setDiagnosticHandler()` to take a `bool`
argument indicating whether to respect the built-in filters.  This
defaults to `false`, so other consumers don't have a behaviour change,
but `LTOCodeGenerator::setDiagnosticHandler()` sets it to `true`.

To make this behaviour testable, I added a `-use-diagnostic-handler`
command-line option to `llvm-lto`.

This fixes PR21108.

llvm-svn: 218784
2014-10-01 18:36:03 +00:00
Peter Collingbourne 10039c02ea LTO: introduce object file-based on-disk module format.
This format is simply a regular object file with the bitcode stored in a
section named ".llvmbc", plus any number of other (non-allocated) sections.

One immediate use case for this is to accommodate compilation processes
which expect the object file to contain metadata in non-allocated sections,
such as the ".go_export" section used by some Go compilers [1], although I
imagine that in the future we could consider compiling parts of the module
(such as large non-inlinable functions) directly into the object file to
improve LTO efficiency.

[1] http://golang.org/doc/install/gccgo#Imports

Differential Revision: http://reviews.llvm.org/D4371

llvm-svn: 218078
2014-09-18 21:28:49 +00:00
Rafael Espindola c435adcde0 Add doInitialization/doFinalization to DataLayoutPass.
With this a DataLayoutPass can be reused for multiple modules.

Once we have doInitialization/doFinalization, it doesn't seem necessary to pass
a Module to the constructor.

Overall this change seems in line with the idea of making DataLayout a required
part of Module. With it the only way of having a DataLayout used is to add it
to the Module.

llvm-svn: 217548
2014-09-10 21:27:43 +00:00
David Blaikie 78fdec5898 unique_ptrify LTOCodeGenerator::NativeObjectFile
llvm-svn: 216927
2014-09-02 18:21:06 +00:00
Rafael Espindola 3560ff2c1f Return a std::unique_ptr when creating a new MemoryBuffer.
llvm-svn: 216583
2014-08-27 20:03:13 +00:00
Craig Topper 3af9722529 Fix some cases were ArrayRefs were being passed by reference. Also remove 'const' from some other ArrayRef uses since its implicitly const already.
llvm-svn: 216524
2014-08-27 05:25:00 +00:00
Rafael Espindola d96d553d76 Pass a MemoryBufferRef when we can avoid taking ownership.
The attached patch simplifies a few interfaces that don't need to take
ownership of a buffer.

For example, both parseAssembly and parseBitcodeFile will parse the
entire buffer before returning. There is no need to take ownership.

Using a MemoryBufferRef makes it obvious in the type signature that
there is no ownership transfer.

llvm-svn: 216488
2014-08-26 21:49:01 +00:00
Rafael Espindola 2ce3882eaf Simplify LTOModule::makeLTOModule a bit. NFC.
Just call parseBitcodeFile instead of getLazyBitcodeModule followed by
materializeAllPermanently.

llvm-svn: 216461
2014-08-26 15:09:32 +00:00
Rafael Espindola 3fd1e9933f Modernize raw_fd_ostream's constructor a bit.
Take a StringRef instead of a "const char *".
Take a "std::error_code &" instead of a "std::string &" for error.

A create static method would be even better, but this patch is already a bit too
big.

llvm-svn: 216393
2014-08-25 18:16:47 +00:00
Rafael Espindola 7cebf36a95 Move some logic to populateLTOPassManager.
This will avoid code duplication in the next commit which calls it directly
from the gold plugin.

llvm-svn: 216211
2014-08-21 20:03:44 +00:00
Rafael Espindola 216e0c0617 Respect LibraryInfo in populateLTOPassManager and use it. NFC.
llvm-svn: 216203
2014-08-21 18:49:52 +00:00
Rafael Espindola e07caad9e7 Handle inlining in populateLTOPassManager like in populateModulePassManager.
No functionality change.

llvm-svn: 216178
2014-08-21 13:35:30 +00:00
Rafael Espindola 208bc533cd Move DisableGVNLoadPRE from populateLTOPassManager to PassManagerBuilder.
llvm-svn: 216174
2014-08-21 13:13:17 +00:00
Craig Topper 71b7b68b74 Repace SmallPtrSet with SmallPtrSetImpl in function arguments to avoid needing to mention the size.
llvm-svn: 216158
2014-08-21 05:55:13 +00:00
Aaron Ballman 474972585a Silencing a -Wcast-qual warning. NFC.
llvm-svn: 216068
2014-08-20 12:54:13 +00:00
Rafael Espindola 48af1c2a1a Don't own the buffer in object::Binary.
Owning the buffer is somewhat inflexible. Some Binaries have sub Binaries
(like Archive) and we had to create dummy buffers just to handle that. It is
also a bad fit for IRObjectFile where the Module wants to own the buffer too.

Keeping this ownership would make supporting IR inside native objects
particularly painful.

This patch focuses in lib/Object. If something elsewhere used to own an Binary,
now it also owns a MemoryBuffer.

This patch introduces a few new types.

* MemoryBufferRef. This is just a pair of StringRefs for the data and name.
  This is to MemoryBuffer as StringRef is to std::string.
* OwningBinary. A combination of Binary and a MemoryBuffer. This is needed
  for convenience functions that take a filename and return both the
  buffer and the Binary using that buffer.

The C api now uses OwningBinary to avoid any change in semantics. I will start
a new thread to see if we want to change it and how.

llvm-svn: 216002
2014-08-19 18:44:46 +00:00
Craig Topper 6230691c91 Revert "Repace SmallPtrSet with SmallPtrSetImpl in function arguments to avoid needing to mention the size."
Getting a weird buildbot failure that I need to investigate.

llvm-svn: 215870
2014-08-18 00:24:38 +00:00
Craig Topper 5229cfd163 Repace SmallPtrSet with SmallPtrSetImpl in function arguments to avoid needing to mention the size.
llvm-svn: 215868
2014-08-17 23:47:00 +00:00
Rafael Espindola a6d0b2f801 Return a std::uinque_ptr. Every caller was already using one.
llvm-svn: 215858
2014-08-17 22:37:39 +00:00
Rafael Espindola f9e52cf015 Don't internalize all but main by default.
This is mostly a cleanup, but it changes a fairly old behavior.

Every "real" LTO user was already disabling the silly internalize pass
and creating the internalize pass itself. The difference with this
patch is for "opt -std-link-opts" and the C api.

Now to get a usable behavior out of opt one doesn't need the funny
looking command line:

opt -internalize -disable-internalize -internalize-public-api-list=foo,bar -std-link-opts

llvm-svn: 214919
2014-08-05 20:10:38 +00:00
Eric Christopher d913448b38 Remove the TargetMachine forwards for TargetSubtargetInfo based
information and update all callers. No functional change.

llvm-svn: 214781
2014-08-04 21:25:23 +00:00
Rafael Espindola 293cbf6ffe Attempt at fixing the windows dll build.
It looks like only direct (i.e., explicitly listed) dependencies are
scanned.

llvm-svn: 214361
2014-07-30 23:39:30 +00:00
Rafael Espindola f21434ccb0 Refactor duplicated code.
llvm-svn: 214328
2014-07-30 19:42:16 +00:00
Rafael Espindola 3cf4af11d5 Add the missing hasLinkOnceODRLinkage predicate.
llvm-svn: 214312
2014-07-30 15:57:51 +00:00
Tim Northover e19bed7d33 AArch64: remove arm64 triple enumerator.
Having both Triple::arm64 and Triple::aarch64 is extremely confusing, and
invites bugs where only one is checked. In reality, the only legitimate
difference between the two (arm64 usually means iOS) is also present in the OS
part of the triple and that's what should be checked.

We still parse the "arm64" triple, just canonicalise it to Triple::aarch64, so
there aren't any LLVM-side test changes.

llvm-svn: 213743
2014-07-23 12:32:47 +00:00
Gerolf Hoflehner f27ae6cdcf MergedLoadStoreMotion pass
Merges equivalent loads on both sides of a hammock/diamond
and hoists into into the header.
Merges equivalent stores on both sides of a hammock/diamond
and sinks it to the footer.
Can enable if conversion and tolerate better load misses
and store operand latencies.

llvm-svn: 213396
2014-07-18 19:13:09 +00:00
NAKAMURA Takumi 04b8b37f56 Prune Redundant libdeps in CMake's target_link_libraries and LLVMBuild.txt.
I checked this with Release+Asserts on x86_64-mingw32. Please restore partially if this were overkill.

llvm-svn: 213064
2014-07-15 11:37:03 +00:00
Rafael Espindola adf21f2a56 Update the MemoryBuffer API to use ErrorOr.
llvm-svn: 212405
2014-07-06 17:43:13 +00:00
Rafael Espindola c75c4fad46 Revert "Convert a few std::strings to StringRef."
This reverts commit r212342.

We can get a StringRef into the current Record, but not one in the bitcode
itself since the string is compressed in it.

llvm-svn: 212356
2014-07-04 20:02:42 +00:00
Rafael Espindola 089a317c64 Ignore llvm specific symbols in the LTOModule.
These are the llvm.* globals and functions.

I don't think it is possible to test this directly since llvm-lto is not
a full linker and will not report duplicated symbols, but this fixes
bootstrap with gold and lto enabled.

llvm-svn: 212354
2014-07-04 19:31:27 +00:00
Rafael Espindola dddd1fd9f4 Implement LTOModule on top of IRObjectFile.
IRObjectFile provides all the logic for producing mangled names and getting
symbols from inline assembly.

LTOModule then adds logic for linking specific tasks, like constructing
llvm.compiler_user or extracting linker options from the bitcode.

The rule of the thumb is that IRObjectFile has the functionality that is
needed by both LTO and llvm-ar.

llvm-svn: 212349
2014-07-04 18:40:36 +00:00
Rafael Espindola 0972d41c73 Avoid mangling names twice. No functionality change.
llvm-svn: 212348
2014-07-04 16:37:02 +00:00
Rafael Espindola f98536a046 Convert a few std::strings to StringRef.
llvm-svn: 212342
2014-07-04 14:12:46 +00:00
Alp Toker be53eebe5a Fix prefix comparison from r212308
llvm-svn: 212310
2014-07-04 02:01:54 +00:00
Alp Toker ac90380b5e Sink undesirable LTO functions into the old C API
We want to encourage users of the C++ LTO API to reuse memory buffers instead
of repeatedly opening and reading the same file contents.

This reverts commit r212305 and implements a tidier scheme.

llvm-svn: 212308
2014-07-04 00:58:41 +00:00
Peter Collingbourne d7f75eeffc Modify LTOModule::isTargetMatch to take a StringRef instead of a MemoryBuffer.
llvm-svn: 212305
2014-07-03 23:49:28 +00:00
Peter Collingbourne 63086fe166 LTO: rename the various makeLTOModule overloads.
This rename makes it easier to identify the specific overload being called
in each particular case and makes future refactorings easier.

Differential Revision: http://reviews.llvm.org/D4370

llvm-svn: 212302
2014-07-03 23:28:00 +00:00
Rafael Espindola 2f0647cfdc Use std::unique_ptr to manage memory. No functionality change.
llvm-svn: 212299
2014-07-03 22:43:03 +00:00
Rafael Espindola c63c714ed1 LTO depends on Object now.
Fixes the build with only the ARM backend enabled. For some reason some
other backend was pulling Object and this went unnoticed.

llvm-svn: 212288
2014-07-03 20:19:03 +00:00
Rafael Espindola 13b69d63e6 Add support for inline asm symbols to IRObjectFile.
This also enables it in llvm-nm so that it can be tested.

llvm-svn: 212282
2014-07-03 18:59:23 +00:00
Rafael Espindola 05cef84640 Use a range loop. No functionality change.
llvm-svn: 211986
2014-06-28 18:44:59 +00:00
Alp Toker e69170a110 Revert "Introduce a string_ostream string builder facilty"
Temporarily back out commits r211749, r211752 and r211754.

llvm-svn: 211814
2014-06-26 22:52:05 +00:00
Alp Toker 614717388c Introduce a string_ostream string builder facilty
string_ostream is a safe and efficient string builder that combines opaque
stack storage with a built-in ostream interface.

small_string_ostream<bytes> additionally permits an explicit stack storage size
other than the default 128 bytes to be provided. Beyond that, storage is
transferred to the heap.

This convenient class can be used in most places an
std::string+raw_string_ostream pair or SmallString<>+raw_svector_ostream pair
would previously have been used, in order to guarantee consistent access
without byte truncation.

The patch also converts much of LLVM to use the new facility. These changes
include several probable bug fixes for truncated output, a programming error
that's no longer possible with the new interface.

llvm-svn: 211749
2014-06-26 00:00:48 +00:00
Rafael Espindola 591c641920 Merge the used symbol scanning of MCObjectStreamer and RecordStreamer.
This completes the refactoring of RecordStreamer.

llvm-svn: 211727
2014-06-25 18:37:33 +00:00
Rafael Espindola e2c6624475 Move expression visitation logic up to MCStreamer.
Remove the duplicate from MCRecordStreamer. No functionality change.

llvm-svn: 211714
2014-06-25 15:45:33 +00:00
Rafael Espindola 624ac24da9 Move some trivial methods up to MCStreamer.
This saves some duplicated boilerplate in RecordStreamer and NullStreamer.

llvm-svn: 211653
2014-06-25 00:27:53 +00:00
Rafael Espindola 49bbfd026a Simplify the handling of .cfi_endproc.
No functionality change.

llvm-svn: 211651
2014-06-25 00:13:59 +00:00
Rafael Espindola c607d8e7be Simplify EmitLabel.
All the "real" streamers were already calling to MCStreamer::EmitLabel
to do part of the work.

llvm-svn: 211646
2014-06-24 23:54:40 +00:00
Rafael Espindola f491704e22 Print a=b as an assignment.
In assembly the expression a=b is parsed as an assignment, so it should be
printed as one.

This remove a truly horrible hack for producing a label with "a=.". It would
be used by codegen but would never be reached by the asm parser. Sorry I
missed this when it was first committed.

llvm-svn: 211639
2014-06-24 22:45:16 +00:00
Rafael Espindola 70d3c20b0f Use the assignment operator.
No functionality change.

llvm-svn: 211319
2014-06-19 22:27:46 +00:00
Rafael Espindola a064b0c476 Set missing options in LTOCodeGenerator::setTargetOptions.
Patch by Tom Roeder, I just added the test.

llvm-svn: 211317
2014-06-19 22:14:12 +00:00
Rafael Espindola db4ed0bdab Remove 'using std::errro_code' from lib.
llvm-svn: 210871
2014-06-13 02:24:39 +00:00
Rafael Espindola 3acea39853 Don't use 'using std::error_code' in include/llvm.
This should make sure that most new uses use the std prefix.

llvm-svn: 210835
2014-06-12 21:46:39 +00:00
Rafael Espindola a6e9c3e43a Remove system_error.h.
This is a minimal change to remove the header. I will remove the occurrences
of "using std::error_code" in a followup patch.

llvm-svn: 210803
2014-06-12 17:38:55 +00:00
Alp Toker 17dd8efe9e Build fix: remove initializeJumpInstrTablesPass() call from LTO
This was incurring an unsatisfied dependency on CodeGen from LTO breaking
shared builds:

Undefined symbols for architecture x86_64:
  "llvm::initializeJumpInstrTablesPass(llvm::PassRegistry&)", referenced from:
      llvm::LTOCodeGenerator::initializeLTOPasses() in LTOCodeGenerator.cpp.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Removed as a temporary measure pending feedback from the author.

llvm-svn: 210400
2014-06-07 20:39:53 +00:00
Tom Roeder 44cb65fff1 Add a new attribute called 'jumptable' that creates jump-instruction tables for functions marked with this attribute.
It includes a pass that rewrites all indirect calls to jumptable functions to pass through these tables.

This also adds backend support for generating the jump-instruction tables on ARM and X86.
Note that since the jumptable attribute creates a second function pointer for a
function, any function marked with jumptable must also be marked with unnamed_addr.

llvm-svn: 210280
2014-06-05 19:29:43 +00:00
Rafael Espindola 64c1e18033 Allow alias to point to an arbitrary ConstantExpr.
This  patch changes GlobalAlias to point to an arbitrary ConstantExpr and it is
up to MC (or the system assembler) to decide if that expression is valid or not.

This reduces our ability to diagnose invalid uses and how early we can spot
them, but it also lets us do things like

@test5 = alias inttoptr(i32 sub (i32 ptrtoint (i32* @test2 to i32),
                                 i32 ptrtoint (i32* @bar to i32)) to i32*)

An important implication of this patch is that the notion of aliased global
doesn't exist any more. The alias has to encode the information needed to
access it in its metadata (linkage, visibility, type, etc).

Another consequence to notice is that getSection has to return a "const char *".
It could return a NullTerminatedStringRef if there was such a thing, but when
that was proposed the decision was to just uses "const char*" for that.

llvm-svn: 210062
2014-06-03 02:41:57 +00:00
Tim Northover 3b0846e8f7 AArch64/ARM64: move ARM64 into AArch64's place
This commit starts with a "git mv ARM64 AArch64" and continues out
from there, renaming the C++ classes, intrinsics, and other
target-local objects for consistency.

"ARM64" test directories are also moved, and tests that began their
life in ARM64 use an arm64 triple, those from AArch64 use an aarch64
triple. Both should be equivalent though.

This finishes the AArch64 merge, and everyone should feel free to
continue committing as normal now.

llvm-svn: 209577
2014-05-24 12:50:23 +00:00
Rafael Espindola 6314ad41d1 Aliases are always definition, delete dead code.
While at it, use a range loop.

llvm-svn: 209519
2014-05-23 15:18:06 +00:00
Rafael Espindola e0098928c9 Delete getAliasedGlobal.
llvm-svn: 209040
2014-05-16 22:37:03 +00:00
Rafael Espindola 05447dd278 Move EmitDwarfAdvanceLineAddr and EmitDwarfAdvanceFrameAddr to the obj streamer.
This lets us delete the MCAsmStreamer implementation. No functionality change.

llvm-svn: 208570
2014-05-12 14:43:25 +00:00
Duncan P. N. Exon Smith 87121f84d2 LTO: Check local linkage first
Since visibility is meaningless for symbols with local linkage, check
local linkage before visibility when setting symbol attributes.

When linkage is `internal` and the visibility is `hidden`, the exposed
attribute is now `LTO_SYMBOL_SCOPE_INTERNAL` instead of
`LTO_SYMBOL_SCOPE_HIDDEN`.  Although the bitfield allows *both* to be
specified, the combination is nonsense anyway.

Given changes (in progress) to drop visibility when a symbol has local
linkage, this almost has no functionality change: it's mostly a cleanup
to clarify the logic.

The exception is when something has `appending` linkage.  Before this
change, such symbols would be advertised as `LTO_SYMBOL_SCOPE_INTERNAL`;
now, they'll be given `LTO_SYMBOL_SCOPE_COMMON`.

Unfortunately this is really awkward to test.  This only changes what we
advertise to linkers (before running LTO), not what the final object
looks like.  In theory I could add `DEBUG` output to `llvm-lto` (and
test with "REQUIRES: asserts"), but follow-up commits to disallow
`internal hidden` simplify this anyway.

<rdar://problem/16141113>

llvm-svn: 208261
2014-05-07 22:53:14 +00:00
Rafael Espindola 9c8c96f08a Use a range loop.
llvm-svn: 207996
2014-05-05 20:06:41 +00:00
Rafael Espindola 9475117f5d Trivial simplification. No functionality change.
llvm-svn: 207967
2014-05-05 14:18:16 +00:00
Rafael Espindola 80df4bb10f Rename member variable to try to fix the bots.
llvm-svn: 207915
2014-05-03 15:28:13 +00:00
Tom Roeder fd1bc602b3 Add an -mattr option to the gold plugin to support subtarget features in LTO
This adds support for an -mattr option to the gold plugin and to llvm-lto. This
allows the caller to specify details of the subtarget architecture, like +aes,
or +ssse3 on x86.  Note that this requires a change to the include/llvm-c/lto.h
interface: it adds a function lto_codegen_set_attr and it increments the
version of the interface.

llvm-svn: 207279
2014-04-25 21:46:51 +00:00
Evgeniy Stepanov 0a951b775e Create MCTargetOptions.
For now it contains a single flag, SanitizeAddress, which enables
AddressSanitizer instrumentation of inline assembly.

Patch by Yuri Gorshenin.

llvm-svn: 206971
2014-04-23 11:16:03 +00:00
Kevin Enderby 96918bc406 Fix the assembler to print a better relocatable expression error
diagnostic that includes location information.

Currently if one has this assembly:

	.quad (0x1234 + (4 * SOME_VALUE))

where SOME_VALUE is undefined ones gets the less than
useful error message with no location information:

% clang -c x.s
clang -cc1as: fatal error: error in backend: expected relocatable expression

With this fix one now gets a more useful error message
with location information:

% clang -c x.s 
x.s:5:8: error: expected relocatable expression
 .quad (0x1234 + (4 * SOME_VALUE))
       ^

To do this I plumbed the SMLoc through the MCObjectStreamer
EmitValue() and EmitValueImpl() interfaces so it could be used
when creating the MCFixup.

rdar://12391022

llvm-svn: 206906
2014-04-22 17:27:29 +00:00
Alp Toker 9844434151 Remove some empty statements
Cleanup only.

llvm-svn: 206710
2014-04-19 23:56:35 +00:00
Duncan P. N. Exon Smith 6ef5f284d6 verify-di: Implement DebugInfoVerifier
Implement DebugInfoVerifier, which steals verification relying on
DebugInfoFinder from Verifier.

  - Adds LegacyDebugInfoVerifierPassPass, a ModulePass which wraps
    DebugInfoVerifier.  Uses -verify-di command-line flag.

  - Change verifyModule() to invoke DebugInfoVerifier as well as
    Verifier.

  - Add a call to createDebugInfoVerifierPass() wherever there was a
    call to createVerifierPass().

This implementation as a module pass should sidestep efficiency issues,
allowing us to turn debug info verification back on.

<rdar://problem/15500563>

llvm-svn: 206300
2014-04-15 16:27:38 +00:00