Commit Graph

70 Commits

Author SHA1 Message Date
Mehdi Amini 3c0e64c9d6 llvm-lto: run the module verifier when doing IR level work
It seems it was only running during CodeGen previously.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266846
2016-04-20 01:04:26 +00:00
Davide Italiano 1eea9bd86c [llvm-lto] Uniform error handling. NFC.
llvm-svn: 266255
2016-04-13 22:08:26 +00:00
Davide Italiano b10e893273 [llvm-lto] clang-format before working on this file.
llvm-svn: 266250
2016-04-13 21:41:35 +00:00
Teresa Johnson 3c35e0999b Clean up calls to WriteBitcodeToFile (NFC)
Remove a default parameter value being passed unnecessarily, which
also reduces the changes required when this parameter is changed in
D18763.

Document the remaining non-default bool value passed for another
parameter.

llvm-svn: 265348
2016-04-04 21:19:31 +00:00
Mehdi Amini 385cf28829 Rename ModuleSummaryIndex::modPathStringEntries() into modulePaths()
It now return the map instead of an iterator.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 264489
2016-03-26 03:35:38 +00:00
Mehdi Amini 155da5b132 Add a test for r263577: "Add missing error handling in llvm-lto"
On Rafael's suggestion!
(also fix a discrepancy between this error message format and the others)

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 263860
2016-03-19 00:17:32 +00:00
Mehdi Amini ef156c6701 Add missing error handling in llvm-lto
Annoyingly, ErrorOr allows to *not check* the error when things go
well. It will crash badly when there is an error though. It should
runtime assert when it is used without being checked!

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 263577
2016-03-15 20:17:55 +00:00
Teresa Johnson 26ab5772b0 [ThinLTO] Renaming of function index to module summary index (NFC)
(Resubmitting after fixing missing file issue)

With the changes in r263275, there are now more than just functions in
the summary. Completed the renaming of data structures (started in
r263275) to reflect the wider scope. In particular, changed the
FunctionIndex* data structures to ModuleIndex*, and renamed related
variables and comments. Also renamed the files to reflect the changes.

A companion clang patch will immediately succeed this patch to reflect
this renaming.

llvm-svn: 263513
2016-03-15 00:04:37 +00:00
Teresa Johnson cec0cae313 Revert "[ThinLTO] Renaming of function index to module summary index (NFC)"
This reverts commit r263490. Missed a file.

llvm-svn: 263493
2016-03-14 21:18:10 +00:00
Teresa Johnson 892920b358 [ThinLTO] Renaming of function index to module summary index (NFC)
With the changes in r263275, there are now more than just functions in
the summary. Completed the renaming of data structures (started in
r263275) to reflect the wider scope. In particular, changed the
FunctionIndex* data structures to ModuleIndex*, and renamed related
variables and comments. Also renamed the files to reflect the changes.

A companion clang patch will immediately succeed this patch to reflect
this renaming.

llvm-svn: 263490
2016-03-14 21:05:56 +00:00
Teresa Johnson 76a1c1d0ba [ThinLTO] Support for reference graph in per-module and combined summary.
Summary:
This patch adds support for including a full reference graph including
call graph edges and other GV references in the summary.

The reference graph edges can be used to make importing decisions
without materializing any source modules, can be used in the plugin
to make file staging decisions for distributed build systems, and is
expected to have other uses.

The call graph edges are recorded in each function summary in the
bitcode via a list of <CalleeValueIds, StaticCount> tuples when no PGO
data exists, or <CalleeValueId, StaticCount, ProfileCount> pairs when
there is PGO, where the ValueId can be mapped to the function GUID via
the ValueSymbolTable. In the function index in memory, the call graph
edges reference the target via the CalleeGUID instead of the
CalleeValueId.

The reference graph edges are recorded in each summary record with a
list of referenced value IDs, which can be mapped to value GUID via the
ValueSymbolTable.

Addtionally, a new summary record type is added to record references
from global variable initializers. A number of bitcode records and data
structures have been renamed to reflect the newly expanded scope of the
summary beyond functions. More cleanup will follow.

Reviewers: joker.eph, davidxl

Subscribers: joker.eph, llvm-commits

Differential Revision: http://reviews.llvm.org/D17212

llvm-svn: 263275
2016-03-11 18:52:24 +00:00
Mehdi Amini 0e83a809a6 Fix library dependency for llvm-lto after r262977
It is a transitive dependency, so static build are OK but not build
with individual DSO for each LLVM library.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 262987
2016-03-09 02:34:13 +00:00
Mehdi Amini 7c4a1a8d48 libLTO: add a ThinLTOCodeGenerator on the model of LTOCodeGenerator.
This is intended to provide a parallel (threaded) ThinLTO scheme
for linker plugin use through the libLTO C API.

The intent of this patch is to provide a first implementation as a
proof-of-concept and allows linker to start supporting ThinLTO by
definiing the libLTO C API. Some part of the libLTO API are left
unimplemented yet. Following patches will add support for these.

The current implementation can link all clang/llvm binaries.

Differential Revision: http://reviews.llvm.org/D17066

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 262977
2016-03-09 01:37:22 +00:00
Petr Pavlu 7ad9ec9fcf [LTO] Fix error reporting from lto_module_create_in_local_context()
Function lto_module_create_in_local_context() would previously
rely on the default LLVMContext being created for it by
LTOModule::makeLTOModule(). This context exits the program on
error and is not arranged to update sLastStringError in
tools/lto/lto.cpp.

Function lto_module_create_in_local_context() now creates an
LLVMContext by itself, sets it up correctly to its needs and then
passes it to LTOModule::createInLocalContext() which takes
ownership of the context and keeps it present for the lifetime of
the returned LTOModule.

Function LTOModule::makeLTOModule() is modified to take a
reference to LLVMContext (instead of a pointer) and no longer
creates a default context when nullptr is passed to it. Method
LTOModule::createInContext() that takes a pointer to LLVMContext
is removed because it allows to pass a nullptr to it. Instead
LTOModule::createFromBuffer() (that takes a reference to
LLVMContext) should be used.

Differential Revision: http://reviews.llvm.org/D17715

llvm-svn: 262330
2016-03-01 13:13:49 +00:00
Chris Bieneman e49730d4ba Remove autoconf support
Summary:
This patch is provided in preparation for removing autoconf on 1/26. The proposal to remove autoconf on 1/26 was discussed on the llvm-dev thread here: http://lists.llvm.org/pipermail/llvm-dev/2016-January/093875.html

"I felt a great disturbance in the [build system], as if millions of [makefiles] suddenly cried out in terror and were suddenly silenced. I fear something [amazing] has happened."
- Obi Wan Kenobi

Reviewers: chandlerc, grosbach, bob.wilson, tstellarAMD, echristo, whitequark

Subscribers: chfast, simoncook, emaste, jholewinski, tberghammer, jfb, danalbert, srhines, arsenm, dschuff, jyknight, dsanders, joker.eph, llvm-commits

Differential Revision: http://reviews.llvm.org/D16471

llvm-svn: 258861
2016-01-26 21:29:08 +00:00
Petr Pavlu eba3039238 [LTO] Fix error reporting when a file passed to libLTO is invalid or non-existent
This addresses PR26060 where function lto_module_create() could return nullptr
but lto_get_error_message() returned an empty string.

The error() call after LTOModule::createFromFile() in llvm-lto is then removed
because any error from this function should go through the diagnostic handler in
llvm-lto which will exit the program. The error() call was added because this
previously did not happen when the file was non-existent. This is fixed by the
patch. (The situation that llvm-lto reports an error when the input file does
not exist is tested by llvm/tools/llvm-lto/error.ll).

Differential Revision: http://reviews.llvm.org/D16106

llvm-svn: 258298
2016-01-20 09:03:42 +00:00
Tobias Edler von Koch 3f4f6f3ed6 Add a change accidentally left out from r258100
Also remove an executable bit introduced by r258083.

llvm-svn: 258101
2016-01-18 23:35:24 +00:00
Rafael Espindola 5a7756044f Add a missing error handling to llvm-lto.
llvm-svn: 257395
2016-01-11 22:08:22 +00:00
NAKAMURA Takumi 8293414718 Let llvm-lto installed. A few tests in clang/test are using it.
llvm-svn: 254963
2015-12-07 23:07:16 +00:00
Rafael Espindola a7612b4fac Modernize the C++ APIs for creating LTO modules.
This is a continuation of r253367.

These functions return is owned by the caller, so they return
std::unique_ptr now.

The call can fail, so the return is wrapped in ErrorOr.

They have a context where to report diagnostics, so they don't need to
take a string out parameter.

With this there are no call to getGlobalContext in lib/LTO.

llvm-svn: 254721
2015-12-04 16:14:31 +00:00
Rafael Espindola 7b8a24e5bb Move a call to getGlobalContext out of lib/LTO.
llvm-svn: 254696
2015-12-04 02:42:28 +00:00
Rafael Espindola 5e128dbcbf Simplify the error handling in llvm-lto a bit.
llvm-svn: 254675
2015-12-04 00:45:57 +00:00
Teresa Johnson 6b92316811 [ThinLTO] Deduplicate function index loading into shared helper (NFC)
Add a shared helper routine to read the function index from a file
and create/return the function index object. Use it in llvm-link and
llvm-lto.

llvm-svn: 253903
2015-11-23 19:19:11 +00:00
Teresa Johnson 6290dbc0f7 [ThinLTO] Handle bitcode without function summary sections gracefully
Summary:
Several fixes to the handling of bitcode files without function summary
sections so that they are skipped during ThinLTO processing in llvm-lto
and the gold plugin when appropriate instead of aborting.

1 Don't assert when trying to add a FunctionInfo that doesn't have
  a summary attached.
2 Skip FunctionInfo structures that don't have attached function summary
  sections when trying to create the combined function summary.
3 In both llvm-lto and gold-plugin, check whether a bitcode file has
  a function summary section before trying to parse the index, and skip
  the bitcode file if it does not.
4 Fix hasFunctionSummaryInMemBuffer in BitcodeReader, which had a bug
  where we returned to early while looking for the summary section.

Also added llvm-lto and gold-plugin based tests for cases where we
don't have function summaries in the bitcode file. I verified that
either the first couple fixes described above are enough to avoid the
crashes, or fixes 1,3,4. But have combined them all here for added
robustness.

Reviewers: joker.eph

Subscribers: llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D14903

llvm-svn: 253796
2015-11-21 21:55:48 +00:00
Tobias Edler von Koch 49c9a6e802 [LTO] Add options to llvm-lto to select output format and dump merged module
This introduces two new options:
- "llvm-lto -save-merged-module -o outfile" dumps the LTO Module to
  outfile.merged.bc prior to CodeGen and after LTO optimizations have been run.
- "llvm-lto -filetype=asm -o outfile" makes llvm-lto emit assembly instead of
  object code in outfile.

Both are intended for use in lit tests.

llvm-svn: 253624
2015-11-20 00:13:05 +00:00
Mehdi Amini 354f520fbc Do not require a Context to extract the FunctionIndex from Bitcode (NFC)
The LLVMContext was only used for Diagnostic. Pass a DiagnosticHandler
instead.

Differential Revision: http://reviews.llvm.org/D14794

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 253540
2015-11-19 05:52:29 +00:00
Yunzhong Gao 8e348cc732 Switch lto codegen to using diagnostic handlers.
This patch removes the std::string& argument from a number of C++ LTO API calls
and instead makes them use the installed diagnostic handler. This would also
improve consistency of diagnostic handling infrastructure: if an LTO client used
lto_codegen_set_diagnostic_handler() to install a custom error handler, we do
not want some error messages to go through the custom error handler, and some
other error messages to go into sLastErrorString.

llvm-svn: 253367
2015-11-17 19:48:12 +00:00
Yunzhong Gao ef436f068c llvm-lto: trivial spelling changes to distinguish custom diagnostic handler and
default diagnostic handler.

Differential Revision: http://reviews.llvm.org/D14520

llvm-svn: 252633
2015-11-10 18:52:48 +00:00
Teresa Johnson f72278f051 Clang format a few prior patches (NFC)
I had clang formatted my earlier patches using the wrong style.
Reformatted with the LLVM style.

llvm-svn: 251812
2015-11-02 18:02:11 +00:00
Teresa Johnson 3da931f87a Pass FunctionInfoIndex by reference to WriteFunctionSummaryToFile (NFC)
Implemented suggestion by dblakie in review for r250704.

llvm-svn: 250723
2015-10-19 19:06:06 +00:00
Teresa Johnson a80b50820b Fix required library for r250699 to BitWriter instead of BitReader.
This should fix the mingw3 bot failure.

llvm-svn: 250703
2015-10-19 15:21:46 +00:00
Teresa Johnson 91a88bba0e llvm-lto support for generating combined function indexes
Summary:
This patch adds support to llvm-lto that mirrors the support added by
r249270 to the gold plugin. This enables better testing of combined
index generation for ThinLTO.

Added a new test, and this support will be used in the test in D13515.

Reviewers: joker.eph

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D13847

llvm-svn: 250699
2015-10-19 14:30:44 +00:00
Duncan P. N. Exon Smith cff5feff6f Reapply "LTO: Disable extra verify runs in release builds"
This reverts commit r247730, effectively reapplying r247729.  This time
I have an lld commit ready to follow.

llvm-svn: 247735
2015-09-15 23:05:59 +00:00
Duncan P. N. Exon Smith 7de73e56a4 Revert "LTO: Disable extra verify runs in release builds"
This temporarily reverts commit r247729, as it caused lld build
failures.  I'll recommit once I have an lld patch ready-to-go.

llvm-svn: 247730
2015-09-15 22:47:38 +00:00
Duncan P. N. Exon Smith 236787838c LTO: Disable extra verify runs in release builds
The verifier currently runs three times in LTO: (1) after parsing, (2)
at the beginning of the optimization pipeline, and (3) at the end of it.

The first run is important, since we're not sure where the bitcode comes
from and it's nice to validate it, but in release builds the extra runs
aren't appropriate.

This commit:
  - Allows these runs to be disabled in LTOCodeGenerator.
  - Adds command-line options to llvm-lto.
  - Adds command-line options to libLTO.dylib, and disables the verifier
    by default in release builds (based on NDEBUG).

This shaves about 3.5% off the runtime of ld64 when linking
verify-uselistorder with -flto -g.

rdar://22509081

llvm-svn: 247729
2015-09-15 22:26:11 +00:00
Yunzhong Gao 46261a74db Add a non-exiting diagnostic handler for LTO.
This is in order to give LTO clients a chance to do some clean-up before
terminating the process.

llvm-svn: 247461
2015-09-11 20:01:53 +00:00
Peter Collingbourne c269ed5115 CodeGen: Introduce splitCodeGen and teach LTOCodeGenerator to use it.
llvm::splitCodeGen is a function that implements the core of parallel LTO
code generation. It uses llvm::SplitModule to split the module into linkable
partitions and spawning one code generation thread per partition. The function
produces multiple object files which can be linked in the usual way.

This has been threaded through to LTOCodeGenerator (and llvm-lto for testing
purposes). Separate patches will add parallel LTO support to the gold plugin
and lld.

Differential Revision: http://reviews.llvm.org/D12260

llvm-svn: 246236
2015-08-27 23:37:36 +00:00
Peter Collingbourne 9c8909dbd1 LTO: Simplify merged module ownership.
This change moves LTOCodeGenerator's ownership of the merged module to a
field of type std::unique_ptr<Module>. This helps simplify parts of the code
and clears the way for the module to be consumed by LLVM CodeGen (see D12132
review comments).

Differential Revision: http://reviews.llvm.org/D12205

llvm-svn: 245891
2015-08-24 22:22:53 +00:00
Peter Collingbourne 44ee84eec5 LTO: Change signature of LTOCodeGenerator::setCodePICModel() to take a Reloc::Model.
This allows us to remove a bunch of code in LTOCodeGenerator and llvm-lto
and has the side effect of improving error handling in the libLTO C API.

llvm-svn: 245756
2015-08-21 22:57:17 +00:00
Peter Collingbourne 5521743918 llvm-lto: Re-order code.
This saves us from needing to asave a pointer, and will be needed for an
upcoming ownership change.

llvm-svn: 245722
2015-08-21 19:09:42 +00:00
Sanjay Patel f9d009b755 add missing dependency on Target lib for tools
This was exposed by r238842 (which was reverted by r238900)
when doing a CMake build with -DBUILD_SHARED_LIBS=ON.

llvm-svn: 238953
2015-06-03 19:07:51 +00:00
Peter Collingbourne 3cc69d90f0 Make the C++ LTO API easier to use from C++ clients.
Start using C++ types such as StringRef and MemoryBuffer in the C++ LTO
API. In doing so, clarify the ownership of the native object file: the caller
now owns it, not the LTOCodeGenerator. The C libLTO library has been modified
to use a derived class of LTOCodeGenerator that owns the object file.

Differential Revision: http://reviews.llvm.org/D10114

llvm-svn: 238776
2015-06-01 20:08:30 +00:00
Peter Collingbourne 070843d60b libLTO, llvm-lto, gold: Introduce flag for controlling optimization level.
This change also introduces a link-time optimization level of 1. This
optimization level runs only the globaldce pass as well as cleanup passes for
passes that run at -O0, specifically simplifycfg which cleans up lowerbitsets.

http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20150316/266951.html

llvm-svn: 232769
2015-03-19 22:01:00 +00:00
Benjamin Kramer f044d3f93b Make helper functions static.
Found by -Wmissing-prototypes. NFC.

llvm-svn: 231664
2015-03-09 16:23:46 +00:00
Manman Ren 6487ce955a [LTO API] add lto_codegen_set_module to set the destination module.
When debugging LTO issues with ld64, we use -save-temps to save the merged
optimized bitcode file, then invoke ld64 again on the single bitcode file to
speed up debugging code generation passes and ld64 stuff after code generation.

llvm linking a single bitcode file via lto_codegen_add_module will generate a
different bitcode file from the single input. With the newly-added
lto_codegen_set_module, we can make sure the destination module is the same as
the input.

lto_codegen_set_module will transfer the ownship of the module to code
generator.

rdar://19024554

llvm-svn: 230290
2015-02-24 00:45:56 +00:00
Akira Hatanaka 23b5f6721d [llvm-lto] Add a line for setting LTOCodeGenerator's CPU string from command
line.

This is needed for a test case I plan to commit later.

llvm-svn: 227532
2015-01-30 01:14:28 +00:00
Duncan P. N. Exon Smith f9abf4fb0c llvm-lto: Add testing coverage for local contexts
Add coverage in `llvm-lto` for the API exposed by libLTO to create
modules in local contexts.

The goal here isn't to test the symbol-related API extensively, just to
confirm that these modules work at all.  (I'll be shifting code around
soon that should be NFC and I realized there was no test coverage.)

llvm-svn: 224408
2014-12-17 02:00:38 +00:00
Arnold Schwaighofer eb1a38fa73 Add an option to the LTO code generator to disable vectorization during LTO
We used to always vectorize (slp and loop vectorize) in the LTO pass pipeline.

r220345 changed it so that we used the PassManager's fields 'LoopVectorize' and
'SLPVectorize' out of the desire to be able to disable vectorization using the
cl::opt flags 'vectorize-loops'/'slp-vectorize' which the before mentioned
fields default to.
Unfortunately, this turns off vectorization because those fields
default to false.
This commit adds flags to the LTO library to disable lto vectorization which
reconciles the desire to optionally disable vectorization during LTO and
the desired behavior of defaulting to enabled vectorization.

We really want tools to set PassManager flags directly to enable/disable
vectorization and not go the route via cl::opt flags *in*
PassManagerBuilder.cpp.

llvm-svn: 220652
2014-10-26 21:50:58 +00:00
Rafael Espindola d12b4a334b Update the error handling of lib/Linker.
Instead of passing a std::string&, use the new diagnostic infrastructure.

llvm-svn: 220608
2014-10-25 04:06:10 +00:00
Duncan P. N. Exon Smith 30c9242caa LTO: Ignore disabled diagnostic remarks
r206400 and r209442 added remarks that are disabled by default.
However, if a diagnostic handler is registered, the remarks are sent
unfiltered to the handler.  This is the right behaviour for clang, since
it has its own filters.

However, the diagnostic handler exposed in the LTO API receives only the
severity and message.  It doesn't have the information to filter by pass
name.  For LTO, disabled remarks should be filtered by the producer.

I've changed `LLVMContext::setDiagnosticHandler()` to take a `bool`
argument indicating whether to respect the built-in filters.  This
defaults to `false`, so other consumers don't have a behaviour change,
but `LTOCodeGenerator::setDiagnosticHandler()` sets it to `true`.

To make this behaviour testable, I added a `-use-diagnostic-handler`
command-line option to `llvm-lto`.

This fixes PR21108.

llvm-svn: 218784
2014-10-01 18:36:03 +00:00