Summary:
The motivation is to support better the -object_path_lto option on
Darwin. The linker needs to write down the generate object files on
disk for later use by lldb or dsymutil (debug info are not present
in the final binary). We're moving this into libLTO so that we can
be smarter when a cache is enabled and hard-link when possible
instead of duplicating the files.
Reviewers: tejohnson, deadalnix, pcc
Subscribers: dexonsmith, llvm-commits
Differential Revision: https://reviews.llvm.org/D27507
llvm-svn: 289631
The linker supports a feature to force load an object from a static
archive if it defines an Objective-C category.
This API supports this feature by looking at every section in the
module to find if a category is defined in the module.
llvm-svn: 275125
This control how the cache is pruned. The cache still has to
be explicitely enabled/disabled by providing a path.
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 268393
This allows the linker to instruct ThinLTO to perform only the
optimization part or only the codegen part of the process.
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 265113
This is intended to provide a parallel (threaded) ThinLTO scheme
for linker plugin use through the libLTO C API.
The intent of this patch is to provide a first implementation as a
proof-of-concept and allows linker to start supporting ThinLTO by
definiing the libLTO C API. Some part of the libLTO API are left
unimplemented yet. Following patches will add support for these.
The current implementation can link all clang/llvm binaries.
Differential Revision: http://reviews.llvm.org/D17066
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 262977
This change moves LTOCodeGenerator's ownership of the merged module to a
field of type std::unique_ptr<Module>. This helps simplify parts of the code
and clears the way for the module to be consumed by LLVM CodeGen (see D12132
review comments).
Differential Revision: http://reviews.llvm.org/D12205
llvm-svn: 245891
This is needed for COFF linkers to distinguish between weak external aliases
and regular symbols with LLVM weak linkage, which are represented as strong
symbols in COFF.
llvm-svn: 241389
Specifically, remove the dependent library interface and replace the existing
linker option interface with a new one that returns a single list of flags.
Differential Revision: http://reviews.llvm.org/D10820
llvm-svn: 241018
This change unifies how LTOModule and the backend obtain linker flags
for globals: via a new TargetLoweringObjectFile member function named
emitLinkerFlagsForGlobal. A new function LTOModule::getLinkerOpts() returns
the list of linker flags as a single concatenated string.
This change affects the C libLTO API: the function lto_module_get_*deplibs now
exposes an empty list, and lto_module_get_*linkeropts exposes a single element
which combines the contents of all observed flags. libLTO should never have
tried to parse the linker flags; it is the linker's job to do so. Because
linkers will need to be able to parse flags in regular object files, it
makes little sense for libLTO to have a redundant mechanism for doing so.
The new API is compatible with the old one. It is valid for a user to specify
multiple linker flags in a single pragma directive like this:
#pragma comment(linker, "/defaultlib:foo /defaultlib:bar")
The previous implementation would not have exposed
either flag via lto_module_get_*deplibs (as the test in
TargetLoweringObjectFileCOFF::getDepLibFromLinkerOpt was case sensitive)
and would have exposed "/defaultlib:foo /defaultlib:bar" as a single flag via
lto_module_get_*linkeropts. This may have been a bug in the implementation,
but it does give us a chance to fix the interface.
Differential Revision: http://reviews.llvm.org/D10548
llvm-svn: 241010
Reverse libLTO's default behaviour for preserving use-list order in
bitcode, and add API for controlling it. The default setting is now
`false` (don't preserve them), which is consistent with `clang`'s
default behaviour.
Users of libLTO should call `lto_codegen_should_embed_uselists(CG,true)`
prior to calling `lto_codegen_write_merged_modules()` whenever the
output file isn't part of the production workflow in order to reproduce
results with subsequent calls to `llc`.
(I haven't added tests since `llvm-lto` (the test tool for LTO) doesn't
support bitcode output, and even if it did: there isn't actually a good
way to test whether a tool has passed the flag. If the order is already
"natural" (if the order will already round-trip) then no use-list
directives are emitted at all. At some point I'll circle back to add
tests to `llvm-as` (etc.) that they actually respect the flag, at which
point I can somehow add a test here as well.)
llvm-svn: 235943
These look like copy/paste errors, and shouldn't have the "prior to"
qualifier. Each API was introduced at the given values of
`LTO_API_VERSION`. The "prior to" in other doxygen comments is because
I couldn't easily differentiate between versions 1 and 2 when I added
these comments.
llvm-svn: 235925
When debugging LTO issues with ld64, we use -save-temps to save the merged
optimized bitcode file, then invoke ld64 again on the single bitcode file.
The saved bitcode file is already internalized, so we can call
lto_codegen_set_should_internalize and skip running internalization again.
rdar://20227235
llvm-svn: 235211
When debugging LTO issues with ld64, we use -save-temps to save the merged
optimized bitcode file, then invoke ld64 again on the single bitcode file to
speed up debugging code generation passes and ld64 stuff after code generation.
llvm linking a single bitcode file via lto_codegen_add_module will generate a
different bitcode file from the single input. With the newly-added
lto_codegen_set_module, we can make sure the destination module is the same as
the input.
lto_codegen_set_module will transfer the ownship of the module to code
generator.
rdar://19024554
llvm-svn: 230290
lto_codegen_compile_optimized. Also add lto_api_version.
Before this commit, we can only dump the optimized bitcode after running
lto_codegen_compile, but it includes some impacts of running codegen passes,
one example is StackProtector pass. We will get assertion failure when running
llc on the optimized bitcode, because StackProtector is effectively run twice.
After splitting lto_codegen_compile, the linker can choose to dump the bitcode
before running lto_codegen_compile_optimized.
lto_api_version is added so ld64 can check for runtime-availability of the new
API.
rdar://19565500
llvm-svn: 228000
Add API for specifying which `LLVMContext` each `lto_module_t` and
`lto_code_gen_t` is in.
In particular, this enables the following flow:
for (auto &File : Files) {
lto_module_t M = lto_module_create_in_local_context(File...);
querySymbols(M);
lto_module_dispose(M);
}
lto_code_gen_t CG = lto_codegen_create_in_local_context();
for (auto &File : FilesToLink) {
lto_module_t M = lto_module_create_in_codegen_context(File..., CG);
lto_codegen_add_module(CG, M);
lto_module_dispose(M);
}
lto_codegen_compile(CG);
lto_codegen_write_merged_modules(CG, ...);
lto_codegen_dispose(CG);
This flow has a few benefits.
- Only one module (two if you count the combined module in the code
generator) is in memory at a time.
- Metadata (and constants) from files that are parsed to query symbols
but not linked into the code generator don't pollute the global
context.
- The first for loop can be parallelized, since each module is in its
own context.
- When the code generator is disposed, the memory from LTO gets freed.
rdar://problem/18767512
llvm-svn: 221733
This adds support for an -mattr option to the gold plugin and to llvm-lto. This
allows the caller to specify details of the subtarget architecture, like +aes,
or +ssse3 on x86. Note that this requires a change to the include/llvm-c/lto.h
interface: it adds a function lto_codegen_set_attr and it increments the
version of the interface.
llvm-svn: 207279
A 'remark' is information that is not an error or a warning, but rather some
additional information provided to the user. In contrast to a 'note' a 'remark'
is an independent diagnostic, whereas a 'note' always depends on another
diagnostic.
A typical use case for remark nodes is information provided to the user, e.g.
information provided by the vectorizer about loops that have been vectorized.
llvm-svn: 202474
This function adds an extra path argument to lto_module_create_from_memory.
The path argument will be passed to makeBuffer to make sure the MemoryBuffer
has a name and the created module has a module identifier.
This is mainly for emitting warning messages from the linker. When we emit
warning message on a module, we can use the module identifier.
rdar://15985737
llvm-svn: 201114
Adding a doxygen comment for each bit of API to indicate at which
LTO_API_VERSION each was available, manually gleaned from successive
git-blames. A few notes:
- LTO_API_VERSION was set to 3 at its introduction.
- I've indicated all the API introduced before LTO_API_VERSION was
around as available "prior to LTO_API_VERSION=3".
- A number of API changes neglected to bump LTO_API_VERSION. These I've
indicated as available at the *next* bump of LTO_API_VERSION.
llvm-svn: 199429
Add a hook in the C API of LTO so that clients of the code generator can set
their own handler for the LLVM diagnostics.
The handler is defined like this:
typedef void (*lto_diagnostic_handler_t)(lto_codegen_diagnostic_severity_t
severity, const char *diag, void *ctxt)
- severity says how bad this is.
- diag is a string that contains the diagnostic message.
- ctxt is the registered context for this handler.
This hook is more general than the lto_get_error_message, since this function
keeps only the latest message and can only be queried when something went wrong
(no warning for instance).
<rdar://problem/15517596>
llvm-svn: 199338
Reapply r199191, reverted in r199197 because it carelessly broke
Other/link-opts.ll. The problem was that calling
createInternalizePass("main") would select
createInternalizePass(bool("main")) instead of
createInternalizePass(ArrayRef<const char *>("main")). This commit
fixes the bug.
The original commit message follows.
Add API to LTOCodeGenerator to specify a strategy for the -internalize
pass.
This is a new attempt at Bill's change in r185882, which he reverted in
r188029 due to problems with the gold linker. This puts the onus on the
linker to decide whether (and what) to internalize.
In particular, running internalize before outputting an object file may
change a 'weak' symbol into an internal one, even though that symbol
could be needed by an external object file --- e.g., with arclite.
This patch enables three strategies:
- LTO_INTERNALIZE_FULL: the default (and the old behaviour).
- LTO_INTERNALIZE_NONE: skip -internalize.
- LTO_INTERNALIZE_HIDDEN: only -internalize symbols with hidden
visibility.
LTO_INTERNALIZE_FULL should be used when linking an executable.
Outputting an object file (e.g., via ld -r) is more complicated, and
depends on whether hidden symbols should be internalized. E.g., for
ld -r, LTO_INTERNALIZE_NONE can be used when -keep_private_externs, and
LTO_INTERNALIZE_HIDDEN can be used otherwise. However,
LTO_INTERNALIZE_FULL is inappropriate, since the output object file will
eventually need to link with others.
lto_codegen_set_internalize_strategy() sets the strategy for subsequent
calls to lto_codegen_write_merged_modules() and lto_codegen_compile*().
<rdar://problem/14334895>
llvm-svn: 199244
Add API to LTOCodeGenerator to specify a strategy for the -internalize
pass.
This is a new attempt at Bill's change in r185882, which he reverted in
r188029 due to problems with the gold linker. This puts the onus on the
linker to decide whether (and what) to internalize.
In particular, running internalize before outputting an object file may
change a 'weak' symbol into an internal one, even though that symbol
could be needed by an external object file --- e.g., with arclite.
This patch enables three strategies:
- LTO_INTERNALIZE_FULL: the default (and the old behaviour).
- LTO_INTERNALIZE_NONE: skip -internalize.
- LTO_INTERNALIZE_HIDDEN: only -internalize symbols with hidden
visibility.
LTO_INTERNALIZE_FULL should be used when linking an executable.
Outputting an object file (e.g., via ld -r) is more complicated, and
depends on whether hidden symbols should be internalized. E.g., for
ld -r, LTO_INTERNALIZE_NONE can be used when -keep_private_externs, and
LTO_INTERNALIZE_HIDDEN can be used otherwise. However,
LTO_INTERNALIZE_FULL is inappropriate, since the output object file will
eventually need to link with others.
lto_codegen_set_internalize_strategy() sets the strategy for subsequent
calls to lto_codegen_write_merged_modules() and lto_codegen_compile*().
<rdar://problem/14334895>
llvm-svn: 199191
There are two ways one could implement hiding of linkonce_odr symbols in LTO:
* LLVM tells the linker which symbols can be hidden if not used from native
files.
* The linker tells LLVM which symbols are not used from other object files,
but will be put in the dso symbol table if present.
GOLD's API is the second option. It was implemented almost 1:1 in llvm by
passing the list down to internalize.
LLVM already had partial support for the first option. It is also very similar
to how ld64 handles hiding these symbols when *not* doing LTO.
This patch then
* removes the APIs for the DSO list.
* marks LTO_SYMBOL_SCOPE_DEFAULT_CAN_BE_HIDDEN all linkonce_odr unnamed_addr
global values and other linkonce_odr whose address is not used.
* makes the gold plugin responsible for handling the API mismatch.
llvm-svn: 193800
This reverts commit r193255 and instead creates an lto_bool_t typedef
that points to bool, _Bool, or unsigned char depending on what is
available. Only recent versions of MSVC provide a stdbool.h header.
Reviewers: rafael.espindola
Differential Revision: http://llvm-reviews.chandlerc.com/D2019
llvm-svn: 193377
Generalize the API so we can distinguish symbols that are needed just for a DSO
symbol table from those that are used from some native .o.
The symbols that are only wanted for the dso symbol table can be dropped if
llvm can prove every other dso has a copy (linkonce_odr) and the address is not
important (unnamed_addr).
llvm-svn: 191922
----
Add new API lto_codegen_compile_parallel().
This API is proposed by Nick Kledzik. The semantic is:
--------------------------------------------------------------------------
Generate code for merged module into an array of native object files. On
success returns a pointer to an array of NativeObjectFile. The count
parameter returns the number of elements in the array. Each element is
a pointer/length for a generated mach-o/ELF buffer. The buffer is owned
by the lto_code_gen_t and will be freed when lto_codegen_dispose() is called,
or lto_codegen_compile() is called again. On failure, returns NULL
(check lto_get_error_message() for details).
extern const struct NativeObjectFile*
lto_codegen_compile_parallel(lto_code_gen_t cg, size_t *count);
---------------------------------------------------------------------------
This API is currently only called on OSX platform. Linux or other Unixes
using GNU gold are not supposed to call this function, because on these systems,
object files are fed back to linker via disk file instead of memory buffer.
In this commit, lto_codegen_compile_parallel() simply calls
lto_codegen_compile() to return a single object file. In the near future,
this function is the entry point for compilation with partition. Linker can
blindly call this function even if partition is turned off; in this case,
compiler will return only one object file.
llvm-svn: 189386
This API is proposed by Nick Kledzik. The semantic is:
--------------------------------------------------------------------------
Generate code for merged module into an array of native object files. On
success returns a pointer to an array of NativeObjectFile. The count
parameter returns the number of elements in the array. Each element is
a pointer/length for a generated mach-o/ELF buffer. The buffer is owned
by the lto_code_gen_t and will be freed when lto_codegen_dispose() is called,
or lto_codegen_compile() is called again. On failure, returns NULL
(check lto_get_error_message() for details).
extern const struct NativeObjectFile*
lto_codegen_compile_parallel(lto_code_gen_t cg, size_t *count);
---------------------------------------------------------------------------
This API is currently only called on OSX platform. Linux or other Unixes
using GNU gold are not supposed to call this function, because on these systems,
object files are fed back to linker via disk file instead of memory buffer.
In this commit, lto_codegen_compile_parallel() simply calls
lto_codegen_compile() to return a single object file. In the near future,
this function is the entry point for compilation with partition. Linker can
blindly call this function even if partition is turned off; in this case,
compiler will return only one object file.
llvm-svn: 189297
In order to appease people (in Apple) who accuse me for committing "huge change" (?) without proper review.
Thank Eric for fixing a compile-warning.
llvm-svn: 188204
1. Add some helper classes for partitions. They are designed in a
way such that the top-level LTO driver will not see much difference
with or without partitioning.
2. Introduce work-dir. Now all intermediate files generated during
LTO phases will be saved under work-dir. User can specify the workdir
via -lto-workdir=/path/to/dir. By default the work-dir will be
erased before linker exit. To keep the workdir, do -lto-keep, or -lto-keep=1.
TODO: Erase the workdir, if the linker exit prematurely.
We are currently not able to remove directory on signal. The support
routines simply ignore directory.
3. Add one new API lto_codegen_get_files_need_remove().
Linker and LTO plugin will communicate via this API about which files
(including directories) need to removed before linker exit.
llvm-svn: 188188
Mehdi Amini