Commit Graph

131 Commits

Author SHA1 Message Date
Peter Collingbourne 78240e067c Fix build failure.
llvm-svn: 232771
2015-03-19 22:12:08 +00:00
Peter Collingbourne 070843d60b libLTO, llvm-lto, gold: Introduce flag for controlling optimization level.
This change also introduces a link-time optimization level of 1. This
optimization level runs only the globaldce pass as well as cleanup passes for
passes that run at -O0, specifically simplifycfg which cleans up lowerbitsets.

http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20150316/266951.html

llvm-svn: 232769
2015-03-19 22:01:00 +00:00
Benjamin Kramer 0a446fd56c Add missing includes. make_unique proliferated everywhere.
llvm-svn: 230909
2015-03-01 21:28:53 +00:00
Manman Ren 6487ce955a [LTO API] add lto_codegen_set_module to set the destination module.
When debugging LTO issues with ld64, we use -save-temps to save the merged
optimized bitcode file, then invoke ld64 again on the single bitcode file to
speed up debugging code generation passes and ld64 stuff after code generation.

llvm linking a single bitcode file via lto_codegen_add_module will generate a
different bitcode file from the single input. With the newly-added
lto_codegen_set_module, we can make sure the destination module is the same as
the input.

lto_codegen_set_module will transfer the ownship of the module to code
generator.

rdar://19024554

llvm-svn: 230290
2015-02-24 00:45:56 +00:00
Rafael Espindola a5ef4905a5 Fix duplicated symbol error.
llvm-svn: 228012
2015-02-03 19:25:53 +00:00
Manman Ren 8121e1db91 [LTO API] split lto_codegen_compile to lto_codegen_optimize and
lto_codegen_compile_optimized. Also add lto_api_version.

Before this commit, we can only dump the optimized bitcode after running
lto_codegen_compile, but it includes some impacts of running codegen passes,
one example is StackProtector pass. We will get assertion failure when running
llc on the optimized bitcode, because StackProtector is effectively run twice.

After splitting lto_codegen_compile, the linker can choose to dump the bitcode
before running lto_codegen_compile_optimized.

lto_api_version is added so ld64 can check for runtime-availability of the new
API.

rdar://19565500

llvm-svn: 228000
2015-02-03 18:39:15 +00:00
Michael J. Spencer 50a20c0e42 [lto] Disable dialog boxes on crash on Windows.
This has to be done in the DLL because the state doesn't cross DLL boundaries.

llvm-svn: 227471
2015-01-29 17:20:41 +00:00
Duncan P. N. Exon Smith d34b613b92 LTO: Export local context symbols
Export symbols in libLTO.dylib for the local context-related functions
added in r221733 (`LTO_API_VERSION=11`)... and add the missing
definition for `lto_codegen_create_in_local_context()`.

llvm-svn: 224567
2014-12-19 07:19:50 +00:00
Duncan P. N. Exon Smith c5800f68bd libLTO: Allow linker to choose context of modules and codegen
Add API for specifying which `LLVMContext` each `lto_module_t` and
`lto_code_gen_t` is in.

In particular, this enables the following flow:

    for (auto &File : Files) {
      lto_module_t M = lto_module_create_in_local_context(File...);
      querySymbols(M);
      lto_module_dispose(M);
    }

    lto_code_gen_t CG = lto_codegen_create_in_local_context();
    for (auto &File : FilesToLink) {
      lto_module_t M = lto_module_create_in_codegen_context(File..., CG);
      lto_codegen_add_module(CG, M);
      lto_module_dispose(M);
    }
    lto_codegen_compile(CG);
    lto_codegen_write_merged_modules(CG, ...);
    lto_codegen_dispose(CG);

This flow has a few benefits.

  - Only one module (two if you count the combined module in the code
    generator) is in memory at a time.

  - Metadata (and constants) from files that are parsed to query symbols
    but not linked into the code generator don't pollute the global
    context.

  - The first for loop can be parallelized, since each module is in its
    own context.

  - When the code generator is disposed, the memory from LTO gets freed.

rdar://problem/18767512

llvm-svn: 221733
2014-11-11 23:19:23 +00:00
Arnold Schwaighofer eb1a38fa73 Add an option to the LTO code generator to disable vectorization during LTO
We used to always vectorize (slp and loop vectorize) in the LTO pass pipeline.

r220345 changed it so that we used the PassManager's fields 'LoopVectorize' and
'SLPVectorize' out of the desire to be able to disable vectorization using the
cl::opt flags 'vectorize-loops'/'slp-vectorize' which the before mentioned
fields default to.
Unfortunately, this turns off vectorization because those fields
default to false.
This commit adds flags to the LTO library to disable lto vectorization which
reconciles the desire to optionally disable vectorization during LTO and
the desired behavior of defaulting to enabled vectorization.

We really want tools to set PassManager flags directly to enable/disable
vectorization and not go the route via cl::opt flags *in*
PassManagerBuilder.cpp.

llvm-svn: 220652
2014-10-26 21:50:58 +00:00
Rafael Espindola d12b4a334b Update the error handling of lib/Linker.
Instead of passing a std::string&, use the new diagnostic infrastructure.

llvm-svn: 220608
2014-10-25 04:06:10 +00:00
Rafael Espindola 77f1f8f170 Remove lto_codegen_set_attr.
It was never exported, so no functionality change.

llvm-svn: 214519
2014-08-01 14:57:05 +00:00
Rafael Espindola adf21f2a56 Update the MemoryBuffer API to use ErrorOr.
llvm-svn: 212405
2014-07-06 17:43:13 +00:00
Rafael Espindola d749fb5129 Change LTOModule`s getTargetTriple and setTargetTriple to use c++ types.
llvm-svn: 212343
2014-07-04 14:19:41 +00:00
Alp Toker ac90380b5e Sink undesirable LTO functions into the old C API
We want to encourage users of the C++ LTO API to reuse memory buffers instead
of repeatedly opening and reading the same file contents.

This reverts commit r212305 and implements a tidier scheme.

llvm-svn: 212308
2014-07-04 00:58:41 +00:00
Peter Collingbourne 63086fe166 LTO: rename the various makeLTOModule overloads.
This rename makes it easier to identify the specific overload being called
in each particular case and makes future refactorings easier.

Differential Revision: http://reviews.llvm.org/D4370

llvm-svn: 212302
2014-07-03 23:28:00 +00:00
Alp Toker ec9b42a907 Remove unused includes following r211294
llvm-svn: 211297
2014-06-19 19:25:49 +00:00
Rafael Espindola 77c50d2394 Use the c++ APIs.
No functionality change.

llvm-svn: 211294
2014-06-19 19:11:22 +00:00
Patrik Hagglund 9be9d8725b Fix gcc -pedantic warning in lto.cpp.
llvm-svn: 207959
2014-05-05 12:24:08 +00:00
Rafael Espindola 83ceb8edfb Move LTOModule and LTOCodeGenerator to the llvm namespace.
llvm-svn: 207911
2014-05-03 14:59:52 +00:00
Rafael Espindola b62e6b4535 Style update: don't duplicate comments, they were getting out of sync.
llvm-svn: 207909
2014-05-03 14:34:48 +00:00
Tom Roeder fd1bc602b3 Add an -mattr option to the gold plugin to support subtarget features in LTO
This adds support for an -mattr option to the gold plugin and to llvm-lto. This
allows the caller to specify details of the subtarget architecture, like +aes,
or +ssse3 on x86.  Note that this requires a change to the include/llvm-c/lto.h
interface: it adds a function lto_codegen_set_attr and it increments the
version of the interface.

llvm-svn: 207279
2014-04-25 21:46:51 +00:00
Duncan P. N. Exon Smith 4680f40d28 Revert "Reapply "LTO: add API to set strategy for -internalize""
This reverts commit r199244.

Conflicts:
	include/llvm-c/lto.h
	include/llvm/LTO/LTOCodeGenerator.h
	lib/LTO/LTOCodeGenerator.cpp

llvm-svn: 205471
2014-04-02 22:05:57 +00:00
Eli Bendersky f0f210052f Refactor TargetOptions initialization into a single place.
The same code (~20 lines) for initializing a TargetOptions object from CodeGen
cmdline flags is duplicated 4 times in 4 different tools. This patch moves it
into a utility function.

Since the CodeGen/CommandFlags.h file defines cl::opt flags in a header, it's
a bit of a touchy situation because we should only link them into tools. So this
patch puts the init function in the header.

llvm-svn: 201699
2014-02-19 17:09:35 +00:00
Manman Ren 03456a176d LTO API: add lto_module_create_from_memory_with_path.
This function adds an extra path argument to lto_module_create_from_memory.
The path argument will be passed to makeBuffer to make sure the MemoryBuffer
has a name and the created module has a module identifier.

This is mainly for emitting warning messages from the linker. When we emit
warning message on a module, we can use the module identifier.

rdar://15985737

llvm-svn: 201114
2014-02-10 23:26:14 +00:00
Yunzhong Gao a88d7abeb1 Adding new LTO APIs to parse metadata nodes and extract linker options and
dependent libraries from a bitcode module.

Differential Revision: http://llvm-reviews.chandlerc.com/D2343

llvm-svn: 199759
2014-01-21 18:31:27 +00:00
Quentin Colombet 5fa1f6f57a [LTO] Add a hook to map LLVM diagnostics into the clients of LTO.
Add a hook in the C API of LTO so that clients of the code generator can set
their own handler for the LLVM diagnostics.
The handler is defined like this:
typedef void (*lto_diagnostic_handler_t)(lto_codegen_diagnostic_severity_t
severity, const char *diag, void *ctxt)
- severity says how bad this is.
- diag is a string that contains the diagnostic message.
- ctxt is the registered context for this handler.

This hook is more general than the lto_get_error_message, since this function
keeps only the latest message and can only be queried when something went wrong
(no warning for instance).

<rdar://problem/15517596>

llvm-svn: 199338
2014-01-15 22:04:35 +00:00
Duncan P. N. Exon Smith 93be7c4fb3 Reapply "LTO: add API to set strategy for -internalize"
Reapply r199191, reverted in r199197 because it carelessly broke
Other/link-opts.ll.  The problem was that calling
createInternalizePass("main") would select
createInternalizePass(bool("main")) instead of
createInternalizePass(ArrayRef<const char *>("main")).  This commit
fixes the bug.

The original commit message follows.

Add API to LTOCodeGenerator to specify a strategy for the -internalize
pass.

This is a new attempt at Bill's change in r185882, which he reverted in
r188029 due to problems with the gold linker.  This puts the onus on the
linker to decide whether (and what) to internalize.

In particular, running internalize before outputting an object file may
change a 'weak' symbol into an internal one, even though that symbol
could be needed by an external object file --- e.g., with arclite.

This patch enables three strategies:

- LTO_INTERNALIZE_FULL: the default (and the old behaviour).
- LTO_INTERNALIZE_NONE: skip -internalize.
- LTO_INTERNALIZE_HIDDEN: only -internalize symbols with hidden
  visibility.

LTO_INTERNALIZE_FULL should be used when linking an executable.

Outputting an object file (e.g., via ld -r) is more complicated, and
depends on whether hidden symbols should be internalized.  E.g., for
ld -r, LTO_INTERNALIZE_NONE can be used when -keep_private_externs, and
LTO_INTERNALIZE_HIDDEN can be used otherwise.  However,
LTO_INTERNALIZE_FULL is inappropriate, since the output object file will
eventually need to link with others.

lto_codegen_set_internalize_strategy() sets the strategy for subsequent
calls to lto_codegen_write_merged_modules() and lto_codegen_compile*().

<rdar://problem/14334895>

llvm-svn: 199244
2014-01-14 18:52:17 +00:00
NAKAMURA Takumi 23c0ab53b2 Revert r199191, "LTO: add API to set strategy for -internalize"
Please update also Other/link-opts.ll, in next time.

llvm-svn: 199197
2014-01-14 09:40:18 +00:00
Duncan P. N. Exon Smith 43ea3478bf LTO: add API to set strategy for -internalize
Add API to LTOCodeGenerator to specify a strategy for the -internalize
pass.

This is a new attempt at Bill's change in r185882, which he reverted in
r188029 due to problems with the gold linker.  This puts the onus on the
linker to decide whether (and what) to internalize.

In particular, running internalize before outputting an object file may
change a 'weak' symbol into an internal one, even though that symbol
could be needed by an external object file --- e.g., with arclite.

This patch enables three strategies:

- LTO_INTERNALIZE_FULL: the default (and the old behaviour).
- LTO_INTERNALIZE_NONE: skip -internalize.
- LTO_INTERNALIZE_HIDDEN: only -internalize symbols with hidden
  visibility.

LTO_INTERNALIZE_FULL should be used when linking an executable.

Outputting an object file (e.g., via ld -r) is more complicated, and
depends on whether hidden symbols should be internalized.  E.g., for
ld -r, LTO_INTERNALIZE_NONE can be used when -keep_private_externs, and
LTO_INTERNALIZE_HIDDEN can be used otherwise.  However,
LTO_INTERNALIZE_FULL is inappropriate, since the output object file will
eventually need to link with others.

lto_codegen_set_internalize_strategy() sets the strategy for subsequent
calls to lto_codegen_write_merged_modules() and lto_codegen_compile*().

<rdar://problem/14334895>

llvm-svn: 199191
2014-01-14 06:37:26 +00:00
Chandler Carruth 07baed53e8 Re-sort #include lines again, prior to moving headers around.
llvm-svn: 199080
2014-01-13 08:04:33 +00:00
Rafael Espindola 282a47037b Use LTO_SYMBOL_SCOPE_DEFAULT_CAN_BE_HIDDEN instead of the "dso list".
There are two ways one could implement hiding of linkonce_odr symbols in LTO:
* LLVM tells the linker which symbols can be hidden if not used from native
  files.
* The linker tells LLVM which symbols are not used from other object files,
  but will be put in the dso symbol table if present.

GOLD's API is the second option. It was implemented almost 1:1 in llvm by
passing the list down to internalize.

LLVM already had partial support for the first option. It is also very similar
to how ld64 handles hiding these symbols when *not* doing LTO.

This patch then
* removes the APIs for the DSO list.
* marks LTO_SYMBOL_SCOPE_DEFAULT_CAN_BE_HIDDEN all linkonce_odr unnamed_addr
  global values and other linkonce_odr whose address is not used.
* makes the gold plugin responsible for handling the API mismatch.

llvm-svn: 193800
2013-10-31 20:51:58 +00:00
Reid Kleckner ddac15108a lto.h: Use lto_bool_t instead of int to restore the ABI
This reverts commit r193255 and instead creates an lto_bool_t typedef
that points to bool, _Bool, or unsigned char depending on what is
available.  Only recent versions of MSVC provide a stdbool.h header.

Reviewers: rafael.espindola

Differential Revision: http://llvm-reviews.chandlerc.com/D2019

llvm-svn: 193377
2013-10-24 22:26:04 +00:00
NAKAMURA Takumi b13d51c6eb llvm-c/lto.h: Avoid use of bool.
llvm-svn: 193255
2013-10-23 17:56:46 +00:00
Rafael Espindola cda2911caa Optimize linkonce_odr unnamed_addr functions during LTO.
Generalize the API so we can distinguish symbols that are needed just for a DSO
symbol table from those that are used from some native .o.

The symbols that are only wanted for the dso symbol table can be dropped if
llvm can prove every other dso has a copy (linkonce_odr) and the address is not
important (unnamed_addr).

llvm-svn: 191922
2013-10-03 18:29:09 +00:00
Rafael Espindola efa02d53ff Fix option parsing in the gold plugin.
This was broken when options were moved up in r191680. No test because this is
specific LLVMgold.so/libLTO.so.

Patch by Tom Roeder!

llvm-svn: 191829
2013-10-02 14:36:23 +00:00
Rafael Espindola 0b385c77f7 Move command line options to the users of libLTO. Fixes --enable-shared build.
Patch by Richard Sandiford.

llvm-svn: 191680
2013-09-30 16:39:19 +00:00
Peter Collingbourne 4ccf0f1bef Move LTO support library to a component, allowing it to be tested
more reliably across platforms.  Patch by Tom Roeder!

llvm-svn: 191343
2013-09-24 23:52:22 +00:00
Shuxin Yang 388b52c0ce Revert 189297, the original commit message is following.
----
  Add new API lto_codegen_compile_parallel().

  This API is proposed by Nick Kledzik. The semantic is:

  --------------------------------------------------------------------------
   Generate code for merged module into an array of native object files. On 
   success returns a pointer to an array of NativeObjectFile.  The count 
   parameter returns the number of elements in the array.  Each element is 
   a pointer/length for a generated mach-o/ELF buffer.  The buffer is owned
   by the lto_code_gen_t and will be freed when lto_codegen_dispose() is called, 
   or lto_codegen_compile() is called again. On failure, returns NULL 
   (check lto_get_error_message() for details).

   extern const struct  NativeObjectFile*
   lto_codegen_compile_parallel(lto_code_gen_t cg, size_t *count);
  ---------------------------------------------------------------------------

  This API is currently only called on OSX platform. Linux or other Unixes
using GNU gold are not supposed to call this function, because on these systems, 
object files are fed back to linker via disk file instead of memory buffer.

  In this commit, lto_codegen_compile_parallel() simply calls
lto_codegen_compile() to return a single object file. In the near future,
this function is the entry point for compilation with partition. Linker can
blindly call this function even if partition is turned off; in this case,
compiler will return only one object file.

llvm-svn: 189386
2013-08-27 17:15:54 +00:00
Shuxin Yang a44362e4e9 Add new API lto_codegen_compile_parallel().
This API is proposed by Nick Kledzik. The semantic is:

  --------------------------------------------------------------------------
   Generate code for merged module into an array of native object files. On 
   success returns a pointer to an array of NativeObjectFile.  The count 
   parameter returns the number of elements in the array.  Each element is 
   a pointer/length for a generated mach-o/ELF buffer.  The buffer is owned
   by the lto_code_gen_t and will be freed when lto_codegen_dispose() is called, 
   or lto_codegen_compile() is called again. On failure, returns NULL 
   (check lto_get_error_message() for details).

   extern const struct  NativeObjectFile*
   lto_codegen_compile_parallel(lto_code_gen_t cg, size_t *count);
  ---------------------------------------------------------------------------

  This API is currently only called on OSX platform. Linux or other Unixes
using GNU gold are not supposed to call this function, because on these systems, 
object files are fed back to linker via disk file instead of memory buffer.

  In this commit, lto_codegen_compile_parallel() simply calls
lto_codegen_compile() to return a single object file. In the near future,
this function is the entry point for compilation with partition. Linker can
blindly call this function even if partition is turned off; in this case,
compiler will return only one object file.

llvm-svn: 189297
2013-08-27 00:03:23 +00:00
Shuxin Yang 1826ae23ff Revert r188188 and r188200.
In order to appease people (in Apple) who accuse me for committing "huge change" (?) without proper review. 

Thank Eric for fixing a compile-warning. 

llvm-svn: 188204
2013-08-12 21:07:31 +00:00
Shuxin Yang 76d082b5cd Misc enhancements to LTO:
1. Add some helper classes for partitions. They are designed in a
     way such that the top-level LTO driver will not see much difference 
     with or without partitioning.

  2. Introduce work-dir. Now all intermediate files generated during 
     LTO phases will be saved under work-dir. User can specify the workdir
     via -lto-workdir=/path/to/dir. By default the work-dir will be 
     erased before linker exit. To keep the workdir, do -lto-keep, or -lto-keep=1.

    TODO: Erase the workdir, if the linker exit prematurely.  
      We are currently not able to remove directory on signal. The support 
      routines simply ignore directory.

  3. Add one new API lto_codegen_get_files_need_remove().
     Linker and LTO plugin will communicate via this API about which files
    (including directories) need to removed before linker exit.

llvm-svn: 188188
2013-08-12 18:29:43 +00:00
Shuxin Yang b6696a93c2 Change public functions of LTOCodeGenerator from ret-false-on-succ to ret-true-on-succ.
As of this revision, all functions of LTOCodeGenerator are consistent in
ret-true-on-succ.

Tested on multiple OSes.

llvm-svn: 187864
2013-08-07 05:19:23 +00:00
Rafael Espindola 3d2ac2e41a Split getOpenFile into getOpenFile and getOpenFileSlice.
The main observation is that we never need both the filesize and the map size.
When mapping a slice of a file, it doesn't make sense to request a null
terminator and that would be the only case where the filesize would be used.

There are other cleanups that should be done in this area:

* A client should not have to pass the size (even an explicit -1) to say if
  it wants a null terminator or not, so we should probably swap the argument
  order.
* The default should be to not require a null terminator. Very few clients
  require this, but many end up asking for it just because it is the default.

llvm-svn: 186984
2013-07-23 20:25:01 +00:00
Bill Wendling 4a8fc8f271 Revert r169656.
The linker will call `lto_codegen_add_must_preserve_symbol' on all globals that
should be kept around. The linker will pretend that a dylib is being created.
<rdar://problem/12528059>

llvm-svn: 169770
2012-12-10 21:33:45 +00:00
Bill Wendling 65a6ee11dd Add the `lto_codegen_set_export_dynamic' function.
This function sets the `_exportDynamic' ivar. When that's set, we export all
symbols (e.g. we don't run the internalize pass). This is equivalent to the
`--export-dynamic' linker flag in GNU land:

--export-dynamic
  When creating a dynamically linked executable, add all symbols to the dynamic
  symbol table. The dynamic symbol table is the set of symbols which are visible
  from dynamic objects at run time. If you do not use this option, the dynamic
  symbol table will normally contain only those symbols which are referenced by
  some dynamic object mentioned in the link. If you use dlopen to load a dynamic
  object which needs to refer back to the symbols defined by the program, rather
  than some other dynamic object, then you will probably need to use this option
  when linking the program itself.

The Darwin linker will support this via the `-export_dynamic' flag. We should
modify clang to support this via the `-rdynamic' flag.

llvm-svn: 169656
2012-12-08 00:18:16 +00:00
Chandler Carruth 4d88a1c233 Sort the #include lines for tools/...
Again, tools are trickier to pick the main module header for than
library source files. I've started to follow the pattern of using
LLVMContext.h when it is included as a stub for program source files.

llvm-svn: 169252
2012-12-04 10:44:52 +00:00
Rafael Espindola 0ecb8faa7c Remove lto_codegen_set_whole_program_optimization. It is a work in progress,
so we don't want it to show up in the stable 3.1 interface.

While at it, add a comment about why LTOCodeGenerator manually creates the
internalize pass.

llvm-svn: 154807
2012-04-16 10:58:38 +00:00
Bill Wendling deffc42d63 s/lto_codegen_whole_program_optimization/lto_codegen_set_whole_program_optimization/
llvm-svn: 154312
2012-04-09 08:32:21 +00:00
Bill Wendling 8a49d049e1 Add a hook to turn on the internalize pass through the LTO interface.
llvm-svn: 154306
2012-04-09 05:26:48 +00:00