Commit Graph

18 Commits

Author SHA1 Message Date
Teresa Johnson ec544c552e [ThinLTO] Default backend threads to heavyweight_hardware_concurrency
Summary:
Changes default backend parallelism from thread::hardware_concurrency to
the new llvm::heavyweight_hardware_concurrency, which for X86 Linux
defaults to the number of physical cores (and will fall back to
thread::hardware_concurrency otherwise). This avoid oversubscribing
the physical cores using hyperthreading.

Reviewers: mehdi_amini, pcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D25775

llvm-svn: 284618
2016-10-19 17:35:01 +00:00
Teresa Johnson 2f159a265a [ThinLTO] Fix bot failure due to unused variable with NDEBUG
Put variable only used in assert under #ifndef NDEBUG.

This should fix bot failure at
http://lab.llvm.org:8011/builders/lld-x86_64-darwin13/builds/28537

llvm-svn: 284039
2016-10-12 20:06:02 +00:00
Teresa Johnson 4b9b379172 [ThinLTO] Don't link module level assembly when importing
Module inline asm was always being linked/concatenated
when running the IRLinker. This is correct for full LTO but not when
we are importing for ThinLTO, as it can result in multiply defined
symbols when the module asm defines a global symbol.

In order to test with llvm-lto2, I had to work around PR30396,
where a symbol that is defined in module assembly but defined in the
LLVM IR appears twice. Added workaround to llvm-lto2 with a FIXME.

Fixes PR30610.

Reviewers: mehdi_amini

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D25359

llvm-svn: 284030
2016-10-12 18:39:29 +00:00
Peter Collingbourne 80186a57d6 LTO: Simplify caching interface.
The NativeObjectOutput class has a design problem: it mixes up the caching
policy with the interface for output streams, which makes the client-side
code hard to follow and would for example make it harder to replace the
cache implementation in an arbitrary client.

This change separates the two aspects by moving the caching policy
to a separate field in Config, replacing NativeObjectOutput with a
NativeObjectStream class which only deals with streams and does not need to
be overridden by most clients and introducing an AddFile callback for adding
files (e.g. from the cache) to the link.

Differential Revision: https://reviews.llvm.org/D24622

llvm-svn: 282299
2016-09-23 21:33:43 +00:00
Davide Italiano 14e9e8af35 [LTO] Add ability to parse AA pipelines.
This is supposed to be a drop in replacement for what lld
provides via --lto-newpm-aa-pipeline.

llvm-svn: 281774
2016-09-16 21:03:21 +00:00
Davide Italiano ec9612da1a [lib/LTO] Add a way to run a custom pipeline
Differential Revision:  https://reviews.llvm.org/D24095

llvm-svn: 280830
2016-09-07 17:46:16 +00:00
Teresa Johnson 26a462877b [ThinLTO] Move loading of cache entry to client
Summary:
Have the cache pass back the path to the cache entry when it
is ready to be loaded, instead of a buffer.

For gold-plugin we can simply pass this file back to gold directly,
which avoids expensive writing of a separate tmp file. Ensure
the cache entry is not deleted on cleanup by adjusting the setting
of the IsTemporary flags.

Moved the loading of the buffer into llvm-lto2 to maintain current
behavior.

Reviewers: mehdi_amini

Subscribers: llvm-commits, mehdi_amini

Differential Revision: https://reviews.llvm.org/D23946

llvm-svn: 279883
2016-08-26 23:29:14 +00:00
Mehdi Amini adc0e26bef [ThinLTO] Add caching to the new LTO API
Add the ability to plug a cache on the LTO API.
I tried to write such that a linker implementation can
control the cache backend. This is intrusive and I'm
not totally happy with it, but I can't figure out a
better design right now.

Differential Revision: https://reviews.llvm.org/D23599

llvm-svn: 279576
2016-08-23 21:30:12 +00:00
Mehdi Amini 458f805468 [LTO] Add the ability to test -thinlto-emit-imports-files through llvm-lto2
Summary:
Start bringing llvm-lto2 to a level where we can test the LTO API
a bit deeper.

Reviewers: tejohnson

Subscribers: llvm-commits, mehdi_amini

Differential Revision: https://reviews.llvm.org/D23681

llvm-svn: 279349
2016-08-19 23:54:40 +00:00
Mehdi Amini eccffada33 [LTO] Change addSaveTemps API: do not add dot to the supplied prefix path
Summary:
It does not play well with directories (end up with a bunch of hidden
files).
Also, do not strip the 0 suffix for the first task, especially since
0 can be used by ThinLTO as well now.

Reviewers: tejohnson

Subscribers: mehdi_amini, pcc, llvm-commits

Differential Revision: https://reviews.llvm.org/D23612

llvm-svn: 279014
2016-08-18 00:12:33 +00:00
Chandler Carruth 5f6d73b1fa [LTO] Fix a use-after-free introduced in r278907 and caught by ASan.
The ASan build bot caught this right away:
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/15580/steps/check-llvm%20asan/logs/stdio

This was also breaking a Windows build bot I'm pretty sure.

llvm-svn: 278912
2016-08-17 07:48:34 +00:00
Mehdi Amini 970800e0c8 [LTO] Introduce an Output class to wrap the output stream creation (NFC)
Summary:
While NFC for now, this will allow more flexibility on the client side
to hold state necessary to back up the stream.
Also when adding caching, this class will grow in complexity.

Note I blindly modified the gold-plugin as I can't compile it.

Reviewers: tejohnson

Subscribers: mehdi_amini, llvm-commits

Differential Revision: https://reviews.llvm.org/D23542

llvm-svn: 278907
2016-08-17 06:23:09 +00:00
Teresa Johnson 27ce116175 Add (hopefully last) remaining missing dependences to llvm-lto2
There are still a few missing symbols reported by:
http://bb.pgr.jp/builders/i686-mingw32-RA-on-linux/builds/15535/steps/build_llvmclang/logs/stdio
http://bb.pgr.jp/builders/i686-mingw32-RA-on-linux/builds/15535/steps/build_llvmclang/logs/stdio

This should hopefully take care of them.

llvm-svn: 278353
2016-08-11 16:29:47 +00:00
Teresa Johnson fdfde19985 More missing llvm-lto2 dependencies
Follow-on to r278341: Update CMakeLists.txt to match LLVMBuild.txt

llvm-svn: 278349
2016-08-11 15:58:49 +00:00
Teresa Johnson a26adfb6a8 Fix bot failure from r278338 due to missing dependences
Add some missing dependences to the llvm-lto2 tool to attempt to appease
missing symbols in link from bot:
http://bb.pgr.jp/builders/i686-mingw32-RA-on-linux/builds/15527

llvm-svn: 278341
2016-08-11 15:23:05 +00:00
Teresa Johnson 9ba95f99f3 Restore "Resolution-based LTO API."
This restores commit r278330, with fixes for a few bot failures:
- Fix a late change I had made to the save temps output file that I
  missed due to existing files sitting on my disk
- Fix a bunch of Windows bot failures with "ambiguous call to overloaded
  function" due to confusion between llvm::make_unique vs
  std::make_unique (preface the new make_unique calls with "llvm::")
- Attempt to fix a modules bot failure by adding a missing include
  to LTO/Config.h.

Original change:

Resolution-based LTO API.

Summary:
This introduces a resolution-based LTO API. The main advantage of this API over
existing APIs is that it allows the linker to supply a resolution for each
symbol in each object, rather than the combined object as a whole. This will
become increasingly important for use cases such as ThinLTO which require us
to process symbol resolutions in a more complicated way than just adjusting
linkage.

Patch by Peter Collingbourne.

Reviewers: rafael, tejohnson, mehdi_amini

Subscribers: lhames, tejohnson, mehdi_amini, llvm-commits

Differential Revision: https://reviews.llvm.org/D20268

llvm-svn: 278338
2016-08-11 14:58:12 +00:00
Teresa Johnson cbf684e6c6 Revert "Resolution-based LTO API."
This reverts commit r278330.

I made a change to the save temps output that is causing issues with the
bots. Didn't realize this because I had older output files sitting on
disk in my test output directory.

llvm-svn: 278331
2016-08-11 13:03:56 +00:00
Teresa Johnson f99573b3ee Resolution-based LTO API.
Summary:
This introduces a resolution-based LTO API. The main advantage of this API over
existing APIs is that it allows the linker to supply a resolution for each
symbol in each object, rather than the combined object as a whole. This will
become increasingly important for use cases such as ThinLTO which require us
to process symbol resolutions in a more complicated way than just adjusting
linkage.

Patch by Peter Collingbourne.

Reviewers: rafael, tejohnson, mehdi_amini

Subscribers: lhames, tejohnson, mehdi_amini, llvm-commits

Differential Revision: https://reviews.llvm.org/D20268

Address review comments

llvm-svn: 278330
2016-08-11 12:56:40 +00:00