Commit Graph

212 Commits

Author SHA1 Message Date
Kristof Beyls a14e9a5576 Remove size 1 from check as that isn't part of what the test is meant to be testing.
This test also runs on e.g. ARM-native builds when the X86 backend is also
built.  This test produces code for the default instruction set, even though it
is in a "X86" sub-directory. Given that this test doesn't seem to be testing
anything architecture-specific, it seems it's best to adapt the check to not
check for an architecture-dependent value (the size of the function), rather
than hard-code the test to target x86.

llvm-svn: 267722
2016-04-27 15:03:09 +00:00
Teresa Johnson 02e98331c0 [ThinLTO] Use valueid instead of bitcode offsets in combined index file
Summary:
With the removal of support for lazy parsing of combined index summary
records (e.g. r267344), we no longer need to include the summary record
bitcode offset in the VST entries for definitions. Change the combined
index format to be similar to the per-module index format in using value
ids to cross-reference from the summary record to the VST entry (rather
than the summary record bitcode offset to cross-reference in the other
direction).

The visible changes are:
1) Add the value id to the combined summary records
2) Remove the summary offset from the combined VST records, which has
the following effects:
- No longer need the VST_CODE_COMBINED_GVDEFENTRY record, as all
  combined index VST entries now only contain the value id and
  corresponding GUID.
- No longer have duplicate VST entries in the case where there are
  multiple definitions of a symbol (e.g. weak/linkonce), as they all
  have the same value id and GUID.

An implication of #2 above is that in order to hook up an alias to the
correct aliasee based on the value id of the aliasee recorded in the
combined index alias record, we need to scan the entries in the index
for that GUID to find the one from the same module (i.e. the case where
there are multiple entries for the aliasee). But the reader no longer
has to maintain a special map to hook up the alias/aliasee.

Reviewers: joker.eph

Subscribers: joker.eph, llvm-commits

Differential Revision: http://reviews.llvm.org/D19481

llvm-svn: 267712
2016-04-27 13:28:35 +00:00
Evgeniy Stepanov d194e51726 [gold] Fix linkInModule and extend common.ll test.
Fix early exit from linkInModule. IRMover::move returns false on
success and true on error.

Add a few more cases of merged common linkage variables with
different sizes and alignments.

llvm-svn: 267437
2016-04-25 18:23:29 +00:00
Rafael Espindola 468d327b34 Also check the IR.
llvm-svn: 267367
2016-04-24 21:42:56 +00:00
Rafael Espindola 2adb06a820 Add a test for how we handle protected visibility.
llvm-svn: 267366
2016-04-24 21:30:18 +00:00
NAKAMURA Takumi d53ab701bf llvm/test/tools/gold/X86/thinlto.ll: Possible fix corresponding to r267318.
llvm-svn: 267334
2016-04-24 08:02:00 +00:00
Teresa Johnson 723cd05f83 [gold] Gate value name discarding under save-temps
Summary:
This removes a couple of flags added to control this behavior, and
simply keeps all value names when save-temps is specified.

Reviewers: rafael

Subscribers: llvm-commits, pcc, davide

Differential Revision: http://reviews.llvm.org/D19384

llvm-svn: 267279
2016-04-23 05:15:59 +00:00
Teresa Johnson 143d15bc29 Re-enable "[gold-plugin] Disable name for values other than GlobalValue"
This restores r266871 with a fix for gold tests relying on the value
names, when using a release compiler, by adding a way to disable the
default discarding. Update affected tests to use the new mechanism so
that value names are preserved as expected, regardless of how the
compiler was built.

llvm-svn: 266881
2016-04-20 15:16:57 +00:00
Teresa Johnson f0bedf5343 Revert "[gold-plugin] Disable name for values other than GlobalValue"
This reverts commit r266871. Setting the default based on the NDEBUG
flag is causing test failures. Need to figure out whether to change this
approach or update tests.

llvm-svn: 266872
2016-04-20 13:18:47 +00:00
Teresa Johnson d3ded4a441 [gold-plugin] Disable name for values other than GlobalValue
Summary:
Applies Mehdi's optimization (r263086) to disable value names other than
for GlobalValues to LTO/ThinLTO performed via the gold-plugin, in the
same manner as it is applied in libLTO.

Reviewers: rafael, joker-eph

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D19269

llvm-svn: 266871
2016-04-20 13:01:37 +00:00
Mehdi Amini f12cbe7dd6 Fix Gold test after r266750 (ModuleLinker: Do not import linkonce/weak as "external_weak")
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266752
2016-04-19 16:21:37 +00:00
Adrian Prantl 75819aedf6 [PR27284] Reverse the ownership between DICompileUnit and DISubprogram.
Currently each Function points to a DISubprogram and DISubprogram has a
scope field. For member functions the scope is a DICompositeType. DIScopes
point to the DICompileUnit to facilitate type uniquing.

Distinct DISubprograms (with isDefinition: true) are not part of the type
hierarchy and cannot be uniqued. This change removes the subprograms
list from DICompileUnit and instead adds a pointer to the owning compile
unit to distinct DISubprograms. This would make it easy for ThinLTO to
strip unneeded DISubprograms and their transitively referenced debug info.

Motivation
----------

Materializing DISubprograms is currently the most expensive operation when
doing a ThinLTO build of clang.

We want the DISubprogram to be stored in a separate Bitcode block (or the
same block as the function body) so we can avoid having to expensively
deserialize all DISubprograms together with the global metadata. If a
function has been inlined into another subprogram we need to store a
reference the block containing the inlined subprogram.

Attached to https://llvm.org/bugs/show_bug.cgi?id=27284 is a python script
that updates LLVM IR testcases to the new format.

http://reviews.llvm.org/D19034
<rdar://problem/25256815>

llvm-svn: 266446
2016-04-15 15:57:41 +00:00
Mehdi Amini 68da426eea Move summary creation out of llvm-as into opt
Summary:
Let keep llvm-as "dumb": it converts textual IR to bitcode. This
commit removes the dependency from llvm-as to libLLVMAnalysis.
We'll add back summary in llvm-as if we get to a textual
representation for it at some point. In the meantime, opt seems
like a better place for that.

Reviewers: tejohnson

Subscribers: joker.eph, llvm-commits

Differential Revision: http://reviews.llvm.org/D19032

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266131
2016-04-12 21:35:18 +00:00
Evgeniy Stepanov 268826a287 [gold] Save bitcode for module partitions (save-temps + split codegen).
llvm-svn: 265583
2016-04-06 18:32:13 +00:00
Adrian Prantl b8089516a5 testcase gardening: update the emissionKind enum to the new syntax. (NFC)
llvm-svn: 265081
2016-04-01 00:16:49 +00:00
Rafael Espindola fe26864440 Fix gold tests for llvm-readobj format change.
llvm-svn: 264306
2016-03-24 16:45:41 +00:00
Teresa Johnson 26ab5772b0 [ThinLTO] Renaming of function index to module summary index (NFC)
(Resubmitting after fixing missing file issue)

With the changes in r263275, there are now more than just functions in
the summary. Completed the renaming of data structures (started in
r263275) to reflect the wider scope. In particular, changed the
FunctionIndex* data structures to ModuleIndex*, and renamed related
variables and comments. Also renamed the files to reflect the changes.

A companion clang patch will immediately succeed this patch to reflect
this renaming.

llvm-svn: 263513
2016-03-15 00:04:37 +00:00
Teresa Johnson cec0cae313 Revert "[ThinLTO] Renaming of function index to module summary index (NFC)"
This reverts commit r263490. Missed a file.

llvm-svn: 263493
2016-03-14 21:18:10 +00:00
Teresa Johnson 892920b358 [ThinLTO] Renaming of function index to module summary index (NFC)
With the changes in r263275, there are now more than just functions in
the summary. Completed the renaming of data structures (started in
r263275) to reflect the wider scope. In particular, changed the
FunctionIndex* data structures to ModuleIndex*, and renamed related
variables and comments. Also renamed the files to reflect the changes.

A companion clang patch will immediately succeed this patch to reflect
this renaming.

llvm-svn: 263490
2016-03-14 21:05:56 +00:00
Teresa Johnson 76a1c1d0ba [ThinLTO] Support for reference graph in per-module and combined summary.
Summary:
This patch adds support for including a full reference graph including
call graph edges and other GV references in the summary.

The reference graph edges can be used to make importing decisions
without materializing any source modules, can be used in the plugin
to make file staging decisions for distributed build systems, and is
expected to have other uses.

The call graph edges are recorded in each function summary in the
bitcode via a list of <CalleeValueIds, StaticCount> tuples when no PGO
data exists, or <CalleeValueId, StaticCount, ProfileCount> pairs when
there is PGO, where the ValueId can be mapped to the function GUID via
the ValueSymbolTable. In the function index in memory, the call graph
edges reference the target via the CalleeGUID instead of the
CalleeValueId.

The reference graph edges are recorded in each summary record with a
list of referenced value IDs, which can be mapped to value GUID via the
ValueSymbolTable.

Addtionally, a new summary record type is added to record references
from global variable initializers. A number of bitcode records and data
structures have been renamed to reflect the newly expanded scope of the
summary beyond functions. More cleanup will follow.

Reviewers: joker.eph, davidxl

Subscribers: joker.eph, llvm-commits

Differential Revision: http://reviews.llvm.org/D17212

llvm-svn: 263275
2016-03-11 18:52:24 +00:00
Evgeniy Stepanov 4dc3c8df45 [gold] Fix common symbols handling.
LLVM Gold plugin decides which instance of a common symbol it wants
based on the symbol size in claim_file_hook. If the file that
contains the chosen instance is later dropped from the link, we end
up with an undefined reference.

This change delays this decision until the set of the included files
is known.

llvm-svn: 263180
2016-03-11 00:51:57 +00:00
Teresa Johnson 5d07531d02 Fix new gold test to specify emulation mode.
The thinlto_linkonceresolution.ll gold linker test introduced in r262727
included a target triple, but didn't set the emulation mode, which is
necessary since the default linker target may be different.

Patch by H.J. Lu

llvm-svn: 262745
2016-03-04 21:19:08 +00:00
Teresa Johnson a17f2cd1a3 [ThinLTO] Ensure prevailing linkonce emitted as weak in ThinLTO backends
Summary:
Since IR files are all compiled into separate independent object files
in ThinLTO mode, the prevailing linkonce symbols must be emitted in its
object file even if it is no longer referenced there, e.g. if no
references remain in the module after inlining, since it may be
referenced by another ThinLTO compiled object file. This is done by
changing LDPR_PREVAILING_DEF_IRONLY* symbols to LDPR_PREVAILING_DEF,
which converts the prevailing linkonce to weak. We also don't need the
other prevailing IRONLY handling for internalization, which is not
currently performed for ThinLTO.

Test case included.

Reviewers: davidxl, rafael

Subscribers: rafael, joker.eph, llvm-commits

Differential Revision: http://reviews.llvm.org/D16173

llvm-svn: 262727
2016-03-04 17:48:35 +00:00
Teresa Johnson 7cffaf3ad0 [ThinLTO] Launch importing backends in parallel threads from gold plugin
Summary:
Launch ThinLTO backends (LTO and codegen pipelines with importing) in
parallel using a ThreadPool, after creating the combined index.
The number of threads is controlled by the existing -jobs gold plugin
option, or the hardware concurrency if not specified.

The old behavior of exiting after creating the combined index can be
invoked via a new thinlto-index-only plugin option.

This commit involves just the ThinLTO-specific pieces of D15390, the NFC
and other restructuring pieces were committed independently:
  r262677: Add hardware_concurrency interface to llvm::thread (NFC)
  r262719: Change split code gen to use ThreadPool
  r262721: Refactor gold-plugin codegen to prepare for ThinLTO threads (NFC)

Reviewers: pcc, joker.eph, rafael

Subscribers: rafael, davidxl, llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D15390

llvm-svn: 262724
2016-03-04 17:06:02 +00:00
Teresa Johnson e1164de5d0 Restore "[ThinLTO] Use MD5 hash in function index." with fix
This restores commit r260408, along with a fix for a bot failure.

The bot failure was caused by dereferencing a unique_ptr in the same
call instruction parameter list where it was passed via std::move.
Apparently due to luck this was not exposed when I built the compiler
with clang, only with gcc.

llvm-svn: 260442
2016-02-10 21:55:02 +00:00
Teresa Johnson 89f38fb5cc Revert "[ThinLTO] Use MD5 hash in function index." due to bot failure
This reverts commit r260408. Bot failure that I need to investigate.

llvm-svn: 260412
2016-02-10 19:11:15 +00:00
Teresa Johnson 0919a84071 [ThinLTO] Use MD5 hash in function index.
Summary:
This patch uses the lower 64-bits of the MD5 hash of a function name as
a GUID in the function index, instead of storing function names. Any
local functions are first given a global name by prepending the original
source file name. This is the same naming scheme and GUID used by PGO in
the indexed profile format.

This change has a couple of benefits. The primary benefit is size
reduction in the combined index file, for example 483.xalancbmk's
combined index file was reduced by around 70%. It should also result in
memory savings for the index file in memory, as the in-memory map is
also indexed by the hash instead of the string.

Second, this enables integration with indirect call promotion, since the
indirect call profile targets are recorded using the same global naming
convention and hash. This will enable the function importer to easily
locate function summaries for indirect call profile targets to enable
their import and subsequent promotion.

The original source file name is recorded in the bitcode in a new
module-level record for use in the ThinLTO backend pipeline.

Reviewers: davidxl, joker.eph

Subscribers: llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D17028

llvm-svn: 260408
2016-02-10 18:57:54 +00:00
Peter Collingbourne df49d1bbb2 WholeProgramDevirt: introduce.
This pass implements whole program optimization of virtual calls in cases
where we know (via bitset information) that the list of callees is fixed. This
includes the following:

- Single implementation devirtualization: if a virtual call has a single
  possible callee, replace all calls with a direct call to that callee.

- Virtual constant propagation: if the virtual function's return type is an
  integer <=64 bits and all possible callees are readnone, for each class and
  each list of constant arguments: evaluate the function, store the return
  value alongside the virtual table, and rewrite each virtual call as a load
  from the virtual table.

- Uniform return value optimization: if the conditions for virtual constant
  propagation hold and each function returns the same constant value, replace
  each virtual call with that constant.

- Unique return value optimization for i1 return values: if the conditions
  for virtual constant propagation hold and a single vtable's function
  returns 0, or a single vtable's function returns 1, replace each virtual
  call with a comparison of the vptr against that vtable's address.

Differential Revision: http://reviews.llvm.org/D16795

llvm-svn: 260312
2016-02-09 22:50:34 +00:00
James Y Knight 992904a0af Fix gold test after r256465.
That commit added a new pass, and this test is sensitive to what the
first pass after verify is called.

llvm-svn: 256532
2015-12-29 03:48:37 +00:00
Teresa Johnson d213aa469e Handle empty Subprogram list when linking metadata.
Use an iterator that handles an empty subprogram list.

Fixes PR25915.

llvm-svn: 256224
2015-12-22 01:17:19 +00:00
Teresa Johnson 3e88a3200c Add testcase for r256161 (PR25907)
llvm-svn: 256174
2015-12-21 18:24:35 +00:00
Teresa Johnson 0e7c82cb69 [ThinLTO/LTO] Don't link in unneeded metadata
Summary:
Third patch split out from http://reviews.llvm.org/D14752.

Only map in needed DISubroutine metadata (imported or otherwise linked
in functions and other DISubroutine referenced by inlined instructions).
This is supported for ThinLTO, LTO and llvm-link --only-needed, with
associated tests for each one.

Depends on D14838.

Reviewers: dexonsmith, joker.eph

Subscribers: davidxl, llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D14843

llvm-svn: 256003
2015-12-18 17:51:37 +00:00
Rafael Espindola 762a6ac0b5 Pass -m elf_x84_64 to gold invocations.
Fixes pr25868.

llvm-svn: 255930
2015-12-17 21:56:27 +00:00
Rafael Espindola caabe22832 Slit lib/Linker in two.
A linker normally has two stages: symbol resolution and "moving stuff".

In lib/Linker there is the complication of lazy linking some globals,
but it was still far more mixed than it needed to.

This splits the linker into a lower level IRMover and the linker proper.
The IRMover just takes a list of globals to move and a callback that
lets the user control what is lazy linked.

The main motivation is that now tools/gold (and soon lld) can use their
own symbol resolution to instruct IRMover what to do.

llvm-svn: 255254
2015-12-10 14:19:35 +00:00
Rafael Espindola 3f85d24df4 Add a passing test.
When a comdat is discarded, any globals defined in it become undefined.

llvm-svn: 254258
2015-11-29 15:52:12 +00:00
Rafael Espindola c73fdcd1b7 Don't depend on the order the IR is copied.
llvm-svn: 254257
2015-11-29 15:22:49 +00:00
Rafael Espindola 94247d0860 Don't depend on the order the IR is copied.
llvm-svn: 254256
2015-11-29 15:08:39 +00:00
Rafael Espindola 8e8183b8bd Test both input file orders.
llvm-svn: 254186
2015-11-27 03:50:34 +00:00
Rafael Espindola 60b57863a0 Add missing file.
llvm-svn: 254185
2015-11-27 03:47:29 +00:00
Rafael Espindola 1d3465f641 Make the test a bit more interesting.
It now covers a regular function replacing an available_externally one.

llvm-svn: 254184
2015-11-27 02:07:37 +00:00
Rafael Espindola b38f7b5adc Add a few passing lto tests.
I found these while trying to get a prototype to bootstrap.

They cover things like
* Handling of non linker visible stuff (append, available_externally)
* Type merging
* Alias to dropped globals
* Dropping linkage when converting to a declaration.

These should hopefully be generally useful for anyone refactoring the
plugin.

llvm-svn: 254174
2015-11-26 19:53:12 +00:00
Rafael Espindola 383d1f6aa6 Make this test a bit more strict.
It now tests with files in both orders.

llvm-svn: 253993
2015-11-24 16:43:53 +00:00
Rafael Espindola 23117e5a7b Add an already passing test.
This tests that a declaration can resolve to an alias.

I broke this locally while prototyping a change and it looks like a nice
test to have.

llvm-svn: 253984
2015-11-24 14:15:50 +00:00
Rafael Espindola d1beb07d39 Have a single way for creating unique value names.
We had two code paths. One would create names like "foo.1" and the other
names like "foo1".

For globals it is important to use "foo.1" to help C++ name demangling.
For locals there is no strong reason to go one way or the other so I
kept the most common mangling (foo1).

llvm-svn: 253804
2015-11-22 00:16:24 +00:00
Teresa Johnson 6290dbc0f7 [ThinLTO] Handle bitcode without function summary sections gracefully
Summary:
Several fixes to the handling of bitcode files without function summary
sections so that they are skipped during ThinLTO processing in llvm-lto
and the gold plugin when appropriate instead of aborting.

1 Don't assert when trying to add a FunctionInfo that doesn't have
  a summary attached.
2 Skip FunctionInfo structures that don't have attached function summary
  sections when trying to create the combined function summary.
3 In both llvm-lto and gold-plugin, check whether a bitcode file has
  a function summary section before trying to parse the index, and skip
  the bitcode file if it does not.
4 Fix hasFunctionSummaryInMemBuffer in BitcodeReader, which had a bug
  where we returned to early while looking for the summary section.

Also added llvm-lto and gold-plugin based tests for cases where we
don't have function summaries in the bitcode file. I verified that
either the first couple fixes described above are enough to avoid the
crashes, or fixes 1,3,4. But have combined them all here for added
robustness.

Reviewers: joker.eph

Subscribers: llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D14903

llvm-svn: 253796
2015-11-21 21:55:48 +00:00
James Molloy ea3bb626d4 [LTO] Appease buildbots take 3
This time I've found a linux box and checked it there. This test now passes.

Because I'd introduced an undefined reference in @bar, gold now returns an error. This doesn't matter for the test itself, because it also emits the remarks the test is checking for. But it does cause LIT to notice a nonzero return code which it faults on.

llvm-svn: 253454
2015-11-18 12:08:24 +00:00
James Molloy ec9698a8c8 [LTO] Buildbot appeasing take 2
Let's try again. This time using the right function signature. It's a real pity I can't run this on a darwin machine...

llvm-svn: 253453
2015-11-18 11:37:32 +00:00
James Molloy 784adffce1 [LTO] Fix up test/tools/gold/X86/remarks.ll
It needs the same fixes as in test/LTO/X86/remarks.ll, but this test appears not to get run on my system (but does on the buildbot). Strange.

llvm-svn: 253452
2015-11-18 11:32:14 +00:00
Teresa Johnson b1cfcd4a53 Support for llvm-bcanalyzer dumping of record array strings.
Summary:
Adds support for automatically detecting and printing strings
represented by Array abbrev operands, analogous to the string dumping
performed for Blob abbrev operands.

Enhanced the ThinLTO combined index test to check for the appropriate
module and function strings.

Reviewers: dexonsmith, joker.eph, davidxl

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D13553

llvm-svn: 249695
2015-10-08 15:56:24 +00:00
Teresa Johnson 403a787e03 Support for function summary index bitcode sections and files.
Summary:
The bitcode format is described in this document:
  https://drive.google.com/file/d/0B036uwnWM6RWdnBLakxmeDdOeXc/view
For more info on ThinLTO see:
  https://sites.google.com/site/llvmthinlto

The first customer is ThinLTO, however the data structures are designed
and named more generally based on prior feedback. There are a few
comments regarding how certain interfaces are used by ThinLTO, and the
options added here to gold currently have ThinLTO-specific names as the
behavior they provoke is currently ThinLTO-specific.

This patch includes support for generating per-module function indexes,
the combined index file via the gold plugin, and several tests
(more are included with the associated clang patch D11908).

Reviewers: dexonsmith, davidxl, joker.eph

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D13107

llvm-svn: 249270
2015-10-04 14:33:43 +00:00
Teresa Johnson 8c8fe5a015 Disable the second verification run when performing LTO through
gold in NDEBUG mode.
Follow on patch for r247729 - LTO: Disable extra verify runs in release
builds.

llvm-svn: 247824
2015-09-16 18:06:45 +00:00
Arnaud A. de Grandmaison ba4e977cff Tweak 2 x86 gold tests so they can run on non-x86 platforms
llvm-svn: 247415
2015-09-11 14:45:34 +00:00
NAKAMURA Takumi ecc73e7806 Fix llvm/test/tools/gold/X86/bad-alias.ll.
llvm-svn: 247391
2015-09-11 08:03:17 +00:00
David Blaikie 223a634f9d Fix the gold test cases after alias changes
llvm-svn: 247381
2015-09-11 03:28:37 +00:00
David Blaikie 2f40830dde [opaque pointer type] Add textual IR support for explicit type parameter for global aliases
update.py:
import fileinput
import sys
import re

alias_match_prefix = r"(.*(?:=|:|^)\s*(?:external |)(?:(?:private|internal|linkonce|linkonce_odr|weak|weak_odr|common|appending|extern_weak|available_externally) )?(?:default |hidden |protected )?(?:dllimport |dllexport )?(?:unnamed_addr |)(?:thread_local(?:\([a-z]*\))? )?alias"
plain = re.compile(alias_match_prefix + r" (.*?))(| addrspace\(\d+\) *)\*($| *(?:%|@|null|undef|blockaddress|addrspacecast|\[\[[a-zA-Z]|\{\{).*$)")
cast  = re.compile(alias_match_prefix + r") ((?:bitcast|inttoptr|addrspacecast)\s*\(.* to (.*?)(| addrspace\(\d+\) *)\*\)\s*(?:;.*)?$)")
gep   = re.compile(alias_match_prefix + r") ((?:getelementptr)\s*(?:inbounds)?\s*\((?P<type>.*), (?P=type)(?:\s*addrspace\(\d+\)\s*)?\* .*\)\s*(?:;.*)?$)")

def conv(line):
  m = re.match(cast, line)
  if m:
    return m.group(1) + " " + m.group(3) + ", " + m.group(2)
  m = re.match(gep, line)
  if m:
    return m.group(1) + " " + m.group(3) + ", " + m.group(2)
  m = re.match(plain, line)
  if m:
    return m.group(1) + ", " + m.group(2) + m.group(3) + "*" + m.group(4) + "\n"
  return line

for line in sys.stdin:
  sys.stdout.write(conv(line))

apply.sh:
for name in "$@"
do
  python3 `dirname "$0"`/update.py < "$name" > "$name.tmp" && mv "$name.tmp" "$name"
  rm -f "$name.tmp"
done

The actual commands:
From llvm/src:
find test/ -name *.ll | xargs ./apply.sh
From llvm/src/tools/clang:
find test/ -name *.mm -o -name *.m -o -name *.cpp -o -name *.c | xargs -I '{}' ../../apply.sh "{}"
From llvm/src/tools/polly:
find test/ -name *.ll | xargs ./apply.sh

llvm-svn: 247378
2015-09-11 03:22:04 +00:00
NAKAMURA Takumi 3c0f9d8170 Tweak llvm/test/tools/gold/X86/parallel.ll to run with pthread-unaware ld.gold on Linux.
If ld.gold is configured without --enable-thread, ld.gold might not load libpthread.so.
Preloading LLVMgold.so loads also libpthread.so.

llvm-svn: 246739
2015-09-03 00:48:59 +00:00
NAKAMURA Takumi 44e3c5f36c Suppress llvm/test/tools/gold/X86/parallel.ll while investigating.
For me,

  Program received signal SIGSEGV, Segmentation fault.
  0x00007ffff7deb0dc in _dl_fixup () from /lib64/ld-linux-x86-64.so.2

llvm-svn: 246641
2015-09-02 10:59:21 +00:00
Peter Collingbourne 87202a4aac gold-plugin: Implement parallel LTO code generation using llvm::splitCodeGen.
Parallelism can be enabled using a new plugin option, jobs=N, where N is
the number of code generation threads.

Differential Revision: http://reviews.llvm.org/D12308

llvm-svn: 246584
2015-09-01 20:40:22 +00:00
Rafael Espindola cad82ccbb2 Split the gold tests into X86 and PowerPC directories.
Patch by Than McIntosh!

llvm-svn: 246328
2015-08-28 20:33:56 +00:00
Rafael Espindola ad3b6bfa2a Pass -m to the linker in this test.
Fixes the test on a ppc host.

llvm-svn: 240431
2015-06-23 18:04:54 +00:00
Rafael Espindola bdf509aaaf Add a triple to the test to fix it on some hosts.
The slp vectorizer doesn't optimize this case in 32 bits.

Fixes PR23453.

llvm-svn: 240289
2015-06-22 15:44:20 +00:00
Rafael Espindola 92200d237a gold-plugin: save the .o when given -save-temps.
The plugin now save the bitcode before and after optimizations and the
.o that is passed to the linker.

llvm-svn: 239726
2015-06-15 13:36:27 +00:00
Rafael Espindola 14ebbc6d99 Add -m -m elf_x86_64 to gold invocations.
Otherwise the tests would fail if the default was not elf_x86_64.

This fixes PR22966.

Patch by H.J. Lu!

llvm-svn: 233124
2015-03-24 22:20:19 +00:00
Peter Collingbourne 070843d60b libLTO, llvm-lto, gold: Introduce flag for controlling optimization level.
This change also introduces a link-time optimization level of 1. This
optimization level runs only the globaldce pass as well as cleanup passes for
passes that run at -O0, specifically simplifycfg which cleans up lowerbitsets.

http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20150316/266951.html

llvm-svn: 232769
2015-03-19 22:01:00 +00:00
Peter Collingbourne 152b936683 gold: Make powerpc support optional for the tests.
Differential Revision: http://reviews.llvm.org/D8400

llvm-svn: 232744
2015-03-19 18:23:31 +00:00
Rafael Espindola f3f185486c Bring r231132 back with a fix.
The issue was that we were always printing the remarks. Fix that and add a test
showing that it prints nothing if -pass-remarks is not given.

Original message:
Correctly handle -pass-remarks in the gold plugin.

llvm-svn: 231273
2015-03-04 18:51:45 +00:00
NAKAMURA Takumi 84a9697c17 Revert r231132, "Correctly handle -pass-remarks in the gold plugin.", for now, to suppress log floodng in LTO.
llvm-svn: 231253
2015-03-04 16:24:28 +00:00
Rafael Espindola 84483d247f Correctly handle -pass-remarks in the gold plugin.
llvm-svn: 231132
2015-03-03 21:11:13 +00:00
Rafael Espindola 503f883b95 Add r230655 back with a fix.
The issue is that now we have a diag handler during optimizations
and get forward every optimization remark, flooding stdout.

The same filtering should probably be done with or without a
custom handler, but for now just ignore remarks.

Original message:

gold-plugin: "Upgrade" debug info and handle its warnings.

The gold plugin never calls MaterializeModule, so any old debug info
was not deleted and could cause crashes.

Now that it is being "upgraded", the plugin also has to handle warnings
and create Modules with a nice id (it shows in the warning).

llvm-svn: 230991
2015-03-02 19:08:03 +00:00
NAKAMURA Takumi 0f480f5010 Revert r230655, "gold-plugin: "Upgrade" debug info and handle its warnings."
It emits *millions of warnings* during selfhosting LTO build, to choke the buildbot with gigbytes of log.

llvm-svn: 230885
2015-03-01 04:16:28 +00:00
Eric Christopher b759340fc8 Remove option.ll as part of the Forward Control Flow Integrity
removal.

llvm-svn: 230844
2015-02-28 10:04:18 +00:00
David Blaikie a79ac14fa6 [opaque pointer type] Add textual IR support for explicit type parameter to load instruction
Essentially the same as the GEP change in r230786.

A similar migration script can be used to update test cases, though a few more
test case improvements/changes were required this time around: (r229269-r229278)

import fileinput
import sys
import re

pat = re.compile(r"((?:=|:|^)\s*load (?:atomic )?(?:volatile )?(.*?))(| addrspace\(\d+\) *)\*($| *(?:%|@|null|undef|blockaddress|getelementptr|addrspacecast|bitcast|inttoptr|\[\[[a-zA-Z]|\{\{).*$)")

for line in sys.stdin:
  sys.stdout.write(re.sub(pat, r"\1, \2\3*\4", line))

Reviewers: rafael, dexonsmith, grosser

Differential Revision: http://reviews.llvm.org/D7649

llvm-svn: 230794
2015-02-27 21:17:42 +00:00
David Blaikie 79e6c74981 [opaque pointer type] Add textual IR support for explicit type parameter to getelementptr instruction
One of several parallel first steps to remove the target type of pointers,
replacing them with a single opaque pointer type.

This adds an explicit type parameter to the gep instruction so that when the
first parameter becomes an opaque pointer type, the type to gep through is
still available to the instructions.

* This doesn't modify gep operators, only instructions (operators will be
  handled separately)

* Textual IR changes only. Bitcode (including upgrade) and changing the
  in-memory representation will be in separate changes.

* geps of vectors are transformed as:
    getelementptr <4 x float*> %x, ...
  ->getelementptr float, <4 x float*> %x, ...
  Then, once the opaque pointer type is introduced, this will ultimately look
  like:
    getelementptr float, <4 x ptr> %x
  with the unambiguous interpretation that it is a vector of pointers to float.

* address spaces remain on the pointer, not the type:
    getelementptr float addrspace(1)* %x
  ->getelementptr float, float addrspace(1)* %x
  Then, eventually:
    getelementptr float, ptr addrspace(1) %x

Importantly, the massive amount of test case churn has been automated by
same crappy python code. I had to manually update a few test cases that
wouldn't fit the script's model (r228970,r229196,r229197,r229198). The
python script just massages stdin and writes the result to stdout, I
then wrapped that in a shell script to handle replacing files, then
using the usual find+xargs to migrate all the files.

update.py:
import fileinput
import sys
import re

ibrep = re.compile(r"(^.*?[^%\w]getelementptr inbounds )(((?:<\d* x )?)(.*?)(| addrspace\(\d\)) *\*(|>)(?:$| *(?:%|@|null|undef|blockaddress|getelementptr|addrspacecast|bitcast|inttoptr|\[\[[a-zA-Z]|\{\{).*$))")
normrep = re.compile(       r"(^.*?[^%\w]getelementptr )(((?:<\d* x )?)(.*?)(| addrspace\(\d\)) *\*(|>)(?:$| *(?:%|@|null|undef|blockaddress|getelementptr|addrspacecast|bitcast|inttoptr|\[\[[a-zA-Z]|\{\{).*$))")

def conv(match, line):
  if not match:
    return line
  line = match.groups()[0]
  if len(match.groups()[5]) == 0:
    line += match.groups()[2]
  line += match.groups()[3]
  line += ", "
  line += match.groups()[1]
  line += "\n"
  return line

for line in sys.stdin:
  if line.find("getelementptr ") == line.find("getelementptr inbounds"):
    if line.find("getelementptr inbounds") != line.find("getelementptr inbounds ("):
      line = conv(re.match(ibrep, line), line)
  elif line.find("getelementptr ") != line.find("getelementptr ("):
    line = conv(re.match(normrep, line), line)
  sys.stdout.write(line)

apply.sh:
for name in "$@"
do
  python3 `dirname "$0"`/update.py < "$name" > "$name.tmp" && mv "$name.tmp" "$name"
  rm -f "$name.tmp"
done

The actual commands:
From llvm/src:
find test/ -name *.ll | xargs ./apply.sh
From llvm/src/tools/clang:
find test/ -name *.mm -o -name *.m -o -name *.cpp -o -name *.c | xargs -I '{}' ../../apply.sh "{}"
From llvm/src/tools/polly:
find test/ -name *.ll | xargs ./apply.sh

After that, check-all (with llvm, clang, clang-tools-extra, lld,
compiler-rt, and polly all checked out).

The extra 'rm' in the apply.sh script is due to a few files in clang's test
suite using interesting unicode stuff that my python script was throwing
exceptions on. None of those files needed to be migrated, so it seemed
sufficient to ignore those cases.

Reviewers: rafael, dexonsmith, grosser

Differential Revision: http://reviews.llvm.org/D7636

llvm-svn: 230786
2015-02-27 19:29:02 +00:00
Rafael Espindola 7360fb6206 gold-plugin: "Upgrade" debug info and handle its warnings.
The gold plugin never calls MaterializeModule, so any old debug info
was not deleted and could cause crashes.

Now that it is being "upgraded", the plugin also has to handle warnings
and create Modules with a nice id (it shows in the warning).

llvm-svn: 230655
2015-02-26 18:24:37 +00:00
Michael Kuperstein 2ccbc08b8e gold-plugin: fix test to allow default visibility on local symbols
GNU ld sets default, not hidden, visibility on local symbols. 
Having default or hidden visibility on local symbols makes no difference in run-time behavior.

Patch by: H.J. Lu <hjl.tools@gmail.com>

llvm-svn: 229297
2015-02-15 09:32:30 +00:00
Chandler Carruth edd92948d1 [gold] Consolidate the gold plugin options and actually search for
a gold binary explicitly. Substitute this binary into the tests rather
than just directly executing the 'ld' binary.

This should allow folks to inject a cross compiling gold binary, or in
my case to use a gold binary built and installed somewhere other than
/usr/bin/ld. It should also allow the tests to find 'ld.gold' so that
things work even if gold isn't the default on the system.

I've only stubbed out support in the makefile to preserve the existing
behavior with none of the fancy logic. If someone else wants to add
logic here, they're welcome to do so.

llvm-svn: 229251
2015-02-14 09:43:57 +00:00
Jan Wen Voung c11b45a2ea Gold-plugin: Broaden scope of get/release_input_file to scope of Module.
Summary:
Move calls to get_input_file and release_input_file out of
getModuleForFile(). Otherwise release_input_file may end up
unmapping a view of the file while the view is still being
used by the Module (on 32-bit hosts).

Fix for PR22482.

Test Plan: Add test using --no-map-whole-files.

Reviewers: rafael, nlewycky

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D7539

llvm-svn: 228842
2015-02-11 16:12:50 +00:00
Rafael Espindola 9e3e53f6dd Fix linking of shared libraries.
In shared libraries the plugin can see non-weak declarations that are still
undefined.

llvm-svn: 226031
2015-01-14 20:08:46 +00:00
Rafael Espindola 0fd9e5f719 Fix handling of extern_weak. This was broken by r225983.
llvm-svn: 226026
2015-01-14 19:43:32 +00:00
Rafael Espindola 5ca7fa15fd Handle a symbol being undefined.
This can happen if:
* It is present in a comdat in one file.
* It is not present in the comdat of the file that is kept.
* Is is not used.

This should fix the LTO boostrap.

Thanks to Takumi NAKAMURA for setting up the bot!

llvm-svn: 225983
2015-01-14 13:53:50 +00:00
Rafael Espindola 83a362cde8 Change the .ll syntax for comdats and add a syntactic sugar.
In order to make comdats always explicit in the IR, we decided to make
the syntax a bit more compact for the case of a GlobalObject in a
comdat with the same name.

Just dropping the $name causes problems for

@foo = globabl i32 0, comdat
$bar = comdat ...

and

declare void @foo() comdat
$bar = comdat ...

So the syntax is changed to

@g1 = globabl i32 0, comdat($c1)
@g2 = globabl i32 0, comdat

and

declare void @foo() comdat($c1)
declare void @foo() comdat

llvm-svn: 225302
2015-01-06 22:55:16 +00:00
Rafael Espindola a4f104b2ce Remember the unmangled name in the plugin.
This allows it to work with non trivial manglings like the one in COFF.

Amusingly, this can be tested with gold, as emit-llvm causes the plugin to
exit before any COFF is generated.

llvm-svn: 223790
2014-12-09 16:50:57 +00:00
Rafael Espindola 7f2fa4fa38 This test requires asserts because of -stats.
Sorry about that.

llvm-svn: 222788
2014-11-25 20:56:56 +00:00
Rafael Espindola 947bdb69cb gold plugin: call llvm_shutdown so that -stats works.
llvm-svn: 222787
2014-11-25 20:52:49 +00:00
Rafael Espindola 6953a3a6e0 Add a disable-output option to the gold plugin.
This corresponds to the opt option and is handy for profiling.

llvm-svn: 222687
2014-11-24 21:18:14 +00:00
Rafael Espindola 301396c911 Fix the test.
It was broken since r221708.

llvm-svn: 221783
2014-11-12 14:23:04 +00:00
Rafael Espindola 919fb535a0 Enable the slp vectorizer in the gold plugin.
llvm-svn: 220887
2014-10-30 00:38:54 +00:00
Rafael Espindola 8391dbd1c6 Enable the loop vectorizer in the gold plugin.
llvm-svn: 220886
2014-10-30 00:11:24 +00:00
Rafael Espindola 4a3b6cf3c1 Replace also-emit-llvm with save-temps.
The also-emit-llvm option only supported getting the IR before optimizations.
This patch replaces it with a more generic save-temps option that saves the IR
both before and after optimizations.

llvm-svn: 220885
2014-10-29 23:54:45 +00:00
Rafael Espindola dfc7ed7ad0 gold plugin: Handle gold selecting a linkonce GV when a weak is present.
The plugin API doesn't have the notion of linkonce, only weak. It is up to the
plugin to figure out if a symbol used only for the symbol table can be dropped.
In particular, it has to avoid dropping a linkonce_odr selected by gold if there
is also a weak_odr.

llvm-svn: 219188
2014-10-07 04:06:13 +00:00
Rafael Espindola cb69583c50 gold plugin: create internal replacement with original linkage first.
The call to copyAttributesFrom will copy the visibility, which might assert
if it were to produce something invalid like "internal hidden". We avoid it
by first creating the replacement with the original linkage and then setting
it to internal affter the call to copyAttributesFrom.

llvm-svn: 219184
2014-10-07 03:19:55 +00:00
Rafael Espindola 93aec71ba9 gold plugin: Remap function arguments when creating a replacement function.
When creating an internal function replacement for use in an alias we were
not remapping the argument uses in the instructions to point to the new
arguments.

llvm-svn: 219177
2014-10-07 00:47:38 +00:00
Peter Collingbourne 10039c02ea LTO: introduce object file-based on-disk module format.
This format is simply a regular object file with the bitcode stored in a
section named ".llvmbc", plus any number of other (non-allocated) sections.

One immediate use case for this is to accommodate compilation processes
which expect the object file to contain metadata in non-allocated sections,
such as the ".go_export" section used by some Go compilers [1], although I
imagine that in the future we could consider compiling parts of the module
(such as large non-inlinable functions) directly into the object file to
improve LTO efficiency.

[1] http://golang.org/doc/install/gccgo#Imports

Differential Revision: http://reviews.llvm.org/D4371

llvm-svn: 218078
2014-09-18 21:28:49 +00:00
Rafael Espindola 51bd8ee309 Internalize common symbols when we can.
This fixes pr20974.

llvm-svn: 217981
2014-09-17 20:41:13 +00:00
Rafael Espindola 71143ed24b Remember to eraseFromParent after replaceAllUsesWith.
llvm-svn: 217536
2014-09-10 19:39:41 +00:00
Rafael Espindola 890db27b67 Handle common linkage correctly in the gold plugin.
This is the plugin version of pr20882.

This handles the case of every common symbol being in the IR. We will need some
support from gold to handle the case where some symbols are in ELF and some in
the IR.

llvm-svn: 217458
2014-09-09 20:08:22 +00:00
Rafael Espindola 262b90321a The gold tests also require ppc to be compiled in.
We could create a tools/gold/PowerPC and a tools/gold/X86, but it doesn't seem
worth it.

llvm-svn: 217267
2014-09-05 19:01:12 +00:00
Rafael Espindola f1d2fc657b Update to not depend on "llvm-objdump -d -symbolize".
llvm-svn: 217047
2014-09-03 16:16:02 +00:00
Rafael Espindola f7ecb11572 Add support for comdats to the gold plugin.
There are two parts to this. First, the plugin needs to tell gold the comdat by
setting comdat_key.

What gets things a bit more complicated is that gold only seems
symbols. In particular, if A is an alias to B, it only sees the symbols
A and B. It can then ask us to keep symbol A but drop symbol B. What
we have to do instead is to create an internal version of B and make A
an alias to that.

At some point some of this logic should be moved to lib/Linker so that
we don't map a Constant to an internal version just to have lib/Linker
map that again to the destination module.

The reason for implementing this in tools/gold for now is simplicity.
With it in place it should be possible to update clang to use comdats
for constructors and destructors on ELF without breaking the LTO
bootstrap. Once that is done I intend to come back and improve the
interface lib/Linker exposes.

llvm-svn: 216302
2014-08-22 23:26:10 +00:00
Rafael Espindola 33466a745e Rewrite the gold plugin to fix pr19901.
There is a fundamental difference between how the gold API and lib/LTO view
the LTO process.

The gold API talks about a particular symbol in a particular file. The lib/LTO
API talks about a symbol in the merged module.

The merged module is then defined in terms of the IR semantics. In particular,
a linkonce_odr GV is only copied if it is used, since it is valid to drop
unused linkonce_odr GVs.

In the testcase in pr19901 both properties collide. What happens is that gold
asks us to keep a particular linkonce_odr symbol, but the IR linker doesn't
copy it to the merged module and we never have a chance to ask lib/LTO to keep
it.

This patch fixes it by having a more direct implementation of the gold API. If
it asks us to keep a symbol, we change the linkage so it is not linkonce. If it
says we can drop a symbol, we do so. All of this before we even send the module
to lib/Linker.

Since now we don't have to produce LTO_SYMBOL_SCOPE_DEFAULT_CAN_BE_HIDDEN,
during symbol resolution we can use a temporary LLVMContext and do lazy
module loading. This allows us to keep the minimum possible amount of
allocated memory around. This should also allow as much parallelism as
we want, since there is no shared context.

llvm-svn: 216215
2014-08-21 20:28:55 +00:00
Rafael Espindola daa117be4c Make the test a bit more strict.
Before it would pass even if @b or @c ended up pointing to a variable named
@a123.

llvm-svn: 215450
2014-08-12 15:55:27 +00:00
Rafael Espindola cb9cd7f913 Add a plugin testcase for merging weak variables.
I initially thought I could implement COMDATs with aliases by just
internalizing GVs instead of dropping them. This is a counter
example: Internalizing one of the @a would make @b and @c point
to different variables.

llvm-svn: 215447
2014-08-12 15:39:14 +00:00
Rafael Espindola 55b3254ea2 Fix using -plugin-opt=apiflie when also using -plugin-opt=emit-llvm.
llvm-svn: 215378
2014-08-11 19:06:54 +00:00
Rafael Espindola cc847b6376 Fix test failure on ARM.
llvm-svn: 215140
2014-08-07 20:33:06 +00:00
Rafael Espindola c03b6e7880 Add a test showing the interaction of linker scripts and plugin.
In particular, the linker script is processed early enough for function g
to be internalized.

llvm-svn: 214916
2014-08-05 19:56:53 +00:00
Rafael Espindola 21ab24edfb Add a small test showing when a linkonce_odr symbol can be hidden.
llvm-svn: 214311
2014-07-30 15:31:21 +00:00
Rafael Espindola 6c472e5e14 gold plugin: Fix handling of corrupted bitcode files.
We should still claim them and tell gold about the error.

llvm-svn: 214214
2014-07-29 20:46:19 +00:00
Rafael Espindola 340aae797d Add a test for the mtriple plugin option.
llvm-svn: 214186
2014-07-29 17:27:07 +00:00
Rafael Espindola 99298fc36a Test the linker plugin handling of llvm.used.
llvm-svn: 214116
2014-07-28 20:42:29 +00:00
Rafael Espindola 6dca51502a Add tests for the various emit-llvm plugin options.
llvm-svn: 214102
2014-07-28 17:37:25 +00:00
Rafael Espindola 2172f51e4e Test the mcpu option.
llvm-svn: 214087
2014-07-28 14:44:33 +00:00
Rafael Espindola 852bad0a95 Start adding some tests for the gold plugin.
These are only used when the 'ld' in the path is gold and the plugin has
been built, but it is already a start to make sure we don't regress features
that cannot be tested with llvm-lto.

llvm-svn: 214058
2014-07-27 23:11:06 +00:00