This restores 59733525d3 (D71913), along
with bot fix 19c76989bb.
The bot failure should be fixed by D73418, committed as
af954e441a.
I also added a fix for non-x86 bot failures by requiring x86 in new test
lld/test/ELF/lto/devirt_vcall_vis_public.ll.
Summary:
Third part in series to support Safe Whole Program Devirtualization
Enablement, see RFC here:
http://lists.llvm.org/pipermail/llvm-dev/2019-December/137543.html
This patch adds type test metadata under -fwhole-program-vtables,
even for classes without hidden visibility. It then changes WPD to skip
devirtualization for a virtual function call when any of the compatible
vtables has public vcall visibility.
Additionally, internal LLVM options as well as lld and gold-plugin
options are added which enable upgrading all public vcall visibility
to linkage unit (hidden) visibility during LTO. This enables the more
aggressive WPD to kick in based on LTO time knowledge of the visibility
guarantees.
Support was added to all flavors of LTO WPD (regular, hybrid and
index-only), and to both the new and old LTO APIs.
Unfortunately it was not simple to split the first and second parts of
this part of the change (the unconditional emission of type tests and
the upgrading of the vcall visiblity) as I needed a way to upgrade the
public visibility on legacy WPD llvm assembly tests that don't include
linkage unit vcall visibility specifiers, to avoid a lot of test churn.
I also added a mechanism to LowerTypeTests that allows dropping type
test assume sequences we now aggressively insert when we invoke
distributed ThinLTO backends with null indexes, which is used in testing
mode, and which doesn't invoke the normal ThinLTO backend pipeline.
Depends on D71907 and D71911.
Reviewers: pcc, evgeny777, steven_wu, espindola
Subscribers: emaste, Prazek, inglorion, arichardson, hiraditya, MaskRay, dexonsmith, dang, davidxl, cfe-commits, llvm-commits
Tags: #clang, #llvm
Differential Revision: https://reviews.llvm.org/D71913
Summary:
This patch could be treated as a rebase of D33960. It also fixes PR35547.
A fix for `llvm/test/Other/close-stderr.ll` is proposed in D68164. Seems
the consensus is that the test is passing by chance and I'm not
sure how important it is for us. So it is removed like in D33960 for now.
The rest of the test fixes are just adding `--crash` flag to `not` tool.
** The reason it fixes PR35547 is
`exit` does cleanup including calling class destructor whereas `abort`
does not do any cleanup. In multithreading environment such as ThinLTO or JIT,
threads may share states which mostly are ManagedStatic<>. If faulting thread
tearing down a class when another thread is using it, there are chances of
memory corruption. This is bad 1. It will stop error reporting like pretty
stack printer; 2. The memory corruption is distracting and nondeterministic in
terms of error message, and corruption type (depending one the timing, it
could be double free, heap free after use, etc.).
Reviewers: rnk, chandlerc, zturner, sepavloff, MaskRay, espindola
Reviewed By: rnk, MaskRay
Subscribers: wuzish, jholewinski, qcolombet, dschuff, jyknight, emaste, sdardis, nemanjai, jvesely, nhaehnle, sbc100, arichardson, jgravelle-google, aheejin, kbarton, fedor.sergeev, asb, rbar, johnrusso, simoncook, apazos, sabuasal, niosHD, jrtc27, zzheng, edward-jones, atanasyan, rogfer01, MartinMosbeck, brucehoult, the_o, PkmX, jocewei, jsji, lenary, s.egerton, pzheng, cfe-commits, MaskRay, filcab, davide, MatzeB, mehdi_amini, hiraditya, steven_wu, dexonsmith, rupprecht, seiya, llvm-commits
Tags: #llvm, #clang
Differential Revision: https://reviews.llvm.org/D67847
down to pass builder in ltobackend.
Currently CodeGenOpts like UnrollLoops/VectorizeLoop/VectorizeSLP in clang
are not passed down to pass builder in ltobackend when new pass manager is
used. This is inconsistent with the behavior when new pass manager is used
and thinlto is not used. Such inconsistency causes slp vectorization pass
not being enabled in ltobackend for O3 + thinlto right now. This patch
fixes that.
Differential Revision: https://reviews.llvm.org/D72386
Test files in the same directory share the same %T. %T is easy to
misuse and cause race conditions (when running concurrently) so it has
been deprecated since D48842 (see docs/CommandGuide/lit.rst).
While here, add `rm -rf %t.dir` so that tests cannot depend on old files lying around.
Reviewed By: jhenderson, ruiu
Differential Revision: https://reviews.llvm.org/D69572
The combination of the two flags doesn't make sense. And other linkers
seem to just ignore --export-dynamic if --relocatable is given, but
we probably should report it as an error to let users know that is
an invalid combination.
Fixes https://bugs.llvm.org/show_bug.cgi?id=43552
Differential Revision: https://reviews.llvm.org/D68441
llvm-svn: 374022
Summary:
When support for ThinLTO was first added to lld, the options that
control it were prefixed with --plugin-opt= for compatibility with
an existing implementation as a linker plugin. This change enables
shorter versions of the options to be used, as follows:
New Existing
-thinlto-emit-imports-files --plugin-opt=thinlto-emit-imports-files
-thinlto-index-only --plugin-opt=thinlto-index-only
-thinlto-index-only= --plugin-opt=thinlto-index-only=
-thinlto-object-suffix-replace= --plugin-opt=thinlto-object-suffix-replace=
-thinlto-prefix-replace= --plugin-opt=thinlto-prefix-replace=
-lto-obj-path= --plugin-opt=obj-path=
The options with the --plugin-opt= prefix have been retained as aliases
for the shorter variants so that they continue to be accepted.
Reviewers: tejohnson, ruiu, espindola
Reviewed By: ruiu
Subscribers: emaste, arichardson, MaskRay, steven_wu, dexonsmith, arphaman, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D67782
llvm-svn: 372798
D64136 and D65584, while fixing STB_WEAK issues and improving our
compatibility with ld.bfd, can cause another STB_WEAK problem related to
LTO:
If %tundef.o has an undefined reference on f,
and %tweakundef.o has a weak undefined reference on f,
%tdef.o has a definition of f
```
ld.lld %tundef.o %tweakundef.o --start-lib %tdef.o --end-lib
```
1) `%tundef.o` doesn't set the `referenced` bit.
2) `%weakundef.o` changes the binding from STB_GLOBAL to STB_WEAK
3) `%tdef.o` is not fetched because the binding is weak.
Step (1) is incorrect. This patch sets the `referenced` bit of Undefined
created by bitcode files.
Reviewed By: ruiu
Differential Revision: https://reviews.llvm.org/D66992
llvm-svn: 370437
lld currently selects the relocation model automatically depending on
the link flags specified, but in some cases it'd be useful to allow
explicitly overriding the relocation model using a flag.
llvm-svn: 366644
This matches the wasm lld and GNU ld behavior.
The ELF linker has special handling for bitcode archives but if that
doesn't kick in we probably want to error out rather than silently
ignore the library.
Differential Revision: https://reviews.llvm.org/D63781
llvm-svn: 364998
Add Triple::riscv64 and Triple::riscv32 to getBitcodeMachineKind for get right
e_machine during LTO.
Reviewed By: ruiu, MaskRay
Differential Revision: https://reviews.llvm.org/D52165
llvm-svn: 364996
r360841 introduced CommonSymbol class. An unintended behavioral change
introduced by that change was that common symbols are not internalized
by LTO under some condition. This patch fixes that issue.
The issue occurred under the following condition:
1. There exists a common symbol
2. At least one DSO is given to lld or -pie is used
If the above conditions are met, Symbol::includeInDynsym() returned a
wrong value for a common symbol.
Fixes https://bugs.llvm.org/show_bug.cgi?id=41978
Differential Revision: https://reviews.llvm.org/D63752
llvm-svn: 364273
Use -fsave-optimization-record=<format> to specify a different format
than the default, which is YAML.
For now, only YAML is supported.
llvm-svn: 363573
This patch implements a limited form of autolinking primarily designed to allow
either the --dependent-library compiler option, or "comment lib" pragmas (
https://docs.microsoft.com/en-us/cpp/preprocessor/comment-c-cpp?view=vs-2017) in
C/C++ e.g. #pragma comment(lib, "foo"), to cause an ELF linker to automatically
add the specified library to the link when processing the input file generated
by the compiler.
Currently this extension is unique to LLVM and LLD. However, care has been taken
to design this feature so that it could be supported by other ELF linkers.
The design goals were to provide:
- A simple linking model for developers to reason about.
- The ability to to override autolinking from the linker command line.
- Source code compatibility, where possible, with "comment lib" pragmas in other
environments (MSVC in particular).
Dependent library support is implemented differently for ELF platforms than on
the other platforms. Primarily this difference is that on ELF we pass the
dependent library specifiers directly to the linker without manipulating them.
This is in contrast to other platforms where they are mapped to a specific
linker option by the compiler. This difference is a result of the greater
variety of ELF linkers and the fact that ELF linkers tend to handle libraries in
a more complicated fashion than on other platforms. This forces us to defer
handling the specifiers to the linker.
In order to achieve a level of source code compatibility with other platforms
we have restricted this feature to work with libraries that meet the following
"reasonable" requirements:
1. There are no competing defined symbols in a given set of libraries, or
if they exist, the program owner doesn't care which is linked to their
program.
2. There may be circular dependencies between libraries.
The binary representation is a mergeable string section (SHF_MERGE,
SHF_STRINGS), called .deplibs, with custom type SHT_LLVM_DEPENDENT_LIBRARIES
(0x6fff4c04). The compiler forms this section by concatenating the arguments of
the "comment lib" pragmas and --dependent-library options in the order they are
encountered. Partial (-r, -Ur) links are handled by concatenating .deplibs
sections with the normal mergeable string section rules. As an example, #pragma
comment(lib, "foo") would result in:
.section ".deplibs","MS",@llvm_dependent_libraries,1
.asciz "foo"
For LTO, equivalent information to the contents of a the .deplibs section can be
retrieved by the LLD for bitcode input files.
LLD processes the dependent library specifiers in the following way:
1. Dependent libraries which are found from the specifiers in .deplibs sections
of relocatable object files are added when the linker decides to include that
file (which could itself be in a library) in the link. Dependent libraries
behave as if they were appended to the command line after all other options. As
a consequence the set of dependent libraries are searched last to resolve
symbols.
2. It is an error if a file cannot be found for a given specifier.
3. Any command line options in effect at the end of the command line parsing apply
to the dependent libraries, e.g. --whole-archive.
4. The linker tries to add a library or relocatable object file from each of the
strings in a .deplibs section by; first, handling the string as if it was
specified on the command line; second, by looking for the string in each of the
library search paths in turn; third, by looking for a lib<string>.a or
lib<string>.so (depending on the current mode of the linker) in each of the
library search paths.
5. A new command line option --no-dependent-libraries tells LLD to ignore the
dependent libraries.
Rationale for the above points:
1. Adding the dependent libraries last makes the process simple to understand
from a developers perspective. All linkers are able to implement this scheme.
2. Error-ing for libraries that are not found seems like better behavior than
failing the link during symbol resolution.
3. It seems useful for the user to be able to apply command line options which
will affect all of the dependent libraries. There is a potential problem of
surprise for developers, who might not realize that these options would apply
to these "invisible" input files; however, despite the potential for surprise,
this is easy for developers to reason about and gives developers the control
that they may require.
4. This algorithm takes into account all of the different ways that ELF linkers
find input files. The different search methods are tried by the linker in most
obvious to least obvious order.
5. I considered adding finer grained control over which dependent libraries were
ignored (e.g. MSVC has /nodefaultlib:<library>); however, I concluded that this
is not necessary: if finer control is required developers can fall back to using
the command line directly.
RFC thread: http://lists.llvm.org/pipermail/llvm-dev/2019-March/131004.html.
Differential Revision: https://reviews.llvm.org/D60274
llvm-svn: 360984
Module IDs can appear in diagnostic messages.
This patch adds some auxiliary symbols to improve their readability.
Differential Revision: https://reviews.llvm.org/D61857
llvm-svn: 360858
Summary:
The gold plugin behavior (creating empty index files for lazy bitcode
files) was added in D46034, but it missed the case when there is no
non-lazy bitcode files, e.g.
ld.lld -shared crti.o crtbeginS.o --start-lib bitcode.o --end-lib ...
crti.o crtbeginS.o are not bitcode, but our distributed build system
wants bitcode.o.thinlto.bc to confirm all expected outputs are created
based on all of the modules provided to the linker.
Differential Revision: https://reviews.llvm.org/D61420
llvm-svn: 359788
Also change some options that have different semantics (cause confusion) in llvm-readelf mode:
-s => -S
-t => --symbols
-sd => --section-data
llvm-svn: 359651
D58391 changed the LTO pipelines to add the tailcall elimination pass.
This caused three LLD tests to fail.
Differential Revision: https://reviews.llvm.org/D59604
llvm-svn: 356593
We allow an archive file without symbol table as a linker input as a
workaround for a very common error in LTO build. But that logic worked
even for an archive file containing non-bitcode files, which is not
expected. This patch limits that workaround to one that contains only
bitcode files.
Differential Revision: https://reviews.llvm.org/D59373
llvm-svn: 356186
Old: PT_LOAD(.data | PT_GNU_RELRO(.data.rel.ro .bss.rel.ro) | .bss)
New: PT_LOAD(PT_GNU_RELRO(.data.rel.ro .bss.rel.ro) | .data .bss)
The placement of | indicates page alignment caused by PT_GNU_RELRO. The
new layout has simpler rules and saves space for many cases.
Old size: roundup(.data) + roundup(.data.rel.ro)
New size: roundup(.data.rel.ro + .bss.rel.ro) + .data
Other advantages:
* At runtime the 3 memory mappings decrease to 2.
* start(PT_TLS) = start(PT_GNU_RELRO) = start(RW PT_LOAD). This
simplifies binary manipulation tools.
GNU strip before 2.31 discards PT_GNU_RELRO if its
address is not equal to the start of its associated PT_LOAD.
This has been fixed by https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=f2731e0c374e5323ce4cdae2bcc7b7fe22da1a6f
But with this change, we will be compatible with GNU strip before 2.31
* Before, .got.plt (non-relro by default) was placed before .got (relro
by default), which made it impossible to have _GLOBAL_OFFSET_TABLE_
(start of .got.plt on x86-64) equal to the end of .got (R_GOT*_FROM_END)
(https://bugs.llvm.org/show_bug.cgi?id=36555). With the new ordering, we
can improve on this regard if we'd like to.
Reviewers: ruiu, espindola, pcc
Subscribers: emaste, arichardson, llvm-commits, joerg, jdoerfert
Differential Revision: https://reviews.llvm.org/D56828
llvm-svn: 356117
Currently we have -Rpass for filtering the remarks that are displayed as
diagnostics, but when using -fsave-optimization-record, there is no way
to filter the remarks while generating them.
This adds support for filtering remarks by passes using a regex.
Ex: `clang -fsave-optimization-record -foptimization-record-passes=inline`
will only emit the remarks coming from the pass `inline`.
This adds:
* `-fsave-optimization-record` to the driver
* `-opt-record-passes` to cc1
* `-lto-pass-remarks-filter` to the LTOCodeGenerator
* `--opt-remarks-passes` to lld
* `-pass-remarks-filter` to llc, opt, llvm-lto, llvm-lto2
* `-opt-remarks-passes` to gold-plugin
Differential Revision: https://reviews.llvm.org/D59268
Original llvm-svn: 355964
llvm-svn: 355984
Currently we have -Rpass for filtering the remarks that are displayed as
diagnostics, but when using -fsave-optimization-record, there is no way
to filter the remarks while generating them.
This adds support for filtering remarks by passes using a regex.
Ex: `clang -fsave-optimization-record -foptimization-record-passes=inline`
will only emit the remarks coming from the pass `inline`.
This adds:
* `-fsave-optimization-record` to the driver
* `-opt-record-passes` to cc1
* `-lto-pass-remarks-filter` to the LTOCodeGenerator
* `--opt-remarks-passes` to lld
* `-pass-remarks-filter` to llc, opt, llvm-lto, llvm-lto2
* `-opt-remarks-passes` to gold-plugin
Differential Revision: https://reviews.llvm.org/D59268
llvm-svn: 355964
This unit test fails if 'zed' appears in your build path, as lld is
echoing out the path of the source file. Change the unit test to
explicitly only match everything after the 'Symbols [' line of the
output, to make sure that this test doesn't inadvertently fail due to
the build path.
llvm-svn: 355748
If the permission does not permit writing and the standard input is a
terminal, rm gives an annoying prompt. chmod u+w to make the output
directory easier to delete.
llvm-svn: 355382
Summary:
The gold linker allowed you to output the ELF files after LTO was run. It did
it by using the 'obj-path' option. This replicates that behavior.
Reviewers: espindola, ruiu, MaskRay, pcc
Reviewed By: MaskRay, pcc
Subscribers: grimar, emaste, inglorion, arichardson, steven_wu, dexonsmith, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D56046
llvm-svn: 354917
llvm-readobj currently has a bug (see PR40097) where it prints '@' at
the end of unversioned dynamic symbols. This bug will be fixed in a
separate later commit, but these tests need fixing first.
Reviewed by: ruiu, Higuoxing
Differential Revision: https://reviews.llvm.org/D56388
llvm-svn: 350614
`--plugin-opt=emit-llvm` is an option for LTO. It makes the linker to
combine all bitcode files and write the result to an output file without
doing codegen. Gold LTO plugin has this option.
This option is being used for some post-link code analysis tools that
have to see a whole program but don't need to see them in the native
machine code.
Differential Revision: https://reviews.llvm.org/D55717
llvm-svn: 349198
Summary: llvm-readobj/readelf accepts both -s and -S as aliases for --sections. However with GNU readelf only -S means --section, and -s means --symbols. I would like to make llvm-readelf more compatible.
Reviewers: MaskRay, espindola
Reviewed By: MaskRay
Subscribers: emaste, arichardson, steven_wu, dexonsmith, llvm-commits
Differential Revision: https://reviews.llvm.org/D54118
llvm-svn: 346164
Summary:
It is difficult to touch a file with a relative mtime across different OSes as POSIX touch -d is rigid. While we may construct relative timestamps with `date`, POSIX date is inadequate to do so as various OSes' date do not agree on a common format (OpenBSD uses `date -r seconds`, FreeBSD uses `date -v-2M` while GNU accepts `-d '-2 min'`)
Just use python os.utime()
Original description:
The case may randomly fail if we test it with command "
while llvm-lit tools/lld/test/ELF/lto/cache.ll; do true; done". It is because the llvmcache-foo file is younger than llvmcache-349F039B8EB076D412007D82778442BED3148C4E and llvmcache-A8107945C65C2B2BBEE8E61AA604C311D60D58D6. But due to timestamp precision reason their timestamp is the same. Given the same timestamp, the file prune policy is to remove bigger size file first, so mostly foo file is removed for its bigger size. And the files size is under threshold after deleting foo file. That's what test case expect.
However sometimes, the precision is enough to measure that timestamp of llvmcache-349F039B8EB076D412007D82778442BED3148C4E and llvmcache-A8107945C65C2B2BBEE8E61AA604C311D60D58D6 are smaller than foo, so llvmcache-349F039B8EB076D412007D82778442BED3148C4E and llvmcache-A8107945C65C2B2BBEE8E61AA604C311D60D58D6 are deleted first. Since the files size is still above the file size threshold after deleting the 2 files, the foo file is also deleted. And then the test case fails, because it expect only one file should be deleted instead of 3.
The fix is to change the timestamp of llvmcache-foo file to meet the thinLTO prune policy.
The same fix is applied to llvm code at https://reviews.llvm.org/D52452.
Patch by Luo Yuanke.
Reviewers: ruiu, craig.topper, smaslov, Jianping, espindola, LuoYuanke, tejohnson
Reviewed By: tejohnson
Subscribers: sbc100, krytarowski, aheejin, llvm-commits, dexonsmith, steven_wu, arichardson, inglorion, emaste, bjope, rupprecht
Differential Revision: https://reviews.llvm.org/D54039
llvm-svn: 346006
Teresa Johnson