Summary: We needed to use `Joined` instead of `Flag`. This wasn't caught
because the relevant test that was copied from LLD-ELF was still
invoking LLD-ELF instead of LLD-MachO...
Differential Revision: https://reviews.llvm.org/D99313
Pretty simple code-wise. Also threw in some refactoring:
* Put the functionStartSection under Writer instead of InStruct, since
it doesn't need to be accessed outside of Writer
* Adjusted the test to put all files under the temp dir instead of at
the top-level
* Added some CHECK-LABELs to make it clearer where the function starts
data is
Differential Revision: https://reviews.llvm.org/D99112
I added just enough to allow us to see a top-level breakdown of time taken. This
is the result of loading the time-trace output into `chrome:://tracing`:
ef5e8234f3/tracing.png
Reviewed By: oontvoo
Differential Revision: https://reviews.llvm.org/D99311
In future patches I will be setting the IsText parameter frequently so I will refactor the args to be in the following order. I have removed the FileSize parameter because it is never used.
```
static ErrorOr<std::unique_ptr<MemoryBuffer>>
getFile(const Twine &Filename, bool IsText = false,
bool RequiresNullTerminator = true, bool IsVolatile = false);
static ErrorOr<std::unique_ptr<MemoryBuffer>>
getFileOrSTDIN(const Twine &Filename, bool IsText = false,
bool RequiresNullTerminator = true);
static ErrorOr<std::unique_ptr<MB>>
getFileAux(const Twine &Filename, uint64_t MapSize, uint64_t Offset,
bool IsText, bool RequiresNullTerminator, bool IsVolatile);
static ErrorOr<std::unique_ptr<WritableMemoryBuffer>>
getFile(const Twine &Filename, bool IsVolatile = false);
```
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D99182
Don't run the 'tar' tool in a cleared environment with only the
LANG variable set, just set LANG on top of the existing environment.
If the 'tar' tool is an MSYS based tool, running it in a Windows
Container hangs if all environment variables are cleared - in
particular, the USERPROFILE variable needs to be kept intact.
This is the same issue fixed as was fixed in other places in
9de63b2e05, but contrary to running
the actual tests, running with an as-cleared-as-possible environment
here is less important.
Differential Revision: https://reviews.llvm.org/D99304
This patch renames the "Initial" member of WasmLimits to the name used
in the spec, "Minimum".
In the core WebAssembly specification, the Limits data type has one
required "min" member and one optional "max" member, indicating the
minimum required size of the corresponding table or memory, and the
maximum size, if any.
Although the WebAssembly spec does instantiate locally-defined tables
and memories with the initial size being equal to the minimum size, it
can't impose such a requirement for imports. It doesn't make sense to
require an initial size for a memory import, for example. The compiler
can only sensibly express the minimum and maximum sizes.
See
https://github.com/WebAssembly/js-types/blob/master/proposals/js-types/Overview.md#naming-of-size-limits
for a related discussion that agrees that the right name of "initial" is
"minimum" when querying the type of a table or memory from JavaScript.
(Of course it still makes sense for JS to speak in terms of an initial
size when it explicitly instantiates memories and tables.)
Differential Revision: https://reviews.llvm.org/D99186
This reverts commit 4876ba5b2d.
Third-attemp relanding D98559, new change:
- explicitly cast enum to underlying type to avoid ambiguity (workaround to clang's bug).
This reverts commit 3c21166a94.
The build is broken (clang-8 host compiler):
lld/MachO/DriverUtils.cpp:271:8: error: use of overloaded operator '<<' is ambiguous (with operand types 'llvm::raw_fd_ostream' and 'lld::macho::DependencyTracker::DepOpCode')
os << opcode;
~~ ^ ~~~~~~
This reverts commit 9670d2e4af.
Second attemp to reland D98559. New changes:
- inline functions removed from cpp file.
- updated tests to use CHECK-DAG instead of CHECK-NEXT
- fixed ambiguous "<<" operator by switching `char` to uint8_t
The only known reason why ICF should not merge otherwise identical
sections with differing associated sections has to do with exception
handling tables. It's not clear what ICF should do when there are other
kinds of associated sections. In every other case when this has come up,
debug info and CF guard metadata, we have opted to make ICF ignore the
associated sections.
For comparison, ELF doesn't do anything for comdat groups. Instead,
.eh_frame is parsed to figure out if a section has an LSDA, and if so,
ICF is disabled.
Another issue is that the order of associated sections is not defined.
We have had issues in the past (crbug.com/1144476) where changing the
order of the .xdata/.pdata sections in the object file lead to large ICF
slowdowns.
To address these issues, I decided it would be best to explicitly
consider only .pdata and .xdata sections during ICF. This makes it easy
to ignore the object file order, and I think it makes the intention of
the code clearer.
I've also made the children() accessor return an empty list for
associated sections. This mostly only affects ICF and GC. This was the
behavior before I made this a linked list, so the behavior change should
be good. This had positive effects on chrome.dll: more .xdata sections
were merged that previously could not be merged because they were
associated with distinct .pdata sections.
Reviewed By: mstorsjo
Differential Revision: https://reviews.llvm.org/D98993
This reverts commit 2554b95db5.
Relanding [lld-macho] Implement -dependency_info (D98559) with changes:
- inline functions removed from cpp file.
- updated tests to not check libSystem.tbd with other input files (because of possible indeterministic ordering)
There is a bug when initial exec is relaxed to local exec.
In the following situation:
InitExec.c
```
extern __thread unsigned TGlobal;
unsigned getConst(unsigned*);
unsigned addVal(unsigned, unsigned*);
unsigned GetAddrT() {
return addVal(getConst(&TGlobal), &TGlobal);
}
```
Def.c
```
__thread unsigned TGlobal;
unsigned getConst(unsigned* A) {
return *A + 3;
}
unsigned addVal(unsigned A, unsigned* B) {
return A + *B;
}
```
The problem is in InitExec.c but Def.c is required if you want to link the example and see the problem.
To compile everything:
```
clang -O3 -mcpu=pwr10 -c InitExec.c
clang -O3 -mcpu=pwr10 -c Def.c
ld.lld InitExec.o Def.o -o IeToLe
```
If you objdump the problem object file:
```
$ llvm-objdump -dr --mcpu=pwr10 InitExec.o
```
you will get the following assembly:
```
0000000000000000 <GetAddrT>:
0: a6 02 08 7c mflr 0
4: f0 ff c1 fb std 30, -16(1)
8: 10 00 01 f8 std 0, 16(1)
c: d1 ff 21 f8 stdu 1, -48(1)
10: 00 00 10 04 00 00 60 e4 pld 3, 0(0), 1
0000000000000010: R_PPC64_GOT_TPREL_PCREL34 TGlobal
18: 14 6a c3 7f add 30, 3, 13
0000000000000019: R_PPC64_TLS TGlobal
1c: 78 f3 c3 7f mr 3, 30
20: 01 00 00 48 bl 0x20
0000000000000020: R_PPC64_REL24_NOTOC getConst
24: 78 f3 c4 7f mr 4, 30
28: 30 00 21 38 addi 1, 1, 48
2c: 10 00 01 e8 ld 0, 16(1)
30: f0 ff c1 eb ld 30, -16(1)
34: a6 03 08 7c mtlr 0
38: 00 00 00 48 b 0x38
0000000000000038: R_PPC64_REL24_NOTOC addVal
```
The lines of interest are:
```
10: 00 00 10 04 00 00 60 e4 pld 3, 0(0), 1
0000000000000010: R_PPC64_GOT_TPREL_PCREL34 TGlobal
18: 14 6a c3 7f add 30, 3, 13
0000000000000019: R_PPC64_TLS TGlobal
1c: 78 f3 c3 7f mr 3, 30
```
Which once linked gets turned into:
```
10010210: ff ff 03 06 00 90 6d 38 paddi 3, 13, -28672, 0
10010218: 00 00 00 60 nop
1001021c: 78 f3 c3 7f mr 3, 30
```
The problem is that register 30 is never set after the optimization.
Therefore it is not correct to relax the above instructions by replacing
the add instruction with a nop.
Instead the add instruction should be replaced with a copy (mr) instruction.
If the add uses the same resgiter as input and as ouput then it is safe to
continue to replace the add with a nop.
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D95262
This reverts commit c53a1322f3.
Test only passes depending on build dir having a lexicographically later name
than the source dir, and doesn't link on mac/win. See
https://reviews.llvm.org/D98559#2640265 onward.
Bug: https://bugs.llvm.org/show_bug.cgi?id=49278
The flag is not well documented, so this implementation is based on observed behaviour.
When specified, `-dependency_info <path>` produced a text file containing information pertaining to the current linkage, such as input files, output file, linker version, etc.
This file's layout is also not documented, but it seems to be a series of null ('\0') terminated strings in the form `<op code><path>`
`<op code>` could be:
`0x00` : linker version
`0x10` : input
`0x11` : files not found(??)
`0x40` : output
`<path>` : is the file path, except for the linker-version case.
(??) This part is a bit unclear. I think it means all the files the linker attempted to look at, but could not find.
Differential Revision: https://reviews.llvm.org/D98559
`--shuffle-sections=<seed>` applies to all sections. The new
`--shuffle-sections=<section-glob>=<seed>` makes shuffling selective. To the
best of my knowledge, the option is only used as debugging, so just drop the
original form.
`--shuffle-sections '.init_array*=-1'` `--shuffle-sections '.fini_array*=-1'`.
reverses static constructors/destructors of the same priority.
Useful to detect some static initialization order fiasco.
`--shuffle-sections '.data*=-1'`
reverses `.data*` sections. Useful to detect unfunded pointer comparison results
of two unrelated objects.
If certain sections have an intrinsic order, the old form cannot be used.
Differential Revision: https://reviews.llvm.org/D98679
Move some functions closer to their uses. Move detailed address-assignment logic out of the otherwise abstract `Writer::run()`. This prepares the ground for a diff to implement branch range extension thunks.
* `SyntheticSections.cpp`
** move `needsBinding()` and `prepareBranchTarget()` into `Writer.cpp`
** move `addNonLazyBindingEntries()` adjacent to its use.
* `Writer.cpp`
** move address-assignment logic from `Writer::run()` into new function `Writer::assignAddresses()`
** move `needsBinding()` and `prepareBranchTarget()` from `SyntheticSections.cpp`
* `Target.h`
** remove orphaned decls of `prepareSymbolRelocation()` and `validateRelocationInfo()` which were moved to other files in earlier diffs.
Differential Revision: https://reviews.llvm.org/D98795
If the number of sections changes, which is common for re-links after
incremental updates, the section order may change drastically.
Special case -1 to reverse input sections. This is a stable transform.
The section order is more resilient to incremental updates. Usually the
code issue (e.g. Static Initialization Order Fiasco, assuming pointer
comparison result of two unrelated objects) is due to the relative order
between two problematic input files A and B. Checking the regular order
and the reversed order is sufficient.
Differential Revision: https://reviews.llvm.org/D98445
This fixes defects in D98223 [lld-macho] implement options -(un)exported_symbol(s_list):
* disallow export of hidden symbols
* verify that whitelisted literal names are defined in the symbol table
* reflect export-status overrides in `nlist` attribute of `N_EXT` or `N_PEXT`
Thanks to @thakis for raising these issues
Differential Revision: https://reviews.llvm.org/D98381
Visual Studios implementation of the C++ Standard Library does not use strerror to produce a message for std::error_code unlike other standard libraries such as libstdc++ or libc++ that might be used.
This patch adds a cmake script that through running a C++ program gets the error messages for the POSIX error codes and passes them onto lit through an optional config parameter.
If the config parameter is not set, or getting the messages failed, due to say a cross compiling configuration without an emulator, it will fall back to using pythons strerror functions.
Differential Revision: https://reviews.llvm.org/D98278
This pleases the codesign
(Otherwise it complains about "function starts data out of place")
Reviewed By: #lld-macho, smeenai
Differential Revision: https://reviews.llvm.org/D98648
The lit test suite uses python 3.6 features. Rather than a strange
python syntax error upon running the lit tests, we will require the
correct version in CMake.
Reviewed By: serge-sans-paille, yln
Differential Revision: https://reviews.llvm.org/D95635
They were previously in SyntheticSections.h, but now there are
a bunch of non-synthetic section names in the list.
Also renamed `__functionStarts` to `__func_starts` for uniformity with
other section names + keeps the name under 16 characters (in case we ever
want to write it out as a real section).
Reviewed By: #lld-macho, compnerd
Differential Revision: https://reviews.llvm.org/D98586
Summary: The exact out-of-range value seems to differ by 8 bytes on the
buildbots compared to my local machine. I'm guessing it has something to
do with what inputs we are getting from llvm-mc. Not sure why, but I
don't think it's super important -- let's just ignore the number, the
out-of-range message is the important thing here
Previously, it was difficult to write code that handled both synthetic
and regular sections generically. We solve this problem by creating a
fake InputSection at the start of every SyntheticSection.
This refactor allows us to handle DSOHandle like a regular Defined
symbol (since Defined symbols must be attached to an InputSection), and
paves the way for supporting `__mh_*header` symbols. Additionally, it
simplifies our binding/rebase code.
I did have to extend Defined a little -- it now has a `linkerInternal`
flag, to indicate that `___dso_handle` should not be in the final symbol
table.
I've also added some additional testing for `___dso_handle`.
Reviewed By: #lld-macho, oontvoo
Differential Revision: https://reviews.llvm.org/D98545
This diff required fixing `getEmbeddedAddend` to apply sign
extension to 32-bit values. We were previously passing around wrong
64-bit addend values that became "right" after being truncated back to
32-bit.
I've also made `getEmbeddedAddend` return a signed int, which is similar
to what LLD-ELF does for its `getImplicitAddend`.
`reportRangeError`, `checkUInt`, and `checkInt` are counterparts of similar
functions in LLD-ELF.
Reviewed By: #lld-macho, thakis
Differential Revision: https://reviews.llvm.org/D98387
The S_[GL]PROC32_ID symbol records are supposed to point to function ID
records. If they don't, they are corrupt. The warning message here was
very technical, but a user has encountered it in the wild. Add some more
information and some more testing.
There are 3 remaining tests that still have `REQUIRE: shell`:
* color-diagnostics.test -- seems necessary for ANSI escape sequence support
* stabs.s -- the shell part could be removed, but I don't think we can support
the test on Windows anyway due to its reliance on `touch` to set the modtime
* framework.s -- uses symlinks, I'm not sure this works on Windows
Addresses PR49512.
Reviewed By: #lld-macho, alexshap
Differential Revision: https://reviews.llvm.org/D98395
SUBTRACTOR relocations are always paired with UNSIGNED
relocations to indicate a pair of symbols whose address difference we
want. Functionally they are like a single relocation: only one pointer
gets written / relocated. Previously, we would handle these pairs by
skipping over the SUBTRACTOR relocation and writing the pointer when
handling the UNSIGNED reloc. This diff reverses things, so we write
while handling SUBTRACTORs and skip over the UNSIGNED relocs instead.
Being able to distinguish between SUBTRACTOR and UNSIGNED relocs in the
write phase (i.e. inside `relocateOne`) is useful for the upcoming range
check diff: we want to check that SUBTRACTOR relocs write signed values,
but UNSIGNED relocs (naturally) write unsigned values.
Reviewed By: #lld-macho, thakis
Differential Revision: https://reviews.llvm.org/D98386
The previous implementation miscalculated the addend, resulting
in an underflow. This meant that every SIGNED_N section relocation would
be associated with the last subsection (since the addend would now be a
huge number). We were "lucky" that this mistake was typically cancelled
out -- 64-to-32-bit-truncation meant that the final value was correct,
as long as subsections were not rearranged.
Reviewed By: #lld-macho, thakis
Differential Revision: https://reviews.llvm.org/D98385
Previously, SyntheticSections.cpp did not have a top-level `using namespace
llvm::MachO` because it caused a naming conflict: `llvm::MachO::Symbol` would
collide with `lld::macho::Symbol`.
`MachO::Symbol` represents the symbols defined in InterfaceFiles (TBDs). By
moving the inclusion of InterfaceFile.h into our .cpp files, we can avoid this
name collision in other files where we are only dealing with LLD's own symbols.
Along the way, I removed all unnecessary "MachO::" prefixes in our code.
Cons of this approach: If TextAPI/MachO/Symbol.h gets included via some other
header file in the future, we could run into this collision again.
Alternative 1: Have either TextAPI/MachO or BinaryFormat/MachO.h use a different
namespace. Most of the benefit of `using namespace llvm::MachO` comes from being
able to use things in BinaryFormat/MachO.h conveniently; if TextAPI was under a
different (and fully-qualified) namespace like `llvm::tapi` that would solve our
problems. Cons: lots of files across llvm-project will need to be updated, and
folks who own the TextAPI code need to agree to the name change.
Alternative 2: Rename our Symbol to something like `LldSymbol`. I think this is
ugly.
Personally I think alternative #1 is ideal, but I'm not sure the effort to do it is
worthwhile, this diff's halfway solution seems good enough to me. Thoughts?
Reviewed By: #lld-macho, oontvoo, MaskRay
Differential Revision: https://reviews.llvm.org/D98149
GNU ld supports `.` and `$` in symbol names while LLD doesn't support them in
`readPrimary` expressions. Using `.` can result in such an error:
```
https://github.com/ClangBuiltLinux/linux/issues/1318
ld.lld: error: ./arch/powerpc/kernel/vmlinux.lds:255: malformed number: .TOC.
>>> __toc_ptr = (DEFINED (.TOC.) ? .TOC. : ADDR (.got)) + 0x8000;
```
Allow `.` (ppc64 special symbol `.TOC.`) and `$` (RISC-V special symbol `__global_pointer$`).
Change `diag[3-5].test` to use an invalid character `^`.
Note: GNU ld allows `~` in non-leading positions of a symbol name. `~`
is not used in practice, conflicts with the unary operator, and can
cause some parsing difficulty, so this patch does not add it.
Differential Revision: https://reviews.llvm.org/D98306
This is the minimal port from ELF. Any extension should easy from here
Test plan: ninja check-all-macho
Reviewed By: #lld-macho, thakis
Differential Revision: https://reviews.llvm.org/D98419
This reverts commit bacf9cf2c5 and
reinstates commit 1a9bd5b813.
Reverting this commit did not appear to make the problem go away, so we
can go ahead and reland it.
Pointer and reference induction variables of range-based for loops are often const, and code authors often lax about qualifying them.
Differential Revision: https://reviews.llvm.org/D98317
The flag doesn't (and shouldn't) have an effect in that case.
ld64 doesn't warn on this, but it seems like a good thing to do.
If it causes problems in practice for some reason, we can revert it.
Also add a dedicated test for install_name.
Differential Revision: https://reviews.llvm.org/D98259
lld doesn't read MH_DEAD_STRIPPABLE_DYLIB to strip dead dylibs yet,
but now it can produce dylibs with it set.
While here, also switch an existing test that looks only at the main Mach-O
header from --all-headers to --private-header.
Differential Revision: https://reviews.llvm.org/D98262
A `local:` version node in a version script can change the effective symbol binding
to STB_LOCAL. The linker needs to communicate the fact to enable WPD
(otherwise LTO does not know that the `!vcall_visibility` metadata has
effectively changed from VCallVisibilityPublic to VCallVisibilityLinkageUnit).
Reviewed By: tejohnson
Differential Revision: https://reviews.llvm.org/D98220
Top-level `using llvm::opt` has been present in `lld/MachO/Driver*.cpp` for some time, so remove lingering `opt::` prefixes.
Differential Revision: https://reviews.llvm.org/D98314
lld policy discourages `auto`. Replace it with a type name whenever reasonable. Retain `auto` to avoid ...
* redundancy, as for decls such as `auto *t = mumble_cast<TYPE *>` or similar that specifies the result type on the RHS
* verbosity, as for iterators
* gratuitous suffering, as for lambdas
Along the way, add `const` when appropriate.
Note: a future diff will ...
* add more `const` qualifiers
* remove `opt::` when we are already `using llvm::opt`
Differential Revision: https://reviews.llvm.org/D98313
Implement command-line options to alter a dylib's exported-symbols list:
* `-exported_symbol*` options override the default export list. The export list is compiled according to the command-line option(s) only.
* `-unexported_symbol*` options hide otherwise public symbols.
* `-*exported_symbol PATTERN` options specify a single literal or glob pattern.
* `-*exported_symbols_list FILE` options specify a file containing a series of lines containing symbol literals or glob patterns. Whitespace and `#`-prefix comments are stripped.
Note: This is a simple implementation of the primary use case. ld64 has much more complexity surrounding interactions with other options, many of which are obscure and undocumented. We will start simple and complexity as necessary.
Differential Revision: https://reviews.llvm.org/D98223
Add first bits for emitting LC_FUNCTION_STARTS.
This is a recommit of f344dfeb with the adjusted test
which should address build bots breakages.
Test plan: make check-all
Differential revision: https://reviews.llvm.org/D97260
This `R_WASM_MEMORY_ADDR_SELFREL_I32` relocation represents an offset
between its relocating address and the symbol address. It's very similar
to `R_X86_64_PC32` but restricted to be used for only data segments.
```
S + A - P
```
A: Represents the addend used to compute the value of the relocatable
field.
P: Represents the place of the storage unit being relocated.
S: Represents the value of the symbol whose index resides in the
relocation entry.
Proposal: https://github.com/WebAssembly/tool-conventions/issues/162
Differential Revision: https://reviews.llvm.org/D96659
This changes the target data layout to make stack align to 16 bytes
on Power10. Before this change, stack was being aligned to 32 bytes.
Reviewed By: #powerpc, nemanjai
Differential Revision: https://reviews.llvm.org/D96265
These are mostly things that ld64 has itself marked obsolete.
In the case of `-sectorder`, it's suggested in ld64's manpage that it
could be deprecated, so let's skip implementing it for now.
Reviewed By: #lld-macho, thakis
Differential Revision: https://reviews.llvm.org/D98066
clang appears to emit symbols in `__debug_aranges`, at least
for arm64... in the examples I've seen, it doesn't seem like those
symbols are referenced outside of `__DWARF`, so I think they're safe to
ignore. But hopefully @clayborg can confirm.
Reviewed By: clayborg
Differential Revision: https://reviews.llvm.org/D98073
We'll need to properly handle object files with multiple source inputs
eventually, but remove the assert for now so we can successfully emit binaries
for testing.
Reviewed By: #lld-macho, smeenai
Differential Revision: https://reviews.llvm.org/D98067
With this, llvm-tblgen no longer tries and fails to allocate 7953 petabyte
when it runs during the build. Instead, `check-llvm` with lld/mac as host
linker now completes without any failures on an m1 mac.
This vector op handling code matches what happens in:
- ld64's OutputFile::applyFixUps() in OutputFile.cpp for kindStoreARM64PageOff12
- lld.ld64.darwinold's offset12KindFromInstruction() in
lld/lib/ReaderWriter/MachO/ArchHandler_arm64.cpp for offset12scale16
- RuntimeDyld's decodeAddend() in
llvm/lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldMachOAArch64.h for
ARM64_RELOC_PAGEOFF12
Fixes PR49444.
Differential Revision: https://reviews.llvm.org/D98053
Previously, lld/mac only ad-hoc codesigned executables on arm64.
Matches ld64 behavior. Part of PR49443. Fixes 14 of 17 failures when running
check-llvm with lld as host linker on an M1 MBP.
Differential Revision: https://reviews.llvm.org/D97994
This patch fixes LLD to allow element sections for tables whose number
is nonzero. We also add a test for linking multiple tables, showing
that nonzero table numbers for the indirect function table,
user-declared imported tables, and local user table definitions work.
Differential Revision: https://reviews.llvm.org/D92321
Since multiple dylibs can be defined in one TBD, this is
necessary to avoid confusion.
Reviewed By: #lld-macho, oontvoo
Differential Revision: https://reviews.llvm.org/D97905
Previously, we were loading re-exports without checking whether
they were compatible with our target. Prior to {D97209}, it meant that
we were defining dylib symbols that were invalid -- usually a silent
failure unless our binary actually used them. D97209 exposed this as an
explicit error.
Along the way, I've extended our TAPI compatibility check to cover the
platform as well, instead of just checking the arch. To this end, I've
replaced MachO::Architecture with MachO::Target in our Config struct.
Reviewed By: #lld-macho, oontvoo
Differential Revision: https://reviews.llvm.org/D97867
The reexport-nested-libs test added in D97438 was a bit wonky.
First, it was linking against libReexportSystem.tbd which targets the
iOS simulator, and which in turn attempted to re-export the iOS
simulator's libSystem. However, due to the way `-syslibroot` works, it
was actually re-exporting the macOS libSystem.
As a result, the test was not actually able to resolve the symbols in
the desired libSystem. I'm guessing that @oontvoo was confused by this
and therefore included those symbols in libReexportSystem.tbd itself.
But this means that the test wasn't actually testing the resolution of
re-exported symbols (though it did at least verify that the re-exported
libraries could be located).
After some consideration, I figured that stub-link.s could be extended
to cover what reexport-nested-libs.s was attempting to do. The test
targets macOS, so we only have one `-syslibroot` and no chance of
confusion.
Reviewed By: #lld-macho, oontvoo
Differential Revision: https://reviews.llvm.org/D97866
Suppose we are linking against libFoo, which re-exports the
implicitly-bound libSystem, which in turn re-exports some
non-explicitly-bound library like `/usr/lib/system/libsystem_c.dylib`.
Then any bindings we have to a symbol in libsystem_c should use
libSystem (and not libFoo) as the umbrella library.
Reviewed By: #lld-macho, smeenai
Differential Revision: https://reviews.llvm.org/D97865
Implemented the option to omit Power10 instructions from save stubs via the
option --no-power10-stubs or --power10-stubs=no on lld. --power10-stubs= will
override the other option. --power10-stubs=auto also exists to use the default
behaviour (ie allow Power10 instructions in stubs).
Differential Revision: https://reviews.llvm.org/D94627
llvm-objdump only uses one MCInstrAnalysis object, so if ARM and Thumb
code is mixed in one object, or if an object is disassembled without
explicitly setting the triple to match the ISA used, then branch and
call targets will be printed incorrectly.
This could be fixed by creating two MCInstrAnalysis objects in
llvm-objdump, like we currently do for SubtargetInfo. However, I don't
think there's any reason we need two separate sub-classes of
MCInstrAnalysis, so instead these can be merged into one, and the ISA
determined by checking the opcode of the instruction.
Differential revision: https://reviews.llvm.org/D97766
We previously defaulted to x86_64 and an unknown platform, which was fine when
we only supported one arch and did no platform checks, but that will no longer
be true going ahead. Therefore, we should require those flags to be specified
whenever the linker is invoked.
Note that LLD-ELF and ld64 both infer the arch from their input object files,
but the usefulness of that is questionable since clang will always specify these
flags, and most of the time `lld` will be invoked via clang.
Reviewed By: #lld-macho, thakis
Differential Revision: https://reviews.llvm.org/D97799
The option-iterating loop should be reserved for options whose command-line
order is important. I think LLD-ELF follows a similar design.
Reviewed By: #lld-macho, smeenai
Differential Revision: https://reviews.llvm.org/D97797
This reverts diff D97610 (commit 0223ab035c) and adds a one-line fix to verify that a `MemoryBufferRef` has sufficient length before reading a 4-byte magic number.
Differential Revision: https://reviews.llvm.org/D97757
Currently, it was delibrately impleneted to not handle this case, but as it has turnt out, we need this feature.
The concrete use case is
`System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa` reexports
/System/Library/Frameworks/AppKit.framework/Versions/C/AppKit , which then rexports
/System/Library/PrivateFrameworks/UIFoundation.framework/Versions/A/UIFoundation
The current implemention uses a global currentTopLevelTapi, which is not reset until it finishes loading the whole tree.
This is a problem because if the top-level is set to Cocoa, then when we get to UIFoundation, it will try to find UIFoundation in the current top level, which is Cocoa and will not find it.
The right thing should be:
- When loading a library from a TBD file, re-exports need to be looked up in the auxiliary documents within the same TBD.
- When loading from an actual dylib, no additional TBD documents need to be examined.
- In no case does a re-export mentioned in one TBD file need to be looked up in a document in an auxiliary document from a different TBD file
Differential Revision: https://reviews.llvm.org/D97438
In AArch32 ARM, the PC reads two instructions ahead of the currently
executiing instruction. This evaluates to 8 in ARM state and 4 in
Thumb state. Branch instructions on AArch32 compensate for this by
subtracting the PC bias from the addend. For a branch to symbol this
will result in an addend of -8 in ARM state and -4 in Thumb state.
The existing ARM Target::inBranchRange function accounted for this
implict addend within the function meaning that if the addend were
to be taken into account by the caller then it would be double
counted. This complicates the interface for all Targets as callers
wanting to account for addends had to account for the ARM PC-bias.
In certain situations such as:
https://github.com/ClangBuiltLinux/linux/issues/1305
the PC-bias compensation code didn't match up. In particular
normalizeExistingThunk() didn't put the PC-bias back in as Arm
thunks did not store the addend.
The simplest fix for the problem is to add the PC bias in
normalizeExistingThunk when restoring the addend. However I think
it is worth refactoring the Arm inBranchRange implementation so
that fewer calls to getPCBias are needed for other Targets. I
wasn't able to remove getPCBias completely but hopefully the
Relocations.cpp code is simpler now.
In principle a test could be written to replicate the linux kernel
build failure but I wasn't able to reproduce with a small example
that I could build up from scratch.
Fixes https://github.com/ClangBuiltLinux/linux/issues/1305
Differential Revision: https://reviews.llvm.org/D97550
A couple of small changes to match the ELF linker in preparation
for adding support string mergings.
Differential Revision: https://reviews.llvm.org/D97654
This adds support for `-undefined dynamic_lookup`, and for
`-undefined warning` and `-undefined suppress` with `-flat_namespace`.
We just replace undefined symbols with a DynamicLookup when we hit them.
With this, `check-llvm` passes when using ld64.lld.darwinnew as host linker.
Differential Revision: https://reviews.llvm.org/D97642
-flat_namespace makes lld emit binaries that use name lookup that's more in
line with other POSIX systems: Instead of looking up symbols as (dylib,name)
pairs by dyld, they're instead looked up just by name.
-flat_namespace has three effects:
1. MH_TWOLEVEL and MH_NNOUNDEFS are no longer set in the Mach-O header
2. All symbols use BIND_SPECIAL_DYLIB_FLAT_LOOKUP as ordinal
3. When a dylib is added to the link, its dependent dylibs are also added,
so that lld can verify that no undefined symbols remain at the end of
a link with -flat_namespace. These transitive dylibs are added for symbol
resolution, but they are not emitted in LC_LOAD_COMMANDs.
-undefined with -flat_namespace still isn't implemented. Before this change,
it was impossible to hit that combination because -flat_namespace caused a
diagnostic. Now that it no longer does, emit a dedicated temporary diagnostic
when both flags are used.
Differential Revision: https://reviews.llvm.org/D97641
For -flat_namespace, lld needs to load dylibs in LC_LOAD_DYLIB.
The current setup meant that libSystem.dylib would cause a LC_LOAD_DYLIB
with libSystem.B.dylib, but that didn't exist in our libsysroot for
tests. So just drop the .B.
See https://reviews.llvm.org/D97641#2595237 and
https://reviews.llvm.org/D97641#2595270
Matches the ELF and COFF ports, which use ld.lld and lld-link, respectively.
While here, also move up `cleanupCallback` to match ELF / COFF.
Differential Revision: https://reviews.llvm.org/D97715
The new Darwin backend for LLD is now able to link reasonably large
real-world programs on x86_64. For instance, we have achieved
self-hosting for the X86_64 target, where all LLD tests pass when
building lld with itself on macOS. As such, we would like to make it the
default back-end.
The new port is now named `ld64.lld`, and the old port remains
accessible as `ld64.lld.darwinold`
This [annoucement email][1] has some context. (But note that, unlike
what the email says, we are no longer doing this as part of the LLVM 12
branch cut -- instead we will go into LLVM 13.)
Numerous mechanical test changes were required to make this change; in
the interest of creating something that's reviewable on Phabricator,
I've split out the boring changes into a separate diff (D95905). I plan to
merge its contents with those in this diff before landing.
(@gkm made the original draft of this diff, and he has agreed to let me
take over.)
[1]: https://lists.llvm.org/pipermail/llvm-dev/2021-January/147665.html
Reviewed By: #lld-macho, thakis
Differential Revision: https://reviews.llvm.org/D95204
There was initially some concern around the correct handling of pcrel
section relocations with r_length != 2. But it looks like there are no such
relocations in practice -- x86_64's pcrel section relocs all have r_length == 2,
and ARM64 doesn't even have pcrel section relocs. So we can replace the TODO
with an assert.
Reviewed By: #lld-macho, thakis
Differential Revision: https://reviews.llvm.org/D97576
If the reference-types feature is enabled, call_indirect will explicitly
reference its corresponding function table via TABLE_NUMBER
relocations against a table symbol.
Also, as before, address-taken functions can also cause the function
table to be created, only with reference-types they additionally cause a
symbol table entry to be emitted.
Differential Revision: https://reviews.llvm.org/D90948
Also a couple of minor cleanups in merge-string.s:
- fix inconsistent use of tabs
- use `.p2align` rather than `.align` since `.p2align` works the
same on all platforms (the meaning of align seems to differ
between platforms according to `AlignmentIsInBytes`.
I noticed these potential cleanups while porting SHF_STRINGS support to
wasm-ld.
Differential Revision: https://reviews.llvm.org/D97647
Only one of the two callers used the lastBinding parameter, so
do that work at that one call site. Extract a ordinalForDylibSymbol()
helper to make this tidy.
No behavior change.
Differential Revision: https://reviews.llvm.org/D97597
Bifurcate the `readFile()` API into ...
* `readRawFile()` which performs no checks, and
* `readLinkableFile()` which enforces minimum length of 20 bytes, same as ld64
There are no new tests because tweaks to existing tests are sufficient.
Differential Revision: https://reviews.llvm.org/D97610
On arm64, UNSIGNED relocs are the only ones that use embedded addends
instead of the ADDEND relocation.
Also ensure that the addend works when UNSIGNED is part of a SUBTRACTOR
pair.
Reviewed By: #lld-macho, alexshap
Differential Revision: https://reviews.llvm.org/D97105
Also add a few asserts to verify that we are indeed handling an
UNSIGNED relocation as the minued. I haven't made it an actual
user-facing error since I don't think llvm-mc is capable of generating
SUBTRACTOR relocations without an associated UNSIGNED.
Reviewed By: #lld-macho, smeenai
Differential Revision: https://reviews.llvm.org/D97103
`llvm-mc` doesn't generate any relocations for subtractions
between local symbols -- they must be global -- so the previous test
wasn't actually testing any relocation logic. I've fixed that and
extended the test to cover r_length=3 relocations as well as both x86_64
and arm64.
Reviewed By: #lld-macho, smeenai
Differential Revision: https://reviews.llvm.org/D97057
Specifically:
- InputChunk::outputOffset -> outSecOffset
- Symbol::get/setVirtualAddress -> get/setVA
- add InputChunk::getOffset helper that takes an offset
These are mostly in preparation for adding support for
SHF_MERGE/SHF_STRINGS but its also good to align with ELF where
possible.
Differential Revision: https://reviews.llvm.org/D97595
Dynamic lookup symbols are symbols that work like dynamic symbols
in ELF: They're not bound to a dylib like normal Mach-O twolevel lookup
symbols, but they live in a global pool and dyld resolves them against
exported symbols from all loaded dylibs.
This adds support for dynamical lookup symbols to lld/mac. They are
represented as DylibSymbols with file set to nullptr.
This also uses this support to implement the -U flag, which makes
a specific symbol that's undefined at the end of the link a
dynamic lookup symbol.
For -U, it'd be sufficient to just to a pass over remaining undefined symbols
at the end of the link and to replace them with dynamic lookup symbols then.
But I'd like to use this code to implement flat_namespace too, and that will
require real support for resolving dynamic lookup symbols in SymbolTable. So
this patch adds this now already.
While writing tests for this, I noticed that we didn't set N_WEAK_DEF in the
symbol table for DylibSymbols, so this fixes that too.
Differential Revision: https://reviews.llvm.org/D97521
For one metadata section usage, each text section references a metadata section.
The metadata sections have a C identifier name to allow the runtime to collect them via `__start_/__stop_` symbols.
Since `__start_`/`__stop_` references are always present from live sections, the
C identifier name sections appear like GC roots, which means they cannot be
discarded by `ld --gc-sections`.
To make such sections GCable, either SHF_LINK_ORDER or a section group is needed.
SHF_LINK_ORDER is not suitable for the references can be inlined into other functions
(See D97430:
Function A (in the section .text.A) references its `__sancov_guard` section.
Function B inlines A (so now .text.B references `__sancov_guard` - this is invalid with the semantics of SHF_LINK_ORDER).
In the linking stage,
if `.text.A` gets discarded, and `__sancov_guard` is retained via the reference from `.text.B`,
the output will be invalid because `__sancov_guard` references the discarded `.text.A`.
LLD errors "sh_link points to discarded section".
)
A section group have size overhead, and is cumbersome when there is just one metadata section.
Add `-z start-stop-gc` to drop the "__start_/__stop_ references retain
non-SHF_LINK_ORDER non-SHF_GROUP C identifier name sections" rule.
We reserve the rights to switch the default in the future.
Reviewed By: phosek, jrtc27
Differential Revision: https://reviews.llvm.org/D96914
When parsing bitcode, convert LTO Symbols to LLD Symbols in order to perform
resolution. The "winning" symbol will then be marked as Prevailing at LTO
compilation time. This is similar to what the other LLD ports do.
This change allows us to handle `linkonce` symbols correctly, and to deal with
duplicate bitcode symbols gracefully. Previously, both scenarios would result in
an assertion failure inside the LTO code, complaining that multiple Prevailing
definitions are not allowed.
While at it, I also added basic logic around visibility. We don't do anything
useful with it yet, but we do check that its value is valid. LLD-ELF appears to
use it only to set FinalDefinitionInLinkageUnit for LTO, which I think is just a
performance optimization.
From my local experimentation, the linker itself doesn't seem to do anything
differently when encountering linkonce / linkonce_odr / weak / weak_odr. So I've
only written a test for one of them. LLD-ELF has more, but they seem to mostly
be testing the intermediate bitcode output of their LTO backend...? I'm far from
an expert here though, so I might very well be missing things.
Reviewed By: #lld-macho, MaskRay, smeenai
Differential Revision: https://reviews.llvm.org/D94342
Remove a stray -lib argument in guardcf-lto.ll; llvm-lib doesn't
support generating import libs from a def file unlike lib.exe.
Previously this worked because the -lib argument was ignored
(printing only a warning).
Differential Revision: https://reviews.llvm.org/D96699
{D95809} introduced a mechanism for synthetic symbol creation of personality
pointers. When multiple section relocations referred to the same personality
pointer, it would deduplicate them. However, it neglected to consider that we
could have symbol relocations that also refer to the same personality pointer.
This diff fixes it.
In practice, this mix of relocations arises when there is a statically-linked
personality routine that is referenced from multiple object files. Within the
same object file, it will be referred to via section relocations, but
(obviously) other object files will refer to it via symbol relocations. Failing
to deduplicate these references resulted in us going over the
3-personality-pointer limit when linking some larger applications.
Fixes llvm.org/PR48389.
Reviewed By: #lld-macho, thakis, alexshap
Differential Revision: https://reviews.llvm.org/D97245
The silent failures had confused me a few times.
I haven't added a similar check for platform yet as we don't yet have logic to
infer the platform automatically, and so adding that check would require
updating dozens of test files.
Reviewed By: #lld-macho, thakis, alexshap
Differential Revision: https://reviews.llvm.org/D97209
I've adjusted the RelocAttrBits to better fit the semantics of
the relocations. In particular:
1. *_UNSIGNED relocations are no longer marked with the `TLV` bit, even
though they can occur within TLV sections. Instead the `TLV` bit is
reserved for relocations that can reference thread-local symbols, and
*_UNSIGNED relocations have their own `UNSIGNED` bit. The previous
implementation caused TLV and regular UNSIGNED semantics to be
conflated, resulting in rebase opcodes being incorrectly emitted for TLV
relocations.
2. I've added a new `POINTER` bit to denote non-relaxable GOT
relocations. This distinction isn't important on x86 -- the GOT
relocations there are either relaxable or non-relaxable loads -- but
arm64 has `GOT_LOAD_PAGE21` which loads the page that the referent
symbol is in (regardless of whether the symbol ends up in the GOT). This
relocation must reference a GOT symbol (so must have the `GOT` bit set)
but isn't itself relaxable (so must not have the `LOAD` bit). The
`POINTER` bit is used for relocations that *must* reference a GOT
slot.
3. A similar situation occurs for TLV relocations.
4. ld64 supports both a pcrel and an absolute version of
ARM64_RELOC_POINTER_TO_GOT. But the semantics of the absolute version
are pretty weird -- it results in the value of the GOT slot being
written, rather than the address. (That means a reference to a
dynamically-bound slot will result in zeroes being written.) The
programs I've tried linking don't use this form of the relocation, so
I've dropped our partial support for it by removing the relevant
RelocAttrBits.
Reviewed By: alexshap
Differential Revision: https://reviews.llvm.org/D97031
/reproduce: now works correctly with:
- /call-graph-ordering-file:
- /def:
- /natvis:
- /order:
- /pdbstream:
I went through all instances of MemoryBuffer::getFile() and made sure
everything that didn't already do so called takeBuffer().
For natvis, that wasn't possible since DebugInfo/PDB wants to take
owernship of the natvis buffer. For that case, I'm manually adding the
tar file entry.
/natvis: and /pdbstream: is slightly awkward, since createResponseFile()
always adds these flags to the response file but createPDB() (which
ultimately adds the files referenced by the flags) is only called if
/debug is also passed. So when using /natvis: without /debug with
/reproduce:, lld won't warn, but when linking using the response
file from the archive, it won't find the natvis file since it's not
in the tar. This isn't a new issue though, and after this patch things
at least work with using /natvis: _with_ debug with /reproduce:.
(Same for /pdbstream:)
Differential Revison: https://reviews.llvm.org/D97212
If the reference-types feature is enabled, call_indirect will explicitly
reference its corresponding function table via `TABLE_NUMBER`
relocations against a table symbol.
Also, as before, address-taken functions can also cause the function
table to be created, only with reference-types they additionally cause a
symbol table entry to be emitted.
We abuse the used-in-reloc flag on symbols to indicate which tables
should end up in the symbol table. We do this because unfortunately
older wasm-ld will carp if it see a table symbol.
Differential Revision: https://reviews.llvm.org/D90948
The special root semantics for identifier-named sections is meant
specifically for the metadata sections. In the context of group
semantics, where group members are always retained or discarded as a
unit, it's natural not to have this semantics apply to a section in a
group, otherwise we would never discard the group defeating the purpose
of using the group in the first place.
This change modifies the GC behavior so that __start_/__stop_ references
don't retain C identifier named sections in section groups which allows
for these groups to be collected. This matches the behavior of BFD ld.
The only kind of existing case that might break is interdependent
metadata sections that are all in a group together, but that group
doesn't contain any other sections referenced by anything except
implicit inclusion in a `__start_` and/or `__stop_`-referenced
identifier-named section, but such cases should be unlikely.
Differential Revision: https://reviews.llvm.org/D96753
See discussion on https://reviews.llvm.org/D93263
-flat_namespace isn't implemented yet, and neither is -undefined dynamic,
so this makes -undefined pretty pointless in lld/MachO for now. But once
we implement -flat_namespace (which we need to do anyways to get check-llvm
to pass with lld as host linker), the code's already there.
Follow-up to https://reviews.llvm.org/D93263#2491865
Differential Revision: https://reviews.llvm.org/D96963
Differential Revision: https://reviews.llvm.org/D95913
Usage: -bundle_loader <executable>
This option specifies the executable that will load the build output file being linked.
When building a bundle, users can use the --bundle_loader to specify an executable
that contains symbols referenced, but not implemented in the bundle.
For relocatable output that needs the indirect function table, identify
the well-known function table. This allows us to properly fix the
limits on the imported table, and in a followup will allow the element
section to reference the indirect function table even if it's not
assigned to table number 0. Adapt tests for import reordering.
Differential Revision: https://reviews.llvm.org/D96770
Before, --importTable forced the creation of an indirect function table,
whether it was needed or not. Now it only imports a table if needed.
Differential Revision: https://reviews.llvm.org/D96872
ST_Data is used to model BFD `BFD_OBJECT`.
A STT_TLS symbol does not have the `BFD_OBJECT` flag in BFD.
This makes sense because a STT_TLS symbol is like in a different address space,
normal data/object properties do not apply on them.
With this change, a STT_TLS symbol will not be displayed as 'O'.
This new behavior matches objdump.
Differential Revision: https://reviews.llvm.org/D96735
Adds a lld test for a case that the handling added for dynamically
exported symbols in 1487747e99 already
fixes. Because isExportDynamic returns true when the symbol is
SharedKind with default visibility, it will treat as dynamically
exported and block devirtualization when the definition of a vtable
comes from a shared library. This is desireable as it is dangerous to
devirtualize in that case, since there could be hidden overrides in the
shared library. Typically that happens when the shared library header
contains available externally definitions, which applications can
override. An example is std::error_category, which is overridden in LLVM
and causing failures after a self build with WPD enabled, because
libstdc++ contains hidden overrides of the virtual base class methods.
The regular LTO case in the new test already worked, but there are
2 fixes in this patch needed for the index-only case and the hybrid
LTO case. For the index-only case, WPD should not simply ignore
available externally vtables. A follow on fix will be made to clang to
emit type metadata for those vtables, which the new test is modeling.
For the hybrid case, we need to ensure when the module is split that any
llvm.*used globals are cloned to the regular LTO split module so
available externally vtable definitions are not prematurely deleted.
Another follow on fix will add the equivalent gold test, which requires
a small fix to the plugin to treat symbols in dynamic libraries the same
way lld already is.
Differential Revision: https://reviews.llvm.org/D96721
This change introduces support for zero flag ELF section groups to lld.
lld already supports COMDAT sections, which in ELF are a special type of
ELF section groups. These are generally useful to enable linker GC where
you want a group of sections to always travel together, that is to be
either retained or discarded as a whole, but without the COMDAT
semantics. Other ELF linkers already support zero flag ELF section
groups and this change helps us reach feature parity.
Differential Revision: https://reviews.llvm.org/D96636
As we don't sort local symbols, don't sort non-local symbols. This makes
non-local symbols appear in their register order, which matches GNU as. The
register order is nice in that you can write tests with interleaved CHECK
prefixes, e.g.
```
// CHECK: something about foo
.globl foo
foo:
// CHECK: something about bar
.globl bar
bar:
```
With the lexicographical order, the user needs to place lexicographical smallest
symbol first or keep CHECK prefixes in one place.
MVP object files may import at most one table, and if they do, it must
be assigned table number zero in the output, as the references to that
table are not relocatable. Ensure that this is the case, even if some
inputs define other tables.
Differential Revision: https://reviews.llvm.org/D96001
This reverts commit ac2be2b6a3.
This causes a whole much of emscripten tests to fail due to newly
undefined symbols appearing. Will investigate and look into re-landing
later.
This fixes two somewhat related issues. Firstly we were never
generating imports for weak functions (even with the `import-functions`
policy for undefined symbols). Adding a direct call to foo in the
`weak-undefined-pic.s` exposed a crash in the linker which this
change fixes.
Secondly we were failing to call `handleWeakUndefines` for the `-pie`
case which is PIC but doesn't set the undefined symbol policy to
`import-functions`. With this change `-pie` binaries will by default
call `handleWeakUndefines` which generates the undefined stub handlers
for any weakly undefined symbols.
Fixes: https://github.com/emscripten-core/emscripten/issues/13337
Differential Revision: https://reviews.llvm.org/D95914
With dynamic linking we have the current limitation that there can be
only a single active data segment (since we use __memory_base as the
load address and we can't do arithmetic in constant expresions).
This change delays the merging of active segments until a little later
in the linking process which means that the grouping of data by section,
and the magic __start/__end symbols work as expected under dynamic
linking.
Differential Revision: https://reviews.llvm.org/D96453
When parsing an object file, LLD interleaves undefined symbol resolution (which
may recursively fetch other lazy objects) with defined symbol resolution.
This may lead to surprising results, e.g. if an object file defines currently
undefined symbols and references another lazy symbol, we may interleave defined
symbols with the lazy fetch, potentially leading to the defined symbols
resolving to different files.
As an example, if both `a.a(a.o)` and `a.a(b.o)` define `foo` (not in COMDAT
group, or in different COMDAT groups) and `__profd_foo` (in COMDAT group
`__profd_foo`). LLD may resolve `foo` to `a.a(a.o)` and `__profd_foo` to
`b.a(b.o)`, i.e. different files.
```
parse ArchiveFile a.a
entry fetches a.a(a.o)
parse ObjectFile a.o
define entry
define foo
reference b
b fetches a.a(b.o)
parse ObjectFile b.o
define prevailing __profd_foo
define (ignored) non-prevailing __profd_foo
```
Assuming a set of interconnected symbols are defined all or none in several lazy
objects. Arguably making them resolve to the same file is preferable than making
them resolve to different files (some are lazy objects).
The main argument favoring the new behavior is the stability. The relative order
between a defined symbol and an undefined symbol does not change the symbol
resolution behavior. Only the relative order between two undefined symbols can
affect fetching behaviors.
---
The real world case is reduced from a Fuchsia PGO usage: `a.a(a.o)` has a
constructor within COMDAT group C5 while `a.a(b.o)` has a constructor within
COMDAT group C2. Because they use different group signatures, they are not
de-duplicated. It is not entirely whether Clang behavior is entirely conforming.
LLD selects the PGO counter section (`__profd_*`) from `a.a(b.o)` and the
constructor section from `a.a(a.o)`. The `__profd_*` is a SHF_LINK_ORDER section
linking to its own non-prevailing constructor section, so LLD errors
`sh_link points to discarded section`. This patch fixes the error.
Differential Revision: https://reviews.llvm.org/D95985
Multi-configuration generators (such as Visual Studio and Xcode) allow the specification of a build flavor at build time instead of config time, so the lit configuration files need to support that - and they do for the most part. There are several places that had one of two issues (or both!):
1) Paths had %(build_mode)s set up, but then not configured, resulting in values that would not work correctly e.g. D:/llvm-build/%(build_mode)s/bin/dsymutil.exe
2) Paths did not have %(build_mode)s set up, but instead contained $(Configuration) (which is the value for Visual Studio at configuration time, for Xcode they would have had the equivalent) e.g. "D:/llvm-build/$(Configuration)/lib".
This seems to indicate that we still have a lot of fragility in the configurations, but also that a number of these paths are never used (at least on Windows) since the errors appear to have been there a while.
This patch fixes the configurations and it has been tested with Ninja and Visual Studio to generate the correct paths. We should consider removing some of these settings altogether.
Reviewed By: JDevlieghere, mehdi_amini
Differential Revision: https://reviews.llvm.org/D96427
This commit regroups commonalities among InputGlobal, InputEvent, and
InputTable into the new InputElement. The subclasses are defined
inline in the new InputElement.h. NFC.
Reviewed By: sbc100
Differential Revision: https://reviews.llvm.org/D94677
The code previously assumed that `getChunk` would return a non-null pointer for
every symbol, but in fact it only returns non-null pointers for DefinedFunction
and DefinedData symbols. This patch fixes the segfault by checking whether
`getChunk` returns a null for each symbol and skipping the mapping output for
any symbols for which it does.
Differential Revision: https://reviews.llvm.org/D88369
This moves the error checking until after all optional
symbols (including the section start/end symbols) have
been created.
Differential Revision: https://reviews.llvm.org/D96318
Since we emit diagnostics for undefineds in Writer::scanRelocations()
and symbols referenced by -u flags aren't referenced by any relocations,
this needs some manual code (similar to the entry point).
Differential Revision: https://reviews.llvm.org/D94371
`extern const bfd_target aarch64_elf64_le_vec;` is a variable in BFD.
It was somehow misused as an emulation by Android.
```
% aarch64-linux-gnu-ld -m aarch64_elf64_le_vec a.o
aarch64-linux-gnu-ld: unrecognised emulation mode: aarch64_elf64_le_vec
Supported emulations: aarch64linux aarch64elf aarch64elf32 aarch64elf32b aarch64elfb armelf armelfb aarch64linuxb aarch64linux32 aarch64linux32b armelfb_linux_eabi armelf_linux_eabi
```
Acked by Stephen Hines, who removed the flag from Android a while back.
Rewritting the path of the sample profile file in response.txt to be relative to the repro tar.
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D96193
addOptionalGlobalSymbols should be addOptionalGlobalSymbol.
Also, remove unnecessary additional argument to make the signature match
the sibling function: addOptionalDataSymbol.
Differential Revision: https://reviews.llvm.org/D96305
This is an initial base commit for ARM64 target arch support. I don't represent that it complete or bug-free, but wish to put it out for review now that some basic things like branch target & load/store address relocs are working.
I can add more tests to this base commit, or add them in follow-up commits.
It is not entirely clear whether I use the "ARM64" (Apple) or "AArch64" (non-Apple) naming convention. Guidance is appreciated.
Differential Revision: https://reviews.llvm.org/D88629
The LSDA pointers are encoded as offsets from the image base,
and arranged in one big contiguous array. Each second-level page records
the offset within that LSDA array which corresponds to the LSDA for its
first CU entry.
Reviewed By: clayborg
Differential Revision: https://reviews.llvm.org/D95810
Note that there is a triple indirection involved with
personalities and compact unwind:
1. Two bits of each CU encoding are used as an offset into the
personality array.
2. Each entry of the personality array is an offset from the image base.
The resulting address (after adding the image base) should point within the
GOT.
3. The corresponding GOT entry contains the actual pointer to the
personality function.
To further complicate things, when the personality function is in the
object file (as opposed to a dylib), its references in
`__compact_unwind` may refer to it via a section + offset relocation
instead of a symbol relocation. Since our GOT implementation can only
create entries for symbols, we have to create a synthetic symbol at the
given section offset.
Reviewed By: clayborg
Differential Revision: https://reviews.llvm.org/D95809
GNU as does not sort local symbols. This has several advantages:
* The .symtab order is roughly the symbol occurrence order.
* The closest preceding STT_SECTION symbol is the definition of a local symbol.
* The closest preceding STT_FILE symbol is the defining file of a local symbol, if there are multiple default-version .file directives. (Not implemented in MC.)
A SHF_LINK_ORDER .gcc_except_table is similar to a .gcc_except_table in
a section group. The associated text section is responsible for retaining it.
LLD still does not support GC of non-group non-SHF_LINK_ORDER .gcc_except_table -
but that is not necessary because we can teach the compiler to set SHF_LINK_ORDER.
The Mach kernel & codesign on arm64 macOS has strict requirements for alignment and sequence of segments and sections. Dyld probably is just as picky, though kernel & codesign reject malformed Mach-O files before dyld ever has a chance.
I developed this diff by incrementally changing alignments & sequences to match the output of ld64. I stopped when my hello-world test program started working: `codesign --verify` succeded, and `execve(2)` didn't immediately fail with `errno == EBADMACHO` = `"Malformed Mach-O file"`.
Differential Revision: https://reviews.llvm.org/D94935
The current diagnostic has confused users. The new wording is adapted from one suggested by Ian Lance Taylor.
Differential Revision: https://reviews.llvm.org/D95917
binutils 2.36 introduced the new section flag SHF_GNU_RETAIN (for ELFOSABI_GNU &
ELFOSABI_FREEBSD) to mark a sections as a GC root. Several LLVM side toolchain
folks (including me) were involved in the design process of SHF_GNU_RETAIN and
were happy with this proposal.
Currently GNU ld only respects SHF_GNU_RETAIN semantics for ELFOSABI_GNU &
ELFOSABI_FREEBSD object files
(https://sourceware.org/bugzilla/show_bug.cgi?id=27282). GNU ld sets EI_OSABI
to ELFOSABI_GNU for relocatable output
(https://sourceware.org/bugzilla/show_bug.cgi?id=27091). In practice the single
value EI_OSABI is neither a good indicator for object file compatibility, nor a
useful mechanism marking used ELF extensions.
For input, we respect SHF_GNU_RETAIN semantics even for ELFOSABI_NONE object
files. This is compatible with how LLD and GNU ld handle (mildly useful) STT_GNU_IFUNC
/ (emitted by GCC, considered misfeature by some folks) STB_GNU_UNIQUE input.
(As of LLVM 12.0.0, the integrated assembler does not set ELFOSABI_GNU for
STT_GNU_IFUNC/STB_GNU_UNIQUE).
Arguably STT_GNU_IFUNC/STB_GNU_UNIQUE probably need indicators in object files
but SHF_GNU_RETAIN is more likely accepted by more OSABI platforms.
For output, we take a step further than GNU ld: we don't promote ELFOSABI_NONE
to ELFOSABI_GNU for all output.
Differential Revision: https://reviews.llvm.org/D95749
In GCC emitted .debug_info sections, R_386_GOTOFF may be used to
relocate DW_AT_GNU_call_site_value values
(https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98946).
R_386_GOTOFF (`S + A - GOT`) is one of the `isStaticLinkTimeConstant` relocation
type which is not PC-relative, so it can be used from non-SHF_ALLOC sections. We
current allow new relocation types as needs come. The diagnostic has caught some
bugs in the past.
Differential Revision: https://reviews.llvm.org/D95994
This extends {D92539} to work even when we are loading archive
members via `-force_load`. I uncovered this issue while trying to
force-load archives containing bitcode -- we were segfaulting.
In addition to fixing the `-force_load` case, this diff also addresses
the behavior of `-ObjC` when LTO bitcode is involved -- we need to
force-load those archive members if they contain ObjC categories.
Reviewed By: #lld-macho, smeenai
Differential Revision: https://reviews.llvm.org/D95265
This makes our error messages more informative. But the bigger motivation is for
LTO symbol resolution, which will be in an upcoming diff. The changes in this
one are largely mechanical.
Reviewed By: #lld-macho, smeenai
Differential Revision: https://reviews.llvm.org/D94316
On z/OS, other error messages are not matched correctly in lit tests.
```
EDC5121I Invalid argument.
EDC5111I Permission denied.
```
This patch adds a lit substitution to fix it.
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D95808
Add per-reloc-type attribute bits and migrate code from per-target file into target independent code, driven by reloc attributes.
Many cleanups
Differential Revision: https://reviews.llvm.org/D95121
The option catches incompatibility between `R_*_IRELATIVE` and DT_TEXTREL/DF_TEXTREL
before glibc 2.29. Newer glibc versions are more common nowadays and I don't
think this option has ever been used. Diagnosing this problem is also
straightforward by reading the stack trace.
Previously, CMake would find any version of Python3. However, the project
claims to require 3.6 or greater, and 3.6 features are being used.
Reviewed By: yln
Differential Revision: https://reviews.llvm.org/D95635
On z/OS, the following error message is not matched correctly in lit tests.
```
EDC5129I No such file or directory.
```
This patch uses a lit config substitution to check for platform specific error messages.
Reviewed By: muiez, jhenderson
Differential Revision: https://reviews.llvm.org/D95246
Identify dynamically exported symbols (--export-dynamic[-symbol=],
--dynamic-list=, or definitions needed to preempt shared objects) and
prevent their LTO visibility from being upgraded.
This helps avoid use of whole program devirtualization when there may
be overrides in dynamic libraries.
Differential Revision: https://reviews.llvm.org/D91583
A default version (@@) is only available for defined symbols.
Currently we use "@@" for undefined symbols too.
This patch fixes the issue and improves our test case.
Differential revision: https://reviews.llvm.org/D95219
Not sure what the difference is, but using the latter appears to cause
issues in standalone builds. See llvm.org/PR48853.
Reviewed By: #lld-macho, compnerd
Differential Revision: https://reviews.llvm.org/D95359
I noticed that this option was not appearing at all in the `--help`
messages for `wasm-ld` or `ld.lld`.
Add help text and make it consistent across all ports.
Differential Revision: https://reviews.llvm.org/D94925
Whilst migrating/retiring some downstream testing, I came across a test
for weak undef IE and LD TLS references, but was unable to find any
equivalent in LLD's upstream testing. There does seem to be some slight
subtle differences that could be worth testing compared to LE TLS
references, in particular that IE can be relaxed to LE in this case,
hence this change.
Differential Revision: https://reviews.llvm.org/D95124
Reviewed by: grimar, MaskRay
Just getting rid of some logspew as I test LLD under existing build
systems.
Reviewed By: #lld-macho, smeenai
Differential Revision: https://reviews.llvm.org/D95213
If foo is referenced in any object file, bitcode file or shared object,
`__wrap_foo` should be retained as the redirection target of sym
(f96ff3c0f8).
If the object file defining foo has foo references, we cannot easily distinguish
the case from cases where foo is not referenced (we haven't scanned
relocations). Retain `__wrap_foo` because we choose to wrap sym references
regardless of whether sym is defined to keep non-LTO/LTO/relocatable links' behaviors similar
https://sourceware.org/bugzilla/show_bug.cgi?id=26358 .
If foo is defined in a shared object, `__wrap_foo` can still be omitted
(`wrap-dynamic-undef.s`).
Reviewed By: andrewng
Differential Revision: https://reviews.llvm.org/D95152
Fixes PR48523. When the linker errors with "output file too large",
one question that comes to mind is how the section sizes differ from
what they were previously. Unfortunately, this information is lost
when the linker exits without writing the output file. This change
makes it so that the error message includes the sizes of the largest
sections.
Reviewed By: MaskRay, grimar, jhenderson
Differential Revision: https://reviews.llvm.org/D94560
This makes the following improvements.
For `SHT_GNU_versym`:
* yaml2obj: set `sh_link` to index of `.dynsym` section automatically.
For `SHT_GNU_verdef`:
* yaml2obj: set `sh_link` to index of `.dynstr` section automatically.
* yaml2obj: set `sh_info` field automatically.
* obj2yaml: don't dump the `Info` field when its value matches the number of version definitions.
For `SHT_GNU_verneed`:
* yaml2obj: set `sh_link` to index of `.dynstr` section automatically.
* yaml2obj: set `sh_info` field automatically.
* obj2yaml: don't dump the `Info` field when its value matches the number of version dependencies.
Also, simplifies few test cases.
Differential revision: https://reviews.llvm.org/D94956
This reverts commit 5b7aef6eb4 and relands
6529d7c5a4.
The ASan error was debugged and determined to be the fault of an invalid
object file input in our test suite, which was fixed by my last change.
LLD's project policy is that it assumes input objects are valid, so I
have added a comment about this assumption to the relocation bounds
check.
The relocation offsets were incorrect. I fixed them with llvm-readobj
-codeview -codeview-subsection-bytes, which has a helpful printout of
the relocations that apply to a given symbol record with their offsets.
With this, I was able to update the relocation offsets in the yaml to
fix the line table and the S_DEFRANGE_REGISTER records.
There is still some remaining inconsistency in yaml2obj and obj2yaml
when round tripping MSVC objects, but that isn't a blocker for relanding
D94267.
Run the ObjCARCContractPass during LTO. The legacy LTO backend (under
LTO/ThinLTOCodeGenerator.cpp) already does this; this diff just adds that
behavior to the new LTO backend. Without that pass, the objc.clang.arc.use
intrinsic will get passed to the instruction selector, which doesn't know how to
handle it.
In order to test both the new and old pass managers, I've also added support for
the `--[no-]lto-legacy-pass-manager` flags.
P.S. Not sure if the ordering of the pass within the pipeline matters...
Reviewed By: fhahn
Differential Revision: https://reviews.llvm.org/D94547
In 8031785f4a the temporary object being built was moved to %t/main.o,
but not all run lines were updated to reflect this. Observe the failure
on this buildbot:
http://lab.llvm.org:8011/#/builders/5/builds/3646/steps/9/logs/stdio
It might pass locally for some people due to a stale %t.o hanging around
the build directory.
```
// a.s
jmp fcntl
// b.s
.globl fcntl
fcntl:
ret
```
`ld.lld -shared --wrap=fcntl a.o b.o` has an `R_X86_64_JUMP_SLOT` referencing
the index 0 undefined symbol, which will cause a glibc `symbol lookup error` at
runtime. This is because `__wrap_fcntl` is not in .dynsym
We use an approximation `!wrap->isUndefined()`, which doesn't set
`isUsedInRegularObj` of `__wrap_fcntl` when `fcntl` is referenced and
`__wrap_fcntl` is undefined.
Fix this by using `sym->referenced`.
This reverts commit 418df4a6ab.
This change broke emscripten tests, I believe because it started
generating 5-byte a wide table index in the call_indirect instruction.
Neither v8 nor wabt seem to be able to handle that. The spec
currently says that this is single 0x0 byte and:
"In future versions of WebAssembly, the zero byte occurring in the
encoding of the call_indirectcall_indirect instruction may be used to
index additional tables."
So we need to revisit this change. For backwards compat I guess
we need to guarantee that __indirect_function_table is always at
address zero. We could also consider making this a single-byte
relocation with and assert if have more than 127 tables (for now).
Differential Revision: https://reviews.llvm.org/D95005
Object files (and the output --relocatable) should never define
__indirect_function_table. It should always be linker synthesized
with the final output executable.
Differential Revision: https://reviews.llvm.org/D94993
R_PPC64_ADDR16_HI represents bits 16-31 of a 32-bit value
R_PPC64_ADDR16_HIGH represents bits 16-31 of a 64-bit value.
In the Linux kernel, `LOAD_REG_IMMEDIATE_SYM` defined in `arch/powerpc/include/asm/ppc_asm.h`
uses @l, @high, @higher, @highest to load the 64-bit value of a symbol.
Fixes https://github.com/ClangBuiltLinux/linux/issues/1260
Such files (Thin-%%%%%%.tmp.o) are supposed to be deleted immediately
after they're used (either by renaming or deletion). However, we've seen
instances on Windows where this doesn't happen, probably due to the
filesystem being flaky. This is effectively a resource leak which has
prevented us from using the ThinLTO cache on Windows.
Since those temporary files are in the thinlto cache directory which we
prune periodically anyway, allowing them to be pruned too seems like a
tidy way to solve the problem.
Differential revision: https://reviews.llvm.org/D94962
Element sections will also need flags, so we shouldn't squat the
WASM_SEGMENT namespace.
Depends on D90948.
Differential Revision: https://reviews.llvm.org/D92315
This patch changes to make call_indirect explicitly refer to the
corresponding function table, residualizing TABLE_NUMBER relocs against
it.
With this change, wasm-ld now sees all references to tables, and can
link multiple tables.
Differential Revision: https://reviews.llvm.org/D90948
The commit 18aa0be36e changed the default GotBaseSymInGotPlt to true
for AArch64. This is different than binutils, where
_GLOBAL_OFFSET_TABLE_ points at the start or .got.
It seems to not intefere with current relocations used by LLVM. However
as indicated by PR#40357 [1] gcc generates R_AARCH64_LD64_GOTPAGE_LO15
for -pie (in fact it also generated the relocation for -fpic).
This change is requires to correctly handle R_AARCH64_LD64_GOTPAGE_LO15
by lld from objects generated by gcc.
[1] https://bugs.llvm.org/show_bug.cgi?id=40357
This patch adds support to wasm-ld for linking multiple table references
together, in a manner similar to wasm globals. The indirect function
table is synthesized as needed.
To manage the transitional period in which the compiler doesn't yet
produce TABLE_NUMBER relocations and doesn't residualize table symbols,
the linker will detect object files which have table imports or
definitions, but no table symbols. In that case it will synthesize
symbols for the defined and imported tables.
As a change, relocatable objects are now written with table symbols,
which can cause symbol renumbering in some of the tests. If no object
file requires an indirect function table, none will be written to the
file. Note that for legacy ObjFile inputs, this test is conservative: as
we don't have relocs for each use of the indirecy function table, we
just assume that any incoming indirect function table should be
propagated to the output.
Differential Revision: https://reviews.llvm.org/D91870
On z/OS, the following error message is not matched correctly in lit tests. This patch updates the CHECK expression to match successfully.
```
EDC5129I No such file or directory.
```
Reviewed By: muiez
Differential Revision: https://reviews.llvm.org/D94239
This patch adds support to wasm-ld for linking multiple table references
together, in a manner similar to wasm globals. The indirect function
table is synthesized as needed.
To manage the transitional period in which the compiler doesn't yet
produce TABLE_NUMBER relocations and doesn't residualize table symbols,
the linker will detect object files which have table imports or
definitions, but no table symbols. In that case it will synthesize
symbols for the defined and imported tables.
As a change, relocatable objects are now written with table symbols,
which can cause symbol renumbering in some of the tests. If no object
file requires an indirect function table, none will be written to the
file. Note that for legacy ObjFile inputs, this test is conservative: as
we don't have relocs for each use of the indirecy function table, we
just assume that any incoming indirect function table should be
propagated to the output.
Differential Revision: https://reviews.llvm.org/D91870
This commit adds table symbol support in a partial way, while still
including some special cases for the __indirect_function_table symbol.
No change in tests.
Differential Revision: https://reviews.llvm.org/D94075
When running in `-r/--relocatable` we output relocations but the
new TLS relocations type was missing from `ObjFile::calcNewAddend`
causing this combination of inputs/flags to crash the linker.
Also avoid creating tls variables in relocatable mode. These variables
are only needed when linking final executables.
Fixes: https://github.com/emscripten-core/emscripten/issues/12934
Fixes: PR48506
Differential Revision: https://reviews.llvm.org/D93554
This is a pretty classic optimization. Instead of processing symbol
records and copying them to temporary storage, do a first pass to
measure how large the module symbol stream will be, and then copy the
data into place in the PDB file. This requires defering relocation until
much later, which accounts for most of the complexity in this patch.
This patch avoids copying the contents of all live .debug$S sections
into heap memory, which is worth about 20% of private memory usage when
making PDBs. However, this is not an unmitigated performance win,
because it can be faster to read dense, temporary, heap data than it is
to iterate symbol records in object file backed memory a second time.
Results on release chrome.dll:
peak mem: 5164.89MB -> 4072.19MB (-1,092.7MB, -21.2%)
wall-j1: 0m30.844s -> 0m32.094s (slightly slower)
wall-j3: 0m20.968s -> 0m20.312s (slightly faster)
wall-j8: 0m19.062s -> 0m17.672s (meaningfully faster)
I gathered similar numbers for a debug, component build of content.dll
in Chrome, and the performance impact of this change was in the noise.
The memory usage reduction was visible and similar.
Because of the new parallelism in the PDB commit phase, more cores makes
the new approach faster. I'm assuming that most C++ developer machines
these days are at least quad core, so I think this is a win.
Differential Revision: https://reviews.llvm.org/D94267
Jez Ng
