Commit Graph

76 Commits

Author SHA1 Message Date
Jez Ng ccbacdded4 [lld-macho] Weak locals should be relaxed too
Reviewed By: #lld-macho, smeenai

Differential Revision: https://reviews.llvm.org/D86746
2020-08-27 17:44:17 -07:00
Jez Ng 2a38dba7dd [lld-macho] Emit binding opcodes for defined symbols that override weak dysyms
These opcodes tell dyld to coalesce the overridden weak dysyms to this
particular symbol definition.

Reviewed By: #lld-macho, smeenai

Differential Revision: https://reviews.llvm.org/D86575
2020-08-27 17:44:16 -07:00
Jez Ng 3da2130e45 [lld-macho] Emit the right header flags for weak bindings/symbols
Reviewed By: #lld-macho, smeenai

Differential Revision: https://reviews.llvm.org/D86574
2020-08-27 17:44:16 -07:00
Jez Ng e263287c79 [lld-macho] Implement weak binding for branch relocations
Since there is no "weak lazy" lookup, function calls to weak symbols are
always non-lazily bound. We emit both regular non-lazy bindings as well
as weak bindings, in order that the weak bindings may overwrite the
non-lazy bindings if an appropriate symbol is found at runtime. However,
the bound addresses will still be written (non-lazily) into the
LazyPointerSection.

Reviewed By: #lld-macho, smeenai

Differential Revision: https://reviews.llvm.org/D86573
2020-08-27 17:44:15 -07:00
Jez Ng cbe27316ef [lld-macho] Implement weak bindings for GOT/TLV
Previously, we were only emitting regular bindings to weak
dynamic symbols; this diff adds support for the weak bindings too, which
can overwrite the regular bindings at runtime. We also treat weak
defined global symbols similarly -- since they can also be interposed at
runtime, they need to be treated as potentially dynamic symbols.

Note that weak bindings differ from regular bindings in that they do not
specify the dylib to do the lookup in (i.e. weak symbol lookup happens
in a flat namespace.)

Differential Revision: https://reviews.llvm.org/D86572
2020-08-26 19:21:09 -07:00
Jez Ng b84d72d893 [lld-macho][NFC] Handle GOT bindings and regular bindings more uniformly
Previously, the BindingEntry struct could only store bindings to offsets
within InputSections. Since the GOTSection and TLVPointerSections are
OutputSections, I handled those in a separate code path. However, this
makes it awkward to support weak bindings properly without code
duplication. This diff allows BindingEntries to point directly to
OutputSections, simplifying the upcoming weak binding implementation.

Along the way, I also converted a bunch of functions taking references
to symbols to take pointers instead. Given how much casting we do for
Symbol (especially in the upcoming weak binding diffs), it's cleaner
this way.

Differential Revision: https://reviews.llvm.org/D86571
2020-08-26 19:21:04 -07:00
Jez Ng 3c9100fb78 [lld-macho] Support dynamic linking of thread-locals
References to symbols in dylibs work very similarly regardless of
whether the symbol is a TLV. The main difference is that we have a
separate `__thread_ptrs` section that acts as the GOT for these
thread-locals.

We can identify thread-locals in dylibs by a flag in their export trie
entries, and we cross-check it with the relocations that refer to them
to ensure that we are not using a GOT relocation to reference a
thread-local (or vice versa).

Reviewed By: #lld-macho, smeenai

Differential Revision: https://reviews.llvm.org/D85081
2020-08-12 19:50:09 -07:00
Jez Ng 98210796e1 [lld-macho] Make __LINKEDIT sections contiguous
codesign (or more specifically libstuff) checks that each section in
__LINKEDIT ends where the next one starts -- no gaps are permitted. This
diff achieves it by aligning every section's start and end points to
WordSize.

Remarks: ld64 appears to satisfy the constraint by adding padding bytes
when generating the __LINKEDIT data, e.g. by emitting BIND_OPCODE_DONE
(which is a 0x0 byte) repeatedly. I think the approach this diff takes
is a bit more elegant, but I'm not sure if it's too restrictive. In
particular, it assumes padding always uses the zero byte. But we can
revisit this later.

Reviewed By: #lld-macho, compnerd

Differential Revision: https://reviews.llvm.org/D84718
2020-07-30 14:30:07 -07:00
Jez Ng 3587de2281 [lld-macho] Support __dso_handle for C++
The C++ ABI requires dylibs to pass a pointer to __cxa_atexit which does
e.g. cleanup of static global variables. The C++ spec says that the pointer
can point to any address in one of the dylib's segments, but in practice
ld64 seems to set it to point to the header, so that's what's implemented
here.

Reviewed By: #lld-macho, smeenai

Differential Revision: https://reviews.llvm.org/D83603
2020-07-30 14:28:41 -07:00
Jez Ng 53eb7fda51 [lld-macho] Support binding dysyms to any section
Previously, we only supported binding dysyms to the GOT. This
diff adds support for binding them to any arbitrary section. C++
programs appear to use this, I believe for vtables and type_info.

This diff also makes our bind opcode encoding a bit smarter -- we now
encode just the differences between bindings, which will make things
more compact.

I was initially concerned about the performance overhead of iterating
over these relocations, but it turns out that the number of such
relocations is small. A quick analysis of my llvm-project build
directory showed that < 1.3% out of ~7M relocations are RELOC_UNSIGNED
bindings to symbols (including both dynamic and static symbols).

Reviewed By: #lld-macho, smeenai

Differential Revision: https://reviews.llvm.org/D83103
2020-07-02 21:21:01 -07:00
Jez Ng 3646ee503d [lld-macho] Refactor segment/section creation, sorting, and merging
Summary:
There were a few issues with the previous setup:

1. The section sorting comparator used a declarative map of section names to
  determine the correct order, but it turns out we need to match on more than
  just names -- in particular, an upcoming diff will sort based on whether the
  S_ZERO_FILL flag is set. This diff changes the sorter to a more imperative but
  flexible form.

2. We were sorting OutputSections stored in a MapVector, which left the
  MapVector in an inconsistent state -- the wrong keys map to the wrong values!
  In practice, we weren't doing key lookups (only container iteration) after the
  sort, so this was fine, but it was still a dubious state of affairs. This diff
  copies the OutputSections to a vector before sorting them.

3. We were adding unneeded OutputSections to OutputSegments and then filtering
  them out later, which meant that we had to remember whether an OutputSegment
  was in a pre- or post-filtered state. This diff only adds the sections to the
  segments if they are needed.

In addition to those major changes, two minor ones worth noting:

1. I renamed all OutputSection variable names to `osec`, to parallel `isec`.
  Previously we were using some inconsistent combination of `osec`, `os`, and
  `section`.

2. I added a check (and a test) for InputSections with names that clashed with
  those of our synthetic OutputSections.

Reviewers: #lld-macho

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D81887
2020-06-21 17:13:59 -07:00
Jez Ng a12e7d406d [lld-macho] Handle GOT relocations of non-dylib symbols
Summary:
Turns out this case is actually really common -- it happens whenever there's
a reference to an `extern` variable that ends up statically linked.

Depends on D80856.

Reviewers: ruiu, pcc, MaskRay, smeenai, alexshap, gkm, Ktwu, christylee

Reviewed By: smeenai

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80857
2020-06-17 20:41:28 -07:00
Jez Ng a2d096df26 [lld-macho] Use uint64_t for getSize() instead of size_t
Summary:
So things work on 32-bit machines. (@vzakhari reported the
breakage starting from D80177).

Reviewers: #lld-macho, vzakhari

Subscribers: llvm-commits, vzakhari

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D81982
2020-06-16 18:42:45 -07:00
Jez Ng a04c133564 [lld-macho] Set __PAGEZERO size to 4GB
That's what ld64 uses for 64-bit targets. I figured it's best to make
this change sooner rather than later since a bunch of our tests are
relying on hardcoded addresses that depend on this value.

Reviewed By: smeenai

Differential Revision: https://reviews.llvm.org/D80177
2020-06-02 13:19:38 -07:00
Jez Ng 55e9eb416e [lld-macho] Support -order_file
The order file indicates how input sections should be sorted within each
output section, based on the symbols contained within those sections.

This diff sets the stage for implementing and testing
`.subsections_via_symbols`, where we will break up InputSections by each
symbol and sort them more granularly.

Reviewed By: smeenai

Differential Revision: https://reviews.llvm.org/D79668
2020-05-19 07:46:57 -07:00
Jez Ng b3e2fc931d [lld-macho] Support calls to functions in dylibs
Summary:
This diff implements lazy symbol binding -- very similar to the PLT
mechanism in ELF.

ELF's .plt section is broken up into two sections in Mach-O:
StubsSection and StubHelperSection. Calls to functions in dylibs will
end up calling into StubsSection, which contains indirect jumps to
addresses stored in the LazyPointerSection (the counterpart to ELF's
.plt.got).

Initially, the LazyPointerSection contains addresses that point into one
of the entry points in the middle of the StubHelperSection. The code in
StubHelperSection will push on the stack an offset into the
LazyBindingSection. The push is followed by a jump to the beginning of
the StubHelperSection (similar to PLT0), which then calls into
dyld_stub_binder. dyld_stub_binder is a non-lazily bound symbol, so this
call looks it up in the GOT.

The stub binder will look up the bind opcodes in the LazyBindingSection
at the given offset. The bind opcodes will tell the binder to update the
address in the LazyPointerSection to point to the symbol, so that
subsequent calls don't have to redo the symbol resolution. The binder
will then jump to the resolved symbol.

Depends on D78269.

Reviewers: ruiu, pcc, MaskRay, smeenai, alexshap, gkm, Ktwu, christylee

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D78270
2020-05-09 20:56:22 -07:00
Kellie Medlin 6cb073133c [lld] Merge Mach-O input sections
Summary: Similar to other formats, input sections in the MachO
implementation are now grouped under output sections. This is primarily
a refactor, although there's some new logic (like resolving the output
section's flags based on its inputs).

Differential Revision: https://reviews.llvm.org/D77893
2020-05-01 16:57:18 -07:00
Jez Ng 9854edd817 [lld-macho] Implement basic export trie
Build the trie by performing a three-way radix quicksort: We start by
sorting the strings by their first characters, then sort the strings
with the same first characters by their second characters, and so on
recursively. Each time the prefixes diverge, we add a node to the trie.
Thanks to @ruiu for the idea.

I used llvm-mc's radix quicksort implementation as a starting point. The
trie offset fixpoint code was taken from
MachONormalizedFileBinaryWriter.cpp.

Differential Revision: https://reviews.llvm.org/D76977
2020-04-29 15:44:27 -07:00
Jez Ng 62b8f32f76 [lld-macho][reland] Add support for emitting dylibs with a single symbol
This got reverted due to UBSAN errors in a diff lower in the stack,
which is being fixed in https://reviews.llvm.org/D79050. This diff is
otherwise identical to the original https://reviews.llvm.org/D76908
(which was committed in 9598778bd1 and reverted in b52bc2653b).

Differential Revision: https://reviews.llvm.org/D79051
2020-04-28 17:08:32 -07:00
Jez Ng 4f0cccdd7a [lld-macho][reland] Add basic symbol table output
This diff implements basic support for writing a symbol table.

Attributes are loosely supported for extern symbols and not at all for
other types.

Initial version by Kellie Medlin <kelliem@fb.com>

Originally committed in a3d95a50ee and reverted in fbae153ca5 due to
UBSAN erroring over unaligned writes. That has been fixed in the
current diff with the following changes:

```
diff --git a/lld/MachO/SyntheticSections.cpp b/lld/MachO/SyntheticSections.cpp
--- a/lld/MachO/SyntheticSections.cpp
+++ b/lld/MachO/SyntheticSections.cpp
@@ -133,6 +133,9 @@ SymtabSection::SymtabSection(StringTableSection &stringTableSection)
     : stringTableSection(stringTableSection) {
   segname = segment_names::linkEdit;
   name = section_names::symbolTable;
+  // TODO: When we introduce the SyntheticSections superclass, we should make
+  // all synthetic sections aligned to WordSize by default.
+  align = WordSize;
 }

 size_t SymtabSection::getSize() const {
diff --git a/lld/MachO/Writer.cpp b/lld/MachO/Writer.cpp
--- a/lld/MachO/Writer.cpp
+++ b/lld/MachO/Writer.cpp
@@ -371,6 +371,7 @@ void Writer::assignAddresses(OutputSegment *seg) {
     ArrayRef<InputSection *> sections = p.second;
     for (InputSection *isec : sections) {
       addr = alignTo(addr, isec->align);
+      // We must align the file offsets too to avoid misaligned writes of
+      // structs.
+      fileOff = alignTo(fileOff, isec->align);
       isec->addr = addr;
       addr += isec->getSize();
       fileOff += isec->getFileSize();
@@ -396,6 +397,7 @@ void Writer::writeSections() {
     uint64_t fileOff = seg->fileOff;
     for (auto &sect : seg->getSections()) {
       for (InputSection *isec : sect.second) {
+        fileOff = alignTo(fileOff, isec->align);
         isec->writeTo(buf + fileOff);
         fileOff += isec->getFileSize();
       }
```

I don't think it's easy to write a test for alignment (that doesn't
involve brittly hard-coding file offsets), so there isn't one... but
UBSAN builds pass now.

Differential Revision: https://reviews.llvm.org/D79050
2020-04-28 17:07:06 -07:00
Shoaib Meenai fbae153ca5 Revert "[lld-macho] Add basic symbol table output"
This reverts commit a3d95a50ee.

Reverting due to UBSan failures:
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/40817/steps/check-lld%20ubsan/logs/stdio
2020-04-28 11:34:03 -07:00
Shoaib Meenai b52bc2653b Revert "[lld-macho] Add support for emitting dylibs with a single symbol"
This reverts commit 9598778bd1.

Reverting due to UBSan failures:
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/40817/steps/check-lld%20ubsan/logs/stdio
2020-04-28 11:34:03 -07:00
Jez Ng 9598778bd1 [lld-macho] Add support for emitting dylibs with a single symbol
Summary:
Add logic for emitting the correct set of load commands and segments
when `-dylib` is passed.

I haven't gotten to implementing a real export trie yet, so we can only
emit a single symbol, but it's enough to replace the YAML test files
introduced in D76252.

Differential Revision: https://reviews.llvm.org/D76908
2020-04-27 13:33:46 -07:00
Jez Ng a3d95a50ee [lld-macho] Add basic symbol table output
This diff implements basic support for writing a symbol table.

- Attributes are loosely supported for extern symbols and not at all for
  other types

Immediate future work will involve implementing section merging.

Initial version by Kellie Medlin <kelliem@fb.com>

Differential Revision: https://reviews.llvm.org/D76742
2020-04-27 13:33:15 -07:00
Jez Ng 6f63216c3d [lld-macho] Extend SyntheticSections to cover all segment load commands
Previously, the special segments `__PAGEZERO` and `__LINKEDIT` were
implemented as special LoadCommands. This diff implements them using
special sections instead which have an `isHidden()` attribute. We do not
emit section headers for hidden sections, but we use their addresses and
file offsets to determine that of their containing segments. In addition
to allowing us to share more segment-related code, this refactor is also
important for the next step of emitting dylibs:

1) dylibs don't have segments like __PAGEZERO, so we need an easy way of
   omitting them w/o messing up segment indices
2) Unlike the kernel, which is happy to run an executable with
   out-of-order segments, dyld requires dylibs to have their segment
   load commands arranged in increasing address order. The refactor
   makes it easier to implement sorting of sections and segments.

Differential Revision: https://reviews.llvm.org/D76839
2020-04-27 12:58:12 -07:00
Jez Ng 060efd24c7 [lld-macho] Add basic support for linking against dylibs
This diff implements:

* dylib loading (much of which is being restored from @pcc and @ruiu's
  original work)
* The GOT_LOAD relocation, which allows us to load non-lazy dylib
  symbols
* Basic bind opcode emission, which tells `dyld` how to populate the GOT

Differential Revision: https://reviews.llvm.org/D76252
2020-04-21 13:43:19 -07:00