Commit Graph

551 Commits

Author SHA1 Message Date
Thomas Lively 15a5e86fb3 [lld][WebAssembly] Allow `atomics` feature with unshared memory
https://github.com/WebAssembly/threads/issues/144 updated the
WebAssembly threads proposal to make atomic operations on unshared memories
valid. This change updates the feature checking in the linker accordingly.
Production WebAssembly engines have recently been updated to allow this
behvaior, but after this change users who accidentally use atomics with unshared
memories on older versions of the engines will get validation errors at runtime
rather than link errors.

Differential Revision: https://reviews.llvm.org/D79530
2020-09-24 20:35:29 -07:00
Alexandre Ganea f2efb5742c [LLD][COFF] Cover usage of LLD-as-a-library in tests
In lit tests, we run each LLD invocation twice (LLD_IN_TEST=2), without shutting down the process in-between. This ensures a full cleanup is properly done between runs.
Only active for the COFF driver for now. Other drivers still use LLD_IN_TEST=1 which executes just one iteration with full cleanup, like before.
When the environment variable LLD_IN_TEST is unset, a shortcut is taken, only one iteration is executed, no cleanup for faster exit, like before.
A public API, lld::safeLldMain(), is also available when using LLD as a library.

Differential Revision: https://reviews.llvm.org/D70378
2020-09-24 15:07:50 -04:00
Fangrui Song 51b75b87db [lld][WebAssembly] Fix -Wunused-variable after D87663 2020-09-18 16:10:39 -07:00
Sam Clegg 3f411e9773 [lld][WebAssembly] Fix --export-all when __stack_pointer is present
With https://reviews.llvm.org/D87537 we made it an error
to import or export a mutable global with the +mutable-globals
feature present.  However the scan was of the entire symbol
table rather than just the imports or exports and the filter
didn't match exaclyt meaning the `__stack_pointer` (a mutable
global) was always triggering with error when the `--export-all`
flag was used.

This also revealed that we didn't have any test coverage for
the `--export-all` flag.

This change fixes the current breakage on the emscripten-releases
roller.

Differential Revision: https://reviews.llvm.org/D87663
2020-09-15 06:17:01 -07:00
Sam Clegg 2c12b056be [lld][WebAssembly] Allow globals imports via import_name/import_module
This feature already exists but was limited to function
symbols.

Differential Revision: https://reviews.llvm.org/D87666
2020-09-14 20:35:03 -07:00
Sam Clegg cc2da5554b [lld][WebAssembly] Add initial support for -Map/--print-map
Differential Revision: https://reviews.llvm.org/D77187
2020-09-12 16:10:51 -07:00
Sam Clegg 04febd30a8 [lld][WebAssembly] Error on import/export of mutable global without `mutable-globals` feature
Also add the +mutable-globals features in clang when
building with `-fPIC` since the linker will generate mutable
globals imports and exports in that case.

Differential Revision: https://reviews.llvm.org/D87537
2020-09-12 14:28:14 -07:00
Samuel Kostial 304264e73d [lld][WebAssembly] Emit all return types of multivalue functions
We previously were incorrectly emitting only the first result type.

Differential Revision: https://reviews.llvm.org/D85783
2020-08-12 13:14:15 -07:00
Wouter van Oortmerssen b52fc59e17 [WebAssembly] Fixed memory.init always using 64-bit ptr
(because the is64 flag was tested incorrectly in LLD).
2020-08-12 12:23:36 -07:00
Wouter van Oortmerssen 582fd474dd [WebAssembly] wasm64: fix memory.init operand types
I had assumed they would all become in i64, but this is not necessary as long as data segments stay 32-bit, see:
https://github.com/WebAssembly/memory64/blob/master/proposals/memory64/Overview.md

Differential Revision: https://reviews.llvm.org/D85552
2020-08-10 10:15:20 -07:00
Sam Clegg b34ec5969f [lld][WebAssembly] Handle weakly referenced symbols when lazy (archive) version is see first
When a weak reference of a lazy symbol occurs we were not correctly
updating the lazy symbol.  We need to tag the existing lazy symbol
as weak and, in the case of a function symbol, give it a signature.

Without the signature we can't then create the dummy function which
is needed when an weakly undefined function is called.

We had tests for weakly referenced lazy symbols but we were only
tests in the case where the reference was seen before the lazy
symbol.

See: https://github.com/WebAssembly/wasi-libc/pull/214

Differential Revision: https://reviews.llvm.org/D85567
2020-08-10 08:16:55 -07:00
Michele Scandale 53880b8cb9 [CMake] Make `intrinsics_gen` dependency unconditional.
The `intrinsics_gen` target exists in the CMake exports since r309389
(see LLVMConfig.cmake.in), hence projects can depend on `intrinsics_gen`
even it they are built separately from LLVM.

Reviewed By: MaskRay, JDevlieghere

Differential Revision: https://reviews.llvm.org/D83454
2020-07-17 16:43:17 -07:00
Wouter van Oortmerssen cc1b9b680f [WebAssembly] 64-bit (function) pointer fixes.
Accounting for the fact that Wasm function indices are 32-bit, but in wasm64 we want uniform 64-bit pointers.
Includes reloc types for 64-bit table indices.

Differential Revision: https://reviews.llvm.org/D83729
2020-07-16 14:10:22 -07:00
Wouter van Oortmerssen 29f8c9f6c2 [WebAssembly] Triple::wasm64 related cleanup
Differential Revision: https://reviews.llvm.org/D83713
2020-07-16 12:01:10 -07:00
Wouter van Oortmerssen 4d135b0446 [WebAssembly] 64-bit memory limits 2020-07-06 12:40:45 -07:00
Sam Clegg 73e575a88e [lld][WebAssembly] Give better error message on bad archive member
Include the archive name as well as the member name when an error
is encountered parsing bitcode archives.

Differential Revision: https://reviews.llvm.org/D82884
2020-06-30 21:36:36 -07:00
Dan Gohman 46a3268312 [WebAssembly] Add warnings for -shared and -pie
The meaning of -shared and -pie are expected to be changed in the
future when Module Linking-style libraries are implemented. Begin
issuing warnings to give people a heads-up that they will be changing.

For compatibility with Emscripten, add a --experimental-pic flag which
disables these warnings.

Differential Revision: https://reviews.llvm.org/D81760
2020-06-25 15:55:46 -07:00
Wouter van Oortmerssen b9a539c010 [WebAssembly] Adding 64-bit versions of __stack_pointer and other globals
We have 6 globals, all of which except for __table_base are 64-bit under wasm64.

Differential Revision: https://reviews.llvm.org/D82130
2020-06-25 15:52:44 -07:00
Sam Clegg 79aad89d8d [WebAssembly] Add support for externalref to MC and wasm-ld
This allows code for handling externref values to be processed by the
assembler and linker.

Differential Revision: https://reviews.llvm.org/D81977
2020-06-22 15:57:24 -07:00
Sam Clegg 21e14cce23 [lld][WebAssembly] Allow ctors functions that return values
Some projects use the constructor attribute on functions that also
return values.  In this case we just ignore them.

The error was reported in the libgpg-error project that marks
gpg_err_init with the `__constructor__` attribute.

Differential Revision: https://reviews.llvm.org/D81962
2020-06-18 13:11:40 -07:00
Wouter van Oortmerssen 3b29376e3f [WebAssembly] Adding 64-bit version of R_WASM_MEMORY_ADDR_* relocs
This adds 4 new reloc types.

A lot of code that previously assumed any memory or offset values could be contained in a uint32_t (and often truncated results from functions returning 64-bit values) have been upgraded to uint64_t. This is not comprehensive: it is only the values that come in contact with the new relocation values and their dependents.

A new tablegen mapping was added to automatically upgrade loads/stores in the assembler, which otherwise has no way to select for these instructions (since they are indentical other than for the offset immediate). It follows a similar technique to https://reviews.llvm.org/D53307

Differential Revision: https://reviews.llvm.org/D81704
2020-06-15 10:07:42 -07:00
Sam Clegg 81443ac1bc [WebAssembly] Add placeholders for R_WASM_TABLE_INDEX_REL_SLEB relocations
Previously in the object format we punted on this and simply wrote
zeros (and didn't include the function in the elem segment).  With
this change we write a meaningful value which is the segment
relative table index of the associated function.

This matches the that wasm-ld produces in `-r` mode.  This inconsistency
between the output the MC object writer and the wasm-ld object
writer could cause warnings to be emitted when reading back in the
output of `wasm-ld -r`.  See:
https://github.com/emscripten-core/emscripten/issues/11217

This only applies to this one relocation type which is only generated
when compiling in PIC mode.

Differential Revision: https://reviews.llvm.org/D80774
2020-05-29 10:57:26 -07:00
Thomas Lively d851fce4cb [lld][WebAssembly] Do not emit initialization for .bss segments
Summary:
This patch fixes a bug where initialization code for .bss segments was
emitted in the memory initialization function even though the .bss
segments were discounted in the datacount section and omitted in the
data section. This was producing invalid binaries due to out-of-bounds
segment indices on the memory.init and data.drop instructions that
were trying to operate on the nonexistent .bss segments.

Reviewers: sbc100

Subscribers: dschuff, jgravelle-google, aheejin, sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80354
2020-05-21 11:33:25 -07:00
Sam Clegg 064e9907ba [lld][WebAssembly] Fix for --relocatable and signature mismatches
This is a followup to https://reviews.llvm.org/D78779.

When signatures mismatch we create set of variant symbols.  Some of
the fields in these symbols were not be initialized correct.
Specifically we were seeing isUsedInRegularObj not being set correctly,
leading to the symbol not getting included in the symbol table
and a crash writing relections in --reloctable mode.

There is larger refactor due here, but this is a minimal change the
fixes the bug at hand.

Differential Revision: https://reviews.llvm.org/D79756
2020-05-13 10:27:09 -07:00
Thomas Lively a1ae9566ea [WebAssembly] Disallow 'shared-mem' rather than 'atomics'
Summary:
The WebAssembly backend automatically lowers atomic operations and TLS
to nonatomic operations and non-TLS data when either are present and
the atomics or bulk-memory features are not present, respectively. The
resulting object is no longer thread-safe, so the linker has to be
told not to allow it to be linked into a module with shared
memory. This was previously done by disallowing the 'atomics' feature,
which prevented any objct with its atomic operations or TLS removed
from being linked with any object containing atomics or TLS, and
therefore preventing it from being linked into a module with shared
memory since shared memory requires atomics.

However, as of https://github.com/WebAssembly/threads/issues/144, the
validation rules are relaxed to allow atomic operations to validate
with unshared memories, which makes it perfectly safe to link an
object with stripped atomics and TLS with another object that still
contains TLS and atomics as long as the resulting module has an
unshared memory. To allow this kind of link, this patch disallows a
pseudo-feature 'shared-mem' rather than 'atomics' to communicate to
the linker that the object is not thread-safe. This means that the
'atomics' feature is available to accurately reflect whether or not an
object has atomics enabled.

As a drive-by tweak, this change also requires that bulk-memory be
enabled in addition to atomics in order to use shared memory. This is
because initializing shared memories requires bulk-memory operations.

Reviewers: aheejin, sbc100

Subscribers: dschuff, jgravelle-google, hiraditya, sunfish, jfb, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D79542
2020-05-08 13:52:39 -07:00
Sam Clegg f03b6e785b [lld][WebAssembly] Honor --allow-undefined for data symbols too
This was originally the way this worked before before
https://reviews.llvm.org/D60882.

In retrospect it seems inconsistent that `--allow-undefined` doesn't
work for all symbols.  See:
https://groups.google.com/g/emscripten-discuss/c/HSRgQiIq1gI/m/Kt9oFWHiAwAJ

I'm also planning a followup change which implement the full
`--unresolved-symbols=..` flags supported by ELF linkers (both ld and
ld.lld) since it seems more standard.

Differential Revision: https://reviews.llvm.org/D79247
2020-05-06 12:39:29 -07:00
Reid Kleckner 932f0276ea [Support] Move LLD's parallel algorithm wrappers to support
Essentially takes the lld/Common/Threads.h wrappers and moves them to
the llvm/Support/Paralle.h algorithm header.

The changes are:
- Remove policy parameter, since all clients use `par`.
- Rename the methods to `parallelSort` etc to match LLVM style, since
  they are no longer C++17 pstl compatible.
- Move algorithms from llvm::parallel:: to llvm::, since they have
  "parallel" in the name and are no longer overloads of the regular
  algorithms.
- Add range overloads
- Use the sequential algorithm directly when 1 thread is requested
  (skips task grouping)
- Fix the index type of parallelForEachN to size_t. Nobody in LLVM was
  using any other parameter, and it made overload resolution hard for
  for_each_n(par, 0, foo.size(), ...) because 0 is int, not size_t.

Remove Threads.h and update LLD for that.

This is a prerequisite for parallel public symbol processing in the PDB
library, which is in LLVM.

Reviewed By: MaskRay, aganea

Differential Revision: https://reviews.llvm.org/D79390
2020-05-05 15:21:05 -07:00
Sam Clegg 0a6c4d8d2e [WebAssmebly] Add support for defined wasm globals in MC and lld
This change add support for defined wasm globals in the .s format,
the MC layer, and wasm-ld

Currently there is no support custom initialization and all wasm
globals are initialized to zero.

Fixes: PR45742

Differential Revision: https://reviews.llvm.org/D79137
2020-04-30 12:43:15 -07:00
Sam Clegg 4b8e2d8e81 [lld][WebAssembly] Fix crash on function signature mismatch with --relocatable
These stub new function were not being added to the symbol table
which in turn meant that we were crashing when trying to output
relocations against them.

Differential Revision: https://reviews.llvm.org/D78779
2020-04-25 10:26:11 -07:00
Wouter van Oortmerssen cbf99e0fba [WebAssembly] Fix faulty logic in verifyRelocTargets 2020-04-16 18:19:56 -07:00
Wouter van Oortmerssen 48139ebc3a [WebAssembly] Add int32 DW_OP_WASM_location variant
This to allow us to add reloctable global indices as a symbol.
Also adds R_WASM_GLOBAL_INDEX_I32 relocation type to support it.

See discussion in https://github.com/WebAssembly/debugging/issues/12
2020-04-16 16:32:17 -07:00
Thomas Lively 6474d1b20e [lld][WebAssembly] Do not require --shared-memory with --relocatable
Summary:
wasm-ld requires --shared-memory to be passed when the atomics feature
is enabled because historically atomic operations were only valid with
shared memories. This change relaxes that requirement for when
building relocatable objects because their memories are not
meaningful. This technically maintains the validity of object files
because the threads spec now allows atomic operations with unshared
memories, although we don't support that elsewhere in the tools yet.

This fixes and Emscripten build issue reported at
https://bugs.chromium.org/p/webp/issues/detail?id=463.

Reviewers: sbc100

Subscribers: dschuff, jgravelle-google, aheejin, sunfish, jfb, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D78072
2020-04-14 13:49:28 -07:00
Sam Clegg 3ea1c62cba [WebAssembly] Emit .llvmcmd and .llvmbc as custom sections
Fixes: https://bugs.llvm.org/show_bug.cgi?id=45362

Differential Revision: https://reviews.llvm.org/D77115
2020-04-14 13:24:18 -07:00
Georgii Rymar 1647ff6e27 [ADT/STLExtras.h] - Add llvm::is_sorted wrapper and update callers.
It can be used to avoid passing the begin and end of a range.
This makes the code shorter and it is consistent with another
wrappers we already have.

Differential revision: https://reviews.llvm.org/D78016
2020-04-14 14:11:02 +03:00
Thomas Lively c496d84b4f [lld][WebAssembly] Handle 4gb max memories
Summary:
A previous change (53211a) had updated the argument parsing to handle
large max memories, but 4294967296 would still wrap to zero after the
options were parsed. This change updates the configuration to use a
64-bit integer to store the max memory to avoid that overflow.

Reviewers: sbc100

Subscribers: dschuff, jgravelle-google, aheejin, sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D77437
2020-04-09 13:06:41 -07:00
Heejin Ahn c09acd5dd1 [WebAssembly] Handle event exports
Summary: This handles exports of events, which was missing.

Reviewers: sbc100

Subscribers: dschuff, jgravelle-google, sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D77566
2020-04-06 11:28:38 -07:00
Kazuaki Ishizaki 7c5fcb3591 [lld] NFC: fix trivial typos in comments
Differential Revision: https://reviews.llvm.org/D72339
2020-04-02 01:21:36 +09:00
Sam Clegg b5767010a8 [lld][WebAssembly] Early error if output file cannot be created.
This matches the behaviour of the ELF driver.

Also move the `createFiles` to be `checkConfig` and report `no input
files` there.   Again this is mostly to match the structure of the ELF
linker better.

Differential Revision: https://reviews.llvm.org/D76960
2020-03-31 21:42:38 -07:00
Fangrui Song eb4663d8c6 [lld][COFF][ELF][WebAssembly] Replace --[no-]threads /threads[:no] with --threads={1,2,...} /threads:{1,2,...}
--no-threads is a name copied from gold.
gold has --no-thread, --thread-count and several other --thread-count-*.

There are needs to customize the number of threads (running several lld
processes concurrently or customizing the number of LTO threads).
Having a single --threads=N is a straightforward replacement of gold's
--no-threads + --thread-count.

--no-threads is used rarely. So just delete --no-threads instead of
keeping it for compatibility for a while.

If --threads= is specified (ELF,wasm; COFF /threads: is similar),
--thinlto-jobs= defaults to --threads=,
otherwise all available hardware threads are used.

There is currently no way to override a --threads={1,2,...}. It is still
a debate whether we should use --threads=all.

Reviewed By: rnk, aganea

Differential Revision: https://reviews.llvm.org/D76885
2020-03-31 08:46:12 -07:00
Alexandre Ganea 09158252f7 [ThinLTO] Allow usage of all hardware threads in the system
Before this patch, it wasn't possible to extend the ThinLTO threads to all SMT/CMT threads in the system. Only one thread per core was allowed, instructed by usage of llvm::heavyweight_hardware_concurrency() in the ThinLTO code. Any number passed to the LLD flag /opt:lldltojobs=..., or any other ThinLTO-specific flag, was previously interpreted in the context of llvm::heavyweight_hardware_concurrency(), which means SMT disabled.

One can now say in LLD:
/opt:lldltojobs=0 -- Use one std::thread / hardware core in the system (no SMT). Default value if flag not specified.
/opt:lldltojobs=N -- Limit usage to N threads, regardless of usage of heavyweight_hardware_concurrency().
/opt:lldltojobs=all -- Use all hardware threads in the system. Equivalent to /opt:lldltojobs=$(nproc) on Linux and /opt:lldltojobs=%NUMBER_OF_PROCESSORS% on Windows. When an affinity mask is set for the process, threads will be created only for the cores selected by the mask.

When N > number-of-hardware-threads-in-the-system, the threads in the thread pool will be dispatched equally on all CPU sockets (tested only on Windows).
When N <= number-of-hardware-threads-on-a-CPU-socket, the threads will remain on the CPU socket where the process started (only on Windows).

Differential Revision: https://reviews.llvm.org/D75153
2020-03-27 10:20:58 -04:00
Dan Gohman 66bfbedbdf [WebAssembly] Support wasm exports with zero-length names.
Zero-length strings are valid export names in WebAssembly, so allow
users to specify them.

Differential Revision: https://reviews.llvm.org/D71793
2020-03-26 16:20:43 -07:00
Paolo Severini aff75e1a1f [lld][Wasm] Wasm-ld emits invalid .debug_ranges entries for non-live symbols
When the debug info contains a relocation against a dead symbol, wasm-ld
may emit spurious range-list terminator entries (entries with Start==0
and End==0). This change fixes this by emitting the WasmRelocation
Addend as End value for a non-live symbol.

Reviewed by: sbc100, dblaikie

Differential Revision: https://reviews.llvm.org/D74781
2020-03-26 14:26:31 -07:00
Heejin Ahn f93426c5b9 [WebAssembly] Move event section before global section
Summary:
https://github.com/WebAssembly/exception-handling/issues/98

Also this moves many parts of code to make code align with the section
order, even if they don't affect the output.

Reviewers: tlively

Subscribers: dschuff, sbc100, hiraditya, sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D76752
2020-03-25 11:49:03 -07:00
Reid Kleckner 213aea4c58 Remove unused Endian.h includes, NFC
Mainly avoids including Host.h everywhere:

$ diff -u <(sort thedeps-before.txt) <(sort thedeps-after.txt) \
    | grep '^[-+] ' | sort | uniq -c | sort -nr
   3141 - /usr/local/google/home/rnk/llvm-project/llvm/include/llvm/Support/Host.h
2020-03-11 15:45:34 -07:00
Sam Clegg 928e9e1723 [lld][WebAssembly] Add support for --rsp-quoting
This also changes to default style to match the host.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D75577
2020-03-04 11:41:33 -08:00
Sam Clegg a57f1a5435 [lld][WebAssembly] Handle mixed strong and weak undefined symbols
When there are both strong and weak references to an undefined
symbol ensure that the strong reference prevails in the output symbol
generating the correct error.

Test case copied from lld/test/ELF/weak-and-strong-undef.s

Differential Revision: https://reviews.llvm.org/D75322
2020-02-28 10:16:10 -08:00
Dan Gohman 197bda587b [WebAssembly] Teach lld how to demangle "__main_argc_argv".
WebAssembly requires that caller and callee signatures match, so it
can't do the usual trick of passing more arguments to main than it
expects. Instead WebAssembly will mangle "main" with argc/argv
parameters as "__main_argc_argv". This patch teaches lld how to
demangle it.

This patch is part of https://reviews.llvm.org/D70700.
2020-02-27 07:55:01 -08:00
Sam Clegg 06f1a5c9c2 [lld][WebAssembly] Allow symbols with explict import names to be undefined at link time.
Differential Revision: https://reviews.llvm.org/D74110
2020-02-19 18:02:49 -08:00
Sam Clegg bd4812776b [WebAssembly] Use llvm::Optional to store optional symbol attributes. NFC.
The changes the in-memory representation of wasm symbols such that their
optional ImportName and ImportModule use llvm::Optional.

ImportName is set whenever WASM_SYMBOL_EXPLICIT_NAME flag is set.
ImportModule (for imports) is currently always set since it defaults to
"env".

In the future we can possibly extent to binary format distingish
import which have explit module names.

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D74109
2020-02-19 17:25:33 -08:00
Sam Clegg b062fe1816 [lld][WebAssembly] Fail if bitcode objects are pulled in after LTO
This can happen if lto::LTO::getRuntimeLibcallSymbols doesn't return
an complete/accurate list of libcalls.  In this case new bitcode
object can be linked in after LTO.

For example the WebAssembly backend currently calls:
  setLibcallName(RTLIB::FPROUND_F32_F16, "__truncsfhf2");

But `__truncsfhf2` is not part of `getRuntimeLibcallSymbols` so if
this symbol is generated during LTO the link will currently fail.

Without this change the linker crashes because the bitcode symbol
makes it all the way to the output phase.

See: https://bugs.llvm.org/show_bug.cgi?id=44353

Differential Revision: https://reviews.llvm.org/D71632
2020-02-11 17:36:15 -08:00
Jonas Devlieghere 3e24242a7d [lld] Replace SmallStr.str().str() with std::string conversion operator.
Use the std::string conversion operator introduced in
d7049213d0.
2020-01-29 21:30:21 -08:00
Benjamin Kramer adcd026838 Make llvm::StringRef to std::string conversions explicit.
This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.

This doesn't actually modify StringRef yet, I'll do that in a follow-up.
2020-01-28 23:25:25 +01:00
Sam Clegg 51b521c07a [lld][WebAssembly] Use a more meaningful name for stub functions
When we generate these stub functions on signature mismatches give
them a more meaningful name so that when people see this in stack
traces is gives a clue as the what is going on.

See: https://github.com/emscripten-core/emscripten/issues/10226

Differential Revision: https://reviews.llvm.org/D72881
2020-01-16 14:55:37 -08:00
Sam Clegg 9cd985815a [lld][WebAssembly] Add libcall symbols to the link when LTO is being used.
This code is copied almost verbatim from the equivalent change to the
ELF linker:

- https://reviews.llvm.org/D50017
- https://reviews.llvm.org/D50475

The upshot is that libraries containing libcall (such as compiler-rt
and libc) can be compiled with LTO.

Fixes PR41384

Differential Revision: https://reviews.llvm.org/D71738
2020-01-10 11:01:05 -08:00
Wei Mi 21a4710c67 [ThinLTO] Pass CodeGenOpts like UnrollLoops/VectorizeLoop/VectorizeSLP
down to pass builder in ltobackend.

Currently CodeGenOpts like UnrollLoops/VectorizeLoop/VectorizeSLP in clang
are not passed down to pass builder in ltobackend when new pass manager is
used. This is inconsistent with the behavior when new pass manager is used
and thinlto is not used. Such inconsistency causes slp vectorization pass
not being enabled in ltobackend for O3 + thinlto right now. This patch
fixes that.

Differential Revision: https://reviews.llvm.org/D72386
2020-01-09 21:13:11 -08:00
Kazuaki Ishizaki 7ae3d33546 [lld] Fix trivial typos in comments
Reviewed By: ruiu, MaskRay

Differential Revision: https://reviews.llvm.org/D72196
2020-01-06 10:25:48 -08:00
Fangrui Song 681b1be774 [lld] Fix -Wrange-loop-analysis warnings
One instance looks like a false positive:

lld/ELF/Relocations.cpp:1622:14: note: use reference type 'const std::pair<ThunkSection *, uint32_t> &' (aka 'cons
t pair<lld:🧝:ThunkSection *, unsigned int> &') to prevent copying
        for (const std::pair<ThunkSection *, uint32_t> ts : isd->thunkSections)

It is not changed in this commit.
2020-01-01 15:41:20 -08:00
Sam Clegg 881d877846 [WebAssembly] Add new `export_name` clang attribute for controlling wasm export names
This is equivalent to the existing `import_name` and `import_module`
attributes which control the import names in the final wasm binary
produced by lld.

This maps the existing

This attribute currently requires a string rather than using the
symbol name for a couple of reasons:

1. Avoid confusion with static and dynamic linking which is
   based on symbol name.  Exporting a function from a wasm module using
   this directive is orthogonal to both static and dynamic linking.
2. Avoids name mangling.

Differential Revision: https://reviews.llvm.org/D70520
2019-12-11 11:54:57 -08:00
James Y Knight d3fec7fb45 LLD: Don't use the stderrOS stream in link before it's reassigned.
Remove the lld::enableColors function, as it just obscures which
stream it's affecting, and replace with explicit calls to the stream's
enable_colors.

Also, assign the stderrOS and stdoutOS globals first in link function,
just to ensure nothing might use them.

(Either change individually fixes the issue of using the old
stream, but both together seems best.)

Follow-up to b11386f9be.

Differential Revision: https://reviews.llvm.org/D70492
2019-11-21 10:55:03 -05:00
Rui Ueyama b11386f9be Make it possible to redirect not only errs() but also outs()
This change is for those who use lld as a library. Context:
https://reviews.llvm.org/D70287

This patch adds a new parmeter to lld::*::link() so that we can pass
an raw_ostream object representing stdout. Previously, lld::*::link()
took only an stderr object.

Justification for making stdoutOS and stderrOS mandatory: I wanted to
make link() functions to take stdout and stderr in that order.
However, if we change the function signature from

  bool link(ArrayRef<const char *> args, bool canExitEarly,
            raw_ostream &stderrOS = llvm::errs());

to

  bool link(ArrayRef<const char *> args, bool canExitEarly,
            raw_ostream &stdoutOS = llvm::outs(),
            raw_ostream &stderrOS = llvm::errs());

, then the meaning of existing code that passes stderrOS silently
changes (stderrOS would be interpreted as stdoutOS). So, I chose to
make existing code not to compile, so that developers can fix their
code.

Differential Revision: https://reviews.llvm.org/D70292
2019-11-18 11:18:06 +09:00
Sam Clegg baff8ec2e1 [WebAssembly][lld] Fix for static linking of PIC code
When statically linking PIC code we create an internalized __memory_base
so that memory-base-relative relocation work correctly.  The value of
this global should be zero, and not the globalBase since the globalBase
offset is already taken into account by getVirtualAddress.

Fixes: https://github.com/emscripten-core/emscripten/issues/9013

Differential Revision: https://reviews.llvm.org/D69600
2019-10-29 18:58:56 -07:00
Sam Clegg 67b055841f [lld][WebAssebmly] Preserve custom import attributes with LTO
Undefined symbols in WebAssembly can come with custom `import-module`
and `import-field` attributes.  However when reading symbols from
bitcode object files during LTO those curtom attributes are not
available.

Once we compile the LTO object and read in the symbol table from the
object file we have access to these custom attributes.  In this case,
when undefined symbols are added and a symbol already exists in the
SymbolTable we can't simple return it, we may need to update the
symbol's attributes.

Fixes: PR43211

Differential Revision: https://reviews.llvm.org/D68959

llvm-svn: 375081
2019-10-17 05:16:54 +00:00
Sam Clegg 6c393e9d74 [lld][WebAssembly] Fix for weak references to data symbols in archives
Fix a bug where were not handling relocations against weakly undefined
data symbol.  Add a test for this case.  Also ensure that the weak
references to data symbols are not pulled in from archive files by
default (but are if `-u <name>` is added to the command line).

Fixes: PR43696

Differential Revision: https://reviews.llvm.org/D69073

llvm-svn: 375077
2019-10-17 03:21:02 +00:00
Thomas Lively 190dacc3cc [WebAssembly] Elide data segments for .bss sections
Summary:
WebAssembly memories are zero-initialized, so when module does not
import its memory initializing .bss sections is guaranteed to be a
no-op. To reduce binary size and initialization time, .bss sections
are simply not emitted into the final binary unless the memory is
imported.

Reviewers: sbc100

Subscribers: dschuff, jgravelle-google, aheejin, sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D68965

llvm-svn: 374940
2019-10-15 19:05:11 +00:00
James Clarke 1ab27c74d4 [lld][WebAssembly] Fix static linking of -fPIC code with external undefined data
Reviewers: ruiu, sbc100

Reviewed By: sbc100

Subscribers: dschuff, jgravelle-google, aheejin, sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D68991

llvm-svn: 374913
2019-10-15 17:05:42 +00:00
Fangrui Song 33c59abf5c [WebAssembly] Wrap definitions in namespace lld { namespace wasm {. NFC
Similar to D68323, but for wasm.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D68759

llvm-svn: 374279
2019-10-10 05:25:39 +00:00
Sam Clegg ad2e12a3d9 [lld][WebAssembly] Refactor markLive.cpp. NFC
This pattern matches the ELF implementation add if also useful as
part of a planned change where running `mark` more than once is needed.

Differential Revision: https://reviews.llvm.org/D68749

llvm-svn: 374275
2019-10-10 03:23:06 +00:00
Martin Storsjo 5ebab1f8f9 [LLD] Simplify the demangleItanium function. NFC.
Instead of returning an optional, just return the input string if
demangling fails, as that's what all callers use anyway.

Differential Revision: https://reviews.llvm.org/D68015

llvm-svn: 373077
2019-09-27 12:24:18 +00:00
Sam Clegg 937b955837 [lld][WebAssembly] Fix static linking of -fPIC code with external undefined functions
Differential Revision: https://reviews.llvm.org/D66784

llvm-svn: 372779
2019-09-24 20:52:12 +00:00
Sam Clegg f6f4b98f03 [lld][WebAssembly] Preserve symbol flags in --relocatable output
Fixes https://github.com/emscripten-core/emscripten/issues/8879

Differential Revision: https://reviews.llvm.org/D67729

llvm-svn: 372660
2019-09-23 21:28:29 +00:00
Thomas Lively 0c3d4cfbad [WebAssembly][NFC] Remove unnecessary braces
llvm-svn: 372358
2019-09-19 21:51:52 +00:00
Thomas Lively 21143b93a6 [WebAssembly] Sort output data sections to place .bss last
Summary:
This was always the intended behavior, but had not been
implemented. This ordering is important for Emscripten when generating
.mem files while compiling to JS, since only zeros at the end of
initialized memory can be dropped.

Fixes https://github.com/emscripten-core/emscripten/issues/8999

Reviewers: sbc100

Subscribers: dschuff, jgravelle-google, aheejin, sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D67736

llvm-svn: 372284
2019-09-19 01:14:59 +00:00
Sam Clegg e40ef12bfa [lld][WebAssembly] Fix use after free of archive path
This was fixed in the ELF backend in https://reviews.llvm.org/D34554.

Differential Revision: https://reviews.llvm.org/D67676

llvm-svn: 372266
2019-09-18 21:51:03 +00:00
Thomas Lively 09768c5d7a [WebAssembly] Initialize memory in start function
Summary:
 - `__wasm_init_memory` is now the WebAssembly start function instead
   of being called from `__wasm_call_ctors` or called directly by the
   runtime.
 - Adds a new synthetic data symbol `__wasm_init_memory_flag` that is
   atomically incremented from zero to one by the thread responsible
   for initializing memory.
 - All threads now unconditionally perform data.drop on all passive
   segments.
 - Removes --passive-segments and --active-segments flags and controls
   segment type based on --shared-memory instead. The deleted flags
   were only present to ameliorate the upgrade path in Emscripten.

Reviewers: sbc100, aheejin

Subscribers: dschuff, jgravelle-google, sunfish, jfb, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D65783

llvm-svn: 370965
2019-09-04 19:50:39 +00:00
Sam Clegg fecfc5920a [lld][WebAssembly] Fix spurious signature mismatch warnings
Summary:
This a follow up on: https://reviews.llvm.org/D62153

Handle the case where there are multiple object files that contain
undefined references to the same function.  We only generate a function
variant if the existing symbol is directly called.

See: https://github.com/emscripten-core/emscripten/issues/8995

Subscribers: dschuff, jgravelle-google, aheejin, sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D67015

llvm-svn: 370509
2019-08-30 19:50:59 +00:00
Dan Gohman 7cb9c8a506 [WebAssembly] Implement NO_STRIP
This patch implements support for the NO_STRIP flag, which will allow
__attribute__((used)) to be implemented.

This accompanies https://reviews.llvm.org/D62542, which moves to setting the
NO_STRIP flag, and will continue to set EXPORTED for Emscripten targets for
compatibility.

Differential Revision: https://reviews.llvm.org/D66968

llvm-svn: 370416
2019-08-29 22:41:05 +00:00
Jacob Gravelle 92ed86d239 [lld][WebAssembly] Support for growable tables
Adds --growable-table flag to handle building wasm modules with tables
that can grow.

Wasm tables that we use to store function pointers. In order to add functions
to that table at runtime, we need to either preallocate space, or grow the table.
In order to specify a table with no maximum size, we need some flag to handle
that case, separately from a potential --max-table-size= flag.

Note that the number of elements in the table isn't knowable until link-time,
so it's unclear if we will want a --max-table-size= flag in the future.

llvm-svn: 370127
2019-08-27 22:58:21 +00:00
Sam Clegg 040ef1091d [lld][WebAssembly] Create optional symbols after handling --export/--undefined
Handling of --export/--undefined can pull in lazy symbols which in turn
can pull in referenced to optional symbols.  We need to delay the
creation of optional symbols until all possible references to them have
been created.

Differential Revision: https://reviews.llvm.org/D66768

llvm-svn: 370012
2019-08-27 04:27:57 +00:00
Sam Clegg 1a1df72a43 [lld][WebAssembly] Store table base in config rather than passing it around. NFC.
I've got another change that makes more use of this value in other
places.

Differential Revision: https://reviews.llvm.org/D66777

llvm-svn: 370010
2019-08-27 04:19:34 +00:00
Sam Clegg cf2b8722d4 [WebAssembly][lld] Fix crash when applying relocations to debug sections
Debug sections are special in that they can contain relocations against
symbols that are not present in the final output (i.e. not live).
However it is also possible to have R_WASM_TABLE_INDEX relocations
against symbols that don't have a table index assigned (since they are
not address taken by actual code.

Fixes: https://github.com/emscripten-core/emscripten/issues/9023

Differential Revision: https://reviews.llvm.org/D66435

llvm-svn: 369423
2019-08-20 18:39:24 +00:00
Sam Clegg e8e914e640 [lld][WebAssembly] Honor --no-export-dynamic even with -shared
Differential Revision: https://reviews.llvm.org/D66359

llvm-svn: 369276
2019-08-19 16:34:51 +00:00
Jonas Devlieghere 6ba7992031 [LLD] Migrate llvm::make_unique to std::make_unique
Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.

Differential revision: https://reviews.llvm.org/D66259

llvm-svn: 368936
2019-08-14 22:28:17 +00:00
Sam Clegg 7185a7301e [lld][WebAssembly] Allow linking of pic code into static binaries
Summary: See https://github.com/emscripten-core/emscripten/issues/9013

Subscribers: dschuff, jgravelle-google, aheejin, sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D65922

llvm-svn: 368719
2019-08-13 17:02:02 +00:00
Sam Clegg dbfea28219 [lld][WebAssembly] Don't create optional symbols when outputing an object file
Summary: This was a bug in rL368310.  I'm working on a test case now.

Differential Revision: https://reviews.llvm.org/D65985

llvm-svn: 368369
2019-08-08 23:56:21 +00:00
Guanzhong Chen c5ccbf52ad [WebAssembly][lld] control __data_end export with config->shared
Summary:
Emscripten expects `__data_end` to show up in PIC code as long as it's not
linked with `--shared`.

Currently, Emscripten breaks with latest LLVM because `__data_end` is controlled
by `config->isPic` instead of `config->shared`.`

Reviewers: tlively, sbc100

Reviewed By: sbc100

Subscribers: dschuff, jgravelle-google, aheejin, sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D65980

llvm-svn: 368361
2019-08-08 22:40:04 +00:00
Sam Clegg ad1cc145e8 [lld][WebAssembly] Use createGlobalVariable helper function. NFC.
Differential Revision: https://reviews.llvm.org/D65911

llvm-svn: 368325
2019-08-08 18:22:03 +00:00
Sam Clegg caa0db1318 [lld][WebAssembly] Add optional symbols after input file handling
This allows undefined references in input files be resolved by the
optional symbols.  Previously we were doing this before input file
reading which means it was working only for command line symbols
references (i.e. -u or --export).

Also use addOptionalDataSymbol for __dso_handle and make all optional
symbols hidden by default.

Differential Revision: https://reviews.llvm.org/D65920

llvm-svn: 368310
2019-08-08 16:58:36 +00:00
Rui Ueyama cac8df1ab9 Re-submit r367649: Improve raw_ostream so that you can "write" colors using operator<<
The original patch broke buildbots, perhaps because it changed the
default setting whether colors are enabled or not.

llvm-svn: 368131
2019-08-07 08:08:17 +00:00
Guanzhong Chen 0cb776e61a [WebAssembly] Fix null pointer in createInitTLSFunction
Summary:
`createSyntheticSymbols`, which creates `WasmSym::InitTLS`, is only called
when `!config->relocatable`, but this condition is not checked when calling
`createInitTLSFunction`.

This diff checks `!config->relocatable` before calling `createInitTLSFunction`.

Fixes https://github.com/emscripten-core/emscripten/issues/9155.

Reviewers: tlively, aheejin, kripken, sbc100

Subscribers: dschuff, jgravelle-google, sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D65785

llvm-svn: 368078
2019-08-06 20:09:04 +00:00
Rui Ueyama 4d41c332ef Revert r367649: Improve raw_ostream so that you can "write" colors using operator<<
This reverts commit r367649 in an attempt to unbreak Windows bots.

llvm-svn: 367658
2019-08-02 07:22:34 +00:00
Rui Ueyama a52f982f1c Improve raw_ostream so that you can "write" colors using operator<<
1. raw_ostream supports ANSI colors so that you can write messages to
the termina with colors. Previously, in order to change and reset
color, you had to call `changeColor` and `resetColor` functions,
respectively.

So, if you print out "error: " in red, for example, you had to do
something like this:

  OS.changeColor(raw_ostream::RED);
  OS << "error: ";
  OS.resetColor();

With this patch, you can write the same code as follows:

  OS << raw_ostream::RED << "error: " << raw_ostream::RESET;

2. Add a boolean flag to raw_ostream so that you can disable colored
output. If you disable colors, changeColor, operator<<(Color),
resetColor and other color-related functions have no effect.

Most LLVM tools automatically prints out messages using colors, and
you can disable it by passing a flag such as `--disable-colors`.
This new flag makes it easy to write code that works that way.

Differential Revision: https://reviews.llvm.org/D65564

llvm-svn: 367649
2019-08-02 04:48:30 +00:00
Guanzhong Chen 87186b2447 [WebAssembly] Set __tls_align to 1 when there is no TLS
Summary:
We want the tool conventions to state that `__tls_align` will be a power of 2.
It makes sense to not have an exception for when there is no TLS.

Reviewers: tlively, sunfish

Reviewed By: tlively

Subscribers: dschuff, sbc100, jgravelle-google, aheejin, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D65177

llvm-svn: 366948
2019-07-24 21:48:14 +00:00
Guanzhong Chen 5204f7611f [WebAssembly] Compute and export TLS block alignment
Summary:
Add immutable WASM global `__tls_align` which stores the alignment
requirements of the TLS segment.

Add `__builtin_wasm_tls_align()` intrinsic to get this alignment in Clang.

The expected usage has now changed to:

    __wasm_init_tls(memalign(__builtin_wasm_tls_align(),
                             __builtin_wasm_tls_size()));

Reviewers: tlively, aheejin, sbc100, sunfish, alexcrichton

Reviewed By: tlively

Subscribers: dschuff, jgravelle-google, hiraditya, cfe-commits, llvm-commits

Tags: #clang, #llvm

Differential Revision: https://reviews.llvm.org/D65028

llvm-svn: 366624
2019-07-19 23:34:16 +00:00
Thomas Lively 6a6f28f7b7 [WebAssembly] Use passive segments by default when memory is shared
Summary:
This change makes it so that passing --shared-memory is all a user
needs to do to get proper multithreaded code. This default can still
be explicitly overridden for any reason using --passive-segments and
--active-segments.

Reviewers: sbc100, quantum

Subscribers: dschuff, jgravelle-google, aheejin, sunfish, jfb, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64950

llvm-svn: 366504
2019-07-18 21:50:24 +00:00
Guanzhong Chen 21aafc2e0c [WebAssembly] fix bug in finding .tdata segment
Summary: Fix bug in `wasm-ld`'s `Writer::createInitTLSFunction` that only finds `.tdata` if it's the first section.

Reviewers: tlively, aheejin, sbc100

Reviewed By: sbc100

Subscribers: dschuff, jgravelle-google, sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64947

llvm-svn: 366500
2019-07-18 21:18:24 +00:00
Sam Clegg accad76c14 [lld][WebAssembly] Fix handling of comdat functions in init array.
When hidden symbols are discarded by comdat rules we still want to
create a local defined symbol, otherwise `Symbol::isDiscarded()` relies
on begin able to check `getChunk->discarded`.

This is a followup on rL362769. The comdat.ll test was previously GC'ing
the `__wasm_call_ctors` functions so `do_init` was not actually being
included in the link.  Once that function was included in triggered the
crash bug that this change addresses.

Fixes: https://github.com/emscripten-core/emscripten/issues/8981

Differential Revision: https://reviews.llvm.org/D64872

llvm-svn: 366358
2019-07-17 18:43:36 +00:00
Guanzhong Chen 42bba4b852 [WebAssembly] Implement thread-local storage (local-exec model)
Summary:
Thread local variables are placed inside a `.tdata` segment. Their symbols are
offsets from the start of the segment. The address of a thread local variable
is computed as `__tls_base` + the offset from the start of the segment.

`.tdata` segment is a passive segment and `memory.init` is used once per thread
to initialize the thread local storage.

`__tls_base` is a wasm global. Since each thread has its own wasm instance,
it is effectively thread local. Currently, `__tls_base` must be initialized
at thread startup, and so cannot be used with dynamic libraries.

`__tls_base` is to be initialized with a new linker-synthesized function,
`__wasm_init_tls`, which takes as an argument a block of memory to use as the
storage for thread locals. It then initializes the block of memory and sets
`__tls_base`. As `__wasm_init_tls` will handle the memory initialization,
the memory does not have to be zeroed.

To help allocating memory for thread-local storage, a new compiler intrinsic
is introduced: `__builtin_wasm_tls_size()`. This instrinsic function returns
the size of the thread-local storage for the current function.

The expected usage is to run something like the following upon thread startup:

    __wasm_init_tls(malloc(__builtin_wasm_tls_size()));

Reviewers: tlively, aheejin, kripken, sbc100

Subscribers: dschuff, jgravelle-google, hiraditya, sunfish, jfb, cfe-commits, llvm-commits

Tags: #clang, #llvm

Differential Revision: https://reviews.llvm.org/D64537

llvm-svn: 366272
2019-07-16 22:00:45 +00:00
Fangrui Song 33fdf82dda [WebAssembly] Rename variale references in comments after VariableName -> variableName change
llvm-svn: 366192
2019-07-16 08:08:17 +00:00
Heejin Ahn 9f96a58ccc [WebAssembly] Rename except_ref type to exnref
Summary:
We agreed to rename `except_ref` to `exnref` for consistency with other
reference types in
https://github.com/WebAssembly/exception-handling/issues/79. This also
renames WebAssemblyInstrExceptRef.td to WebAssemblyInstrRef.td in order
to use the file for other reference types in future.

Reviewers: dschuff

Subscribers: sbc100, jgravelle-google, hiraditya, sunfish, jfb, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64703

llvm-svn: 366145
2019-07-15 22:49:25 +00:00
Thomas Lively 26a6b95da9 [WebAssembly] i32.const operands should be signed
Summary:
This was causing large addresses to be emitted as negative numbers,
which rightfully caused crashes in binaryen.

Reviewers: aheejin, dschuff

Subscribers: sbc100, jgravelle-google, sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64612

llvm-svn: 365930
2019-07-12 17:55:07 +00:00