Summary:
The WebAssembly backend automatically lowers atomic operations and TLS
to nonatomic operations and non-TLS data when either are present and
the atomics or bulk-memory features are not present, respectively. The
resulting object is no longer thread-safe, so the linker has to be
told not to allow it to be linked into a module with shared
memory. This was previously done by disallowing the 'atomics' feature,
which prevented any objct with its atomic operations or TLS removed
from being linked with any object containing atomics or TLS, and
therefore preventing it from being linked into a module with shared
memory since shared memory requires atomics.
However, as of https://github.com/WebAssembly/threads/issues/144, the
validation rules are relaxed to allow atomic operations to validate
with unshared memories, which makes it perfectly safe to link an
object with stripped atomics and TLS with another object that still
contains TLS and atomics as long as the resulting module has an
unshared memory. To allow this kind of link, this patch disallows a
pseudo-feature 'shared-mem' rather than 'atomics' to communicate to
the linker that the object is not thread-safe. This means that the
'atomics' feature is available to accurately reflect whether or not an
object has atomics enabled.
As a drive-by tweak, this change also requires that bulk-memory be
enabled in addition to atomics in order to use shared memory. This is
because initializing shared memories requires bulk-memory operations.
Reviewers: aheejin, sbc100
Subscribers: dschuff, jgravelle-google, hiraditya, sunfish, jfb, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D79542
Essentially takes the lld/Common/Threads.h wrappers and moves them to
the llvm/Support/Paralle.h algorithm header.
The changes are:
- Remove policy parameter, since all clients use `par`.
- Rename the methods to `parallelSort` etc to match LLVM style, since
they are no longer C++17 pstl compatible.
- Move algorithms from llvm::parallel:: to llvm::, since they have
"parallel" in the name and are no longer overloads of the regular
algorithms.
- Add range overloads
- Use the sequential algorithm directly when 1 thread is requested
(skips task grouping)
- Fix the index type of parallelForEachN to size_t. Nobody in LLVM was
using any other parameter, and it made overload resolution hard for
for_each_n(par, 0, foo.size(), ...) because 0 is int, not size_t.
Remove Threads.h and update LLD for that.
This is a prerequisite for parallel public symbol processing in the PDB
library, which is in LLVM.
Reviewed By: MaskRay, aganea
Differential Revision: https://reviews.llvm.org/D79390
This change add support for defined wasm globals in the .s format,
the MC layer, and wasm-ld
Currently there is no support custom initialization and all wasm
globals are initialized to zero.
Fixes: PR45742
Differential Revision: https://reviews.llvm.org/D79137
These stub new function were not being added to the symbol table
which in turn meant that we were crashing when trying to output
relocations against them.
Differential Revision: https://reviews.llvm.org/D78779
Summary:
wasm-ld requires --shared-memory to be passed when the atomics feature
is enabled because historically atomic operations were only valid with
shared memories. This change relaxes that requirement for when
building relocatable objects because their memories are not
meaningful. This technically maintains the validity of object files
because the threads spec now allows atomic operations with unshared
memories, although we don't support that elsewhere in the tools yet.
This fixes and Emscripten build issue reported at
https://bugs.chromium.org/p/webp/issues/detail?id=463.
Reviewers: sbc100
Subscribers: dschuff, jgravelle-google, aheejin, sunfish, jfb, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D78072
It can be used to avoid passing the begin and end of a range.
This makes the code shorter and it is consistent with another
wrappers we already have.
Differential revision: https://reviews.llvm.org/D78016
Summary:
A previous change (53211a) had updated the argument parsing to handle
large max memories, but 4294967296 would still wrap to zero after the
options were parsed. This change updates the configuration to use a
64-bit integer to store the max memory to avoid that overflow.
Reviewers: sbc100
Subscribers: dschuff, jgravelle-google, aheejin, sunfish, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D77437
This matches the behaviour of the ELF driver.
Also move the `createFiles` to be `checkConfig` and report `no input
files` there. Again this is mostly to match the structure of the ELF
linker better.
Differential Revision: https://reviews.llvm.org/D76960
--no-threads is a name copied from gold.
gold has --no-thread, --thread-count and several other --thread-count-*.
There are needs to customize the number of threads (running several lld
processes concurrently or customizing the number of LTO threads).
Having a single --threads=N is a straightforward replacement of gold's
--no-threads + --thread-count.
--no-threads is used rarely. So just delete --no-threads instead of
keeping it for compatibility for a while.
If --threads= is specified (ELF,wasm; COFF /threads: is similar),
--thinlto-jobs= defaults to --threads=,
otherwise all available hardware threads are used.
There is currently no way to override a --threads={1,2,...}. It is still
a debate whether we should use --threads=all.
Reviewed By: rnk, aganea
Differential Revision: https://reviews.llvm.org/D76885
Before this patch, it wasn't possible to extend the ThinLTO threads to all SMT/CMT threads in the system. Only one thread per core was allowed, instructed by usage of llvm::heavyweight_hardware_concurrency() in the ThinLTO code. Any number passed to the LLD flag /opt:lldltojobs=..., or any other ThinLTO-specific flag, was previously interpreted in the context of llvm::heavyweight_hardware_concurrency(), which means SMT disabled.
One can now say in LLD:
/opt:lldltojobs=0 -- Use one std::thread / hardware core in the system (no SMT). Default value if flag not specified.
/opt:lldltojobs=N -- Limit usage to N threads, regardless of usage of heavyweight_hardware_concurrency().
/opt:lldltojobs=all -- Use all hardware threads in the system. Equivalent to /opt:lldltojobs=$(nproc) on Linux and /opt:lldltojobs=%NUMBER_OF_PROCESSORS% on Windows. When an affinity mask is set for the process, threads will be created only for the cores selected by the mask.
When N > number-of-hardware-threads-in-the-system, the threads in the thread pool will be dispatched equally on all CPU sockets (tested only on Windows).
When N <= number-of-hardware-threads-on-a-CPU-socket, the threads will remain on the CPU socket where the process started (only on Windows).
Differential Revision: https://reviews.llvm.org/D75153
When the debug info contains a relocation against a dead symbol, wasm-ld
may emit spurious range-list terminator entries (entries with Start==0
and End==0). This change fixes this by emitting the WasmRelocation
Addend as End value for a non-live symbol.
Reviewed by: sbc100, dblaikie
Differential Revision: https://reviews.llvm.org/D74781
When there are both strong and weak references to an undefined
symbol ensure that the strong reference prevails in the output symbol
generating the correct error.
Test case copied from lld/test/ELF/weak-and-strong-undef.s
Differential Revision: https://reviews.llvm.org/D75322
WebAssembly requires that caller and callee signatures match, so it
can't do the usual trick of passing more arguments to main than it
expects. Instead WebAssembly will mangle "main" with argc/argv
parameters as "__main_argc_argv". This patch teaches lld how to
demangle it.
This patch is part of https://reviews.llvm.org/D70700.
The changes the in-memory representation of wasm symbols such that their
optional ImportName and ImportModule use llvm::Optional.
ImportName is set whenever WASM_SYMBOL_EXPLICIT_NAME flag is set.
ImportModule (for imports) is currently always set since it defaults to
"env".
In the future we can possibly extent to binary format distingish
import which have explit module names.
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D74109
This can happen if lto::LTO::getRuntimeLibcallSymbols doesn't return
an complete/accurate list of libcalls. In this case new bitcode
object can be linked in after LTO.
For example the WebAssembly backend currently calls:
setLibcallName(RTLIB::FPROUND_F32_F16, "__truncsfhf2");
But `__truncsfhf2` is not part of `getRuntimeLibcallSymbols` so if
this symbol is generated during LTO the link will currently fail.
Without this change the linker crashes because the bitcode symbol
makes it all the way to the output phase.
See: https://bugs.llvm.org/show_bug.cgi?id=44353
Differential Revision: https://reviews.llvm.org/D71632
This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.
This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.
This doesn't actually modify StringRef yet, I'll do that in a follow-up.
down to pass builder in ltobackend.
Currently CodeGenOpts like UnrollLoops/VectorizeLoop/VectorizeSLP in clang
are not passed down to pass builder in ltobackend when new pass manager is
used. This is inconsistent with the behavior when new pass manager is used
and thinlto is not used. Such inconsistency causes slp vectorization pass
not being enabled in ltobackend for O3 + thinlto right now. This patch
fixes that.
Differential Revision: https://reviews.llvm.org/D72386
One instance looks like a false positive:
lld/ELF/Relocations.cpp:1622:14: note: use reference type 'const std::pair<ThunkSection *, uint32_t> &' (aka 'cons
t pair<lld:🧝:ThunkSection *, unsigned int> &') to prevent copying
for (const std::pair<ThunkSection *, uint32_t> ts : isd->thunkSections)
It is not changed in this commit.
This is equivalent to the existing `import_name` and `import_module`
attributes which control the import names in the final wasm binary
produced by lld.
This maps the existing
This attribute currently requires a string rather than using the
symbol name for a couple of reasons:
1. Avoid confusion with static and dynamic linking which is
based on symbol name. Exporting a function from a wasm module using
this directive is orthogonal to both static and dynamic linking.
2. Avoids name mangling.
Differential Revision: https://reviews.llvm.org/D70520
Remove the lld::enableColors function, as it just obscures which
stream it's affecting, and replace with explicit calls to the stream's
enable_colors.
Also, assign the stderrOS and stdoutOS globals first in link function,
just to ensure nothing might use them.
(Either change individually fixes the issue of using the old
stream, but both together seems best.)
Follow-up to b11386f9be.
Differential Revision: https://reviews.llvm.org/D70492
This change is for those who use lld as a library. Context:
https://reviews.llvm.org/D70287
This patch adds a new parmeter to lld::*::link() so that we can pass
an raw_ostream object representing stdout. Previously, lld::*::link()
took only an stderr object.
Justification for making stdoutOS and stderrOS mandatory: I wanted to
make link() functions to take stdout and stderr in that order.
However, if we change the function signature from
bool link(ArrayRef<const char *> args, bool canExitEarly,
raw_ostream &stderrOS = llvm::errs());
to
bool link(ArrayRef<const char *> args, bool canExitEarly,
raw_ostream &stdoutOS = llvm::outs(),
raw_ostream &stderrOS = llvm::errs());
, then the meaning of existing code that passes stderrOS silently
changes (stderrOS would be interpreted as stdoutOS). So, I chose to
make existing code not to compile, so that developers can fix their
code.
Differential Revision: https://reviews.llvm.org/D70292
When statically linking PIC code we create an internalized __memory_base
so that memory-base-relative relocation work correctly. The value of
this global should be zero, and not the globalBase since the globalBase
offset is already taken into account by getVirtualAddress.
Fixes: https://github.com/emscripten-core/emscripten/issues/9013
Differential Revision: https://reviews.llvm.org/D69600
Undefined symbols in WebAssembly can come with custom `import-module`
and `import-field` attributes. However when reading symbols from
bitcode object files during LTO those curtom attributes are not
available.
Once we compile the LTO object and read in the symbol table from the
object file we have access to these custom attributes. In this case,
when undefined symbols are added and a symbol already exists in the
SymbolTable we can't simple return it, we may need to update the
symbol's attributes.
Fixes: PR43211
Differential Revision: https://reviews.llvm.org/D68959
llvm-svn: 375081
Fix a bug where were not handling relocations against weakly undefined
data symbol. Add a test for this case. Also ensure that the weak
references to data symbols are not pulled in from archive files by
default (but are if `-u <name>` is added to the command line).
Fixes: PR43696
Differential Revision: https://reviews.llvm.org/D69073
llvm-svn: 375077
Summary:
WebAssembly memories are zero-initialized, so when module does not
import its memory initializing .bss sections is guaranteed to be a
no-op. To reduce binary size and initialization time, .bss sections
are simply not emitted into the final binary unless the memory is
imported.
Reviewers: sbc100
Subscribers: dschuff, jgravelle-google, aheejin, sunfish, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D68965
llvm-svn: 374940
This pattern matches the ELF implementation add if also useful as
part of a planned change where running `mark` more than once is needed.
Differential Revision: https://reviews.llvm.org/D68749
llvm-svn: 374275
Instead of returning an optional, just return the input string if
demangling fails, as that's what all callers use anyway.
Differential Revision: https://reviews.llvm.org/D68015
llvm-svn: 373077
Summary:
This was always the intended behavior, but had not been
implemented. This ordering is important for Emscripten when generating
.mem files while compiling to JS, since only zeros at the end of
initialized memory can be dropped.
Fixes https://github.com/emscripten-core/emscripten/issues/8999
Reviewers: sbc100
Subscribers: dschuff, jgravelle-google, aheejin, sunfish, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D67736
llvm-svn: 372284
Summary:
- `__wasm_init_memory` is now the WebAssembly start function instead
of being called from `__wasm_call_ctors` or called directly by the
runtime.
- Adds a new synthetic data symbol `__wasm_init_memory_flag` that is
atomically incremented from zero to one by the thread responsible
for initializing memory.
- All threads now unconditionally perform data.drop on all passive
segments.
- Removes --passive-segments and --active-segments flags and controls
segment type based on --shared-memory instead. The deleted flags
were only present to ameliorate the upgrade path in Emscripten.
Reviewers: sbc100, aheejin
Subscribers: dschuff, jgravelle-google, sunfish, jfb, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D65783
llvm-svn: 370965
Thomas Lively