Commit Graph

204 Commits

Author SHA1 Message Date
Kazuaki Ishizaki 7c5fcb3591 [lld] NFC: fix trivial typos in comments
Differential Revision: https://reviews.llvm.org/D72339
2020-04-02 01:21:36 +09:00
Sam Clegg b5767010a8 [lld][WebAssembly] Early error if output file cannot be created.
This matches the behaviour of the ELF driver.

Also move the `createFiles` to be `checkConfig` and report `no input
files` there.   Again this is mostly to match the structure of the ELF
linker better.

Differential Revision: https://reviews.llvm.org/D76960
2020-03-31 21:42:38 -07:00
Fangrui Song eb4663d8c6 [lld][COFF][ELF][WebAssembly] Replace --[no-]threads /threads[:no] with --threads={1,2,...} /threads:{1,2,...}
--no-threads is a name copied from gold.
gold has --no-thread, --thread-count and several other --thread-count-*.

There are needs to customize the number of threads (running several lld
processes concurrently or customizing the number of LTO threads).
Having a single --threads=N is a straightforward replacement of gold's
--no-threads + --thread-count.

--no-threads is used rarely. So just delete --no-threads instead of
keeping it for compatibility for a while.

If --threads= is specified (ELF,wasm; COFF /threads: is similar),
--thinlto-jobs= defaults to --threads=,
otherwise all available hardware threads are used.

There is currently no way to override a --threads={1,2,...}. It is still
a debate whether we should use --threads=all.

Reviewed By: rnk, aganea

Differential Revision: https://reviews.llvm.org/D76885
2020-03-31 08:46:12 -07:00
Alexandre Ganea 09158252f7 [ThinLTO] Allow usage of all hardware threads in the system
Before this patch, it wasn't possible to extend the ThinLTO threads to all SMT/CMT threads in the system. Only one thread per core was allowed, instructed by usage of llvm::heavyweight_hardware_concurrency() in the ThinLTO code. Any number passed to the LLD flag /opt:lldltojobs=..., or any other ThinLTO-specific flag, was previously interpreted in the context of llvm::heavyweight_hardware_concurrency(), which means SMT disabled.

One can now say in LLD:
/opt:lldltojobs=0 -- Use one std::thread / hardware core in the system (no SMT). Default value if flag not specified.
/opt:lldltojobs=N -- Limit usage to N threads, regardless of usage of heavyweight_hardware_concurrency().
/opt:lldltojobs=all -- Use all hardware threads in the system. Equivalent to /opt:lldltojobs=$(nproc) on Linux and /opt:lldltojobs=%NUMBER_OF_PROCESSORS% on Windows. When an affinity mask is set for the process, threads will be created only for the cores selected by the mask.

When N > number-of-hardware-threads-in-the-system, the threads in the thread pool will be dispatched equally on all CPU sockets (tested only on Windows).
When N <= number-of-hardware-threads-on-a-CPU-socket, the threads will remain on the CPU socket where the process started (only on Windows).

Differential Revision: https://reviews.llvm.org/D75153
2020-03-27 10:20:58 -04:00
Reid Kleckner 213aea4c58 Remove unused Endian.h includes, NFC
Mainly avoids including Host.h everywhere:

$ diff -u <(sort thedeps-before.txt) <(sort thedeps-after.txt) \
    | grep '^[-+] ' | sort | uniq -c | sort -nr
   3141 - /usr/local/google/home/rnk/llvm-project/llvm/include/llvm/Support/Host.h
2020-03-11 15:45:34 -07:00
Sam Clegg 928e9e1723 [lld][WebAssembly] Add support for --rsp-quoting
This also changes to default style to match the host.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D75577
2020-03-04 11:41:33 -08:00
Sam Clegg bd4812776b [WebAssembly] Use llvm::Optional to store optional symbol attributes. NFC.
The changes the in-memory representation of wasm symbols such that their
optional ImportName and ImportModule use llvm::Optional.

ImportName is set whenever WASM_SYMBOL_EXPLICIT_NAME flag is set.
ImportModule (for imports) is currently always set since it defaults to
"env".

In the future we can possibly extent to binary format distingish
import which have explit module names.

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D74109
2020-02-19 17:25:33 -08:00
Jonas Devlieghere 3e24242a7d [lld] Replace SmallStr.str().str() with std::string conversion operator.
Use the std::string conversion operator introduced in
d7049213d0.
2020-01-29 21:30:21 -08:00
Benjamin Kramer adcd026838 Make llvm::StringRef to std::string conversions explicit.
This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.

This doesn't actually modify StringRef yet, I'll do that in a follow-up.
2020-01-28 23:25:25 +01:00
Sam Clegg 9cd985815a [lld][WebAssembly] Add libcall symbols to the link when LTO is being used.
This code is copied almost verbatim from the equivalent change to the
ELF linker:

- https://reviews.llvm.org/D50017
- https://reviews.llvm.org/D50475

The upshot is that libraries containing libcall (such as compiler-rt
and libc) can be compiled with LTO.

Fixes PR41384

Differential Revision: https://reviews.llvm.org/D71738
2020-01-10 11:01:05 -08:00
Fangrui Song 681b1be774 [lld] Fix -Wrange-loop-analysis warnings
One instance looks like a false positive:

lld/ELF/Relocations.cpp:1622:14: note: use reference type 'const std::pair<ThunkSection *, uint32_t> &' (aka 'cons
t pair<lld:🧝:ThunkSection *, unsigned int> &') to prevent copying
        for (const std::pair<ThunkSection *, uint32_t> ts : isd->thunkSections)

It is not changed in this commit.
2020-01-01 15:41:20 -08:00
James Y Knight d3fec7fb45 LLD: Don't use the stderrOS stream in link before it's reassigned.
Remove the lld::enableColors function, as it just obscures which
stream it's affecting, and replace with explicit calls to the stream's
enable_colors.

Also, assign the stderrOS and stdoutOS globals first in link function,
just to ensure nothing might use them.

(Either change individually fixes the issue of using the old
stream, but both together seems best.)

Follow-up to b11386f9be.

Differential Revision: https://reviews.llvm.org/D70492
2019-11-21 10:55:03 -05:00
Rui Ueyama b11386f9be Make it possible to redirect not only errs() but also outs()
This change is for those who use lld as a library. Context:
https://reviews.llvm.org/D70287

This patch adds a new parmeter to lld::*::link() so that we can pass
an raw_ostream object representing stdout. Previously, lld::*::link()
took only an stderr object.

Justification for making stdoutOS and stderrOS mandatory: I wanted to
make link() functions to take stdout and stderr in that order.
However, if we change the function signature from

  bool link(ArrayRef<const char *> args, bool canExitEarly,
            raw_ostream &stderrOS = llvm::errs());

to

  bool link(ArrayRef<const char *> args, bool canExitEarly,
            raw_ostream &stdoutOS = llvm::outs(),
            raw_ostream &stderrOS = llvm::errs());

, then the meaning of existing code that passes stderrOS silently
changes (stderrOS would be interpreted as stdoutOS). So, I chose to
make existing code not to compile, so that developers can fix their
code.

Differential Revision: https://reviews.llvm.org/D70292
2019-11-18 11:18:06 +09:00
Fangrui Song 33c59abf5c [WebAssembly] Wrap definitions in namespace lld { namespace wasm {. NFC
Similar to D68323, but for wasm.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D68759

llvm-svn: 374279
2019-10-10 05:25:39 +00:00
Sam Clegg f6f4b98f03 [lld][WebAssembly] Preserve symbol flags in --relocatable output
Fixes https://github.com/emscripten-core/emscripten/issues/8879

Differential Revision: https://reviews.llvm.org/D67729

llvm-svn: 372660
2019-09-23 21:28:29 +00:00
Thomas Lively 09768c5d7a [WebAssembly] Initialize memory in start function
Summary:
 - `__wasm_init_memory` is now the WebAssembly start function instead
   of being called from `__wasm_call_ctors` or called directly by the
   runtime.
 - Adds a new synthetic data symbol `__wasm_init_memory_flag` that is
   atomically incremented from zero to one by the thread responsible
   for initializing memory.
 - All threads now unconditionally perform data.drop on all passive
   segments.
 - Removes --passive-segments and --active-segments flags and controls
   segment type based on --shared-memory instead. The deleted flags
   were only present to ameliorate the upgrade path in Emscripten.

Reviewers: sbc100, aheejin

Subscribers: dschuff, jgravelle-google, sunfish, jfb, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D65783

llvm-svn: 370965
2019-09-04 19:50:39 +00:00
Jacob Gravelle 92ed86d239 [lld][WebAssembly] Support for growable tables
Adds --growable-table flag to handle building wasm modules with tables
that can grow.

Wasm tables that we use to store function pointers. In order to add functions
to that table at runtime, we need to either preallocate space, or grow the table.
In order to specify a table with no maximum size, we need some flag to handle
that case, separately from a potential --max-table-size= flag.

Note that the number of elements in the table isn't knowable until link-time,
so it's unclear if we will want a --max-table-size= flag in the future.

llvm-svn: 370127
2019-08-27 22:58:21 +00:00
Sam Clegg 040ef1091d [lld][WebAssembly] Create optional symbols after handling --export/--undefined
Handling of --export/--undefined can pull in lazy symbols which in turn
can pull in referenced to optional symbols.  We need to delay the
creation of optional symbols until all possible references to them have
been created.

Differential Revision: https://reviews.llvm.org/D66768

llvm-svn: 370012
2019-08-27 04:27:57 +00:00
Sam Clegg e8e914e640 [lld][WebAssembly] Honor --no-export-dynamic even with -shared
Differential Revision: https://reviews.llvm.org/D66359

llvm-svn: 369276
2019-08-19 16:34:51 +00:00
Sam Clegg 7185a7301e [lld][WebAssembly] Allow linking of pic code into static binaries
Summary: See https://github.com/emscripten-core/emscripten/issues/9013

Subscribers: dschuff, jgravelle-google, aheejin, sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D65922

llvm-svn: 368719
2019-08-13 17:02:02 +00:00
Sam Clegg dbfea28219 [lld][WebAssembly] Don't create optional symbols when outputing an object file
Summary: This was a bug in rL368310.  I'm working on a test case now.

Differential Revision: https://reviews.llvm.org/D65985

llvm-svn: 368369
2019-08-08 23:56:21 +00:00
Guanzhong Chen c5ccbf52ad [WebAssembly][lld] control __data_end export with config->shared
Summary:
Emscripten expects `__data_end` to show up in PIC code as long as it's not
linked with `--shared`.

Currently, Emscripten breaks with latest LLVM because `__data_end` is controlled
by `config->isPic` instead of `config->shared`.`

Reviewers: tlively, sbc100

Reviewed By: sbc100

Subscribers: dschuff, jgravelle-google, aheejin, sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D65980

llvm-svn: 368361
2019-08-08 22:40:04 +00:00
Sam Clegg ad1cc145e8 [lld][WebAssembly] Use createGlobalVariable helper function. NFC.
Differential Revision: https://reviews.llvm.org/D65911

llvm-svn: 368325
2019-08-08 18:22:03 +00:00
Sam Clegg caa0db1318 [lld][WebAssembly] Add optional symbols after input file handling
This allows undefined references in input files be resolved by the
optional symbols.  Previously we were doing this before input file
reading which means it was working only for command line symbols
references (i.e. -u or --export).

Also use addOptionalDataSymbol for __dso_handle and make all optional
symbols hidden by default.

Differential Revision: https://reviews.llvm.org/D65920

llvm-svn: 368310
2019-08-08 16:58:36 +00:00
Rui Ueyama cac8df1ab9 Re-submit r367649: Improve raw_ostream so that you can "write" colors using operator<<
The original patch broke buildbots, perhaps because it changed the
default setting whether colors are enabled or not.

llvm-svn: 368131
2019-08-07 08:08:17 +00:00
Rui Ueyama 4d41c332ef Revert r367649: Improve raw_ostream so that you can "write" colors using operator<<
This reverts commit r367649 in an attempt to unbreak Windows bots.

llvm-svn: 367658
2019-08-02 07:22:34 +00:00
Rui Ueyama a52f982f1c Improve raw_ostream so that you can "write" colors using operator<<
1. raw_ostream supports ANSI colors so that you can write messages to
the termina with colors. Previously, in order to change and reset
color, you had to call `changeColor` and `resetColor` functions,
respectively.

So, if you print out "error: " in red, for example, you had to do
something like this:

  OS.changeColor(raw_ostream::RED);
  OS << "error: ";
  OS.resetColor();

With this patch, you can write the same code as follows:

  OS << raw_ostream::RED << "error: " << raw_ostream::RESET;

2. Add a boolean flag to raw_ostream so that you can disable colored
output. If you disable colors, changeColor, operator<<(Color),
resetColor and other color-related functions have no effect.

Most LLVM tools automatically prints out messages using colors, and
you can disable it by passing a flag such as `--disable-colors`.
This new flag makes it easy to write code that works that way.

Differential Revision: https://reviews.llvm.org/D65564

llvm-svn: 367649
2019-08-02 04:48:30 +00:00
Guanzhong Chen 87186b2447 [WebAssembly] Set __tls_align to 1 when there is no TLS
Summary:
We want the tool conventions to state that `__tls_align` will be a power of 2.
It makes sense to not have an exception for when there is no TLS.

Reviewers: tlively, sunfish

Reviewed By: tlively

Subscribers: dschuff, sbc100, jgravelle-google, aheejin, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D65177

llvm-svn: 366948
2019-07-24 21:48:14 +00:00
Guanzhong Chen 5204f7611f [WebAssembly] Compute and export TLS block alignment
Summary:
Add immutable WASM global `__tls_align` which stores the alignment
requirements of the TLS segment.

Add `__builtin_wasm_tls_align()` intrinsic to get this alignment in Clang.

The expected usage has now changed to:

    __wasm_init_tls(memalign(__builtin_wasm_tls_align(),
                             __builtin_wasm_tls_size()));

Reviewers: tlively, aheejin, sbc100, sunfish, alexcrichton

Reviewed By: tlively

Subscribers: dschuff, jgravelle-google, hiraditya, cfe-commits, llvm-commits

Tags: #clang, #llvm

Differential Revision: https://reviews.llvm.org/D65028

llvm-svn: 366624
2019-07-19 23:34:16 +00:00
Thomas Lively 6a6f28f7b7 [WebAssembly] Use passive segments by default when memory is shared
Summary:
This change makes it so that passing --shared-memory is all a user
needs to do to get proper multithreaded code. This default can still
be explicitly overridden for any reason using --passive-segments and
--active-segments.

Reviewers: sbc100, quantum

Subscribers: dschuff, jgravelle-google, aheejin, sunfish, jfb, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64950

llvm-svn: 366504
2019-07-18 21:50:24 +00:00
Guanzhong Chen 42bba4b852 [WebAssembly] Implement thread-local storage (local-exec model)
Summary:
Thread local variables are placed inside a `.tdata` segment. Their symbols are
offsets from the start of the segment. The address of a thread local variable
is computed as `__tls_base` + the offset from the start of the segment.

`.tdata` segment is a passive segment and `memory.init` is used once per thread
to initialize the thread local storage.

`__tls_base` is a wasm global. Since each thread has its own wasm instance,
it is effectively thread local. Currently, `__tls_base` must be initialized
at thread startup, and so cannot be used with dynamic libraries.

`__tls_base` is to be initialized with a new linker-synthesized function,
`__wasm_init_tls`, which takes as an argument a block of memory to use as the
storage for thread locals. It then initializes the block of memory and sets
`__tls_base`. As `__wasm_init_tls` will handle the memory initialization,
the memory does not have to be zeroed.

To help allocating memory for thread-local storage, a new compiler intrinsic
is introduced: `__builtin_wasm_tls_size()`. This instrinsic function returns
the size of the thread-local storage for the current function.

The expected usage is to run something like the following upon thread startup:

    __wasm_init_tls(malloc(__builtin_wasm_tls_size()));

Reviewers: tlively, aheejin, kripken, sbc100

Subscribers: dschuff, jgravelle-google, hiraditya, sunfish, jfb, cfe-commits, llvm-commits

Tags: #clang, #llvm

Differential Revision: https://reviews.llvm.org/D64537

llvm-svn: 366272
2019-07-16 22:00:45 +00:00
Sam Clegg fd11ce32bb [WebAssembly] Import __stack_pointer when building -pie binaries
The -pie binary doesn't know that layout ahead of time so needs to
import the stack pointer from the embedder, just like we do already
for shared libraries.

This change is needed in order to address:
https://github.com/emscripten-core/emscripten/issues/8915

Differential Revision: https://reviews.llvm.org/D64516

llvm-svn: 365771
2019-07-11 13:13:25 +00:00
Rui Ueyama 136d27ab4d [Coding style change][lld] Rename variables for non-ELF ports
This patch does the same thing as r365595 to other subdirectories,
which completes the naming style change for the entire lld directory.

With this, the naming style conversion is complete for lld.

Differential Revision: https://reviews.llvm.org/D64473

llvm-svn: 365730
2019-07-11 05:40:30 +00:00
Nico Weber a780276301 lld, llvm-dlltool, llvm-lib: Use getAsString() instead of getSpelling() for printing unknown args
Since OPT_UNKNOWN args never have any values and consist only of
spelling (and are never aliased), this doesn't make any difference in
practice, but it's more consistent with Arg's guidance to use
getAsString() for diagnostics, and it matches what clang does.

Also tweak two tests to use an unknown option that contains '=' for
additional coverage while here. (The new tests pass fine with the old
code too though.)

llvm-svn: 365200
2019-07-05 12:31:32 +00:00
Nico Weber cf1a11ded2 Make joined instances of JoinedOrSeparate flags point to the unaliased args, like all other arg types do
This fixes an 8-year-old regression. r105763 made it so that aliases
always refer to the unaliased option – but it missed the "joined" branch
of JoinedOrSeparate flags. (r162231 then made the Args classes
non-virtual, and r169344 moved them from clang to llvm.)

Back then, there was no JoinedOrSeparate flag that was an alias, so it
wasn't observable. Now /U in CLCompatOptions is a JoinedOrSeparate alias
in clang, and warn_slash_u_filename incorrectly used the aliased arg id
(using the unaliased one isn't really a regression since that warning
checks if the undefined macro contains slash or backslash and only then
emits the warning – and no valid use will pass "-Ufoo/bar" or similar).

Also, lld has many JoinedOrSeparate aliases, and due to this bug it had
to explicitly call `getUnaliasedOption()` in a bunch of places, even
though that shouldn't be necessary by design. After this fix in Option,
these calls really don't have an effect any more, so remove them.

No intended behavior change.

(I accidentally fixed this bug while working on PR29106 but then
wondered why the warn_slash_u_filename broke. When I figured it out, I
thought it would make sense to land this in a separate commit.)

Differential Revision: https://reviews.llvm.org/D64156

llvm-svn: 365186
2019-07-05 11:45:24 +00:00
Thomas Lively 6004d9a13d [WebAssembly] Add option to emit passive segments
Summary:
Adds `--passive-segments` and `--active-segments` flags to control
what kind of segments are emitted. For now the default is always
to emit active segments so this is not a breaking change, but in
the future the default will be changed to passive segments when
shared memory is requested and active segments otherwise. When
passive segments are emitted, corresponding memory.init and
data.drop instructions are emitted in a `__wasm_init_memory`
function that is automatically called at the beginning of
`__wasm_call_ctors`.

Reviewers: sbc100, aheejin, dschuff

Subscribers: azakai, dschuff, jgravelle-google, sunfish, jfb, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59343

llvm-svn: 365088
2019-07-03 22:04:54 +00:00
Guanzhong Chen e15dc95466 [wasm-ld] Add __global_base symbol to mark the value of --global-base
Summary:
This is needed for address sanitizer on Emscripten. As everything in
memory starts at the value passed to --global-base, everything before
that can be used as shadow memory.

This symbol is added so that the library for the ASan runtime can know
where the shadow memory ends and real memory begins.

This is split from D63742.

Reviewers: tlively, aheejin, sbc100

Subscribers: sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D63833

llvm-svn: 364467
2019-06-26 20:12:33 +00:00
Sam Clegg 61d70e4a93 [WebAssembly] Error on archives without a symbol index
This is fairly common with wasm since GNU ar (most likely the system ar)
doesn't support the wasm object format so user who don't override AR
will end up with archives without an index.  We don't want to silently
ignore this issue.

In the future we could choose to instead behave like the ELF backend and
read the symbols from each object file in the archive if they are all of
the same type.  However, error'ing out seem like a conservative approach
for now.

Fixes: PR42376

Differential Revision: https://reviews.llvm.org/D63739

llvm-svn: 364338
2019-06-25 17:49:35 +00:00
Sam Clegg 7d4ec5af6c [WebAssembly] Don't export __data_end and __heap_base by default.
These can still be exported via --export if needed.

Differential Revision: https://reviews.llvm.org/D62744

llvm-svn: 362276
2019-05-31 22:51:59 +00:00
Sam Clegg 59f959ff33 [WebAssembly] Relax signature checking for undefined functions that are not called directly
When function signatures don't match and the undefined function is not
called directly (i.e. only has its address taken) we don't issue a
warning or create a runtime thunk for the undefined function.

Instead in this case we simply use the defined version of the function.
This is possible since checking signatures of dynamic calls happens
at runtime so any invalid usage will still result in a runtime error.

This is needed to allow C++ programs to link without generating
warnings.  Its not uncommon in C++ for vtables to be populated by
function address whee the signature of the function is not known in the
compilation unit.  In this case clang declares the method as void(void)
and relies on the vtable caller casting the data back to the correct
signature.

Fixes: https://bugs.llvm.org/show_bug.cgi?id=40412

Differential Revision: https://reviews.llvm.org/D62153

llvm-svn: 361678
2019-05-24 22:45:08 +00:00
Sam Clegg a5ca34e6b3 [WebAssebmly] Add support for --wrap
The code for implementing this features is taken almost verbatim
from the ELF backend.

Fixes: https://bugs.llvm.org/show_bug.cgi?id=41681

Differential Revision: https://reviews.llvm.org/D62380

llvm-svn: 361639
2019-05-24 14:14:25 +00:00
Sam Clegg 35be7ff80c [WebAssembly] Add support for -emit-relocs
This can be useful for post-link tools and for testing.  Sometimes
it can be useful to produces a regular executable but with relocations
preserved.

Differential Revision: https://reviews.llvm.org/D62378

llvm-svn: 361635
2019-05-24 13:28:27 +00:00
Sam Clegg 4bce63a0e7 Reland: [WebAssembly] Add __start_/_stop_ symbols for data sections
This is a reland of rL361235.

Fixes https://bugs.llvm.org/show_bug.cgi?id=41565

Differential Revision: https://reviews.llvm.org/D61876

llvm-svn: 361476
2019-05-23 10:06:03 +00:00
Sam Clegg c0a4f45bcb Revert "[WebAssembly] Add __start_/_stop_ symbols for data sections"
This reverts commit 7804dbddcc.

This change broke a bunch of tests of the WebAssembly waterfall.
Will hopefully reland with increased test coverage.

llvm-svn: 361273
2019-05-21 17:16:33 +00:00
Rui Ueyama 35150bb534 [WebAssembly] Add --reproduce.
--reproduce is a convenient option for debugging. If you invoke lld
with `--reproduce=repro.tar`, it creates `repro.tar` with all input
files and the command line options given to the linker, so that it is
very easy to run lld with the exact same inputs.

ELF and Windows lld have this option.

This patch add that option to lld/wasm.

Differential Revision: https://reviews.llvm.org/D62170

llvm-svn: 361244
2019-05-21 11:52:14 +00:00
Sam Clegg 7804dbddcc [WebAssembly] Add __start_/_stop_ symbols for data sections
Fixes https://bugs.llvm.org/show_bug.cgi?id=41565

Differential Revision: https://reviews.llvm.org/D61876

llvm-svn: 361236
2019-05-21 10:07:30 +00:00
Sam Clegg a1282a39ba [WebAssembly] Handle command line options consistently with the ELF backend.
Differential Revision: https://reviews.llvm.org/D61645

llvm-svn: 360266
2019-05-08 16:20:05 +00:00
Sam Clegg 0cfaa2470f [WebAssembly] Ensure ArchiveName is set even in the presence of --whole-archive.
Differential Revision: https://reviews.llvm.org/D60431

llvm-svn: 357966
2019-04-09 05:41:52 +00:00
Sam Clegg 09137be7f8 [WebAssembly] Apply data relocations at runtime in shared objects
See: https://github.com/WebAssembly/tool-conventions/blob/master/DynamicLinking.md

Data section relocations in wasm shared libraries are applied by the
library itself at static constructor time.  This change adds a new
synthetic function that applies relocations to relevant memory locations
on startup.

Differential Revision: https://reviews.llvm.org/D59278

llvm-svn: 357715
2019-04-04 18:40:51 +00:00
Sam Clegg 492f752969 [WebAssembly] Initial implementation of PIC code generation
This change implements lowering of references global symbols in PIC
mode.

This change implements lowering of global references in PIC mode using a
new @GOT reference type. @GOT references can be used with function or
data symbol names combined with the get_global instruction. In this case
the linker will insert the wasm global that stores the address of the
symbol (either in memory for data symbols or in the wasm table for
function symbols).

For now I'm continuing to use the R_WASM_GLOBAL_INDEX_LEB relocation
type for this type of reference which means that this relocation type
can refer to either a global or a function or data symbol. We could
choose to introduce specific relocation types for GOT entries in the
future.  See the current dynamic linking proposal:

https://github.com/WebAssembly/tool-conventions/blob/master/DynamicLinking.md

Differential Revision: https://reviews.llvm.org/D54647

llvm-svn: 357022
2019-03-26 19:46:15 +00:00
Thomas Lively 82de51a3ae Reland "[WebAssembly] Add linker options to control feature checking"
Do not pipe binary data between processes in lit tests this time,
since it turns out that can break on Windows.

This reverts commit 84c8652fc3.

llvm-svn: 356975
2019-03-26 04:11:05 +00:00
Thomas Lively 84c8652fc3 Revert "[WebAssembly] Add linker options to control feature checking"
This reverts commit 5991328c96.

llvm-svn: 356932
2019-03-25 18:10:26 +00:00
Thomas Lively 5991328c96 [WebAssembly] Add linker options to control feature checking
Summary:
Adds --check-features and --no-check-features. The default for now is
to enable the checking, but this might change in the future.

Also adds --features=foo,bar for precisely controlling the features
used in the output binary.

Depends on D59173.

Reviewers: sbc100, aheejin

Subscribers: dschuff, jgravelle-google, sunfish, jdoerfert, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59274

llvm-svn: 356805
2019-03-22 20:43:06 +00:00
Sam Clegg d425d6b08b [WebAssembly] Handle undefined data symbols in shared libraries
When linking shared libraries, we import a mutable wasm global
to represent the address of each undefined data symbol.

This is a step towards supporting dynamic linking and shared
libraries.

Differential Revision: https://reviews.llvm.org/D59270

llvm-svn: 355988
2019-03-12 21:53:23 +00:00
Sam Clegg 6540e57002 [WebAssembly] Don't generate invalid modules when function signatures mismatch
Previously we could emit a warning and generate a potentially invalid
wasm module (due to call sites and functions having conflicting
signatures). Now, rather than create invalid binaries we handle such
cases by creating stub functions containing unreachable, effectively
turning these into runtime errors rather than validation failures.

Differential Revision: https://reviews.llvm.org/D57909

llvm-svn: 354528
2019-02-20 23:19:31 +00:00
Sam Clegg 230dc11d24 [WebAssembly] Refactor handling of weak undefined functions. NFC.
Also add to the docs.

This is refactor in preparation for https://reviews.llvm.org/D57909

Differential Revision: https://reviews.llvm.org/D57920

llvm-svn: 353478
2019-02-07 22:42:16 +00:00
Dan Gohman 9b84eeaa3e [WebAssembly] Fix imported function symbol names that differ from their import names in the .o format
Add a flag to allow symbols to have a wasm import name which differs from the
linker symbol name, allowing the linker to link code using the import_module
attribute.

Differential Revision: https://reviews.llvm.org/D57632

llvm-svn: 353473
2019-02-07 22:00:48 +00:00
Sam Clegg 1f3f774f10 [WebAssembly] Implement --trace and --trace-symbol
Differential Revision: https://reviews.llvm.org/D57725

llvm-svn: 353264
2019-02-06 02:35:18 +00:00
Sam Clegg e320cea5b9 [WebAssembly] Fix undefined weak function symbols in LTO builds
Summary: Fixes PR40219

Subscribers: dschuff, mehdi_amini, inglorion, jgravelle-google, aheejin, sunfish, steven_wu, dexonsmith, llvm-commits

Differential Revision: https://reviews.llvm.org/D57420

llvm-svn: 352575
2019-01-30 00:25:52 +00:00
Sam Clegg 37b4ee523b [WebAssembly] Don't load weak undefined symbols from archive files
Summary: Fixes PR40494

Subscribers: dschuff, jgravelle-google, aheejin, sunfish, llvm-commits

Differential Revision: https://reviews.llvm.org/D57370

llvm-svn: 352554
2019-01-29 22:26:31 +00:00
Chandler Carruth 2946cd7010 Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636
2019-01-19 08:50:56 +00:00
Heejin Ahn e915a71f18 [WebAssembly] Add support for the event section
Summary:
This adds support for the 'event section' specified in the exception
handling proposal.

Wasm exception handling binary model spec:
https://github.com/WebAssembly/exception-handling/blob/master/proposals/Exceptions.md#changes-to-the-binary-model

Reviewers: sbc100, ruiu

Subscribers: dschuff, jgravelle-google, sunfish, llvm-commits

Differential Revision: https://reviews.llvm.org/D54875

llvm-svn: 348703
2018-12-08 06:17:43 +00:00
Sam Clegg 0f90191faa [WebAssembly] Allow undefined symbols when building shared libraries
Differential Revision: https://reviews.llvm.org/D55043

llvm-svn: 347909
2018-11-29 20:07:13 +00:00
Sam Clegg 45218f4af9 [WebAssembly] Remove `using` statements from header files. NFC.
Differential Revision: https://reviews.llvm.org/D54758

llvm-svn: 347621
2018-11-27 01:08:16 +00:00
Sam Clegg 2dad4e2c6d [WebAssembly] Import the stack pointer when building shared libraries
Differential Revision: https://reviews.llvm.org/D54558

llvm-svn: 346974
2018-11-15 18:15:54 +00:00
Sam Clegg d5f650631c [WebAssembly] Refactor config setting and checking. NFC.
This matches the way the ELF backend does it.

Differential Revision: https://reviews.llvm.org/D54559

llvm-svn: 346972
2018-11-15 18:09:41 +00:00
Sam Clegg bfb75348e2 [WebAssembly] Initial support for shared objects (-shared)
Based on the initial spec proposal:
https://github.com/WebAssembly/tool-conventions/blob/master/DynamicLinking.md

The llvm/codegen side of this is still missing but I believe this change is
still worth landing as an incremental step

Differential Revision: https://reviews.llvm.org/D54249

llvm-svn: 346918
2018-11-15 00:37:21 +00:00
Sam Clegg 37125f080f [WebAssembly] Respect `--no-mangle` in more locations
`--no-demangle` now also applies to the name section.  This change
was motivated by the rust team that have a slightly different name
mangling scheme to the standard C++ itanium one and prefer to do their
de-mangling as a post-link setp.

Patch by Alex Crichton!

Differential Revision: https://reviews.llvm.org/D54279

llvm-svn: 346516
2018-11-09 16:57:41 +00:00
Derek Schuff 3bea8bcae5 [WebAssembly] Support creation and import of shared memories
Used for WebAssembly threads proposal. Add a flag --shared-memory
which sets the IS_SHARED bit in WasmLimits

Differential Revision: https://reviews.llvm.org/D54130

llvm-svn: 346248
2018-11-06 17:59:32 +00:00
Rui Ueyama 9ae88c64fa Eliminate dependency to formatv(). NFC.
llvm-svn: 344177
2018-10-10 20:29:29 +00:00
Fangrui Song 2043a58abe Adapt OptTable::PrintHelp change in D51009
Summary: Before, OptTable::PrintHelp append "[options] <inputs>" to its parameter `Help`. It is more flexible to change its semantic to `Usage` and let user customize the usage line.

Reviewers: rupprecht, ruiu, espindola

Reviewed By: rupprecht

Subscribers: emaste, sbc100, arichardson, aheejin, llvm-commits

Differential Revision: https://reviews.llvm.org/D53054

llvm-svn: 344099
2018-10-10 00:15:36 +00:00
Derek Schuff 371842b853 [WebAssembly] Refactor use of signatures
Update use of WebAssemblySignature to go along with D52580

Differential Revision: https://reviews.llvm.org/D52622

llvm-svn: 343734
2018-10-03 22:25:32 +00:00
Sam Clegg 305b0343ce [WebAssembly] Add --[no]-export-dynamic to replace --export-default
In a very recent change I introduced a --no-export-default flag
but after conferring with others it seems that this feature already
exists in gnu GNU ld and lld in the form the --export-dynamic flag
which is off by default.

This change replaces export-default with export-dynamic and also
changes the default to match the traditional linker behaviour.

Now, by default, only the entry point is exported.  If other symbols
are required by the embedder then --export-dynamic or --export can
be used to export all visibility hidden symbols or individual
symbols respectively.

This change touches a lot of tests that were relying on symbols
being exported by default.  I imagine it will also effect many
users but do think the change is worth it match of the traditional
behaviour and flag names.

Differential Revision: https://reviews.llvm.org/D52587

llvm-svn: 343265
2018-09-27 21:06:25 +00:00
Sam Clegg 4aad12ce0b [WebAssembly] Update Config member to match command line option
Subscribers: dschuff, jgravelle-google, aheejin, sunfish, llvm-commits

Differential Revision: https://reviews.llvm.org/D50976

llvm-svn: 343157
2018-09-27 00:46:54 +00:00
Sam Clegg 4e62418b25 [WebAssembly] Add --export-default/--no-export-default options
These option control weather or not symbols marked as visibility
default are exported in the output binary.

By default this is true, but emscripten prefers to control the
exported symbol list explicitly at link time and ignore the
symbol attributes.

Differential Revision: https://reviews.llvm.org/D52003

llvm-svn: 343034
2018-09-25 21:50:15 +00:00
Heejin Ahn 4821ebf73f [WebAssembly] clang-format (NFC)
Summary: This patch runs clang-format on all wasm-only files.

Reviewers: sbc100

Subscribers: dschuff, jgravelle-google, sunfish, llvm-commits

Differential Revision: https://reviews.llvm.org/D51449

llvm-svn: 340970
2018-08-29 21:03:16 +00:00
Rui Ueyama 41831204c7 Rename a function to follow the LLVM coding style.
llvm-svn: 340716
2018-08-27 06:18:10 +00:00
Nico Weber 386bf1216e win: Omit ".exe" from lld warning and error messages.
This is a minor follow-up to https://reviews.llvm.org/D49189. On Windows, lld
used to print "lld-link.exe: error: ...". Now it just prints "lld-link: error:
...". This matches what link.exe does (it prints "LINK : ...") and makes lld's
output less dependent on the host system.

https://reviews.llvm.org/D51133

llvm-svn: 340487
2018-08-22 23:52:13 +00:00
Sam Clegg 30161dc28d [WebAssembly] Don't compress LEBs by default
LEB compression breaks debug info so we don't want to enable
it by default, even at high optimization levels.

Differential Revision: https://reviews.llvm.org/D50729

llvm-svn: 340073
2018-08-17 19:42:46 +00:00
Sam Clegg cb06300ad2 [WebAssembly] --export should fetch lazy symbols
--export now implies --undefined

This is really a requirement from emscripten but I think it
makes sense in general too.

Differential Revision: https://reviews.llvm.org/D50287

llvm-svn: 339047
2018-08-06 19:45:12 +00:00
Sam Clegg 47e2b6b29e [WebAssembly] Don't error when --undefined symbols are not found
This matches the behavior of the ELF linker where -u/--undefined
means symbols will get pulled in from archives but won't result
in link error if they are missing.

Also, don't actually great symbol table entries for the undefined
symbols, again matching more closely the ELF linker.

This also results in simplification of the code.

Differential Revision: https://reviews.llvm.org/D50279

llvm-svn: 338938
2018-08-04 00:04:06 +00:00
Sam Clegg 8adf7ac5c1 [WebAssembly] Add support for --whole-archive.
Subscribers: dschuff, jgravelle-google, aheejin, sunfish, llvm-commits

Differential Revision: https://reviews.llvm.org/D49706

llvm-svn: 337777
2018-07-23 23:51:19 +00:00
Nico Weber bbfe0b79e2 Omit path to lld binary from lld's error, warning, and log output.
lld currently prepends the absolute path to itself to every diagnostic it
emits. This path can be longer than the diagnostic, and makes the actual error
message hard to read.

There isn't a good reason for printing this path: if you want to know which lld
you're running, pass -v to clang – chances are that if you're unsure of this,
you're not only unsure when it errors out. Some people want an indication that
the diagnostic is from the linker though, so instead print just the basename of
the linker's path.

Before:

```
$ out/bin/clang -target x86_64-unknown-linux -x c++ /dev/null -fuse-ld=lld 
/Users/thakis/src/llvm-mono/out/bin/ld.lld: error: cannot open crt1.o: No such file or directory
/Users/thakis/src/llvm-mono/out/bin/ld.lld: error: cannot open crti.o: No such file or directory
/Users/thakis/src/llvm-mono/out/bin/ld.lld: error: cannot open crtbegin.o: No such file or directory
/Users/thakis/src/llvm-mono/out/bin/ld.lld: error: unable to find library -lgcc
/Users/thakis/src/llvm-mono/out/bin/ld.lld: error: unable to find library -lgcc_s
/Users/thakis/src/llvm-mono/out/bin/ld.lld: error: unable to find library -lc
/Users/thakis/src/llvm-mono/out/bin/ld.lld: error: unable to find library -lgcc
/Users/thakis/src/llvm-mono/out/bin/ld.lld: error: unable to find library -lgcc_s
/Users/thakis/src/llvm-mono/out/bin/ld.lld: error: cannot open crtend.o: No such file or directory
/Users/thakis/src/llvm-mono/out/bin/ld.lld: error: cannot open crtn.o: No such file or directory
clang: error: linker command failed with exit code 1 (use -v to see invocation)
```

After:

```
$ out/bin/clang -target x86_64-unknown-linux -x c++ /dev/null -fuse-ld=lld 
ld.lld: error: cannot open crt1.o: No such file or directory
ld.lld: error: cannot open crti.o: No such file or directory
ld.lld: error: cannot open crtbegin.o: No such file or directory
ld.lld: error: unable to find library -lgcc
ld.lld: error: unable to find library -lgcc_s
ld.lld: error: unable to find library -lc
ld.lld: error: unable to find library -lgcc
ld.lld: error: unable to find library -lgcc_s
ld.lld: error: cannot open crtend.o: No such file or directory
ld.lld: error: cannot open crtn.o: No such file or directory
clang: error: linker command failed with exit code 1 (use -v to see invocation)
```

https://reviews.llvm.org/D49189

llvm-svn: 337634
2018-07-20 23:09:12 +00:00
Sam Clegg 52b2456a95 [WebAssembly] Fix --export of LTO symbols
Differential Revision: https://reviews.llvm.org/D48697

llvm-svn: 335881
2018-06-28 17:21:46 +00:00
Sam Clegg cefbf9aca1 [WebAssembly] LTO: Fix signatures of undefined functions in bitcode
Function symbols that come from bitcode have not signatures.
After LTO when the real symbols are read in we need to make
sure that we set the signature on the existing symbol.
the signature-less undefined functions.

Differential Revision: https://reviews.llvm.org/D48693

llvm-svn: 335875
2018-06-28 16:53:53 +00:00
Sam Clegg ffd0aaf810 Revert "[WebAssembly] Error on mismatched function signature in final output"
This caused a lot of issues on the WebAssembly waterfall.
In particular, until with the signature of `main`.  We
probably want a better solution for main before we re-land.

Reverts rL335192

llvm-svn: 335355
2018-06-22 15:13:10 +00:00
Sam Clegg 1369dfa30b [WebAssembly] Error on mismatched function signature in final output
During symbol resolution, emit warnings for function signature
mismatches.  During GC, if any mismatched symbol is marked as live
then generate an error.

This means that we only error out if the mismatch is written to the
final output.  i.e. if we would generate an invalid wasm file.

Differential Revision: https://reviews.llvm.org/D48394

llvm-svn: 335192
2018-06-21 00:12:25 +00:00
Sam Clegg 177b458c8a [WebAssembly] Add --export-all flag
This causes all symbols to be exported in the final wasm binary
even if they were not compiled with default visibility.

This feature is useful for the emscripten toolchain that has a
corresponding EXPORT_ALL feature which allows the JS code to
interact with all C function.

Differential Revision: https://reviews.llvm.org/D47806

llvm-svn: 334157
2018-06-07 01:27:07 +00:00
Sam Clegg c729c1b47d [WebAssembly] Initial support for LTO
Differential Revision: https://reviews.llvm.org/D47162

llvm-svn: 333570
2018-05-30 18:07:52 +00:00
Sam Clegg e9ce661e32 [WebAssembly] Add support for response file parsing
Differential Revision: https://reviews.llvm.org/D47500

llvm-svn: 333499
2018-05-30 03:51:56 +00:00
Sam Clegg fb983cda8a [WebAssembly] Add option to remove LEB padding at relocate sites
This change adds the ability for lld to remove LEB padding from
code section. This effectively shrinks the size of the resulting
binary in proportion to the number of code relocations.

Since there will be a performance cost this is currently only active for
-O1 and above. Some toolchains may instead want to perform this
compression as a post linker step (for example running a binary through
binaryen will automatically compress these values).

I imagine we might want to make this the default in the future.

Differential Revision: https://reviews.llvm.org/D46416

llvm-svn: 332783
2018-05-18 23:28:05 +00:00
Benjamin Kramer 5455038d98 [lld] Make helpers static. NFC.
llvm-svn: 332408
2018-05-15 22:01:54 +00:00
Sam Clegg 65d6380cd6 [WebAssembly] Allow signautre of entry function to be flexible
Since we a no longer using this function for the wasm start
section we don't actually care what its signature is.

Differential Revision: https://reviews.llvm.org/D46594

llvm-svn: 332308
2018-05-14 23:01:16 +00:00
Sam Clegg 668447677a [WebAssembly] Add a flag to control merging data segments
Merging data segments produces smaller code sizes because each segment
has some boilerplate. Therefore, merging data segments is generally the
right approach, especially with wasm where binaries are typically
delivered over the network.

However, when analyzing wasm binaries, it can be helpful to get a
conservative picture of which functions are using which data
segments[0]. Perhaps there is a large data segment that you didn't
expect to be included in the wasm, introduced by some library you're
using, and you'd like to know which library it was. In this scenario,
merging data segments only makes the analysis worse.

Alternatively, perhaps you will remove some dead functions by-hand[1]
that can't be statically proven dead by the compiler or lld, and
removing these functions might make some data garbage collect-able, and
you'd like to run `--gc-sections` again so that this now-unused data can
be collected. If the segments were originally merged, then a single use
of the merged data segment will entrench all of the data.

[0] https://github.com/rustwasm/twiggy
[1] https://github.com/fitzgen/wasm-snip

Patch by Nick Fitzgerald!

Differential Revision: https://reviews.llvm.org/D46417

llvm-svn: 332013
2018-05-10 18:23:51 +00:00
Nico Weber cac2b3349e lld-link: Add --color-diagnostics(={always,never,auto})?, --no-color-diagnostics flags.
This is most useful when using lld-link on a non-Win host (but it might become
useful on Windows too if lld also grows a fansi-escape-codes flag).

Also make the help for --color-diagnostic mention the valid values in ELF and
wasm, and print the flag name with two dashes in diags, since the one-dash form
is seen as a list of many one-letter flags in some contexts.

https://reviews.llvm.org/D46693

llvm-svn: 332012
2018-05-10 18:19:02 +00:00
Sam Clegg 752494bfe3 [WebAssembly] Check function signatures by default
But only produce a warning (for now) unless --fatal-warnings
is passed.

Differential Revision: https://reviews.llvm.org/D46484

llvm-svn: 331574
2018-05-05 01:23:07 +00:00
Sam Clegg a0f095ebd7 [WebAssembly] Add --stack-first option which places the shadow stack at start of linear memory
Fixes https://bugs.llvm.org/show_bug.cgi?id=37181

Differential Revision: https://reviews.llvm.org/D46141

llvm-svn: 331467
2018-05-03 17:21:53 +00:00
Nicholas Wilson 358af38d37 [WebAssembly] Implement -print-gc-sections, to better test GC of globals
Differential Revision: https://reviews.llvm.org/D44311

llvm-svn: 330456
2018-04-20 17:28:12 +00:00
Nicholas Wilson 6c7fe30a1c [WebAssembly] Implement --print-gc-sections for synthetic functions
Enables cleaning up confusion between which name variables are mangled
and which are unmangled, and --print-gc-sections then excersises and
tests that.

Differential Revision: https://reviews.llvm.org/D44440

llvm-svn: 330449
2018-04-20 17:09:18 +00:00
Rui Ueyama 96ba8befa9 [WebAssembly] Remove StackPointerGlobal member variable from the driver.
Since InputGlobal makes a copy of a given object, we can use a temporary
object allocated on the stack here.

Differential Revision: https://reviews.llvm.org/D43924

llvm-svn: 329337
2018-04-05 19:37:48 +00:00