Commit Graph

12127 Commits

Author SHA1 Message Date
Martin Storsjo 6863dfaa21 [MinGW] Implement the -v and --version flags for the MinGW driver
Differential Revision: https://reviews.llvm.org/D62030

llvm-svn: 361016
2019-05-17 11:07:42 +00:00
Martin Storsjo 9ae0d222dc [MinGW] Implement --help for the MinGW driver
Differential Revision: https://reviews.llvm.org/D62029

llvm-svn: 361015
2019-05-17 11:07:38 +00:00
Martin Storsjo 2c52ddf31f [MinGW] Allow requesting PDB output without giving a file name
When integrating PDB output in mingw targeting build systems, it
might be a lot of extra work to specify unique file names for
the pdb output. Therefore allow omitting the actual file name
and let it implicitly be the same name as the linker output, with
a pdb extension.

As the current form of the pdb option takes a separate parameter value,
e.g. "-pdb out.pdb", it is impractical to leave out the parameter value.

Therefore, introduce a second syntax for the option, with an equals
sign, like -pdb=out.pdb, where the value easily can be omitted.

The form -pdb= for requesting pdb files with an implicit name should
work fine, even though it looks a bit unconventional in that form.

Differential Revision: https://reviews.llvm.org/D62004

llvm-svn: 361014
2019-05-17 11:07:33 +00:00
Fangrui Song ef9b8e03fd [ELF][test] Reorganize some R_*_NONE tests
Change R_{386,AARCH64}_NONE yaml2obj tests/icf10.test to use assembly

Add relocation-none-{arm,x86_64}.s.
  Check the referenced section survives under --gc-sections.
  Check -r copies R_X86_64_NONE R_AARCH64_NONE. (Elf*_Rel arches currently have a bug)
  Delete the dtrace tests as they are covered by the R_X86_64_NONE test.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D62051

llvm-svn: 361013
2019-05-17 10:44:05 +00:00
Ben Dunbobbin 1d16515fb4 [ELF] Implement Dependent Libraries Feature
This patch implements a limited form of autolinking primarily designed to allow
either the --dependent-library compiler option, or "comment lib" pragmas (
https://docs.microsoft.com/en-us/cpp/preprocessor/comment-c-cpp?view=vs-2017) in
C/C++ e.g. #pragma comment(lib, "foo"), to cause an ELF linker to automatically
add the specified library to the link when processing the input file generated
by the compiler.

Currently this extension is unique to LLVM and LLD. However, care has been taken
to design this feature so that it could be supported by other ELF linkers.

The design goals were to provide:

- A simple linking model for developers to reason about.
- The ability to to override autolinking from the linker command line.
- Source code compatibility, where possible, with "comment lib" pragmas in other
  environments (MSVC in particular).

Dependent library support is implemented differently for ELF platforms than on
the other platforms. Primarily this difference is that on ELF we pass the
dependent library specifiers directly to the linker without manipulating them.
This is in contrast to other platforms where they are mapped to a specific
linker option by the compiler. This difference is a result of the greater
variety of ELF linkers and the fact that ELF linkers tend to handle libraries in
a more complicated fashion than on other platforms. This forces us to defer
handling the specifiers to the linker.

In order to achieve a level of source code compatibility with other platforms
we have restricted this feature to work with libraries that meet the following
"reasonable" requirements:

1. There are no competing defined symbols in a given set of libraries, or
   if they exist, the program owner doesn't care which is linked to their
   program.
2. There may be circular dependencies between libraries.

The binary representation is a mergeable string section (SHF_MERGE,
SHF_STRINGS), called .deplibs, with custom type SHT_LLVM_DEPENDENT_LIBRARIES
(0x6fff4c04). The compiler forms this section by concatenating the arguments of
the "comment lib" pragmas and --dependent-library options in the order they are
encountered. Partial (-r, -Ur) links are handled by concatenating .deplibs
sections with the normal mergeable string section rules. As an example, #pragma
comment(lib, "foo") would result in:

.section ".deplibs","MS",@llvm_dependent_libraries,1
         .asciz "foo"

For LTO, equivalent information to the contents of a the .deplibs section can be
retrieved by the LLD for bitcode input files.

LLD processes the dependent library specifiers in the following way:

1. Dependent libraries which are found from the specifiers in .deplibs sections
   of relocatable object files are added when the linker decides to include that
   file (which could itself be in a library) in the link. Dependent libraries
   behave as if they were appended to the command line after all other options. As
   a consequence the set of dependent libraries are searched last to resolve
   symbols.
2. It is an error if a file cannot be found for a given specifier.
3. Any command line options in effect at the end of the command line parsing apply
   to the dependent libraries, e.g. --whole-archive.
4. The linker tries to add a library or relocatable object file from each of the
   strings in a .deplibs section by; first, handling the string as if it was
   specified on the command line; second, by looking for the string in each of the
   library search paths in turn; third, by looking for a lib<string>.a or
   lib<string>.so (depending on the current mode of the linker) in each of the
   library search paths.
5. A new command line option --no-dependent-libraries tells LLD to ignore the
   dependent libraries.

Rationale for the above points:

1. Adding the dependent libraries last makes the process simple to understand
   from a developers perspective. All linkers are able to implement this scheme.
2. Error-ing for libraries that are not found seems like better behavior than
   failing the link during symbol resolution.
3. It seems useful for the user to be able to apply command line options which
   will affect all of the dependent libraries. There is a potential problem of
   surprise for developers, who might not realize that these options would apply
   to these "invisible" input files; however, despite the potential for surprise,
   this is easy for developers to reason about and gives developers the control
   that they may require.
4. This algorithm takes into account all of the different ways that ELF linkers
   find input files. The different search methods are tried by the linker in most
   obvious to least obvious order.
5. I considered adding finer grained control over which dependent libraries were
   ignored (e.g. MSVC has /nodefaultlib:<library>); however, I concluded that this
   is not necessary: if finer control is required developers can fall back to using
   the command line directly.

RFC thread: http://lists.llvm.org/pipermail/llvm-dev/2019-March/131004.html.

Differential Revision: https://reviews.llvm.org/D60274

llvm-svn: 360984
2019-05-17 03:44:15 +00:00
Rui Ueyama bbf154cf9c Move symbol resolution code out of SymbolTable class.
This is the last patch of the series of patches to make it possible to
resolve symbols without asking SymbolTable to do so.

The main point of this patch is the introduction of
`elf::resolveSymbol(Symbol *Old, Symbol *New)`. That function resolves
or merges given symbols by examining symbol types and call
replaceSymbol (which memcpy's New to Old) if necessary.

With the new function, we have now separated symbol resolution from
symbol lookup. If you already have a Symbol pointer, you can directly
resolve the symbol without asking SymbolTable to do that.

Now that the nice abstraction become available, I can start working on
performance improvement of the linker. As a starter, I'm thinking of
making --{start,end}-lib faster.

--{start,end}-lib is currently unnecessarily slow because it looks up
the symbol table twice for each symbol.

 - The first hash table lookup/insertion occurs when we instantiate a
   LazyObject file to insert LazyObject symbols.

 - The second hash table lookup/insertion occurs when we create an
   ObjFile from LazyObject file. That overwrites LazyObject symbols
   with Defined symbols.

I think it is not too hard to see how we can now eliminate the second
hash table lookup. We can keep LazyObject symbols in Step 1, and then
call elf::resolveSymbol() to do Step 2.

Differential Revision: https://reviews.llvm.org/D61898

llvm-svn: 360975
2019-05-17 01:55:20 +00:00
Bob Haarman 5ff1eb6418 Revert r358069 "Discard debuginfo for object files empty after GC"
The change broke some scenarios where debug information is still
needed, although MarkLive cannot see it, including the
Chromium/Android build. Reverting to unbreak that build.

llvm-svn: 360955
2019-05-16 23:33:06 +00:00
Sam Clegg d029bf0f8c [WebAssembly] Move code and data section generation to finalizeContent. NFC.
Previously these sections were being generated during their
constructors.  This moves the work to finalizeContent, and also does
the same for the relocation sections because their contents depends
on the final layout too.

This change is part of a larger refactor to how we deal with synthetic
sections: https://reviews.llvm.org/D61811

Differential Revision: https://reviews.llvm.org/D61971

llvm-svn: 360941
2019-05-16 21:36:06 +00:00
Sam Clegg 1a53ff2a13 [WebAssembly] Don't generate empty type sections.
Differential Revision: https://reviews.llvm.org/D61991

llvm-svn: 360940
2019-05-16 21:22:43 +00:00
Yvan Roux 671fc5f3f4 [lld-link] Add missing "REQUIRES: x86" to COFF/lib-module-asm.ll test.
This should fix ARM bots.

llvm-svn: 360875
2019-05-16 11:29:30 +00:00
Igor Kudrin 261a1bc129 Add missing "REQUIRES: x86" to the lto/irmover-warning.ll test.
llvm-svn: 360866
2019-05-16 09:04:52 +00:00
Igor Kudrin 4669cf2750 [LTO] Improve readability of module IDs
Module IDs can appear in diagnostic messages.
This patch adds some auxiliary symbols to improve their readability.

Differential Revision: https://reviews.llvm.org/D61857

llvm-svn: 360858
2019-05-16 05:23:25 +00:00
Rui Ueyama 54ee6df247 Pemove SymbolTable::addBitcode as it is redundant.
Differential Revision: https://reviews.llvm.org/D61897

llvm-svn: 360846
2019-05-16 03:54:50 +00:00
Rui Ueyama d668873bfe Consistently return `Symbol *` from SymbolTable's add-family functions.
llvm-svn: 360845
2019-05-16 03:54:41 +00:00
Rui Ueyama 943cd00580 De-template parseFile() and SymbolTable's add-family functions.
Differential Revision: https://reviews.llvm.org/D61896

llvm-svn: 360844
2019-05-16 03:45:13 +00:00
Rui Ueyama 5c073a94f9 Introduce CommonSymbol.
Previously, we handled common symbols as a kind of Defined symbol,
but what we were doing for common symbols is pretty different from
regular defined symbols.

Common symbol and defined symbol are probably as different as shared
symbol and defined symbols are different.

This patch introduces CommonSymbol to represent common symbols.
After symbols are resolved, they are converted to Defined symbols
residing in a .bss section.

Differential Revision: https://reviews.llvm.org/D61895

llvm-svn: 360841
2019-05-16 03:29:03 +00:00
Rui Ueyama 7d4761928e Simplify SymbolTable::add{Defined,Undefined,...} functions.
SymbolTable's add-family functions have lots of parameters because
when they have to create a new symbol, they forward given arguments
to Symbol's constructors. Therefore, the functions take at least as
many arguments as their corresponding constructors.

This patch simplifies the add-family functions. Now, the functions
take a symbol instead of arguments to construct a symbol. If there's
no existing symbol, a given symbol is memcpy'ed to the symbol table.
Otherwise, the functions attempt to merge the existing and a given
new symbol.

I also eliminated `CanOmitFromDynSym` parameter, so that the functions
take really one argument.

Symbol classes are trivially constructible, so looks like constructing
them to pass to add-family functions is as cheap as passing a lot of
arguments to the functions. A quick benchmark showed that this patch
seems performance-neutral.

This is a preparation for
http://lists.llvm.org/pipermail/llvm-dev/2019-April/131902.html

Differential Revision: https://reviews.llvm.org/D61855

llvm-svn: 360838
2019-05-16 02:14:00 +00:00
Sam Clegg e7ab59eda9 [ELF] Fix typo in header guard. NFC.
llvm-svn: 360832
2019-05-16 00:45:50 +00:00
Sam Clegg 5e6ea24497 [lld] Remove 2>&1 from --trace-symbol tests
The tracing goes to stdout so this is not needed.

Also remove the "not" from the final check in ELF/trace-symbols.s.
According the comment the check is that we don't crash, so we should
be checking for success here. Previously this step is error'ing with
undefined symbols because it didn't include all the needed objects.

Differential Revision: https://reviews.llvm.org/D61928

llvm-svn: 360794
2019-05-15 19:01:51 +00:00
Sam Clegg 697f2149f1 [WebAssembly] LTO: Honor comdat groups when loading bitcode files
But don't apply comdat groups when loading the LTO object files.
This is basically the same logic used by the ELF linker.

Differential Revision: https://reviews.llvm.org/D61924

llvm-svn: 360782
2019-05-15 16:03:28 +00:00
Fangrui Song e041d15f5e [LLD][ELF] Add the -z ifunc-noplt option
Patch by Mark Johnston!

Summary:
When the option is configured, ifunc calls do not go through the PLT;
rather, they appear as regular function calls with relocations
referencing the ifunc symbol, and the resolver is invoked when
applying the relocation.  This is intended for use in freestanding
environments where text relocations are permissible and is incompatible
with the -z text option.  The option is motivated by ifunc usage in the
FreeBSD kernel, where ifuncs are used to elide CPU feature flag bit
checks in hot paths.  Instead of replacing the cost of a branch with that
of an indirect function call, the -z ifunc-noplt option is used to ensure
that ifunc calls carry no hidden overhead relative to normal function
calls.

Test Plan:
I added a couple of regression tests and tested the FreeBSD kernel
build using the latest lld sources.

To demonstrate the effects of the change, I used a micro-benchmark
which results in frequent invocations of a FreeBSD kernel ifunc.  The
benchmark was run with and without IBRS enabled, and with and without
-zifunc-noplt configured.  The observed speedup is small and consistent,
and is significantly larger with IBRS enabled:

https://people.freebsd.org/~markj/ifunc-noplt/noibrs.txt
https://people.freebsd.org/~markj/ifunc-noplt/ibrs.txt

Reviewed By: ruiu, MaskRay

Differential Revision: https://reviews.llvm.org/D61613

llvm-svn: 360685
2019-05-14 15:25:21 +00:00
Fangrui Song ed4dbe6326 [ELF] --gdb-index: fix SIGSEGV when a DWARFAddressRange has invalid SectionIndex
See D61891: llvm had a bug that might create invalid (DW_AT_low_pc,DW_AT_high_pc) pairs or range list entries due to missing DW_AT_addr_base.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D61889

llvm-svn: 360679
2019-05-14 14:41:20 +00:00
Rui Ueyama 2dd5283d2a Move SymbolTable::addFile to InputFiles.cpp.
The symbol table used to be a container of vectors of input files,
but that's no longer the case because the vectors are moved out of
SymbolTable and are now global variables.

Therefore, addFile doesn't have to belong to any class. This patch
moves the function out of the class.

This patch is a preparation for my RFC [1].

[1] http://lists.llvm.org/pipermail/llvm-dev/2019-April/131902.html

Differential Revision: https://reviews.llvm.org/D61854

llvm-svn: 360666
2019-05-14 12:03:13 +00:00
Fangrui Song e1cb2c0f40 [Object] Change ObjectFile::getSectionContents to return Expected<ArrayRef<uint8_t>>
Change
std::error_code getSectionContents(DataRefImpl, StringRef &) const;
to
Expected<ArrayRef<uint8_t>> getSectionContents(DataRefImpl) const;

Many object formats use ArrayRef<uint8_t> as the underlying type, which
is generally better than StringRef to represent binary data, so change
the type to decrease the number of type conversions.

Reviewed By: ruiu, sbc100

Differential Revision: https://reviews.llvm.org/D61781

llvm-svn: 360648
2019-05-14 04:22:51 +00:00
Sam Clegg 0b13ca2f5e [WebAssembly] Refactor relocation processing. NFC.
This is the remaining NFC part of https://reviews.llvm.org/D61539 which
was reverted.

Differential Revision: https://reviews.llvm.org/D61800

llvm-svn: 360598
2019-05-13 16:42:52 +00:00
Peter Smith 4e21c770ec [ELF] Full support for -n (--nmagic) and -N (--omagic) via common page
The -n (--nmagic) disables page alignment, and acts as a -Bstatic
The -N (--omagic) does what -n does but also marks the executable segment as
writeable. As page alignment is disabled headers are not allocated unless
explicit in the linker script.

To disable page alignment in LLD we choose to set the page sizes to 1 so
that any alignment based on the page size does nothing. To set the
Target->PageSize to 1 we implement -z common-page-size, which has the side
effect of allowing the user to set the value as well.

Setting the page alignments to 1 does mean that any use of
CONSTANT(MAXPAGESIZE) or CONSTANT(COMMONPAGESIZE) in a linker script will
return 1, unlike in ld.bfd. However given that -n and -N disable paging
these probably shouldn't be used in a linker script where -n or -N is in
use.

Differential Revision: https://reviews.llvm.org/D61688

llvm-svn: 360593
2019-05-13 16:01:26 +00:00
Reid Kleckner c10f80eb7b [COFF] Update LLD yaml test cases to include .bss size
These yaml test cases appear to have been affected by PR41836

Right now what happens is that these empty .bss sections are merged into
.data, then the .data output section ends up having a zero virtual size,
and it is discarded from the output after addresses are assigned.
However, we've already assigned OutputSections to Chunks, so we don't
correctly report the zero-sized chunks that were in there as having been
discarded. Soon, we will report them as discarded, so these test cases
need to be updated to have a non-zero size so they aren't discarded.

llvm-svn: 360476
2019-05-10 22:12:51 +00:00
Fangrui Song 6150407951 [llvm-objdump] Print st_other
Add support for ".hidden" ".internal" ".protected" and " 0x%02x" for
other st_other bits used by some architectures.

Reviewed By: sfertile

Differential Revision: https://reviews.llvm.org/D61718

llvm-svn: 360439
2019-05-10 16:24:57 +00:00
Fangrui Song 6cdd68e386 [PPC64] Define getThunkSectionSpacing() based on the range of R_PPC64_REL24
Suggested by Sean Fertile and Peter Smith.

Thunk section spacing decrease the total number of thunks. I measured a
decrease of 1% or less in some large programs, with no perceivable
slowdown in link time. Override getThunkSectionSpacing() to enable it.
0x2000000 is the farthest point R_PPC64_REL24 can reach. I tried several
numbers and found 0x2000000 works the best. Numbers near 0x2000000 work
as well but let's just use the simpler number.

As demonstrated by the updated tests, this essentially changes placement
of most thunks to the end of the output section. We leverage this
property to fix PR40740 reported by Alfredo Dal'Ava Júnior:

The output section .init consists of input sections from several object
files (crti.o crtbegin.o crtend.o crtn.o). Sections other than the last
one do not have a terminator. With this patch, we create the thunk after
the last .init input section and thus fix the issue. This is not
foolproof but works quite well for such sections (with no terminator) in
practice.

Reviewed By: ruiu, sfertile

Differential Revision: https://reviews.llvm.org/D61720

llvm-svn: 360405
2019-05-10 05:51:00 +00:00
Sam Clegg ea38ac5ba3 [WebAssembly] Don't assume that strongly defined symbols are DSO-local
The current PIC model for WebAssembly is more like ELF in that it
allows symbol interposition.

This means that more functions end up being addressed via the GOT
and fewer directly added to the wasm table.

One effect is a reduction in the number of wasm table entries similar
to the previous attempt in https://reviews.llvm.org/D61539 which was
reverted.

Differential Revision: https://reviews.llvm.org/D61772

llvm-svn: 360402
2019-05-10 01:52:08 +00:00
Reid Kleckner 4c64256b51 [COFF] Simplify Chunk::writeTo and remove OutputSectionOff, NFC
Summary:
Prior to this change, every implementation of writeTo would add
OutputSectionOff to the output section buffer start before writing data.
Instead, do this math in the caller, so that it can be written once
instead of many times.

The output section offset is always equivalent to the difference between
the chunk RVA and the output section RVA, so we can replace the one
remaining usage of OutputSectionOff with that subtraction.

This doesn't change the size of SectionChunk because of alignment
requirements, but I will rearrange the fields in a follow-up change to
accomplish that.

Reviewers: ruiu, aganea

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D61696

llvm-svn: 360376
2019-05-09 21:21:22 +00:00
Sam Clegg 08fa01a9b7 Revert "[WebAssembly] Don't generate unused table entries."
This reverts commit b33fdb7768.

This change apparently broke am emscripten test.

llvm-svn: 360367
2019-05-09 19:34:32 +00:00
Ben Dunbobbin 3edca1ac1a [LLD][NFC] Refactor: BuildID hash size now computed in one place.
Differential Revision: https://reviews.llvm.org/D61078

llvm-svn: 360316
2019-05-09 08:08:09 +00:00
Fangrui Song bd086817f6 [ELF] Initialize Target before it may be dereferenced by findAux when reporting "duplicate symbol" error
for (InputFile *F : Files)
      Symtab->addFile<ELFT>(F); // if there is a duplicate symbol error

    ...

    Target = getTarget();

When parsing .debug_info in the object file (for better diagnostics),
DWARF.cpp findAux may dereference the null pointer Target

    auto *DR = dyn_cast<Defined>(&File->getRelocTargetSym(Rel));
    if (!DR) {
      // Broken debug info may point to a non-defined symbol,
      // some asan object files may also contain R_X86_64_NONE
      RelType Type = Rel.getType(Config->IsMips64EL);
      if (Type != Target->NoneRel) /// Target is null

Move the assignment of Target to an earlier place to fix this.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D61712

llvm-svn: 360305
2019-05-09 01:45:53 +00:00
Bob Haarman f3fb7fac32 [lld-link] initialize targets and asmparsers before invoking lib
Summary:
When using lld-link to build static libraries containing object files
with module assembly, the program would crash with "Assertion `T &&
T->hasMCAsmParser()' failed". This change causes the code in lld-link
that initialized Targets, TargetInfos, and AsmParsers (which already
existed) to be run before entering the lib building path (which needs
it). This avoids the error (and is what llvm-lib and llvm-ar do, too).

Fixes PR41803.

Reviewers: ruiu, rnk, hans

Reviewed By: ruiu

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D61699

llvm-svn: 360295
2019-05-08 22:11:02 +00:00
Sam Clegg a1282a39ba [WebAssembly] Handle command line options consistently with the ELF backend.
Differential Revision: https://reviews.llvm.org/D61645

llvm-svn: 360266
2019-05-08 16:20:05 +00:00
Reid Kleckner 34e9c41164 [COFF] Store Chunk RVAs and section offsets as uint32_t
Saves 8 bytes on SectionChunk, one of the most commonly allocated data
structures.

llvm-svn: 360188
2019-05-07 20:30:41 +00:00
Simon Atanasyan 3bdb81c26d [mips] Fix ld instruction in PLT entries on MIPS64
Use `ld` and `daddiu` instructions in MIPS64 PLT records. That fixes a
segmentation fault.

Patch by Qiao Pengcheng.

Differential Revision: https://reviews.llvm.org/D61586

llvm-svn: 360187
2019-05-07 20:26:23 +00:00
Simon Atanasyan 494b763cfc [mips] Rename test case. NFC
This test case checks MIPS PLT records for N64 ABI. For the N32 ABI case
there is a separate test case `mips-plt-n32.s`.

llvm-svn: 360186
2019-05-07 20:26:12 +00:00
Sam Clegg b33fdb7768 [WebAssembly] Don't generate unused table entries.
When generating PIC output only relocations of type
R_WASM_TABLE_INDEX_REL_SLEB should generate table entries.

R_WASM_TABLE_INDEX_I32 get resolved at runtime via the auto-generated
__wasm_apply_relocs functions.

R_WASM_TABLE_INDEX_SLEB are not allowed in PIC code.

Differential Revision: https://reviews.llvm.org/D61539

llvm-svn: 360165
2019-05-07 15:46:30 +00:00
Nico Weber 4b81e9f8d1 lld-link: Allow /? as option prefix, like -? is allowed
link.exe seems to allow `/?foo` and `-?foo` in addition to `/foo` and `-foo`.

Since lld-link already supports the `-?foo` spelling, support `/?foo` as well.

Differential Revision: https://reviews.llvm.org/D61375

llvm-svn: 360150
2019-05-07 14:15:35 +00:00
Nico Weber 54743d5767 Add typo correction for command-line flags to ELF and COFF lld drivers
For lld-link, unknown '/'-style flags are treated as filenames on POSIX
systems, so only '-'-style flags get typo correction for now. This
matches clang-cl.

PR37006.

Differential Revision: https://reviews.llvm.org/D61443

llvm-svn: 360145
2019-05-07 13:48:30 +00:00
George Rimar 72f7a98876 [LLD][ELF] - Remove symbol-name-offset.elf binary from test cases.
It was possible to convert the test case to YAML test.
After that, we have only one binary test left in LLD/ELF.

llvm-svn: 360139
2019-05-07 12:59:44 +00:00
Fangrui Song f3994e4dfa [ELF] Reorder BitcodeFiles.empty() to call thinLTOCreateEmptyIndexFiles() in only one place
It makes the --plugin-opt=obj-path= and --plugin-opt=thinlto-index-only=
behavior more consistent - the files will be created in the
BitcodeFiles.empty() case, but I assume whether it behaves this way is
not required by anyone.

LTOObj->run() cannot run with empty BitcodeFiles. There would be an error:

    ld.lld: error: No available targets are compatible with triple ""

Differential Revision: https://reviews.llvm.org/D61635

llvm-svn: 360129
2019-05-07 10:40:26 +00:00
Fangrui Song 912251e82f [PPC64] toc-indirect to toc-relative relaxation
This is based on D54720 by Sean Fertile.

When accessing a global symbol which is not defined in the translation unit,
compilers will generate instructions that load the address from the toc entry.

If the symbol is defined, non-preemptable, and addressable with a 32-bit
signed offset from the toc pointer, the address can be computed
directly. e.g.

    addis 3, 2, .LC0@toc@ha  # R_PPC64_TOC16_HA
    ld    3, .LC0@toc@l(3)   # R_PPC64_TOC16_LO_DS, load the address from a .toc entry
    ld/lwa 3, 0(3)           # load the value from the address

    .section .toc,"aw",@progbits
    .LC0: .tc var[TC],var

can be relaxed to

    addis 3,2,var@toc@ha     # this may be relaxed to a nop,
    addi  3,3,var@toc@l      # then this becomes addi 3,2,var@toc
    ld/lwa 3, 0(3)           # load the value from the address

We can delete the test ppc64-got-indirect.s as its purpose is covered by
newly added ppc64-toc-relax.s and ppc64-toc-relax-constants.s

Reviewed By: ruiu, sfertile

Differential Revision: https://reviews.llvm.org/D60958

llvm-svn: 360112
2019-05-07 04:26:05 +00:00
Sam Clegg 5f8c2edef3 [WebAssembly] Add more test coverage for reloctions against section symbols
The only known user of this relocation type and symbol type is
the debug info sections, but we were not testing the `--relocatable`
output path.

This change adds a minimal test case to cover relocations against
section symbols includes `--relocatable` output.

Differential Revision: https://reviews.llvm.org/D61623

llvm-svn: 360110
2019-05-07 03:53:16 +00:00
Fangrui Song 12fb52007b [AMDGPU][test] Define local symbols used in amdgpu-relocs.s
Differential Revision: https://reviews.llvm.org/D61594

llvm-svn: 360046
2019-05-06 14:17:59 +00:00
Alexander Kornienko 8b92ec521d [lld] A better version of the fix in r359942.
Thanks to George Rimar for the suggestion.

llvm-svn: 360040
2019-05-06 12:11:30 +00:00
Alexander Kornienko 3f796f974d [lld] Specify output file explicitly
The test shouldn't try to create `a.out` in the current directory, which can be
read-only (and it is in our test setup).

llvm-svn: 359942
2019-05-03 23:11:32 +00:00
Reid Kleckner 0a1b1d6e62 Shrink SectionChunk by combining Relocs and SectionName sizes
SectionChunk is one of the most frequently allocated data structures in
LLD, since there are about four per function when optimizations and
debug info are enabled (.text, .pdata, .xdata, .debug$S).

A PE COFF file cannot be larger than 2GB, so there is an inherent limit
on the length of the section name and the number of relocations.
Decompose the ArrayRef and StringRef into pointer and size, and put them
back together in the accessors for section name and relocation list.

I plan to gather complete performance numbers later by padding
SectionChunk with dead data and measuring performance after all the size
optimizations are done.

llvm-svn: 359923
2019-05-03 20:17:14 +00:00
Fangrui Song d45df09435 [ELF] Place SHT_NOTE sections with the same alignment into one PT_NOTE
Summary:
While the generic ABI requires notes to be 8-byte aligned in ELF64, many
vendor-specific notes (from Linux, NetBSD, Solaris, etc) use 4-byte
alignment.

In a PT_NOTE segment, if 4-byte aligned notes are followed by an 8-byte
aligned note, the possible 4-byte padding may make consumers fail to
parse the 8-byte aligned note. See PR41000 for a recent report about
.note.gnu.property (NT_GNU_PROPERTY_TYPE_0).
(Note, for NT_GNU_PROPERTY_TYPE_0, the consumers should probably migrate
to PT_GNU_PROPERTY, but the alignment issue affects other notes as well.)

To fix the issue, don't mix notes with different alignments in one
PT_NOTE. If compilers emit 4-byte aligned notes before 8-byte aligned
notes, we'll create at most 2 segments.

sh_size%sh_addralign=0 is actually implied by the rule for linking
unrecognized sections (in generic ABI), so we don't have to check that.
Notes that match in name, type and attribute flags are concatenated into
a single output section. The compilers have to ensure
sh_size%sh_addralign=0 to make concatenated notes parsable.

An alternative approach is to create a PT_NOTE for each SHT_NOTE, but
we'll have to incur the sizeof(Elf64_Phdr)=56 overhead every time a new
note section is introduced.

Reviewers: ruiu, jakehehrlich, phosek, jhenderson, pcc, espindola

Subscribers: emaste, arichardson, krytarowski, fedor.sergeev, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D61296

llvm-svn: 359853
2019-05-03 00:35:49 +00:00
Nico Weber 81862f82ee lld-link: Add /force:multipleres extension to make dupe resource diag non-fatal
As a side benefit, lld-link now reports more than one duplicate resource
entry before exiting with an error even if the new flag is not passed.

llvm-svn: 359829
2019-05-02 21:21:55 +00:00
Fangrui Song 0178cff279 [ELF] --plugin-opt=thinlto-index-only: create empty index files even if all bitcode files are lazy
Summary:
The gold plugin behavior (creating empty index files for lazy bitcode
files) was added in D46034, but it missed the case when there is no
non-lazy bitcode files, e.g.

    ld.lld -shared crti.o crtbeginS.o --start-lib bitcode.o --end-lib ...

crti.o crtbeginS.o are not bitcode, but our distributed build system
wants bitcode.o.thinlto.bc to confirm all expected outputs are created
based on all of the modules provided to the linker.

Differential Revision: https://reviews.llvm.org/D61420

llvm-svn: 359788
2019-05-02 14:05:20 +00:00
Fangrui Song 8be28cdc52 [Object] Change getSectionName() to return Expected<StringRef>
Summary:
It currently receives an output parameter and returns
std::error_code. Expected<StringRef> fits for this purpose perfectly.

Differential Revision: https://reviews.llvm.org/D61421

llvm-svn: 359774
2019-05-02 10:32:03 +00:00
Fangrui Song d8c2fa99c1 [ELF] Delete a cant-write test from test/lto/thinlto-index-only.ll
The test is performed by thinlto-cant-write-index.ll

llvm-svn: 359769
2019-05-02 09:29:24 +00:00
Nico Weber 413517ecfe lld-link: Make "duplicate resource" error message a bit more concise
Reduces the error message from:
    lld-link: error: failed to parse .res file: duplicate resource: type STRINGTABLE (ID 6)/name ID 3/language 1033, in test1.res and in test2.res

To:
    lld-link: error: duplicate resource: type STRINGTABLE (ID 6)/name ID 3/language 1033, in test1.res and in test2.res

Make sure every error message emitted by cvtres contains the name of at
least one ".res" file, so that removing the "failed to parse .res file"
string doesn't lose information.

Differential Revision: https://reviews.llvm.org/D61388

llvm-svn: 359749
2019-05-02 01:52:24 +00:00
Fangrui Song c669ef1721 Delete trailing \r. NFC
llvm-svn: 359745
2019-05-02 00:39:03 +00:00
Ben Dunbobbin 6e32dd6cfd [LLD] Emit dynamic relocations for references to script symbols in -pie links
https://reviews.llvm.org/D55423 caused LLD to stop emitting dynamic relocations for references to script symbols in -pie links.

This patch fixes that regression.

https://reviews.llvm.org/D61298

llvm-svn: 359683
2019-05-01 14:07:31 +00:00
Fangrui Song 5387c2cd17 [llvm-objdump] Print newlines before and after "Disassembly of section ...:"
This improves readability and the behavior is consistent with GNU objdump.

The new test test/tools/llvm-objdump/X86/disassemble-section-name.s
checks we print newlines before and after "Disassembly of section ...:"

Differential Revision: https://reviews.llvm.org/D61127

llvm-svn: 359668
2019-05-01 10:40:48 +00:00
Fangrui Song 4672e52e22 [llvm-readobj] llvm-readobj --elf-output-style=GNU => llvm-readelf
While updating the test, change -l -S to -S -l as the output of -S goes
before -l.

llvm-svn: 359653
2019-05-01 06:02:16 +00:00
Fangrui Song b159906a9a [test] Change llvm-readobj -long-option to --long-option or well-known short options. NFC
Also change some options that have different semantics (cause confusion) in llvm-readelf mode:

-s => -S
-t => --symbols
-sd => --section-data

llvm-svn: 359651
2019-05-01 05:49:01 +00:00
Andrew Ng 24896d304d [LLD][ELF] /DISCARD/ output sections should not be orphans
/DISCARD/ output sections were being treated as orphans. As a result, if
a /DISCARD/ output section has been assigned a PHDR, it could cause
incorrect assignment of sections to segments.

Differential Revision: https://reviews.llvm.org/D61186

llvm-svn: 359565
2019-04-30 14:31:22 +00:00
Andrew Ng 0f4c58f6f4 [LLD][ELF] Fix getRankProximity to "ignore" not live sections
This is a follow up to r358979 which made findOrphanPos only consider
live sections. Unfortunately, this required change to getRankProximity,
used by findOrphanPos, was missed.

Differential Revision: https://reviews.llvm.org/D61197

llvm-svn: 359554
2019-04-30 12:27:06 +00:00
Nico Weber 99bad37013 Add more lld release notes
llvm-svn: 359518
2019-04-29 23:26:21 +00:00
Nico Weber aec5dcc457 Add some lld-link 9.0 release notes
llvm-svn: 359412
2019-04-29 00:42:05 +00:00
George Rimar dee900ae59 [LLD][ELF] - Do not remove empty sections referenced in LOADADDR/ADDR commands.
This is https://bugs.llvm.org//show_bug.cgi?id=38750.

If script references empty sections in LOADADDR/ADDR commands

.empty  : { *(.empty ) }
.text   : AT(LOADADDR (.empty) + SIZEOF (.empty)) { *(.text) }
then an empty section will be removed and LOADADDR/ADDR will evaluate to null.
It is not that user may expect from using of the generic script, what is a common case.

Differential revision: https://reviews.llvm.org/D54621

llvm-svn: 359279
2019-04-26 06:59:30 +00:00
Fangrui Song 98b70f6705 [ELF] Change std::max<uint64_t> to uint32_t for section alignment
Summary:
We use `uint32_t SectionBase::Alignment` and `uint32_t
PhdrEntry::p_align` despite alignments being 64 bits in ELF64.
Fix the std::max template arguments accordingly.

The currently 160-byte InputSection will become 168 bytes if we make SectionBase::Alignment uint64_t.

Differential Revision: https://reviews.llvm.org/D61171

llvm-svn: 359268
2019-04-26 04:07:58 +00:00
Sam Clegg b685ddf288 [WebAssembly] Always take into account added when applying runtime relocations
The code we generate for applying data relocations at runtime omitted
the symbols with GOT entries.

Also refactor the code to reduce duplication.

Differential Revision: https://reviews.llvm.org/D61111

llvm-svn: 359207
2019-04-25 17:11:54 +00:00
George Rimar 44f2d74aa5 [LLD][ELF] - Convert symbol-index.s testcase to a YAML test case. NFCI.
This removes one more binary object from the inputs and fixes the
test case description.

Previously it said that:
"symbol-index.elf has incorrect type of .symtab section.
There is no symbol bodies because of that and any symbol index becomes incorrect."

But the real reason of the failture was not the incorrect type of a symbol table,
but invalid index of the symbol used in a relocation, what happened because
previous test tried to read .symtab as a SHT_RELA section.

llvm-svn: 359197
2019-04-25 15:08:52 +00:00
George Rimar e043417268 [LLD][ELF] - Move the test to a correct folder, remove excessive input. NFCI.
This test should live in `invalid` folder.

Also it was possible to avoid adding input
with use of `-docnum=x` yaml2obj argument.

llvm-svn: 359194
2019-04-25 14:53:23 +00:00
Nico Weber c0838af754 lld-link: Implement /swaprun: flag
r191276 added this to old LLD, but it never made it to new LLD -- except
that the flag was in Options.td, so it was silently ignored. I figured
it should be easy to implement, so I did that instead of removing the
flags from Options.td.

I then discovered that link.exe also supports comma-separated lists of
'cd' and 'net', which made the parsing code a bit annoying.

The Alias technique in Options.td is to get nice help output.

Differential Revision: https://reviews.llvm.org/D61067

llvm-svn: 359192
2019-04-25 14:02:26 +00:00
Nico Weber 23cb79ff93 llvm-cvtres: Make new dupe resource error a bit friendlier
For well-known type IDs, include the name of the type.

To not duplicate the ID->name map, make llvm-readobj call this new
function as well.  It has slightly different output, so this also
requires updating a few tests.

Differential Revision: https://reviews.llvm.org/D61086

llvm-svn: 359153
2019-04-24 23:26:30 +00:00
Reid Kleckner 54c8182a3f [COFF] Don't emit .gfids sections when CFG is off
Put them on the list of GuardFidChunks instead of the main Chunks list,
even with CFG is off. It will be ignored if CFG is disabled.

llvm-svn: 359137
2019-04-24 20:38:37 +00:00
Sam Clegg f7f00ebc27 [docs] Copy-edit lld/docs/WebAssembly.rst
Fixes small typos in WebAssembly documentation. I first noticed the
sub-heading "Bahavior", and then decided to review the whole file.

Patch by Christoph Siedentop!

Differential Revision: https://reviews.llvm.org/D60987

llvm-svn: 359103
2019-04-24 15:13:35 +00:00
Fangrui Song 5929553868 [ELF] Delete a redundant SHT_NOBITS -> SHT_PROGBITS after D60131
Differential Revision: https://reviews.llvm.org/D61006

llvm-svn: 359099
2019-04-24 14:44:07 +00:00
Fangrui Song 513d3658e7 [PPC64] Consider localentry offset when computing branch distance
Summary:
We don't take localentry offset into account, and thus may fail to
create a long branch when the gap is just a few bytes smaller than 2^25.

relocation R_PPC64_REL24 out of range: 33554432 is not in [-33554432, 33554431]
relocation R_PPC64_REL24 out of range: 33554436 is not in [-33554432, 33554431]

Fix that by adding the offset to the symbol VA.

Differential Revision: https://reviews.llvm.org/D61058

llvm-svn: 359094
2019-04-24 14:03:30 +00:00
George Rimar fa34952a00 [LLD][ELF] - Remove binding.elf binary from test case. NFCI.
This introduces YAML based invalid-binding.test instead.

llvm-svn: 359086
2019-04-24 12:16:39 +00:00
George Rimar 1ee7bee224 [LLD][ELD] - Remove excessive lines from test. NFC.
They are not used.

llvm-svn: 359084
2019-04-24 12:00:09 +00:00
Fangrui Song 74780852dc [ELF] Fix a gcc -Wextra warning
Extracted from D61046.

warning: enumeral and non-enumeral type in conditional expression [-Wextra]

Cast SHF_ALLOC to avoid that.

llvm-svn: 359070
2019-04-24 05:33:33 +00:00
Sam Clegg 99cf58339f [WebAssembly] Fix typo in relocation checking
Runtime relocation are generated for relocations of type
R_WASM_MEMORY_ADDR_I32 when in PIC mode (either -shared or -pie).

Followup on https://reviews.llvm.org/D60882.

Differential Revision: https://reviews.llvm.org/D60992

llvm-svn: 358995
2019-04-23 14:49:38 +00:00
George Rimar bb2079b7e1 [LLD][ELF] - Remove dynamic-section-sh_size.elf binary, convert test to yaml. NFCI.
dynamic-section-sh_size.elf was introduced in D25090.
Now it is possible to use yaml2obj instead. 
That is what this patch does.

Also I added one more case of a possibly broken .dynamic
section just in case.

llvm-svn: 358990
2019-04-23 14:21:31 +00:00
George Rimar 95203efc1a [LLD][ELF] - Remove file-class.a binary from inputs and improve the test case.
file-class.a was used to diagnose the "corrupted ELF file: invalid file class"
error when the object was fetched from the archive.

file-class.a contained an object of 16 bytes size. I replaced it with
an echo call (because it is impossible to use yaml2obj for that, and I am
not sure it is worth to support), and also increased its size to 18 bytes.
That allowed to also test a case when such object is a regular input and not an
archive member (we have a bit different logic for these cases).

llvm-svn: 358985
2019-04-23 13:27:54 +00:00
Andrew Ng ccba42c7eb [ELF] Change default output section type to SHT_PROGBITS
This fixes an issue where a symbol only section at the start of a
PT_LOAD segment, causes incorrect alignment of the file offset for the
start of the segment which results in the output of an invalid ELF.

SHT_PROGBITS was the default output section type in the past.

Differential Revision: https://reviews.llvm.org/D60131

llvm-svn: 358981
2019-04-23 12:38:52 +00:00
Alexandre Ganea 2769d58628 [LLD][COFF] Fix /linkrepro with output options that take a filename or path
The following options: /pdb, /out or /implib now emit in the repro.tar/response.txt only a filename stripped from its path, to avoid non-existent paths on the reproducer's machine.

Differential Revision: https://reviews.llvm.org/D59530

llvm-svn: 358980
2019-04-23 12:30:49 +00:00
Andrew Ng 98c858a23b [ELF] Change findOrphanPos to only consider live sections
This patch changes the behaviour of findOrphanPos to only consider live
sections when placing orphan sections. This used to be how it behaved in
the past.

Differential Revision: https://reviews.llvm.org/D60273

llvm-svn: 358979
2019-04-23 12:17:15 +00:00
Fangrui Song 003c18a39c [PPC][PPC64] Improve some llvm-objdump -d -D tests
Various improvement:

Some offsets in disassembly are incorrect after several layout adjustment. Fix them.
llvm-objdump -D should not be used. -D dumps unrelated non-text sections. Replace them with llvm-objdump -d, llvm-readelf -x, etc
Many llvm-objdump -d tests use {{.*}} . Add the option --no-show-raw-insn to avoid check hex bytes.

ppc64-long-branch.s does not need a shared object. Delete it.
Make ppc64-ifunc.s check 2 ifuncs.

Reviewers: ruiu, espindola

Subscribers: emaste, nemanjai, arichardson, kbarton, jsji, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60998

llvm-svn: 358975
2019-04-23 11:47:28 +00:00
George Rimar 6684a351ed [LLD][ELF] - Fix mips-invalid-options-descriptor.s test case and convert to YAML.
It was initially introduced in D25229 to report the "zero option descriptor size"
error message. In following commits it was broken and did not report this
error anymore. I think that happened because elf object was a result of fuzzing
and it was broken in many ways.

This patch converts this test to a YAML, removes a binary and hence fixes the
original intention.

llvm-svn: 358972
2019-04-23 11:19:51 +00:00
George Rimar b7dc13f4c0 [LLD][ELF] - Remove multiple-eh-relocs.elf binary from the inputs. NFCI.
Introduced multiple-relocations-sections.test based on YAML instead.

llvm-svn: 358966
2019-04-23 10:03:09 +00:00
George Rimar 545f621a7c [LLD][ELF] - Update test case after llvm-mc change.
rL358960 "[llvm-mc] - Properly set the the address align field of the compressed sections."
changed the sh_addralign field of the "zlib" compressed debug sections.

llvm-svn: 358961
2019-04-23 09:30:26 +00:00
Fangrui Song d986e41fe4 [PPC64] Allow R_PPC64_DTPREL* to preemptable local-dynamic symbols
Similar to D60945.

Differential Revision: https://reviews.llvm.org/D60994

llvm-svn: 358950
2019-04-23 06:31:44 +00:00
Fangrui Song 32c0ebe615 Use llvm::stable_sort
Make some small adjustment while touching the code: make parameters
const, use less_first(), etc.

Differential Revision: https://reviews.llvm.org/D60989

llvm-svn: 358943
2019-04-23 02:42:06 +00:00
Sam Clegg 7cdec273dd [WebAssembly] Error on relocations against undefined data symbols.
We can't (currently) meaningfully resolve certain types of relocations
against undefined data symbols.  Previously when `--allow-undefined` was
used we were treating such relocation much like weak data symbols and
simply inserting zeros.  This change turns such use cases in to an
error.

This means that `--allow-undefined` is no longer effective for data
symbols.

Fixes https://bugs.llvm.org/show_bug.cgi?id=40364

Differential Revision: https://reviews.llvm.org/D60882

llvm-svn: 358899
2019-04-22 16:12:54 +00:00
George Rimar 81ffc08a8a [LLD][ELF] - Remove a binary from the inputs. NFCI.
section-index.elf was removed and the corresponding test
was replaced with a yaml2obj based test.

llvm-svn: 358889
2019-04-22 14:53:32 +00:00
George Rimar 3275742898 [LLD][ELF] - Do not forget to use ch_addralign field after decompressing the sections.
LLD did not use ELF::Chdr::ch_addralign for decompressed sections.
This resulted in a broken output.

Fixes https://bugs.llvm.org/show_bug.cgi?id=40482.

Differential revision: https://reviews.llvm.org/D60959

llvm-svn: 358885
2019-04-22 13:40:42 +00:00
George Rimar f902250fc1 [LLD][ELF] - Handle quoted strings in the linker scripts correctly.
This is the https://bugs.llvm.org/show_bug.cgi?id=41356,

Seems it is kind of unusual case but it is possible to
have sections that require quotes for their namings.
Like "aaa bbb".

This patch adds support for those.

Differential revision: https://reviews.llvm.org/D60901

llvm-svn: 358874
2019-04-22 07:57:07 +00:00
Sam Clegg 7868fb6fdd [WebAssembly] Fix R_WASM_FUNCTION_OFFSET_I32 relocation warnings
We were incorrectly used the symbol table version of the function rather
than the object-local version when checking the existing relocation
value.

This was causing erroneous warnings for comat symbols defined in
multiple object.s

Fixes: https://bugs.llvm.org/show_bug.cgi?id=40503

Differential Revision: https://reviews.llvm.org/D60928

llvm-svn: 358871
2019-04-22 05:26:44 +00:00
Fangrui Song bc4b159bb1 [ELF][X86] Allow R_386_TLS_LDO_32 and R_X86_64_DTPOFF{32,64} to preemptable local-dynamic symbols
Summary:
Fixes PR35242. A simplified reproduce:

    thread_local int i; int f() { return i; }

% {g++,clang++} -fPIC -shared -ftls-model=local-dynamic -fuse-ld=lld a.cc
ld.lld: error: can't create dynamic relocation R_X86_64_DTPOFF32 against symbol: i in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output

In isStaticLinkTimeConstant(), Syn.IsPreemptible is true, so it is not
seen as a constant. The error is then issued in processRelocAux().

A symbol of the local-dynamic TLS model cannot be preempted but it can
preempt symbols of the global-dynamic TLS model in other DSOs.
So it makes some sense that the variable is not static.

This patch fixes the linking error by changing getRelExpr() on
R_386_TLS_LDO_32 and R_X86_64_DTPOFF{32,64} from R_ABS to R_DTPREL.
R_PPC64_DTPREL_* and R_MIPS_TLS_DTPREL_* need similar fixes, but they are not handled in this patch.

As a bonus, we use `if (Expr == R_ABS && !Config->Shared)` to find
ld-to-le opportunities. R_ABS is overloaded here for such STT_TLS symbols.
A dedicated R_DTPREL is clearer.

Differential Revision: https://reviews.llvm.org/D60945

llvm-svn: 358870
2019-04-22 03:10:40 +00:00
Fangrui Song e1f3191a0d [ELF][X86] Rename R_RELAX_TLS_GD_TO_IE_END to R_RELAX_TLS_GD_TO_IE_GOTPLT
Summary:
This relocation type is used by R_386_TLS_GD. Its formula is the same as
R_GOTPLT (e.g R_X86_64_GOT{32,64} R_386_TLS_GOTIE). Rename it to be clearer.

Differential Revision: https://reviews.llvm.org/D60941

llvm-svn: 358868
2019-04-22 02:48:37 +00:00
George Rimar 5f7665969a [LLD][ELF] - Fix mistype. NFC.
Change the tripple name from
aarch64-linux-gnux to -triple=aarch64-linux-gnu

llvm-svn: 358810
2019-04-20 10:50:27 +00:00
Thomas Lively 84771e2d21 [WebAssembly] Emit the DataCount section when bulk memory is enabled
Summary:
The DataCount section is necessary for the bulk memory operations
memory.init and data.drop to validate, but it is not recognized by
engines that do not support bulk memory, so emit the section only if
bulk-memory is enabled.

Reviewers: aheejin, dschuff, sbc100

Subscribers: jgravelle-google, sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60637

llvm-svn: 358798
2019-04-19 23:40:36 +00:00
Reid Kleckner a30920c31f [COFF] Pack Name in Symbol as is done in ELF
Summary:
This assumes all symbols are <4GB long, so we can store them as a 32-bit
integer. This reorders the fields so the length appears first, packing
with the other bitfield data in the base Symbol object.

This saved 70MB / 3.60% of heap allocations when linking
browser_tests.exe with no PDB. It's not much as a percentage, but worth
doing. I didn't do performance measurements, I don't think it will be
measurable in time.

Reviewers: ruiu, inglorion, amccarth, aganea

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60297

llvm-svn: 358794
2019-04-19 22:51:49 +00:00
Martin Storsjo 2c1f3ea538 [MinGW] Add an --appcontainer flag, passed through to lld-link
GNU ld doesn't have such a flag though, so this is a lld specific
option.

Differential Revision: https://reviews.llvm.org/D60860

llvm-svn: 358759
2019-04-19 13:50:43 +00:00
Ali Tamur 783d84bb39 [llvm] Prevent duplicate files in debug line header in dwarf 5: another attempt
Another attempt to land the changes in debug line header to prevent duplicate
files in Dwarf 5. I rolled back my previous commit because of a mistake in
generating the object file in a test. Meanwhile, I addressed some offline
comments and changed the implementation; the largest difference is that
MCDwarfLineTableHeader does not keep DwarfVersion but gets it as a parameter. I
also merged the patch to fix two lld tests that will strt to fail into this
patch.

Original Commit:

https://reviews.llvm.org/D59515

Original Message:
Motivation: In previous dwarf versions, file name indexes started from 1, and
the primary source file was not explicit. Dwarf 5 standard (6.2.4) prescribes
the primary source file to be explicitly given an entry with an index number 0.

The current implementation honors the specification by just duplicating the
main source file, once with index number 0, and later maybe with another
index number. While this is compliant with the letter of the standard, the
duplication causes problems for consumers of this information such as lldb.
(Some files are duplicated, where only some of them have a line table although
all refer to the same file)

With this change, dwarf 5 debug line section files always start from 0, and
the zeroth entry is not duplicated whenever possible. This requires different
handling of dwarf 4 and dwarf 5 during generation (e.g. when a function returns
an index zero for a file name, it signals an error in dwarf 4, but not in dwarf
5) However, I think the minor complication is worth it, because it enables all
consumers (lldb, gdb, dwarfdump, objdump, and so on) to treat all files in the
file name list homogenously.

llvm-svn: 358732
2019-04-19 02:26:56 +00:00
Fangrui Song 196a440411 [wasm] Simplify. NFC
llvm-svn: 358663
2019-04-18 13:33:29 +00:00
George Rimar 55e1987f81 [LLD][ELF] - Convert out-of-order-section-in-region.s to *.test. NFCI.
This is consistent with the our others tests that has large scripts.

llvm-svn: 358659
2019-04-18 12:13:41 +00:00
George Rimar 69186f82f3 [LLD][ELF] - A fix for "linker script assignment loses relative nature of section" bug.
This is https://bugs.llvm.org//show_bug.cgi?id=39857.
I added the comment with much more details to the bug page,
the short version is below.

The following script and code demonstrates the issue:

aliasto__text = __text;
 SECTIONS {
  .text 0x1000 : { __text = . ; *(.text) }
 }
...
call aliasto__text

LLD fails with "cannot refer to absolute symbol: aliasto__text" error.
It happens because at the moment of scanning the relocations
we do not yet assign the correct/final/any section value for the symbol aliasto__text.
I made a change to Relocations.cpp to fix that.

Also, I had to remove the symbol-location.s test case completely, because now it does not
trigger any error. Since now all linker scripts symbols are resolved to constants, no
errors can be triggered at all it seems. I checked that it is consistent with the behavior
of bfd and gold (they do not trigger errors for the case from symbol-location.s), so it should
be OK. I.e. at least it is probably not the best possible, but natural behavior we obtained.

Differential revision: https://reviews.llvm.org/D55423

llvm-svn: 358652
2019-04-18 10:00:37 +00:00
Fangrui Song 3deff86657 [ELF] Respect NonAlloc when copying flags from the previous sections
Summary:
If the output section contains only symbol assignments, we copy flags
from the previous sections. Don't set SHF_ALLOC if NonAlloc is true.

We also have to change the type from SHT_NOBITS to SHT_PROGBITS.
In ld.bfd, bfd_elf_get_default_section_type maps non-alloctable sections to SHT_PROGBITS.
Non-alloctable SHT_NOBITS sections do not make sense.

Fixes PR38626

Differential Revision: https://reviews.llvm.org/D59986

llvm-svn: 358650
2019-04-18 09:22:05 +00:00
George Rimar da49faf15e [LLD][ELF] - Fix the different behavior of the linker script symbols on different platforms.
This generalizes code and also fixes the broken behavior shown in
one of our test cases for some targets, like x86-64.

The issue occurs when the forward declarations are used in the script.
One of the samples is:

SECTIONS {
  foo = ADDR(.text) - ABSOLUTE(ADDR(.text));
};

In that case, we have a broken output when output target does
not use thunks. That happens because thunks creating code
(called from maybeAddThunks)
calls Script->assignAddresses() at least one more time,
what fixups the values. As a result final symbols values can
be different on AArch64 and x86, for example.

In this patch, I generalize and rename maybeAddThunks to
finalizeAddressDependentContent and now it is used and called
by all targets.

Differential revision: https://reviews.llvm.org/D55550

llvm-svn: 358646
2019-04-18 08:15:54 +00:00
Fangrui Song 957c356ffe [ELF] Place SectionPiece::{Live,Hash} bit fields together
Summary:
We access Live and OutputOff (which may share the same memory location)
concurrently in 2 parallelForEachN loops. Separating them avoids subtle
data races like D41884/PR35788. This patch places Live and Hash
together.

2 reasons this is appealing:

1) Hash is immutable. Live is almost read-only - only written once in MarkLive.cpp where
   Hash is not accessed
2) we already discard low bits of Hash to decide ShardID. It doesn't
   matter much if we make 32-bit Hash to 31-bit.

   For a huge internal clang -O3 executable (1.6GiB),
   `Strings` in StringTableBuilder::finalizeStringTable contains at most 310253 elements.
   The expected number of pair-wise collisions 2^(-31) * C(310253,2) ~= 22.41 is too small to have a negative impact on performance.
   Actually, my benchmark shows there is actually a minor performance improvement.

Differential Revision: https://reviews.llvm.org/D60765

llvm-svn: 358645
2019-04-18 07:46:09 +00:00
Rui Ueyama 14ef9b30b6 lld: elf: Fix sections with explict addresses in regions
Patch by Gabriel Smith.

The address for a section would be evaluated before the region was
switched to. Because of this, the position within the region would not
be updated. After the region is swapped to the dot would be set to the
out of date position within the region, undoing the section address
evaluation.

To fix this, the region is swapped to before the section's address is
evaluated. As part of the fallout of this, expandMemoryRegions needed
to be gated in setDot on the condition that the evaluated address is
less than the dot. This is for the case where sections are not listed
from lowest address to highest address.

Finally, a test for the case where sections are listed "out of order"
was added.

Differential Revision: https://reviews.llvm.org/D60744

llvm-svn: 358638
2019-04-18 02:32:12 +00:00
Fangrui Song 2bc3a19a49 [ELF] Use llvm::bsearch. NFC
Differential Revision: https://reviews.llvm.org/D60813

llvm-svn: 358565
2019-04-17 08:00:46 +00:00
Rui Ueyama 5081e41bda Fix a crash bug caused by a nested call of parallelForEach.
parallelForEach is not reentrant. We use parallelForEach to call
each section's writeTo(), so calling the same function within writeTo()
is not safe.

Fixes https://bugs.llvm.org/show_bug.cgi?id=41508

Differential Revision: https://reviews.llvm.org/D60757

llvm-svn: 358547
2019-04-17 02:12:47 +00:00
Rui Ueyama 7f8ca6e367 lld: Fix initial Mach-O load commands size calculation omitting LC_FUNCTION_STARTS
Patch by Nicholas Allegra.

The Mach-O writer calculates the size of load commands multiple times.

First, Util::assignAddressesToSections() (in MachONormalizedFileFromAtoms.cpp)
calculates the size using headerAndLoadCommandsSize() (in
MachONormalizedFileBinaryWriter.cpp), which creates a temporary
MachOFileLayout for the NormalizedFile, only to retrieve its
headerAndLoadCommandsSize.  Later, writeBinary() (in
MachONormalizedFileBinaryWriter.cpp) creates a new layout and uses the offsets
from that layout to actually write out everything in the NormalizedFile.

But the NormalizedFile changes between the first computation and the second.
When Util::assignAddressesToSections is called, file.functionStarts is always
empty because Util::addFunctionStarts has not yet been called. Yet
MachOFileLayout decides whether to include a LC_FUNCTION_STARTS command based
on whether file.functionStarts is nonempty. Therefore, the initial computation
always omits it.

Because padding for the __TEXT segment (to make its size a multiple of the
page size) is added between the load commands and the first section, LLD still
generates a valid binary as long as the amount of padding happens to be large
enough to fit LC_FUNCTION_STARTS command, which it usually is.

However, it's easy to reproduce the issue by adding a section of a precise
size. Given foo.c:

  __attribute__((section("__TEXT,__foo")))
  char foo[0xd78] = {0};

Run:

  clang -dynamiclib -o foo.dylib foo.c -fuse-ld=lld -install_name
  /usr/lib/foo.dylib
  otool -lvv foo.dylib

This should produce:

  truncated or malformed object (offset field of section 1 in LC_SEGMENT_64
  command 0 not past the headers of the file)

This commit:

 - Changes MachOFileLayout to always assume LC_FUNCTION_STARTS is present for
   the initial computation, as long as generating LC_FUNCTION_STARTS is
   enabled. It would be slightly better to check whether there are actually
   any functions, since no LC_FUNCTION_STARTS will be generated if not, but it
   doesn't cause a problem if the initial computation is too high.

 - Adds a test.

 - Adds an assert in MachOFileLayout::writeSectionContent() that we are not
   writing section content into the load commands region (which would happen
   if the offset was calculated too low due to the initial load commands size
   calculation being too low).  Adds an assert in
   MachOFileLayout::writeLoadCommands to validate a similar situation where
   two size-of-load-commands computations are expected to be equivalent.

llvm-svn: 358545
2019-04-17 01:47:16 +00:00
Peter Collingbourne 97d25e068f ELF: Move build id computation to Writer. NFCI.
With partitions, each partition should have the same build id. This means
that the build id needs to be only computed once, otherwise we will end up
with different build ids in each partition as a result of the file contents
changing. This change moves the computation of the build id into Writer so
that it only happens once.

Differential Revision: https://reviews.llvm.org/D60342

llvm-svn: 358536
2019-04-16 22:45:14 +00:00
Fangrui Song f10065b68b [MachO] Add -macho to llvm-objdump commands
llvm-svn: 358473
2019-04-16 03:51:53 +00:00
Douglas Yung 45129145b9 Fix test on Windows that uses a different path separator character.
llvm-svn: 358453
2019-04-15 21:43:28 +00:00
Bob Haarman 8b1ec798b5 [LLD][COFF] use offset in archive to disambiguate archive members
Summary:
Archives can contain multiple members with the same name. This would
cause ThinLTO links to fail ("Expected at most one ThinLTO module per
bitcode file"). This change implements the same strategy we use in
the ELF linker: make the offset in the archive part of the module
name so that names are unique.

Reviewers: pcc, mehdi_amini, ruiu

Reviewed By: ruiu

Subscribers: eraman, steven_wu, dexonsmith, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60549

llvm-svn: 358440
2019-04-15 19:48:32 +00:00
Martin Storsjo cdf126ebec [COFF] Link crtend.o as the last object file
When faced with command line options such as "crtbegin.o appmain.o
-lsomelib crtend.o", GNU ld pulls in all necessary object files from
somelib before proceeding to crtend.o.

LLD operates differently, only loading object files from any
referenced static libraries after processing all input object files.

This uses a similar hack as in the ELF linker. Here, it moves crtend.o
to the end of the vector of object files. This makes sure that
terminator chunks for sections such as .eh_frame gets ordered last,
fixing DWARF exception handling for libgcc and gcc's crtend.o.

Differential Revision: https://reviews.llvm.org/D60628

llvm-svn: 358394
2019-04-15 10:57:44 +00:00
Fangrui Song abc068fc59 [ELF] Fix typo: .symtab_shndxr -> .symtab_shndx
The typo was introduced to llvm MC in rL204769 (fixed in rL358247) and then to lld.

Also, for relocatable-many-sections.s, the size of .symtab changed at some point and the formula needs update.

llvm-svn: 358248
2019-04-12 02:20:52 +00:00
Hans Wennborg 136a6a612a Fix two sphinx warnings
llvm-svn: 358155
2019-04-11 07:31:03 +00:00
Sam Clegg 0d9f609d82 [WebAssembly] Assign GOT entries symbols used in data relocations
Differential Revision: https://reviews.llvm.org/D60492

llvm-svn: 358090
2019-04-10 15:06:17 +00:00
Xing GUO 8ab7414580 [llvm-readobj] Should declare `ListScope` for `verneed` entries.
Summary: YAML mappings require keys to be unique. See: https://yaml.org/spec/1.2/spec.html#id2764652

Reviewers: jhenderson, grimar, rupprecht, espindola, ruiu

Reviewed By: ruiu

Subscribers: ruiu, emaste, arichardson, MaskRay, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60438

llvm-svn: 358078
2019-04-10 12:47:21 +00:00
Rui Ueyama 3a8bb7cd2c Discard debuginfo for object files empty after GC
Patch by Robert O'Callahan.

Rust projects tend to link in all object files from all dependent
libraries and rely on --gc-sections to strip unused code and data.
Unfortunately --gc-sections doesn't currently strip any debuginfo
associated with GC'ed sections, so lld links in the full debuginfo from
all dependencies even if almost all that code has been discarded. See
https://github.com/rust-lang/rust/issues/56068 for some details.

Properly stripping debuginfo for discarded sections would be difficult,
but a simple approach that helps significantly is to mark debuginfo
sections as live only if their associated object file has at least one
live code/data section. This patch does that. In a (contrived but not
totally artificial) Rust testcase linked above, it reduces the final
binary size from 46MB to 5.1MB.

Differential Revision: https://reviews.llvm.org/D54747

llvm-svn: 358069
2019-04-10 10:37:10 +00:00
Rui Ueyama 0c01607bbf Rename a variable and add a comment.
llvm-svn: 358049
2019-04-10 06:32:05 +00:00
Chih-Mao Chen 23e23836d5 Move tests in r357996 to correct location within monorepo
llvm-svn: 357997
2019-04-09 11:46:47 +00:00
Chih-Mao Chen 00100b5d0d [RISCV] Rewrite tests to use llvm-mc/llvm-objdump
Previously the tests in lld for RISC-V were given in yaml format as
LLVM's RISC-V assembler support was incomplete. Now that the
assembler/disassembler has matured we can rewrite all tests to use
LLVM's tools.

Differential Revision: https://reviews.llvm.org/D60415

llvm-svn: 357996
2019-04-09 11:39:50 +00:00
Chih-Mao Chen 949570ce39 [RISCV] Fix range check for HI20/LO12/RVC_LUI relocations
The code previously specified a 32-bit range for R_RISCV_HI20 and
R_RISCV_LO12_[IS], however this is incorrect as the maximum offset on
RV64 that can be formed from the immediate of lui and the displacement
of an I-type or S-type instruction is -0x80000800 to 0x7ffff7ff. There
is also the same issue with a c.lui and LO12 pair, whose actual
addressable range should be -0x20800 to 0x1f7ff.

The tests will be included in the next patch that converts all RISC-V
tests to use llvm-mc instead of yaml2obj, as assembler support has
matured enough to write tests in them.

Differential Revision: https://reviews.llvm.org/D60414

llvm-svn: 357995
2019-04-09 11:39:23 +00:00
Rui Ueyama f432fa6eee De-template SymbolTable::addShared.
Because of r357925, this member function doesn't have to be a
template of ELFT.

llvm-svn: 357982
2019-04-09 08:52:00 +00:00
Sam Clegg 0cfaa2470f [WebAssembly] Ensure ArchiveName is set even in the presence of --whole-archive.
Differential Revision: https://reviews.llvm.org/D60431

llvm-svn: 357966
2019-04-09 05:41:52 +00:00
Peter Collingbourne d3e207057f ELF: Move verneed tracking data structures out of VersionNeedSection.
For partitions I intend to use the same set of version indexes in
each partition for simplicity. Since each partition will need its own
VersionNeedSection this will require moving the verneed tracking out of
VersionNeedSection. The way I've done this is to move most of the tracking
into SharedFile. What will eventually become the per-partition tracking
still lives in VersionNeedSection.

As a bonus the code gets a little simpler and more consistent with how we
handle verdef.

Differential Revision: https://reviews.llvm.org/D60307

llvm-svn: 357926
2019-04-08 17:48:05 +00:00
Peter Collingbourne cc1618e668 ELF: De-template SharedFile. NFCI.
Differential Revision: https://reviews.llvm.org/D60305

llvm-svn: 357925
2019-04-08 17:35:55 +00:00
Rui Ueyama 4af8d47d05 Fix -emit-reloc against local symbols.
Previously, we drop symbols starting with .L from the symbol table, so
if there is a relocation that refers a .L symbol, it ended up
referencing a null -- which happened to be interpreted as an absolute
symbol.

This patch copies all symbols including local ones if -emit-reloc is
given.

Fixes https://bugs.llvm.org/show_bug.cgi?id=41385

Differential Revision: https://reviews.llvm.org/D60306

llvm-svn: 357885
2019-04-08 06:45:07 +00:00
Peter Collingbourne 883ab235ee ELF: De-template ELFFileBase. NFCI.
Differential Revision: https://reviews.llvm.org/D60304

llvm-svn: 357806
2019-04-05 20:16:26 +00:00
Peter Collingbourne ad4376e8af ELF: Simplify. NFCI.
Differential Revision: https://reviews.llvm.org/D60299

llvm-svn: 357739
2019-04-05 01:31:40 +00:00
Rui Ueyama 7c28937baf Remove redundant parameters. NFC.
llvm-svn: 357738
2019-04-05 01:30:09 +00:00
Sam Clegg a116d91ba2 [WebAssembly] Include function in wasm table when used in R_WASM_TABLE_INDEX_REL_SLEB
This should have been part of rL357710 but was overlooked because
in our test code the function in question was also used in other
relocations that caused it to be added to the table anyway.

Differential Revision: https://reviews.llvm.org/D60296

llvm-svn: 357737
2019-04-05 00:35:12 +00:00
Sam Clegg 09137be7f8 [WebAssembly] Apply data relocations at runtime in shared objects
See: https://github.com/WebAssembly/tool-conventions/blob/master/DynamicLinking.md

Data section relocations in wasm shared libraries are applied by the
library itself at static constructor time.  This change adds a new
synthetic function that applies relocations to relevant memory locations
on startup.

Differential Revision: https://reviews.llvm.org/D59278

llvm-svn: 357715
2019-04-04 18:40:51 +00:00
Sam Clegg 2a7cac932b [WebAssembly] Add new explicit relocation types for PIC relocations
See https://github.com/WebAssembly/tool-conventions/pull/106

Differential Revision: https://reviews.llvm.org/D59907

llvm-svn: 357710
2019-04-04 17:43:50 +00:00
Joseph Tremoulet 9f598ac706 [COFF] Fix delay import directory iterator
Summary:
Take the Index into account in `getDelayImportTable`, otherwise we
always return the entry for the first delay DLL reference.

Reviewers: ruiu

Reviewed By: ruiu

Subscribers: hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60081

llvm-svn: 357697
2019-04-04 14:13:28 +00:00
Peter Collingbourne 8238604259 ELF: Move SymtabSHNDX and getSectionIndex() to ObjFile. NFCI.
Differential Revision: https://reviews.llvm.org/D60244

llvm-svn: 357670
2019-04-04 03:13:51 +00:00
Reid Kleckner e10d00419a [codeview] Remove Type member from CVRecord
Summary:
Now CVType and CVSymbol are effectively type-safe wrappers around
ArrayRef<uint8_t>. Make the kind() accessor load it from the
RecordPrefix, which is the same for types and symbols.

Reviewers: zturner, aganea

Subscribers: hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60018

llvm-svn: 357658
2019-04-04 00:28:48 +00:00
George Rimar ff3397247f [LLD] - Update the test cases after yaml2obj change.
https://reviews.llvm.org/D60122 (r357595) changed the
symbols description format.

This change updates the LLD tests.

llvm-svn: 357596
2019-04-03 14:54:22 +00:00
Reid Kleckner cc525c97b7 [COFF] Reduce the size of Chunk and SectionChunk, NFC
Summary:
Reorder the fields in both to use padding more efficiently, and add more
comments on the purpose of the fields.

Replace `std::vector<SectionChunk*> AssociativeChildren` with a
singly-linked list. This avoids the separate vector allocation to list
associative children, and shrinks the 3 pointers used for the typically
empty vector down to 1.

In the end, this reduces the sum of heap allocations used to link
browser_tests.exe with NO PDB by 13.10%, going from 2,248,728 KB to
1,954,071 KB of heap. These numbers exclude memory mapped files, which
are of course a significant factor in LLD's memory usage.

Reviewers: ruiu, mstorsjo, aganea

Subscribers: jdoerfert, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59797

llvm-svn: 357535
2019-04-02 22:11:58 +00:00
Alexandre Ganea 19775a4c67 [LLD][COFF] Move type merging structures out of PDB.cpp. NFC
Introduce a new TypeMerger class, out of some type-merge-specific structures from PDB.cpp

No changes intended / this is only moving code around.

This patch is step 3. in "Proposed commit strategy" in D59226

Differential Revision: https://reviews.llvm.org/D60070

llvm-svn: 357525
2019-04-02 20:43:19 +00:00
Matthew Voss 3c023420d1 [NFC][LLD] Specify namespaces explicity to fix build failure on GCC 5 after r357383
llvm-svn: 357421
2019-04-01 19:23:56 +00:00
Peter Collingbourne a9e847238e ELF: Perform per-section .ARM.exidx processing during combineEhFrameSections(). NFCI.
And rename the function to combineEhSections(). This makes the processing
of .ARM.exidx even more similar to .eh_frame and means that we can avoid an
additional loop over InputSections.

Differential Revision: https://reviews.llvm.org/D60026

llvm-svn: 357417
2019-04-01 18:01:18 +00:00
Alexandre Ganea 30c2f20e55 Fix builder.
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fuzzer/builds/24702/steps/check-fuzzer/logs/stdio

llvm-svn: 357391
2019-04-01 14:37:36 +00:00
Alexandre Ganea bf55c4e3e3 [LLD][COFF] Early dependency detection
We introduce a new class hierarchy for debug types merging (in DebugTypes.h). The end-goal is to parallelize the type merging - please see the plan in D59226.

Previously, dependency discovery was done on the fly, much later, during the type merging loop. Unfortunately, parallelizing the type merging requires the dependencies to be merged in first, before any dependent ObjFile, thus this early discovery.

The overall intention for this path is to discover debug information dependencies at a much earlier stage, when processing input files. Currently, two types of dependency are supported: PDB type servers (when compiling with MSVC /Zi) and precompiled headers OBJs (when compiling with MSVC /Yc and /Yu). Once discovered, an explicit link is added into the dependent ObjFile, through the new debug types class hierarchy introduced in DebugTypes.h.

Differential Revision: https://reviews.llvm.org/D59053

llvm-svn: 357383
2019-04-01 13:36:59 +00:00
Fangrui Song d83fb24533 [ELF] Rename SyntheticSection::empty to more appropriate isNeeded() with opposite meaning
Summary:
Some synthetic sections can be empty while still being needed, thus they
can't be removed by removeUnusedSyntheticSections(). Rename this member
function to more appropriate isNeeded() with the opposite meaning.

No functional change intended.

Reviewers: ruiu, espindola

Reviewed By: ruiu

Subscribers: jhenderson, grimar, emaste, arichardson, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59982

llvm-svn: 357377
2019-04-01 08:16:08 +00:00
Rui Ueyama a77ea59c4d Simplify. NFC.
llvm-svn: 357373
2019-04-01 00:25:17 +00:00
Rui Ueyama 68b9f45fee Replace `typedef A B` with `using B = A`. NFC.
I did this using Perl.

Differential Revision: https://reviews.llvm.org/D60003

llvm-svn: 357372
2019-04-01 00:11:24 +00:00
Sam Clegg e3a845e25e Re-land "[WebAssembly] Improve invalid relocation error message""
See https://reviews.llvm.org/D59860

The initial version of this change effected more than just the
error message.  This version is scoped down to only effect the error
itself.

llvm-svn: 357328
2019-03-29 22:56:39 +00:00
Alexandre Ganea b13f064b5d Fix build following r357308 : Ensure only live thunks are considered when creating import modules
llvm-svn: 357316
2019-03-29 21:24:19 +00:00
Reid Kleckner ba708619ad Don't copy the .drective section with std::string
Both COFF and bitcode input files expose these as stable strings.

llvm-svn: 357314
2019-03-29 21:00:22 +00:00
Thomas Lively 06391f34bd [WebAssembly] "atomics" feature requires shared memory
Summary:
Makes it a linker error if the "atomics" feature is used but the user
does not opt in to shared memory or if "atomics" is disallowed but the
user does opt in to shared memory. Also check that an appropriate max
memory size is supplied if shared memory is used.

Reviewers: sbc100, aheejin

Subscribers: dschuff, jgravelle-google, sunfish, jfb, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59281

llvm-svn: 357310
2019-03-29 20:43:49 +00:00
Alexandre Ganea 09cca5b243 [LLD][COFF] Generate import modules & COFF groups in PDB
Generate import modules for each imported DLL, along with its symbol stream.
Also create COFF groups in the * Linker * module, one for each PartialSection (input, unmerged sections)
Currently COFF groups are disabled for MINGW because it significantly increases PDB sizes. We could enable that later with an option.

The overall objective for this change is to support code hot patching tools. Such tools need to know the import libraries used, from the PDB alone.

Differential Revision: https://reviews.llvm.org/D54802

llvm-svn: 357308
2019-03-29 20:25:34 +00:00
Alexandre Ganea 347a45ccd5 [LLD][COFF] Improve checkFailIfMismatch()
As suggested by ruiu here (https://reviews.llvm.org/D58910#1425484), defer a call to toString(File) until it's really needed (if there's an error)

Differential Revision: https://reviews.llvm.org/D59411

llvm-svn: 357305
2019-03-29 19:58:58 +00:00
Fangrui Song 8048fe2b8c [ELF][MachO][wasm] Simplify range-style std::find{,_if} with STLExtras.h utilities. NFC
llvm-svn: 357269
2019-03-29 16:21:16 +00:00
Derek Schuff 0c9ea10530 Revert "[WebAssembly] Improve invalid relocation error message"
This reverts commit 0805ec5f7b.

llvm-svn: 357225
2019-03-29 00:05:00 +00:00
Derek Schuff ac727e8647 Revert "[WebAssembly] Improve invalid relocation error message"
Also Revert "[WebAssembly] Fix typo from rL357143"

This reverts commit 0805ec5f7b and
a9958fc30d
(rL357143 and rL357144)

They cause failures with address-taken extern (JS) functions
in emscripten.

llvm-svn: 357224
2019-03-29 00:04:58 +00:00
Rui Ueyama 6528f2eac9 Remove a duplicate assignment. NFC.
llvm-svn: 357223
2019-03-28 23:53:38 +00:00
Rumeet Dhindsa 45bd9b2567 Fixed the lld test using ld-lld command to use ld.lld instead.
Differential Revision: https://reviews.llvm.org/D59962

llvm-svn: 357216
2019-03-28 22:14:46 +00:00
Reid Kleckner 1600490af1 [COFF] Optimize range extension thunk insertion memory usage
Summary:
This avoids allocating O(#relocs) of intermediate data for each section
when range extension thunks aren't needed for that section. This also
removes a std::vector from SectionChunk, which further reduces its size.

Instead, this change adds the range extension thunk symbols to the
object files that contain sections that need extension thunks. By adding
them to the symbol table of the parent object, that means they now have
a symbol table index. Then we can then modify the original relocation,
after copying it to read-write memory, to use the new symbol table
index.

This makes linking browser_tests.exe with no PDB 10.46% faster, moving
it from 11.364s to 10.288s averaged over five runs.

Reviewers: mstorsjo, ruiu

Subscribers: aganea, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59902

llvm-svn: 357200
2019-03-28 18:30:03 +00:00
Rui Ueyama f28825bc06 Create an instance of Target after reading all input files. NFC.
This change itself doesn't mean anything, but it helps D59780 because
in patch, we don't know whether we need to create a CET-aware PLT or
not until we read all input files.

llvm-svn: 357194
2019-03-28 17:38:53 +00:00
Rui Ueyama 8521ba37d7 Make a member function a non-member function.
Since this member function doesn't use anything in the class,
it doesn't have to be a member of the class.

llvm-svn: 357193
2019-03-28 17:35:00 +00:00
Rui Ueyama 676d25ab94 De-template X86_64TargetInfo. NFC.
llvm-svn: 357191
2019-03-28 17:31:12 +00:00
Benjamin Kramer ba2ea93ad1 Make helper functions static. NFC.
llvm-svn: 357187
2019-03-28 17:18:42 +00:00
Rui Ueyama c694633a12 Make TargetInfo const. NFC.
We do not mutate a TargetInfo instance after creating it. This change
makes it explicit.

llvm-svn: 357185
2019-03-28 17:05:09 +00:00
Peter Smith 3ce9af9370 [ELF][ARM] Recommit Redesign of .ARM.exidx handling to use a SyntheticSection
Recommit r356666 with fixes for buildbot failure, as well as handling for
--emit-relocs, which we decide not to emit any relocation sections as the
table is already position independent and an offline tool can deduce the
relocations.

Instead of creating extra Synthetic .ARM.exidx sections to account for
gaps in the table, create a single .ARM.exidx SyntheticSection that can
derive the contents of the gaps from a sorted list of the executable
InputSections. This has the benefit of moving the ARM specific code for
SyntheticSections in SHF_LINK_ORDER processing and the table merging code
into the ARM specific SyntheticSection. This also makes it easier to create
EXIDX_CANTUNWIND table entries for executable InputSections that don't
have an associated .ARM.exidx section.

Fixes pr40277

Differential Revision: https://reviews.llvm.org/D59216

llvm-svn: 357160
2019-03-28 11:10:20 +00:00
Sam Clegg a9958fc30d [WebAssembly] Fix typo from rL357143
llvm-svn: 357144
2019-03-28 02:04:31 +00:00
Sam Clegg 0805ec5f7b [WebAssembly] Improve invalid relocation error message
This message now matches the equivalent message in the ELF linker.

Differential Revision: https://reviews.llvm.org/D59860

llvm-svn: 357143
2019-03-28 02:02:07 +00:00
Rui Ueyama a0a50a7a5b Inline a trivial function. NFC.
I found that hiding this particular actual expression doesn't help
readers understand the code. So I remove and inline that function.

llvm-svn: 357140
2019-03-28 01:37:48 +00:00
Rui Ueyama 432030e843 [ELF] Dump symbols ordered by profiled guided section layout to file.
Patch by Tiancong Wang.

In D36351, Call-Chain Clustering (C3) heuristic is implemented with
option --call-graph-ordering-file <file>.
This patch adds a flag --print-symbol-order=<file> to LLD, and when
specified, it prints out the symbols ordered by the heuristics to the
file. The symbols printout is helpful to those who want to understand
the heuristics and want to reproduce the ordering with
--symbol-ordering-file in later pass.

Differential Revision: https://reviews.llvm.org/D59311

llvm-svn: 357133
2019-03-27 23:52:22 +00:00
Andrew Ng e6b6ab2c66 [LLD] Restore tests that use "-" as output
No longer require workarounds for output to "-" (stdout) for
Windows. These workarounds were just hiding the actual problem which has
been fixed in r357058.

Differential Revision: https://reviews.llvm.org/D59824

llvm-svn: 357072
2019-03-27 15:30:52 +00:00
Ali Tamur cea9548028 Revert "[lld] Reapply "Prevent duplicate files in debug line header in dwarf 5.""
This reverts commit rL357020

Reason: rL357018, which this commit depend on breaks the test:
llvm/test/tools/llvm-objdump/embedded-source.test on some architectures.

llvm-svn: 357025
2019-03-26 19:57:13 +00:00
Sam Clegg 492f752969 [WebAssembly] Initial implementation of PIC code generation
This change implements lowering of references global symbols in PIC
mode.

This change implements lowering of global references in PIC mode using a
new @GOT reference type. @GOT references can be used with function or
data symbol names combined with the get_global instruction. In this case
the linker will insert the wasm global that stores the address of the
symbol (either in memory for data symbols or in the wasm table for
function symbols).

For now I'm continuing to use the R_WASM_GLOBAL_INDEX_LEB relocation
type for this type of reference which means that this relocation type
can refer to either a global or a function or data symbol. We could
choose to introduce specific relocation types for GOT entries in the
future.  See the current dynamic linking proposal:

https://github.com/WebAssembly/tool-conventions/blob/master/DynamicLinking.md

Differential Revision: https://reviews.llvm.org/D54647

llvm-svn: 357022
2019-03-26 19:46:15 +00:00
Ali Tamur 5c2f176ccc [lld] Reapply "Prevent duplicate files in debug line header in dwarf 5."
Reapply rL356943; the previous attempt was reverted because the patch rL356941
that this depended on had broken a test.

Original commit message:

[lld] Prevent duplicate files in debug line header in dwarf 5.

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59671

llvm-svn: 357020
2019-03-26 18:59:52 +00:00
Thomas Lively 82de51a3ae Reland "[WebAssembly] Add linker options to control feature checking"
Do not pipe binary data between processes in lit tests this time,
since it turns out that can break on Windows.

This reverts commit 84c8652fc3.

llvm-svn: 356975
2019-03-26 04:11:05 +00:00
Fangrui Song 210949a221 [ELF] Change GOT*_FROM_END (relative to end(.got)) to GOTPLT* (start(.got.plt))
Summary:
This should address remaining issues discussed in PR36555.

Currently R_GOT*_FROM_END are exclusively used by x86 and x86_64 to
express relocations types relative to the GOT base. We have
_GLOBAL_OFFSET_TABLE_ (GOT base) = start(.got.plt) but end(.got) !=
start(.got.plt)

This can have problems when _GLOBAL_OFFSET_TABLE_ is used as a symbol, e.g.
glibc dl_machine_dynamic assumes _GLOBAL_OFFSET_TABLE_ is start(.got.plt),
which is not true.

  extern const ElfW(Addr) _GLOBAL_OFFSET_TABLE_[] attribute_hidden;
  return _GLOBAL_OFFSET_TABLE_[0]; // R_X86_64_GOTPC32

In this patch, we

* Change all GOT*_FROM_END to GOTPLT* to fix the problem.
* Add HasGotPltOffRel to denote whether .got.plt should be kept even if
  the section is empty.
* Simplify GotSection::empty and GotPltSection::empty by setting
  HasGotOffRel and HasGotPltOffRel according to GlobalOffsetTable early.

The change of R_386_GOTPC makes X86::writePltHeader simpler as we don't
have to compute the offset start(.got.plt) - Ebx (it is constant 0).

We still diverge from ld.bfd (at least in most cases) and gold in that
.got.plt and .got are not adjacent, but the advantage doing that is
unclear.

Reviewers: ruiu, sivachandra, espindola

Subscribers: emaste, mehdi_amini, arichardson, dexonsmith, jdoerfert, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59594

llvm-svn: 356968
2019-03-25 23:46:19 +00:00
Rui Ueyama e6c24299d2 Use a class instead of lambda-based callbacks to organize garbage collector.
lld's mark-sweep garbage collector was written in the visitor pattern.
There are functions that traverses a given graph, and the functions calls
callback functions to dispatch according to node type.

The code was originaly pretty simple, and lambdas worked pretty
well. However, as we add more features to the garbage collector, that became
more like a callback hell. We now have a callback function that wraps
another callback function, for example. It is not easy to follow the flow of
the control.

This patch rewrites it as a regular class. What was once a lambda is now a
regular class member function. I think this change fixes the readability
issue.

No functionality change intended.

Differential Revision: https://reviews.llvm.org/D59800

llvm-svn: 356966
2019-03-25 23:28:47 +00:00
Ali Tamur 800fe977d0 Revert "[lld] Prevent duplicate files in debug line header in dwarf 5."
This reverts commit 5aa7478a55.

Some things are broken; I will fix and try to commit again.

llvm-svn: 356949
2019-03-25 21:06:02 +00:00
Ali Tamur 5aa7478a55 [lld] Prevent duplicate files in debug line header in dwarf 5.
Summary:
Changes to a couple of tests that will start failing after https://reviews.llvm.org/D59515 is submitted.

Reviewers: echristo, ruiu, espindola

Reviewed By: echristo, ruiu

Subscribers: emaste, arichardson, MaskRay, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59671

llvm-svn: 356943
2019-03-25 20:18:27 +00:00
Thomas Lively 84c8652fc3 Revert "[WebAssembly] Add linker options to control feature checking"
This reverts commit 5991328c96.

llvm-svn: 356932
2019-03-25 18:10:26 +00:00
Alexandre Ganea 74d5b33222 [LLD][COFF] Separate module descriptors creation from type/symbol merging
Take module DBI creation out of PDBLinker::addObjFile() into its own function.

This is groundwork towards parallelizable type merging, as proposed in D59226.

Differential Revision: https://reviews.llvm.org/D59261

llvm-svn: 356815
2019-03-22 22:07:27 +00:00
Rui Ueyama d2e0ed7755 Simplify PltSection.
Previously, `Entries` contains pairs of symbols and their indices.
The indices are always 0, x, 2x, 3x, ..., where x is the size of
relocation entry. We didn't have to store that values because we can
compute them when we consume them.

llvm-svn: 356812
2019-03-22 21:17:25 +00:00
Thomas Lively 5991328c96 [WebAssembly] Add linker options to control feature checking
Summary:
Adds --check-features and --no-check-features. The default for now is
to enable the checking, but this might change in the future.

Also adds --features=foo,bar for precisely controlling the features
used in the output binary.

Depends on D59173.

Reviewers: sbc100, aheejin

Subscribers: dschuff, jgravelle-google, sunfish, jdoerfert, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59274

llvm-svn: 356805
2019-03-22 20:43:06 +00:00
Fangrui Song dc6c0cf94d [DWARF] Refactor RelocVisitor and fix computation of SHT_RELA-typed relocation entries
Summary: This is the lld-side change of D57939

Reviewers: echristo, dblaikie, ruiu, espindola

Reviewed By: echristo, ruiu

Subscribers: jdoerfert, emaste, arichardson, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D57940

llvm-svn: 356730
2019-03-22 02:43:17 +00:00
Reid Kleckner 05ea3a6be3 Fix lld wasm tests after r356610
Apparently stdout is not opened in binary mode, so the executable gets
corrupted when piping to obj2yaml.

llvm-svn: 356694
2019-03-21 18:24:05 +00:00
Peter Smith 54dab70bb7 [ELF][ARM] Revert Redesign of .ARM.exidx handling to use a SyntheticSection
There is a reproducible buildbot failure (segfault) on the 2 stage
clang-cmake-armv8-lld bot. Reverting while I investigate.

Differential Revision: https://reviews.llvm.org/D59216

llvm-svn: 356684
2019-03-21 17:17:54 +00:00
Peter Smith d3511a214e [ELF][ARM] Redesign of .ARM.exidx handling to use a SyntheticSection
Instead of creating extra Synthetic .ARM.exidx sections to account for
gaps in the table, create a single .ARM.exidx SyntheticSection that can
derive the contents of the gaps from a sorted list of the executable
InputSections. This has the benefit of moving the ARM specific code for
SyntheticSections in SHF_LINK_ORDER processing and the table merging code
into the ARM specific SyntheticSection. This also makes it easier to create
EXIDX_CANTUNWIND table entries for executable InputSections that don't
have an associated .ARM.exidx section.

Fixes pr40277

Differential Revision: https://reviews.llvm.org/D59216

llvm-svn: 356666
2019-03-21 14:06:40 +00:00
Thomas Lively f6f4f84378 [WebAssembly] Target features section
Summary:
Implements a new target features section in assembly and object files
that records what features are used, required, and disallowed in
WebAssembly objects. The linker uses this information to ensure that
all objects participating in a link are feature-compatible and records
the set of used features in the output binary for use by optimizers
and other tools later in the toolchain.

The "atomics" feature is always required or disallowed to prevent
linking code with stripped atomics into multithreaded binaries. Other
features are marked used if they are enabled globally or on any
function in a module.

Future CLs will add linker flags for ignoring feature compatibility
checks and for specifying the set of allowed features, implement using
the presence of the "atomics" feature to control the type of memory
and segments in the linked binary, and add front-end flags for
relaxing the linkage policy for atomics.

Reviewers: aheejin, sbc100, dschuff

Subscribers: jgravelle-google, hiraditya, sunfish, mgrang, jfb, jdoerfert, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59173

llvm-svn: 356610
2019-03-20 20:26:45 +00:00
Robert Lougher 364cb6b5d7 [TailCallElim] Update tests for LTO pipeline change
D58391 changed the LTO pipelines to add the tailcall elimination pass.
This caused three LLD tests to fail.

Differential Revision: https://reviews.llvm.org/D59604

llvm-svn: 356593
2019-03-20 19:05:05 +00:00
Serge Guelton a1aa7a4054 Python 2/3 compatibility
This should be the only change required to have lld's python code base compatible with both Python 2 and Python 3

Differential Revision: https://reviews.llvm.org/D59538

llvm-svn: 356538
2019-03-20 07:42:13 +00:00
Fangrui Song 1092fc9057 [ELF] Allow sh_entsize to be unrelated to sh_addralign and not a power of 2
Summary:
This implements Rui Ueyama's idea in PR39044.
I've checked that ld.bfd and gold do not have the power-of-2 requirement
and do not require sh_entsize to be a multiple of sh_align.

Now on the updated test merge-entsize.s, all the 3 linkers happily
create .rodata that is not 3-byte aligned.

This has a use case in Linux arch/x86/crypto/sha512-avx2-asm.S
It uses sh_entsize of 640, which is not a power of 2.
See https://github.com/ClangBuiltLinux/linux/issues/417

Reviewers: ruiu, espindola

Reviewed By: ruiu

Subscribers: nickdesaulniers, E5ten, emaste, arichardson, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59478

llvm-svn: 356428
2019-03-18 23:49:18 +00:00
Alexandre Ganea 4aeea4cc42 [DebugInfo][PDB] Don't write empty debug streams
Before, empty debug streams were written as 8 bytes (4 bytes signature + 4 bytes for the GlobalRefs count).

With this patch, unused empty streams aren't emitted anymore. Modules now encode 65535 as an 'unused stream' value, by convention.
Also fix the * Linker * contrib section which wasn't correctly emitted previously.

Differential Revision: https://reviews.llvm.org/D59502

llvm-svn: 356395
2019-03-18 19:13:23 +00:00
Nico Weber 2b1dca79f2 lld-link: Run conflict-mangled.test on all systems
It seems to pass fine on my Mac, and it running it only on Windows made
me miss it in r355959 and required r355959.

When the test was added in r288992 we still used Win-only
UnDecorateSymbolName() for demangling. Now we use LLVM's
microsoftDemangle() which is cross-platform.

Differential Revision: https://reviews.llvm.org/D59497

llvm-svn: 356380
2019-03-18 16:51:23 +00:00
Siva Chandra 1915e2be93 [ELF] Emit weak-undef symbols in .dynsym of a PIE binary only if linked against shared libs.
Reviewers: espindola

Subscribers: emaste, arichardson, MaskRay, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59275

llvm-svn: 356374
2019-03-18 15:32:57 +00:00
George Rimar 738146ab33 [LLD][ELF] - Replace one of the tests with a YAML version.
This removes one more binary from the inputs.

Differential revision: https://reviews.llvm.org/D59085

llvm-svn: 356334
2019-03-17 15:46:03 +00:00
Fangrui Song 6778b53e95 [ELF] De-virtualize findOrphanPos, excludeLibs and handleARMTlsRelocation
llvm-svn: 356331
2019-03-17 13:53:42 +00:00
Sam Clegg 632c217921 [WebAssembly] Error on R_WASM_MEMORY_ADDR relocations against undefined data symbols.
For these types of relocations an absolute memory address is
required which is not possible for undefined data symbols.  For symbols
that can be undefined at link time (i.e. external data symbols in
shared libraries) a different type of relocation (i.e. via a GOT) will
be needed.

Differential Revision: https://reviews.llvm.org/D59337

llvm-svn: 356310
2019-03-16 01:18:12 +00:00