Commit Graph

12523 Commits

Author SHA1 Message Date
Fangrui Song 89f7859641 [ELF][test] Improve LTO tests
Add file-level comments
Delete insignificant addresses to make them more tolerant to layout changes
Simplify test output

llvm-svn: 371292
2019-09-07 08:20:09 +00:00
Matthew Voss c177919409 Update lld tests dynamic-list.s and symbol-override.s to use llvm-nm
The following tests failed on Windows bots due to nm not being
available:

  lld/test/ELF/dynamic-list.s
  lld/test/ELF/symbol-override.s

llvm-svn: 371267
2019-09-06 22:51:46 +00:00
Fangrui Song 2682bc3c9d [ELF] Replace error() with errorOrWarn() for the ASSERT command
Summary:
ld.bfd produces an output with --noinhibit-exec when an ASSERT fails.
Use errorOrWarn() so that we can produce an output as well.

An interesting case is that symbol assignments may execute multiple
times, so we probably want to suppress errors for non-final runs.

Reviewed By: peter.smith

Differential Revision: https://reviews.llvm.org/D67285

llvm-svn: 371225
2019-09-06 16:30:22 +00:00
Fangrui Song 8d30c1dcec Reland D66717 [ELF] Do not ICF two sections with different output sections (by SECTIONS commands)
Recommit r370635 (reverted by r371202), with one change: move addOrphanSections() before ICF.

Before, orphan sections in two different partitions may be folded and
moved to the main partition.

Now, InputSection->OutputSection assignment for orphans happens before
ICF. ICF does not fold input sections with different output sections.

With the PR43241 reproduce,
`llvm-objcopy --extract-partition libvr.so libchrome__combined.so libvr.so` => no error

Updated description:

Fixes PR39418. Complements D47241 (the non-linker-script case).

processSectionCommands() assigns input sections to output sections.
ICF is called before it, so .text.foo and .text.bar may be folded even if
their output sections are made different by SECTIONS commands.

```
markLive<ELFT>()
doIcf<ELFT>()                      // During ICF, we don't know the output sections
writeResult()
  combineEhSections<ELFT>()
  script->processSectionCommands() // InputSection -> OutputSection assignment
```

This patch splits processSectionCommands() into processSectionCommands()
and processSymbolAssignments(), and moves
processSectionCommands()/addOrphanSections() before ICF:

```
markLive<ELFT>()
combineEhSections<ELFT>()
script->processSectionCommands()
script->addOrphanSections();
doIcf<ELFT>()                      // should remove folded input sections
writeResult()
  script->processSymbolAssignments()
```

An alternative approach is to unfold a section `sec` in
processSectionCommands() when we find `sec` and `sec->repl` belong to
different output sections. I feel this patch is superior because this
can fold more sections and the decouple of
SectionCommand/SymbolAssignment gives flexibility:

* An ExprValue can't be evaluated before its section is assigned to an
  output section -> we can delete getOutputSectionVA and simplify
  another place where we had to check if the output section is null.
  Moreover, a case in linkerscript/early-assign-symbol.s can be handled
  now.
* processSectionCommands/processSymbolAssignments can be freely moved
  around.

llvm-svn: 371216
2019-09-06 15:57:44 +00:00
Fangrui Song 5d9f419a2e Revert "Revert r370635, it caused PR43241."
This reverts commit 50d2dca22b3b05d0ee4883b0cbf93d7d15f241fc.

llvm-svn: 371215
2019-09-06 15:57:24 +00:00
Fangrui Song 8f10a5f42d [ELF][test] Simplify and reorganize tests
Add file-level comments
Delete insignificant addresses to make them more tolerant to layout changes
Simplify test output
Delete simple Inputs/*.s files

Delete version-script-copy-rel.s - covered by verdef-defaultver.s
Delete version-wildcard.test - covered by version-script-glob.s

llvm-svn: 371213
2019-09-06 15:10:31 +00:00
Nico Weber 8455294f2a Revert r370635, it caused PR43241.
llvm-svn: 371202
2019-09-06 13:23:42 +00:00
Fangrui Song 70e002b50b [ELF][test] Update test after r371185
llvm-svn: 371189
2019-09-06 09:39:48 +00:00
Simon Atanasyan ff2172c9f4 [mips] Make another set of test cases more tolerant to exact symbol addresses. NFC
llvm-svn: 371174
2019-09-06 07:23:02 +00:00
Fangrui Song a2028f73c2 Update SHT_LLVM_PART_EHDR test after r371157
llvm-svn: 371160
2019-09-06 01:18:01 +00:00
Fangrui Song 6dc2bd70bb [ELF] Initialize PhdrEntry::p_align to maxPageSize for PT_LOAD
```
Writer<ELFT>::run
  assignFileOffsets
    setFileOffset
      computeFileOffset
        os->ptLoad->p_align may be smaller than config->maxPageSize
  setPhdrs
    p_align = max(p_align, config->maxPageSize)
```

If we move the config->maxPageSize logic to the constructor of
PhdrEntry, computeFileOffset can be simplified.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D67211

llvm-svn: 371085
2019-09-05 16:32:31 +00:00
Simon Atanasyan 2172f3f49d [mips] Make a few test cases more tolerant to exact symbol addresses. NFC
llvm-svn: 371065
2019-09-05 13:37:16 +00:00
Rui Ueyama e99dc4ba57 Align output segments correctly
Previously, segments were aligned according to their first section's
alignment requirements. That was not correct, but segments are also
aligned to a page boundary, and a page boundary is usually much larger
than a section alignment requirement, so no one noticed this bug before.

Now, lld has --nmagic option which sets maxPageSize to 1 to effectively
disable page alignment, which reveals the issue.

Fixes https://bugs.llvm.org/show_bug.cgi?id=43212

Differential Revision: https://reviews.llvm.org/D67152

llvm-svn: 371013
2019-09-05 05:30:24 +00:00
Martin Storsjo d581dd5013 [LLD] [COFF] Implement MinGW default manifest handling
In mingw environments, resources are normally compiled to resource
object files directly, instead of letting the linker convert them to
COFF format.

Since some time, GCC supports the notion of a default manifest object.
When invoking the linker, GCC looks for the default manifest object
file, and if found in the expected path, it is added to linker commands.

The default manifest is one that indicates support for the latest known
versions of windows, to implicitly unlock the modern behaviours of certain
APIs.

Not all mingw/gcc distributions include this file, but e.g. in msys2,
the default manifest object is distributed in a separate package (which
can be but might not always be installed).

This means that even if user projects only use one single resource
object file, the linker can end up with two resource object files,
and thus needs to support merging them.

The default manifest has a language id of zero, and GNU ld has got
logic for dropping a manifest with a zero language id, if there's
another manifest present with a nonzero language id. If there are
multiple manifests with a nonzero language id, the merging process
errors out.

Differential Revision: https://reviews.llvm.org/D66825

llvm-svn: 370974
2019-09-04 20:34:00 +00:00
Thomas Lively 09768c5d7a [WebAssembly] Initialize memory in start function
Summary:
 - `__wasm_init_memory` is now the WebAssembly start function instead
   of being called from `__wasm_call_ctors` or called directly by the
   runtime.
 - Adds a new synthetic data symbol `__wasm_init_memory_flag` that is
   atomically incremented from zero to one by the thread responsible
   for initializing memory.
 - All threads now unconditionally perform data.drop on all passive
   segments.
 - Removes --passive-segments and --active-segments flags and controls
   segment type based on --shared-memory instead. The deleted flags
   were only present to ameliorate the upgrade path in Emscripten.

Reviewers: sbc100, aheejin

Subscribers: dschuff, jgravelle-google, sunfish, jfb, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D65783

llvm-svn: 370965
2019-09-04 19:50:39 +00:00
Fangrui Song 7afffb54ea [ELF] Don't shrink RelrSection
Fixes PR43214.

The size of SHT_RELR may oscillate between 2 numbers (see D53003 for a
similar --pack-dyn-relocs=android issue). This can happen if the shrink
of SHT_RELR causes it to take more words to encode relocation offsets
(this can happen with thunks or segments with overlapping p_offset
ranges), and the expansion of SHT_RELR causes it to take fewer words to
encode relocation offsets.

To avoid the issue, add padding 1s to the end of the relocation section
if its size would decrease. Trailing 1s do not decode to more relocations.

Reviewed By: peter.smith

Differential Revision: https://reviews.llvm.org/D67164

llvm-svn: 370923
2019-09-04 16:27:35 +00:00
Fangrui Song 520bdf79b5 [ELF] Fix spell corrector: don't call elf::InputFile::getSymbols() on shared objects
Exposed by pr34872.s

llvm-svn: 370875
2019-09-04 11:02:58 +00:00
Fangrui Song b4745fad24 [ELF] Add a spell corrector for "undefined symbol" diagnostics
Non-undefined symbols with Levenshtein distance 1 or a transposition are
suggestion candidates. This is probably good enough and it can suggest
some missing/superfluous qualifiers: const, restrict, volatile, & and &&
ref-qualifier, e.g.

   error: undefined symbol: foo(int*)
   >>> referenced by b.o:(.text+0x1)
  +>>> did you mean: foo(int const*)
  +>>> defined in: a.o

   error: undefined symbol: foo(int*&)
   >>> referenced by b.o:(.text+0x1)
  +>>> did you mean: foo(int*)
  +>>> defined in: b.o

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D67039

llvm-svn: 370853
2019-09-04 09:04:26 +00:00
Bob Haarman 7dc5e7a0a4 reland "[lld-link] implement -start-lib and -end-lib"
Summary:
This is a re-land of r370487 with a fix for the use-after-free bug
that rev contained.

This implements -start-lib and -end-lib flags for lld-link, analogous
to the similarly named options in ld.lld. Object files after
-start-lib are included in the link only when needed to resolve
undefined symbols. The -end-lib flag goes back to the normal behavior
of always including object files in the link. This mimics the
semantics of static libraries, but without needing to actually create
the archive file.

Reviewers: ruiu, smeenai, MaskRay

Reviewed By: ruiu, MaskRay

Subscribers: akhuang, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D66848

llvm-svn: 370816
2019-09-03 20:32:16 +00:00
Ed Maste 5289bbe4d4 ld.lld.1: explain long options may use one or two dashes
Obtained from FreeBSD r329003

llvm-svn: 370800
2019-09-03 17:58:30 +00:00
Ed Maste 174e083345 ld.lld.1: stylistic changes suggested by igor
igor is an automated man page "proofreader" from FreeBSD - see
http://www.wonkity.com/~wblock/igor/igor.pdf

No content change.

llvm-svn: 370799
2019-09-03 17:58:24 +00:00
Martin Storsjo a66fc1c99f [LLD] [COFF] Demangle itanium symbols in mingw mode
Differential Revision: https://reviews.llvm.org/D67051

llvm-svn: 370654
2019-09-02 13:25:46 +00:00
Fangrui Song d8bc6a48ea [ELF] Do not ICF two sections with different output sections (by SECTIONS commands)
Fixes PR39418. Complements D47241 (the non-linker-script case).

processSectionCommands() assigns input sections to output sections.
ICF is called before it, so .text.foo and .text.bar may be folded even if
their output sections are made different by SECTIONS commands.

```
markLive<ELFT>()
doIcf<ELFT>()                      // During ICF, we don't know the output sections
writeResult()
  combineEhSections<ELFT>()
  script->processSectionCommands() // InputSection -> OutputSection assignment
```

This patch splits processSectionCommands() into processSectionCommands() and
processSymbolAssignments(), and moves processSectionCommands() before ICF:

```
markLive<ELFT>()
combineEhSections<ELFT>()
script->processSectionCommands()
doIcf<ELFT>()                      // should remove folded input sections
writeResult()
  script->processSymbolAssignments()
```

An alternative approach is to unfold a section `sec` in
processSectionCommands() when we find `sec` and `sec->repl` belong to
different output sections. I feel this patch is superior because this
can fold more sections and the decouple of
SectionCommand/SymbolAssignment gives flexibility:

* An ExprValue can't be evaluated before its section is assigned to an
  output section -> we can delete getOutputSectionVA and simplify
  another place where we had to check if the output section is null.
  Moreover, a case in linkerscript/early-assign-symbol.s can be handled
  now.
* processSectionCommands/processSymbolAssignments can be freely moved
  around.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D66717

llvm-svn: 370635
2019-09-02 10:33:58 +00:00
Fangrui Song 4514ac7cfb [ELF] Align SHT_LLVM_PART_EHDR to a maximum page size boundary
Fixes https://bugs.chromium.org/p/chromium/issues/detail?id=998712

SHT_LLVM_PART_EHDR marks the start of a partition. The partition
sections will be extracted to a separate file. Align to the next maximum
page size boundary so that we can find the ELF header at the start. We
cannot benefit from overlapping p_offset ranges with the previous
segment anyway.

It seems we lack some llvm-objcopy --extract-main-partition and
--extract-partition sanity checks. It may place EHDR at the start
even if p_offset if non zero. Anyway, the lld change is justified for
the reasons above.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D67032

llvm-svn: 370629
2019-09-02 08:49:50 +00:00
Vlad Tsyrklevich 802aab5de8 Revert "[lld-link] implement -start-lib and -end-lib"
This reverts commit r370487 as it is causing ASan/MSan failures on
sanitizer-x86_64-linux-fast

llvm-svn: 370550
2019-08-30 23:24:41 +00:00
Sam Clegg fecfc5920a [lld][WebAssembly] Fix spurious signature mismatch warnings
Summary:
This a follow up on: https://reviews.llvm.org/D62153

Handle the case where there are multiple object files that contain
undefined references to the same function.  We only generate a function
variant if the existing symbol is directly called.

See: https://github.com/emscripten-core/emscripten/issues/8995

Subscribers: dschuff, jgravelle-google, aheejin, sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D67015

llvm-svn: 370509
2019-08-30 19:50:59 +00:00
Martin Storsjo 8c189e50c7 [LLD] [COFF] Add a missing REQUIRES line to a recently added test. NFC.
This should fix failing buildbots like
http://lab.llvm.org:8011/builders/clang-cmake-aarch64-lld/builds/7180.

llvm-svn: 370491
2019-08-30 17:26:54 +00:00
Bob Haarman fd7569c8e3 [lld-link] implement -start-lib and -end-lib
Summary:
This implements -start-lib and -end-lib flags for lld-link, analogous
to the similarly named options in ld.lld. Object files after
-start-lib are included in the link only when needed to resolve
undefined symbols. The -end-lib flag goes back to the normal behavior
of always including object files in the link. This mimics the
semantics of static libraries, but without needing to actually create
the archive file.

Reviewers: ruiu, smeenai, MaskRay

Reviewed By: ruiu, MaskRay

Subscribers: akhuang, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D66848

llvm-svn: 370487
2019-08-30 16:50:10 +00:00
Fangrui Song 688183ec54 [ELF] Set `referenced` bit of Undefined created by BitcodeFile
D64136 and D65584, while fixing STB_WEAK issues and improving our
compatibility with ld.bfd, can cause another STB_WEAK problem related to
LTO:

If %tundef.o has an undefined reference on f,
and %tweakundef.o has a weak undefined reference on f,
%tdef.o has a definition of f

```
ld.lld %tundef.o %tweakundef.o --start-lib %tdef.o --end-lib
```

1) `%tundef.o` doesn't set the `referenced` bit.
2) `%weakundef.o` changes the binding from STB_GLOBAL to STB_WEAK
3) `%tdef.o` is not fetched because the binding is weak.

Step (1) is incorrect. This patch sets the `referenced` bit of Undefined
created by bitcode files.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D66992

llvm-svn: 370437
2019-08-30 07:10:30 +00:00
Martin Storsjo 3d3a9b3b41 [LLD] [COFF] Support merging resource object files
Extend WindowsResourceParser to support using a ResourceSectionRef for
loading resources from an object file.

Only allow merging resource object files in mingw mode; keep the
existing error on multiple resource objects in link mode.

If there only is one resource object file and no .res resources,
don't parse and recreate the .rsrc section, but just link it in without
inspecting it. This allows users to produce any .rsrc section (outside
of what the parser supports), just like before. (I don't have a specific
need for this, but it reduces the risk of this new feature.)

Separate out the .rsrc section chunks in InputFiles.cpp, and only include
them in the list of section chunks to link if we've determined that there
only was one single resource object. (We need to keep other chunks from
those object files, as they can legitimately contain other sections as
well, in addition to .rsrc section chunks.)

Differential Revision: https://reviews.llvm.org/D66824

llvm-svn: 370436
2019-08-30 06:56:33 +00:00
Dan Gohman 7cb9c8a506 [WebAssembly] Implement NO_STRIP
This patch implements support for the NO_STRIP flag, which will allow
__attribute__((used)) to be implemented.

This accompanies https://reviews.llvm.org/D62542, which moves to setting the
NO_STRIP flag, and will continue to set EXPORTED for Emscripten targets for
compatibility.

Differential Revision: https://reviews.llvm.org/D66968

llvm-svn: 370416
2019-08-29 22:41:05 +00:00
Nico Weber 3c1996a489 lld: Make a test not fail if "repro" is part of the build directory name
r268231 made it so that the name of the --reproduce archive
is no longer listed in the response file. Previously, with
"--reproduce repro.tar" the response file would contain

  repro/home/.../llvm-build-dir/.../foo.o

but after that change it contained

  home/.../llvm-build-dir/.../foo.o

instead. The test added for this in r268231 checked that
the response file doesn't contain the string "repro", but
if the build dir is named e.g. "llvm-build-repro" then the
test fails because of that.

Change the assert to check that "repro" doesn't exist at the
beginning of the line instead. I verified that the test
still fails with r268231 reverted.

The test technically still fails if someone builds llvm in a directory
'/repro' below the root directory. Don't do that :)

llvm-svn: 370211
2019-08-28 14:33:35 +00:00
Fangrui Song 523f999acf [ELF][RISCV] Allow PT_LOAD to have overlapping p_offset ranges on EM_RISCV
Port the D64906 technique to RISC-V. It deletes 3 alignments at
PT_LOAD boundaries for the default case: the size of a RISC-V binary
decreases by at most 12kb.

llvm-svn: 370192
2019-08-28 12:06:06 +00:00
Rui Ueyama 91864f82c7 [mach-o] Extend LC_DATA_IN_CODE support to x86_64
Patch by LemonBoy

Differential Revision: https://reviews.llvm.org/D62185

llvm-svn: 370183
2019-08-28 10:11:50 +00:00
Fangrui Song 54a6f6839b [ELF][AMDGPU][SPARC] Allow PT_LOAD to have overlapping p_offset ranges on EM_AMDGPU and EM_SPARCV9
llvm-svn: 370180
2019-08-28 09:45:06 +00:00
Fangrui Song 8fbe81fb29 [ELF][RISCV] Assign st_shndx of __global_pointer$ to 1 if .sdata does not exist
This essentially reverts the code change of D63132 and switches to a simpler approach.

In an executable/shared object, st_shndx of a symbol can be:

1) SHN_UNDEF: undefined symbol (or canonical PLT)
2) SHN_ABS: absolute symbol
3) any other value (usually a regular section index) represents a relative symbol.
  The actual value does not matter.

Many ld.so (musl, all archs except MIPS of FreeBSD rtld-elf) even treat 2) and 3)
the same. If .sdata does not exist, it does not matter what value/section
__global_pointer$ has, as long as it is relative (otherwise there will be a pedantic
lld error. See D63132). Just set the st_shndx arbitrarily to 1.

Dummy st_shndx=1 may be used by __rela_iplt_start, linker-script-defined symbols outside a section, __dso_handle, etc.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D66798

llvm-svn: 370172
2019-08-28 09:01:03 +00:00
Vlad Tsyrklevich 57076d3199 Revert "Change the X86 datalayout to add three address spaces for 32 bit signed,"
This reverts commit r370083 because it caused check-lld failures on
sanitizer-x86_64-linux-fast.

llvm-svn: 370142
2019-08-28 01:08:54 +00:00
Jacob Gravelle 92ed86d239 [lld][WebAssembly] Support for growable tables
Adds --growable-table flag to handle building wasm modules with tables
that can grow.

Wasm tables that we use to store function pointers. In order to add functions
to that table at runtime, we need to either preallocate space, or grow the table.
In order to specify a table with no maximum size, we need some flag to handle
that case, separately from a potential --max-table-size= flag.

Note that the number of elements in the table isn't knowable until link-time,
so it's unclear if we will want a --max-table-size= flag in the future.

llvm-svn: 370127
2019-08-27 22:58:21 +00:00
Amy Huang 1299945b81 Change the X86 datalayout to add three address spaces for 32 bit signed,
32 bit unsigned, and 64 bit pointers.

llvm-svn: 370083
2019-08-27 17:46:53 +00:00
Fangrui Song 024bf27ddf [ELF][ARM] Allow PT_LOAD to have overlapping p_offset ranges on EM_ARM
Port the D64906 technique to ARM. It deletes 3 alignments at
PT_LOAD boundaries for the default case: the size of an arm binary
decreases by at most 12kb.

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D66749

llvm-svn: 370049
2019-08-27 11:52:36 +00:00
Fangrui Song fdbc2bd2aa [ELF][ARM] Add --no-show-raw-insn and -soname to some ARM tests
Delete some insignificant addresses to make it simpler for layout
changes.

llvm-svn: 370048
2019-08-27 11:50:27 +00:00
Sam Clegg 040ef1091d [lld][WebAssembly] Create optional symbols after handling --export/--undefined
Handling of --export/--undefined can pull in lazy symbols which in turn
can pull in referenced to optional symbols.  We need to delay the
creation of optional symbols until all possible references to them have
been created.

Differential Revision: https://reviews.llvm.org/D66768

llvm-svn: 370012
2019-08-27 04:27:57 +00:00
Sam Clegg 1a1df72a43 [lld][WebAssembly] Store table base in config rather than passing it around. NFC.
I've got another change that makes more use of this value in other
places.

Differential Revision: https://reviews.llvm.org/D66777

llvm-svn: 370010
2019-08-27 04:19:34 +00:00
Richard Trieu f837bb4a34 Copy test data so tests don't traverse test directories. NFC
llvm-svn: 369984
2019-08-26 22:41:05 +00:00
Fangrui Song 1681ceb2c4 [ELF] EhFrameSection: postpone FDE liveness check to finalizeSections
EhFrameSection::addSection checks liveness of FDE early. This makes it
infeasible to move combineEhSections() before ICF.

Postpone the check to EhFrameSection::finalizeContents(). This is what
ARMExidxSyntheticSection does and it will make a subsequent patch D66717
simpler.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D66727

llvm-svn: 369890
2019-08-26 10:32:12 +00:00
Fangrui Song debcac9fef [ELF] Make LinkerScript::assignAddresses iterative
PR42990. For `SECTIONS { b = a; . = 0xff00 + (a >> 8); a = .; }`,
we currently set st_value(a)=0xff00 while st_value(b)=0xffff.

The following call tree demonstrates the problem:

```
link<ELF64LE>(Args);
  Script->declareSymbols(); // insert a and b as absolute Defined
  Writer<ELFT>().run();
    Script->processSectionCommands();
      addSymbol(cmd);       // a and b are re-inserted. LinkerScript::getSymbolValue
                            // is lazily called by subsequent evaluation
    finalizeSections();
      forEachRelSec(scanRelocations<ELFT>);
        processRelocAux     // another problem PR42506, not affected by this patch
      finalizeAddressDependentContent(); // loop executed once
        script->assignAddresses(); // a = 0, b = 0xff00
    script->assignAddresses(); // a = 0xff00, _end = 0xffff
```

We need another assignAddresses() to finalize the value of `a`.

This patch

1) modifies assignAddress() to track the original section/value of each
  symbol and return a symbol whose section/value has changed.
2) moves the post-finalizeSections assignAddress() inside the loop
  of finalizeAddressDependentContent() and makes it iterative.
  Symbol assignment may not converge so we make a few attempts before
  bailing out.

Note, assignAddresses() must be called at least twice. The penultimate
call finalized section addresses while the last finalized symbol values.
It is somewhat obscure and there was no comment.
linkerscript/addr-zero.test tests this.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D66279

llvm-svn: 369889
2019-08-26 10:23:31 +00:00
Fangrui Song 8e5184af71 [ELF] Error if --strip-all and --emit-relocs are used together
--strip-all suppresses the creation of in.symtab
This can cause a null pointer dereference in OutputSection::finalize()

  // --emit-relocs => copyRelocs is true
  if (!config->copyRelocs || (type != SHT_RELA && type != SHT_REL))
    return;
  ...
  link = in.symTab->getParent()->sectionIndex; // in.symTab is null

Let's just disallow the combination. In some cases the combination can
cause GNU linkers to fail:

* ld.bfd: final link failed: invalid operation
* gold: internal error in set_no_output_symtab_entry, at ../../gold/object.h:1814

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D66704

llvm-svn: 369878
2019-08-26 06:23:53 +00:00
Fangrui Song 76f005535a [ELF] Delete a redundant dyn_cast<InputSection>. NFC
llvm-svn: 369868
2019-08-25 14:41:18 +00:00
Fangrui Song 6d5a8c92bf [ELF] Simplify with less_second. NFC
llvm-svn: 369844
2019-08-24 08:40:20 +00:00
Fangrui Song 62083ec157 [ELF] Make member function Writer<ELFT>::removeEmptyPTLoad non-member. NFC
llvm-svn: 369838
2019-08-24 06:31:34 +00:00
Fangrui Song af47d0021c [ELF] Align the first section of a PT_LOAD even if its type is SHT_NOBITS
Reported at https://reviews.llvm.org/D64930#1642223

If the only section of a PT_LOAD is a SHT_NOBITS section (e.g. .bss), we
may not align its sh_offset. p_offset of the PT_LOAD will be set to
sh_offset, and we will get p_offset!=p_vaddr (mod p_align).  If such
executable is mapped by the Linux kernel, it will segfault.

After D64906, this may happen the non-linker script case.

The linker script case has had this issue for a long time.
This was fixed by rL321657 (but the test linkerscript/nobits-offset.s
failed to test a SHT_NOBITS section), but broken by rL345154.

Reviewed By: peter.smith

Differential Revision: https://reviews.llvm.org/D66658

llvm-svn: 369828
2019-08-24 00:41:15 +00:00
Peter Smith 7d6aa7eb7f [ELF] Mention contents of reproduce archive and add help description.
Building on D60557 mention the name of the linker generated contents of
the reproduce archive, response.txt and version.txt.

Also write a shorter description in the ld.lld --help that is closer to
the documentation.

Differential Revision: https://reviews.llvm.org/D66641

llvm-svn: 369762
2019-08-23 14:41:25 +00:00
Rui Ueyama 72d1089a3a Explain --reproduce option
I think --reproduce is no longer a debug-only option but a useful
option that a common user may want to use. So, this patch updates
the description of the option in the manual page.

Differential Revision: https://reviews.llvm.org/D60557

llvm-svn: 369740
2019-08-23 08:52:55 +00:00
Rui Ueyama 6ef01c3e2b Add a description about multiple linker scripts
Differential Revision: https://reviews.llvm.org/D66630

llvm-svn: 369737
2019-08-23 07:50:09 +00:00
Benjamin Kramer b3a991df3c Fight a bit against global initializers. NFC.
llvm-svn: 369695
2019-08-22 19:43:27 +00:00
Amy Huang a1c022c791 [COFF] Add libcall symbols to the link when LTO is being used
llvm-svn: 369694
2019-08-22 19:40:07 +00:00
Bob Haarman 5375b94e36 [lld-link] implement -lto-obj-path
Summary:
This adds the -lto-obj-path option to lld-link. This can be
used to specify a path at which to write a native object file for
the full LTO part when using LTO unit splitting.

Reviewers: ruiu, tejohnson, pcc, rnk

Reviewed By: ruiu, rnk

Subscribers: mehdi_amini, steven_wu, dexonsmith, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D65964

llvm-svn: 369559
2019-08-21 18:24:59 +00:00
Fangrui Song 2d37bf843c [ELF][ARM] Simplify some llvm-objdump tests with both ARM/Thumb states
llvm-objdump can switch between ARM/Thumb states after D60927.

In a few lld tests, we run both

* llvm-objdump -d -triple=thumbv7a-none-linux-gnueabi %t
* llvm-objdump -d -triple=armv7a-none-linux-gnueabi %t

to test ARM/Thumb parts of the same file. In many cases we can just
run one command. There is a problem that prevents us from cleaning
more tests (e.g. test/ELF/arm-thumb-interwork-thunk.s):

In llvm-objdump, while we have ARM/Thumb (primary and secondary)
MCDisassembler and MCSubtargetInfo, we have just one MCInstrAnalysis
which is used to resolve the targets of calls in both ARM/Thumb parts.

    // ThumbMCInstrAnalysis evaluating ARM parts or ARMMCInstrAnalysis evaluating Thumb parts
    // will have incorrect offsets.
    // An example of llvm-objdump -d -triple=thumbv7a on ARM part:
    1304: 3d ff ff fa  blx     #-780                 # no <...>
    1308: 06 00 00 ea  b       #24 <arm_caller+0x24> # wrong target due to wrong offset

Reviewed By: peter.smith

Differential Revision: https://reviews.llvm.org/D66539

llvm-svn: 369535
2019-08-21 15:30:55 +00:00
George Rimar a50f115d36 [LLD][ELF] - Simplify the bad-archive.s test case.
This removes the precompiled binary and improves the
check of the error reported.

Differential revision: https://reviews.llvm.org/D66523

llvm-svn: 369516
2019-08-21 11:19:50 +00:00
Fangrui Song 2d337fdc95 Reland D65242 "[ELF] More dynamic relocation packing""
This fixed a bug in r369488. When config->isRela is false, i->r_addend
is not initialized (see encodeDynamicReloc). So we should check
config->isRela before accessing r_addend:

- if (j - i < 3 || i->r_addend)
+ if (j - i < 3 || (config->isRela && i->r_addend != 0))

Original description:

Currently, with Android dynamic relocation packing, only relative
relocations are grouped together. This patch implements similar
packing for non-relative relocations.

The implementation groups non-relative relocations with the same
r_info and r_addend, if using RELA. By requiring a minimum group
size of 3, this achieves smaller relocation sections. Building Android
for an ARM32 device, I see the total size of /system/lib decrease by
392 KB.

Grouping by r_info also allows the runtime dynamic linker to implement
an 1-entry cache to reduce the number of symbol lookup required. With
such 1-entry cache implemented on Android, I'm seeing 10% to 20%
reduction in total time spent in runtime linker for several executables
that I tested.

As a simple correctness check, I've also built x86_64 Android and booted
successfully.

Differential Revision: https://reviews.llvm.org/D65242
Patch by Vic Yang

llvm-svn: 369507
2019-08-21 09:21:37 +00:00
Fangrui Song b2895a8cdc Revert D65242 "[ELF] More dynamic relocation packing"
This reverts r369488 and r369489. The change broke build bots:

http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap-ubsan/builds/14511
http://lab.llvm.org:8011/builders/lld-x86_64-freebsd/builds/34407

llvm-svn: 369497
2019-08-21 06:50:08 +00:00
Fangrui Song d840a9cbed [ELF][test] Add CHECK lines omitted in r369488
Add append .o to some object file names

llvm-svn: 369489
2019-08-21 03:15:57 +00:00
Fangrui Song 35f9a84a15 [ELF] More dynamic relocation packing
Currently, with Android dynamic relocation packing, only relative
relocations are grouped together. This patch implements similar
packing for non-relative relocations.

The implementation groups non-relative relocations with the same
r_info and r_addend, if using RELA. By requiring a minimum group
size of 3, this achieves smaller relocation sections. Building Android
for an ARM32 device, I see the total size of /system/lib decrease by
392 KB.

Grouping by r_info also allows the runtime dynamic linker to implement
an 1-entry cache to reduce the number of symbol lookup required. With
such 1-entry cache implemented on Android, I'm seeing 10% to 20%
reduction in total time spent in runtime linker for several executables
that I tested.

As a simple correctness check, I've also built x86_64 Android and booted
successfully.

Differential Revision: https://reviews.llvm.org/D66491
Patch by Vic Yang!

llvm-svn: 369488
2019-08-21 03:02:08 +00:00
Martin Storsjo 08a5a0aa25 [COFF] Check errorCount before committing the output file
This avoids producing an output file if errors appeared late in the
linking process (e.g. while fixing relocations, or as in the test,
while checking for multiple resources). If an output file is produced,
build tools might not retry building it on rebuilds, even if a previous
build failed due to the error return code.

Differential Revision: https://reviews.llvm.org/D66491

llvm-svn: 369445
2019-08-20 21:08:14 +00:00
Martin Storsjo 8a91aa53a0 [COFF] Print the file name on errors writing the pdb file
This avoids confusing contextless error messages such as "No such file
or directory" if e.g. the pdb output file should be written to a
nonexistent directory. (This can happen with linkrepro scripts, at least
old ones.)

Differential Revision: https://reviews.llvm.org/D66466

llvm-svn: 369425
2019-08-20 18:56:48 +00:00
Sam Clegg cf2b8722d4 [WebAssembly][lld] Fix crash when applying relocations to debug sections
Debug sections are special in that they can contain relocations against
symbols that are not present in the final output (i.e. not live).
However it is also possible to have R_WASM_TABLE_INDEX relocations
against symbols that don't have a table index assigned (since they are
not address taken by actual code.

Fixes: https://github.com/emscripten-core/emscripten/issues/9023

Differential Revision: https://reviews.llvm.org/D66435

llvm-svn: 369423
2019-08-20 18:39:24 +00:00
Martin Storsjo 6540e55067 [COFF] Require an explicit -implib option for creating implibs in mingw mode
GNU ld doesn't produce implibs unless explicitly requested.

Differential Revision: https://reviews.llvm.org/D66367

llvm-svn: 369363
2019-08-20 10:14:54 +00:00
Martin Storsjo dadc6f2488 [COFF] Allow using custom .edata from input object files
This is used by Wine for manually crafting export tables.

If the input object contains .edata sections, GNU ld references them
in the export directory instead of synthesizing an export table using
either export directives or the normal auto export mechanism. (AFAIK,
historically, way way back, GNU ld didn't support synthesizing the
export table - one was supposed to generate it using dlltool and link
it in instead.)

If faced with --out-implib and --output-def, GNU ld still populates
those output files with the same export info as it would have generated
otherwise, disregarding the input .edata. As this isn't an intended
usage combination, I'm not adding checks for that in tests.

Differential Revision: https://reviews.llvm.org/D65903

llvm-svn: 369358
2019-08-20 09:53:06 +00:00
Fangrui Song 12d83b4270 [ELF][PPC] Allow PT_LOAD to have overlapping p_offset ranges on EM_PPC
Ported the D64906 technique to EM_PPC.

Delete ppc-rela.s that is covered by ppc32-abs-pic.s

llvm-svn: 369351
2019-08-20 09:20:05 +00:00
Fangrui Song 9c371309f3 [ELF][X86] Allow PT_LOAD to have overlapping p_offset ranges on EM_386
Ported the D64906 technique to EM_386.

If `sh_addralign(.tdata) < sh_addralign(.tbss)`,
we can potentially make `p_vaddr(PT_TLS)%p_align(PT_TLS) != 0`.

ld.so that are known to have problems if p_vaddr%p_align!=0:

* FreeBSD 13.0-CURRENT rtld-elf
* glibc https://sourceware.org/bugzilla/show_bug.cgi?id=24606

New test i386-tls-vaddr-align.s checks our workaround makes p_vaddr%p_align = 0.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D65865

llvm-svn: 369347
2019-08-20 08:43:47 +00:00
Fangrui Song f66b767abe [ELF][AArch64] Allow PT_LOAD to have overlapping p_offset ranges
Ported the D64906 technique to AArch64. It deletes 3 alignments at
PT_LOAD boundaries for the default case: the size of an aarch64 binary
decreases by at most 192kb.

If `sh_addralign(.tdata) < sh_addralign(.tbss)`,
we can potentially make `p_vaddr(PT_TLS)%p_align(PT_TLS) != 0`.

ld.so that are known to have problems if p_vaddr%p_align!=0:

* musl<=1.1.22
* FreeBSD 13.0-CURRENT (and before) rtld-elf arm64

New test aarch64-tls-vaddr-align.s checks that our workaround makes p_vaddr%p_align = 0.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D64930

llvm-svn: 369344
2019-08-20 08:34:56 +00:00
Fangrui Song 01c7f4b606 [ELF][PPC] Allow PT_LOAD to have overlapping p_offset ranges
This change affects the non-linker script case (precisely, when the
`SECTIONS` command is not used). It deletes 3 alignments at PT_LOAD
boundaries for the default case: the size of a powerpc64 binary can be
decreased by at most 192kb. The technique can be ported to other
targets.

Let me demonstrate the idea with a maxPageSize=65536 example:

When assigning the address to the first output section of a new PT_LOAD,
if the end p_vaddr of the previous PT_LOAD is 0x10020, we advance to
the next multiple of maxPageSize: 0x20000. The new PT_LOAD will thus
have p_vaddr=0x20000. Because p_offset and p_vaddr are congruent modulo
maxPageSize, p_offset will be 0x20000, leaving a p_offset gap [0x10020,
0x20000) in the output.

Alternatively, if we advance to 0x20020, the new PT_LOAD will have
p_vaddr=0x20020. We can pick either 0x10020 or 0x20020 for p_offset!
Obviously 0x10020 is the choice because it leaves no gap. At runtime,
p_vaddr will be rounded down by pagesize (65536 if
pagesize=maxPageSize). This PT_LOAD will load additional initial
contents from p_offset ranges [0x10000,0x10020), which will also be
loaded by the previous PT_LOAD. This is fine if -z noseparate-code is in
effect or if we are not transiting between executable and non-executable
segments.

ld.bfd -z noseparate-code leverages this technique to keep output small.
This patch implements the technique in lld, which is mostly effective on
targets with large defaultMaxPageSize (AArch64/MIPS/PPC: 65536). The 3
removed alignments can save almost 3*65536 bytes.

Two places that rely on p_vaddr%pagesize = 0 have to be updated.

1) We used to round p_memsz(PT_GNU_RELRO) up to commonPageSize (defaults
  to 4096 on all targets). Now p_vaddr%commonPageSize may be non-zero.
  The updated formula takes account of that factor.
2) Our TP offsets formulae are only correct if p_vaddr%p_align = 0.
  Fix them. See the updated comments in InputSection.cpp for details.

  On targets that we enable the technique (only PPC64 now),
  we can potentially make `p_vaddr(PT_TLS)%p_align(PT_TLS) != 0`
  if `sh_addralign(.tdata) < sh_addralign(.tbss)`

  This exposes many problems in ld.so implementations, especially the
  offsets of dynamic TLS blocks. Known issues:

  FreeBSD 13.0-CURRENT rtld-elf (i386/amd64/powerpc/arm64)
  glibc (HEAD) i386 and x86_64 https://sourceware.org/bugzilla/show_bug.cgi?id=24606
  musl<=1.1.22 on TLS Variant I architectures (aarch64/powerpc64/...)

  So, force p_vaddr%p_align = 0 by rounding dot up to p_align(PT_TLS).

The technique will be enabled (with updated tests) for other targets in
subsequent patches.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D64906

llvm-svn: 369343
2019-08-20 08:34:25 +00:00
Sam Clegg e8e914e640 [lld][WebAssembly] Honor --no-export-dynamic even with -shared
Differential Revision: https://reviews.llvm.org/D66359

llvm-svn: 369276
2019-08-19 16:34:51 +00:00
Fangrui Song ec4540d8a6 [ELF] Simplify processRelocAux and allow a corner-case error
After D66007/r369262, if the control flow reaches `if (sym.isUndefined())`, we know:

* The relocation is not a link-time constant => symbol is preemptable => Undefined or SharedSymbol
* Not an undef weak.
* -no-pie.
* The symbol type is neither STT_OBJECT nor STT_FUNC.

ld.lld --export-dynamic --unresolved-symbols=ignore-all %t.o can satisfy
these conditions. Delete the isUndefined() test so that we error
`symbol '...' has no type`, because we don't know the type to make the
decision to create copy relocation/canonical PLT.

llvm-svn: 369271
2019-08-19 15:52:29 +00:00
Fangrui Song ac0e6c6502 [ELF] Move (copy relocation/canonical PLT) before error checking
In processRelocAux(), we handle errors before copy relocation/canonical PLT.
This makes error checking a bit complex because we have to check for
conditions that will be allowed by copy relocation/canonical PLT.

Instead, move copy relocation/canonical PLT before error checking. This
simplifies the previous clumsy error checking code

`config->shared || (config->pie && expr == R_ABS && type != target->symbolicRel)`

to the simple `config->isPic`. Some diagnostics can be reported in
different ways. The code motion changes diagnostics for some contrived
test cases:

* copy-rel-pie-error.s -> copy-rel-pie2.s:
  It was rejected before but accepted now. ld.bfd also accepts the case.
* copy-errors.s: "cannot preempt symbol" changes to "symbol 'bar' has no type"
* got32{,x}-i386.s: the suggestion changes from "-fPIC or -Wl,-z,notext" to "-fPIE"
* x86-64-dyn-rel-error5.s: one diagnostic changes for -pie case

Reviewed By: peter.smith

Differential Revision: https://reviews.llvm.org/D66007

llvm-svn: 369262
2019-08-19 14:30:12 +00:00
Fangrui Song c8a1dfc484 [ELF][Hexagon] Improve error message for unknown relocations
Like rLLD354040

Previously, for unknown relocation types, in -no-pie/-pie mode, we got something like:

    foo.o: unrecognized relocation ...

In -shared mode:

    error: can't create dynamic relocation ... against symbol: yyy in readonly segment

Delete the default case from Hexagon::getRelExpr and add the error there. We will get consistent error message like `error: unknown relocation (1024) against symbol foo`

Reviewed By: sidneym

Differential Revision: https://reviews.llvm.org/D66275

llvm-svn: 369260
2019-08-19 14:07:14 +00:00
Sid Manning a0a4c6b722 [lld][Hexagon] Add GOTREL relocations.
Add GOTREL relocation support. (S + A - GOT)

Differential Revision: https://reviews.llvm.org/D66260

llvm-svn: 369258
2019-08-19 13:32:32 +00:00
Peter Smith 2cafd872fb [ELF][ARM] Add a test that maxes out the thunk convergence limit
Add a test that takes the maximum amount of passes permitted to converge.
This will make sure that any symbol defined in a linker script gets the
correct value and that any other convergence limit involving symbol address
doesn't restrict Thunk convergence.

Differential Revision: https://reviews.llvm.org/D66346

llvm-svn: 369246
2019-08-19 09:46:52 +00:00
Fangrui Song a560bbf7a0 [ELF] Replace local variable hasExportDynamic with config->exportDynamic. NFC
llvm-svn: 369187
2019-08-17 10:04:18 +00:00
Fangrui Song d5d79dfd56 [ELF][PPC] Fix getRelExpr for R_PPC64_REL16_HI
Fixes https://github.com/ClangBuiltLinux/linux/issues/640

R_PPC64_REL16_HI was incorrectly computed as an R_ABS relocation.
rLLD368964 made it a linker failure. Change it to use R_PC to fix the
failures.

Add ppc64-reloc-rel.s for these R_PPC64_REL* tests.

llvm-svn: 369184
2019-08-17 06:28:03 +00:00
Fangrui Song 7ca1e0c825 [ELF][Hexagon] Replace R_HEXAGON_GOT with R_GOTPLT
R_GOTPLT is relative to .got.plt since D59594. Since R_HEXAGON_GOT
relocations always have 0 r_addend, they can use R_GOTPLT instead.

Reviewed By: sidneym

Differential Revision: https://reviews.llvm.org/D66274

llvm-svn: 369128
2019-08-16 17:01:56 +00:00
Sid Manning 2d3ebeb813 [lld][Hexagon]Support HEX_32 when building shared objects
Differential Revision: https://reviews.llvm.org/D66105

llvm-svn: 369121
2019-08-16 15:35:02 +00:00
JF Bastien 7a210d65ed Fix lld on GCC 5.1 after the C++14 move
Summary:
libstdc++ in GCC 5.1 has some bugs. The move to C++14 in D66195 triggered one
such bug caused by the new constexpr support in C++14, and the implementation
doing SFINAE wrong with the comparator to std::stable_sort.

Here's a small repro: https://godbolt.org/z/2QC3-n

The fix is to inline the lambdas directly into the llvm::stable_sort call
instead of erasing them through a std::function. The code is more readable as
well.

Reviewers: thakis, ruiu, espindola

Subscribers: emaste, arichardson, MaskRay, jkorous, dexonsmith, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D66306

llvm-svn: 369023
2019-08-15 17:47:19 +00:00
Fangrui Song e54ed4789f [ELF][AArch64] Improve error message for unknown relocations
Like rLLD354040.

Previously, for unrecognized relocation types, in -no-pie/-pie mode, we got something like:

  foo.o: unrecognized relocation ...

In -shared mode:

  error: can't create dynamic relocation ... against symbol: yyy in readonly segment

Delete the default case from AArch64::getRelExpr and add the error there.

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D66277

llvm-svn: 368983
2019-08-15 10:02:54 +00:00
Fangrui Song 1542ff5282 [ELF][PPC] Improve error message for unknown relocations
Like rLLD354040.

Previously, for unrecognized relocation types, in -no-pie mode:

  foo.o: unrecognized reloc 256

In -pie/-shared mode:

  error: can't create dynamic relocation R_PPC_xxx against symbol: yyy in readonly segment

llvm-svn: 368964
2019-08-15 05:22:23 +00:00
Jonas Devlieghere 6ba7992031 [LLD] Migrate llvm::make_unique to std::make_unique
Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.

Differential revision: https://reviews.llvm.org/D66259

llvm-svn: 368936
2019-08-14 22:28:17 +00:00
Fangrui Song e220c67b7a [ELF] --gdb-index: fix odd variable name cUs after r365730 and replace lower_bound with partition_point. NFC
llvm-svn: 368845
2019-08-14 12:56:30 +00:00
Fangrui Song b9138bc52b [ELF][test] Update silent-ignore.test
Some options are implemented now:

--no-warn-common : r263413
--allow-shlib-undefined : r352826

Some are ignored but were not reflected in this test.

llvm-svn: 368837
2019-08-14 12:20:21 +00:00
Martin Storsjo 0e5530abfc [MinGW] Correct handling different forms of a few options
Support the equals form of the long --entry=<symbol> option,
add a test for the -e<symbol> form.

Add tests for single dash forms of -exclude-all-symbols and
-export-all-symbols.

Support single-dash forms of -out-implib and -output-def, support
the equals form of --output-def=<file>. (We previously had a test
to explicitly disallow -out-implib, but it turns out that GNU ld
actually does support it just fine, despite also matching the
-o<file> option.)

Disallow the double-dashed --u form, add a test for -u<symbol>.

Differential Revision: https://reviews.llvm.org/D66066

llvm-svn: 368816
2019-08-14 09:35:44 +00:00
Martin Storsjo ce26ad7a9d [MinGW] Restructure Options.td to use multiclass where sensible. NFC.
Differential Revision: https://reviews.llvm.org/D66065

llvm-svn: 368815
2019-08-14 09:35:40 +00:00
Martin Storsjo df7ec1ff7b [MinGW] Remove stray/inconsistent comment chars in test file. NFC.
Test directives don't need to be in comments in this file.

llvm-svn: 368814
2019-08-14 09:35:36 +00:00
Fangrui Song f1d538cce5 [ELF] Initialize 2 fields of Symbol in SymbolTable::insert
A new symbol is added to elf::symtab in 3 steps:

1) SymbolTable::insert creates a placeholder.
2) Symbol::mergeProperties
3) Symbol::replace

Fields referenced by steps 2) and 3) should be initialized in
SymbolTable::insert.  `traced` and `referenced` were missed previously.
This did not cause problems because compilers generated code that
initialized them (bit fields) to 0.

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D66130

llvm-svn: 368784
2019-08-14 01:52:47 +00:00
Sam Clegg 7185a7301e [lld][WebAssembly] Allow linking of pic code into static binaries
Summary: See https://github.com/emscripten-core/emscripten/issues/9013

Subscribers: dschuff, jgravelle-google, aheejin, sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D65922

llvm-svn: 368719
2019-08-13 17:02:02 +00:00
Jordan Rupprecht 7859641626 [lld][test] Update test to print ELF note description data
llvm-svn: 368710
2019-08-13 16:11:09 +00:00
Fangrui Song dc06b0bc9a [ELF] Don't special case symbolic relocations with 0 addend to ifunc in writable locations
Currently the following 3 relocation types do not trigger the creation
of a canonical PLT (which changes STT_GNU_IFUNC to STT_FUNC and
redirects all references):

1) GOT-generating (`needsGot`)
2) PLT-generating (`needsPlt`)
3) R_ABS with 0 addend in a writable location. This is used for
  for ifunc function pointers in writable sections such as .data and .toc.

This patch deletes case 3) to simplify the R_*_IRELATIVE generating
logic added in D57371. Other advantages:

* It is guaranteed no more than 1 R_*_IRELATIVE is created for an ifunc.
* PPC64: no need to special case ifunc in toc-indirect to toc-relative relaxation. See D65755

The deleted elf::addIRelativeRelocs demonstrates that one-pass scan
through relocations makes several optimizations difficult. This is
something we can think about in the future.

Reviewed By: peter.smith

Differential Revision: https://reviews.llvm.org/D65995

llvm-svn: 368661
2019-08-13 09:43:40 +00:00
Fangrui Song c6cd62352c [ELF] Simplify handling of exportDynamic and isPreemptible
In Writer::includeInDynSym(), exportDynamic is used by a Defined with
protected or default visibility, to record whether it is required to be
exported into .dynsym. It is set when any of the following conditions
hold:

1) There is an interposable symbol from a DSO (Undefined or SharedSymbol with default visibility)
2) If -shared or --export-dynamic is specified, any symbol in an object file/bitcode sets this property, unless suppressed by canBeOmittedFromSymbolTable().
3) --dynamic-list when producing an executable

4) protected symbol from a DSO preempted by copy relocation/canonical PLT when
  --ignore-{data,function}-address-equality is specified
5) ifunc is exported when -z ifunc-noplt is specified

Bullet points 4) and 5) are irrelevant in this patch.

Bullet 3) does not play well with 1) and 2). When -shared is specified,
exportDynamic of most symbols is true. This makes it incapable to record
--dynamic-list marked symbols. We thus have obscure:

    if (!config->shared)
      b->exportDynamic = true;
    else if (b->includeInDynsym())
      b->isPreemptible = true;

This patch adds another bit `Symbol::inDynamicList` to record
3). We can thus simplify handleDynamicList() by unifying the DSO and
  executable cases. It also allows us to simplify isPreemptible - now
the field is only used in finalizeSections() and later stages.

Reviewed By: peter.smith

Differential Revision: https://reviews.llvm.org/D66091

llvm-svn: 368659
2019-08-13 09:12:52 +00:00
Fangrui Song ab04ad6af7 [ELF] Rename odd variable names "New" after r365730. NFC
New -> newSym or newFlags

Reviewed By: atanasyan

Differential Revision: https://reviews.llvm.org/D66127

llvm-svn: 368651
2019-08-13 06:19:39 +00:00
Fangrui Song 892cdc73be [ELF][test] Add dynamic-list-preempt2.s
When producing a DSO, the isPreemptible property of a Defined with
default or protected visibility is affected by the --dynamic-list file,
but not by interposable symbols in other DSOs.

llvm-svn: 368649
2019-08-13 05:08:46 +00:00
Bob Haarman 6e18c7f8d4 [lld] Remove unnecessary "class Lazy"
llvm-svn: 368644
2019-08-13 01:02:30 +00:00
Fangrui Song 3cafdfddcb [ELF] Remove unnecessary assignment to `isPreemptible` in replaceWithDefined()
After r368535, it is no longer used in the handling of VER_NDX_LOCAL.
Drop it.

llvm-svn: 368550
2019-08-12 10:01:17 +00:00
Fangrui Song cfdd4589f1 [ELF] Remove redundant isDefined() in Symbol::computeBinding() and delete one redundant call site
After r367869, VER_NDX_LOCAL can only be assigned to Defined and
CommonSymbol.  CommonSymbol becomes Defined after replaceCommonSymbols(),
thus `versionId == VER_NDX_LOCAL` will imply `isDefined()`.

In maybeReportUndefined(), computeBinding() is called when the symbol is
unknown to be Undefined. computeBinding() != STB_LOCAL will always be
true.

llvm-svn: 368536
2019-08-11 17:03:00 +00:00
Fangrui Song 635eda8bb0 [ELF] Remove redundant !isPreemptible in Symbol::computeBinding()
!isPreemptible was added in r343668 to fix PR39104: symbols redefined by
replaceWithDefined() might be incorrectly considered STB_LOCAL if a
version script specified `local: *;`.

After r367869 (`config->defaultSymbolVersion` was removed), we will
assign VER_NDX_LOCAL to only regular Defined and CommonSymbol, not
Defined created by replaceWithDefined() (because scanVersionScript() is
called before scanRelocations()). The !isPreemptible is thus redundant
and can be deleted.

llvm-svn: 368535
2019-08-11 16:12:07 +00:00
Fangrui Song 38c5788548 [ELF] Remove unnecessary assignment to `used` in replaceWithDefined
`Symbol::used` is used by Undefined and SharedSymbol to record if a
.symtab entry is needed. It is of no use for Defined.

llvm-svn: 368533
2019-08-11 13:32:46 +00:00
Igor Kudrin b65016ddac [ELF] For VS-style diagnostics, prefer printing full paths in the header.
The filename part in the message header is used by Visual Studio
to fill Error List so that a user can click on an item and jump
to the mentioned location. If we use only the name of a source file
and not the full path, Visual Studio might be unable to find the right
file or, even worse, show a wrong one.

Differential Revision: https://reviews.llvm.org/D65875

llvm-svn: 368409
2019-08-09 08:29:03 +00:00
Fangrui Song 179dc276eb [ELF] Expand regions for gaps due to explicit address
If the dot gets moved by an explicit section address, an empty gap between sections could be created. The encompassing region for the section being parsed needs to be expanded to include the gap.

Differential Revision: https://reviews.llvm.org/D65722

Patch by Gabriel Smith!

llvm-svn: 368379
2019-08-09 01:25:49 +00:00
David Blaikie 84b55e61dd DebugInfo: Explicitly handle errors when parsing unit DIEs
This ensures these errors produce a non-zero exit and improves the
context (providing the name of the input object and section being
parsed).

llvm-svn: 368378
2019-08-09 01:14:36 +00:00
Peter Collingbourne 1fa1cd200b ELF: Move sections referred to by __start_/__stop_ symbols into the main partition.
In the case where C identifier sections have SHF_LINK_ORDER they will most
likely be placed in the same partition as the section that they are associated
with. But unless this happens to be the main partition, this will cause them
to be excluded from the range covered by the __start_ and __stop_ symbols,
which may lead to incorrect program behaviour. So we need to move them
all into the main partition so that they will be covered by the __start_
and __stop_ symbols.

We may want to refine this approach later and allow different __start_/__stop_
symbol values for different partitions. This would only make sense for
relocations from SHT_NOTE sections since they are duplicated into each
partition.

Differential Revision: https://reviews.llvm.org/D65909

llvm-svn: 368375
2019-08-09 00:57:54 +00:00
Sam Clegg dbfea28219 [lld][WebAssembly] Don't create optional symbols when outputing an object file
Summary: This was a bug in rL368310.  I'm working on a test case now.

Differential Revision: https://reviews.llvm.org/D65985

llvm-svn: 368369
2019-08-08 23:56:21 +00:00
Guanzhong Chen c5ccbf52ad [WebAssembly][lld] control __data_end export with config->shared
Summary:
Emscripten expects `__data_end` to show up in PIC code as long as it's not
linked with `--shared`.

Currently, Emscripten breaks with latest LLVM because `__data_end` is controlled
by `config->isPic` instead of `config->shared`.`

Reviewers: tlively, sbc100

Reviewed By: sbc100

Subscribers: dschuff, jgravelle-google, aheejin, sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D65980

llvm-svn: 368361
2019-08-08 22:40:04 +00:00
Sam Clegg ad1cc145e8 [lld][WebAssembly] Use createGlobalVariable helper function. NFC.
Differential Revision: https://reviews.llvm.org/D65911

llvm-svn: 368325
2019-08-08 18:22:03 +00:00
Sam Clegg caa0db1318 [lld][WebAssembly] Add optional symbols after input file handling
This allows undefined references in input files be resolved by the
optional symbols.  Previously we were doing this before input file
reading which means it was working only for command line symbols
references (i.e. -u or --export).

Also use addOptionalDataSymbol for __dso_handle and make all optional
symbols hidden by default.

Differential Revision: https://reviews.llvm.org/D65920

llvm-svn: 368310
2019-08-08 16:58:36 +00:00
Fangrui Song ba4bda657e [ELF][AArch64] Delete two unused RUN lines from aarch64-movw-tprel.s after D65882
llvm-svn: 368298
2019-08-08 13:57:53 +00:00
Peter Smith d4695e1d75 [ELF][AArch64] Support for movz, movk tprel relocations
This patch Implements the R_AARCH64_TLSLE_MOVW_TPREL_G*[_NC]. These are
logically the same calculation as the existing TLSLE relocations with
the result written back to mov[nz] and movk instructions. A typical code
sequence is:
movz x0, #:tprel_g2:foo    // bits [47:32] of R_TLS with overflow check
movk x0, #:tprel_g1_nc:foo // bits [31:16] of R_TLS with no overflow check
movk x0, #:tprel_g0_nc:foo // bits [15:0] of R_TLS with no overflow check

This type of code sequence is usually used with a large code model.

Differential Revision: https://reviews.llvm.org/D65882

Fixes: PR42853
llvm-svn: 368293
2019-08-08 13:38:09 +00:00
David Blaikie 8a480d2cbf gdb-index: Wire up str_offsets section to avoid incorrect error message about offsets_base
There's still a need for a deeper fix to the way libDebugInfoDWARF error
messages are propagated up to lld - if lld had exited non-zero on this
error message we would've found the issue sooner.

llvm-svn: 368229
2019-08-07 22:49:14 +00:00
David Blaikie 77ddf6af2b Add a test demonstrating DWARF parse failures are not causing lld to exit non-zero
This bug was/is masking other issues - committing this to demonstrate
the problem/track fixing it.

llvm-svn: 368220
2019-08-07 22:05:33 +00:00
David Blaikie fa2f4395e9 API update for change to LLVM's lib/DebugInfo/DWARF
llvm-svn: 368190
2019-08-07 17:18:18 +00:00
Igor Kudrin 0e1fea6754 Try to fix windows build bots after r368153.
llvm-svn: 368169
2019-08-07 13:22:29 +00:00
Igor Kudrin da41e2107e [ELF] Fix splitting messages for duplicate symbols.
D65213 (rL367536) does not work for the case when a source file path
includes subdirectories.

Differential Revision: https://reviews.llvm.org/D65810

llvm-svn: 368153
2019-08-07 11:32:43 +00:00
Rui Ueyama e6a33e1f11 Handle /align option.
Differential Revision: https://reviews.llvm.org/D65736

llvm-svn: 368145
2019-08-07 10:16:21 +00:00
Rui Ueyama 6c5fc94093 Simplify error message output. NFC.
Differential Revision: https://reviews.llvm.org/D65855

llvm-svn: 368144
2019-08-07 10:11:24 +00:00
Fangrui Song 5ebee2558b [ELF][X86] Add --no-show-raw-insn and -soname to some i386 tests
llvm-svn: 368142
2019-08-07 09:43:54 +00:00
Rui Ueyama cac8df1ab9 Re-submit r367649: Improve raw_ostream so that you can "write" colors using operator<<
The original patch broke buildbots, perhaps because it changed the
default setting whether colors are enabled or not.

llvm-svn: 368131
2019-08-07 08:08:17 +00:00
Guanzhong Chen 0cb776e61a [WebAssembly] Fix null pointer in createInitTLSFunction
Summary:
`createSyntheticSymbols`, which creates `WasmSym::InitTLS`, is only called
when `!config->relocatable`, but this condition is not checked when calling
`createInitTLSFunction`.

This diff checks `!config->relocatable` before calling `createInitTLSFunction`.

Fixes https://github.com/emscripten-core/emscripten/issues/9155.

Reviewers: tlively, aheejin, kripken, sbc100

Subscribers: dschuff, jgravelle-google, sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D65785

llvm-svn: 368078
2019-08-06 20:09:04 +00:00
Fangrui Song c55c0598f9 [ELF][PPC] Don't relax ifunc toc-indirect accesses to toc-relative
Fixes PR42759.

```
// If ifunc is taken address in -fPIC code, it may have a toc entry
.section .toc,"aw",@progbits
  .quad ifunc

// ifunc may be defined as STT_GNU_IFUNC in another object file
.type ifunc, %gnu_indirect_function
```

If ifunc is non-preemptable (e.g. when linking an executable), the toc
entry will be relocated by R_PPC64_IRELATIVE.

R_*_IRELATIVE represents the symbolic value of a
non-preemptable ifunc (not associated with a canonical PLT) in a writable location. It has an unknown value at
link time, so we cannot apply toc-indirect to toc-relative relaxation.

Reviewed By: luporl, sfertile

Differential Revision: https://reviews.llvm.org/D65755

llvm-svn: 368057
2019-08-06 16:57:54 +00:00
Peter Smith 7f320d4bf0 [ELF][ARM] Fix /DISCARD/ of section with .ARM.exidx section
The combineEhSections runs, by design, before processSectionCommands so
that input exception sections like .ARM.exidx and .eh_frame are not assigned
to OutputSections. Unfortunately if /DISCARD/ removes InputSections that
have associated .ARM.exidx sections without discarding the .ARM.exidx
synthetic section then we will end up crashing when trying to sort the
InputSections in ascending address order.

We fix this by filtering out the sections that have been discarded prior
to processing the InputSections in finalizeContents().

fixes pr42890

Differential Revision: https://reviews.llvm.org/D65759

llvm-svn: 368041
2019-08-06 14:13:38 +00:00
Fangrui Song e49c417ed1 [ELF] Make binding (weak or non-weak) logic consistent for Undefined and SharedSymbol
This is a case missed by D64136. If %t1.o has a weak reference on foo,
and %t2.so has a non-weak reference on foo:

```
0. ld.lld %t1.o %t2.so          # ok; STB_WEAK; accepted since D64136
1. ld.lld %t2.so %t1.o          # undefined symbol: foo; STB_GLOBAL
2. gold %t1.o %t2.so            # ok; STB_WEAK
3. gold %t2.so %t1.o            # undefined reference to 'foo'; STB_GLOBAL
4. ld.bfd %t1.o %t2.so          # undefined reference to `foo'; STB_WEAK
5. ld.bfd %t2.so %t1.o          # undefined reference to `foo'; STB_WEAK
```

It can be argued that in both cases, the binding of the undefined foo
should be set to STB_WEAK, because the binding should not be affected by
referenced from shared objects.

--allow-shlib-undefined doesn't suppress errors (3,4,5), but -shared or
--noinhibit-exec allows ld.bfd/gold to produce a binary:

```
3. gold -shared %t2.so %t1.o    # ok; STB_GLOBAL
4. ld.bfd -shared %t2.so %t1.o  # ok; STB_WEAK
5. ld.bfd -shared %t1.o %t1.o   # ok; STB_WEAK
```

If %t2.so has DT_NEEDED entries, ld.bfd will load them (lld/gold don't
have the behavior). If one of the DSO defines foo and it is in the
link-time search path (e.g. DT_NEEDED entry is an absolute path, via
-rpath=, via -rpath-link=, etc),
`ld.bfd %t1.o %t2.so` and `ld.bfd %t1.o %t2.so` will not error.

In this patch, we make Undefined and SharedSymbol share the same binding
computing logic. Case 1 will be allowed:

```
0. ld.lld %t1.o %t2.so          # ok; STB_WEAK; accepted since D64136
1. ld.lld %t2.so %t1.o          # ok; STB_WEAK; changed by this patch
```

In the future, we can explore the option that turns both (0,1) into
errors if --no-allow-shlib-undefined (default when linking an
executable) is in action.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D65584

llvm-svn: 368038
2019-08-06 14:03:45 +00:00
Igor Kudrin 5978c9249b [MachO] Update LLD to use 64-bit offsets with DataExtractor (3/5)
Differential Revision: https://reviews.llvm.org/D65639

llvm-svn: 368032
2019-08-06 13:37:34 +00:00
Fangrui Song 8981dc3058 [ELF][test] Reorganize some tls-*.s tests
Some tls-*.s tests do not test generic TLS behavior but rather are x86 specific.
Rename them to i386-*.s or x86-64-*.s

Delete tls-static.s: covered by tls-opt.s
Delete tls-opt-no-plt.s: add --implicit-check-not=.plt to x86-64-tls-gdie.s to cover it

Rename tls-dynamic-i686.s to i386-tls-dynamic.s
Rename tls-i686.s to i386-tls-le.s
Rename tls-opt-i686.s to i386-tls-opt.s
Rename tls-opt-iele-i686-nopic.s to i386-tls-opt-iele-nopic.s

Rename tls-dynamic.s to x86-64-tls-dynamic.s . IE should be split off in the future.
Rename tls-error.s to x86-64-reloc-tpoff32-error.s
Rename tls-opt-gdie.s to x86-64-tls-gdie.s
Rename tls-opt-x86_64-noplt.s to x86-64-tls-opt-noplt.s
Rename tls-opt-local.s => x86-64-tls-ie-opt-local.s . It can be merged with x86-64-tls-ie-local.s in the future.

llvm-svn: 367877
2019-08-05 15:34:01 +00:00
Fangrui Song e28a70daf4 [ELF] Consistently prioritize non-* wildcards overs "*" in version scripts
We prioritize non-* wildcards overs VER_NDX_LOCAL/VER_NDX_GLOBAL "*".
This patch generalizes the rule to "*" of other versions and thus fixes PR40176.
I don't feel strongly about this GNU linkers' behavior but the
generalization simplifies code.

Delete `config->defaultSymbolVersion` which was used to special case
VER_NDX_LOCAL/VER_NDX_GLOBAL "*".

In `SymbolTable::scanVersionScript`, custom versions are handled the same
way as VER_NDX_LOCAL/VER_NDX_GLOBAL. So merge
`config->versionScript{Locals,Globals}` into `config->versionDefinitions`.
Overall this seems to simplify the code.

In `SymbolTable::assign{Exact,Wildcard}Versions`,
`sym->verdefIndex == config->defaultSymbolVersion` is changed to
`verdefIndex == UINT32_C(-1)`.
This allows us to give duplicate assignment diagnostics for
`{ global: foo; };` `V1 { global: foo; };`

In test/linkerscript/version-script.s:
  vs_index of an undefined symbol changes from 0 to 1. This doesn't matter (arguably 1 is better because the binding is STB_GLOBAL) because vs_index of an undefined symbol is ignored.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D65716

llvm-svn: 367869
2019-08-05 14:31:39 +00:00
Nilanjana Basu da60fc813c Changing representation of .cv_def_range directives in Codeview debug info assembly format for better readability
llvm-svn: 367867
2019-08-05 14:16:58 +00:00
Nilanjana Basu b5e4d7de17 Revert "Changing representation of .cv_def_range directives in Codeview debug info assembly format for better readability"
This reverts commit a885afa9fa.

llvm-svn: 367861
2019-08-05 13:55:21 +00:00
Nilanjana Basu a885afa9fa Changing representation of .cv_def_range directives in Codeview debug info assembly format for better readability
llvm-svn: 367850
2019-08-05 13:11:51 +00:00
Martin Storsjo c92b3563c6 [MinGW] Add an lld specific option for requesting to delay load libraries
With GNU tools, delayload is handled completely differently. (One
creates a specific delayload import library using dlltool and then
links against it instead of the normal import library.)

Instead of requiring using -Xlink=-delayload:lib.dll, we can provide
an lld specific option for this.

Differential Revision: https://reviews.llvm.org/D65728

llvm-svn: 367837
2019-08-05 11:57:06 +00:00
Martin Storsjo a0cbe16ed5 [COFF] Omit automatically imported symbols from the symbol table
These symbols actually point to the symbol's IAT entry, which
obviously is different from the symbol itself (which is imported
from a different module and doesn't exist in the current one).

Omitting this symbol helps gdb inspect automatically imported
symbols, see https://sourceware.org/bugzilla/show_bug.cgi?id=24574
for discussion on the matter.

Surprisingly, those extra symbols don't seem to be an issue for
gdb when the sources have been built with clang, only with gcc.
The actual logic in gdb that this depends on still is unknown, but
omitting these symbols from the symbol table is the right thing to
do in any case.

Differential Revision: https://reviews.llvm.org/D65727

llvm-svn: 367836
2019-08-05 11:57:00 +00:00
Fangrui Song d9b948b6eb Rename F_{None,Text,Append} to OF_{None,Text,Append}. NFC
F_{None,Text,Append} are kept for compatibility since r334221.

llvm-svn: 367800
2019-08-05 05:43:48 +00:00
Fangrui Song 8f6cf97bee [ELF][test] Delete redundant version-script-*.s tests
Delete version-script-missing.s: it is covered by version-script-noundef.s
Delete version-script-anonymous-local.s: it is covered by version-script-{glob,weak}.s etc
Delete version-script-no-warn{,2}.s: add --fatal-warnings to some version-script.s commands instead

llvm-svn: 367778
2019-08-04 09:05:21 +00:00
Fangrui Song 25ab1c6471 [ELF] Move R_*_IRELATIVE from .rel[a].plt to .rel[a].dyn unless --pack-dyn-relocs=android[+relr]
An R_*_IRELATIVE represents the address of a STT_GNU_IFUNC symbol
(redirected at runtime) which is non-preemptable and is not associated
with a canonical PLT (associated with a symbol with a section index of
SHN_UNDEF but a non-zero st_value).

.rel[a].plt [DT_JMPREL, DT_JMPREL+DT_JMPRELSZ) contains relocations that
can be lazily resolved. R_*_IRELATIVE are always eagerly resolved, so
conceptually they do not belong to .rela.plt. "iplt" is mostly a misnomer.

glibc powerpc and powerpc64 do not resolve R_*_IRELATIVE if they are in .rela.plt.

    // a.o - synthesized PLT call stub has an R_*_IRELATIVE
    void ifunc(); int main() { ifunc(); }
    // b.o
    static void real() {}
    asm (".type ifunc, %gnu_indirect_function");
    void *ifunc() { return &real; }

The lld-linked executable crashes. ld.bfd places R_*_IRELATIVE in
.rela.dyn and the executable works.

glibc i386, x86_64, and aarch64 have logic
(glibc/sysdeps/*/dl-machine.h:elf_machine_lazy_rel) to eagerly resolve
R_*_IRELATIVE in .rel[a].plt so the lld-linked executable works.

Move R_*_IRELATIVE from .rel[a].plt to .rel[a].dyn to fix the crashes on
glibc powerpc/powerpc64. This also helps simplifying ifunc
implementation in FreeBSD rtld-elf powerpc64.

If --pack-dyn-relocs=android[+relr] is specified, the Android packed
dynamic relocation format is used for .rela.dyn. We cannot name
in.relaIplt ".rela.dyn" because the output section will have mixed
formats. This can be improved in the future.

Reviewed By: pcc

Differential Revision: https://reviews.llvm.org/D65651

llvm-svn: 367745
2019-08-03 02:26:52 +00:00
Martin Storsjo 397a516a52 [COFF] Clarify a comment. NFC.
It's the __delayLoadHelper2 function that overwrites the jump table
slot, not this thunk.

llvm-svn: 367674
2019-08-02 11:08:15 +00:00
Martin Storsjo 5f0077d238 [COFF] Avoid loading objects for mingw autoimport, when a defined alias exists
This avoids a spurious and confusing log message in cases where
both e.g. "alias" and "__imp_alias" exist.

Differential Revision: https://reviews.llvm.org/D65598

llvm-svn: 367673
2019-08-02 11:02:34 +00:00
Rui Ueyama 4d41c332ef Revert r367649: Improve raw_ostream so that you can "write" colors using operator<<
This reverts commit r367649 in an attempt to unbreak Windows bots.

llvm-svn: 367658
2019-08-02 07:22:34 +00:00
Rui Ueyama c1981b2b26 Add an assert() to catch possible regexp errors.
llvm-svn: 367651
2019-08-02 05:11:19 +00:00
Rui Ueyama 96a7a225f5 Add a comment for --vs-diagnostics.
llvm-svn: 367650
2019-08-02 05:04:27 +00:00
Rui Ueyama a52f982f1c Improve raw_ostream so that you can "write" colors using operator<<
1. raw_ostream supports ANSI colors so that you can write messages to
the termina with colors. Previously, in order to change and reset
color, you had to call `changeColor` and `resetColor` functions,
respectively.

So, if you print out "error: " in red, for example, you had to do
something like this:

  OS.changeColor(raw_ostream::RED);
  OS << "error: ";
  OS.resetColor();

With this patch, you can write the same code as follows:

  OS << raw_ostream::RED << "error: " << raw_ostream::RESET;

2. Add a boolean flag to raw_ostream so that you can disable colored
output. If you disable colors, changeColor, operator<<(Color),
resetColor and other color-related functions have no effect.

Most LLVM tools automatically prints out messages using colors, and
you can disable it by passing a flag such as `--disable-colors`.
This new flag makes it easy to write code that works that way.

Differential Revision: https://reviews.llvm.org/D65564

llvm-svn: 367649
2019-08-02 04:48:30 +00:00
Rui Ueyama 966b9a3b9d Fix an unused variable warning.
llvm-svn: 367643
2019-08-02 02:51:20 +00:00
Martin Storsjo 90b4388f56 [COFF] Fix wholearchive with thin archives
The Archive object created when loading an archive specified with
wholearchive got cleaned up immediately, when the owning std::unique_ptr
went out of scope, even if persisted StringRefs pointed to memory that
belonged to the archive, which no longer was mapped in memory.

This hasn't been an issue with regular (as opposed to thin) archives,
as references to the member objects has kept the mapping for the whole
archive file alive - but with thin archives, all such references point
to other files.

Add the std::unique_ptr to the arena allocator, to retain it as long
as necessary.

This fixes (the last issue raised in) PR42388.

Differential Revision: https://reviews.llvm.org/D65565

llvm-svn: 367599
2019-08-01 18:47:27 +00:00
Fangrui Song 5391f158c2 [ELF] Add -z separate-code and pad the last page of last PF_X PT_LOAD with traps only if -z separate-code is specified
This patch

1) adds -z separate-code and -z noseparate-code (default).
2) changes the condition that the last page of last PF_X PT_LOAD is
 padded with trap instructions.
 Current condition (after D33630): if there is no `SECTIONS` commands.
 After this change: if -z separate-code is specified.

-z separate-code was introduced to ld.bfd in 2018, to place the text
segment in its own pages. There is no overlap in pages between an
executable segment and a non-executable segment:

1) RX cannot load initial contents from R or RW(or non-SHF_ALLOC).
2) R and RW(or non-SHF_ALLOC) cannot load initial contents from RX.

lld's current status:

- Between R and RX: in `Writer<ELFT>::fixSectionAlignments()`, the start of a
  segment is always aligned to maxPageSize, so the initial contents loaded by R
  and RX do not overlap. I plan to allow overlaps in D64906 if -z noseparate-code
  is in effect.
- Between RX and RW(or non-SHF_ALLOC if RW doesn't exist):
  we currently unconditionally pad the last page to commonPageSize
  (defaults to 4096 on all targets we support).
  This patch will make it effective only if -z separate-code is specified.

-z separate-code is a dubious feature that intends to reduce the number
of ROP gadgets (which is actually ineffective because attackers can find
plenty of gadgets in the text segment, no need to find gadgets in
non-code regions).

With the overlapping PT_LOAD technique D64906, -z noseparate-code
removes two more alignments at segment boundaries than -z separate-code.
This saves at most defaultCommonPageSize*2 bytes, which are significant
on targets with large defaultCommonPageSize (AArch64/MIPS/PPC: 65536).

Issues/feedback on alignment at segment boundaries to help understand
the implication:

* binutils PR24490 (the situation on ld.bfd is worse because they have
  two R-- on both sides of R-E so more alignments.)

* In binutils, the 2018-02-27 commit "ld: Add --enable-separate-code" made -z separate-code the default on Linux.
  d969dea983
  In musl-cross-make, binutils is configured with --disable-separate-code
  to address size regressions caused by -z separate-code. (lld actually has the same
  issue, which I plan to fix in a future patch. The ld.bfd x86 status is
  worse because they default to max-page-size=0x200000).

* https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=237676 people want
  smaller code size. This patch will remove one alignment boundary.

* Stef O'Rear: I'm opposed to any kind of page alignment at the
  text/rodata line (having a partial page of text aliased as rodata and
  vice versa has no demonstrable harm, and I actually care about small
  systems).

So, make -z noseparate-code the default.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D64903

llvm-svn: 367537
2019-08-01 09:58:25 +00:00
Igor Kudrin 07ceadda25 [ELF] With --vs-diagnostics, print a separate message for each location of a duplicate symbol.
We extract and print the source location in the message header so that
Visual Studio is able to parse it and jump there. As duplicate symbols
are defined in several locations, it is more convenient to have separate
error messages, which allows a user to easily access all the locations.

Differential Revision: https://reviews.llvm.org/D65213

llvm-svn: 367536
2019-08-01 09:58:03 +00:00
Fangrui Song 0cfa72eaec [ELF][X86] Improve tests
* Add --no-show-raw-insn to llvm-objdump -d tests
* When linking an executable with %t.so, the path %t.so will be recorded
  in the DT_NEEDED entry if %t.so doesn't have DT_SONAME. .dynstr will
  have varying lengths on different systems. Add -soname so that the
  string in .dynstr is of fixed length to make tests more robust.
* Rename i386-tls-initial-exec-local.s to i386-tls-ie-local.s
* Refactor tls-initial-exec-local.s to x86-64-tls-ie-local.s

llvm-svn: 367533
2019-08-01 09:25:34 +00:00
Igor Kudrin 510086b5e5 [ELF] Fix finding locations in messages for undefined hidden symbols.
Previously, when `--vs-diagnostics` was used, the linker printed
something like

  hidden(undef.s): error: undefined hidden symbol: foo
  >>> referenced by undef.s:15

Differential Revision: https://reviews.llvm.org/D65499

llvm-svn: 367515
2019-08-01 05:23:45 +00:00
Rui Ueyama d524c17029 Return early. NFC.
llvm-svn: 367200
2019-07-29 05:24:51 +00:00