Commit Graph

418 Commits

Author SHA1 Message Date
Eric Beckmann c8dba240b1 Revert "Revert "Revert "Revert "Switch external cvtres.exe for llvm's own resource library.""""
This reverts commit 147f45ff24456aea59575fa4ac16c8fa554df46a.

Revert "Revert "Revert "Revert "Replace trivial use of external rc.exe by writing our own .res file.""""

This reverts commit 61a90a67ed54a1f0dfeab457b65abffa129569e4.

The patches were intially reverted because they were causing a failure
on CrWinClangLLD.  Unfortunately, this was done haphazardly and didn't
compile, so the revert was reverted again quickly to fix this.  One that
was done, the revert of the revert was itself reverted.  This allowed me
to finally fix the actual bug in r307452.  This patch re-enables the
code path that had originally been causing the bug, now that it (should)
be fixed.

llvm-svn: 307460
2017-07-08 03:06:10 +00:00
Zachary Turner 41f6a453e4 Delete the pdb diff test.
This is failing on Linux for unknown reasons, so I need to
get the bots green while I investigate.

llvm-svn: 307459
2017-07-08 02:54:19 +00:00
Eric Beckmann 7c865f004b Add name offset flags, for parity with cvtres.exe.
Summary:
The original cvtres.exe sets the high bit when an identifier offset
points to a string.  Even though this is not mentioned in the spec, and
in fact does not seem to cause errors with most cases, for some reason
this causes a failure in Chromium where the new resource file is not
verified as a new version.  This patch sets this high bit flag, and also
adds a test case to check that the output of our library is always
identical to original cvtres.

Reviewers: zturner, ruiu

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D35099

llvm-svn: 307452
2017-07-07 23:23:53 +00:00
Zachary Turner 3a11fdf8ce [PDB] More changes to bring lld PDBs to parity with MSVC.
1) Don't write a /src/headerblock stream.  This appears to be
   written conditionally by MSVC, but it's not clear what the
   condition is.  For now, just remove it since we dont' know
   what it is anyway and the particular pdb we've checked in
   for the test doesn't have one.
2) Write a valid timestamp for the PDB file signature.  This
   leads to non-reproducible builds, but it matches the default
   behavior of link, so it should be out default as well.  If
   we need reproducibility, we should add a separate command
   line option for it that is off by default.
3) Write an empty FPO stream.  MSVC seems to always write an
   FPO stream.  This change makes the stream directory match
   up, although we still need to make the contents of the FPO
   stream match.

llvm-svn: 307436
2017-07-07 20:25:39 +00:00
Zachary Turner fe71c546e7 [llvm-pdbutil] Fix build.
Some platforms require an explicit specialization of std::hash
for PdbRaw_FeaturesSig.  Also a test involving case sensitivity
needed to be fixed.  For now that particular check just accepts
any path even if they're completely different.  Long term we
should output paths in the correct case to match MSVC.

llvm-svn: 307426
2017-07-07 19:00:06 +00:00
Zachary Turner 448dea419c Use windows path syntax when writing PDB module name.
Without this we would just append whatever the user
wrote on the command line, so if we're in C:\foo
and we run lld-link bar/baz.obj, we would write
C:\foo\bar/baz.obj in various places in the PDB.
MSVC linker does not do this, so we shouldn't either.
This fixes some differences in the diff test, so we
update the test as well.

Differential Revision: https://reviews.llvm.org/D35092

llvm-svn: 307423
2017-07-07 18:46:14 +00:00
Zachary Turner c1e93e5fa4 Fix some differences between lld and MSVC generated PDBs.
A couple of things were different about our generated PDBs.

1) We were outputting the wrong Version on the PDB Stream.
   The version we were setting was newer than what MSVC is setting.
   It's not clear what the implications are, but we change LLD
   to use PdbImplVC70, as MSVC does.
2) For the optional debug stream indices in the DBI Stream, we
   were outputting 0 to mean "the stream is not present".  MSVC
   outputs uint16_t(-1), which is the "correct" way to specify
   that a stream is not present.  So we fix that as well.
3) We were setting the PDB Stream signature to 0.  This is supposed
   to be the result of calling time(nullptr).  Although this leads
   to non-deterministic builds, a better way to solve that is by
   having a command line option explicitly for generating a
   reproducible build, and have the default behavior of lld-link
   match the default behavior of link.

To test this, I'm making use of the new and improved `pdb diff`
sub command.  To make it suitable for writing tests against, I had
to modify the diff subcommand slightly to print less verbose output.
Previously it would always print | <column> | <value1> | <value2> |
which is quite verbose, and the values are fragile.  All we really
want to know is "did we produce the same value as link?"  So I added
command line options to print a single character representing the
result status (different, identical, equivalent), and another to
hide the value display.  Note that just inspecting the diff output
used to write the test, you can see some things that are obviously
wrong.  That is just reflective of the fact that this is the state
of affairs today, not that we're asserting that this is "correct".
We can use this as a starting point to discover differences, fix
them, and update the test.

Differential Revision: https://reviews.llvm.org/D35086

llvm-svn: 307422
2017-07-07 18:45:56 +00:00
Zachary Turner cea99709c4 Fix lld tests after r307356.
llvm-svn: 307360
2017-07-07 05:41:25 +00:00
Reid Kleckner ffa5180f39 Fix pdb-scopes.test
llvm-svn: 307280
2017-07-06 16:41:15 +00:00
Reid Kleckner 3f85192930 [PDB] Fill in "Parent" and "End" fields of scope-like symbol records
Summary:
There are a variety of records that open scopes: function scopes, block
scopes, and inlined call site scopes. These symbol records contain
Parent and End fields with the offsets of other symbol records. The End
field contains the offset of the matching S_END or S_INLINESITE_END
record. The Parent field contains the offset of the parent record, or 0
if this is a top-level scope (i.e. a function).

With this change, `llvm-pdbutil pretty -all` no longer crashes on PDBs
produced by LLD. I haven't tried a real debugger yet.

Reviewers: zturner, ruiu

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D34898

llvm-svn: 307278
2017-07-06 16:39:32 +00:00
Eric Beckmann f6090b620e Revert "Revert "Revert "Switch external cvtres.exe for llvm's own resource library."""
This reverts commit ae21ee0b6cacbc1efaf4d42502e71da2f0eb45c3.

The initial revert was done in order to prevent ongoing errors on
chromium bots such as CrWinClangLLD.  However, this was done haphazardly
and I didn't realize there were test and compilation failures, so this
revert was reverted.  Now that those have been fixed, we can revert the
revert of the revert.

llvm-svn: 307227
2017-07-05 23:46:06 +00:00
Eric Beckmann 81979b038f Revert "Revert "Revert "Replace trivial use of external rc.exe by writing our own .res file."""
This reverts commit 5fecbbbe5049665d86834cf69d8f75db4f392308.

The initial revert was done in order to prevent ongoing errors on
chromium bots such as CrWinClangLLD.  However, this was done haphazardly
and I didn't realize there were test and compilation failures, so this
revert was reverted.  Now that those have been fixed, we can revert the
revert of the revert.

llvm-svn: 307226
2017-07-05 23:45:50 +00:00
Eric Beckmann 1d50926e71 Revert "Revert "Replace trivial use of external rc.exe by writing our own .res file.""
This reverts commit 8c8dce3b8f15d6ebaefc35ce88f15a85c8cdbd6e.

llvm-svn: 307191
2017-07-05 19:04:48 +00:00
Eric Beckmann 0eafa581a3 Revert "Revert "Switch external cvtres.exe for llvm's own resource library.""
This reverts commit 165e578e47f1cd38191120aad23a9020fb5476dd.

Forgot to run tests on this.

llvm-svn: 307190
2017-07-05 19:04:33 +00:00
Eric Beckmann 36793a0ecf Revert "Switch external cvtres.exe for llvm's own resource library."
This reverts commit 600d52c278e123dd08bee24c1f00932b55add8de.

This patch still seems to break CrWinClangLLD, reverting until I can
find root problem.

llvm-svn: 307189
2017-07-05 18:59:16 +00:00
Eric Beckmann 8cc9fd31e6 Revert "Replace trivial use of external rc.exe by writing our own .res file."
This patch still seems to break CrWinClangLLD, reverting this once more
until I can discover root problem.

This reverts commit 3dbbc8ce43be50ffde2b1c655c6d3a25796fe78b.

llvm-svn: 307188
2017-07-05 18:59:01 +00:00
Martin Storsjo 94fce50950 [COFF] Add initial support for linking ARM64 binaries
A plain empty entry point function that returns 0 seems to produce
a binary that loads and runs fine in wine.

Differential Revision: https://reviews.llvm.org/D34833

llvm-svn: 306963
2017-07-01 20:29:27 +00:00
Eric Beckmann 602afcf7e9 Revert "Revert "Replace trivial use of external rc.exe by writing our own .res file.""
Summary:
This reverts commit 51931072a7c9a52540baf76fc30ef391d2529a2f.

This revert was originally done because the integrations of the new
WindowsResource library into LLD was causing error in chromium, due to
bugs in how resource sections were handled.  These bugs were fixed,
meaning that the features may be reintegrated.

Subscribers: hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D34922

llvm-svn: 306941
2017-07-01 03:59:54 +00:00
Zachary Turner af8c75a8c0 [llvm-pdbutil] Output the symbol offset when dumping.
Type records have a unique type index, but symbol records do
not.  Instead, symbol records refer to other symbol records
by referencing their offset in the symbol stream.  In a sense
this is the analogue of the TypeIndex, but we are not printing
it in the dumper.  Printing it not only gives us more useful
information when manually investigating the contents of a PDB,
but also allows us to write better tests by enabling us to
verify that fields that reference other symbol records do
so correctly.

Differential Revision: https://reviews.llvm.org/D34906

llvm-svn: 306890
2017-06-30 21:35:00 +00:00
Eric Beckmann 5de7361097 Tighten up tests for .rsrc section emission.
Summary:
There have been bugs with the WindowsResource library, such as incorrect
symbols for addresses.  Directly checking the .rsrc in the final PE will
help ensure this doesn't happen again.

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D34900

llvm-svn: 306854
2017-06-30 18:22:11 +00:00
Martin Storsjo e24f434eb2 Update a test case after LLVM r306794
llvm-svn: 306796
2017-06-30 07:02:59 +00:00
Eric Beckmann d40dd64ff0 Revert "Replace trivial use of external rc.exe by writing our own .res file."
This reverts commit d4c7e9fc63c10dbab0c30186ef8575474a704496.

This is done in order to address the failure of CrWinClangLLD etc. bots.
These throw an error of "side-by-side configuration is incorrect" during
compilation, which sounds suspiciously related to these manifest
changes.

Revert "Switch external cvtres.exe for llvm's own resource library."

This reverts commit 71fe8ef283a9dab9a3f21432c98466cbc23990d1.

llvm-svn: 306618
2017-06-29 00:17:26 +00:00
Reid Kleckner a1001b8f38 [COFF] Allow debug info to relocate against discarded symbols
Summary:
In order to do this without switching on the symbol kind multiple times,
I created Defined::getChunkAndOffset and use that instead of
SymbolBody::getRVA in the inner relocation loop.

Now we get the symbol's chunk before switching over relocation types, so
we can test if it has been discarded outside the inner relocation type
switch. This also simplifies application of section relative
relocations. Previously we would switch on symbol kind to compute the
RVA, then the relocation type, and then the symbol kind again to get the
output section so we could subtract that from the symbol RVA. Now we
*always* have an OutputSection, so applying SECREL and SECTION
relocations isn't as much of a special case.

I'm still not quite happy with the cleanliness of this code. I'm not
sure what offsets and bases we should be using during the relocation
processing loop: VA, RVA, or OutputSectionOffset.

Reviewers: ruiu, pcc

Reviewed By: ruiu

Subscribers: majnemer, inglorion, llvm-commits, aprantl

Differential Revision: https://reviews.llvm.org/D34650

llvm-svn: 306566
2017-06-28 17:06:35 +00:00
Eric Beckmann 01822fa1c1 Add more stringent tests for the resource section of executables.
Summary: The testing on the resource section of executables produced by lld has been very lax, and allowed a major bug to go unnoticed when we switched from shelling out to cvtres.exe to using llvm's own library.  These additional tests should cover all the major failure points.

Reviewers: zturner, ruiu

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D34664

llvm-svn: 306465
2017-06-27 21:28:55 +00:00
Eric Beckmann 2a81089116 Replace trivial use of external rc.exe by writing our own .res file.
This patch removes the dependency on the external rc.exe tool by writing
a simple .res file using our own library. In this patch I also added an
explicit definition for the .res file magic.  Furthermore, I added a
unittest for embeded manifests and fixed a bug exposed by the test.

llvm-svn: 306311
2017-06-26 17:43:30 +00:00
Reid Kleckner eb8c0f9d51 [COFF] Fix SECREL and SECTION relocations against common symbols
Summary:
They do the obvious thing: provide the section index of .bss and the
offset of the symbol in .bss.

Reviewers: ruiu

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D34628

llvm-svn: 306304
2017-06-26 16:45:36 +00:00
Reid Kleckner 892d2a5767 Add .yaml as an lld test suffix
Over time we've started to add inputs and test cases using the .yaml
extension, which seems to be preferred over the .objtxt extension that
we were using initially. One nice thing about using .yaml is that it
triggers existing editor highlighting and formatting support.

Fix two pdb*.yaml test cases that I added that weren't being run as part
of check-lld.

llvm-svn: 306303
2017-06-26 16:42:44 +00:00
Reid Kleckner 502d4ce2e4 [COFF] Improve synthetic symbol handling
Summary:
The main change is that we can have SECREL and SECTION relocations
against ___safe_se_handler_table, which is important for handling the
debug info in the MSVCRT.

Previously we were using DefinedRelative for __safe_se_handler_table and
__ImageBase, and after we implement CFGuard, we plan to extend it to
handle __guard_fids_table, __guard_longjmp_table, and more.  However,
DefinedRelative is really only suitable for implementing __ImageBase,
because it lacks a Chunk, which you need in order to figure out the
output section index and output section offset when resolving SECREl and
SECTION relocations.

This change renames DefinedRelative to DefinedSynthetic and gives it a
Chunk. One wart is that __ImageBase doesn't have a chunk. It points to
the PE header, effectively. We could split DefinedRelative and
DefinedSynthetic if we think that's cleaner and creates fewer special
cases.

I also added safeseh.s, which checks that we don't emit a safe seh table
entries pointing to garbage collected handlers and that we don't emit a
table at all when there are no handlers.

Reviewers: ruiu

Reviewed By: ruiu

Subscribers: inglorion, pcc, llvm-commits, aprantl

Differential Revision: https://reviews.llvm.org/D34577

llvm-svn: 306293
2017-06-26 15:39:52 +00:00
Reid Kleckner 8456411e3b [COFF] Fix SECTION and SECREL relocation handling for absolute symbols
Summary:
For SECTION relocations against absolute symbols, MSVC emits the largest
output section index plus one. I've implemented that by threading a
global variable through DefinedAbsolute that is filled in by the Writer.
A more library-oriented approach would be to thread the Writer through
Chunk::writeTo and SectionChunk::applyRel*, but Rui seems to prefer
doing it this way.

MSVC rejects SECREL relocations against absolute symbols, but only when
the relocation is in a real output section. When the relocation is in a
CodeView debug info section destined for the PDB, it seems that this
relocation error is suppressed, and absolute symbols become zeros in the
object file. This is easily implemented by checking the input section
from which we're applying relocations.

This should fix errors about __safe_se_handler_table and
__guard_fids_table when linking the CRT and generating a PDB.

Reviewers: ruiu

Subscribers: aprantl, llvm-commits

Differential Revision: https://reviews.llvm.org/D34541

llvm-svn: 306071
2017-06-22 23:33:04 +00:00
Zachary Turner 7df69958f8 [llvm-pdbutil] Rename "raw" to "dump".
Now you run llvm-pdbutil dump <options>.  This is a followup
after having renamed the tool, whereas before raw was obviously
just the style of dumping, whereas now "dump" is the action to
perform with the "util".

llvm-svn: 306055
2017-06-22 20:57:39 +00:00
Reid Kleckner d0e6e24a53 [PDB] Add symbols to the PDB
Summary:
The main complexity in adding symbol records is that we need to
"relocate" all the type indices. Type indices do not have anything like
relocations, an opaque data structure describing where to find existing
type indices for fixups. The linker just has to "know" where the type
references are in the symbol records. I added an overload of
`discoverTypeIndices` that works on symbol records, and it seems to be
able to link the standard library.

Reviewers: zturner, ruiu

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D34432

llvm-svn: 305933
2017-06-21 17:25:56 +00:00
Rui Ueyama 28ea8c7ad7 [COFF] Set MajorLinkerVersion to 14 instead of 0.
This works around a strange interaction with Authenticode signatures,
in which a signed PE executable with {Major,Minor}LinkerVersion = 0.0
fails to validate on Windows 7 (but is OK on Windows 10). Setting the
linker version to 14.0 (which is what VS2015 outputs) makes it work
again.

Patch by Simon Tatham <simon.tatham@arm.com>.

llvm-svn: 305929
2017-06-21 16:42:08 +00:00
Rui Ueyama 2f740f74bb Define __guard_{iat,longjmp}_{count,table} symbols.
VC2017 contains these new symbols as undefined symobls. They are used
for /guard:cf. Since we do not support the control flow guard, but we
want to at least ignore these symbols so that we can link against VS2017
libraries.

Fixes https://bugs.chromium.org/p/chromium/issues/detail?id=727193.

llvm-svn: 305876
2017-06-21 02:26:19 +00:00
Reid Kleckner 91ef9de643 [codeview] YAMLize all section offsets and indices in symbol records
We forgot to serialize these because llvm-readobj didn't dump them. They
are typically all zeros in an object file. The linker fills them in with
relocations before adding them to the PDB. Now we can properly round
trip these symbols through pdb2yaml -> yaml2pdb.

I made these fields optional with a zero default so that we can elide
them from our test cases.

llvm-svn: 305857
2017-06-20 21:19:22 +00:00
Reid Kleckner f5bb738f75 [PDB] Don't emit debug info associated with dead chunks
Summary:
Previously we didn't add debug info chunks to the SparseChunks array, so
they didn't participate in section GC. Now we do.

Reviewers: ruiu

Subscribers: aprantl, llvm-commits

Differential Revision: https://reviews.llvm.org/D34356

llvm-svn: 305811
2017-06-20 17:14:09 +00:00
Reid Kleckner adea0ce7ca Fix build and PDB test on Linux
llvm-svn: 305716
2017-06-19 17:27:31 +00:00
Reid Kleckner 44cdb10964 [PDB] Start emitting source file and line information
Summary:
This is a first step towards getting line info to show up in VS and
windbg. So far, only llvm-pdbutil can parse the PDBs that we produce.
cvdump doesn't like something about our file checksum tables. I'll have
to dig into that next.

This patch adds a new DebugSubsectionRecordBuilder which takes bytes
directly from some other producer, such as a linker, and sticks it into
the PDB. Line tables only need to be relocated. No data needs to be
rewritten.

File checksums and string tables, on the other hand, need to be re-done.

Reviewers: zturner, ruiu

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D34257

llvm-svn: 305713
2017-06-19 17:21:45 +00:00
Eric Beckmann d135e8c039 Switch external cvtres.exe for llvm's own resource library.
In this patch, I flip the switch in DriverUtils from using the external
cvtres.exe tool to using the Windows Resource library in llvm.

I also fixed a bug where .rsrc sections were marked as discardable
memory and therefore were placed in the wrong order in the final PE.

Furthermore, I modified WindowsResource to write the coff directly to a
memory buffer instead of to file, also had it use the machine types
already declared in COFF.h instead creating my own enum.

Finally, I flipped the switch to allow all unit tests that had
previously run only on windows due to a winres dependency to run
cross-platform.

Reviewers: zturner, ruiu

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D34265

llvm-svn: 305592
2017-06-16 21:13:24 +00:00
Reid Kleckner f8b992dd08 Give associative-comdat lld test a triple
llvm-svn: 305585
2017-06-16 20:51:42 +00:00
Reid Kleckner 79ac99b3e8 [COFF] Drop unused comdat sections when GC is turned off
Summary:
Adds a "Discarded" bool to SectionChunk to indicate if the section was
discarded by COMDAT deduplication. The Writer still just checks
`isLive()`.

Fixes PR33446

Reviewers: ruiu

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D34288

llvm-svn: 305582
2017-06-16 20:47:19 +00:00
Zachary Turner 56ed9deafc Remove some non-determinstic numbers from pdb test.
llvm-svn: 305520
2017-06-15 22:41:45 +00:00
Zachary Turner 93f986a7dc Fix pdb embedded paths.
Obviously the paths are going to be machine dependent, so
don't try to match them in the test.

llvm-svn: 305518
2017-06-15 22:30:00 +00:00
Zachary Turner 6305545527 Resubmit "[llvm-pdbutil] rewrite the "raw" output style."
This resubmits commit c0c249e9f2ef83e1d1e5f166b50673d92f3579d7.

It was broken due to some weird template issues, which have
since been fixed.

llvm-svn: 305517
2017-06-15 22:24:24 +00:00
Zachary Turner da504b794c Revert "[llvm-pdbutil] rewrite the "raw" output style."
This reverts commit 83ea17ebf2106859a51fbc2a86031b44d33696ad.

This is failing due to some strange template problems, so reverting
until it can be straightened out.

llvm-svn: 305505
2017-06-15 20:55:51 +00:00
Saleem Abdulrasool bc7ff7027e COFF: add support for lib mode usage
When link is invoked with `/def:` and no input files, it behaves as if
`lib.exe` was invoked.  Emulate this behaviour, generating the import
library from the def file that was passed.  Because there is no input to
actually generate the dll, we simply process the def file early and exit
once we have created the import library.

llvm-svn: 305502
2017-06-15 20:39:58 +00:00
Zachary Turner b560fdf3b8 [llvm-pdbutil] rewrite the "raw" output style.
After some internal discussions, we agreed that the raw output style had
outlived its usefulness. It was originally created before we had even
thought of dumping to YAML, and it was intended to give us some insight
into the internals of a PDB file. Now we have YAML mode which does
almost exactly this but is more powerful in that it can round-trip back
to a PDB, which the raw mode could not do. So the raw mode had become
purely a maintenance burden.

One option was to just delete it. However, its original goal was to be
as readable as possible while staying close to the "metal" - i.e.
presenting the output in a way that maps directly to the underlying file
format. We don't actually need that last requirement anymore since it's
covered by the yaml mode, so we could repurpose "raw" mode to actually
just be as readable as possible.

This patch implements about 80% of the functionality previously in raw
mode, but in a completely different style that is more akin to what
cvdump outputs. Records are very compressed, often times appearing on
just one line. One nice thing about this is that it makes full record
matching easier, because you can grep for indices, names, and leaf types
on a single line often.

See the tests for some examples of what the new output looks like.

Note that this patch actually regresses the functionality of raw mode in
a few areas, but only because the patch was already unreasonably large
and going 100% would have been even worse. Specifically, this patch is
missing:

The ability to dump module debug subsections (checksums, lines, etc)
The ability to dump section headers
Aside from that everything is here. While goign through the tests fixing
them all up, I found many duplicate tests. They've been deleted. In
subsequent patches I will go through and re-add the missing
functionality.

Differential Revision: https://reviews.llvm.org/D34191

llvm-svn: 305495
2017-06-15 19:34:41 +00:00
Zachary Turner a8cfc29c9a Resubmit "[codeview] Make obj2yaml/yaml2obj support .debug$S..."
This was originally reverted because of some non-deterministic
failures on certain buildbots.  Luckily ASAN eventually caught
this as a stack-use-after-scope, so the fix is included in
this patch.

llvm-svn: 305393
2017-06-14 15:59:27 +00:00
Zachary Turner 0085dce221 Revert "[codeview] Make obj2yaml/yaml2obj support .debug$S..."
This is causing failures on linux bots with an invalid stream
read.  It doesn't repro in any configuration on Windows, so
reverting until I have a chance to investigate on Linux.

llvm-svn: 305371
2017-06-14 06:24:24 +00:00
Zachary Turner a3da4467fa [codeview] Make obj2yaml/yaml2obj support .debug$S/T sections.
This allows us to use yaml2obj and obj2yaml to round-trip CodeView
symbol and type information without having to manually specify the bytes
of the section. This makes for much easier to maintain tests. See the
tests under lld/COFF in this patch for example. Before they just said
SectionData: <blob> whereas now we can use meaningful record
descriptions. Note that it still supports the SectionData yaml field,
which could be useful for initializing a section to invalid bytes for
testing, for example.

Differential Revision: https://reviews.llvm.org/D34127

llvm-svn: 305366
2017-06-14 05:31:00 +00:00
Reid Kleckner 8cbdd0c0f2 [PDB] Add a module descriptor for every object file
Summary:
Expose the module descriptor index and fill it in for section
contributions.

Reviewers: zturner

Subscribers: llvm-commits, ruiu, hiraditya

Differential Revision: https://reviews.llvm.org/D34126

llvm-svn: 305296
2017-06-13 15:49:13 +00:00
Zachary Turner fa18b2f68c Fix broken LLD test.
llvm-svn: 305260
2017-06-13 00:30:09 +00:00
Zachary Turner 68ea80d0a7 Slightly better fix for dealing with no-id-stream PDBs.
The last fix required the user to manually add the required
feature.  This caused an LLD test to fail because I failed to
update LLD.  In practice we can hide this logic so it can just
be transparently added when we write the PDB.

llvm-svn: 305236
2017-06-12 21:46:51 +00:00
Peter Collingbourne 89061b2224 IR: Replace the "Linker Options" module flag with "llvm.linker.options" named metadata.
The new metadata is easier to manipulate than module flags.

Differential Revision: https://reviews.llvm.org/D31349

llvm-svn: 305227
2017-06-12 20:10:48 +00:00
Zachary Turner bd336e44d8 Rename llvm-pdbdump -> llvm-pdbutil.
This is to reflect the evolving nature of the tool as being
useful for more than just dumping PDBs, as it can do many other
things.

Differential Revision: https://reviews.llvm.org/D34062

llvm-svn: 305106
2017-06-09 20:46:17 +00:00
Reid Kleckner c2eaccb0af Revert "Merge IAT and ILT."
This reverts r303374. It breaks Chrome's IAT patching code:
http://crbug.com/729077

llvm-svn: 304584
2017-06-02 18:49:38 +00:00
Reid Kleckner 5a1ed789e1 Add test case for LLD/COFF for stdcall exports in .def files
The .def file parser changes I reverted broke this test case, and
exported "__imp__foo" instead of "__imp__foo@8". This was
http://crbug.com/728726.

llvm-svn: 304572
2017-06-02 17:48:06 +00:00
Rui Ueyama 69ae29b1d1 Do not allow delay-importing data symbols.
If you pass /delayload:<dllname> to the COFF linker, it creates thunks
so that DLLs are loaded when they are used for the first time instead of
load-time.

This mechanism do not work for data symbols as there's no way to trap
acccesses to data imported from DLLs. (Technically, I think if we do not
initially map dllimport tables in memory, we could actually trap accesses
and delay-load data symbols, but that's not what Windows do.)

This patch is to report an error when you try to delay-load data symbols.

Fixes https://bugs.llvm.org/show_bug.cgi?id=33106

Differential Revision: https://reviews.llvm.org/D33557

llvm-svn: 303890
2017-05-25 18:03:34 +00:00
Rui Ueyama 9aa82f76ac Garbage collect dllimported symbols.
This is a different implementation than r303225 (which was reverted
in r303270, re-submitted in r303304 and then re-reverted in r303527).

In the previous patch, I tried to add Live bit to each dllimported
symbol. It turned out that it didn't work with "oldnames.lib" which
contains a lot of weak aliases to dllimported symbols.

The way we handle weak aliases is to check if undefined symbols
can be resolved using weak aliases, and if so, memcpy the Defined
symbols to weak Undefined symbols, so that any references to weak
aliases automatically see defined symbols instead of undefined ones.

This memcpy happens before MarkLive kicks in.

That means we may have multiple copies of dllimported symbols. So
turning on one instance's Live bit is not enough.

This patch moves the Live bit to dllimport file. Since multiple
copies of dllsymbols still point to the same file, we can use it as the
central repository to keep track of liveness.

Differential Revision: https://reviews.llvm.org/D33520

llvm-svn: 303814
2017-05-24 22:30:06 +00:00
Rui Ueyama b6632d9cd1 Revert r303304: Re-submit r303225: Garbage collect dllimported symbols.
This reverts commit r303304 because it looks like the change
introduced a crash bug. At least after that change, LLD with thinlto
crashes when linking Chromium.

llvm-svn: 303527
2017-05-22 06:01:37 +00:00
Rui Ueyama a674943211 Set IMAGE_DLL_CHARACTERISTICS_NO_BIND.
Our output is not compatible with the Binding feature, so make it
explicit that.

Differential Revision: https://reviews.llvm.org/D33336

llvm-svn: 303378
2017-05-18 20:26:58 +00:00
Rui Ueyama 9fbfd76fe3 Merge IAT and ILT.
Previously, LLD-produced executables had IAT (Import Address Table) and
ILT (Import Lookup Table) as separate chunks of data, although their
contents are identical. My interpretation of the COFF spec when I wrote
the COFF linker is that they need to be separate tables even though they
are the same.

But Peter found that the Windows loader is fine with executables in
which IAT and ILT are merged. This is a patch to merge IAT and ILT.
I confirmed that an lld-link self-hosted with this patch works fine.

Fixes https://bugs.llvm.org/show_bug.cgi?id=33064

Differential Revision: https://reviews.llvm.org/D33326

llvm-svn: 303374
2017-05-18 19:59:22 +00:00
Zachary Turner 8a7508970a [COFF] Fix interaction between /DEBUG and /PDB
When /DEBUG is not specified, /PDB should be ignored.  When
/DEBUG is specified, a PDB should be output regardless of
whether or not /PDB is specified.  /PDB just overrides the
default name.

This patch implements this behavior, and adds some tests, while
also removing a dead option /DEBUGPDB which was unused in any
code.

Differential Revision: https://reviews.llvm.org/D33302

llvm-svn: 303352
2017-05-18 15:15:10 +00:00
Rui Ueyama cd41bc8dec Re-submit r303225: Garbage collect dllimported symbols.
This reverts re-submits r303225 which was reverted in r303270 because it
broke the sanitizer-windows bot.

The reason of the failure is that we were writing dead symbols to the
symbol table. I fixed the issue.

llvm-svn: 303304
2017-05-17 21:36:08 +00:00
Hans Wennborg e67c5f6b52 Revert r303225 "Garbage collect dllimported symbols."
and follow-up r303226 "Fix Windows buildbots."

This broke the sanitizer-windows buildbot.

> Previously, the garbage collector (enabled by default or by explicitly
> passing /opt:ref) did not kill dllimported symbols. As a result,
> dllimported symbols could be added to resulting executables' dllimport
> list even if no one was actually using them.
>
> This patch implements dllexported symbol garbage collection. Just like
> COMDAT sections, dllimported symbols now have Live bits to manage their
> liveness, and MarkLive marks reachable dllimported symbols.
>
> Fixes https://bugs.llvm.org/show_bug.cgi?id=32950
>
> Reviewers: pcc
>
> Subscribers: llvm-commits
>
> Differential Revision: https://reviews.llvm.org/D33264

llvm-svn: 303270
2017-05-17 16:22:03 +00:00
Rui Ueyama de83fec029 Fix Windows buildbots.
llvm-svn: 303226
2017-05-17 01:05:56 +00:00
Rui Ueyama 02df7a6cf1 Garbage collect dllimported symbols.
Summary:
Previously, the garbage collector (enabled by default or by explicitly
passing /opt:ref) did not kill dllimported symbols. As a result,
dllimported symbols could be added to resulting executables' dllimport
list even if no one was actually using them.

This patch implements dllexported symbol garbage collection. Just like
COMDAT sections, dllimported symbols now have Live bits to manage their
liveness, and MarkLive marks reachable dllimported symbols.

Fixes https://bugs.llvm.org/show_bug.cgi?id=32950

Reviewers: pcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D33264

llvm-svn: 303225
2017-05-17 00:35:50 +00:00
Rui Ueyama 35e828775f Merge a test YAML file and a test file.
llvm-svn: 303135
2017-05-16 01:16:20 +00:00
Rui Ueyama 3969acf8a4 Port recent changes to ELF/MapFile.cpp to COFF.
llvm-svn: 301754
2017-04-29 18:50:39 +00:00
Saleem Abdulrasool 5c98b74536 COFF: actually synthesize CONST imports properly
CONSTANT imports expect both the `_imp_` prefixed and non-prefixed
symbols should be added to the symbol table.  This allows for linking
symbols like _NSConcreteGlobalBlock in WinObjC.  The previous change
would generate the import library properly by handling the option but
would not consume the generated entry properly.

llvm-svn: 301657
2017-04-28 17:06:40 +00:00
Rui Ueyama a85572ebf0 COFF ICF: Merge only functions. Do not merge read-only data.
This seems to be the behavior of the MSVC linker. Previously, this
incompatibility caused nasty issues in chromium build a few times.

Differential Revision: https://reviews.llvm.org/D30363

llvm-svn: 301598
2017-04-27 23:03:22 +00:00
Rui Ueyama 95fe854332 Handle _LINK_ env string as command line parameters.
"_LINK_" environment varaible should be appended to the command line.
https://msdn.microsoft.com/en-us/library/6y6t9esh.aspx

Fixes https://bugs.llvm.org/show_bug.cgi?id=32756

llvm-svn: 301264
2017-04-24 22:20:03 +00:00
Bob Haarman 4110816253 [coff] for /msvclto, pass archive members with prevailing symbols first
Summary: When using /msvclto, lld and MSVC's linker both do their own symbol resolution. This can cause them to select different archive members, which can result in undefined references. This change avoids that situation by extracting archive members that are selected by lld and passing those to link.exe before any archives, so that MSVC's uses those objects for symbol resolution instead of different archive members.

Reviewers: pcc, rnk, ruiu

Reviewed By: pcc

Subscribers: llvm-commits, mehdi_amini

Differential Revision: https://reviews.llvm.org/D32317

llvm-svn: 301045
2017-04-21 21:38:01 +00:00
Saleem Abdulrasool 72fbd346c6 COFF: add support for CONSTANT exports
The CONSTANT export type is marked as obsolete, but link still supports
this.  Furthermore, WinObjC uses this for certain exports.  Add support
for this export type.

llvm-svn: 301013
2017-04-21 18:05:46 +00:00
Bob Haarman 63220d5203 [coff] fix test for msvclto
llvm-svn: 300616
2017-04-18 22:29:36 +00:00
Rui Ueyama 5c5bc7bc0a Accept not only lld-link but also LLD-LINK, for example.
Filenames are case-insensitive on Windows, so when we dispatch based
on argv0, we need to handle it case-insensitively.

Fixes https://bugs.llvm.org/show_bug.cgi?id=32637.

llvm-svn: 300087
2017-04-12 18:29:52 +00:00
Reid Kleckner 6e545ffc4e [PDB] Emit index/offset pairs for TPI and IPI streams
Summary:
This lets PDB readers lookup type record data by type index in O(log n)
time. It also enables makes `cvdump -t` work on PDBs produced by LLD.
cvdump will not dump a PDB that doesn't have an index-to-offset table.

The table is sorted by type index, and has an entry every 8KB. Looking
up a type record by index is a binary search of this table, followed by
a scan of at most 8KB.

Reviewers: ruiu, zturner, inglorion

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D31636

llvm-svn: 299958
2017-04-11 16:26:15 +00:00
Reid Kleckner c9bc745518 Remove .rdata section size checks from pdb.test, they are not the same on Windows and Unix
llvm-svn: 299795
2017-04-07 19:58:33 +00:00
Reid Kleckner f519ef0990 [PDB] Re-add RAW FileCheck testing to pdb.test
Update the expected output to match the current llvm-pdbdump raw output.

llvm-svn: 299793
2017-04-07 18:38:38 +00:00
Saleem Abdulrasool 671029daec COFF: support the /appcontainer flag
The /appcontainer flag indicates that the module may only be used inside
an application container (for isolation).  This has been supported by
link.exe since Windows 8.0.  It sets an additional bit in the PE DLL
Characteristics flag to indicate the behavioural change.

llvm-svn: 299728
2017-04-06 23:07:53 +00:00
Bob Haarman ac8f7fc07b [COFF] support /ERRORLIMIT option
Summary:
This adds support for reporting multiple errors in a single invocation of lld-link. The limit defaults to 20 and can be changed with the /ERRORLIMIT command line parameter, or set to unlimited by passing a value of 0.

This is a new attempt after r295507, which was reverted because opening files raced with exiting early, causing the test to be flaky. This version avoids the race by exiting before calling enqueuePath.

Reviewers: pcc, ruiu

Reviewed By: ruiu

Subscribers: llvm-commits, dblaikie

Differential Revision: https://reviews.llvm.org/D31688

llvm-svn: 299496
2017-04-05 00:43:54 +00:00
Bob Haarman bf24cb7189 COFF: tests for thinlto with mangled names
Summary: This change adds tests for r299268.

Reviewers: pcc, rnk

Reviewed By: pcc

Subscribers: mehdi_amini, llvm-commits, Prazek

Differential Revision: https://reviews.llvm.org/D31549

llvm-svn: 299272
2017-03-31 22:17:02 +00:00
Rui Ueyama 9a3e73383b Add parsing of mllvm command line options.
Patch by Alberto Magni
Original patch by Kyungwoo Lee <kyulee@microsoft.com>

Differential Revision: https://reviews.llvm.org/D31388

llvm-svn: 299106
2017-03-30 20:10:40 +00:00
Bob Haarman 2d2b5b09e0 COFF: include archive name in LTO object name
Summary: In the ELF linker, we create the buffer identifier for bitcode files by appending the object name to the archive name. This change makes the COFF linker do the same. Without the change, ThinLTO builds can fail with an error message about multiple ThinLTO modules per object file, caused by object files contained in different archives having the same name.

Reviewers: pcc, ruiu

Reviewed By: pcc

Subscribers: mehdi_amini

Differential Revision: https://reviews.llvm.org/D31402

llvm-svn: 298942
2017-03-28 21:20:06 +00:00
Reid Kleckner 5d57752c81 [PDB] Split item and type records when merging type streams
Summary: MSVC does this when producing a PDB.

Reviewers: ruiu

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D31316

llvm-svn: 298717
2017-03-24 17:26:38 +00:00
Reid Kleckner 77d3aa476e Add a /nopdb flag to disable PDB generation
This will be used in the sanitizer test suite, which wants to use DWARF
line tables.

At some point we should reconsider how LLD handles the long section
names required by DWARF debug sections.

llvm-svn: 298544
2017-03-22 19:49:12 +00:00
Reid Kleckner 3a0dea5243 Update LLD tests for codeview dumping changes
llvm-svn: 298475
2017-03-22 01:43:49 +00:00
Reid Kleckner 13bdbfb622 [COFF] Put the PDB next to the image if the user doesn't pass /PDB:
Summary: This is compatible with MSVC link.exe.

Reviewers: ruiu

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D31224

llvm-svn: 298467
2017-03-22 00:57:14 +00:00
Reid Kleckner 7668182e44 [COFF] Don't let /def override /out filename
Summary:
This also delays setting the output filename based on the first input
argument until after processing /def.

Fixes PR32354

Reviewers: ruiu, pcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D31152

llvm-svn: 298327
2017-03-21 00:12:51 +00:00
Peter Collingbourne db7447deed COFF: Do not create empty temporary archives for /msvclto.
The MSVC linker doesn't support them.

Differential Revision: https://reviews.llvm.org/D31067

llvm-svn: 298034
2017-03-17 02:04:22 +00:00
Rui Ueyama e1bf1365d7 Do not pass archive files containing non-native object files.
The MSVC linker doesn't like archive files containing non-native object
files.

When we are doing an LTO build, we may create archive files containing
both LLVM bitcode files and native object files. For example, if a
project contains assembly files and C++ files, we create native object
files for the assembly files and LLVM bitcode files for the C++ files.

With the /msvclto option, LLD passes archive files to the MSVC linker.
Previously, we didn't pass archive files if they contain at least one
bitcode files. That wasn't correct because the native object files that
weren't passed to the MSVC linker may be needed to complete linking.

In this patch, we create new temporary archive files to strip bitcode
files.

Differential Revision: https://reviews.llvm.org/D31053

llvm-svn: 297997
2017-03-16 21:19:36 +00:00
Zachary Turner 337e884643 Fix lld pdb test.
It was assuming various things about the PDB like file size
which are going to be too high maintenance to maintain in
a test.

llvm-svn: 297908
2017-03-15 22:58:10 +00:00
Rui Ueyama 7d271ae21d Fix wrong assertion failure.
Previously, if you have foo=bar in a definition file, this assertion
could fire because when symbols are read from file they could be mangled.
It seems that due to historical reasons underscore mangling scheme is
really ad-hoc, and I cannot find a clean way to handle this. I had
to just de-mangle symbols to search again.

llvm-svn: 297357
2017-03-09 04:47:33 +00:00
Rui Ueyama 9accea6feb Pass archive files to link.exe if they contain at least one native object file.
Some archive files created during chromium build contains both BC
and native files. If that's the case, we want to pass the archive
file to link.exe. Otherwise, the MSVC linker would complain that
there's an unresolved symbol in a given set of files.

I cannot explain why link.exe doesn't complain about the presence
of bitcode files in this case, but it seems link.exe doesn't touch BC.

llvm-svn: 297229
2017-03-07 21:26:10 +00:00
Rui Ueyama e0341db179 Do not pass archive files containing bitcode files to the MSVC Linker.
If /msvclto is specified, we compile bitcode files and pass it to the
MSVC linker, stripping all bitcode files. We haven't stripped archive
files, because I was thinking that the MSVC linker wouldn't touch files
in archive files. When we pass an object file to link.exe, all symbols
have been resolved already, so link.exe shoulnd't need any of the files
in archives.

It turns out that even though link.exe doesn't need to do that, it
seems to try to read each file in all archives. And if there's a non-
COFF file in an archive, it exists with an error message. So we need
to remove archives from the command line too.

llvm-svn: 297191
2017-03-07 19:45:53 +00:00
Bob Haarman 851b1f866b [COFF] added test for thinlto
Summary: Creates bitcode files suitable for use with ThinLTO, then checks that the linker can build an executable from them.

Reviewers: ruiu, pcc

Reviewed By: pcc

Subscribers: mehdi_amini, Prazek, llvm-commits

Differential Revision: https://reviews.llvm.org/D30277

llvm-svn: 296042
2017-02-23 23:42:24 +00:00
Rui Ueyama 85d54b050e Fix /msvclto.
Previously, bitcode files in library paths were passed to the MSVC linker.
This patch strips them.

llvm-svn: 295913
2017-02-23 00:26:42 +00:00
Rui Ueyama 99861f4730 Attempt to placate MSVC buildbot.
I really do not understand what is going on on some Windows buildbots,
but FileCheck command on some buildbot behaves like long lines were
truncated. I'll try to find a cause of the issue, but let me relax the
test so that they'll succeed on all buildbots.

llvm-svn: 295798
2017-02-22 00:32:56 +00:00
David Blaikie 4cdfe69982 Revert "[COFF] support /ERRORLIMIT option"
Behavior races on ErrorCount. If the enqueued paths are evaluated
eagerly (in enqueuePath) then the behavior is as the test expects. But
they may not be evaluated until the future is waited on, in run() -
which is after the early return/exit on ErrorCount. (this causes the
test to fail (because in the "/ERRORCOUNT:XYZ" test, no other errors
are printed), at least for me, on linux)

This reverts commit r295507.

llvm-svn: 295590
2017-02-19 02:25:47 +00:00
Bob Haarman b96b10102a [COFF] support /ERRORLIMIT option
Summary: This adds support for reporting multiple errors in a single invocation of lld-link. The limit defaults to 20 and can be changed with the /ERRORLIMIT command line parameter, or set to unlimited by passing a value of 0.

Reviewers: pcc, ruiu

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D29691

llvm-svn: 295507
2017-02-17 22:46:06 +00:00
Bob Haarman c87da86707 added missing "savetemps/" in test/COFF/savetemps.ll
llvm-svn: 294664
2017-02-09 23:31:47 +00:00