Commit Graph

780 Commits

Author SHA1 Message Date
Martin Storsjo 31fe4cd25d [MinGW] Support dllexport on i386
In MinGW configurations (GCC, or clang with a *-windows-gnu target),
the -export directives in the object file contains the undecorated
symbol name, while it is decorated in MSVC configurations. (On the
command line, link.exe takes an undecorated symbol name for the
-export argument though.)

Differential Revision: https://reviews.llvm.org/D37772

llvm-svn: 313174
2017-09-13 19:29:39 +00:00
Martin Storsjo 8278ba51d3 [COFF] Add support for the -wholearchive option
This fixes PR31824.

Differential Revision: https://reviews.llvm.org/D37709

llvm-svn: 313123
2017-09-13 07:28:03 +00:00
Peter Collingbourne 052e855e2b COFF: Implement ThinLTO cache and cache pruning support.
Differential Revision: https://reviews.llvm.org/D37607

llvm-svn: 312770
2017-09-08 00:50:50 +00:00
Peter Collingbourne 9e26e97955 COFF: PDB: Allow multiple modules with the same name.
It is possible for two modules to have the same name if they are
archive members with the same name, or if we are doing LTO (in which
case all modules will have the name "lto.tmp").

Differential Revision: https://reviews.llvm.org/D37589

llvm-svn: 312744
2017-09-07 20:39:46 +00:00
Eric Beckmann 0aa4b7d4c5 Fix crbug 759265 by suppressing llvm mt warnings.
Summary:
Previous would throw warning whenever libxml2 is not installed.  Now
only give this warning if merging manifest fails.

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D37240

llvm-svn: 312604
2017-09-06 01:50:36 +00:00
Nico Weber a05cbb8b95 lld-link: Add --rsp-quoting= flag.
This ports https://reviews.llvm.org/D19425 from clang /
https://reviews.llvm.org/D22015 from the ELF port to COFF lld. This can be
useful when linking COFF files on a posix host.

https://reviews.llvm.org/D37452

llvm-svn: 312594
2017-09-05 23:46:45 +00:00
Saleem Abdulrasool 3170ad7431 COFF: handle multiply defined symbols with different storage
If a symbol is locally defined and is DLL imported in another
translation unit, and the object with the locally defined version is
loaded prior to the imported version, then the linker will fail to
resolve the definition of the thunk and return the locally defined
symbol.  This will then be attempted to be cast to an import thunk,
which will clearly fail.

Only return the thunk if the symbol is inserted or a thunk is created.
Otherwise, report a duplication error.

llvm-svn: 312386
2017-09-01 22:12:10 +00:00
Zachary Turner abb17cc084 [llvm-pdbutil] Support dumping CodeView from object files.
We have llvm-readobj for dumping CodeView from object files, and
llvm-pdbutil has always been more focused on PDB.  However,
llvm-pdbutil has a lot of useful options for summarizing debug
information in aggregate and presenting high level statistical
views.  Furthermore, it's arguably better as a testing tool since
we don't have to write tests to conform to a state-machine like
structure where you match multiple lines in succession, each
depending on a previous match.  llvm-pdbutil dumps much more
concisely, so it's possible to use single-line matches in many
cases where as with readobj tests you have to use multi-line
matches with an implicit state machine.

Because of this, I'm adding object file support to llvm-pdbutil.
In fact, this mirrors the cvdump tool from Microsoft, which also
supports both object files and pdb files.  In the future we could
perhaps rename this tool llvm-cvutil.

In the meantime, this allows us to deep dive into object files
the same way we already can with PDB files.

llvm-svn: 312358
2017-09-01 20:06:56 +00:00
Rui Ueyama c1e101f356 Revert r312171: Support nested static libraries.
This reverts commit r312171 because it is pointed out that that's not a
correct fix (see https://bugs.llvm.org/show_bug.cgi?id=32674#c14) and
also because it broke buildbots.

llvm-svn: 312174
2017-08-30 21:22:59 +00:00
Rui Ueyama a68748659a Support nested static libraries.
MSVC link.exe supports nested static libraries. That is, an .a file can
contain other .a file as its member. It is reported that MySQL actually
depends on this feature.

Fixes https://bugs.llvm.org/show_bug.cgi?id=32674

llvm-svn: 312171
2017-08-30 20:55:41 +00:00
Eric Beckmann 87c6acf38a Integrate manifest merging library into LLD.
Summary: Now that the llvm-mt manifest merging libraries are complete, we may use them to merge manifests instead of needing to shell out to mt.exe.

Subscribers: mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D36255

llvm-svn: 311424
2017-08-22 03:15:28 +00:00
Zachary Turner d1de2f4f5e [llvm-pdbutil] Add support for dumping detailed module stats.
This adds support for dumping a summary of module symbols
and CodeView debug chunks.  This option prints a table for
each module of all of the symbols that occurred in the module
and the number of times it occurred and total byte size.  Then
at the end it prints the totals for the entire file.

Additionally, this patch adds the -jmc (just my code) option,
which suppresses modules which are from external libraries or
linker imports, so that you can focus only on the object files
and libraries that originate from your own source code.

llvm-svn: 311338
2017-08-21 14:53:25 +00:00
Martin Storsjo 92f32d0c53 [COFF] Don't produce weak aliases in import libraries
When creating an import library from lld, the cases with
Name != ExtName shouldn't end up as a weak alias, but as a real
export of the new name, which is what actually is exported from
the DLL.

This restores the behaviour of renamed exports to what it was in
4.0.

Differential Revision: https://reviews.llvm.org/D36634

llvm-svn: 310992
2017-08-16 05:23:00 +00:00
Martin Storsjo a50275cfe5 [COFF] Fix the name type for stdcall functions in import libraries
Since SVN r303491 and r304573, LLD used the COFFImportLibrary
functions from LLVM. These only had two names, Name and ExtName,
which wasn't enough to convey all the details of stdcall functions.

Stdcall functions got the wrong symbol name in the import library
itself in r303491, which is why it was reverted in r304561. When
re-landed and fixed in r304573 (after adding a test in r304572),
the symbol name itself in the import library ended up right, but the
name type of the import library entry was wrong.

This had the effect that linking to the import library succeeded
(contrary to in r303491, where linking to such an import library
failed), but at runtime, the symbol wouldn't be found in the DLL
(since the caller linked to the stdcall decorated name).

Differential Revision: https://reviews.llvm.org/D36545

llvm-svn: 310989
2017-08-16 05:13:25 +00:00
Zachary Turner 024323cb12 [LLD COFF/PDB] Incrementally update the build id.
Previously, our algorithm to compute a build id involved hashing the
executable and storing that as the GUID in the CV Debug Record chunk,
and setting the age to 1.

This breaks down in one very obvious case: a user adds some newlines to
a file, rebuilds, but changes nothing else. This causes new line
information and new file checksums to get written to the PDB, meaning
that the debug info is different, but the generated code would be the
same, so we would write the same build over again with an age of 1.

Anyone using a symbol cache would have a problem now, because the
debugger would open the executable, look at the age and guid, find a
matching PDB in the symbol cache and then load it. It would never copy
the new PDB to the symbol cache.

This patch implements the canonical Windows algorithm for updating
a build id, which is to check the existing executable first, and
re-use an existing GUID while bumping the age if it already
exists.

Differential Revision: https://reviews.llvm.org/D36758

llvm-svn: 310961
2017-08-15 21:31:41 +00:00
Martin Storsjo d2752aa9ec [COFF] Add support for aligncomm directives
These are emitted for comm symbols in object files, when targeting
a GNU environment.

Alternatively, just ignore them since we already align CommonChunk
to the natural size of the content (up to 32 bytes). That would only
trade away the possibility to overalign small symbols, which doesn't
sound like something that might not need to be handled?

Differential Revision: https://reviews.llvm.org/D36304

llvm-svn: 310871
2017-08-14 19:07:27 +00:00
Zachary Turner 302dc8bccf [PDB] Ignore all S_UDT symbols when writing PDBs.
We don't have the right algorithm for copying S_UDT symbols
from object files to the globals stream, and having it wrong
is worse than not having it at all, since it breaks display
of local variables of UDT types (for example, "dv Foo" fails
in our current implementation, but succeeds if the S_UDT records
are omitted).  Omit them until we fix the algorithm.

llvm-svn: 310867
2017-08-14 18:44:58 +00:00
Zachary Turner b57884e818 Fix some broken tests.
These were pending in a separate patch but I forgot to squash them
before comitting, and this one didn't go through.

llvm-svn: 310764
2017-08-11 21:14:01 +00:00
Zachary Turner ee9906d884 [LLD/PDB] Write actual records to the globals stream.
Previously we were writing an empty globals stream.  Windows
tools interpret this as "private symbols are not present in
this PDB", even when they are, so we need to fix this.  Regardless,
without it we don't have information about global variables, so
we need to fix it anyway.  This patch does that.

With this patch, the "lm" command in WinDbg correctly reports
that we have private symbols available, but the "dv" command
still refuses to display local variables.

Differential Revision: https://reviews.llvm.org/D36535

llvm-svn: 310743
2017-08-11 19:00:03 +00:00
Zachary Turner 5448dabbdd [PDB] Fix an issue writing the publics stream.
In the refactor to merge the publics and globals stream, a bug
was introduced that wrote the wrong value for one of the fields
of the PublicsStreamHeader.  This caused debugging in WinDbg
to break.

We had no way of dumping any of these fields, so in addition to
fixing the bug I've added dumping support for them along with a
test that verifies the correct value is written.

llvm-svn: 310439
2017-08-09 04:23:59 +00:00
Zachary Turner 946204c83e [PDB] Merge Global and Publics Builders.
The publics stream and globals stream are very similar. They both
contain a list of hash buckets that refer into a single shared stream,
the symbol record stream. Because of the need for each builder to manage
both an independent hash stream as well as a single shared record
stream, making the two builders be independent entities is not the right
design. This patch merges them into a single class, of which only a
single instance is needed to create all 3 streams.  PublicsStreamBuilder
and GlobalsStreamBuilder are now merged into the single GSIStreamBuilder
class, which writes all 3 streams at once.

Note that this patch does not contain any functionality change. So we're
still not yet writing any records to the globals stream. All we're doing
is making it so that when we do start writing records to the globals,
this refactor won't have to be part of that patch.

Differential Revision: https://reviews.llvm.org/D36489

llvm-svn: 310438
2017-08-09 04:23:25 +00:00
Zachary Turner 59e3ae827d [PDB] Fix linking of function symbols and local variables.
The compiler outputs PROC32_ID symbols into the object files
for functions, and these symbols have an embedded type index
which, when copied to the PDB, refer to the IPI stream.  However,
the symbols themselves are also converted into regular symbols
(e.g. S_GPROC32_ID -> S_GPROC32), and type indices in the regular
symbol records refer to the TPI stream.  So this patch applies
two fixes to function records.
  1. It converts ID symbols to the proper non-ID record type.
  2. After remapping the type index from the object file's index
     space to the PDB file/IPI stream's index space, it then
     remaps that index to the TPI stream's index space by.

Besides functions, during the remapping process we were also
discarding symbol record types which we did not recognize.
In particular, we were discarding S_BPREL32 records, which is
what MSVC uses to describe local variables on the stack.  So
this patch fixes that as well by copying them to the PDB.

Differential Revision: https://reviews.llvm.org/D36426

llvm-svn: 310394
2017-08-08 18:34:44 +00:00
Zachary Turner 676386ff30 [lld] Write the DataCRC to the output PDB.
llvm-svn: 310297
2017-08-07 20:23:45 +00:00
Zachary Turner 6b7db9a66c More PDB buildbot fixes.
llvm-svn: 310131
2017-08-04 21:18:17 +00:00
Zachary Turner dd6f4368d6 Fix broken PDB tests.
llvm-svn: 310130
2017-08-04 21:15:12 +00:00
Zachary Turner fb1cd5090c [llvm-pdbutil] Dump image section headers.
Image section headers are stored in the DBI stream, but we
had no way to dump them.  This patch adds dumping support,
along with some tests that LLD actually dumps them correctly.

Differential Revision: https://reviews.llvm.org/D36332

llvm-svn: 310107
2017-08-04 20:02:38 +00:00
Reid Kleckner a5d3909678 [PDB] Loosen checks for section contribution sizes
The PDB debug data directory entry has an absolute path in it. This will
make it different on every machine.

llvm-svn: 309989
2017-08-03 21:20:41 +00:00
Reid Kleckner 175af4bcc7 [PDB] Fix section contributions
Summary:
PDB section contributions are supposed to use output section indices and
offsets, not input section indices and offsets.

This allows the debugger to look up the index of the module that it
should look up in the modules stream for symbol information. With this
change, windbg can now find line tables, but it still cannot print local
variables.

Fixes PR34048

Reviewers: zturner

Subscribers: hiraditya, ruiu, llvm-commits

Differential Revision: https://reviews.llvm.org/D36285

llvm-svn: 309987
2017-08-03 21:15:09 +00:00
Reid Kleckner 9dcd3e78ae [PDB] Improve rsds test to check that we fill in the file offset as well as the RVA
llvm-svn: 309902
2017-08-02 23:32:26 +00:00
Zachary Turner 8d927b6bf9 [lld/pdb] Add an empty globals stream.
We don't write any actual symbols to this stream yet, but for
now we just create the stream and hook it up to the appropriate
places and give it a valid header.

Differential Revision: https://reviews.llvm.org/D35290

llvm-svn: 309608
2017-07-31 19:36:08 +00:00
Reid Kleckner eacdf04fdd [PDB] Write public symbol records and the publics hash table
Summary:
MSVC link.exe records all external symbol names in the publics stream.
It provides similar functionality to an ELF .symtab.

Reviewers: zturner, ruiu

Subscribers: hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D35871

llvm-svn: 309303
2017-07-27 18:25:59 +00:00
Nico Weber 268f89d540 lld: /manifestuac:no shouldn't disable /manifestdependency:
Matches link.exe
https://reviews.llvm.org/D35872

llvm-svn: 309231
2017-07-26 23:38:10 +00:00
Martin Storsjo 38608c0975 [COFF, ARM64] Handle ADRP immediate offsets in relocations
Also handle overflow correctly in LDR/STR relocations. Even if the
offset range of a 8 byte LDR instruction is 15 bit (even if the immediate
itself is 12 bit) due to a 3 bit shift, only include up to 12 bits of offset
after doing the relocation, by limiting the range of the immediate by the
number of shifted bits.

Differential Revision: https://reviews.llvm.org/D35792

llvm-svn: 309175
2017-07-26 20:51:47 +00:00
Martin Storsjo 82eaf6cb50 [COFF] Add support for delay loading DLLs on ARM
Differential Revision: https://reviews.llvm.org/D35768

llvm-svn: 309017
2017-07-25 20:00:37 +00:00
Nico Weber 578363ac08 Attempt to fix lld tests on Windows after 308998.
The test used /manifestinput: without /manifest:embed, which isn't actually
supported.  Just remove this part of the test for now; if it's important to
check this the llvm-readobj part should be extended to check this.

llvm-svn: 309002
2017-07-25 18:39:38 +00:00
Nico Weber a7a2c44e70 lld: only write .manifest files if /manifest is passed, PR33925
Also emit an error if /manifestinput: is used without /manifest:embed.
Increases compatibility with link.exe

https://reviews.llvm.org/D35842

llvm-svn: 308998
2017-07-25 18:08:03 +00:00
Martin Storsjo 405b5bcc36 [COFF] Correctly set the thumb bit in DLL export addresses
The same adjustment is already done for the entry point in
Writer.cpp and for relocations that point to executable code
in Chunks.cpp.

Differential Revision: https://reviews.llvm.org/D35767

llvm-svn: 308953
2017-07-25 06:10:44 +00:00
Martin Storsjo 25712667cb [COFF] Add a test for producing DLLs and import libraries for ARM64
This is a test for LLVM SVN r308951.

Differential Revision: https://reviews.llvm.org/D35814

llvm-svn: 308952
2017-07-25 06:08:25 +00:00
Martin Storsjo 5ae7649a4e [COFF] Support 128 bit SIMD/FP ldr/str in IMAGE_REL_ARM64_PAGEOFFSET_12L
Also extend the tests for IMAGE_REL_ARM64_PAGEOFFSET_12L to test
all 8/16/32/64 bit GPR and 8/16/32/64/128 SIMD/FP bit ldr/str variants,
and a ldr with an existing offset.

Differential revision: https://reviews.llvm.org/D35647

llvm-svn: 308631
2017-07-20 16:48:33 +00:00
Martin Storsjo b3c97b9623 [COFF] Align import address chunks to the pointer size
This fixes cases on ARM64 when importing from more than one DLL,
in case the imports from the first DLL ended up unaligned.

When fixing up a IMAGE_REL_ARM64_PAGEOFFSET_12L, which shifts the
offset by the load/store size, check that the shift doesn't discard
any bits. (This would also detect if the import address chunks were
unaligned.)

Differential revision: https://reviews.llvm.org/D35640

llvm-svn: 308585
2017-07-20 05:49:54 +00:00
Reid Kleckner d91ca7666f [PDB] Emit TPI and IPI hashes
llvm-svn: 308497
2017-07-19 17:26:07 +00:00
Saleem Abdulrasool 2334adeb02 test: remove duplicated test
This test is folded into implib-name.  Don't bother with the racy test.
The use of %T results in left-overs from previous tests which write out
the same import library as this test.

llvm-svn: 308409
2017-07-19 02:39:00 +00:00
Saleem Abdulrasool ace2fa7da4 COFF: improve link conformance for import names
Improve the link conformance for the import name embedded into the
import library.  This requires the associated change to the LLVM portion
for the DEF file parser.  The import file generation embeds a different
name based on whether the driver is invoked as "link" or "lib".
Furthermore, the LIBRARY keyword in the DEF file influences the import
name.  The behaviour can be summarised according to the following table:

      | LIBRARY w/ ext |   LIBRARY w/o ext   | no LIBRARY
 -----+----------------+---------------------+------------------
 LINK | {value}        | {value}.{.dll/.exe} | {output name}
  LIB | {value}        | {value}.dll         | {output name}.dll

llvm-svn: 308407
2017-07-19 02:01:27 +00:00
Saleem Abdulrasool 08c83ea0f3 test: add COFF test for library extension handling
Ensure that an extension other than .dll, .exe is preserved as per the
behaviour of link.

llvm-svn: 308386
2017-07-18 22:25:07 +00:00
Jon Chesterfield e0ca2ff070 [LLD] Mark a number of x86 only tests to require x86
Noticed while testing for an out of tree target. There are probably more tests that should be so marked.
I'm not sure who owns these tests so I've added a few names I recognise from the recent history.

With advice from probinson, ruiu, rafael and dramatically improved by davidb. Thank you all!

Differential Revision: https://reviews.llvm.org/D34685

llvm-svn: 308335
2017-07-18 18:40:50 +00:00
Shoaib Meenai 9a61a791ac [COFF] Accept discarded relocations in DWARF debug sections
DWARF debug sections can also contain relocations against symbols in
discared segments. LLD should accept such relocations.

Differential Revision: https://reviews.llvm.org/D35526

llvm-svn: 308315
2017-07-18 15:11:05 +00:00
Reid Kleckner 651db91c2d [PDB] Merge in types and items from type servers (/Zi)
Summary:
Object files compiled with /Zi emit type information into a type server
PDB. The .debug$S section will contain a single TypeServer2Record with
the absolute path and GUID of the type server. LLD needs to load the
type server PDB and merge all types and items it finds in it into the
destination PDB.

Depends on D35495

Reviewers: ruiu, inglorion

Subscribers: zturner, llvm-commits

Differential Revision: https://reviews.llvm.org/D35504

llvm-svn: 308235
2017-07-18 00:21:25 +00:00
Reid Kleckner 67653ee086 [codeview] Fix YAML for LF_TYPESERVER2 by hoisting PDB_UniqueId
Summary:
We were treating the GUIDs in TypeServer2Record as strings, and the
non-ASCII bytes in the GUID would not round-trip through YAML.

We already had the PDB_UniqueId type portably represent a Windows GUID,
but we need to hoist that up to the DebugInfo/CodeView library so that
we can use it in the TypeServer2Record as well as in PDB parsing code.

Reviewers: inglorion, amccarth

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D35495

llvm-svn: 308234
2017-07-17 23:59:44 +00:00
Reid Kleckner a842cd75e2 [codeview] Remove TypeServerHandler and PDBTypeServerHandler
Summary:
Instead of wiring these through the CVTypeVisitor interface, clients
should inspect the CVTypeArray before visiting it and potentially load
up the type server's TPI stream if they need it.

No tests relied on this functionality because LLD was the only client.

Reviewers: ruiu

Subscribers: mgorny, hiraditya, zturner, llvm-commits

Differential Revision: https://reviews.llvm.org/D35394

llvm-svn: 308212
2017-07-17 20:28:06 +00:00
Reid Kleckner af88a910fd [CodeView] Dump BuildInfoSym and ProcSym type indices
I need to print the type index in hex so that I can match it in
FileCheck for a test I'm writing.

llvm-svn: 308107
2017-07-15 18:10:39 +00:00
Reid Kleckner 3b8acb2c5a [COFF] Bounds check relocations
Summary:
This would have caught the invalid object file I used in my test case in
r307726. The OOB was only caught by ASan later, which is slow and
doesn't work on some platforms. LLD should do some basic input
validation itself. This check isn't perfect, so relocations can reach
OOB by up to seven bytes, but it's better than what we had and probably
cheap.

Reviewers: ruiu

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D35371

llvm-svn: 307948
2017-07-13 20:29:59 +00:00
Reid Kleckner 6597c28d76 [PDB] Fix type server handling for archives
Summary:
This fixes type indices for SDK or CRT static archives. Previously we'd
try to look next to the archive object file path, which would not exist
on the local machine.

Also error out if we can't resolve a type server record. Hypothetically
we can recover from this error by discarding debug info for this object,
but that is not yet implemented.

Reviewers: ruiu, amccarth

Subscribers: aprantl, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D35369

llvm-svn: 307946
2017-07-13 20:12:23 +00:00
Reid Kleckner 03b5baf480 Re-land "[PDB] Use a more appropriate message for a fatal error"
This re-commits r307726 with valid line table relocations.

llvm-svn: 307829
2017-07-12 18:49:43 +00:00
Adrian McCarthy a6c21919d0 Temporary fix to green up the build bots on behalf of zturner who's on vacation.
llvm-svn: 307823
2017-07-12 18:10:02 +00:00
Vitaly Buka e970c2b706 Revert "[PDB] Use a more appropriate message for a fatal error"
Revert "[PDB] Tweak bad type index error handling"

check-lld with asan detects use-after-poison.

This reverts commits r307733 and r307726.

llvm-svn: 307752
2017-07-12 04:34:17 +00:00
Reid Kleckner 2bf9ab2d62 [PDB] Tweak bad type index error handling
Translate invalid type indices to a sentinel value instead of skipping
the record. Skipping records isn't a good recovery method, because we
can skip a scope open or close record, which will confuse the scope
management code.

We currently have lots of invalid type indices on Microsoft-provided
standard libraries, because the LF_TYPESERVER2 records contain absolute
paths that are only valid on their build servers. Our type server
handlers need to look at other things (GUIDs) to find these type server
PDBs.

llvm-svn: 307726
2017-07-11 22:37:50 +00:00
Martin Storsjo 2779165d8c [COFF] Add initial support for some ARM64 relocations and import thunks
This is enough to link a working hello world executable, with
a call to an imported function, a string constant passed to
the imported function, and loads from a global variable.

Differential Revision: https://reviews.llvm.org/D34964

llvm-svn: 307629
2017-07-11 07:22:44 +00:00
Zachary Turner 7eaf1d96ad [lld/pdb] Create an empty public symbol record stream.
This is part of the continuing effort to increase parity between
LLD and MSVC PDBs.  link still doesn't like our PDBs, so the most
obvious thing to check was whether adding an empty publics stream
would get it to do something else.  It still fails in the same way
but at least this removes one more variable from the equation.
The next logical step would be to try creating an empty globals
stream.

Differential Revision: https://reviews.llvm.org/D35224

llvm-svn: 307598
2017-07-10 22:40:20 +00:00
Zachary Turner 6708e0b45e [lld/pdb] Add some basic linker module symbols.
Differential Revision: https://reviews.llvm.org/D35152

llvm-svn: 307590
2017-07-10 21:01:37 +00:00
Zachary Turner a9d944fd6f Resubmit "Add pdb-diff test."
This was originally reverted because of two issues.
  1) Printing ANSI color escape codes even when outputting to
     a file
  2) Module name comparisons were failing when comparing a PDB
     generated on one machine to a PDB generated on another
     machine.

I attempted to fix #2 by adding command line options which let
you specify prefixes to strip from the beginning of embedded
paths, which effectively lets us specify a path to "base" each
PDB from and only compare the parts under the base.  But this is
tricky because PDB paths always use Windows path syntax, even
when they are created on non-Windows hosts.  A problem still
existed when constructing the prefix to strip, where we were
accidentally using a host-specific path separator instead of
a Windows path separator.

This resubmission fixes the issue on Linux (and I have verified
that the test now passes on Linux).

llvm-svn: 307571
2017-07-10 19:16:49 +00:00
Zachary Turner ba3836bc78 Revert "Build fixes for pdb-diff test."
This reverts commit 180af3fdbdb17ec35b45ec1f925fd743b28d37e1.

This is still breaking due to linux-specific path differences.

llvm-svn: 307559
2017-07-10 17:32:47 +00:00
Zachary Turner 6da7a3058e Fix pdb-diff test.
A test was checked in on Friday that worked by checking in an
object file and PDB generated locally by MSVC, and then having
the test run lld-link on the object file and diffing LLD's PDB
against the checked in PDB.

This failed because part of the diffing algorithm involves
determining if two modules are the same, and if so drilling into
the module and diffing individual fields of the module.  The
only thing we can use to make this determination though is the
"name" of the module, which is a path to where the module (obj
file) was read from on the machine where it was linked.  This
fails for obvious reasons when comparing a PDB generated on one
machine to a PDB on another machine.

The fix employed here is to add two command line options to the
diff subcommand, which allow the user to specify a "binary root
path".  The bin root path, if specified, is stripped from the
beginning of any embedded PDB paths.  The test is updated to
specify the user's local test output directory for the left
PDB, and is hardcoded to the location where the original PDB
was created for the right PDB.  This way all the equivalence
comparisons should succeed.

llvm-svn: 307555
2017-07-10 16:52:15 +00:00
Eric Beckmann c8dba240b1 Revert "Revert "Revert "Revert "Switch external cvtres.exe for llvm's own resource library.""""
This reverts commit 147f45ff24456aea59575fa4ac16c8fa554df46a.

Revert "Revert "Revert "Revert "Replace trivial use of external rc.exe by writing our own .res file.""""

This reverts commit 61a90a67ed54a1f0dfeab457b65abffa129569e4.

The patches were intially reverted because they were causing a failure
on CrWinClangLLD.  Unfortunately, this was done haphazardly and didn't
compile, so the revert was reverted again quickly to fix this.  One that
was done, the revert of the revert was itself reverted.  This allowed me
to finally fix the actual bug in r307452.  This patch re-enables the
code path that had originally been causing the bug, now that it (should)
be fixed.

llvm-svn: 307460
2017-07-08 03:06:10 +00:00
Zachary Turner 41f6a453e4 Delete the pdb diff test.
This is failing on Linux for unknown reasons, so I need to
get the bots green while I investigate.

llvm-svn: 307459
2017-07-08 02:54:19 +00:00
Eric Beckmann 7c865f004b Add name offset flags, for parity with cvtres.exe.
Summary:
The original cvtres.exe sets the high bit when an identifier offset
points to a string.  Even though this is not mentioned in the spec, and
in fact does not seem to cause errors with most cases, for some reason
this causes a failure in Chromium where the new resource file is not
verified as a new version.  This patch sets this high bit flag, and also
adds a test case to check that the output of our library is always
identical to original cvtres.

Reviewers: zturner, ruiu

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D35099

llvm-svn: 307452
2017-07-07 23:23:53 +00:00
Zachary Turner 3a11fdf8ce [PDB] More changes to bring lld PDBs to parity with MSVC.
1) Don't write a /src/headerblock stream.  This appears to be
   written conditionally by MSVC, but it's not clear what the
   condition is.  For now, just remove it since we dont' know
   what it is anyway and the particular pdb we've checked in
   for the test doesn't have one.
2) Write a valid timestamp for the PDB file signature.  This
   leads to non-reproducible builds, but it matches the default
   behavior of link, so it should be out default as well.  If
   we need reproducibility, we should add a separate command
   line option for it that is off by default.
3) Write an empty FPO stream.  MSVC seems to always write an
   FPO stream.  This change makes the stream directory match
   up, although we still need to make the contents of the FPO
   stream match.

llvm-svn: 307436
2017-07-07 20:25:39 +00:00
Zachary Turner fe71c546e7 [llvm-pdbutil] Fix build.
Some platforms require an explicit specialization of std::hash
for PdbRaw_FeaturesSig.  Also a test involving case sensitivity
needed to be fixed.  For now that particular check just accepts
any path even if they're completely different.  Long term we
should output paths in the correct case to match MSVC.

llvm-svn: 307426
2017-07-07 19:00:06 +00:00
Zachary Turner 448dea419c Use windows path syntax when writing PDB module name.
Without this we would just append whatever the user
wrote on the command line, so if we're in C:\foo
and we run lld-link bar/baz.obj, we would write
C:\foo\bar/baz.obj in various places in the PDB.
MSVC linker does not do this, so we shouldn't either.
This fixes some differences in the diff test, so we
update the test as well.

Differential Revision: https://reviews.llvm.org/D35092

llvm-svn: 307423
2017-07-07 18:46:14 +00:00
Zachary Turner c1e93e5fa4 Fix some differences between lld and MSVC generated PDBs.
A couple of things were different about our generated PDBs.

1) We were outputting the wrong Version on the PDB Stream.
   The version we were setting was newer than what MSVC is setting.
   It's not clear what the implications are, but we change LLD
   to use PdbImplVC70, as MSVC does.
2) For the optional debug stream indices in the DBI Stream, we
   were outputting 0 to mean "the stream is not present".  MSVC
   outputs uint16_t(-1), which is the "correct" way to specify
   that a stream is not present.  So we fix that as well.
3) We were setting the PDB Stream signature to 0.  This is supposed
   to be the result of calling time(nullptr).  Although this leads
   to non-deterministic builds, a better way to solve that is by
   having a command line option explicitly for generating a
   reproducible build, and have the default behavior of lld-link
   match the default behavior of link.

To test this, I'm making use of the new and improved `pdb diff`
sub command.  To make it suitable for writing tests against, I had
to modify the diff subcommand slightly to print less verbose output.
Previously it would always print | <column> | <value1> | <value2> |
which is quite verbose, and the values are fragile.  All we really
want to know is "did we produce the same value as link?"  So I added
command line options to print a single character representing the
result status (different, identical, equivalent), and another to
hide the value display.  Note that just inspecting the diff output
used to write the test, you can see some things that are obviously
wrong.  That is just reflective of the fact that this is the state
of affairs today, not that we're asserting that this is "correct".
We can use this as a starting point to discover differences, fix
them, and update the test.

Differential Revision: https://reviews.llvm.org/D35086

llvm-svn: 307422
2017-07-07 18:45:56 +00:00
Zachary Turner cea99709c4 Fix lld tests after r307356.
llvm-svn: 307360
2017-07-07 05:41:25 +00:00
Reid Kleckner ffa5180f39 Fix pdb-scopes.test
llvm-svn: 307280
2017-07-06 16:41:15 +00:00
Reid Kleckner 3f85192930 [PDB] Fill in "Parent" and "End" fields of scope-like symbol records
Summary:
There are a variety of records that open scopes: function scopes, block
scopes, and inlined call site scopes. These symbol records contain
Parent and End fields with the offsets of other symbol records. The End
field contains the offset of the matching S_END or S_INLINESITE_END
record. The Parent field contains the offset of the parent record, or 0
if this is a top-level scope (i.e. a function).

With this change, `llvm-pdbutil pretty -all` no longer crashes on PDBs
produced by LLD. I haven't tried a real debugger yet.

Reviewers: zturner, ruiu

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D34898

llvm-svn: 307278
2017-07-06 16:39:32 +00:00
Eric Beckmann f6090b620e Revert "Revert "Revert "Switch external cvtres.exe for llvm's own resource library."""
This reverts commit ae21ee0b6cacbc1efaf4d42502e71da2f0eb45c3.

The initial revert was done in order to prevent ongoing errors on
chromium bots such as CrWinClangLLD.  However, this was done haphazardly
and I didn't realize there were test and compilation failures, so this
revert was reverted.  Now that those have been fixed, we can revert the
revert of the revert.

llvm-svn: 307227
2017-07-05 23:46:06 +00:00
Eric Beckmann 81979b038f Revert "Revert "Revert "Replace trivial use of external rc.exe by writing our own .res file."""
This reverts commit 5fecbbbe5049665d86834cf69d8f75db4f392308.

The initial revert was done in order to prevent ongoing errors on
chromium bots such as CrWinClangLLD.  However, this was done haphazardly
and I didn't realize there were test and compilation failures, so this
revert was reverted.  Now that those have been fixed, we can revert the
revert of the revert.

llvm-svn: 307226
2017-07-05 23:45:50 +00:00
Eric Beckmann 1d50926e71 Revert "Revert "Replace trivial use of external rc.exe by writing our own .res file.""
This reverts commit 8c8dce3b8f15d6ebaefc35ce88f15a85c8cdbd6e.

llvm-svn: 307191
2017-07-05 19:04:48 +00:00
Eric Beckmann 0eafa581a3 Revert "Revert "Switch external cvtres.exe for llvm's own resource library.""
This reverts commit 165e578e47f1cd38191120aad23a9020fb5476dd.

Forgot to run tests on this.

llvm-svn: 307190
2017-07-05 19:04:33 +00:00
Eric Beckmann 36793a0ecf Revert "Switch external cvtres.exe for llvm's own resource library."
This reverts commit 600d52c278e123dd08bee24c1f00932b55add8de.

This patch still seems to break CrWinClangLLD, reverting until I can
find root problem.

llvm-svn: 307189
2017-07-05 18:59:16 +00:00
Eric Beckmann 8cc9fd31e6 Revert "Replace trivial use of external rc.exe by writing our own .res file."
This patch still seems to break CrWinClangLLD, reverting this once more
until I can discover root problem.

This reverts commit 3dbbc8ce43be50ffde2b1c655c6d3a25796fe78b.

llvm-svn: 307188
2017-07-05 18:59:01 +00:00
Martin Storsjo 94fce50950 [COFF] Add initial support for linking ARM64 binaries
A plain empty entry point function that returns 0 seems to produce
a binary that loads and runs fine in wine.

Differential Revision: https://reviews.llvm.org/D34833

llvm-svn: 306963
2017-07-01 20:29:27 +00:00
Eric Beckmann 602afcf7e9 Revert "Revert "Replace trivial use of external rc.exe by writing our own .res file.""
Summary:
This reverts commit 51931072a7c9a52540baf76fc30ef391d2529a2f.

This revert was originally done because the integrations of the new
WindowsResource library into LLD was causing error in chromium, due to
bugs in how resource sections were handled.  These bugs were fixed,
meaning that the features may be reintegrated.

Subscribers: hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D34922

llvm-svn: 306941
2017-07-01 03:59:54 +00:00
Zachary Turner af8c75a8c0 [llvm-pdbutil] Output the symbol offset when dumping.
Type records have a unique type index, but symbol records do
not.  Instead, symbol records refer to other symbol records
by referencing their offset in the symbol stream.  In a sense
this is the analogue of the TypeIndex, but we are not printing
it in the dumper.  Printing it not only gives us more useful
information when manually investigating the contents of a PDB,
but also allows us to write better tests by enabling us to
verify that fields that reference other symbol records do
so correctly.

Differential Revision: https://reviews.llvm.org/D34906

llvm-svn: 306890
2017-06-30 21:35:00 +00:00
Eric Beckmann 5de7361097 Tighten up tests for .rsrc section emission.
Summary:
There have been bugs with the WindowsResource library, such as incorrect
symbols for addresses.  Directly checking the .rsrc in the final PE will
help ensure this doesn't happen again.

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D34900

llvm-svn: 306854
2017-06-30 18:22:11 +00:00
Martin Storsjo e24f434eb2 Update a test case after LLVM r306794
llvm-svn: 306796
2017-06-30 07:02:59 +00:00
Eric Beckmann d40dd64ff0 Revert "Replace trivial use of external rc.exe by writing our own .res file."
This reverts commit d4c7e9fc63c10dbab0c30186ef8575474a704496.

This is done in order to address the failure of CrWinClangLLD etc. bots.
These throw an error of "side-by-side configuration is incorrect" during
compilation, which sounds suspiciously related to these manifest
changes.

Revert "Switch external cvtres.exe for llvm's own resource library."

This reverts commit 71fe8ef283a9dab9a3f21432c98466cbc23990d1.

llvm-svn: 306618
2017-06-29 00:17:26 +00:00
Reid Kleckner a1001b8f38 [COFF] Allow debug info to relocate against discarded symbols
Summary:
In order to do this without switching on the symbol kind multiple times,
I created Defined::getChunkAndOffset and use that instead of
SymbolBody::getRVA in the inner relocation loop.

Now we get the symbol's chunk before switching over relocation types, so
we can test if it has been discarded outside the inner relocation type
switch. This also simplifies application of section relative
relocations. Previously we would switch on symbol kind to compute the
RVA, then the relocation type, and then the symbol kind again to get the
output section so we could subtract that from the symbol RVA. Now we
*always* have an OutputSection, so applying SECREL and SECTION
relocations isn't as much of a special case.

I'm still not quite happy with the cleanliness of this code. I'm not
sure what offsets and bases we should be using during the relocation
processing loop: VA, RVA, or OutputSectionOffset.

Reviewers: ruiu, pcc

Reviewed By: ruiu

Subscribers: majnemer, inglorion, llvm-commits, aprantl

Differential Revision: https://reviews.llvm.org/D34650

llvm-svn: 306566
2017-06-28 17:06:35 +00:00
Eric Beckmann 01822fa1c1 Add more stringent tests for the resource section of executables.
Summary: The testing on the resource section of executables produced by lld has been very lax, and allowed a major bug to go unnoticed when we switched from shelling out to cvtres.exe to using llvm's own library.  These additional tests should cover all the major failure points.

Reviewers: zturner, ruiu

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D34664

llvm-svn: 306465
2017-06-27 21:28:55 +00:00
Eric Beckmann 2a81089116 Replace trivial use of external rc.exe by writing our own .res file.
This patch removes the dependency on the external rc.exe tool by writing
a simple .res file using our own library. In this patch I also added an
explicit definition for the .res file magic.  Furthermore, I added a
unittest for embeded manifests and fixed a bug exposed by the test.

llvm-svn: 306311
2017-06-26 17:43:30 +00:00
Reid Kleckner eb8c0f9d51 [COFF] Fix SECREL and SECTION relocations against common symbols
Summary:
They do the obvious thing: provide the section index of .bss and the
offset of the symbol in .bss.

Reviewers: ruiu

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D34628

llvm-svn: 306304
2017-06-26 16:45:36 +00:00
Reid Kleckner 892d2a5767 Add .yaml as an lld test suffix
Over time we've started to add inputs and test cases using the .yaml
extension, which seems to be preferred over the .objtxt extension that
we were using initially. One nice thing about using .yaml is that it
triggers existing editor highlighting and formatting support.

Fix two pdb*.yaml test cases that I added that weren't being run as part
of check-lld.

llvm-svn: 306303
2017-06-26 16:42:44 +00:00
Reid Kleckner 502d4ce2e4 [COFF] Improve synthetic symbol handling
Summary:
The main change is that we can have SECREL and SECTION relocations
against ___safe_se_handler_table, which is important for handling the
debug info in the MSVCRT.

Previously we were using DefinedRelative for __safe_se_handler_table and
__ImageBase, and after we implement CFGuard, we plan to extend it to
handle __guard_fids_table, __guard_longjmp_table, and more.  However,
DefinedRelative is really only suitable for implementing __ImageBase,
because it lacks a Chunk, which you need in order to figure out the
output section index and output section offset when resolving SECREl and
SECTION relocations.

This change renames DefinedRelative to DefinedSynthetic and gives it a
Chunk. One wart is that __ImageBase doesn't have a chunk. It points to
the PE header, effectively. We could split DefinedRelative and
DefinedSynthetic if we think that's cleaner and creates fewer special
cases.

I also added safeseh.s, which checks that we don't emit a safe seh table
entries pointing to garbage collected handlers and that we don't emit a
table at all when there are no handlers.

Reviewers: ruiu

Reviewed By: ruiu

Subscribers: inglorion, pcc, llvm-commits, aprantl

Differential Revision: https://reviews.llvm.org/D34577

llvm-svn: 306293
2017-06-26 15:39:52 +00:00
Reid Kleckner 8456411e3b [COFF] Fix SECTION and SECREL relocation handling for absolute symbols
Summary:
For SECTION relocations against absolute symbols, MSVC emits the largest
output section index plus one. I've implemented that by threading a
global variable through DefinedAbsolute that is filled in by the Writer.
A more library-oriented approach would be to thread the Writer through
Chunk::writeTo and SectionChunk::applyRel*, but Rui seems to prefer
doing it this way.

MSVC rejects SECREL relocations against absolute symbols, but only when
the relocation is in a real output section. When the relocation is in a
CodeView debug info section destined for the PDB, it seems that this
relocation error is suppressed, and absolute symbols become zeros in the
object file. This is easily implemented by checking the input section
from which we're applying relocations.

This should fix errors about __safe_se_handler_table and
__guard_fids_table when linking the CRT and generating a PDB.

Reviewers: ruiu

Subscribers: aprantl, llvm-commits

Differential Revision: https://reviews.llvm.org/D34541

llvm-svn: 306071
2017-06-22 23:33:04 +00:00
Zachary Turner 7df69958f8 [llvm-pdbutil] Rename "raw" to "dump".
Now you run llvm-pdbutil dump <options>.  This is a followup
after having renamed the tool, whereas before raw was obviously
just the style of dumping, whereas now "dump" is the action to
perform with the "util".

llvm-svn: 306055
2017-06-22 20:57:39 +00:00
Reid Kleckner d0e6e24a53 [PDB] Add symbols to the PDB
Summary:
The main complexity in adding symbol records is that we need to
"relocate" all the type indices. Type indices do not have anything like
relocations, an opaque data structure describing where to find existing
type indices for fixups. The linker just has to "know" where the type
references are in the symbol records. I added an overload of
`discoverTypeIndices` that works on symbol records, and it seems to be
able to link the standard library.

Reviewers: zturner, ruiu

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D34432

llvm-svn: 305933
2017-06-21 17:25:56 +00:00
Rui Ueyama 28ea8c7ad7 [COFF] Set MajorLinkerVersion to 14 instead of 0.
This works around a strange interaction with Authenticode signatures,
in which a signed PE executable with {Major,Minor}LinkerVersion = 0.0
fails to validate on Windows 7 (but is OK on Windows 10). Setting the
linker version to 14.0 (which is what VS2015 outputs) makes it work
again.

Patch by Simon Tatham <simon.tatham@arm.com>.

llvm-svn: 305929
2017-06-21 16:42:08 +00:00
Rui Ueyama 2f740f74bb Define __guard_{iat,longjmp}_{count,table} symbols.
VC2017 contains these new symbols as undefined symobls. They are used
for /guard:cf. Since we do not support the control flow guard, but we
want to at least ignore these symbols so that we can link against VS2017
libraries.

Fixes https://bugs.chromium.org/p/chromium/issues/detail?id=727193.

llvm-svn: 305876
2017-06-21 02:26:19 +00:00
Reid Kleckner 91ef9de643 [codeview] YAMLize all section offsets and indices in symbol records
We forgot to serialize these because llvm-readobj didn't dump them. They
are typically all zeros in an object file. The linker fills them in with
relocations before adding them to the PDB. Now we can properly round
trip these symbols through pdb2yaml -> yaml2pdb.

I made these fields optional with a zero default so that we can elide
them from our test cases.

llvm-svn: 305857
2017-06-20 21:19:22 +00:00
Reid Kleckner f5bb738f75 [PDB] Don't emit debug info associated with dead chunks
Summary:
Previously we didn't add debug info chunks to the SparseChunks array, so
they didn't participate in section GC. Now we do.

Reviewers: ruiu

Subscribers: aprantl, llvm-commits

Differential Revision: https://reviews.llvm.org/D34356

llvm-svn: 305811
2017-06-20 17:14:09 +00:00
Reid Kleckner adea0ce7ca Fix build and PDB test on Linux
llvm-svn: 305716
2017-06-19 17:27:31 +00:00
Reid Kleckner 44cdb10964 [PDB] Start emitting source file and line information
Summary:
This is a first step towards getting line info to show up in VS and
windbg. So far, only llvm-pdbutil can parse the PDBs that we produce.
cvdump doesn't like something about our file checksum tables. I'll have
to dig into that next.

This patch adds a new DebugSubsectionRecordBuilder which takes bytes
directly from some other producer, such as a linker, and sticks it into
the PDB. Line tables only need to be relocated. No data needs to be
rewritten.

File checksums and string tables, on the other hand, need to be re-done.

Reviewers: zturner, ruiu

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D34257

llvm-svn: 305713
2017-06-19 17:21:45 +00:00
Eric Beckmann d135e8c039 Switch external cvtres.exe for llvm's own resource library.
In this patch, I flip the switch in DriverUtils from using the external
cvtres.exe tool to using the Windows Resource library in llvm.

I also fixed a bug where .rsrc sections were marked as discardable
memory and therefore were placed in the wrong order in the final PE.

Furthermore, I modified WindowsResource to write the coff directly to a
memory buffer instead of to file, also had it use the machine types
already declared in COFF.h instead creating my own enum.

Finally, I flipped the switch to allow all unit tests that had
previously run only on windows due to a winres dependency to run
cross-platform.

Reviewers: zturner, ruiu

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D34265

llvm-svn: 305592
2017-06-16 21:13:24 +00:00
Reid Kleckner f8b992dd08 Give associative-comdat lld test a triple
llvm-svn: 305585
2017-06-16 20:51:42 +00:00
Reid Kleckner 79ac99b3e8 [COFF] Drop unused comdat sections when GC is turned off
Summary:
Adds a "Discarded" bool to SectionChunk to indicate if the section was
discarded by COMDAT deduplication. The Writer still just checks
`isLive()`.

Fixes PR33446

Reviewers: ruiu

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D34288

llvm-svn: 305582
2017-06-16 20:47:19 +00:00
Zachary Turner 56ed9deafc Remove some non-determinstic numbers from pdb test.
llvm-svn: 305520
2017-06-15 22:41:45 +00:00
Zachary Turner 93f986a7dc Fix pdb embedded paths.
Obviously the paths are going to be machine dependent, so
don't try to match them in the test.

llvm-svn: 305518
2017-06-15 22:30:00 +00:00
Zachary Turner 6305545527 Resubmit "[llvm-pdbutil] rewrite the "raw" output style."
This resubmits commit c0c249e9f2ef83e1d1e5f166b50673d92f3579d7.

It was broken due to some weird template issues, which have
since been fixed.

llvm-svn: 305517
2017-06-15 22:24:24 +00:00
Zachary Turner da504b794c Revert "[llvm-pdbutil] rewrite the "raw" output style."
This reverts commit 83ea17ebf2106859a51fbc2a86031b44d33696ad.

This is failing due to some strange template problems, so reverting
until it can be straightened out.

llvm-svn: 305505
2017-06-15 20:55:51 +00:00
Saleem Abdulrasool bc7ff7027e COFF: add support for lib mode usage
When link is invoked with `/def:` and no input files, it behaves as if
`lib.exe` was invoked.  Emulate this behaviour, generating the import
library from the def file that was passed.  Because there is no input to
actually generate the dll, we simply process the def file early and exit
once we have created the import library.

llvm-svn: 305502
2017-06-15 20:39:58 +00:00
Zachary Turner b560fdf3b8 [llvm-pdbutil] rewrite the "raw" output style.
After some internal discussions, we agreed that the raw output style had
outlived its usefulness. It was originally created before we had even
thought of dumping to YAML, and it was intended to give us some insight
into the internals of a PDB file. Now we have YAML mode which does
almost exactly this but is more powerful in that it can round-trip back
to a PDB, which the raw mode could not do. So the raw mode had become
purely a maintenance burden.

One option was to just delete it. However, its original goal was to be
as readable as possible while staying close to the "metal" - i.e.
presenting the output in a way that maps directly to the underlying file
format. We don't actually need that last requirement anymore since it's
covered by the yaml mode, so we could repurpose "raw" mode to actually
just be as readable as possible.

This patch implements about 80% of the functionality previously in raw
mode, but in a completely different style that is more akin to what
cvdump outputs. Records are very compressed, often times appearing on
just one line. One nice thing about this is that it makes full record
matching easier, because you can grep for indices, names, and leaf types
on a single line often.

See the tests for some examples of what the new output looks like.

Note that this patch actually regresses the functionality of raw mode in
a few areas, but only because the patch was already unreasonably large
and going 100% would have been even worse. Specifically, this patch is
missing:

The ability to dump module debug subsections (checksums, lines, etc)
The ability to dump section headers
Aside from that everything is here. While goign through the tests fixing
them all up, I found many duplicate tests. They've been deleted. In
subsequent patches I will go through and re-add the missing
functionality.

Differential Revision: https://reviews.llvm.org/D34191

llvm-svn: 305495
2017-06-15 19:34:41 +00:00
Zachary Turner a8cfc29c9a Resubmit "[codeview] Make obj2yaml/yaml2obj support .debug$S..."
This was originally reverted because of some non-deterministic
failures on certain buildbots.  Luckily ASAN eventually caught
this as a stack-use-after-scope, so the fix is included in
this patch.

llvm-svn: 305393
2017-06-14 15:59:27 +00:00
Zachary Turner 0085dce221 Revert "[codeview] Make obj2yaml/yaml2obj support .debug$S..."
This is causing failures on linux bots with an invalid stream
read.  It doesn't repro in any configuration on Windows, so
reverting until I have a chance to investigate on Linux.

llvm-svn: 305371
2017-06-14 06:24:24 +00:00
Zachary Turner a3da4467fa [codeview] Make obj2yaml/yaml2obj support .debug$S/T sections.
This allows us to use yaml2obj and obj2yaml to round-trip CodeView
symbol and type information without having to manually specify the bytes
of the section. This makes for much easier to maintain tests. See the
tests under lld/COFF in this patch for example. Before they just said
SectionData: <blob> whereas now we can use meaningful record
descriptions. Note that it still supports the SectionData yaml field,
which could be useful for initializing a section to invalid bytes for
testing, for example.

Differential Revision: https://reviews.llvm.org/D34127

llvm-svn: 305366
2017-06-14 05:31:00 +00:00
Reid Kleckner 8cbdd0c0f2 [PDB] Add a module descriptor for every object file
Summary:
Expose the module descriptor index and fill it in for section
contributions.

Reviewers: zturner

Subscribers: llvm-commits, ruiu, hiraditya

Differential Revision: https://reviews.llvm.org/D34126

llvm-svn: 305296
2017-06-13 15:49:13 +00:00
Zachary Turner fa18b2f68c Fix broken LLD test.
llvm-svn: 305260
2017-06-13 00:30:09 +00:00
Zachary Turner 68ea80d0a7 Slightly better fix for dealing with no-id-stream PDBs.
The last fix required the user to manually add the required
feature.  This caused an LLD test to fail because I failed to
update LLD.  In practice we can hide this logic so it can just
be transparently added when we write the PDB.

llvm-svn: 305236
2017-06-12 21:46:51 +00:00
Peter Collingbourne 89061b2224 IR: Replace the "Linker Options" module flag with "llvm.linker.options" named metadata.
The new metadata is easier to manipulate than module flags.

Differential Revision: https://reviews.llvm.org/D31349

llvm-svn: 305227
2017-06-12 20:10:48 +00:00
Zachary Turner bd336e44d8 Rename llvm-pdbdump -> llvm-pdbutil.
This is to reflect the evolving nature of the tool as being
useful for more than just dumping PDBs, as it can do many other
things.

Differential Revision: https://reviews.llvm.org/D34062

llvm-svn: 305106
2017-06-09 20:46:17 +00:00
Reid Kleckner c2eaccb0af Revert "Merge IAT and ILT."
This reverts r303374. It breaks Chrome's IAT patching code:
http://crbug.com/729077

llvm-svn: 304584
2017-06-02 18:49:38 +00:00
Reid Kleckner 5a1ed789e1 Add test case for LLD/COFF for stdcall exports in .def files
The .def file parser changes I reverted broke this test case, and
exported "__imp__foo" instead of "__imp__foo@8". This was
http://crbug.com/728726.

llvm-svn: 304572
2017-06-02 17:48:06 +00:00
Rui Ueyama 69ae29b1d1 Do not allow delay-importing data symbols.
If you pass /delayload:<dllname> to the COFF linker, it creates thunks
so that DLLs are loaded when they are used for the first time instead of
load-time.

This mechanism do not work for data symbols as there's no way to trap
acccesses to data imported from DLLs. (Technically, I think if we do not
initially map dllimport tables in memory, we could actually trap accesses
and delay-load data symbols, but that's not what Windows do.)

This patch is to report an error when you try to delay-load data symbols.

Fixes https://bugs.llvm.org/show_bug.cgi?id=33106

Differential Revision: https://reviews.llvm.org/D33557

llvm-svn: 303890
2017-05-25 18:03:34 +00:00
Rui Ueyama 9aa82f76ac Garbage collect dllimported symbols.
This is a different implementation than r303225 (which was reverted
in r303270, re-submitted in r303304 and then re-reverted in r303527).

In the previous patch, I tried to add Live bit to each dllimported
symbol. It turned out that it didn't work with "oldnames.lib" which
contains a lot of weak aliases to dllimported symbols.

The way we handle weak aliases is to check if undefined symbols
can be resolved using weak aliases, and if so, memcpy the Defined
symbols to weak Undefined symbols, so that any references to weak
aliases automatically see defined symbols instead of undefined ones.

This memcpy happens before MarkLive kicks in.

That means we may have multiple copies of dllimported symbols. So
turning on one instance's Live bit is not enough.

This patch moves the Live bit to dllimport file. Since multiple
copies of dllsymbols still point to the same file, we can use it as the
central repository to keep track of liveness.

Differential Revision: https://reviews.llvm.org/D33520

llvm-svn: 303814
2017-05-24 22:30:06 +00:00
Rui Ueyama b6632d9cd1 Revert r303304: Re-submit r303225: Garbage collect dllimported symbols.
This reverts commit r303304 because it looks like the change
introduced a crash bug. At least after that change, LLD with thinlto
crashes when linking Chromium.

llvm-svn: 303527
2017-05-22 06:01:37 +00:00
Rui Ueyama a674943211 Set IMAGE_DLL_CHARACTERISTICS_NO_BIND.
Our output is not compatible with the Binding feature, so make it
explicit that.

Differential Revision: https://reviews.llvm.org/D33336

llvm-svn: 303378
2017-05-18 20:26:58 +00:00
Rui Ueyama 9fbfd76fe3 Merge IAT and ILT.
Previously, LLD-produced executables had IAT (Import Address Table) and
ILT (Import Lookup Table) as separate chunks of data, although their
contents are identical. My interpretation of the COFF spec when I wrote
the COFF linker is that they need to be separate tables even though they
are the same.

But Peter found that the Windows loader is fine with executables in
which IAT and ILT are merged. This is a patch to merge IAT and ILT.
I confirmed that an lld-link self-hosted with this patch works fine.

Fixes https://bugs.llvm.org/show_bug.cgi?id=33064

Differential Revision: https://reviews.llvm.org/D33326

llvm-svn: 303374
2017-05-18 19:59:22 +00:00
Zachary Turner 8a7508970a [COFF] Fix interaction between /DEBUG and /PDB
When /DEBUG is not specified, /PDB should be ignored.  When
/DEBUG is specified, a PDB should be output regardless of
whether or not /PDB is specified.  /PDB just overrides the
default name.

This patch implements this behavior, and adds some tests, while
also removing a dead option /DEBUGPDB which was unused in any
code.

Differential Revision: https://reviews.llvm.org/D33302

llvm-svn: 303352
2017-05-18 15:15:10 +00:00
Rui Ueyama cd41bc8dec Re-submit r303225: Garbage collect dllimported symbols.
This reverts re-submits r303225 which was reverted in r303270 because it
broke the sanitizer-windows bot.

The reason of the failure is that we were writing dead symbols to the
symbol table. I fixed the issue.

llvm-svn: 303304
2017-05-17 21:36:08 +00:00
Hans Wennborg e67c5f6b52 Revert r303225 "Garbage collect dllimported symbols."
and follow-up r303226 "Fix Windows buildbots."

This broke the sanitizer-windows buildbot.

> Previously, the garbage collector (enabled by default or by explicitly
> passing /opt:ref) did not kill dllimported symbols. As a result,
> dllimported symbols could be added to resulting executables' dllimport
> list even if no one was actually using them.
>
> This patch implements dllexported symbol garbage collection. Just like
> COMDAT sections, dllimported symbols now have Live bits to manage their
> liveness, and MarkLive marks reachable dllimported symbols.
>
> Fixes https://bugs.llvm.org/show_bug.cgi?id=32950
>
> Reviewers: pcc
>
> Subscribers: llvm-commits
>
> Differential Revision: https://reviews.llvm.org/D33264

llvm-svn: 303270
2017-05-17 16:22:03 +00:00
Rui Ueyama de83fec029 Fix Windows buildbots.
llvm-svn: 303226
2017-05-17 01:05:56 +00:00
Rui Ueyama 02df7a6cf1 Garbage collect dllimported symbols.
Summary:
Previously, the garbage collector (enabled by default or by explicitly
passing /opt:ref) did not kill dllimported symbols. As a result,
dllimported symbols could be added to resulting executables' dllimport
list even if no one was actually using them.

This patch implements dllexported symbol garbage collection. Just like
COMDAT sections, dllimported symbols now have Live bits to manage their
liveness, and MarkLive marks reachable dllimported symbols.

Fixes https://bugs.llvm.org/show_bug.cgi?id=32950

Reviewers: pcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D33264

llvm-svn: 303225
2017-05-17 00:35:50 +00:00
Rui Ueyama 35e828775f Merge a test YAML file and a test file.
llvm-svn: 303135
2017-05-16 01:16:20 +00:00
Rui Ueyama 3969acf8a4 Port recent changes to ELF/MapFile.cpp to COFF.
llvm-svn: 301754
2017-04-29 18:50:39 +00:00
Saleem Abdulrasool 5c98b74536 COFF: actually synthesize CONST imports properly
CONSTANT imports expect both the `_imp_` prefixed and non-prefixed
symbols should be added to the symbol table.  This allows for linking
symbols like _NSConcreteGlobalBlock in WinObjC.  The previous change
would generate the import library properly by handling the option but
would not consume the generated entry properly.

llvm-svn: 301657
2017-04-28 17:06:40 +00:00
Rui Ueyama a85572ebf0 COFF ICF: Merge only functions. Do not merge read-only data.
This seems to be the behavior of the MSVC linker. Previously, this
incompatibility caused nasty issues in chromium build a few times.

Differential Revision: https://reviews.llvm.org/D30363

llvm-svn: 301598
2017-04-27 23:03:22 +00:00
Rui Ueyama 95fe854332 Handle _LINK_ env string as command line parameters.
"_LINK_" environment varaible should be appended to the command line.
https://msdn.microsoft.com/en-us/library/6y6t9esh.aspx

Fixes https://bugs.llvm.org/show_bug.cgi?id=32756

llvm-svn: 301264
2017-04-24 22:20:03 +00:00
Bob Haarman 4110816253 [coff] for /msvclto, pass archive members with prevailing symbols first
Summary: When using /msvclto, lld and MSVC's linker both do their own symbol resolution. This can cause them to select different archive members, which can result in undefined references. This change avoids that situation by extracting archive members that are selected by lld and passing those to link.exe before any archives, so that MSVC's uses those objects for symbol resolution instead of different archive members.

Reviewers: pcc, rnk, ruiu

Reviewed By: pcc

Subscribers: llvm-commits, mehdi_amini

Differential Revision: https://reviews.llvm.org/D32317

llvm-svn: 301045
2017-04-21 21:38:01 +00:00
Saleem Abdulrasool 72fbd346c6 COFF: add support for CONSTANT exports
The CONSTANT export type is marked as obsolete, but link still supports
this.  Furthermore, WinObjC uses this for certain exports.  Add support
for this export type.

llvm-svn: 301013
2017-04-21 18:05:46 +00:00
Bob Haarman 63220d5203 [coff] fix test for msvclto
llvm-svn: 300616
2017-04-18 22:29:36 +00:00
Rui Ueyama 5c5bc7bc0a Accept not only lld-link but also LLD-LINK, for example.
Filenames are case-insensitive on Windows, so when we dispatch based
on argv0, we need to handle it case-insensitively.

Fixes https://bugs.llvm.org/show_bug.cgi?id=32637.

llvm-svn: 300087
2017-04-12 18:29:52 +00:00
Reid Kleckner 6e545ffc4e [PDB] Emit index/offset pairs for TPI and IPI streams
Summary:
This lets PDB readers lookup type record data by type index in O(log n)
time. It also enables makes `cvdump -t` work on PDBs produced by LLD.
cvdump will not dump a PDB that doesn't have an index-to-offset table.

The table is sorted by type index, and has an entry every 8KB. Looking
up a type record by index is a binary search of this table, followed by
a scan of at most 8KB.

Reviewers: ruiu, zturner, inglorion

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D31636

llvm-svn: 299958
2017-04-11 16:26:15 +00:00
Reid Kleckner c9bc745518 Remove .rdata section size checks from pdb.test, they are not the same on Windows and Unix
llvm-svn: 299795
2017-04-07 19:58:33 +00:00
Reid Kleckner f519ef0990 [PDB] Re-add RAW FileCheck testing to pdb.test
Update the expected output to match the current llvm-pdbdump raw output.

llvm-svn: 299793
2017-04-07 18:38:38 +00:00
Saleem Abdulrasool 671029daec COFF: support the /appcontainer flag
The /appcontainer flag indicates that the module may only be used inside
an application container (for isolation).  This has been supported by
link.exe since Windows 8.0.  It sets an additional bit in the PE DLL
Characteristics flag to indicate the behavioural change.

llvm-svn: 299728
2017-04-06 23:07:53 +00:00
Bob Haarman ac8f7fc07b [COFF] support /ERRORLIMIT option
Summary:
This adds support for reporting multiple errors in a single invocation of lld-link. The limit defaults to 20 and can be changed with the /ERRORLIMIT command line parameter, or set to unlimited by passing a value of 0.

This is a new attempt after r295507, which was reverted because opening files raced with exiting early, causing the test to be flaky. This version avoids the race by exiting before calling enqueuePath.

Reviewers: pcc, ruiu

Reviewed By: ruiu

Subscribers: llvm-commits, dblaikie

Differential Revision: https://reviews.llvm.org/D31688

llvm-svn: 299496
2017-04-05 00:43:54 +00:00
Bob Haarman bf24cb7189 COFF: tests for thinlto with mangled names
Summary: This change adds tests for r299268.

Reviewers: pcc, rnk

Reviewed By: pcc

Subscribers: mehdi_amini, llvm-commits, Prazek

Differential Revision: https://reviews.llvm.org/D31549

llvm-svn: 299272
2017-03-31 22:17:02 +00:00
Rui Ueyama 9a3e73383b Add parsing of mllvm command line options.
Patch by Alberto Magni
Original patch by Kyungwoo Lee <kyulee@microsoft.com>

Differential Revision: https://reviews.llvm.org/D31388

llvm-svn: 299106
2017-03-30 20:10:40 +00:00
Bob Haarman 2d2b5b09e0 COFF: include archive name in LTO object name
Summary: In the ELF linker, we create the buffer identifier for bitcode files by appending the object name to the archive name. This change makes the COFF linker do the same. Without the change, ThinLTO builds can fail with an error message about multiple ThinLTO modules per object file, caused by object files contained in different archives having the same name.

Reviewers: pcc, ruiu

Reviewed By: pcc

Subscribers: mehdi_amini

Differential Revision: https://reviews.llvm.org/D31402

llvm-svn: 298942
2017-03-28 21:20:06 +00:00
Reid Kleckner 5d57752c81 [PDB] Split item and type records when merging type streams
Summary: MSVC does this when producing a PDB.

Reviewers: ruiu

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D31316

llvm-svn: 298717
2017-03-24 17:26:38 +00:00
Reid Kleckner 77d3aa476e Add a /nopdb flag to disable PDB generation
This will be used in the sanitizer test suite, which wants to use DWARF
line tables.

At some point we should reconsider how LLD handles the long section
names required by DWARF debug sections.

llvm-svn: 298544
2017-03-22 19:49:12 +00:00
Reid Kleckner 3a0dea5243 Update LLD tests for codeview dumping changes
llvm-svn: 298475
2017-03-22 01:43:49 +00:00
Reid Kleckner 13bdbfb622 [COFF] Put the PDB next to the image if the user doesn't pass /PDB:
Summary: This is compatible with MSVC link.exe.

Reviewers: ruiu

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D31224

llvm-svn: 298467
2017-03-22 00:57:14 +00:00
Reid Kleckner 7668182e44 [COFF] Don't let /def override /out filename
Summary:
This also delays setting the output filename based on the first input
argument until after processing /def.

Fixes PR32354

Reviewers: ruiu, pcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D31152

llvm-svn: 298327
2017-03-21 00:12:51 +00:00
Peter Collingbourne db7447deed COFF: Do not create empty temporary archives for /msvclto.
The MSVC linker doesn't support them.

Differential Revision: https://reviews.llvm.org/D31067

llvm-svn: 298034
2017-03-17 02:04:22 +00:00
Rui Ueyama e1bf1365d7 Do not pass archive files containing non-native object files.
The MSVC linker doesn't like archive files containing non-native object
files.

When we are doing an LTO build, we may create archive files containing
both LLVM bitcode files and native object files. For example, if a
project contains assembly files and C++ files, we create native object
files for the assembly files and LLVM bitcode files for the C++ files.

With the /msvclto option, LLD passes archive files to the MSVC linker.
Previously, we didn't pass archive files if they contain at least one
bitcode files. That wasn't correct because the native object files that
weren't passed to the MSVC linker may be needed to complete linking.

In this patch, we create new temporary archive files to strip bitcode
files.

Differential Revision: https://reviews.llvm.org/D31053

llvm-svn: 297997
2017-03-16 21:19:36 +00:00
Zachary Turner 337e884643 Fix lld pdb test.
It was assuming various things about the PDB like file size
which are going to be too high maintenance to maintain in
a test.

llvm-svn: 297908
2017-03-15 22:58:10 +00:00
Rui Ueyama 7d271ae21d Fix wrong assertion failure.
Previously, if you have foo=bar in a definition file, this assertion
could fire because when symbols are read from file they could be mangled.
It seems that due to historical reasons underscore mangling scheme is
really ad-hoc, and I cannot find a clean way to handle this. I had
to just de-mangle symbols to search again.

llvm-svn: 297357
2017-03-09 04:47:33 +00:00
Rui Ueyama 9accea6feb Pass archive files to link.exe if they contain at least one native object file.
Some archive files created during chromium build contains both BC
and native files. If that's the case, we want to pass the archive
file to link.exe. Otherwise, the MSVC linker would complain that
there's an unresolved symbol in a given set of files.

I cannot explain why link.exe doesn't complain about the presence
of bitcode files in this case, but it seems link.exe doesn't touch BC.

llvm-svn: 297229
2017-03-07 21:26:10 +00:00
Rui Ueyama e0341db179 Do not pass archive files containing bitcode files to the MSVC Linker.
If /msvclto is specified, we compile bitcode files and pass it to the
MSVC linker, stripping all bitcode files. We haven't stripped archive
files, because I was thinking that the MSVC linker wouldn't touch files
in archive files. When we pass an object file to link.exe, all symbols
have been resolved already, so link.exe shoulnd't need any of the files
in archives.

It turns out that even though link.exe doesn't need to do that, it
seems to try to read each file in all archives. And if there's a non-
COFF file in an archive, it exists with an error message. So we need
to remove archives from the command line too.

llvm-svn: 297191
2017-03-07 19:45:53 +00:00
Bob Haarman 851b1f866b [COFF] added test for thinlto
Summary: Creates bitcode files suitable for use with ThinLTO, then checks that the linker can build an executable from them.

Reviewers: ruiu, pcc

Reviewed By: pcc

Subscribers: mehdi_amini, Prazek, llvm-commits

Differential Revision: https://reviews.llvm.org/D30277

llvm-svn: 296042
2017-02-23 23:42:24 +00:00
Rui Ueyama 85d54b050e Fix /msvclto.
Previously, bitcode files in library paths were passed to the MSVC linker.
This patch strips them.

llvm-svn: 295913
2017-02-23 00:26:42 +00:00
Rui Ueyama 99861f4730 Attempt to placate MSVC buildbot.
I really do not understand what is going on on some Windows buildbots,
but FileCheck command on some buildbot behaves like long lines were
truncated. I'll try to find a cause of the issue, but let me relax the
test so that they'll succeed on all buildbots.

llvm-svn: 295798
2017-02-22 00:32:56 +00:00
David Blaikie 4cdfe69982 Revert "[COFF] support /ERRORLIMIT option"
Behavior races on ErrorCount. If the enqueued paths are evaluated
eagerly (in enqueuePath) then the behavior is as the test expects. But
they may not be evaluated until the future is waited on, in run() -
which is after the early return/exit on ErrorCount. (this causes the
test to fail (because in the "/ERRORCOUNT:XYZ" test, no other errors
are printed), at least for me, on linux)

This reverts commit r295507.

llvm-svn: 295590
2017-02-19 02:25:47 +00:00
Bob Haarman b96b10102a [COFF] support /ERRORLIMIT option
Summary: This adds support for reporting multiple errors in a single invocation of lld-link. The limit defaults to 20 and can be changed with the /ERRORLIMIT command line parameter, or set to unlimited by passing a value of 0.

Reviewers: pcc, ruiu

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D29691

llvm-svn: 295507
2017-02-17 22:46:06 +00:00
Bob Haarman c87da86707 added missing "savetemps/" in test/COFF/savetemps.ll
llvm-svn: 294664
2017-02-09 23:31:47 +00:00
Bob Haarman 751f169657 list paths explicitly instead of using * in
test/COFF/savetemps.ll

Reviewers: dyung, ruiu

Reviewed By: dyung, ruiu

Differential Revision: https://reviews.llvm.org/D29788

llvm-svn: 294643
2017-02-09 22:52:21 +00:00
Rui Ueyama 7a6f391451 Attempt to fix lld-x86_64-win7 bot.
The test is failing on the bot because "/subsystem:console" was
truncated for some reason. I don't know why that is happening on
that machine (it is not reproducible on my Windows machine).
In this patch, I'm trying to tame it by making the output shorter.

llvm-svn: 294502
2017-02-08 19:22:34 +00:00
Bob Haarman 69b196dd91 [COFF] added support for /lldsavetemps
Summary: This adds an option to save temporary files generated during link-time optimization. This can be useful for debugging.

Reviewers: ruiu, pcc

Reviewed By: ruiu, pcc

Subscribers: mehdi_amini

Differential Revision: https://reviews.llvm.org/D29518

llvm-svn: 294498
2017-02-08 18:36:41 +00:00
Saleem Abdulrasool b6394287e2 COFF: ensure that we handle bad arguments
The parameter to /debugtype is user provided and may be invalid.  Ensure
that we gracefully handle bad input.

llvm-svn: 294280
2017-02-07 04:28:05 +00:00
Saleem Abdulrasool 0acd6dd6ce COFF: prevent nullptr dereference
If `/debugtypes` is used to omit the codeview information, we would not
have constructed the debug info codeview record which is used to tie the
PDB to the binary.  In such a case, rub out the GUID and Age fields.

llvm-svn: 294279
2017-02-07 04:28:02 +00:00
Rui Ueyama a5ed3226d6 Attempt to fix a flakey test.
I believe that the test is flakey because the order of stdout
and stderr is not deterministic.

llvm-svn: 294238
2017-02-06 21:11:33 +00:00
Rui Ueyama 1e0b158dbd Add an option to use the MSVC linker to link LTO-generated object files.
This patch defines a new command line option, /MSVCLTO, to LLD.
If that option is given, LLD invokes link.exe to link LTO-generated
object files. This is hacky but useful because link.exe can create
PDB files.

Differential Revision: https://reviews.llvm.org/D29526

llvm-svn: 294234
2017-02-06 20:47:55 +00:00
Bob Haarman cde5e5b600 refactor COFF linker to use new LTO API
Summary: The COFF linker previously implemented link-time optimization using an API which has now been marked as legacy. This change refactors the COFF linker to use the new LTO API, which is also used by the ELF linker.

Reviewers: pcc, ruiu

Reviewed By: pcc

Subscribers: mgorny, mehdi_amini

Differential Revision: https://reviews.llvm.org/D29059

llvm-svn: 293967
2017-02-02 23:58:14 +00:00
Rui Ueyama eb1ad400f9 Print alignment in decimal instead of hexadecimal.
Previously, we were printing out something like this for
sections/symbols with alignment 16

  0000000000201000 0000000000000182    10 .data

which I think confusing. I think printing it in decimal is better.

Differential Revision: https://reviews.llvm.org/D29258

llvm-svn: 293685
2017-01-31 20:42:34 +00:00
Bob Haarman 25c731874d COFF: add test for /INCLUDE directives specified in bitcode
Summary: MSVC allows linker options to be specified in source code. One of these is the /INCLUDE directive, which specifies that a symbol must be added to the symbol table, even if it otherwise wouldn't be. Existing tests cover the case where the linker is given an object file with an /INCLUDE directive, but we also need to cover the case where /INCLUDE is specified in a bitcode file (as would happen when using LTO). This new test covers that case.

Reviewers: pcc, ruiu

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D29096

llvm-svn: 293107
2017-01-25 23:07:40 +00:00
Zachary Turner 18de36bc90 Fix failing lld pdb test.
llvm-svn: 292673
2017-01-20 22:57:24 +00:00
Peter Collingbourne 6f24fdb6a0 COFF: Change the /lldmap output format to be more like the ELF linker.
Differential Revision: https://reviews.llvm.org/D28717

llvm-svn: 291990
2017-01-14 03:14:46 +00:00
Rui Ueyama 5289662e4a PDB: Merge type info.
This patch is to merge type info in multiple .debug$T sections.

One mystery that needs to be solved is that it is not clear how
the MSVC linker uses TPI and IPI streams. Both streams contain
type info, and it is not obvious what kind of record should go
which.

dumppdb command in microsoft-pdb repository prints out IPI stream
contents as "IDs" and TPI stream as "TYPES", but looks like the tool
don't really care about which stream type recrods were read from.

For now, in this patch, I emit all type records to TPI stream.
It might just work with other tools. If not, we need to investigate
it more.

llvm-svn: 291739
2017-01-12 03:09:25 +00:00
Rui Ueyama 0697ae2ad9 Revert r291526: Re-enable /linkrepro test on Windows.
This broke the following two bots:

lld-x86_64-win7: the test failed because a diff command is not available
on that bot. That's a configuration error of the bot and will be fixed soon.

llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast: "tar xf" failed on that
bot. I suspect that it is due to the maximum path limitation on Windows.
A build directory contains a buildbot name, so it's longer than usual on
that machine. On Windows, many filesystem operations fail if a path is
longer than 255 characters. I'll try to address that in another patch.

llvm-svn: 291527
2017-01-10 02:08:37 +00:00
Rui Ueyama e98516ff99 Re-enable /linkrepro test on Windows.
I think generated tar files are more compatible with old tar commands
because of r291494 and r291340, so I want to enable this test on buildbots.

llvm-svn: 291526
2017-01-10 01:37:37 +00:00
Simon Pilgrim bb643ab5bc Attempt to fix windows buildbot
llvm-svn: 291348
2017-01-07 15:44:27 +00:00
Rui Ueyama 7f1f912794 Use TarWriter to create tar archives instead of cpio.
This is how we use TarWriter in LLD. Now LLD does not append
a file extension, so you need to pass `--reproduce foo.tar`
instead of `--reproduce foo`.

Differential Revision: https://reviews.llvm.org/D28103

llvm-svn: 291210
2017-01-06 02:33:53 +00:00
Saleem Abdulrasool 8d29e62de7 test: loosen a test for the time being
The GUID should match between the RSDS and the PDB.  This should repair
the build bots, though we should be ensuring that the GUIDs match.
Unfortunately, different build bots seem to be getting different GUIDs.

llvm-svn: 290981
2017-01-04 18:17:28 +00:00
Saleem Abdulrasool df8a13b257 COFF: tie the execute and the PDB together
The PDB GUID, Age, and version are tied together by the RSDS record in
the binary.  Pass along the BuildId information into the createPDB to
allow us to tie the binary and the PDB together.

llvm-svn: 290975
2017-01-04 17:56:54 +00:00
Peter Collingbourne 6ee0b4e9f5 COFF: Open and map input files asynchronously on Windows.
Profiling revealed that the majority of lld's execution time on Windows was
spent opening and mapping input files. We can reduce this cost significantly
by performing these operations asynchronously.

This change introduces a queue for all operations on input file data. When
we discover that we need to load a file (for example, when we find a lazy
archive for an undefined symbol, or when we read a linker directive to
load a file from disk), the file operation is launched using a future and
the symbol resolution operation is enqueued.  This implies another change
to symbol resolution semantics, but it seems to be harmless ("ninja All"
in Chromium still succeeds).

To measure the perf impact of this change I linked Chromium's chrome_child.dll
with both thin and fat archives.

Thin archives:

Before (median of 5 runs): 19.50s
After: 10.93s

Fat archives:

Before: 12.00s
After: 9.90s

On Linux I found that doing this asynchronously had a negative effect on
performance, probably because the cost of mapping a file is small enough that
it becomes outweighed by the cost of managing the futures. So on non-Windows
platforms I use the deferred execution strategy.

Differential Revision: https://reviews.llvm.org/D27768

llvm-svn: 289760
2016-12-15 04:02:23 +00:00
Peter Collingbourne c4e9482164 COFF: We no longer require lib.exe to test DLL exports.
llvm-svn: 289745
2016-12-15 00:11:17 +00:00
Peter Collingbourne 8b65e51b73 COFF: Load inputs immediately instead of adding them to a queue.
This patch replaces the symbol table's object and archive queues, as well as
the convergent loop in the linker driver, with a design more similar to the
ELF linker where symbol resolution directly causes input files to be added to
the link, including input files arising from linker directives. Effectively
this removes the last vestiges of the old parallel input file loader.

Differential Revision: https://reviews.llvm.org/D27660

llvm-svn: 289409
2016-12-11 22:15:25 +00:00
Rui Ueyama 327705d17d Create a TPI stream only when /debugpdb is given.
Adding type records to TPI stream is too time consuming.
It is reported that linking chrome_child.dll took 5 minutes.

llvm-svn: 289330
2016-12-10 17:23:23 +00:00
Peter Collingbourne 79a5e6b1b7 COFF: New symbol table design.
This ports the ELF linker's symbol table design, introduced in r268178,
to the COFF linker.

Differential Revision: http://reviews.llvm.org/D21166

llvm-svn: 289280
2016-12-09 21:55:24 +00:00
Hans Wennborg 1818e65aad COFF: Support both /lldmap and /lldmap:<file>
The former option bases the filename on the output name, e.g. if the
link output is a.exe, the map will be written to a.map. This matches the
behaviour of link.exe's /MAP option and is useful for creating a map
file of each executable when building a large project.

Differential Revision: https://reviews.llvm.org/D27595

llvm-svn: 289271
2016-12-09 20:54:44 +00:00
Rui Ueyama 26186c7e0f Add type records to TPI stream.
I don't think the data I add to a TPI stream in this patch is correct,
but at least it can be displayed using llvm-pdbdump. Until I add more
streams to a PDB file, I'm not able to know whether the data will be
accepted by MSVC tools or not.

llvm-svn: 289183
2016-12-09 04:46:54 +00:00
Rui Ueyama a45d45e231 COFF: Define overloaded toString functions.
Previously, we had different way to stringize SymbolBody and InputFile
to construct error messages. This patch defines overloaded function
toString() so that we don't need to memorize all these different
function names.

With that change, it is now easy to include demangled names in error
messages. Now, if there is a symbol name conflict, we'll print out
both mangled and demangled names.

llvm-svn: 288992
2016-12-07 23:17:02 +00:00
Rui Ueyama 27498b5dd5 Fix a bug in ICF involving COFF associative sections.
Associative sections are sections that need to be linked if their associated
sections are linked. Associative sections are used to append auxiliary data
such as debug info.

Previously, we compared all associative sections when comparing two comdat
sections. Because usually assocative sections are not mergeable sections,
we missed a lot of mergeable sections. MSVC linker doesn't seem to check
the identity of associative sections.

This patch makes LLD to ignore associative sections when doing ICF.

llvm-svn: 288483
2016-12-02 07:46:12 +00:00
Rui Ueyama c5cb737584 Dump Codeview type information correctly.
llvm-svn: 288298
2016-12-01 01:22:48 +00:00
Rui Ueyama 24625fd9b2 Dump not only type records but symbol records.
llvm-svn: 287723
2016-11-22 23:51:34 +00:00
Rui Ueyama be939b3ff6 Do plumbing work for CodeView debug info.
Previously, we discarded .debug$ sections. This patch adds them to
files so that PDB.cpp can access them.

This patch also adds a debug option, /dumppdb, to dump debug info
fed to createPDB so that we can verify that valid data has been passed.

llvm-svn: 287555
2016-11-21 17:22:35 +00:00
Rui Ueyama c91f716643 PDB: Add "* Linker *" module.
The added module contains nothing, but it is still useful as a test
to ensure that we are emitting modules that can be read back.

llvm-svn: 287071
2016-11-16 01:10:46 +00:00
Rui Ueyama 569851f1d6 Partially revert r287009: Remove trailing whitespace.
This reverts part of r287009 because I accidentally changed binary files.

llvm-svn: 287010
2016-11-15 19:09:13 +00:00
Rui Ueyama d7cfb1b6dd Remove trailing whitespace.
llvm-svn: 287009
2016-11-15 19:07:17 +00:00
Rui Ueyama f83806a8ad Identify object files compiled with cl.exe /GL.
Object files compiled with cl.exe /GL contain intermediate code for LTO.
We can't (and don't want to) interpret such code, but we should print
out a user-friendly error message.

Differential Revision: https://reviews.llvm.org/D26647

llvm-svn: 286921
2016-11-15 01:01:51 +00:00
Rui Ueyama ec2595d2e1 Attempt to fix freaky bot.
I don't really understand what is failing on lld-x86_64-darwin13 bot,
but this patch should at least reduces the number of moving parts.

llvm-svn: 286876
2016-11-14 19:50:10 +00:00
Rui Ueyama 09e0b5f2c9 Emit Section Contributions.
Differential Revision: https://reviews.llvm.org/D26211

llvm-svn: 286684
2016-11-12 00:00:51 +00:00
Davide Italiano c520e18ce0 [lld/COFF] Don't round alignment if it's already a power-of-two.
This matches link.exe behaviour.

Differential Revision:  https://reviews.llvm.org/D26372

llvm-svn: 286553
2016-11-11 02:53:48 +00:00
Rui Ueyama 20df4ecf4f Re-commit r285607: Emit Section Map stream."
Removed some test fields that overspecified test conditions.

llvm-svn: 285638
2016-10-31 21:09:21 +00:00
Rui Ueyama e7b54c2ea5 Revert r285607: Emit Section Map stream.
That change broke buildbots.

llvm-svn: 285618
2016-10-31 18:50:13 +00:00
Rui Ueyama 86a1ef388b Emit Section Map stream.
This is how we use DbiStreamBuilder::addSectionMap.

Differential Revision: https://reviews.llvm.org/D26113

llvm-svn: 285607
2016-10-31 17:40:40 +00:00
Hans Wennborg cb8f2ef644 LTO: Don't follow lazy references when seeing new definitions in the combined LTO object
Following the lazy reference might bring in an object file that depends
on bitcode files that weren't part of the LTO step.

Differential Revision: https://reviews.llvm.org/D25461

llvm-svn: 283989
2016-10-12 09:00:06 +00:00
Rui Ueyama e05e9502c5 Fix a test to pacify llvm-clang-lld-x86_64-scei-ps4-windows10pro bot.
I don't really understand why we get a larger .rodata section only
on this bot. I guess it may be picking up a library which contains
a .rodata. I removed the specific values since their values are not
important for this test case.

llvm-svn: 283931
2016-10-11 20:15:36 +00:00
Rui Ueyama 9f66f8277d Re-submit r283825: Add section header stream to PDB.
It was reverted because the change that depends on was reverted.
Now it was submitted as r283925, so we can submit this as well.

llvm-svn: 283926
2016-10-11 19:45:07 +00:00
Rui Ueyama 9aa4ab6f9b Revert "Add section header stream to PDB." because it depends on r283823.
The change this patch depends on was reverted.

llvm-svn: 283837
2016-10-11 01:01:40 +00:00
Rui Ueyama 55505954fe Add section header stream to PDB.
Differential Revision: https://reviews.llvm.org/D25357

llvm-svn: 283825
2016-10-10 23:44:10 +00:00
Rui Ueyama 1343fac3ca Add an empty DBI stream.
llvm-svn: 283504
2016-10-06 22:52:01 +00:00
Rui Ueyama 8d3fb5da7b Add exact number of streams for reserved stream #s.
llvm-svn: 283397
2016-10-05 22:08:58 +00:00
Rui Ueyama d381c9842b Add an empty IPI stream.
With this, "llvm-pdbdump yaml -ipi-stream" prints out an IPI stream.
Previously it crashed because it can't handle the case where IPI
stream doesn't exist.

llvm-svn: 283392
2016-10-05 21:37:25 +00:00
Rui Ueyama 129795497d Change for LLVM upstream change r282944.
llvm-svn: 282945
2016-09-30 20:53:45 +00:00
Martin Storsjo 63762b58c0 [COFF] Add support for IMAGE_REL_ARM_SECREL
Handle this in the exact same way as IMAGE_REL_AMD64_SECREL
and IMAGE_REL_I386_SECREL.

Differential revision: https://reviews.llvm.org/D24608

llvm-svn: 282531
2016-09-27 19:27:17 +00:00
Rui Ueyama bb542b3f0c Partially fill Info stream with proper values.
llvm-svn: 281795
2016-09-16 22:51:17 +00:00
Rui Ueyama b28c6d495d Use functions in DebugInfoPDB to create dummy PDB file.
I don't think we are creating valid PDB file here,
but it is okay because we have never created valid PDBs before.

llvm-svn: 281704
2016-09-16 04:32:33 +00:00
Rui Ueyama 783afd667d Check /pdb output using llvm-pdbdump.
test/COFF/rsds.test checks only RSDS directory in a DLL and
didn't check the validity of the PDF file produced.
(Technically the produced PDB is not valid because it is really
a stub, but this test is still good to have.)

llvm-svn: 281678
2016-09-15 23:14:40 +00:00
Saleem Abdulrasool c1b5f19262 test: adjust the GUID
Although the GUID seems to be stable across test runs now, it seems to be
unstable across hosts.  Lets be a bit more lax about the reading of the RSDS
record.

llvm-svn: 281083
2016-09-09 19:41:46 +00:00
Saleem Abdulrasool 88f6407542 COFF: make builds more reproducible
Change the way we calculate the build id to use MD5 to give reproducible build
ids. Previously we would generate random bytes for the build id GUID.

llvm-svn: 281079
2016-09-09 19:26:03 +00:00
Saleem Abdulrasool 8fcff934ba COFF: add beginnings of debug directory creation
The IMAGE_FILE_HEADER structure contains a (RVA, size) to an array of
COFF_DEBUG_DIRECTORY records. Each one of these records contains an RVA to a OMF
Debug Directory. These OMF debug directories are derived into newer types such
as PDB70, PDB20, etc. This constructs a PDB70 structure which will allow us to
associate a GUID with a build to actually tie debug information.

llvm-svn: 280012
2016-08-29 21:20:46 +00:00
Saleem Abdulrasool 45a574130e COFF ARM: Fix typos in the relocation test
Fix comments.  NFC.

Patch by Martin Storsjö!

llvm-svn: 277929
2016-08-06 18:02:51 +00:00
Saleem Abdulrasool fd063e796f COFF ARM: Apply an existing offset in MOV32T relocations
Don't blindly OR in the new value, but clear the existing one, since it can be
nonzero. Read out the existing value before, and add into the desired offset.
(The add is done outside of the applyMOV, to handle potential overflow between
the two.)

Patch by Martin Storsjö!

llvm-svn: 277846
2016-08-05 18:20:31 +00:00
Saleem Abdulrasool eea45bc478 COFF ARM: Error out if 24 bit thumb branches are out of range
In the ELF linker, the same situation already errors out with "relocation
R_ARM_THM_CALL out of range".

Patch by Martin Storsjö!

llvm-svn: 277838
2016-08-05 17:33:24 +00:00
Saleem Abdulrasool 8202c6dbdf COFF ARM: Clear the J1 and J2 bits when applying relocations to 24 bit branches
The opcode for the bl branches can initially be F000 F800, i.e.
the J1 and J2 bits are already set. Therefore mask these bits out
before or'ing in the new bits.

Patch by Martin Storsjö!

llvm-svn: 277836
2016-08-05 17:28:21 +00:00
Rafael Espindola cc83ad18f7 Disable test on windows.
llvm-svn: 276742
2016-07-26 12:38:18 +00:00
Peter Collingbourne feee2103c6 COFF: Implement /linkrepro flag.
This flag is implemented similarly to --reproduce in the ELF linker.

This patch implements /linkrepro by moving the cpio writer and associated
utility functions to lldCore, and using that implementation in both linkers.

One COFF-specific detail is that we store the object file from which the
resource files were created in our reproducer, rather than the resource
files themselves. This allows the reproducer to be used on non-Windows
systems for example.

Differential Revision: https://reviews.llvm.org/D22418

llvm-svn: 276719
2016-07-26 02:00:42 +00:00
Chris Bieneman ca5170e3e4 [yaml2obj] Updating LLD tests to account for LLVM r273915
This removed the -format flag from yaml2obj in favor of YAML tags.

llvm-svn: 273920
2016-06-27 20:17:08 +00:00
Rui Ueyama 440138c4c8 [COFF] Add /section command line flag.
llvm-svn: 273134
2016-06-20 03:39:39 +00:00
Nico Weber 984cd3535d Reland test change.
llvm-svn: 266943
2016-04-21 01:33:04 +00:00
Nico Weber a7a848f10d Revert unintentionally commited bits in r266935.
llvm-svn: 266942
2016-04-21 01:31:37 +00:00
Nico Weber 848334704d unbreak COFF/out.test after r266929
llvm-svn: 266935
2016-04-20 23:30:14 +00:00
Nico Weber 5660de7877 lld-link: Fix default output name with /dll flag.
If /dll is passed, the default output filename should be foo.dll,
not foo.exe.
http://reviews.llvm.org/D19321

llvm-svn: 266929
2016-04-20 22:34:15 +00:00
Rui Ueyama afb1901e42 COFF: Support /manifestinput command line option.
Manifest file is a separate or embedded XML file having metadata
of an executable. As it is XML, it can contain various types of
information. Probably the most popular one is to request escalated
priviledges.

Usually the linker creates an XML file and embed that file into
an executable. However, there's a way to supply an XML file from
command line. /manifestniput is it.

Apparently it is over-designed here, but if you supply two or more
manifest files, then the linker needs to merge the files into a
single XML file. A good news is that we don't need to do that ourselves.
MT.exe command can do that, so we call the command from the linker
in this patch.

llvm-svn: 266704
2016-04-19 01:21:58 +00:00
David Majnemer 18558b6d9e [COFF] Correct tests with bogus alignment
llvm-svn: 263838
2016-03-18 21:47:12 +00:00
David Majnemer 07f7fe5a4c [COFF] Fix invalid alignment in tests
Some COFF tests used INT_MIN for the alignment of the directive section.
This is invalid; replace the alignment with something more sensible: 1.

llvm-svn: 263723
2016-03-17 16:56:31 +00:00
David Majnemer 272de1e4d1 [COFF] Don't trust a symbol's section number
This fixes a test which exposed an ASan issue.

We assumed that a symbol's section number had a corresponding section
without performing validation.

llvm-svn: 263558
2016-03-15 16:47:28 +00:00
David Majnemer 22dff0aafc [COFF] Don't hard-code the load configuration size
The load configuration directory is a structure whose size varies as the
OS gains additional functionality.  To account for this, the structure's
layout begins with a size field; this allows loaders to know which
fields are available.

However, LLD hard-coded the sizes (112 bytes for 64-bit and 64 for
32-bit).  This means that we might not inform the loader of all the
pertinent fields or we might claim that there are more fields than are
actually present.

To correctly account for this, the size field must be loaded from the
_load_config_used symbol.

N.B.  The COFF spec is either wrong or out of date, the load
configuration directory is not correctly documented in the
specification: it omits the size field.

llvm-svn: 263543
2016-03-15 09:48:27 +00:00
David Majnemer 22a19a9ace [COFF] Use the correct size of the TLS directory
The TLS directory has a different layout depending on the bitness of the
machine the image will run on.  LLD would always use the 64-bit TLS
directory for the data directory entry instead of an appropriately sized
TLS directory.

llvm-svn: 263539
2016-03-15 06:41:02 +00:00
Rui Ueyama 436d91d835 COFF: Temporarily remove flaky a test.
This test is flaky for more than half a year or so on buildbots
and has been causing confusion. Remove it while I'm investing the
cause.

llvm-svn: 261709
2016-02-24 00:23:11 +00:00
Rui Ueyama 4b00e7310f COFF: Update test for recent llvm-objdump change (r257539).
This test now correctly checks if the specified forwarding-to
symbol exists in the export table.

llvm-svn: 257540
2016-01-12 23:29:42 +00:00
Rui Ueyama 84425d7289 COFF: Implement DLL symbol forwarding.
DLL export tables usually contain dllexport'ed symbol RVAs so that
applications which use the DLLs can find symbols from the DLLs.
However, there's a minor feature to "forward" DLL symbols to other
DLLs.

If you set an RVA to a string whose form is "<dllname>.<symbolname>"
(e.g. "KERNEL32.ExitProcess") instead of symbol RVA to the export
table, the loader interprets that as a forwarder symbol, and resolve
that symbol from the specified DLL.

This patch implements that feature.

llvm-svn: 257243
2016-01-09 01:22:00 +00:00
Rui Ueyama 43e12900d9 COFF: Non-external COMDAT sections sholud not be merged by ICF.
If a section symbol is not external, that COMDAT section should never
be merge with other sections in other compilation unit. Previously,
we didn't take visibility into account.

Note that COMDAT sections with non-external visibility makes sense
because they can be removed by dead-stripping.

Fixes https://llvm.org/bugs/show_bug.cgi?id=25686

llvm-svn: 254578
2015-12-03 02:23:33 +00:00
Rui Ueyama 75656eebb8 COFF: /OPT should accept comma-separated multiple arguments.
/OPT:foo,bar is equivalent to /OPT:foo /OPT:bar.

Reported by alexchandel. https://llvm.org/bugs/show_bug.cgi?id=25228

llvm-svn: 250728
2015-10-19 19:40:43 +00:00
Rui Ueyama 548d22c073 COFF: ICF should not merge sectinos if their alignments are not the same.
There's actually a room to improve this patch. Instead of not merging
sections that have different alignements, we can choose the section that
has the largest alignment requirement among all sections that are otherwise
considered the same. Then all section alignments are satisfied, so we can
merge them.

I don't know if that improvement could make any difference for real-world
input, so I'll leave it alone. Would be interesting to revisit later.

llvm-svn: 248581
2015-09-25 16:50:12 +00:00
Rui Ueyama 9640173e16 COFF: Add /nosymtab command line option.
This is an LLD extension to MSVC link.exe command line. MSVC linker
does not write symbol tables for executables. We do unless no /debug
option is given.

There's a situation that we want to enable debug info but don't want
to emit the symbol table. One example is when we are comparing output
file size. With this patch, you can tell the linker to not create
a symbol table by just specifying /nosymtab.

llvm-svn: 248225
2015-09-21 23:43:31 +00:00
Rui Ueyama 5f38915624 COFF: Fix ICF regression.
This patch fixes a regression introduced by r247964. Relocations that
are referring the same symbol should be considered equal, but they
were not if they were pointing to non-section chunks.

llvm-svn: 248132
2015-09-20 20:19:12 +00:00
Rui Ueyama 4dbff20c91 COFF: Fix bug that not all symbols were written to symtab if /opt:noref.
Only live symbols are written to the symbol table. Because isLive()
returned false if dead-stripping was disabled entirely, only
non-COMDAT sections were written to the symbol table. This patch fixes
the issue.

llvm-svn: 247856
2015-09-16 21:40:47 +00:00
Rui Ueyama 3b153e6541 COFF: Fix bug that /opt:noicf was ignored.
llvm-svn: 247854
2015-09-16 21:30:55 +00:00
Rui Ueyama b02a320f5e COFF: Enable ICF by default.
MSVC linker enables ICF as long as /opt:ref is eanbled, so do we.

llvm-svn: 247817
2015-09-16 16:41:38 +00:00
Rui Ueyama c04d5dbf20 COFF: Rename /opt:lldicf -> /opt:icf.
Now that ICF is complete, we can rename this option so that
the driver accepts the MSVC-compatible command line option.

llvm-svn: 247816
2015-09-16 16:33:57 +00:00
Rui Ueyama 9cb2870ce0 ICF: Improve ICF to reduce more sections than before.
This is a patch to make LLD to be on par with MSVC in terms of ICF
effectiveness. MSVC produces a 27.14MB executable when linking LLD.
LLD previously produced a 27.61MB when self-linking. Now the size
is reduced to 27.11MB. Note that without ICF the size is 29.63MB.

In r247387, I implemented an algorithm that handles section graphs
as cyclic graphs and merge them using SCC. The algorithm did not
always work as intended as I demonstrated in r247721. The new
algortihm implemented in this patch is different from the previous
one. If you are interested the details, you want to read the file
comment of ICF.cpp.

llvm-svn: 247770
2015-09-16 03:26:31 +00:00
Rui Ueyama c26bbfe6b6 COFF: Add a test for ICF which LLD cannot handle yet.
In this test, we have two functions, foo and bar. MSVC linker can
choose one and discard the other using ICF. LLD cannot. I add this
test as a TODO.

foo and bar are conceptually equivalent to the following:

  void foo() { foo(); }
  void bar() { foo(); }

foo and bar are effectively the same function. If foo and bar are
compiled to the same instructions, both their contents (foo and bar)
and relocation targets (foo) become the same, so from the ICF point
of view, they are reducible. But their graphs are not isomorphic!
LLD's ICF algorithm cannot handle this case yet.

llvm-svn: 247721
2015-09-15 21:17:12 +00:00
Rui Ueyama 8e186df2f5 COFF: Corrected error message if a section failed to load.
There is no sense to use Name in these lines as it is not initialized yet.

Patch from Igor Kudrin!

llvm-svn: 247531
2015-09-13 20:22:22 +00:00
Rui Ueyama 5b93aa51de COFF: Teach ICF to merge cyclic graphs.
Previously, LLD's ICF couldn't merge cyclic graphs. That was unfortunate
because, in COFF, cyclic graphs are not exceptional at all. That is
pretty common.

In this patch, sections are grouped by Tarjan's strongly connected
component algorithm to get acyclic graphs. And then we try to merge
SCCs whose outdegree is zero, and remove them from the graph. This
makes other SCCs to have outdegree zero, so we can repeat the
process until all SCCs are removed. When comparing two SCCs, we handle
cycles properly.

This algorithm works better than previous one. Previously, self-linking
produced a 29.0MB executable. It now produces a 27.7MB. There's still some
gap compared to MSVC linker which produces a 27.1MB executable for the
same input. So the gap is narrowed, but still LLD is not on par with MSVC.
I'll investigate that later.

llvm-svn: 247387
2015-09-11 04:29:03 +00:00
David Blaikie 947bf5e0bb [opaque pointer type] update test cases for explicit pointee types on global aliases
llvm-svn: 247379
2015-09-11 03:22:12 +00:00
Rui Ueyama 6295b27184 COFF: /delayload:<DLLNAME> is case-insensitive.
llvm-svn: 246770
2015-09-03 14:49:47 +00:00
Rui Ueyama 6f5ec97dc0 COFF: Attempt to fix a flaky test.
I don't understand why the previous code is pretty flaky and
the new code is at least less flaky, but the original test
occasionally failed on the second run of lib.exe.

My guess was that lib.exe was failing because the output of
the echo command executed immediately before lib.exe was not
flushed to a file, but as far as I can say, the file
descriptor is properly closed in TestRunner.py, so this's
probably not correct. Other theory is that, on Windows, file
output is not guaranteed to be visible to other processes even
if a process flushes file descriptors, but I'd think that's
unlikely. So honestly I don't know the cause yet.

llvm-svn: 246621
2015-09-02 08:10:37 +00:00
Rui Ueyama bfbd277a1c COFF: Preserve original spelling of DLL file name.
This patch fixes a subtle incompatibility with MSVC linker.
MSVC linker preserves the original spelling of a DLL in the
import descriptor table. LLD previously converted all
characters to lowercase. Usually this difference is benign,
but if a program explicitly checks for DLL file names, the
program could fail.

llvm-svn: 246620
2015-09-02 07:27:31 +00:00
Ed Maste 6194363b31 Exclude COFF/export-exe.test on non-Windows
llvm-svn: 246559
2015-09-01 17:26:33 +00:00
Rui Ueyama 100ffacf2c COFF: .exe files should be able to export functions.
In r246424, I made a change that disables non-DLL to export
symbols. It turned out that the change was not correct. Both
DLLs and executables are able to export symbols (although the
latter is relatively rare). This change restores the feature.

llvm-svn: 246537
2015-09-01 09:15:58 +00:00
Rui Ueyama dfff2542b8 COFF: Make import libraries compatible with MSVC link.
I have totally no idea why, but MSVC linker is sensitive about
file names of archive members. If we do not make import library
file names to the same as the DLL name, MSVC link *crashes*
when it is processing the library file. This patch is to set
the same name.

llvm-svn: 246535
2015-09-01 08:08:57 +00:00
Rui Ueyama 84f9218417 COFF: Set "Data" bit for data symbols in the import descriptor.
llvm-svn: 246533
2015-09-01 06:46:10 +00:00
Rui Ueyama f10a32014d COFF: Improve dllexported name mangling compatibility.
The rules for dllexported symbols are overly complicated due to
x86 name decoration, fuzzy symbol resolution, and the fact that
one symbol can be resolved by so many different names. The rules
are probably intended to be "intuitive", so that users don't have
to understand the name mangling schemes, but it seems that it can
lead to unintended symbol exports.

To make it clear what I'm trying to do with this patch, let me
write how the export rules are subtle and complicated.

 - x86 name decoration: If machine type is i386 and export name
   is given by a command line option, like /export:foo, the
   real symbol name the linker has to search for is _foo because
   all symbols are decorated with "_" prefixes. This doesn't happen
   on non-x86 machines. This automatic name decoration happens only
   when the name is not C++ mangled.

   However, the symbol name exported from DLLs are ones without "_"
   on all platforms.

   Moreover, if the option is given via .drectve section, no
   symbol decoration is done (the reason being that the .drectve
   section is created by a compiler and the compiler should always
   know the exact name of the symbol, I guess).

 - Fuzzy symbol resolution: In addition to x86 name decoration,
   the linker has to look for cdecl or C++ mangled symbols
   for a given /export. For example, it searches for not only
   _foo but also _foo@<number> or ??foo@... for /export:foo.

Previous implementation didn't get it right. I'm trying to make
it as compatible with MSVC linker as possible with this patch
however the rules are. The new code looks a bit messy to me, but
I don't think it can be simpler due to the ad-hoc-ness of the rules.

llvm-svn: 246424
2015-08-31 08:43:21 +00:00
Peter Collingbourne df5783b7a5 COFF: Implement parallel LTO code generation.
This is exposed via a new flag /opt:lldltojobs=N, where N is the number of
code generation threads.

Differential Revision: http://reviews.llvm.org/D12309

llvm-svn: 246342
2015-08-28 22:16:09 +00:00
Rui Ueyama d020c65d24 COFF: Attempt to unbreak buildbots.
This is an attempt to unbreak http://reviews.llvm.org/P122.

llvm-svn: 246000
2015-08-26 02:52:05 +00:00
Rui Ueyama 473aa4088d COFF: Add a test for ICF and circular references.
ICF is a feature to merge sections not by name (which is the regular
COMDAT merging) but by contents. If two or more sections have the
identical contents and relocations, ICF merges them to save space.
Accessors or templated functions tend to have the same contents, and
ICF can hold them.

If we consider sections as vertices and relocations as edges, the
problem is to find as many isomorphic graphs as possile from input
graphs. MSVC linker is smart enough to identify isomorphic graphs
even if they contain circles (GNU gold cannot handle circles
according to http://research.google.com/pubs/pub36912.html, so this
is impressive).

Circular references are not uncommon in COFF object files.
One example is .pdata. .pdata sections contain exception handler info
for functions, so they naturally have relocations for the functions.
The functions in turn have references to the .pdata sections so that
the functions and their .pdata are linked together. As a result, they
form circles.

This is a test case for circular graphs. LLD is not able to handle
this test case yet. I'll add code soon.

llvm-svn: 245827
2015-08-24 08:11:53 +00:00
Rui Ueyama 28103266c6 COFF: Update a test for ICF.
The old test files were just compiler outputs, so it was hard to
debug if something goes wrong. The new test file is carefully
hand-crafted to trigger ICF to avoid that.

llvm-svn: 245826
2015-08-24 07:52:08 +00:00
Rui Ueyama 7171c82194 COFF: Allow forward reference for weak externals
Previously, weak external symbols could reference only symbols that
appeared before them. Although that covers almost all use cases
of weak externals, there are object files out there which contains
weak externals that have forward references.

This patch supports such weak externals.

llvm-svn: 245258
2015-08-17 23:35:43 +00:00
Rui Ueyama 27716bb9d9 COFF: Fix a broken test.
This test couldn't be run more than once because lib.exe does not work
if the files already exist.

llvm-svn: 245214
2015-08-17 11:06:21 +00:00
Rui Ueyama d157088adb COFF: Fix the order of the DLL import entry.
There are some DLLs whose initializers depends on other DLLs'
initializers. The initialization order matters for them.

MSVC linker uses the order of the libraries from the command line.
LLD used ASCII-betical order. So they were incompatible.
This patch makes LLD compatible with MSVC.

llvm-svn: 245201
2015-08-17 08:30:31 +00:00
Peter Collingbourne 526ff15546 COFF: Introduce flag /opt:lldlto=N for controlling LTO optimization level.
Differential Revision: http://reviews.llvm.org/D12024

llvm-svn: 245027
2015-08-14 04:47:07 +00:00
Rafael Espindola 40dfae36fc This test needs lib.exe.
llvm-svn: 244844
2015-08-13 00:03:47 +00:00
Rui Ueyama fa071e13aa COFF: Align sections to 512-byte boundaries on disk.
Sections must start at page boundaries in memory, but they
can be aligned to sector boundaries (512-bytes) on disk.
We aligned them to 4096-byte boundaries even on disk, so we
wasted disk space a bit.

llvm-svn: 244691
2015-08-11 23:09:00 +00:00
Rui Ueyama 6872455c6d COFF: Make test commands shorter. NFC.
llvm-svn: 244227
2015-08-06 16:47:46 +00:00
Rui Ueyama 251b0e268b COFF: Remove the old COFF linker and make link an alias to link2.
It's time to remove old COFF linker because the new one is now complete.

llvm-svn: 244226
2015-08-06 16:19:35 +00:00
Rafael Espindola 4280a96468 Handle writeImportLibrary failing.
We were printing an error but exiting with 0.

Not sure how to test this. We could add a no-winlib feature,
but that is probably not worth it.

llvm-svn: 244109
2015-08-05 20:03:57 +00:00
Rui Ueyama ba7c041f21 COFF: ARM: Implepment BLX23T relocation and fix Branch20T.
I fed the same test to MSVC linker and got the same output,
so I believe this implementation is correct.

llvm-svn: 244102
2015-08-05 19:40:07 +00:00
David Majnemer bf1e5c0ee9 COFF: Reenable the lld test disabled in r243758.
llvm-svn: 243762
2015-07-31 17:41:11 +00:00
Reid Kleckner 60dde1fbda Disable objdump disassembly test after r243755
Right now PE image section addresses are RVAs and symbol addresses are
VAs.  We should probably fix this by changing section addresses to match
symbol addresses. Fixing this might take a few hours, so temporarily
disable the objdump part of this test.

llvm-svn: 243758
2015-07-31 16:41:23 +00:00
Peter Collingbourne e7107ec03b COFF: When resolving _load_config_used, add it as a GC root.
This fixes the cases where the symbol is defined in a comdat or by bitcode.

Differential Revision: http://reviews.llvm.org/D11673

llvm-svn: 243735
2015-07-31 05:33:34 +00:00
Rui Ueyama 7ed10b84cc Remove trailing whitespace.
llvm-svn: 243683
2015-07-30 20:47:17 +00:00
David Majnemer 13ac40ea6e COFF: Sort output sections which start with .debug to the end of the file
We want to convince the NT loader not to map these sections into memory.
A good first step is to move them to the end of the executable.

Differential Revision: http://reviews.llvm.org/D11655

llvm-svn: 243680
2015-07-30 20:26:55 +00:00
Rui Ueyama ff88d5a26f COFF: Add /safeseh command line option.
If /safeseh is specified, all input files must be compatible with Safe SEH.

llvm-svn: 243565
2015-07-29 20:25:40 +00:00
Rui Ueyama 8bc43a142b COFF: ARM: Fix relocations to thumb code.
Windows ARM is the thumb ARM environment, and pointers to thumb code
needs to have its LSB set. When we apply relocations, we need to
adjust the LSB if it points to an executable section.

llvm-svn: 243560
2015-07-29 19:25:00 +00:00
Rui Ueyama 56965f4b32 COFF: ARM: Fix DLL import table.
The previous test was testing -flavor link. This patch correctly
tests link2 and fixes a bug that we didn't emit import thunks.

llvm-svn: 243559
2015-07-29 19:24:58 +00:00
Rui Ueyama 308915ade2 COFF: Fix seh.test.
seh.test was testing -flavor link. This should have been testing
-flavor link2.

llvm-svn: 243557
2015-07-29 19:17:15 +00:00
Rui Ueyama eb26e1d03c COFF: Fix SECREL and SECTION relocations.
SECREL should sets the 32-bit offset of the target from the beginning
of *target's* output section. Previously, the offset from the beginning
of source's output section was used instead.

SECTION means the target section's index, and not the source section's
index. This patch fixes that issue too.

llvm-svn: 243535
2015-07-29 16:30:45 +00:00
Rui Ueyama 29f74c312a COFF: Set load config table entry on non-x86.
llvm-svn: 243532
2015-07-29 16:30:31 +00:00
Rui Ueyama 506f6d1ae1 COFF: _tls_used is __tls_used on x86.
llvm-svn: 243495
2015-07-28 22:56:02 +00:00
Rui Ueyama 9420dee328 COFF: Fix export symbol names for x86.
I don't fully understand the rationale behind the name mangling
scheme used for the DLL export table and the import library.
Why only leading "_" is dropped for the import library while
both "_" and "@" are dropped from DLL symbol table? But this seems
to be what MSVC linker does.

llvm-svn: 243490
2015-07-28 22:34:24 +00:00
Rui Ueyama cfb874a3d4 COFF: Do not ignore /merge if /debug is specified.
Previously, we ignore /merge option if /debug is specified
because I thought that was MSVC linker did. This was wrong.
/merge shouldn't be ignored even in debug mode.

llvm-svn: 243375
2015-07-28 03:24:23 +00:00
Rui Ueyama d68e211be5 COFF: /HighEntropyVA is on by default only on 64-bit.
llvm-svn: 243374
2015-07-28 03:15:57 +00:00
Rui Ueyama 4d54534627 COFF: Add /LargeAddressAware command line option.
llvm-svn: 243373
2015-07-28 03:12:00 +00:00
Rui Ueyama 7e387a68de COFF: Fix 32-bit delay-import address table.
The address table entry is 32-bit wide on 32-bit and 64-bit on 64-bit.
Previously, it was 64-bit even on 32-bit.

llvm-svn: 243372
2015-07-28 02:54:18 +00:00
Rui Ueyama a8eed749a2 COFF: Write import library symbols to a symbol table.
Previously no __imp_ symbols nor dllimport thunk functions were
written to a symbol table.

llvm-svn: 243350
2015-07-27 23:40:20 +00:00
Rui Ueyama 83175276ef COFF: Add REQUIRES arm to tests which use the ARM disassembler.
llvm-svn: 243346
2015-07-27 23:27:03 +00:00
Rui Ueyama 5c437cd1e9 COFF: Fix image base address for 32-bit.
0x140000000 or 0x180000000 are not correct image base addresses
for 32-bit. They are actually much smaller.

llvm-svn: 243228
2015-07-25 21:42:33 +00:00
Rui Ueyama 3dd9372d2b COFF: ARM: Support import functions.
llvm-svn: 243205
2015-07-25 03:39:29 +00:00
Rui Ueyama cde7a7907e COFF: ARM: Implement BLX23T relocation.
llvm-svn: 243204
2015-07-25 03:25:28 +00:00
Rui Ueyama 3d9c8639c3 COFF: ARM: Implement BRANCH24T relocation.
llvm-svn: 243202
2015-07-25 03:19:34 +00:00
Rui Ueyama 237fca1451 COFF: ARM: Implement MOV32T relocation.
llvm-svn: 243201
2015-07-25 03:03:46 +00:00
Rui Ueyama a265b01353 COFF: ARM: Set correct entry point address.
llvm-svn: 243199
2015-07-25 02:25:14 +00:00
Rui Ueyama 1c341a54de COFF: Do not align import thunks on 16-byte boundaries on x86.
Looks like MSVC linker aligns them only on x64.

llvm-svn: 243194
2015-07-25 01:16:04 +00:00
Rui Ueyama cd3f99b6c5 COFF: Implement Safe SEH support for x86.
An object file compatible with Safe SEH contains a .sxdata section.
The section contains a list of symbol table indices, each of which
is an exception handler function. A safe SEH-enabled executable
contains a list of exception handler RVAs. So, what the linker has
to do to support Safe SEH is basically to read the .sxdata section,
interpret the contents as a list of symbol indices, unique-fy and
sort their RVAs, and then emit that list to .rdata. This patch
implements that feature.

llvm-svn: 243182
2015-07-24 23:51:14 +00:00
Rui Ueyama 2296dc137c COFF: Fix base relocation type for x86.
llvm-svn: 243178
2015-07-24 23:24:45 +00:00
Rui Ueyama 3cb895c930 COFF: Fix __ImageBase symbol relocation.
__ImageBase is a special symbol whose value is the image base address.
Previously, we handled __ImageBase symbol as an absolute symbol.

Absolute symbols point to specific locations in memory and the locations
never change even if an image is base-relocated. That means that we
don't have base relocation entries for absolute symbols.

This is not a case for __ImageBase. If an image is base-relocated, its
base address changes, and __ImageBase needs to be shifted as well.
So we have to have base relocations for __ImageBase. That means that
__ImageBase is not really an absolute symbol but a different kind of
symbol.

In this patch, I introduced a new type of symbol -- DefinedRelative.
DefinedRelative is similar to DefinedAbsolute, but it has not a VA but RVA
and is a subject of base relocation. Currently only __ImageBase is of
the new symbol type.

llvm-svn: 243176
2015-07-24 22:58:44 +00:00
Sean Silva 1bc87776ec Add missing REQUIRES to this test.
An internal builder was failing.

llvm-svn: 242452
2015-07-16 22:12:44 +00:00
Rui Ueyama afad42f9ea COFF: Set Load Configuration entry in Data Directory.
Load Configuration field points to a structure containing information
for SEH. That data strucutre is not created by the linker but provided
by an external file. What we have to do is just to set __load_config_used
address to the header.

llvm-svn: 242427
2015-07-16 18:30:35 +00:00
Nico Weber f26ff282df Fix test/COFF/dll.test after r242342.
r242342 removed the RUN line that wrote t2.obj, but another RUN line further
down still reads t2.obj, so add it back.

llvm-svn: 242415
2015-07-16 16:45:01 +00:00
Rui Ueyama 759c8aa9a0 COFF: Fix offset in x86 delay-load thunks.
llvm-svn: 242353
2015-07-15 23:01:36 +00:00
Rui Ueyama ef0e647581 COFF: Implement x86 delay-load thunks.
llvm-svn: 242343
2015-07-15 22:26:57 +00:00
Rui Ueyama 8765fbae15 COFF: Fix mangled dllexported names.
If a symbol is exported as /export:foo, and foo is resolved as a
mangled name (_foo@<number> or ?foo@@Y...), that mangled name should
be written to the export table. Previously, we wrote the original
name to the export table.

llvm-svn: 242342
2015-07-15 22:21:08 +00:00
Rui Ueyama 33fb2cb11b COFF: Fix base relocations for __imp_ symbols on x86.
Because thunks for dllimported symbols contain absolute addresses on x86,
they need to be relocated at load-time. This bug was a cause of crashes
in DLL initialization routines.

llvm-svn: 242259
2015-07-15 00:25:38 +00:00
Rui Ueyama a50387f1b3 COFF: Fix entry name inference for x86.
Entry name selection rule is already complicated on x64, but it's more
complicated on x86 because of the underscore name mangling scheme.

If one of _main, _main@<number> (a C function) or ?main@@... (a C++ function)
is defined, entry name is _mainCRTStartup. If _wmain, _wmain@<number or
?wmain@@... is defined, entry name is _wmainCRTStartup. And so on.

llvm-svn: 242110
2015-07-14 02:58:13 +00:00
Rui Ueyama 6d24908fe7 COFF: Fix x86 delay-load helper function name.
If /delayload option is given, we have to resolve __delayLoadHelper2
since the function is the dynamic loader to delay-load DLLs.
The function name is mangled in x86 as ___delayLoadHelper2@8.

llvm-svn: 242078
2015-07-13 22:31:45 +00:00
Rui Ueyama c851ccc3bd COFF: Fix locally-imported symbol's base relocations.
Base relocations are RVA and not VA, so we shouldn't add ImageBase.

llvm-svn: 241883
2015-07-10 04:30:54 +00:00
Rui Ueyama 270960f5cd COFF: Find C++ mangled name for symbols starting with underscore.
Symbol foo is mangled as _foo in C and ?foo@@... in C++ on x86.
findMangle has to remove prefix underscore before mangle a given name
as a C++ symbol.

llvm-svn: 241874
2015-07-09 23:03:51 +00:00
Rui Ueyama bbdec4fc82 COFF: Fix dllexported symbol names on x86.
Symbol names are usually mangled by appending "_" prefix on x86.
But the mangled name is not used in DLL export table. The export
table contains unmangled names.

llvm-svn: 241872
2015-07-09 22:51:41 +00:00
Rui Ueyama d4b351f0de COFF: Fix locally-imported symbol's size for x86.
llvm-svn: 241860
2015-07-09 21:15:58 +00:00
Rui Ueyama 93b4571187 COFF: Implement base relocations for x86.
With this patch, LLD is now able to self-link an .exe file for x86
that runs correctly, although I don't think some headers (particularly
SEH) are not correct. DLL support is coming soon.

llvm-svn: 241857
2015-07-09 20:36:59 +00:00
Rui Ueyama a841bb0f5d COFF: Fix import symbol name mangling.
For IMPORT_NAME_NOPREFIX symbols, we should remove only one prefix character.

llvm-svn: 241854
2015-07-09 20:22:41 +00:00
Rui Ueyama ea533cde30 COFF: Infer machine type earlier than before.
Previously, we infer machine type at the very end of linking after
all symbols are resolved. That's actually too late because machine
type affects how we mangle symbols (whether or not we need to
add "_").

For example, /entry:foo adds "_foo" to the symbol table if x86 but
"foo" if x64.

This patch moves the code to infer machine type, so that machine
type is inferred based on input files given via the command line
(but not based on .directives files).

llvm-svn: 241843
2015-07-09 19:54:13 +00:00
David Majnemer 3a62d3d456 COFF: Fill in the type and storage class in the symbol table
We can use the type and storage class from the symbol's original object
file to fill in the linked executable's symbol table.

llvm-svn: 241828
2015-07-09 17:43:50 +00:00
Rui Ueyama 7c3e23fffd COFF: Fix import thunks and name mangling for x86.
With this patch, LLD is now able to correctly link a "hello world"
program written in assembly for 32-bit x86.

llvm-svn: 241771
2015-07-09 01:25:49 +00:00
Rui Ueyama 25522f5d4a COFF: Support 32-bit x86 DLL import table.
llvm-svn: 241767
2015-07-09 00:45:50 +00:00
Rui Ueyama 1c79ce9a4c COFF: Implement dllimported symbol name mangling.
Symbols exported by DLLs are listed in import library files.
Exported names may be mangled by "Import Name Type" field as
described in PE/COFF spec 7.3. This patch implements that
mangling scheme.

llvm-svn: 241719
2015-07-08 20:22:50 +00:00
Peter Collingbourne 04a4711565 COFF: Set parent name for bitcode files.
Differential Revision: http://reviews.llvm.org/D10983

llvm-svn: 241713
2015-07-08 19:14:33 +00:00
David Majnemer 2c345a337c COFF: Emit a symbol table if /debug is specified
Providing a symbol table in the executable is quite useful when
debugging a fully-linked executable without having to reconstruct one
from DWARF.

Differential Revision: http://reviews.llvm.org/D11023

llvm-svn: 241689
2015-07-08 16:37:50 +00:00
Rui Ueyama 4e1536c155 COFF: Fix AMD64_SECTION relocation.
llvm-svn: 241658
2015-07-08 01:47:28 +00:00
Rui Ueyama 11863b4ae1 COFF: Support x86 file header and relocations.
llvm-svn: 241657
2015-07-08 01:45:29 +00:00
Rui Ueyama 84936e0b43 COFF: Check for incompatible machine types.
llvm-svn: 241647
2015-07-07 23:39:18 +00:00
Peter Collingbourne f5339ec035 COFF: Improve undefined symbol diagnostics.
We now report the names of any files containing undefined symbol references.

Differential Revision: http://reviews.llvm.org/D10982

llvm-svn: 241612
2015-07-07 18:38:39 +00:00
Peter Collingbourne 8e17451d54 COFF: Fix bug involving archives defining a symbol multiple times.
Previously we were unnecessarily loading lazy symbols if they appeared in an
archive multiple times, as can happen with comdat symbols. This change fixes
the bug by only loading symbols from archives at load time if the original
symbol was undefined.

Differential Revision: http://reviews.llvm.org/D10980

llvm-svn: 241538
2015-07-07 02:15:25 +00:00
Rui Ueyama 92a8c82076 COFF: Set TLS table header field.
TLS table header field is supposed to have address and size of TLS table.
The linker doesn't have to understand what TLS table is. TLS table's name
is always "_tls_used", so if there's that symbol, the linker simply sets
that symbol's RVA to the header. The size of the TLS table is always 40 bytes.

llvm-svn: 241426
2015-07-06 01:48:01 +00:00
Rui Ueyama 6600eb18cd COFF: Implement /merge option.
/merge:.foo=.bar makes the linker to merge section .foo with section .bar.

llvm-svn: 241396
2015-07-04 23:37:32 +00:00
Peter Collingbourne 2612a32ce5 COFF: Numerous fixes for interaction between LTO and weak externals.
We were previously hitting assertion failures in the writer in cases where
a regular object file defined a weak external symbol that was defined by
a bitcode file. Because /export and /entry name mangling were implemented
using weak externals, the same problem affected mangled symbol names in
bitcode files.

The underlying cause of the problem was that weak external symbols were
being resolved before doing LTO, so the symbol table may have contained stale
references to bitcode symbols. The fix here is to defer weak external symbol
resolution until after LTO.

Also implement support for weak external symbols in bitcode files
by modelling them as replaceable DefinedBitcode symbols.

Differential Revision: http://reviews.llvm.org/D10940

llvm-svn: 241391
2015-07-04 05:28:41 +00:00
Rui Ueyama b0398827c2 COFF: Fix bug in garbage collector.
GC root may have non-regular defined symbols, such as DefinedImportThunk,
so this cast<> was a wrong assumption.

llvm-svn: 241382
2015-07-04 01:10:32 +00:00
Rui Ueyama 4b8cdd20fb COFF: Don't print warning message for identical /export options.
llvm-svn: 241379
2015-07-03 23:23:29 +00:00
Peter Collingbourne da2f094bbb COFF: Fix the case where an object defines a weak external and its alias.
This worked before, but only by accident, and only with assertions disabled.
We ended up storing a DefinedRegular symbol in the WeakAlias field,
and never using it as an Undefined.

Differential Revision: http://reviews.llvm.org/D10934

llvm-svn: 241376
2015-07-03 22:03:36 +00:00
Rui Ueyama d8111f2c2e COFF: Fix ordinal-only delay-imported symbols.
DLLs can export symbols only by ordinal, and DLLs are also able to be
delay-loaded. The combination of the two is valid. I didn't expect
that combination. This patch implements that feature.

With this patch, LLD is now able to link a working executable of Chrome
for 64-bit debug build. The browser seemed to be working fine. Chrome is
good for testing because of its variety and size. It contains various
open-source libraries written by various people. The largest file in
Chrome is chrome.dll whose size is 496MB. LLD can link it in 24 seconds.
MSVC linker takes 48 seconds. So it is exactly 2x faster. (I measured
that with debug info and ICF being turned off.)

With this achievement, I think I can say that the new COFF linker is
now mostly feature complete for x86-64 Windows. I believe there are
still many lingering bugs, though.

llvm-svn: 241318
2015-07-03 04:32:49 +00:00
Rui Ueyama 7a247ee242 COFF: Fix a bug that /delayload was case-sensitive.
llvm-svn: 241316
2015-07-03 01:40:14 +00:00
Rui Ueyama 49d6cd35ad COFF: Fix /base option.
Previously, __ImageBase symbol got a different value than the one
specified by /base:<number> because the symbol was created in the
SymbolTable's constructor. When the constructor is called,
no command line options are processed yet, so the symbol was
created always with the initial value. This caused wrong relocations
and thus caused mysterious crashes of some executables linked by LLD.

llvm-svn: 241313
2015-07-03 00:02:19 +00:00
Rui Ueyama 7a333c66be COFF: Fix locally-imported symbols.
Previously, pointers pointed by locally-imported symbols were broken.
It has only 4 bytes although the correct size is 8 byte. This patch
fixes that bug.

llvm-svn: 241295
2015-07-02 20:33:50 +00:00
Rui Ueyama 85225b0a36 COFF: Infer entry point as early as possible, but not too early.
On Windows, we have four different main functions, {w,}{main,WinMain}.
The linker has to choose a corresponding entry point function among
{w,}{main,WinMain}CRTStartup. These entry point functions are defined
in the standard library. The linker resolves one of them by looking at
which main function is defined and adding a corresponding undefined
symbol to the symbol table.

Object files containing entry point functions conflicts each other.
For example, we cannot resolve both mainCRTStartup and WinMainCRTStartup
because other symbols defined in the files conflict.

Previously, we inferred CRT function name at the very end of name
resolution. I found that that is sometimes too late. If the linker
already linked one of these four archive member objects, it's too late
to change the decision.

The right thing to do here is to infer entry point name after adding
all symbols from command line files and before adding any other files
(which are specified by directive sections). This patch does that.

llvm-svn: 241236
2015-07-02 03:15:15 +00:00
Rui Ueyama 8d3010a1a6 COFF: Change the order of adding symbols to the symbol table.
Previously, the order of adding symbols to the symbol table was simple.
We have a list of all input files. We read each file from beginning of
the list and add all symbols in it to the symbol table.

This patch changes that order. Now all archive files are added to the
symbol table first, and then all the other object files are added.
This shouldn't change the behavior in single-threading, and make room
to parallelize in multi-threading.

In the first step, only lazy symbols are added to the symbol table
because archives contain only Lazy symbols. Member object files
found to be necessary are queued. In the second step, defined and
undefined symbols are added from object files. Adding an undefined
symbol to the symbol table may cause more member files to be added
to the queue. We simply continue reading all object files until the
queue is empty.

Finally, new archive or object files may be added to the queues by
object files' directive sections (which contain new command line
options).

The above process is repeated until we get no new files.

Symbols defined both in object files and in archives can make results
undeterministic. If an archive is read before an object, a new member
file gets linked, while in the other way, no new file would be added.
That is the most popular cause of an undeterministic result or linking
failure as I observed. Separating phases of adding lazy symbols and
undefined symbols makes that deterministic. Adding symbols in each
phase should be parallelizable.

llvm-svn: 241107
2015-06-30 19:35:21 +00:00
Peter Collingbourne a5cb389337 Add layout/triple to fix test on platforms where names are mangled.
llvm-svn: 241031
2015-06-30 01:15:54 +00:00
Peter Collingbourne f7b27d15f2 COFF: Implement SymbolBody::getDebugName() for DefinedBitcode symbols.
Differential Revision: http://reviews.llvm.org/D10827

llvm-svn: 241029
2015-06-30 00:47:52 +00:00
Rui Ueyama 2d5e917bce COFF: Handle mangled entry symbol name.
Compilers recognize "main" function and don't mangle its name.
But if you use a different function as a user-defined entry name,
and if you didn't define that function with extern C, your entry
point function name is mangled. And the linker has to be able to
find that. This is relatively rare but can happen.

llvm-svn: 240953
2015-06-29 14:43:07 +00:00
Rui Ueyama 6b79ed128a COFF: Fix /export.
Mangled dllexported symbols may be defined in a library.
If that's the case, we have to read a member file from the library.

llvm-svn: 240947
2015-06-29 14:27:10 +00:00
Rui Ueyama 45044f47d3 COFF: Fix logic to find default entry name or subsystem.
The previous logic to find default entry name or subsystem does not
seem correct (i.e. was not compatible with MSVC linker). Previously,
default entry name was inferred from CRT functions and user-defined
entry functions. Subsystem was inferred from CRT functions.

Default entry name and subsystem are now inferred based on the
following table. Note that we no longer use CRT functions to infer
them.

               Entry name           Subsystem
  main         mainCRTStartup       console
  wmain        wmainCRTStartup      console
  WinMain      WinMainCRTStartup    windows
  wWinMain     wWinMainCRTStartup   windows

llvm-svn: 240922
2015-06-29 01:03:53 +00:00
Rui Ueyama f5313b3498 COFF: Allow mangled symbols as arguments for /export.
Usually dllexported symbols are defined with 'extern "C"',
so identifying them is easy. We can just do hash table lookup
to look up exported symbols.

However, C++ non-member functions are also allowed to be exported,
and they can be specified with unmangled name. So, if /export:foo
is given, we need to look up not only "foo" but also its all
mangled names. In MSVC mangling scheme, that means that we need to
look up any symbol which starts with "?foo@@Y".

In this patch, we scan the entire symbol table to search for
a mangled symbol. The symbol table is a DenseMap, and that doesn't
support table lookup by string prefix. This is of course very
inefficient. But that should be probably OK because the user
should always add 'extern "C"' to dllexported symbols.

llvm-svn: 240919
2015-06-28 22:16:41 +00:00
Rui Ueyama 091b1a6585 COFF: Fix flaky test.
This test was flaky because stdout and stderr can be mixed.

llvm-svn: 240918
2015-06-28 22:06:53 +00:00
Rui Ueyama a8b60458ea COFF: Add /noentry flag.
This option is sometimes used to create a resource-only DLL that
doesn't need any initialization.

llvm-svn: 240915
2015-06-28 19:56:30 +00:00
Rui Ueyama fe6d11c0db Fix broken test.
llvm-svn: 240914
2015-06-28 19:38:10 +00:00
Rui Ueyama 95925fd1ab COFF: Support /force flag.
This option is to ignore remaining undefined symbols and force
the linker to create an output file anyways.

The existing code assumes that there's no undefined symbol after
reportRemainingUndefines(). That assumption is legitimate.
I also don't want to mess up the existing code for this minor feature.
In order to keep it as is, remaining undefined symbols are replaced
with dummy defined symbols.

llvm-svn: 240913
2015-06-28 19:35:15 +00:00
Rui Ueyama 06cf3df2d5 COFF: Handle LINK environment variable.
If LINK is defined and not empty, it's supposed to contain
command line options.

llvm-svn: 240900
2015-06-28 02:35:31 +00:00
Rui Ueyama 871847e32d COFF: Fix ICF correctness bug.
When comparing two COMDAT sections, we need to take section values
and associative sections into account. This patch fixes that bug.
It fixes a crash bug of llvm-tblgen when linked with /opt:lldicf.

One thing I don't understand yet is that this logic seems to be
too strict. MSVC linker is able to create more compact executables
(which of course work correctly). With this ICF algorithm, LLD is
able to make executable smaller, but the outputs are larger than
MSVC's. There must be something I'm missing here.

llvm-svn: 240897
2015-06-28 01:30:54 +00:00
Rui Ueyama 810551a694 COFF: Add base relocation for delay-import table.
Because the address table of the delay-import table contains
absolute address, it needs to be added to the base relocation
table.

llvm-svn: 240844
2015-06-26 22:05:32 +00:00
Rui Ueyama 382dc96e29 COFF: Fix delay-import tables.
There were a few issues with the previous delay-import tables.

 - "Attribute" field should have been 1 instead of 0.
   (I don't know the meaning of this field, though.)
 - LEA and CALL operands had wrong addresses.
 - Address tables are in .didat (which is read-only).
   They should have been in .data.

llvm-svn: 240837
2015-06-26 21:40:15 +00:00
Peter Collingbourne be54955bba COFF: Implement /lldmap flag.
This flag can be used to produce a map file, which is essentially a list
of objects linked into the final output file together with the RVAs of
their symbols. Because our format differs from MSVC's we expose it as a
separate flag.

Differential Revision: http://reviews.llvm.org/D10773

llvm-svn: 240812
2015-06-26 18:58:24 +00:00
Rui Ueyama 7383562bc9 COFF: Align DLL import thunks on 16-byte boundaries.
llvm-svn: 240806
2015-06-26 18:28:56 +00:00
Rui Ueyama 923cb64801 COFF: Add a test for r240719.
llvm-svn: 240758
2015-06-26 03:50:27 +00:00
Rui Ueyama 32f8e1cb4e COFF: Change symbol resolution order for entry and /include.
We were resolving entry symbols and /include'd symbols after all other
symbols are resolved. But looks like it's too late. I found that it
causes some program to fail to link.

Let's say we have an object file A which defines symbols X and Y in an
archive. We also have another file B after A which defines X, Y and
_DLLMainCRTStartup in another archive. They conflict each other, so
either A or B can be linked.

If we have _DLLMainCRTStartup as an undefined symbol, file B is always
chosen. If not, there's a chance that A is chosen. If the linker
find it needs _DllMainCRTStartup after that, it's too late.

This patch adds undefined symbols to the symbol table as soon as
possible to fix the issue.

llvm-svn: 240757
2015-06-26 03:44:00 +00:00
Rui Ueyama ccde19d77e COFF: Fix local absolute symbols.
Absolute symbols were always handled as external symbols, so if two
or more object files define the same absolute symbol, they would
conflict even if the symbol is private to each file.
This patch fixes that bug.

llvm-svn: 240756
2015-06-26 03:09:23 +00:00
Rui Ueyama a29948873f COFF: Don't read non-x64 object files.
Currently the new LLD supports only x86-64.

llvm-svn: 240749
2015-06-26 00:42:21 +00:00
Rui Ueyama f799edef28 COFF: Rename /opt:icf -> /opt:lldicf.
ICF implemented in LLD is so experimental that we don't want to
enable that even if /opt:icf option is passed. I'll rename it back
once the feature is complete.

llvm-svn: 240721
2015-06-25 23:26:58 +00:00
Rui Ueyama 5817ebb0c8 COFF: Fix lexer for the module-definition file.
Previously it would hang if there's a stray punctuation (e.g. ?).

llvm-svn: 240697
2015-06-25 21:06:00 +00:00
Rui Ueyama 88e0f9206b COFF: Fix a bug of __imp_ symbol.
The change I made in r240620 was not correct. If a symbol foo is
defined, and if you use __imp_foo, __imp_foo symbol is automatically
defined as a pointer (not just an alias) to foo.

Now that we need to create a chunk for automatically-created symbols.
I defined LocalImportChunk class for them.

llvm-svn: 240622
2015-06-25 03:31:47 +00:00
Rui Ueyama d766653534 COFF: Handle undefined symbols starting with __imp_ in a special way.
MSVC linker is able to link an object file created from the following code.
Note that __imp_hello is not defined anywhere.

  void hello() { printf("Hello\n"); }
  extern void (*__imp_hello)();
  int main() { __imp_hello(); }

Function symbols exported from DLLs are automatically mangled by appending
__imp_ prefix, so they have two names (original one and with the prefix).
This "feature" seems to simulate that behavior even for non-DLL symbols.

This is in my opnion very odd feature. Even MSVC linker warns if you use this.
I'm adding that anyway for the sake of compatibiltiy.

llvm-svn: 240620
2015-06-25 02:21:44 +00:00
Rui Ueyama ddf71fc370 COFF: Initial implementation of Identical COMDAT Folding.
Identical COMDAT Folding (ICF) is an optimization to reduce binary
size by merging COMDAT sections that contain the same metadata,
actual data and relocations. MSVC link.exe and many other linkers
have this feature. LLD achieves on per with MSVC in terms produced
binary size with this patch.

This technique is pretty effective. For example, LLD's size is
reduced from 64MB to 54MB by enaling this optimization.

The algorithm implemented in this patch is extremely inefficient.
It puts all COMDAT sections into a set to identify duplicates.
Time to self-link with/without ICF are 3.3 and 320 seconds,
respectively. So this option roughly makes LLD 100x slower.
But it's okay as I wanted to achieve correctness first.
LLD is still able to link itself with this optimization.
I'm going to make it more efficient in followup patches.

Note that this optimization is *not* entirely safe. C/C++ require
different functions have different addresses. If your program
relies on that property, your program wouldn't work with ICF.
However, it's not going to be an issue on Windows because MSVC
link.exe turns ICF on by default. As long as your program works
with default settings (or not passing /opt:noicf), your program
would work with LLD too.

llvm-svn: 240519
2015-06-24 04:36:52 +00:00
Peter Collingbourne c7b685d997 COFF: Ignore debug symbols.
Differential Revision: http://reviews.llvm.org/D10675

llvm-svn: 240487
2015-06-24 00:05:50 +00:00
Rui Ueyama 0d2e999050 COFF: Make link order compatible with MSVC link.exe.
Previously, we added files in directive sections to the symbol
table as we read the sections, so the link order was depth-first.
That's not compatible with MSVC link.exe nor the old LLD.

This patch is to queue files so that new files are added to the
end of the queue and processed last. Now addFile() doesn't parse
files nor resolve symbols. You need to call run() to process
queued files.

llvm-svn: 240483
2015-06-23 23:56:39 +00:00
Rui Ueyama a77336bd5d COFF: Support delay-load import tables.
DLLs are usually resolved at process startup, but you can
delay-load them by passing /delayload option to the linker.

If a /delayload is specified, the linker has to create data
which is similar to regular import table.
One notable difference is that the pointers in a delay-load
import table are originally pointing to thunks that resolves
themselves. Each thunk loads a DLL, resolve its name, and then
overwrites the pointer with the result so that subsequent
function calls directly call a desired function. The linker
has to emit thunks.

llvm-svn: 240250
2015-06-21 22:31:52 +00:00
Rui Ueyama 4d769c3a57 COFF: Support exception table.
.pdata section contains a list of triplets of function start address,
function end address and its unwind information. Linkers have to
sort section contents by function start address and set the section
address to the file header (so that runtime is able to find it and
do binary search.)

This change seems to resolve all but one remaining test failures in
check{,-clang,-lld} when building the entire stuff with clang-cl and
lld-link.

llvm-svn: 240231
2015-06-21 04:00:54 +00:00
Rui Ueyama 5e31d0b2e9 COFF: Fix common symbol alignment.
llvm-svn: 240217
2015-06-20 07:25:45 +00:00
Rui Ueyama efb7e1aa29 COFF: Fix a common symbol bug.
This is a case that one mistake caused a very mysterious bug.
I made a mistake to calculate addresses of common symbols, so
each common symbol pointed not to the beginning of its location
but to the end of its location. (Ouch!)

Common symbols are aligned on 16 byte boundaries. If a common
symbol is small enough to fit between the end of its real
location and whatever comes next, this bug didn't cause any harm.

However, if a common symbol is larger than that, its memory
naturally overlapped with other symbols. That means some
uninitialized variables accidentally shared memory. Because
totally unrelated memory writes mutated other varaibles, it was
hard to debug.

It's surprising that LLD was able to link itself and all LLD
tests except gunit tests passed with this nasty bug.

With this fix, the new COFF linker is able to pass all tests
for LLVM, Clang and LLD if I use MSVC cl.exe as a compiler.
Only three tests are failing when used with clang-cl.

llvm-svn: 240216
2015-06-20 07:21:57 +00:00
Rui Ueyama f00df0af2d COFF: Fix precedence between LIB and /libpath.
/libpath should take precedence over LIB.
Previously, LIB took precedence over /libpath.

llvm-svn: 240182
2015-06-19 22:39:48 +00:00
Rui Ueyama 165b254e06 COFF: Add search paths in the correct order.
Previously, we added search paths in reverse order.

llvm-svn: 240180
2015-06-19 21:44:32 +00:00
Rui Ueyama 573bf7de9c COFF: Continue reading object files until converge.
In this linker model, adding an undefined symbol may trigger chain
reactions. It may trigger a Lazy symbol to read a new file.
A new file may contain a directive section, which may contain various
command line options.

Previously, we didn't handle chain reactions well. We visited /include'd
symbols only once, so newly-added /include symbols were ignored.
This patch fixes that bug.

Now, the symbol table is versioned; every time the symbol table is
updated, the version number is incremented. We repeat adding undefined
symbols until the version number does not change. It is guaranteed to
converge -- the number of undefined symbol in the system is finite,
and adding the same undefined symbol more than once is basically no-op.

llvm-svn: 240177
2015-06-19 21:12:48 +00:00
Rui Ueyama 4d2834bd7b COFF: Don't add new undefined symbols for /alternatename.
Alternatename option is in the form of /alternatename:<from>=<to>.
It's effect is to resolve <from> as <to> if <from> is still undefined
at end of name resolution.

If <from> is not undefined but completely a new symbol, alternatename
shouldn't do anything. Previously, it introduced a new undefined
symbol for <from>, which resulted in undefined symbol error.

llvm-svn: 240161
2015-06-19 19:23:43 +00:00
Rui Ueyama 08d5e1875f COFF: Handle /include in .drectve.
We don't want to insert a new symbol to the symbol table while reading
a .drectve section because it's going to be too complicated.
That we are reading a directive section means that we are currently
reading some object file. Adding a new undefined symbol to the symbol
table can trigger a library file to read a new file, so it would make
the call stack too deep.

In this patch, I add new symbol names to a list to resolve them later.

llvm-svn: 240076
2015-06-18 23:20:11 +00:00
Rui Ueyama e8d56b5258 COFF: Allow identical alternatename options.
Alternatename option is in the form of /alternatename:<from>=<to>.
It is an error if there are two options having the same <from> but
different <to>. It is *not* an error if both are the same.

llvm-svn: 240075
2015-06-18 23:04:26 +00:00
Rui Ueyama 562daa8148 COFF: Unknown options in .drectve section is an error.
We skip unknown options in the command line with a warning message
being printed out, but we shouldn't do that for .drectve section.
The section is not visible to the user. We should handle unknown
options as an error.

llvm-svn: 240067
2015-06-18 21:50:38 +00:00
Rui Ueyama b95188cb2c COFF: Add /implib option.
llvm-svn: 240045
2015-06-18 20:27:09 +00:00
Rui Ueyama 2edb35a264 COFF: Handle /alternatename in .drectve section.
llvm-svn: 240037
2015-06-18 19:09:30 +00:00
Peter Collingbourne 8b2492f2a0 COFF: Implement DLL symbol exports for bitcode files.
Differential Revision: http://reviews.llvm.org/D10530

llvm-svn: 239994
2015-06-18 05:22:15 +00:00
Rui Ueyama ae36985af7 COFF: Fix entry point inference bug.
Previously, LLD couldn't find a default entry point if it's
defined by a library.

llvm-svn: 239982
2015-06-18 00:40:33 +00:00
Rui Ueyama 24c5fd0419 COFF: Support /manifest{,uac,dependency,file} options.
The linker has to create an XML file for each executable.
This patch supports that feature.

You can optionally embed an XML file to an executable as .rsrc
section. If you choose to do that (by passing /manifest:embed
option), the linker has to create a textual resource file
containing an XML file, compile that using rc.exe to a binary
resource file, conver that resource file to a COFF file using
cvtres.exe, and then link that COFF file. This patch implements
that feature too.

llvm-svn: 239978
2015-06-18 00:12:42 +00:00
Rui Ueyama 151d862d97 COFF: Create import library files.
On Windows, we have to create a .lib file for each .dll.
When linking against DLLs, the linker doesn't use the DLL files,
but instead read a list of dllexported symbols from corresponding
lib files.

A library file containing descriptors of a DLL is called an
import library file.

lib.exe has a feature to create an import library file from a
module-definition file. In this patch, we create a module-definition
file and pass that to lib.exe.

We eventually want to create an import library file by ourselves
to eliminate dependency to lib.exe. For now, we just use the MSVC
tool.

llvm-svn: 239937
2015-06-17 20:40:43 +00:00
Rui Ueyama 39026589b9 COFF: Fix a test which was failing with debug build.
llvm-svn: 239931
2015-06-17 19:28:01 +00:00
Rui Ueyama 1f373704e3 COFF: Support module-definition files.
Module-definition files (.def files) are yet another way to
specify parameters to the linker. You can write a list of dllexported
symbols in module-definition files instead of using /export command
line option. It also supports a few more directives.

The parser code is taken from lib/Driver/WinLinkModuleDef.cpp
with the following modifications.

 - variable names are updated to comply with the LLVM coding style.
 - Instead of returning parsing results as "directive" objects,
   it updates Config object directly.

llvm-svn: 239929
2015-06-17 19:19:25 +00:00
Rui Ueyama 97dff9ee3a COFF: Support creating DLLs.
DLL files are in the same format as executables but they have export tables.
The format of the export table is described in PE/COFF spec section 5.3.

A new class, EdataContents, takes care of creating chunks for export tables.
What we need to do is to parse command line flags for dllexports, and then
instantiate the class to create chunks. For the writer, export table chunks
are opaque data -- it just add chunks to .edata section.

llvm-svn: 239869
2015-06-17 00:16:33 +00:00
Rui Ueyama 1e974577d4 COFF: Fix tests.
I was accidentally testing not -flavor link2 but -flavor link.

llvm-svn: 239868
2015-06-16 23:51:58 +00:00
Rui Ueyama 6592ff8c93 COFF: Add miscellaneous boolean flags.
llvm-svn: 239864
2015-06-16 23:13:00 +00:00
Rui Ueyama bc2cc7d0b8 COFF: Fix .reloc section attributes.
llvm-svn: 239738
2015-06-15 18:03:47 +00:00
Rui Ueyama 59e9578f20 COFF: Fix resource table size.
The size field shouldn't include trailing padding.

llvm-svn: 239712
2015-06-15 01:35:56 +00:00
Rui Ueyama 588e832d0a COFF: Support base relocations.
PE/COFF executables/DLLs usually contain data which is called
base relocations. Base relocations are a list of addresses that
need to be fixed by the loader if load-time relocation is needed.

Base relocations are in .reloc section.

We emit one base relocation entry for each IMAGE_REL_AMD64_ADDR64
relocation.

In order to save disk space, base relocations are grouped by page.
Each group is called a block. A block starts with a 32-bit page
address followed by 16-bit offsets in the page. That is more
efficient representation of addresses than just an array of 32-bit
addresses.

llvm-svn: 239710
2015-06-15 01:23:58 +00:00
Rui Ueyama 2bf6a12238 COFF: Support Windows resource files.
Resource files are data files containing i18n messages, icon images, etc.
MSVC has a tool to convert a resource file to a regular COFF file so that
you can just link that file to embed resources to an executable.

However, you can directly pass resource files to the linker. If you do that,
the linker invokes the tool automatically. This patch implements that feature.

llvm-svn: 239704
2015-06-14 21:50:50 +00:00
Peter Collingbourne 1b6fd1f5fd COFF: Symbol resolution for common and comdat symbols defined in bitcode.
In the case where either a bitcode file and a regular file or two bitcode
files export a common or comdat symbol with the same name, the linker needs
to pick one of them following COFF semantics. This patch implements a design
for resolving such symbols that pushes most of the work onto either LLD's
regular mechanism for resolving common or comdat symbols or the IR linker's
mechanism for doing the same.

We modify SymbolBody::compare to always prefer non-bitcode symbols, so that
during the initial phase of symbol resolution, the symbol table always contains
a regular symbol in any case where we need to choose between a regular and
a bitcode symbol. In SymbolTable::addCombinedLTOObject, we force export
any bitcode symbols that were initially pre-empted by a regular symbol,
and later use SymbolBody::compare to choose between the regular symbol in
the symbol table and the regular symbol from the combined LTO object file.

This design seems to be sound, so long as the resolution mechanism is defined
to be commutative and associative modulo arbitrary choices between symbols
(which seems to be the case for COFF).

Differential Revision: http://reviews.llvm.org/D10329

llvm-svn: 239563
2015-06-11 21:49:54 +00:00
Peter Collingbourne 73b75e3d0c COFF: Handle references from LTO object to lazy symbols correctly.
The code generator may create references to runtime library symbols such as
__chkstk which were not visible via LTOModule. Handle these cases by loading
the object file from the library, but abort if we end up having loaded any
bitcode objects.

Because loading the object file may have introduced new undefined references,
call reportRemainingUndefines again to detect and report them.

Differential Revision: http://reviews.llvm.org/D10332

llvm-svn: 239386
2015-06-09 04:29:54 +00:00
Peter Collingbourne d9e4e98cce COFF: Allow the combined LTO object to define new symbols.
The LLVM code generator can sometimes synthesize symbols, such as SSE
constants, that are not visible via the LTOModule interface. Allow such
symbols so long as they have definitions.

Differential Revision: http://reviews.llvm.org/D10331

llvm-svn: 239385
2015-06-09 02:53:09 +00:00