Commit Graph

694 Commits

Author SHA1 Message Date
Rui Ueyama 55505954fe Add section header stream to PDB.
Differential Revision: https://reviews.llvm.org/D25357

llvm-svn: 283825
2016-10-10 23:44:10 +00:00
Rui Ueyama 1343fac3ca Add an empty DBI stream.
llvm-svn: 283504
2016-10-06 22:52:01 +00:00
Rui Ueyama 8d3fb5da7b Add exact number of streams for reserved stream #s.
llvm-svn: 283397
2016-10-05 22:08:58 +00:00
Rui Ueyama d381c9842b Add an empty IPI stream.
With this, "llvm-pdbdump yaml -ipi-stream" prints out an IPI stream.
Previously it crashed because it can't handle the case where IPI
stream doesn't exist.

llvm-svn: 283392
2016-10-05 21:37:25 +00:00
Mehdi Amini ec4fb5ba97 Use StringRef in StringSaver API (NFC)
llvm-svn: 283290
2016-10-05 01:32:41 +00:00
Rui Ueyama ab8e88b686 Remove unused parameter.
I do not fully understand how to use these classes yet, but
seems like these arguments are not used, since without them
all tests still pass. In order to simplify the situation,
I'll remove them now.

llvm-svn: 283174
2016-10-04 01:03:03 +00:00
Rui Ueyama c79ecdd876 Use StringSwtich::Cases for multiple choices.
llvm-svn: 282964
2016-09-30 22:01:25 +00:00
Rui Ueyama 129795497d Change for LLVM upstream change r282944.
llvm-svn: 282945
2016-09-30 20:53:45 +00:00
Rui Ueyama 5c82eeaf23 Change for LLVM change r282940.
llvm-svn: 282942
2016-09-30 20:39:04 +00:00
Martin Storsjo 63762b58c0 [COFF] Add support for IMAGE_REL_ARM_SECREL
Handle this in the exact same way as IMAGE_REL_AMD64_SECREL
and IMAGE_REL_I386_SECREL.

Differential revision: https://reviews.llvm.org/D24608

llvm-svn: 282531
2016-09-27 19:27:17 +00:00
Rui Ueyama 3e9d6bbad6 Create FileOutputBuffer lazily.
So that it is clear that FileOutputBuffer does not depend on
PDB file builder. Eventually we will have to to get the file size
info from the file builder to create a file with the exact size.
NFC.

llvm-svn: 282454
2016-09-26 23:53:55 +00:00
Rui Ueyama bb542b3f0c Partially fill Info stream with proper values.
llvm-svn: 281795
2016-09-16 22:51:17 +00:00
Ismail Donmez e7174ffcd8 Fix shared library build
llvm-svn: 281728
2016-09-16 14:10:23 +00:00
Rui Ueyama b28c6d495d Use functions in DebugInfoPDB to create dummy PDB file.
I don't think we are creating valid PDB file here,
but it is okay because we have never created valid PDBs before.

llvm-svn: 281704
2016-09-16 04:32:33 +00:00
Rui Ueyama 1d99ab3925 Create PDB.h and move code to remove unnecessary #includes.
llvm-svn: 281670
2016-09-15 22:24:51 +00:00
Rui Ueyama 7f38299919 Use SuperBlock struct defined in MSFCommon.h.
llvm-svn: 281643
2016-09-15 18:55:18 +00:00
Saleem Abdulrasool 88f6407542 COFF: make builds more reproducible
Change the way we calculate the build id to use MD5 to give reproducible build
ids. Previously we would generate random bytes for the build id GUID.

llvm-svn: 281079
2016-09-09 19:26:03 +00:00
Rui Ueyama fa12b8bd1a Remove temoprary files.
Previously, we created temporary files using llvm::sys::fs::createTemporaryFile
and removed them using llvm::FileRemover. This is error-prone as it is easy to
forget creating FileRemover instances after creating temporary files.
There is actually a temporary file leak bug.

This patch introduces a new class, TemporaryFile, to manage temporary files
in the RAII style.

Differential Revision: https://reviews.llvm.org/D24176

llvm-svn: 280510
2016-09-02 17:34:17 +00:00
Ivan Krasin 8baccc2f19 Fix UBSan bot by not passing NULL into memcpy src.
Summary:
UBSan complains like the following:
tools/lld/COFF/Writer.cpp:97:15: runtime error: null pointer passed as argument 2, which is declared to never be null

The reason is that the vector could be empty.

Reviewers: rsmith

Subscribers: Eugene.Zelenko, kcc

Differential Revision: https://reviews.llvm.org/D24050

llvm-svn: 280259
2016-08-31 17:23:05 +00:00
Saleem Abdulrasool dcbf2cbfca COFF: disambiguate make_unique (NFC)
This disambiguates `llvm::make_unqiue` and `std::make_unique` for the Windows
buildbots.

llvm-svn: 280014
2016-08-29 21:33:01 +00:00
Saleem Abdulrasool 8fcff934ba COFF: add beginnings of debug directory creation
The IMAGE_FILE_HEADER structure contains a (RVA, size) to an array of
COFF_DEBUG_DIRECTORY records. Each one of these records contains an RVA to a OMF
Debug Directory. These OMF debug directories are derived into newer types such
as PDB70, PDB20, etc. This constructs a PDB70 structure which will allow us to
associate a GUID with a build to actually tie debug information.

llvm-svn: 280012
2016-08-29 21:20:46 +00:00
Saleem Abdulrasool 9f8bc029bd COFF: hoist a local variable
Create a local variable for the rdata section.  NFC.

llvm-svn: 279407
2016-08-21 23:05:43 +00:00
Saleem Abdulrasool 15f6f6cb97 COFF: reorder the table construction
Reorder the table setup to mirror the indices corresponding to them.  This means
that the table values are filled out as per the enumeration ordering.  Doing so
makes it easier to identify a particular table.  NFC.

llvm-svn: 278199
2016-08-10 04:37:56 +00:00
Saleem Abdulrasool a2cca7e2b4 COFF: handle /debugtype option
Add the support infrastructure for the /debugtype option which takes a comma
delimited list of debug info to generate. The defaults are based on other
options potentially (/driver or /profile). This sets up the infrastructure to
allow us to emit RSDS records to get "build id" equivalents on COFF (similar to
binutils).

llvm-svn: 278056
2016-08-08 22:02:44 +00:00
Benjamin Kramer df8f196f9a Unpollute the global namespace. lld edition.
llvm-svn: 277926
2016-08-06 13:52:37 +00:00
Saleem Abdulrasool fd063e796f COFF ARM: Apply an existing offset in MOV32T relocations
Don't blindly OR in the new value, but clear the existing one, since it can be
nonzero. Read out the existing value before, and add into the desired offset.
(The add is done outside of the applyMOV, to handle potential overflow between
the two.)

Patch by Martin Storsjö!

llvm-svn: 277846
2016-08-05 18:20:31 +00:00
Saleem Abdulrasool eea45bc478 COFF ARM: Error out if 24 bit thumb branches are out of range
In the ELF linker, the same situation already errors out with "relocation
R_ARM_THM_CALL out of range".

Patch by Martin Storsjö!

llvm-svn: 277838
2016-08-05 17:33:24 +00:00
Saleem Abdulrasool 8202c6dbdf COFF ARM: Clear the J1 and J2 bits when applying relocations to 24 bit branches
The opcode for the bl branches can initially be F000 F800, i.e.
the J1 and J2 bits are already set. Therefore mask these bits out
before or'ing in the new bits.

Patch by Martin Storsjö!

llvm-svn: 277836
2016-08-05 17:28:21 +00:00
Kevin Enderby 0fa9fd0f28 Needed change to lld for the changes to libObject/Archive interfaces now returning Expected<>
for the llvm trunk change in r277656

llvm-svn: 277657
2016-08-03 21:58:48 +00:00
Nico Weber 204f7dc56e lld-link: Include the name of bad input files in several "input file is bad" diagnostics.
Also change a few getName() calls to getShortName() calls.
https://reviews.llvm.org/D23123
Part of PR28553.

llvm-svn: 277616
2016-08-03 18:07:28 +00:00
Nico Weber c9d9b31ba7 Revert an unintentional change from r277599
llvm-svn: 277600
2016-08-03 14:38:52 +00:00
Nico Weber 2e36772caf Revert 277594, it caused PR28827
llvm-svn: 277599
2016-08-03 14:37:57 +00:00
Peter Collingbourne feee2103c6 COFF: Implement /linkrepro flag.
This flag is implemented similarly to --reproduce in the ELF linker.

This patch implements /linkrepro by moving the cpio writer and associated
utility functions to lldCore, and using that implementation in both linkers.

One COFF-specific detail is that we store the object file from which the
resource files were created in our reproducer, rather than the resource
files themselves. This allows the reproducer to be used on non-Windows
systems for example.

Differential Revision: https://reviews.llvm.org/D22418

llvm-svn: 276719
2016-07-26 02:00:42 +00:00
Rui Ueyama bb57954241 COFF: Update error messages so that they start with lowercase letters.
llvm-svn: 275513
2016-07-15 01:12:24 +00:00
Rui Ueyama 5694ebbc8b Remove unnecessary explicit call of Twine ctor.
llvm-svn: 275512
2016-07-15 01:06:40 +00:00
Rui Ueyama 659a4f2f38 Make check() always return a value.
Previously, one of two check functions didn't return a value.
It was confusing. This patch makes both functions return values.

llvm-svn: 275511
2016-07-15 01:06:38 +00:00
Rui Ueyama 0d09a865c6 COFF: Remove `void error()` functions and use fatal instead.
This change makes the control flow more explicit.

llvm-svn: 275504
2016-07-15 00:40:46 +00:00
Rui Ueyama 1a3fd13776 COFF: Remove unnecessary explicit calls of Twine ctor.
llvm-svn: 275501
2016-07-14 23:43:36 +00:00
Rui Ueyama 606047939c COFF: Rename noreturn error -> fatal.
This new name is also consistent with ELF.

llvm-svn: 275500
2016-07-14 23:37:14 +00:00
Rui Ueyama 7e75866c51 COFF: Rename non-noreturn error -> check.
The new name is consistent with ELF.

llvm-svn: 275499
2016-07-14 23:37:10 +00:00
Peter Collingbourne 765aa2d1c2 COFF: Update remaining #include paths.
llvm-svn: 275480
2016-07-14 21:30:37 +00:00
Lang Hames 622ef17f5d [lld] Update LLD for Archive::child_iterator change in LLVM r275361.
llvm-svn: 275362
2016-07-14 02:35:18 +00:00
Saleem Abdulrasool f27d4068a5 COFF: drop the dependency on LIB.EXE for implibs
lld currently relies on lib.exe in order to generate an empty import library.
The "empty" import library consists of 5 members:
  - first linker member
  - second linker member
  - Import Descriptor
  - NULL Import Descriptor
  - NULl Thunk

The first two entries (first and second linker members) are string tables which
are never updated.  Therefore, they may as well as not be present.  A subsequent
change to add that is probably warranted.  However, this does not prevent the
use of the linker.

The Import Descriptor is the content which is most important.  It provides an
Import Name Table entry for the library (as specified by the LIBRARY directive
in the DEF file).  Additionally, it contains undefined references to the NULL
Import Descriptor and the library NULL Thunk Data.  This ensures that the linker
will pull in the subsequent objects from the import library for the link.  The
Import Descriptor has a single symbol (__IMPORT_DESCRIPTOR_<Library>) which
contains 3 relocations, one to the INT (Import Name Table) entry, one to the ILT
(Import Lookup Table) entry, and one to the IAT (Import Address Table) entry.

The NULL Import Descriptor is the last import descriptor and terminates the
import descriptor array.  It contains a single symbol
(__NULL_IMPORT_DESCRIPTOR).

The NULL Thunk contains a single symbol (\x7f<Library>_NULL_THUNK_DATA) and
provides the terminator for the ILT and IAT.

These files are currently constructed manually following the example of the
Short Import Library format.  This is arguably less than ideal, and it may be
possible to use MCAssembler and feed it the fragments to construct the object.

The major difference between the LIB (LINK) generated objects and the ones
generated here is that they are all one section shorter (.debug$S) as they do
not contain the debug information and one symbol shorter (@comp.id) as they do
not contain the RICH signature.

Move the logic related to the librarian into a new source file (Librarian.cpp).

llvm-svn: 275242
2016-07-13 03:19:27 +00:00
Saleem Abdulrasool 0561bd5b47 COFF: remove unused function (touchFile)
Remove some dead code.  NFC.

llvm-svn: 274900
2016-07-08 18:36:56 +00:00
Peter Collingbourne 66ec178c6c Fix logic error in check() function.
llvm-svn: 274195
2016-06-30 00:32:24 +00:00
Peter Collingbourne eadba86c3e COFF: Switch to new archive writer interface (D21721).
Differential Revision: http://reviews.llvm.org/D21722

llvm-svn: 274184
2016-06-29 22:27:45 +00:00
Kevin Enderby b9e053cfd7 Matching change for lld for the llvm change of Archive::create() from ErrorOr<...> to Expected<...> in r274160.
llvm-svn: 274161
2016-06-29 20:36:11 +00:00
Rui Ueyama 440138c4c8 [COFF] Add /section command line flag.
llvm-svn: 273134
2016-06-20 03:39:39 +00:00
Benjamin Kramer bd521201b7 Apply clang-tidy's misc-move-constructor-init to lld.
No functionality change intended.

llvm-svn: 271686
2016-06-03 16:57:13 +00:00
Rafael Espindola a5cefffc33 Update for llvm change.
llvm-svn: 270907
2016-05-26 20:31:06 +00:00
Eugene Zelenko 1a299eab32 Fix Clang-tidy misc-unused-using-decls and Include What You Use warnings.
Differential revision: http://reviews.llvm.org/D19348

llvm-svn: 267008
2016-04-21 17:14:10 +00:00
Nico Weber a7a848f10d Revert unintentionally commited bits in r266935.
llvm-svn: 266942
2016-04-21 01:31:37 +00:00
Nico Weber 848334704d unbreak COFF/out.test after r266929
llvm-svn: 266935
2016-04-20 23:30:14 +00:00
Nico Weber 5660de7877 lld-link: Fix default output name with /dll flag.
If /dll is passed, the default output filename should be foo.dll,
not foo.exe.
http://reviews.llvm.org/D19321

llvm-svn: 266929
2016-04-20 22:34:15 +00:00
Duncan P. N. Exon Smith 28d8d04657 LTO: Adapt to LLVM API changes in r266713
llvm-svn: 266714
2016-04-19 04:57:40 +00:00
Rui Ueyama afb1901e42 COFF: Support /manifestinput command line option.
Manifest file is a separate or embedded XML file having metadata
of an executable. As it is XML, it can contain various types of
information. Probably the most popular one is to request escalated
priviledges.

Usually the linker creates an XML file and embed that file into
an executable. However, there's a way to supply an XML file from
command line. /manifestniput is it.

Apparently it is over-designed here, but if you supply two or more
manifest files, then the linker needs to merge the files into a
single XML file. A good news is that we don't need to do that ourselves.
MT.exe command can do that, so we call the command from the linker
in this patch.

llvm-svn: 266704
2016-04-19 01:21:58 +00:00
Duncan P. N. Exon Smith 2c66b23d79 LTO: Merge debug info types when linking bitcode
Make sure lld enables ODR type uniquing for debug info when invoking
LTO, as a follow-up to LLVM r266549.

llvm-svn: 266555
2016-04-17 07:35:38 +00:00
Rui Ueyama 34b60f1c40 Do not use llvm::getGlobalContext().
llvm-svn: 266375
2016-04-14 21:41:44 +00:00
Mehdi Amini 03a5042e4c Revert "Do not use llvm::getGlobalContext(), trying to nuke it from LLVM"
This reverts commit r266365 and r266367, the contexts in the two
places have to match.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266373
2016-04-14 21:31:46 +00:00
Mehdi Amini 9a15293ec1 Use fully qualified name to refer to LLVMContext
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266367
2016-04-14 21:09:10 +00:00
Mehdi Amini f1e7ff9d0e Do not use llvm::getGlobalContext(), trying to nuke it from LLVM
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266365
2016-04-14 20:53:39 +00:00
Davide Italiano 3774afba07 [COFF] Simplify the code leveraging implicit conversion.
Suggested by: David Blaikie!

llvm-svn: 266140
2016-04-12 22:05:38 +00:00
Davide Italiano eeae124faf [COFF] SmallVector<char, 0> -> SmallString<0>.
This way we're consistent between ELF and COFF.

llvm-svn: 265885
2016-04-09 23:00:31 +00:00
Kevin Enderby 439453679c Needed change to lld for the change to createBinary() now returning Expected<...>
With the llvm change in r265606 this is the matching needed change to the lld
code now that createBinary() is returning Expected<...> .

llvm-svn: 265607
2016-04-06 22:15:23 +00:00
Reid Kleckner 9cd77ce169 [coff] Accept and ignore another link.exe flag for compatibility
This flag disables link.exe's crash handler so that normal windows error
reporting and crash dumping occurs. For now it is reasonable for LLD to
ignore the flag.

Chromium is currently using this flag to collect minidumps of link.exe
crashing, and it breaks the LLD build.

llvm-svn: 264439
2016-03-25 18:09:29 +00:00
David Majnemer 08af4e56e1 [COFF] Don't call memcpy with a NULL argument
Some declarations of memcpy (like glibc's for example) are attributed
with notnull which makes it UB for NULL to get passed in, even if the
memcpy count is zero.

To account for this, guard the memcpy with an appropriate precondition.

This should fix the last UBSan bug, exposed by the test suite, in the
COFF linker.

llvm-svn: 263919
2016-03-20 23:10:12 +00:00
David Majnemer 2aa29ee7c1 [COFF] Remove undefined behavior from ObjectFile::createWeakExternal
LLD type-punned an integral type and a pointer type using a pointer
field.  This is problematic because the pointer type has alignment
greater than some of the integral values.

This would be less problematic if a union was used but it turns out the
integral values are only present for a short, transient, amount of time.

Let's remove this undefined behavior by skipping the punning altogether
by storing the state in a separate memory location: a vector which
informs us which symbols to process for weak externs.

llvm-svn: 263918
2016-03-20 22:56:31 +00:00
David Majnemer 93bbc7cd66 [COFF] Use coff_section::getAlignment
Use LLVM's section alignment calculation instead of having LLD calculate
it.

llvm-svn: 263724
2016-03-17 16:58:08 +00:00
David Majnemer 272de1e4d1 [COFF] Don't trust a symbol's section number
This fixes a test which exposed an ASan issue.

We assumed that a symbol's section number had a corresponding section
without performing validation.

llvm-svn: 263558
2016-03-15 16:47:28 +00:00
David Majnemer 22dff0aafc [COFF] Don't hard-code the load configuration size
The load configuration directory is a structure whose size varies as the
OS gains additional functionality.  To account for this, the structure's
layout begins with a size field; this allows loaders to know which
fields are available.

However, LLD hard-coded the sizes (112 bytes for 64-bit and 64 for
32-bit).  This means that we might not inform the loader of all the
pertinent fields or we might claim that there are more fields than are
actually present.

To correctly account for this, the size field must be loaded from the
_load_config_used symbol.

N.B.  The COFF spec is either wrong or out of date, the load
configuration directory is not correctly documented in the
specification: it omits the size field.

llvm-svn: 263543
2016-03-15 09:48:27 +00:00
David Majnemer 1706719972 [COFF] Remove an unused function, getFileOff
The function was not used and was not functional: all paths would lead
to report_fatal_error or endless stack recursion.

llvm-svn: 263542
2016-03-15 09:48:18 +00:00
David Majnemer 22a19a9ace [COFF] Use the correct size of the TLS directory
The TLS directory has a different layout depending on the bitness of the
machine the image will run on.  LLD would always use the 64-bit TLS
directory for the data directory entry instead of an appropriately sized
TLS directory.

llvm-svn: 263539
2016-03-15 06:41:02 +00:00
Rui Ueyama 02aa766897 Update the documents of the new LLD.
This patch merges the documents for ELF and COFF into one
and puts it into docs directory.

llvm-svn: 263336
2016-03-12 06:06:40 +00:00
Rui Ueyama a453c0a5ad Merge DarwinLdDriver and Driver.
Now that DarwinLdDriver is the only derived class of Driver.
This patch merges them and actually removed the class because
they can now just be non-member functions. This change simplifies
a common header, Driver.h.

http://reviews.llvm.org/D17788

llvm-svn: 262502
2016-03-02 19:08:05 +00:00
Rafael Espindola 393877d51b Fix BUILD_SHARED_LIBS build.
llvm-svn: 262345
2016-03-01 15:56:53 +00:00
Rui Ueyama 417553d393 Make the entry point function calls consistent. NFC.
llvm-svn: 262191
2016-02-28 19:54:51 +00:00
Rui Ueyama 29d8eef440 Rename so that the function name is consistent between ELF and COFF.
llvm-svn: 261914
2016-02-25 18:49:11 +00:00
Rui Ueyama 489a806965 Update for LLVM function name change.
llvm-svn: 257801
2016-01-14 20:53:50 +00:00
Rui Ueyama 84425d7289 COFF: Implement DLL symbol forwarding.
DLL export tables usually contain dllexport'ed symbol RVAs so that
applications which use the DLLs can find symbols from the DLLs.
However, there's a minor feature to "forward" DLL symbols to other
DLLs.

If you set an RVA to a string whose form is "<dllname>.<symbolname>"
(e.g. "KERNEL32.ExitProcess") instead of symbol RVA to the export
table, the loader interprets that as a forwarder symbol, and resolve
that symbol from the specified DLL.

This patch implements that feature.

llvm-svn: 257243
2016-01-09 01:22:00 +00:00
Rui Ueyama dba6b576cf COFF: Rename RoundUpToAlignment -> align.
llvm-svn: 257220
2016-01-08 22:24:26 +00:00
Pete Cooper f154b678c6 Set the folder for libraries to 'lld libraries'. NFC.
In a UI such as XCode, LLVM source files are in 'libraries' while clang
files are in 'clang libraries'.

This change moves the lld source to 'lld libraries' to make code browsing easier.

It should be NFC as the build itself is still the same, just the structure in a
UI differs.

llvm-svn: 257001
2016-01-07 00:14:04 +00:00
Rui Ueyama 1763c0d3bc COFF: Create an empty but valid PDF file.
MSVC linker considers PDB files created with this patch valid.
So you don't have to remove PDB files created by lld before
running MSVC linker.

This patch has no test since llvm-pdbdump dislikes PDB files
with no metadata streams.

llvm-svn: 255039
2015-12-08 18:39:55 +00:00
Rui Ueyama e737824d8a COFF: Create a PDB file with the correct file signature.
Before this patch, we created an empty PDB file if /debug option is
specified. For MSVC linker, such PDB file is completely broken, and
linker exits without doing anything as soon as it finds an empty PDB
file.

A PDB file created in this patch has the correct file signature.
MSVC linker still thinks that the file is broken, but it then removes
and replaces with its output.

This is an initial patch to support PDB in LLD. We aim to support
PDB in order to make it 100% compatible with MSVC linker. PDB support
is the last missing piece.

llvm-svn: 254796
2015-12-04 23:11:05 +00:00
Rafael Espindola 3d49d7aa22 Update for llvm change.
llvm-svn: 254725
2015-12-04 16:22:09 +00:00
Rafael Espindola 35dad13714 Update for LLVM api change.
llvm-svn: 254697
2015-12-04 02:42:47 +00:00
Rui Ueyama 43e12900d9 COFF: Non-external COMDAT sections sholud not be merged by ICF.
If a section symbol is not external, that COMDAT section should never
be merge with other sections in other compilation unit. Previously,
we didn't take visibility into account.

Note that COMDAT sections with non-external visibility makes sense
because they can be removed by dead-stripping.

Fixes https://llvm.org/bugs/show_bug.cgi?id=25686

llvm-svn: 254578
2015-12-03 02:23:33 +00:00
Rui Ueyama df06d203e5 Update documents.
llvm-svn: 253728
2015-11-20 22:47:42 +00:00
Peter Collingbourne 70507e59fe COFF: Destroy LTOModules as they are linked.
This should help reduce memory consumption during LTO.

Differential Revision: http://reviews.llvm.org/D14672

llvm-svn: 253397
2015-11-17 23:30:59 +00:00
Yunzhong Gao fbee89263f Fixing build failures caused by r253367. Sorry for breaking the build.
llvm-svn: 253374
2015-11-17 20:48:12 +00:00
Rafael Espindola 7bcaec83be Fix a few windows only tests.
This argument must be non-null.

llvm-svn: 252336
2015-11-06 19:57:37 +00:00
Kevin Enderby 35dfc95efe These are the matching changes needed to the lld project for the changes to llvm
in r252192 that changed the Archive and Child interfaces in libObject. These include
Rafael Espindola’s many suggested updates.

llvm-svn: 252193
2015-11-05 19:25:47 +00:00
Rafael Espindola 543f29d1a9 Don't implicitly construct an Archive::child_iterator.
llvm-svn: 252166
2015-11-05 14:34:56 +00:00
Rui Ueyama df985afa14 COFF: De-parallelize ICF for now.
There was a threading issue in the ICF code for COFF. That seems like
a venign bug in the sense that it doesn't produce an incorrect output,
but it oftentimes misses reducible sections. As a result, mergeable
sections could remain in outputs, which makes the output nondeterministic.

Basically the algorithm we are using for ICF is this: We group sections
so that identical sections will eventually be in the same group. Initially,
all sections are in one group. We split the group by relocation targets
until we get a convergence (if relocation targets are in different gruops,
the sections are different). Once a group is split, they will never be
merged.

Each section has a group ID. That variable itself is atomic, so there's
no threading issue at the level that we can use thread sanitizer.
The point is, when we split a group, we re-assign new group IDs to group
of sections. That are multiple separate writes to atomic varaibles.
Thus, splitting a group is not an atomic operation, and there's a small
chance that the other thread observes inconsistent group IDs.

Over-splitting is always "safe", so it will never create incorrect output.

I suspect that the nondeterminism stems from that point. However, I
cannot prove or fix that at this moment, so I'm going to avoid using
threads here.

llvm-svn: 251300
2015-10-26 16:20:00 +00:00
Craig Topper f88d22970d Update lld to match llvm r250901. OptTable constructor now takes an ArrayRef. NFC
llvm-svn: 250904
2015-10-21 16:31:56 +00:00
Rui Ueyama 75656eebb8 COFF: /OPT should accept comma-separated multiple arguments.
/OPT:foo,bar is equivalent to /OPT:foo /OPT:bar.

Reported by alexchandel. https://llvm.org/bugs/show_bug.cgi?id=25228

llvm-svn: 250728
2015-10-19 19:40:43 +00:00
Rui Ueyama 43155d0d48 [LLD] Fix Clang-tidy modernize-use-nullptr warnings; other minor cleanups.
Patch from Eugene Zelenko!

llvm-svn: 249111
2015-10-02 00:36:00 +00:00
Rui Ueyama 548d22c073 COFF: ICF should not merge sectinos if their alignments are not the same.
There's actually a room to improve this patch. Instead of not merging
sections that have different alignements, we can choose the section that
has the largest alignment requirement among all sections that are otherwise
considered the same. Then all section alignments are satisfied, so we can
merge them.

I don't know if that improvement could make any difference for real-world
input, so I'll leave it alone. Would be interesting to revisit later.

llvm-svn: 248581
2015-09-25 16:50:12 +00:00
Rui Ueyama c9e746b9e6 COFF: Fix local varaible type.
This is intended to be 64-bit integer, but size_t is not guranteed
to be the same or larger type than uint64_t.

llvm-svn: 248580
2015-09-25 16:38:13 +00:00
Rui Ueyama de88072a00 COFF: Rename Ptr -> Repl.
This pointer points to a replacement for this chunk. Ptr was not a good name.

llvm-svn: 248579
2015-09-25 16:20:24 +00:00
Rui Ueyama c28a08b8d2 COFF: Remove duplicate parameter from hash value calculation.
llvm-svn: 248526
2015-09-24 19:00:42 +00:00
Rui Ueyama 9640173e16 COFF: Add /nosymtab command line option.
This is an LLD extension to MSVC link.exe command line. MSVC linker
does not write symbol tables for executables. We do unless no /debug
option is given.

There's a situation that we want to enable debug info but don't want
to emit the symbol table. One example is when we are comparing output
file size. With this patch, you can tell the linker to not create
a symbol table by just specifying /nosymtab.

llvm-svn: 248225
2015-09-21 23:43:31 +00:00
Rui Ueyama 97d92736f5 COFF: Improve section hash value.
std::distance(C->Relocs.end(), C->Relocs.begin()) is the same as NumRelocs
which is already added to the hash value. What we are missing here is the
section size.

llvm-svn: 248202
2015-09-21 19:41:38 +00:00
Rui Ueyama 3cb1f5c860 COFF: Rename A.replaceWith(B) -> B.replace(A). NFC.
llvm-svn: 248197
2015-09-21 19:36:51 +00:00
Rui Ueyama 98a98cffb6 COFF: Do not call std::async with std::launch::async if multithreading is disabled.
llvm-svn: 248193
2015-09-21 19:12:36 +00:00
Rui Ueyama 5f38915624 COFF: Fix ICF regression.
This patch fixes a regression introduced by r247964. Relocations that
are referring the same symbol should be considered equal, but they
were not if they were pointing to non-section chunks.

llvm-svn: 248132
2015-09-20 20:19:12 +00:00
Rui Ueyama 997b357ac1 COFF: Run InputFile::parse() in background using std::async().
Previously, InputFile::parse() was run in batch. We construct a list
of all input files and call parse() on each file using parallel_for_each.
That means we cannot start parsing files until we get a complete list
of input files, although InputFile::parse() is safe to call from anywhere.

This patch makes it asynchronous. As soon as we add a file to the symbol
table, we now start parsing the file using std::async().

This change shortens self-hosting time (650 ms) by 28 ms. It's about 4%
improvement.

llvm-svn: 248109
2015-09-20 03:11:16 +00:00
Rui Ueyama f49712a853 COFF: Fix race condition.
NextID is updated inside parallel_for_each, so it needs mutual exclusion.

llvm-svn: 248106
2015-09-20 01:44:44 +00:00
Rui Ueyama 3cfd2bff1e Remove dead code.
llvm-svn: 248105
2015-09-20 01:19:36 +00:00
Rui Ueyama 1cce300843 COFF: Change Symbol::Body type from atomic pointer to regular pointer.
I made the field an atomic pointer in hope that we would be able to
parallelize the symbol resolver soon, but that's not going to happen
soon. This patch reverts that change for the sake of readability.

llvm-svn: 248104
2015-09-20 00:00:05 +00:00
Rui Ueyama 63bbe84b27 COFF: Make Chunk::writeTo() const. NFC.
This should improve code readability especially because this function
is called inside parallel_for_each.

llvm-svn: 248103
2015-09-19 23:28:57 +00:00
Rui Ueyama ebb0ebff4b COFF: Fix thread-safety bug.
LTOModule doesn't seem to be thread-safe, so guard that with mutex.

llvm-svn: 248102
2015-09-19 23:14:51 +00:00
Rui Ueyama a5f0f758d3 COFF: Move markLive() from Writer.cpp to its own file.
Conceptually, garbage collection is not part of Writer,
so move the function out of the file.

llvm-svn: 248099
2015-09-19 21:36:28 +00:00
Rui Ueyama 0652c59506 COFF: Actually parallelize InputFile::parse().
This is a follow-up patch to r248078.

llvm-svn: 248098
2015-09-19 21:33:26 +00:00
Rui Ueyama 27e9e6540c Remove unused #includes.
llvm-svn: 248081
2015-09-19 02:28:32 +00:00
Rui Ueyama f4d05d7a80 COFF: Parallelize InputFile::parse().
InputFile::parse() can be called in parallel with other calls of
the same function. By doing that, time to self-link improves from
741 ms to 654 ms or 12% faster.

This is probably the last low hanging fruit in terms of parallelism.
Input file parsing and symbol table insertion takes 450 ms in total.
If we want to optimize further, we probably have to parallelize
symbol table insertion using concurrent hashmap or something.
That's doable, but that's not easy, especially if you want to keep
the exact same semantics and linking order. I'm not going to do that
at least soon.

Anyway, compared to r248019 (the change before the first attempt for
parallelism), we achieved 36% performance improvement from 1022 ms
to 654 ms. MSVC linker takes 3.3 seconds to link the same program.
MSVC's ICF feature is very slow for some reason, but even if we
disable the feature, it still takes about 1.2 seconds.
Our number is probably good enough.

llvm-svn: 248078
2015-09-19 01:48:26 +00:00
Rui Ueyama 8197a4e0bf COFF: Use parallel_sort in Writer::sortExceptionTable().
This patch saves 4 ms out of 5 ms. Very small improvement,
but maybe better than nothing.

llvm-svn: 248063
2015-09-18 23:17:34 +00:00
Rui Ueyama 49e72e69e5 Fix build error that std::atomic is not copy-constructible.
llvm-svn: 248061
2015-09-18 22:58:12 +00:00
Rui Ueyama e629a45531 COFF: Address review comments.
- Fix race condition of `Redo`
- Avoid std::distance

llvm-svn: 248058
2015-09-18 22:31:15 +00:00
Rui Ueyama e0e0796d83 COFF: Parallelize Writer::writeSections().
Self-hosting took 801 ms on my machine. Of which this function took
69 ms. Now it takes 37 ms. That is about 4% overall performance
improvement.

llvm-svn: 248052
2015-09-18 22:07:10 +00:00
Rui Ueyama e8d1c59756 Style fix to make it look consistent. NFC.
llvm-svn: 248044
2015-09-18 21:17:44 +00:00
Rui Ueyama aa95e5a4cc COFF: Parallelize ICF.
The LLD's ICF algorithm is highly parallelizable. This patch does that
using parallel_for_each.

ICF accounted for about one third of total execution time. Previously,
it took 324 ms when self-hosting. Now it takes only 62 ms.

Of course your mileage may vary. My machine is a beefy 24-core Xeon machine,
so you may not see this much speedup. But this optimization should be
effective even for 2-core machine, since I saw speedup (324 ms -> 189 ms)
when setting parallelism parameter to 2.

llvm-svn: 248038
2015-09-18 21:06:34 +00:00
Rui Ueyama 603d51104b COFF: Reorder comparisons.
This change makes equalsConstant a bit faster (193ms -> 163ms).

llvm-svn: 247965
2015-09-18 02:40:54 +00:00
Rui Ueyama 8c73dfb6bf COFF: Remove useless micro-optimization.
This patch simplifies code by removing micro-optimization that doesn't
contribute to speed.

llvm-svn: 247964
2015-09-18 02:15:34 +00:00
Rui Ueyama c9a6e827bd COFF: Optimize ICF by not creating temporary vectors.
Previously, ICF created a vector for each SectionChunk. The vector
contained pointers to successors, which are namely associative sections
and COMDAT relocation targets. The reason I created vectors is because
I thought that that would make section comparison faster.

It did make the comparison faster. When self-linking, for example, it
saved about 10 ms on each iteration. The time we spent on constructing
the vectors was 124 ms. If we iterate more than 12 times, return from
the investment exceeds the initial cost.

In reality, it usually needs 5 iterations. So we shouldn't construct
the vectors.

llvm-svn: 247963
2015-09-18 01:51:37 +00:00
Rui Ueyama 7d8263bf1d COFF: Optimize ICF by comparing relocations before section contents.
equalsConstants() is the heaviest function in ICF, and that consumes
more than half of total ICF execution time. Of which, section content
comparison accounts for roughly one third.

Previously, we compared section contents at the beginning of the
function after comparing their checksums. The comparison is very
likely to succeed because when the control reaches that comparison,
their checksums are always equal. And because checksums are 64-bit
CRC, they are unlikely to collide.

We compared relocations and associative sections after that.
If they are different, the time we spent on byte-by-byte comparison
of section contents were wasted.

This patch moves the comparison at the end of function. If the
comparison fails, the time we spent on relocation comparison are
wasted, but as I wrote it's very unlikely to happen.

LLD took 1198 ms to link itself to produce a 27.11 MB executable.
Of which, ICF accounted for 536 ms. This patch cuts it by 90 ms,
which is 17% speedup of ICF and 7.5% speedup overall. All numbers
are median of ten runs.

llvm-svn: 247961
2015-09-18 01:30:56 +00:00
Rui Ueyama 4151972c22 Enable extra LTO verification only when build type is debug.
llvm-svn: 247956
2015-09-17 22:54:08 +00:00
Rui Ueyama 63dd8766ab COFF: Remove DefinedSymbol::isLive() and markLive(). NFC.
Basically the concept of "liveness" is for sections (or chunks in LLD
terminology) and not for symbols. Symbols are always available or live,
or otherwise it indicates a link failure.

Previously, we had isLive() and markLive() methods for DefinedSymbol.
They are confusing methods. What they actually did is to act as a proxy
to backing section chunks. We can simplify eliminate these methods
and call section chunk's methods directly.

llvm-svn: 247869
2015-09-16 23:55:52 +00:00
Rui Ueyama 66c06ceaca COFF: ICF: Print out the number of iterations. NFC.
llvm-svn: 247868
2015-09-16 23:55:39 +00:00
Rui Ueyama 4dbff20c91 COFF: Fix bug that not all symbols were written to symtab if /opt:noref.
Only live symbols are written to the symbol table. Because isLive()
returned false if dead-stripping was disabled entirely, only
non-COMDAT sections were written to the symbol table. This patch fixes
the issue.

llvm-svn: 247856
2015-09-16 21:40:47 +00:00
Rui Ueyama 3b153e6541 COFF: Fix bug that /opt:noicf was ignored.
llvm-svn: 247854
2015-09-16 21:30:55 +00:00
Rui Ueyama 4bce7bcc88 COFF: Output messages for /verbose to stdout instead of stderr.
This patch also makes the message less verbose.

llvm-svn: 247853
2015-09-16 21:30:40 +00:00
Rui Ueyama b02a320f5e COFF: Enable ICF by default.
MSVC linker enables ICF as long as /opt:ref is eanbled, so do we.

llvm-svn: 247817
2015-09-16 16:41:38 +00:00
Rui Ueyama c04d5dbf20 COFF: Rename /opt:lldicf -> /opt:icf.
Now that ICF is complete, we can rename this option so that
the driver accepts the MSVC-compatible command line option.

llvm-svn: 247816
2015-09-16 16:33:57 +00:00
Rui Ueyama 92298d5418 COFF: Create ICF class to move code from SectionChunk to ICF. NFC.
This patch defines ICF class and defines ICF-related functions as
members of the class. By doing this we can move code that are
related only to ICF from SectionChunk to the newly-defined class.
This also eliminates a global variable "NextID".

llvm-svn: 247802
2015-09-16 14:19:10 +00:00
Rui Ueyama 9cb2870ce0 ICF: Improve ICF to reduce more sections than before.
This is a patch to make LLD to be on par with MSVC in terms of ICF
effectiveness. MSVC produces a 27.14MB executable when linking LLD.
LLD previously produced a 27.61MB when self-linking. Now the size
is reduced to 27.11MB. Note that without ICF the size is 29.63MB.

In r247387, I implemented an algorithm that handles section graphs
as cyclic graphs and merge them using SCC. The algorithm did not
always work as intended as I demonstrated in r247721. The new
algortihm implemented in this patch is different from the previous
one. If you are interested the details, you want to read the file
comment of ICF.cpp.

llvm-svn: 247770
2015-09-16 03:26:31 +00:00
Duncan P. N. Exon Smith a11f81973a LTO: Adjust to LLVM r247735
Perhaps lld wants to disable the verifier sometimes during COFF LTO, but
for now just match behaviour from before r247735.

llvm-svn: 247736
2015-09-15 23:06:16 +00:00
Rui Ueyama c48f78ca5a Fix typo.
llvm-svn: 247645
2015-09-15 00:35:41 +00:00
Rui Ueyama 13563d8645 Fix style.
llvm-svn: 247644
2015-09-15 00:33:11 +00:00
Rui Ueyama 8e186df2f5 COFF: Corrected error message if a section failed to load.
There is no sense to use Name in these lines as it is not initialized yet.

Patch from Igor Kudrin!

llvm-svn: 247531
2015-09-13 20:22:22 +00:00
Rui Ueyama 5b93aa51de COFF: Teach ICF to merge cyclic graphs.
Previously, LLD's ICF couldn't merge cyclic graphs. That was unfortunate
because, in COFF, cyclic graphs are not exceptional at all. That is
pretty common.

In this patch, sections are grouped by Tarjan's strongly connected
component algorithm to get acyclic graphs. And then we try to merge
SCCs whose outdegree is zero, and remove them from the graph. This
makes other SCCs to have outdegree zero, so we can repeat the
process until all SCCs are removed. When comparing two SCCs, we handle
cycles properly.

This algorithm works better than previous one. Previously, self-linking
produced a 29.0MB executable. It now produces a 27.7MB. There's still some
gap compared to MSVC linker which produces a 27.1MB executable for the
same input. So the gap is narrowed, but still LLD is not on par with MSVC.
I'll investigate that later.

llvm-svn: 247387
2015-09-11 04:29:03 +00:00
Rui Ueyama 3c28ba38de COFF: Split doICF(). No functionality change.
llvm-svn: 246934
2015-09-05 23:06:32 +00:00
Rui Ueyama ef907ec82d COFF: Implement a better algorithm for ICF.
Identical COMDAT Folding is a feature to merge COMDAT sections
by contents. Two sections are considered the same if their contents,
relocations, attributes, etc, are all the same.

An interesting fact is that MSVC linker takes "iterations" parameter
for ICF because the algorithm they are using is iterative. Merging
two sections could make more sections to be mergeable because
different relocations could now point to the same section. ICF is
repeated until we get a convergence (until no section can be merged).
This algorithm is not fast. Usually it needs three iterations until a
convergence is obtained.

In the new algorithm implemented in this patch, we consider sections
and relocations as a directed acyclic graph, and we try to merge
sections whose outdegree is zero. Sections with outdegree zero are then
removed from the graph, which makes  other sections to have outdegree
zero. We repeat that until all sections are processed. In this
algorithm, we don't iterate over the same sections many times.

There's an apparent issue in the algorithm -- the section graph is
not guaranteed to be acyclic. It's actually pretty often cyclic.
So this algorithm cannot eliminate all possible duplicates.
That's OK for now because the previous algorithm was not able to
eliminate cycles too. I'll address the issue in a follow-up patch.

llvm-svn: 246878
2015-09-04 21:35:54 +00:00
Rui Ueyama 434de7a33f Remove unused variable.
llvm-svn: 246874
2015-09-04 21:05:30 +00:00
Rui Ueyama 2dcc23580e COFF: Use section content checksum for ICF.
Previously, we calculated our own hash values for section contents.
Of coruse that's slow because we had to access all bytes in sections.
Fortunately, COFF objects usually contain hash values for COMDAT
sections. We can use that to speed up Identical COMDAT Folding.

llvm-svn: 246869
2015-09-04 20:45:50 +00:00
Rui Ueyama 31e66e32b4 COFF: Ignore /GUARDSYM option.
The option is added in MSVC 2015, and there's no documentation about
what the option is. This patch is to ignore the option for now, so that
at least LLD is usable with MSVC 2015.

llvm-svn: 246780
2015-09-03 16:20:47 +00:00
Rui Ueyama 6295b27184 COFF: /delayload:<DLLNAME> is case-insensitive.
llvm-svn: 246770
2015-09-03 14:49:47 +00:00
Rafael Espindola fb497d79f6 Remove an allocator which was used for just one allocation.
llvm-svn: 246662
2015-09-02 16:07:11 +00:00
Rui Ueyama bfbd277a1c COFF: Preserve original spelling of DLL file name.
This patch fixes a subtle incompatibility with MSVC linker.
MSVC linker preserves the original spelling of a DLL in the
import descriptor table. LLD previously converted all
characters to lowercase. Usually this difference is benign,
but if a program explicitly checks for DLL file names, the
program could fail.

llvm-svn: 246620
2015-09-02 07:27:31 +00:00
Davide Italiano 491d3bfd43 [COFF] Remove dead private field. Also fixes build with -Werror.
llvm-svn: 246553
2015-09-01 16:05:53 +00:00
Rui Ueyama 100ffacf2c COFF: .exe files should be able to export functions.
In r246424, I made a change that disables non-DLL to export
symbols. It turned out that the change was not correct. Both
DLLs and executables are able to export symbols (although the
latter is relatively rare). This change restores the feature.

llvm-svn: 246537
2015-09-01 09:15:58 +00:00
Rui Ueyama dfff2542b8 COFF: Make import libraries compatible with MSVC link.
I have totally no idea why, but MSVC linker is sensitive about
file names of archive members. If we do not make import library
file names to the same as the DLL name, MSVC link *crashes*
when it is processing the library file. This patch is to set
the same name.

llvm-svn: 246535
2015-09-01 08:08:57 +00:00
Rui Ueyama 84f9218417 COFF: Set "Data" bit for data symbols in the import descriptor.
llvm-svn: 246533
2015-09-01 06:46:10 +00:00
Rui Ueyama f10a32014d COFF: Improve dllexported name mangling compatibility.
The rules for dllexported symbols are overly complicated due to
x86 name decoration, fuzzy symbol resolution, and the fact that
one symbol can be resolved by so many different names. The rules
are probably intended to be "intuitive", so that users don't have
to understand the name mangling schemes, but it seems that it can
lead to unintended symbol exports.

To make it clear what I'm trying to do with this patch, let me
write how the export rules are subtle and complicated.

 - x86 name decoration: If machine type is i386 and export name
   is given by a command line option, like /export:foo, the
   real symbol name the linker has to search for is _foo because
   all symbols are decorated with "_" prefixes. This doesn't happen
   on non-x86 machines. This automatic name decoration happens only
   when the name is not C++ mangled.

   However, the symbol name exported from DLLs are ones without "_"
   on all platforms.

   Moreover, if the option is given via .drectve section, no
   symbol decoration is done (the reason being that the .drectve
   section is created by a compiler and the compiler should always
   know the exact name of the symbol, I guess).

 - Fuzzy symbol resolution: In addition to x86 name decoration,
   the linker has to look for cdecl or C++ mangled symbols
   for a given /export. For example, it searches for not only
   _foo but also _foo@<number> or ??foo@... for /export:foo.

Previous implementation didn't get it right. I'm trying to make
it as compatible with MSVC linker as possible with this patch
however the rules are. The new code looks a bit messy to me, but
I don't think it can be simpler due to the ad-hoc-ness of the rules.

llvm-svn: 246424
2015-08-31 08:43:21 +00:00
Peter Collingbourne df5783b7a5 COFF: Implement parallel LTO code generation.
This is exposed via a new flag /opt:lldltojobs=N, where N is the number of
code generation threads.

Differential Revision: http://reviews.llvm.org/D12309

llvm-svn: 246342
2015-08-28 22:16:09 +00:00
Rui Ueyama 40f4d86f5f COFF: Create short import files instead of using lib.exe.
lib.exe has a feature to create import library files (which contain
short import files) from module-definition files. Previously, we were
using that feature, but it turned out that the feature is not complete
for us.

There seems no way to specify "Import Types" in module-definition file.
lib.exe always adds "_" to given symbols and specify IMPORT_NAME_UNDECORATE.
We need more fine-grainded control on that value.

This patch teaches LLD to create short import files itself.
We are still using lib.exe, but the use of the tool is limited to create
empty import library files. We then create short import files and add them
to the empty files as new members.

This patch does not intend to change the functionality. LLD produces
the same import libraries as before. I'll make another change to create
different import libraries in a follow-up patch.

llvm-svn: 246292
2015-08-28 10:52:05 +00:00
Rui Ueyama c0c74e1b8a COFF: Print out module-definition files if /verbose is given.
This is useful for testing.

llvm-svn: 246032
2015-08-26 12:37:54 +00:00
Rui Ueyama 2caf407107 COFF: Show real command line options if /verbose is given.
llvm-svn: 246023
2015-08-26 07:12:08 +00:00
Peter Collingbourne 0bb50d92b6 COFF: Update for LTO API change.
llvm-svn: 245892
2015-08-24 22:22:58 +00:00
Rui Ueyama 6a883b086e COFF: Improve debug helper function.
SectionChunk::getDebugName crashed if the symbol was a nullptr.

llvm-svn: 245677
2015-08-21 07:01:10 +00:00
Rui Ueyama d2d2360222 COFF: Fix /lldmap option.
isLive returns false if it's not COMDAT, so check for that condition.

llvm-svn: 245676
2015-08-21 07:01:08 +00:00
Rui Ueyama 04ec69aa9e COFF: Use ErrorOr::operator* instead of ErrorOr::get.
This patch is to make COFF coding style consistent with ELF. NFC.

llvm-svn: 245282
2015-08-18 09:18:15 +00:00
Rui Ueyama 570752c7ac Do not use unique pointers. NFC.
These unique pointers have the exact same lifetime as automatic
variables, so use automatic variables instead.

llvm-svn: 245281
2015-08-18 09:13:25 +00:00
Rui Ueyama 95b781d863 COFF: Do not handle __NULL_IMPORT_DESCRIPTOR differently than the other symbols.
__NULL_IMPORT_DESCRIPTOR is a symbol used by MSVC liner to construct
the import descriptor table. We do not use the symbol. Previously,
we had code to skip that symbol. That code does not actually do
anything meaningful because no one is referencing the symbol, the
symbol would naturally be ignored. This patch stops recognizing
the symbol.

llvm-svn: 245280
2015-08-18 09:06:41 +00:00
Rui Ueyama 7171c82194 COFF: Allow forward reference for weak externals
Previously, weak external symbols could reference only symbols that
appeared before them. Although that covers almost all use cases
of weak externals, there are object files out there which contains
weak externals that have forward references.

This patch supports such weak externals.

llvm-svn: 245258
2015-08-17 23:35:43 +00:00
Rui Ueyama d157088adb COFF: Fix the order of the DLL import entry.
There are some DLLs whose initializers depends on other DLLs'
initializers. The initialization order matters for them.

MSVC linker uses the order of the libraries from the command line.
LLD used ASCII-betical order. So they were incompatible.
This patch makes LLD compatible with MSVC.

llvm-svn: 245201
2015-08-17 08:30:31 +00:00
Rui Ueyama e73e418bb4 COFF: Simplify Writer::createImportTables.
A short import library has up to two symbols, so we don't have
to do a for-loop and type dispatch in createImportTables.

llvm-svn: 245200
2015-08-17 07:27:45 +00:00
Rafael Espindola beee25e484 Make these headers as being c++.
llvm-svn: 245050
2015-08-14 14:12:54 +00:00
Peter Collingbourne 526ff15546 COFF: Introduce flag /opt:lldlto=N for controlling LTO optimization level.
Differential Revision: http://reviews.llvm.org/D12024

llvm-svn: 245027
2015-08-14 04:47:07 +00:00
Rafael Espindola 5c546a1437 COFF: In chunks, store the offset from the start of the output section. NFC.
This is more convenient than the offset from the start of the file as we
don't have to worry about it changing when we move the output section.

This is a port of r245008 from ELF.

llvm-svn: 245018
2015-08-14 03:30:59 +00:00
Rafael Espindola f0461ba985 Update for llvm api change.
llvm-svn: 244856
2015-08-13 01:07:08 +00:00
Rafael Espindola bdc8f2fb83 Update for llvm api change.
llvm-svn: 244849
2015-08-13 00:31:46 +00:00
Rui Ueyama fa071e13aa COFF: Align sections to 512-byte boundaries on disk.
Sections must start at page boundaries in memory, but they
can be aligned to sector boundaries (512-bytes) on disk.
We aligned them to 4096-byte boundaries even on disk, so we
wasted disk space a bit.

llvm-svn: 244691
2015-08-11 23:09:00 +00:00
Rui Ueyama 3c4737db54 COFF: Ignore /editandcontinue option.
llvm-svn: 244626
2015-08-11 16:46:08 +00:00
Rui Ueyama bc9891f01d COFF: Update README.
It's no longer "new" because the old COFF linker has been removed.
Update the introduction accordingly.

llvm-svn: 244565
2015-08-11 03:44:51 +00:00
Rui Ueyama 857b30356f Move file-local classes to an anonymous namespace. NFC.
llvm-svn: 244525
2015-08-10 23:02:57 +00:00
Rui Ueyama 107db55ac4 COFF: Define symbols for MSVC 2015 Control Flow Protection.
MSVC 2015's load configuration object (__load_config_used) contains
references to these symbols. I don't fully understand how it works,
but looks like these symbols are linker-defined ones. So I define them
here in the Driver. With this patch, LLD can self-host with MSVC 2015.

This patch is to link MSVC 2015-produced object files. It does not
implement Control Flow Protection. If I understand correctly, the
linker has to create a bitmap of function entry point addresses for
the CFG runtime. We don't do that yet. Produced executables will not
be protected by CFG.

llvm-svn: 244425
2015-08-09 21:01:06 +00:00
Rui Ueyama 27e470abae COFF: Do not fall through if /lib is processed.
llvm-svn: 244424
2015-08-09 20:45:17 +00:00
Rui Ueyama 234afc4a0e Remove unused `using`.
llvm-svn: 244422
2015-08-09 20:38:58 +00:00
Rui Ueyama 611add25e3 COFF: Simplify.
SymbolTable::find(mangle(X)) is equivalent to SymbolTable::findUnderscore(X)
except that the latter is slightly efficient as that doesn't allocate a new
string.

llvm-svn: 244377
2015-08-08 00:23:37 +00:00
Rui Ueyama 8ebdc8cedc COFF: Handle _load_config_used in the same way as other special symbols.
Handling the symbol this way is consistent with other symbols, such as
_tls_used. NFC.

llvm-svn: 244367
2015-08-07 22:43:53 +00:00
Rui Ueyama 237c8ff796 Remove unused variable.
llvm-svn: 244365
2015-08-07 22:40:13 +00:00
Rui Ueyama 1574b8ecfb COFF: Update a comment.
llvm-svn: 244258
2015-08-06 19:57:21 +00:00
Rafael Espindola b835ae8e4a Port the error functions from ELF to COFF.
This has a few advantages

* Less C++ code (about 300 lines less).
* Less machine code (about 14 KB of text on a linux x86_64 build).
* It is more debugger friendly. Just set a breakpoint on the exit function and
  you get the complete lld stack trace of when the error was found.
* It is a more robust API. The errors are handled early and we don't get a
  std::error_code hot potato being passed around.
* In most cases the error function in a better position to print diagnostics
  (it has more context).

llvm-svn: 244215
2015-08-06 14:58:50 +00:00
Rui Ueyama 9b000812a4 COFF: ARM: Sort .pdata section correctly.
On ARM, exception handler entries in .pdata section are 8 byte long.

llvm-svn: 244191
2015-08-06 03:45:27 +00:00
Rui Ueyama cb8474edae COFF, ELF2: Pass output file path implicitly using Config global variable.
Various parameters are passed implicitly using Config global variable
already. Output file path is no different from others, so there was no
special reason to handle that differnetly.

This patch changes the signature of writeResult(SymbolTable *, StringRef)
to writeResult(SymbolTable *).

llvm-svn: 244180
2015-08-05 23:51:50 +00:00
Rui Ueyama 685c41cd39 COFF: Simplify Writer interface by hiding Writer class.
llvm-svn: 244175
2015-08-05 23:43:53 +00:00
Rafael Espindola 4280a96468 Handle writeImportLibrary failing.
We were printing an error but exiting with 0.

Not sure how to test this. We could add a no-winlib feature,
but that is probably not worth it.

llvm-svn: 244109
2015-08-05 20:03:57 +00:00
Rui Ueyama 67fcd1a0c7 COFF: Fix bad #includes.
Writer.h is intended to be included only by Writer.cpp and Driver.cpp.
Use of the header in other files are bad.

llvm-svn: 244106
2015-08-05 19:51:28 +00:00
Rui Ueyama ba7c041f21 COFF: ARM: Implepment BLX23T relocation and fix Branch20T.
I fed the same test to MSVC linker and got the same output,
so I believe this implementation is correct.

llvm-svn: 244102
2015-08-05 19:40:07 +00:00
Rui Ueyama 18edc83b6c COFF: Fix error message. Space was missing.
llvm-svn: 243794
2015-07-31 21:51:25 +00:00
Reid Kleckner 981576daf7 Add some help strings for /dll and /debug so they show up in /?
llvm-svn: 243757
2015-07-31 16:40:38 +00:00
Peter Collingbourne e7107ec03b COFF: When resolving _load_config_used, add it as a GC root.
This fixes the cases where the symbol is defined in a comdat or by bitcode.

Differential Revision: http://reviews.llvm.org/D11673

llvm-svn: 243735
2015-07-31 05:33:34 +00:00
Rui Ueyama 7b276e2fb4 COFF: Move code for Identical COMDAT Folding to ICF.cpp.
llvm-svn: 243701
2015-07-30 22:57:21 +00:00
Rui Ueyama f69ecc1212 COFF: Handle all COMDAT sections as non-GC root.
I don't remember why I thought that only functions are subject
of garbage collection, but the comment here said so, which is
not correct. Moreover, the code just below the comment does not
do what the comment says -- it handles non-COMDAT, non-function
sections as GC root. As a result, it just handles non-COMDAT
sections as GC root.

This patch cleans that up by removing SectionChunk::isRoot and
use isCOMDAT instead.

llvm-svn: 243700
2015-07-30 22:48:45 +00:00
David Majnemer 13ac40ea6e COFF: Sort output sections which start with .debug to the end of the file
We want to convince the NT loader not to map these sections into memory.
A good first step is to move them to the end of the executable.

Differential Revision: http://reviews.llvm.org/D11655

llvm-svn: 243680
2015-07-30 20:26:55 +00:00
Rui Ueyama e0d68e381b Remove unused #includes.
llvm-svn: 243588
2015-07-29 22:53:29 +00:00
Rui Ueyama 966acb2e2f COFF: Suppress "Duplicate entry" warning of lib.exe
We create a module-definition file and give that to lib.exe to
create an import library file. A module-definition has to be
syntactically and semantically correct, of course.

There was a case that we created a module-definition file that
lib.exe would complain for duplicate entries. If a user gives
an unmangled and mangled name for the same symbol, we would end
up having two duplicate lines for the mangled name in a module-
definition file.

This patch fixes that issue by uniquefying entries by mangled
symbol name.

llvm-svn: 243587
2015-07-29 22:38:27 +00:00
Rui Ueyama 4323831732 COFF: Fix command line option spelling.
llvm-svn: 243573
2015-07-29 21:01:15 +00:00
Rui Ueyama 46682630f4 COFF: Ignore /ThrowNew command line option.
This command line option is added since MSVC 2015. Our wild guess is
that the flag is for LTCG and we can safely ignore that.

llvm-svn: 243568
2015-07-29 20:29:15 +00:00
Rui Ueyama ff88d5a26f COFF: Add /safeseh command line option.
If /safeseh is specified, all input files must be compatible with Safe SEH.

llvm-svn: 243565
2015-07-29 20:25:40 +00:00
Rui Ueyama 8bc43a142b COFF: ARM: Fix relocations to thumb code.
Windows ARM is the thumb ARM environment, and pointers to thumb code
needs to have its LSB set. When we apply relocations, we need to
adjust the LSB if it points to an executable section.

llvm-svn: 243560
2015-07-29 19:25:00 +00:00
Rui Ueyama 56965f4b32 COFF: ARM: Fix DLL import table.
The previous test was testing -flavor link. This patch correctly
tests link2 and fixes a bug that we didn't emit import thunks.

llvm-svn: 243559
2015-07-29 19:24:58 +00:00
Rui Ueyama eb26e1d03c COFF: Fix SECREL and SECTION relocations.
SECREL should sets the 32-bit offset of the target from the beginning
of *target's* output section. Previously, the offset from the beginning
of source's output section was used instead.

SECTION means the target section's index, and not the source section's
index. This patch fixes that issue too.

llvm-svn: 243535
2015-07-29 16:30:45 +00:00
Rui Ueyama ca577fd536 COFF: Remove unused command line option.
llvm-svn: 243533
2015-07-29 16:30:34 +00:00
Rui Ueyama 29f74c312a COFF: Set load config table entry on non-x86.
llvm-svn: 243532
2015-07-29 16:30:31 +00:00
Rui Ueyama 506f6d1ae1 COFF: _tls_used is __tls_used on x86.
llvm-svn: 243495
2015-07-28 22:56:02 +00:00
Rui Ueyama 9420dee328 COFF: Fix export symbol names for x86.
I don't fully understand the rationale behind the name mangling
scheme used for the DLL export table and the import library.
Why only leading "_" is dropped for the import library while
both "_" and "@" are dropped from DLL symbol table? But this seems
to be what MSVC linker does.

llvm-svn: 243490
2015-07-28 22:34:24 +00:00
Rui Ueyama 635e85a647 COFF: Update README to mention that it now supports 32-bit x86.
The linker is now able to link not only LLVM/Clang/LLD for x86 but
even larger programs. I confirmed that it successsfully linked Chrome
for x86. Because the browser is a pretty large program, I think I can
say that the linker is now mostly feature complete. (I'm pretty sure
that there are hidden bugs somewhere, but they shouldn't be significant.)

llvm-svn: 243377
2015-07-28 03:40:58 +00:00
Rui Ueyama cfb874a3d4 COFF: Do not ignore /merge if /debug is specified.
Previously, we ignore /merge option if /debug is specified
because I thought that was MSVC linker did. This was wrong.
/merge shouldn't be ignored even in debug mode.

llvm-svn: 243375
2015-07-28 03:24:23 +00:00
Rui Ueyama d68e211be5 COFF: /HighEntropyVA is on by default only on 64-bit.
llvm-svn: 243374
2015-07-28 03:15:57 +00:00
Rui Ueyama 4d54534627 COFF: Add /LargeAddressAware command line option.
llvm-svn: 243373
2015-07-28 03:12:00 +00:00
Rui Ueyama 7e387a68de COFF: Fix 32-bit delay-import address table.
The address table entry is 32-bit wide on 32-bit and 64-bit on 64-bit.
Previously, it was 64-bit even on 32-bit.

llvm-svn: 243372
2015-07-28 02:54:18 +00:00
Rui Ueyama 2832ce95a5 COFF: Skip non-DWARF debug info sections.
Leaving them in an executable is basically harmless but wastes disk space.
Because no one is using non-DWARF debug info linked by LLD, we can just
remove them.

llvm-svn: 243364
2015-07-28 01:06:58 +00:00
Rui Ueyama e44524de65 Use SmallDenseMap instead of std::map where we don't care about order of keys.
llvm-svn: 243358
2015-07-28 00:17:25 +00:00
Rui Ueyama a8eed749a2 COFF: Write import library symbols to a symbol table.
Previously no __imp_ symbols nor dllimport thunk functions were
written to a symbol table.

llvm-svn: 243350
2015-07-27 23:40:20 +00:00
Rui Ueyama 5e706b3ee3 COFF: Use short identifiers. NFC.
llvm-svn: 243229
2015-07-25 21:54:50 +00:00
Rui Ueyama 5c437cd1e9 COFF: Fix image base address for 32-bit.
0x140000000 or 0x180000000 are not correct image base addresses
for 32-bit. They are actually much smaller.

llvm-svn: 243228
2015-07-25 21:42:33 +00:00
Rui Ueyama 3dd9372d2b COFF: ARM: Support import functions.
llvm-svn: 243205
2015-07-25 03:39:29 +00:00
Rui Ueyama cde7a7907e COFF: ARM: Implement BLX23T relocation.
llvm-svn: 243204
2015-07-25 03:25:28 +00:00
Rui Ueyama 3d9c8639c3 COFF: ARM: Implement BRANCH24T relocation.
llvm-svn: 243202
2015-07-25 03:19:34 +00:00
Rui Ueyama 237fca1451 COFF: ARM: Implement MOV32T relocation.
llvm-svn: 243201
2015-07-25 03:03:46 +00:00
Rui Ueyama a265b01353 COFF: ARM: Set correct entry point address.
llvm-svn: 243199
2015-07-25 02:25:14 +00:00
Rui Ueyama 3afd5bfd7b COFF: Handle base relocation as a tuple of relocation type and RVA. NFC.
On x64 and x86, we use only one base relocation type, so we handled
base relocations just as a list of RVAs. That doesn't work well for
ARM becuase we have to handle two types of base relocations on ARM.
This patch changes the type of base relocation from uint32_t to
{reltype, uint32_t} to make it easy to port this code to ARM.

llvm-svn: 243197
2015-07-25 01:44:32 +00:00
Rui Ueyama 28df04211c COFF: Split ImportThunkChunk into x86 and x64. NFC.
This change should make it easy to port this code to ARM.

llvm-svn: 243195
2015-07-25 01:16:06 +00:00
Rui Ueyama 1c341a54de COFF: Do not align import thunks on 16-byte boundaries on x86.
Looks like MSVC linker aligns them only on x64.

llvm-svn: 243194
2015-07-25 01:16:04 +00:00
Rui Ueyama 35ccb0f7d4 COFF: Don't assume !is64() means i386.
In many places we assumed that is64() means AMD64 and i386 otherwise.
This assumption is not sound because Windows also supports ARM.
The linker doesn't support ARM yet, but this is a first step.

llvm-svn: 243188
2015-07-25 00:20:06 +00:00
Rui Ueyama cd3f99b6c5 COFF: Implement Safe SEH support for x86.
An object file compatible with Safe SEH contains a .sxdata section.
The section contains a list of symbol table indices, each of which
is an exception handler function. A safe SEH-enabled executable
contains a list of exception handler RVAs. So, what the linker has
to do to support Safe SEH is basically to read the .sxdata section,
interpret the contents as a list of symbol indices, unique-fy and
sort their RVAs, and then emit that list to .rdata. This patch
implements that feature.

llvm-svn: 243182
2015-07-24 23:51:14 +00:00
Rui Ueyama 2296dc137c COFF: Fix base relocation type for x86.
llvm-svn: 243178
2015-07-24 23:24:45 +00:00
Rui Ueyama 3cb895c930 COFF: Fix __ImageBase symbol relocation.
__ImageBase is a special symbol whose value is the image base address.
Previously, we handled __ImageBase symbol as an absolute symbol.

Absolute symbols point to specific locations in memory and the locations
never change even if an image is base-relocated. That means that we
don't have base relocation entries for absolute symbols.

This is not a case for __ImageBase. If an image is base-relocated, its
base address changes, and __ImageBase needs to be shifted as well.
So we have to have base relocations for __ImageBase. That means that
__ImageBase is not really an absolute symbol but a different kind of
symbol.

In this patch, I introduced a new type of symbol -- DefinedRelative.
DefinedRelative is similar to DefinedAbsolute, but it has not a VA but RVA
and is a subject of base relocation. Currently only __ImageBase is of
the new symbol type.

llvm-svn: 243176
2015-07-24 22:58:44 +00:00
Rui Ueyama afad42f9ea COFF: Set Load Configuration entry in Data Directory.
Load Configuration field points to a structure containing information
for SEH. That data strucutre is not created by the linker but provided
by an external file. What we have to do is just to set __load_config_used
address to the header.

llvm-svn: 242427
2015-07-16 18:30:35 +00:00
Rui Ueyama 759c8aa9a0 COFF: Fix offset in x86 delay-load thunks.
llvm-svn: 242353
2015-07-15 23:01:36 +00:00
Rui Ueyama ef0e647581 COFF: Implement x86 delay-load thunks.
llvm-svn: 242343
2015-07-15 22:26:57 +00:00
Rui Ueyama 8765fbae15 COFF: Fix mangled dllexported names.
If a symbol is exported as /export:foo, and foo is resolved as a
mangled name (_foo@<number> or ?foo@@Y...), that mangled name should
be written to the export table. Previously, we wrote the original
name to the export table.

llvm-svn: 242342
2015-07-15 22:21:08 +00:00
Rui Ueyama 33fb2cb11b COFF: Fix base relocations for __imp_ symbols on x86.
Because thunks for dllimported symbols contain absolute addresses on x86,
they need to be relocated at load-time. This bug was a cause of crashes
in DLL initialization routines.

llvm-svn: 242259
2015-07-15 00:25:38 +00:00
Rafael Espindola 9f141efc57 Use getChildOffset instead of getBuffer for identifying a member.
I am adding support for thin archives. On those, getting the buffer
involves reading another file.

Since we only need an id in here, use the member offset in the archive.

llvm-svn: 242205
2015-07-14 21:28:07 +00:00
Rui Ueyama 159fb4cd84 Revert "Make COFF linker work when it's built by clang again."
This reverts commit r242006. The original issue in Clang was fixed in r242009,
so we can now safely use std::atomic_flag.

llvm-svn: 242112
2015-07-14 03:09:59 +00:00
Rui Ueyama a50387f1b3 COFF: Fix entry name inference for x86.
Entry name selection rule is already complicated on x64, but it's more
complicated on x86 because of the underscore name mangling scheme.

If one of _main, _main@<number> (a C function) or ?main@@... (a C++ function)
is defined, entry name is _mainCRTStartup. If _wmain, _wmain@<number or
?wmain@@... is defined, entry name is _wmainCRTStartup. And so on.

llvm-svn: 242110
2015-07-14 02:58:13 +00:00
Rui Ueyama 6d24908fe7 COFF: Fix x86 delay-load helper function name.
If /delayload option is given, we have to resolve __delayLoadHelper2
since the function is the dynamic loader to delay-load DLLs.
The function name is mangled in x86 as ___delayLoadHelper2@8.

llvm-svn: 242078
2015-07-13 22:31:45 +00:00
Rui Ueyama cb71c72ccc COFF: Inline Defined::getRVA because it's very hot.
llvm-svn: 242075
2015-07-13 22:01:27 +00:00
Rui Ueyama e59a530a6c COFF: Split createSymbolAndSymbolTable to small functions. NFC.
llvm-svn: 242066
2015-07-13 20:56:31 +00:00
Nico Weber 9262da26d0 Make COFF linker work when it's built by clang again.
clang-cl doesn't compile std::atomic_flag correctly (PR24101).  Since the COFF
linker doesn't use threads yet, just revert r241420 and r241481 for now to
work around this clang-cl bug.

llvm-svn: 242006
2015-07-13 00:55:26 +00:00
Rui Ueyama c851ccc3bd COFF: Fix locally-imported symbol's base relocations.
Base relocations are RVA and not VA, so we shouldn't add ImageBase.

llvm-svn: 241883
2015-07-10 04:30:54 +00:00
Rui Ueyama 270960f5cd COFF: Find C++ mangled name for symbols starting with underscore.
Symbol foo is mangled as _foo in C and ?foo@@... in C++ on x86.
findMangle has to remove prefix underscore before mangle a given name
as a C++ symbol.

llvm-svn: 241874
2015-07-09 23:03:51 +00:00
Rui Ueyama bbdec4fc82 COFF: Fix dllexported symbol names on x86.
Symbol names are usually mangled by appending "_" prefix on x86.
But the mangled name is not used in DLL export table. The export
table contains unmangled names.

llvm-svn: 241872
2015-07-09 22:51:41 +00:00
Rui Ueyama d4b351f0de COFF: Fix locally-imported symbol's size for x86.
llvm-svn: 241860
2015-07-09 21:15:58 +00:00
Rui Ueyama 93b4571187 COFF: Implement base relocations for x86.
With this patch, LLD is now able to self-link an .exe file for x86
that runs correctly, although I don't think some headers (particularly
SEH) are not correct. DLL support is coming soon.

llvm-svn: 241857
2015-07-09 20:36:59 +00:00
Rui Ueyama a841bb0f5d COFF: Fix import symbol name mangling.
For IMPORT_NAME_NOPREFIX symbols, we should remove only one prefix character.

llvm-svn: 241854
2015-07-09 20:22:41 +00:00
Rui Ueyama 39d9efb772 COFF: Fix command line options for external commands.
llvm-svn: 241853
2015-07-09 20:22:39 +00:00
Rui Ueyama ea533cde30 COFF: Infer machine type earlier than before.
Previously, we infer machine type at the very end of linking after
all symbols are resolved. That's actually too late because machine
type affects how we mangle symbols (whether or not we need to
add "_").

For example, /entry:foo adds "_foo" to the symbol table if x86 but
"foo" if x64.

This patch moves the code to infer machine type, so that machine
type is inferred based on input files given via the command line
(but not based on .directives files).

llvm-svn: 241843
2015-07-09 19:54:13 +00:00
Rui Ueyama 57aa69ee97 COFF: Make /machine:{i386,amd64} aliases to {x86,x64}.
MSVC linker accepts these aliases.

llvm-svn: 241840
2015-07-09 19:43:49 +00:00