Commit Graph

673 Commits

Author SHA1 Message Date
Rui Ueyama 67aea737f3 Use check() to report an error.
llvm-svn: 305033
2017-06-08 23:43:44 +00:00
Sam Clegg c1e7a0dde5 Add BinaryFormat to lld libraries
Without this, when building with shared BUILD_SHARED_LIBS=ON
I get errors such as:

lib/Core/Reader.cpp:40: error: undefined reference to
'llvm::identify_magic(llvm::StringRef)'

Differential Revision: https://reviews.llvm.org/D34004

llvm-svn: 304932
2017-06-07 18:06:11 +00:00
Zachary Turner 264b5d9e88 Move Object format code to lib/BinaryFormat.
This creates a new library called BinaryFormat that has all of
the headers from llvm/Support containing structure and layout
definitions for various types of binary formats like dwarf, coff,
elf, etc as well as the code for identifying a file from its
magic.

Differential Revision: https://reviews.llvm.org/D33843

llvm-svn: 304864
2017-06-07 03:48:56 +00:00
Reid Kleckner c2eaccb0af Revert "Merge IAT and ILT."
This reverts r303374. It breaks Chrome's IAT patching code:
http://crbug.com/729077

llvm-svn: 304584
2017-06-02 18:49:38 +00:00
Reid Kleckner 146eb7a65f Re-land "COFF: migrate def parser from LLD to LLVM"
This reverts commit r304561 and re-lands r303490 & co.

The fix was to use "SymbolName" when translating LLD's internal export
list to lib/Object's short export struct. The SymbolName reflects the
actual symbol name, which may include fastcall and stdcall mangling bits
not included in the /EXPORT or .def file EXPORTS name:

@@ -434,8 +434,7 @@ std::vector<COFFShortExport> createCOFFShortExportFromConfig() {
   std::vector<COFFShortExport> Exports;
   for (Export &E1 : Config->Exports) {
     COFFShortExport E2;
-    E2.Name = E1.Name;
+    // Use SymbolName, which will have any stdcall or fastcall qualifiers.
+    E2.Name = E1.SymbolName;
     E2.ExtName = E1.ExtName;
     E2.Ordinal = E1.Ordinal;
     E2.Noname = E1.Noname;

llvm-svn: 304573
2017-06-02 17:53:06 +00:00
Reid Kleckner d249e4a188 Revert "COFF: migrate def parser from LLD to LLVM"
This reverts commits r303490, r303491, r303493, and r303494.

This caused http://crbug.com/728726. Essentially, exporting stdcall
functions doesn't appear to work after this change. Reduced test case
soon.

llvm-svn: 304561
2017-06-02 16:26:24 +00:00
Zachary Turner ebd3ae8371 [CodeView] Properly align symbol records on read/write.
Object files have symbol records not aligned to any particular
boundary (e.g. 1-byte aligned), while PDB files have symbol
records padded to 4-byte aligned boundaries.  Since they share
the same reading / writing code, we have to provide an option to
specify the alignment and propagate it up to the producer or
consumer who knows what the alignment is supposed to be for the
given container type.

Added a test for this by modifying the existing PDB -> YAML -> PDB
round-tripping code to round trip symbol records as well as types.

Differential Revision: https://reviews.llvm.org/D33785

llvm-svn: 304484
2017-06-01 21:52:41 +00:00
Zachary Turner 2897e0306e [lld] Fix a bug where we continually re-follow type servers.
Originally this was intended to be set up so that when linking
a PDB which refers to a type server, it would only visit the
PDB once, and on subsequent visitations it would just skip it
since all the records had already been added.

Due to some C++ scoping issues, this was not occurring and it
was revisiting the type server every time, which caused every
record to end up being thrown away on all subsequent visitations.

This doesn't affect the performance of linking clang-cl generated
object files because we don't use type servers, but when linking
object files and libraries generated with /Zi via MSVC, this means
only 1 object file has to be linked instead of N object files, so
the speedup is quite large.

llvm-svn: 303920
2017-05-25 21:16:03 +00:00
Rui Ueyama 236e781011 Use MD5::hash(). NFC.
llvm-svn: 303893
2017-05-25 18:17:43 +00:00
Rui Ueyama 69ae29b1d1 Do not allow delay-importing data symbols.
If you pass /delayload:<dllname> to the COFF linker, it creates thunks
so that DLLs are loaded when they are used for the first time instead of
load-time.

This mechanism do not work for data symbols as there's no way to trap
acccesses to data imported from DLLs. (Technically, I think if we do not
initially map dllimport tables in memory, we could actually trap accesses
and delay-load data symbols, but that's not what Windows do.)

This patch is to report an error when you try to delay-load data symbols.

Fixes https://bugs.llvm.org/show_bug.cgi?id=33106

Differential Revision: https://reviews.llvm.org/D33557

llvm-svn: 303890
2017-05-25 18:03:34 +00:00
Rui Ueyama 0e8521c05a Reduce indentation. NFC.
llvm-svn: 303815
2017-05-24 22:36:11 +00:00
Rui Ueyama 9aa82f76ac Garbage collect dllimported symbols.
This is a different implementation than r303225 (which was reverted
in r303270, re-submitted in r303304 and then re-reverted in r303527).

In the previous patch, I tried to add Live bit to each dllimported
symbol. It turned out that it didn't work with "oldnames.lib" which
contains a lot of weak aliases to dllimported symbols.

The way we handle weak aliases is to check if undefined symbols
can be resolved using weak aliases, and if so, memcpy the Defined
symbols to weak Undefined symbols, so that any references to weak
aliases automatically see defined symbols instead of undefined ones.

This memcpy happens before MarkLive kicks in.

That means we may have multiple copies of dllimported symbols. So
turning on one instance's Live bit is not enough.

This patch moves the Live bit to dllimport file. Since multiple
copies of dllsymbols still point to the same file, we can use it as the
central repository to keep track of liveness.

Differential Revision: https://reviews.llvm.org/D33520

llvm-svn: 303814
2017-05-24 22:30:06 +00:00
Rui Ueyama 96cbf8bca6 Fix the sanitizer-windows bot.
Looks like r303801 broke the sanitizer-windows bot. I don't fully
understand what is going on, so I'll partially revert that patch.

llvm-svn: 303805
2017-05-24 20:32:23 +00:00
Rui Ueyama 27abe98cfa Close the gap between ELF and COFF ICF implementations. NFC.
We originally wrote the ICF code for COFF and ported it to ELF.
They started diverging since then. This patch closes the gap.

llvm-svn: 303801
2017-05-24 19:56:29 +00:00
Rui Ueyama f04c04837c Improve parallelism of ICF.
This is the only place we use threads for ICF. The intention of this code
was to split an input vector into 256 shards and process them in parallel.
What the code was actually doing was to split an input into 257 shards,
process the first 256 shards in parallel, and the remaining one in serial.

That means this code takes ceil(256/n)+1 instead of ceil(256/n) where n
is the number of available CPU cores. The former converges to 2 while
the latter converges to 1.

This patches fixes the above issue.

llvm-svn: 303797
2017-05-24 19:22:34 +00:00
Rui Ueyama 8df7fa782e Rewrite badly-formatted switch. NFC.
llvm-svn: 303777
2017-05-24 17:12:53 +00:00
Rui Ueyama 216d6834ba Make key-value types explicit. NFC.
llvm-svn: 303776
2017-05-24 17:12:32 +00:00
Rui Ueyama d6489635f4 Change the control flow so that the function is a bit more readable. NFC.
llvm-svn: 303775
2017-05-24 17:12:10 +00:00
Zachary Turner bb64231d2d Don't do a full scan of the type stream before processing records.
LazyRandomTypeCollection is designed for random access, and in
order to provide this it lazily indexes ranges of types.  In the
case of types from an object file, there is no partial index
to build off of, so it has to index the full stream up front.
However, merging types only requires sequential access, and when
that is needed, this extra work is simply wasted.  Changing the
algorithm to work on sequential arrays of types rather than
random access type collections eliminates this up front scan.

llvm-svn: 303707
2017-05-24 00:26:27 +00:00
Zachary Turner d4136e945e Implement various flavors of type merging.
Previous algotirhm assumed that types and ids are in a single
unified stream.  For inputs that come from object files, this
is the case.  But if the input is already a PDB, or is the result
of a previous merge, then the types and ids will already have
been split up, in which case we need an algorithm that can
accept operate on independent streams of types and ids that
refer across stream boundaries to each other.

Differential Revision: https://reviews.llvm.org/D33417

llvm-svn: 303577
2017-05-22 21:07:43 +00:00
Rui Ueyama b6632d9cd1 Revert r303304: Re-submit r303225: Garbage collect dllimported symbols.
This reverts commit r303304 because it looks like the change
introduced a crash bug. At least after that change, LLD with thinlto
crashes when linking Chromium.

llvm-svn: 303527
2017-05-22 06:01:37 +00:00
Martell Malone 1e39e5e964 COFF: migrate def parser from LLD to LLVM [2/2]
This is split up into two commits.
This commit removes the DEF parser from LLD
See the previous commit for the creation in LLVM.

Reviewers: ruiu

Differential Revision: https://reviews.llvm.org/D32689

llvm-svn: 303491
2017-05-20 19:56:44 +00:00
Zachary Turner 526f4f2aa8 Resubmit "[CodeView] Provide a common interface for type collections."
This was originally reverted because it was a breaking a bunch
of bots and the breakage was not surfacing on Windows.  After much
head-scratching this was ultimately traced back to a bug in the
lit test runner related to its pipe handling.  Now that the bug
in lit is fixed, Windows correctly reports these test failures,
and as such I have finally (hopefully) fixed all of them in this
patch.

llvm-svn: 303446
2017-05-19 19:26:58 +00:00
Zachary Turner 1dfcf8d92c Revert "[CodeView] Provide a common interface for type collections."
This is a squash of ~5 reverts of, well, pretty much everything
I did today.  Something is seriously broken with lit on Windows
right now, and as a result assertions that fire in tests are
triggering failures.  I've been breaking non-Windows bots all
day which has seriously confused me because all my tests have
been passing, and after running lit with -a to view the output
even on successful runs, I find out that the tool is crashing
and yet lit is still reporting it as a success!

At this point I don't even know where to start, so rather than
leave the tree broken for who knows how long, I will get this
back to green, and then once lit is fixed on Windows, hopefully
hopefully fix the remaining set of problems for real.

llvm-svn: 303409
2017-05-19 05:57:45 +00:00
Zachary Turner b32ec02b80 [CodeView] Raise the source to ID map out of the TypeStreamMerger.
This map will be needed to rewrite symbol streams after re-writing
the corresponding type streams.

llvm-svn: 303390
2017-05-18 23:04:08 +00:00
Zachary Turner 0c60f269fc [CodeView] Provide a common interface for type collections.
Right now we have multiple notions of things that represent collections of
types. Most commonly used are TypeDatabase, which is supposed to keep
mappings from TypeIndex to type name when reading a type stream, which
happens when reading PDBs. And also TypeTableBuilder, which is used to
build up a collection of types dynamically which we will later serialize
(i.e. when writing PDBs).

But often you just want to do some operation on a collection of types, and
you may want to do the same operation on any kind of collection. For
example, you might want to merge two TypeTableBuilders or you might want
to merge two type streams that you loaded from various files.

This dichotomy between reading and writing is responsible for a lot of the
existing code duplication and overlapping responsibilities in the existing
CodeView library classes. For example, after building up a
TypeTableBuilder with a bunch of type records, if we want to dump it we
have to re-invent a bunch of extra glue because our dumper takes a
TypeDatabase or a CVTypeArray, which are both incompatible with
TypeTableBuilder.

This patch introduces an abstract base class called TypeCollection which
is shared between the various type collection like things. Wherever we
previously stored a TypeDatabase& in some common class, we now store a
TypeCollection&.

The advantage of this is that all the details of how the collection are
implemented, such as lazy deserialization of partial type streams, is
completely transparent and you can just treat any collection of types the
same regardless of where it came from.

Differential Revision: https://reviews.llvm.org/D33293

llvm-svn: 303388
2017-05-18 23:03:06 +00:00
Rui Ueyama a674943211 Set IMAGE_DLL_CHARACTERISTICS_NO_BIND.
Our output is not compatible with the Binding feature, so make it
explicit that.

Differential Revision: https://reviews.llvm.org/D33336

llvm-svn: 303378
2017-05-18 20:26:58 +00:00
Rui Ueyama 9fbfd76fe3 Merge IAT and ILT.
Previously, LLD-produced executables had IAT (Import Address Table) and
ILT (Import Lookup Table) as separate chunks of data, although their
contents are identical. My interpretation of the COFF spec when I wrote
the COFF linker is that they need to be separate tables even though they
are the same.

But Peter found that the Windows loader is fine with executables in
which IAT and ILT are merged. This is a patch to merge IAT and ILT.
I confirmed that an lld-link self-hosted with this patch works fine.

Fixes https://bugs.llvm.org/show_bug.cgi?id=33064

Differential Revision: https://reviews.llvm.org/D33326

llvm-svn: 303374
2017-05-18 19:59:22 +00:00
Peter Collingbourne 03638d091a COFF: Replace DLLNames maps with vectors.
The import lists are already binned by DLL name, so there's no need to
deduplicate here.

Differential Revision: https://reviews.llvm.org/D33330

llvm-svn: 303371
2017-05-18 18:53:39 +00:00
Rui Ueyama 01f93335a0 Use make<> everywhere in COFF to make it consistent with ELF.
We've been using make<> to allocate new objects in ELF. We have
the same function in COFF, but we didn't use it widely due to
negligence. This patch uses the function in COFF to close the gap
between ELF and COFF.

llvm-svn: 303357
2017-05-18 17:03:49 +00:00
Zachary Turner 8a7508970a [COFF] Fix interaction between /DEBUG and /PDB
When /DEBUG is not specified, /PDB should be ignored.  When
/DEBUG is specified, a PDB should be output regardless of
whether or not /PDB is specified.  /PDB just overrides the
default name.

This patch implements this behavior, and adds some tests, while
also removing a dead option /DEBUGPDB which was unused in any
code.

Differential Revision: https://reviews.llvm.org/D33302

llvm-svn: 303352
2017-05-18 15:15:10 +00:00
Rui Ueyama cd41bc8dec Re-submit r303225: Garbage collect dllimported symbols.
This reverts re-submits r303225 which was reverted in r303270 because it
broke the sanitizer-windows bot.

The reason of the failure is that we were writing dead symbols to the
symbol table. I fixed the issue.

llvm-svn: 303304
2017-05-17 21:36:08 +00:00
Hans Wennborg e67c5f6b52 Revert r303225 "Garbage collect dllimported symbols."
and follow-up r303226 "Fix Windows buildbots."

This broke the sanitizer-windows buildbot.

> Previously, the garbage collector (enabled by default or by explicitly
> passing /opt:ref) did not kill dllimported symbols. As a result,
> dllimported symbols could be added to resulting executables' dllimport
> list even if no one was actually using them.
>
> This patch implements dllexported symbol garbage collection. Just like
> COMDAT sections, dllimported symbols now have Live bits to manage their
> liveness, and MarkLive marks reachable dllimported symbols.
>
> Fixes https://bugs.llvm.org/show_bug.cgi?id=32950
>
> Reviewers: pcc
>
> Subscribers: llvm-commits
>
> Differential Revision: https://reviews.llvm.org/D33264

llvm-svn: 303270
2017-05-17 16:22:03 +00:00
Rui Ueyama 02df7a6cf1 Garbage collect dllimported symbols.
Summary:
Previously, the garbage collector (enabled by default or by explicitly
passing /opt:ref) did not kill dllimported symbols. As a result,
dllimported symbols could be added to resulting executables' dllimport
list even if no one was actually using them.

This patch implements dllexported symbol garbage collection. Just like
COMDAT sections, dllimported symbols now have Live bits to manage their
liveness, and MarkLive marks reachable dllimported symbols.

Fixes https://bugs.llvm.org/show_bug.cgi?id=32950

Reviewers: pcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D33264

llvm-svn: 303225
2017-05-17 00:35:50 +00:00
Peter Collingbourne c6f07c423d Move lib/LibDriver -> lib/ToolDrivers/llvm-lib. NFCI.
This reorganisation prevents us from cluttering up the top-level lib directory
with more driver libraries such as llvm-dlltool (see D29892).

llvm-svn: 302995
2017-05-13 22:06:46 +00:00
Zachary Turner 3a57fbd6db [Support] Move Parallel algorithms from LLD to LLVM.
Differential Revision: https://reviews.llvm.org/D33024

llvm-svn: 302748
2017-05-11 00:03:52 +00:00
Zachary Turner 092c767745 [Core] Make parallel algorithms match C++ Parallelism TS.
Differential Revision: https://reviews.llvm.org/D33016

llvm-svn: 302613
2017-05-10 01:16:22 +00:00
Rui Ueyama 88172fb603 Use the same terminology as ELF.
This patch do s/color/class/g.

llvm-svn: 302326
2017-05-05 23:52:24 +00:00
Zachary Turner 5634ccf476 lld Fixups for TypeDatabase changes.
llvm-svn: 302304
2017-05-05 22:06:06 +00:00
Rafael Espindola a0f30da62e Revert "Fix misnamed #include. Remove _NC suffix from ELF relocations TLSDESC_{LD64,ADD}_LO12"
This reverts commit r301964. It broke the build.

llvm-svn: 301969
2017-05-02 20:19:42 +00:00
Joel Jones f387eb62b9 Fix misnamed #include. Remove _NC suffix from ELF relocations TLSDESC_{LD64,ADD}_LO12
llvm-svn: 301964
2017-05-02 19:31:40 +00:00
Zachary Turner e204a6c9a3 Rename pdb::StringTable -> pdb::PDBStringTable.
With the forthcoming codeview::StringTable which a pdb::StringTable
would hold an instance of as one member, this ambiguity becomes
confusing.  Rename to PDBStringTable to avoid this.

llvm-svn: 301948
2017-05-02 18:00:13 +00:00
Rui Ueyama 3781137909 Fix comments.
llvm-svn: 301778
2017-04-30 20:58:20 +00:00
Rui Ueyama 3969acf8a4 Port recent changes to ELF/MapFile.cpp to COFF.
llvm-svn: 301754
2017-04-29 18:50:39 +00:00
Hans Wennborg ca31b9f89c Flush output in log()
This change was motivated by output from lld-link.exe and link.exe
getting intermixed. There's already a flush() call in message(), so
there's precedence.

llvm-svn: 301693
2017-04-28 21:35:02 +00:00
Saleem Abdulrasool 5c98b74536 COFF: actually synthesize CONST imports properly
CONSTANT imports expect both the `_imp_` prefixed and non-prefixed
symbols should be added to the symbol table.  This allows for linking
symbols like _NSConcreteGlobalBlock in WinObjC.  The previous change
would generate the import library properly by handling the option but
would not consume the generated entry properly.

llvm-svn: 301657
2017-04-28 17:06:40 +00:00
Rui Ueyama a85572ebf0 COFF ICF: Merge only functions. Do not merge read-only data.
This seems to be the behavior of the MSVC linker. Previously, this
incompatibility caused nasty issues in chromium build a few times.

Differential Revision: https://reviews.llvm.org/D30363

llvm-svn: 301598
2017-04-27 23:03:22 +00:00
Rui Ueyama 0417172387 Fix comments again.
I thought I fixed the page size, but there were still errors.
This patch also contains fixes for grammatical errors.
Thanks pcc for proofreading!

llvm-svn: 301454
2017-04-26 20:20:05 +00:00
Rui Ueyama 6cef17ce04 Fix comment.
llvm-svn: 301450
2017-04-26 19:50:49 +00:00
Rui Ueyama 1a7f6afe7d Add comments about Widnows .reloc section.
llvm-svn: 301287
2017-04-25 03:31:10 +00:00