Commit Graph

8486 Commits

Author SHA1 Message Date
Rui Ueyama 236e781011 Use MD5::hash(). NFC.
llvm-svn: 303893
2017-05-25 18:17:43 +00:00
Rui Ueyama 69ae29b1d1 Do not allow delay-importing data symbols.
If you pass /delayload:<dllname> to the COFF linker, it creates thunks
so that DLLs are loaded when they are used for the first time instead of
load-time.

This mechanism do not work for data symbols as there's no way to trap
acccesses to data imported from DLLs. (Technically, I think if we do not
initially map dllimport tables in memory, we could actually trap accesses
and delay-load data symbols, but that's not what Windows do.)

This patch is to report an error when you try to delay-load data symbols.

Fixes https://bugs.llvm.org/show_bug.cgi?id=33106

Differential Revision: https://reviews.llvm.org/D33557

llvm-svn: 303890
2017-05-25 18:03:34 +00:00
Rui Ueyama 0e8521c05a Reduce indentation. NFC.
llvm-svn: 303815
2017-05-24 22:36:11 +00:00
Rui Ueyama 9aa82f76ac Garbage collect dllimported symbols.
This is a different implementation than r303225 (which was reverted
in r303270, re-submitted in r303304 and then re-reverted in r303527).

In the previous patch, I tried to add Live bit to each dllimported
symbol. It turned out that it didn't work with "oldnames.lib" which
contains a lot of weak aliases to dllimported symbols.

The way we handle weak aliases is to check if undefined symbols
can be resolved using weak aliases, and if so, memcpy the Defined
symbols to weak Undefined symbols, so that any references to weak
aliases automatically see defined symbols instead of undefined ones.

This memcpy happens before MarkLive kicks in.

That means we may have multiple copies of dllimported symbols. So
turning on one instance's Live bit is not enough.

This patch moves the Live bit to dllimport file. Since multiple
copies of dllsymbols still point to the same file, we can use it as the
central repository to keep track of liveness.

Differential Revision: https://reviews.llvm.org/D33520

llvm-svn: 303814
2017-05-24 22:30:06 +00:00
Rafael Espindola a28414d7ec Simplify MipsRldMapSection::writeTo.
It is not clear why a synthetic section wants to use padding defined
in the linker script. The padding is for the space between sections.

It was also missing a test.

llvm-svn: 303812
2017-05-24 22:04:32 +00:00
Rui Ueyama 96cbf8bca6 Fix the sanitizer-windows bot.
Looks like r303801 broke the sanitizer-windows bot. I don't fully
understand what is going on, so I'll partially revert that patch.

llvm-svn: 303805
2017-05-24 20:32:23 +00:00
Rui Ueyama 27abe98cfa Close the gap between ELF and COFF ICF implementations. NFC.
We originally wrote the ICF code for COFF and ported it to ELF.
They started diverging since then. This patch closes the gap.

llvm-svn: 303801
2017-05-24 19:56:29 +00:00
Rui Ueyama f04c04837c Improve parallelism of ICF.
This is the only place we use threads for ICF. The intention of this code
was to split an input vector into 256 shards and process them in parallel.
What the code was actually doing was to split an input into 257 shards,
process the first 256 shards in parallel, and the remaining one in serial.

That means this code takes ceil(256/n)+1 instead of ceil(256/n) where n
is the number of available CPU cores. The former converges to 2 while
the latter converges to 1.

This patches fixes the above issue.

llvm-svn: 303797
2017-05-24 19:22:34 +00:00
Kamil Rytarowski e739e49c0f Replace std::call_once with llvm:call_once
Summary:
This is required on some platforms, as GNU libstdc++ std::call_once is known to be buggy.

This fixes operation of LLD on at least NetBSD and perhaps OpenBSD and Linux PowerPC.

The same change has been introduced to LLVM and LLDB.

Reviewers: ruiu

Reviewed By: ruiu

Subscribers: emaste, #lld

Tags: #lld

Differential Revision: https://reviews.llvm.org/D33508

llvm-svn: 303788
2017-05-24 18:31:48 +00:00
Rui Ueyama 466c82b74f Simplify a variable type by using StringRef instead of CachedHashStringRef.
A variable `ComdatGroup` is not supposed to contain a large number of
items. Even when linking clang, it ends up having only 300K strings.
It doesn't make sense to use CachedHashStringRef for this hash table.
This patch has neutral or slightly positive impact on performance while
reducing code complexity.

llvm-svn: 303787
2017-05-24 18:22:27 +00:00
Rafael Espindola 55b169bf5d Move writeTo to OutputSectionCommand.
This reduces how many times we have to map from OutputSection to
OutputSectionCommand. It is a required step to moving
clearOutputSections earlier.

In order to always use writeTo in OutputSectionCommand we have to call
fabricateDefaultCommands for -r links and move section compression
after it.

llvm-svn: 303784
2017-05-24 18:08:04 +00:00
Rafael Espindola 27dd659be1 Move clearOutputSections earlier. NFC.
llvm-svn: 303781
2017-05-24 17:54:28 +00:00
Rui Ueyama 8df7fa782e Rewrite badly-formatted switch. NFC.
llvm-svn: 303777
2017-05-24 17:12:53 +00:00
Rui Ueyama 216d6834ba Make key-value types explicit. NFC.
llvm-svn: 303776
2017-05-24 17:12:32 +00:00
Rui Ueyama d6489635f4 Change the control flow so that the function is a bit more readable. NFC.
llvm-svn: 303775
2017-05-24 17:12:10 +00:00
Dmitry Mikulin fd0c844fbb Do not track section types of previous sections, always use PROGBITS for dummy sections.
Fix for PR33029.

llvm-svn: 303770
2017-05-24 16:48:31 +00:00
Zachary Turner bb64231d2d Don't do a full scan of the type stream before processing records.
LazyRandomTypeCollection is designed for random access, and in
order to provide this it lazily indexes ranges of types.  In the
case of types from an object file, there is no partial index
to build off of, so it has to index the full stream up front.
However, merging types only requires sequential access, and when
that is needed, this extra work is simply wasted.  Changing the
algorithm to work on sequential arrays of types rather than
random access type collections eliminates this up front scan.

llvm-svn: 303707
2017-05-24 00:26:27 +00:00
Rafael Espindola aeb4b7d217 Clear OutSec->Sections.
Once the dummy linker script is created, we want it to be used for
everything to avoid having two redundant representations that can get
out of sync.

We were already clearing OutputSections. With this patch we clear the
Sections vector of every OutputSection.

llvm-svn: 303703
2017-05-23 22:54:06 +00:00
Rafael Espindola 6a1aa8d9b4 Use more strict types. NFC.
By the time we get to linker scripts, all special InputSectionBase
should have been combined into synthetic sections, which are a type of
InputSection. The net result is that we can use InputSection in a few
places that were using InputSectionBase.

llvm-svn: 303702
2017-05-23 22:47:31 +00:00
Peter Collingbourne 968fe93803 ELF: The later of --build-id and --build-id= wins.
If the compiler driver passes --build-id and the user uses -Wl to
pass --build-id= then the user's flag should take precedence.

Differential Revision: https://reviews.llvm.org/D33461

llvm-svn: 303689
2017-05-23 21:16:48 +00:00
Evgeniy Stepanov f12ac5b776 [lld] Infer relocation model from module flags in relocatable LTO link.
Fix for PR33096.

llvm-svn: 303579
2017-05-22 21:11:44 +00:00
Zachary Turner d4136e945e Implement various flavors of type merging.
Previous algotirhm assumed that types and ids are in a single
unified stream.  For inputs that come from object files, this
is the case.  But if the input is already a PDB, or is the result
of a previous merge, then the types and ids will already have
been split up, in which case we need an algorithm that can
accept operate on independent streams of types and ids that
refer across stream boundaries to each other.

Differential Revision: https://reviews.llvm.org/D33417

llvm-svn: 303577
2017-05-22 21:07:43 +00:00
Rui Ueyama b6632d9cd1 Revert r303304: Re-submit r303225: Garbage collect dllimported symbols.
This reverts commit r303304 because it looks like the change
introduced a crash bug. At least after that change, LLD with thinlto
crashes when linking Chromium.

llvm-svn: 303527
2017-05-22 06:01:37 +00:00
Martell Malone 1e39e5e964 COFF: migrate def parser from LLD to LLVM [2/2]
This is split up into two commits.
This commit removes the DEF parser from LLD
See the previous commit for the creation in LLVM.

Reviewers: ruiu

Differential Revision: https://reviews.llvm.org/D32689

llvm-svn: 303491
2017-05-20 19:56:44 +00:00
Zachary Turner 526f4f2aa8 Resubmit "[CodeView] Provide a common interface for type collections."
This was originally reverted because it was a breaking a bunch
of bots and the breakage was not surfacing on Windows.  After much
head-scratching this was ultimately traced back to a bug in the
lit test runner related to its pipe handling.  Now that the bug
in lit is fixed, Windows correctly reports these test failures,
and as such I have finally (hopefully) fixed all of them in this
patch.

llvm-svn: 303446
2017-05-19 19:26:58 +00:00
Zachary Turner 1dfcf8d92c Revert "[CodeView] Provide a common interface for type collections."
This is a squash of ~5 reverts of, well, pretty much everything
I did today.  Something is seriously broken with lit on Windows
right now, and as a result assertions that fire in tests are
triggering failures.  I've been breaking non-Windows bots all
day which has seriously confused me because all my tests have
been passing, and after running lit with -a to view the output
even on successful runs, I find out that the tool is crashing
and yet lit is still reporting it as a success!

At this point I don't even know where to start, so rather than
leave the tree broken for who knows how long, I will get this
back to green, and then once lit is fixed on Windows, hopefully
hopefully fix the remaining set of problems for real.

llvm-svn: 303409
2017-05-19 05:57:45 +00:00
Zachary Turner b32ec02b80 [CodeView] Raise the source to ID map out of the TypeStreamMerger.
This map will be needed to rewrite symbol streams after re-writing
the corresponding type streams.

llvm-svn: 303390
2017-05-18 23:04:08 +00:00
Zachary Turner 0c60f269fc [CodeView] Provide a common interface for type collections.
Right now we have multiple notions of things that represent collections of
types. Most commonly used are TypeDatabase, which is supposed to keep
mappings from TypeIndex to type name when reading a type stream, which
happens when reading PDBs. And also TypeTableBuilder, which is used to
build up a collection of types dynamically which we will later serialize
(i.e. when writing PDBs).

But often you just want to do some operation on a collection of types, and
you may want to do the same operation on any kind of collection. For
example, you might want to merge two TypeTableBuilders or you might want
to merge two type streams that you loaded from various files.

This dichotomy between reading and writing is responsible for a lot of the
existing code duplication and overlapping responsibilities in the existing
CodeView library classes. For example, after building up a
TypeTableBuilder with a bunch of type records, if we want to dump it we
have to re-invent a bunch of extra glue because our dumper takes a
TypeDatabase or a CVTypeArray, which are both incompatible with
TypeTableBuilder.

This patch introduces an abstract base class called TypeCollection which
is shared between the various type collection like things. Wherever we
previously stored a TypeDatabase& in some common class, we now store a
TypeCollection&.

The advantage of this is that all the details of how the collection are
implemented, such as lazy deserialization of partial type streams, is
completely transparent and you can just treat any collection of types the
same regardless of where it came from.

Differential Revision: https://reviews.llvm.org/D33293

llvm-svn: 303388
2017-05-18 23:03:06 +00:00
Rafael Espindola a46f688e8f Use linker script commands in writeMapFile.
This converts the last (chronologically) user of OutputSections to use
the linker script commands instead.

The idea is to convert all uses after fabricateDefaultCommands, so
that we have a single representation.

llvm-svn: 303384
2017-05-18 21:30:14 +00:00
Rui Ueyama a674943211 Set IMAGE_DLL_CHARACTERISTICS_NO_BIND.
Our output is not compatible with the Binding feature, so make it
explicit that.

Differential Revision: https://reviews.llvm.org/D33336

llvm-svn: 303378
2017-05-18 20:26:58 +00:00
Rui Ueyama 9fbfd76fe3 Merge IAT and ILT.
Previously, LLD-produced executables had IAT (Import Address Table) and
ILT (Import Lookup Table) as separate chunks of data, although their
contents are identical. My interpretation of the COFF spec when I wrote
the COFF linker is that they need to be separate tables even though they
are the same.

But Peter found that the Windows loader is fine with executables in
which IAT and ILT are merged. This is a patch to merge IAT and ILT.
I confirmed that an lld-link self-hosted with this patch works fine.

Fixes https://bugs.llvm.org/show_bug.cgi?id=33064

Differential Revision: https://reviews.llvm.org/D33326

llvm-svn: 303374
2017-05-18 19:59:22 +00:00
Peter Collingbourne 03638d091a COFF: Replace DLLNames maps with vectors.
The import lists are already binned by DLL name, so there's no need to
deduplicate here.

Differential Revision: https://reviews.llvm.org/D33330

llvm-svn: 303371
2017-05-18 18:53:39 +00:00
Rafael Espindola 1b41409079 Use a forward declaration. NFC.
llvm-svn: 303363
2017-05-18 17:26:00 +00:00
Rui Ueyama 01f93335a0 Use make<> everywhere in COFF to make it consistent with ELF.
We've been using make<> to allocate new objects in ELF. We have
the same function in COFF, but we didn't use it widely due to
negligence. This patch uses the function in COFF to close the gap
between ELF and COFF.

llvm-svn: 303357
2017-05-18 17:03:49 +00:00
Rafael Espindola a6465bbb72 Alternative way to detemplate GotSection.
GetSection is a template because write calls relocate.

relocate has two parts. The non alloc code really has to be a
template, as it is looking a raw input file data.

The alloc part is only a template because of getSize.

This patch folds the value of getSize early, detemplates
getRelocTargetVA and splits relocate into a templated non alloc case
and a regular function for the alloc case. This has the nice advantage
of making sure we collect all the information we need for relocations
before getting to InputSection::relocateNonAlloc.

Since we know got is alloc, it can just call the function directly and
avoid the template.

llvm-svn: 303355
2017-05-18 16:45:36 +00:00
Rafael Espindola 246c1c47ea Fix flag to start with 1 << 0. NFC.
Thanks to Andrew Ng for noticing it.

llvm-svn: 303354
2017-05-18 16:20:12 +00:00
Zachary Turner 8a7508970a [COFF] Fix interaction between /DEBUG and /PDB
When /DEBUG is not specified, /PDB should be ignored.  When
/DEBUG is specified, a PDB should be output regardless of
whether or not /PDB is specified.  /PDB just overrides the
default name.

This patch implements this behavior, and adds some tests, while
also removing a dead option /DEBUGPDB which was unused in any
code.

Differential Revision: https://reviews.llvm.org/D33302

llvm-svn: 303352
2017-05-18 15:15:10 +00:00
Peter Smith d54f368e6a [ELF] Support R_ARM_SBREL32 Relocation
This change adds support for the R_ARM_SBREL32 relocation. The relocation
is a base relative relocation that is produced by clang/llvm when -frwpi
is used. The use case for the -frwpi option is position independent data
for embedded systems that do not have a GOT. With -frwpi all data is
accessed via an offset from a base register (usually r9), where r9 is set
at run time to where the data has been loaded. The base of the data is
known as the static base.

The ARM ABI defines the static base as:
B(S) is the addressing origin of the output segment defining the symbol S.
The origin is not required to be the base address of the segment. For
simplicity we choose to use the base address of the segment.

The ARM procedure call standard only defines a read write variant using
R_ARM_SBREL32 relocations. The read-only data is accessed via pc-relative
offsets from the code, this is implemented in clang as -fropi.

Fixes PR32924

Differential Revision: https://reviews.llvm.org/D33280

llvm-svn: 303337
2017-05-18 09:12:21 +00:00
Rui Ueyama cd41bc8dec Re-submit r303225: Garbage collect dllimported symbols.
This reverts re-submits r303225 which was reverted in r303270 because it
broke the sanitizer-windows bot.

The reason of the failure is that we were writing dead symbols to the
symbol table. I fixed the issue.

llvm-svn: 303304
2017-05-17 21:36:08 +00:00
Hans Wennborg e67c5f6b52 Revert r303225 "Garbage collect dllimported symbols."
and follow-up r303226 "Fix Windows buildbots."

This broke the sanitizer-windows buildbot.

> Previously, the garbage collector (enabled by default or by explicitly
> passing /opt:ref) did not kill dllimported symbols. As a result,
> dllimported symbols could be added to resulting executables' dllimport
> list even if no one was actually using them.
>
> This patch implements dllexported symbol garbage collection. Just like
> COMDAT sections, dllimported symbols now have Live bits to manage their
> liveness, and MarkLive marks reachable dllimported symbols.
>
> Fixes https://bugs.llvm.org/show_bug.cgi?id=32950
>
> Reviewers: pcc
>
> Subscribers: llvm-commits
>
> Differential Revision: https://reviews.llvm.org/D33264

llvm-svn: 303270
2017-05-17 16:22:03 +00:00
George Rimar ec84ffc529 [ELF] - Detemplate Thunk creation.
Nothing special here, just detemplates code that became possible 
to detemplate after recent commits in a straghtforward way.

Differential revision: https://reviews.llvm.org/D33234

llvm-svn: 303237
2017-05-17 07:10:59 +00:00
Rui Ueyama de83fec029 Fix Windows buildbots.
llvm-svn: 303226
2017-05-17 01:05:56 +00:00
Rui Ueyama 02df7a6cf1 Garbage collect dllimported symbols.
Summary:
Previously, the garbage collector (enabled by default or by explicitly
passing /opt:ref) did not kill dllimported symbols. As a result,
dllimported symbols could be added to resulting executables' dllimport
list even if no one was actually using them.

This patch implements dllexported symbol garbage collection. Just like
COMDAT sections, dllimported symbols now have Live bits to manage their
liveness, and MarkLive marks reachable dllimported symbols.

Fixes https://bugs.llvm.org/show_bug.cgi?id=32950

Reviewers: pcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D33264

llvm-svn: 303225
2017-05-17 00:35:50 +00:00
George Rimar d8daafd0fb [ELF] - Update for LLVM's r303163 change. NFCi.
llvm-svn: 303164
2017-05-16 12:34:51 +00:00
George Rimar 390242d0e9 [ELF] - Detemplate elf::addSyntheticLocal(). NFC.
llvm-svn: 303155
2017-05-16 10:11:36 +00:00
George Rimar 69b17c35d9 [ELF] - Detemplate access to SymTab, DynSymTab, GnuHashTab. NFC.
Follow up for r303150.

llvm-svn: 303153
2017-05-16 10:04:42 +00:00
George Rimar f45f681609 [ELF] - Detemplate GnuHashTableSection and SymbolTableSection sections.
SymbolTableBaseSection was introduced.

Detemplation of SymbolTableSection should allow to detemplate more things.

Differential revision: https://reviews.llvm.org/D33124

llvm-svn: 303150
2017-05-16 08:53:30 +00:00
George Rimar ab94768cc1 [ELF] - Use llvm::to_integer() instead of StringRef::getAsInteger().
Switch to llvm::to_integer() everywhere in LLD instead of 
StringRef::getAsInteger() because API of latter is confusing. 
It returns true on error and false otherwise what makes reading 
the code incomfortable.

Differential revision: https://reviews.llvm.org/D33187

llvm-svn: 303149
2017-05-16 08:19:25 +00:00
Rui Ueyama 35e828775f Merge a test YAML file and a test file.
llvm-svn: 303135
2017-05-16 01:16:20 +00:00
Rafael Espindola d51400b60a Disable threads in a few tests.
They are too slow otherwise. We track the issue in pr32942.

llvm-svn: 303097
2017-05-15 18:29:14 +00:00