This reverts commit 600d52c278e123dd08bee24c1f00932b55add8de.
This patch still seems to break CrWinClangLLD, reverting until I can
find root problem.
llvm-svn: 307189
This patch still seems to break CrWinClangLLD, reverting this once more
until I can discover root problem.
This reverts commit 3dbbc8ce43be50ffde2b1c655c6d3a25796fe78b.
llvm-svn: 307188
A plain empty entry point function that returns 0 seems to produce
a binary that loads and runs fine in wine.
Differential Revision: https://reviews.llvm.org/D34833
llvm-svn: 306963
Summary:
This reverts commit 51931072a7c9a52540baf76fc30ef391d2529a2f.
This revert was originally done because the integrations of the new
WindowsResource library into LLD was causing error in chromium, due to
bugs in how resource sections were handled. These bugs were fixed,
meaning that the features may be reintegrated.
Subscribers: hiraditya, llvm-commits
Differential Revision: https://reviews.llvm.org/D34922
llvm-svn: 306941
This reverts commit d4c7e9fc63c10dbab0c30186ef8575474a704496.
This is done in order to address the failure of CrWinClangLLD etc. bots.
These throw an error of "side-by-side configuration is incorrect" during
compilation, which sounds suspiciously related to these manifest
changes.
Revert "Switch external cvtres.exe for llvm's own resource library."
This reverts commit 71fe8ef283a9dab9a3f21432c98466cbc23990d1.
llvm-svn: 306618
Summary:
In order to do this without switching on the symbol kind multiple times,
I created Defined::getChunkAndOffset and use that instead of
SymbolBody::getRVA in the inner relocation loop.
Now we get the symbol's chunk before switching over relocation types, so
we can test if it has been discarded outside the inner relocation type
switch. This also simplifies application of section relative
relocations. Previously we would switch on symbol kind to compute the
RVA, then the relocation type, and then the symbol kind again to get the
output section so we could subtract that from the symbol RVA. Now we
*always* have an OutputSection, so applying SECREL and SECTION
relocations isn't as much of a special case.
I'm still not quite happy with the cleanliness of this code. I'm not
sure what offsets and bases we should be using during the relocation
processing loop: VA, RVA, or OutputSectionOffset.
Reviewers: ruiu, pcc
Reviewed By: ruiu
Subscribers: majnemer, inglorion, llvm-commits, aprantl
Differential Revision: https://reviews.llvm.org/D34650
llvm-svn: 306566
This patch removes the dependency on the external rc.exe tool by writing
a simple .res file using our own library. In this patch I also added an
explicit definition for the .res file magic. Furthermore, I added a
unittest for embeded manifests and fixed a bug exposed by the test.
llvm-svn: 306311
Summary:
They do the obvious thing: provide the section index of .bss and the
offset of the symbol in .bss.
Reviewers: ruiu
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D34628
llvm-svn: 306304
Summary:
The main change is that we can have SECREL and SECTION relocations
against ___safe_se_handler_table, which is important for handling the
debug info in the MSVCRT.
Previously we were using DefinedRelative for __safe_se_handler_table and
__ImageBase, and after we implement CFGuard, we plan to extend it to
handle __guard_fids_table, __guard_longjmp_table, and more. However,
DefinedRelative is really only suitable for implementing __ImageBase,
because it lacks a Chunk, which you need in order to figure out the
output section index and output section offset when resolving SECREl and
SECTION relocations.
This change renames DefinedRelative to DefinedSynthetic and gives it a
Chunk. One wart is that __ImageBase doesn't have a chunk. It points to
the PE header, effectively. We could split DefinedRelative and
DefinedSynthetic if we think that's cleaner and creates fewer special
cases.
I also added safeseh.s, which checks that we don't emit a safe seh table
entries pointing to garbage collected handlers and that we don't emit a
table at all when there are no handlers.
Reviewers: ruiu
Reviewed By: ruiu
Subscribers: inglorion, pcc, llvm-commits, aprantl
Differential Revision: https://reviews.llvm.org/D34577
llvm-svn: 306293
Summary:
For SECTION relocations against absolute symbols, MSVC emits the largest
output section index plus one. I've implemented that by threading a
global variable through DefinedAbsolute that is filled in by the Writer.
A more library-oriented approach would be to thread the Writer through
Chunk::writeTo and SectionChunk::applyRel*, but Rui seems to prefer
doing it this way.
MSVC rejects SECREL relocations against absolute symbols, but only when
the relocation is in a real output section. When the relocation is in a
CodeView debug info section destined for the PDB, it seems that this
relocation error is suppressed, and absolute symbols become zeros in the
object file. This is easily implemented by checking the input section
from which we're applying relocations.
This should fix errors about __safe_se_handler_table and
__guard_fids_table when linking the CRT and generating a PDB.
Reviewers: ruiu
Subscribers: aprantl, llvm-commits
Differential Revision: https://reviews.llvm.org/D34541
llvm-svn: 306071
Summary:
The main complexity in adding symbol records is that we need to
"relocate" all the type indices. Type indices do not have anything like
relocations, an opaque data structure describing where to find existing
type indices for fixups. The linker just has to "know" where the type
references are in the symbol records. I added an overload of
`discoverTypeIndices` that works on symbol records, and it seems to be
able to link the standard library.
Reviewers: zturner, ruiu
Subscribers: llvm-commits, hiraditya
Differential Revision: https://reviews.llvm.org/D34432
llvm-svn: 305933
This works around a strange interaction with Authenticode signatures,
in which a signed PE executable with {Major,Minor}LinkerVersion = 0.0
fails to validate on Windows 7 (but is OK on Windows 10). Setting the
linker version to 14.0 (which is what VS2015 outputs) makes it work
again.
Patch by Simon Tatham <simon.tatham@arm.com>.
llvm-svn: 305929
VC2017 contains these new symbols as undefined symobls. They are used
for /guard:cf. Since we do not support the control flow guard, but we
want to at least ignore these symbols so that we can link against VS2017
libraries.
Fixes https://bugs.chromium.org/p/chromium/issues/detail?id=727193.
llvm-svn: 305876
The option numbers in the macro were off by one which
leads to some confusion. There are actually 12 arguments
to this macro.
Differential Revision: https://reviews.llvm.org/D34413
llvm-svn: 305823
Summary:
Previously we didn't add debug info chunks to the SparseChunks array, so
they didn't participate in section GC. Now we do.
Reviewers: ruiu
Subscribers: aprantl, llvm-commits
Differential Revision: https://reviews.llvm.org/D34356
llvm-svn: 305811
This is patch for GSoC project, bash-completion for clang.
To use this on bash, please run `source clang/utils/bash-autocomplete.sh`.
bash-autocomplete.sh is code for bash-completion.
In this patch, Options.td was mainly changed in order to add value class
in Options.inc.
llvm-svn: 305805
Summary:
This is a first step towards getting line info to show up in VS and
windbg. So far, only llvm-pdbutil can parse the PDBs that we produce.
cvdump doesn't like something about our file checksum tables. I'll have
to dig into that next.
This patch adds a new DebugSubsectionRecordBuilder which takes bytes
directly from some other producer, such as a linker, and sticks it into
the PDB. Line tables only need to be relocated. No data needs to be
rewritten.
File checksums and string tables, on the other hand, need to be re-done.
Reviewers: zturner, ruiu
Subscribers: llvm-commits, hiraditya
Differential Revision: https://reviews.llvm.org/D34257
llvm-svn: 305713
In this patch, I flip the switch in DriverUtils from using the external
cvtres.exe tool to using the Windows Resource library in llvm.
I also fixed a bug where .rsrc sections were marked as discardable
memory and therefore were placed in the wrong order in the final PE.
Furthermore, I modified WindowsResource to write the coff directly to a
memory buffer instead of to file, also had it use the machine types
already declared in COFF.h instead creating my own enum.
Finally, I flipped the switch to allow all unit tests that had
previously run only on windows due to a winres dependency to run
cross-platform.
Reviewers: zturner, ruiu
Subscribers: llvm-commits, hiraditya
Differential Revision: https://reviews.llvm.org/D34265
llvm-svn: 305592
Summary:
Adds a "Discarded" bool to SectionChunk to indicate if the section was
discarded by COMDAT deduplication. The Writer still just checks
`isLive()`.
Fixes PR33446
Reviewers: ruiu
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D34288
llvm-svn: 305582
When link is invoked with `/def:` and no input files, it behaves as if
`lib.exe` was invoked. Emulate this behaviour, generating the import
library from the def file that was passed. Because there is no input to
actually generate the dll, we simply process the def file early and exit
once we have created the import library.
llvm-svn: 305502
Summary:
Expose the module descriptor index and fill it in for section
contributions.
Reviewers: zturner
Subscribers: llvm-commits, ruiu, hiraditya
Differential Revision: https://reviews.llvm.org/D34126
llvm-svn: 305296
Without this, when building with shared BUILD_SHARED_LIBS=ON
I get errors such as:
lib/Core/Reader.cpp:40: error: undefined reference to
'llvm::identify_magic(llvm::StringRef)'
Differential Revision: https://reviews.llvm.org/D34004
llvm-svn: 304932
This creates a new library called BinaryFormat that has all of
the headers from llvm/Support containing structure and layout
definitions for various types of binary formats like dwarf, coff,
elf, etc as well as the code for identifying a file from its
magic.
Differential Revision: https://reviews.llvm.org/D33843
llvm-svn: 304864
This reverts commit r304561 and re-lands r303490 & co.
The fix was to use "SymbolName" when translating LLD's internal export
list to lib/Object's short export struct. The SymbolName reflects the
actual symbol name, which may include fastcall and stdcall mangling bits
not included in the /EXPORT or .def file EXPORTS name:
@@ -434,8 +434,7 @@ std::vector<COFFShortExport> createCOFFShortExportFromConfig() {
std::vector<COFFShortExport> Exports;
for (Export &E1 : Config->Exports) {
COFFShortExport E2;
- E2.Name = E1.Name;
+ // Use SymbolName, which will have any stdcall or fastcall qualifiers.
+ E2.Name = E1.SymbolName;
E2.ExtName = E1.ExtName;
E2.Ordinal = E1.Ordinal;
E2.Noname = E1.Noname;
llvm-svn: 304573
This reverts commits r303490, r303491, r303493, and r303494.
This caused http://crbug.com/728726. Essentially, exporting stdcall
functions doesn't appear to work after this change. Reduced test case
soon.
llvm-svn: 304561
Object files have symbol records not aligned to any particular
boundary (e.g. 1-byte aligned), while PDB files have symbol
records padded to 4-byte aligned boundaries. Since they share
the same reading / writing code, we have to provide an option to
specify the alignment and propagate it up to the producer or
consumer who knows what the alignment is supposed to be for the
given container type.
Added a test for this by modifying the existing PDB -> YAML -> PDB
round-tripping code to round trip symbol records as well as types.
Differential Revision: https://reviews.llvm.org/D33785
llvm-svn: 304484
Originally this was intended to be set up so that when linking
a PDB which refers to a type server, it would only visit the
PDB once, and on subsequent visitations it would just skip it
since all the records had already been added.
Due to some C++ scoping issues, this was not occurring and it
was revisiting the type server every time, which caused every
record to end up being thrown away on all subsequent visitations.
This doesn't affect the performance of linking clang-cl generated
object files because we don't use type servers, but when linking
object files and libraries generated with /Zi via MSVC, this means
only 1 object file has to be linked instead of N object files, so
the speedup is quite large.
llvm-svn: 303920
If you pass /delayload:<dllname> to the COFF linker, it creates thunks
so that DLLs are loaded when they are used for the first time instead of
load-time.
This mechanism do not work for data symbols as there's no way to trap
acccesses to data imported from DLLs. (Technically, I think if we do not
initially map dllimport tables in memory, we could actually trap accesses
and delay-load data symbols, but that's not what Windows do.)
This patch is to report an error when you try to delay-load data symbols.
Fixes https://bugs.llvm.org/show_bug.cgi?id=33106
Differential Revision: https://reviews.llvm.org/D33557
llvm-svn: 303890
This is a different implementation than r303225 (which was reverted
in r303270, re-submitted in r303304 and then re-reverted in r303527).
In the previous patch, I tried to add Live bit to each dllimported
symbol. It turned out that it didn't work with "oldnames.lib" which
contains a lot of weak aliases to dllimported symbols.
The way we handle weak aliases is to check if undefined symbols
can be resolved using weak aliases, and if so, memcpy the Defined
symbols to weak Undefined symbols, so that any references to weak
aliases automatically see defined symbols instead of undefined ones.
This memcpy happens before MarkLive kicks in.
That means we may have multiple copies of dllimported symbols. So
turning on one instance's Live bit is not enough.
This patch moves the Live bit to dllimport file. Since multiple
copies of dllsymbols still point to the same file, we can use it as the
central repository to keep track of liveness.
Differential Revision: https://reviews.llvm.org/D33520
llvm-svn: 303814
This is the only place we use threads for ICF. The intention of this code
was to split an input vector into 256 shards and process them in parallel.
What the code was actually doing was to split an input into 257 shards,
process the first 256 shards in parallel, and the remaining one in serial.
That means this code takes ceil(256/n)+1 instead of ceil(256/n) where n
is the number of available CPU cores. The former converges to 2 while
the latter converges to 1.
This patches fixes the above issue.
llvm-svn: 303797
LazyRandomTypeCollection is designed for random access, and in
order to provide this it lazily indexes ranges of types. In the
case of types from an object file, there is no partial index
to build off of, so it has to index the full stream up front.
However, merging types only requires sequential access, and when
that is needed, this extra work is simply wasted. Changing the
algorithm to work on sequential arrays of types rather than
random access type collections eliminates this up front scan.
llvm-svn: 303707
Previous algotirhm assumed that types and ids are in a single
unified stream. For inputs that come from object files, this
is the case. But if the input is already a PDB, or is the result
of a previous merge, then the types and ids will already have
been split up, in which case we need an algorithm that can
accept operate on independent streams of types and ids that
refer across stream boundaries to each other.
Differential Revision: https://reviews.llvm.org/D33417
llvm-svn: 303577
This reverts commit r303304 because it looks like the change
introduced a crash bug. At least after that change, LLD with thinlto
crashes when linking Chromium.
llvm-svn: 303527
This is split up into two commits.
This commit removes the DEF parser from LLD
See the previous commit for the creation in LLVM.
Reviewers: ruiu
Differential Revision: https://reviews.llvm.org/D32689
llvm-svn: 303491
This was originally reverted because it was a breaking a bunch
of bots and the breakage was not surfacing on Windows. After much
head-scratching this was ultimately traced back to a bug in the
lit test runner related to its pipe handling. Now that the bug
in lit is fixed, Windows correctly reports these test failures,
and as such I have finally (hopefully) fixed all of them in this
patch.
llvm-svn: 303446
This is a squash of ~5 reverts of, well, pretty much everything
I did today. Something is seriously broken with lit on Windows
right now, and as a result assertions that fire in tests are
triggering failures. I've been breaking non-Windows bots all
day which has seriously confused me because all my tests have
been passing, and after running lit with -a to view the output
even on successful runs, I find out that the tool is crashing
and yet lit is still reporting it as a success!
At this point I don't even know where to start, so rather than
leave the tree broken for who knows how long, I will get this
back to green, and then once lit is fixed on Windows, hopefully
hopefully fix the remaining set of problems for real.
llvm-svn: 303409
Right now we have multiple notions of things that represent collections of
types. Most commonly used are TypeDatabase, which is supposed to keep
mappings from TypeIndex to type name when reading a type stream, which
happens when reading PDBs. And also TypeTableBuilder, which is used to
build up a collection of types dynamically which we will later serialize
(i.e. when writing PDBs).
But often you just want to do some operation on a collection of types, and
you may want to do the same operation on any kind of collection. For
example, you might want to merge two TypeTableBuilders or you might want
to merge two type streams that you loaded from various files.
This dichotomy between reading and writing is responsible for a lot of the
existing code duplication and overlapping responsibilities in the existing
CodeView library classes. For example, after building up a
TypeTableBuilder with a bunch of type records, if we want to dump it we
have to re-invent a bunch of extra glue because our dumper takes a
TypeDatabase or a CVTypeArray, which are both incompatible with
TypeTableBuilder.
This patch introduces an abstract base class called TypeCollection which
is shared between the various type collection like things. Wherever we
previously stored a TypeDatabase& in some common class, we now store a
TypeCollection&.
The advantage of this is that all the details of how the collection are
implemented, such as lazy deserialization of partial type streams, is
completely transparent and you can just treat any collection of types the
same regardless of where it came from.
Differential Revision: https://reviews.llvm.org/D33293
llvm-svn: 303388
Our output is not compatible with the Binding feature, so make it
explicit that.
Differential Revision: https://reviews.llvm.org/D33336
llvm-svn: 303378
Previously, LLD-produced executables had IAT (Import Address Table) and
ILT (Import Lookup Table) as separate chunks of data, although their
contents are identical. My interpretation of the COFF spec when I wrote
the COFF linker is that they need to be separate tables even though they
are the same.
But Peter found that the Windows loader is fine with executables in
which IAT and ILT are merged. This is a patch to merge IAT and ILT.
I confirmed that an lld-link self-hosted with this patch works fine.
Fixes https://bugs.llvm.org/show_bug.cgi?id=33064
Differential Revision: https://reviews.llvm.org/D33326
llvm-svn: 303374
The import lists are already binned by DLL name, so there's no need to
deduplicate here.
Differential Revision: https://reviews.llvm.org/D33330
llvm-svn: 303371
We've been using make<> to allocate new objects in ELF. We have
the same function in COFF, but we didn't use it widely due to
negligence. This patch uses the function in COFF to close the gap
between ELF and COFF.
llvm-svn: 303357
When /DEBUG is not specified, /PDB should be ignored. When
/DEBUG is specified, a PDB should be output regardless of
whether or not /PDB is specified. /PDB just overrides the
default name.
This patch implements this behavior, and adds some tests, while
also removing a dead option /DEBUGPDB which was unused in any
code.
Differential Revision: https://reviews.llvm.org/D33302
llvm-svn: 303352
This reverts re-submits r303225 which was reverted in r303270 because it
broke the sanitizer-windows bot.
The reason of the failure is that we were writing dead symbols to the
symbol table. I fixed the issue.
llvm-svn: 303304
and follow-up r303226 "Fix Windows buildbots."
This broke the sanitizer-windows buildbot.
> Previously, the garbage collector (enabled by default or by explicitly
> passing /opt:ref) did not kill dllimported symbols. As a result,
> dllimported symbols could be added to resulting executables' dllimport
> list even if no one was actually using them.
>
> This patch implements dllexported symbol garbage collection. Just like
> COMDAT sections, dllimported symbols now have Live bits to manage their
> liveness, and MarkLive marks reachable dllimported symbols.
>
> Fixes https://bugs.llvm.org/show_bug.cgi?id=32950
>
> Reviewers: pcc
>
> Subscribers: llvm-commits
>
> Differential Revision: https://reviews.llvm.org/D33264
llvm-svn: 303270
Summary:
Previously, the garbage collector (enabled by default or by explicitly
passing /opt:ref) did not kill dllimported symbols. As a result,
dllimported symbols could be added to resulting executables' dllimport
list even if no one was actually using them.
This patch implements dllexported symbol garbage collection. Just like
COMDAT sections, dllimported symbols now have Live bits to manage their
liveness, and MarkLive marks reachable dllimported symbols.
Fixes https://bugs.llvm.org/show_bug.cgi?id=32950
Reviewers: pcc
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D33264
llvm-svn: 303225
This reorganisation prevents us from cluttering up the top-level lib directory
with more driver libraries such as llvm-dlltool (see D29892).
llvm-svn: 302995
With the forthcoming codeview::StringTable which a pdb::StringTable
would hold an instance of as one member, this ambiguity becomes
confusing. Rename to PDBStringTable to avoid this.
llvm-svn: 301948
This change was motivated by output from lld-link.exe and link.exe
getting intermixed. There's already a flush() call in message(), so
there's precedence.
llvm-svn: 301693
CONSTANT imports expect both the `_imp_` prefixed and non-prefixed
symbols should be added to the symbol table. This allows for linking
symbols like _NSConcreteGlobalBlock in WinObjC. The previous change
would generate the import library properly by handling the option but
would not consume the generated entry properly.
llvm-svn: 301657
This seems to be the behavior of the MSVC linker. Previously, this
incompatibility caused nasty issues in chromium build a few times.
Differential Revision: https://reviews.llvm.org/D30363
llvm-svn: 301598
I thought I fixed the page size, but there were still errors.
This patch also contains fixes for grammatical errors.
Thanks pcc for proofreading!
llvm-svn: 301454
Summary: When using /msvclto, lld and MSVC's linker both do their own symbol resolution. This can cause them to select different archive members, which can result in undefined references. This change avoids that situation by extracting archive members that are selected by lld and passing those to link.exe before any archives, so that MSVC's uses those objects for symbol resolution instead of different archive members.
Reviewers: pcc, rnk, ruiu
Reviewed By: pcc
Subscribers: llvm-commits, mehdi_amini
Differential Revision: https://reviews.llvm.org/D32317
llvm-svn: 301045
The CONSTANT export type is marked as obsolete, but link still supports
this. Furthermore, WinObjC uses this for certain exports. Add support
for this export type.
llvm-svn: 301013
Summary:
Fixes PR32689: /msvclto creates response files with lines
that are too long for msvc's linker (LNK1170).
Reviewers: hans, rnk, ruiu
Subscribers: llvm-commits, mehdi_amini
Differential Revision: https://reviews.llvm.org/D32185
llvm-svn: 300612
Start using it in LLD to avoid needing to read bitcode again just to get the
target triple, and in llvm-lto2 to avoid printing symbol table information
that is inappropriate for the target.
Differential Revision: https://reviews.llvm.org/D32038
llvm-svn: 300300
Summary:
lld-link allows the number of parallel ThinLTO jobs to be specified
using /opt:lldltojobs=N. If left unspecified, the implementation
conservatively defaults to 1. This leads to very long link times. This
change makes it so that the default is to automatically set the
parallelism, as we do in the ELF linker.
Reviewers: ruiu, hans
Reviewed By: ruiu, hans
Subscribers: pcc, mehdi_amini, Prazek, llvm-commits
Differential Revision: https://reviews.llvm.org/D31986
llvm-svn: 300089
The /appcontainer flag indicates that the module may only be used inside
an application container (for isolation). This has been supported by
link.exe since Windows 8.0. It sets an additional bit in the PE DLL
Characteristics flag to indicate the behavioural change.
llvm-svn: 299728
Summary:
This adds support for reporting multiple errors in a single invocation of lld-link. The limit defaults to 20 and can be changed with the /ERRORLIMIT command line parameter, or set to unlimited by passing a value of 0.
This is a new attempt after r295507, which was reverted because opening files raced with exiting early, causing the test to be flaky. This version avoids the race by exiting before calling enqueuePath.
Reviewers: pcc, ruiu
Reviewed By: ruiu
Subscribers: llvm-commits, dblaikie
Differential Revision: https://reviews.llvm.org/D31688
llvm-svn: 299496
Summary:
The TypeTableBuilder provides stable storage for type records. We don't
need to copy all of the bytes into a flat vector before adding it to the
TpiStreamBuilder.
This makes addTypeRecord take an ArrayRef<uint8_t> and a hash code to go
with it, which seems like a simplification.
Reviewers: ruiu, zturner, inglorion
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D31634
llvm-svn: 299406
Introduce symbol table data structures that can be potentially written to
disk, have the LTO library build those data structures using temporarily
constructed modules and redirect the LTO library implementation to go through
those data structures. This allows us to remove the LLVMContext and Modules
owned by InputFile.
With this change I measured a peak memory consumption decrease from 5.4GB to
2.8GB in a no-op incremental ThinLTO link of Chromium on Linux. The impact on
memory consumption is larger in COFF linkers where we are currently forced
to materialize all metadata in order to read linker options. Peak memory
consumption linking a large piece of Chromium for Windows with full LTO and
debug info decreases from >64GB (OOM) to 15GB.
Part of PR27551.
Differential Revision: https://reviews.llvm.org/D31364
llvm-svn: 299168
Summary: In the ELF linker, we create the buffer identifier for bitcode files by appending the object name to the archive name. This change makes the COFF linker do the same. Without the change, ThinLTO builds can fail with an error message about multiple ThinLTO modules per object file, caused by object files contained in different archives having the same name.
Reviewers: pcc, ruiu
Reviewed By: pcc
Subscribers: mehdi_amini
Differential Revision: https://reviews.llvm.org/D31402
llvm-svn: 298942
Summary: MSVC does this when producing a PDB.
Reviewers: ruiu
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D31316
llvm-svn: 298717
This will be used in the sanitizer test suite, which wants to use DWARF
line tables.
At some point we should reconsider how LLD handles the long section
names required by DWARF debug sections.
llvm-svn: 298544
Summary:
This also delays setting the output filename based on the first input
argument until after processing /def.
Fixes PR32354
Reviewers: ruiu, pcc
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D31152
llvm-svn: 298327
In doing so, clean up the MD5 interface a little. Most
existing users only care about the lower 8 bytes of an MD5,
but for some users that care about the upper and lower,
there wasn't a good interface. Furthermore, consumers
of the MD5 checksum were required to handle endianness
details on their own, so it seems reasonable to abstract
this into a nicer interface that just gives you the right
value.
Differential Revision: https://reviews.llvm.org/D31105
llvm-svn: 298322
The Archive object owns the memory buffers of any thin archive members, so we
need to make sure the object is still in scope when we access archive members.
Differential Revision: https://reviews.llvm.org/D31066
llvm-svn: 298033
The MSVC linker doesn't like archive files containing non-native object
files.
When we are doing an LTO build, we may create archive files containing
both LLVM bitcode files and native object files. For example, if a
project contains assembly files and C++ files, we create native object
files for the assembly files and LLVM bitcode files for the C++ files.
With the /msvclto option, LLD passes archive files to the MSVC linker.
Previously, we didn't pass archive files if they contain at least one
bitcode files. That wasn't correct because the native object files that
weren't passed to the MSVC linker may be needed to complete linking.
In this patch, we create new temporary archive files to strip bitcode
files.
Differential Revision: https://reviews.llvm.org/D31053
llvm-svn: 297997
Previously we did not have support for writing detailed
module information for each module, as well as the symbol
records. This patch adds support for this, and in doing
so enables the ability to construct minimal PDBs from
just a few lines of YAML. A test is added to illustrate
this functionality.
llvm-svn: 297900
Previously, if you have foo=bar in a definition file, this assertion
could fire because when symbols are read from file they could be mangled.
It seems that due to historical reasons underscore mangling scheme is
really ad-hoc, and I cannot find a clean way to handle this. I had
to just de-mangle symbols to search again.
llvm-svn: 297357
Some archive files created during chromium build contains both BC
and native files. If that's the case, we want to pass the archive
file to link.exe. Otherwise, the MSVC linker would complain that
there's an unresolved symbol in a given set of files.
I cannot explain why link.exe doesn't complain about the presence
of bitcode files in this case, but it seems link.exe doesn't touch BC.
llvm-svn: 297229
If /msvclto is specified, we compile bitcode files and pass it to the
MSVC linker, stripping all bitcode files. We haven't stripped archive
files, because I was thinking that the MSVC linker wouldn't touch files
in archive files. When we pass an object file to link.exe, all symbols
have been resolved already, so link.exe shoulnd't need any of the files
in archives.
It turns out that even though link.exe doesn't need to do that, it
seems to try to read each file in all archives. And if there's a non-
COFF file in an archive, it exists with an error message. So we need
to remove archives from the command line too.
llvm-svn: 297191
Prior to MSVC 2015 we had to manually include this header any
time we were going to include <thread> or <future> due to a
bug in MSVC's STL implementation. This has been fixed in MSVC
for some time now, and we require VS 2015 minimum, so we can
remove this across all subprojects.
llvm-svn: 296906
After several smaller patches to get most of the core improvements
finished up, this patch is a straight move and header fixup of
the source.
Differential Revision: https://reviews.llvm.org/D30266
llvm-svn: 296810
Before the endianness was specified on each call to read
or write of the StreamReader / StreamWriter, but in practice
it's extremely rare for streams to have data encoded in
multiple different endiannesses, so we should optimize for the
99% use case.
This makes the code cleaner and more general, but otherwise
has NFC.
llvm-svn: 296415
This was reverted because it was breaking some builds, and
because of incorrect error code usage. Since the CL was
large and contained many different things, I'm resubmitting
it in pieces.
This portion is NFC, and consists of:
1) Renaming classes to follow a consistent naming convention.
2) Fixing the const-ness of the interface methods.
3) Adding detailed doxygen comments.
4) Fixing a few instances of passing `const BinaryStream& X`. These
are now passed as `BinaryStreamRef X`.
llvm-svn: 296394
r296215, "[PDB] General improvements to Stream library."
r296217, "Disable BinaryStreamTest.StreamReaderObject temporarily."
r296220, "Re-enable BinaryStreamTest.StreamReaderObject."
r296244, "[PDB] Disable some tests that are breaking bots."
r296249, "Add static_cast to silence -Wc++11-narrowing."
std::errc::no_buffer_space should be used for OS-oriented errors for socket transmission.
(Seek discussions around llvm/xray.)
I could substitute s/no_buffer_space/others/g, but I revert whole them ATM.
Could we define and use LLVM errors there?
llvm-svn: 296258
This adds various new functionality and cleanup surrounding the
use of the Stream library. Major changes include:
* Renaming of all classes for more consistency / meaningfulness
* Addition of some new methods for reading multiple values at once.
* Full suite of unit tests for reader / writer functionality.
* Full set of doxygen comments for all classes.
* Streams now store their own endianness.
* Fixed some bugs in a few of the classes that were discovered
by the unit tests.
llvm-svn: 296215
This is part of a larger effort to get the Stream code moved
up to Support. I don't want to do it in one large patch, in
part because the changes are so big that it will treat everything
as file deletions and add, losing history in the process.
Aside from that though, it's just a good idea in general to
make small changes.
So this change only changes the names of the Stream related
source files, and applies necessary source fix ups.
llvm-svn: 296211
I added this log message to test the /msvclto option, but
this output might confuse FileCheck. This patch attempts to fix
it by removing it.
llvm-svn: 295793
LLD is a multi-threaded program. errs() or outs() are not guaranteed
to be thread-safe (they are actually not).
LLD's message(), log() or error() are thread-safe. We should use them.
llvm-svn: 295787
Behavior races on ErrorCount. If the enqueued paths are evaluated
eagerly (in enqueuePath) then the behavior is as the test expects. But
they may not be evaluated until the future is waited on, in run() -
which is after the early return/exit on ErrorCount. (this causes the
test to fail (because in the "/ERRORCOUNT:XYZ" test, no other errors
are printed), at least for me, on linux)
This reverts commit r295507.
llvm-svn: 295590
Summary: This adds support for reporting multiple errors in a single invocation of lld-link. The limit defaults to 20 and can be changed with the /ERRORLIMIT command line parameter, or set to unlimited by passing a value of 0.
Reviewers: pcc, ruiu
Reviewed By: ruiu
Differential Revision: https://reviews.llvm.org/D29691
llvm-svn: 295507
Some PDBs or object files can contain references to other PDBs
where the real type information lives. When this happens,
all type indices in the original PDB are meaningless because
their records are not there.
With this patch we add the ability to pull type info from those
secondary PDBs.
Differential Revision: https://reviews.llvm.org/D29973
llvm-svn: 295382
LLVM defines `PTHREAD_LIB` which is used by AddLLVM.cmake and various projects
to correctly link the threading library when needed. Unfortunately
`PTHREAD_LIB` is defined by LLVM's `config-ix.cmake` file which isn't installed
and therefore can't be used when configuring out-of-tree builds. This causes
such builds to fail since `pthread` isn't being correctly linked.
This patch attempts to fix that problem by renaming and exporting
`LLVM_PTHREAD_LIB` as part of`LLVMConfig.cmake`. I renamed `PTHREAD_LIB`
because It seemed likely to cause collisions with downstream users of
`LLVMConfig.cmake`.
llvm-svn: 294690
Summary: This adds an option to save temporary files generated during link-time optimization. This can be useful for debugging.
Reviewers: ruiu, pcc
Reviewed By: ruiu, pcc
Subscribers: mehdi_amini
Differential Revision: https://reviews.llvm.org/D29518
llvm-svn: 294498
If `/debugtypes` is used to omit the codeview information, we would not
have constructed the debug info codeview record which is used to tie the
PDB to the binary. In such a case, rub out the GUID and Age fields.
llvm-svn: 294279
This patch defines a new command line option, /MSVCLTO, to LLD.
If that option is given, LLD invokes link.exe to link LTO-generated
object files. This is hacky but useful because link.exe can create
PDB files.
Differential Revision: https://reviews.llvm.org/D29526
llvm-svn: 294234
Summary: The COFF linker previously implemented link-time optimization using an API which has now been marked as legacy. This change refactors the COFF linker to use the new LTO API, which is also used by the ELF linker.
Reviewers: pcc, ruiu
Reviewed By: pcc
Subscribers: mgorny, mehdi_amini
Differential Revision: https://reviews.llvm.org/D29059
llvm-svn: 293967
Previously, mergeTypeStreams returns only true or false, so it was
impossible to know the reason if it failed. This patch changes the
function signature so that it returns an Error object.
Differential Revision: https://reviews.llvm.org/D29362
llvm-svn: 293820
Previously, we were printing out something like this for
sections/symbols with alignment 16
0000000000201000 0000000000000182 10 .data
which I think confusing. I think printing it in decimal is better.
Differential Revision: https://reviews.llvm.org/D29258
llvm-svn: 293685
Summary: This copies over some functionality we have in ELF/Error.{cpp,h} and makes it available in COFF/Error.{cpp,h}
Reviewers: pcc, rafael, ruiu
Subscribers:
Differential Revision: https://reviews.llvm.org/D28692
llvm-svn: 292240
Previous code had a bug that if the program exits with an assert() or
fail() before the control reaches end of writeMapFile(), it leaves a
temporary file, because FileRemover's dtor isn't called in that case.
I could fix that by removeFileOnSignal() and other functions, but
I think we can simply write to the result file directly. I think
that is straightforward and easy to understand.
Additionally, that allows something like `-Map /dev/null` or a bash
hack such as `-Map >(grep symbol-im-looking-for)`. Previously,
that kind of things didn't work.
Differential Revision: https://reviews.llvm.org/D28714
llvm-svn: 292041
Summary: When we encouter a relocation type we don't know how to handle, this change causes us to print out the hexadecimal value of the relocation type. This makes troubleshooting a little easier.
Reviewers: ruiu, zturner
Differential Revision: https://reviews.llvm.org/D28576
llvm-svn: 291962
This patch is to merge type info in multiple .debug$T sections.
One mystery that needs to be solved is that it is not clear how
the MSVC linker uses TPI and IPI streams. Both streams contain
type info, and it is not obvious what kind of record should go
which.
dumppdb command in microsoft-pdb repository prints out IPI stream
contents as "IDs" and TPI stream as "TYPES", but looks like the tool
don't really care about which stream type recrods were read from.
For now, in this patch, I emit all type records to TPI stream.
It might just work with other tools. If not, we need to investigate
it more.
llvm-svn: 291739
Previously the type dumper itself was passed around to a lot of different
places and manipulated in ways that were more appropriate on the type
database. For example, the entire TypeDumper was passed into the symbol
dumper, when all the symbol dumper wanted to do was lookup the name of a
TypeIndex so it could print it. That's what the TypeDatabase is for --
mapping type indices to names.
Another example is how if the user runs llvm-pdbdump with the option to
dump symbols but not types, we still have to visit all types so that we
can print minimal information about the type of a symbol, but just without
dumping full symbol records. The way we did this before is by hacking it
up so that we run everything through the type dumper with a null printer,
so that the output goes to /dev/null. But really, we don't need to dump
anything, all we want to do is build the type database. Since
TypeDatabaseVisitor now exists independently of TypeDumper, we can do
this. We just build a custom visitor callback pipeline that includes a
database visitor but not a dumper.
All the hackery around printers etc goes away. After this patch, we could
probably even delete the entire CVTypeDumper class since really all it is
at this point is a thin wrapper that hides the details of how to build a
useful visitation pipeline. It's not a priority though, so CVTypeDumper
remains for now.
After this patch we will be able to easily plug in a different style of
type dumper by only implementing the proper visitation methods to dump
one-line output and then sticking it on the pipeline.
Differential Revision: https://reviews.llvm.org/D28524
llvm-svn: 291724
We were starting to get some name clashes between llvm-pdbdump
and the common CodeView framework, so I took this opportunity
to rename a bunch of files to more accurately describe their
usage. This also helps in llvm-pdbdump to distinguish
between different files and whether they are used for pretty
dump mode or raw dump mode.
llvm-svn: 291627
This is how we use TarWriter in LLD. Now LLD does not append
a file extension, so you need to pass `--reproduce foo.tar`
instead of `--reproduce foo`.
Differential Revision: https://reviews.llvm.org/D28103
llvm-svn: 291210
The PDB GUID, Age, and version are tied together by the RSDS record in
the binary. Pass along the BuildId information into the createPDB to
allow us to tie the binary and the PDB together.
llvm-svn: 290975
Assert that the size of the MD5 result is the same size as the signature
field being populated. Use the sizeof operator to determine the size of
the field being written rather than hardcoding it to the magic number
16. NFC.
llvm-svn: 290764
I thought for a while about how to remove it, but it looks like we
can just copy the file for now. Of course I'm not happy about that,
but it's just less than 50 lines of code, and we already have
duplicate code in Error.h and some other places. I want to solve
them all at once later.
Differential Revision: https://reviews.llvm.org/D27819
llvm-svn: 290062
File system operations were still dominating the profile on Windows. In this
case we were spending a significant amount of our time repeatedly searching
for libraries as a result of processing linker directives. Address this
by caching whether we have already found a library with a given name. For
chrome_child.dll:
Before: 10.53s
After: 6.88s
Differential Revision: https://reviews.llvm.org/D27840
llvm-svn: 289915
Profiling revealed that the majority of lld's execution time on Windows was
spent opening and mapping input files. We can reduce this cost significantly
by performing these operations asynchronously.
This change introduces a queue for all operations on input file data. When
we discover that we need to load a file (for example, when we find a lazy
archive for an undefined symbol, or when we read a linker directive to
load a file from disk), the file operation is launched using a future and
the symbol resolution operation is enqueued. This implies another change
to symbol resolution semantics, but it seems to be harmless ("ninja All"
in Chromium still succeeds).
To measure the perf impact of this change I linked Chromium's chrome_child.dll
with both thin and fat archives.
Thin archives:
Before (median of 5 runs): 19.50s
After: 10.93s
Fat archives:
Before: 12.00s
After: 9.90s
On Linux I found that doing this asynchronously had a negative effect on
performance, probably because the cost of mapping a file is small enough that
it becomes outweighed by the cost of managing the futures. So on non-Windows
platforms I use the deferred execution strategy.
Differential Revision: https://reviews.llvm.org/D27768
llvm-svn: 289760
Enable building lld as a standalone project. This is motivated by the desire to
package lld for inclusion in a linux distribution. This allows building lld
against an existing paired llvm installation. Now that lld is usable on x86_64,
it makes sense to revive this configuration to allow distributions to package
it.
llvm-svn: 289421
This patch replaces the symbol table's object and archive queues, as well as
the convergent loop in the linker driver, with a design more similar to the
ELF linker where symbol resolution directly causes input files to be added to
the link, including input files arising from linker directives. Effectively
this removes the last vestiges of the old parallel input file loader.
Differential Revision: https://reviews.llvm.org/D27660
llvm-svn: 289409
Using a set here caused us to take about 1 second longer to write the symbol
table when linking chrome_child.dll. With this I consistently get better
performance on Windows with the new symbol table.
Before r289280 and with r289183 reverted (median of 5 runs): 17.65s
After this change: 17.33s
On Linux things look even better:
Before: 10.700480444s
After: 5.735681610s
Differential Revision: https://reviews.llvm.org/D27648
llvm-svn: 289408
This ports the ELF linker's symbol table design, introduced in r268178,
to the COFF linker.
Differential Revision: http://reviews.llvm.org/D21166
llvm-svn: 289280
The former option bases the filename on the output name, e.g. if the
link output is a.exe, the map will be written to a.map. This matches the
behaviour of link.exe's /MAP option and is useful for creating a map
file of each executable when building a large project.
Differential Revision: https://reviews.llvm.org/D27595
llvm-svn: 289271
Profiling revealed that we were spending 5% of our time linking
chrome_child.dll just in this call to toString().
Differential Revision: https://reviews.llvm.org/D27628
llvm-svn: 289270
I don't think the data I add to a TPI stream in this patch is correct,
but at least it can be displayed using llvm-pdbdump. Until I add more
streams to a PDB file, I'm not able to know whether the data will be
accepted by MSVC tools or not.
llvm-svn: 289183
Previously, we had different way to stringize SymbolBody and InputFile
to construct error messages. This patch defines overloaded function
toString() so that we don't need to memorize all these different
function names.
With that change, it is now easy to include demangled names in error
messages. Now, if there is a symbol name conflict, we'll print out
both mangled and demangled names.
llvm-svn: 288992
LLD used to take 11.73 seconds to link Clang. Now it is 6.94 seconds.
MSVC link takes 83.02 seconds. Note that ICF is enabled by default on
Windows, so a low latency ICF is more important than in ELF.
llvm-svn: 288487
Associative sections are sections that need to be linked if their associated
sections are linked. Associative sections are used to append auxiliary data
such as debug info.
Previously, we compared all associative sections when comparing two comdat
sections. Because usually assocative sections are not mergeable sections,
we missed a lot of mergeable sections. MSVC linker doesn't seem to check
the identity of associative sections.
This patch makes LLD to ignore associative sections when doing ICF.
llvm-svn: 288483
rL287555 introduces a link error when building with BUILD_SHARED_LIBS:
undefined reference to llvm::codeview::CVSymbolDumper::dump(),
and more...
The functions are available in libDebugInfoCodeView, from LLVM.
Patch by Visoiu Mistrih Francis!
llvm-svn: 287837
Previously, we discarded .debug$ sections. This patch adds them to
files so that PDB.cpp can access them.
This patch also adds a debug option, /dumppdb, to dump debug info
fed to createPDB so that we can verify that valid data has been passed.
llvm-svn: 287555
This patch updates a couple places where add_dependencies was being explicitly called to add dependencies on intrinsics_gen to instead use the DEPENDS named parameter. This cleanup is needed for a patch I'm working on to add a dependency debugging mode to the build system.
llvm-svn: 287205
createManifestRes was generating a MemoryBuffer from a TemporaryFile,
keeping the data but removing the file, before passing the file path
to CVTRES.exe, leading to the following error:
CVTRES : fatal error CVT1101: cannot open 'C:\Users\user\AppData\
Local\Temp\lld-output-resource-bfee19.res' for reading
With this, we instead create a new TemporaryFile before passing it to cvtres.
Patch from Rudy Pons!
llvm-svn: 287034
Object files compiled with cl.exe /GL contain intermediate code for LTO.
We can't (and don't want to) interpret such code, but we should print
out a user-friendly error message.
Differential Revision: https://reviews.llvm.org/D26647
llvm-svn: 286921
The MSVC linker relies on this invariant to produce a valid import
table. More ASan tests pass in a stage 2 build now. They still fail when
using LLD since there are no PDBs for the dynamic ASan runtime.
llvm-svn: 286499
Following the lazy reference might bring in an object file that depends
on bitcode files that weren't part of the LTO step.
Differential Revision: https://reviews.llvm.org/D25461
llvm-svn: 283989
With this, "llvm-pdbdump yaml -ipi-stream" prints out an IPI stream.
Previously it crashed because it can't handle the case where IPI
stream doesn't exist.
llvm-svn: 283392
I do not fully understand how to use these classes yet, but
seems like these arguments are not used, since without them
all tests still pass. In order to simplify the situation,
I'll remove them now.
llvm-svn: 283174
Handle this in the exact same way as IMAGE_REL_AMD64_SECREL
and IMAGE_REL_I386_SECREL.
Differential revision: https://reviews.llvm.org/D24608
llvm-svn: 282531
So that it is clear that FileOutputBuffer does not depend on
PDB file builder. Eventually we will have to to get the file size
info from the file builder to create a file with the exact size.
NFC.
llvm-svn: 282454
Change the way we calculate the build id to use MD5 to give reproducible build
ids. Previously we would generate random bytes for the build id GUID.
llvm-svn: 281079
Previously, we created temporary files using llvm::sys::fs::createTemporaryFile
and removed them using llvm::FileRemover. This is error-prone as it is easy to
forget creating FileRemover instances after creating temporary files.
There is actually a temporary file leak bug.
This patch introduces a new class, TemporaryFile, to manage temporary files
in the RAII style.
Differential Revision: https://reviews.llvm.org/D24176
llvm-svn: 280510
Summary:
UBSan complains like the following:
tools/lld/COFF/Writer.cpp:97:15: runtime error: null pointer passed as argument 2, which is declared to never be null
The reason is that the vector could be empty.
Reviewers: rsmith
Subscribers: Eugene.Zelenko, kcc
Differential Revision: https://reviews.llvm.org/D24050
llvm-svn: 280259
The IMAGE_FILE_HEADER structure contains a (RVA, size) to an array of
COFF_DEBUG_DIRECTORY records. Each one of these records contains an RVA to a OMF
Debug Directory. These OMF debug directories are derived into newer types such
as PDB70, PDB20, etc. This constructs a PDB70 structure which will allow us to
associate a GUID with a build to actually tie debug information.
llvm-svn: 280012
Reorder the table setup to mirror the indices corresponding to them. This means
that the table values are filled out as per the enumeration ordering. Doing so
makes it easier to identify a particular table. NFC.
llvm-svn: 278199
Add the support infrastructure for the /debugtype option which takes a comma
delimited list of debug info to generate. The defaults are based on other
options potentially (/driver or /profile). This sets up the infrastructure to
allow us to emit RSDS records to get "build id" equivalents on COFF (similar to
binutils).
llvm-svn: 278056
Don't blindly OR in the new value, but clear the existing one, since it can be
nonzero. Read out the existing value before, and add into the desired offset.
(The add is done outside of the applyMOV, to handle potential overflow between
the two.)
Patch by Martin Storsjö!
llvm-svn: 277846
The opcode for the bl branches can initially be F000 F800, i.e.
the J1 and J2 bits are already set. Therefore mask these bits out
before or'ing in the new bits.
Patch by Martin Storsjö!
llvm-svn: 277836
This flag is implemented similarly to --reproduce in the ELF linker.
This patch implements /linkrepro by moving the cpio writer and associated
utility functions to lldCore, and using that implementation in both linkers.
One COFF-specific detail is that we store the object file from which the
resource files were created in our reproducer, rather than the resource
files themselves. This allows the reproducer to be used on non-Windows
systems for example.
Differential Revision: https://reviews.llvm.org/D22418
llvm-svn: 276719
Eric Beckmann