With the forthcoming codeview::StringTable which a pdb::StringTable
would hold an instance of as one member, this ambiguity becomes
confusing. Rename to PDBStringTable to avoid this.
llvm-svn: 301948
This change was motivated by output from lld-link.exe and link.exe
getting intermixed. There's already a flush() call in message(), so
there's precedence.
llvm-svn: 301693
CONSTANT imports expect both the `_imp_` prefixed and non-prefixed
symbols should be added to the symbol table. This allows for linking
symbols like _NSConcreteGlobalBlock in WinObjC. The previous change
would generate the import library properly by handling the option but
would not consume the generated entry properly.
llvm-svn: 301657
This seems to be the behavior of the MSVC linker. Previously, this
incompatibility caused nasty issues in chromium build a few times.
Differential Revision: https://reviews.llvm.org/D30363
llvm-svn: 301598
I thought I fixed the page size, but there were still errors.
This patch also contains fixes for grammatical errors.
Thanks pcc for proofreading!
llvm-svn: 301454
Summary: When using /msvclto, lld and MSVC's linker both do their own symbol resolution. This can cause them to select different archive members, which can result in undefined references. This change avoids that situation by extracting archive members that are selected by lld and passing those to link.exe before any archives, so that MSVC's uses those objects for symbol resolution instead of different archive members.
Reviewers: pcc, rnk, ruiu
Reviewed By: pcc
Subscribers: llvm-commits, mehdi_amini
Differential Revision: https://reviews.llvm.org/D32317
llvm-svn: 301045
The CONSTANT export type is marked as obsolete, but link still supports
this. Furthermore, WinObjC uses this for certain exports. Add support
for this export type.
llvm-svn: 301013
Summary:
Fixes PR32689: /msvclto creates response files with lines
that are too long for msvc's linker (LNK1170).
Reviewers: hans, rnk, ruiu
Subscribers: llvm-commits, mehdi_amini
Differential Revision: https://reviews.llvm.org/D32185
llvm-svn: 300612
Start using it in LLD to avoid needing to read bitcode again just to get the
target triple, and in llvm-lto2 to avoid printing symbol table information
that is inappropriate for the target.
Differential Revision: https://reviews.llvm.org/D32038
llvm-svn: 300300
Summary:
lld-link allows the number of parallel ThinLTO jobs to be specified
using /opt:lldltojobs=N. If left unspecified, the implementation
conservatively defaults to 1. This leads to very long link times. This
change makes it so that the default is to automatically set the
parallelism, as we do in the ELF linker.
Reviewers: ruiu, hans
Reviewed By: ruiu, hans
Subscribers: pcc, mehdi_amini, Prazek, llvm-commits
Differential Revision: https://reviews.llvm.org/D31986
llvm-svn: 300089
The /appcontainer flag indicates that the module may only be used inside
an application container (for isolation). This has been supported by
link.exe since Windows 8.0. It sets an additional bit in the PE DLL
Characteristics flag to indicate the behavioural change.
llvm-svn: 299728
Summary:
This adds support for reporting multiple errors in a single invocation of lld-link. The limit defaults to 20 and can be changed with the /ERRORLIMIT command line parameter, or set to unlimited by passing a value of 0.
This is a new attempt after r295507, which was reverted because opening files raced with exiting early, causing the test to be flaky. This version avoids the race by exiting before calling enqueuePath.
Reviewers: pcc, ruiu
Reviewed By: ruiu
Subscribers: llvm-commits, dblaikie
Differential Revision: https://reviews.llvm.org/D31688
llvm-svn: 299496
Summary:
The TypeTableBuilder provides stable storage for type records. We don't
need to copy all of the bytes into a flat vector before adding it to the
TpiStreamBuilder.
This makes addTypeRecord take an ArrayRef<uint8_t> and a hash code to go
with it, which seems like a simplification.
Reviewers: ruiu, zturner, inglorion
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D31634
llvm-svn: 299406
Introduce symbol table data structures that can be potentially written to
disk, have the LTO library build those data structures using temporarily
constructed modules and redirect the LTO library implementation to go through
those data structures. This allows us to remove the LLVMContext and Modules
owned by InputFile.
With this change I measured a peak memory consumption decrease from 5.4GB to
2.8GB in a no-op incremental ThinLTO link of Chromium on Linux. The impact on
memory consumption is larger in COFF linkers where we are currently forced
to materialize all metadata in order to read linker options. Peak memory
consumption linking a large piece of Chromium for Windows with full LTO and
debug info decreases from >64GB (OOM) to 15GB.
Part of PR27551.
Differential Revision: https://reviews.llvm.org/D31364
llvm-svn: 299168
Summary: In the ELF linker, we create the buffer identifier for bitcode files by appending the object name to the archive name. This change makes the COFF linker do the same. Without the change, ThinLTO builds can fail with an error message about multiple ThinLTO modules per object file, caused by object files contained in different archives having the same name.
Reviewers: pcc, ruiu
Reviewed By: pcc
Subscribers: mehdi_amini
Differential Revision: https://reviews.llvm.org/D31402
llvm-svn: 298942
Summary: MSVC does this when producing a PDB.
Reviewers: ruiu
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D31316
llvm-svn: 298717
This will be used in the sanitizer test suite, which wants to use DWARF
line tables.
At some point we should reconsider how LLD handles the long section
names required by DWARF debug sections.
llvm-svn: 298544
Summary:
This also delays setting the output filename based on the first input
argument until after processing /def.
Fixes PR32354
Reviewers: ruiu, pcc
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D31152
llvm-svn: 298327
In doing so, clean up the MD5 interface a little. Most
existing users only care about the lower 8 bytes of an MD5,
but for some users that care about the upper and lower,
there wasn't a good interface. Furthermore, consumers
of the MD5 checksum were required to handle endianness
details on their own, so it seems reasonable to abstract
this into a nicer interface that just gives you the right
value.
Differential Revision: https://reviews.llvm.org/D31105
llvm-svn: 298322
The Archive object owns the memory buffers of any thin archive members, so we
need to make sure the object is still in scope when we access archive members.
Differential Revision: https://reviews.llvm.org/D31066
llvm-svn: 298033
The MSVC linker doesn't like archive files containing non-native object
files.
When we are doing an LTO build, we may create archive files containing
both LLVM bitcode files and native object files. For example, if a
project contains assembly files and C++ files, we create native object
files for the assembly files and LLVM bitcode files for the C++ files.
With the /msvclto option, LLD passes archive files to the MSVC linker.
Previously, we didn't pass archive files if they contain at least one
bitcode files. That wasn't correct because the native object files that
weren't passed to the MSVC linker may be needed to complete linking.
In this patch, we create new temporary archive files to strip bitcode
files.
Differential Revision: https://reviews.llvm.org/D31053
llvm-svn: 297997
Previously we did not have support for writing detailed
module information for each module, as well as the symbol
records. This patch adds support for this, and in doing
so enables the ability to construct minimal PDBs from
just a few lines of YAML. A test is added to illustrate
this functionality.
llvm-svn: 297900
Previously, if you have foo=bar in a definition file, this assertion
could fire because when symbols are read from file they could be mangled.
It seems that due to historical reasons underscore mangling scheme is
really ad-hoc, and I cannot find a clean way to handle this. I had
to just de-mangle symbols to search again.
llvm-svn: 297357
Some archive files created during chromium build contains both BC
and native files. If that's the case, we want to pass the archive
file to link.exe. Otherwise, the MSVC linker would complain that
there's an unresolved symbol in a given set of files.
I cannot explain why link.exe doesn't complain about the presence
of bitcode files in this case, but it seems link.exe doesn't touch BC.
llvm-svn: 297229
If /msvclto is specified, we compile bitcode files and pass it to the
MSVC linker, stripping all bitcode files. We haven't stripped archive
files, because I was thinking that the MSVC linker wouldn't touch files
in archive files. When we pass an object file to link.exe, all symbols
have been resolved already, so link.exe shoulnd't need any of the files
in archives.
It turns out that even though link.exe doesn't need to do that, it
seems to try to read each file in all archives. And if there's a non-
COFF file in an archive, it exists with an error message. So we need
to remove archives from the command line too.
llvm-svn: 297191
Prior to MSVC 2015 we had to manually include this header any
time we were going to include <thread> or <future> due to a
bug in MSVC's STL implementation. This has been fixed in MSVC
for some time now, and we require VS 2015 minimum, so we can
remove this across all subprojects.
llvm-svn: 296906
After several smaller patches to get most of the core improvements
finished up, this patch is a straight move and header fixup of
the source.
Differential Revision: https://reviews.llvm.org/D30266
llvm-svn: 296810
Before the endianness was specified on each call to read
or write of the StreamReader / StreamWriter, but in practice
it's extremely rare for streams to have data encoded in
multiple different endiannesses, so we should optimize for the
99% use case.
This makes the code cleaner and more general, but otherwise
has NFC.
llvm-svn: 296415
This was reverted because it was breaking some builds, and
because of incorrect error code usage. Since the CL was
large and contained many different things, I'm resubmitting
it in pieces.
This portion is NFC, and consists of:
1) Renaming classes to follow a consistent naming convention.
2) Fixing the const-ness of the interface methods.
3) Adding detailed doxygen comments.
4) Fixing a few instances of passing `const BinaryStream& X`. These
are now passed as `BinaryStreamRef X`.
llvm-svn: 296394
r296215, "[PDB] General improvements to Stream library."
r296217, "Disable BinaryStreamTest.StreamReaderObject temporarily."
r296220, "Re-enable BinaryStreamTest.StreamReaderObject."
r296244, "[PDB] Disable some tests that are breaking bots."
r296249, "Add static_cast to silence -Wc++11-narrowing."
std::errc::no_buffer_space should be used for OS-oriented errors for socket transmission.
(Seek discussions around llvm/xray.)
I could substitute s/no_buffer_space/others/g, but I revert whole them ATM.
Could we define and use LLVM errors there?
llvm-svn: 296258
This adds various new functionality and cleanup surrounding the
use of the Stream library. Major changes include:
* Renaming of all classes for more consistency / meaningfulness
* Addition of some new methods for reading multiple values at once.
* Full suite of unit tests for reader / writer functionality.
* Full set of doxygen comments for all classes.
* Streams now store their own endianness.
* Fixed some bugs in a few of the classes that were discovered
by the unit tests.
llvm-svn: 296215
This is part of a larger effort to get the Stream code moved
up to Support. I don't want to do it in one large patch, in
part because the changes are so big that it will treat everything
as file deletions and add, losing history in the process.
Aside from that though, it's just a good idea in general to
make small changes.
So this change only changes the names of the Stream related
source files, and applies necessary source fix ups.
llvm-svn: 296211
I added this log message to test the /msvclto option, but
this output might confuse FileCheck. This patch attempts to fix
it by removing it.
llvm-svn: 295793
LLD is a multi-threaded program. errs() or outs() are not guaranteed
to be thread-safe (they are actually not).
LLD's message(), log() or error() are thread-safe. We should use them.
llvm-svn: 295787
Behavior races on ErrorCount. If the enqueued paths are evaluated
eagerly (in enqueuePath) then the behavior is as the test expects. But
they may not be evaluated until the future is waited on, in run() -
which is after the early return/exit on ErrorCount. (this causes the
test to fail (because in the "/ERRORCOUNT:XYZ" test, no other errors
are printed), at least for me, on linux)
This reverts commit r295507.
llvm-svn: 295590
Summary: This adds support for reporting multiple errors in a single invocation of lld-link. The limit defaults to 20 and can be changed with the /ERRORLIMIT command line parameter, or set to unlimited by passing a value of 0.
Reviewers: pcc, ruiu
Reviewed By: ruiu
Differential Revision: https://reviews.llvm.org/D29691
llvm-svn: 295507
Some PDBs or object files can contain references to other PDBs
where the real type information lives. When this happens,
all type indices in the original PDB are meaningless because
their records are not there.
With this patch we add the ability to pull type info from those
secondary PDBs.
Differential Revision: https://reviews.llvm.org/D29973
llvm-svn: 295382
LLVM defines `PTHREAD_LIB` which is used by AddLLVM.cmake and various projects
to correctly link the threading library when needed. Unfortunately
`PTHREAD_LIB` is defined by LLVM's `config-ix.cmake` file which isn't installed
and therefore can't be used when configuring out-of-tree builds. This causes
such builds to fail since `pthread` isn't being correctly linked.
This patch attempts to fix that problem by renaming and exporting
`LLVM_PTHREAD_LIB` as part of`LLVMConfig.cmake`. I renamed `PTHREAD_LIB`
because It seemed likely to cause collisions with downstream users of
`LLVMConfig.cmake`.
llvm-svn: 294690
Summary: This adds an option to save temporary files generated during link-time optimization. This can be useful for debugging.
Reviewers: ruiu, pcc
Reviewed By: ruiu, pcc
Subscribers: mehdi_amini
Differential Revision: https://reviews.llvm.org/D29518
llvm-svn: 294498
If `/debugtypes` is used to omit the codeview information, we would not
have constructed the debug info codeview record which is used to tie the
PDB to the binary. In such a case, rub out the GUID and Age fields.
llvm-svn: 294279
This patch defines a new command line option, /MSVCLTO, to LLD.
If that option is given, LLD invokes link.exe to link LTO-generated
object files. This is hacky but useful because link.exe can create
PDB files.
Differential Revision: https://reviews.llvm.org/D29526
llvm-svn: 294234
Summary: The COFF linker previously implemented link-time optimization using an API which has now been marked as legacy. This change refactors the COFF linker to use the new LTO API, which is also used by the ELF linker.
Reviewers: pcc, ruiu
Reviewed By: pcc
Subscribers: mgorny, mehdi_amini
Differential Revision: https://reviews.llvm.org/D29059
llvm-svn: 293967
Previously, mergeTypeStreams returns only true or false, so it was
impossible to know the reason if it failed. This patch changes the
function signature so that it returns an Error object.
Differential Revision: https://reviews.llvm.org/D29362
llvm-svn: 293820
Previously, we were printing out something like this for
sections/symbols with alignment 16
0000000000201000 0000000000000182 10 .data
which I think confusing. I think printing it in decimal is better.
Differential Revision: https://reviews.llvm.org/D29258
llvm-svn: 293685
Summary: This copies over some functionality we have in ELF/Error.{cpp,h} and makes it available in COFF/Error.{cpp,h}
Reviewers: pcc, rafael, ruiu
Subscribers:
Differential Revision: https://reviews.llvm.org/D28692
llvm-svn: 292240
Previous code had a bug that if the program exits with an assert() or
fail() before the control reaches end of writeMapFile(), it leaves a
temporary file, because FileRemover's dtor isn't called in that case.
I could fix that by removeFileOnSignal() and other functions, but
I think we can simply write to the result file directly. I think
that is straightforward and easy to understand.
Additionally, that allows something like `-Map /dev/null` or a bash
hack such as `-Map >(grep symbol-im-looking-for)`. Previously,
that kind of things didn't work.
Differential Revision: https://reviews.llvm.org/D28714
llvm-svn: 292041
Summary: When we encouter a relocation type we don't know how to handle, this change causes us to print out the hexadecimal value of the relocation type. This makes troubleshooting a little easier.
Reviewers: ruiu, zturner
Differential Revision: https://reviews.llvm.org/D28576
llvm-svn: 291962
This patch is to merge type info in multiple .debug$T sections.
One mystery that needs to be solved is that it is not clear how
the MSVC linker uses TPI and IPI streams. Both streams contain
type info, and it is not obvious what kind of record should go
which.
dumppdb command in microsoft-pdb repository prints out IPI stream
contents as "IDs" and TPI stream as "TYPES", but looks like the tool
don't really care about which stream type recrods were read from.
For now, in this patch, I emit all type records to TPI stream.
It might just work with other tools. If not, we need to investigate
it more.
llvm-svn: 291739
Previously the type dumper itself was passed around to a lot of different
places and manipulated in ways that were more appropriate on the type
database. For example, the entire TypeDumper was passed into the symbol
dumper, when all the symbol dumper wanted to do was lookup the name of a
TypeIndex so it could print it. That's what the TypeDatabase is for --
mapping type indices to names.
Another example is how if the user runs llvm-pdbdump with the option to
dump symbols but not types, we still have to visit all types so that we
can print minimal information about the type of a symbol, but just without
dumping full symbol records. The way we did this before is by hacking it
up so that we run everything through the type dumper with a null printer,
so that the output goes to /dev/null. But really, we don't need to dump
anything, all we want to do is build the type database. Since
TypeDatabaseVisitor now exists independently of TypeDumper, we can do
this. We just build a custom visitor callback pipeline that includes a
database visitor but not a dumper.
All the hackery around printers etc goes away. After this patch, we could
probably even delete the entire CVTypeDumper class since really all it is
at this point is a thin wrapper that hides the details of how to build a
useful visitation pipeline. It's not a priority though, so CVTypeDumper
remains for now.
After this patch we will be able to easily plug in a different style of
type dumper by only implementing the proper visitation methods to dump
one-line output and then sticking it on the pipeline.
Differential Revision: https://reviews.llvm.org/D28524
llvm-svn: 291724
We were starting to get some name clashes between llvm-pdbdump
and the common CodeView framework, so I took this opportunity
to rename a bunch of files to more accurately describe their
usage. This also helps in llvm-pdbdump to distinguish
between different files and whether they are used for pretty
dump mode or raw dump mode.
llvm-svn: 291627
This is how we use TarWriter in LLD. Now LLD does not append
a file extension, so you need to pass `--reproduce foo.tar`
instead of `--reproduce foo`.
Differential Revision: https://reviews.llvm.org/D28103
llvm-svn: 291210
The PDB GUID, Age, and version are tied together by the RSDS record in
the binary. Pass along the BuildId information into the createPDB to
allow us to tie the binary and the PDB together.
llvm-svn: 290975
Assert that the size of the MD5 result is the same size as the signature
field being populated. Use the sizeof operator to determine the size of
the field being written rather than hardcoding it to the magic number
16. NFC.
llvm-svn: 290764
I thought for a while about how to remove it, but it looks like we
can just copy the file for now. Of course I'm not happy about that,
but it's just less than 50 lines of code, and we already have
duplicate code in Error.h and some other places. I want to solve
them all at once later.
Differential Revision: https://reviews.llvm.org/D27819
llvm-svn: 290062
File system operations were still dominating the profile on Windows. In this
case we were spending a significant amount of our time repeatedly searching
for libraries as a result of processing linker directives. Address this
by caching whether we have already found a library with a given name. For
chrome_child.dll:
Before: 10.53s
After: 6.88s
Differential Revision: https://reviews.llvm.org/D27840
llvm-svn: 289915
Profiling revealed that the majority of lld's execution time on Windows was
spent opening and mapping input files. We can reduce this cost significantly
by performing these operations asynchronously.
This change introduces a queue for all operations on input file data. When
we discover that we need to load a file (for example, when we find a lazy
archive for an undefined symbol, or when we read a linker directive to
load a file from disk), the file operation is launched using a future and
the symbol resolution operation is enqueued. This implies another change
to symbol resolution semantics, but it seems to be harmless ("ninja All"
in Chromium still succeeds).
To measure the perf impact of this change I linked Chromium's chrome_child.dll
with both thin and fat archives.
Thin archives:
Before (median of 5 runs): 19.50s
After: 10.93s
Fat archives:
Before: 12.00s
After: 9.90s
On Linux I found that doing this asynchronously had a negative effect on
performance, probably because the cost of mapping a file is small enough that
it becomes outweighed by the cost of managing the futures. So on non-Windows
platforms I use the deferred execution strategy.
Differential Revision: https://reviews.llvm.org/D27768
llvm-svn: 289760
Enable building lld as a standalone project. This is motivated by the desire to
package lld for inclusion in a linux distribution. This allows building lld
against an existing paired llvm installation. Now that lld is usable on x86_64,
it makes sense to revive this configuration to allow distributions to package
it.
llvm-svn: 289421
This patch replaces the symbol table's object and archive queues, as well as
the convergent loop in the linker driver, with a design more similar to the
ELF linker where symbol resolution directly causes input files to be added to
the link, including input files arising from linker directives. Effectively
this removes the last vestiges of the old parallel input file loader.
Differential Revision: https://reviews.llvm.org/D27660
llvm-svn: 289409
Using a set here caused us to take about 1 second longer to write the symbol
table when linking chrome_child.dll. With this I consistently get better
performance on Windows with the new symbol table.
Before r289280 and with r289183 reverted (median of 5 runs): 17.65s
After this change: 17.33s
On Linux things look even better:
Before: 10.700480444s
After: 5.735681610s
Differential Revision: https://reviews.llvm.org/D27648
llvm-svn: 289408
This ports the ELF linker's symbol table design, introduced in r268178,
to the COFF linker.
Differential Revision: http://reviews.llvm.org/D21166
llvm-svn: 289280
The former option bases the filename on the output name, e.g. if the
link output is a.exe, the map will be written to a.map. This matches the
behaviour of link.exe's /MAP option and is useful for creating a map
file of each executable when building a large project.
Differential Revision: https://reviews.llvm.org/D27595
llvm-svn: 289271
Profiling revealed that we were spending 5% of our time linking
chrome_child.dll just in this call to toString().
Differential Revision: https://reviews.llvm.org/D27628
llvm-svn: 289270
I don't think the data I add to a TPI stream in this patch is correct,
but at least it can be displayed using llvm-pdbdump. Until I add more
streams to a PDB file, I'm not able to know whether the data will be
accepted by MSVC tools or not.
llvm-svn: 289183
Previously, we had different way to stringize SymbolBody and InputFile
to construct error messages. This patch defines overloaded function
toString() so that we don't need to memorize all these different
function names.
With that change, it is now easy to include demangled names in error
messages. Now, if there is a symbol name conflict, we'll print out
both mangled and demangled names.
llvm-svn: 288992
LLD used to take 11.73 seconds to link Clang. Now it is 6.94 seconds.
MSVC link takes 83.02 seconds. Note that ICF is enabled by default on
Windows, so a low latency ICF is more important than in ELF.
llvm-svn: 288487
Associative sections are sections that need to be linked if their associated
sections are linked. Associative sections are used to append auxiliary data
such as debug info.
Previously, we compared all associative sections when comparing two comdat
sections. Because usually assocative sections are not mergeable sections,
we missed a lot of mergeable sections. MSVC linker doesn't seem to check
the identity of associative sections.
This patch makes LLD to ignore associative sections when doing ICF.
llvm-svn: 288483
rL287555 introduces a link error when building with BUILD_SHARED_LIBS:
undefined reference to llvm::codeview::CVSymbolDumper::dump(),
and more...
The functions are available in libDebugInfoCodeView, from LLVM.
Patch by Visoiu Mistrih Francis!
llvm-svn: 287837
Previously, we discarded .debug$ sections. This patch adds them to
files so that PDB.cpp can access them.
This patch also adds a debug option, /dumppdb, to dump debug info
fed to createPDB so that we can verify that valid data has been passed.
llvm-svn: 287555
This patch updates a couple places where add_dependencies was being explicitly called to add dependencies on intrinsics_gen to instead use the DEPENDS named parameter. This cleanup is needed for a patch I'm working on to add a dependency debugging mode to the build system.
llvm-svn: 287205
createManifestRes was generating a MemoryBuffer from a TemporaryFile,
keeping the data but removing the file, before passing the file path
to CVTRES.exe, leading to the following error:
CVTRES : fatal error CVT1101: cannot open 'C:\Users\user\AppData\
Local\Temp\lld-output-resource-bfee19.res' for reading
With this, we instead create a new TemporaryFile before passing it to cvtres.
Patch from Rudy Pons!
llvm-svn: 287034
Object files compiled with cl.exe /GL contain intermediate code for LTO.
We can't (and don't want to) interpret such code, but we should print
out a user-friendly error message.
Differential Revision: https://reviews.llvm.org/D26647
llvm-svn: 286921
The MSVC linker relies on this invariant to produce a valid import
table. More ASan tests pass in a stage 2 build now. They still fail when
using LLD since there are no PDBs for the dynamic ASan runtime.
llvm-svn: 286499
Following the lazy reference might bring in an object file that depends
on bitcode files that weren't part of the LTO step.
Differential Revision: https://reviews.llvm.org/D25461
llvm-svn: 283989
With this, "llvm-pdbdump yaml -ipi-stream" prints out an IPI stream.
Previously it crashed because it can't handle the case where IPI
stream doesn't exist.
llvm-svn: 283392
I do not fully understand how to use these classes yet, but
seems like these arguments are not used, since without them
all tests still pass. In order to simplify the situation,
I'll remove them now.
llvm-svn: 283174
Handle this in the exact same way as IMAGE_REL_AMD64_SECREL
and IMAGE_REL_I386_SECREL.
Differential revision: https://reviews.llvm.org/D24608
llvm-svn: 282531
So that it is clear that FileOutputBuffer does not depend on
PDB file builder. Eventually we will have to to get the file size
info from the file builder to create a file with the exact size.
NFC.
llvm-svn: 282454
Change the way we calculate the build id to use MD5 to give reproducible build
ids. Previously we would generate random bytes for the build id GUID.
llvm-svn: 281079
Previously, we created temporary files using llvm::sys::fs::createTemporaryFile
and removed them using llvm::FileRemover. This is error-prone as it is easy to
forget creating FileRemover instances after creating temporary files.
There is actually a temporary file leak bug.
This patch introduces a new class, TemporaryFile, to manage temporary files
in the RAII style.
Differential Revision: https://reviews.llvm.org/D24176
llvm-svn: 280510
Summary:
UBSan complains like the following:
tools/lld/COFF/Writer.cpp:97:15: runtime error: null pointer passed as argument 2, which is declared to never be null
The reason is that the vector could be empty.
Reviewers: rsmith
Subscribers: Eugene.Zelenko, kcc
Differential Revision: https://reviews.llvm.org/D24050
llvm-svn: 280259
The IMAGE_FILE_HEADER structure contains a (RVA, size) to an array of
COFF_DEBUG_DIRECTORY records. Each one of these records contains an RVA to a OMF
Debug Directory. These OMF debug directories are derived into newer types such
as PDB70, PDB20, etc. This constructs a PDB70 structure which will allow us to
associate a GUID with a build to actually tie debug information.
llvm-svn: 280012
Reorder the table setup to mirror the indices corresponding to them. This means
that the table values are filled out as per the enumeration ordering. Doing so
makes it easier to identify a particular table. NFC.
llvm-svn: 278199
Add the support infrastructure for the /debugtype option which takes a comma
delimited list of debug info to generate. The defaults are based on other
options potentially (/driver or /profile). This sets up the infrastructure to
allow us to emit RSDS records to get "build id" equivalents on COFF (similar to
binutils).
llvm-svn: 278056
Don't blindly OR in the new value, but clear the existing one, since it can be
nonzero. Read out the existing value before, and add into the desired offset.
(The add is done outside of the applyMOV, to handle potential overflow between
the two.)
Patch by Martin Storsjö!
llvm-svn: 277846
The opcode for the bl branches can initially be F000 F800, i.e.
the J1 and J2 bits are already set. Therefore mask these bits out
before or'ing in the new bits.
Patch by Martin Storsjö!
llvm-svn: 277836
This flag is implemented similarly to --reproduce in the ELF linker.
This patch implements /linkrepro by moving the cpio writer and associated
utility functions to lldCore, and using that implementation in both linkers.
One COFF-specific detail is that we store the object file from which the
resource files were created in our reproducer, rather than the resource
files themselves. This allows the reproducer to be used on non-Windows
systems for example.
Differential Revision: https://reviews.llvm.org/D22418
llvm-svn: 276719
lld currently relies on lib.exe in order to generate an empty import library.
The "empty" import library consists of 5 members:
- first linker member
- second linker member
- Import Descriptor
- NULL Import Descriptor
- NULl Thunk
The first two entries (first and second linker members) are string tables which
are never updated. Therefore, they may as well as not be present. A subsequent
change to add that is probably warranted. However, this does not prevent the
use of the linker.
The Import Descriptor is the content which is most important. It provides an
Import Name Table entry for the library (as specified by the LIBRARY directive
in the DEF file). Additionally, it contains undefined references to the NULL
Import Descriptor and the library NULL Thunk Data. This ensures that the linker
will pull in the subsequent objects from the import library for the link. The
Import Descriptor has a single symbol (__IMPORT_DESCRIPTOR_<Library>) which
contains 3 relocations, one to the INT (Import Name Table) entry, one to the ILT
(Import Lookup Table) entry, and one to the IAT (Import Address Table) entry.
The NULL Import Descriptor is the last import descriptor and terminates the
import descriptor array. It contains a single symbol
(__NULL_IMPORT_DESCRIPTOR).
The NULL Thunk contains a single symbol (\x7f<Library>_NULL_THUNK_DATA) and
provides the terminator for the ILT and IAT.
These files are currently constructed manually following the example of the
Short Import Library format. This is arguably less than ideal, and it may be
possible to use MCAssembler and feed it the fragments to construct the object.
The major difference between the LIB (LINK) generated objects and the ones
generated here is that they are all one section shorter (.debug$S) as they do
not contain the debug information and one symbol shorter (@comp.id) as they do
not contain the RICH signature.
Move the logic related to the librarian into a new source file (Librarian.cpp).
llvm-svn: 275242
Manifest file is a separate or embedded XML file having metadata
of an executable. As it is XML, it can contain various types of
information. Probably the most popular one is to request escalated
priviledges.
Usually the linker creates an XML file and embed that file into
an executable. However, there's a way to supply an XML file from
command line. /manifestniput is it.
Apparently it is over-designed here, but if you supply two or more
manifest files, then the linker needs to merge the files into a
single XML file. A good news is that we don't need to do that ourselves.
MT.exe command can do that, so we call the command from the linker
in this patch.
llvm-svn: 266704
With the llvm change in r265606 this is the matching needed change to the lld
code now that createBinary() is returning Expected<...> .
llvm-svn: 265607
This flag disables link.exe's crash handler so that normal windows error
reporting and crash dumping occurs. For now it is reasonable for LLD to
ignore the flag.
Chromium is currently using this flag to collect minidumps of link.exe
crashing, and it breaks the LLD build.
llvm-svn: 264439
Some declarations of memcpy (like glibc's for example) are attributed
with notnull which makes it UB for NULL to get passed in, even if the
memcpy count is zero.
To account for this, guard the memcpy with an appropriate precondition.
This should fix the last UBSan bug, exposed by the test suite, in the
COFF linker.
llvm-svn: 263919
LLD type-punned an integral type and a pointer type using a pointer
field. This is problematic because the pointer type has alignment
greater than some of the integral values.
This would be less problematic if a union was used but it turns out the
integral values are only present for a short, transient, amount of time.
Let's remove this undefined behavior by skipping the punning altogether
by storing the state in a separate memory location: a vector which
informs us which symbols to process for weak externs.
llvm-svn: 263918
This fixes a test which exposed an ASan issue.
We assumed that a symbol's section number had a corresponding section
without performing validation.
llvm-svn: 263558
The load configuration directory is a structure whose size varies as the
OS gains additional functionality. To account for this, the structure's
layout begins with a size field; this allows loaders to know which
fields are available.
However, LLD hard-coded the sizes (112 bytes for 64-bit and 64 for
32-bit). This means that we might not inform the loader of all the
pertinent fields or we might claim that there are more fields than are
actually present.
To correctly account for this, the size field must be loaded from the
_load_config_used symbol.
N.B. The COFF spec is either wrong or out of date, the load
configuration directory is not correctly documented in the
specification: it omits the size field.
llvm-svn: 263543
The TLS directory has a different layout depending on the bitness of the
machine the image will run on. LLD would always use the 64-bit TLS
directory for the data directory entry instead of an appropriately sized
TLS directory.
llvm-svn: 263539
Now that DarwinLdDriver is the only derived class of Driver.
This patch merges them and actually removed the class because
they can now just be non-member functions. This change simplifies
a common header, Driver.h.
http://reviews.llvm.org/D17788
llvm-svn: 262502
DLL export tables usually contain dllexport'ed symbol RVAs so that
applications which use the DLLs can find symbols from the DLLs.
However, there's a minor feature to "forward" DLL symbols to other
DLLs.
If you set an RVA to a string whose form is "<dllname>.<symbolname>"
(e.g. "KERNEL32.ExitProcess") instead of symbol RVA to the export
table, the loader interprets that as a forwarder symbol, and resolve
that symbol from the specified DLL.
This patch implements that feature.
llvm-svn: 257243
In a UI such as XCode, LLVM source files are in 'libraries' while clang
files are in 'clang libraries'.
This change moves the lld source to 'lld libraries' to make code browsing easier.
It should be NFC as the build itself is still the same, just the structure in a
UI differs.
llvm-svn: 257001
MSVC linker considers PDB files created with this patch valid.
So you don't have to remove PDB files created by lld before
running MSVC linker.
This patch has no test since llvm-pdbdump dislikes PDB files
with no metadata streams.
llvm-svn: 255039
Before this patch, we created an empty PDB file if /debug option is
specified. For MSVC linker, such PDB file is completely broken, and
linker exits without doing anything as soon as it finds an empty PDB
file.
A PDB file created in this patch has the correct file signature.
MSVC linker still thinks that the file is broken, but it then removes
and replaces with its output.
This is an initial patch to support PDB in LLD. We aim to support
PDB in order to make it 100% compatible with MSVC linker. PDB support
is the last missing piece.
llvm-svn: 254796
If a section symbol is not external, that COMDAT section should never
be merge with other sections in other compilation unit. Previously,
we didn't take visibility into account.
Note that COMDAT sections with non-external visibility makes sense
because they can be removed by dead-stripping.
Fixes https://llvm.org/bugs/show_bug.cgi?id=25686
llvm-svn: 254578
There was a threading issue in the ICF code for COFF. That seems like
a venign bug in the sense that it doesn't produce an incorrect output,
but it oftentimes misses reducible sections. As a result, mergeable
sections could remain in outputs, which makes the output nondeterministic.
Basically the algorithm we are using for ICF is this: We group sections
so that identical sections will eventually be in the same group. Initially,
all sections are in one group. We split the group by relocation targets
until we get a convergence (if relocation targets are in different gruops,
the sections are different). Once a group is split, they will never be
merged.
Each section has a group ID. That variable itself is atomic, so there's
no threading issue at the level that we can use thread sanitizer.
The point is, when we split a group, we re-assign new group IDs to group
of sections. That are multiple separate writes to atomic varaibles.
Thus, splitting a group is not an atomic operation, and there's a small
chance that the other thread observes inconsistent group IDs.
Over-splitting is always "safe", so it will never create incorrect output.
I suspect that the nondeterminism stems from that point. However, I
cannot prove or fix that at this moment, so I'm going to avoid using
threads here.
llvm-svn: 251300
There's actually a room to improve this patch. Instead of not merging
sections that have different alignements, we can choose the section that
has the largest alignment requirement among all sections that are otherwise
considered the same. Then all section alignments are satisfied, so we can
merge them.
I don't know if that improvement could make any difference for real-world
input, so I'll leave it alone. Would be interesting to revisit later.
llvm-svn: 248581
This is an LLD extension to MSVC link.exe command line. MSVC linker
does not write symbol tables for executables. We do unless no /debug
option is given.
There's a situation that we want to enable debug info but don't want
to emit the symbol table. One example is when we are comparing output
file size. With this patch, you can tell the linker to not create
a symbol table by just specifying /nosymtab.
llvm-svn: 248225
std::distance(C->Relocs.end(), C->Relocs.begin()) is the same as NumRelocs
which is already added to the hash value. What we are missing here is the
section size.
llvm-svn: 248202
This patch fixes a regression introduced by r247964. Relocations that
are referring the same symbol should be considered equal, but they
were not if they were pointing to non-section chunks.
llvm-svn: 248132
Previously, InputFile::parse() was run in batch. We construct a list
of all input files and call parse() on each file using parallel_for_each.
That means we cannot start parsing files until we get a complete list
of input files, although InputFile::parse() is safe to call from anywhere.
This patch makes it asynchronous. As soon as we add a file to the symbol
table, we now start parsing the file using std::async().
This change shortens self-hosting time (650 ms) by 28 ms. It's about 4%
improvement.
llvm-svn: 248109
I made the field an atomic pointer in hope that we would be able to
parallelize the symbol resolver soon, but that's not going to happen
soon. This patch reverts that change for the sake of readability.
llvm-svn: 248104
Zachary Turner