Commit Graph

210 Commits

Author SHA1 Message Date
Rui Ueyama f52496e1e0 Rename SymbolBody -> Symbol
Now that we have only SymbolBody as the symbol class. So, "SymbolBody"
is a bit strange name now. This is a mechanical change generated by

  perl -i -pe s/SymbolBody/Symbol/g $(git grep -l SymbolBody lld/ELF lld/COFF)

nd clang-format-diff.

Differential Revision: https://reviews.llvm.org/D39459

llvm-svn: 317370
2017-11-03 21:21:47 +00:00
Rui Ueyama 616cd99194 [COFF] Merge Symbol and SymbolBody.
llvm-svn: 317007
2017-10-31 16:10:24 +00:00
Bob Haarman b8a59c8aa5 [lld] unified COFF and ELF error handling on new Common/ErrorHandler
Summary:
The COFF linker and the ELF linker have long had similar but separate
Error.h and Error.cpp files to implement error handling. This change
introduces new error handling code in Common/ErrorHandler.h, changes the
COFF and ELF linkers to use it, and removes the old, separate
implementations.

Reviewers: ruiu

Reviewed By: ruiu

Subscribers: smeenai, jyknight, emaste, sdardis, nemanjai, nhaehnle, mgorny, javed.absar, kbarton, fedor.sergeev, llvm-commits

Differential Revision: https://reviews.llvm.org/D39259

llvm-svn: 316624
2017-10-25 22:28:38 +00:00
Reid Kleckner 8aa32ffbad [codeview] Fix handling of S_HEAPALLOCSITE
The type index is from the TPI stream, not the IPI stream. Fix the
dumper, fix type index discovery, and add a test in LLD.

Also improve the log message we emit when we fail to rewrite type
indices in LLD. That's how I found this bug.

llvm-svn: 316461
2017-10-24 17:02:40 +00:00
Reid Kleckner 5d82f68d17 [PDB] Fix logging of bad type indices
llvm-svn: 316386
2017-10-23 22:44:51 +00:00
Peter Collingbourne 75257bc2ec COFF: Add type server pdb files to linkrepro tar file.
Differential Revision: https://reviews.llvm.org/D38977

llvm-svn: 316233
2017-10-20 19:48:26 +00:00
Reid Kleckner 0a2016471d [COFF] Remove unused variable NFC
llvm-svn: 313330
2017-09-15 01:07:08 +00:00
Zachary Turner 024323cb12 [LLD COFF/PDB] Incrementally update the build id.
Previously, our algorithm to compute a build id involved hashing the
executable and storing that as the GUID in the CV Debug Record chunk,
and setting the age to 1.

This breaks down in one very obvious case: a user adds some newlines to
a file, rebuilds, but changes nothing else. This causes new line
information and new file checksums to get written to the PDB, meaning
that the debug info is different, but the generated code would be the
same, so we would write the same build over again with an age of 1.

Anyone using a symbol cache would have a problem now, because the
debugger would open the executable, look at the age and guid, find a
matching PDB in the symbol cache and then load it. It would never copy
the new PDB to the symbol cache.

This patch implements the canonical Windows algorithm for updating
a build id, which is to check the existing executable first, and
re-use an existing GUID while bumping the age if it already
exists.

Differential Revision: https://reviews.llvm.org/D36758

llvm-svn: 310961
2017-08-15 21:31:41 +00:00
Zachary Turner 302dc8bccf [PDB] Ignore all S_UDT symbols when writing PDBs.
We don't have the right algorithm for copying S_UDT symbols
from object files to the globals stream, and having it wrong
is worse than not having it at all, since it breaks display
of local variables of UDT types (for example, "dv Foo" fails
in our current implementation, but succeeds if the S_UDT records
are omitted).  Omit them until we fix the algorithm.

llvm-svn: 310867
2017-08-14 18:44:58 +00:00
Zachary Turner b57884e818 Fix some broken tests.
These were pending in a separate patch but I forgot to squash them
before comitting, and this one didn't go through.

llvm-svn: 310764
2017-08-11 21:14:01 +00:00
Zachary Turner f6822c869c [pdb] Fix linker module symbols to work with dbgeng.
The linker module contains a symbol of type S_COMPILE3 which
contains various information about the compiler and linker used
to create the PDB, such as the name of the linker, the target
machine, and the linker version.  Interestingly, if we set the
version string to 0.0.0.0, then when trying to view local
variables WinDbg emits an error that private symbols are not
present.  By setting this to a valid MSVC linker version string,
local variables can display.

As such, even though it is not representative of LLVM's version
information, we need this for compatibility.

llvm-svn: 310755
2017-08-11 20:46:47 +00:00
Zachary Turner 28e31ee45e Output S_SECTION symbols to the Linker module.
PDBs need to contain 1 module for each object file/compiland,
and a special one synthesized by the linker.  This one contains
a symbol record for each output section in the executable with
its address information.  This patch adds such symbols to the
linker module.  Note that we also are supposed to add an
S_COFFGROUP symbol for what appears to be each input section that
contributes to each output section, but it's not entirely clear
how to generate these yet, so I'm leaving that for a separate
patch.

llvm-svn: 310754
2017-08-11 20:46:28 +00:00
Zachary Turner 964e096345 Remove unused lambda capture.
llvm-svn: 310752
2017-08-11 20:37:49 +00:00
Zachary Turner ee9906d884 [LLD/PDB] Write actual records to the globals stream.
Previously we were writing an empty globals stream.  Windows
tools interpret this as "private symbols are not present in
this PDB", even when they are, so we need to fix this.  Regardless,
without it we don't have information about global variables, so
we need to fix it anyway.  This patch does that.

With this patch, the "lm" command in WinDbg correctly reports
that we have private symbols available, but the "dv" command
still refuses to display local variables.

Differential Revision: https://reviews.llvm.org/D36535

llvm-svn: 310743
2017-08-11 19:00:03 +00:00
Zachary Turner 946204c83e [PDB] Merge Global and Publics Builders.
The publics stream and globals stream are very similar. They both
contain a list of hash buckets that refer into a single shared stream,
the symbol record stream. Because of the need for each builder to manage
both an independent hash stream as well as a single shared record
stream, making the two builders be independent entities is not the right
design. This patch merges them into a single class, of which only a
single instance is needed to create all 3 streams.  PublicsStreamBuilder
and GlobalsStreamBuilder are now merged into the single GSIStreamBuilder
class, which writes all 3 streams at once.

Note that this patch does not contain any functionality change. So we're
still not yet writing any records to the globals stream. All we're doing
is making it so that when we do start writing records to the globals,
this refactor won't have to be part of that patch.

Differential Revision: https://reviews.llvm.org/D36489

llvm-svn: 310438
2017-08-09 04:23:25 +00:00
Zachary Turner 59e3ae827d [PDB] Fix linking of function symbols and local variables.
The compiler outputs PROC32_ID symbols into the object files
for functions, and these symbols have an embedded type index
which, when copied to the PDB, refer to the IPI stream.  However,
the symbols themselves are also converted into regular symbols
(e.g. S_GPROC32_ID -> S_GPROC32), and type indices in the regular
symbol records refer to the TPI stream.  So this patch applies
two fixes to function records.
  1. It converts ID symbols to the proper non-ID record type.
  2. After remapping the type index from the object file's index
     space to the PDB file/IPI stream's index space, it then
     remaps that index to the TPI stream's index space by.

Besides functions, during the remapping process we were also
discarding symbol record types which we did not recognize.
In particular, we were discarding S_BPREL32 records, which is
what MSVC uses to describe local variables on the stack.  So
this patch fixes that as well by copying them to the PDB.

Differential Revision: https://reviews.llvm.org/D36426

llvm-svn: 310394
2017-08-08 18:34:44 +00:00
Zachary Turner 676386ff30 [lld] Write the DataCRC to the output PDB.
llvm-svn: 310297
2017-08-07 20:23:45 +00:00
Reid Kleckner 175af4bcc7 [PDB] Fix section contributions
Summary:
PDB section contributions are supposed to use output section indices and
offsets, not input section indices and offsets.

This allows the debugger to look up the index of the module that it
should look up in the modules stream for symbol information. With this
change, windbg can now find line tables, but it still cannot print local
variables.

Fixes PR34048

Reviewers: zturner

Subscribers: hiraditya, ruiu, llvm-commits

Differential Revision: https://reviews.llvm.org/D36285

llvm-svn: 309987
2017-08-03 21:15:09 +00:00
Zachary Turner 8d927b6bf9 [lld/pdb] Add an empty globals stream.
We don't write any actual symbols to this stream yet, but for
now we just create the stream and hook it up to the appropriate
places and give it a valid header.

Differential Revision: https://reviews.llvm.org/D35290

llvm-svn: 309608
2017-07-31 19:36:08 +00:00
Reid Kleckner eacdf04fdd [PDB] Write public symbol records and the publics hash table
Summary:
MSVC link.exe records all external symbol names in the publics stream.
It provides similar functionality to an ELF .symtab.

Reviewers: zturner, ruiu

Subscribers: hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D35871

llvm-svn: 309303
2017-07-27 18:25:59 +00:00
Rui Ueyama acd632d338 Add {Obj,Import,Bitcode}File::Instances to COFF input files.
We did the same thing for ELF in r309152, and I want to maintain
COFF and ELF as close as possible.

llvm-svn: 309239
2017-07-27 00:45:26 +00:00
Rui Ueyama e1b48e099c Rename ObjectFile ObjFile for COFF as well.
llvm-svn: 309228
2017-07-26 23:05:24 +00:00
Reid Kleckner d91ca7666f [PDB] Emit TPI and IPI hashes
llvm-svn: 308497
2017-07-19 17:26:07 +00:00
Reid Kleckner f8522d81c3 Fix pdb-type-server-simple test on non-Windows
Assume that the LF_TYPESERVER2 record contains Windows-style paths.  In
any case, 'sys::path::filename(Path, Style::windows)' will work on
Unix-style paths.

llvm-svn: 308241
2017-07-18 00:33:53 +00:00
Reid Kleckner 651db91c2d [PDB] Merge in types and items from type servers (/Zi)
Summary:
Object files compiled with /Zi emit type information into a type server
PDB. The .debug$S section will contain a single TypeServer2Record with
the absolute path and GUID of the type server. LLD needs to load the
type server PDB and merge all types and items it finds in it into the
destination PDB.

Depends on D35495

Reviewers: ruiu, inglorion

Subscribers: zturner, llvm-commits

Differential Revision: https://reviews.llvm.org/D35504

llvm-svn: 308235
2017-07-18 00:21:25 +00:00
Reid Kleckner 67653ee086 [codeview] Fix YAML for LF_TYPESERVER2 by hoisting PDB_UniqueId
Summary:
We were treating the GUIDs in TypeServer2Record as strings, and the
non-ASCII bytes in the GUID would not round-trip through YAML.

We already had the PDB_UniqueId type portably represent a Windows GUID,
but we need to hoist that up to the DebugInfo/CodeView library so that
we can use it in the TypeServer2Record as well as in PDB parsing code.

Reviewers: inglorion, amccarth

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D35495

llvm-svn: 308234
2017-07-17 23:59:44 +00:00
Reid Kleckner a842cd75e2 [codeview] Remove TypeServerHandler and PDBTypeServerHandler
Summary:
Instead of wiring these through the CVTypeVisitor interface, clients
should inspect the CVTypeArray before visiting it and potentially load
up the type server's TPI stream if they need it.

No tests relied on this functionality because LLD was the only client.

Reviewers: ruiu

Subscribers: mgorny, hiraditya, zturner, llvm-commits

Differential Revision: https://reviews.llvm.org/D35394

llvm-svn: 308212
2017-07-17 20:28:06 +00:00
Reid Kleckner 0faf6d7beb [PDB] Put LLD's PDB writing code in a class NFC
Summary:
We've accumulated about five or so data structures that are widely
referenced:
- PDBBuilder
- Type table
- Id table
- PDB string table
- Type server handler

I'm about to rewrite type server handling, and I need a new class in LLD
where I can put its state. By creating a new PDBLinker class, I hope to
put it there next.

Reviewers: ruiu

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D35392

llvm-svn: 307979
2017-07-14 00:14:58 +00:00
Reid Kleckner 6597c28d76 [PDB] Fix type server handling for archives
Summary:
This fixes type indices for SDK or CRT static archives. Previously we'd
try to look next to the archive object file path, which would not exist
on the local machine.

Also error out if we can't resolve a type server record. Hypothetically
we can recover from this error by discarding debug info for this object,
but that is not yet implemented.

Reviewers: ruiu, amccarth

Subscribers: aprantl, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D35369

llvm-svn: 307946
2017-07-13 20:12:23 +00:00
Reid Kleckner 03b5baf480 Re-land "[PDB] Use a more appropriate message for a fatal error"
This re-commits r307726 with valid line table relocations.

llvm-svn: 307829
2017-07-12 18:49:43 +00:00
Vitaly Buka e970c2b706 Revert "[PDB] Use a more appropriate message for a fatal error"
Revert "[PDB] Tweak bad type index error handling"

check-lld with asan detects use-after-poison.

This reverts commits r307733 and r307726.

llvm-svn: 307752
2017-07-12 04:34:17 +00:00
Reid Kleckner 992263c14e [PDB] Use a more appropriate message for a fatal error
llvm-svn: 307733
2017-07-11 23:44:51 +00:00
Reid Kleckner 2bf9ab2d62 [PDB] Tweak bad type index error handling
Translate invalid type indices to a sentinel value instead of skipping
the record. Skipping records isn't a good recovery method, because we
can skip a scope open or close record, which will confuse the scope
management code.

We currently have lots of invalid type indices on Microsoft-provided
standard libraries, because the LF_TYPESERVER2 records contain absolute
paths that are only valid on their build servers. Our type server
handlers need to look at other things (GUIDs) to find these type server
PDBs.

llvm-svn: 307726
2017-07-11 22:37:50 +00:00
Zachary Turner 7eaf1d96ad [lld/pdb] Create an empty public symbol record stream.
This is part of the continuing effort to increase parity between
LLD and MSVC PDBs.  link still doesn't like our PDBs, so the most
obvious thing to check was whether adding an empty publics stream
would get it to do something else.  It still fails in the same way
but at least this removes one more variable from the equation.
The next logical step would be to try creating an empty globals
stream.

Differential Revision: https://reviews.llvm.org/D35224

llvm-svn: 307598
2017-07-10 22:40:20 +00:00
Zachary Turner 77f23b9c21 Fix pdb-linker-module test on linux.
lld-link is apparently a symlink to lld.  Our pdb writing code
uses llvm::sys::fs::getMainExecutable() and writes that value
into the PDB, which will resolve symlinks etc.  But what we
really want is the exact value of argv[0], which matches what
the MS linker does.  Fix this by just writing argv[0] instead
of calling getMainExecutable.

llvm-svn: 307592
2017-07-10 21:09:11 +00:00
Zachary Turner 6708e0b45e [lld/pdb] Add some basic linker module symbols.
Differential Revision: https://reviews.llvm.org/D35152

llvm-svn: 307590
2017-07-10 21:01:37 +00:00
Zachary Turner 3a11fdf8ce [PDB] More changes to bring lld PDBs to parity with MSVC.
1) Don't write a /src/headerblock stream.  This appears to be
   written conditionally by MSVC, but it's not clear what the
   condition is.  For now, just remove it since we dont' know
   what it is anyway and the particular pdb we've checked in
   for the test doesn't have one.
2) Write a valid timestamp for the PDB file signature.  This
   leads to non-reproducible builds, but it matches the default
   behavior of link, so it should be out default as well.  If
   we need reproducibility, we should add a separate command
   line option for it that is off by default.
3) Write an empty FPO stream.  MSVC seems to always write an
   FPO stream.  This change makes the stream directory match
   up, although we still need to make the contents of the FPO
   stream match.

llvm-svn: 307436
2017-07-07 20:25:39 +00:00
Zachary Turner 448dea419c Use windows path syntax when writing PDB module name.
Without this we would just append whatever the user
wrote on the command line, so if we're in C:\foo
and we run lld-link bar/baz.obj, we would write
C:\foo\bar/baz.obj in various places in the PDB.
MSVC linker does not do this, so we shouldn't either.
This fixes some differences in the diff test, so we
update the test as well.

Differential Revision: https://reviews.llvm.org/D35092

llvm-svn: 307423
2017-07-07 18:46:14 +00:00
Zachary Turner c1e93e5fa4 Fix some differences between lld and MSVC generated PDBs.
A couple of things were different about our generated PDBs.

1) We were outputting the wrong Version on the PDB Stream.
   The version we were setting was newer than what MSVC is setting.
   It's not clear what the implications are, but we change LLD
   to use PdbImplVC70, as MSVC does.
2) For the optional debug stream indices in the DBI Stream, we
   were outputting 0 to mean "the stream is not present".  MSVC
   outputs uint16_t(-1), which is the "correct" way to specify
   that a stream is not present.  So we fix that as well.
3) We were setting the PDB Stream signature to 0.  This is supposed
   to be the result of calling time(nullptr).  Although this leads
   to non-deterministic builds, a better way to solve that is by
   having a command line option explicitly for generating a
   reproducible build, and have the default behavior of lld-link
   match the default behavior of link.

To test this, I'm making use of the new and improved `pdb diff`
sub command.  To make it suitable for writing tests against, I had
to modify the diff subcommand slightly to print less verbose output.
Previously it would always print | <column> | <value1> | <value2> |
which is quite verbose, and the values are fragile.  All we really
want to know is "did we produce the same value as link?"  So I added
command line options to print a single character representing the
result status (different, identical, equivalent), and another to
hide the value display.  Note that just inspecting the diff output
used to write the test, you can see some things that are obviously
wrong.  That is just reflective of the fact that this is the state
of affairs today, not that we're asserting that this is "correct".
We can use this as a starting point to discover differences, fix
them, and update the test.

Differential Revision: https://reviews.llvm.org/D35086

llvm-svn: 307422
2017-07-07 18:45:56 +00:00
Zachary Turner 6c4bfba8f3 [PDB] Teach libpdb to write DBI Stream ECNames.
Based strictly on the name, this seems to have something to do
width edit & continue.  The goal of this patch has nothing to do
with supporting edit and continue though.  msvc link.exe writes
very basic information into this area even when *not* compiling
with support for E&C, and so the goal here is to bring lld-link
to parity.  Since we cannot know what assumptions standard tools
make about the content of PDB files, we need to be as close as
possible.

This ECNames data structure is a standard PDB string hash table.
link.exe puts a single string into this hash table, which is the
full path to the PDB file on disk.  It then references this string
from the module descriptor for the compiler generated `* Linker *`
module.

With this patch, lld-link will generate the exact same sequence of
bytes as MSVC link for this subsection for a given object file
input (as reported by `llvm-pdbutil bytes -ec`).

llvm-svn: 307356
2017-07-07 05:04:36 +00:00
Reid Kleckner 3f85192930 [PDB] Fill in "Parent" and "End" fields of scope-like symbol records
Summary:
There are a variety of records that open scopes: function scopes, block
scopes, and inlined call site scopes. These symbol records contain
Parent and End fields with the offsets of other symbol records. The End
field contains the offset of the matching S_END or S_INLINESITE_END
record. The Parent field contains the offset of the parent record, or 0
if this is a top-level scope (i.e. a function).

With this change, `llvm-pdbutil pretty -all` no longer crashes on PDBs
produced by LLD. I haven't tried a real debugger yet.

Reviewers: zturner, ruiu

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D34898

llvm-svn: 307278
2017-07-06 16:39:32 +00:00
Reid Kleckner d0e6e24a53 [PDB] Add symbols to the PDB
Summary:
The main complexity in adding symbol records is that we need to
"relocate" all the type indices. Type indices do not have anything like
relocations, an opaque data structure describing where to find existing
type indices for fixups. The linker just has to "know" where the type
references are in the symbol records. I added an overload of
`discoverTypeIndices` that works on symbol records, and it seems to be
able to link the standard library.

Reviewers: zturner, ruiu

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D34432

llvm-svn: 305933
2017-06-21 17:25:56 +00:00
Reid Kleckner f5bb738f75 [PDB] Don't emit debug info associated with dead chunks
Summary:
Previously we didn't add debug info chunks to the SparseChunks array, so
they didn't participate in section GC. Now we do.

Reviewers: ruiu

Subscribers: aprantl, llvm-commits

Differential Revision: https://reviews.llvm.org/D34356

llvm-svn: 305811
2017-06-20 17:14:09 +00:00
Reid Kleckner adea0ce7ca Fix build and PDB test on Linux
llvm-svn: 305716
2017-06-19 17:27:31 +00:00
Reid Kleckner 44cdb10964 [PDB] Start emitting source file and line information
Summary:
This is a first step towards getting line info to show up in VS and
windbg. So far, only llvm-pdbutil can parse the PDBs that we produce.
cvdump doesn't like something about our file checksum tables. I'll have
to dig into that next.

This patch adds a new DebugSubsectionRecordBuilder which takes bytes
directly from some other producer, such as a linker, and sticks it into
the PDB. Line tables only need to be relocated. No data needs to be
rewritten.

File checksums and string tables, on the other hand, need to be re-done.

Reviewers: zturner, ruiu

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D34257

llvm-svn: 305713
2017-06-19 17:21:45 +00:00
Zachary Turner 7ba50b7151 Remove /dumppdb option from LLD.
This option is no longer needed.

Differential Revision: https://reviews.llvm.org/D34302

llvm-svn: 305610
2017-06-16 23:41:44 +00:00
Reid Kleckner 8cbdd0c0f2 [PDB] Add a module descriptor for every object file
Summary:
Expose the module descriptor index and fill it in for section
contributions.

Reviewers: zturner

Subscribers: llvm-commits, ruiu, hiraditya

Differential Revision: https://reviews.llvm.org/D34126

llvm-svn: 305296
2017-06-13 15:49:13 +00:00
Zachary Turner ebd3ae8371 [CodeView] Properly align symbol records on read/write.
Object files have symbol records not aligned to any particular
boundary (e.g. 1-byte aligned), while PDB files have symbol
records padded to 4-byte aligned boundaries.  Since they share
the same reading / writing code, we have to provide an option to
specify the alignment and propagate it up to the producer or
consumer who knows what the alignment is supposed to be for the
given container type.

Added a test for this by modifying the existing PDB -> YAML -> PDB
round-tripping code to round trip symbol records as well as types.

Differential Revision: https://reviews.llvm.org/D33785

llvm-svn: 304484
2017-06-01 21:52:41 +00:00
Zachary Turner 2897e0306e [lld] Fix a bug where we continually re-follow type servers.
Originally this was intended to be set up so that when linking
a PDB which refers to a type server, it would only visit the
PDB once, and on subsequent visitations it would just skip it
since all the records had already been added.

Due to some C++ scoping issues, this was not occurring and it
was revisiting the type server every time, which caused every
record to end up being thrown away on all subsequent visitations.

This doesn't affect the performance of linking clang-cl generated
object files because we don't use type servers, but when linking
object files and libraries generated with /Zi via MSVC, this means
only 1 object file has to be linked instead of N object files, so
the speedup is quite large.

llvm-svn: 303920
2017-05-25 21:16:03 +00:00
Zachary Turner bb64231d2d Don't do a full scan of the type stream before processing records.
LazyRandomTypeCollection is designed for random access, and in
order to provide this it lazily indexes ranges of types.  In the
case of types from an object file, there is no partial index
to build off of, so it has to index the full stream up front.
However, merging types only requires sequential access, and when
that is needed, this extra work is simply wasted.  Changing the
algorithm to work on sequential arrays of types rather than
random access type collections eliminates this up front scan.

llvm-svn: 303707
2017-05-24 00:26:27 +00:00
Zachary Turner d4136e945e Implement various flavors of type merging.
Previous algotirhm assumed that types and ids are in a single
unified stream.  For inputs that come from object files, this
is the case.  But if the input is already a PDB, or is the result
of a previous merge, then the types and ids will already have
been split up, in which case we need an algorithm that can
accept operate on independent streams of types and ids that
refer across stream boundaries to each other.

Differential Revision: https://reviews.llvm.org/D33417

llvm-svn: 303577
2017-05-22 21:07:43 +00:00
Zachary Turner 526f4f2aa8 Resubmit "[CodeView] Provide a common interface for type collections."
This was originally reverted because it was a breaking a bunch
of bots and the breakage was not surfacing on Windows.  After much
head-scratching this was ultimately traced back to a bug in the
lit test runner related to its pipe handling.  Now that the bug
in lit is fixed, Windows correctly reports these test failures,
and as such I have finally (hopefully) fixed all of them in this
patch.

llvm-svn: 303446
2017-05-19 19:26:58 +00:00
Zachary Turner 1dfcf8d92c Revert "[CodeView] Provide a common interface for type collections."
This is a squash of ~5 reverts of, well, pretty much everything
I did today.  Something is seriously broken with lit on Windows
right now, and as a result assertions that fire in tests are
triggering failures.  I've been breaking non-Windows bots all
day which has seriously confused me because all my tests have
been passing, and after running lit with -a to view the output
even on successful runs, I find out that the tool is crashing
and yet lit is still reporting it as a success!

At this point I don't even know where to start, so rather than
leave the tree broken for who knows how long, I will get this
back to green, and then once lit is fixed on Windows, hopefully
hopefully fix the remaining set of problems for real.

llvm-svn: 303409
2017-05-19 05:57:45 +00:00
Zachary Turner b32ec02b80 [CodeView] Raise the source to ID map out of the TypeStreamMerger.
This map will be needed to rewrite symbol streams after re-writing
the corresponding type streams.

llvm-svn: 303390
2017-05-18 23:04:08 +00:00
Zachary Turner 0c60f269fc [CodeView] Provide a common interface for type collections.
Right now we have multiple notions of things that represent collections of
types. Most commonly used are TypeDatabase, which is supposed to keep
mappings from TypeIndex to type name when reading a type stream, which
happens when reading PDBs. And also TypeTableBuilder, which is used to
build up a collection of types dynamically which we will later serialize
(i.e. when writing PDBs).

But often you just want to do some operation on a collection of types, and
you may want to do the same operation on any kind of collection. For
example, you might want to merge two TypeTableBuilders or you might want
to merge two type streams that you loaded from various files.

This dichotomy between reading and writing is responsible for a lot of the
existing code duplication and overlapping responsibilities in the existing
CodeView library classes. For example, after building up a
TypeTableBuilder with a bunch of type records, if we want to dump it we
have to re-invent a bunch of extra glue because our dumper takes a
TypeDatabase or a CVTypeArray, which are both incompatible with
TypeTableBuilder.

This patch introduces an abstract base class called TypeCollection which
is shared between the various type collection like things. Wherever we
previously stored a TypeDatabase& in some common class, we now store a
TypeCollection&.

The advantage of this is that all the details of how the collection are
implemented, such as lazy deserialization of partial type streams, is
completely transparent and you can just treat any collection of types the
same regardless of where it came from.

Differential Revision: https://reviews.llvm.org/D33293

llvm-svn: 303388
2017-05-18 23:03:06 +00:00
Zachary Turner 5634ccf476 lld Fixups for TypeDatabase changes.
llvm-svn: 302304
2017-05-05 22:06:06 +00:00
Rafael Espindola a0f30da62e Revert "Fix misnamed #include. Remove _NC suffix from ELF relocations TLSDESC_{LD64,ADD}_LO12"
This reverts commit r301964. It broke the build.

llvm-svn: 301969
2017-05-02 20:19:42 +00:00
Joel Jones f387eb62b9 Fix misnamed #include. Remove _NC suffix from ELF relocations TLSDESC_{LD64,ADD}_LO12
llvm-svn: 301964
2017-05-02 19:31:40 +00:00
Zachary Turner e204a6c9a3 Rename pdb::StringTable -> pdb::PDBStringTable.
With the forthcoming codeview::StringTable which a pdb::StringTable
would hold an instance of as one member, this ambiguity becomes
confusing.  Rename to PDBStringTable to avoid this.

llvm-svn: 301948
2017-05-02 18:00:13 +00:00
Reid Kleckner 13fc411e39 [PDB] Save one type record copy
Summary:
The TypeTableBuilder provides stable storage for type records. We don't
need to copy all of the bytes into a flat vector before adding it to the
TpiStreamBuilder.

This makes addTypeRecord take an ArrayRef<uint8_t> and a hash code to go
with it, which seems like a simplification.

Reviewers: ruiu, zturner, inglorion

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D31634

llvm-svn: 299406
2017-04-04 00:56:34 +00:00
Reid Kleckner 5d57752c81 [PDB] Split item and type records when merging type streams
Summary: MSVC does this when producing a PDB.

Reviewers: ruiu

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D31316

llvm-svn: 298717
2017-03-24 17:26:38 +00:00
Zachary Turner ea4e60754e [pdb] Write the module info and symbol record streams.
Previously we did not have support for writing detailed
module information for each module, as well as the symbol
records.  This patch adds support for this, and in doing
so enables the ability to construct minimal PDBs from
just a few lines of YAML.  A test is added to illustrate
this functionality.

llvm-svn: 297900
2017-03-15 22:18:53 +00:00
Zachary Turner d9dc2829ea [Support] Move Stream library from MSF -> Support.
After several smaller patches to get most of the core improvements
finished up, this patch is a straight move and header fixup of
the source.

Differential Revision: https://reviews.llvm.org/D30266

llvm-svn: 296810
2017-03-02 20:52:51 +00:00
Zachary Turner 695ed56ba5 [PDB] Make streams carry their own endianness.
Before the endianness was specified on each call to read
or write of the StreamReader / StreamWriter, but in practice
it's extremely rare for streams to have data encoded in
multiple different endiannesses, so we should optimize for the
99% use case.

This makes the code cleaner and more general, but otherwise
has NFC.

llvm-svn: 296415
2017-02-28 00:04:07 +00:00
Zachary Turner 120faca41b [PDB] Partial resubmit of r296215, which improved PDB Stream Library.
This was reverted because it was breaking some builds, and
because of incorrect error code usage.  Since the CL was
large and contained many different things, I'm resubmitting
it in pieces.

This portion is NFC, and consists of:

1) Renaming classes to follow a consistent naming convention.
2) Fixing the const-ness of the interface methods.
3) Adding detailed doxygen comments.
4) Fixing a few instances of passing `const BinaryStream& X`.  These
   are now passed as `BinaryStreamRef X`.

llvm-svn: 296394
2017-02-27 22:11:43 +00:00
NAKAMURA Takumi 05a75e40da Revert r296215, "[PDB] General improvements to Stream library." and followings.
r296215, "[PDB] General improvements to Stream library."
r296217, "Disable BinaryStreamTest.StreamReaderObject temporarily."
r296220, "Re-enable BinaryStreamTest.StreamReaderObject."
r296244, "[PDB] Disable some tests that are breaking bots."
r296249, "Add static_cast to silence -Wc++11-narrowing."

std::errc::no_buffer_space should be used for OS-oriented errors for socket transmission.
(Seek discussions around llvm/xray.)

I could substitute s/no_buffer_space/others/g, but I revert whole them ATM.

Could we define and use LLVM errors there?

llvm-svn: 296258
2017-02-25 17:04:23 +00:00
Zachary Turner af299ea5d4 [PDB] General improvements to Stream library.
This adds various new functionality and cleanup surrounding the
use of the Stream library.  Major changes include:

* Renaming of all classes for more consistency / meaningfulness
* Addition of some new methods for reading multiple values at once.
* Full suite of unit tests for reader / writer functionality.
* Full set of doxygen comments for all classes.
* Streams now store their own endianness.
* Fixed some bugs in a few of the classes that were discovered
  by the unit tests.

llvm-svn: 296215
2017-02-25 00:44:30 +00:00
Zachary Turner d2684b7969 [PDB] Rename Stream related source files.
This is part of a larger effort to get the Stream code moved
up to Support.  I don't want to do it in one large patch, in
part because the changes are so big that it will treat everything
as file deletions and add, losing history in the process.
Aside from that though, it's just a good idea in general to
make small changes.

So this change only changes the names of the Stream related
source files, and applies necessary source fix ups.

llvm-svn: 296211
2017-02-25 00:33:34 +00:00
Zachary Turner 7b327d051b [pdb] Add the ability to resolve TypeServer PDBs.
Some PDBs or object files can contain references to other PDBs
where the real type information lives.  When this happens,
all type indices in the original PDB are meaningless because
their records are not there.

With this patch we add the ability to pull type info from those
secondary PDBs.

Differential Revision: https://reviews.llvm.org/D29973

llvm-svn: 295382
2017-02-16 23:35:45 +00:00
Saleem Abdulrasool 0acd6dd6ce COFF: prevent nullptr dereference
If `/debugtypes` is used to omit the codeview information, we would not
have constructed the debug info codeview record which is used to tie the
PDB to the binary.  In such a case, rub out the GUID and Age fields.

llvm-svn: 294279
2017-02-07 04:28:02 +00:00
Rui Ueyama a9b29615fb Re-submit r293820: Return Error instead of bool from mergeTypeStreams().
llvm-svn: 293847
2017-02-02 00:47:10 +00:00
Rui Ueyama 7d07a1652d Revert r293820: Return Error instead of bool from mergeTypeStreams().
It broke buildbots.

llvm-svn: 293824
2017-02-01 22:28:43 +00:00
Rui Ueyama 00d4f49717 Return Error instead of bool from mergeTypeStreams().
Previously, mergeTypeStreams returns only true or false, so it was
impossible to know the reason if it failed. This patch changes the
function signature so that it returns an Error object.

Differential Revision: https://reviews.llvm.org/D29362

llvm-svn: 293820
2017-02-01 22:09:34 +00:00
Adrian McCarthy 6b6b8c4fb9 NFC: Rename (PDB) RawSession to NativeSession
This eliminates one overload on the term Raw.

Differential Revision: https://reviews.llvm.org/D29098

llvm-svn: 293104
2017-01-25 22:38:55 +00:00
Zachary Turner 760ad4da60 [pdb] Write the Named Stream mapping to Yaml and binary.
Differential Revision: https://reviews.llvm.org/D28919

llvm-svn: 292665
2017-01-20 22:42:09 +00:00
Rui Ueyama 5289662e4a PDB: Merge type info.
This patch is to merge type info in multiple .debug$T sections.

One mystery that needs to be solved is that it is not clear how
the MSVC linker uses TPI and IPI streams. Both streams contain
type info, and it is not obvious what kind of record should go
which.

dumppdb command in microsoft-pdb repository prints out IPI stream
contents as "IDs" and TPI stream as "TYPES", but looks like the tool
don't really care about which stream type recrods were read from.

For now, in this patch, I emit all type records to TPI stream.
It might just work with other tools. If not, we need to investigate
it more.

llvm-svn: 291739
2017-01-12 03:09:25 +00:00
Zachary Turner 629cb7d8cc [CodeView] Finish decoupling TypeDatabase from TypeDumper.
Previously the type dumper itself was passed around to a lot of different
places and manipulated in ways that were more appropriate on the type
database. For example, the entire TypeDumper was passed into the symbol
dumper, when all the symbol dumper wanted to do was lookup the name of a
TypeIndex so it could print it. That's what the TypeDatabase is for --
mapping type indices to names.

Another example is how if the user runs llvm-pdbdump with the option to
dump symbols but not types, we still have to visit all types so that we
can print minimal information about the type of a symbol, but just without
dumping full symbol records. The way we did this before is by hacking it
up so that we run everything through the type dumper with a null printer,
so that the output goes to /dev/null. But really, we don't need to dump
anything, all we want to do is build the type database. Since
TypeDatabaseVisitor now exists independently of TypeDumper, we can do
this. We just build a custom visitor callback pipeline that includes a
database visitor but not a dumper.

All the hackery around printers etc goes away. After this patch, we could
probably even delete the entire CVTypeDumper class since really all it is
at this point is a thin wrapper that hides the details of how to build a
useful visitation pipeline. It's not a priority though, so CVTypeDumper
remains for now.

After this patch we will be able to easily plug in a different style of
type dumper by only implementing the proper visitation methods to dump
one-line output and then sticking it on the pipeline.

Differential Revision: https://reviews.llvm.org/D28524

llvm-svn: 291724
2017-01-11 23:24:22 +00:00
Zachary Turner a9054ddd9c [CodeView/PDB] Rename a bunch of files.
We were starting to get some name clashes between llvm-pdbdump
and the common CodeView framework, so I took this opportunity
to rename a bunch of files to more accurately describe their
usage.  This also helps in llvm-pdbdump to distinguish
between different files and whether they are used for pretty
dump mode or raw dump mode.

llvm-svn: 291627
2017-01-11 00:35:43 +00:00
Saleem Abdulrasool df8a13b257 COFF: tie the execute and the PDB together
The PDB GUID, Age, and version are tied together by the RSDS record in
the binary.  Pass along the BuildId information into the createPDB to
allow us to tie the binary and the PDB together.

llvm-svn: 290975
2017-01-04 17:56:54 +00:00
Rui Ueyama 327705d17d Create a TPI stream only when /debugpdb is given.
Adding type records to TPI stream is too time consuming.
It is reported that linking chrome_child.dll took 5 minutes.

llvm-svn: 289330
2016-12-10 17:23:23 +00:00
Rui Ueyama 26186c7e0f Add type records to TPI stream.
I don't think the data I add to a TPI stream in this patch is correct,
but at least it can be displayed using llvm-pdbdump. Until I add more
streams to a PDB file, I'm not able to know whether the data will be
accepted by MSVC tools or not.

llvm-svn: 289183
2016-12-09 04:46:54 +00:00
Rui Ueyama c5cb737584 Dump Codeview type information correctly.
llvm-svn: 288298
2016-12-01 01:22:48 +00:00
Rui Ueyama 24625fd9b2 Dump not only type records but symbol records.
llvm-svn: 287723
2016-11-22 23:51:34 +00:00
Rui Ueyama be939b3ff6 Do plumbing work for CodeView debug info.
Previously, we discarded .debug$ sections. This patch adds them to
files so that PDB.cpp can access them.

This patch also adds a debug option, /dumppdb, to dump debug info
fed to createPDB so that we can verify that valid data has been passed.

llvm-svn: 287555
2016-11-21 17:22:35 +00:00
Rui Ueyama c91f716643 PDB: Add "* Linker *" module.
The added module contains nothing, but it is still useful as a test
to ensure that we are emitting modules that can be read back.

llvm-svn: 287071
2016-11-16 01:10:46 +00:00
Vitaly Buka 4cf112c6f2 Fix heap-use-after-free coff::createPDB
Summary:
getInputSections returns std::vector by value and ArrayRef
pointed to destroyed temporarily.

Reviewers: ruiu

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D26630

llvm-svn: 286880
2016-11-14 20:21:41 +00:00
Rui Ueyama 09e0b5f2c9 Emit Section Contributions.
Differential Revision: https://reviews.llvm.org/D26211

llvm-svn: 286684
2016-11-12 00:00:51 +00:00
Rui Ueyama 6294a240dc Fix -Wcast-qual warnings.
llvm-svn: 286001
2016-11-04 17:41:29 +00:00
Rui Ueyama 20df4ecf4f Re-commit r285607: Emit Section Map stream."
Removed some test fields that overspecified test conditions.

llvm-svn: 285638
2016-10-31 21:09:21 +00:00
Rui Ueyama e7b54c2ea5 Revert r285607: Emit Section Map stream.
That change broke buildbots.

llvm-svn: 285618
2016-10-31 18:50:13 +00:00
Rui Ueyama 86a1ef388b Emit Section Map stream.
This is how we use DbiStreamBuilder::addSectionMap.

Differential Revision: https://reviews.llvm.org/D26113

llvm-svn: 285607
2016-10-31 17:40:40 +00:00
Rui Ueyama 9f66f8277d Re-submit r283825: Add section header stream to PDB.
It was reverted because the change that depends on was reverted.
Now it was submitted as r283925, so we can submit this as well.

llvm-svn: 283926
2016-10-11 19:45:07 +00:00
Rui Ueyama 9aa4ab6f9b Revert "Add section header stream to PDB." because it depends on r283823.
The change this patch depends on was reverted.

llvm-svn: 283837
2016-10-11 01:01:40 +00:00
Rui Ueyama 55505954fe Add section header stream to PDB.
Differential Revision: https://reviews.llvm.org/D25357

llvm-svn: 283825
2016-10-10 23:44:10 +00:00
Rui Ueyama 1343fac3ca Add an empty DBI stream.
llvm-svn: 283504
2016-10-06 22:52:01 +00:00
Rui Ueyama 8d3fb5da7b Add exact number of streams for reserved stream #s.
llvm-svn: 283397
2016-10-05 22:08:58 +00:00
Rui Ueyama d381c9842b Add an empty IPI stream.
With this, "llvm-pdbdump yaml -ipi-stream" prints out an IPI stream.
Previously it crashed because it can't handle the case where IPI
stream doesn't exist.

llvm-svn: 283392
2016-10-05 21:37:25 +00:00
Rui Ueyama ab8e88b686 Remove unused parameter.
I do not fully understand how to use these classes yet, but
seems like these arguments are not used, since without them
all tests still pass. In order to simplify the situation,
I'll remove them now.

llvm-svn: 283174
2016-10-04 01:03:03 +00:00
Rui Ueyama 129795497d Change for LLVM upstream change r282944.
llvm-svn: 282945
2016-09-30 20:53:45 +00:00
Rui Ueyama 5c82eeaf23 Change for LLVM change r282940.
llvm-svn: 282942
2016-09-30 20:39:04 +00:00
Rui Ueyama 3e9d6bbad6 Create FileOutputBuffer lazily.
So that it is clear that FileOutputBuffer does not depend on
PDB file builder. Eventually we will have to to get the file size
info from the file builder to create a file with the exact size.
NFC.

llvm-svn: 282454
2016-09-26 23:53:55 +00:00
Rui Ueyama bb542b3f0c Partially fill Info stream with proper values.
llvm-svn: 281795
2016-09-16 22:51:17 +00:00
Rui Ueyama b28c6d495d Use functions in DebugInfoPDB to create dummy PDB file.
I don't think we are creating valid PDB file here,
but it is okay because we have never created valid PDBs before.

llvm-svn: 281704
2016-09-16 04:32:33 +00:00
Rui Ueyama 1d99ab3925 Create PDB.h and move code to remove unnecessary #includes.
llvm-svn: 281670
2016-09-15 22:24:51 +00:00
Rui Ueyama 7f38299919 Use SuperBlock struct defined in MSFCommon.h.
llvm-svn: 281643
2016-09-15 18:55:18 +00:00
Rui Ueyama 0d09a865c6 COFF: Remove `void error()` functions and use fatal instead.
This change makes the control flow more explicit.

llvm-svn: 275504
2016-07-15 00:40:46 +00:00
Rui Ueyama 1a3fd13776 COFF: Remove unnecessary explicit calls of Twine ctor.
llvm-svn: 275501
2016-07-14 23:43:36 +00:00
Rui Ueyama 7e75866c51 COFF: Rename non-noreturn error -> check.
The new name is consistent with ELF.

llvm-svn: 275499
2016-07-14 23:37:10 +00:00
Rui Ueyama 1763c0d3bc COFF: Create an empty but valid PDF file.
MSVC linker considers PDB files created with this patch valid.
So you don't have to remove PDB files created by lld before
running MSVC linker.

This patch has no test since llvm-pdbdump dislikes PDB files
with no metadata streams.

llvm-svn: 255039
2015-12-08 18:39:55 +00:00
Rui Ueyama e737824d8a COFF: Create a PDB file with the correct file signature.
Before this patch, we created an empty PDB file if /debug option is
specified. For MSVC linker, such PDB file is completely broken, and
linker exits without doing anything as soon as it finds an empty PDB
file.

A PDB file created in this patch has the correct file signature.
MSVC linker still thinks that the file is broken, but it then removes
and replaces with its output.

This is an initial patch to support PDB in LLD. We aim to support
PDB in order to make it 100% compatible with MSVC linker. PDB support
is the last missing piece.

llvm-svn: 254796
2015-12-04 23:11:05 +00:00