Commit Graph

519 Commits

Author SHA1 Message Date
Zachary Turner 0119e38491 Fix some of the PDB tests.
They were unintentionally calling DIA directly, which requires
Windows.  We need to pass the -native flag, and this then required
fixing up one or two tests.

llvm-svn: 341731
2018-09-07 23:36:08 +00:00
Zachary Turner da4b63ab9a [PDB] Support pointer types in the native reader.
In order to start testing this, I've added a new mode to
llvm-pdbutil which is only really useful for writing tests.
It just dumps the value of raw fields in record format.
This isn't really ideal and it won't allow us to test some
important cases, but it's better than nothing for now.

llvm-svn: 341729
2018-09-07 23:21:33 +00:00
Zachary Turner 5d629966a9 [PDB] Rename some files in the native reader.
By calling these NativeType<foo>.cpp, they will all be sorted
together, and it also distinguishes the types from the symbols.

llvm-svn: 341609
2018-09-07 00:12:56 +00:00
Zachary Turner 8ab7dd6028 [PDB] Create a SymbolCache class.
Part of the responsibility of the native PDB reader is to cache
symbols the first time they are accessed, so they can then be
looked up by an ID.  Furthermore, we need to resolve type indices
to records that we vend to the user, and other things.  Previously
this code was all thrown together a bit haphazardly in the native
session class, but it makes sense to collect all of this into a
single class whose sole responsibility is to manage the collection
of known symbols.

llvm-svn: 341608
2018-09-07 00:12:34 +00:00
Zachary Turner 5cda1b802d Fix some warnings.
llvm-svn: 341508
2018-09-06 00:06:20 +00:00
Zachary Turner 7999b4fa48 [PDB] Refactor the PDB symbol classes to fix a reuse bug.
The way DIA SDK works is that when you request a symbol, it
gets assigned an internal identifier that is unique for the
life of the session.  You can then use this identifier to
get back the same symbol, with all of the same internal state
that it had before, even if you "destroyed" the original
copy of the object you had.

This didn't work properly in our native implementation, and
if you destroyed an object for a particular symbol, then
requested the same symbol again, it would get assigned a new
ID and you'd get a fresh copy of the object.  In order to fix
this some refactoring had to happen to properly reuse cached
objects.  Some unittests are added to verify that symbol
reuse is taking place, making use of the new unittest input
feature.

llvm-svn: 341503
2018-09-05 23:30:38 +00:00
Alexandre Ganea 6a7efef4af [DebugInfo] Common behavior for error types
Following D50807, and heading towards D50664, this intermediary change does the following:

1. Upgrade all custom Error types in llvm/trunk/lib/DebugInfo/ to use the new StringError behavior (D50807).
2. Implement std::is_error_code_enum and make_error_code() for DebugInfo error enumerations.
3. Rename GenericError -> PDBError (the file will be renamed in a subsequent commit)
4. Update custom error messages to follow the same formatting: (\w\s*)+\.
5. Keep generic "file not found" (ENOENT) errors as they are in PDB code. Previously, there used to be a custom enumeration for that purpose.
6. Remove a few extraneous LF in log() implementations. Printing LF is a responsability at a higher level, not at the error level.

Differential Revision: https://reviews.llvm.org/D51499

llvm-svn: 341228
2018-08-31 17:41:58 +00:00
Reid Kleckner bd5d71229d [codeview] Use push_macro to avoid conflicts instead of a prefix
Summary:
This prefix was added in r333421, and it changed our dumper output to
say things like "CVRegEAX" instead of just "EAX". That's a functional
change that I'd rather avoid.

I tested GCC, Clang, and MSVC, and all of them support #pragma
push_macro. They don't issue warnings whem the macro is not defined
either.

I don't have a Mac so I can't test the real termios.h header, but I
looked at the termios.h sources online and looked for other conflicts.
I saw only the CR* macros, so those are the ones we work around.

Reviewers: zturner, JDevlieghere

Subscribers: hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D50851

llvm-svn: 339907
2018-08-16 17:34:31 +00:00
Alexandre Ganea 741cc3531a [llvm-pdbutil] Support PDBs without a DBI stream
Differential Revision: https://reviews.llvm.org/D50258

llvm-svn: 339045
2018-08-06 19:35:00 +00:00
Mandeep Singh Grang 20239b18bb [llvm] Change 2 instances of std::sort to llvm::sort
llvm-svn: 337192
2018-07-16 17:26:37 +00:00
Benjamin Kramer 9fc944ae36 [PDB] memicmp only exists on Windows, use StringRef::compare_lower instead
llvm-svn: 336469
2018-07-06 21:56:57 +00:00
Zachary Turner 648bebdc67 [PDB] One more fix for hasing GSI records.
The reference implementation uses a case-insensitive string
comparison for strings of equal length.  This will cause the
string "tEo" to compare less than "VUo".  However we were using
a case sensitive comparison, which would generate the opposite
outcome.  Switch to a case insensitive comparison.  Also, when
one of the strings contains non-ascii characters, fallback to
a straight memcmp.

The only way to really test this is with a DIA test.  Before this
patch, the test will fail (but succeed if link.exe is used instead
of lld-link).  After the patch, it succeeds even with lld-link.

llvm-svn: 336464
2018-07-06 21:01:42 +00:00
Zachary Turner 1f200adfa7 [PDB] Sort globals symbols by name in GSI hash buckets.
It seems like the debugger first computes a symbol's bucket,
and then does a binary search of entries in the bucket using the
symbol's name in order to find it.  If the bucket entries are not
in sorted order, this obviously won't work.  After this patch a
couple of simple test cases show that we generate an exactly
identical GSI hash stream, which is very nice.

llvm-svn: 336405
2018-07-06 02:33:58 +00:00
Zachary Turner ee8010abe3 Move some code from PDBFileBuilder to MSFBuilder.
The code to emit the pieces of the MSF file were actually in
PDBFileBuilder.  Move this to MSFBuilder so that we can
theoretically emit an MSF without having a PDB file.

llvm-svn: 335789
2018-06-27 21:18:15 +00:00
Jonas Devlieghere 43dce3edbe [CodeView] Add prefix to CodeView registers.
Adds CVReg to CodeView register names to prevent a duplicate symbol with
CR3 defined in termios.h, as suggested by Zachary on the mailing list.

http://lists.llvm.org/pipermail/llvm-dev/2018-May/123372.html

Differential revision: https://reviews.llvm.org/D47478

rdar://39863705

llvm-svn: 333421
2018-05-29 14:35:34 +00:00
Zachary Turner 194be871b9 [LLD/PDB] Emit first section contribution for DBI Module Descriptor.
Part of the DBI stream is a list of variable length structures
describing each module that contributes to the final executable.

One member of this structure is a section contribution entry that
describes the first section contribution in the output file for
the given module.

We have been leaving this structure unpopulated until now, so with
this patch it is now filled out correctly.

Differential Revision: https://reviews.llvm.org/D45832

llvm-svn: 330457
2018-04-20 18:00:46 +00:00
Zachary Turner bee6c22414 [llvm-pdbutil] Dump first section contribution for each module.
The DBI stream contains a list of module descriptors.  At the
beginning of each descriptor is a structure representing the first
section contribution in the output file for that module.  LLD
currently doesn't fill out this structure at all, but link.exe
does.  So as a precursor to emitting this data in LLD, we first
need a way to dump it so that it can be checked.

This patch adds support for the dumping, and verifies via a test
that LLD emits bogus information.

llvm-svn: 330208
2018-04-17 20:06:43 +00:00
Zachary Turner d8d97de514 [PDB] Correctly use the target machine when writing DBI stream.
Using Config->is64() will treat ARM64 as Amd64, which is incorrect.
Furthermore, there are more esoteric architectures that could
theoretically be encountered.  Just set it directly to the machine
type, which we already know anyway.

llvm-svn: 330157
2018-04-16 20:42:06 +00:00
Zachary Turner e3fe669855 Resubmit "Fix some incorrect fields in our generated PDBs."
This fixes the failing tests.  They simply hadn't been updated
to match the new output resulting from this patch.

llvm-svn: 330145
2018-04-16 18:17:13 +00:00
Zachary Turner 52c80e3860 Revert "Fix some incorrect fields in our generated PDBs."
There are a couple of failing tests which slipped under my radar
so I'm reverting this while I attempt to fix.

llvm-svn: 330133
2018-04-16 16:55:41 +00:00
Zachary Turner 1b06cc7817 Fix some incorrect fields in our generated PDBs.
Most of these are pretty trivial and obvious. Setting the toolchain
version to 14.11 is perhaps a little questionable, but we've been bitten
in the past where one of our version fields sidn't match MSVC's, and I
definitely don't want to go through that diagnosis again as it was
pretty time consuming and hard to track down.

I found all of these by using llvm-pdbutil export to dump the dbi and
pdb streams to a file, then using fc followed by llvm-pdbutil explain to
explain the mismatched bytes.

There are still some more, these are just the low hanging fruit.

Differential Revision: https://reviews.llvm.org/D45276

llvm-svn: 330130
2018-04-16 16:27:49 +00:00
Aaron Smith 3dca0bedbb [DebugInfoPDB] Add DIA implementations of findSymbolByRVA and findSymbolByAddr
llvm-svn: 329724
2018-04-10 17:33:18 +00:00
Aaron Smith c0a5c01aeb [PDB] Remove dead code and run clang format; NFC
llvm-svn: 329712
2018-04-10 15:25:04 +00:00
Hiroshi Inoue 9ff2380ea6 [NFC] fix trivial typos in comments and error message
"is is" -> "is", "are are" -> "are"

llvm-svn: 329546
2018-04-09 04:37:53 +00:00
Zachary Turner 15b2bdfd8b [llvm-pdbutil] Add the ability to explain binary files.
Using this, you can use llvm-pdbutil to export the contents of a
stream to a binary file, then run explain on the binary file so
that it treats the offset as an offset into the stream instead
of an offset into a file.  This makes it easy to compare the
contents of the same stream from two different files.

llvm-svn: 329207
2018-04-04 17:29:09 +00:00
Nico Weber 086b1c8118 Minor no-op cmake file style fix.
llvm-svn: 329137
2018-04-04 00:50:22 +00:00
Aaron Smith 47f18b91bb [DebugInfoPDB] Add a few missing definitions to PDBTypes.h
The missing definitions are from cvconst.h shipped with DIA SDK.

Correct the url to MSDN for MemoryTypeEnum and set the underlying 
type of PDB_StackFrameType and PDB_MemoryType to uint16_t.

llvm-svn: 329104
2018-04-03 19:41:27 +00:00
Zachary Turner d11328a1bb [llvm-pdbutil] Add an export subcommand.
This command can dump the binary contents of a stream to a file.
This is useful when you want to do side-by-side comparisons of
a specific stream from two PDBs to examine the differences between
them.  You can export both of them to a file, then open them up
side by side in a hex editor (for example), so as to eliminate any
differences that might arise from the contents being on different
blocks in the PDB.

In subsequent patches I plan to improve the "explain" subcommand
so that you can explain the contents of a binary file that isn't
necessarily a full PDB, but one of these dumped streams, by telling
the subcommand how to interpret the contents.

llvm-svn: 329002
2018-04-02 18:35:21 +00:00
Zachary Turner d5cf5cf637 [llvm-pdbutil] Dig deeper into the PDB and DBI streams when explaining.
This will show more detail when using `llvm-pdbutil explain` on an
offset in the DBI or PDB streams.  Specifically, it will dig into
individual header fields and substreams to give a more precise
description of what the byte represents.

llvm-svn: 328878
2018-03-30 17:16:50 +00:00
Aaron Smith f13938382c [DebugInfoPDB] Print the method name along with the variant value
Before this change, using dumpProperties() with PDBSymbolData
would look like this:

  get_locationType: 3
  1

After this change:

  get_locationType: 3
  get_value: 1

llvm-svn: 328590
2018-03-26 22:53:38 +00:00
Aaron Smith 1af50bcf89 [DebugInfoPDB] Add methods to get the compiland and line numbers with PDBSymbolData
llvm-svn: 328587
2018-03-26 22:17:12 +00:00
Aaron Smith ed81a9db29 [DebugInfoPDB] Add DIA implementation of findLineNumbersByRVA
This method is used to find line numbers for PDBSymbolData
that have an invalid virtual address.

llvm-svn: 328586
2018-03-26 22:13:22 +00:00
Aaron Smith 53708a5e9e [DebugInfoPDB] Add DIA implementation of addressForVA and addressForRVA
These are used in finding line numbers for PDBSymbolData

llvm-svn: 328585
2018-03-26 22:10:02 +00:00
Zachary Turner f228276262 [PDB] Resubmit "Support embedding natvis files in PDBs."
This was reverted several times due to what ultimately turned out
to be incompatibilities in our serialized hash table format.

Several changes went in prior to this to fix those issues since
they were more fundamental and independent of supporting injected
sources, so now that those are fixed this change should hopefully
pass.

llvm-svn: 328363
2018-03-23 19:57:25 +00:00
Zachary Turner a6fb536e5b [PDB] Make our PDBs look more like MS PDBs.
When investigating bugs in PDB generation, the first step is
often to do the same link with link.exe and then compare PDBs.

But comparing PDBs is hard because two completely different byte
sequences can both be correct, so it hampers the investigation when
you also have to spend time figuring out not just which bytes are
different, but also if the difference is meaningful.

This patch fixes a couple of cases related to string table emission,
hash table emission, and the order in which we emit strings that
makes more of our bytes the same as the bytes generated by MS PDBs.

Differential Revision: https://reviews.llvm.org/D44810

llvm-svn: 328348
2018-03-23 18:43:39 +00:00
Zachary Turner 71d36ad9f9 [Codeview/PDB] Rename some methods for clarity.
NFC, this just renames some methods to better express what they
do, and also adds a few helper methods to add some symmetry to the
API in a few places (for example there was a getStringFromId but not
a getIdFromString method in the string table).

llvm-svn: 328221
2018-03-22 17:37:28 +00:00
Aaron Smith 523de05a1f [DIA] Add IPDBSectionContrib interfaces and DIA implementation
To resolve symbol context at a particular address, we need to
determine the compiland for the address. We are able to determine
the parent compiland of PDBSymbolFunc, PDBSymbolTypeUDT,
PDBSymbolTypeEnum symbols indirectly through line information. 
However no such information is availabile for PDBSymbolData, 
i.e. variables.

The Section Contribution table from PDBs has information about
each compiland's contribution to sections by address. For example,
a piece of a contribution looks like,

  VA         RelativeVA  Sect No.  Offset    Length    Compiland
  14000087B0 000087B0    0001      000077B0  000000BB  exe_main.obj

So given an address, it's possible to determine its compiland with
this information.

llvm-svn: 328178
2018-03-22 04:08:15 +00:00
Aaron Smith 58a32a478f [PDB] Get more DIA table enumerators
Rename the original function and make it a static template.

llvm-svn: 328177
2018-03-22 03:57:06 +00:00
Zachary Turner eb62999455 [PDB] Don't ignore bucket 0 when writing the PDB string table.
The hash table is a list of buckets, and the *value* stored in
the bucket cannot be 0 since that is reserved.  However, the code
here was incorrectly skipping over the 0'th bucket entirely.
The 0'th bucket is perfectly fine, just none of these buckets
can contain the value 0.

As a result, whenever there was a string where hash(S) % Size
was equal to 0, we would write the value in the next bucket
instead.  We never caught this in our tests due to *another*
bug, which is that we would iterate the entire list of buckets
looking for the value, only using the hash value as a starting
point.  However, the real algorithm stops when it finds 0 in
a bucket since it takes that to mean "the item is not in the
hash table".

The unit test is updated to carefully construct a set of hash
values that will cause one item to hash to 0 mod bucket count,
and the reader is also updated to return an error indicating that
the item is not found when it encounters a 0 bucket.

llvm-svn: 328162
2018-03-21 22:23:59 +00:00
Reid Kleckner 8562c1a198 [PDB] Remove unused private variable, re-applying r327900 after relanding more natvis changes[4~
llvm-svn: 328156
2018-03-21 21:47:26 +00:00
Zachary Turner fced530650 Revert "Resubmit "Support embedding natvis files in PDBs.""
This is still failing on a different bot this time due to some
issue related to hashing absolute paths.  Reverting until I can
figure it out.

llvm-svn: 328014
2018-03-20 18:37:03 +00:00
Zachary Turner 132d7a134f Resubmit "Support embedding natvis files in PDBs."
The issue causing this to fail in certain configurations
should be fixed.

It was due to the fact that DIA apparently expects there to be
a null string at ID 1 in the string table.  I'm not sure why this
is important but it seems to make a difference, so set it.

llvm-svn: 328002
2018-03-20 17:06:39 +00:00
Aaron Smith da61120749 [PDB] Add a method to get the full path of the source file for PDBSymbolCompiland
Summary:
Redefine PDBSymbolCompiland::getSourceFileName() to return the filename (w/o directory) of the source file that is used to compile the compiland. This is because the result returned previously is ambiguous. It could be the filename, relative path or full path of the source file. 

Move the implementation of SymbolFilePDB::GetSourceFileNameForPDBCompiland() into a new method PDBSymbolCompiland::getSourceFileFullPath(). 

Reviewers: zturner, rnk, llvm-commits

Reviewed By: zturner

Differential Revision: https://reviews.llvm.org/D44458

llvm-svn: 327910
2018-03-19 21:20:04 +00:00
Aaron Smith 06173e8b46 [PDB] Add exclusive methods to derived symbol class
Summary: This commit adds two methods to the PDBSymboFunc class used in parsing symbols. getLineNumbers() is used to determine a Function symbol's declaration and getCompilandId() is used to initialize the SymbolContext field sc.comp_unit.

Reviewers: zturner, rnk, llvm-commits

Reviewed By: zturner

Differential Revision: https://reviews.llvm.org/D44457

llvm-svn: 327909
2018-03-19 21:18:39 +00:00
Zachary Turner a21558897b Revert "Support embedding natvis files in PDBs."
This is causing a test failure on a certain bot, so I'm removing
this temporarily until we can figure out the source of the error.

llvm-svn: 327903
2018-03-19 20:41:59 +00:00
Zachary Turner 426885b10c Remove an unused private variable.
llvm-svn: 327900
2018-03-19 20:22:48 +00:00
Zachary Turner de53aaf132 Support embedding natvis files in PDBs.
Natvis is a debug language supported by Visual Studio for
specifying custom visualizers.  The /NATVIS option is an
undocumented link.exe flag which will take a .natvis file
and "inject" it into the PDB.  This way, you can ship the
debug visualizers for a program along with the PDB, which
is very useful for postmortem debugging.

This is implemented by adding a new "named stream" to the
PDB with a special name of /src/files/<natvis file name>
and simply copying the contents of the xml into this file.

Additionally, we need to emit a single stream named
/src/headerblock which contains a hash table of embedded
files to records describing them.

This patch adds this functionality, including the /NATVIS
option to lld-link.

Differential Revision: https://reviews.llvm.org/D44328

llvm-svn: 327895
2018-03-19 19:53:51 +00:00
Zachary Turner edbcbe0b62 [PDB] Fix a bug where we were serializing hash tables incorrectly.
There was some code that tried to calculate the number of 4-byte
words required to hold N bits, but it was instead computing the
number of bytes required to hold N bits.  This was leading to
extraneous data being output into the hash table, which would
cause certain operations in DIA (the Microsoft PDB reader) to
fail.

llvm-svn: 327675
2018-03-15 22:31:00 +00:00
Zachary Turner ebf03f6c46 Refactor the PDB HashTable class.
It previously only worked when the key and value types were
both 4 byte integers.  We now have a use case for a non trivial
value type, so we need to extend it to support arbitrary value
types, which means templatizing it.

llvm-svn: 327647
2018-03-15 17:38:26 +00:00
Aaron Smith 40198f5905 [DebugInfo] Add a new method IPDBSession::findLineNumbersBySectOffset
Summary:
Some PDB symbols do not have a valid VA or RVA but have Addr by Section and Offset. For example, a variable in thread-local storage has the following properties:

     get_addressOffset: 0
     get_addressSection: 5
     get_lexicalParentId: 2
     get_name: g_tls
     get_symIndexId: 12
     get_typeId: 4
     get_dataKind: 6
     get_symTag: 7
     get_locationType: 2

This change provides a new method to locate line numbers by Section and Offset from those symbols.

Reviewers: zturner, rnk, llvm-commits

Subscribers: asmith, JDevlieghere

Differential Revision: https://reviews.llvm.org/D44407

llvm-svn: 327601
2018-03-15 06:04:51 +00:00
Zachary Turner 679aeadda1 [PDB] Support dumping injected sources via the DIA reader.
Injected sources are basically a way to add actual source file content
to your PDB. Presumably you could use this for shipping your source code
with your debug information, but in practice I can only find this being
used for embedding natvis files inside of PDBs.

In order to effectively test LLVM's natvis file injection, we need a way
to dump the injected sources of a PDB in a way that is authoritative
(i.e. based on Microsoft's understanding of the PDB format, and not
LLVM's). To this end, I've added support for dumping injected sources
via DIA. I made a PDB file that used the /natvis option to generate a
test case.

Differential Revision: https://reviews.llvm.org/D44405

llvm-svn: 327428
2018-03-13 17:46:06 +00:00
Aaron Smith 25409ddf2a [DebugInfoPDB] Add DIA implementation for getSrcLineOnTypeDefn
Summary: This helps to determine the line number for a PDB type with definition

Reviewers: zturner, llvm-commits, rnk

Reviewed By: zturner

Subscribers: rengolin, JDevlieghere

Differential Revision: https://reviews.llvm.org/D44119

llvm-svn: 326857
2018-03-07 00:33:09 +00:00
Zachary Turner c6a75a69f1 [PDB] Defer writing the build id until the rest of the PDB is written.
For now this is NFC, but this small refactor opens the door to
letting us embed a hash of the PDB in the build id field of the
PDB.

Differential Revision: https://reviews.llvm.org/D43913

llvm-svn: 326453
2018-03-01 18:00:29 +00:00
Aaron Smith 89a19ac38d [PDB] Check the result of setLoadAddress()
Summary: Change setLoadAddress() to return true or false on failure.

Reviewers: zturner, llvm-commits

Reviewed By: zturner

Differential Revision: https://reviews.llvm.org/D43638

llvm-svn: 325843
2018-02-23 00:02:27 +00:00
Aaron Smith 9161a6cb25 [PDB] Fix buildbot failure from missing include for DIAEnumLineNumbers
llvm-svn: 325826
2018-02-22 20:00:07 +00:00
Aaron Smith fbe65404fd [PDB] Implement more find methods for PDB symbols
Summary:
Add additional find methods on PDB raw symbols.

findChildrenByAddr()
findChildrenByVA()
findInlineFramesByAddr()
findInlineFramesByVA()
findInlineLines()
findInlineLinesByAddr()
findInlineLinesByRVA()
findInlineLinesByVA()




Reviewers: zturner, llvm-commits

Reviewed By: zturner

Differential Revision: https://reviews.llvm.org/D43637

llvm-svn: 325824
2018-02-22 19:47:43 +00:00
Zachary Turner cafd476836 Fix emission of PDB string table.
This was originally reported as a bug with the symptom being "cvdump
crashes when printing an LLD-linked PDB that has an S_FILESTATIC record
in it". After some additional investigation, I determined that this was
a symptom of a larger problem, and in fact the real problem was in the
way we emitted the global PDB string table. As evidence of this, you can
take any lld-generated PDB, run cvdump -stringtable on it, and it would
return no results.

My hypothesis was that cvdump could not *find* the string table to begin
with. Normally it would do this by looking in the "named stream map",
finding the string /names, and using its value as the stream index. If
this lookup fails, then cvdump would fail to load the string table.

To test this hypothesis, I looked at the name stream map generated by a
link.exe PDB, and I emitted exactly those bytes into an LLD-generated
PDB. Suddenly, cvdump could read our string table!

This code has always been hacky and we knew there was something we
didn't understand. After all, there were some comments to the effect of
"we have to emit strings in a specific order, otherwise things don't
work". The key to fixing this was finally understanding this.

The way it works is that it makes use of a generic serializable hash map
that maps integers to other integers. In this case, the "key" is the
offset into a buffer, and the value is the stream number. If you index
into the buffer at the offset specified by a given key, you find the
name. The underlying cause of all these problems is that we were using
the identity function for the hash. i.e. if a string's offset in the
buffer was 12, the hash value was 12. Instead, we need to hash the
string *at that offset*. There is an additional catch, in that we have
to compute the hash as a uint32 and then truncate it to uint16.

Making this work is a little bit annoying, because we use the same hash
table in other places as well, and normally just using the identity
function for the hash function is actually what's desired. I'm not
totally happy with the template goo I came up with, but it works in any
case.

The reason we never found this bug through our own testing is because we
were building a /parallel/ hash table (in the form of an
llvm::StringMap<>) and doing all of our lookups and "real" hash table
work against that. I deleted all of that code and now everything goes
through the real hash table. Then, to test it, I added a unit test which
adds 7 strings and queries the associated values. I test every possible
insertion order permutation of these 7 strings, to verify that it really
does work as expected.

Differential Revision: https://reviews.llvm.org/D43326

llvm-svn: 325386
2018-02-16 20:46:04 +00:00
Aaron Smith 53a1a1616c Fix pretty printing the unspecified param of a variadic function
Summary:
 - Fix a bug in PrettyBuiltinDumper that returns "void" as the name for
  an unspecified builtin type. Since the unspecified param of a variadic
  function is considered a builtin of unspecified type in PDBs, we set
  "..." for its name.

  - Provide a method to determine if a PDBSymbolFunc is variadic in
  PrettyFunctionDumper since PDBSymbolFunc::getArgument() doesn't return the
  last unspecified-type param.

  - Add a pretty-func-dumper.test to test pretty dumping of variadic
  functions.

Reviewers: zturner, llvm-commits

Reviewed By: zturner

Differential Revision: https://reviews.llvm.org/D41801

llvm-svn: 322608
2018-01-17 01:22:03 +00:00
Michael Zolotukhin 0c169bf7f7 Remove redundant includes from lib/DebugInfo.
llvm-svn: 320620
2017-12-13 21:30:49 +00:00
Zachary Turner bd159d32c4 Don't #include MemoryBuffer.h from Host.h.
It turns out this #include isn't used from Host.h anyway,
but by having it it causes circular include dependencies.
This issues only surfaced while I was working on a separate
patch, so I'm submitting this first so that it's independent
of the other, unrelated patch.

llvm-svn: 318489
2017-11-17 01:00:35 +00:00
Reid Kleckner b5d17d8d30 Fix my typo of PDB_TableType
llvm-svn: 318447
2017-11-16 19:41:12 +00:00
Reid Kleckner 4ca69bdac6 Fix -Wreturn-type falling off the end of a function in new DIA code
llvm-svn: 318444
2017-11-16 19:32:53 +00:00
Aaron Smith 89bca9e566 [DebugInfo/PDB] Adding getUndecoratedNameEx and IPDB interfaces for IDiaEnumTables and IDiaTable.
Initial changes to support debugging PE/COFF files with LLDB on Windows through DIA SDK.
There is another set of changes required on the LLDB side before this does anything.

Differential Revision: https://reviews.llvm.org/D39517

llvm-svn: 318403
2017-11-16 14:33:09 +00:00
Aaron Smith c6ef575909 Test commit. Add a missing dash to the standard llvm file header; NFC.
llvm-svn: 318400
2017-11-16 13:42:28 +00:00
Rafael Espindola e0df357dbd Convert FileOutputBuffer to Expected. NFC.
llvm-svn: 317649
2017-11-08 01:05:44 +00:00
Reid Kleckner 145090f124 [PDB] Handle an empty globals hash table with no buckets
llvm-svn: 316722
2017-10-27 00:45:51 +00:00
Peter Collingbourne 75257bc2ec COFF: Add type server pdb files to linkrepro tar file.
Differential Revision: https://reviews.llvm.org/D38977

llvm-svn: 316233
2017-10-20 19:48:26 +00:00
Hans Wennborg dc8d6f2527 Fix -Wcovered-switch-default warnings from r314821
llvm-svn: 314826
2017-10-03 18:44:12 +00:00
Hans Wennborg 660531085a CodeView: Provide a .def file with the register ids
The list of register ids was previously written out in a couple of dirrent
places. This puts it in a .def file and also adds a few more registers (e.g.
the x87 regs) which should lead to more readable dumps, but I didn't include
the whole list since that seems unnecessary.

X86_MC::initLLVMToSEHAndCVRegMapping is pretty ugly, but at least it's not
relying on magic constants anymore. The TODO of using tablegen still stands.

Differential revision: https://reviews.llvm.org/D38480

llvm-svn: 314821
2017-10-03 18:27:22 +00:00
Peter Collingbourne 9e26e97955 COFF: PDB: Allow multiple modules with the same name.
It is possible for two modules to have the same name if they are
archive members with the same name, or if we are doing LTO (in which
case all modules will have the name "lto.tmp").

Differential Revision: https://reviews.llvm.org/D37589

llvm-svn: 312744
2017-09-07 20:39:46 +00:00
Peter Collingbourne 8ad3aab4e5 Remove dead code. NFCI.
llvm-svn: 312740
2017-09-07 19:17:30 +00:00
Zachary Turner abb17cc084 [llvm-pdbutil] Support dumping CodeView from object files.
We have llvm-readobj for dumping CodeView from object files, and
llvm-pdbutil has always been more focused on PDB.  However,
llvm-pdbutil has a lot of useful options for summarizing debug
information in aggregate and presenting high level statistical
views.  Furthermore, it's arguably better as a testing tool since
we don't have to write tests to conform to a state-machine like
structure where you match multiple lines in succession, each
depending on a previous match.  llvm-pdbutil dumps much more
concisely, so it's possible to use single-line matches in many
cases where as with readobj tests you have to use multi-line
matches with an implicit state machine.

Because of this, I'm adding object file support to llvm-pdbutil.
In fact, this mirrors the cvdump tool from Microsoft, which also
supports both object files and pdb files.  In the future we could
perhaps rename this tool llvm-cvutil.

In the meantime, this allows us to deep dive into object files
the same way we already can with PDB files.

llvm-svn: 312358
2017-09-01 20:06:56 +00:00
Zachary Turner 99c6982bcd [llvm-pdbutil] Print detailed S_UDT stats.
This adds a new command line option, -udt-stats, which breaks
down the stats of S_UDT records.  These are one of the biggest
contributors to the size of /DEBUG:FASTLINK PDBs, so they need
some additional tools to be able to analyze their usage.  This
option will dig into each S_UDT record and determine what kind
of record it points to, and then break down the statistics by
the target type.  The goal here is to identify how our object
files differ from MSVC object files in S_UDT records, so that
we can output fewer of them and reach size parity.

llvm-svn: 312276
2017-08-31 20:43:22 +00:00
Zachary Turner d76dc2d31e [lld/pdb] Speed up construction of publics & globals addr map.
computeAddrMap function calls std::stable_sort with a comparison
function that computes deserialized symbols every time its called.
In the result deserializeAs<PublicSym32> is called 20-30 times per
symbol. It's much faster to calculate it beforehand and pass a
pointer to it to the comparison function.

Patch by Alex Telishev
Differential Revision: https://reviews.llvm.org/D36941

llvm-svn: 311373
2017-08-21 20:08:40 +00:00
Zachary Turner d1de2f4f5e [llvm-pdbutil] Add support for dumping detailed module stats.
This adds support for dumping a summary of module symbols
and CodeView debug chunks.  This option prints a table for
each module of all of the symbols that occurred in the module
and the number of times it occurred and total byte size.  Then
at the end it prints the totals for the entire file.

Additionally, this patch adds the -jmc (just my code) option,
which suppresses modules which are from external libraries or
linker imports, so that you can focus only on the object files
and libraries that originate from your own source code.

llvm-svn: 311338
2017-08-21 14:53:25 +00:00
Benjamin Kramer 49a49fe816 Move helper classes into anonymous namespaces.
No functionality change intended.

llvm-svn: 311288
2017-08-20 13:03:48 +00:00
Zachary Turner ee9906d884 [LLD/PDB] Write actual records to the globals stream.
Previously we were writing an empty globals stream.  Windows
tools interpret this as "private symbols are not present in
this PDB", even when they are, so we need to fix this.  Regardless,
without it we don't have information about global variables, so
we need to fix it anyway.  This patch does that.

With this patch, the "lm" command in WinDbg correctly reports
that we have private symbols available, but the "dv" command
still refuses to display local variables.

Differential Revision: https://reviews.llvm.org/D36535

llvm-svn: 310743
2017-08-11 19:00:03 +00:00
Zachary Turner 5448dabbdd [PDB] Fix an issue writing the publics stream.
In the refactor to merge the publics and globals stream, a bug
was introduced that wrote the wrong value for one of the fields
of the PublicsStreamHeader.  This caused debugging in WinDbg
to break.

We had no way of dumping any of these fields, so in addition to
fixing the bug I've added dumping support for them along with a
test that verifies the correct value is written.

llvm-svn: 310439
2017-08-09 04:23:59 +00:00
Zachary Turner 946204c83e [PDB] Merge Global and Publics Builders.
The publics stream and globals stream are very similar. They both
contain a list of hash buckets that refer into a single shared stream,
the symbol record stream. Because of the need for each builder to manage
both an independent hash stream as well as a single shared record
stream, making the two builders be independent entities is not the right
design. This patch merges them into a single class, of which only a
single instance is needed to create all 3 streams.  PublicsStreamBuilder
and GlobalsStreamBuilder are now merged into the single GSIStreamBuilder
class, which writes all 3 streams at once.

Note that this patch does not contain any functionality change. So we're
still not yet writing any records to the globals stream. All we're doing
is making it so that when we do start writing records to the globals,
this refactor won't have to be part of that patch.

Differential Revision: https://reviews.llvm.org/D36489

llvm-svn: 310438
2017-08-09 04:23:25 +00:00
Adrian McCarthy b41f03e768 Enable llvm-pdbutil to list enumerations using native PDB reader
This extends the native reader to enable llvm-pdbutil to list the enums in a
PDB and it includes a simple test. It does not yet list the values in the
enumerations, which requires an actual implementation of
NativeEnumSymbol::FindChildren.

To exercise this code, use a command like:

    llvm-pdbutil pretty -native -enums foo.pdb

Differential Revision: https://reviews.llvm.org/D35738

llvm-svn: 310144
2017-08-04 22:37:58 +00:00
Reid Kleckner 175af4bcc7 [PDB] Fix section contributions
Summary:
PDB section contributions are supposed to use output section indices and
offsets, not input section indices and offsets.

This allows the debugger to look up the index of the module that it
should look up in the modules stream for symbol information. With this
change, windbg can now find line tables, but it still cannot print local
variables.

Fixes PR34048

Reviewers: zturner

Subscribers: hiraditya, ruiu, llvm-commits

Differential Revision: https://reviews.llvm.org/D36285

llvm-svn: 309987
2017-08-03 21:15:09 +00:00
Zachary Turner 9fb9d71d3e [pdb/lld] Write a valid FPM.
The PDB reserves certain blocks for the FPM that describe which
blocks in the file are allocated and which are free.  We weren't
filling that out at all, and in some cases we were even stomping
it with incorrect data.  This patch writes a correct FPM.

Differential Revision: https://reviews.llvm.org/D36235

llvm-svn: 309896
2017-08-02 22:31:39 +00:00
Zachary Turner c3d8eec9e9 [pdbutil] Add a command to dump the FPM.
Recently problems have been discovered in the way we write the FPM
(free page map).  In order to fix this, we first need to establish
a baseline about what a correct FPM looks like using an MSVC
generated PDB, so that we can then make our own generated PDBs
match.  And in order to do this, the dumper needs a mode where it
can dump an FPM so that we can write tests for it.

This patch adds a command to dump the FPM, as well as a test against
a known-good PDB.

llvm-svn: 309894
2017-08-02 22:25:52 +00:00
Zachary Turner 8d927b6bf9 [lld/pdb] Add an empty globals stream.
We don't write any actual symbols to this stream yet, but for
now we just create the stream and hook it up to the appropriate
places and give it a valid header.

Differential Revision: https://reviews.llvm.org/D35290

llvm-svn: 309608
2017-07-31 19:36:08 +00:00
Reid Kleckner ef443296a4 [PDB] Initialize the std::array<ulittle32_t> used for the gsi bitmap
With ASan, we would write about 512 bytes of malloc fill value to the
PDB, with some random bits ORed in here and there. Dumping the PDB would
always fail reliably.

llvm-svn: 309331
2017-07-27 23:13:05 +00:00
Reid Kleckner eacdf04fdd [PDB] Write public symbol records and the publics hash table
Summary:
MSVC link.exe records all external symbol names in the publics stream.
It provides similar functionality to an ELF .symtab.

Reviewers: zturner, ruiu

Subscribers: hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D35871

llvm-svn: 309303
2017-07-27 18:25:59 +00:00
Reid Kleckner 037bcd9345 [PDB] Remove stale GSI.h header that I intended to remove in the previous commit
llvm-svn: 309069
2017-07-26 00:58:49 +00:00
Reid Kleckner 14d90fd05c [PDB] Improve GSI hash table dumping for publics and globals
The PDB "symbol stream" actually contains symbol records for the publics
and the globals stream. The globals and publics streams are essentially
hash tables that point into a single stream of records. In order to
match cvdump's behavior, we need to only dump symbol records referenced
from the hash table. This patch implements that, and then implements
global stream dumping, since it's just a subset of public stream
dumping.

Now we shouldn't see S_PROCREF or S_GDATA32 records when dumping
publics, and instead we should see those record in the globals stream.

llvm-svn: 309066
2017-07-26 00:40:36 +00:00
Tim Northover fe6be421a7 Revert "Debug: handle dumping the D language."
Reid beat me to it.

llvm-svn: 308902
2017-07-24 17:47:46 +00:00
Tim Northover c7bd8255b9 Debug: handle dumping the D language.
Mostly just to silence a warning about an unhandled case. There don't seem to
be any tests for this operator (at least that I could find).

llvm-svn: 308901
2017-07-24 17:39:44 +00:00
Reid Kleckner e2ba971302 Add missing case to switch
llvm-svn: 308894
2017-07-24 16:30:44 +00:00
Reid Kleckner c85041fe00 Fix DebugInfo/PDB build by adding missing changes
llvm-svn: 308765
2017-07-21 18:32:00 +00:00
Reid Kleckner 686f121a5d [PDB] Dump extra info about the publics stream
This includes the hash table, the address map, and the thunk table and
section offset table. The last two are only used for incremental
linking, which LLD doesn't support, so they are less interesting. The
hash table is particularly important to get right, since this is the one
of the streams that debuggers use to translate addresses to symbols.

llvm-svn: 308764
2017-07-21 18:28:55 +00:00
Reid Kleckner c50349d4c6 [PDB] Finish and simplify TPI hashing
Summary:
This removes the CVTypeVisitor updater and verifier classes. They were
made dead by the minimal type dumping refactoring. Replace them with a
single function that takes a type record and produces a hash. Call this
from the minimal type dumper and compare the hash.

I also noticed that the microsoft-pdb reference repository uses a basic
CRC32 for records that aren't special. We already have an implementation
of that CRC ready to use, because it's used in COFF for ICF.

I'll make LLD call this hashing utility in a follow-up change. We might
also consider using this same hash in type stream merging, so that we
don't have to hash our records twice.

Reviewers: inglorion, ruiu

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D35515

llvm-svn: 308240
2017-07-18 00:33:45 +00:00
Reid Kleckner 67653ee086 [codeview] Fix YAML for LF_TYPESERVER2 by hoisting PDB_UniqueId
Summary:
We were treating the GUIDs in TypeServer2Record as strings, and the
non-ASCII bytes in the GUID would not round-trip through YAML.

We already had the PDB_UniqueId type portably represent a Windows GUID,
but we need to hoist that up to the DebugInfo/CodeView library so that
we can use it in the TypeServer2Record as well as in PDB parsing code.

Reviewers: inglorion, amccarth

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D35495

llvm-svn: 308234
2017-07-17 23:59:44 +00:00
Reid Kleckner a842cd75e2 [codeview] Remove TypeServerHandler and PDBTypeServerHandler
Summary:
Instead of wiring these through the CVTypeVisitor interface, clients
should inspect the CVTypeArray before visiting it and potentially load
up the type server's TPI stream if they need it.

No tests relied on this functionality because LLD was the only client.

Reviewers: ruiu

Subscribers: mgorny, hiraditya, zturner, llvm-commits

Differential Revision: https://reviews.llvm.org/D35394

llvm-svn: 308212
2017-07-17 20:28:06 +00:00
Reid Kleckner 6597c28d76 [PDB] Fix type server handling for archives
Summary:
This fixes type indices for SDK or CRT static archives. Previously we'd
try to look next to the archive object file path, which would not exist
on the local machine.

Also error out if we can't resolve a type server record. Hypothetically
we can recover from this error by discarding debug info for this object,
but that is not yet implemented.

Reviewers: ruiu, amccarth

Subscribers: aprantl, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D35369

llvm-svn: 307946
2017-07-13 20:12:23 +00:00
Reid Kleckner 0962cb2e3a Fix non-Windows build after PDB native builtin type change
Some C++14 features slipped in along with an extra member qualification.

llvm-svn: 307835
2017-07-12 19:46:35 +00:00
Adrian McCarthy 8d090fc531 [PDB] Enable NativeSession to create symbols for built-in types on demand
Summary:
There is a reserved range of type indexes for built-in types (like integers).
This will create a symbol for a built-in type if the caller askes for one by
type index.  This is also plumbing for being able to recall symbols by type
index in general, but user-defined types will come in subsequent patches.

Reviewers: rnk, zturner

Subscribers: mgorny, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D35163

llvm-svn: 307834
2017-07-12 19:38:11 +00:00
Zachary Turner 7eaf1d96ad [lld/pdb] Create an empty public symbol record stream.
This is part of the continuing effort to increase parity between
LLD and MSVC PDBs.  link still doesn't like our PDBs, so the most
obvious thing to check was whether adding an empty publics stream
would get it to do something else.  It still fails in the same way
but at least this removes one more variable from the equation.
The next logical step would be to try creating an empty globals
stream.

Differential Revision: https://reviews.llvm.org/D35224

llvm-svn: 307598
2017-07-10 22:40:20 +00:00