Commit Graph

1521 Commits

Author SHA1 Message Date
Paul Robinson 96545db374 [DebugInfo/DWARF] Remove redundant iterator type. NFC
llvm-svn: 338759
2018-08-02 19:29:38 +00:00
Paul Robinson 2c25f345d7 [DebugInfo/DWARF] [4/4] Unify handling of compile and type units. NFC
This is patch 4 of 4 NFC refactorings to handle type units and compile
units more consistently and with less concern about the object-file
section that they came from.

Patch 4 combines separate DWARFUnitVectors for compile and type units
into a single DWARFUnitVector that contains both.  For now the
implementation distinguishes compile units from type units by putting
all compile units at the front of the vector, reflecting the DWARF v4
distinction between .debug_info and .debug_types sections.  A future
patch will change this to allow the free mixing of unit kinds, as is
specified by DWARF v5.

Differential Revision: https://reviews.llvm.org/D49744

llvm-svn: 338633
2018-08-01 20:54:11 +00:00
Paul Robinson 11307fab93 [DebugInfo/DWARF] [3/4] Rename DWARFUnitSection to DWARFUnitVector. NFC
This is patch 3 of 4 NFC refactorings to handle type units and compile
units more consistently and with less concern about the object-file
section that they came from.

Patch 3 simply renames DWARFUnitSection to DWARFUnitVector, as the
object-file section of a unit is nearly irrelevant now.

Differential Revision: https://reviews.llvm.org/D49743

llvm-svn: 338632
2018-08-01 20:49:44 +00:00
Paul Robinson 7f33094486 [DebugInfo/DWARF] [2/4] Type units no longer in a std::deque. NFC
This is patch 2 of 4 NFC refactorings to handle type units and compile
units more consistently and with less concern about the object-file
section that they came from.

Patch 2 takes the existing std::deque<DWARFUnitSection> for type units
and makes it a simple DWARFUnitSection, simplifying the handling of
type units and making it more consistent with compile units.

Differential Revision: https://reviews.llvm.org/D49742

llvm-svn: 338629
2018-08-01 20:46:46 +00:00
Paul Robinson 143eaeab53 [DebugInfo/DWARF] [1/4] De-templatize DWARFUnitSection. NFC
This is patch 1 of 4 NFC refactorings to handle type units and compile
units more consistently and with less concern about the object-file
section that they came from.

Patch 1 replaces the templated DWARFUnitSection with a non-templated
version. That is, instead of being a SmallVector of pointers to a
specific unit kind, it is not a SmallVector of pointers to the base
class for both type and compile units.  Virtual methods are magic.

Differential Revision: https://reviews.llvm.org/D49741

llvm-svn: 338628
2018-08-01 20:43:47 +00:00
Victor Leschuk 58d3399d8a [DWARF] Support for .debug_addr (consumer)
This patch implements basic support for parsing
  and dumping DWARFv5 .debug_addr section.

llvm-svn: 338447
2018-07-31 22:19:19 +00:00
Alexandre Ganea ee8a720051 [CodeView] Minimal support for S_UNAMESPACE records
Differential Revision: https://reviews.llvm.org/D50007

llvm-svn: 338417
2018-07-31 19:15:50 +00:00
Alexandre Ganea 0bb8e89187 This fixes a crash when a second pass is required for the Codeview Type merging *and* the index points outside of the table (which should lead to an error being printed).
This occurs currently until MS precompiled headers .obj is added (see D45213)

Differential Revision: https://reviews.llvm.org/D50006

llvm-svn: 338308
2018-07-30 21:14:25 +00:00
Fangrui Song f78650a8de Remove trailing space
sed -Ei 's/[[:space:]]+$//' include/**/*.{def,h,td} lib/**/*.{cpp,h}

llvm-svn: 338293
2018-07-30 19:41:25 +00:00
Wolfgang Pieb 1d56b4ae40 [DWARF v5] Don't report an error when the .debug_rnglists section is empty or non-existent. Fixes PR38297.
Reviewer: JDevlieghere

Differential Revision: https://reviews.llvm.org/D49815
 

llvm-svn: 337993
2018-07-26 01:12:41 +00:00
Fangrui Song 5bad9d835a [DWARF] Use deque in place of SmallVector to fix use-after-free issue
Summary: SmallVector's elements are moved when resizing and cause use-after-free.

Reviewers: probinson, dblaikie

Subscribers: JDevlieghere, llvm-commits

Differential Revision: https://reviews.llvm.org/D49702

llvm-svn: 337772
2018-07-23 23:27:45 +00:00
Wolfgang Pieb 790d86cefc Embed a template specialization in a namespace to work around a gcc bug.
llvm-svn: 337770
2018-07-23 23:14:23 +00:00
Wolfgang Pieb 439801ba1d [DWARF v5] Refactor range lists dumping by using a more generic way of handling tables of lists.
The intent is to use it for location list tables as well. Change is almost NFC with the exception
of the spelling of some strings used during dumping (all lowercase now).

Reviewer: JDevlieghere

Differential Revision: https://reviews.llvm.org/D49500

llvm-svn: 337763
2018-07-23 22:37:17 +00:00
Mandeep Singh Grang 20239b18bb [llvm] Change 2 instances of std::sort to llvm::sort
llvm-svn: 337192
2018-07-16 17:26:37 +00:00
Jonas Devlieghere 327e7a1608 [dwarfdump] Add pretty printer for accelerator table based on Atom.
For instance, When dumping .apple_types, the second atom represents the
DW_TAG. In addition to printing the raw value, we now also pretty print
the value if the ATOM tells us how.

llvm-svn: 337026
2018-07-13 17:21:51 +00:00
Fangrui Song 24452316c6 [DebugInfo] Fix getPreviousSibling after r336823
llvm-svn: 336837
2018-07-11 19:09:37 +00:00
Jonas Devlieghere 3f27e57ade [DebugInfo] Make children iterator bidirectional
Make the DIE iterator bidirectional so we can move to the previous
sibling of a DIE.

Differential revision: https://reviews.llvm.org/D49173

llvm-svn: 336823
2018-07-11 17:11:11 +00:00
Rui Ueyama 0230f7c763 Use StringRef instead of `const char *`.
I don't think there's a need to use `const char *`. In most (probably all?)
cases, we need a length of a name later, so discarding a length will
lead to a wasted effort.

Differential Revision: https://reviews.llvm.org/D49046

llvm-svn: 336612
2018-07-09 22:26:49 +00:00
Maksim Panchenko fa762cc19b [DebugInfo] Change default value of FDEPointerEncoding
Summary:
If the encoding is not specified in CIE augmentation string, then it
should be DW_EH_PE_absptr instead of DW_EH_PE_omit.

Reviewers: ruiu, MaskRay, plotfi, rafauler

Reviewed By: MaskRay

Subscribers: rafauler, JDevlieghere, llvm-commits

Differential Revision: https://reviews.llvm.org/D49000

llvm-svn: 336577
2018-07-09 18:45:38 +00:00
Benjamin Kramer 9fc944ae36 [PDB] memicmp only exists on Windows, use StringRef::compare_lower instead
llvm-svn: 336469
2018-07-06 21:56:57 +00:00
Zachary Turner 648bebdc67 [PDB] One more fix for hasing GSI records.
The reference implementation uses a case-insensitive string
comparison for strings of equal length.  This will cause the
string "tEo" to compare less than "VUo".  However we were using
a case sensitive comparison, which would generate the opposite
outcome.  Switch to a case insensitive comparison.  Also, when
one of the strings contains non-ascii characters, fallback to
a straight memcmp.

The only way to really test this is with a DIA test.  Before this
patch, the test will fail (but succeed if link.exe is used instead
of lld-link).  After the patch, it succeeds even with lld-link.

llvm-svn: 336464
2018-07-06 21:01:42 +00:00
Zachary Turner 1f200adfa7 [PDB] Sort globals symbols by name in GSI hash buckets.
It seems like the debugger first computes a symbol's bucket,
and then does a binary search of entries in the bucket using the
symbol's name in order to find it.  If the bucket entries are not
in sorted order, this obviously won't work.  After this patch a
couple of simple test cases show that we generate an exactly
identical GSI hash stream, which is very nice.

llvm-svn: 336405
2018-07-06 02:33:58 +00:00
Zachary Turner 68e1919d14 [CodeView] Correctly compute the name of S_PROCREF symbols.
We have a function which switches on the type of a symbol record
to return a hardcoded offset into the record that contains the
symbol name.  Not all symbols have names to begin with, and for
those records we return -1 for the offset.

Names are used for various things.  Importantly for this particular
bug, a hash of the record name is used as a key for certain hash
tables which are serialied into the PDB file.  One of these hash
tables is for the global symbol stream, which is basically a
collection of S_PROCREF symbols which contain the name of the
symbol, a module, and an address offset.

However, for S_PROCREF symbols, the function to return the offset
of the name was returning -1: basically it wasn't implemented.
As a result of this, all global symbols were hashing to the same
value, essentially it was as if every single global symbol's name
was the empty string.

This manifests in the VS debugger when you try to call a function
(global or member, doesn't matter) through the immediate window
and the debugger simply reports an error because it can't find the
function.  This makes perfect sense, because it is hashing the name
for real, looking in the global symbol hash table, and there is only
1 entry there which corresponds to a symbol whose name is the empty
string.

Fixing this fixes the MSVC debugger in this case.

llvm-svn: 336024
2018-06-29 22:19:02 +00:00
Paul Robinson 50f8ca38ee Pass DWARFUnit to verifier by reference not by value. I am moderately
sure this should not cause a memory leak.

llvm-svn: 336007
2018-06-29 19:17:44 +00:00
Zachary Turner ee8010abe3 Move some code from PDBFileBuilder to MSFBuilder.
The code to emit the pieces of the MSF file were actually in
PDBFileBuilder.  Move this to MSFBuilder so that we can
theoretically emit an MSF without having a PDB file.

llvm-svn: 335789
2018-06-27 21:18:15 +00:00
Kamil Rytarowski a8448ad098 Handle NetBSD specific path in findDebugBinary()
Summary:
The NetBSD Operating System installs debuginfo
files into /usr/libdata/debug, rather than other path
like in some other popular distribution.

This change makes llvm-symbolizer functional with
the basesystem executables.

Reviewers: joerg, vitalybuka

Reviewed By: vitalybuka

Subscribers: JDevlieghere, llvm-commits

Differential Revision: https://reviews.llvm.org/D48525

llvm-svn: 335511
2018-06-25 18:49:13 +00:00
Wolfgang Pieb 61d8c8d9b3 [DWARF] Improved error reporting for range lists.
Errors found processing the DW_AT_ranges attribute are propagated by lower level 
routines and reported by their callers.

Reviewer: JDevlieghere

Differential Revision: https://reviews.llvm.org/D48344

llvm-svn: 335188
2018-06-20 22:56:37 +00:00
Pavel Labath 4adc88ed25 [DWARF/AccelTable] Remove getDIESectionOffset for DWARF v5 entries
Summary:
This method was not correct for entries in DWO files as it assumed it
could just add up the CU and DIE offsets to get the absolute DIE offset.
This is not correct for the DWO files, as here the CU offset will
reference the skeleton unit, whereas the DIE offset will be the offset
in the full unit in the DWO file.

Unfortunately, this means that we are not able to determine the absolute
DIE offset using the information in the .debug_names section alone,
which means we have to offload some of this work to the users of this
class.

To demonstrate how this can be done, I've added/fixed the ability to
lookup entries using accelerator tables in DWO files in llvm-dwarfdump.
To make this happen, I've needed to make two extra changes in other
classes:
- made the DWARFContext method to lookup a CU based on the section
  offset public. I've needed this functionality to lookup a CU, and this
  seems like a useful thing in general.
- made DWARFUnit::getDWOId call extractDIEsIfNeeded. Before this, the
  DWOId was filled in only if the root DIE happened to be parsed
  before we called the accessor. Since the lazy parsing is supposed to
  happen under the hood, calling extractDIEsIfNeeded seems appropriate.

Reviewers: JDevlieghere, aprantl, dblaikie

Subscribers: mgrang, llvm-commits

Differential Revision: https://reviews.llvm.org/D48009

llvm-svn: 334578
2018-06-13 08:14:27 +00:00
Pavel Labath d6ca063907 DWARFAcceleratorTable: Add an iterator-based api for accessing names in the index
Summary:
Back when we were introducing the DWARF v5 name index, there was a
short discussion whether we shouldn't have a nicer api for iterating
over the index. At that time, I did not find it necessary since the
iteration over names was done only from within the index itself (and I
figured the internal implementation can deal with a slightly rough
interface).

However, now I ran into a use for this kind of API in LLDB (for finding
all names matching a regular expression), so it looked like a nice
opportunity to introduce one. To make the API more useful, I've made the
NameTableEntry class a bit smarter: it now stores the string section
reference (so it can return its name) and its position in the name index
(mainly useful for dumping/logging).

I also convert the internal users to use the new API, which also gives
test coverage for the added code.

Reviewers: JDevlieghere, aprantl, dblaikie

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D47590

llvm-svn: 333738
2018-06-01 10:33:11 +00:00
Pavel Labath 59870af66f DWARFAcceleratorTable: fix equal_range iterators
Summary:
Both (Apple and DWARF5) implementations of the iterators had bugs which
resulted in crashes if one attempted to iterate through the accelerator
tables all the way.

For the Apple tables, the issue was that we did not clear the DataOffset
field when we reached the end, which made our iterator compare unequal
to the "end" iterator. For the Dwarf5 tables, the problem was that we
incremented the CurrentIndex pointer and then used the incremented
(possibly invalid) pointer to check whether we have reached the end of
the index list.

The reason these bugs went undetected is because their only user
(dwarfdump) only ever searched for the first match. Besides allowing us
to test this fix, changing llvm-dwarfdump --find to display all matches
seems like a good improvement (it makes the behavior consistent with the
--name option), so I change llvm-dwarfdump to do that.

The existing tests would be sufficient to test this fix with the new
llvm-dwarfdump behavior, but I add a special test that demonstrates that
the tool indeed displays multiple results. The find.test test needed to
be tweaked a bit as the tool now does not print the ".debug_info
contents" header (also consistent with how --name works).

Reviewers: JDevlieghere, aprantl, dblaikie

Subscribers: mgrang, llvm-commits

Differential Revision: https://reviews.llvm.org/D47543

llvm-svn: 333635
2018-05-31 08:47:00 +00:00
Jonas Devlieghere 43dce3edbe [CodeView] Add prefix to CodeView registers.
Adds CVReg to CodeView register names to prevent a duplicate symbol with
CR3 defined in termios.h, as suggested by Zachary on the mailing list.

http://lists.llvm.org/pipermail/llvm-dev/2018-May/123372.html

Differential revision: https://reviews.llvm.org/D47478

rdar://39863705

llvm-svn: 333421
2018-05-29 14:35:34 +00:00
Jonas Devlieghere cb547cbb5c [dwarfdump] Make -c and -p work together
When requesting to dump both the parent chain and children, we used to
print the DIE more than once because we propagated the dump options to
the parent without clearing the respective flags. This commit fixes this
oversight and adds a test.

rdar://39415292

Differential revision: https://reviews.llvm.org/D47263

llvm-svn: 333350
2018-05-26 19:39:56 +00:00
Jonas Devlieghere 63eca15e95 [DebugInfo] Invert DIE order for range errors.
When printing an error for an invalid address range in a DIE, we used to
print the child above the parent, which is counter intuitive. This patch
reverses the order and indents the child to mimic the way we print the
debug info section.

llvm-svn: 333006
2018-05-22 17:38:03 +00:00
Jonas Devlieghere 7e0b023302 [DebugInfo] Fix location list check in the verifier
We weren't properly verifying location lists because we tried obtaining
the offset as a constant.

llvm-svn: 333005
2018-05-22 17:37:27 +00:00
Paul Robinson 543c0e1d50 [DWARFv5] Put the DWO ID in its place.
In DWARF v5, the DWO ID is in the (split/skeleton) CU header, not an
attribute on the CU DIE.

This changes the size of those headers, so use the parsed size whenever
we have one, for simplicitly.

Differential Revision: https://reviews.llvm.org/D47158

llvm-svn: 333004
2018-05-22 17:27:31 +00:00
Jonas Devlieghere c111382aa8 [DebugInfo] Use absolute addresses in location lists
Rather than relying on the user to do the address calculating in
DW_AT_location we should just dump the absolute address.

rdar://problem/38513870

Differential revision: https://reviews.llvm.org/D47152

llvm-svn: 332873
2018-05-21 19:36:54 +00:00
James Henderson 004b729ed1 [DWARF] Refactor callback usage for .debug_line error handling
Change the "recoverable" error callback to take an Error instaed of a
string.

Reviewed by: JDevlieghere

Differential Revision: https://reviews.llvm.org/D46831

llvm-svn: 332845
2018-05-21 15:30:54 +00:00
Wolfgang Pieb 20e1546655 Fixing buildbot error introduced with r332759.
llvm-svn: 332772
2018-05-18 21:44:28 +00:00
Wolfgang Pieb 401b5ecfea Addressing a couple of compiler warnings introduced with r332759.
llvm-svn: 332766
2018-05-18 20:51:16 +00:00
Wolfgang Pieb da71639cdb Fixing build error introduced with r332759.
llvm-svn: 332762
2018-05-18 20:35:13 +00:00
Wolfgang Pieb ad60559be7 [DWARF v5] Improved support for .debug_rnglists (consumer). Enables any consumer to
extract DWARF v5 encoded rangelists.

Reviewer: JDevlieghere

Differential Revision: https://reviews.llvm.org/D45549

llvm-svn: 332759
2018-05-18 20:12:54 +00:00
Zachary Turner c762666e87 Resubmit [pdb] Change /DEBUG:GHASH to emit 8 byte hashes."
This fixes the remaining failing tests, so resubmitting with no
functional change.

llvm-svn: 332676
2018-05-17 22:55:15 +00:00
Zachary Turner 1de9fce151 Revert "[pdb] Change /DEBUG:GHASH to emit 8 byte hashes."
A few tests haven't been properly updated, so reverting while
I have time to investigate proper fixes.

llvm-svn: 332672
2018-05-17 21:49:25 +00:00
Zachary Turner 3c4c8a0937 [pdb] Change /DEBUG:GHASH to emit 8 byte hashes.
Previously we emitted 20-byte SHA1 hashes.  This is overkill
for identifying debug info records, and has the negative side
effect of making object files bigger and links slower.  By
using only the last 8 bytes of a SHA1, we get smaller object
files and ~10% faster links.

This modifies the format of the .debug$H section by adding a new
value for the hash algorithm field, so that the linker will still
work when its object files have an old format.

Differential Revision: https://reviews.llvm.org/D46855

llvm-svn: 332669
2018-05-17 21:22:48 +00:00
Reid Kleckner f40f85868e [codeview] Include record prefix in global type hashing
The prefix includes type kind, which is important to preserve. Two
different type leafs can easily have the same interior record contents
as another type.

We ran into this issue in PR37492 where a bitfield type record collided
with a const modifier record. Their contents were bitwise identical, but
their kinds were different.

llvm-svn: 332664
2018-05-17 20:47:22 +00:00
Pavel Labath 80827f10a1 Reapply "DWARFVerifier: Check "completeness" of .debug_names section"
This is a resubmit of r331868 (D46583), which was reverted due to
failures on the PS4 bot.

These have been resolved with r332246/D46748.

llvm-svn: 332349
2018-05-15 13:24:10 +00:00
Paul Robinson 5f53f07b66 [DWARF] Factor out a DWARFUnitHeader class. NFC
Extract information related to a "unit header" from DWARFUnit into a
new DWARFUnitHeader class, and add a DWARFUnit member for the header.
This is one step in the direction of allowing type units in the
.debug_info section for DWARF v5.

Differential Revision: https://reviews.llvm.org/D46707

llvm-svn: 332289
2018-05-14 20:32:31 +00:00
Pavel Labath 2a6afe5f87 [CodeGen/AccelTable]: Handle -dwarf-linkage-names=Abstract correctly
Summary:
If we are not emitting a linkage name in the .debug_info sections, we
should not add it into the index either. This makes sure our index is
consistent with the actual debug info.

I am also explicitly setting the --dwarf-linkage-names=All in the
name-collsions test as that one would now fail on targets where this
defaults to "Abstract" (in fact, it would have failed already if there
wasn't a bug in the DWARF verifier, which I fix as well).

Reviewers: probinson, aprantl, JDevlieghere

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D46748

llvm-svn: 332246
2018-05-14 14:13:20 +00:00
Wolfgang Pieb f2b6915ed4 [DWARF] Fixing a bug in DWARF v5 string offsets tables where the length encoded the contribution
length excluding the table header. Instead it must encode the contribution length minus the length
field itself.

Reviewer: JDevliegehere

Differential Revision: https://reviews.llvm.org/D45922

llvm-svn: 332030
2018-05-10 20:02:34 +00:00
James Henderson 11a9de74c9 Fix signed/unsigned comparison warning and print format
The print format was causing at least 2 unit-test failures from r331971.

The signed/unsigned comparison warnings only appeared to affect two lines but
it was unclear whether it might just pop up on other lines, so I have been
explicit in all the literals in the tests.

There were other bot unit-test failures that I am still investigating.

llvm-svn: 331978
2018-05-10 12:15:43 +00:00
James Henderson a3acf99e59 [DWARF] Rework debug line parsing to use llvm::Error and callbacks
Reviewed by: dblaikie, JDevlieghere, espindola

Differential Revision: https://reviews.llvm.org/D44560

Summary:
The .debug_line parser previously reported errors by printing to stderr and
return false. This is not particularly helpful for clients of the library code,
as it prevents them from handling the errors in a manner based on the calling
context. This change switches to using llvm::Error and callbacks to indicate
what problems were detected during parsing, and has updated clients to handle
the errors in a location-specific manner. In general, this means that they
continue to do the same thing to external users. Below, I have outlined what
the known behaviour changes are, relating to this change.

There are two levels of "errors" in the new error mechanism, to broadly
distinguish between different fail states of the parser, since not every
failure will prevent parsing of the unit, or of subsequent unit. Malformed
table errors that prevent reading the remainder of the table (reported by
returning them) and other minor issues representing problems with parsing that
do not prevent attempting to continue reading the table (reported by calling a
specified callback funciton). The only example of this currently is when the
last sequence of a unit is unterminated. However, I think it would be good to
change the handling of unrecognised opcodes to report as minor issues as well,
rather than just printing to the stream if --verbose is used (this would be a
subsequent change however).

I have substantially extended the DwarfGenerator to be able to handle
custom-crafted .debug_line sections, allowing for comprehensive unit-testing
of the parser code. For now, I am just adding unit tests to cover the basic
error reporting, and positive cases, and do not currently intend to test every
part of the parser, although the framework should be sufficient to do so at a
later point.

Known behaviour changes:
  - The dump function in DWARFContext now does not attempt to read subsequent
  tables when searching for a specific offset, if the unit length field of a
  table before the specified offset is a reserved value.
  - getOrParseLineTable now returns a useful Error if an invalid offset is
  encountered, rather than simply a nullptr.
  - The parse functions no longer use `WithColor::warning` directly to report
  errors, allowing LLD to call its own warning function.
  - The existing parse error messages have been updated to not specifically
  include "warning" in their message, allowing consumers to determine what
  severity the problem is.
  - If the line table version field appears to have a value less than 2, an
  informative error is returned, instead of just false.
  - If the line table unit length field uses a reserved value, an informative
  error is returned, instead of just false.
  - Dumping of .debug_line.dwo sections is now implemented the same as regular
  .debug_line sections.
  - Verbose dumping of .debug_line[.dwo] sections now prints the prologue, if
  there is a prologue error, just like non-verbose dumping.

As a helper for the generator code, I have re-added emitInt64 to the
AsmPrinter code. This previously existed, but was removed way back in r100296,
presumably because it was dead at the time.

This change also requires a change to LLD, which will be committed separately.

llvm-svn: 331971
2018-05-10 10:51:33 +00:00
Pavel Labath e0207a60dd Revert "DWARFVerifier: Check "completeness" of .debug_names section"
The new verifier check has found an error in the
debug-names-name-collisions.ll test on the PS4 bot:

error: Name Index @ 0x0: Entry @ 0xdc: mismatched Name of DIE @ 0x23: index - _ZN3foo3fooE; debug_info - foo.

Reverting while I investigate whether this is a bug in the verifier or
the generator.

This reverts commit r331868.

llvm-svn: 331869
2018-05-09 12:26:19 +00:00
Pavel Labath 3280e0467f DWARFVerifier: Check "completeness" of .debug_names section
Summary:
This patch implements a check which makes sure all entries required by
the DWARF v5 specification are present in the Name Index. The algorithm
tries to follow the wording of Section 6.1.1.1 of the spec as closely as
possible.

The main deviation from it is that instead of a whitelist-based approach
in the spec "The name index must contain an entry for each debugging
information entry that defines a named subprogram, label, variable,
type, or namespace" I chose a blacklist-based one, where I consider
everything to be "in" and then remove the entries that don't make sense.
I did this because it has more potential for catching interesting cases
and the above is a bit vague (it uses plain words like "variable" and
"subprogram", but the rest of the section speaks about specific TAGs).

This approach has raised some interesting questions, the main one being
whether enumerator values should be indexed. The consensus seems to be
that they should, although it does not follow from section 6.1.1.1.
For the time being I made the verifier ignore these, as LLVM does not do
this yet, and I wanted to get a clean run when verifying generated debug
info.

Another interesting case was the DW_TAG_imported_declaration. It was not
immediately clear to me whether this should go in or not, but currently
it is not indexed, and (unlike the enumerators) in does not seem to cause
problems for LLDB, so I've also ignored it.

Reviewers: JDevlieghere, aprantl, dblaikie

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D46583

llvm-svn: 331868
2018-05-09 12:06:17 +00:00
Fangrui Song bd088560a8 [DebugInfo] Accept `S` in augmentation strings in CIE.
glibc libc.a(sigaction.o) compiled from sysdeps/unix/sysv/linux/x86_64/sigaction.c uses "zRS".

llvm-svn: 331738
2018-05-08 06:21:12 +00:00
David Blaikie aa537da89f llvm-symbolizer: Handle function definitions nested within other functions
LLVM always puts function definition DIEs at the top level, but under
some circumstances GCC does not (at least in this case with member
functions of a function-local type).

To ensure that doesn't appear as though the local type's member function
is unduly inlined within the outer function - ensure the inline
discovery DIE parent walk stops at the first DW_TAG_subprogram.

llvm-svn: 331291
2018-05-01 18:08:45 +00:00
Jonas Devlieghere 4bbcb5ab04 [DebugInfo] Prevent infinite recursion for malformed DWARF
This prevents infinite recursion in DWARFDie::findRecursively for
malformed DWARF where a DIE references itself.

This fixes PR36257.

Differential revision: https://reviews.llvm.org/D43092

llvm-svn: 331200
2018-04-30 17:02:41 +00:00
Zachary Turner 194be871b9 [LLD/PDB] Emit first section contribution for DBI Module Descriptor.
Part of the DBI stream is a list of variable length structures
describing each module that contributes to the final executable.

One member of this structure is a section contribution entry that
describes the first section contribution in the output file for
the given module.

We have been leaving this structure unpopulated until now, so with
this patch it is now filled out correctly.

Differential Revision: https://reviews.llvm.org/D45832

llvm-svn: 330457
2018-04-20 18:00:46 +00:00
Andrew Ng 7a2fa74ab0 [DebugInfo] Use WithColor for more debug line warnings
Updated two more debug line related warnings to use WithColor. This was
necessary to ensure consistent output order of the warnings on Windows
for debug line tests.

Differential Revision: https://reviews.llvm.org/D45871

llvm-svn: 330440
2018-04-20 15:29:47 +00:00
Zachary Turner bee6c22414 [llvm-pdbutil] Dump first section contribution for each module.
The DBI stream contains a list of module descriptors.  At the
beginning of each descriptor is a structure representing the first
section contribution in the output file for that module.  LLD
currently doesn't fill out this structure at all, but link.exe
does.  So as a precursor to emitting this data in LLD, we first
need a way to dump it so that it can be checked.

This patch adds support for the dumping, and verifies via a test
that LLD emits bogus information.

llvm-svn: 330208
2018-04-17 20:06:43 +00:00
Zachary Turner d8d97de514 [PDB] Correctly use the target machine when writing DBI stream.
Using Config->is64() will treat ARM64 as Amd64, which is incorrect.
Furthermore, there are more esoteric architectures that could
theoretically be encountered.  Just set it directly to the machine
type, which we already know anyway.

llvm-svn: 330157
2018-04-16 20:42:06 +00:00
Zachary Turner e3fe669855 Resubmit "Fix some incorrect fields in our generated PDBs."
This fixes the failing tests.  They simply hadn't been updated
to match the new output resulting from this patch.

llvm-svn: 330145
2018-04-16 18:17:13 +00:00
Zachary Turner 52c80e3860 Revert "Fix some incorrect fields in our generated PDBs."
There are a couple of failing tests which slipped under my radar
so I'm reverting this while I attempt to fix.

llvm-svn: 330133
2018-04-16 16:55:41 +00:00
Brock Wyma 94ece8fbc9 [CodeView] Initial support for emitting S_THUNK32 symbols for compiler...
When emitting CodeView debug information, compiler-generated thunk routines
should be emitted using S_THUNK32 symbols instead of S_GPROC32_ID symbols so
Visual Studio can properly step into the user code.  This initial support only
handles standard thunk ordinals.

Differential Revision: https://reviews.llvm.org/D43838

llvm-svn: 330132
2018-04-16 16:53:57 +00:00
Zachary Turner 1b06cc7817 Fix some incorrect fields in our generated PDBs.
Most of these are pretty trivial and obvious. Setting the toolchain
version to 14.11 is perhaps a little questionable, but we've been bitten
in the past where one of our version fields sidn't match MSVC's, and I
definitely don't want to go through that diagnosis again as it was
pretty time consuming and hard to track down.

I found all of these by using llvm-pdbutil export to dump the dbi and
pdb streams to a file, then using fc followed by llvm-pdbutil explain to
explain the mismatched bytes.

There are still some more, these are just the low hanging fruit.

Differential Revision: https://reviews.llvm.org/D45276

llvm-svn: 330130
2018-04-16 16:27:49 +00:00
Jonas Devlieghere 6be1f01935 [Support] Extend WithColor helpers
Although printing warnings and errors to stderr is by far the most
common case, this patch makes it possible to specify any stream.

llvm-svn: 330094
2018-04-15 08:44:15 +00:00
Jonas Devlieghere 84e99265d6 [DebugInfo] Use WithColor to print errors/warnings
Use the convenience methods from WithColor to consistently print errors
and warnings in libDebugInfo.

llvm-svn: 330092
2018-04-14 22:07:23 +00:00
Mandeep Singh Grang 0f035ebed2 [DebugInfo] Change std::sort to llvm::sort in response to r327219
r327219 added wrappers to std::sort which randomly shuffle the container before
sorting.  This will help in uncovering non-determinism caused due to undefined
sorting order of objects having the same key.

To make use of that infrastructure we need to invoke llvm::sort instead of
std::sort.

Note: This patch is one of a series of patches to replace *all* std::sort to
llvm::sort.  Refer the comments section in D44363 for a list of all the
required patches.

llvm-svn: 330061
2018-04-13 19:50:51 +00:00
Aaron Smith 3dca0bedbb [DebugInfoPDB] Add DIA implementations of findSymbolByRVA and findSymbolByAddr
llvm-svn: 329724
2018-04-10 17:33:18 +00:00
Aaron Smith c0a5c01aeb [PDB] Remove dead code and run clang format; NFC
llvm-svn: 329712
2018-04-10 15:25:04 +00:00
Alexandre Ganea 08df84e4f0 [DebugInfo][COFF] Fix reading variable-length encoded records
While reading Codeview records which contain variable-length encoded integers,
such as LF_BCLASS, LF_ENUMERATE, LF_MEMBER, LF_VBCLASS or LF_IVBCLASS,
the record's size would be improperly calculated in cases where the value was
indeed of a variable length (>= LF_NUMERIC). This caused a bad alignement on
the next record, which would/might crash later on.

Differential Revision: https://reviews.llvm.org/D45104

llvm-svn: 329659
2018-04-10 01:58:45 +00:00
Alexandre Ganea 3241cec577 Fix line endings (CR/LF -> LF) introduced by rL329613
reviewer: zturner
llvm-svn: 329646
2018-04-10 00:09:15 +00:00
Alexandre Ganea d9e96741c4 [Debuginfo][COFF] Minimal serialization support for precompiled types records
This change adds support for the LF_PRECOMP and LF_ENDPRECOMP records required
to read/write Microsoft precompiled types .objs.
See https://en.wikipedia.org/wiki/Precompiled_header#Microsoft_Visual_C_and_C++

This also adds handling for the .debug$P section, which is actually a .debug$T
section in disguise, found only in precompiled .objs.

Differential Revision: https://reviews.llvm.org/D45283

llvm-svn: 329613
2018-04-09 20:17:56 +00:00
Hiroshi Inoue 9ff2380ea6 [NFC] fix trivial typos in comments and error message
"is is" -> "is", "are are" -> "are"

llvm-svn: 329546
2018-04-09 04:37:53 +00:00
Pavel Labath c9f07b06a1 DWARFVerifier: validate information in name index entries
Summary:
This patch add checks to verify that the information in the name index
entries is consistent with the debug_info section. Specifically, we
check that entries point to valid DIEs, and their names, tags, and
compile units match the information in the debug_info sections.

These checks are only run if the previous checks did not find any errors
in the name index headers. Attempting to proceed with the checks anyway
would likely produce a lot of spurious errors and the verification code
would need to be very careful to avoid crashing.

I also add a couple of more checks to the abbreviation-validation code
to verify that some attributes are always present (an index without a
DW_IDX_die_offset attribute is fairly useless).

The entry verification works only on indexes without any type units - I
haven't attempted to extend it to type units, as we don't even have a
DWARF v5-compatible type unit generator at the moment.

Reviewers: JDevlieghere, aprantl, dblaikie

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D45323

llvm-svn: 329392
2018-04-06 13:34:12 +00:00
Pavel Labath 54ca2d688a [debug_loc] Fix typo in DWARFExpression constructor
Summary:
The positions of the DwarfVersion and AddressSize arguments were
reversed, which caused parsing for dwarf opcodes which contained
address-size-dependent operands (such as DW_OP_addr). Amusingly enough,
none of the address-size asserts fired, as dwarf version was always 4,
which is a valid address size.

I ran into this when constructing weird inputs for the DWARF verifier. I
I add a test case as hand-written dwarf -- I am not sure how to trigger
this differently, as having a DW_OP_addr inside a location list is a
fairly non-standard thing to do.

Fixing this error exposed a bug in the debug_loc.dwo parser, which was
always being constructed with an address size of 0. I fix that as well
by following the pattern in the non-dwo parser of picking up the address
size from the first compile unit (which is technically not correct, but
probably good enough in practice).

Reviewers: JDevlieghere, aprantl, dblaikie

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D45324

llvm-svn: 329381
2018-04-06 08:49:57 +00:00
Wolfgang Pieb 3fb9e3f398 [DWARF v5][NFC]: Refactor DebugRnglists to prepare for the support of the DW_AT_ranges
attribute in conjunction with .debug_rnglists.

Reviewers: JDevlieghere
 
Differential Revision: https://reviews.llvm.org/D45307

llvm-svn: 329345
2018-04-05 21:01:49 +00:00
Zachary Turner 15b2bdfd8b [llvm-pdbutil] Add the ability to explain binary files.
Using this, you can use llvm-pdbutil to export the contents of a
stream to a binary file, then run explain on the binary file so
that it treats the offset as an offset into the stream instead
of an offset into a file.  This makes it easy to compare the
contents of the same stream from two different files.

llvm-svn: 329207
2018-04-04 17:29:09 +00:00
Nico Weber 086b1c8118 Minor no-op cmake file style fix.
llvm-svn: 329137
2018-04-04 00:50:22 +00:00
Aaron Smith 47f18b91bb [DebugInfoPDB] Add a few missing definitions to PDBTypes.h
The missing definitions are from cvconst.h shipped with DIA SDK.

Correct the url to MSDN for MemoryTypeEnum and set the underlying 
type of PDB_StackFrameType and PDB_MemoryType to uint16_t.

llvm-svn: 329104
2018-04-03 19:41:27 +00:00
Zachary Turner d11328a1bb [llvm-pdbutil] Add an export subcommand.
This command can dump the binary contents of a stream to a file.
This is useful when you want to do side-by-side comparisons of
a specific stream from two PDBs to examine the differences between
them.  You can export both of them to a file, then open them up
side by side in a hex editor (for example), so as to eliminate any
differences that might arise from the contents being on different
blocks in the PDB.

In subsequent patches I plan to improve the "explain" subcommand
so that you can explain the contents of a binary file that isn't
necessarily a full PDB, but one of these dumped streams, by telling
the subcommand how to interpret the contents.

llvm-svn: 329002
2018-04-02 18:35:21 +00:00
Mandeep Singh Grang fe1d28e83d [DebugInfo] Change std::sort to llvm::sort in response to r327219
Summary:
r327219 added wrappers to std::sort which randomly shuffle the container before sorting.
This will help in uncovering non-determinism caused due to undefined sorting
order of objects having the same key.

To make use of that infrastructure we need to invoke llvm::sort instead of std::sort.

Note: This patch is one of a series of patches to replace *all* std::sort to llvm::sort.
Refer the comments section in D44363 for a list of all the required patches.

Reviewers: echristo, zturner, samsonov

Reviewed By: echristo

Subscribers: JDevlieghere, llvm-commits

Differential Revision: https://reviews.llvm.org/D45134

llvm-svn: 328935
2018-04-01 16:18:49 +00:00
Zachary Turner d5cf5cf637 [llvm-pdbutil] Dig deeper into the PDB and DBI streams when explaining.
This will show more detail when using `llvm-pdbutil explain` on an
offset in the DBI or PDB streams.  Specifically, it will dig into
individual header fields and substreams to give a more precise
description of what the byte represents.

llvm-svn: 328878
2018-03-30 17:16:50 +00:00
Zachary Turner 3203e27473 [MSF] Default to FPM2, and always mark FPM pages allocated.
There are two FPMs in an MSF file, the idea being that for
incremental updates you can write to the alternate one and then
atomically swap them on commit.  LLVM defaulted to using FPM1
on the first commit, but this differs from Microsoft's behavior
which is to default to using FPM2 on the first commit.  To
eliminate some byte-level file differences, this patch changes
LLVM's default to also be FPM2.

Additionally, LLVM was trying to be "smart" about marking FPM
pages allocated.  In addition to marking every page belonging
to the alternate FPM as unallocated, LLVM also marked pages at
the end of the main FPM which were not needed as unallocated.

In order to match the behavior of Microsoft-generated PDBs, we
now always mark every FPM block as allocated, regardless of
whether it is in the main FPM or the alt FPM, and regardless of
whether or not it describes blocks which are actually in the file.

This has the side benefit of simplifying our code.

llvm-svn: 328812
2018-03-29 18:34:15 +00:00
Pavel Labath ea0f841c3b .debug_names: Correctly align the AugmentationStringSize field
We should align the value of the field, not the overall section offset.

This distinction matters if one of the debug_names contributions is not
of size which is a multiple of four. The dwarf producers may choose to
emit rounded contributions, but they are not required to do so. In the
latter case, without this patch we would corrupt the parsing state, as
we would adjust the offset even if subsequent contributions contained
correctly rounded augmentation strings.

llvm-svn: 328796
2018-03-29 15:12:45 +00:00
Pavel Labath 2d1fc4375f .debug_names: Parse DW_IDX_die_offset as a reference
Before this patch we were parsing the attributes as section offsets, as
that is what apple_names is doing. However, this is not correct as DWARF
v5 specifies that this attribute should use the Reference form class.

This also updates all the testcases (except the ones that deliberately
pass a different form) to use the correct form class.

llvm-svn: 328773
2018-03-29 13:47:57 +00:00
Wolfgang Pieb ab068eaa57 [DWARF][DWARF v5]: Adding support for dumping DW_RLE_offset_pair and DW_RLE_base_address
Reviewers: dblakie, aprantl

Differential Revision: https://reviews.llvm.org/D44811

llvm-svn: 328662
2018-03-27 20:27:36 +00:00
Aaron Smith f13938382c [DebugInfoPDB] Print the method name along with the variant value
Before this change, using dumpProperties() with PDBSymbolData
would look like this:

  get_locationType: 3
  1

After this change:

  get_locationType: 3
  get_value: 1

llvm-svn: 328590
2018-03-26 22:53:38 +00:00
Aaron Smith 1af50bcf89 [DebugInfoPDB] Add methods to get the compiland and line numbers with PDBSymbolData
llvm-svn: 328587
2018-03-26 22:17:12 +00:00
Aaron Smith ed81a9db29 [DebugInfoPDB] Add DIA implementation of findLineNumbersByRVA
This method is used to find line numbers for PDBSymbolData
that have an invalid virtual address.

llvm-svn: 328586
2018-03-26 22:13:22 +00:00
Aaron Smith 53708a5e9e [DebugInfoPDB] Add DIA implementation of addressForVA and addressForRVA
These are used in finding line numbers for PDBSymbolData

llvm-svn: 328585
2018-03-26 22:10:02 +00:00
Paul Robinson 82e4864730 Use correct format specifier.
Review comment on r328235 by James Henderson.

llvm-svn: 328578
2018-03-26 19:55:01 +00:00
Zachary Turner f228276262 [PDB] Resubmit "Support embedding natvis files in PDBs."
This was reverted several times due to what ultimately turned out
to be incompatibilities in our serialized hash table format.

Several changes went in prior to this to fix those issues since
they were more fundamental and independent of supporting injected
sources, so now that those are fixed this change should hopefully
pass.

llvm-svn: 328363
2018-03-23 19:57:25 +00:00
Zachary Turner a6fb536e5b [PDB] Make our PDBs look more like MS PDBs.
When investigating bugs in PDB generation, the first step is
often to do the same link with link.exe and then compare PDBs.

But comparing PDBs is hard because two completely different byte
sequences can both be correct, so it hampers the investigation when
you also have to spend time figuring out not just which bytes are
different, but also if the difference is meaningful.

This patch fixes a couple of cases related to string table emission,
hash table emission, and the order in which we emit strings that
makes more of our bytes the same as the bytes generated by MS PDBs.

Differential Revision: https://reviews.llvm.org/D44810

llvm-svn: 328348
2018-03-23 18:43:39 +00:00
Paul Robinson 7947468e69 [DWARF] Replace assert with diagnostic. PR36868.
llvm-svn: 328235
2018-03-22 19:37:56 +00:00
Zachary Turner 71d36ad9f9 [Codeview/PDB] Rename some methods for clarity.
NFC, this just renames some methods to better express what they
do, and also adds a few helper methods to add some symmetry to the
API in a few places (for example there was a getStringFromId but not
a getIdFromString method in the string table).

llvm-svn: 328221
2018-03-22 17:37:28 +00:00
Pavel Labath 79cd942c23 DWARFVerifier: verify debug_names abbreviation table
Summary:
This commit adds checks of the abbreviation table in a DWARF v5 Name
Index. The most interesting/useful check is the one which checks that
each index attributes is encoded using the correct form class, but it
also checks for the more obvious errors like unknown
forms/tags/attributes and duplicated attributes.

Reviewers: JDevlieghere, aprantl, dblaikie

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D44736

llvm-svn: 328202
2018-03-22 14:50:44 +00:00
Aaron Smith 523de05a1f [DIA] Add IPDBSectionContrib interfaces and DIA implementation
To resolve symbol context at a particular address, we need to
determine the compiland for the address. We are able to determine
the parent compiland of PDBSymbolFunc, PDBSymbolTypeUDT,
PDBSymbolTypeEnum symbols indirectly through line information. 
However no such information is availabile for PDBSymbolData, 
i.e. variables.

The Section Contribution table from PDBs has information about
each compiland's contribution to sections by address. For example,
a piece of a contribution looks like,

  VA         RelativeVA  Sect No.  Offset    Length    Compiland
  14000087B0 000087B0    0001      000077B0  000000BB  exe_main.obj

So given an address, it's possible to determine its compiland with
this information.

llvm-svn: 328178
2018-03-22 04:08:15 +00:00
Aaron Smith 58a32a478f [PDB] Get more DIA table enumerators
Rename the original function and make it a static template.

llvm-svn: 328177
2018-03-22 03:57:06 +00:00
Zachary Turner eb62999455 [PDB] Don't ignore bucket 0 when writing the PDB string table.
The hash table is a list of buckets, and the *value* stored in
the bucket cannot be 0 since that is reserved.  However, the code
here was incorrectly skipping over the 0'th bucket entirely.
The 0'th bucket is perfectly fine, just none of these buckets
can contain the value 0.

As a result, whenever there was a string where hash(S) % Size
was equal to 0, we would write the value in the next bucket
instead.  We never caught this in our tests due to *another*
bug, which is that we would iterate the entire list of buckets
looking for the value, only using the hash value as a starting
point.  However, the real algorithm stops when it finds 0 in
a bucket since it takes that to mean "the item is not in the
hash table".

The unit test is updated to carefully construct a set of hash
values that will cause one item to hash to 0 mod bucket count,
and the reader is also updated to return an error indicating that
the item is not found when it encounters a 0 bucket.

llvm-svn: 328162
2018-03-21 22:23:59 +00:00
Reid Kleckner 8562c1a198 [PDB] Remove unused private variable, re-applying r327900 after relanding more natvis changes[4~
llvm-svn: 328156
2018-03-21 21:47:26 +00:00
Rafael Espindola c51dc906ea Handle abbr_offset with relocations.
This is mostly just plumbing to get a DWARFDataExtractor where we
compute abbr_offset so we can use getRelocatedValue.

This is part of PR36793.

llvm-svn: 328154
2018-03-21 21:31:25 +00:00
Pavel Labath 9025f9559d [dwarf] Unify unknown dwarf enum formatting code
Summary:
We have had at least three pieces of code (in DWARFAbbreviationDeclaration,
DWARFAcceleratorTable and DWARFDie) that have hand-rolled support for
dumping unknown dwarf enum values. While not terrible, they are a bit
distracting and enable small differences to creep in (Unknown_ffff vs.
Unknown_0xffff). I ended up needing to add a fourth place
(DWARFVerifier), so it seems it would be a good time to centralize.

This patch creates an alternative to the XXXString dumping functions in
the BinaryFormat library, which formats an unknown value as
DW_TYPE_unknown_1234, instead of just an empty string. It is based on
the formatv function, as that allows us to avoid materializing the
string for unknown values (and because this way I don't have to invent a
name for the new functions :P).

In this patch I add formatters for dwarf attributes, forms, tags, and
index attributes as these are the ones in use currently, but adding
other enums is straight-forward.

Reviewers: dblaikie, JDevlieghere, aprantl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D44570

llvm-svn: 328090
2018-03-21 11:46:37 +00:00
Zachary Turner fced530650 Revert "Resubmit "Support embedding natvis files in PDBs.""
This is still failing on a different bot this time due to some
issue related to hashing absolute paths.  Reverting until I can
figure it out.

llvm-svn: 328014
2018-03-20 18:37:03 +00:00
Zachary Turner 132d7a134f Resubmit "Support embedding natvis files in PDBs."
The issue causing this to fail in certain configurations
should be fixed.

It was due to the fact that DIA apparently expects there to be
a null string at ID 1 in the string table.  I'm not sure why this
is important but it seems to make a difference, so set it.

llvm-svn: 328002
2018-03-20 17:06:39 +00:00
Aaron Smith da61120749 [PDB] Add a method to get the full path of the source file for PDBSymbolCompiland
Summary:
Redefine PDBSymbolCompiland::getSourceFileName() to return the filename (w/o directory) of the source file that is used to compile the compiland. This is because the result returned previously is ambiguous. It could be the filename, relative path or full path of the source file. 

Move the implementation of SymbolFilePDB::GetSourceFileNameForPDBCompiland() into a new method PDBSymbolCompiland::getSourceFileFullPath(). 

Reviewers: zturner, rnk, llvm-commits

Reviewed By: zturner

Differential Revision: https://reviews.llvm.org/D44458

llvm-svn: 327910
2018-03-19 21:20:04 +00:00
Aaron Smith 06173e8b46 [PDB] Add exclusive methods to derived symbol class
Summary: This commit adds two methods to the PDBSymboFunc class used in parsing symbols. getLineNumbers() is used to determine a Function symbol's declaration and getCompilandId() is used to initialize the SymbolContext field sc.comp_unit.

Reviewers: zturner, rnk, llvm-commits

Reviewed By: zturner

Differential Revision: https://reviews.llvm.org/D44457

llvm-svn: 327909
2018-03-19 21:18:39 +00:00
Zachary Turner a21558897b Revert "Support embedding natvis files in PDBs."
This is causing a test failure on a certain bot, so I'm removing
this temporarily until we can figure out the source of the error.

llvm-svn: 327903
2018-03-19 20:41:59 +00:00
Zachary Turner 426885b10c Remove an unused private variable.
llvm-svn: 327900
2018-03-19 20:22:48 +00:00
Zachary Turner de53aaf132 Support embedding natvis files in PDBs.
Natvis is a debug language supported by Visual Studio for
specifying custom visualizers.  The /NATVIS option is an
undocumented link.exe flag which will take a .natvis file
and "inject" it into the PDB.  This way, you can ship the
debug visualizers for a program along with the PDB, which
is very useful for postmortem debugging.

This is implemented by adding a new "named stream" to the
PDB with a special name of /src/files/<natvis file name>
and simply copying the contents of the xml into this file.

Additionally, we need to emit a single stream named
/src/headerblock which contains a hash table of embedded
files to records describing them.

This patch adds this functionality, including the /NATVIS
option to lld-link.

Differential Revision: https://reviews.llvm.org/D44328

llvm-svn: 327895
2018-03-19 19:53:51 +00:00
Pavel Labath 906b777a6a DWARFVerifier: Enhance validation of .debug_names hash tables
Summary:
This patch adds more checks to the .debug_names validator. Specifically,
they check for:
- buckets claiming to be non-empty but pointing to mismatched hashes
  (most consumers would interpret this as an empty bucket, but it
  questionable whether the generator meant that)
- hashes that are not reachable from any bucket
- names with incorrect hashes

Together, these checks ensure that any name in the index can be reached
through the hash table using the regular lookup algorithm. We also warn
if we encounter a name index without a hash table.

Reviewers: JDevlieghere, aprantl, dblaikie

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D44433

llvm-svn: 327699
2018-03-16 10:02:16 +00:00
Zachary Turner edbcbe0b62 [PDB] Fix a bug where we were serializing hash tables incorrectly.
There was some code that tried to calculate the number of 4-byte
words required to hold N bits, but it was instead computing the
number of bytes required to hold N bits.  This was leading to
extraneous data being output into the hash table, which would
cause certain operations in DIA (the Microsoft PDB reader) to
fail.

llvm-svn: 327675
2018-03-15 22:31:00 +00:00
Zachary Turner ebf03f6c46 Refactor the PDB HashTable class.
It previously only worked when the key and value types were
both 4 byte integers.  We now have a use case for a non trivial
value type, so we need to extend it to support arbitrary value
types, which means templatizing it.

llvm-svn: 327647
2018-03-15 17:38:26 +00:00
Aaron Smith 40198f5905 [DebugInfo] Add a new method IPDBSession::findLineNumbersBySectOffset
Summary:
Some PDB symbols do not have a valid VA or RVA but have Addr by Section and Offset. For example, a variable in thread-local storage has the following properties:

     get_addressOffset: 0
     get_addressSection: 5
     get_lexicalParentId: 2
     get_name: g_tls
     get_symIndexId: 12
     get_typeId: 4
     get_dataKind: 6
     get_symTag: 7
     get_locationType: 2

This change provides a new method to locate line numbers by Section and Offset from those symbols.

Reviewers: zturner, rnk, llvm-commits

Subscribers: asmith, JDevlieghere

Differential Revision: https://reviews.llvm.org/D44407

llvm-svn: 327601
2018-03-15 06:04:51 +00:00
Pavel Labath 322711f529 DWARF: Unify form size handling code
Summary:
This patch replaces the two switches which are deducing the size of
various forms with a single implementation. I have put the new
implementation into BinaryFormat, to avoid introducing dependencies
between the two independent libraries (DebugInfo and CodeGen) that need
this functionality.

Reviewers: aprantl, JDevlieghere, dblaikie

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D44418

llvm-svn: 327486
2018-03-14 09:39:54 +00:00
Eugene Zemtsov 82d60d6b29 Handle mixed-OS paths in DWARF reader
Make sure that DWARF line information generated by Windows can be properly read by Posix OS and vice versa.

Differential Revision: https://reviews.llvm.org/D44290

llvm-svn: 327430
2018-03-13 17:54:29 +00:00
Zachary Turner 679aeadda1 [PDB] Support dumping injected sources via the DIA reader.
Injected sources are basically a way to add actual source file content
to your PDB. Presumably you could use this for shipping your source code
with your debug information, but in practice I can only find this being
used for embedding natvis files inside of PDBs.

In order to effectively test LLVM's natvis file injection, we need a way
to dump the injected sources of a PDB in a way that is authoritative
(i.e. based on Microsoft's understanding of the PDB format, and not
LLVM's). To this end, I've added support for dumping injected sources
via DIA. I made a PDB file that used the /natvis option to generate a
test case.

Differential Revision: https://reviews.llvm.org/D44405

llvm-svn: 327428
2018-03-13 17:46:06 +00:00
Jonas Devlieghere b4c85cf4a4 [DebugInfo] Replace unreachable with None
Invalid user input should not trigger assertions and unreachables. We
already return an Option so we should just return None here.

Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=5532

llvm-svn: 327274
2018-03-12 14:45:08 +00:00
Pavel Labath 47c3472c41 [DebugInfo/AccelTable] Fix inconsistency in getDIEOffset implementations
Summary:
Even though the getDIEOffset offset function was common for the two
accelerator table implementations, it was doing two different things:
for the Apple tables, it was returning the die offset relative to the
start of the section, whereas for DWARF v5 tables, it was relative to
the start of the CU.

I resolve this by renaming the function to getDIESectionOffset to make
it obvious what the function returns, and change the DWARF
implementation to return the section offset. I also keep the CU-relative
accessor, but only in the DWARF implementation (there is no way to get
this information for the Apple tables). This was not caught by existing
tests because the hand-written inputs also erroneously used section
offsets instead of CU-relative ones.

While looking at this, I noticed that the Apple implementation was not
fully correct either -- the header contains a DIEOffsetBase field, which
should be added to offsets encoded with the DW_FORM_ref*** family, but
this was not being used. This went unnoticed because all current writers
set this field to zero anyway. I fix this as well and add a hand-written
test which demonstrates the issue.

Reviewers: JDevlieghere, dblaikie

Subscribers: aprantl, llvm-commits

Differential Revision: https://reviews.llvm.org/D44202

llvm-svn: 327116
2018-03-09 11:58:59 +00:00
Jonas Devlieghere 6921753994 [Support] Move syntax highlighting into support
Move the DWARF syntax highlighting into support. This has several
advantages, most notably that this makes the WithColor RAII wrapper
available outside libDebugInfo. Furthermore, several projects all have
their own code for handling colored output. This provides a place to
centralize it.

Differential revision: https://reviews.llvm.org/D44215

llvm-svn: 327108
2018-03-09 09:56:24 +00:00
Benjamin Kramer 70e6faaa0d [DebugInfo] Move RangeListEntries instead of copying.
This is needed for correctness as RangeListEntry is not copy-assignable,
which std::vector might rely on.

llvm-svn: 327067
2018-03-08 21:31:10 +00:00
Zachary Turner 145bc6e0d3 Fix compilation failure with MSVC.
llvm-svn: 327063
2018-03-08 21:07:30 +00:00
Wolfgang Pieb a0729d4126 [DWARF v5] Support for verbose dumping of .debug_rnglist entries
Adding verbose dumping to the recent implementation of dumping of v5 range list entries. 
We're capturing the entries as is as they come in during extraction, including their file offset,
so we can dump them in more detail.
The offset table entries which are table-relative are shown as is (as in non-verbose mode)
and with the actual file offset they map to.

Reviewers: dblaikie, aprantl, jdevlieghere, jhenderson

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D43366

llvm-svn: 327059
2018-03-08 20:52:35 +00:00
Pavel Labath b136c3934e DWARFVerifier: Basic verification of .debug_names
Summary:
This patch adds basic .debug_names verification capabilities to the
DWARF verifier. Right now, it checks that the headers and abbreviation
tables of the individual name indexes can be parsed correctly, it
verifies the buckets table and the cross-checks the CU lists for
consistency. I intend to add further checks in follow-up patches.

Reviewers: JDevlieghere, aprantl, probinson, dblaikie

Subscribers: vleschuk, echristo, clayborg, llvm-commits

Differential Revision: https://reviews.llvm.org/D44211

llvm-svn: 327011
2018-03-08 15:34:42 +00:00
James Henderson 667026297d [DWARF] Don't attempt to parse line tables at invalid offsets
Whilst working on improvements to the error handling of the debug line
parsing code, I noticed that if an invalid offset were to be specified
in a call to getOrParseLineTable(), an entry in the LineTableMap would
still be created, even if the offset was not within the section range.
The immediate parsing attempt afterwards would fail (it would end up
getting a version of 0), and thereafter, any subsequent calls to
getOrParseLineTable or getLineTable would return the default-
constructed, invalid line table. In reality, we shouldn't even attempt
to parse this table, and we should always return a nullptr from these
two functions for this situation.

I have tested this via a unit test, which required some new framework
for unit testing debug line. My plan is to add quite a few more unit
tests for the new error reporting mechanism that will follow shortly,
hence the reason why the supporting code for the tests are written the
way they are - I intend to extend the DwarfGenerator class to support
generating debug line. At that point, I'll make sure that there are a
few positive test cases for this and the parsing code too.

Differential Revision: https://reviews.llvm.org/D44200

Reviewers: JDevlieghere, aprantl
llvm-svn: 326995
2018-03-08 10:53:34 +00:00
Rafael Auler 86fb7bf2bc Reland "[DebugInfo] Support DWARF expressions in eh_frame"
Summary:
Original change was D43313 (r326932) and reverted by r326953 because it
broke an LLD test and a windows build. The LLD test was already fixed in
lld commit r326944 (thanks maskray). This is the original change with
the windows build fixed.

llvm-svn: 326970
2018-03-08 00:46:53 +00:00
Eugene Zemtsov c4a13015fd Fix build broken by r326959
Adding Demangle to link time dependencies of Symbolize

llvm-svn: 326964
2018-03-08 00:07:26 +00:00
Eugene Zemtsov cd72cbc667 Use itaniumDemangle in llvm-symbolizer
Currently on Windows (_MSC_VER) LLVMSymbolizer supports only Microsoft mangling.
This fix just explicitly uses itaniumDemangle when mangled name starts with _Z.

Differential Revision: https://reviews.llvm.org/D44192

llvm-svn: 326959
2018-03-07 23:07:34 +00:00
Rui Ueyama 6aa8b3491f Revert r326932: [DebugInfo] Support DWARF expressions in eh_frame
This reverts commit rr326932 because it broke lld/test/ELF/eh-frame-hdr-augmentation.s.

llvm-svn: 326953
2018-03-07 22:29:48 +00:00
Rafael Auler 7fdf44440c [DebugInfo] Support DWARF expressions in eh_frame
This patch enhances DWARFDebugFrame with the capability of parsing and
printing DWARF expressions in CFI instructions. It also makes FDEs and
CIEs accessible to lib users, so they can process them in client tools
that rely on LLVM. To make it self-contained with a test case, it
teaches llvm-readobj to be able to dump EH frames and checks they are
correct in a unit test. The llvm-readobj code is Maksim Panchenko's work
(maksfb).

Reviewers: JDevlieghere, espindola

Reviewed By: JDevlieghere

Differential Revision: https://reviews.llvm.org/D43313

llvm-svn: 326932
2018-03-07 19:19:51 +00:00
Jonas Devlieghere bf8596f9cf [dwarfdump] Only print CU relative offset in verbose mode
Instead of only printing the CU-relative offset in non-verbose mode, it
makes more sense to only printed the resolved address. In verbose mode
we still print both.

Differential revision: https://reviews.llvm.org/D44148

rdar://33525475

llvm-svn: 326903
2018-03-07 16:28:53 +00:00
Aaron Smith 25409ddf2a [DebugInfoPDB] Add DIA implementation for getSrcLineOnTypeDefn
Summary: This helps to determine the line number for a PDB type with definition

Reviewers: zturner, llvm-commits, rnk

Reviewed By: zturner

Subscribers: rengolin, JDevlieghere

Differential Revision: https://reviews.llvm.org/D44119

llvm-svn: 326857
2018-03-07 00:33:09 +00:00
Francis Ricci fe6cbceec2 [llvm-symbolizer] Use correct path when resolving .gnu_debuglink in .debug
Summary:
The symbolizer was checking for .debug as a subdirectory of the
binary file itself, not of the directory containing the binary. This led to
a failure to find split debug info when it was contained in a .debug directory.

Reviewers: rnk, glider, zturner

Subscribers: llvm-commits, aprantl

Differential Revision: https://reviews.llvm.org/D44025

llvm-svn: 326630
2018-03-02 22:56:45 +00:00
Zachary Turner c6a75a69f1 [PDB] Defer writing the build id until the rest of the PDB is written.
For now this is NFC, but this small refactor opens the door to
letting us embed a hash of the PDB in the build id field of the
PDB.

Differential Revision: https://reviews.llvm.org/D43913

llvm-svn: 326453
2018-03-01 18:00:29 +00:00
Reid Kleckner 3acdc67734 [CodeView] Lower __restrict and other pointer qualifiers correctly
Qualifiers on a pointer or reference type may apply to either the
pointee or the pointer itself. Consider 'const char *' and 'char *
const'. In the first example, the pointee data may not be modified
without casts, and in the second example, the pointer may not be updated
to point to new data.

In the general case, qualifiers are applied to types with LF_MODIFIER
records, which support the usual const and volatile qualifiers as well
as the __unaligned extension qualifier.

However, LF_POINTER records, which are used for pointers, references,
and member pointers, have flags for qualifiers applying to the
*pointer*. In fact, this is the only way to represent the restrict
qualifier, which can only apply to pointers, and cannot qualify regular
data types.

This patch causes LLVM to correctly fold 'const' and 'volatile' pointer
qualifiers into the pointer record, as well as adding support for
'__restrict' qualifiers in the same place.

Based on a patch from Aaron Smith

Differential Revision: https://reviews.llvm.org/D43060

llvm-svn: 326260
2018-02-27 22:08:15 +00:00
Reid Kleckner 22d838cd31 [codeview] Remove unused variable
llvm-svn: 326253
2018-02-27 21:46:40 +00:00
Pavel Labath d99072bc97 Implement equal_range for the DWARF v5 accelerator table
Summary:
This patch implements the name lookup functionality of the .debug_names
accelerator table and hooks it up to "llvm-dwarfdump -find". To make the
interface of the two kinds of accelerator tables more consistent, I've
created an abstract "DWARFAcceleratorTable::Entry" class, which provides
a consistent interface to access the common functionality of the table
entries (such as getting the die offset, die tag, etc.). I've also
modified the apple table to vend entries conforming to this interface.

Reviewers: JDevlieghere, aprantl, probinson, dblaikie

Subscribers: vleschuk, clayborg, echristo, llvm-commits

Differential Revision: https://reviews.llvm.org/D43067

llvm-svn: 326003
2018-02-24 00:35:21 +00:00
Scott Linder 16c7bdaf32 [DebugInfo] Support DWARF v5 source code embedding extension
In DWARF v5 the Line Number Program Header is extensible, allowing values with
new content types. In this extension a content type is added,
DW_LNCT_LLVM_source, which contains the embedded source code of the file.

Add new optional attribute for !DIFile IR metadata called source which contains
source text. Use this to output the source to the DWARF line table of code
objects. Analogously extend METADATA_FILE in Bitcode and .file directive in ASM
to support optional source.

Teach llvm-dwarfdump and llvm-objdump about the new values. Update the output
format of llvm-dwarfdump to make room for the new attribute on file_names
entries, and support embedded sources for the -source option in llvm-objdump.

Differential Revision: https://reviews.llvm.org/D42765

llvm-svn: 325970
2018-02-23 23:01:06 +00:00
Aaron Smith 89a19ac38d [PDB] Check the result of setLoadAddress()
Summary: Change setLoadAddress() to return true or false on failure.

Reviewers: zturner, llvm-commits

Reviewed By: zturner

Differential Revision: https://reviews.llvm.org/D43638

llvm-svn: 325843
2018-02-23 00:02:27 +00:00
Aaron Smith 9161a6cb25 [PDB] Fix buildbot failure from missing include for DIAEnumLineNumbers
llvm-svn: 325826
2018-02-22 20:00:07 +00:00
Aaron Smith fbe65404fd [PDB] Implement more find methods for PDB symbols
Summary:
Add additional find methods on PDB raw symbols.

findChildrenByAddr()
findChildrenByVA()
findInlineFramesByAddr()
findInlineFramesByVA()
findInlineLines()
findInlineLinesByAddr()
findInlineLinesByRVA()
findInlineLinesByVA()




Reviewers: zturner, llvm-commits

Reviewed By: zturner

Differential Revision: https://reviews.llvm.org/D43637

llvm-svn: 325824
2018-02-22 19:47:43 +00:00
Jonas Devlieghere 7d4a974d8b [dwarfdump] Fix spurious verification errors for DW_AT_location attributes
Verifying any DWARF file that is optimized and contains at least one tag
with a DW_AT_location with a location list offset as a
DW_AT_form_dataXXX results in dwarfdump spuriously claiming that the
location list is invalid.

Differential revision: https://reviews.llvm.org/D40199

llvm-svn: 325430
2018-02-17 13:06:37 +00:00
Zachary Turner cafd476836 Fix emission of PDB string table.
This was originally reported as a bug with the symptom being "cvdump
crashes when printing an LLD-linked PDB that has an S_FILESTATIC record
in it". After some additional investigation, I determined that this was
a symptom of a larger problem, and in fact the real problem was in the
way we emitted the global PDB string table. As evidence of this, you can
take any lld-generated PDB, run cvdump -stringtable on it, and it would
return no results.

My hypothesis was that cvdump could not *find* the string table to begin
with. Normally it would do this by looking in the "named stream map",
finding the string /names, and using its value as the stream index. If
this lookup fails, then cvdump would fail to load the string table.

To test this hypothesis, I looked at the name stream map generated by a
link.exe PDB, and I emitted exactly those bytes into an LLD-generated
PDB. Suddenly, cvdump could read our string table!

This code has always been hacky and we knew there was something we
didn't understand. After all, there were some comments to the effect of
"we have to emit strings in a specific order, otherwise things don't
work". The key to fixing this was finally understanding this.

The way it works is that it makes use of a generic serializable hash map
that maps integers to other integers. In this case, the "key" is the
offset into a buffer, and the value is the stream number. If you index
into the buffer at the offset specified by a given key, you find the
name. The underlying cause of all these problems is that we were using
the identity function for the hash. i.e. if a string's offset in the
buffer was 12, the hash value was 12. Instead, we need to hash the
string *at that offset*. There is an additional catch, in that we have
to compute the hash as a uint32 and then truncate it to uint16.

Making this work is a little bit annoying, because we use the same hash
table in other places as well, and normally just using the identity
function for the hash function is actually what's desired. I'm not
totally happy with the template goo I came up with, but it works in any
case.

The reason we never found this bug through our own testing is because we
were building a /parallel/ hash table (in the form of an
llvm::StringMap<>) and doing all of our lookups and "real" hash table
work against that. I deleted all of that code and now everything goes
through the real hash table. Then, to test it, I added a unit test which
adds 7 strings and queries the associated values. I test every possible
insertion order permutation of these 7 strings, to verify that it really
does work as expected.

Differential Revision: https://reviews.llvm.org/D43326

llvm-svn: 325386
2018-02-16 20:46:04 +00:00
David Blaikie 3b6de6fe1c Revert "Rewrite the cached map used for locating the most precise DIE among inlined subroutines for a given address."
Seeing some inlining missing in internal uses of symbolizer. I'll work
on a reproduction, tests, improvements & recommit as soon as possible.

(Chandler would like it to be known that this improvement did make
check-llvm 4x faster... - so there's certainly some fairly good
motivation to push on fixing/figuring this out & getting it back in)

This reverts commit r321345.

llvm-svn: 324981
2018-02-13 01:52:30 +00:00
Adrian Prantl a5eee4de07 Simplify switch statement (NFC)
llvm-svn: 324945
2018-02-12 22:09:57 +00:00
Adrian Prantl 520789e139 Fix the syntax highlighting of strings in dwarfdump.
llvm-svn: 324936
2018-02-12 21:11:23 +00:00
Adrian Prantl baf9c20b02 Factor out common condition into an easier to understand helper function (NFC).
llvm-svn: 324935
2018-02-12 21:11:14 +00:00
Paul Robinson ceafcd41cf [DWARFv5] Fix dumper to show the file table starts at index 0.
Emitting the correct (root of compilation) file at index 0 will be
posted for review later; I wanted to get this minor change out of the
way first.

llvm-svn: 324669
2018-02-08 23:08:02 +00:00
Paul Robinson 0a22709f06 [DWARF] Regularize dumping strings from line tables.
The major visible difference here is that in line-table dumps,
directory and file names are wrapped in double-quotes; previously,
directory names got single quotes and file names were not quoted at
all.

The improvement in this patch is that when a DWARF v5 line table
header has indirect strings, in a verbose dump these will all have
their section[offset] printed as well as the name itself.  This
matches the format used for dumping strings in the .debug_info
section.

Differential Revision: https://reviews.llvm.org/D42802

llvm-svn: 324270
2018-02-05 20:43:15 +00:00
James Henderson 10392cdbf7 Fix more print format specifiers in debug_rnglists dumping
See also r324096.

I have made the assumption that DWARF64 is not an issue for the time
being with these fixes.

llvm-svn: 324223
2018-02-05 10:47:13 +00:00
Simon Pilgrim 3d5ee7aa41 Fix MSVC signed/unsigned comparison warning. NFCI.
llvm-svn: 324171
2018-02-03 12:38:56 +00:00
James Henderson 2465633234 Fix type sizes that were causing incorrect string formatting
llvm-svn: 324096
2018-02-02 15:09:31 +00:00
James Henderson c2dfd502a2 Add missing new files from r324077
Differential Revision: https://reviews.llvm.org/D42481

llvm-svn: 324078
2018-02-02 12:45:57 +00:00
James Henderson 3fcc74500a [DWARF v5] Add limited support for dumping .debug_rnglists
This change adds support to llvm-dwarfdump for dumping DWARF5
.debug_rnglists sections in regular ELF files.

It is not complete, in that several DW_RLE_* encodings are currently
not supported, but does dump the headert and the basic ranges for
DW_RLE_start_length and DW_RLE_start_end encodings.

Obvious next steps are to add verbose dumping that dumps the raw
encodings, rather than the interpreted contents, to add -verify support
of the section (e.g. to show that the correct number of offsets are
specified), add dumping of .debug_rnglists.dwo, and to add support for
other encodings.

Reviewed by: dblaikie, JDevlieghere

Differential Revision: https://reviews.llvm.org/D42481

llvm-svn: 324077
2018-02-02 12:35:52 +00:00
Zachary Turner 07d803777c [CodeView] Micro-optimizations to speed up type merging.
Based on a profile, a couple of hot spots were identified in the
main type merging loop.  The code was simplified, a few loops
were re-arranged, and some outlined functions were inlined.  This
speeds up type merging by a decent amount, shaving around 3-4 seconds
off of a 40 second link in my test case.

Differential Revision: https://reviews.llvm.org/D42559

llvm-svn: 323790
2018-01-30 17:12:04 +00:00
Paul Robinson d0c89f851b Stop tracking .debug_line_str in DWARFUnit. NFC.
llvm-svn: 323701
2018-01-29 22:02:56 +00:00
Paul Robinson bf750c80e9 [DWARFv5] Re-enable dumping a line table with no CU.
r323476 added support for DW_FORM_line_strp, and incorrectly made that
depend on having a DWARFUnit available.  We shouldn't be tracking
.debug_line_str in DWARFUnit after all.  After this patch, I can do an
NFC follow up and undo a bunch of the "plumbing" part of r323476.

Differential Revision: https://reviews.llvm.org/D42609

llvm-svn: 323691
2018-01-29 20:57:43 +00:00
Pavel Labath e7264106d4 Fix windows test failure caused by r323638
The test was failing because of an incorrect sizeof check in the name
index parsing code. This code was meant to check that we have enough
input to parse the fixed-size part of the dwarf header, which it did by
comparing the input to sizeof(Header). Originally struct Header only
contained the fixed-size part, but during review, we've moved additional
members into it, which rendered the sizeof check invalid.

I resolve this by moving the fixed-size part to a separate struct and
updating the sizeof-expression to use that.

llvm-svn: 323648
2018-01-29 13:53:48 +00:00
Pavel Labath 3460957ea3 Fix build broken by r323641
The call to ScopedPrinter::printNumber with size_t argument was
ambiguous (I think) on 32-bit builds. Explicitly cast to a 64-bit int to
avoid this.

llvm-svn: 323642
2018-01-29 11:53:46 +00:00
Pavel Labath 394e805668 Refactor dwarfdump -apple-names output
Summary:
This modifies the dwarfdump output to align it with the new .debug_names
dump. It also renames two header fields to match similar fields in the
dwarf5 header.

A couple of tests needed to be updated to match new output. The changes
were fairly straight-forward, although not really automatable.

Reviewers: JDevlieghere, aprantl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D42415

llvm-svn: 323641
2018-01-29 11:33:17 +00:00
Pavel Labath 3c9a918c9e [DebugInfo] Basic .debug_names dumping support
Summary:
This commit renames DWARFAcceleratorTable to AppleAcceleratorTable to free up
the first name as an interface for the different accelerator tables.
Then I add a DWARFDebugNames class for the dwarf5 table.

Presently, the only common functionality of the two classes is the dump()
method, because this is the only method that was necessary to implement
dwarfdump -debug-names; and because the rest of the
AppleAcceleratorTable interface does not directly transfer to the dwarf5
tables (the main reason for that is that the present interface assumes
the tables are homogeneous, but the dwarf5 tables can have different
keys associated with each entry).

I expect to make the common interface richer as I add more functionality
to the new class (and invent a way to represent it in generic way).

In terms of sharing the implementation, I found the format of the two
tables sufficiently different to frustrate any attempts to have common
parsing or dumping code, so presently the implementations share just low
level code for formatting dwarf constants.

Reviewers: vleschuk, JDevlieghere, clayborg, aprantl, probinson, echristo, dblaikie

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D42297

llvm-svn: 323638
2018-01-29 11:08:32 +00:00
Jonas Devlieghere 92ac9d3e1b [Support] Move DJB hash to support. NFC
This patch moves the DJB hash to support. This is consistent with other
hashing algorithms living there. The hash is used by the DWARF
accelerator tables. We're doing this now because the hashing function is
needed by dsymutil and we don't want to link against libBinaryFormat.

Differential revision: https://reviews.llvm.org/D42594

llvm-svn: 323616
2018-01-28 11:05:10 +00:00
Paul Robinson 0115844c2f [DWARFv5] Classify all the new forms. NFC.
Move standard forms from a switch statement to the table of forms;
fill in all the missing ones defined in DWARF v5.  I'm guessing at
classifications in a couple of cases where v5 forms aren't actually
supported yet, but whoever adds support for the forms can fix the
classifications as needed.

llvm-svn: 323481
2018-01-25 23:06:36 +00:00
Paul Robinson b6aa01ca99 [DWARFv5] Support DW_FORM_line_strp in llvm-dwarfdump.
This form is like DW_FORM_strp, but points to .debug_line_str instead
of .debug_str as the string section.  It's intended to be used from
the line-table header, and allows string-pooling of directory and
filenames across compilation units.

Differential Revision: https://reviews.llvm.org/D42553

llvm-svn: 323476
2018-01-25 22:02:36 +00:00
Pavel Labath 9b36fd2541 Rename DwarfAcceleratorTable to AppleAcceleratorTable. NFC
This frees up the first name to be used as an base class for the
apple table and the dwarf5 .debug_names accel table. The rename  was
split off from D42297 (adding of debug_names support), which is still
under review.

llvm-svn: 323113
2018-01-22 13:17:23 +00:00
Paul Robinson 8181d23b3d [DWARFv5] Number the line-table's directory array correctly.
The compilation directory has always been #0, but as of DWARF v5 it is
explicitly listed in the line-table section instead of implicitly
being a reference to the compile_unit DIE's DW_AT_comp_dir attribute.
This means the dumper should number the dumped array starting with 0
or 1 depending on the DWARF version of the line table.

References in the generated DWARF are correct, it's just the dumper
that was wrong.  Also some assembler-coded tests were similarly
confused about directory numbers.

llvm-svn: 322884
2018-01-18 20:33:35 +00:00
Zachary Turner 1bc2ce6b9b Speed up iteration of CodeView record streams.
There's some abstraction overhead in the underlying
mechanisms that were being used, and it was leading to an
abundance of small but not-free copies being made.  This
showed up on a profile.  Eliminating this and going back to
a low-level byte-based implementation speeds up lld with
/DEBUG between 10 and 15%.

Differential Revision: https://reviews.llvm.org/D42148

llvm-svn: 322871
2018-01-18 18:35:01 +00:00
Aaron Smith 53a1a1616c Fix pretty printing the unspecified param of a variadic function
Summary:
 - Fix a bug in PrettyBuiltinDumper that returns "void" as the name for
  an unspecified builtin type. Since the unspecified param of a variadic
  function is considered a builtin of unspecified type in PDBs, we set
  "..." for its name.

  - Provide a method to determine if a PDBSymbolFunc is variadic in
  PrettyFunctionDumper since PDBSymbolFunc::getArgument() doesn't return the
  last unspecified-type param.

  - Add a pretty-func-dumper.test to test pretty dumping of variadic
  functions.

Reviewers: zturner, llvm-commits

Reviewed By: zturner

Differential Revision: https://reviews.llvm.org/D41801

llvm-svn: 322608
2018-01-17 01:22:03 +00:00
Jonas Devlieghere 6f24c8778c [DebugInfo] Unify dumping of address ranges
Summary:
This patch unifies the printing of address ranges as [0x0, 0x1).

rdar://34822059

Reviewers: aprantl, dblaikie

Subscribers: mehdi_amini, llvm-commits

Differential Revision: https://reviews.llvm.org/D42056

llvm-svn: 322543
2018-01-16 11:17:57 +00:00
Adrian Prantl 146ed408f4 dwarfdump: Match the --uuid output with that of Darwin dwarfdump.
This option is widely used by scripts and there is no reason to break them.

rdar://problem/36032398

llvm-svn: 321901
2018-01-05 21:44:17 +00:00
Zachary Turner de6a487d70 [MSF] Fix FPM interval calcluation
We have some code to try to determine how many pieces an MSF
Free Page Map is split into, and this code had an off by one
error which would cause the calculation to be incorrect when
there were exactly 4096*k + 1 blocks in an MSF file.

Original investigation and patch outline by Colden Cullen.

Differential Revision: https://reviews.llvm.org/D41742

llvm-svn: 321880
2018-01-05 18:12:14 +00:00
Jonas Devlieghere cbf651f739 [DebugInfo] Don't crash when given invalid DWARFv5 line table prologue.
This patch replaces an assertion with an explicit check for the validity
of the FORM parameters. The assertion was triggered when the DWARFv5
line table contained a zero address size.

This fixes OSS-Fuzz Issue 4644
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4644

Differential revision: https://reviews.llvm.org/D41615

llvm-svn: 321863
2018-01-05 10:03:02 +00:00
Chandler Carruth 54a5ad3681 Rewrite the cached map used for locating the most precise DIE among
inlined subroutines for a given address.

This is essentially the hot path of llvm-symbolizer when extracting
inlined frames during symbolization. Previously, we would read every
subprogram and every inlined subroutine, building a std::map across the
entire PC space to the best DIE, and then do only a handful of queries
as we symbolized a backtrace. A huge fraction of the time was spent
building the map itself.

This patch changes it two a two-level system. First, we just build a map
from PC-interval to DWARF subprograms. These are required to be disjoint
and so constructing this is pretty easy. Second, we build a map *just*
for the inlined subroutines within the subprogram containing the query
address. This allows us to look at far fewer DIEs and build a *much*
smaller set of cached maps in the llvm-symbolizer case where only a few
address get symbolized during the entire run.

It also builds both interval maps in a very different way. It constructs
a single flat vector of pairs that maps from offset -> index. The
indices point into collections of DIE objects, but can also be
"tombstones" (-1) to mark gaps. In the case of subprograms, this mostly
just simplifies the data structure a bit. For inlined subroutines,
because we carefully split them as we build the map, we end up in many
cases having no holes and not having to store both start and stop
offsets.

Finally, the PC ranges for the inlined subroutines are compressed into
32-bits by making them relative to the base PC of the outer subprogram.
This means that if you have a single function body with over 2gb of
executable code in it, we will stop mapping address past the first 2gb
of that function into inlined subroutines and just give you the
subprogram. This doesn't seem like a problem. ;]

All of this combines to make llvm-symbolizer *well* over 2x faster for
symbolizing backtraces out of LLVM's unittests. Death-test heavy unit
tests are running >2x faster. I'm still going to look at completely
disabling symbolization there, but figured while I had a good benchmark
we should make symbolization a bit better.

Sadly, the logic to build the flat interval map for the inlined
subroutines is fairly complex. I'm not super happy about this and
welcome any simplifying suggestions.

Huge thanks to Dave Blaikie who helped walk me through what the various
things I needed to do in DWARF to make this work.

Differential Revision: https://reviews.llvm.org/D40987

llvm-svn: 321345
2017-12-22 06:41:23 +00:00
Wolfgang Pieb b4ba1aa486 [DWARF] Fix formatting bug with r321295. This fixes a MIPS buildbot failure.
llvm-svn: 321330
2017-12-22 01:12:24 +00:00
Wolfgang Pieb 6ecd6a8088 [DWARF v5] Rework of string offsets table reader
Reorganizes the DWARF consumer to derive the string offsets table 
contribution's format from the contribution header instead of 
(incorrectly) from the unit's format.

Reviewers: JDevliegehere, aprantl

Differential Revision: https://reviews.llvm.org/D41146
 

llvm-svn: 321295
2017-12-21 19:38:13 +00:00
Adrian Prantl 0e6694d111 Silence a bunch of implicit fallthrough warnings
llvm-svn: 321114
2017-12-19 22:05:25 +00:00
Paul Robinson a06f8dcca6 Recommit "[DWARFv5] Dump an MD5 checksum in the line-table header."
Adds missing support for DW_FORM_data16.

Update of r320852/r320886, fixing the unittest again, this time use a
raw char string for the test data.

Differential Revision: https://reviews.llvm.org/D41090

llvm-svn: 321011
2017-12-18 19:08:35 +00:00
Paul Robinson 6d0484f2b6 Revert "Recommit "[DWARFv5] Dump an MD5 checksum in the line-table header.""
This reverts commit 0afef672f63f0e4e91938656bc73424a8c058bfc.
Still failing at runtime on bots.

llvm-svn: 320888
2017-12-15 23:21:52 +00:00
Paul Robinson 5c8f7d7de4 Recommit "[DWARFv5] Dump an MD5 checksum in the line-table header."
Adds missing support for DW_FORM_data16.

Update of r320852, fixing the unittest to use a hand-coded struct
instead of std::array to guarantee data layout.

Differential Revision: https://reviews.llvm.org/D41090

llvm-svn: 320886
2017-12-15 22:57:17 +00:00
Paul Robinson 67ca67d1b2 Revert "[DWARFv5] Dump an MD5 checksum in the line-table header."
Unit test fails on some bots.

llvm-svn: 320857
2017-12-15 20:29:25 +00:00
Paul Robinson 72546fe87b [DWARFv5] Dump an MD5 checksum in the line-table header.
Adds missing support for DW_FORM_data16.

Differential Revision: https://reviews.llvm.org/D41090

llvm-svn: 320852
2017-12-15 19:52:34 +00:00
Zachary Turner 0d07a8e948 [COFF] Teach LLD to use the COFF .debug$H section.
This adds the /DEBUG:GHASH option to LLD which will look for
the existence of .debug$H sections in linker inputs and use them
to accelerate type merging.  The clang-cl side has already been
added, so this completes the work necessary to begin experimenting
with this feature.

Differential Revision: https://reviews.llvm.org/D40980

llvm-svn: 320719
2017-12-14 18:07:04 +00:00
Zachary Turner 048f8f99bf [CodeView] Teach clang to emit the .debug$H COFF section.
Currently this is an LLVM extension to the COFF spec which is
experimental and intended to speed up linking.  For now it is
behind a hidden cl::opt flag, but in the future we can move it
to a "real" cc1 flag and have the driver pass it through whenever
it is appropriate.

The patch to actually make use of this section in lld will come
in a followup.

Differential Revision: https://reviews.llvm.org/D40917

llvm-svn: 320649
2017-12-13 22:33:58 +00:00
Michael Zolotukhin 0c169bf7f7 Remove redundant includes from lib/DebugInfo.
llvm-svn: 320620
2017-12-13 21:30:49 +00:00
Jonas Devlieghere ba915897da [dwarfdump] Fix off-by-one bug in accelerator table extractor.
This fixes a bug where the verifier was complaining about empty
accelerator tables. When the table is empty, its size is not a valid
offset as it points after the end of the section.

This patch also makes the extractor return llvm:Error instead of bool
for better error reporting in the verifier.

Differential revision: https://reviews.llvm.org/D41063

rdar://35932007

llvm-svn: 320399
2017-12-11 18:22:47 +00:00
Adrian Prantl 01fb31cc89 dwarfdump: Add support for the --diff option.
--diff      Emit the output in a diff-friendly way by omitting offsets and
            addresses.

<rdar://problem/34502625>

llvm-svn: 320214
2017-12-08 23:32:47 +00:00
Zachary Turner ecd2684ed7 [DebugInfo] Fix register variables not showing up in pdb.
Previously, when linking against libcmt from the MSVC runtime,
lld-link /verbose would show "Ignoring unknown symbol record
with kind 0x1006".  It turns out this was because
TypeIndexDiscovery did not handle S_REGISTER records, so these
records were not getting properly remapped.

Patch by: Alexnadre Ganea
Differential Revision: https://reviews.llvm.org/D40919

llvm-svn: 320108
2017-12-07 22:51:16 +00:00
Zachary Turner 376d437776 Teach llvm-pdbutil to dump types from object files.
llvm-svn: 319859
2017-12-05 23:58:18 +00:00
Zachary Turner 023f88ef42 Fix -Wmissing-braces error.
llvm-svn: 319855
2017-12-05 23:19:33 +00:00
Zachary Turner 87b78e9d1e [CodeView] Add support for content hashing CodeView type records.
Currently nothing uses this, but this at least gets the core
algorithm in, and adds some test to demonstrate correctness.

Differential Revision: https://reviews.llvm.org/D40736

llvm-svn: 319854
2017-12-05 23:08:58 +00:00
Paul Robinson ab69b477a9 [DebugInfo] Bail out if making no progress dumping line tables.
llvm-svn: 319564
2017-12-01 18:25:30 +00:00
Zachary Turner f0e4c6a819 Simplify the DenseSet used for hashing CodeView records.
This was storing the hash alongside the key so that the hash
doesn't need to be re-computed every time, but in doing so it
was allocating a structure to keep the key size small in the
DenseMap.  This is a noble goal, but it also leads to a pointer
indirection on every probe, and this cost of this pointer
indirection ends up being higher than the cost of having a
slightly larger entry in the hash table.  Removing this not only
simplifies the code, but yields a small but noticeable
performance improvement in the type merging algorithm.

llvm-svn: 319493
2017-11-30 23:00:30 +00:00
Zachary Turner ca6dbf1440 Split TypeTableBuilder into two classes.
llvm-svn: 319456
2017-11-30 18:39:50 +00:00
Zachary Turner 52d036e693 [CodeView] Factor some code out of TypeTableBuilder.
This class had some code that would automatically remap type
indices before hashing and serializing.  The only caller of
this method was the TypeStreamMerger anyway, and the method
doesn't make general sense, and prevents making certain future
improvements to the class.  So, factoring this up one level
into the TypeStreamMerger where it belongs.

llvm-svn: 319377
2017-11-29 22:41:56 +00:00
Zachary Turner 3e3936da93 Make TypeTableBuilder inherit from TypeCollection.
A couple of places in LLD were passing references to
TypeTableCollections around, which makes it hard to change the
implementation at runtime.  However, these cases only needed to
iterate over the types in the collection, and TypeCollection
already provides a handy abstract interface for this purpose.

By implementing this interface, we can get rid of the need to
pass TypeTableBuilder references around, which should allow us
to swap the implementation at runtime in subsequent patches.

llvm-svn: 319345
2017-11-29 19:35:21 +00:00
Adrian Prantl 5da51f435a llvm-dwarfdump: honor the --show-children option when dumping a specific DIE.
llvm-svn: 319271
2017-11-29 01:12:22 +00:00
Zachary Turner 4c1fa68590 Fix a warning.
llvm-svn: 319263
2017-11-29 00:13:44 +00:00
Zachary Turner 29b081dcd1 [NFC] Minor cleanups in CodeView TypeTableBuilder.
llvm-svn: 319260
2017-11-28 23:57:13 +00:00
Rafael Espindola bba7f862d8 Fix non assert build warnings.
llvm-svn: 319200
2017-11-28 18:50:08 +00:00
Zachary Turner 6900de1dfb [CodeView] Refactor / Rewrite TypeSerializer and TypeTableBuilder.
The motivation behind this patch is that future directions require us to
be able to compute the hash value of records independently of actually
using them for de-duplication.

The current structure of TypeSerializer / TypeTableBuilder being a
single entry point that takes an unserialized type record, and then
hashes and de-duplicates it is not flexible enough to allow this.

At the same time, the existing TypeSerializer is already extremely
complex for this very reason -- it tries to be too many things. In
addition to serializing, hashing, and de-duplicating, ti also supports
splitting up field list records and adding continuations. All of this
functionality crammed into this one class makes it very complicated to
work with and hard to maintain.

To solve all of these problems, I've re-written everything from scratch
and split the functionality into separate pieces that can easily be
reused. The end result is that one class TypeSerializer is turned into 3
new classes SimpleTypeSerializer, ContinuationRecordBuilder, and
TypeTableBuilder, each of which in isolation is simple and
straightforward.

A quick summary of these new classes and their responsibilities are:

- SimpleTypeSerializer : Turns a non-FieldList leaf type into a series of
  bytes. Does not do any hashing. Every time you call it, it will
  re-serialize and return bytes again. The same instance can be re-used
  over and over to avoid re-allocations, and in exchange for this
  optimization the bytes returned by the serializer only live until the
  caller attempts to serialize a new record.

- ContinuationRecordBuilder : Turns a FieldList-like record into a series
  of fragments. Does not do any hashing. Like SimpleTypeSerializer,
  returns references to privately owned bytes, so the storage is
  invalidated as soon as the caller tries to re-use the instance. Works
  equally well for LF_FIELDLIST as it does for LF_METHODLIST, solving a
  long-standing theoretical limitation of the previous implementation.

- TypeTableBuilder : Accepts sequences of bytes that the user has already
  serialized, and inserts them by de-duplicating with a hash table. For
  the sake of convenience and efficiency, this class internally stores a
  SimpleTypeSerializer so that it can accept unserialized records. The
  same is not true of ContinuationRecordBuilder. The user is required to
  create their own instance of ContinuationRecordBuilder.

Differential Revision: https://reviews.llvm.org/D40518

llvm-svn: 319198
2017-11-28 18:33:17 +00:00
Greg Clayton d6b67eb15c Fixed the ability to recursively get an attribute value from a DWARFDie.
The previous implementation would only look 1 DW_AT_specification or DW_AT_abstract_origin deep. This means DWARFDie::getName() would fail in certain cases. I ran into such a case while creating a tool that used the LLVM DWARF parser to generate a symbolication format so I have seen this in the wild.

Differential Revision: https://reviews.llvm.org/D40156

llvm-svn: 319104
2017-11-27 22:12:44 +00:00