Commit Graph

2117 Commits

Author SHA1 Message Date
Amy Huang 7669f3c0f6 Recommit "[CodeView] Emit static data members as S_CONSTANTs."
We used to only emit static const data members in CodeView as
S_CONSTANTS when they were used; this patch makes it so they are always emitted.

This changes CodeViewDebug.cpp to find the static const members from the
class debug info instead of creating DIGlobalVariables in the IR
whenever a static const data member is used.

Bug: https://bugs.llvm.org/show_bug.cgi?id=47580

Differential Revision: https://reviews.llvm.org/D89072

This reverts commit 504615353f.
2020-10-28 16:35:59 -07:00
Amy Huang 504615353f Revert "[CodeView] Emit static data members as S_CONSTANTs."
Seems like there's an assert in here that we shouldn't be running into.

This reverts commit 515973222e.
2020-10-27 11:29:58 -07:00
Amy Huang 515973222e [CodeView] Emit static data members as S_CONSTANTs.
We used to only emit static const data members in CodeView as
S_CONSTANTS when they were used; this patch makes it so they are always emitted.

I changed CodeViewDebug.cpp to find the static const members from the
class debug info instead of creating DIGlobalVariables in the IR
whenever a static const data member is used.

Bug: https://bugs.llvm.org/show_bug.cgi?id=47580

Differential Revision: https://reviews.llvm.org/D89072
2020-10-26 15:30:35 -07:00
David Blaikie 0ec5baa132 llvm-dwarfdump: Support verbose printing DW_OP_convert to print the CU local offset before the resolved absolute offset 2020-10-23 18:50:15 -07:00
David Blaikie a67d164a82 Revert several changes related to llvm-symbolizer exiting non-zero on failure.
Seems users have enough different uses of the symbolizer where they
might have unknown binaries and offsets such that "best effort" behavior
is all that's expected of llvm-symbolizer - so even erroring on unknown
executables and out of bounds offsets might not be suitable.

This reverts commit 1de0199748.
This reverts commit a7b209a6d4.
This reverts commit 338dd138ea.
2020-10-21 15:21:44 -07:00
Luqman Aden 51892a42da [COFF][ARM] Fix CodeView for Windows on 32bit ARM targets.
Create the LLVM / CodeView register mappings for the 32-bit ARM Window targets.

Reviewed By: compnerd

Differential Revision: https://reviews.llvm.org/D89622
2020-10-19 22:16:16 -07:00
David Blaikie a7b209a6d4 llvm-symbolizer: Exit non-zero when DWARF parsing errors have been rendered 2020-10-14 23:42:00 -07:00
David Blaikie 9670a45c98 libDebugInfoDWARF: Don't try to parse loclist[.dwo] headers when parsing debug_info[.dwo]
There's no way to know whether there's a loclist contribution to parse
if there's no loclistx encoding - and if there is one, there's no need
to walk back from the loclist_base (or, uin the case of
info.dwo/loclist.dwo - starting at 0 in the contribution) to parse the
header, instead rely on the DWARF32/64 and address size in the CU
that's already available.

This would come up in split DWARF (non-split wouldn't try to read a
loclist header in the absence of a loclist_base) when one unit had
location lists and another does not (because the loclists.dwo section
would be non-empty in that case - in the case where it's empty the
parsing would silently skip).

Simplify the testing a bit, rather than needing a whole dwp, etc - by
creating a malformed loclists.dwo section (and use single file Split
DWARF) that would trip up any attempt to parse it - but no attempt
should be made.
2020-10-13 22:28:59 -07:00
Greg Clayton a4b842e294 Show register names in DWARF unwind info.
Register context information was already being passed into the DWARFDebugFrame code that dumps unwind information but it wasn't being used. This change adds the ability to dump registers names of a valid MC register context was passed in and if it knows about the register. Updated the tests to use the newly returned register names.

Differential Revision: https://reviews.llvm.org/D88767
2020-10-05 15:34:33 -07:00
David Blaikie 6d0be74af5 llvm-dwarfdump: Don't try to parse rnglist tables when dumping CUs
It's not possible to do this in complete generality - a CU using a
sec_offset DW_AT_ranges has no way of knowing where its rnglists
contribution starts, so should not attempt to parse any full rnglist
table/header to do so. And even using FORM_rnglistx there's no need to
parse the header - the offset can be computed using the CU's DWARF
format (32 or 64) to compute offset entry sizes, and then the list
parsed at that offset without ever trying to find a rnglist contribution
header immediately prior to the rnglists_base.
2020-10-04 19:18:14 -07:00
David Blaikie 92c45e4ee2 llvm-dwarfdump: Add support for DW_RLE_startx_endx 2020-10-04 17:50:43 -07:00
David Blaikie 628a319475 llvm-dwarfdump: Print addresses in debug_line to the parsed address size 2020-10-04 16:05:49 -07:00
David Blaikie ea83e0b17e llvm-dwarfdump: Dump address forms in their encoded length rather than always in 64 bits
Few places did this already - refactor them all into a common helper.
2020-10-04 15:48:57 -07:00
David Blaikie 8036cf7f54 llvm-dwarfdump: Skip tombstoned address ranges
Make the dumper & API a bit more informative by using the new tombstone
addresses to filter out or otherwise render more explicitly dead code
ranges.
2020-10-04 13:43:29 -07:00
Reid Kleckner 5519e4da83 Re-land "[PDB] Merge types in parallel when using ghashing"
Stored Error objects have to be checked, even if they are success
values.

This reverts commit 8d250ac3cd.
Relands commit 49b3459930655d879b2dc190ff8fe11c38a8be5f..

Original commit message:
-----------------------------------------

This makes type merging much faster (-24% on chrome.dll) when multiple
threads are available, but it slightly increases the time to link (+10%)
when /threads:1 is passed. With only one more thread, the new type
merging is faster (-11%). The output PDB should be identical to what it
was before this change.

To give an idea, here is the /time output placed side by side:
                              BEFORE    | AFTER
  Input File Reading:           956 ms  |  968 ms
  Code Layout:                  258 ms  |  190 ms
  Commit Output File:             6 ms  |    7 ms
  PDB Emission (Cumulative):   6691 ms  | 4253 ms
    Add Objects:               4341 ms  | 2927 ms
      Type Merging:            2814 ms  | 1269 ms  -55%!
      Symbol Merging:          1509 ms  | 1645 ms
    Publics Stream Layout:      111 ms  |  112 ms
    TPI Stream Layout:          764 ms  |   26 ms  trivial
    Commit to Disk:            1322 ms  | 1036 ms  -300ms
----------------------------------------- --------
Total Link Time:               8416 ms    5882 ms  -30% overall

The main source of the additional overhead in the single-threaded case
is the need to iterate all .debug$T sections up front to check which
type records should go in the IPI stream. See fillIsItemIndexFromDebugT.
With changes to the .debug$H section, we could pre-calculate this info
and eliminate the need to do this walk up front. That should restore
single-threaded performance back to what it was before this change.

This change will cause LLD to be much more parallel than it used to, and
for users who do multiple links in parallel, it could regress
performance. However, when the user is only doing one link, it's a huge
improvement. In the future, we can use NT worker threads to avoid
oversaturating the machine with work, but for now, this is such an
improvement for the single-link use case that I think we should land
this as is.

Algorithm
----------

Before this change, we essentially used a
DenseMap<GloballyHashedType, TypeIndex> to check if a type has already
been seen, and if it hasn't been seen, insert it now and use the next
available type index for it in the destination type stream. DenseMap
does not support concurrent insertion, and even if it did, the linker
must be deterministic: it cannot produce different PDBs by using
different numbers of threads. The output type stream must be in the same
order regardless of the order of hash table insertions.

In order to create a hash table that supports concurrent insertion, the
table cells must be small enough that they can be updated atomically.
The algorithm I used for updating the table using linear probing is
described in this paper, "Concurrent Hash Tables: Fast and General(?)!":
https://dl.acm.org/doi/10.1145/3309206

The GHashCell in this change is essentially a pair of 32-bit integer
indices: <sourceIndex, typeIndex>. The sourceIndex is the index of the
TpiSource object, and it represents an input type stream. The typeIndex
is the index of the type in the stream. Together, we have something like
a ragged 2D array of ghashes, which can be looked up as:
  tpiSources[tpiSrcIndex]->ghashes[typeIndex]

By using these side tables, we can omit the key data from the hash
table, and keep the table cell small. There is a cost to this: resolving
hash table collisions requires many more loads than simply looking at
the key in the same cache line as the insertion position. However, most
supported platforms should have a 64-bit CAS operation to update the
cell atomically.

To make the result of concurrent insertion deterministic, the cell
payloads must have a priority function. Defining one is pretty
straightforward: compare the two 32-bit numbers as a combined 64-bit
number. This means that types coming from inputs earlier on the command
line have a higher priority and are more likely to appear earlier in the
final PDB type stream than types from an input appearing later on the
link line.

After table insertion, the non-empty cells in the table can be copied
out of the main table and sorted by priority to determine the ordering
of the final type index stream. At this point, item and type records
must be separated, either by sorting or by splitting into two arrays,
and I chose sorting. This is why the GHashCell must contain the isItem
bit.

Once the final PDB TPI stream ordering is known, we need to compute a
mapping from source type index to PDB type index. To avoid starting over
from scratch and looking up every type again by its ghash, we save the
insertion position of every hash table insertion during the first
insertion phase. Because the table does not support rehashing, the
insertion position is stable. Using the array of insertion positions
indexed by source type index, we can replace the source type indices in
the ghash table cells with the PDB type indices.

Once the table cells have been updated to contain PDB type indices, the
mapping for each type source can be computed in parallel. Simply iterate
the list of cell positions and replace them with the PDB type index,
since the insertion positions are no longer needed.

Once we have a source to destination type index mapping for every type
source, there are no more data dependencies. We know which type records
are "unique" (not duplicates), and what their final type indices will
be. We can do the remapping in parallel, and accumulate type sizes and
type hashes in parallel by type source.

Lastly, TPI stream layout must be done serially. Accumulate all the type
records, sizes, and hashes, and add them to the PDB.

Differential Revision: https://reviews.llvm.org/D87805
2020-09-30 15:44:38 -07:00
Reid Kleckner 8d250ac3cd Revert "[PDB] Merge types in parallel when using ghashing"
This reverts commit 49b3459930.
2020-09-30 14:55:32 -07:00
Reid Kleckner 49b3459930 [PDB] Merge types in parallel when using ghashing
This makes type merging much faster (-24% on chrome.dll) when multiple
threads are available, but it slightly increases the time to link (+10%)
when /threads:1 is passed. With only one more thread, the new type
merging is faster (-11%). The output PDB should be identical to what it
was before this change.

To give an idea, here is the /time output placed side by side:
                              BEFORE    | AFTER
  Input File Reading:           956 ms  |  968 ms
  Code Layout:                  258 ms  |  190 ms
  Commit Output File:             6 ms  |    7 ms
  PDB Emission (Cumulative):   6691 ms  | 4253 ms
    Add Objects:               4341 ms  | 2927 ms
      Type Merging:            2814 ms  | 1269 ms  -55%!
      Symbol Merging:          1509 ms  | 1645 ms
    Publics Stream Layout:      111 ms  |  112 ms
    TPI Stream Layout:          764 ms  |   26 ms  trivial
    Commit to Disk:            1322 ms  | 1036 ms  -300ms
----------------------------------------- --------
Total Link Time:               8416 ms    5882 ms  -30% overall

The main source of the additional overhead in the single-threaded case
is the need to iterate all .debug$T sections up front to check which
type records should go in the IPI stream. See fillIsItemIndexFromDebugT.
With changes to the .debug$H section, we could pre-calculate this info
and eliminate the need to do this walk up front. That should restore
single-threaded performance back to what it was before this change.

This change will cause LLD to be much more parallel than it used to, and
for users who do multiple links in parallel, it could regress
performance. However, when the user is only doing one link, it's a huge
improvement. In the future, we can use NT worker threads to avoid
oversaturating the machine with work, but for now, this is such an
improvement for the single-link use case that I think we should land
this as is.

Algorithm
----------

Before this change, we essentially used a
DenseMap<GloballyHashedType, TypeIndex> to check if a type has already
been seen, and if it hasn't been seen, insert it now and use the next
available type index for it in the destination type stream. DenseMap
does not support concurrent insertion, and even if it did, the linker
must be deterministic: it cannot produce different PDBs by using
different numbers of threads. The output type stream must be in the same
order regardless of the order of hash table insertions.

In order to create a hash table that supports concurrent insertion, the
table cells must be small enough that they can be updated atomically.
The algorithm I used for updating the table using linear probing is
described in this paper, "Concurrent Hash Tables: Fast and General(?)!":
https://dl.acm.org/doi/10.1145/3309206

The GHashCell in this change is essentially a pair of 32-bit integer
indices: <sourceIndex, typeIndex>. The sourceIndex is the index of the
TpiSource object, and it represents an input type stream. The typeIndex
is the index of the type in the stream. Together, we have something like
a ragged 2D array of ghashes, which can be looked up as:
  tpiSources[tpiSrcIndex]->ghashes[typeIndex]

By using these side tables, we can omit the key data from the hash
table, and keep the table cell small. There is a cost to this: resolving
hash table collisions requires many more loads than simply looking at
the key in the same cache line as the insertion position. However, most
supported platforms should have a 64-bit CAS operation to update the
cell atomically.

To make the result of concurrent insertion deterministic, the cell
payloads must have a priority function. Defining one is pretty
straightforward: compare the two 32-bit numbers as a combined 64-bit
number. This means that types coming from inputs earlier on the command
line have a higher priority and are more likely to appear earlier in the
final PDB type stream than types from an input appearing later on the
link line.

After table insertion, the non-empty cells in the table can be copied
out of the main table and sorted by priority to determine the ordering
of the final type index stream. At this point, item and type records
must be separated, either by sorting or by splitting into two arrays,
and I chose sorting. This is why the GHashCell must contain the isItem
bit.

Once the final PDB TPI stream ordering is known, we need to compute a
mapping from source type index to PDB type index. To avoid starting over
from scratch and looking up every type again by its ghash, we save the
insertion position of every hash table insertion during the first
insertion phase. Because the table does not support rehashing, the
insertion position is stable. Using the array of insertion positions
indexed by source type index, we can replace the source type indices in
the ghash table cells with the PDB type indices.

Once the table cells have been updated to contain PDB type indices, the
mapping for each type source can be computed in parallel. Simply iterate
the list of cell positions and replace them with the PDB type index,
since the insertion positions are no longer needed.

Once we have a source to destination type index mapping for every type
source, there are no more data dependencies. We know which type records
are "unique" (not duplicates), and what their final type indices will
be. We can do the remapping in parallel, and accumulate type sizes and
type hashes in parallel by type source.

Lastly, TPI stream layout must be done serially. Accumulate all the type
records, sizes, and hashes, and add them to the PDB.

Differential Revision: https://reviews.llvm.org/D87805
2020-09-30 14:22:48 -07:00
David Blaikie 0328feb086 DebugInfo: Filter DWARFv5 TUs out of the debug_info unit list when CUs requested
Since DWARFv5 places TUs in debug_info, some of DWARFContext's APIs have
become a bit erroneous, including TUs in the CU list by accident.
Correct that by providing compile_units (& dwo_compile_units) that
filter out the type units from the debug_info units.

Differential Revision: https://reviews.llvm.org/D87935
2020-09-23 22:15:53 -07:00
Jonas Devlieghere e1ef7183c6 [dwarfdump] Warn for tags with DW_CHILDREN_yes but no children.
Flag DIEs that have DW_CHILDREN_yes set in their abbreviation but don't
actually have any children.

rdar://59809554

Differential revision: https://reviews.llvm.org/D88048
2020-09-23 22:12:04 -07:00
David Blaikie ad68a8b952 DebugInfo: Cleanup RLE dumping, using a length-constrained DataExtractor rather than carrying the end offset separately 2020-09-18 19:32:38 -07:00
David Blaikie 51a505340d DebugInfo: Simplify line table parsing to take all the units together, rather than CUs and TUs separately 2020-09-18 11:18:23 -07:00
David Blaikie e0802fe016 DebugInfo: Tidy up initializing multi-section contributions in DWARFContext 2020-09-18 10:54:43 -07:00
Simon Pilgrim ed53ff4cde SymbolizableObjectFile.h - remove unnecessary includes. NFCI.
Use forward declarations where possible, move includes down to SymbolizableObjectFile.cpp and avoid duplicate includes.
2020-09-17 13:18:53 +01:00
Reid Kleckner e47d2927de Include (Type|Symbol)Record.h less
Most clients only need CVType and CVSymbol, not structs for every type
and symbol. Move CVSymbol and CVType to CVRecord.h to accomplish this.
Update some of the common headers that need CVSymbol and CVType to use
the new location.
2020-09-16 09:59:03 -07:00
Petr Hosek 9c73e55510 Revert "[DebugInfo] Remove dots from getFilenameByIndex return value"
This is failing on Windows bots due to path separator normalization.

This reverts commit 042c235068.
2020-09-15 10:06:47 -07:00
Petr Hosek 042c235068 [DebugInfo] Remove dots from getFilenameByIndex return value
When concatenating directory with filename in getFilenameByIndex, we
might end up with a path that contains extra dots. For example, if the
input is /path and ./example, we would return /path/./example. Run
sys::path::remove_dots on the output to eliminate unnecessary dots.

Differential Revision: https://reviews.llvm.org/D87657
2020-09-14 20:19:06 -07:00
David Blaikie 69da27c749 llvm-symbolizer: Add optional "start file" to match "start line"
Since a function might have portions of its code coming from multiple
different files, "start line" is ambiguous (it can't just be resolved
relative to the file/line specified). Add start file to disambiguate it.
2020-09-08 15:40:58 -07:00
Xing GUO 67ce11405b [llvm-dwarfdump] Warn user when it encounters no null terminated strings.
When llvm-dwarfdump encounters no null terminated strings, we should
warn user about it rather than ignore it and print nothing.

Before this patch, when llvm-dwarfdump dumps a .debug_str section whose
content is "abc", it prints:

```
.debug_str contents:
```

After this patch:

```
.debug_str contents:
warning: no null terminated string at offset 0x0
```

Reviewed By: jhenderson, MaskRay

Differential Revision: https://reviews.llvm.org/D86998
2020-09-03 08:49:57 +08:00
Xing GUO 369f9169a5 [DebugInfo] Simplify string table dumpers.
This patch adds a helper function DumpStrSection to simplify codes.
Besides, nonprintable chars in debug_str and debug_str.dwo sections
are printed as escaped chars.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D86918
2020-09-02 08:41:10 +08:00
Jordan Rupprecht 202766947e [NFC] Fix unused var in release builds.
This was always unused, but the change in D86354 upgraded this to a compiler warning.
2020-09-01 16:38:24 -07:00
David Blaikie f7a49d2aa6 [WIP][DebugInfo] Lazily parse debug_loclist offsets
Parsing DWARFv5 debug_loclist offsets when a CU is parsed is weighing
down memory usage of symbolizers that don't need to parse this data at
all. There's not much benefit to caching these anyway - since they are
O(1) lookup and reading once you know where the offset list starts (and
can do bounds checking with the offset list size too).

In general, I think it might be time to start paying down some of the
technical debt of loc/loclist/range/rnglist parsing to try to unify it a
bit more.

eg:

* Currently DWARFUnit has: RangeSection, RangeSectionBase, LocSection,
  LocSectionBase, LocTable, RngListTable, LoclistTableHeader (be nice if
  these were all wrapped up in two variables - one for loclists, one for
  rnglists)

* rnglists and loclists are handled differently (see:
  LoclistTableHeader, but no RnglistTableHeader)

* maybe all these types could be less stateful - lazily parse what they
  need to, even reparsing rather than caching because it doesn't seem
  too expensive, for instance. (though admittedly so long as it's
  constantcost/overead per compilatiton that's probably adequate)

* Maybe implementing and using a DWARFDataExtractor that can be
  sub-ranged (so we could slice it up to just the single contribution) -
  though maybe that's not so useful because loc/ranges need to refer to
  it by absolute, not contribution-relative mechanisms

Differential Revision: https://reviews.llvm.org/D86110
2020-08-18 10:49:39 -07:00
Igor Kudrin 95fad44e34 [DebugInfo] Avoid an infinite loop with a truncated pre-v5 .debug_str_offsets.dwo.
dumpStringOffsetsSection() expects the size of a contribution to be
correctly aligned. The patch adds the corresponding verifications for
pre-v5 cases.

Differential Revision: https://reviews.llvm.org/D85739
2020-08-14 13:11:37 +07:00
Igor Kudrin 9ceb192e14 [llvm-dwarfdump] Avoid crashing if an abbreviation offset is invalid.
Note that DWARFUnit::getAbbreviations() returns nullptr if the
abbreviations could not be read, but callers used the returned
pointer without checking.

Differential Revision: https://reviews.llvm.org/D85738
2020-08-12 16:01:53 +07:00
David Stenberg 91bd9db2cd [DebugInfo] Allow GNU macro extension to be read
Allow the GNU .debug_macro extension to be parsed and printed by
llvm-dwarfdump. In an upcoming patch support will be added for emitting
that format also.

Reviewed By: dblaikie

Differential Revision: https://reviews.llvm.org/D82974
2020-08-11 13:30:52 +02:00
James Henderson ca05601cd2 [DebugInfo] Don't error for zero-length arange entries
Although the DWARF specification states that .debug_aranges entries
can't have length zero, these can occur in the wild. There's no
particular reason to enforce this part of the spec, since functionally
they have no impact. The patch removes the error and introduces a new
warning for premature terminator entries which does not stop parsing.

This is a relanding of cb3a598c87, adding the missing obj2yaml part
that was needed.

Fixes https://bugs.llvm.org/show_bug.cgi?id=46805. See also
https://reviews.llvm.org/D71932 which originally introduced the error.

Reviewed by: ikudrin, dblaikie, Higuoxing

Differential Revision: https://reviews.llvm.org/D85313
2020-08-10 14:57:52 +01:00
Nico Weber bc5d68dd8a Revert "[DebugInfo] Don't error for zero-length arange entries"
This reverts commit cb3a598c87.
Breaks build of check-llvm dep obj2yaml everywhere.
2020-08-10 08:20:35 -04:00
James Henderson cb3a598c87 [DebugInfo] Don't error for zero-length arange entries
Although the DWARF specification states that .debug_aranges entries
can't have length zero, these can occur in the wild. There's no
particular reason to enforce this part of the spec, since functionally
they have no impact. The patch removes the error and introduces a new
warning for premature terminator entries which does not stop parsing.

Fixes https://bugs.llvm.org/show_bug.cgi?id=46805. See also
https://reviews.llvm.org/D71932 which originally introduced the error.

Reviewed by: ikudrin, dblaikie

Differential Revision: https://reviews.llvm.org/D85313
2020-08-10 12:48:31 +01:00
Raphael Isemann 1de43bd6df Revert "PDBExtras.h - remove unnecessary raw_ostream forward declaration. NFCI."
This reverts commit 87c5437afd.

The commit includes several headers in the middle of a function, which
breaks pretty much everything.
2020-08-06 15:15:43 +02:00
Simon Pilgrim 87c5437afd PDBExtras.h - remove unnecessary raw_ostream forward declaration. NFCI.
We already need to include raw_ostream.h, also add missing StringRef.h and cstdint implicit dependencies.

Remove unnecessary includes from PDBExtras.cpp
2020-08-06 11:28:42 +01:00
Greg Clayton e1de85f9f4 Add verification for DW_AT_decl_file and DW_AT_call_file.
LTO builds have been creating invalid DWARF and one of the errors was a file index that was out of bounds. "llvm-dwarfdump --verify" will check all file indexes for line tables already, but there are no checks for the validity of file indexes in attributes.

The verification will verify if there is a DW_AT_decl_file/DW_AT_call_file that:
- there is a line table for the compile unit
- the file index is valid
- the encoding is appropriate

Tests are added that test all of the above conditions.

Differential Revision: https://reviews.llvm.org/D84817
2020-08-05 15:30:13 -07:00
Amy Huang 724bf4ee23 [Symbolize][PDB] Switch llvm-symbolizer to use PDB_ReaderType::Native.
Since native PDB reading has been implemented for symbolizing,
switch to using the native PDB reader by default, unless
LLVM_ENABLE_DIA_SDK is on.

Bug: https://bugs.llvm.org/show_bug.cgi?id=41795

Differential Revision: https://reviews.llvm.org/D84286
2020-07-22 17:17:57 -07:00
Amy Huang 0881d0bed3 [PDB][NativeSession] Clean up some things in NativeSession.
-Use the actual sect/offset to keep track of symbols in the cache so they don't get created multiple times with different addresses.
-Remove getSymTag from PDBFunctionSymbol/PDBPublicSymbol because it's already implemented in the base class
-Merge the symbolizer test files for DIA and native, since the tests are the same.
-Implement getCompilandId for NativeLineNumber

Reviewed By: amccarth

Differential Revision: https://reviews.llvm.org/D84208
2020-07-21 16:54:52 -07:00
Fangrui Song 3ab0f53ef3 [DebugInfo] Respect relocations when decoding DW_EH_PE_sdata4 & DW_EH_PE_sdata8 and support R_ARM_REL32
The addresses in llvm-dwarfdump --eh-frame output for object files are closer to readelf -wf output now.
2020-07-18 09:00:50 -07:00
Igor Kudrin ec9f0c7d4a [DebugInfo] Fix a possible crash when reading a malformed .debug_*lists section.
DWARFListTableHeader::length() handles the zero value of HeaderData.Length
in a special way, which makes the result different from the calculated
value of FullLength, which leads to triggering an assertion. The patch
moves the assertion a bit later when `FullLength` is already checked for
minimal allowed value.

Differential Revision: https://reviews.llvm.org/D82886
2020-07-14 16:35:17 +07:00
Oliver Stannard 57909b0a53 [llvm-objdump] Add entry_value and stack_value opcodes
Add the DW_OP_entry_value and DW_OP_stack_value opcodes to the DWARF
expression printer.

Differential revision: https://reviews.llvm.org/D74843
2020-07-14 10:24:59 +01:00
Oliver Stannard 1d7311e052 [llvm-objdump] Add simple memory expressions to variable display
Add the DW_OP_breg0..DW_OP_breg31 and DW_OP_bregx opcodes to the DWARF
expression printer.

Differential revision: https://reviews.llvm.org/D74841
2020-07-14 10:24:59 +01:00
Alexandre Ganea 23cd70d71c [PDB] Fix out-of-bounds acces when sorting GSI buckets
When building in Debug on Windows-MSVC after b7402edce3, a lot of tests were failing because we were dereferencing an element past the end of HashRecords. This happened towards the end of the table, in unused slots.
2020-07-10 10:55:27 -04:00
Benjamin Kramer d36b8414bd [DebugInfo] Fix pessimizing move. NFC.
DWARFDebugPubTable.cpp:80:31: warning: moving a temporary object prevents copy elision [-Wpessimizing-move]
2020-07-09 14:23:46 +02:00
Igor Kudrin ca4d8da0c3 [DebugInfo] Add more checks to parsing .debug_pub* sections.
The patch adds checking for various potential issues in parsing name
lookup tables and reporting them as recoverable errors, similarly as we
do for other tables.

Differential Revision: https://reviews.llvm.org/D83050
2020-07-09 19:15:31 +07:00
Igor Kudrin 68f5a8b204 [DebugInfo] Do not hang when parsing a malformed .debug_pub* section.
The parsing method did not check reading errors and might easily fall
into an infinite loop on an invalid input because of that.

Differential Revision: https://reviews.llvm.org/D83049
2020-07-09 19:15:11 +07:00
Oliver Stannard dc4a6f5db4 [llvm-objdump] Display locations of variables alongside disassembly
This adds the --debug-vars option to llvm-objdump, which prints
locations (registers/memory) of source-level variables alongside the
disassembly based on DWARF info. A vertical line is printed for each
live-range, with a label at the top giving the variable name and
location, and the position and length of the line indicating the program
counter range in which it is valid.

Differential revision: https://reviews.llvm.org/D70720
2020-07-09 09:58:00 +01:00
Georgii Rymar bee8cdcabd [DebugInfo/DWARF] - Test invalid CFI opcodes properly and refine related `CFIProgram::parse` code.
There are following issues with `CFIProgram::parse` code:

1) Invalid CFI opcodes were never tested. And currently a test would fail
when the `LLVM_ENABLE_ABI_BREAKING_CHECKS` is enabled. It happens because
the `DataExtractor::Cursor C` remains unchecked when the
"Invalid extended CFI opcode" error is reported:

```
.eh_frame section at offset 0x1128 address 0x0:
Program aborted due to an unhandled Error:
Error value was Success. (Note: Success values must still be checked prior to being destroyed).
```

2) It is impossible to reach the "Invalid primary CFI opcode" error with the current code.
There are 3 possible primary opcode values and all of them are handled. Hence this error
should be replaced with llvm_unreachable.

3) Errors currently reported are upper-case.

This patch refines the code in the `CFIProgram::parse` method to fix all issues mentioned
and adds unit tests for all possible invalid extended CFI opcodes.

Differential revision: https://reviews.llvm.org/D82868
2020-07-08 12:10:23 +03:00
Amy Huang 9ee90a4905 [NativeSession] Add column numbers to NativeLineNumber.
Summary:
This adds column numbers if they are present, and otherwise
sets the column number to be zero.

Bug: https://bugs.llvm.org/show_bug.cgi?id=41795

Reviewers: amccarth

Subscribers: hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D81950
2020-07-07 09:59:22 -07:00
James Henderson 9e09a54c69 [DebugInfo] Use Cursor to detect errors in debug line prologue parser
Previously, the debug line parser would keep attempting to read data
even if it had run out of data to read. This meant errors in parsing
would often end up being reported as something else, such as an unknown
version or malformed directory/filename table. This patch fixes the
issues by using the Cursor API to capture errors.

Reviewed by: labath

Differential Revision: https://reviews.llvm.org/D83043
2020-07-03 11:52:06 +01:00
Reid Kleckner b7402edce3 [PDB] Defer public serialization until PDB writing
This reduces peak memory on my test case from 1960.14MB to 1700.63MB
(-260MB, -13.2%) with no measurable impact on CPU time. I'm currently
working with a publics stream that is about 277MB. Before this change,
we would allocate 277MB of heap memory, serialize publics into them,
hold onto that heap memory, open the PDB, and commit into it.  After
this change, we defer the serialization until commit time.

In the last change I made to public writing, I re-sorted the list of
publics multiple times in place to avoid allocating new temporary data
structures. Deferring serialization until later requires that we don't
reorder the publics. Instead of sorting the publics, I partially
construct the hash table data structures, store a publics index in them,
and then sort the hash table data structures. Later, I replace the index
with the symbol record offset.

This change also addresses a FIXME and moves the list of global and
public records from GSIHashStreamBuilder to GSIStreamBuilder. Now that
publics aren't being serialized, it makes even less sense to store them
as a list of CVSymbol records. The hash table used to deduplicate
globals is moved as well, since that is specific to globals, and not
publics.

Reviewed By: aganea, hans

Differential Revision: https://reviews.llvm.org/D81296
2020-06-30 11:28:04 -07:00
Georgii Rymar 1e820e82b1 [DebugInfo/DWARF] - Do not hang when CFI are truncated.
Currently when the .eh_frame section is truncated so that
CFI instructions can't be read, it is possible to enter
an infinite loop.

It happens because `CFIProgram::parse` does not handle errors properly.
This patch fixes the issue.

Differential revision: https://reviews.llvm.org/D82017
2020-06-23 14:39:24 +03:00
James Henderson 9782c922cb [DebugInfo] Print line table extended opcode bytes if parsing fails
Previously, if there was an error whilst parsing the operands of an
extended opcode, the operands would be treated as zero and printed. This
could potentially be slightly confusing. This patch changes the
behaviour to print the raw bytes instead.

Reviewed by: ikudrin

Differential Revision: https://reviews.llvm.org/D81570
2020-06-23 10:04:02 +01:00
Greg Clayton ccf5a44917 Fix the verification of DIEs with DW_AT_ranges.
Summary: Previous code would try to verify DW_AT_ranges and if any ranges would overlap, it would stop attributing any ranges after this to the DIE which caused incorrect errors to be reported that a DIE's address ranges were not contained in the parent DIE's ranges. Added a fix and a test.

Reviewers: aprantl, labath, probinson, JDevlieghere, jhenderson

Subscribers: hiraditya, MaskRay, cmtice, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D79962
2020-06-22 13:13:48 -07:00
Anatoly Trosinenko 359fae6eb0 [DebugInfo] Explicitly permit addr_size = 0x02 when parsing DWARF data
Current LLVM implementation uses `MCAsmInfo::CodePointerSize` as addr_size when emitting the DWARF data. llvm-dwarfdump, on the other hand, handles `addr_size`s of 4 and 8 properly and considers all other sizes as an error. This works for most of mainline targets except for MSP430 and AVR.

msp430-gcc v8.3.1 emits DWARF32 with addr_size = 4 (DWARF32 does not imply addr_size = 4, 32 refers to internal offset width of 4 bytes) that is handled by llvm-dwarfdump already. Still, emitting 2-byte target pointers on MSP430 seems correct as well (but not for MSP430X that is supported by msp430-gcc but not by LLVM and has 20-bit address space).

This patch make it possible for MSP430 debug info support to be tested with llvm-dwarfdump.

Differential Revision: https://reviews.llvm.org/D82055
2020-06-22 13:11:55 +03:00
Eric Christopher cf23852587 [Target] As part of using inclusive language within the llvm project,
migrate away from the use of blacklist and whitelist.

This change affects an internal llvm command line option.
2020-06-20 00:06:39 -07:00
Alexandre Ganea 24eff42ba4 [CodeView] Add TypeCollection::replaceType to replace type records post-merging
The API is not called in this patch. This is to simply/support https://reviews.llvm.org/D80833
2020-06-18 09:17:14 -04:00
James Henderson b21794a91c [DebugInfo] Unify Cursor usage for all debug line opcodes
This is a natural extension of the previous changes to use the Cursor
class independently in the standard and extended opcode paths, and in
turn allows delaying error handling until the entire line has been
printed in verbose mode, removing interleaved output in some cases.

Reviewed by: MaskRay, JDevlieghere

Differential Revision: https://reviews.llvm.org/D81562
2020-06-17 09:19:24 +01:00
Igor Kudrin ccbd7e8d46 [DebugInfo] Support parsing and dumping of DWARF64 macro units.
Differential Revision: https://reviews.llvm.org/D81844
2020-06-17 12:57:54 +07:00
Georgii Rymar 66fb3c39cb [DebugInfo/DWARF] - Report .eh_frame sections of version != 1.
Specification (https://refspecs.linuxbase.org/LSB_5.0.0/LSB-Core-generic/LSB-Core-generic/ehframechpt.html#AEN1349)
says that the value of Version field for .eh_frame should be 1.

Though we accept other values and might perform an attempt to read
it as a .debug_frame because of that, what is wrong.

This patch adds a version check.

Differential revision: https://reviews.llvm.org/D81469
2020-06-16 15:46:26 +03:00
Amy Huang f8170d8715 [NativeSession] Implement findLineNumbersByAddress in NativeSession,
which takes an address and a length and returns all lines within that
address range.
2020-06-15 17:05:39 -07:00
James Henderson 1a78904752 [DebugInfo] Report errors for truncated debug line standard opcode
Standard opcodes usually have ULEB128 arguments, so it is generally not
possible to recover from such errors. This patch causes the parser to
stop parsing the table in such situations.

Also don't emit the operands or add data to the table if there is an
error reading these opcodes.

Reviewed by: JDevlieghere

Differential Revision: https://reviews.llvm.org/D81470
2020-06-15 11:50:12 +01:00
David Blaikie 5146fc15fc llvm-dwarfdump: Include unit count in DWP index header dumping
And add comma separators (to be consistent with recent
changes/improvements to the dumping of other section headers) while I'm
here.
2020-06-12 12:40:02 -07:00
Reid Kleckner 1c03389c29 Re-land "Migrate the rest of COFFObjectFile to Error"
This reverts commit 101fbc0138.

Remove leftover debugging attribute.

Update LLDB as well, which was missed before.
2020-06-11 14:46:16 -07:00
Pavel Labath 9ed452f370 [llvm/DWARFDebugLine] Remove spurious full stop from warning messages
Other warnings messages don't have a trailing full stop.
2020-06-11 13:14:21 +02:00
Pavel Labath fccaa89e23 [llvm/DWARFDebugLine] Fix a typo in one warning message 2020-06-11 13:04:52 +02:00
Pavel Labath 6f55b5a101 [DWARFDebugLine] Use truncating data extractors for prologue parsing
Summary:
This makes the code easier to reason about, as it will behave the same
way regardless of whether there is any more data coming after the
presumed end of the prologue.

Reviewers: jhenderson, dblaikie, probinson, ikudrin

Subscribers: hiraditya, MaskRay, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D77557
2020-06-10 16:12:53 +02:00
Fangrui Song 81cca98768 [DebugInfo] Drop unneeded format() calls (fix -Wformat-security) after 3b7ec64d59 2020-06-09 09:56:13 -07:00
James Henderson 3b7ec64d59 [DebugInfo] Fix printing of unrecognised standard opcodes
The verbose printing of unrecognised standard opcodes was broken in
multiple ways (additional blank lines, a closing parenthesis without
opening parenthesis and so on). This patch fixes it, and makes the
output more consistent with other opcodes.
2020-06-09 14:32:20 +01:00
James Henderson e3547ade68 [DebugInfo] Improve new line printing in debug line verbose output
The new line printing for debug line verbose output was inconsistent.
For new rows in the matrix, a blank line followed, whilst the
DW_LNS_copy opcode actually resulted in two blank lines. There was also
potential inconsistency in the blank lines at the end of the table. This
patch mostly resolves these issues - no blank lines appear in the output
except for a single line after the prologue and at table end to separate
it from any subsquent table, plus some instances after error messages.

Also add a unit test for verbose output to test the fine details of new
line placement and other aspects of verbose output.

Reviewed by: dblaikie

Differential Revision: https://reviews.llvm.org/D81102
2020-06-09 14:27:16 +01:00
James Henderson dbd26fe0b6 [DebugInfo] Print non-verbose output at some point as verbose output
Verbose and non-verbose parsing of .debug_line produced their output at
different points in the program. The most obvious impact of this was
that error messages were produced at different times, but it also
potentially reduced what clients could do by customising the stream or
warning/error handlers.

This change makes the two variants consistent by printing non-verbose
output inline, the same as verbose output.

Testing of the error messages has been modified to check the messages
always appear in the same location to illustrate the behaviour.

Reviewed by: JDevlieghere, dblaikie, MaskRay, labath

Differential Revision: https://reviews.llvm.org/D80989
2020-06-09 14:24:53 +01:00
James Henderson 6378167409 [DebugInfo] Remove unnecessary flushes and add additional testing
The flushes previously existed to help ensure consistent error message
output when stdout and stderr were passed to the same location. This is
no longer necessary as errs() is now tied to outs().

Reviewed by: dblaikie, MaskRay, JDevlieghere, labath

Differential Revision: https://reviews.llvm.org/D80803
2020-06-09 13:50:58 +01:00
James Henderson 5777570d24 [DebugInfo] Check for errors when reading data for extended opcode
Previously, if an extended opcode was truncated, it would manifest as an
"unexpected line op length error" which wasn't quite accurate. This
change checks for errors any time data is read whilst parsing an
extended opcode, and reports any errors detected.

Reviewed by: MaskRay, labath, aprantl

Differential Revision: https://reviews.llvm.org/D80797
2020-06-09 09:56:37 +01:00
Nico Weber 101fbc0138 Revert "Migrate the rest of COFFObjectFile to Error"
This reverts commit b5289656b8.
__attribute__((optnone)) doesn't build with msvc, see
http://lab.llvm.org:8011/builders/clang-x64-windows-msvc/builds/16326
2020-06-05 21:20:11 -04:00
Reid Kleckner b5289656b8 Migrate the rest of COFFObjectFile to Error 2020-06-05 16:29:05 -07:00
Fangrui Song 9be3567df2 [llvm-dwarfdump] Add a table header for -debug-line -verbose output
Like non-verbose output, so that it is easy to recognize the `Line,Column,File,ISA,Discriminator` column values.

Reviewed By: JDevlieghere, jhenderson

Differential Revision: https://reviews.llvm.org/D80874
2020-06-04 08:56:17 -07:00
Igor Kudrin c3902b62e6 [DebugInfo] Report the format of type units [10/10]
Differential Revision: https://reviews.llvm.org/D80523
2020-06-02 17:55:31 +07:00
Igor Kudrin def677194c [DebugInfo] Report the format of location and range lists [9/10]
Differential Revision: https://reviews.llvm.org/D80523
2020-06-02 17:55:31 +07:00
Igor Kudrin ec9d7ea4f5 [DebugInfo] Report the format of tables in .debug_pub* sections [8/10]
Differential Revision: https://reviews.llvm.org/D80523
2020-06-02 17:55:31 +07:00
Igor Kudrin da913259c7 [DebugInfo] Report the format of line tables [7/10]
Differential Revision: https://reviews.llvm.org/D80523
2020-06-02 17:55:31 +07:00
Igor Kudrin 4933ab2ccb [DebugInfo] Report the format of call frame information entries [6/10]
Differential Revision: https://reviews.llvm.org/D80523
2020-06-02 17:55:30 +07:00
Igor Kudrin 2ea94c8a42 [DebugInfo] Report the format of address range tables [5/10]
Differential Revision: https://reviews.llvm.org/D80523
2020-06-02 17:55:30 +07:00
Igor Kudrin 48232a4858 [DebugInfo] Report the format of address tables [4/10]
Differential Revision: https://reviews.llvm.org/D80523
2020-06-02 17:55:30 +07:00
Igor Kudrin ef7ada04b1 [DebugInfo] Report the format of compilation units [3/10]
Differential Revision: https://reviews.llvm.org/D80523
2020-06-02 17:55:30 +07:00
Igor Kudrin 6aed60c01c [DebugInfo] Report the format of .debug_names [2/10]
Differential Revision: https://reviews.llvm.org/D80523
2020-06-02 17:55:30 +07:00
Igor Kudrin 5e296e3db7 [DebugInfo] Extract a helper function to return the DWARF format name, NFC [1/10]
Differential Revision: https://reviews.llvm.org/D80523
2020-06-02 17:55:30 +07:00
Sterling Augustine f027cfa37e For --relativenames, ignore directory 0, which is the comp_dir.
Update for upstream comments. Improve test by writing all the debug
info by hand.

Reviewers: dblaikie, jhenderson

Subscribers: hiraditya, MaskRay, rupprecht, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80168
2020-06-01 13:13:37 -07:00
James Henderson e8bcf4ef07 [DebugInfo] Add use of truncating data extractor to debug line parsing
This will ensure that nothing can ever start parsing data from a future
sequence and part-read data will be returned as 0 instead.

Reviewed by: aprantl, labath

Differential Revision: https://reviews.llvm.org/D80796
2020-06-01 12:33:21 +01:00
Igor Kudrin cbec419b3e [DebugInfo] Separate fields with commas in headers of type units (3/3).
For most tables, we already use commas in headers. This set of patches
unifies dumping the remaining ones.

Differential Revision: https://reviews.llvm.org/D80806
2020-06-01 17:40:28 +07:00
Igor Kudrin 2a7af30482 [DebugInfo] Separate fields with commas in headers of compile units (2/3).
For most tables, we already use commas in headers. This set of patches
unifies dumping the remaining ones.

Differential Revision: https://reviews.llvm.org/D80806
2020-06-01 17:40:24 +07:00
Igor Kudrin 937403d684 [DebugInfo] Separate fields with commas in headers of .debug_pub* tables (1/3).
For most tables, we already use commas in headers. This set of patches
unifies dumping the remaining ones.

Differential Revision: https://reviews.llvm.org/D80806
2020-06-01 17:39:48 +07:00
Sourabh Singh Tomar e7102eed20 [DWARF5] Added support for .debug_macro.dwo section in llvm-dwarfdump
This patch extends the parsing and dumping support of llvm-dwarfdump
for debug_macro.dwo section.

Following forms are supported:

 - DW_MACRO_define
 - DW_MACRO_undef
 - DW_MACRO_start_file
 - DW_MACRO_end_file
 - DW_MACRO_define_strx
 - DW_MACRO_undef_strx
 - DW_MACRO_define_strp
 - DW_MACRO_undef_strp

Reviewed by: ikudrin, dblaikie

Differential Revision: https://reviews.llvm.org/D78500
2020-05-30 00:12:50 +05:30
Simon Pilgrim f6417f5db8 FileOutputBuffer.h - remove unused includes. NFC.
Move dependent includes down to source files where necessary.
2020-05-28 14:38:12 +01:00
Georgii Rymar 84c6433586 [DebugInfo] - Fix typo in comment. NFC.
I've forgot to address this bit when landed D80476.
2020-05-27 12:21:19 +03:00
Georgii Rymar 2e365ca2f7 [DebugInfo/llvm-objdump] - Print "ZERO terminator" for terminator entries when dumping .eh_frame.
A CIE with the Length == 0 is a terminator:
https://refspecs.linuxfoundation.org/LSB_5.0.0/LSB-Core-generic/LSB-Core-generic/ehframechpt.html

And GNU objdump recognizes them and prints the following for such entries:

"00000000 ZERO terminator"

This patch teaches llvm-objdump to do the same. I had to update tests to use
"CHECK-NEXT" too.

(Note: it looks perhaps not right that printing is done inside the DebugInfo library,
I'd expect to see the change in the llvm-objdump's code somewhere instead,
but that is how it done atm).

Differential revision: https://reviews.llvm.org/D80476
2020-05-26 12:52:42 +03:00
Georgii Rymar 2569787e44 [DebugInfo] - Fix multiple issues in DWARFDebugFrame::parse().
I've noticed an issue with "Data.getRelocatedValue(...)" call.

it might silently ignore an error when a content is truncated.
That leads to an infinite loop in the code (e.g. llvm-readobj hangs).

After fixing the issue I've found that actually we always tried
to read past the end of a section, even when a content was valid.
It happened because the terminator CIE (a CIE with the length == 0)
was never handled. At first I've tried just to stop adding the terminator
entry (and return), but it does not seem to be correct, because tools like
llvm-objdump might want to print something for such entries
(see comments in the code and test cases).

This patch fixes issues mentioned, provides new test cases for
both llvm-readobj and lib/DebugInfo and adds FIXMEs to existent
test cases related.

Differential revision: https://reviews.llvm.org/D80299
2020-05-26 12:13:13 +03:00