Commit Graph

1254 Commits

Author SHA1 Message Date
Reid Kleckner 037bcd9345 [PDB] Remove stale GSI.h header that I intended to remove in the previous commit
llvm-svn: 309069
2017-07-26 00:58:49 +00:00
Spyridoula Gravani dc635f40bb [DWARF] Generalized verification of .apple_names accelerator table to be applicable to any acceleration table. Added verification for .apple_types, .apple_namespaces and .apple_objc sections.
Differential Revision: https://reviews.llvm.org/D35853

llvm-svn: 309068
2017-07-26 00:52:31 +00:00
Reid Kleckner 14d90fd05c [PDB] Improve GSI hash table dumping for publics and globals
The PDB "symbol stream" actually contains symbol records for the publics
and the globals stream. The globals and publics streams are essentially
hash tables that point into a single stream of records. In order to
match cvdump's behavior, we need to only dump symbol records referenced
from the hash table. This patch implements that, and then implements
global stream dumping, since it's just a subset of public stream
dumping.

Now we shouldn't see S_PROCREF or S_GDATA32 records when dumping
publics, and instead we should see those record in the globals stream.

llvm-svn: 309066
2017-07-26 00:40:36 +00:00
NAKAMURA Takumi 7ddaf3cf88 DWARFVerifier.cpp: Fix -m32 in r308928. Use PRIx64.
llvm-svn: 308949
2017-07-25 05:03:17 +00:00
Spyridoula Gravani e0ba415740 [DWARF] Added verification check for die ranges. If highPC is an address, then it should be greater than lowPC for each range.
Differential Revision: https://reviews.llvm.org/D35733

llvm-svn: 308928
2017-07-24 21:04:11 +00:00
Rafael Espindola 87c3f4a938 Move DWARFSectionMap to a .cpp file.
Thanks to Paul Robinson for the suggestion.

llvm-svn: 308913
2017-07-24 19:34:26 +00:00
Tim Northover fe6be421a7 Revert "Debug: handle dumping the D language."
Reid beat me to it.

llvm-svn: 308902
2017-07-24 17:47:46 +00:00
Tim Northover c7bd8255b9 Debug: handle dumping the D language.
Mostly just to silence a warning about an unhandled case. There don't seem to
be any tests for this operator (at least that I could find).

llvm-svn: 308901
2017-07-24 17:39:44 +00:00
Reid Kleckner e2ba971302 Add missing case to switch
llvm-svn: 308894
2017-07-24 16:30:44 +00:00
Reid Kleckner 898ddf61c0 [codeview] Emit 'D' as the cv source language for D code
This matches DMD:
522263965c/src/ddmd/backend/cv8.c (L199)

Fixes PR33899.

llvm-svn: 308890
2017-07-24 16:16:42 +00:00
Reid Kleckner c85041fe00 Fix DebugInfo/PDB build by adding missing changes
llvm-svn: 308765
2017-07-21 18:32:00 +00:00
Reid Kleckner 686f121a5d [PDB] Dump extra info about the publics stream
This includes the hash table, the address map, and the thunk table and
section offset table. The last two are only used for incremental
linking, which LLD doesn't support, so they are less interesting. The
hash table is particularly important to get right, since this is the one
of the streams that debuggers use to translate addresses to symbols.

llvm-svn: 308764
2017-07-21 18:28:55 +00:00
Spyridoula Gravani c6ef9873ac [DWARF] Generalized verification of .debug_abbrev to be applicable to both .debug_abbrev and .debug_abbrev.dwo sections.
Differential Revision: https://reviews.llvm.org/D35698

llvm-svn: 308703
2017-07-21 00:51:32 +00:00
Spyridoula Gravani 364b535234 [DWARF] Added check that verifies that no abbreviation declaration has more than one attribute with the same name.
SUMMARY

This patch adds a verification check on the abbreviation declarations in the .debug_abbrev section.
The check makes sure that no abbreviation declaration has more than one attributes with the same name.

Differential Revision: https://reviews.llvm.org/D35643

llvm-svn: 308579
2017-07-20 02:06:52 +00:00
Reid Kleckner 388f88070e Use llvm::make_unique once more to avoid ADL ambiguity with std::make_unique
llvm-svn: 308552
2017-07-19 23:42:53 +00:00
Rafael Espindola 2e942fbaef Use llvm::make_unique to try to fix the windows build.
llvm-svn: 308551
2017-07-19 23:38:54 +00:00
Rafael Espindola 3ee9e11acb Remove some leftover DWARFContextInMemory.
Not sure how I missed these on the previous commit.

llvm-svn: 308550
2017-07-19 23:34:59 +00:00
Rafael Espindola c398e67fed Use delegation instead of inheritance.
This changes DwarfContext to delegate to DwarfObject instead of having
pure virtual methods.

With this DwarfContextInMemory is replaced with an implementation of
DwarfObject that is local to a .cpp file.

llvm-svn: 308543
2017-07-19 22:27:28 +00:00
Spyridoula Gravani f6bd788dda [DWARF] Modification of code for the verification of .debug_info section.
Summary:
This patch modifies the handleDebugInfo() function so that we verify the contents of each unit
in the .debug_info section only if its header has been successfully verified.

This change will allow for more/different verification checks depending on the type of the unit since from
dwarf5, the .debug_info section may consist of different types of units.

Subscribers: aprantl

Differential Revision: https://reviews.llvm.org/D35521

llvm-svn: 308245
2017-07-18 01:00:26 +00:00
Reid Kleckner c50349d4c6 [PDB] Finish and simplify TPI hashing
Summary:
This removes the CVTypeVisitor updater and verifier classes. They were
made dead by the minimal type dumping refactoring. Replace them with a
single function that takes a type record and produces a hash. Call this
from the minimal type dumper and compare the hash.

I also noticed that the microsoft-pdb reference repository uses a basic
CRC32 for records that aren't special. We already have an implementation
of that CRC ready to use, because it's used in COFF for ICF.

I'll make LLD call this hashing utility in a follow-up change. We might
also consider using this same hash in type stream merging, so that we
don't have to hash our records twice.

Reviewers: inglorion, ruiu

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D35515

llvm-svn: 308240
2017-07-18 00:33:45 +00:00
Reid Kleckner 67653ee086 [codeview] Fix YAML for LF_TYPESERVER2 by hoisting PDB_UniqueId
Summary:
We were treating the GUIDs in TypeServer2Record as strings, and the
non-ASCII bytes in the GUID would not round-trip through YAML.

We already had the PDB_UniqueId type portably represent a Windows GUID,
but we need to hoist that up to the DebugInfo/CodeView library so that
we can use it in the TypeServer2Record as well as in PDB parsing code.

Reviewers: inglorion, amccarth

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D35495

llvm-svn: 308234
2017-07-17 23:59:44 +00:00
Reid Kleckner 3167480ca6 [codeview] Don't use the type visitor to merge types
Summary:
This didn't do much to speed things up, but it implements a FIXME, and I
think it's a nice simplification. We don't need the record kind switch.
We're doing that ourselves.

Reviewers: ruiu, inglorion

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D35496

llvm-svn: 308213
2017-07-17 20:31:38 +00:00
Reid Kleckner a842cd75e2 [codeview] Remove TypeServerHandler and PDBTypeServerHandler
Summary:
Instead of wiring these through the CVTypeVisitor interface, clients
should inspect the CVTypeArray before visiting it and potentially load
up the type server's TPI stream if they need it.

No tests relied on this functionality because LLD was the only client.

Reviewers: ruiu

Subscribers: mgorny, hiraditya, zturner, llvm-commits

Differential Revision: https://reviews.llvm.org/D35394

llvm-svn: 308212
2017-07-17 20:28:06 +00:00
Reid Kleckner af88a910fd [CodeView] Dump BuildInfoSym and ProcSym type indices
I need to print the type index in hex so that I can match it in
FileCheck for a test I'm writing.

llvm-svn: 308107
2017-07-15 18:10:39 +00:00
Eric Christopher f73870eefa Remove set but not used variables from the debug info verifier code.
llvm-svn: 307987
2017-07-14 01:40:47 +00:00
Spyridoula Gravani 890eedc4e4 [DWARF] Introduce verification for the unit header chain in .debug_info section to llvm-dwarfdump.
This patch adds verification checks for the unit header chain in the .debug_info section.
Specifically, for each unit in the .debug_info section, the verifier checks that:

The unit length is valid (i.e. the unit can actually fit in the .debug_info section)
The dwarf version of the unit is valid
The address size is valid (4 or 8)
The unit type (if the unit is in dwarf5) is valid
The debug_abbrev_offset is valid

llvm-svn: 307975
2017-07-13 23:25:24 +00:00
Reid Kleckner 6597c28d76 [PDB] Fix type server handling for archives
Summary:
This fixes type indices for SDK or CRT static archives. Previously we'd
try to look next to the archive object file path, which would not exist
on the local machine.

Also error out if we can't resolve a type server record. Hypothetically
we can recover from this error by discarding debug info for this object,
but that is not yet implemented.

Reviewers: ruiu, amccarth

Subscribers: aprantl, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D35369

llvm-svn: 307946
2017-07-13 20:12:23 +00:00
Wolfgang Pieb 515d0e5001 [DWARF] Fixing a bug with processing of DWARF v5 indexed strings in Mach-O objects.
Code to convert MachO - specific section debug section names to standard DWARF v5
section names was in the wrong place.

Differential Revision: https://reviews.llvm.org/D35321

llvm-svn: 307872
2017-07-13 01:03:28 +00:00
Rafael Espindola 5e5dfa1fc5 Don't expose a map in the DWARFContext interface.
Doing so is leaking an implementation detail.

I have an implementation that uses the lld infrastructure and doesn't
use a map or object::SectionRef.

llvm-svn: 307846
2017-07-12 21:08:24 +00:00
Reid Kleckner 0962cb2e3a Fix non-Windows build after PDB native builtin type change
Some C++14 features slipped in along with an extra member qualification.

llvm-svn: 307835
2017-07-12 19:46:35 +00:00
Adrian McCarthy 8d090fc531 [PDB] Enable NativeSession to create symbols for built-in types on demand
Summary:
There is a reserved range of type indexes for built-in types (like integers).
This will create a symbol for a built-in type if the caller askes for one by
type index.  This is also plumbing for being able to recall symbols by type
index in general, but user-defined types will come in subsequent patches.

Reviewers: rnk, zturner

Subscribers: mgorny, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D35163

llvm-svn: 307834
2017-07-12 19:38:11 +00:00
Reid Kleckner 8d8888ff42 [codeview] Change readobj symbol dumping format
Avoid duplicating DictScope with hand-written names everywhere.  Print
the S_-prefixed symbol kind for every record. This should make it easier
to search for certain kinds of records when debugging PDB linking.

llvm-svn: 307732
2017-07-11 23:41:41 +00:00
Reid Kleckner 447f677133 [codeview] Fix type index discovery for four symbol records
I encountered these when linking LLD, which uses atls.lib. Those objects
appear to use these uncommon symbol records:

  0x115E S_HEAPALLOCSITE
  0x113D S_ENVBLOCK
  0x1113 S_GTHREAD32
  0x1153 S_FILESTATIC

llvm-svn: 307725
2017-07-11 22:37:25 +00:00
Zachary Turner 7eaf1d96ad [lld/pdb] Create an empty public symbol record stream.
This is part of the continuing effort to increase parity between
LLD and MSVC PDBs.  link still doesn't like our PDBs, so the most
obvious thing to check was whether adding an empty publics stream
would get it to do something else.  It still fails in the same way
but at least this removes one more variable from the equation.
The next logical step would be to try creating an empty globals
stream.

Differential Revision: https://reviews.llvm.org/D35224

llvm-svn: 307598
2017-07-10 22:40:20 +00:00
George Rimar c7147b368a [DWARF] - Rename variable. NFC.
Variable was called 'Name' and contained text
name of relocation type. Problem was that
outside of this error handling scope we already
have different 'Name' variable that contains
section name.

Change helps to avoid confusion. 

llvm-svn: 307530
2017-07-10 10:04:51 +00:00
George Rimar 4be1388ebb [DWARF] - Remove unused variables. NFC.
llvm-svn: 307528
2017-07-10 09:36:44 +00:00
Zachary Turner 3a11fdf8ce [PDB] More changes to bring lld PDBs to parity with MSVC.
1) Don't write a /src/headerblock stream.  This appears to be
   written conditionally by MSVC, but it's not clear what the
   condition is.  For now, just remove it since we dont' know
   what it is anyway and the particular pdb we've checked in
   for the test doesn't have one.
2) Write a valid timestamp for the PDB file signature.  This
   leads to non-reproducible builds, but it matches the default
   behavior of link, so it should be out default as well.  If
   we need reproducibility, we should add a separate command
   line option for it that is off by default.
3) Write an empty FPO stream.  MSVC seems to always write an
   FPO stream.  This change makes the stream directory match
   up, although we still need to make the contents of the FPO
   stream match.

llvm-svn: 307436
2017-07-07 20:25:39 +00:00
Zachary Turner c1e93e5fa4 Fix some differences between lld and MSVC generated PDBs.
A couple of things were different about our generated PDBs.

1) We were outputting the wrong Version on the PDB Stream.
   The version we were setting was newer than what MSVC is setting.
   It's not clear what the implications are, but we change LLD
   to use PdbImplVC70, as MSVC does.
2) For the optional debug stream indices in the DBI Stream, we
   were outputting 0 to mean "the stream is not present".  MSVC
   outputs uint16_t(-1), which is the "correct" way to specify
   that a stream is not present.  So we fix that as well.
3) We were setting the PDB Stream signature to 0.  This is supposed
   to be the result of calling time(nullptr).  Although this leads
   to non-deterministic builds, a better way to solve that is by
   having a command line option explicitly for generating a
   reproducible build, and have the default behavior of lld-link
   match the default behavior of link.

To test this, I'm making use of the new and improved `pdb diff`
sub command.  To make it suitable for writing tests against, I had
to modify the diff subcommand slightly to print less verbose output.
Previously it would always print | <column> | <value1> | <value2> |
which is quite verbose, and the values are fragile.  All we really
want to know is "did we produce the same value as link?"  So I added
command line options to print a single character representing the
result status (different, identical, equivalent), and another to
hide the value display.  Note that just inspecting the diff output
used to write the test, you can see some things that are obviously
wrong.  That is just reflective of the fact that this is the state
of affairs today, not that we're asserting that this is "correct".
We can use this as a starting point to discover differences, fix
them, and update the test.

Differential Revision: https://reviews.llvm.org/D35086

llvm-svn: 307422
2017-07-07 18:45:56 +00:00
Zachary Turner f3b4b2d89d [llvm-pdbutil] Improve diff mode.
We're getting to the point that some MS tools (e.g. DIA) can recognize
our PDBs but others (e.g. link.exe) cannot. I think the way forward is
to improve our tooling to help us find differences more easily. For
example, if we can compile the same program with clang-cl and cl and
have a tool tell us all the places where the PDBs differ, this could
tell us what we're doing wrong. It's tricky though, because there are a
lot of "benign" differences in a PDB. For example, if the string table
in one PDB consists of "foo" followed by "bar" and in the other PDB it
consists of "bar" followed by "foo", this is not necessarily a critical
difference, as long as the uses of these strings also refer to the
correct location. On the other hand, if the second PDB doesn't even
contain the string "foo" at all, this is a critical difference.

diff mode has been in llvm-pdbutil for quite a while, but because of the
above challenge along with some others, it's been hard to make it
useful. I think this patch addresses that. It looks for all the same
things, but it now prints the output in tabular format (carefully
formatted and aligned into tables and fields), and it highlights
critical differences in red, non-critical differences in yellow, and
identical fields in green.  This makes it easy to spot the places we
differ, and the general concept of outputting arbitrary fields in
tabular format can be extended to provide analysis into many of the
different types of information that show up in a PDB.

Differential Revision: https://reviews.llvm.org/D35039

llvm-svn: 307421
2017-07-07 18:45:37 +00:00
Rafael Espindola 7413496f8d Fix variable names. NFC.
llvm-svn: 307406
2017-07-07 15:20:55 +00:00
Rafael Espindola 77e87b3444 Reduce code duplication.
By addding a mapNameToDWARFSection we only need to check section names
in one place.

llvm-svn: 307359
2017-07-07 05:36:53 +00:00
Zachary Turner 6c4bfba8f3 [PDB] Teach libpdb to write DBI Stream ECNames.
Based strictly on the name, this seems to have something to do
width edit & continue.  The goal of this patch has nothing to do
with supporting edit and continue though.  msvc link.exe writes
very basic information into this area even when *not* compiling
with support for E&C, and so the goal here is to bring lld-link
to parity.  Since we cannot know what assumptions standard tools
make about the content of PDB files, we need to be as close as
possible.

This ECNames data structure is a standard PDB string hash table.
link.exe puts a single string into this hash table, which is the
full path to the PDB file on disk.  It then references this string
from the module descriptor for the compiler generated `* Linker *`
module.

With this patch, lld-link will generate the exact same sequence of
bytes as MSVC link for this subsection for a given object file
input (as reported by `llvm-pdbutil bytes -ec`).

llvm-svn: 307356
2017-07-07 05:04:36 +00:00
Hiroshi Inoue ddb34d84c9 fix trivial typos in comments; NFC
llvm-svn: 307004
2017-07-03 06:32:59 +00:00
Eugene Zelenko 4fcfc19976 [CodeView, PDB] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC).
llvm-svn: 306911
2017-06-30 23:06:03 +00:00
Zachary Turner af8c75a8c0 [llvm-pdbutil] Output the symbol offset when dumping.
Type records have a unique type index, but symbol records do
not.  Instead, symbol records refer to other symbol records
by referencing their offset in the symbol stream.  In a sense
this is the analogue of the TypeIndex, but we are not printing
it in the dumper.  Printing it not only gives us more useful
information when manually investigating the contents of a PDB,
but also allows us to write better tests by enabling us to
verify that fields that reference other symbol records do
so correctly.

Differential Revision: https://reviews.llvm.org/D34906

llvm-svn: 306890
2017-06-30 21:35:00 +00:00
Zachary Turner 02a267758e [llvm-pdbutil] Add the ability to dump the dependency tree for a type
Previously we had the -type-index option which would dump the record of
a single, but we had no way to follow the dependency graph backwards and
also dump all dependent types.

Having this option makes test-writing better, because we can limit the
test to only those records that are of importance for the thing we're
trying to test, which allows us to use things like CHECK-NEXT to reduce
fragility.

Differential Revision: https://reviews.llvm.org/D34899

llvm-svn: 306852
2017-06-30 18:15:47 +00:00
Spyridoula Gravani 837c110cb1 [DWARF] Added verification checks for the .apple_names section.
This patch verifies the number of atoms, the validity of the form for each atom, as well as the validity of the
hashdata. For hashdata, we're verifying that the hashdata offset is correct and that the offset in the .debug_info for
each DIE in the hashdata is also valid.

llvm-svn: 306735
2017-06-29 20:13:05 +00:00
Paul Robinson 17536b935a [DWARF] NFC: DWARFDataExtractor combines relocs with DataExtractor.
Requires callers to directly associate relocations with a DataExtractor
used to read data from a DWARF section, which helps a callee not make
assumptions about which section it is reading.
This is the next step in reducing DWARFFormValue's dependence on DWARFUnit.

Differential Revision: https://reviews.llvm.org/D34704

llvm-svn: 306699
2017-06-29 16:52:08 +00:00
George Rimar b25a09d0f5 [DWARF] - Fix message reporting about broken relocation.
Because of mistake introduced in r306517,
wrong variable ("name" instead of "Name") was used
in error message.
As a result it reported section name instead of
relocation name.

This file still needs cleanup to match LLVM coding style
and more tests I think.

llvm-svn: 306677
2017-06-29 14:05:18 +00:00
Eugene Zelenko 8456b16ea9 [CodeView] Fix some Clang-tidy modernize-use-using and Include What You Use warnings; other minor fixes (NFC).
llvm-svn: 306616
2017-06-29 00:05:44 +00:00
Adrian McCarthy bf0afc3246 Introduce symbol cache to PDB NativeSession
Instead of creating symbols directly in the findChildren methods of the native
symbol implementations, they will rely on the NativeSession to act as a factory
for these types.  This lets NativeSession cache the NativeRawSymbols in its
new symbol cache and makes that cache the source of unique IDs for the symbols.

Right now, this affects only NativeCompilandSymbols.  There's no external
change yet, so I think the existing tests are still sufficient.  Coming soon
are patches to extend this to built-in types and enums.

llvm-svn: 306610
2017-06-28 22:47:40 +00:00
George Rimar 1af3cb2912 Recommit "[ELF] - Add ability for DWARFContextInMemory to exit early when any error happen."
With fix in include folder character case:
#include "llvm/Codegen/AsmPrinter.h" -> #include "llvm/CodeGen/AsmPrinter.h"

Original commit message:

Change introduces error reporting policy for DWARFContextInMemory.
New callback provided by client is able to handle error on it's
side and return Halt or Continue.

That allows to either keep current behavior when parser prints all errors
but continues parsing object or implement something very different, like
stop parsing on a first error and report an error in a client style.

Differential revision: https://reviews.llvm.org/D34328

llvm-svn: 306517
2017-06-28 08:21:19 +00:00
George Rimar 7a82cffd68 Revert r306512 "[ELF] - Add ability for DWARFContextInMemory to exit early when any error happen."
It broke BB:

[13/106] 13 0.022 Generating VCSRevision.h
[25/106] 24 1.209 Building CXX object unittests/DebugInfo/DWARF/CMakeFiles/DebugInfoDWARFTests.dir/DWARFDebugInfoTest.cpp.o
FAILED: unittests/DebugInfo/DWARF/CMakeFiles/DebugInfoDWARFTests.dir/DWARFDebugInfoTest.cpp.o 
/home/bb/bin/g++  -DGTEST_HAS_RTTI=0 -DLLVM_BUILD_GLOBAL_ISEL -D_DEBUG -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Iunittests/DebugInfo/DWARF -I../llvm-project/llvm/unittests/DebugInfo/DWARF -Iinclude -I../llvm-project/llvm/include -I../llvm-project/llvm/utils/unittest/googletest/include -I../llvm-project/llvm/utils/unittest/googlemock/include -fPIC -fvisibility-inlines-hidden -m32 -std=c++11 -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wno-maybe-uninitialized -Wdelete-non-virtual-dtor -Wno-comment -ffunction-sections -fdata-sections -O3    -UNDEBUG  -Wno-variadic-macros -fno-exceptions -fno-rtti -MD -MT unittests/DebugInfo/DWARF/CMakeFiles/DebugInfoDWARFTests.dir/DWARFDebugInfoTest.cpp.o -MF unittests/DebugInfo/DWARF/CMakeFiles/DebugInfoDWARFTests.dir/DWARFDebugInfoTest.cpp.o.d -o unittests/DebugInfo/DWARF/CMakeFiles/DebugInfoDWARFTests.dir/DWARFDebugInfoTest.cpp.o -c ../llvm-project/llvm/unittests/DebugInfo/DWARF/DWARFDebugInfoTest.cpp
../llvm-project/llvm/unittests/DebugInfo/DWARF/DWARFDebugInfoTest.cpp:18:37: fatal error: llvm/Codegen/AsmPrinter.h: No such file or directory
 #include "llvm/Codegen/AsmPrinter.h"
                                     ^
compilation terminated.

llvm-svn: 306513
2017-06-28 07:06:17 +00:00
George Rimar 397a70425b [ELF] - Add ability for DWARFContextInMemory to exit early when any error happen.
Change introduces error reporting policy for DWARFContextInMemory.
New callback provided by client is able to handle error on it's
side and return Halt or Continue.

That allows to either keep current behavior when parser prints all errors
but continues parsing object or implement something very different, like
stop parsing on a first error and report an error in a client style.

Differential revision: https://reviews.llvm.org/D34328

llvm-svn: 306512
2017-06-28 06:57:20 +00:00
Paul Robinson d66ee0f9a7 [DWARF] NFC: Make string-offset handling more like address-table handling;
do the indirection and relocation all in the same method.

llvm-svn: 306418
2017-06-27 15:40:18 +00:00
Paul Robinson 36e85a867b [DWARF] NFC: Give DwarfFormat a 1-byte base type.
In particular this reduces DWARFFormParams from 64 to 32 bits; pass it
around by value.

llvm-svn: 306324
2017-06-26 19:52:32 +00:00
Paul Robinson 75c068c50b [DWARF] NFC: Collect info used by DWARFFormValue into a helper.
Some forms have sizes that depend on the DWARF version, DWARF format
(32/64-bit), or the size of an address.  Collect these into a struct
to simplify passing them around.  Require callers to provide one when
they query a form's size.

Differential Revision: http://reviews.llvm.org/D34570

llvm-svn: 306315
2017-06-26 18:43:01 +00:00
Zachary Turner 1affd805fc [pdb] Fix reading of llvm-generated PDBs by cvdump.
If you dump a pdb to yaml, and then round-trip it back to a pdb,
and run cvdump -l <file> on the new pdb, cvdump will generate
output such as this.

*** LINES

** Module: "d:\src\llvm\test\DebugInfo\PDB\Inputs\empty.obj"

Error: Line number corrupted: invalid file id 0
  <Unknown> (MD5), 0001:00000010-0000001A, line/addr pairs = 3

        5 00000010      6 00000013      7 00000018

Note the error message about the corrupted line number.

It turns out that the problem is that cvdump cannot find the
/names stream (e.g. the global string table), and the reason it
can't find the /names stream is because it doesn't understand
the NameMap that we serialize which tells pdb consumers which
stream has the string table.

Some experimentation shows that if we add items to the hash
table in a specific order before serializing it, cvdump can read
it. This suggests that either we're using the wrong hash function,
or we're serializing something incorrectly, but it will take some
deeper investigation to figure out how / why.  For now, this at
least allows cvdump to read our line information (and incidentally,
produces an identical byte sequence to what Microsoft tools
produce when writing the named stream map).

Differential Revision: https://reviews.llvm.org/D34491

llvm-svn: 306233
2017-06-25 03:51:42 +00:00
Zachary Turner fa33282774 [llvm-pdbutil] Dump raw bytes of module symbols and debug chunks.
llvm-svn: 306179
2017-06-23 23:08:57 +00:00
Eugene Zelenko 2db0cfa617 [DebugInfo] Fix some Clang-tidy modernize-use-using and Include What You Use warnings; other minor fixes (NFC).
llvm-svn: 306169
2017-06-23 21:57:40 +00:00
Zachary Turner c2f5b4bfd9 [llvm-pdbutil] Dump raw bytes of type and id records.
llvm-svn: 306167
2017-06-23 21:50:54 +00:00
Zachary Turner dd73968256 [llvm-pdbutil] Dump raw bytes of various DBI stream subsections.
llvm-svn: 306160
2017-06-23 21:11:54 +00:00
Zachary Turner 6c3e41bbd3 [llvm-pdbutil] Dump raw bytes of pdb name map.
This patch dumps the raw bytes of the pdb name map which contains
the mapping of stream name to stream index for the string table
and other reserved streams.

llvm-svn: 306148
2017-06-23 20:18:38 +00:00
Zachary Turner 0b36c3ebd0 [llvm-pdbutil] Add a function for formatting MSF data.
The goal here is to make it possible to display absolute
file offsets when dumping byets from an MSF.  The problem is
that when dumping bytes from an MSF, often the bytes will
cross a block boundary and encounter a discontinuity.  We
can't use the normal formatBinary() function for this because
this would just treat the sequence as entirely ascending, and
not account out-of-order blocks.

This patch adds a formatMsfData() function to our printer, and
then uses this function to improve the output of the -stream-data
command line option for dumping bytes from a particular stream.

Test coverage is also expanded to make sure to include all possible
scenarios of offsets, sizes, and crossing block boundaries.

llvm-svn: 306141
2017-06-23 18:52:13 +00:00
Adrian McCarthy 4aedc81b8c Fix build break by using llvm::make_unique instead of std::make_unique.
llvm-svn: 306043
2017-06-22 18:57:51 +00:00
Adrian McCarthy 31bcb6f680 Add IDs and clone methods to NativeRawSymbol
All NativeRawSymbols will have a unique symbol ID (retrievable via
getSymIndexId).  For now, these are initialized to 0, but soon the
NativeSession will be responsible for creating the raw symbols, and it will
assign unique IDs.

The symbol cache in the NativeSession will also require the ability to clone
raw symbols, so I've provided implementations for that as well.

llvm-svn: 306042
2017-06-22 18:43:18 +00:00
Adrian McCarthy 6a4b080a5f Make IPDBSession::getGlobalScope a non-const method
There doesn't seem to be a compelling reason why this method should be const
other than it was possible with the DIA implementation.  The native session
is going to act as a symbol factory and cache.  This could be acheived with
mutable (and the existing const_cast), but it seems cleaner to accept that
this method affects the state of the session.

This change eliminates an existing const_cast.

llvm-svn: 306041
2017-06-22 18:42:23 +00:00
Wolfgang Pieb 258927e3da [DWARF] Support for DW_FORM_strx3 and complete support for DW_FORM_strx{1,2,4}
(consumer).

Reviewer: aprantl

Differential Revision:  https://reviews.llvm.org/D34418

llvm-svn: 305944
2017-06-21 19:37:44 +00:00
Reid Kleckner d0e6e24a53 [PDB] Add symbols to the PDB
Summary:
The main complexity in adding symbol records is that we need to
"relocate" all the type indices. Type indices do not have anything like
relocations, an opaque data structure describing where to find existing
type indices for fixups. The linker just has to "know" where the type
references are in the symbol records. I added an overload of
`discoverTypeIndices` that works on symbol records, and it seems to be
able to link the standard library.

Reviewers: zturner, ruiu

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D34432

llvm-svn: 305933
2017-06-21 17:25:56 +00:00
Zachary Turner 297b6eb20d [PDB] Don't write uninitialized bytes to a PDB file.
There were certain fields that we didn't know how to write, as
well as various padding bytes that we would ignore.  This leads
to garbage data in the PDB.  While not strictly necessary, we
should initialize these bytes to something meaningful, as it
makes for easier binary comparison between PDBs.

llvm-svn: 305819
2017-06-20 18:50:55 +00:00
David Blaikie 6ab0eb4764 Remove convenient but probably not worthwhile macro for lambda workaround
Cleanup from r305405

llvm-svn: 305731
2017-06-19 19:01:08 +00:00
Reid Kleckner 44cdb10964 [PDB] Start emitting source file and line information
Summary:
This is a first step towards getting line info to show up in VS and
windbg. So far, only llvm-pdbutil can parse the PDBs that we produce.
cvdump doesn't like something about our file checksum tables. I'll have
to dig into that next.

This patch adds a new DebugSubsectionRecordBuilder which takes bytes
directly from some other producer, such as a linker, and sticks it into
the PDB. Line tables only need to be relocated. No data needs to be
rewritten.

File checksums and string tables, on the other hand, need to be re-done.

Reviewers: zturner, ruiu

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D34257

llvm-svn: 305713
2017-06-19 17:21:45 +00:00
Reid Kleckner 18d90e17ad [CodeView] Fix dumping of public symbol record flags
I noticed nonsensical type information while dumping PDBs produced by
MSVC.

llvm-svn: 305708
2017-06-19 16:54:51 +00:00
Zachary Turner 26dbc5420d Delete TypeDatabase.
Merge the functionality into the random access type collection.
This class was only being used in 2 places, so getting rid of it
simplifies the code.

llvm-svn: 305653
2017-06-18 20:52:45 +00:00
Zachary Turner b0fdd214b7 Don't crash if a type record can't be found.
This was a regression introduced in a previous patch.  Adding
back the code that handles this case.

llvm-svn: 305617
2017-06-17 00:02:24 +00:00
Zachary Turner ad859bd472 [CodeView] Fix random access of type names.
Suppose we had a type index offsets array with a boundary at type index
N. Then you request the name of the type with index N+1, and that name
requires the name of index N-1 (think a parameter list, for example). We
didn't handle this, and we would print something like (<unknown UDT>,
<unknown UDT>).

The fix for this is not entirely trivial, and speaks to a larger
problem. I think we need to kill TypeDatabase, or at the very least kill
TypeDatabaseVisitor. We need a thing that doesn't do any caching
whatsoever, just given a type index it can compute the type name "the
slow way". The reason for the bug is that we don't have anything like
that. Everything goes through the type database, and if we've visited a
record, then we're "done". It doesn't know how to do the expensive thing
of re-visiting dependent records if they've not yet been visited.

What I've done here is more or less copied the code (albeit greatly
simplified) from TypeDatabaseVisitor, but wrapped it in an interface
that just returns a std::string. The logic of caching the name is now in
LazyRandomTypeCollection. Eventually I'd like to move the record
database here as well and the visited record bitfield here as well, at
which point we can actually just delete TypeDatabase. I don't see any
reason for it if a "sequential" collection is just a special case of a
random access collection with an empty partial offsets array.

Differential Revision: https://reviews.llvm.org/D34297

llvm-svn: 305612
2017-06-16 23:42:44 +00:00
Zachary Turner 59224cba2e Remove some dead code / includes.
I'm trying to get rid of the TypeDatabase class, so the first
step is to minimize its footprint.

llvm-svn: 305611
2017-06-16 23:42:15 +00:00
Spyridoula Gravani 32614fcf42 [DWARF] Corrected behavior for when no .apple_names section is present in the object.
The verifier should not output any message in such a case.
Added test case with no .apple_name section in the file to verify new functionality.
Made existing test case more specific.

llvm-svn: 305597
2017-06-16 22:03:21 +00:00
Zachary Turner 4e950647fb [llvm-pdbutil] Add support for dumping lines and inlinee lines.
llvm-svn: 305529
2017-06-15 23:56:19 +00:00
Zachary Turner 0e327d0360 [llvm-pdbutil] Add back support for dumping file checksums.
When dumping module source files, also dump checksums.

llvm-svn: 305526
2017-06-15 23:12:41 +00:00
Zachary Turner f8a2e04812 [llvm-pdbutil] Add back the ability to dump hashes and index offsets.
This was regressed in a previous patch that re-wrote the dumper,
and I'm incrementally adding back the pieces that are missing.

llvm-svn: 305524
2017-06-15 23:04:42 +00:00
Zachary Turner 6305545527 Resubmit "[llvm-pdbutil] rewrite the "raw" output style."
This resubmits commit c0c249e9f2ef83e1d1e5f166b50673d92f3579d7.

It was broken due to some weird template issues, which have
since been fixed.

llvm-svn: 305517
2017-06-15 22:24:24 +00:00
Zachary Turner da504b794c Revert "[llvm-pdbutil] rewrite the "raw" output style."
This reverts commit 83ea17ebf2106859a51fbc2a86031b44d33696ad.

This is failing due to some strange template problems, so reverting
until it can be straightened out.

llvm-svn: 305505
2017-06-15 20:55:51 +00:00
Spyridoula Gravani 7a27a26db0 [DWARF] Removed dead code. The verifier functionality is provided by
the DWARFVerifier class (as it should).

llvm-svn: 305503
2017-06-15 20:40:08 +00:00
Zachary Turner b560fdf3b8 [llvm-pdbutil] rewrite the "raw" output style.
After some internal discussions, we agreed that the raw output style had
outlived its usefulness. It was originally created before we had even
thought of dumping to YAML, and it was intended to give us some insight
into the internals of a PDB file. Now we have YAML mode which does
almost exactly this but is more powerful in that it can round-trip back
to a PDB, which the raw mode could not do. So the raw mode had become
purely a maintenance burden.

One option was to just delete it. However, its original goal was to be
as readable as possible while staying close to the "metal" - i.e.
presenting the output in a way that maps directly to the underlying file
format. We don't actually need that last requirement anymore since it's
covered by the yaml mode, so we could repurpose "raw" mode to actually
just be as readable as possible.

This patch implements about 80% of the functionality previously in raw
mode, but in a completely different style that is more akin to what
cvdump outputs. Records are very compressed, often times appearing on
just one line. One nice thing about this is that it makes full record
matching easier, because you can grep for indices, names, and leaf types
on a single line often.

See the tests for some examples of what the new output looks like.

Note that this patch actually regresses the functionality of raw mode in
a few areas, but only because the patch was already unreasonably large
and going 100% would have been even worse. Specifically, this patch is
missing:

The ability to dump module debug subsections (checksums, lines, etc)
The ability to dump section headers
Aside from that everything is here. While goign through the tests fixing
them all up, I found many duplicate tests. They've been deleted. In
subsequent patches I will go through and re-add the missing
functionality.

Differential Revision: https://reviews.llvm.org/D34191

llvm-svn: 305495
2017-06-15 19:34:41 +00:00
Galina Kistanova 3c0505d30c Specified ReportError as noreturn friendly to old compilers.
llvm-svn: 305405
2017-06-14 17:32:53 +00:00
Zachary Turner a8cfc29c9a Resubmit "[codeview] Make obj2yaml/yaml2obj support .debug$S..."
This was originally reverted because of some non-deterministic
failures on certain buildbots.  Luckily ASAN eventually caught
this as a stack-use-after-scope, so the fix is included in
this patch.

llvm-svn: 305393
2017-06-14 15:59:27 +00:00
Zachary Turner 0085dce221 Revert "[codeview] Make obj2yaml/yaml2obj support .debug$S..."
This is causing failures on linux bots with an invalid stream
read.  It doesn't repro in any configuration on Windows, so
reverting until I have a chance to investigate on Linux.

llvm-svn: 305371
2017-06-14 06:24:24 +00:00
Zachary Turner a3da4467fa [codeview] Make obj2yaml/yaml2obj support .debug$S/T sections.
This allows us to use yaml2obj and obj2yaml to round-trip CodeView
symbol and type information without having to manually specify the bytes
of the section. This makes for much easier to maintain tests. See the
tests under lld/COFF in this patch for example. Before they just said
SectionData: <blob> whereas now we can use meaningful record
descriptions. Note that it still supports the SectionData yaml field,
which could be useful for initializing a section to invalid bytes for
testing, for example.

Differential Revision: https://reviews.llvm.org/D34127

llvm-svn: 305366
2017-06-14 05:31:00 +00:00
Spyridoula Gravani e41823bb89 Added partial verification for .apple_names accelerator table in llvm-dwarfdump output.
This patch adds code which verifies that each bucket in the .apple_names
accelerator table is either empty or has a valid hash index.

Differential Revision: https://reviews.llvm.org/D34177

llvm-svn: 305344
2017-06-14 00:17:55 +00:00
Galina Kistanova 41def9b72c Reverted r305339 as MSVC is not happy with noreturn in lambda.
llvm-svn: 305343
2017-06-13 23:57:51 +00:00
Galina Kistanova 680c7605a7 Specified LLVM_ATTRIBUTE_NORETURN for ReportError.
llvm-svn: 305339
2017-06-13 23:39:42 +00:00
Reid Kleckner 8cbdd0c0f2 [PDB] Add a module descriptor for every object file
Summary:
Expose the module descriptor index and fill it in for section
contributions.

Reviewers: zturner

Subscribers: llvm-commits, ruiu, hiraditya

Differential Revision: https://reviews.llvm.org/D34126

llvm-svn: 305296
2017-06-13 15:49:13 +00:00
Zachary Turner 68ea80d0a7 Slightly better fix for dealing with no-id-stream PDBs.
The last fix required the user to manually add the required
feature.  This caused an LLD test to fail because I failed to
update LLD.  In practice we can hide this logic so it can just
be transparently added when we write the PDB.

llvm-svn: 305236
2017-06-12 21:46:51 +00:00
Zachary Turner 990d0c8158 [llvm-pdbdump] Don't fail on PDBs with no ID stream.
Older PDBs don't have this.  Its presence is detected by using
the various "feature" flags that come at the end of the PDB
Stream.  Detect this, and don't try to dump the ID stream if the
features tells us it's not present.

llvm-svn: 305235
2017-06-12 21:34:53 +00:00
Zachary Turner d334cebac4 Fix a null pointer dereference in llvm-pdbutil pretty.
Static data members were causing a problem because I mistakenly
assumed all members would affect a class's layout and so the
Layout member would be non-null.

llvm-svn: 305229
2017-06-12 20:46:35 +00:00
Sylvestre Ledru 337804d86a Same expressions on both sides of the return
Summary:
I guess we want PointerToMemberFunction & PointerToDataMember


Fix coverity cid 1376038 


Reviewers: zturner

Reviewed By: zturner

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D34110

llvm-svn: 305219
2017-06-12 18:53:46 +00:00
David Blaikie a91885a08c dwarfdump: Handle relocs to zlib (.zdebug*) compressed sections
llvm-svn: 305152
2017-06-10 19:32:50 +00:00
Galina Kistanova 038f9854ec Added llvm_unreachable as ReportError cannot be specified as noreturn.
llvm-svn: 305143
2017-06-10 07:50:14 +00:00
Zachary Turner 3226fe95bb [pdb] Support CoffSymbolRVA debug subsection.
llvm-svn: 305108
2017-06-09 20:46:52 +00:00
Zachary Turner 7e62cd17d6 Allow VarStreamArray to use stateful extractors.
Previously extractors tried to be stateless with any additional
context information needed in order to parse items being passed
in via the extraction method.  This led to quite cumbersome
implementation challenges and awkwardness of use.  This patch
brings back support for stateful extractors, making the
implementation and usage simpler.

llvm-svn: 305093
2017-06-09 17:54:36 +00:00
Bob Haarman fdf499bf2d [codeview] use 32-bit integer for RelocOffset in DebugLinesSubsection
Summary:
RelocOffset is a 32-bit value, but we previously truncated it to 16 bits.

Fixes PR33335.

Reviewers: zturner, hiraditya!

Reviewed By: zturner

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D33968

llvm-svn: 305043
2017-06-09 01:18:10 +00:00
Zachary Turner 28c22c83e3 [pdb] Don't crash on unknown debug subsections.
More and more unknown debug subsection kinds are being discovered
so we should make it possible to dump these and display the
bytes.

llvm-svn: 305041
2017-06-09 00:53:59 +00:00
Zachary Turner deb391309c [CodeView] Support remaining debug subsection types
This adds support for Symbols, StringTable, and FrameData subsection
types.  Even though these subsections rarely if ever appear in a PDB
file (they are usually in object files), there's no theoretical reason
why they *couldn't* appear in a PDB.  The real issue though is that in
order to add support for dumping and writing them (which will be useful
for object files), we need a way to test them.  And since there is no
support for reading and writing them to / from object files yet, making
PDB support them is the best way to both add support for the underlying
format and add support for tests at the same time.  Later, when we go
to add support for reading / writing them from object files, we'll need
only minimal changes in the underlying read/write code.

llvm-svn: 305037
2017-06-09 00:28:08 +00:00
Zachary Turner 1bf7762049 [llvm-pdbdump] Support native ordering of subsections in raw mode.
This is the same change for the YAML Output style applied to the
raw output style.  Previously we would queue up all subsections
until every one had been read, and then output them in a pre-
determined order.  This was because some subsections need to be
read first in order to properly dump later subsections.  This
patch allows them to be dumped in the order they appear.

Differential Revision: https://reviews.llvm.org/D34015

llvm-svn: 305034
2017-06-08 23:49:01 +00:00
Zachary Turner 15eb237fd3 [PDB] Don't crash on /debug:fastlink PDBs.
Apparently support for /debug:fastlink PDBs isn't part of the
DIA SDK (!), and it was causing llvm-pdbdump to crash because
we weren't checking for a null pointer return value.  This
manifests when calling findChildren on the IDiaSymbol, and
it returns E_NOTIMPL.

llvm-svn: 304982
2017-06-08 16:00:40 +00:00
NAKAMURA Takumi 92c99cd6dc Update libdeps to add BinaryFormat, introduced in r304864.
llvm-svn: 304869
2017-06-07 04:48:49 +00:00
Zachary Turner 264b5d9e88 Move Object format code to lib/BinaryFormat.
This creates a new library called BinaryFormat that has all of
the headers from llvm/Support containing structure and layout
definitions for various types of binary formats like dwarf, coff,
elf, etc as well as the code for identifying a file from its
magic.

Differential Revision: https://reviews.llvm.org/D33843

llvm-svn: 304864
2017-06-07 03:48:56 +00:00
Zachary Turner 1bfb9f47af Fix uninitialized read.
llvm-svn: 304846
2017-06-06 23:54:23 +00:00
Adrian Prantl 318d1195f2 Introduce -brief command line option to llvm-dwarfdump
This patch introduces a new command line option, called brief, to
llvm-dwarfdump.  When -brief is used, the attribute forms for the
.debug_info section will not be emitted to output.

Patch by Spyridoula Gravani!

rdar://problem/21474365
Differential Revision: https://reviews.llvm.org/D33867

llvm-svn: 304844
2017-06-06 23:28:45 +00:00
Chandler Carruth aaeada6c75 Fix another ordering constraint with windows.h and comment about
a revers constraint that we got right (by chance).

llvm-svn: 304792
2017-06-06 12:43:20 +00:00
Chandler Carruth 6bda14b313 Sort the remaining #include lines in include/... and lib/....
I did this a long time ago with a janky python script, but now
clang-format has built-in support for this. I fed clang-format every
line with a #include and let it re-sort things according to the precise
LLVM rules for include ordering baked into clang-format these days.

I've reverted a number of files where the results of sorting includes
isn't healthy. Either places where we have legacy code relying on
particular include ordering (where possible, I'll fix these separately)
or where we have particular formatting around #include lines that
I didn't want to disturb in this patch.

This patch is *entirely* mechanical. If you get merge conflicts or
anything, just ignore the changes in this patch and run clang-format
over your #include lines in the files.

Sorry for any noise here, but it is important to keep these things
stable. I was seeing an increasing number of patches with irrelevant
re-ordering of #include lines because clang-format was used. This patch
at least isolates that churn, makes it easy to skip when resolving
conflicts, and gets us to a clean baseline (again).

llvm-svn: 304787
2017-06-06 11:49:48 +00:00
Wolfgang Pieb 77d3e938f8 [DWARF] Adding support for the DWARF v5 string offsets table (consumer/reader part only).
Reviewers: dblaikie, aprantl

Differential Revision: https://reviews.llvm.org/D32779

llvm-svn: 304759
2017-06-06 01:22:34 +00:00
Zachary Turner 88101dadcc [CodeView] Fix endianness bug.
We should be outputting in little endian, but we were writing
in host endianness.

llvm-svn: 304741
2017-06-05 22:12:23 +00:00
Zachary Turner 349c18f837 [CodeView] Handle Cross Module Imports and Exports.
While it's not entirely clear why a compiler or linker might
put this information into an object or PDB file, one has been
spotted in the wild which was causing llvm-pdbdump to crash.

This patch adds support for reading-writing these sections.
Since I don't know how to get one of the native tools to
generate this kind of debug info, the only test here is one
in which we feed YAML into the tool to produce a PDB and
then spit out YAML from the resulting PDB and make sure that
it matches.

llvm-svn: 304738
2017-06-05 21:40:33 +00:00
Zachary Turner 5b74ff33e7 [PDB] Fix use after free.
Previously MappedBlockStream owned its own BumpPtrAllocator that
it would allocate from when a read crossed a block boundary.  This
way it could still return the user a contiguous buffer of the
requested size.  However, It's not uncommon to open a stream, read
some stuff, close it, and then save the information for later.
After all, since the entire file is mapped into memory, the data
should always be available as long as the file is open.

Of course, the exception to this is when the data isn't *in* the
file, but rather in some buffer that we temporarily allocated to
present this contiguous view.  And this buffer would get destroyed
as soon as the strema was closed.

The fix here is to force the user to specify the allocator, this
way it can provide an allocator that has whatever lifetime it
chooses.

Differential Revision: https://reviews.llvm.org/D33858

llvm-svn: 304623
2017-06-03 00:33:35 +00:00
Zachary Turner 92dcdda623 [CodeView] Support CodeView subsections in any order.
Previously we would expect certain subsections to appear
in a certain order because some subsections would reference
other subsections, but in practice we need to support
arbitrary orderings since some object file and PDB file
producers generate them this way.  This also paves the
way for supporting Yaml <-> Object File conversion of
CodeView, since Object Files typically have quite a
large number of subsections in their debug info.

Differential Revision: https://reviews.llvm.org/D33807

llvm-svn: 304588
2017-06-02 19:49:14 +00:00
Zachary Turner afb81a83a9 Fix 2 more -Wreorder warnings.
llvm-svn: 304494
2017-06-01 23:24:50 +00:00
Zachary Turner ebd3ae8371 [CodeView] Properly align symbol records on read/write.
Object files have symbol records not aligned to any particular
boundary (e.g. 1-byte aligned), while PDB files have symbol
records padded to 4-byte aligned boundaries.  Since they share
the same reading / writing code, we have to provide an option to
specify the alignment and propagate it up to the producer or
consumer who knows what the alignment is supposed to be for the
given container type.

Added a test for this by modifying the existing PDB -> YAML -> PDB
round-tripping code to round trip symbol records as well as types.

Differential Revision: https://reviews.llvm.org/D33785

llvm-svn: 304484
2017-06-01 21:52:41 +00:00
Adrian Prantl f4bc1f77b7 [DWARF] Introduce Dump Options
This commit introduces a structure that holds all the flags that
control the pretty printing of dwarf output.

Patch by Spyridoula Gravani!

Differential Revision: https://reviews.llvm.org/D33749

llvm-svn: 304446
2017-06-01 18:18:23 +00:00
Zachary Turner d427383cb8 [CodeView] Move CodeView YAML code to ObjectYAML.
This is the beginning of an effort to move the codeview yaml
reader / writer into ObjectYAML so that it can be shared.
Currently the only consumer / producer of CodeView YAML is
llvm-pdbdump, but CodeView can exist outside of PDB files, and
indeed is put into object files and passed to the linker to
produce PDB files.  Furthermore, there are subtle differences
in the types of records that show up in object file CodeView
vs PDB file CodeView, but they are otherwise 99% the same.

By having this code in ObjectYAML, we can have llvm-pdbdump
reuse this code, while teaching obj2yaml and yaml2obj to use
this syntax for dealing with object files that can contain
CodeView.

This patch only adds support for CodeView type information
to ObjectYAML.  Subsequent patches will add support for
CodeView symbol information.

llvm-svn: 304248
2017-05-30 21:53:05 +00:00
Galina Kistanova 8c1e2f9108 Added missing break.
llvm-svn: 304230
2017-05-30 19:02:49 +00:00
Zachary Turner 591312c5c1 [CodeView] Add more DebugSubsection implementations.
This adds implementations for Symbols and FrameData, and renames
the existing codeview::StringTable class to conform to the
DebugSectionStringTable convention.

llvm-svn: 304222
2017-05-30 17:13:33 +00:00
Zachary Turner 8c099fe06e [CodeView] Rename ModuleDebugFragment -> DebugSubsection.
This is more concise, and matches the terminology used in other
parts of the codebase more closely.

llvm-svn: 304218
2017-05-30 16:36:15 +00:00
George Rimar a25d329b33 Recommit "[DWARF] - Make collectAddressRanges() return section index in addition to Low/High PC"
With fix of uninitialized variable.

Original commit message:

This change is intended to use for LLD in D33183. 
Problem we have in LLD when building .gdb_index is that we need to know section which address range belongs to.

Previously it was solved on LLD side by providing fake section addresses with use of llvm::LoadedObjectInfo
interface. We assigned file offsets as addressed. Then after obtaining ranges lists, for each range we had to find section ID's.
That not only was slow, but also complicated implementation and was the reason of incorrect behavior when
sections share the same offsets, like D33176 shows.

This patch makes DWARF parsers to return section index as well. That solves problem mentioned above.

Differential revision: https://reviews.llvm.org/D33184

llvm-svn: 304078
2017-05-27 18:10:23 +00:00
George Rimar 1f9cab6b1c Revert r304002 "[DWARF] - Make collectAddressRanges() return section index in addition to Low/High PC"
Revert it again. Now another bot unhappy: http://lab.llvm.org:8011/builders/clang-s390x-linux/builds/8750

llvm-svn: 304011
2017-05-26 17:36:23 +00:00
George Rimar bc223c63cc [DWARF] - Make collectAddressRanges() return section index in addition to Low/High PC
This change is intended to use for LLD in D33183. 
Problem we have in LLD when building .gdb_index is that we need to know section which address range belongs to.

Previously it was solved on LLD side by providing fake section addresses with use of llvm::LoadedObjectInfo
interface. We assigned file offsets as addressed. Then after obtaining ranges lists, for each range we had to find section ID's.
That not only was slow, but also complicated implementation and was the reason of incorrect behavior when
sections share the same offsets, like D33176 shows.

This patch makes DWARF parsers to return section index as well. That solves problem mentioned above.

Differential revision: https://reviews.llvm.org/D33184

llvm-svn: 304002
2017-05-26 16:26:18 +00:00
George Rimar a8403a64ea Revert "[DWARF] - Make collectAddressRanges() return section index in addition to Low/High PC"
Broked BB again:

TEST 'LLVM :: DebugInfo/X86/dbg-value-regmask-clobber.ll' FAILED
...
LLVM ERROR: Section was outside of section table.

llvm-svn: 303984
2017-05-26 13:20:09 +00:00
George Rimar 655b7b63f6 Recommit r303978 "[DWARF] - Make collectAddressRanges() return section index in addition to Low/High PC"
With fix of test compilation.

Initial commit message:

This change is intended to use for LLD in D33183. 
Problem we have in LLD when building .gdb_index is that we need to know section 
which address range belongs to.

Previously it was solved on LLD side by providing fake section addresses
with use of llvm::LoadedObjectInfo interface. We assigned file offsets as addressed.
Then after obtaining ranges lists, for each range we had to find section ID's.
That not only was slow, but also complicated implementation and was the reason 
of incorrect behavior when
sections share the same offsets, like D33176 shows.

This patch makes DWARF parsers to return section index as well. 
That solves problem mentioned above.

Differential revision: https://reviews.llvm.org/D33184

llvm-svn: 303983
2017-05-26 13:13:50 +00:00
George Rimar 7d5f12185a Revert r303978 "[DWARF] - Make collectAddressRanges() return section index in addition to Low/High PC"
It failed BB.

llvm-svn: 303981
2017-05-26 12:53:41 +00:00
George Rimar 732f268aa0 [DWARF] - Make collectAddressRanges() return section index in addition to Low/High PC
This change is intended to use for LLD in D33183. 
Problem we have in LLD when building .gdb_index is that we need to know section 
which address range belongs to.

Previously it was solved on LLD side by providing fake section addresses
with use of llvm::LoadedObjectInfo interface. We assigned file offsets as addressed.
Then after obtaining ranges lists, for each range we had to find section ID's.
That not only was slow, but also complicated implementation and was the reason 
of incorrect behavior when
sections share the same offsets, like D33176 shows.

This patch makes DWARF parsers to return section index as well. 
That solves problem mentioned above.

Differential revision: https://reviews.llvm.org/D33184

llvm-svn: 303978
2017-05-26 12:46:41 +00:00
Zachary Turner f2110283c6 Remove unused member.
llvm-svn: 303942
2017-05-25 23:47:56 +00:00
Zachary Turner fed467eefb [CV Type Merging] Find nested type indices faster.
Merging two type streams is one of the most time consuming
parts of generating a PDB, and as such it needs to be as
fast as possible.  The visitor abstractions used for interoperating
nicely with many different types of inputs and outputs have
been used widely and help greatly for testability and implementing
tools, but the abstractions build up and get in the way of
performance.

This patch removes all of the visitation stuff from the type
stream merger, essentially re-inventing the leaf / member switch
and loop, but at a very low level.  This allows us many other
optimizations, such as not actually deserializing *any* records
(even member records which don't describe their own length), as
the operation of "figure out how long this record is" is somewhat
faster than "figure out how long this record *and* get all its
fields out".  Furthermore, whereas before we had to deserialize,
re-write type indices, then re-serialize, now we don't have to
do any of those 3 steps.  We just find out where the type indices
are and pull them directly out of the byte stream and re-write
them.

This is worth a 50-60% performance increase.  On top of all other
optimizations that have been applied this week, I now get the
following numbers when linking lld.exe and lld.pdb

MSVC: 25.67s
Before This Patch: 18.59s
After This Patch: 8.92s

So this is a huge performance win.

Differential Revision: https://reviews.llvm.org/D33564

llvm-svn: 303935
2017-05-25 23:36:16 +00:00
Zachary Turner 2897e0306e [lld] Fix a bug where we continually re-follow type servers.
Originally this was intended to be set up so that when linking
a PDB which refers to a type server, it would only visit the
PDB once, and on subsequent visitations it would just skip it
since all the records had already been added.

Due to some C++ scoping issues, this was not occurring and it
was revisiting the type server every time, which caused every
record to end up being thrown away on all subsequent visitations.

This doesn't affect the performance of linking clang-cl generated
object files because we don't use type servers, but when linking
object files and libraries generated with /Zi via MSVC, this means
only 1 object file has to be linked instead of N object files, so
the speedup is quite large.

llvm-svn: 303920
2017-05-25 21:16:03 +00:00
Zachary Turner 7f97c362a4 [CodeView Type Merging] Don't keep re-allocating temp serializer.
Previously, every time we wanted to serialize a field list record, we
would create a new copy of FieldListRecordBuilder, which would in turn
create a temporary instance of TypeSerializer, which itself had a
std::vector<> that was about 128K in size. So this 128K allocation was
happening every time. We can re-use the same instance over and over, we
just have to clear its internal hash table and seen records list between
each run. This saves us from the constant re-allocations.

This is worth an ~18.5% speed increase (3.75s -> 3.05s) in my tests.

Differential Revision: https://reviews.llvm.org/D33506

llvm-svn: 303919
2017-05-25 21:15:37 +00:00
Bob Haarman 55256ada25 [pdb] pad source file name buffer at the end instead of the beginning
Summary:
DbiStreamBuilder calculated the offset of the source file names inside
the file info substream as the size of the file info substream minus
the size of the file names. Since the file info substream is padded to
a multiple of 4 bytes, this caused the first file name to be aligned
on a 4-byte boundary. By contrast, DbiModuleList would read the file
names immediately after the file name offset table, without skipping
to the next 4-byte boundary. This change makes it so that the file
names are written to the location where DbiModuleList expects them,
and puts any necessary padding for the file info substream after the
file names instead of before it.

Reviewers: amccarth, rnk, zturner

Reviewed By: amccarth, zturner

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D33475

llvm-svn: 303917
2017-05-25 21:12:15 +00:00
Zachary Turner c4e4b7e31e Fix a bug in MappedBlockStream.
It was using the number of blocks of the entire PDB file as the number
of blocks of each stream that was created.  This was only an issue in
the readLongestContiguousChunk function, which  was never called prior.
This bug surfaced when I updated an algorithm to use this function and
the algorithm broke.

llvm-svn: 303916
2017-05-25 21:12:00 +00:00
Zachary Turner dda25b128c [CodeView Type Merging] Avoid record deserialization when possible.
A profile shows the majority of time doing type merging is spent
deserializing records from sequences of bytes into friendly C++ structures
that we can easily access members of in order to find the type indices to
re-write.

Records are prefixed with their length, however, and most records have
type indices that appear at fixed offsets in the record. For these
records, we can save some cycles by just looking at the right place in the
byte sequence and re-writing the value, then skipping the record in the
type stream. This saves us from the costly deserialization of examining
every field, including potentially null terminated strings which are the
slowest, even though it was unnecessary to begin with.

In addition, we apply another optimization. Previously, after
deserializing a record and re-writing its type indices, we would
unconditionally re-serialize it in order to compute the hash of the
re-written record. This would result in an alloc and memcpy for every
record. If no type indices were re-written, however, this was an
unnecessary allocation. In this patch re-writing is made two phase. The
first phase discovers the indices that need to be rewritten and their new
values. This information is passed through to the de-duplication code,
which only copies and re-writes type indices in the serialized byte
sequence if at least one type index is different.

Some records have type indices which only appear after variable length
strings, or which have lists of type indices, or various other situations
that can make it tricky to make this optimization. While I'm not giving up
on optimizing these cases as well, for now we can get the easy cases out
of the way and lay the groundwork for more complicated cases later.

This patch yields another 50% speedup on top of the already large speedups
submitted over the past 2 days. In two tests I have run, I went from 9
seconds to 3 seconds, and from 16 seconds to 8 seconds.

Differential Revision: https://reviews.llvm.org/D33480

llvm-svn: 303914
2017-05-25 21:06:28 +00:00
Zachary Turner bb64231d2d Don't do a full scan of the type stream before processing records.
LazyRandomTypeCollection is designed for random access, and in
order to provide this it lazily indexes ranges of types.  In the
case of types from an object file, there is no partial index
to build off of, so it has to index the full stream up front.
However, merging types only requires sequential access, and when
that is needed, this extra work is simply wasted.  Changing the
algorithm to work on sequential arrays of types rather than
random access type collections eliminates this up front scan.

llvm-svn: 303707
2017-05-24 00:26:27 +00:00
Zachary Turner 7daf62e743 [CodeView] Eliminate redundant hashes and allocations.
When writing field list records, we would construct a temporary
type serializer that shared a bump ptr allocator with the rest
of the application, so anything allocated from here would live
forever.  Furthermore, this temporary serializer had all the
properties of a full blown serializer including record hashing
and de-duplication.

These features are required when you're merging multiple type
streams into each other, because different streams may contain
identical records, but records from the same type stream will
never collide with each other.  So all of this hashing was
unnecessary.

To solve this, two fixes are made:

1) The temporary serializer keeps its own bump ptr allocator
instead of sharing a global one.  When it's finished, all of
its memory is freed.

2) Instead of using the same temporary serializer for the life
of an entire type stream, we use it only for the life of a single
field list record and delete it when the field list record is
completed.  This way the hash table will not grow as other
records from the same type stream get inserted.  Further improvements
could eliminate hashing entirely from this codepath.

This reduces the link time by 85% in my test, from 1 minute to 9
seconds.

llvm-svn: 303676
2017-05-23 18:56:23 +00:00
Reid Kleckner 36238b15d7 Speculative build fix for non-Windows
llvm-svn: 303667
2017-05-23 18:28:13 +00:00
Reid Kleckner ded38803c5 [PDB] Hash types up front when merging types instead of using StringMap
Summary:
First, StringMap uses llvm::HashString, which is only good for short
identifiers and really bad for large blobs of binary data like type
records. Moving to `DenseMap<StringRef, TypeIndex>` with some tricks for
memory allocation fixes that.

Unfortunately, that didn't buy very much performance. Profiling showed
that we spend a long time during DenseMap growth rehashing existing
entries. Also, in general, DenseMap is faster when the keys are small.
This change takes that to the logical conclusion by introducing a small
wrapper value type around a pointer to key data. The key data contains a
precomputed hash, the original record data (pointer and size), and the
type index, which is the "value" of our original map.

This reduces the time to produce llvm-as.exe and llvm-as.pdb from ~15s
on my machine to 3.5s, which is about a 4x improvement.

Reviewers: zturner, inglorion, ruiu

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D33428

llvm-svn: 303665
2017-05-23 18:23:59 +00:00
Zachary Turner bf35e6ab2a Revert "Make TypeSerializer's StringMap use the same allocator."
This reverts commit e34ccb7b57da25cc89ded913d8638a2906d1110a.

This is causing failures on the ASAN bots.

llvm-svn: 303640
2017-05-23 15:50:37 +00:00
David Blaikie 15d85fc537 libDebugInfo: Support symbolizing using DWP files
llvm-svn: 303609
2017-05-23 06:48:53 +00:00
David Blaikie 37d1cff491 FIX: Remove debugging assert left in previous commit
Sorry for the bot noise.

llvm-svn: 303592
2017-05-23 00:31:24 +00:00
David Blaikie f9803fb4bb libDebugInfo: Avoid independently parsing the same .dwo file for two separate CUs residing there
NFC, just an optimization. Will be building on this for DWP support
shortly.

llvm-svn: 303591
2017-05-23 00:30:42 +00:00
Zachary Turner d4136e945e Implement various flavors of type merging.
Previous algotirhm assumed that types and ids are in a single
unified stream.  For inputs that come from object files, this
is the case.  But if the input is already a PDB, or is the result
of a previous merge, then the types and ids will already have
been split up, in which case we need an algorithm that can
accept operate on independent streams of types and ids that
refer across stream boundaries to each other.

Differential Revision: https://reviews.llvm.org/D33417

llvm-svn: 303577
2017-05-22 21:07:43 +00:00
Zachary Turner 12f8c31c04 Make TypeSerializer's StringMap use the same allocator.
llvm-svn: 303576
2017-05-22 21:07:14 +00:00
David Blaikie d2f3a941e0 libDebugInfo/DWARF: Apply relocations for debug_addr addresses in object files
llvm-symbolizer would fail to symbolize addresses in unlinked object
files when handling .dwo file data because the addresses would not be
relocated in the same way as the ranges in the skeleton CU in the object
file.

Fix that so object files can be symbolized the same as executables.

llvm-svn: 303532
2017-05-22 07:02:47 +00:00
David Blaikie 8d039d40c5 llvm-symbolizer: Support multiple CUs in a single DWO file
llvm-svn: 303482
2017-05-20 03:32:49 +00:00
Zachary Turner 526f4f2aa8 Resubmit "[CodeView] Provide a common interface for type collections."
This was originally reverted because it was a breaking a bunch
of bots and the breakage was not surfacing on Windows.  After much
head-scratching this was ultimately traced back to a bug in the
lit test runner related to its pipe handling.  Now that the bug
in lit is fixed, Windows correctly reports these test failures,
and as such I have finally (hopefully) fixed all of them in this
patch.

llvm-svn: 303446
2017-05-19 19:26:58 +00:00
Zachary Turner 1dfcf8d92c Revert "[CodeView] Provide a common interface for type collections."
This is a squash of ~5 reverts of, well, pretty much everything
I did today.  Something is seriously broken with lit on Windows
right now, and as a result assertions that fire in tests are
triggering failures.  I've been breaking non-Windows bots all
day which has seriously confused me because all my tests have
been passing, and after running lit with -a to view the output
even on successful runs, I find out that the tool is crashing
and yet lit is still reporting it as a success!

At this point I don't even know where to start, so rather than
leave the tree broken for who knows how long, I will get this
back to green, and then once lit is fixed on Windows, hopefully
hopefully fix the remaining set of problems for real.

llvm-svn: 303409
2017-05-19 05:57:45 +00:00
Zachary Turner 47fdc73771 Don't crash if someone tries to visit an empty type stream.
llvm-svn: 303408
2017-05-19 05:18:09 +00:00
Zachary Turner 59ab6a3816 [CodeView] Reduce memory usage in TypeSerializer.
We were using a BumpPtrAllocator to allocate stable storage for
a record, then trying to insert that into a hash table.  If a
collision occurred, the bytes were never inserted and the
allocation was unnecessary.  At the cost of an extra hash
computation, check first if it exists, and only if it does do
we allocate and insert.

llvm-svn: 303407
2017-05-19 04:56:48 +00:00
Zachary Turner 8f1d87a79a Fix crasher in CodeView test.
Apparently this was always broken, but previously we were more
graceful about it and we would print "unknown udt" if we couldn't
find the type index, whereas now we just segfault because we
assume it's valid.  But this exposed a real bug, which is that
we weren't looking in the right place.  So fix that, and also
fix this crash at the same time.

llvm-svn: 303397
2017-05-19 00:56:39 +00:00
Zachary Turner 7b62d7ccc0 Fix some build errors and warnings.
llvm-svn: 303391
2017-05-18 23:12:42 +00:00
Zachary Turner b32ec02b80 [CodeView] Raise the source to ID map out of the TypeStreamMerger.
This map will be needed to rewrite symbol streams after re-writing
the corresponding type streams.

llvm-svn: 303390
2017-05-18 23:04:08 +00:00
Zachary Turner 8fb441ab9c [llvm-pdbdump] Add the ability to merge PDBs.
Merging PDBs is a feature that will be used heavily by
the linker.  The functionality already exists but does not
have deep test coverage because it's not easily exposed through
any tools.  This patch aims to address that by adding the
ability to merge PDBs via llvm-pdbdump.  It takes arbitrarily
many PDBs and outputs a single PDB.

Using this new functionality, a test is added for merging
type records.  Future patches will add the ability to merge
symbol records, module information, etc.

llvm-svn: 303389
2017-05-18 23:03:41 +00:00
Zachary Turner 0c60f269fc [CodeView] Provide a common interface for type collections.
Right now we have multiple notions of things that represent collections of
types. Most commonly used are TypeDatabase, which is supposed to keep
mappings from TypeIndex to type name when reading a type stream, which
happens when reading PDBs. And also TypeTableBuilder, which is used to
build up a collection of types dynamically which we will later serialize
(i.e. when writing PDBs).

But often you just want to do some operation on a collection of types, and
you may want to do the same operation on any kind of collection. For
example, you might want to merge two TypeTableBuilders or you might want
to merge two type streams that you loaded from various files.

This dichotomy between reading and writing is responsible for a lot of the
existing code duplication and overlapping responsibilities in the existing
CodeView library classes. For example, after building up a
TypeTableBuilder with a bunch of type records, if we want to dump it we
have to re-invent a bunch of extra glue because our dumper takes a
TypeDatabase or a CVTypeArray, which are both incompatible with
TypeTableBuilder.

This patch introduces an abstract base class called TypeCollection which
is shared between the various type collection like things. Wherever we
previously stored a TypeDatabase& in some common class, we now store a
TypeCollection&.

The advantage of this is that all the details of how the collection are
implemented, such as lazy deserialization of partial type streams, is
completely transparent and you can just treat any collection of types the
same regardless of where it came from.

Differential Revision: https://reviews.llvm.org/D33293

llvm-svn: 303388
2017-05-18 23:03:06 +00:00
Zachary Turner 5a83fb153f Fix some minor issues in PDB parsing library.
1) Until now I'd never seen a valid PDB where the DBI stream and
   the PDB Stream disagreed on the "Age" field.  Because of that,
   we had code to assert that they matched.  Recently though I was
   given a PDB where they disagreed, so this assumption has proven
   to be incorrect.  Remove this check.

2) We were walking the entire list of hash values for types up front
   and then throwing away the values.  For large PDBs this was a
   significant slow down.  Remove this.

With this patch, I can dump the list of all compilands from a
1.5GB PDB file in just a few seconds.

llvm-svn: 303351
2017-05-18 15:14:44 +00:00
George Rimar 47f84b1a3c [DWARF] - Simplify RelocVisitor implementation.
We do not need to store relocation width field.
Patch removes relative code, that simplifies implementation.

Differential revision: https://reviews.llvm.org/D33274

llvm-svn: 303335
2017-05-18 08:25:11 +00:00
George Rimar f98b9ac5da [lib/Object] - Minor API update for llvm::Decompressor.
I revisited Decompressor API (issue with it was triggered during D32865 review)
and found it is probably provides more then we really need.

Issue was about next method's signature:

Error decompress(SmallString<32> &Out);
It is too strict. At first I wanted to change it to decompress(SmallVectorImpl<char> &Out),
but then found it is still not flexible because sticks to SmallVector.

During reviews was suggested to use templating to simplify code. Patch do that.

Differential revision: https://reviews.llvm.org/D33200

llvm-svn: 303331
2017-05-18 08:00:01 +00:00
Bob Haarman de33a63784 [llvm-pdbdump] in yaml2pdb, generate default output filename if none given
Summary:
llvm-pdbdump yaml2pdb used to fail with a misleading error
message ("An I/O error occurred on the file system") if no output file
was specified. This change adds an assert to PDBFileBuilder to check
that an output file name is specified, and makes llvm-pdbdump generate
an output file name based on the input file name if no output file
name is explicitly specified.

Reviewers: amccarth, zturner

Reviewed By: zturner

Subscribers: fhahn, llvm-commits

Differential Revision: https://reviews.llvm.org/D33296

llvm-svn: 303299
2017-05-17 20:46:48 +00:00
Zachary Turner 1d795c451e [CodeView] Simplify the use of visiting type records & streams.
There is often a lot of boilerplate code required to visit a type
record or type stream.  The #1 use case is that you have a sequence
of bytes that represent one or more records, and you want to
deserialize each one, switch on it, and call a callback with the
deserialized record that the user can examine.  Currently this
requires at least 6 lines of code:

  codeview::TypeVisitorCallbackPipeline Pipeline;
  Pipeline.addCallbackToPipeline(Deserializer);
  Pipeline.addCallbackToPipeline(MyCallbacks);

  codeview::CVTypeVisitor Visitor(Pipeline);
  consumeError(Visitor.visitTypeRecord(Record));

With this patch, it becomes one line of code:

  consumeError(codeview::visitTypeRecord(Record, MyCallbacks));

This is done by having the deserialization happen internally inside
of the visitTypeRecord function.  Since this is occasionally not
desirable, the function provides a 3rd parameter that can be used
to change this behavior.

Hopefully this can significantly reduce the barrier to entry
to using the visitation infrastructure.

Differential Revision: https://reviews.llvm.org/D33245

llvm-svn: 303271
2017-05-17 16:39:06 +00:00
George Rimar fed9f09f48 [DWARF] - Cleanup relocations proccessing.
RelocAddrMap was a pair of <width, address>, where width is relocation size (4/8/x, x < 8), 
and width field was never used in code.

Relocations proccessing loop had checks for width field. Does not look like DWARF parser
should do that. There is probably no much sense to validate relocations during proccessing 
them in parser.

Patch removes relocation's width relative code from DWARFContext.

Differential revision: https://reviews.llvm.org/D33194

llvm-svn: 303251
2017-05-17 12:10:51 +00:00
George Rimar 41e656768d [DWARF] - Add RelocAddrEntry for cleanup. NFCi.
Was mentioned as possible cleanup during review of D33184.

llvm-svn: 303171
2017-05-16 14:05:45 +00:00
George Rimar 4671f2e08c [DWARF] - Use DWARFAddressRange struct instead of uint64_t pair for DWARFAddressRangesVector.
Recommit of r303159 "[DWARF] - Use DWARFAddressRange struct instead of uint64_t pair for DWARFAddressRangesVector"
All places were shitched to use DWARFAddressRange now.

Suggested during review of D33184.

llvm-svn: 303163
2017-05-16 12:30:59 +00:00
George Rimar 3824cca7b3 Revert r303159 "[DWARF] - Use DWARFAddressRange struct instead of uint64_t pair for DWARFAddressRangesVector."
Something went wrong, it broke BB.
http://green.lab.llvm.org/green//job/clang-stage1-cmake-RA-incremental_build/38477/consoleFull#-200034420049ba4694-19c4-4d7e-bec5-911270d8a58c

llvm-svn: 303162
2017-05-16 12:05:03 +00:00
George Rimar 8680b6ee9c [DWARF] - Use DWARFAddressRange struct instead of uint64_t pair for DWARFAddressRangesVector.
Suggested during review of D33184.

llvm-svn: 303159
2017-05-16 11:54:19 +00:00
George Rimar 958b01aa69 [DWARF] - Speedup handling of relocations in DWARFContextInMemory.
I am working on a speedup of building .gdb_index in LLD and 
noticed that relocations that are proccessed in DWARFContextInMemory often uses
the same symbol in a row. This patch introduces caching to reduce the relocations
proccessing time.

For benchmark,
I took debug LLC binary objects configured with -ggnu-pubnames and linked it using LLD.

Link time without --gdb-index is about 4,45s.
Link time with --gdb-index: a) Without patch: 19,16s b) With patch: 15,52s
That means time spent on --gdb-index in this configuration is 
19,16s - 4,45s = 14,71s (without patch) vs 15,52s - 4,45s = 11,07s (with patch).

Differential revision: https://reviews.llvm.org/D31136

llvm-svn: 303051
2017-05-15 11:45:28 +00:00
Zachary Turner dd3a739d52 [CodeView] Add a random access type visitor.
This adds a visitor that is capable of accessing type
records randomly and caching intermediate results that it
learns about during partial linear scans.  This yields
amortized O(1) access to a type stream even though type
streams cannot normally be indexed.

Differential Revision: https://reviews.llvm.org/D33009

llvm-svn: 302936
2017-05-12 19:18:12 +00:00
Wolfgang Pieb 15fa44698c [DWARF] Fix a parsing issue with type unit headers.
Reviewers: dblaikie

Differential Revision: https://reviews.llvm.org/D32987

llvm-svn: 302574
2017-05-09 19:38:38 +00:00
Aaron Ballman f22f885b66 Removing a file that is not necessary (and was causing link diagnostics with MSVC 2015); NFC.
llvm-svn: 302531
2017-05-09 14:22:48 +00:00
Diana Picus e8da53f4e0 Revert "[Dwarf] Disable reference verification for now (PR32972)"
This reverts commit r302520 because it break the unit tests.

llvm-svn: 302524
2017-05-09 13:05:43 +00:00
Renato Golin 94d6c8fb36 [Dwarf] Disable reference verification for now (PR32972)
There is no other explanation about why this only started happening
now, even though it crashes on old code (supposedly reachable from
here).

The only common factor between the failing bots is that they use GCC
(4.9 and 5.3) to compile Clang, while the others use Clang 3.8, but the
failure is while building the tests, as an assertion, on Clang.

Commenting it out for now in hope the bots will go back green, but we
should keep looking for the real cause, and update bugzilla.

llvm-svn: 302520
2017-05-09 12:36:50 +00:00
Greg Clayton 58a2e0d90b Add const to "DWARFDie &Die" in a few functions as they can't change the DWARFDie.
llvm-svn: 302471
2017-05-08 21:29:17 +00:00
Eugene Zemtsov 3b52dbd934 Fix typo
llvm-svn: 302470
2017-05-08 21:20:53 +00:00
Greg Clayton 5404f114d3 Fix typo "veify" to "verify".
llvm-svn: 302466
2017-05-08 20:53:00 +00:00
Zachary Turner 1dacb24222 [CodeView] Add support for random access type visitors.
Previously type visitation was done strictly sequentially, and
TypeIndexes were computed by incrementing the TypeIndex of the
last visited record.  This works fine for situations like dumping,
but not when you want to visit types in random order.  For example,
in a debug session someone might lookup a symbol by name, find that
it has TypeIndex 10,000 and then want to go straight to TypeIndex
10,000.

In order to make this work, the visitation framework needs a mode
where it can plumb TypeIndices through the callback pipeline.  This
patch adds such a mode.  In doing so, it is necessary to provide
an alternative implementation of TypeDatabase that supports random
access, so that is done as well.

Nothing actually uses these random access capabilities yet, but
this will be done in subsequent patches.

Differential Revision: https://reviews.llvm.org/D32928

llvm-svn: 302454
2017-05-08 18:38:43 +00:00
Zachary Turner 8c74673388 [CodeView] Reserve TypeDatabase records up front.
Most of the time we know exactly how many type records we
have in a list, and we want to use the visitor to deserialize
them into actual records in a database.  Previously we were
just using push_back() every time without reserving the space
up front in the vector.  This is obviously terrible from a
performance standpoint, and it's not uncommon to have PDB
files with half a million type records, where the performance
degredation was quite noticeable.

llvm-svn: 302302
2017-05-05 22:02:37 +00:00
George Rimar 2122ff64c6 [llvm-dwarfdump] - Print an error message if section decompression failed.
llvm-dwarfdump currently prints no message if decompression fails 
for some reason. I noticed that during work on one of LLD patches 
where LLD produced an broken output. It was a bit confusing to see
no output for section dumped and no any error message at all.

Patch adds error message for such cases.

Differential revision: https://reviews.llvm.org/D32865

llvm-svn: 302221
2017-05-05 10:52:39 +00:00
Zachary Turner bedc85fb4b [pdb] Don't verify TPI hash values up front.
Verifying the hash values as we are currently doing
results in iterating every type record before the user
even tries to access the first one, and the API user
has no control over, or ability to hook into this
process.

As a result, when the user wants to iterate over types
to print them or index them, this results in a second
iteration over the same list of types.  When there's
upwards of 1,000,000 type records, this is obviously
quite undesirable.

This patch raises the verification outside of TpiStream
, and llvm-pdbdump hooks a hash verification visitor
into the normal dumping process.  So we still verify
the hash records, but we can do it while not requiring
a second iteration over the type stream.

Differential Revision: https://reviews.llvm.org/D32873

llvm-svn: 302206
2017-05-04 23:53:54 +00:00
Zachary Turner 1eb9a0297c [PDB] Don't build the entire source file list up front.
I tried to run llvm-pdbdump on a very large (~1.5GB) PDB to
try and identify show-stopping performance problems.  This
patch addresses the first such problem.

When loading the DBI stream, before anyone has even tried to
access a single record, we build an in memory map of every
source file for every module.  In the particular PDB I was
using, this was over 85 million files.  Specifically, the
complexity is O(m*n) where m is the number of modules and
n is the average number of source files (including headers)
per module.

The whole reason for doing this was so that we could have
constant time access to any module and any of its source
file lists.  However, we can still get O(1) access to the
source file list for a given module with a simple O(m)
precomputation, and access to the list of modules is
already O(1) anyway.

So this patches reduces the O(m*n) up-front precomputation
to an O(m) one, where n is ~6,500 and n*m is about 85 million
in my pathological test case.

Differential Revision: https://reviews.llvm.org/D32870

llvm-svn: 302205
2017-05-04 23:53:29 +00:00
Greg Clayton 48ff66a280 Don't return an invalid line table if the DW_AT_stmt_list value is not in the .debug_line section.
llvm-svn: 302180
2017-05-04 18:29:44 +00:00
Paul Robinson ae2e6f37f3 clang-format and restyle DWARFFormValue before working on it. NFC
llvm-svn: 302086
2017-05-03 21:53:21 +00:00
Zachary Turner 4f145b2a59 Remove unused private field.
llvm-svn: 302069
2017-05-03 19:42:06 +00:00
Greg Clayton c5b2d561e8 Break verification down into smaller functions to keep code clean.
Adrian requested that we break things down to make things clean in the DWARFVerifier. This patch breaks everything down into nice individual functions and cleans up the code quite a bit and prepares us for the next round of verifiers.

Differential Revision: https://reviews.llvm.org/D32812

llvm-svn: 302062
2017-05-03 18:25:46 +00:00
Davide Italiano 2e23ce4cad [CodeView] Remove constructor initialization of a removed field.
I should've staged this with my last commit.

llvm-svn: 302059
2017-05-03 18:02:46 +00:00
Zachary Turner cf468d86f3 [CodeView] Use actual strings for dealing with checksums and lines.
The raw CodeView format references strings by "offsets", but it's
confusing what table the offset refers to.  In the case of line
number information, it's an offset into a buffer of records,
and an indirection is required to get another offset into a
different table to find the final string.  And in the case of
checksum information, there is no indirection, and the offset
refers directly to the location of the string in another buffer.

This would be less confusing if we always just referred to the
strings by their value, and have the library be smart enough
to correctly resolve the offsets on its own from the right
location.

This patch makes that possible.  When either reading or writing,
all the user deals with are strings, and the library does the
appropriate translations behind the scenes.

llvm-svn: 302053
2017-05-03 17:11:40 +00:00
Zachary Turner 2d5c2cd3ce [llvm-readobj] Update readobj to re-use parsing code.
llvm-readobj hand rolls some CodeView parsing code for string
tables, so this patch updates it to re-use some of the newly
introduced parsing code in LLVMDebugInfoCodeView.

Differential Revision: https://reviews.llvm.org/D32772

llvm-svn: 302052
2017-05-03 17:11:11 +00:00
Greg Clayton b8c162b53c Create DWARFVerifier.cpp and .h and move all DWARF verification code over into it.
Adrian requested we create a DWARFVerifier.cpp file to contain all of the DWARF verification stuff. This change simply moves the functionality over into DWARFVerifier.h and DWARFVerifier.cpp, renames the DWARFVerifier methods to start with lower case, and switches DWARFContext.cpp over to using the new functionality.

Differential Revision: https://reviews.llvm.org/D32809

llvm-svn: 302044
2017-05-03 16:02:29 +00:00
Zachary Turner c504ae3cef Resubmit r301986 and r301987 "Add codeview::StringTable"
This was reverted due to a "missing" file, but in reality
what happened was that I renamed a file, and then due to
a merge conflict both the old file and the new file got
added to the repository.  This led to an unused cpp file
being in the repo and not referenced by any CMakeLists.txt
but #including a .h file that wasn't in the repo.  In an
even more unfortunate coincidence, CMake didn't report the
unused cpp file because it was in a subdirectory of the
folder with the CMakeLists.txt, and not in the same directory
as any CMakeLists.txt.

The presence of the unused file was then breaking certain
tools that determine file lists by globbing rather than
by what's specified in CMakeLists.txt

In any case, the fix is to just remove the unused file from
the patch set.

llvm-svn: 302042
2017-05-03 15:58:37 +00:00
Greg Clayton 8df55b43e1 Verify that no compile units share the same line table in "llvm-dwarfdump --verify"
Check to make sure no compile units have the same DW_AT_stmt_list values. Report a verification error if they do.

Differential Revision: https://reviews.llvm.org/D32771

llvm-svn: 302039
2017-05-03 15:45:31 +00:00
Daniel Jasper dff096f217 Revert r301986 (and subsequent r301987).
The patch is failing to add StringTableStreamBuilder.h, but that isn't
even discovered because the corresponding StringTableStreamBuilder.cpp
isn't added to any CMakeLists.txt file and thus never built. I think
this patch is just incomplete.

llvm-svn: 302002
2017-05-03 07:29:25 +00:00
Zachary Turner 59e83892e0 Fix use after free in BinaryStream library.
This was reported by the ASAN bot, and it turned out to be
a fairly fundamental problem with the design of VarStreamArray
and the way it passes context information to the extractor.

The fix was cumbersome, and I'm not entirely pleased with it,
so I plan to revisit this design in the future when I'm not
pressed to get the bots green again.  For now, this fixes
the issue by storing the context information by value instead
of by reference, and introduces some impossibly-confusing
template magic to make things "work".

llvm-svn: 301999
2017-05-03 05:34:00 +00:00
Zachary Turner 67736594f7 Fix type conversion error.
llvm-svn: 301987
2017-05-02 23:41:51 +00:00
Zachary Turner 7dba20bd2b Make codeview::StringTable.
Previously we had knowledge of how to serialize and deserialize
a string table inside of DebugInfo/PDB, but the string table
that it serializes contains a piece that is actually considered
CodeView and can appear outside of a PDB.  We already have logic
in llvm-readobj and MCCodeView to read and write this format,
so it doesn't make sense to duplicate the logic in DebugInfoPDB
as well.

This patch makes codeview::StringTable (for writing) and
codeview::StringTableRef (for reading), updates DebugInfoPDB
to use these classes for its own writing, and updates llvm-readobj
to additionally use StringTableRef for reading.

It's a bit more difficult to get MCCodeView to use this for
writing, but it's a logical next step.

llvm-svn: 301986
2017-05-02 23:36:17 +00:00
Greg Clayton 6707046f90 Add line table verification to lldb-dwarfdump --verify
This patch verifies the .debug_line:
- verify all addresses in a line table sequence have ascending addresses
- verify that all line table file indexes are valid

Unit tests added for both cases.

Differential Revision: https://reviews.llvm.org/D32765

llvm-svn: 301984
2017-05-02 22:48:52 +00:00
Paul Robinson 2bc3873fe6 [DWARFv5] Parse new line-table header format.
The directory and file tables now have form-based content descriptors.
Parse these and extract the per-directory/file records based on the
descriptors.  For now we support only DW_FORM_string (inline) for the
path names; follow-up work will add support for indirect forms (i.e.,
DW_FORM_strp, strx<N>, and line_strp).

Differential Revision: http://reviews.llvm.org/D32713

llvm-svn: 301978
2017-05-02 21:40:47 +00:00
Greg Clayton c7695a8e45 Verify that all references point to actual DIEs in "llvm-dwarfdump --verify"
LTO and other fancy linking previously led to DWARF that contained invalid references. We already validate that CU relative references fall into the CU, and the DW_FORM_ref_addr references fall inside the .debug_info section, but we didn't validate that the references pointed to correct DIE offsets. This new verification will ensure that all references refer to actual DIEs and not an offset in between.

This caught a bug in DWARFUnit::getDIEForOffset() where if you gave it any offset, it would match the DIE that mathes the offset _or_ the next DIE. This has been fixed.

Differential Revision: https://reviews.llvm.org/D32722

llvm-svn: 301971
2017-05-02 20:28:33 +00:00
Zachary Turner e204a6c9a3 Rename pdb::StringTable -> pdb::PDBStringTable.
With the forthcoming codeview::StringTable which a pdb::StringTable
would hold an instance of as one member, this ambiguity becomes
confusing.  Rename to PDBStringTable to avoid this.

llvm-svn: 301948
2017-05-02 18:00:13 +00:00
Paul Robinson ba1c91564b Make DWARFDebugLine use StringRef for directory/file tables. NFC
Differential Revision: http://reviews.llvm.org/D32728

llvm-svn: 301940
2017-05-02 17:37:32 +00:00
Zachary Turner edef14510e [PDB/CodeView] Read/write codeview inlinee line information.
Previously we wrote line information and file checksum
information, but we did not write information about inlinee
lines and functions.  This patch adds support for that.

llvm-svn: 301936
2017-05-02 16:56:09 +00:00
Paul Robinson 9d4eb6922e Stylistic makeover of DWARFDebugLine before working on it. NFC
Rename parameters and locals to CamelCase, doxygenize the header, and
run clang-format on the whole thing.

llvm-svn: 301883
2017-05-01 23:27:55 +00:00
Zachary Turner 8a2ebfb1cd [CodeView] Write CodeView line information.
Differential Revision: https://reviews.llvm.org/D32716

llvm-svn: 301882
2017-05-01 23:27:42 +00:00
Greg Clayton 48432cfbeb Adds initial llvm-dwarfdump --verify support with unit tests.
lldb-dwarfdump gets a new "--verify" option that will verify a single file's DWARF debug info and will print out any errors that it finds. It will return an non-zero exit status if verification fails, and a zero exit status if verification succeeds. Adding the --quiet option will suppress any output the STDOUT or STDERR.

The first part of the verify does the following:

- verifies that all CU relative references (DW_FORM_ref1, DW_FORM_ref2, DW_FORM_ref4, DW_FORM_ref8, DW_FORM_ref_udata) have valid CU offsets
- verifies that all DW_FORM_ref_addr references have valid .debug_info offsets
- verifies that all DW_AT_ranges attributes have valid .debug_ranges offsets
- verifies that all DW_AT_stmt_list attributes have valid .debug_line offsets
- verifies that all DW_FORM_strp attributes have valid .debug_str offsets

Unit tests were added for each of the above cases.

Differential Revision: https://reviews.llvm.org/D32707

llvm-svn: 301844
2017-05-01 22:07:02 +00:00
Zachary Turner 7cc13e557c [PDB/CodeView] Rename some classes.
In preparation for introducing writing capabilities for each of
these classes, I would like to adopt a Foo / FooRef naming
convention, where Foo indicates that the class can manipulate and
serialize Foos, and FooRef indicates that it is an immutable view of
an existing Foo.  In other words, Foo is a writer and FooRef is a
reader.  This patch names some existing readers to conform to the
FooRef convention, while offering no functional change.

llvm-svn: 301810
2017-05-01 16:46:39 +00:00
Zachary Turner 5b6e4e0aed [llvm-pdbdump] Abstract some of the YAML/Raw printing code.
There is a lot of duplicate code for printing line info between
YAML and the raw output printer.  This introduces a base class
that can be shared between the two, and makes some minor
cleanups in the process.

llvm-svn: 301728
2017-04-29 01:13:21 +00:00
Zachary Turner 05bd9f3713 [llvm-readobj] Use LLVMDebugInfoCodeView to parse line tables.
The llvm-readobj parsing code currently exists in our CodeView
library, so we use that to parse instead of re-writing the logic
in the tool.

llvm-svn: 301718
2017-04-28 23:41:36 +00:00
George Rimar 96a3de2729 [DWARF] - Fix mistype in dump output of pub* tables. NFC.
There was a garbage character in output introduced by myself in
r290040 "[DWARF] - Introduce DWARFDebugPubTable class for dumping pub* sections."

llvm-svn: 301631
2017-04-28 08:54:10 +00:00
Zachary Turner c37cb0c6a5 [CodeView] Isolate Debug Info Fragments into standalone classes.
Previously parsing of these were all grouped together into a
single master class that could parse any type of debug info
fragment.

With writing forthcoming, the complexity of each individual
fragment is enough to warrant them having their own classes so
that reading and writing of each fragment type can be grouped
together, but isolated from the code for reading and writing
other fragment types.

In doing so, I found a place where parsing code was duplicated
for the FileChecksums fragment, across llvm-readobj and the
CodeView library, and one of the implementations had a bug.
Now that the codepaths are merged, the bug is resolved.

Differential Revision: https://reviews.llvm.org/D32547

llvm-svn: 301557
2017-04-27 16:12:16 +00:00
Zachary Turner e509447418 [Support] Make BinaryStreamArray extractors stateless.
Instead, we now pass a context memeber through the extraction
process.

llvm-svn: 301556
2017-04-27 16:11:47 +00:00
Zachary Turner 67c5601404 Rename some PDB classes.
We have a lot of very similarly named classes related to
dealing with module debug info.  This patch has NFC, it just
renames some classes to be more descriptive (albeit slightly
more to type).  The mapping from old to new class names is as
follows:

   Old          |        New
ModInfo         | DbiModuleDescriptor
ModuleSubstream | ModuleDebugFragment
ModStream       | ModuleDebugStream

With the corresponding Builder classes renamed accordingly.

Differential Revision: https://reviews.llvm.org/D32506

llvm-svn: 301555
2017-04-27 16:11:19 +00:00
George Rimar e6ef4488e1 [llvm-dwarfdump] - Change format for .gdb_index dump.
It is useful to output size of ranges when address ranges
section of .gdb_index is dumped.

It helps to compare outputs produced by different linkers,
for example. In that case address ranges can look very different,
when they are the same at fact. Difference comes from different 
low address because of different address of .text.

Differential revision: https://reviews.llvm.org/D32492

llvm-svn: 301527
2017-04-27 10:00:13 +00:00
Zachary Turner da307b64dd [llvm-pdbdump] Allow sorting / filtering by immediate padding
llvm-svn: 301358
2017-04-25 20:22:29 +00:00
Zachary Turner ee3b9c2558 [llvm-pdbdump] Dump File / Line Info to YAML.
We were already parsing and dumping this to the human readable
format, but not to the YAML format.  This does so, in preparation
for reading it in and reconstructing the line information from
YAML.

llvm-svn: 301357
2017-04-25 20:22:02 +00:00
Zachary Turner 1690164cac [llvm-pdbdump] Re-write the record layout code to be more resilient.
This reworks the way virtual bases are handled, and also the way
padding is detected across multiple levels of aggregates, producing
a much more accurate result.

llvm-svn: 301203
2017-04-24 17:47:24 +00:00
George Rimar ca53211beb [DWARF] - Take relocations in account when extracting ranges from .debug_ranges
I found this when investigated "Bug 32319 - .gdb_index is broken/incomplete" for LLD.

When we have object file with .debug_ranges section it may be filled with zeroes.
Relocations are exist in file to relocate this zeroes into real values later, but until that
a pair of zeroes is treated as terminator. And DWARF parser thinks there is no ranges at all
when I am trying to collect address ranges for building .gdb_index.

Solution implemented in this patch is to take relocations in account when parsing ranges.

Differential revision: https://reviews.llvm.org/D32228

llvm-svn: 301170
2017-04-24 10:19:45 +00:00
George Rimar f8a9642526 [DWARF] - Refactoring: localize handling of relocations in a single place.
This is splitted from D32228,
currently DWARF parsers code has few places that applied relocations values manually.
These places has similar duplicated code. Patch introduces separate method that can be
used to obtain relocated value. That helps to reduce code and simplifies things.

Differential revision: https://reviews.llvm.org/D32284

llvm-svn: 300956
2017-04-21 09:12:18 +00:00
Dehao Chen db569bae55 Code style change as suggested in https://reviews.llvm.org/D32177 (NFC)
llvm-svn: 300753
2017-04-19 20:52:21 +00:00
Dehao Chen a364f09f18 Using address range map to speedup finding inline stack for address.
Summary:
In the current implementation, to find inline stack for an address incurs expensive linear search in 2 places:

* linear search for the top-level DIE
* recursive linear traverse the DIE tree to find the path to the leaf DIE

In this patch, a map is built from address to its corresponding leaf DIE. The inline stack is built by traversing from the leaf DIE up to the root DIE. This speeds up batch symbolization by ~10X without noticible memory overhead.

Reviewers: dblaikie

Reviewed By: dblaikie

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D32177

llvm-svn: 300742
2017-04-19 20:09:38 +00:00
Dehao Chen e0b77b24d9 Revert r300697 which causes buildbot failure.
llvm-svn: 300708
2017-04-19 15:28:58 +00:00
Dehao Chen 74f3e0d426 Using address range map to speedup finding inline stack for address.
Summary:
In the current implementation, to find inline stack for an address incurs expensive linear search in 2 places:

* linear search for the top-level DIE
* recursive linear traverse the DIE tree to find the path to the leaf DIE

In this patch, a map is built from address to its corresponding leaf DIE. The inline stack is built by traversing from the leaf DIE up to the root DIE. This speeds up batch symbolization by ~10X without noticible memory overhead.

Reviewers: dblaikie

Reviewed By: dblaikie

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D32177

llvm-svn: 300697
2017-04-19 14:50:57 +00:00
Dehao Chen ef700d550e Add GNU_discriminator support for inline callsites in llvm-symbolizer.
Summary: LLVM symbolize cannot recognize GNU_discriminator for inline callsites. This patch adds support for it.

Reviewers: dblaikie

Reviewed By: dblaikie

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D32134

llvm-svn: 300486
2017-04-17 20:10:39 +00:00
Zachary Turner 4dc4f01a86 [llvm-pdbdump] Recursively dump class layout.
llvm-svn: 300258
2017-04-13 21:11:00 +00:00
George Rimar d4998b0344 [DWARF] - Simplify (use dyn_cast instead of isa + cast).
This addresses post commit review comments for r300039.

llvm-svn: 300188
2017-04-13 09:52:50 +00:00
Zachary Turner 75999dff93 Fix initialization order of class members.
llvm-svn: 300137
2017-04-12 23:27:43 +00:00
Zachary Turner 9e7dda3c6d [llvm-pdbdump] Minor prepatory refactor of Class Def Dumper.
In a followup patch I intend to introduce an additional dumping
mode which dumps a graphical representation of a class's layout.
In preparation for this, the text-based layout printer needs to
be split out from the graphical layout printer, and both need
to be able to use the same code for printing the intro and outro
of a class's definition (e.g. base class list, etc).

This patch does so, and in the process introduces a skeleton
definition for the graphical printer, while currently making
the graphical printer just print nothing.

NFC

llvm-svn: 300134
2017-04-12 23:18:51 +00:00
Zachary Turner c883a8c6dc [llvm-pdbdump] More advanced class definition dumping.
Previously the dumping of class definitions was very primitive,
and it made it hard to do more than the most trivial of output
formats when dumping.  As such, we would only dump one line for
each field, and then dump non-layout items like nested types
and enums.

With this patch, we do a complete analysis of the object
hierarchy including aggregate types, bases, virtual bases,
vftable analysis, etc.  The only immediately visible effects
of this are that a) we can now dump a line for the vfptr where
before we would treat that as padding, and b) we now don't
treat virtual bases that come at the end of a class as padding
since we have a more detailed analysis of the class's storage
usage.

In subsequent patches, we should be able to use this analysis
to display a complete graphical view of a class's layout including
recursing arbitrarily deep into an object's base class / aggregate
member hierarchy.

llvm-svn: 300133
2017-04-12 23:18:21 +00:00
Krasimir Georgiev 4ed589d8d6 [DWARF] Fix compiler warnings in DWARFContext.cpp, NFCi
llvm-svn: 300051
2017-04-12 11:33:26 +00:00
George Rimar 702dac6d35 [DWARF] - Refactoring of DWARFContextInMemory implementation.
This change is basically relative to D31136, where I initially wanted to
implement some relocations handling optimization which shows it can give
significant boost. Though even without any caching algorithm looks
code can have some cleanup at first.

Refactoring separates out code for taking symbol address, used in relocations
computation.

Differential revision: https://reviews.llvm.org/D31747

llvm-svn: 300039
2017-04-12 08:59:15 +00:00
Reid Kleckner 6e545ffc4e [PDB] Emit index/offset pairs for TPI and IPI streams
Summary:
This lets PDB readers lookup type record data by type index in O(log n)
time. It also enables makes `cvdump -t` work on PDBs produced by LLD.
cvdump will not dump a PDB that doesn't have an index-to-offset table.

The table is sorted by type index, and has an entry every 8KB. Looking
up a type record by index is a binary search of this table, followed by
a scan of at most 8KB.

Reviewers: ruiu, zturner, inglorion

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D31636

llvm-svn: 299958
2017-04-11 16:26:15 +00:00
Vassil Vassilev e1f12fadc0 Remove unused functions. Remove static qualifier from functions in header files. NFC.
llvm-svn: 299947
2017-04-11 14:55:32 +00:00
Adrian McCarthy 08eb343cce Improves pretty printing of variable types in llvm-pdbdump
* Adds support for pointers to arrays, which was missing
* Adds some tests
* Improves consistency of const and volatile qualifiers
* Eliminates non-composable special case code for arrays and function by using
  a more general recursive approach
* Has a hack for getting the calling convention into the right spot for
  pointer-to-functions

Given the rapid changes happenning in llvm-pdbdump, this may be difficult to
merge.

Differential Revision: https://reviews.llvm.org/D31832

llvm-svn: 299848
2017-04-10 16:43:09 +00:00
Zachary Turner 1b1a70f172 General usability improvements to generic PDB library.
1. Added some asserts to make sure concrete symbol types don't
   get constructed with RawSymbols that have an incompatible
   SymTag enum value.
2. Added new forwarding macros that auto-define an Id/Sym method
   pair whenever there is a method that returns a SymIndexId.
   Previously we would just provide one method that returned only
   the SymIndexId and it was up to the caller to use the Session
   object to get a pointer to the symbol.  Now we automatically
   get both the method that returns the Id, as well as a method
   that returns the pointer directly with just one macro.
3. Added some methods for dumping straight to stdout that can
   be used from inside the debugger for diagnostics during a
   debug session.
4. Added a clone() method and a cast<T>() method to PDBSymbol
   that can shorten some usage patterns.

llvm-svn: 299831
2017-04-10 06:14:09 +00:00
Reid Kleckner 13fc411e39 [PDB] Save one type record copy
Summary:
The TypeTableBuilder provides stable storage for type records. We don't
need to copy all of the bytes into a flat vector before adding it to the
TpiStreamBuilder.

This makes addTypeRecord take an ArrayRef<uint8_t> and a hash code to go
with it, which seems like a simplification.

Reviewers: ruiu, zturner, inglorion

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D31634

llvm-svn: 299406
2017-04-04 00:56:34 +00:00
Reid Kleckner c4b5d794f1 [codeview] Cope with unsorted streams in type merging
Summary:
MASM can produce type streams that are not topologically sorted. It can
even produce type streams with circular references, but those are not
common in practice.

Reviewers: inglorion, ruiu

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D31629

llvm-svn: 299403
2017-04-03 23:58:15 +00:00
Reid Kleckner 1c3b5087b7 [codeview] Add support for label type records
MASM can produce these type records.

llvm-svn: 299388
2017-04-03 21:25:20 +00:00
Reid Kleckner acd9a6f09d [codeview] Fix buggy BeginIndexMapSize assertion
This assert is just trying to test that processing each record adds
exactly one entry to the index map. The assert logic was wrong when the
first record in the type stream was a field list.

I've simplified the code by moving the LF_FIELDLIST-specific logic into
the callback for that record type.

llvm-svn: 299035
2017-03-29 22:51:22 +00:00
Adrian McCarthy 4d93d66ddd Re-land: "Make NativeExeSymbol a concrete subclass of NativeRawSymbol [PDB]"
This should work on all platforms now that r299006 has landed.  Tested locally
on Windows and Linux.

This moves exe symbol-specific method implementations out of NativeRawSymbol
into a concrete subclass. Also adds implementations for hasCTypes and
hasPrivateSymbols and a simple test to ensure the native reader can access the
summary information for the executable from the PDB.

Original Differential Revision: https://reviews.llvm.org/D31059

llvm-svn: 299019
2017-03-29 19:27:08 +00:00
Reid Kleckner 5d57752c81 [PDB] Split item and type records when merging type streams
Summary: MSVC does this when producing a PDB.

Reviewers: ruiu

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D31316

llvm-svn: 298717
2017-03-24 17:26:38 +00:00
Reid Kleckner a5d187b0ff [PDB] Use two DBs when dumping the IPI stream
Summary:
When dumping these records from an object file section, we should use
only one type database. However, when dumping from a PDB, we should use
two: one for the type stream and one for the IPI stream.

Certain type records that normally live in the .debug$T object file
section get moved over to the IPI stream of the PDB file and they get
new indices.

So far, I've noticed that the MSVC linker always moves these records
into IPI:
- LF_FUNC_ID
- LF_MFUNC_ID
- LF_STRING_ID
- LF_SUBSTR_LIST
- LF_BUILDINFO
- LF_UDT_MOD_SRC_LINE

These records have index fields that can point into TPI or IPI. In
particular, LF_SUBSTR_LIST and LF_BUILDINFO point to LF_STRING_ID
records to describe compilation command lines.

I've modified the dumper to have an optional pointer to the item DB, and
to do type name lookup of these fields in that DB. See printItemIndex.
The result is that our pdbdump-headers.test is more faithful to the PDB
contents and the output is less confusing.

Reviewers: ruiu

Subscribers: amccarth, zturner, llvm-commits

Differential Revision: https://reviews.llvm.org/D31309

llvm-svn: 298649
2017-03-23 21:36:25 +00:00
Adrian McCarthy 3c0328e011 Somehow this still breaks because of ANSI color codes in test output on Linux.
Reverting until I can figure out the root cause.

Revert "Re-land:  Make NativeExeSymbol a concrete subclass of NativeRawSymbol [PDB]"

This reverts commit f461a70cc376f0f91c8b4917be79479cc86330a5.

llvm-svn: 298626
2017-03-23 17:18:50 +00:00
Adrian McCarthy 997a15c3c3 Re-land: Make NativeExeSymbol a concrete subclass of NativeRawSymbol [PDB]
The new test should pass on all platforms now that llvm-pdbdump has the
`-color-output` option.

This moves exe symbol-specific method implementations out of NativeRawSymbol
into a concrete subclass. Also adds implementations for hasCTypes and
hasPrivateSymbols and a simple test to ensure the native reader can access
the summary information for the executable from the PDB.

Original Differential Revision: https://reviews.llvm.org/D31059

llvm-svn: 298623
2017-03-23 16:45:20 +00:00
Reid Kleckner c573acd9e9 [codeview] Move type index remapping logic to type merger
Summary:
This removes the 'remapTypeIndices' method on every TypeRecord class. My
original idea was that this would be the beginning of some kind of
generic entry point that would enumerate all of the TypeIndices inside
of a TypeRecord, so that we could write generic graph algorithms for
them without duplicating the knowledge of which fields are type index
fields everywhere. This never happened, and nothing else uses this
method. I need to change the API to deal with merging into IPI streams,
so let's move it into the file that uses it first.

Reviewers: zturner, ruiu

Reviewed By: zturner, ruiu

Subscribers: mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D31267

llvm-svn: 298564
2017-03-23 00:14:23 +00:00
Reid Kleckner 45928018c5 [codeview] Use separate records for LF_SUBSTR_LIST and LF_ARGLIST
They are structurally the same, but now we need to distinguish them
because one record lives in the IPI stream and the other lives in TPI.

llvm-svn: 298474
2017-03-22 01:37:38 +00:00
Zachary Turner 2d9c082033 Revert "Make NativeExeSymbol a concrete subclass of NativeRawSymbol [PDB]"
For some reason this is causing ANSI color codes to be printed
even when run through FileCheck.

llvm-svn: 298026
2017-03-17 00:46:42 +00:00
Zachary Turner 2ed2aa75bf [pdb] Fix an uninitialized read, and add a test for it.
This was originally reported in pr32249, uncovered by PTVS-Studio.
There was no code coverage for this path because it was
difficult to construct odd-case PDB files that were not generated
by cl.

Now that we can write construct minimal PDB files from YAML,
it's easy to construct fragments that generate whatever we want.

In this patch I add a test that creates 2 type records.  One
with a unique name, and one without.  I verify that we can go
from PDB to Yaml with no errors.  In a future patch I'd like
to add something like llvm-pdbdump raw -lookup-type that will
just dump one record and nothing else, which should make it
a bit cleaner to find this kind of thing.

llvm-svn: 298017
2017-03-17 00:15:55 +00:00
Zachary Turner 42cb87f401 [PDB] It is not an error getting the "Invalid" Annotation opcode.
The linker can insert invalid opcodes to indicate padding
bytes, and we should not fail in this case.

llvm-svn: 298016
2017-03-17 00:15:27 +00:00
Adrian McCarthy 21b54cf632 Make NativeExeSymbol a concrete subclass of NativeRawSymbol [PDB]
This moves exe symbol-specific method implementations out of NativeRawSymbol
into a concrete subclass. Also adds implementations for hasCTypes and
hasPrivateSymbols and a simple test to ensure the native reader can access
the summary information for the executable from the PDB.

Differential Revision: https://reviews.llvm.org/D31059

llvm-svn: 298005
2017-03-16 22:28:39 +00:00