Commit Graph

84 Commits

Author SHA1 Message Date
Zachary Turner c883a8c6dc [llvm-pdbdump] More advanced class definition dumping.
Previously the dumping of class definitions was very primitive,
and it made it hard to do more than the most trivial of output
formats when dumping.  As such, we would only dump one line for
each field, and then dump non-layout items like nested types
and enums.

With this patch, we do a complete analysis of the object
hierarchy including aggregate types, bases, virtual bases,
vftable analysis, etc.  The only immediately visible effects
of this are that a) we can now dump a line for the vfptr where
before we would treat that as padding, and b) we now don't
treat virtual bases that come at the end of a class as padding
since we have a more detailed analysis of the class's storage
usage.

In subsequent patches, we should be able to use this analysis
to display a complete graphical view of a class's layout including
recursing arbitrarily deep into an object's base class / aggregate
member hierarchy.

llvm-svn: 300133
2017-04-12 23:18:21 +00:00
Zachary Turner d9dc2829ea [Support] Move Stream library from MSF -> Support.
After several smaller patches to get most of the core improvements
finished up, this patch is a straight move and header fixup of
the source.

Differential Revision: https://reviews.llvm.org/D30266

llvm-svn: 296810
2017-03-02 20:52:51 +00:00
Zachary Turner 7329569a05 Re-enable BinaryStreamTest.StreamReaderObject.
This was failing because I was using memcmp to compare two
objects that included padding bytes, which were uninitialized.

llvm-svn: 296681
2017-03-01 21:30:06 +00:00
Zachary Turner 097e9574d1 Disable BinaryStreamTest.StreamReaderObject.
llvm-svn: 296672
2017-03-01 20:58:28 +00:00
Zachary Turner d2fd4aeb56 [PDB] Fix and re-enable BinaryStreamArray test.
This was due to the test stream choosing an arbitrary partition
index for introducing the discontinuity rather than choosing
an index that would be correctly aligned for the type of data.

Also added an assertion into FixedStreamArray so that this will
be caught on all bots in the future, and not just the UBSan bot.

llvm-svn: 296661
2017-03-01 19:29:11 +00:00
Zachary Turner 081b1bceae [PDB] Re-add BinaryStreamTest.
This re-adds all the binary stream tests.  This was reverted due
to some misaligned reads.  For now the offending test is
disabled while I investigate.

llvm-svn: 296643
2017-03-01 17:22:36 +00:00
Vedant Kumar b0baffc9a6 Remove unittests/DebugInfo/PDB/BinaryStreamTest.cpp (from r296555)
It breaks the ToT UBSan bots:

/Users/vk/Desktop/llvm/include/llvm/DebugInfo/MSF/BinaryStreamArray.h:246:12: runtime error: reference binding to misaligned address 0x7f925540939a for type 'const int', which requires 4 byte alignment
0x7f925540939a: note: pointer points here
 05 00  00 00 01 00 00 00 02 00  00 00 03 00 00 00 00 00  00 00 00 00 00 00 00 00  70 98 50 06 01 00
              ^
0  DebugInfoPDBTests                   0x0000000106263cbd llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 45
1  DebugInfoPDBTests                   0x00000001062628ff llvm::sys::RunSignalHandlers() + 159
2  DebugInfoPDBTests                   0x0000000106264593 SignalHandler(int) + 179
3  libsystem_platform.dylib            0x0000000107bb3fba _sigtramp + 26
4  libsystem_pthread.dylib             0x0000000107bd82c8 _pthread_keys + 9720
5  libsystem_c.dylib                   0x0000000107947f83 abort + 127
6  libclang_rt.ubsan_osx_dynamic.dylib 0x0000000106bb5fc2 __sanitizer::Abort() + 66
7  DebugInfoPDBTests                   0x000000010613f880 llvm::FixedStreamArrayIterator<int>::operator+=(long) + 0
8  DebugInfoPDBTests                   0x000000010613f615 llvm::FixedStreamArrayIterator<int>::operator*() const + 37
9  DebugInfoPDBTests                   0x000000010613f3cb std::__1::enable_if<__is_forward_iterator<llvm::FixedStreamArrayIterator<int> >::value, void>::type std::__1::vector<int, std::__1::allocator<int> >::__construct_at_end<llvm::FixedStreamArrayIterator<int> >(llvm::FixedStreamArrayIterator<int>, llvm::FixedStreamArrayIterator<int>, unsigned long) + 251
10 DebugInfoPDBTests                   0x000000010613f292 std::__1::vector<int, std::__1::allocator<int> >::vector<llvm::FixedStreamArrayIterator<int> >(llvm::FixedStreamArrayIterator<int>, std::__1::enable_if<(__is_forward_iterator<llvm::FixedStreamArrayIterator<int> >::value) && (is_constructible<int, std::__1::iterator_traits<llvm::FixedStreamArrayIterator<int> >::reference>::value), llvm::FixedStreamArrayIterator<int> >::type) + 226
11 DebugInfoPDBTests                   0x000000010613ddb7 std::__1::vector<int, std::__1::allocator<int> >::vector<llvm::FixedStreamArrayIterator<int> >(llvm::FixedStreamArrayIterator<int>, std::__1::enable_if<(__is_forward_iterator<llvm::FixedStreamArrayIterator<int> >::value) && (is_constructible<int, std::__1::iterator_traits<llvm::FixedStreamArrayIterator<int> >::reference>::value), llvm::FixedStreamArrayIterator<int> >::type) + 87
12 DebugInfoPDBTests                   0x000000010613d4af (anonymous namespace)::BinaryStreamTest_StreamReaderIntegerArray_Test::TestBody() + 1279
13 DebugInfoPDBTests                   0x00000001062780f3 testing::Test::Run() + 179
14 DebugInfoPDBTests                   0x0000000106279594 testing::TestInfo::Run() + 308
15 DebugInfoPDBTests                   0x000000010627a6a3 testing::TestCase::Run() + 307
16 DebugInfoPDBTests                   0x00000001062849d4 testing::internal::UnitTestImpl::RunAllTests() + 756
17 DebugInfoPDBTests                   0x0000000106284558 testing::UnitTest::Run() + 152
18 DebugInfoPDBTests                   0x0000000106266fa5 main + 117
19 libdyld.dylib                       0x00000001078506a5 start + 1
zsh: abort      ./unittests/DebugInfo/PDB/DebugInfoPDBTests

llvm-svn: 296641
2017-03-01 17:10:03 +00:00
Zachary Turner 1fa70d71f2 [PDB] Remove use of std error codes.
I already created a BinaryStreamError class for this purpose,
so update the code to use that on the remaining occurrences
of errc values.

This should also address the issue which led to r296583.

llvm-svn: 296640
2017-03-01 17:02:41 +00:00
NAKAMURA Takumi 1a1d959423 (Rewroking r296581) PDB/BinaryStreamTest.cpp: Appease mingw to avoid std::errc::no_buffer_space.
Unfortunately, mingw's libstdc++ doesn't provide winsock2 errors.
That said, we should avoid raising OS-oriented error code in our code.

For now, I suggest to define custom error from std::error_category.
See also; https://reviews.llvm.org/D20592

llvm-svn: 296583
2017-03-01 05:11:41 +00:00
NAKAMURA Takumi 4913a7921e Revert r296581, "PDB/BinaryStreamTest.cpp: Appease mingw to avoid std::errc::no_buffer_space."
Wrong commit -- I have unstaged changes.

llvm-svn: 296582
2017-03-01 05:11:37 +00:00
NAKAMURA Takumi 623ab91b53 PDB/BinaryStreamTest.cpp: Appease mingw to avoid std::errc::no_buffer_space.
Unfortunately, mingw's libstdc++ doesn't provide winsock2 errors.
That said, we should avoid raising OS-oriented error code in our code.

For now, I suggest to define custom error from std::error_category.
See also; https://reviews.llvm.org/D20592

llvm-svn: 296581
2017-03-01 05:06:31 +00:00
Zachary Turner b75c5c564c Move constexpr arrays out of class definition.
GCC Linker doesn't seem to like this.

llvm-svn: 296560
2017-03-01 01:17:31 +00:00
Zachary Turner 1d855afc5f Fix signed / unsigned comparison warning.
llvm-svn: 296557
2017-03-01 01:07:10 +00:00
Zachary Turner 9538bc4578 [PDB] Add an additional test for BinaryStreamRef.
A bug was uncovered where if you have a StreamRef whose ViewOffset
is > 0, then when you call readLongestContiguousChunk it will
succeed even when it shouldn't, and it always return you a
buffer that was taken as if the ViewOffset was 0.

Fixed this bug and added a test for it.

llvm-svn: 296556
2017-03-01 01:04:16 +00:00
Zachary Turner 24ba722f05 [PDB] Add tests for BinaryStream.
llvm-svn: 296555
2017-03-01 01:03:53 +00:00
Zachary Turner d0b44fa788 [PDB] Add BinaryStreamError.
This migrates the stream code away from MSFError to using its
own custom Error class.

llvm-svn: 296494
2017-02-28 17:49:34 +00:00
Zachary Turner 695ed56ba5 [PDB] Make streams carry their own endianness.
Before the endianness was specified on each call to read
or write of the StreamReader / StreamWriter, but in practice
it's extremely rare for streams to have data encoded in
multiple different endiannesses, so we should optimize for the
99% use case.

This makes the code cleaner and more general, but otherwise
has NFC.

llvm-svn: 296415
2017-02-28 00:04:07 +00:00
Zachary Turner 120faca41b [PDB] Partial resubmit of r296215, which improved PDB Stream Library.
This was reverted because it was breaking some builds, and
because of incorrect error code usage.  Since the CL was
large and contained many different things, I'm resubmitting
it in pieces.

This portion is NFC, and consists of:

1) Renaming classes to follow a consistent naming convention.
2) Fixing the const-ness of the interface methods.
3) Adding detailed doxygen comments.
4) Fixing a few instances of passing `const BinaryStream& X`.  These
   are now passed as `BinaryStreamRef X`.

llvm-svn: 296394
2017-02-27 22:11:43 +00:00
NAKAMURA Takumi 05a75e40da Revert r296215, "[PDB] General improvements to Stream library." and followings.
r296215, "[PDB] General improvements to Stream library."
r296217, "Disable BinaryStreamTest.StreamReaderObject temporarily."
r296220, "Re-enable BinaryStreamTest.StreamReaderObject."
r296244, "[PDB] Disable some tests that are breaking bots."
r296249, "Add static_cast to silence -Wc++11-narrowing."

std::errc::no_buffer_space should be used for OS-oriented errors for socket transmission.
(Seek discussions around llvm/xray.)

I could substitute s/no_buffer_space/others/g, but I revert whole them ATM.

Could we define and use LLVM errors there?

llvm-svn: 296258
2017-02-25 17:04:23 +00:00
Daniel Jasper 6249d4d337 Add static_cast to silence -Wc++11-narrowing.
llvm-svn: 296249
2017-02-25 07:53:36 +00:00
Zachary Turner 4284ce1348 [PDB] Disable some tests that are breaking bots.
This has to do with big endian, but I can't fix it until
Monday.  The code itself is fine, just the tests are wrong.
Disabling 3 tests for now.

llvm-svn: 296244
2017-02-25 05:57:57 +00:00
Zachary Turner 3e2b6925b9 Re-enable BinaryStreamTest.StreamReaderObject.
I had an invalid pointer / size calculation that was causing
a stack smash.  Should be fixed now.

llvm-svn: 296220
2017-02-25 01:20:08 +00:00
Zachary Turner 42d450733b Disable BinaryStreamTest.StreamReaderObject temporarily.
This is crashing on some bots, so I need some time to investigate.

llvm-svn: 296217
2017-02-25 00:52:59 +00:00
Zachary Turner af299ea5d4 [PDB] General improvements to Stream library.
This adds various new functionality and cleanup surrounding the
use of the Stream library.  Major changes include:

* Renaming of all classes for more consistency / meaningfulness
* Addition of some new methods for reading multiple values at once.
* Full suite of unit tests for reader / writer functionality.
* Full set of doxygen comments for all classes.
* Streams now store their own endianness.
* Fixed some bugs in a few of the classes that were discovered
  by the unit tests.

llvm-svn: 296215
2017-02-25 00:44:30 +00:00
Zachary Turner d2684b7969 [PDB] Rename Stream related source files.
This is part of a larger effort to get the Stream code moved
up to Support.  I don't want to do it in one large patch, in
part because the changes are so big that it will treat everything
as file deletions and add, losing history in the process.
Aside from that though, it's just a good idea in general to
make small changes.

So this change only changes the names of the Stream related
source files, and applies necessary source fix ups.

llvm-svn: 296211
2017-02-25 00:33:34 +00:00
Adrian McCarthy 3e0c0688e9 Fix unit tests after r296049.
llvm-svn: 296055
2017-02-24 00:25:17 +00:00
Zachary Turner 181fe17b6f Don't assume little endian in StreamReader / StreamWriter.
In an effort to generalize this so it can be used by more than
just PDB code, we shouldn't assume little endian.

llvm-svn: 295525
2017-02-18 01:35:33 +00:00
Zachary Turner 7b327d051b [pdb] Add the ability to resolve TypeServer PDBs.
Some PDBs or object files can contain references to other PDBs
where the real type information lives.  When this happens,
all type indices in the original PDB are meaningless because
their records are not there.

With this patch we add the ability to pull type info from those
secondary PDBs.

Differential Revision: https://reviews.llvm.org/D29973

llvm-svn: 295382
2017-02-16 23:35:45 +00:00
Adrian McCarthy 1f5f06435d Fix for r293104, which renamed a directory.
llvm-svn: 293105
2017-01-25 22:48:57 +00:00
Zachary Turner 760ad4da60 [pdb] Write the Named Stream mapping to Yaml and binary.
Differential Revision: https://reviews.llvm.org/D28919

llvm-svn: 292665
2017-01-20 22:42:09 +00:00
Zachary Turner f04d6e8d52 [PDB] Rename some files to be more intuitive.
llvm-svn: 292663
2017-01-20 22:41:15 +00:00
Zachary Turner 11036a909f [pdb] Add HashTable data structure.
This was being parsed / serialized ad-hoc inside the code
for a specific PDB stream.  But this data structure is used
in multiple ways / places within the PDB format.  To be able
to re-use it we need to raise this code out and make it more
generic.  In doing so, a number of bugs are fixed in the
original implementation, and support is added for growing
the hash table and deleting items from the hash table,
which had either been omitted or incorrect implemented in
the initial version.

Differential Revision: https://reviews.llvm.org/D28715

llvm-svn: 292535
2017-01-19 23:31:24 +00:00
Rui Ueyama dcd32937dc PDB: Add a class to create the /names stream contents.
This patch adds a new class NameHashTableBuilder which creates /names streams.
This patch contains a test to confirm that a stream created by
NameHashTableBuilder can be read by NameHashTable reader class.

Differential Revision: https://reviews.llvm.org/D28707

llvm-svn: 292040
2017-01-15 00:36:02 +00:00
David Majnemer 9bca03bf81 [PDB] Validate superblock addresses
- Validate the address of the block map.
- Validate the address of the free block map.

llvm-svn: 290053
2016-12-18 00:41:10 +00:00
Zachary Turner 9f664051ab [pdb] Fix unit test compilation.
llvm-svn: 281560
2016-09-14 23:17:08 +00:00
Zachary Turner a3225b0451 [msf] Resubmit "Rename Msf -> MSF".
Previously this change was submitted from a Windows machine, so
changes made to the case of filenames and directory names did
not survive the commit, and as a result the CMake source file
names and the on-disk file names did not match on case-sensitive
file systems.

I'm resubmitting this patch from a Linux system, which hopefully
allows the case changes to make it through unfettered.

llvm-svn: 277213
2016-07-29 20:56:36 +00:00
Zachary Turner 334aec4dd2 Revert "[msf] Rename Msf to MSF."
This reverts commit 4d1557ffac41e079bcb1abbcf04f512474dcd6fe.

llvm-svn: 277194
2016-07-29 18:38:47 +00:00
Zachary Turner a010f5cef0 [msf] Rename Msf to MSF.
In a previous patch, it was suggested to use all caps instead of
rolling caps for initialisms, so this patch changes everything
to do this.

llvm-svn: 277190
2016-07-29 18:24:26 +00:00
Zachary Turner 6c9dbbd661 [pdb] Fix another narrowing conversion on x64 builds.
llvm-svn: 277026
2016-07-28 19:47:04 +00:00
Zachary Turner d66889cbae [pdb] Refactor library to more clearly separate reading/writing
Reviewed By: amccarth, ruiu
Differential Revision: https://reviews.llvm.org/D22693

llvm-svn: 277019
2016-07-28 19:12:28 +00:00
Zachary Turner 199f48a5f0 Get rid of IMsfStreamData class.
This was a pure virtual base class whose purpose was to abstract
away the notion of how you retrieve the layout of a discontiguous
stream of blocks in an Msf file.  This led to too many layers of
abstraction making it difficult to figure out what was going on
and extend things.  Ultimately, a stream's layout is decided by
its length and the array of block numbers that it lives on.  So
rather than have an abstract base class which can return this in
any number of ways, it's more straightforward to simply store them
as fields of a trivial struct, and also to give a more appropriate
name.

This patch does that.  It renames IMsfStreamData to MsfStreamLayout,
and deletes the 2 concrete implementations, DirectoryStreamData
and IndexedStreamData.  MsfStreamLayout is a trivial struct
with the necessary data.

llvm-svn: 277018
2016-07-28 19:11:09 +00:00
Reid Kleckner 84149c45bf Fix dangling reference to temporary in use of ArrayRef
Fixes tests locally for me with MSVC 2015.

llvm-svn: 277015
2016-07-28 18:17:01 +00:00
Zachary Turner 78d1c32dd6 Make DebugInfoMsf a dependency of DebugInfoPDBTests.
For some reason this doesn't cause linker errors with MSVC or
clang-cl, but the bots seem to be failing with other compilers.

llvm-svn: 276463
2016-07-22 20:05:30 +00:00
Zachary Turner bac69d33d0 [msf] Create LLVMDebugInfoMsf
This provides a better layering of responsibilities among different
aspects of PDB writing code.  Some of the MSF related code was
contained in CodeView, and some was in PDB prior to this.  Further,
we were often saying PDB when we meant MSF, and the two are
actually independent of each other since in theory you can have
other types of data besides PDB data in an MSF.  So, this patch
separates the MSF specific code into its own library, with no
dependencies on anything else, and DebugInfoCodeView and
DebugInfoPDB take dependencies on DebugInfoMsf.

llvm-svn: 276458
2016-07-22 19:56:05 +00:00
Zachary Turner b927e02e1b [pdb] Teach MsfBuilder and other classes about the Free Page Map.
Block 1 and 2 of an MSF file are bit vectors that represent the
list of blocks allocated and free in the file.  We had been using
these blocks to write stream data and other data, so we mark them
as the free page map now.  We don't yet serialize these pages to
the disk, but at least we make a note of what it is, and avoid
writing random data to them.

Doing this also necessitated cleaning up some of the tests to be
more general and hardcode fewer values, which is nice.

llvm-svn: 275629
2016-07-15 22:17:19 +00:00
Zachary Turner faa554b2fd [pdb] Use MsfBuilder to handle the writing PDBs.
Previously we would read a PDB, then write some of it back out,
but write the directory, super block, and other pertinent metadata
back out unchanged.  This generates incorrect PDBs since the amount
of data written was not always the same as the amount of data read.

This patch changes things to use the newly introduced `MsfBuilder`
class to write out a correct and accurate set of Msf metadata for
the data *actually* written, which opens up the door for adding and
removing type records, symbol records, and other types of data to
an existing PDB.

llvm-svn: 275627
2016-07-15 22:16:56 +00:00
Zachary Turner f52a899f4a [pdb] Introduce MsfBuilder for laying out PDB files.
Reviewed by: ruiu
Differential Revision: https://reviews.llvm.org/D22308

llvm-svn: 275611
2016-07-15 20:43:38 +00:00
David Majnemer 6211b1f1f9 [llvm-pdbdump] Propagate errors a little more consistently
PDBFile::getBlockData didn't really return any indication that it
failed.  It merely returned an empty buffer.

llvm-svn: 275009
2016-07-10 03:34:47 +00:00
Zachary Turner 352cfe41cd Try to fix compilation error in DebugInfoPDBTests.
llvm-svn: 274881
2016-07-08 16:57:14 +00:00
NAKAMURA Takumi f1b183c402 DebugInfoPDBTests:MappedBlockStreamTest.TestWriteThenRead: Avoid assigning temporary object to ArrayRef.
llvm-svn: 272457
2016-06-11 06:37:28 +00:00