Commit Graph

247 Commits

Author SHA1 Message Date
Zachary Turner 1dfcf8d92c Revert "[CodeView] Provide a common interface for type collections."
This is a squash of ~5 reverts of, well, pretty much everything
I did today.  Something is seriously broken with lit on Windows
right now, and as a result assertions that fire in tests are
triggering failures.  I've been breaking non-Windows bots all
day which has seriously confused me because all my tests have
been passing, and after running lit with -a to view the output
even on successful runs, I find out that the tool is crashing
and yet lit is still reporting it as a success!

At this point I don't even know where to start, so rather than
leave the tree broken for who knows how long, I will get this
back to green, and then once lit is fixed on Windows, hopefully
hopefully fix the remaining set of problems for real.

llvm-svn: 303409
2017-05-19 05:57:45 +00:00
Zachary Turner b32ec02b80 [CodeView] Raise the source to ID map out of the TypeStreamMerger.
This map will be needed to rewrite symbol streams after re-writing
the corresponding type streams.

llvm-svn: 303390
2017-05-18 23:04:08 +00:00
Zachary Turner 0c60f269fc [CodeView] Provide a common interface for type collections.
Right now we have multiple notions of things that represent collections of
types. Most commonly used are TypeDatabase, which is supposed to keep
mappings from TypeIndex to type name when reading a type stream, which
happens when reading PDBs. And also TypeTableBuilder, which is used to
build up a collection of types dynamically which we will later serialize
(i.e. when writing PDBs).

But often you just want to do some operation on a collection of types, and
you may want to do the same operation on any kind of collection. For
example, you might want to merge two TypeTableBuilders or you might want
to merge two type streams that you loaded from various files.

This dichotomy between reading and writing is responsible for a lot of the
existing code duplication and overlapping responsibilities in the existing
CodeView library classes. For example, after building up a
TypeTableBuilder with a bunch of type records, if we want to dump it we
have to re-invent a bunch of extra glue because our dumper takes a
TypeDatabase or a CVTypeArray, which are both incompatible with
TypeTableBuilder.

This patch introduces an abstract base class called TypeCollection which
is shared between the various type collection like things. Wherever we
previously stored a TypeDatabase& in some common class, we now store a
TypeCollection&.

The advantage of this is that all the details of how the collection are
implemented, such as lazy deserialization of partial type streams, is
completely transparent and you can just treat any collection of types the
same regardless of where it came from.

Differential Revision: https://reviews.llvm.org/D33293

llvm-svn: 303388
2017-05-18 23:03:06 +00:00
Zachary Turner d9a626332e [BinaryStream] Reduce the amount of boiler plate needed to use.
Often you have an array and you just want to use it.  With the current
design, you have to first construct a `BinaryByteStream`, and then create
a `BinaryStreamRef` from it.  Worse, the `BinaryStreamRef` holds a pointer
to the `BinaryByteStream`, so you can't just create a temporary one to
appease the compiler, you have to actually hold onto both the `ArrayRef`
as well as the `BinaryByteStream` *AND* the `BinaryStreamReader` on top of
that.  This makes for very cumbersome code, often requiring one to store a
`BinaryByteStream` in a class just to circumvent this.

At the cost of some added complexity (not exposed to users, but internal
to the library), we can do better than this.  This patch allows us to
construct `BinaryStreamReaders` and `BinaryStreamWriters` directly from
source data (e.g. `StringRef`, `MutableArrayRef<uint8_t>`, etc).  Not only
does this reduce the amount of code you have to type and make it more
obvious how to use it, but it solves real lifetime issues when it's
inconvenient to hold onto a `BinaryByteStream` for a long time.

The additional complexity is in the form of an added layer of indirection.
Whereas before we simply stored a `BinaryStream*` in the ref, we now store
both a `BinaryStream*` **and** a `std::shared_ptr<BinaryStream>`.  When
the user wants to construct a `BinaryStreamRef` directly from an
`ArrayRef` etc, we allocate an internal object that holds ownership over a
`BinaryByteStream` and forwards all calls, and store this in the
`shared_ptr<>`.  This also maintains the ref semantics, as you can copy it
by value and references refer to the same underlying stream -- the one
being held in the object stored in the `shared_ptr`.

Differential Revision: https://reviews.llvm.org/D33293

llvm-svn: 303294
2017-05-17 20:23:31 +00:00
Eric Beckmann 674deed94e Fix the Endianness bug by adding the little endian UTF marker.
Summary: Quick fix

Reviewers: zturner, uweigand

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D33014

llvm-svn: 302573
2017-05-09 19:35:45 +00:00
Eric Beckmann cd704cb6c4 Hopefully one last commit to fix this patch, addresses string reference
issues.

llvm-svn: 302401
2017-05-08 02:47:42 +00:00
Eric Beckmann efef15a0c7 Update llvm-readobj -coff-resources to display tree structure.
Summary: Continue making updates to llvm-readobj to display resource sections.  This is necessary for testing the up and coming cvtres tool.

Reviewers: zturner

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D32609

llvm-svn: 302399
2017-05-08 02:47:07 +00:00
Eric Beckmann 03de7c1501 Revert "Hopefully one last commit to fix this patch, addresses string reference"
Summary:
This reverts commit 56beec1b1cfc6d263e5eddb7efff06117c0724d2.

Revert "Quick fix to D32609, it seems .o files are not transferred in all cases."

This reverts commit 7652eecd29cfdeeab7f76f687586607a99ff4e36.

Revert "Update llvm-readobj -coff-resources to display tree structure."

This reverts commit 422b62c4d302cfc92401418c2acd165056081ed7.

Reviewers: zturner

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D32958

llvm-svn: 302397
2017-05-08 02:25:03 +00:00
Eric Beckmann 626680a09e Hopefully one last commit to fix this patch, addresses string reference
issues.

llvm-svn: 302395
2017-05-08 01:48:55 +00:00
Eric Beckmann 33fca46ec3 Update llvm-readobj -coff-resources to display tree structure.
Summary: Continue making updates to llvm-readobj to display resource sections.  This is necessary for testing the up and coming cvtres tool.

Reviewers: zturner

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D32609

llvm-svn: 302386
2017-05-07 22:47:22 +00:00
Zachary Turner 8c74673388 [CodeView] Reserve TypeDatabase records up front.
Most of the time we know exactly how many type records we
have in a list, and we want to use the visitor to deserialize
them into actual records in a database.  Previously we were
just using push_back() every time without reserving the space
up front in the vector.  This is obviously terrible from a
performance standpoint, and it's not uncommon to have PDB
files with half a million type records, where the performance
degredation was quite noticeable.

llvm-svn: 302302
2017-05-05 22:02:37 +00:00
Zachary Turner 2d5c2cd3ce [llvm-readobj] Update readobj to re-use parsing code.
llvm-readobj hand rolls some CodeView parsing code for string
tables, so this patch updates it to re-use some of the newly
introduced parsing code in LLVMDebugInfoCodeView.

Differential Revision: https://reviews.llvm.org/D32772

llvm-svn: 302052
2017-05-03 17:11:11 +00:00
Zachary Turner edef14510e [PDB/CodeView] Read/write codeview inlinee line information.
Previously we wrote line information and file checksum
information, but we did not write information about inlinee
lines and functions.  This patch adds support for that.

llvm-svn: 301936
2017-05-02 16:56:09 +00:00
Zachary Turner 7cc13e557c [PDB/CodeView] Rename some classes.
In preparation for introducing writing capabilities for each of
these classes, I would like to adopt a Foo / FooRef naming
convention, where Foo indicates that the class can manipulate and
serialize Foos, and FooRef indicates that it is an immutable view of
an existing Foo.  In other words, Foo is a writer and FooRef is a
reader.  This patch names some existing readers to conform to the
FooRef convention, while offering no functional change.

llvm-svn: 301810
2017-05-01 16:46:39 +00:00
Zachary Turner 5b6e4e0aed [llvm-pdbdump] Abstract some of the YAML/Raw printing code.
There is a lot of duplicate code for printing line info between
YAML and the raw output printer.  This introduces a base class
that can be shared between the two, and makes some minor
cleanups in the process.

llvm-svn: 301728
2017-04-29 01:13:21 +00:00
Zachary Turner 4346ae1a05 [llvm-readobj] Fix incorrect printing of CV column info.
llvm-svn: 301720
2017-04-29 00:03:32 +00:00
Zachary Turner 05bd9f3713 [llvm-readobj] Use LLVMDebugInfoCodeView to parse line tables.
The llvm-readobj parsing code currently exists in our CodeView
library, so we use that to parse instead of re-writing the logic
in the tool.

llvm-svn: 301718
2017-04-28 23:41:36 +00:00
Zachary Turner 8d6396d3b0 [llvm-readobj] Dump COFF Resources section.
This patch dumps the raw bytes of the .rsrc sections that
are present in COFF object and executable files.  Subsequent
patches will parse this information and dump in a more human
readable format.

Differential Revision: https://reviews.llvm.org/D32463
Patch By: Eric Beckmann

llvm-svn: 301578
2017-04-27 19:38:38 +00:00
Zachary Turner c37cb0c6a5 [CodeView] Isolate Debug Info Fragments into standalone classes.
Previously parsing of these were all grouped together into a
single master class that could parse any type of debug info
fragment.

With writing forthcoming, the complexity of each individual
fragment is enough to warrant them having their own classes so
that reading and writing of each fragment type can be grouped
together, but isolated from the code for reading and writing
other fragment types.

In doing so, I found a place where parsing code was duplicated
for the FileChecksums fragment, across llvm-readobj and the
CodeView library, and one of the implementations had a bug.
Now that the codepaths are merged, the bug is resolved.

Differential Revision: https://reviews.llvm.org/D32547

llvm-svn: 301557
2017-04-27 16:12:16 +00:00
Zachary Turner 67c5601404 Rename some PDB classes.
We have a lot of very similarly named classes related to
dealing with module debug info.  This patch has NFC, it just
renames some classes to be more descriptive (albeit slightly
more to type).  The mapping from old to new class names is as
follows:

   Old          |        New
ModInfo         | DbiModuleDescriptor
ModuleSubstream | ModuleDebugFragment
ModStream       | ModuleDebugStream

With the corresponding Builder classes renamed accordingly.

Differential Revision: https://reviews.llvm.org/D32506

llvm-svn: 301555
2017-04-27 16:11:19 +00:00
Reid Kleckner c4b5d794f1 [codeview] Cope with unsorted streams in type merging
Summary:
MASM can produce type streams that are not topologically sorted. It can
even produce type streams with circular references, but those are not
common in practice.

Reviewers: inglorion, ruiu

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D31629

llvm-svn: 299403
2017-04-03 23:58:15 +00:00
Shoaib Meenai 7375448893 [llvm-readobj] Prefer ILT to IAT for reading COFF imports
We're seeing binutils ld produce binaries where the import address
table's NameRVA entry is actually a VA instead (i.e. it's already base
relocated), which llvm-readobj then chokes on. Both dumpbin and the
Windows loader are able to handle these binaries correctly, however, and
we can make llvm-readobj handle them correctly too by iterating the
import lookup table (which doesn't have a relocated NameRVA) rather than
the import address table.

The import lookup table and the import address table are supposed to be
identical on disk, and prior to r277298 the import lookup table would be
used by `llvm-readobj -coff-imports` anyway, so this shouldn't have any
functional change (except in the case of our malformed binaries). The
import lookup table can apparently be missing when using old Borland
linkers, so fall back to the import address table in that case.

Resolves PR31766.

Differential Revision: https://reviews.llvm.org/D31362

llvm-svn: 298812
2017-03-26 17:10:11 +00:00
Reid Kleckner 5d57752c81 [PDB] Split item and type records when merging type streams
Summary: MSVC does this when producing a PDB.

Reviewers: ruiu

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D31316

llvm-svn: 298717
2017-03-24 17:26:38 +00:00
Zachary Turner d9dc2829ea [Support] Move Stream library from MSF -> Support.
After several smaller patches to get most of the core improvements
finished up, this patch is a straight move and header fixup of
the source.

Differential Revision: https://reviews.llvm.org/D30266

llvm-svn: 296810
2017-03-02 20:52:51 +00:00
Zachary Turner 695ed56ba5 [PDB] Make streams carry their own endianness.
Before the endianness was specified on each call to read
or write of the StreamReader / StreamWriter, but in practice
it's extremely rare for streams to have data encoded in
multiple different endiannesses, so we should optimize for the
99% use case.

This makes the code cleaner and more general, but otherwise
has NFC.

llvm-svn: 296415
2017-02-28 00:04:07 +00:00
Zachary Turner 120faca41b [PDB] Partial resubmit of r296215, which improved PDB Stream Library.
This was reverted because it was breaking some builds, and
because of incorrect error code usage.  Since the CL was
large and contained many different things, I'm resubmitting
it in pieces.

This portion is NFC, and consists of:

1) Renaming classes to follow a consistent naming convention.
2) Fixing the const-ness of the interface methods.
3) Adding detailed doxygen comments.
4) Fixing a few instances of passing `const BinaryStream& X`.  These
   are now passed as `BinaryStreamRef X`.

llvm-svn: 296394
2017-02-27 22:11:43 +00:00
NAKAMURA Takumi 05a75e40da Revert r296215, "[PDB] General improvements to Stream library." and followings.
r296215, "[PDB] General improvements to Stream library."
r296217, "Disable BinaryStreamTest.StreamReaderObject temporarily."
r296220, "Re-enable BinaryStreamTest.StreamReaderObject."
r296244, "[PDB] Disable some tests that are breaking bots."
r296249, "Add static_cast to silence -Wc++11-narrowing."

std::errc::no_buffer_space should be used for OS-oriented errors for socket transmission.
(Seek discussions around llvm/xray.)

I could substitute s/no_buffer_space/others/g, but I revert whole them ATM.

Could we define and use LLVM errors there?

llvm-svn: 296258
2017-02-25 17:04:23 +00:00
Zachary Turner af299ea5d4 [PDB] General improvements to Stream library.
This adds various new functionality and cleanup surrounding the
use of the Stream library.  Major changes include:

* Renaming of all classes for more consistency / meaningfulness
* Addition of some new methods for reading multiple values at once.
* Full suite of unit tests for reader / writer functionality.
* Full set of doxygen comments for all classes.
* Streams now store their own endianness.
* Fixed some bugs in a few of the classes that were discovered
  by the unit tests.

llvm-svn: 296215
2017-02-25 00:44:30 +00:00
Zachary Turner d2684b7969 [PDB] Rename Stream related source files.
This is part of a larger effort to get the Stream code moved
up to Support.  I don't want to do it in one large patch, in
part because the changes are so big that it will treat everything
as file deletions and add, losing history in the process.
Aside from that though, it's just a good idea in general to
make small changes.

So this change only changes the names of the Stream related
source files, and applies necessary source fix ups.

llvm-svn: 296211
2017-02-25 00:33:34 +00:00
Zachary Turner 181fe17b6f Don't assume little endian in StreamReader / StreamWriter.
In an effort to generalize this so it can be used by more than
just PDB code, we shouldn't assume little endian.

llvm-svn: 295525
2017-02-18 01:35:33 +00:00
Zachary Turner 7b327d051b [pdb] Add the ability to resolve TypeServer PDBs.
Some PDBs or object files can contain references to other PDBs
where the real type information lives.  When this happens,
all type indices in the original PDB are meaningless because
their records are not there.

With this patch we add the ability to pull type info from those
secondary PDBs.

Differential Revision: https://reviews.llvm.org/D29973

llvm-svn: 295382
2017-02-16 23:35:45 +00:00
Rui Ueyama a9b29615fb Re-submit r293820: Return Error instead of bool from mergeTypeStreams().
llvm-svn: 293847
2017-02-02 00:47:10 +00:00
Zachary Turner 629cb7d8cc [CodeView] Finish decoupling TypeDatabase from TypeDumper.
Previously the type dumper itself was passed around to a lot of different
places and manipulated in ways that were more appropriate on the type
database. For example, the entire TypeDumper was passed into the symbol
dumper, when all the symbol dumper wanted to do was lookup the name of a
TypeIndex so it could print it. That's what the TypeDatabase is for --
mapping type indices to names.

Another example is how if the user runs llvm-pdbdump with the option to
dump symbols but not types, we still have to visit all types so that we
can print minimal information about the type of a symbol, but just without
dumping full symbol records. The way we did this before is by hacking it
up so that we run everything through the type dumper with a null printer,
so that the output goes to /dev/null. But really, we don't need to dump
anything, all we want to do is build the type database. Since
TypeDatabaseVisitor now exists independently of TypeDumper, we can do
this. We just build a custom visitor callback pipeline that includes a
database visitor but not a dumper.

All the hackery around printers etc goes away. After this patch, we could
probably even delete the entire CVTypeDumper class since really all it is
at this point is a thin wrapper that hides the details of how to build a
useful visitation pipeline. It's not a priority though, so CVTypeDumper
remains for now.

After this patch we will be able to easily plug in a different style of
type dumper by only implementing the proper visitation methods to dump
one-line output and then sticking it on the pipeline.

Differential Revision: https://reviews.llvm.org/D28524

llvm-svn: 291724
2017-01-11 23:24:22 +00:00
Zachary Turner a9054ddd9c [CodeView/PDB] Rename a bunch of files.
We were starting to get some name clashes between llvm-pdbdump
and the common CodeView framework, so I took this opportunity
to rename a bunch of files to more accurately describe their
usage.  This also helps in llvm-pdbdump to distinguish
between different files and whether they are used for pretty
dump mode or raw dump mode.

llvm-svn: 291627
2017-01-11 00:35:43 +00:00
Zachary Turner 4efa0a4201 [CodeView] Hook up CodeViewRecordIO to type serialization path.
Previously support had been added for using CodeViewRecordIO
to read (deserialize) CodeView type records.  This patch adds
support for writing those same records.  With this patch,
reading and writing of CodeView type records finally uses a single
codepath.

Differential Revision: https://reviews.llvm.org/D26253

llvm-svn: 286304
2016-11-08 22:24:53 +00:00
Zachary Turner 4d49eb9fa0 [CodeView] Refactor serialization to use StreamInterface.
This was all using ArrayRef<>s before which presents a problem
when you want to serialize to or deserialize from an actual
PDB stream.  An ArrayRef<> is really just a special case of
what can be handled with StreamInterface though (e.g. by using
a ByteStream), so changing this to use StreamInterface allows
us to plug in a PDB stream and get all the record serialization
and deserialization for free on a MappedBlockStream.

Subsequent patches will try to remove TypeTableBuilder and
TypeRecordBuilder in favor of class that operate on
Streams as well, which should allow us to completely merge
the reading and writing codepaths for both types and symbols.

Differential Revision: https://reviews.llvm.org/D25831

llvm-svn: 284762
2016-10-20 18:31:19 +00:00
Zachary Turner 0d8407447d Refactor Symbol visitor code.
Type visitor code had already been refactored previously to
decouple the visitor and the visitor callback interface.  This
was necessary for having the flexibility to visit in different
ways (for example, dumping to yaml, reading from yaml, dumping
to ScopedPrinter, etc).

This patch merely implements the same visitation pattern for
symbol records that has already been implemented for type records.

llvm-svn: 283609
2016-10-07 21:34:46 +00:00
Sanjoy Das 23f06e53d8 [Stackmap] Added callsite counts to emitted function information.
Summary:
It was previously not possible for tools to use solely the stackmap
information emitted to reconstruct the return addresses of callsites in
the map, which is necessary to use the information to walk a stack. This
patch adds per-function callsite counts when emitting the stackmap
section in order to resolve the problem. Note that this slightly alters
the stackmap format, so external tools parsing these maps will need to
be updated.

**Problem Details:**
Records only store their offset from the beginning of the function they
belong to. While these records and the functions are output in program
order, it is not possible to determine where the end of one function's
records are without the callsite count when processing the records to
compute return addresses.

Patch by Kavon Farvardin!

Reviewers: atrick, ributzka, sanjoy

Subscribers: nemanjai

Differential Revision: https://reviews.llvm.org/D23487

llvm-svn: 281532
2016-09-14 20:22:03 +00:00
Saleem Abdulrasool 015280211b CodeView: extract the OMF Directory Header
The DebugDirectory contains a pointer to the CodeView info structure which is a
derivative of the OMF debug directory.  The structure has evolved a bit over
time, and PDB 2.0 used a slightly different definition from PDB 7.0.  Both of
these are specific to CodeView and not COFF.  Reflect this by moving the
structure definitions into the DebugInfo/CodeView headers.  Define a generic
DebugInfo union type that can be used to pass around a reference to the
DebugInfo irrespective of the versioning.  NFC.

llvm-svn: 278075
2016-08-09 00:25:12 +00:00
Zachary Turner a3225b0451 [msf] Resubmit "Rename Msf -> MSF".
Previously this change was submitted from a Windows machine, so
changes made to the case of filenames and directory names did
not survive the commit, and as a result the CMake source file
names and the on-disk file names did not match on case-sensitive
file systems.

I'm resubmitting this patch from a Linux system, which hopefully
allows the case changes to make it through unfettered.

llvm-svn: 277213
2016-07-29 20:56:36 +00:00
Zachary Turner 334aec4dd2 Revert "[msf] Rename Msf to MSF."
This reverts commit 4d1557ffac41e079bcb1abbcf04f512474dcd6fe.

llvm-svn: 277194
2016-07-29 18:38:47 +00:00
Zachary Turner a010f5cef0 [msf] Rename Msf to MSF.
In a previous patch, it was suggested to use all caps instead of
rolling caps for initialisms, so this patch changes everything
to do this.

llvm-svn: 277190
2016-07-29 18:24:26 +00:00
Zachary Turner d66889cbae [pdb] Refactor library to more clearly separate reading/writing
Reviewed By: amccarth, ruiu
Differential Revision: https://reviews.llvm.org/D22693

llvm-svn: 277019
2016-07-28 19:12:28 +00:00
Zachary Turner bac69d33d0 [msf] Create LLVMDebugInfoMsf
This provides a better layering of responsibilities among different
aspects of PDB writing code.  Some of the MSF related code was
contained in CodeView, and some was in PDB prior to this.  Further,
we were often saying PDB when we meant MSF, and the two are
actually independent of each other since in theory you can have
other types of data besides PDB data in an MSF.  So, this patch
separates the MSF specific code into its own library, with no
dependencies on anything else, and DebugInfoCodeView and
DebugInfoPDB take dependencies on DebugInfoMsf.

llvm-svn: 276458
2016-07-22 19:56:05 +00:00
Rafael Espindola 48975881ab Delete some dead code.
Found by gcc 6.

llvm-svn: 273303
2016-06-21 19:48:12 +00:00
Zachary Turner 01ee3dae04 Resubmit "[pdb] Change type visitor pattern to be dynamic."
There was a regression introduced during type stream merging when
visiting a field list record.  This has been fixed in this patch.

llvm-svn: 272929
2016-06-16 18:22:27 +00:00
Zachary Turner 73b0b2f555 Revert "[pdb] Change type visitor pattern to be dynamic."
This reverts commit fb0dd311e1ad945827b8ffd5354f4810e2be1579.

This breaks some llvm-readobj tests.

llvm-svn: 272927
2016-06-16 18:09:04 +00:00
Zachary Turner 1f6372c429 [pdb] Change type visitor pattern to be dynamic.
This allows better catching of compiler errors since we can use
the override keyword to verify that methods are actually
overridden.

Also in this patch I've changed from storing a boolean Error
code everywhere to returning an llvm::Error, to propagate richer
error information up the call stack.

Reviewed By: ruiu, rnk
Differential Revision: http://reviews.llvm.org/D21410

llvm-svn: 272926
2016-06-16 18:00:28 +00:00
Zachary Turner 5acb4ac6d7 Add support for writing through StreamInterface.
This adds method and tests for writing to a PDB stream.  With
this, even a PDB stream which is discontiguous can be treated
as a sequential stream of bytes for the purposes of writing.

Reviewed By: ruiu
Differential Revision: http://reviews.llvm.org/D21157

llvm-svn: 272369
2016-06-10 05:09:12 +00:00
Zachary Turner a96cce64a5 [codeview] Dump line number and column information.
To facilitate this, a couple of changes had to be made:

1. `ModuleSubstream` got moved from `DebugInfo/PDB` to
`DebugInfo/CodeView`, and various codeview related types are defined
there.  It turns out `DebugInfo/CodeView/Line.h` already defines many of
these structures, but this is really old code that is not endian aware,
doesn't interact well with `StreamInterface` and not very helpful for
getting stuff out of a PDB.  Eventually we should migrate the old readobj
`COFFDumper` code to these new structures, or at least merge their
functionality somehow.

2. A `ModuleSubstream` visitor is introduced.  Depending on where your
module substream array comes from, different subsets of record types can
be expected.  We are already hand parsing these substream arrays in many
places especially in `COFFDumper.cpp`.  In the future we can migrate these
paths to the visitor as well, which should reduce a lot of code in
`COFFDumper.cpp`.

Differential Revision: http://reviews.llvm.org/D20936
Reviewed By: ruiu, majnemer

llvm-svn: 271621
2016-06-03 03:25:59 +00:00
David Majnemer a2a2a73c90 Rename IMAGE_DEBUG_TYPE_NO_TIMESTAMP to IMAGE_DEBUG_TYPE_REPRO
This matches the COFF spec

llvm-svn: 271549
2016-06-02 17:32:11 +00:00
Reid Kleckner 2da433ea99 [COFF] Expose the PE debug data directory and dump it
This directory is used to find if there is a PDB associated with an
executable. I plan to use this functionality to teach llvm-symbolizer
whether it should use DIA or DWARF to symbolize a given DLL.

Reviewers: majnemer

Differential Revision: http://reviews.llvm.org/D20885

llvm-svn: 271539
2016-06-02 17:10:43 +00:00
Reid Kleckner fbdbe9e22b [codeview] Improve readability of type record assembly
Adds the method MCStreamer::EmitBinaryData, which is usually an alias
for EmitBytes. In the MCAsmStreamer case, it is overridden to emit hex
dump output like this:
        .byte   0x0e, 0x00, 0x08, 0x10
        .byte   0x03, 0x00, 0x00, 0x00
        .byte   0x00, 0x00, 0x00, 0x00
        .byte   0x00, 0x10, 0x00, 0x00

Also, when verbose asm comments are enabled, this patch prints the dump
output for each comment before its record, like this:
        # ArgList (0x1000) {
        #   TypeLeafKind: LF_ARGLIST (0x1201)
        #   NumArgs: 0
        #   Arguments [
        #   ]
        # }
        .byte   0x06, 0x00, 0x01, 0x12
        .byte   0x00, 0x00, 0x00, 0x00

This should make debugging easier and testing more convenient.

Reviewers: aaboud

Subscribers: majnemer, zturner, amccarth, aaboud, llvm-commits

Differential Revision: http://reviews.llvm.org/D20711

llvm-svn: 271313
2016-05-31 18:45:36 +00:00
Reid Kleckner 3b3f490f9c [codeview] Add a CVTypeDumper::dump(ArrayRef<uint8_t>) overload
This is a convenient wrapper when the type record is already laid out as
bytes in memory.

llvm-svn: 271309
2016-05-31 18:15:23 +00:00
David Majnemer 61f179815a Don't dereference a symbol iterator before checking for the end case
llvm-svn: 271173
2016-05-29 06:18:08 +00:00
David Majnemer cab2b461ef [COFFDumper] Validate that the next offset is not too large
llvm-svn: 271147
2016-05-28 20:04:50 +00:00
David Majnemer efaaf410a6 [COFFDumper] Make sure there is sufficient padding left in the string table
llvm-svn: 271146
2016-05-28 20:04:48 +00:00
David Majnemer 37dafba713 [COFFDumper] Make sure there is sufficient padding left in the checksum
llvm-svn: 271143
2016-05-28 19:45:58 +00:00
David Majnemer a6d93fd73b [llvm-readobj] Validate the string table offset before using it
llvm-svn: 271139
2016-05-28 19:45:49 +00:00
David Majnemer 78b0d72769 Use consume instead of manually using drop_front
llvm-svn: 271137
2016-05-28 19:17:48 +00:00
Zachary Turner 0d43c1c339 [pdb] Finish conversion to zero copy pdb access.
This converts remaining uses of ByteStream, which was still
left in the symbol stream and type stream, to using the new
StreamInterface zero-copy classes.

RecordIterator is finally deleted, so this is the only way left
now.  Additionally, more error checking is added when iterating
the various streams.

With this, the transition to zero copy pdb access is complete.

llvm-svn: 271101
2016-05-28 05:21:57 +00:00
Zachary Turner dc9ca0f1ba Remove unused global variable.
llvm-svn: 270610
2016-05-24 20:48:18 +00:00
Zachary Turner 537014ce15 Remove unused variable breaking -Werror builds.
llvm-svn: 270529
2016-05-24 03:32:34 +00:00
Zachary Turner aaad57440d Make a symbol visitor and use it to dump CV symbols.
Differential Revision: http://reviews.llvm.org/D20534
Reviewed By: rnk

llvm-svn: 270511
2016-05-23 23:41:13 +00:00
Zachary Turner a78ecd1e6c [codeview] Refactor symbol records to use same pattern as types.
This will pave the way to introduce a full fledged symbol visitor
similar to how we have a type visitor, thus allowing the same
dumping code to be used in llvm-readobj and llvm-pdbdump.

Differential Revision: http://reviews.llvm.org/D20384
Reviewed By: rnk

llvm-svn: 270475
2016-05-23 18:49:06 +00:00
Rafael Espindola 010148a834 Delete default in fully covered switch.
llvm-svn: 269872
2016-05-18 00:31:06 +00:00
Zachary Turner 63a2846e84 [codeview] Some cleanup of Symbol Records.
* Reworks the CVSymbolTypes.def to work similarly to TypeRecords.def.
* Moves some enums from SymbolRecords.h to CodeView.h to maintain
  consistency with how we do type records.
* Generalize a few simple things like the record prefix
* Define the leaf enum and the kind enum similar to how we do with tyep
  records.

Differential Revision: http://reviews.llvm.org/D20342
Reviewed By: amccarth, rnk

llvm-svn: 269867
2016-05-17 23:50:21 +00:00
Reid Kleckner 0b269748a6 [codeview] Add type stream merging prototype
Summary:
This code is intended to be used as part of LLD's PDB writing. Until
that exists, this is exposed via llvm-readobj for testing purposes.

Type stream merging uses the following algorithm:

- Begin with a new empty stream, and a new empty hash table that maps
  from type record contents to new type index.
- For each new type stream, maintain a map from source type index to
  destination type index.
- For each record, copy it and rewrite its type indices to be valid in
  the destination type stream.
- If the new type record is not already present in the destination
  stream hash table, append it to the destination type stream, assign it
  the next type index, and update the two hash tables.
- If the type record already exists in the destination stream, discard
  it and update the type index map to forward the source type index to
  the existing destination type index.

Reviewers: zturner, ruiu

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D20122

llvm-svn: 269521
2016-05-14 00:02:53 +00:00
Zachary Turner 38cc8b3f21 Make CodeView record serialization more generic.
This introduces a variadic template and some helper macros to
safely and correctly deserialize many types of common record
fields while maintaining error checking.

Differential Revision: http://reviews.llvm.org/D20183
Reviewed By: rnk, amccarth

llvm-svn: 269315
2016-05-12 17:45:44 +00:00
Reid Kleckner 4a14bcac41 [codeview] Move dumper into lib/DebugInfo/CodeView
So that we can call it from llvm-pdbdump.

llvm-svn: 268580
2016-05-05 00:34:33 +00:00
Reid Kleckner 7960de99db [codeview] Add a type visitor to help abstract away type stream handling
Summary:
Port the dumper in llvm-readobj over to it.

I'm planning to use this visitor to power type stream merging.

While we're at it, try to switch from StringRef to ArrayRef<uint8_t> in some
places.

Reviewers: zturner, amccarth

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D19899

llvm-svn: 268535
2016-05-04 19:39:28 +00:00
Zachary Turner 2d02ceefdc Move CodeViewTypeStream to DebugInfo/CodeView
Ability to parse codeview type streams is also needed by
DebugInfoPDB for parsing PDBs, so moving this into a library
gives us this option.  Since DebugInfoPDB had already hand
rolled some code to do this, that code is now convereted over
to using this common abstraction.

Differential Revision: http://reviews.llvm.org/D19887
Reviewed By: dblaikie, amccarth

llvm-svn: 268454
2016-05-03 22:18:17 +00:00
Reid Kleckner bc041b82d4 [codeview] Maintain the type enum-to-classname mapping in the .def file
This way it will be easy to stamp out something like a type visitor.

llvm-svn: 268347
2016-05-03 00:45:14 +00:00
Zachary Turner 88bb163f81 Move llvm-readobj/StreamWriter to Support.
We wish to re-use this from llvm-pdbdump, and it provides a nice
way to print structured data in scoped format that could prove
useful for many other dumping tools as well.  Moving to support
and changing name to ScopedPrinter to better reflect its purpose.

llvm-svn: 268342
2016-05-03 00:28:04 +00:00
Adrian McCarthy 963a331a7e NFC: An iterator for stepping through CodeView type stream in llvm-readobj
This is a small refactoring step toward moving CodeView type stream logic from llvm-readobj to a library. It abstracts the logic of stepping through the stream into an iterator class and updates llvm-readobj to use that iterator. This has no functional change; llvm-readobj produces identical output.

The next step is to abstract the parsing of the different leaf types and then move that and the iterator into a library.

Since this is my first contrib outside LLDB, please let me know if I'm messing up on any of the LLVM style guidelines, idioms, or patterns.

Differential Revision: http://reviews.llvm.org/D19746

llvm-svn: 268334
2016-05-02 23:45:03 +00:00
Reid Kleckner 5cb2b6b1be [codeview] Isolate type dumping from object file state
This isolates the state we use for type dumping from the knowledge of
object files. We can use CVTypeDumper to dump types from anywhere in
memory now.

NFC

Reviewers: zturner

Differential Revision: http://reviews.llvm.org/D19824

llvm-svn: 268300
2016-05-02 20:30:47 +00:00
Reid Kleckner b2bd28128d [codeview] Don't dump type stream bytes unless asked to
llvm-svn: 268271
2016-05-02 18:10:00 +00:00
Kevin Enderby 81e8b7d949 Thread Expected<...> up from libObject’s getName() for symbols to allow llvm-objdump to produce a good error message.
Produce another specific error message for a malformed Mach-O file when a symbol’s
string index is past the end of the string table.  The existing test case in test/Object/macho-invalid.test
for macho-invalid-symbol-name-past-eof now reports the error with the message indicating
that a symbol at a specific index has a bad sting index and that bad string index value.
 
Again converting interfaces to Expected<> from ErrorOr<> does involve
touching a number of places. Where the existing code reported the error with a
string message or an error code it was converted to do the same.  There is some
code for this that could be factored into a routine but I would like to leave that for
the code owners post-commit to do as they want for handling an llvm::Error.  An
example of how this could be done is shown in the diff in
lib/ExecutionEngine/RuntimeDyld/RuntimeDyldImpl.h which had a Check() routine
already for std::error_code so I added one like it for llvm::Error .

Also there some were bugs in the existing code that did not deal with the
old ErrorOr<> return values.  So now with Expected<> since they must be
checked and the error handled, I added a TODO and a comment:
“// TODO: Actually report errors helpfully” and a call something like
consumeError(NameOrErr.takeError()) so the buggy code will not crash
since needed to deal with the Error.

Note there fixes needed to lld that goes along with this that I will commit right after this.
So expect lld not to built after this commit and before the next one.

llvm-svn: 266919
2016-04-20 21:24:34 +00:00
Reid Kleckner f9f1c5619a [codeview] Dump char16_t and char32_t simple types
llvm-svn: 266465
2016-04-15 18:26:45 +00:00
Duncan P. N. Exon Smith 91d3cfed78 Revert "Fix Clang-tidy modernize-deprecated-headers warnings in remaining files; other minor fixes."
This reverts commit r265454 since it broke the build.  E.g.:

  http://lab.llvm.org:8080/green/job/clang-stage1-cmake-RA-incremental_build/22413/

llvm-svn: 265459
2016-04-05 20:45:04 +00:00
Eugene Zelenko 1760dc2a23 Fix Clang-tidy modernize-deprecated-headers warnings in remaining files; other minor fixes.
Some Include What You Use suggestions were used too.

Use anonymous namespaces in source files.

Differential revision: http://reviews.llvm.org/D18778

llvm-svn: 265454
2016-04-05 20:19:49 +00:00
Reid Kleckner 4ab992d1b4 [codeview] Dump def range lengths in hex
It makes it easier to correlate with assembly dumps, which are typically
given with hex offsets.

llvm-svn: 260619
2016-02-11 23:40:14 +00:00
Reid Kleckner 98762d2429 [codeview] Dump a missing field and change its signedness
llvm-svn: 259904
2016-02-05 19:15:45 +00:00
David Majnemer a4859dfa46 Correctly handle {Always,Never}StepIntoLine
llvm-svn: 259806
2016-02-04 17:57:12 +00:00
David Majnemer 4d123512a2 Add support for S_DEFRANGE and S_DEFRANGE_SUBFIELD
llvm-svn: 259805
2016-02-04 17:37:30 +00:00
David Majnemer 6f01e05d7e Make the dumper's output for variable ranges easier to read
llvm-svn: 259804
2016-02-04 17:29:13 +00:00
David Majnemer d74490f2cc Print the OffsetStart field's relocation
llvm-svn: 259723
2016-02-03 22:45:21 +00:00
David Majnemer ac10cfde97 [llvm-readobj] Add support for dumping S_DEFRANGE symbols
llvm-svn: 259719
2016-02-03 22:36:46 +00:00
Reid Kleckner 1fcd610c94 [codeview] Wire up the .cv_inline_linetable directive
This directive emits the binary annotations that describe line and code
deltas in inlined call sites. Single-stepping through inlined frames in
windbg now works.

llvm-svn: 259535
2016-02-02 17:41:18 +00:00
Reid Kleckner 828883b86c [CodeView] Fix dumping the is_stmt bit from the line table
Bug pointed out by George Rimar.

llvm-svn: 259205
2016-01-29 16:39:04 +00:00
Reid Kleckner 2214ed8937 Reland "[CodeView] Use assembler directives for line tables"
This reverts commit r259126 and relands r259117.

This time with updated library dependencies.

llvm-svn: 259130
2016-01-29 00:49:42 +00:00
Reid Kleckner 00d9639c24 Revert "[CodeView] Use assembler directives for line tables"
This reverts commit r259117.

The LineInfo constructor is defined in the codeview library and we have
to link against it now. Doing that isn't trivial, so reverting for now.

llvm-svn: 259126
2016-01-29 00:13:28 +00:00
Reid Kleckner c62e379d22 [CodeView] Use assembler directives for line tables
Adds a new family of .cv_* directives to LLVM's variant of GAS syntax:

- .cv_file: Similar to DWARF .file directives

- .cv_loc: Similar to the DWARF .loc directive, but starts with a
  function id. CodeView line tables are emitted by function instead of
  by compilation unit, so we needed an extra field to communicate this.
  Rather than overloading the .loc direction further, we decided it was
  better to have our own directive.

- .cv_stringtable: Emits the codeview string table at the current
  position. Currently this just contains the filenames as
  null-terminated strings.

- .cv_filechecksums: Emits the file checksum table for all files used
  with .cv_file so far. There is currently no support for emitting
  actual checksums, just filenames.

This moves the line table emission code down into the assembler.  This
is in preparation for implementing the inlined call site line table
format. The inline line table format encoding algorithm requires knowing
the absolute code offsets, so it must run after the assembler has laid
out the code.

David Majnemer collaborated on this patch.

llvm-svn: 259117
2016-01-28 23:31:52 +00:00
Reid Kleckner 400f39308c [readobj] Print CodeOffset first, it's easier to read
llvm-svn: 258368
2016-01-20 23:21:14 +00:00
Rafael Espindola a79078c3ce Use new function name. NFC.
llvm-svn: 258079
2016-01-18 20:55:24 +00:00
Reid Kleckner af49ecb286 [codeview] Handle multiple frame descriptions in readobj
llvm-svn: 257943
2016-01-15 22:09:13 +00:00
Reid Kleckner c31f530cb7 [codeview] Dump the file checksum substream
llvm-svn: 257910
2016-01-15 18:06:25 +00:00
Reid Kleckner 8e7275cc73 [codeview] Translate file table offsets to filenames and print them
llvm-svn: 257846
2016-01-15 00:11:21 +00:00
Reid Kleckner 43842b554c [codeview] Dump function callees and add more labels to inlinee info
I kept forgetting which number is the line delta and which is the code
delta.

llvm-svn: 257813
2016-01-14 21:50:05 +00:00
Rui Ueyama da00f2fdf4 Update to use new name alignTo().
llvm-svn: 257804
2016-01-14 21:06:47 +00:00