Commit Graph

551 Commits

Author SHA1 Message Date
Zachary Turner 93839cb4ac [pdb] Parse and dump section map and section contribs
Differential Revision: http://reviews.llvm.org/D20876
Reviewed By: rnk, ruiu

llvm-svn: 271488
2016-06-02 05:07:49 +00:00
David Majnemer a7c29321be [PDB] Make ModStream::symbols report errors
llvm-svn: 271417
2016-06-01 18:13:04 +00:00
Zachary Turner 90b8b8db2e [pdb] Add unit tests for PDB MappedBlockStream and zero copy
Differential Revision: http://reviews.llvm.org/D20837
Reviewed By: ruiu

llvm-svn: 271346
2016-05-31 22:41:52 +00:00
Kevin Enderby 9acb109930 Change llvm-objdump, llvm-nm and llvm-size when reporting an object file error
when the object is from a slice of a Mach-O Universal Binary use something like
"foo.o (for architecture i386)" as part of the error message when expected.

Also fixed places in these tools that were ignoring object file errors from
MachOUniversalBinary::getAsObjectFile() when the code moved on to see if
the slice was an archive.

To do this MachOUniversalBinary::getAsObjectFile() and
MachOUniversalBinary::getObjectForArch() were changed from returning
ErrorOr<...> to Expected<...> then that was threaded up to its users.

Converting these interfaces to Expected<> from ErrorOr<> does involve
touching a number of places. To contain the changes for now the use of
errorToErrorCode() is still used in two places yet to be fully converted.

llvm-svn: 271332
2016-05-31 20:35:34 +00:00
Reid Kleckner fbdbe9e22b [codeview] Improve readability of type record assembly
Adds the method MCStreamer::EmitBinaryData, which is usually an alias
for EmitBytes. In the MCAsmStreamer case, it is overridden to emit hex
dump output like this:
        .byte   0x0e, 0x00, 0x08, 0x10
        .byte   0x03, 0x00, 0x00, 0x00
        .byte   0x00, 0x00, 0x00, 0x00
        .byte   0x00, 0x10, 0x00, 0x00

Also, when verbose asm comments are enabled, this patch prints the dump
output for each comment before its record, like this:
        # ArgList (0x1000) {
        #   TypeLeafKind: LF_ARGLIST (0x1201)
        #   NumArgs: 0
        #   Arguments [
        #   ]
        # }
        .byte   0x06, 0x00, 0x01, 0x12
        .byte   0x00, 0x00, 0x00, 0x00

This should make debugging easier and testing more convenient.

Reviewers: aaboud

Subscribers: majnemer, zturner, amccarth, aaboud, llvm-commits

Differential Revision: http://reviews.llvm.org/D20711

llvm-svn: 271313
2016-05-31 18:45:36 +00:00
Reid Kleckner 3b3f490f9c [codeview] Add a CVTypeDumper::dump(ArrayRef<uint8_t>) overload
This is a convenient wrapper when the type record is already laid out as
bytes in memory.

llvm-svn: 271309
2016-05-31 18:15:23 +00:00
David Majnemer ba1439229a Make sure we don't add an empty string to the stringmap
llvm-svn: 271172
2016-05-29 06:18:06 +00:00
David Majnemer c6cb2ec36e [SymbolDumper] Validate the string table offset before using it
llvm-svn: 271145
2016-05-28 20:04:46 +00:00
David Majnemer b343310b4f [SymbolDumper] Validate the string table offset before using it
llvm-svn: 271142
2016-05-28 19:45:56 +00:00
David Majnemer 328b6d3903 Tighten some of the name map checks further
llvm-svn: 271130
2016-05-28 18:03:37 +00:00
David Majnemer 869631f987 Bounds check the number of bitmap blocks in the name map
llvm-svn: 271105
2016-05-28 05:59:25 +00:00
David Majnemer 7e950b261a Make sure the directory contains info for all streams
llvm-svn: 271103
2016-05-28 05:59:19 +00:00
Zachary Turner 0d43c1c339 [pdb] Finish conversion to zero copy pdb access.
This converts remaining uses of ByteStream, which was still
left in the symbol stream and type stream, to using the new
StreamInterface zero-copy classes.

RecordIterator is finally deleted, so this is the only way left
now.  Additionally, more error checking is added when iterating
the various streams.

With this, the transition to zero copy pdb access is complete.

llvm-svn: 271101
2016-05-28 05:21:57 +00:00
David Majnemer 74b1fb00f7 Don't discard errors
llvm-svn: 271056
2016-05-27 22:07:50 +00:00
Zachary Turner 7dd42598be [pdb] Fix size check when reading stream bytes.
We were accidentally bounds checking the read against the output
ArrayRef instead of against the size of the read.

llvm-svn: 271040
2016-05-27 20:17:33 +00:00
David Majnemer 6c13db402f Make sure data is available before dereferencing it
llvm-svn: 271032
2016-05-27 18:50:02 +00:00
Zachary Turner 1de49c9ffd Resubmit "[pdb] Allow zero-copy read support for symbol streams.""
Due to differences in template instantiation rules, it is not
portable to static_assert(false) inside of an invalid specialization
of a template.  Instead I just =delete the method so that it can't
be used, and leave a comment that it must be explicitly specialized.

llvm-svn: 271027
2016-05-27 18:47:20 +00:00
Chad Rosier 6c247c8cc8 Revert "[pdb] Allow zero-copy read support for symbol streams."
This reverts commit r271024 due to error: static_assert failed
"You must either provide a specialization of VarStreamArrayExtractor
or a custom extractor"

llvm-svn: 271026
2016-05-27 18:31:02 +00:00
Zachary Turner 3a9a23ae62 [pdb] Allow zero-copy read support for symbol streams.
This reduces the amount of memory used by llvm-pdbdump by roughly
1/3 of the size of the PDB file.

Differential Revision: http://reviews.llvm.org/D20724
Reviewed By: ruiu

llvm-svn: 271025
2016-05-27 18:20:20 +00:00
David Majnemer 836937ed79 Make sure these error codes are marked as checked
llvm-svn: 271013
2016-05-27 16:16:56 +00:00
David Majnemer 9efba74778 Make sure there are enough blocks for the stream
llvm-svn: 271012
2016-05-27 16:16:48 +00:00
David Majnemer 5d842ea68e Make sure the directory block array fits in the file
llvm-svn: 271011
2016-05-27 16:16:45 +00:00
David Majnemer 878cadb663 Validate the blocksize before using it
The blocksize could be zero on disk causing later checks to divide by
zero.

llvm-svn: 271008
2016-05-27 15:57:38 +00:00
Benjamin Kramer 82de7d323d Apply clang-tidy's misc-move-constructor-init throughout LLVM.
No functionality change intended, maybe a tiny performance improvement.

llvm-svn: 270997
2016-05-27 14:27:24 +00:00
Zachary Turner b393d95359 [codeview] Remove StreamReader copying method.
Since we want to move toward zero-copy access to stream data, we
want to remove all instances of copying operations.  So get rid
of some of those here.

Differential Revision: http://reviews.llvm.org/D20720
Reviewed By: ruiu

llvm-svn: 270960
2016-05-27 03:51:53 +00:00
Zachary Turner 8dbe3629a0 [codeview,pdb] Try really hard to conserve memory when reading.
PDBs can be extremely large.  We're already mapping the entire
PDB into the process's address space, but to make matters worse
the blocks of the PDB are not arranged contiguously.  So, when
we have something like an array or a string embedded into the
stream, we have to make a copy.  Since it's convenient to use
traditional data structures to iterate and manipulate these
records, we need the memory to be contiguous.

As a result of this, we were using roughly twice as much memory
as the file size of the PDB, because every stream was copied
out and re-stitched together contiguously.

This patch addresses this by improving the MappedBlockStream
to allocate from a BumpPtrAllocator only when a read requires
a discontiguous read.  Furthermore, it introduces some data
structures backed by a stream which can iterate over both
fixed and variable length records of a PDB.  Since everything
is backed by a stream and not a buffer, we can read almost
everything from the PDB with zero copies.

Differential Revision: http://reviews.llvm.org/D20654
Reviewed By: ruiu

llvm-svn: 270951
2016-05-27 01:54:44 +00:00
Zachary Turner d5d37dcf83 [codeview] Move StreamInterface and StreamReader to libcodeview.
We have need to reuse this functionality, including making
additional generic stream types that are smarter about how and
when they copy memory versus referencing the original memory.
So all of these structures belong in the common library
rather than being pdb specific.

llvm-svn: 270751
2016-05-25 20:37:03 +00:00
Zachary Turner d3076ab36f [llvm-pdbdump] Decipher the remaining PDB streams.
We know at least know the meaning of every stream of the
PDB file.  Yay!

llvm-svn: 270669
2016-05-25 05:49:48 +00:00
Zachary Turner c9972c64f5 [llvm-pdbdump] Dump the IPI stream and all records.
llvm-svn: 270661
2016-05-25 04:35:22 +00:00
Rui Ueyama b12b158f20 pdbdump: fix bug in name hash table.
name_ids() did not return all IDs but only the first NameCount items.
The number of non-zero entries in IDs vector is NameCount, but it
does not mean that all non-zero entries are at the beginning of IDs
vector.

Differential Revision: http://reviews.llvm.org/D20611

llvm-svn: 270656
2016-05-25 04:07:17 +00:00
Zachary Turner c59261ca37 [llvm-pdbdump] Stream 0 isn't actually the MSF superblock.
Oddly enough, I realized we don't actually know what stream
0 is (if anything).

llvm-svn: 270655
2016-05-25 03:53:16 +00:00
Zachary Turner 85ed80b9e6 [llvm-pdbdump] Dump stream summary list.
Try to figure out what each stream is, and dump its name.

This gives us a better picture of what streams we still don't
understand.

llvm-svn: 270653
2016-05-25 03:43:17 +00:00
Zachary Turner 172d59c105 [codeview] Add support for new types and symbols.
This patch adds support for:

S_EXPORT
LF_BITFIELD

With this patch, I have run through a couple of gigabytes of PDB
files and cannot find a type or symbol that we do not understand.

llvm-svn: 270637
2016-05-25 00:12:48 +00:00
Zachary Turner 9f054d424f [codeview] Add support for S_EXPORT symbol.
llvm-svn: 270636
2016-05-25 00:12:40 +00:00
Zachary Turner 4caa1bf0bd [codeview] Add support for new type records.
This adds support for parsing and dumping the following
symbol types:

S_LPROCREF
S_ENVBLOCK
S_COMPILE2
S_REGISTER
S_COFFGROUP
S_SECTION
S_THUNK32
S_TRAMPOLINE

As of this patch, the test PDB files no longer have any unknown
symbol types.

llvm-svn: 270628
2016-05-24 22:58:46 +00:00
Zachary Turner 96e60f7573 [llvm-pdbdump] Rework command line options.
When dumping huge PDB files, too many of the options were grouped
together so you would get neverending spew of output.  This patch
introduces more granular display options so you can only dump the
fields you actually care about.

llvm-svn: 270607
2016-05-24 20:31:48 +00:00
Peter Collingbourne 4718f8b5f1 Add FIXMEs to all derived classes of std::error_category.
This helps make clear that we're moving away from std::error_code.

Differential Revision: http://reviews.llvm.org/D20592

llvm-svn: 270604
2016-05-24 20:13:46 +00:00
Zachary Turner 9e33e6f89b [codeview, pdb] Dump symbol records in publics stream
Differential Revision: http://reviews.llvm.org/D20580
Reviewed By: ruiu

llvm-svn: 270597
2016-05-24 18:55:14 +00:00
Zachary Turner 00d847b19e Fix build errors
llvm-svn: 270587
2016-05-24 17:44:29 +00:00
Zachary Turner cac29ae038 Dump symbol record details in llvm-pdbdump
This makes use of the newly introduced `CVSymbolVisitor` to dump details
of each type of symbol record in the symbol streams.  Future patches will
bring this visitor based dumping to the publics stream, as well as
creating a `SymbolDumpDelegate` to print more information about
relocations etc.

Differential Revision: http://reviews.llvm.org/D20545
Reviewed By: ruiu

llvm-svn: 270585
2016-05-24 17:30:25 +00:00
George Rimar 401e4e570e Recommit r270547 ([llvm-dwarfdump] - Teach dwarfdump to decompress debug sections in zlib style.)
Fix was:
1) Had to regenerate dwarfdump-test-zlib.elf-x86-64, dwarfdump-test-zlib-gnu.elf-x86-64
(because llvm-symbolizer-zlib.test uses that inputs for its purposes and failed).
2) Updated llvm-symbolizer-zlib.test (updated used call function address to match new files + 
added one more check for newly created dwarfdump-test-zlib-gnu.elf-x86-64 binary input).
3) Updated comment in dwarfdump-test-zlib.cc.

Original commit message:
[llvm-dwarfdump] - Teach dwarfdump to decompress debug sections in zlib style.

Before this llvm-dwarfdump only recognized zlib-gnu compression style of headers,
this patch adds support for zlib style. 
It looks reasonable to support both styles for dumping,
even if we are not going to suport generating of deprecated gnu one.

Differential revision: http://reviews.llvm.org/D20470

llvm-svn: 270557
2016-05-24 12:48:46 +00:00
George Rimar f059dd4f76 Revert r270543 ("Recommit r270540")
Failed build bot in another test.
I am sorry for noise.
http://lab.llvm.org:8080/green/job/clang-stage1-cmake-RA-incremental_check/23679/testReport/junit/LLVM/DebugInfo/llvm_symbolizer_zlib_test/

llvm-svn: 270547
2016-05-24 11:03:10 +00:00
George Rimar e9b2e19109 Recommit r270540
fix: forgot to commit the updated dwarfdump-test-zlib.elf-x86-64

Original commit message:
[llvm-dwarfdump] - Teach dwarfdump to decompress debug sections in zlib style.

Before this llvm-dwarfdump only recognized zlib-gnu compression style of headers,
this patch adds support for zlib style. 
It looks reasonable to support both styles for dumping,
even if we are not going to suport generating of deprecated gnu one.

Differential revision: http://reviews.llvm.org/D20470

llvm-svn: 270543
2016-05-24 10:46:43 +00:00
George Rimar 6a6185fd78 Revert r270540 "[llvm-dwarfdump] - Teach dwarfdump to decompress debug sections in zlib style."
it broked bot:
http://lab.llvm.org:8011/builders/clang-s390x-linux/builds/5036

llvm-svn: 270541
2016-05-24 09:44:44 +00:00
George Rimar 6bcbf4c572 [llvm-dwarfdump] - Teach dwarfdump to decompress debug sections in zlib style.
Before this llvm-dwarfdump only recognized zlib-gnu compression style of headers,
this patch adds support for zlib style. 
It looks reasonable to support both styles for dumping,
even if we are not going to suport generating of deprecated gnu one.

Differential revision: http://reviews.llvm.org/D20470

llvm-svn: 270540
2016-05-24 09:28:36 +00:00
Zachary Turner 3e78e2d43f Remove unused variable.
llvm-svn: 270516
2016-05-24 00:06:04 +00:00
Zachary Turner aaad57440d Make a symbol visitor and use it to dump CV symbols.
Differential Revision: http://reviews.llvm.org/D20534
Reviewed By: rnk

llvm-svn: 270511
2016-05-23 23:41:13 +00:00
Rui Ueyama 2a58779198 Fix struct member names and simplify. NFC.
llvm-svn: 270289
2016-05-20 22:59:05 +00:00
Rui Ueyama 0fcd82605e pdbdump: print out symbol names referred by publics stream.
DBI stream contains a stream number of the symbol record stream.
Symbol record streams is an array of length-type-value members.
Each member represents one symbol.

Publics stream contains offsets to the symbol record stream.
This patch is to print out all symbols that are referenced by
the publics stream.

Note that even with this patch, llvm-pdbdump cannot dump all the
information in a publics stream since it contains more information
than symbol names. I'll improve it in followup patches.

Differential Revision: http://reviews.llvm.org/D20480

llvm-svn: 270262
2016-05-20 19:55:17 +00:00
Reid Kleckner e1587bce96 Fix -Wmicrosoft-enum-value warning
llvm-svn: 270110
2016-05-19 20:20:22 +00:00