Commit Graph

51 Commits

Author SHA1 Message Date
Teresa Johnson e1164de5d0 Restore "[ThinLTO] Use MD5 hash in function index." with fix
This restores commit r260408, along with a fix for a bot failure.

The bot failure was caused by dereferencing a unique_ptr in the same
call instruction parameter list where it was passed via std::move.
Apparently due to luck this was not exposed when I built the compiler
with clang, only with gcc.

llvm-svn: 260442
2016-02-10 21:55:02 +00:00
Teresa Johnson 89f38fb5cc Revert "[ThinLTO] Use MD5 hash in function index." due to bot failure
This reverts commit r260408. Bot failure that I need to investigate.

llvm-svn: 260412
2016-02-10 19:11:15 +00:00
Teresa Johnson 0919a84071 [ThinLTO] Use MD5 hash in function index.
Summary:
This patch uses the lower 64-bits of the MD5 hash of a function name as
a GUID in the function index, instead of storing function names. Any
local functions are first given a global name by prepending the original
source file name. This is the same naming scheme and GUID used by PGO in
the indexed profile format.

This change has a couple of benefits. The primary benefit is size
reduction in the combined index file, for example 483.xalancbmk's
combined index file was reduced by around 70%. It should also result in
memory savings for the index file in memory, as the in-memory map is
also indexed by the hash instead of the string.

Second, this enables integration with indirect call promotion, since the
indirect call profile targets are recorded using the same global naming
convention and hash. This will enable the function importer to easily
locate function summaries for indirect call profile targets to enable
their import and subsequent promotion.

The original source file name is recorded in the bitcode in a new
module-level record for use in the ThinLTO backend pipeline.

Reviewers: davidxl, joker.eph

Subscribers: llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D17028

llvm-svn: 260408
2016-02-10 18:57:54 +00:00
Filipe Cabecinhas a2b0ac40cf Error out when faced with value names containing '\0'
Bug found with afl-fuzz.

llvm-svn: 252048
2015-11-04 14:53:36 +00:00
Filipe Cabecinhas 7aae2f23c8 Don't assert if materializing before seeing any function bodies
This assert was reachable from user input. A minimized test case (no
FUNCTION_BLOCK_ID record) is attached.

Bug found with afl-fuzz

llvm-svn: 251910
2015-11-03 13:48:26 +00:00
Filipe Cabecinhas 14323e02e6 Revert "Don't assert if materializing before seeing any function bodies"
This reverts r251667 since it broke the bots.

llvm-svn: 251671
2015-10-30 00:00:58 +00:00
Filipe Cabecinhas bcfd1f0c56 Don't assert if materializing before seeing any function bodies
This assert was reachable from user input. A minimized test case (no
FUNCTION_BLOCK_ID record) is attached.

Bug found with afl-fuzz

llvm-svn: 251667
2015-10-29 23:37:28 +00:00
Filipe Cabecinhas b70fd8719e Make sure the CastInst is valid before trying to create it
Bug found with afl-fuzz.

llvm-svn: 249396
2015-10-06 12:37:54 +00:00
Derek Schuff b4c1c28c6e Fix testing for end of stream in bitstream reader.
This fixes a bug found while working on the bitcode reader. In
particular, the method BitstreamReader::AtEndOfStream doesn't always
behave correctly when processing a data streamer. The method
fillCurWord doesn't properly set CurWord/BitsInCurWord if the data
streamer was already at eof, but GetBytes had not yet set the
ObjectSize field of the streaming memory object.

This patch fixes this problem, and provides a test to show that
this problem has been fixed.

Patch by Karl Schimpf.

Differential Revision: http://reviews.llvm.org/D11391

llvm-svn: 243890
2015-08-03 18:01:50 +00:00
Rafael Espindola 4e7212177f Fix for PR23310: llvm-dis crashes when trying to upgrade an intrinsic.
When trying to upgrade @llvm.x86.sse2.psrl.dq while parsing a module,
BitcodeReader adds the function to its worklist twice, resulting in a
crash when accessing it the second time.

This patch replaces the worklist vector by a map.

Patch by Philip Pfaffe.

llvm-svn: 241281
2015-07-02 16:22:40 +00:00
Rafael Espindola c6afe0d4e9 Improve handling of end of file in the bitcode reader.
Before this patch the bitcode reader would read a module from a file
that contained in order:

* Any number of non MODULE_BLOCK sub blocks.
* One MODULE_BLOCK
* Any number of non MODULE_BLOCK sub blocks.
* 4 '\n' characters to handle OS X's ranlib.

Since we support lazy reading of modules, any information that is relevant
for the module has to be in the MODULE_BLOCK or before it. We don't gain
anything from checking what is after.

This patch then changes the reader to stop once the MODULE_BLOCK has been
successfully parsed.

This avoids the ugly special case for .bc files in an archive and makes it
easier to embed bitcode files.

llvm-svn: 239845
2015-06-16 20:03:39 +00:00
Rafael Espindola 35f6faed67 Add a test for padded bitcode files.
llvm-svn: 239829
2015-06-16 16:36:15 +00:00
Filipe Cabecinhas da86b6d409 [BitcodeReader] Diagnose type mismatches with aliases
Bug found with AFL fuzz.

llvm-svn: 238895
2015-06-03 01:30:13 +00:00
Filipe Cabecinhas 7b3995885d [Bitcode] Minimize the test to not conflict with others
Source for the test:
@bloom = global <3 x i32> <i32 0, i32 1, i32 42>

Plus bit twiddling to set the vector numelts to 0 (in the bc file).

llvm-svn: 238894
2015-06-03 01:30:08 +00:00
Filipe Cabecinhas 8e42190d20 [BitcodeReader] Check vector size before trying to create a VectorType
Bug found with AFL fuzz

llvm-svn: 238891
2015-06-03 00:05:30 +00:00
Filipe Cabecinhas 14e686774d [BitcodeReader] Change an assert to a call to a call to Error()
It's reachable from user input.

Bug found with AFL fuzz.

llvm-svn: 238633
2015-05-30 00:17:20 +00:00
Filipe Cabecinhas 6a92a3fe34 [BitcodeReader] Change assert to report_fatal_error
It can be triggered by user input.

Bug found with AFL fuzz.

llvm-svn: 238272
2015-05-27 01:05:40 +00:00
Filipe Cabecinhas 8cd99e9a5a [BitstreamReader] Make sure the Array operand type is an encoding
Bug found with AFL fuzz.

llvm-svn: 238269
2015-05-27 00:48:43 +00:00
Filipe Cabecinhas bc6a909384 [BitcodeReader] Make sure abbrev records have at least one operand (record code)
Bug found with AFL fuzz.

llvm-svn: 238265
2015-05-26 23:52:21 +00:00
Filipe Cabecinhas 0eb8a59a67 [BitcodeReader] Sanity check on Comdat ID
Shouldn't be an assert, since user input can trigger it.

Bug found with AFL fuzz.

llvm-svn: 238261
2015-05-26 23:00:56 +00:00
Filipe Cabecinhas fc93be21ea Change a reachable unreachable to a fatal error.
Summary:
Also tagged a FIXME comment, and added information about why it breaks.

Bug found using AFL fuzz.

Reviewers: rafael, craig.topper

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D9729

llvm-svn: 237709
2015-05-19 18:18:10 +00:00
Filipe Cabecinhas 32af542194 [BitcodeReader] Error out if we read an invalid function argument type
Bug found with AFL fuzz.

llvm-svn: 237650
2015-05-19 01:21:06 +00:00
Filipe Cabecinhas f3fa99c48e [BitcodeReader] It's a malformed block if CodeLenWidth is too big
Bug found with AFL fuzz.

llvm-svn: 237646
2015-05-19 00:34:17 +00:00
Filipe Cabecinhas 4708a02a78 [BitcodeReader] Make sure the type of the inserted value matches the type of the aggregate at those indices
Bug found with AFL-fuzz.

llvm-svn: 237628
2015-05-18 22:27:11 +00:00
Filipe Cabecinhas 11bb8495f6 Extract the load/store type verification to a separate function.
Summary:
Added isLoadableOrStorableType to PointerType.

We were doing some checks in some places, occasionally assert()ing instead
of telling the caller. With this patch, I'm putting all type checking in
the same place for load/store type instructions, and verifying the same
thing every time.

I also added a check for load/store of a function type.

Applied extracted check to Load, Store, and Cmpxcg.

I don't have exhaustive tests for all of these, but all Error() calls in
TypeCheckLoadStoreInst are being tested (in invalid.test).

Reviewers: dblaikie, rafael

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D9785

llvm-svn: 237619
2015-05-18 21:48:55 +00:00
Filipe Cabecinhas 1c299d05e6 [BitcodeReader] Don't allow INSERTVAL/EXTRACTVAL with 0 indices
This would trigger an assertion later.

Bug found with AFL fuzz.

llvm-svn: 237494
2015-05-16 00:33:12 +00:00
Filipe Cabecinhas f8a16a952d Don't overflow GCTable
Summary: Bug found with AFL fuzz.

Reviewers: rafael, dexonsmith

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D9361

llvm-svn: 236200
2015-04-30 04:09:41 +00:00
Filipe Cabecinhas 9a19e56306 Make sure Op->getType() is a PointerType before we cast<> it.
Bug found with AFL fuzz.

llvm-svn: 236193
2015-04-30 01:13:31 +00:00
Filipe Cabecinhas bad0779f63 Make sure we don't resize(0) when we get a fwdref with Idx == UINT_MAX
Make it an error instead.

Bug found with AFL fuzz.

llvm-svn: 236190
2015-04-30 00:52:42 +00:00
Filipe Cabecinhas d8a1bcd0ad Check that we have a valid PointerType element type before calling get()
Same as r236073 but for PointerType.

Bug found with AFL fuzz.

llvm-svn: 236079
2015-04-29 02:27:28 +00:00
Filipe Cabecinhas 1351cba720 Turn an assert into report_fatal_error since it's reachable based on user input
Bug found with AFL fuzz.

llvm-svn: 236076
2015-04-29 01:58:31 +00:00
Filipe Cabecinhas f15fb032ef Make sure that isValidElementType(Type) before calling {Array,Struct}Type::get(Type)
Bug found with AFL fuzz.

llvm-svn: 236073
2015-04-29 01:27:01 +00:00
Filipe Cabecinhas b435d0f439 Relax an assert when there's a type mismatch in forward references
Summary:
We don't seem to need to assert here, since this function's callers expect
to get a nullptr on error. This way we don't assert on user input.

Bug found with AFL fuzz.

Reviewers: rafael

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D9308

llvm-svn: 236027
2015-04-28 20:18:47 +00:00
David Blaikie 5ea1f7b744 [opaque pointer type] bitcode: add explicit callee type to invoke instructions
llvm-svn: 235735
2015-04-24 18:06:06 +00:00
Filipe Cabecinhas ff1e234fb8 [BitcodeReader] Fix asserts when we read a non-vector type for insert/extract/shuffle
Added some additional checking for vector types + tests.

Bug found with AFL fuzz.

llvm-svn: 235710
2015-04-24 11:30:15 +00:00
Filipe Cabecinhas 6621cb7478 Be more strict about the operand for the array type in BitcodeReader
Summary: Bug found with AFL fuzz.

Reviewers: rafael

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D9016

llvm-svn: 235596
2015-04-23 13:38:21 +00:00
Filipe Cabecinhas ee48feadfd Verify sizes when trying to read a BitcodeAbbrevOp
Summary:
Make sure the abbrev operands are valid and that we can read/skip them
afterwards.

Bug found with AFL fuzz.

Reviewers: rafael

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D9030

llvm-svn: 235595
2015-04-23 13:25:35 +00:00
Filipe Cabecinhas ea79c5b4f7 Have more strict type checks when creating BinOp nodes in BitcodeReader
Summary: Bug found with AFL.

Reviewers: rafael, bkramer

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D9015

llvm-svn: 235489
2015-04-22 09:06:21 +00:00
David Blaikie dbe6e0f171 [opaque pointer type] Explicit pointee type for call instruction
Use an extra bit in the CCInfo to flag the newer version of the
instructiont hat includes the type explicitly.

Tested the newer error cases I added, but didn't add tests for the finer
granularity improvements to existing error paths.

llvm-svn: 235160
2015-04-17 06:40:14 +00:00
Filipe Cabecinhas 2e206eb65f Revert "Verify sizes when trying to read a VBR"
This reverts r234984 since it seems to break some bots (most of them
seemed arm*-selfhost).

llvm-svn: 234998
2015-04-15 11:10:17 +00:00
Filipe Cabecinhas 7dc896fcce Verify sizes when trying to read a VBR
Also added an assert to ReadVBR64.

llvm-svn: 234984
2015-04-15 08:48:08 +00:00
Filipe Cabecinhas 225542713b Error out of ParseBitcodeInto(Module*) if we haven't read a Module
Summary:
Without this check the following case failed:

Skip a SubBlock which is not a MODULE_BLOCK_ID nor a BLOCKINFO_BLOCK_ID
Got to end of file

TheModule would still be == nullptr, and we would subsequentially fail
when materializing the Module (assert at the start of
BitcodeReader::MaterializeModule).

Bug found with AFL.

Reviewers: dexonsmith, rafael

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D9014

llvm-svn: 234887
2015-04-14 14:07:15 +00:00
David Blaikie 12cf5d70e8 Add testing for mismatched explicit type on a gep operator when loading from bitcode
llvm-svn: 232427
2015-03-16 22:03:50 +00:00
David Blaikie c695cc7e58 Add testing for mismatched explicit type on a load instruction when loading from bitcode
llvm-svn: 232424
2015-03-16 21:48:46 +00:00
David Blaikie 675e8cb09e Test bitcode parsing error-handling for incorrect explicit type
(turns out I had regressed this when sinking handling of this type down
into GetElementPtrInst::Create - since that asserted before the error
handling was performed)

llvm-svn: 232420
2015-03-16 21:35:48 +00:00
JF Bastien 30bf96bfe7 Use common parse routine to read alignment values from bitcode
While fuzzing LLVM bitcode files, I discovered that (1) the bitcode reader doesn't check that alignments are no larger than 2**29; (2) downstream code doesn't check the range; and (3) for values out of range, corresponding large memory requests (based on alignment size) will fail. This code fixes the bitcode reader to check for valid alignments, fixing this problem.

This CL fixes alignment value on global variables, functions, and instructions: alloca, load, load atomic, store, store atomic.

Patch by Karl Schimpf (kschimpf@google.com).

llvm-svn: 230180
2015-02-22 19:32:03 +00:00
Filipe Cabecinhas ecf8f7f49b [Bitcode reader] Fix a few assertions when reading invalid files
Summary:
When creating {insert,extract}value instructions from a BitcodeReader, we
weren't verifying the fields were valid.

Bugs found with afl-fuzz

Reviewers: rafael

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D7325

llvm-svn: 229345
2015-02-16 00:03:11 +00:00
Filipe Cabecinhas fcd044b692 Check bit widths before trying to get a type.
Added a test case for it.
Also added run lines for the test case in r227566.

Bugs found with afl-fuzz

llvm-svn: 227589
2015-01-30 18:13:50 +00:00
Filipe Cabecinhas d0858e1037 [bitcode reader] Fix an assert on invalid type tables
Bug found with afl-fuzz

llvm-svn: 227566
2015-01-30 10:57:58 +00:00
Filipe Cabecinhas de968ecb05 [Bitcode] Diagnose errors instead of asserting from bad input
Eventually we can make some of these pass the error along to the caller.

Reports a fatal error if:
We find an invalid abbrev record
We try to get an invalid abbrev number
We can't fill the current word due to an EOF

Fixed an invalid bitcode test to check for output with FileCheck

Bugs found with afl-fuzz

llvm-svn: 226986
2015-01-24 04:15:05 +00:00