Commit Graph

404 Commits

Author SHA1 Message Date
Rong Xu 1288a19421 [PGO] Differentiate Clang instrumentation and IR level instrumentation profiles
This patch uses one bit in profile version to differentiate Clang
instrumentation and IR level instrumentation profiles.

PGOInstrumenation generates a COMDAT variable __llvm_profile_raw_version so
that the compiler runtime can set the right profile kind.
PGOInstrumenation now checks this bit to make sure it's an IR level
instrumentation profile.

Differential Revision: http://reviews.llvm.org/D15540

llvm-svn: 260146
2016-02-08 21:07:46 +00:00
Xinliang David Li a82d6c0a4b [PGO] Enable compression in pgo instrumentation
This reduces sizes of instrumented object files, final binaries,
process images, and raw profile data.

The format of the indexed profile data remain the same.

Differential Revision: http://reviews.llvm.org/D16388 
 

llvm-svn: 260117
2016-02-08 18:13:49 +00:00
Benjamin Kramer 85c824f131 Move classes defined in a cpp file into an anonymous namespace.
No functionality change intended.

llvm-svn: 259883
2016-02-05 13:50:53 +00:00
Xinliang David Li eb7d7f8729 Function name change /NFC
llvm-svn: 259851
2016-02-04 23:59:09 +00:00
Easwaran Raman d68aae24e4 Refactor profile summary support code. NFC.
Summary computation is not just for instrumented profiling and so I have moved
the ProfileSummary class to ProfileCommon.h (named so to allow code unrelated
to summary but common to instrumented and sampled profiling to be placed there)

Differential Revision: http://reviews.llvm.org/D16661

llvm-svn: 259846
2016-02-04 23:34:31 +00:00
Xinliang David Li 402477d2ba [PGO] Add interfaces to annotate instr with VP data
Add interfaces to do value profile data IR annnotation
  and read. Needed by both FE and IR based PGO.

llvm-svn: 259813
2016-02-04 19:11:43 +00:00
Xinliang David Li 1e4c809c6c [PGO] Profile interface cleanup
- Remove unused valuemapper parameter
  - add totalcount optional parameter

llvm-svn: 259756
2016-02-04 05:29:51 +00:00
Xinliang David Li 3c88288927 Fix a typo in comment
llvm-svn: 259631
2016-02-03 06:24:11 +00:00
Xinliang David Li a398d2d94a Fix uninitiazed variable use problem
llvm-svn: 259630
2016-02-03 06:23:16 +00:00
Xinliang David Li 6c93ee8d36 [PGO] Profile summary reader/writer support
With this patch, the profile summary data will be available in indexed
profile data file so that profiler reader/compiler optimizer can start
to make use of.

Differential Revision: http://reviews.llvm.org/D16258

llvm-svn: 259626
2016-02-03 04:08:18 +00:00
Vedant Kumar 00dab22853 [Profiling] Add a -sparse mode to llvm-profdata merge
Add an option to llvm-profdata merge for writing out sparse indexed
profiles. These profiles omit InstrProfRecords for functions which are
never executed.

Differential Revision: http://reviews.llvm.org/D16727

llvm-svn: 259258
2016-01-29 22:54:45 +00:00
Yaron Keren eb2a25467e Annotate dump() methods with LLVM_DUMP_METHOD, addressing Richard Smith r259192 post commit comment.
clang part in r259232, this is the LLVM part of the patch.

llvm-svn: 259240
2016-01-29 20:50:44 +00:00
Xinliang David Li 731637567e [PGO] allow pgo name collector to disable compression (for testing)/NFC
llvm-svn: 258876
2016-01-26 23:13:00 +00:00
Chris Bieneman e49730d4ba Remove autoconf support
Summary:
This patch is provided in preparation for removing autoconf on 1/26. The proposal to remove autoconf on 1/26 was discussed on the llvm-dev thread here: http://lists.llvm.org/pipermail/llvm-dev/2016-January/093875.html

"I felt a great disturbance in the [build system], as if millions of [makefiles] suddenly cried out in terror and were suddenly silenced. I fear something [amazing] has happened."
- Obi Wan Kenobi

Reviewers: chandlerc, grosbach, bob.wilson, tstellarAMD, echristo, whitequark

Subscribers: chfast, simoncook, emaste, jholewinski, tberghammer, jfb, danalbert, srhines, arsenm, dschuff, jyknight, dsanders, joker.eph, llvm-commits

Differential Revision: http://reviews.llvm.org/D16471

llvm-svn: 258861
2016-01-26 21:29:08 +00:00
Eugene Zelenko 6ac3f739ca Fix Clang-tidy modernize-use-nullptr and modernize-use-override warnings; other minor fixes.
Differential revision: reviews.llvm.org/D16568

llvm-svn: 258831
2016-01-26 18:48:36 +00:00
Xinliang David Li e1574f0cdf [PGO] Remove use of static variable. /NFC
Make the variable a member of  the writer trait object owned
now by the writer. Also use a different generator interface
to pass the infoObject from the writer. 

llvm-svn: 258544
2016-01-22 20:25:56 +00:00
Xinliang David Li 46ad363ba4 Revert 258486 -- for a better fix coming soon
llvm-svn: 258538
2016-01-22 19:53:31 +00:00
Xinliang David Li 876c2024e2 [PGO] eliminate use of static variable
llvm-svn: 258486
2016-01-22 05:48:40 +00:00
Xinliang David Li 59411db520 [PGO] Add a new interface to be used by Indirect Call Promotion
llvm-svn: 258271
2016-01-20 01:26:34 +00:00
Xinliang David Li 0a83b1b994 Fix a coverage reading bug
function record pointer is not advanced when
duplicate entry is found.

Test case to be added.

llvm-svn: 258188
2016-01-19 21:18:12 +00:00
Xinliang David Li 42a13308a1 [Coverage] move a local var to be BinaryCoverageReader's member
The symtab is logically referenced beyond the call to the create
method. This changes makes sure its lifetime matches that of
the reader.

llvm-svn: 258036
2016-01-18 06:48:01 +00:00
Xinliang David Li 285d7bd4e7 Fix -Wmismatched-tags warning/error
llvm-svn: 257924
2016-01-15 19:22:41 +00:00
Xinliang David Li b606638526 [PGO] Commonize (more) index profile file and buffer writer.
The file and buffer writer code are mostly shared except for the
stream back-patching. This is because raw_string_ostream does not
support seek like interface. The result is that the data patching
code needs to be pushed to the caller which is not quite readable 
(passing around offset, value etc). This also makes future enhancement
(which needs more patching) more difficult (and can make impl messy).

In this patch, two types of streams needed by the writer are now
unified with same set of interfaces under ProfOStream class. The patch
method is added so that common implementation becomes cleaner. It
also enables future enhancement. Should be NFC.

llvm-svn: 257921
2016-01-15 19:01:04 +00:00
Xinliang David Li 565b301380 [PGO] Move profile summary interface/impl into InstrProf.[*] /NFC
llvm-svn: 257819
2016-01-14 22:10:49 +00:00
Xinliang David Li 84a2df39e0 Rename local variable to avoid conflict
llvm-svn: 257748
2016-01-14 06:38:52 +00:00
Xinliang David Li d5d8887d28 Cleanup: shorten prefix to consistent with other decls /NFC
llvm-svn: 257744
2016-01-14 06:21:25 +00:00
Xinliang David Li a6b2c4f721 [PGO] clean up and documentation
Introduce enum for indexed format versions and 
document indexed format change history.

llvm-svn: 257737
2016-01-14 02:47:01 +00:00
Xinliang David Li e7268c1a9a Add virtual dtor
llvm-svn: 257734
2016-01-14 02:10:49 +00:00
Xinliang David Li a9d784666e [Coverage] introduce class hierarchy (funcRecordReader) to support multiple versions of coverage data
[resubmit after fixing build bot failures: qualify make_unique and eliminate -Wcovered-switch-default warning.
With the planned size reduction change, the coverage format version is expected to be bumped up. This patch adds necessary support such that backward compatibility can be kept with maximal code sharing. Reading different versions of coverage data just requires instantiating the reader according to the version.

No functional change is intended.

Differiential Revision: http://reviews.llvm.org/D16133

llvm-svn: 257708
2016-01-13 23:29:33 +00:00
Xinliang David Li e62595c4a7 Revert r257699 -- windows buildbot failure TBI
llvm-svn: 257703
2016-01-13 23:12:53 +00:00
Xinliang David Li aab986f873 [Coverage] introduce class hierarchy (funcRecordReader) to support multiple versions of coverage data
With the planned size reduction change, the coverage format version is expected to be bumped up. This patch adds necessary support such that backward compatibility can be kept with maximal code sharing. Reading different versions of coverage data just requires instantiating the reader according to the version.

No functional change is intended.

Differiential Revision: http://reviews.llvm.org/D16133

llvm-svn: 257699
2016-01-13 22:58:42 +00:00
Xinliang David Li 81f18a58f1 [Coverage] Refactor coverage mapping reader code
(Resubmit after fixing a typo that breaks test on big endian 
 machines)

In this refactoring, member functions are introduced to access
CovMap header/func record members and hide layout details. This
will enable further code restructuring to support reading multiple
versions of coverage mapping data with shared/templatized code. 
(When coveremap format version changes, backward compatibtility
should be preserved).

llvm-svn: 257571
2016-01-13 04:36:15 +00:00
Xinliang David Li c8c39ea822 Rollback r257551 -- unexpected test failures TBI
llvm-svn: 257564
2016-01-13 02:46:40 +00:00
Xinliang David Li b92bc6dff2 [Coverage] Refactor coverage mapping reader code /NFC
(Resubmit after fixing build bot failures)

In this refactoring, member functions are introduced to access
CovMap header/func record members and hide layout details. This
will enable further code restructuring to support reading multiple
versions of coverage mapping data with shared/templatized code. 
(When coveremap format version changes, backward compatibtility
should be preserved).

llvm-svn: 257551
2016-01-13 00:53:46 +00:00
Xinliang David Li 8c65278179 Rollback r257547 -- buildbot failure TBI
llvm-svn: 257549
2016-01-13 00:27:24 +00:00
Xinliang David Li c3498b07db [Coverage] Refactor coverage mapping reader code /NFC
In this refactoring, member functions are introduced to access
CovMap header/func record members and hide layout details. This
will enable further code restructuring to support reading multiple
versions of coverage mapping data with shared/templatized code. 
(When coveremap format version changes, backward compatibtility
should be preserved).

llvm-svn: 257547
2016-01-13 00:16:43 +00:00
Nathan Slingerland 7bee316890 [Support] Add saturating multiply-add support function
Summary: Add SaturatingMultiplyAdd convenience function template since A + (X * Y) comes up frequently when doing weighted arithmetic.

Reviewers: davidxl, silvas

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D15385

llvm-svn: 257532
2016-01-12 22:34:00 +00:00
Xinliang David Li 8a5bdb5dce Move coveragemap_error enum into coverage namespace and InstrProf.h /NFC
llvm-svn: 257295
2016-01-10 21:56:33 +00:00
Xinliang David Li 062cde9cc3 [PGO] Ensure vp data in indexed profile always sorted
Done in  InstrProfWriter to eliminate the need for client
code to do the sorting. The operation is done once and reused 
many times so it is more efficient. Update unit test to remove
sorting. Also update expected output of affected tests.

llvm-svn: 257145
2016-01-08 05:45:21 +00:00
Xinliang David Li 51dc04cff2 [PGO] Fix a bug in InstProfWriter addRecord
For a new record with weight != 1, only edge profiling
counters are scaled, VP data is not properly scaled.

This patch refactors the code and fixes the problem.
Also added sort by count interface (for follow up patch).

llvm-svn: 257143
2016-01-08 03:49:59 +00:00
Xinliang David Li 1054a85a28 [PGO] Minor refactoring /NFC
Move common defs into common header files.

llvm-svn: 257108
2016-01-07 22:46:29 +00:00
Xinliang David Li 204efe2de8 [PGO] Simplify string parsing
Patch Suggested by Vedant.

llvm-svn: 256785
2016-01-04 22:09:26 +00:00
Xinliang David Li 120fe2e898 [PGO] Refactor string writer code
For readability and code sharing.
(Adapted from Suggestions by Vedant).

llvm-svn: 256784
2016-01-04 22:01:02 +00:00
Xinliang David Li 0c677878d3 [PGO]: Use efficient 'join' API for uncompressed string
llvm-svn: 256781
2016-01-04 21:31:09 +00:00
Xinliang David Li 13ea29b5a1 [PGO]: reserve space for string to avoid excessive memory realloc/copy (non linear)
llvm-svn: 256776
2016-01-04 20:26:05 +00:00
Xinliang David Li 946558df2d [PGO] Code refactoring to use header struct def /NFC
llvm-svn: 256712
2016-01-03 18:57:40 +00:00
Xinliang David Li 37c1fa047d [PGO] simple refactoring (NFC)
llvm-svn: 256695
2016-01-03 04:38:13 +00:00
Xinliang David Li e413f1a0fc [PGO]: Implement Func PGO name string compression
This is part of the effort/prepration to reduce the size
instr-pgo (object, binary, memory footprint, and raw data).

The functionality is currently off by default and not yet
used by any clients.

llvm-svn: 256667
2015-12-31 07:57:16 +00:00
Benjamin Kramer 7a5c8c8fe3 [ProfileData] Make helper function static.
No functional change.

llvm-svn: 256375
2015-12-24 10:03:37 +00:00
Xinliang David Li 6005728843 Fix a latent UAF bug in profwriter
llvm-svn: 256116
2015-12-20 08:46:18 +00:00
Xinliang David Li a716cc5c33 [PGO] Improve Indexed Profile Reader efficiency
With the support of value profiling added, the Indexed prof
reader gets less efficient. The prof reader initialization
used to be just reading the file header, but with VP support
added, initialization needs to walk through all profile keys
of ondisk hash table resulting in very poor locality and large
memory increase (keys are stored together with the profile data
in the mapped profile buffer). Even worse, when the reader is 
used by the compiler (not llvm-profdata too), the penalty becomes
very high as compilation of each single module requires touching
profile data buffer for the whole program. 

In this patch, the icall target values (MD5hash) are no longer eargerly 
converted back to name strings when the data is read into memory. New
interface is added to to profile reader so that InstrProfSymtab can be
lazily created for Indexed profile reader on-demand. Creating of the 
symtab is intended to be used by llvm-profdata tool for symbolic dumping
of  VP data. It can be used with compiler (for legacy out of tree uses)
too but not recommended due to compile time and memory reasons 
mentioned above.

Some other cleanups are also included: Function Addr to md5 map is now
consolated into InstrProfSymtab. InstrProfStringtab is no longer used and
eliminated.

llvm-svn: 256114
2015-12-20 06:22:13 +00:00
Xinliang David Li 5c24da5d8e Minor clean up -- move large single use method out of header(NFC)
llvm-svn: 256113
2015-12-20 05:15:45 +00:00
Xinliang David Li 020f22d810 [PGO] Cleanup: Move large member functions out of line (NFC)
llvm-svn: 256058
2015-12-18 23:06:37 +00:00
Xinliang David Li 49ee76d082 [PGO] Simplify computehash interface (NFC)
llvm-svn: 256047
2015-12-18 22:22:12 +00:00
Xinliang David Li 50de45dcc1 [PGO] InstrPGO and coverage code refactoring (NFC)
Introduce a new class InstrProfSymtab to abstract
the PGO symbol table for prof and coverage reader.
The symtab is is to lookup function's PGO name
using function keys. The first user of the class
is CoverageMapping Reader. More will follow.

llvm-svn: 255862
2015-12-17 00:53:37 +00:00
Nathan Slingerland 48dd080c77 [PGO] Handle and report overflow during profile merge for all types of data
Summary: Surface counter overflow when merging profile data. Merging still occurs on overflow but counts saturate to the maximum representable value. Overflow is reported to the user.

Reviewers: davidxl, dnovillo, silvas

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D15547

llvm-svn: 255825
2015-12-16 21:45:43 +00:00
Xinliang David Li 38b9a32fcd Initialize all bytes in vp data (msan error)
llvm-svn: 255680
2015-12-15 21:57:08 +00:00
Xinliang David Li 4ec401406e Coverage code refactoring /NFC
llvm-svn: 255670
2015-12-15 19:44:45 +00:00
Nathan Slingerland 7f5b47ddd4 [llvm-profdata] Add support for weighted merge of profile data (2nd try)
Summary:
This change adds support for specifying a weight when merging profile data with the llvm-profdata tool.
Weights are specified by using the --weighted-input=<weight>,<filename> option. Input files not specified
with this option (normal positional list after options) are given a default weight of 1.

Adding support for arbitrary weighting of input profile data allows for relative importance to be placed on the
input data from multiple training runs.

Both sampled and instrumented profiles are supported.

Reviewers: davidxl, dnovillo, bogner, silvas

Subscribers: silvas, davidxl, llvm-commits

Differential Revision: http://reviews.llvm.org/D15306

llvm-svn: 255659
2015-12-15 17:37:09 +00:00
Xinliang David Li e3bf4fd394 [PGO] Value profiling text format reader/writer support
This patch adds the missing functionality in parsable
text format support for value profiling.

Differential Revision: http://reviews.llvm.org/D15212

llvm-svn: 255523
2015-12-14 18:44:01 +00:00
Xinliang David Li d1bab96045 [PGO] Stop using invalid char in instr variable names.
Before the patch, -fprofile-instr-generate compile will fail
if no integrated-as is specified when the file contains
any static functions (the -S output is also invalid).

This is the second try. The fix in this patch is very localized.
Only profile symbol names of profile symbols with internal 
linkage are fixed up while initializer of name syms are not 
changes. This means there is no format change nor version bump.

llvm-svn: 255434
2015-12-12 17:28:03 +00:00
Xinliang David Li a86545b0b5 [PGO] Revert r255365: solution incomplete, not handling lambda yet
llvm-svn: 255369
2015-12-11 20:23:22 +00:00
Xinliang David Li c79283ef29 [PGO] Stop using invalid char in instr variable names.
Before the patch, -fprofile-instr-generate compile will fail
if no integrated-as is specified when the file contains
any static functions (the -S output is also invalid).

This patch fixed the issue. With the change, the index format
version will be bumped up by 1. Backward compatibility is 
preserved with this change.

Differential Revision: http://reviews.llvm.org/D15243

llvm-svn: 255365
2015-12-11 19:53:19 +00:00
Xinliang David Li d922c26c02 [PGO] Read VP raw data without depending on the Value field
Before this patch, each function's on-disk VP data is 'pointed'
to by the Value field of per-function ProfileData structue, and 
read relies on this field (relocated with ValueDataDelta field)
to read the value data. However this means the Value field needs
to be updated during runtime before dumping, which creates undesirable
data races.

With this patch, the reading of VP data no longer depends on Value
field. There is no format change. ValueDataDelta header field becomes
obsolute but will be kept for compatibility reason (will be removed
next time the raw format change is needed).

llvm-svn: 255329
2015-12-11 06:53:53 +00:00
Vedant Kumar 2491dd118f [ProfileData] clang-format TextInstrProfReader::hasFormat. NFC.
llvm-svn: 255317
2015-12-11 00:40:05 +00:00
Xinliang David Li 2d4803e81b Format fix (NFC)
llvm-svn: 255313
2015-12-10 23:48:05 +00:00
Nathan Slingerland 51abea7442 [ProfileData] Add unit test infrastructure for sample profile reader/writer
Summary:
Adds support for in-memory round-trip of sample profile data along with basic
round trip unit tests. This will also make it easier to include unit tests for
future changes to sample profiling.

Reviewers: davidxl, dnovillo, silvas

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D15211

llvm-svn: 255264
2015-12-10 17:21:42 +00:00
Xinliang David Li 307902e297 [PGO] Add version to getPGOFuncName method
Different version of indexed format may use different
name uniquing schemes for static functions. Pass the
version info to the name interface so that different
schmes can be picked (for profile lookup).

llvm-svn: 254838
2015-12-05 05:16:36 +00:00
Nathan Slingerland cb921a1d88 Revert "[llvm-profdata] Add support for weighted merge of profile data"
This reverts commit b7250858d96b8ce567681214273ac0e62713c661.

Reverting in order to investigate Windows test failure.

llvm-svn: 254687
2015-12-04 02:13:58 +00:00
Xinliang David Li 01cb9bd7b3 [PGO] Unify VP data format between raw and indexed profile (Reader)
With the latest refactoring and code sharing patches landed, 
it is possible to unify the value profile implementation between
raw and indexed profile. This is the patch in raw profile reader 
that uses the common interface. 

Differential Revision: http://reviews.llvm.org/D15056

llvm-svn: 254677
2015-12-04 01:02:10 +00:00
Nathan Slingerland 2a3dbe8be2 [llvm-profdata] Add support for weighted merge of profile data
This change adds support for an optional weight when merging profile data with the llvm-profdata tool.
Weights are specified by adding an option ':<weight>' suffix to the input file names.

Adding support for arbitrary weighting of input profile data allows for relative importance to be placed on the
input data from multiple training runs.

Both sampled and instrumented profiles are supported.

Reviewers: dnovillo, bogner, davidxl

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D14547

llvm-svn: 254669
2015-12-04 00:00:20 +00:00
Nathan Slingerland aa5702d92b [llvm-profdata] Change instr prof counter overflow to saturate rather than discard
Summary: This changes overflow handling during instrumentation profile merge. Rathar than throwing away records that would result in counter overflow, merged counts are instead clamped to the maximum representable value. A warning about counter overflow is still surfaced to the user as before.

Reviewers: dnovillo, davidxl, silvas

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D14893

llvm-svn: 254525
2015-12-02 18:19:24 +00:00
Xinliang David Li a28306db0c [PGO] Add support for reading multiple versions of indexed profile format profile data
Profile readers using incompatible on-disk hash table format can now share the same 
implementation and interfaces. 

Differential Revision: http://reviews.llvm.org/D15100

llvm-svn: 254458
2015-12-01 20:26:26 +00:00
Xinliang David Li 0e6a36e17e Use nullptr (NFC)
llvm-svn: 254447
2015-12-01 19:47:32 +00:00
Xinliang David Li b75544a6df [PGO] Move value profile format related structures and APIs to common file
This is the last step to enable profile runtime to share the same value prof
data format and reader/writer code with llvm host tools. The VP related 
data structures are moved to a section in InstrProfData.inc enabled with macro
INSTR_PROF_VALUE_PROF_DATA, and common API implementations are enabled with
INSTR_PROF_COMMON_API_IMPL. There should be no functional change.

llvm-svn: 254235
2015-11-28 19:07:09 +00:00
Xinliang David Li 017fffbd2a [PGO] Add return code for vp rt record init routine to indicate error condition
llvm-svn: 254220
2015-11-28 05:47:34 +00:00
Xinliang David Li 4cccee52ce [PGO] Allow value profile writer interface to allocated target buffer
Raw profile writer needs to write all data of one kind in one continuous block,
so the buffer needs to be pre-allocated and passed to the writer method in
pieces for function profile data. The change adds the support for raw value data
writing.

llvm-svn: 254219
2015-11-28 05:37:01 +00:00
Xinliang David Li be969c2942 Function name cleanup (NFC)
llvm-svn: 254218
2015-11-28 05:06:00 +00:00
Xinliang David Li 8e32f4d5b6 [PGO] Extract VP data integrity check code into a helper function (NFC)
llvm-svn: 254217
2015-11-28 04:56:07 +00:00
Xinliang David Li ed966771da [PGO] Implement ValueProfiling Closure interfaces for runtime value profile data
This is one of the many steps to commonize value profiling support between profile
runtime and compiler/llvm tools.

After this change, profiler runtime now can share the same C APIs to do VP
serialization/deseriazation with LLVM host tools (and produces value data
in identical format between indexed and raw profile).

It is not yet enabled in profiler runtime yet.

Also added a unit test case to test runtime profile data serialization/deserialization
interfaces implemented using common closure code.

llvm-svn: 254110
2015-11-25 23:31:18 +00:00
Xinliang David Li e809231b22 [PGO] Regroup functions in better order (NFC)
llvm-svn: 254080
2015-11-25 19:13:00 +00:00
Xinliang David Li f47cf5505f [PGO] Convert InstrProfRecord based serialization methods to use common C methods
1. Convert serialization methods using InstrProfRecord as source into C (impl)
   interfaces using Closure.
2. Reimplement InstrProfRecord serialization method to use new C interface
   as dummy wrapper. 

Now it is ready to implement wrapper for runtime value profile data.

(The new code need better source location -- but not changed in this patch to
 minimize diffs. )

llvm-svn: 254057
2015-11-25 06:23:38 +00:00
Xinliang David Li ac5b860633 [PGO] convert a subset of C++ interfaces into C (for sharing) (NFC)
llvm-svn: 254056
2015-11-25 04:29:24 +00:00
Xinliang David Li 4f18bef998 Move member functions closer to others of the same class (NFC)
llvm-svn: 254055
2015-11-25 03:24:37 +00:00
Xinliang David Li 4945b16708 Fix function naming (NFC)
llvm-svn: 254045
2015-11-25 00:08:49 +00:00
Xinliang David Li 28b700373e [PGO] Add mapper callback to interfaces retrieving value data for site (NFC)
This allows cleaner implementation and merging retrieving/mapping in
one pass.

llvm-svn: 254038
2015-11-24 23:36:52 +00:00
Xinliang David Li 759dc628c0 [PGO] Small interface change to be profile rt ready
Convert two C++ static member functions to be C APIs. This
is one of the many steps to get ready to share VP writer code
with profiler runtime. 

llvm-svn: 253999
2015-11-24 18:15:46 +00:00
Xinliang David Li 1b85d4c961 Minor refactor to make VP writing more efficient
llvm-svn: 253994
2015-11-24 17:03:24 +00:00
Xinliang David Li c667683d2e [PGO] In llvm-profdata text dump, add comment lines as annotations
llvm-svn: 253930
2015-11-23 22:31:22 +00:00
Xinliang David Li 6f7c19a494 [PGO] Add --text option for llvm-profdata show|merge commands
The new option is similar to the SampleProfile dump option.

- dump raw/indexed format into text profile format
- merge the profile and output into text profile format.

Note that Value Profiling data text format is not yet designed. 
That functionality will be added later.

Differential Revision: http://reviews.llvm.org/D14894

llvm-svn: 253913
2015-11-23 20:47:38 +00:00
Reid Kleckner 437b1b3ea5 Fix the Windows build, include <tuple> for std::tie
llvm-svn: 253698
2015-11-20 19:29:40 +00:00
Nathan Slingerland a731829788 [llvm-profdata] Add merge() to InstrProfRecord
Summary:
This change refactors two aspects of InstrProfRecord:

1) Add a merge() method to InstrProfRecord (previously InstrProfWriter combineInstrProfRecords()) in order to better encapsulate this functionality and to make the InstrProfRecord and SampleRecord APIs more consistent.

2) Make InstrProfRecord mergeValueProfData() a private method since it is only ever called internally by merge().

Reviewers: dnovillo, bogner, davidxl

Subscribers: silvas, vsk, llvm-commits

Differential Revision: http://reviews.llvm.org/D14786

llvm-svn: 253695
2015-11-20 19:12:43 +00:00
Diego Novillo 379cc5e71b SamplePGO - Tweak debugging output for function samples. NFC.
llvm-svn: 253612
2015-11-19 22:18:30 +00:00
Diego Novillo ef548d2918 SamplePGO - Sort samples by source location when emitting as text.
When dumping function samples or writing them out as text format, it
helps if the samples are emitted sorted by source location. The sorting
of the maps is a bit slow, so we only do it on demand.

llvm-svn: 253568
2015-11-19 15:33:08 +00:00
Xinliang David Li cfb1456572 Minor cleanups (from review feedback)
1. remove uneeded header inclusion
2. use reinterpret_cast instead of c ctyle
3. other format change

llvm-svn: 253515
2015-11-18 22:42:27 +00:00
Betul Buyukkurt 6fac1741c9 [PGO] Value profiling support
This change introduces an instrumentation intrinsic instruction for
value profiling purposes, the lowering of the instrumentation intrinsic
and raw reader updates. The raw profile data files for llvm-profdata
testing are updated.

llvm-svn: 253484
2015-11-18 18:14:55 +00:00
Xinliang David Li 99556877ae [PGO] Move value profile data definitions out of IndexedInstrProf
Move the data structure defintions out of the namespace. The defs will
be shared by raw format. [NFC]

llvm-svn: 253394
2015-11-17 23:00:40 +00:00
Nathan Slingerland e6e30d5e88 [llvm-profdata] Improve error messaging when merging mismatched profile data
Summary:
This change tries to make the root cause of instrumented profile data merge failures clearer.

Previous:

$ llvm-profdata merge test_0.profraw test_1.profraw -o test_merged.profdata
test_1.profraw: foo: Function count mismatch
test_1.profraw: bar: Function count mismatch
test_1.profraw: baz: Function count mismatch
...

Changed:

$ llvm-profdata merge test_0.profraw test_1.profraw -o test_merged.profdata
test_1.profraw: foo: Function basic block count change detected (counter mismatch)
Make sure that all profile data to be merged is generated from the same binary.
test_1.profraw: bar: Function basic block count change detected (counter mismatch)
test_1.profraw: baz: Function basic block count change detected (counter mismatch)
...

Reviewers: dnovillo, davidxl, bogner

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D14739

llvm-svn: 253384
2015-11-17 22:08:53 +00:00
Diego Novillo ba920be4a2 SamplePGO - Move debug/dump function bodies out of header files. NFC.
No point polluting the header declarations with debugging code.

llvm-svn: 253361
2015-11-17 19:04:46 +00:00
Xinliang David Li b8c3ad1d05 Fix unaligned memory read issue exposed by ubsan
Indexed profile data as designed today does not guarantee
counter data to be well aligned, so reading needs to use
the slower form (with memcpy). This is less than ideal and 
should be improved in the future (i.e., with fixed length
function key instead of variable length name key).

llvm-svn: 253309
2015-11-17 03:47:21 +00:00
Diego Novillo 8e415a821f SamplePGO - Add dump routines for LineLocation, SampleRecord and FunctionSamples
llvm-svn: 253071
2015-11-13 20:24:28 +00:00
Nathan Slingerland 4f82366759 [llvm-profdata] Add check for text profile formats and improve error reporting (2nd try)
Summary:
This change addresses two possible instances of user error / confusion when
merging sampled profile data.

Previously any input that didn't match the raw or processed instrumented format
would automatically be interpreted as instrumented profile text format data.
No error would be reported during the merge.

Example:
If foo-sampled.profdata and bar-sampled.profdata are binary sampled profiles:

Old behavior:
$ llvm-profdata merge foo-sampled.profdata bar-sampled.profdata -output foobar-sampled.profdata
$ llvm-profdata show -sample foobar-sampled.profdata
error: foobar-sampled.profdata:1: Expected 'mangled_name:NUM:NUM', found  lprofi

This change adds basic checks for valid input data when assuming text input.
It also makes error messages related to file format validity more specific about
the assumbed profile data type.

New behavior:
$ llvm-profdata merge foo-sampled.profdata bar-sampled.profdata -o foobar-sampled.profdata
error: foo.profdata: Unrecognized instrumentation profile encoding format
Perhaps you forgot to use the -sample option?

Reviewers: bogner, davidxl, dnovillo

Subscribers: davidxl, llvm-commits

Differential Revision: http://reviews.llvm.org/D14558

llvm-svn: 253009
2015-11-13 03:47:58 +00:00
Nathan Slingerland 911ced6bf3 reverting r252916 to investigate test failure
llvm-svn: 252921
2015-11-12 18:39:26 +00:00
Nathan Slingerland f0e107e38a [llvm-profdata] Add check for text profile formats and improve error reporting
Summary:
This change addresses two possible instances of user error / confusion when
merging sampled profile data.

Previously any input that didn't match the raw or processed instrumented format
would automatically be interpreted as instrumented profile text format data.
No error would be reported during the merge.

Example:
If foo-sampled.profdata and bar-sampled.profdata are binary sampled profiles:

Old behavior:
$ llvm-profdata merge foo-sampled.profdata bar-sampled.profdata -output foobar-sampled.profdata
$ llvm-profdata show -sample foobar-sampled.profdata
error: foobar-sampled.profdata:1: Expected 'mangled_name:NUM:NUM', found  lprofi

This change adds basic checks for valid input data when assuming text input.
It also makes error messages related to file format validity more specific about
the assumbed profile data type.

New behavior:
$ llvm-profdata merge foo-sampled.profdata bar-sampled.profdata -o foobar-sampled.profdata
error: foo.profdata: Unrecognized instrumentation profile encoding format
Perhaps you forgot to use the -sample option?

Reviewers: bogner, davidxl, dnovillo

Subscribers: davidxl, llvm-commits

Differential Revision: http://reviews.llvm.org/D14558

llvm-svn: 252916
2015-11-12 18:06:18 +00:00
Diego Novillo 4b6bdb538e SamplePGO - Move FunctionSamples::print() to a better location. NFC.
The class is declared in SampleProf.h, so a better home for this is
SampleProf.cpp.

llvm-svn: 252915
2015-11-12 17:58:14 +00:00
Xinliang David Li 4c3ab815ea Fix problems in coding style
llvm-svn: 252829
2015-11-12 00:32:17 +00:00
David Blaikie 1070a09f17 unique_ptrify the AllocValueProfData helper function introduced in r252783
llvm-svn: 252799
2015-11-11 20:44:52 +00:00
Xinliang David Li 4d1bef3f76 Refactoring and fix another instance of asan error
llvm-svn: 252783
2015-11-11 19:31:53 +00:00
Aaron Ballman 470b5f1a79 Silencing a signed vs unsigned type mismatch warning.
llvm-svn: 252732
2015-11-11 14:57:28 +00:00
Xinliang David Li 6021b75a1f Fix asan warning (NFC)
llvm-svn: 252617
2015-11-10 17:11:33 +00:00
Xinliang David Li ee4158957b [PGO] Make indexed value profile data more compact
- Make indexed value profile data more compact by peeling out 
  the per-site value count field into its own smaller sized array.
- Introduced formal data structure definitions to specify value 
  profile data layout in indexed format. Previously the layout 
  of the data is only assumed in the client code (scattered in 
  three different places : size computation, EmitData, and ReadData
- The new data structure  serves as a central place for layout documentation.
- Add interfaces to force BE output for value profile data (testing purpose)
- Add byte swap unit tests

Differential Revision: http://reviews.llvm.org/D14401

llvm-svn: 252563
2015-11-10 00:24:45 +00:00
Xinliang David Li 441959d296 [PGO] Instr func name var creation code refactoring
Move the code from cfe to LLMV and become shared interfaces.
There is no functional change.

llvm-svn: 252433
2015-11-09 00:01:22 +00:00
Xinliang David Li 6aa216c21c Code style fix (caused by wrongly default clang-format style) (NFC)
llvm-svn: 252276
2015-11-06 07:54:21 +00:00
Xinliang David Li 192c748027 [PGO] Use template file to define runtime structures
With this change, instrumentation code and reader/write
code related to profile data structs are kept strictly
in-sync. THis will be extended to cfe and compile-rt 
references as well.

Differential Revision: http://reviews.llvm.org/D13843

llvm-svn: 252113
2015-11-05 00:47:26 +00:00
Xinliang David Li 2004f003b6 [PGO] Value profiling (index format) code cleanup and testing
1. Added a set of public interfaces in InstrProfRecord
    class to access (read/write) value profile data.
 2. Changed IndexedProfile reader and writer code to 
    use the newly defined interfaces and hide implementation
    details.
 3. Added a couple of unittests for value profiling:
   - Test new interfaces to get and set value profile data
   - Test value profile data merging with various scenarios.

 No functional change is expected. The new interfaces will also
 make it possible to change on-disk format of value prof data
 to be more compact (to be submitted). 

llvm-svn: 251771
2015-11-02 05:08:23 +00:00
Xinliang David Li cf4a128c6e [PGO] RawProf Reader code cleanup
Add a couple of helper methods to make the primary
raw profile reader interface's implementation more
readable. It also hides more format details. This
patch has no functional change.

llvm-svn: 251546
2015-10-28 19:34:04 +00:00
Xinliang David Li 140f4c4f29 [PGO] Indexed Prof Reader refactoring (NFC)
Change InstrProfReaderIndex from typedef into a wrapper
class with helper methods. This makes the index profile
reader code more readable. It also hides the implementation 
detail of the index format and make it more flexible to allow 
support different (or more than one) format in the future.

llvm-svn: 251491
2015-10-28 04:20:31 +00:00
Xinliang David Li 83bc4220ce Add helper functions and remove hard coded references to instProf related name/name-prefixes
This is a clean up patch that defines instr prof section and variable 
name prefixes in a common header with access helper functions. 
clang FE change will be done as a follow up once this patch is in.

Differential Revision: http://reviews.llvm.org/D13919

llvm-svn: 251058
2015-10-22 20:32:12 +00:00
Dehao Chen 100424124b Tolerate negative offset when matching sample profile.
In some cases (as illustrated in the unittest), lineno can be less than the heade_lineno because the function body are included from some other files. In this case, offset will be negative. This patch makes clang still able to match the profile to IR in this situation.

http://reviews.llvm.org/D13914

llvm-svn: 250873
2015-10-21 01:22:27 +00:00
Xinliang David Li dab183ed40 Minor Instr PGO code restructuring
1. Key constant values (version, magic) and data structures related to raw and 
   indexed profile format are moved into one centralized file: InstrProf.h.
2. Utility function such as MD5Hash computation is also moved to the common 
   header to allow sharing with other components in the future.
3. A header data structure is introduced for Indexed format so that the reader
   and writer can always be in sync.
4. Added some comments to document different places where multiple definition
   of the data structure must be kept in sync (reader/writer, runtime, lowering 
   etc).  No functional change is intended.

Differential Revision:  http://reviews.llvm.org/D13758

llvm-svn: 250638
2015-10-18 01:02:29 +00:00
Diego Novillo b93483dbce Sample profiles - Re-arrange binary format to emit head samples only on top functions.
The number of samples collected at the head of a function only make
sense for top-level functions (i.e., those actually called as opposed to
being inlined inside another).

Head samples essentially count the time spent inside the function's
prologue.  This clearly doesn't make sense for inlined functions, so we
were always emitting 0 in those.

llvm-svn: 250539
2015-10-16 18:54:35 +00:00
Diego Novillo 38be33302c Sample Profiles - Adjust integer types. Mostly NFC.
This adjusts all integers in the reader/writer to reflect the types
stored on profile files. They should all be unsigned 32-bit or 64-bit
values. Changed all associated internal types to be uint32_t or
uint64_t.

The only place that needed some adjustments is in the sample profile
transformation. Altough the weight read from the profile are 64-bit
values, the internal API for branch weights only accepts 32-bit values.
The pass now saturates weights that overflow uint32_t.

llvm-svn: 250427
2015-10-15 16:36:21 +00:00
Diego Novillo bb5605ca3a Sample profiles - Add documentation for binary profile encoding. NFC.
This adds documentation for the binary profile encoding and moves the
documentation for the text encoding into the header file
SampleProfReader.h.

llvm-svn: 250309
2015-10-14 18:36:30 +00:00
Diego Novillo 43396fa8db Sample profile reader - remove dead code. NFC.
This removes old remnants from the gcov reader. I missed these when I
re-wrote it recently.

llvm-svn: 250242
2015-10-13 22:48:48 +00:00
Diego Novillo 760c5a8f45 Sample profiles - Add a name table to the binary encoding.
Binary encoded profiles used to encode all function names inline at
every reference.  This is clearly suboptimal in terms of space.  This
patch fixes this by adding a name table to the header of the file.

llvm-svn: 250241
2015-10-13 22:48:46 +00:00
Xinliang David Li c758387e9b Fix a couple of comments; NFC
llvm-svn: 250177
2015-10-13 16:35:59 +00:00
Nathan Slingerland 5e896ce2d1 [ProfileData] Test commit for slingn
This is a test of the LLVM commit system. In the event of a real commit there would be some useful code changes.

llvm-svn: 249972
2015-10-11 13:30:56 +00:00
Diego Novillo a7f1e8ef83 Add inline stack streaming to binary sample profiles.
With this patch we can now read and write inline stacks in sample
profiles to the binary encoded profiles.

In a subsequent patch, I will add a string table to the binary encoding.
Right now function names are emitted as strings every time we find them.
This is too bloated and will produce large files in applications with
lots of inlining.

llvm-svn: 249861
2015-10-09 17:54:24 +00:00
Diego Novillo aae1ed8e08 Re-apply r249644: Handle inline stacks in gcov-encoded sample profiles.
This fixes memory allocation problems by making the merge operation keep
the profile readers around until the merged profile has been emitted.
This is needed to prevent the inlined function names to disappear from
the function profiles. Since all the names are kept as references, once
the reader disappears, the names are also deallocated.

Additionally, XFAIL on big-endian architectures. The test case uses a
gcov file generated on a little-endian system.

llvm-svn: 249724
2015-10-08 19:40:37 +00:00
Diego Novillo a082040ded Revert "Handle inline stacks in gcov-encoded sample profiles."
This reverts commit r249644.

The buildbots are failing the new test I added. Investigating.

llvm-svn: 249648
2015-10-08 01:17:26 +00:00
Diego Novillo b7fca57493 Handle inline stacks in gcov-encoded sample profiles.
This patch adds support for reading sample profiles with inline stacks.
Inline stacks in a profile are generated when the sampled binary has
samples in inlined functions.

For instance, if main() calls foo() and foo() calls bar(), and bar() is
inlined into foo() and foo() inlined into main(), the profile may look
something like:

main total:364084 head:0
  [ ... ]
  2.3: _Z3fool total:243786
    1: 60149
    1.2: 38568
    1.4: 46511
    1.7: _Z3bari total:98558
      1.1: 52672
      1.2: 45886

At line 2, discriminator 3, main() calls foo(). In turn, foo() calls
bar() at line 1, discriminator 7.

In the textual format, this stacking of inline calls is represented
with indentation.

With this change, LLVM can now read sample profile files generated by
the create_gcov tool from https://github.com/google/autofdo.

llvm-svn: 249644
2015-10-08 00:39:11 +00:00
Diego Novillo 91cbed84d9 Remove AutoFDO profile handling for GCC's LIPO. NFC.
Given the work we are doing on ThinLTO, we will never need to support
module groups and working sets in GCC's implementation of LIPO. These
are currently dead code, and will continue to be so.

llvm-svn: 249351
2015-10-05 21:08:05 +00:00
Justin Bogner 75df7187f3 InstrProf: Don't call std::unique twice here
llvm-svn: 248872
2015-09-30 02:02:08 +00:00
Dehao Chen 6722688eaa http://reviews.llvm.org/D13145
Support hierarachical sample profile format.

llvm-svn: 248865
2015-09-30 00:42:46 +00:00
Justin Bogner 9e9a057a9b InstrProf: Support for value profiling in the indexed profile format
Add support to the indexed instrprof reader and writer for the format
that will be used for value profiling.

Patch by Betul Buyukkurt, with minor modifications.

llvm-svn: 248833
2015-09-29 22:13:58 +00:00
Diego Novillo 3376a78781 GCC AutoFDO profile reader - Initial support.
This adds enough machinery to support reading simple GCC AutoFDO
profiles. It now supports reading flat profiles (no function calls).
Subsequent patches will add support for:

- Inlined calls (in particular, the inline call stack is not traversed
  to accumulate samples).

- Working sets and modules. These are used mostly for GCC's LIPO
  optimizations, so they're not needed in LLVM atm. I'm not sure that
  we will ever need them. For now, I've if0'd around the calls.

The patch also adds support in GCOV.h for gcov version V704 (generated
by GCC's profile conversion tool).

llvm-svn: 247874
2015-09-17 00:17:24 +00:00
Alexander Kornienko f00654e31b Revert r240137 (Fixed/added namespace ending comments using clang-tidy. NFC)
Apparently, the style needs to be agreed upon first.

llvm-svn: 240390
2015-06-23 09:49:53 +00:00
Justin Bogner 3a7d44cb81 Re-apply "InstrProf: When reading, copy the data instead of taking a reference. NFC"
This version fixes a missing include that MSVC noticed and
clarifies the ownership of the counter buffer that's passed to
InstrProfRecord.

This restores r240206, which was reverted in r240208.

Patch by Betul Buyukkurt.

llvm-svn: 240360
2015-06-22 23:58:05 +00:00
Frederic Riss ebc162a766 [Object] Search for architecures by name in MachOUniversalBinary::getObjectForArch()
The reason we need to search by name rather than by Triple::ArchType
is to handle subarchitecture correclty. There is no different ArchType
for the x86_64h architecture (it identifies itself as x86_64), or for
the various ARM subarches. The only way to get to the subarch slice
in an universal binary is to search by name.

This issue led to hard to debug and transient symbolication failures
in Asan tests (it mostly works, because the files are very similar).

This also affects the Profiling infrastucture as it is the other user
of that API.

Reviewers: samsonov, bogner

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D10604

llvm-svn: 240339
2015-06-22 21:33:24 +00:00
Justin Bogner fefdac1ee4 Revert "InstrProf: When reading, copy the data instead of taking a reference. NFC"
Seems like MSVC doesn't like this:

  InstrProf.h(49) : error C2614: 'llvm::InstrProfRecord' : illegal member initialization: 'Hash' is not a base or member

This reverts r240206.

llvm-svn: 240208
2015-06-20 01:37:56 +00:00
Justin Bogner 3f5eb6610c InstrProf: When reading, copy the data instead of taking a reference. NFC
This consolidates the logic to read instrprof records into the on disk
hash table's lookup trait and makes us copy the counter data instead
of taking references to it as we read. This will simplify further
changes to the format.

Patch by Betul Buyukkurt.

llvm-svn: 240206
2015-06-20 01:26:04 +00:00
Alexander Kornienko 70bc5f1398 Fixed/added namespace ending comments using clang-tidy. NFC
The patch is generated using this command:

tools/clang/tools/extra/clang-tidy/tool/run-clang-tidy.py -fix \
  -checks=-*,llvm-namespace-comment -header-filter='llvm/.*|clang/.*' \
  llvm/lib/


Thanks to Eugene Kosov for the original patch!

llvm-svn: 240137
2015-06-19 15:57:42 +00:00
Duncan P. N. Exon Smith 51149d5589 modules: Add explicit dependency on intrinsics_gen
`LLVM_ENABLE_MODULES` builds sometimes fail because `Intrinsics.td`
needs to regenerate `Instrinsics.h` before anyone can include anything
from the LLVM_IR module.  Represent the dependency explicitly to prevent
that.

llvm-svn: 239796
2015-06-16 00:44:12 +00:00
Rafael Espindola 74f293249d Don't use std::errc.
As noted on Errc.h:

// * std::errc is just marked with is_error_condition_enum. This means that
//   common patters like AnErrorCode == errc::no_such_file_or_directory take
//   4 virtual calls instead of two comparisons.

And on some libstdc++ those virtual functions conclude that

------------------------
int main() {
  std::error_code foo = std::make_error_code(std::errc::no_such_file_or_directory);
  return foo == std::errc::no_such_file_or_directory;
}
-------------------------

should exit with 0.

llvm-svn: 239683
2015-06-13 17:23:04 +00:00
Justin Bogner d49d8eec12 InstrProf: Fix reading of consecutive 32 bit coverage maps
When we generate coverage data, we explicitly set each coverage map's
alignment to 8 (See InstrProfiling::lowerCoverageData), but when we
read the coverage data, we assume consecutive maps are exactly
adjacent. When we're dealing with 32 bit, maps can end on a 4 byte
boundary, causing us to think the padding is part of the next record.

Fix this by adjusting the buffer to an appropriately aligned address
between records.

This is pretty awkward to test, as it requires a binary with multiple
coverage maps to hit, so we'd need to check in multiple source files
and a binary blob as inputs.

llvm-svn: 239129
2015-06-05 01:23:42 +00:00
Justin Bogner 82a645174a InstrProf: Treat functions with a coverage map but no profile as unreached
If we have a coverage mapping but no profile data for a function,
calling it mismatched is misleading. This can just as easily be
unreachable code that was stripped from the binary. Instead, treat
these the same as functions where we have an explicit "zero" coverage
map by setting the count to zero for each mapped region.

llvm-svn: 237298
2015-05-13 22:03:04 +00:00
Justin Bogner 5a5c381ba9 InstrProf: Simplify looking up sections for coverage data
llvm-svn: 236685
2015-05-07 00:31:58 +00:00
Justin Bogner 367a9f28c1 InstrProf: Give coverage its own errors instead of piggy backing on instrprof
Since the coverage mapping reader and the instrprof reader were
emitting a shared set of error codes, the error messages you'd get
back from llvm-cov were ambiguous about what was actually wrong. Add
another error category to fix this.

I've also improved the wording on a couple of the instrprof errors,
for consistency.

llvm-svn: 236665
2015-05-06 23:19:35 +00:00
Justin Bogner 0b13086366 InstrProf: Remove a function that just returns its argument (NFC)
llvm-svn: 236664
2015-05-06 23:15:55 +00:00
Justin Bogner 0b4c484fb9 InstrProf: Strip filename prefixes from the names we display for coverage
For consumers of coverage data, any filename prefixes we store in the
profile data are just noise. Strip this prefix if it exists.

llvm-svn: 236558
2015-05-05 23:44:48 +00:00
Benjamin Kramer 799003bf8c Re-sort includes with sort-includes.py and insert raw_ostream.h where it's used.
llvm-svn: 232998
2015-03-23 19:32:43 +00:00
Justin Bogner a438717638 InstrProf: Fix CoverageMappingReader on big endian
This makes the reader check the endianness of the object file its
given and behave appropriately. For the test I dug up a really old
linker and created a ppc-apple-darwin file for llvm-cov to read.

llvm-svn: 232422
2015-03-16 21:40:18 +00:00
Justin Bogner 7b33cc9dbc InstrProf: Do a better job of reading coverage mapping data.
This code was casting regions of a memory buffer to a couple of
different structs. This is wrong in a few ways:

1. It breaks aliasing rules.
2. If the buffer isn't aligned, it hits undefined behaviour.
3. It completely ignores endianness differences.
4. The structs being defined for this aren't specifying their padding
   properly, so this doesn't even represent the data properly on some
   platforms.

This commit is mostly NFC, except that it fixes reading coverage for
32 bit binaries as a side effect of getting rid of the mispadded
structs. I've included a test for that.

I've also baked in that we only handle little endian more explicitly,
since that was true in practice already. I'll fix this to handle
endianness properly in a followup commit.

llvm-svn: 232346
2015-03-16 06:55:45 +00:00
Justin Bogner 4379535e3f InstrProf: Teach llvm-cov to handle universal binaries when given -arch
llvm-svn: 231902
2015-03-11 02:30:51 +00:00
Justin Bogner f95ca0758c InstrProf: Allow hexadecimal function hashes in proftext format
llvm-svn: 231685
2015-03-09 18:54:49 +00:00
Benjamin Kramer 0a446fd56c Add missing includes. make_unique proliferated everywhere.
llvm-svn: 230909
2015-03-01 21:28:53 +00:00
Justin Bogner 43e51634bb InstrProf: Simplify the construction of BinaryCoverageReader
Creating BinaryCoverageReader is a strange and complicated dance where
the constructor sets error codes that member functions will later
read, and the object is in an invalid state if readHeader isn't
immediately called after construction.

Instead, make the constructor private and add a static create method
to do the construction properly. This also has the benefit of removing
readHeader completely and simplifying the interface of the object.

llvm-svn: 230676
2015-02-26 20:06:28 +00:00
Justin Bogner e84891a459 InstrProf: Rename ObjectFileCoverageMappingReader to BinaryCoverageReader
The current name is long and confusing. A shorter one is both easier
to understand and easier to work with.

llvm-svn: 230675
2015-02-26 20:06:24 +00:00
Justin Bogner 3588686baf InstrProf: Remove dead code in CoverageMappingReader
Remove a default argument that's never passed and a constructor that's
never called.

llvm-svn: 230563
2015-02-25 22:44:50 +00:00
Justin Bogner c4f5a5e863 Disallow implicit conversions from None to integer types
This fixes an error introduced in r228934 where None was converted to
an int instead of the int being converted to an Optional as intended.
We make that sort of mistake a compile error by changing NoneType into
a scoped enum.

Finally, provide a static NoneType called None to avoid forcing all
users to spell it NoneType::None.

llvm-svn: 229980
2015-02-20 07:28:28 +00:00
Justin Bogner 11ae7789ba InstrProf: Don't combine expansion regions with code regions
This was leading to duplicate counts when a code region happened to
overlap exactly with an expansion. The combining behaviour only makes
sense for code regions.

llvm-svn: 229723
2015-02-18 19:01:06 +00:00
Justin Bogner 428c605dff InstrProf: Handle unknown functions if they consist only of zero-regions
This comes up when we generate coverage for a function but don't end
up emitting the function at all - dead static functions or inline
functions that aren't referenced in a particular TU, for example. In
these cases we'd like to show that the function was never called,
which is trivially true.

llvm-svn: 229717
2015-02-18 18:40:46 +00:00
Justin Bogner 1d29c08095 InstrProf: Make CoverageMapping testable and add a basic unit test
Make CoverageMapping easier to create, so that we can write targeted
unit tests for its internals, and add a some infrastructure to write
these tests. Finally, add a simple unit test for basic functionality.

llvm-svn: 229709
2015-02-18 18:01:14 +00:00
Justin Bogner 2b6c537bdc Re-apply "InstrProf: Add unit tests for the profile reader and writer"
Have the InstrProfWriter return a MemoryBuffer instead of a
std::string. This fixes the alignment issues the reader would hit, and
it's a more appropriate type for this anyway.

I've also removed an ugly helper function that's not needed since
we're allowing initializer lists now, and updated some error code
checks based on MSVC's issues with r229473.

This reverts r229483, reapplying r229478.

llvm-svn: 229602
2015-02-18 01:58:17 +00:00
Benjamin Kramer 6cd780ff21 Prefer SmallVector::append/insert over push_back loops.
Same functionality, but hoists the vector growth out of the loop.

llvm-svn: 229500
2015-02-17 15:29:18 +00:00
Justin Bogner 5b3ad88646 Revert "InstrProf: Add unit tests for the profile reader and writer"
This added API to the InstrProfWriter to write to a string so I could
write unittests without using temp files. This doesn't really work,
since the format has tighter alignment requirements than a char.

This reverts r229478 and its follow-up, r229481.

llvm-svn: 229483
2015-02-17 09:21:43 +00:00
Justin Bogner 218d0689a9 Re-apply "InstrProf: Add unit tests for the profile reader and writer"
Add these tests again, but use va_list instead of initializer lists.

This reverts r229456, reapplying r229455.

llvm-svn: 229478
2015-02-17 07:50:59 +00:00
Justin Bogner fcb2de694a Revert "InstrProf: Add unit tests for the profile reader and writer"
Looks like the bots don't like my initializer lists.

This reverts r229455

llvm-svn: 229456
2015-02-16 23:31:07 +00:00
Justin Bogner f83e895fa7 InstrProf: Add unit tests for the profile reader and writer
This required some minor API to be added to these types to avoid
needing temp files.

Also, I've used initializer lists in the tests, as MSVC 2013 claims to
support them. I'll redo this without them if the bots complain.

llvm-svn: 229455
2015-02-16 23:27:48 +00:00
Justin Bogner ab89ed7dd5 InstrProf: Use ErrorOr for IndexedInstrProfReader::create (NFC)
The other InstrProfReader::create factories were updated to return
ErrorOr in r221120, and it's odd for these APIs not to match.

llvm-svn: 229433
2015-02-16 21:28:58 +00:00
Benjamin Kramer 40957cc2ce Fix accidental bit flip.
llvm-svn: 228936
2015-02-12 16:30:00 +00:00
Benjamin Kramer 71e1eb5ab4 CoverageMapping: Bitvectorize code. No functionality change.
llvm-svn: 228934
2015-02-12 16:18:07 +00:00
Zachary Turner 3bd47cee78 Use ADDITIONAL_HEADER_DIRS in all LLVM CMake projects.
This allows IDEs to recognize the entire set of header files for
each of the core LLVM projects.

Differential Revision: http://reviews.llvm.org/D7526
Reviewed By: Chris Bieneman

llvm-svn: 228798
2015-02-11 03:28:02 +00:00
Justin Bogner 70e0c09e6c InstrProf: Use a stable sort when reading coverage regions
Keeping regions that start at the same location in insertion order
makes this logic easier to test / more deterministic.

llvm-svn: 228083
2015-02-04 00:12:18 +00:00
Justin Bogner 26b3142d34 InstrProf: Make CounterMappingRegions less confusing to construct
Creating empty and expansion regions is awkward with the current API.
Expose static methods to make this simpler.

llvm-svn: 228075
2015-02-03 23:59:33 +00:00
Justin Bogner de15817ea2 InstrProf: Remove CoverageMapping::HasCodeBefore, it isn't used
It's not entirely clear to me what this field was meant for, but it's
always false. Remove it.

llvm-svn: 228034
2015-02-03 21:35:36 +00:00
Justin Bogner 195a4f08ea InstrProf: Simplify RawCoverageMappingReader's API slightly
This is still kind of a weird API, but dropping the (partial) update
of the passed in CoverageMappingRecord makes it a little easier to
understand and use.

llvm-svn: 227900
2015-02-03 00:20:11 +00:00
Justin Bogner 346359daac InstrProf: Simplify some logic by using ArrayRef::slice (NFC)
llvm-svn: 227898
2015-02-03 00:00:00 +00:00
Justin Bogner 3c0f124a74 llvm-cov: Only combine segments if they overlap exactly
If two coverage segments cover the same area we need to combine them,
as per r218432. OTOH, just because they start at the same place
doesn't mean they cover the same area. This fixes the check to be more
exact about this.

This is pretty hard to test right now. The frontend doesn't currently
emit regions that start at the same place but don't overlap, but some
upcoming work changes this.

llvm-svn: 227017
2015-01-24 20:58:52 +00:00
Alexander Kornienko 8c0809c7f8 Replace size method call of containers to empty method where appropriate
This patch was generated by a clang tidy checker that is being open sourced.
The documentation of that checker is the following:

/// The emptiness of a container should be checked using the empty method
/// instead of the size method. It is not guaranteed that size is a
/// constant-time function, and it is generally more efficient and also shows
/// clearer intent to use empty. Furthermore some containers may implement the
/// empty method but not implement the size method. Using empty whenever
/// possible makes it easier to switch to another container in the future.

Patch by Gábor Horváth!

llvm-svn: 226161
2015-01-15 11:41:30 +00:00
Chandler Carruth d9903888d9 [cleanup] Re-sort all the #include lines in LLVM using
utils/sort_includes.py.

I clearly haven't done this in a while, so more changed than usual. This
even uncovered a missing include from the InstrProf library that I've
added. No functionality changed here, just mechanical cleanup of the
include order.

llvm-svn: 225974
2015-01-14 11:23:27 +00:00
Justin Bogner d5fca92425 llvm-cov: Sink some reporting logic into CoverageMapping
This teaches CoverageMapping::getCoveredFunctions to filter to a
particular file and uses that to replace most of the logic found in
llvm-cov report.

llvm-svn: 221962
2014-11-14 01:50:32 +00:00
Diego Novillo fcd556074c Use ErrorOr for the ::create factory on instrumented and sample profilers.
Summary:
As discussed in
http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20141027/242445.html,
the creation of reader and writer instances is better done using
ErrorOr. There are no functional changes, but several callers needed to
be adjusted.

Reviewers: bogner

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D6076

llvm-svn: 221120
2014-11-03 00:51:45 +00:00
Diego Novillo d5336ae269 Add show and merge tools for sample PGO profiles.
Summary:
This patch extends the 'show' and 'merge' commands in llvm-profdata to handle
sample PGO formats. Using the 'merge' command it is now possible to convert
one sample PGO format to another.

The only format that is currently not working is 'gcc'. I still need to
implement support for it in lib/ProfileData.

The changes in the sample profile support classes are needed for the
merge operation.

Reviewers: bogner

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D6065

llvm-svn: 221032
2014-11-01 00:56:55 +00:00
Lang Hames f04de6ec48 [Object] Modify OwningBinary's interface to separate inspection from ownership.
The getBinary and getBuffer method now return ordinary pointers of appropriate
const-ness. Ownership is transferred by calling takeBinary(), which returns a
pair of the Binary and a MemoryBuffer.

llvm-svn: 221003
2014-10-31 21:37:49 +00:00
Diego Novillo c572e92c76 Add profile writing capabilities for sampling profiles.
Summary:
This patch finishes up support for handling sampling profiles in both
text and binary formats. The new binary format uses uleb128 encoding to
represent numeric values. This makes profiles files about 25% smaller.

The profile writer class can write profiles in the existing text and the
new binary format. In subsequent patches, I will add the capability to
read (and perhaps write) profiles in the gcov format used by GCC.

Additionally, I will be adding support in llvm-profdata to manipulate
sampling profiles.

There was a bit of refactoring needed to separate some code that was in
the reader files, but is actually common to both the reader and writer.

The new test checks that reading the same profile encoded as text or
raw, produces the same results.

Reviewers: bogner, dexonsmith

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D6000

llvm-svn: 220915
2014-10-30 18:00:06 +00:00
Benjamin Kramer 2c99e413ba Reduce double set lookups. NFC.
llvm-svn: 219505
2014-10-10 15:32:50 +00:00
Rafael Espindola 802912743e Remove bogus std::error_code returns form SectionRef.
There are two methods in SectionRef that can fail:

* getName: The index into the string table can be invalid.
* getContents: The section might point to invalid contents.

Every other method will always succeed and returning and std::error_code just
complicates the code. For example, a section can have an invalid alignment,
but if we are able to get to the section structure at all and create a
SectionRef, we will always be able to read that invalid alignment.

llvm-svn: 219314
2014-10-08 15:28:58 +00:00
Benjamin Kramer e12a6bac32 Eliminate some deep std::vector copies. NFC.
llvm-svn: 218999
2014-10-03 18:33:16 +00:00
Justin Bogner ad69e64761 InstrProf: Avoid linear search in a hot loop
Every time we were adding or removing an expression when generating a
coverage mapping we were doing a linear search to try and deduplicate
the list. The indices in the list are important, so we can't just
replace it by a DenseMap entirely, but an auxilliary DenseMap for fast
lookup massively improves the performance issues I was seeing here.

llvm-svn: 218892
2014-10-02 17:14:18 +00:00
Justin Bogner f9535c418f Reapply "InstrProf: Don't keep a large sparse list around just to zero it"
When I was preparing r218879 for commit, I removed an early return
that I decided was just noise. It wasn't. This is r218879 no-crash
edition.

This reverts commit r218881, reapplying r218879.

llvm-svn: 218887
2014-10-02 16:43:31 +00:00
Justin Bogner 70b5c562ce Revert "InstrProf: Don't keep a large sparse list around just to zero it"
This seems to be crashing on some buildbots. Reverting to investigate.

This reverts commit r218879.

llvm-svn: 218881
2014-10-02 16:15:27 +00:00
Justin Bogner d6a9e4b3be InstrProf: Don't keep a large sparse list around just to zero it
The Terms vector here represented a polynomial of of all possible
counters, and is used to simplify expressions when generating coverage
mapping. There are a few problems with this:

1. Keeping the vector as a member is wasteful, since we clear it every
   time we use it.
2. Most expressions refer to a subset of the counters, so we end up
   iterating over a large number of zeros doing nothing a lot of the
   time.

This updates the user of the vector to store the terms locally, and
uses a sort and combine approach so that we only operate on counters
that are actually used in a given expression. For small cases this
makes very little difference, but in cases with a very large number of
counted regions this is a significant performance fix.

llvm-svn: 218879
2014-10-02 16:04:03 +00:00
Justin Bogner 5eec02a399 InstrProf: Simplify counting a file's regions when writing coverage (NFC)
When writing a coverage mapping we iterate through the mapping regions
in order of FileID, but we were then repeatedly searching from the
beginning of the list to count the number of regions with a given
FileID.

It is simpler and more efficient to search forward from the current
iterator to find the number of regions.

llvm-svn: 218842
2014-10-02 00:31:00 +00:00
Alex Lorenz cb1702d45a llvm-cov: Use the number of executed functions for the function coverage metric.
This commit fixes llvm-cov's function coverage metric by using the number of executed functions instead of the number of fully covered functions.

Differential Revision: http://reviews.llvm.org/D5196

llvm-svn: 218672
2014-09-30 12:45:13 +00:00
Justin Bogner b35a72ae9e llvm-cov: Combine segments that cover the same location
If we have multiple coverage counts for the same segment, we need to
add them up rather than arbitrarily choosing one. This fixes that and
adds a test with template instantiations to exercise it.

llvm-svn: 218432
2014-09-25 00:34:18 +00:00
Justin Bogner 19a93ba814 llvm-cov: Allow creating CoverageMappings from filenames
llvm-svn: 218185
2014-09-20 17:19:52 +00:00
Justin Bogner 953e2407ed llvm-cov: Disentangle the coverage data logic from the display (NFC)
This splits the logic for actually looking up coverage information
from the logic that displays it. These were tangled rather thoroughly
so this change is a bit large, but it mostly consists of moving things
around. The coverage lookup logic itself now lives in the library,
rather than being spread between the library and the tool.

llvm-svn: 218184
2014-09-20 15:31:56 +00:00
Justin Bogner f584649ae3 llvm-cov: Move some reader debug output out of the tool.
This debug output is really for testing CoverageMappingReader, not the
llvm-cov tool. Move it to where it can be more useful.

llvm-svn: 218183
2014-09-20 15:31:51 +00:00
Chris Bieneman 1efe80109a Converting InstrProf's error_category to a ManagedStatic to avoid static constructors and destructors.
llvm-svn: 218168
2014-09-19 23:19:24 +00:00