Commit Graph

8227 Commits

Author SHA1 Message Date
Eric Beckmann 455210e18f Revert "llvm-mt: implement simple merging of manifests, not factoring namespaces."
This reverts commit 813308e240792ca70ed2f998f21df24a5061ada0.

llvm-svn: 309050
2017-07-25 23:06:46 +00:00
Eric Beckmann 780fd409fb llvm-mt: implement simple merging of manifests, not factoring namespaces.
Summary:
Does a simple merge, where mergeable elements are combined, all others
are appended.  Does not apply trickly namespace rules.

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D35753

llvm-svn: 309047
2017-07-25 22:50:25 +00:00
Petr Hosek 76fb627baf Revert "Reland "[LLVM][llvm-objcopy] Added basic plumbing to get things started""
This reverts commit 0d9d9250483761eb2f50236830161b0e2137d483.

llvm-svn: 309045
2017-07-25 22:39:52 +00:00
Petr Hosek 5e87de3e4e Reland "[LLVM][llvm-objcopy] Added basic plumbing to get things started"
As discussed on llvm-dev I've implemented the first basic steps towards
llvm-objcopy/llvm-objtool (name pending).

This change adds the ability to copy (without modification) 64-bit
little endian ELF executables that have SHT_PROGBITS, SHT_NOBITS,
SHT_NULL and SHT_STRTAB sections.

Patch by Jake Ehrlich

Differential Revision: https://reviews.llvm.org/D33964

llvm-svn: 309043
2017-07-25 22:38:08 +00:00
Petr Hosek a1ddfbb119 Revert "Reland "[LLVM][llvm-objcopy] Added basic plumbing to get things started""
This reverts commit 960873b10dd071298c817ba74ef2228f94ead7a1.

llvm-svn: 309037
2017-07-25 21:55:00 +00:00
Petr Hosek 4e0a4b3674 Reland "[LLVM][llvm-objcopy] Added basic plumbing to get things started"
As discussed on llvm-dev I've implemented the first basic steps towards
llvm-objcopy/llvm-objtool (name pending).

This change adds the ability to copy (without modification) 64-bit
little endian ELF executables that have SHT_PROGBITS, SHT_NOBITS,
SHT_NULL and SHT_STRTAB sections.

Patch by Jake Ehrlich

Differential Revision: https://reviews.llvm.org/D33964

llvm-svn: 309032
2017-07-25 21:16:33 +00:00
Teresa Johnson a83c3f7879 [LTO] Prevent dead stripping and internalization of symbols with sections
Summary:
ELF linkers generate __start_<secname> and __stop_<secname> symbols
when there is a value in a section <secname> where the name is a valid
C identifier.  If dead stripping determines that the values declared
in section <secname> are dead, and we then internalize (and delete)
such a symbol, programs that reference the corresponding start and end
section symbols will get undefined reference linking errors.

To fix this, add the section name to the IRSymtab entry when a symbol is
defined in a specific section. Then use this in the gold-plugin to mark
the symbol as external and visible from outside the summary when the
section name is a valid C identifier.

Reviewers: pcc

Subscribers: mehdi_amini, inglorion, eraman, llvm-commits

Differential Revision: https://reviews.llvm.org/D35639

llvm-svn: 309009
2017-07-25 19:42:32 +00:00
NAKAMURA Takumi 7c1c01068d llvm-rc: Fixup for r308940. This should use LLVMSupport.
llvm-svn: 308941
2017-07-25 00:51:40 +00:00
Marek Sokolowski 2ce2fa481d Add an empty shell of llvm-rc.
This starts the development on one of MS Visual Studio binutils,
Resource Converter. The tool compiles resource scripts (.rc)
into binary resource files (.res).

The current implementation does nothing but parse the command
line arguments. It is going to be extended in the future.

Differential Revision: https://reviews.llvm.org/D35810

llvm-svn: 308940
2017-07-25 00:25:18 +00:00
Reid Kleckner 898ddf61c0 [codeview] Emit 'D' as the cv source language for D code
This matches DMD:
522263965c/src/ddmd/backend/cv8.c (L199)

Fixes PR33899.

llvm-svn: 308890
2017-07-24 16:16:42 +00:00
Petr Hosek c348f9fcd5 Revert "Reland "[LLVM][llvm-objcopy] Added basic plumbing to get things started""
This reverts commit 2b52298eb28ba4d3eca113353a348c02a6ef1f93.

llvm-svn: 308822
2017-07-22 02:43:50 +00:00
Petr Hosek badc76623c Reland "[LLVM][llvm-objcopy] Added basic plumbing to get things started"
As discussed on llvm-dev I've implemented the first basic steps towards
llvm-objcopy/llvm-objtool (name pending).

This change adds the ability to copy (without modification) 64-bit
little endian ELF executables that have SHT_PROGBITS, SHT_NOBITS,
SHT_NULL and SHT_STRTAB sections.

Patch by Jake Ehrlich

Differential Revision: https://reviews.llvm.org/D33964

llvm-svn: 308821
2017-07-22 02:33:45 +00:00
Petr Hosek b13608f5e8 Revert "[LLVM][llvm-objcopy] Added basic plumbing to get things started"
This reverts commit 2f423248e140b94b8377660d4d2fe9364f30febe.

llvm-svn: 308806
2017-07-21 23:39:39 +00:00
Petr Hosek 0b746d3417 Reland "[LLVM][llvm-objcopy] Added basic plumbing to get things started"
As discussed on llvm-dev I've implemented the first basic steps towards
llvm-objcopy/llvm-objtool (name pending).

This change adds the ability to copy (without modification) 64-bit
little endian ELF executables that have SHT_PROGBITS, SHT_NOBITS,
SHT_NULL and SHT_STRTAB sections.

Patch by Jake Ehrlich

Differential Revision: https://reviews.llvm.org/D33964

llvm-svn: 308803
2017-07-21 23:27:40 +00:00
Reid Kleckner 686f121a5d [PDB] Dump extra info about the publics stream
This includes the hash table, the address map, and the thunk table and
section offset table. The last two are only used for incremental
linking, which LLD doesn't support, so they are less interesting. The
hash table is particularly important to get right, since this is the one
of the streams that debuggers use to translate addresses to symbols.

llvm-svn: 308764
2017-07-21 18:28:55 +00:00
Adrian Prantl 5f16f54af3 De-brief doxygen comments
llvm-svn: 308757
2017-07-21 16:51:17 +00:00
Adrian Prantl 6f87d1147a Use default initializers (NFC)
llvm-svn: 308756
2017-07-21 16:51:16 +00:00
Adrian Prantl 4af0a865e0 Canonicalize the spelling of RefDie (NFC)
llvm-svn: 308755
2017-07-21 16:51:09 +00:00
Adrian Prantl 98dc599790 dsymutil: strip unused types from imported DW_TAG_modules
This patch teaches dsymutil to strip types from the imported
DW_TAG_module inside of an object file (not inside the PCM) if they
can be resolved to the full definition inside the PCM. This reduces
the size of the .dSYM from WebCore from webkit.org by almost 2/3.

<rdar://problem/33047213>

llvm-svn: 308710
2017-07-21 02:07:33 +00:00
Kevin Enderby 3e95bd2239 Add error handling to the dyld compact export entries in libObject.
lld needs a matching change for this will be my next commit.
Expect it to fail build until that matching commit is picked up by the bots.

Like the changes in r296527 for dyld bind entires and the changes in
r298883 for lazy bind, weak bind and rebase entries the export
entries are the last of the dyld compact info to have error handling added.

This follows the model of iterators that can fail that Lang Hanes
designed when fixing the problem for bad archives r275316 (or r275361).

So that iterating through the exports now terminates if there is an error
and returns an llvm::Error with an error message in all cases for malformed
input.

This change provides the plumbing for the error handling, all the needed
testing of error conditions and test cases for all of the unique error messages.

llvm-svn: 308690
2017-07-20 23:08:41 +00:00
Eric Beckmann 7d50c389c4 Implement parsing and writing of a single xml manifest file.
Summary: Implement parsing and writing of a single xml manifest file.

Subscribers: mgorny, llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D35425

llvm-svn: 308679
2017-07-20 21:42:04 +00:00
Mandeep Singh Grang d41ac895bb [COFF, ARM64, CodeView] Add support to emit CodeView debug info for ARM64 COFF
Reviewers: compnerd, ruiu, rnk, zturner

Reviewed By: rnk

Subscribers: majnemer, aemerson, aprantl, javed.absar, kristof.beyls, llvm-commits

Differential Revision: https://reviews.llvm.org/D35518

llvm-svn: 308665
2017-07-20 20:20:00 +00:00
Petr Hosek d00e47fd71 Revert "[LLVM][llvm-objcopy] Added basic plumbing to get things started"
This reverts commit 98f9792e7ca5bbd9eb43bda72bf497957cfb6eb8.

llvm-svn: 308569
2017-07-20 00:13:32 +00:00
Petr Hosek 77da6f167c [LLVM][llvm-objcopy] Added basic plumbing to get things started
As discussed on llvm-dev I've implemented the first basic steps towards
llvm-objcopy/llvm-objtool (name pending).

This change adds the ability to copy (without modification) 64-bit
little endian ELF executables that have SHT_PROGBITS, SHT_NOBITS,
SHT_NULL and SHT_STRTAB sections.

Patch by Jake Ehrlich

Differential Revision: https://reviews.llvm.org/D33964

llvm-svn: 308559
2017-07-19 23:51:13 +00:00
Rafael Espindola 3ee9e11acb Remove some leftover DWARFContextInMemory.
Not sure how I missed these on the previous commit.

llvm-svn: 308550
2017-07-19 23:34:59 +00:00
Rafael Espindola c398e67fed Use delegation instead of inheritance.
This changes DwarfContext to delegate to DwarfObject instead of having
pure virtual methods.

With this DwarfContextInMemory is replaced with an implementation of
DwarfObject that is local to a .cpp file.

llvm-svn: 308543
2017-07-19 22:27:28 +00:00
Adam Nemet 817e90f11e [opt-viewer] Reduce memory consumption by another 20-25%
The Args field of the remark which consists of a list of mappings in YAML is
translated into a list of (small) dicts on Python.  An empty dict is 280 bytes
on my system so we can save memory by using a tuple of tuples instead.

Making a tuple of tuples rather than a list of tuples allows Args to be shared
with the key of the remark.  This is actually an even greater saving. (Keys
are alive throughout the entire run in all_remarks.)

Here are a few opt-stats runs with different input sizes while measuring heap
usage with heapy.  Avg remark size is simply estimated as
heap-size / # of remarks:

  | # of files             |   60 |  114 |  308 |  605 | 1370 |
  | # of remarks           |  20K |  37K | 146K | 180K | 640K |
  | total file size (MB)   |   22 |   51 |  219 |  202 | 1034 |
  |------------------------+------+------+------+------+------|
  | Avg remark size before | 4339 | 4792 | 4761 | 4096 | 4607 |
  | Avg remark size after  | 3446 | 3641 | 3567 | 3146 | 3347 |
  | Rate                   | 0.79 | 0.76 | 0.75 | 0.77 | 0.73 |

Differential Revision: https://reviews.llvm.org/D35611

llvm-svn: 308538
2017-07-19 22:04:59 +00:00
Adam Nemet 37d18f1d8a [opt-viewer] Print allocated memory per remark in opt-stats.py
If heapy is installed print the "average" in-memory remark size.  This is
estimated by dividing the total heap size by the number of unique remarks.

llvm-svn: 308537
2017-07-19 22:04:58 +00:00
Adam Nemet 1d5f5b3b59 [opt-viewer] Reduce memory consumption
The observation is that we have a lot of similar remarks with lots of
identical strings (e.g. file paths, text from the remark).  Storing a copy of
each of those strings in memory is wasteful.  This makes all the strings in
the remark interned which maintains a single immutable instance that is
referenced everywhere.

I get an average 20% heap size reduction with this but it's possible that this
varies with the typical length of the file paths used.  (I used heapy to
report the heap size.)  Runtime is same or a tiny bit better.

  | # of files            |   60 |  114 |  308 |  605 | 1370 |
  | # of remarks          |  20K |  37K | 146K | 180K | 640K |
  | total file size (MB)  |   22 |   51 |  219 |  202 | 1034 |
  |-----------------------+------+------+------+------+------|
  | Heap size before (MB) |  106 |  226 |  894 |  934 | 3573 |
  | Heap size after       |   86 |  179 |  694 |  739 | 2798 |
  | Rate                  | 0.81 | 0.79 | 0.78 | 0.79 | 0.78 |
  |-----------------------+------+------+------+------+------|
  | Average remark size   | 4.30 | 4.84 | 4.75 | 4.11 | 4.37 |
  | Mem2disk ratio        | 3.91 | 3.51 | 3.17 | 3.66 | 2.71 |

Differential Revision: https://reviews.llvm.org/D35534

llvm-svn: 308536
2017-07-19 22:04:56 +00:00
Petr Hosek eb04da3a56 [yaml2obj][ELF] Add support for program headers
This change adds basic support for program headers.

I need to do some testing which requires generating program headers but
I can't use ld.lld or clang to produce programs that have headers. I'd
also like to test some strange things that those programs may never
produce.

Patch by Jake Ehrlich

Differential Revision: https://reviews.llvm.org/D35276

llvm-svn: 308520
2017-07-19 20:38:46 +00:00
Sylvestre Ledru 2b536542e4 [Solaris] enable --whole-archive for shared-library build, disable --version-script for Solaris-ld
Shared-library build on Solaris requires --whole-archive to be specified (option accepted by all available linkers).

At the same time, --version-script can not be handled by Solaris-ld, so it should be skipped.
-M is of no use here, since there is no syntax in Solaris-ld mapfiles that allows to version all global symbols,
not just the named ones (at least this is my impression from digging deep into the docs).

Patch by Fedor Sergeev <fedor.sergeev@oracle.com>

llvm-svn: 308490
2017-07-19 16:07:51 +00:00
Petr Hosek bf5155015c [llvm-readobj] Install llvm-readelf alias
Install an llvm-readelf symlink to llvm-readobj.
When invoked as *readelf*, default to -elf-output-style=GNU.

Patch by Roland McGrath

Differential Revision: https://reviews.llvm.org/D33869

llvm-svn: 308408
2017-07-19 02:09:37 +00:00
Petr Hosek ef54b49d16 [llvm-readobj] Accept (and ignore) --wide (-W) command-line switch
Accept and ignore --wide/-W. In GNU readelf this switch is
necessary to get the output format that's consistent between
32-bit and 64-bit targets. llvm-readobj always produces that
output format.

Patch by Roland McGrath

Differential Revision: https://reviews.llvm.org/D33873

llvm-svn: 308396
2017-07-18 23:58:15 +00:00
Petr Hosek 4565de8fc3 [llvm-readobj] Accept -S as an alias for --sections
In GNU readelf, the short option for --sections is upper-case -S.

Note that GNU uses lower-case -s to mean --symbols, while LLVM
uses -s to mean --sections and -t to mean --symbols (-t has yet a
different meaning in GNU). So command-line uses with -S can now
be compatible, but uses with -s or -t are still incompatible.

Patch by Roland McGrath

Differential Revision: https://reviews.llvm.org/D33872

llvm-svn: 308392
2017-07-18 23:35:22 +00:00
Martell Malone 1079ef8dfe llvm: add llvm-dlltool support to the archiver
A PE COFF spec compliant import library generator.
Intended to be used with mingw-w64.

Supports:
PE COFF spec (section 8, Import Library Format)
PE COFF spec (Aux Format 3: Weak Externals)

Reviewed By: ruiu
Differential Revision: https://reviews.llvm.org/D29892

This reapplies rL308329, which was reverted in rL308374

llvm-svn: 308379
2017-07-18 21:26:38 +00:00
Rui Ueyama 6db83a3af3 Revert r308329: llvm: add llvm-dlltool support to the archiver
This reverts commit r308329 because it broke buildbots.

llvm-svn: 308374
2017-07-18 21:07:13 +00:00
Brian Gesiak a88699706b [opt-viewer] Handle file names that contain '#'
Summary:
When using opt-viewer.py with files with '#' in their name, such as
'foo#bar.cpp', opt-viewer.py would generate links such as
'/path/to/foo#bar.cpp.opt.yaml#L42'. In this case, the link is
interpreted by browsers as a link to the file '/path/to/foo', and to the
section within that file with ID 'bar.cpp.opt.yaml#L42'.

To work around this issue, replace '#' with '_' in file names and links
in opt-viewer.py.

Reviewers: anemet, davidxl

Reviewed By: davidxl

Subscribers: llvm-commits, fhahn

Differential Revision: https://reviews.llvm.org/D34646

llvm-svn: 308346
2017-07-18 19:25:34 +00:00
Martell Malone afe8549269 llvm: add llvm-dlltool support to the archiver
A PE COFF spec compliant import library generator.
Intended to be used with mingw-w64.

Supports:
PE COFF spec (section 8, Import Library Format)
PE COFF spec (Aux Format 3: Weak Externals)

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D29892

llvm-svn: 308329
2017-07-18 17:39:11 +00:00
Eric Beckmann d782ca3a3c Revert "Adding temporary debugging info to llvm-mt to solve fedora failure."
This reverts commit 223ef99f839f6b056272bcf4390841fcb26dda3c.

llvm-svn: 308263
2017-07-18 03:38:04 +00:00
Eric Beckmann e35d6565c0 Revert "Adding yet more debug info to fix fedora issue."
This reverts commit f3aaaac609f801df6c12655ec203455be7094627.

llvm-svn: 308262
2017-07-18 03:37:49 +00:00
Eric Beckmann 94c7487936 Revert "Adding extra test info for llvm-mt."
This reverts commit 66093fd60b848572f676023b8387bff69b151511.

llvm-svn: 308261
2017-07-18 03:37:34 +00:00
Eric Beckmann 21c7e9e346 Adding extra test info for llvm-mt.
llvm-svn: 308252
2017-07-18 01:13:10 +00:00
Reid Kleckner c50349d4c6 [PDB] Finish and simplify TPI hashing
Summary:
This removes the CVTypeVisitor updater and verifier classes. They were
made dead by the minimal type dumping refactoring. Replace them with a
single function that takes a type record and produces a hash. Call this
from the minimal type dumper and compare the hash.

I also noticed that the microsoft-pdb reference repository uses a basic
CRC32 for records that aren't special. We already have an implementation
of that CRC ready to use, because it's used in COFF for ICF.

I'll make LLD call this hashing utility in a follow-up change. We might
also consider using this same hash in type stream merging, so that we
don't have to hash our records twice.

Reviewers: inglorion, ruiu

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D35515

llvm-svn: 308240
2017-07-18 00:33:45 +00:00
Reid Kleckner 67653ee086 [codeview] Fix YAML for LF_TYPESERVER2 by hoisting PDB_UniqueId
Summary:
We were treating the GUIDs in TypeServer2Record as strings, and the
non-ASCII bytes in the GUID would not round-trip through YAML.

We already had the PDB_UniqueId type portably represent a Windows GUID,
but we need to hoist that up to the DebugInfo/CodeView library so that
we can use it in the TypeServer2Record as well as in PDB parsing code.

Reviewers: inglorion, amccarth

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D35495

llvm-svn: 308234
2017-07-17 23:59:44 +00:00
Eric Beckmann 7828c88525 Adding yet more debug info to fix fedora issue.
llvm-svn: 308232
2017-07-17 23:36:13 +00:00
Eric Beckmann 4bf5dac62c Adding temporary debugging info to llvm-mt to solve fedora failure.
llvm-svn: 308227
2017-07-17 22:46:10 +00:00
Eric Beckmann 8299850670 Create empty shell of llvm-mt.
Summary:
This is the first patch towards creating the llvm-mt tool for merging
Windows manifests.  This is a reimplementation of mt.exe.

Reviewers: zturner, ruiu, rnk

Subscribers: llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D35333

llvm-svn: 308224
2017-07-17 21:35:12 +00:00
Reid Kleckner a842cd75e2 [codeview] Remove TypeServerHandler and PDBTypeServerHandler
Summary:
Instead of wiring these through the CVTypeVisitor interface, clients
should inspect the CVTypeArray before visiting it and potentially load
up the type server's TPI stream if they need it.

No tests relied on this functionality because LLD was the only client.

Reviewers: ruiu

Subscribers: mgorny, hiraditya, zturner, llvm-commits

Differential Revision: https://reviews.llvm.org/D35394

llvm-svn: 308212
2017-07-17 20:28:06 +00:00
Adam Nemet 659d7dbb0e [opt-viewer] Accept directories that are searched for opt.yaml files
This allows to pass the build directory where all the opt.yaml files are
rather than find | xargs which may invoke opt-viewer multiple times producing
incomplete html output.

The patch generalizes the same functionality from opt-diff.

Differential Revision: https://reviews.llvm.org/D35491

llvm-svn: 308200
2017-07-17 18:00:41 +00:00
Reid Kleckner af88a910fd [CodeView] Dump BuildInfoSym and ProcSym type indices
I need to print the type index in hex so that I can match it in
FileCheck for a test I'm writing.

llvm-svn: 308107
2017-07-15 18:10:39 +00:00
George Rimar efd3ffb2b6 [llvm-readobj] - Teach readobj to print DT_FILTER dynamic tag in human readable form.
Nothing special here, output format is similar to the format
used by binutils readelf and ELF Tool Chain readelf.

Differential revision: https://reviews.llvm.org/D35351

llvm-svn: 308033
2017-07-14 16:00:16 +00:00
Adam Nemet c5bcc587ae [opt-viewer] Flush stdout after progress update
Without this, there was no progress shown during parsing but only during
rendering on macOS.

llvm-svn: 308000
2017-07-14 04:54:26 +00:00
Martin Storsjo 8c0317da02 [llvm-objdump] Properly print MachO aarch64 addend relocations
Previously such relocations fell into the last case for local
symbols, using the relocation addend as symbol index, leading to
a crash.

Differential Revision: https://reviews.llvm.org/D35239

llvm-svn: 307927
2017-07-13 17:03:02 +00:00
Hiroshi Inoue e9dea6e613 fix typos in comments and error messges; NFC
llvm-svn: 307885
2017-07-13 06:48:39 +00:00
Martin Storsjo fa5183b028 [llvm-objdump] Correctly distinguish between the MachO upper/lower16 relocations
All other code in MachODump.cpp uses the same comparison,
((r_length & 0x1) == 1), for distinguishing between the two,
while the code in llvm-objdump.cpp seemed to be incorrect.

Differential Revision: https://reviews.llvm.org/D35240

llvm-svn: 307882
2017-07-13 05:54:08 +00:00
Davide Italiano 7876f2d34c [opt-viewer] Don't except when debug info is not available.
For records without DebugLoc we simply skip the caller location in
map_remarks.

Fixes PR33764.

Patch by Simon Whittaker!

llvm-svn: 307880
2017-07-13 04:19:13 +00:00
Davide Italiano 4d6ef11e48 [sanstats] Print the correct line information.
The instrumentation tracks the return address and not that of the
call so we remove one to compensate. Thanks for Peter Collingbourne
for confirming the analysis of the problem.

llvm-svn: 307871
2017-07-13 00:49:03 +00:00
Sam Clegg 9c07f94a1f [WebAssembly] Expose the offset of each data segment
Summary:
This allows tools like lld that process relocations
to apply data relocation correctly. This information
is required because relocation are stored as section
offset.

Subscribers: jfb, dschuff, jgravelle-google, aheejin

Differential Revision: https://reviews.llvm.org/D35234

llvm-svn: 307741
2017-07-12 00:24:54 +00:00
Xinliang David Li 801b5319c5 [ProfileData] Add new option to dump topn hottest functions
Differential Revision: http://reviews.llvm.org/D35155

llvm-svn: 307702
2017-07-11 20:30:43 +00:00
Philip Pfaffe b4d500f1fb [PM] Another post-commit fix in NewPMDriver
There were two errors in the parsing of opt's command line options for
extension point pipelines. The EP callbacks are not supposed to return a
value. To check the pipeline text for correctness, I now try to parse it
into a temporary PM object, and print a message on failure. This solves
the compile time error for the lambda return type, as well as correctly
handles unparsable pipelines now.

llvm-svn: 307649
2017-07-11 11:17:44 +00:00
Hiroshi Inoue 0ca79dcf4b fix typos in comments; NFC
llvm-svn: 307626
2017-07-11 06:04:59 +00:00
Vedant Kumar 7fa7510580 [llvm-cov] Add a cl::opt to control the number of threads
When an output directory is specified, llvm-cov spawns some threads to
speed up the process of writing out file reports. Add an option which
allows users to control how many threads llvm-cov uses.

A CommandGuide.rst update + test is included.

llvm-svn: 307609
2017-07-11 01:23:29 +00:00
Andrew Wilkins 098b147347 [cmake] Check for Haiku when setting LIB_NAMES for GNU ld
Haiku uses GNU ld for linking, but is not captured in the
conditional when setting LIB_NAMES. This causes a shared
library with no symbols on Haiku. This patch simply adds
a check for whether the CMake system name is Haiku in
addition to the existing checks.

Patch by Jérôme Duval.

Differential Revision: https://reviews.llvm.org/D34998

llvm-svn: 307607
2017-07-11 01:17:44 +00:00
Sam Clegg 14612fbf25 [WebAssembly] Add more details to llvm-readobj for wasm files
Differential Revision: https://reviews.llvm.org/D35208

llvm-svn: 307588
2017-07-10 20:47:12 +00:00
Zachary Turner a9d944fd6f Resubmit "Add pdb-diff test."
This was originally reverted because of two issues.
  1) Printing ANSI color escape codes even when outputting to
     a file
  2) Module name comparisons were failing when comparing a PDB
     generated on one machine to a PDB generated on another
     machine.

I attempted to fix #2 by adding command line options which let
you specify prefixes to strip from the beginning of embedded
paths, which effectively lets us specify a path to "base" each
PDB from and only compare the parts under the base.  But this is
tricky because PDB paths always use Windows path syntax, even
when they are created on non-Windows hosts.  A problem still
existed when constructing the prefix to strip, where we were
accidentally using a host-specific path separator instead of
a Windows path separator.

This resubmission fixes the issue on Linux (and I have verified
that the test now passes on Linux).

llvm-svn: 307571
2017-07-10 19:16:49 +00:00
Zachary Turner ba3836bc78 Revert "Build fixes for pdb-diff test."
This reverts commit 180af3fdbdb17ec35b45ec1f925fd743b28d37e1.

This is still breaking due to linux-specific path differences.

llvm-svn: 307559
2017-07-10 17:32:47 +00:00
Zachary Turner e80c0bf6d7 Build fixes for pdb-diff test.
llvm-svn: 307556
2017-07-10 17:01:28 +00:00
Zachary Turner 6da7a3058e Fix pdb-diff test.
A test was checked in on Friday that worked by checking in an
object file and PDB generated locally by MSVC, and then having
the test run lld-link on the object file and diffing LLD's PDB
against the checked in PDB.

This failed because part of the diffing algorithm involves
determining if two modules are the same, and if so drilling into
the module and diffing individual fields of the module.  The
only thing we can use to make this determination though is the
"name" of the module, which is a path to where the module (obj
file) was read from on the machine where it was linked.  This
fails for obvious reasons when comparing a PDB generated on one
machine to a PDB on another machine.

The fix employed here is to add two command line options to the
diff subcommand, which allow the user to specify a "binary root
path".  The bin root path, if specified, is stripped from the
beginning of any embedded PDB paths.  The test is updated to
specify the user's local test output directory for the left
PDB, and is hardcoded to the location where the original PDB
was created for the right PDB.  This way all the equivalence
comparisons should succeed.

llvm-svn: 307555
2017-07-10 16:52:15 +00:00
Dehao Chen 396f62444d Use emplace_back to replace size() and resize().
Summary: This speeds-up thin-link for ~29% for large programs.

Reviewers: tejohnson

Reviewed By: tejohnson

Subscribers: grandinj, sanjoy, llvm-commits

Differential Revision: https://reviews.llvm.org/D35145

llvm-svn: 307543
2017-07-10 15:31:53 +00:00
Philip Pfaffe 8c84699f28 [PM] Fix a warning.
The DebugLogging argument was unused in the EP callbacks registration.

llvm-svn: 307536
2017-07-10 13:54:23 +00:00
Philip Pfaffe a3b8416880 [PM] Fix r307532: Get rid of a dangling reference.
Escaping lambda by-reference capture of local variable caused a dangling
reference.

llvm-svn: 307534
2017-07-10 12:48:51 +00:00
Philip Pfaffe 730f2f9bb6 [PM] Enable registration of out-of-tree passes with PassBuilder
Summary:
This patch adds a callback registration API to the PassBuilder,
enabling registering out-of-tree passes with it.

Through the Callback API, callers may register callbacks with the
various stages at which passes are added into pass managers, including
parsing of a pass pipeline as well as at extension points within the
default -O pipelines.

Registering utilities like `require<>` and `invalidate<>` needs to be
handled manually by the caller, but a helper is provided.

Additionally, adding passes at pipeline extension points is exposed
through the opt tool. This patch adds a `-passes-ep-X` commandline
option for every extension point X, which opt parses into pipelines
inserted into that extension point.

Reviewers: chandlerc

Reviewed By: chandlerc

Subscribers: lksbhm, grosser, davide, mehdi_amini, llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D33464

llvm-svn: 307532
2017-07-10 10:57:55 +00:00
David Blaikie 98cce00371 llvm-profdata: Reduce memory usage by using Error callback rather than member
Reduces llvm-profdata memory usage on a large profile from 7.8GB to 5.1GB.

The ProfData API now supports reporting all the errors/warnings rather
than only the first, though llvm-profdata ignores everything after the
first for now to preserve existing behavior. (if there's a desire for
other behavior, happy to implement that - but might be as well left for
a separate patch)

Reviewers: davidxl

Differential Revision: https://reviews.llvm.org/D35149

llvm-svn: 307516
2017-07-10 03:04:59 +00:00
Craig Topper 95d2347ae1 [IR] Make use of Type::isPtrOrPtrVectorTy/isIntOrIntVectorTy/isFPOrFPVectorTy to shorten code. NFC
llvm-svn: 307491
2017-07-09 07:04:00 +00:00
Eric Beckmann c8dba240b1 Revert "Revert "Revert "Revert "Switch external cvtres.exe for llvm's own resource library.""""
This reverts commit 147f45ff24456aea59575fa4ac16c8fa554df46a.

Revert "Revert "Revert "Revert "Replace trivial use of external rc.exe by writing our own .res file.""""

This reverts commit 61a90a67ed54a1f0dfeab457b65abffa129569e4.

The patches were intially reverted because they were causing a failure
on CrWinClangLLD.  Unfortunately, this was done haphazardly and didn't
compile, so the revert was reverted again quickly to fix this.  One that
was done, the revert of the revert was itself reverted.  This allowed me
to finally fix the actual bug in r307452.  This patch re-enables the
code path that had originally been causing the bug, now that it (should)
be fixed.

llvm-svn: 307460
2017-07-08 03:06:10 +00:00
Zachary Turner fe71c546e7 [llvm-pdbutil] Fix build.
Some platforms require an explicit specialization of std::hash
for PdbRaw_FeaturesSig.  Also a test involving case sensitivity
needed to be fixed.  For now that particular check just accepts
any path even if they're completely different.  Long term we
should output paths in the correct case to match MSVC.

llvm-svn: 307426
2017-07-07 19:00:06 +00:00
Zachary Turner 448dea419c Use windows path syntax when writing PDB module name.
Without this we would just append whatever the user
wrote on the command line, so if we're in C:\foo
and we run lld-link bar/baz.obj, we would write
C:\foo\bar/baz.obj in various places in the PDB.
MSVC linker does not do this, so we shouldn't either.
This fixes some differences in the diff test, so we
update the test as well.

Differential Revision: https://reviews.llvm.org/D35092

llvm-svn: 307423
2017-07-07 18:46:14 +00:00
Zachary Turner c1e93e5fa4 Fix some differences between lld and MSVC generated PDBs.
A couple of things were different about our generated PDBs.

1) We were outputting the wrong Version on the PDB Stream.
   The version we were setting was newer than what MSVC is setting.
   It's not clear what the implications are, but we change LLD
   to use PdbImplVC70, as MSVC does.
2) For the optional debug stream indices in the DBI Stream, we
   were outputting 0 to mean "the stream is not present".  MSVC
   outputs uint16_t(-1), which is the "correct" way to specify
   that a stream is not present.  So we fix that as well.
3) We were setting the PDB Stream signature to 0.  This is supposed
   to be the result of calling time(nullptr).  Although this leads
   to non-deterministic builds, a better way to solve that is by
   having a command line option explicitly for generating a
   reproducible build, and have the default behavior of lld-link
   match the default behavior of link.

To test this, I'm making use of the new and improved `pdb diff`
sub command.  To make it suitable for writing tests against, I had
to modify the diff subcommand slightly to print less verbose output.
Previously it would always print | <column> | <value1> | <value2> |
which is quite verbose, and the values are fragile.  All we really
want to know is "did we produce the same value as link?"  So I added
command line options to print a single character representing the
result status (different, identical, equivalent), and another to
hide the value display.  Note that just inspecting the diff output
used to write the test, you can see some things that are obviously
wrong.  That is just reflective of the fact that this is the state
of affairs today, not that we're asserting that this is "correct".
We can use this as a starting point to discover differences, fix
them, and update the test.

Differential Revision: https://reviews.llvm.org/D35086

llvm-svn: 307422
2017-07-07 18:45:56 +00:00
Zachary Turner f3b4b2d89d [llvm-pdbutil] Improve diff mode.
We're getting to the point that some MS tools (e.g. DIA) can recognize
our PDBs but others (e.g. link.exe) cannot. I think the way forward is
to improve our tooling to help us find differences more easily. For
example, if we can compile the same program with clang-cl and cl and
have a tool tell us all the places where the PDBs differ, this could
tell us what we're doing wrong. It's tricky though, because there are a
lot of "benign" differences in a PDB. For example, if the string table
in one PDB consists of "foo" followed by "bar" and in the other PDB it
consists of "bar" followed by "foo", this is not necessarily a critical
difference, as long as the uses of these strings also refer to the
correct location. On the other hand, if the second PDB doesn't even
contain the string "foo" at all, this is a critical difference.

diff mode has been in llvm-pdbutil for quite a while, but because of the
above challenge along with some others, it's been hard to make it
useful. I think this patch addresses that. It looks for all the same
things, but it now prints the output in tabular format (carefully
formatted and aligned into tables and fields), and it highlights
critical differences in red, non-critical differences in yellow, and
identical fields in green.  This makes it easy to spot the places we
differ, and the general concept of outputting arbitrary fields in
tabular format can be extended to provide analysis into many of the
different types of information that show up in a PDB.

Differential Revision: https://reviews.llvm.org/D35039

llvm-svn: 307421
2017-07-07 18:45:37 +00:00
Zachary Turner 6c4bfba8f3 [PDB] Teach libpdb to write DBI Stream ECNames.
Based strictly on the name, this seems to have something to do
width edit & continue.  The goal of this patch has nothing to do
with supporting edit and continue though.  msvc link.exe writes
very basic information into this area even when *not* compiling
with support for E&C, and so the goal here is to bring lld-link
to parity.  Since we cannot know what assumptions standard tools
make about the content of PDB files, we need to be as close as
possible.

This ECNames data structure is a standard PDB string hash table.
link.exe puts a single string into this hash table, which is the
full path to the PDB file on disk.  It then references this string
from the module descriptor for the compiler generated `* Linker *`
module.

With this patch, lld-link will generate the exact same sequence of
bytes as MSVC link for this subsection for a given object file
input (as reported by `llvm-pdbutil bytes -ec`).

llvm-svn: 307356
2017-07-07 05:04:36 +00:00
Lang Hames 4ce98662e7 [ORC] Errorize the ORC APIs.
This patch updates the ORC layers and utilities to return and propagate
llvm::Errors where appropriate. This is necessary to allow ORC to safely handle
error cases in cross-process and remote JITing.

llvm-svn: 307350
2017-07-07 02:59:13 +00:00
David Blaikie cf9d52c690 Prototype: Reduce llvm-profdata merge memory usage further
The InstrProfWriter already stores the name and hash of the record in
the nested maps it uses for lookup while merging - this data is
duplicated in the value within the maps.

Refactor the InstrProfRecord to use a nested struct for the counters
themselves so that InstrProfWriter can use this nested struct alone
without the name or hash duplicated there.

This work is incomplete, but enough to demonstrate the value (around a
50% decrease in memory usage for a large test case (10GB -> 5GB)).
Though most of that decrease is probably from removing the
SoftInstrProfError as well, but I haven't implemented a replacement for
it yet. (it needs to go with the counters, because the operations on the
counters - merging, etc, are where the failures are - unlike the
name/hash which are totally unused by those counter-related operations
and thus easy to split out)

Ongoing discussion about removing SoftInstrProfError as a field of the
InstrProfRecord is happening on the thread that added it - including
the possibility of moving back towards an earlier version of that
proposed patch that passed SoftInstrProfError through the various APIs,
rather than as a member of InstrProfRecord.

Reviewers: davidxl

Differential Revision: https://reviews.llvm.org/D34838

llvm-svn: 307298
2017-07-06 19:00:12 +00:00
Peter Collingbourne c855615831 Bitcode: Include any strings added to the string table in the module hash.
Differential Revision: https://reviews.llvm.org/D35037

llvm-svn: 307286
2017-07-06 17:56:01 +00:00
Adam Nemet 8d10129e59 [opt-viewer] Move under tools, install it
We weren't installing opt-viewer and co before, this fixes the omission.  I am
also moving the tools from utils/ to tools/.  I believe that this is more
appropriate since these tools have matured greatly in the past year through
contributions by multiple people (thanks!) so they are ready to become
external tools.

The tools are installed under <install>/share/opt-viewer/.

I am *not* adding the llvm- prefix.  If people feel strongly about adding
that, this is probably a good time since the new location will require some
mental adjustment anyway.

Fixes PR33521

Differential Revision: https://reviews.llvm.org/D35048

llvm-svn: 307285
2017-07-06 17:51:15 +00:00
David Blaikie fd8777ed88 Fix -Wunused-function by making function declarations in a header non-static
Also avoids ODR violations by ensuring names used in headers find the
same entity, not different, file-local entities in each translation
unit.

llvm-svn: 307237
2017-07-06 05:33:32 +00:00
Eric Beckmann f6090b620e Revert "Revert "Revert "Switch external cvtres.exe for llvm's own resource library."""
This reverts commit ae21ee0b6cacbc1efaf4d42502e71da2f0eb45c3.

The initial revert was done in order to prevent ongoing errors on
chromium bots such as CrWinClangLLD.  However, this was done haphazardly
and I didn't realize there were test and compilation failures, so this
revert was reverted.  Now that those have been fixed, we can revert the
revert of the revert.

llvm-svn: 307227
2017-07-05 23:46:06 +00:00
Zachary Turner 91dcab417c Fix std::min ambiguity between uint32 and size_t.
llvm-svn: 307205
2017-07-05 21:59:20 +00:00
Zachary Turner 8120ebf4c7 [llvm-pdbutil] Add the ability to truncate stream purpose names.
This will be useful for aligning fields to a fixed with in
subsequent patches.

llvm-svn: 307204
2017-07-05 21:54:58 +00:00
Eric Beckmann 0eafa581a3 Revert "Revert "Switch external cvtres.exe for llvm's own resource library.""
This reverts commit 165e578e47f1cd38191120aad23a9020fb5476dd.

Forgot to run tests on this.

llvm-svn: 307190
2017-07-05 19:04:33 +00:00
Eric Beckmann 36793a0ecf Revert "Switch external cvtres.exe for llvm's own resource library."
This reverts commit 600d52c278e123dd08bee24c1f00932b55add8de.

This patch still seems to break CrWinClangLLD, reverting until I can
find root problem.

llvm-svn: 307189
2017-07-05 18:59:16 +00:00
Zachary Turner eae44dfee9 [PDB] Add a test that verifies every known type record.
We had a lot of one-off tests for this type and that type,
or "every type that happens to be generated by this program
I built".  Eventually I got a bug report filed where we were
crashing on a type that was not covered by any of these tests.
So this test carefully constructs a minimal C++ program that
will cause every type we support to be emitted.  This ensures
full coverage for type records.

Differential Revision: https://reviews.llvm.org/D34915

llvm-svn: 307187
2017-07-05 18:43:25 +00:00
Alexander Kornienko 656466ed0b Fix dangling StringRefs found by clang-tidy misc-dangling-handle check.
llvm-svn: 307085
2017-07-04 15:13:02 +00:00
Lang Hames 5b51816020 [Orc] Remove the memory manager argument to addModule, and de-templatize the
symbol resolver argument.

De-templatizing the symbol resolver is part of the ongoing simplification of
ORC layer API.

Removing the memory management argument (and delegating construction of memory
managers for RTDyldObjectLinkingLayer to a functor passed in to the constructor)
allows us to build JITs whose base object layers need not be compatible with
RTDyldObjectLinkingLayer's memory mangement scheme. For example, a 'remote
object layer' that sends fully relocatable objects directly to the remote does
not need a memory management scheme at all (that will be handled by the remote).

llvm-svn: 307058
2017-07-04 04:42:30 +00:00
Brian Gesiak 4ef3daafef [ORE] Add diagnostics hotness threshold
Summary:
Add an option to prevent diagnostics that do not meet a minimum hotness
threshold from being output. When generating optimization remarks for
large codebases with a ton of cold code paths, this option can be used
to limit the optimization remark output at a reasonable size. Discussion of
this change can be read here:
http://lists.llvm.org/pipermail/llvm-dev/2017-June/114377.html

Reviewers: anemet, davidxl, hfinkel

Reviewed By: anemet

Subscribers: qcolombet, javed.absar, fhahn, eraman, llvm-commits

Differential Revision: https://reviews.llvm.org/D34867

llvm-svn: 306912
2017-06-30 23:14:53 +00:00
Zachary Turner de6a9796ea Remove spurious semicolons.
llvm-svn: 306891
2017-06-30 21:48:57 +00:00
Zachary Turner af8c75a8c0 [llvm-pdbutil] Output the symbol offset when dumping.
Type records have a unique type index, but symbol records do
not.  Instead, symbol records refer to other symbol records
by referencing their offset in the symbol stream.  In a sense
this is the analogue of the TypeIndex, but we are not printing
it in the dumper.  Printing it not only gives us more useful
information when manually investigating the contents of a PDB,
but also allows us to write better tests by enabling us to
verify that fields that reference other symbol records do
so correctly.

Differential Revision: https://reviews.llvm.org/D34906

llvm-svn: 306890
2017-06-30 21:35:00 +00:00
Richard Smith d0c0c13447 Fix ODR violations due to abuse of LLVM_YAML_IS_(FLOW_)?SEQUENCE_VECTOR
This is a short-term fix for PR33650 aimed to get the modules build bots green again.

Remove all the places where we use the LLVM_YAML_IS_(FLOW_)?SEQUENCE_VECTOR
macros to try to locally specialize a global template for a global type. That's
not how C++ works.

Instead, we now centrally define how to format vectors of fundamental types and
of string (std::string and StringRef). We use flow formatting for the former
cases, since that's the obvious right thing to do; in the latter case, it's
less clear what the right choice is, but flow formatting is really bad for some
cases (due to very long strings), so we pick block formatting. (Many of the
cases that were using flow formatting for strings are improved by this change.)

Other than the flow -> block formatting change for some vectors of strings,
this should result in no functionality change.

Differential Revision: https://reviews.llvm.org/D34907

Corresponding updates to clang, clang-tools-extra, and lld to follow.

llvm-svn: 306878
2017-06-30 20:56:57 +00:00
Zachary Turner 02a267758e [llvm-pdbutil] Add the ability to dump the dependency tree for a type
Previously we had the -type-index option which would dump the record of
a single, but we had no way to follow the dependency graph backwards and
also dump all dependent types.

Having this option makes test-writing better, because we can limit the
test to only those records that are of importance for the thing we're
trying to test, which allows us to use things like CHECK-NEXT to reduce
fragility.

Differential Revision: https://reviews.llvm.org/D34899

llvm-svn: 306852
2017-06-30 18:15:47 +00:00
Brian Gesiak 44e5f6c4ac [ORE] Unify spelling as "diagnostics hotness"
Summary:
To enable profile hotness information in diagnostics output, Clang takes
the option `-fdiagnostics-show-hotness` -- that's "diagnostics", with an
"s" at the end. Clang also defines `CodeGenOptions::DiagnosticsWithHotness`.

LLVM, on the other hand, defines
`LLVMContext::getDiagnosticHotnessRequested` -- that's "diagnostic", not
"diagnostics". It's a small difference, but it's confusing, typo-inducing, and
frustrating.

Add a new method with the spelling "diagnostics", and "deprecate" the
old spelling.

Reviewers: anemet, davidxl

Reviewed By: anemet

Subscribers: llvm-commits, mehdi_amini

Differential Revision: https://reviews.llvm.org/D34864

llvm-svn: 306848
2017-06-30 18:13:59 +00:00
Martin Storsjo 43c854535e [llvm-readobj] Improve printouts for COFF ARM64 binaries
Differential Revision: https://reviews.llvm.org/D34835

llvm-svn: 306795
2017-06-30 07:02:13 +00:00