Commit Graph

10799 Commits

Author SHA1 Message Date
Sean Fertile fd75ee9154 [Object][XCOFF] Add an XCOFF dumper for llvm-readobj.
Patch adds support for dumping of file headers with llvm-readobj. XCOFF
object files are added to test dumping a well formed file, and dumping
both negative timestamps and negative symbol counts, both of which are
allowed in the XCOFF definition.

Differential Revision: https://reviews.llvm.org/D60878

llvm-svn: 359878
2019-05-03 12:57:07 +00:00
Jordan Rupprecht 51a1418768 Revert [llvm-nm] Fix handling of symbol types + [llvm-nm] Generalize symbol types
This reverts r359311 and r359312 (git commit 0bf06a8f59 and 5f184f1780)

llvm-svn: 359830
2019-05-02 21:42:46 +00:00
Nico Weber 81862f82ee lld-link: Add /force:multipleres extension to make dupe resource diag non-fatal
As a side benefit, lld-link now reports more than one duplicate resource
entry before exiting with an error even if the new flag is not passed.

llvm-svn: 359829
2019-05-02 21:21:55 +00:00
George Rimar 366212726a [yaml2obj] - Make interface of `NameToIdxMap` class be human friendly and fix users.
This patch inverses the values returned by `addName` and
`lookup` methods of the class mentioned so that they
now return true on success and false on failure.
Also, it does minor code cleanup.

Differential revision: https://reviews.llvm.org/D61190

llvm-svn: 359818
2019-05-02 19:28:04 +00:00
James Henderson e4a89a1bee [llvm-strip]Add --no-strip-all to disable --strip-all behaviour (including default stripping)
If certain switches are not specified, llvm-strip behaves as if
--strip-all were specified. This means that for testing, when we don't
want the stripping behaviour, we have to specify one of these switches,
which can be confusing. This change adds --no-strip-all to allow an
alternative way of suppressing the default stripping, in a less
confusing manner.

Reviewed by: jakehehrlich, MaskRay

Differential Revision: https://reviews.llvm.org/D61377

llvm-svn: 359781
2019-05-02 11:53:02 +00:00
Fangrui Song facbfe0690 [llvm-readobj] Delete and inline relocAddressLess
It is used only once in COFFDumper.cpp. Deleting it from the public
interface seems better.

llvm-svn: 359775
2019-05-02 10:49:27 +00:00
Fangrui Song 8be28cdc52 [Object] Change getSectionName() to return Expected<StringRef>
Summary:
It currently receives an output parameter and returns
std::error_code. Expected<StringRef> fits for this purpose perfectly.

Differential Revision: https://reviews.llvm.org/D61421

llvm-svn: 359774
2019-05-02 10:32:03 +00:00
Fangrui Song 5387c2cd17 [llvm-objdump] Print newlines before and after "Disassembly of section ...:"
This improves readability and the behavior is consistent with GNU objdump.

The new test test/tools/llvm-objdump/X86/disassemble-section-name.s
checks we print newlines before and after "Disassembly of section ...:"

Differential Revision: https://reviews.llvm.org/D61127

llvm-svn: 359668
2019-05-01 10:40:48 +00:00
George Rimar f5345a3f4c [yaml2obj] - Report when unknown section is referenced from program header declaration block.
Previously we did not report this.
Also this removes multiple lookups in the map
what cleanups the code.

Differential revision: https://reviews.llvm.org/D61322

llvm-svn: 359663
2019-05-01 09:45:55 +00:00
Fangrui Song e29e30b139 [llvm-readobj] Change -long-option to --long-option in tests. NFC
We use both -long-option and --long-option in tests. Switch to --long-option for consistency.

In the "llvm-readelf" mode, -long-option is discouraged as it conflicts with grouped short options and it is not accepted by GNU readelf.

While updating the tests, change llvm-readobj -s to llvm-readobj -S to reduce confusion ("s" is --section-headers in llvm-readobj but --symbols in llvm-readelf).

llvm-svn: 359649
2019-05-01 05:27:20 +00:00
Fangrui Song aa1f2c50a8 [llvm-objcopy] Simplify SHT_NOBITS -> SHT_PROGBITS promotion
GNU objcopy uses bfd_elf_get_default_section_type to decide the candidate section type,
which roughly translates to our [a] (I assume SEC_COMMON implies SHF_ALLOC):

  (!(Sec.Flags & ELF::SHF_ALLOC) || Flags & (SectionFlag::SecContents | SectionFlag::SecLoad)))

Then, it updates the section type in bfd/elf.c:elf_fake_sections if:

  if (this_hdr->sh_type == SHT_NULL)
    this_hdr->sh_type = sh_type; // common case
  else if (this_hdr->sh_type == SHT_NOBITS
           && sh_type == SHT_PROGBITS
           && (asect->flags & SEC_ALLOC) != 0)  // uncommon case
    ...
    this_hdr->sh_type = sh_type;

If the following condition is met the uncommon branch is executed:

  if (elf_section_type (osec) == SHT_NULL
      && (osec->flags == isec->flags
	  || (final_link
	      && ((osec->flags ^ isec->flags)
		  & ~(SEC_LINK_ONCE | SEC_LINK_DUPLICATES | SEC_RELOC)) == 0)))

I suggest we just ignore this clause and follow the common case
behavior, which is done in this patch. Rationales to do so:

If --set-section-flags is a no-op (osec->flags == isec->flags)
(corresponds to the "readonly" test in set-section-flags.test), GNU
objcopy will require (Sec.Flags & ELF::SHF_ALLOC). [a] is essentially:

  Flags & (SectionFlag::SecContents | SectionFlag::SecLoad)

This special case is not really useful. Non-SHF_ALLOC SHT_NOBITS
sections do not make much sense and it doesn't matter if they are
SHT_NOBITS or SHT_PROGBITS.

For all other RUN lines in set-section-flags.test, the new behavior
matches GNU objcopy, i.e. this patch improves compatibility.

Differential Revision: https://reviews.llvm.org/D60189

llvm-svn: 359639
2019-05-01 00:39:31 +00:00
Rong Xu 998b97f6f1 [llvm-profdata] Add overlap command to compute similarity b/w two profile files
Add overlap functionality to llvm-profdata tool to compute the similarity
between two profile files.

Differential Revision: https://reviews.llvm.org/D60977

llvm-svn: 359612
2019-04-30 21:19:12 +00:00
Dan Gohman 3b5b9d0e72 [WebAssembly] Support EXPLICIT_NAME symbols in llvm-readobj
Teach llvm-readobj about WASM_SYMBOL_EXPLICIT_NAME.

Differential Revision: https://reviews.llvm.org/D61323

Reviewer: sbc100
llvm-svn: 359602
2019-04-30 19:30:24 +00:00
Jordan Rupprecht 96bbb1dc2b [llvm-objcopy] Add RISC-V support for -B/-O
Reviewers: jorgbrown, espindola, alexshap, jhenderson

Subscribers: emaste, arichardson, fedor.sergeev, jakehehrlich, kito-cheng, shiva0217, MaskRay, rogfer01, rkruppe, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D61272

llvm-svn: 359568
2019-04-30 15:21:36 +00:00
Eugene Leviant fd0831d0f5 [llvm-nm] Add --special-syms no-op flag
Differential revision: https://reviews.llvm.org/D60502

llvm-svn: 359563
2019-04-30 13:51:48 +00:00
Hans Wennborg 6014107702 vs integration: vs2019 support
llvm-svn: 359557
2019-04-30 12:41:33 +00:00
George Rimar 67f590e286 [llvm-objcopy] - Check dynamic relocation sections for broken references.
This is a fix for https://bugs.llvm.org/show_bug.cgi?id=41371.

Currently, it is possible to break the sh_link field of the dynamic relocation section
by removing the section it refers to. The patch fixes an issue and adds 2 test cases.

One of them shows that it does not seem possible to break the sh_info field.
I added an assert to verify this.

Differential revision: https://reviews.llvm.org/D60825

llvm-svn: 359552
2019-04-30 11:02:09 +00:00
David Bolvansky 59b6889238 Revert r359520
llvm-svn: 359544
2019-04-30 10:09:28 +00:00
David Bolvansky 1a52eaf773 [PDB] Fixed null pointer dereference
Reviewers: zturner, rnk

Reviewed By: rnk

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D61279

llvm-svn: 359520
2019-04-29 23:57:20 +00:00
Lang Hames eb14dc7585 [ORC] Replace the LLJIT/LLLazyJIT Create methods with Builder utilities.
LLJITBuilder and LLLazyJITBuilder construct LLJIT and LLLazyJIT instances
respectively. Over time these will allow more configurable options to be
added while remaining easy to use in the default case, which for default
in-process JITing is now:

auto J = ExitOnErr(LLJITBuilder.create());

llvm-svn: 359511
2019-04-29 22:37:27 +00:00
Simon Pilgrim 1303f90d49 [llvm-pdbutil] FunctionDumper::dump(PDBSymbolTypeFunctionArg) - fix null dereference warning
Reported in https://www.viva64.com/en/b/0629/

llvm-svn: 359488
2019-04-29 19:40:12 +00:00
Quentin Colombet 2d977935a2 [llvm-extract] Expose the group extraction feature of the BlockExtractor
This patch extends the `-bb` option to be able to use the group
extraction feature from the BlockExtractor.
In particular, `-bb=func:bb` is modified to support a list of basic
blocks per function: `-bb=func:bb1[;bb2...]` that will be extracted
together if at all possible (region must be single entry.)

Differential Revision: https://reviews.llvm.org/D60973

llvm-svn: 359464
2019-04-29 16:14:03 +00:00
George Rimar bc4d3c43a3 [yaml2obj] - Simplify and reduce the code. NFC.
This inlines 2 single line static methods
and simplifies the code.

It is also possible to remove the `Is64Bit`
variable since it is used only once,
but I am not sure it will be better for readability.

llvm-svn: 359445
2019-04-29 12:25:01 +00:00
George Rimar bd8bfd333f [yaml2obj] - Replace a loop with write_zeros(). NFCI.
This looks better.

llvm-svn: 359444
2019-04-29 12:05:53 +00:00
George Rimar 4828811a2d [yaml2obj] - Cleanup and simplify the code. NFCI.
The current code has the following problems:
`initSymtabSectionHeader` and `initStrtabSectionHeader` method
names saying us they are going to initialize the section headers.
Though for a few cases sh_flags field is initialized outside of them.
It does not look clean. This patch moves initialization of the
sh_flags inside these methods.

Also, it removes an excessive variable, what together with the above
change hopefully makes the code a bit more readable.

llvm-svn: 359443
2019-04-29 11:54:10 +00:00
Russell Gallop d8212edf68 vs integration: Use llvm-lib for librarian
This uses llvm-lib.exe for the librarian instead of Visual Studio
provided lib.exe. Without this it is not possible to create static
libraries with -flto using the plugin.

Original patch by Steven Noonan

This fixes: PR41147

Differential Revision: https://reviews.llvm.org/D61193

llvm-svn: 359430
2019-04-29 10:10:17 +00:00
Fangrui Song b521d1b887 [llvm-nm] Simplify and fix a buffer overflow
* char SymbolAddrStr[18] can't hold "%" PRIo64 which may need 22 characters.
* Use range-based for
* Delete unnecessary typedef
* format(...).print(Str, sizeof(Str)) + outs() << Str => outs() << format(...)
* Use cascading outs() << .. << ..
* Use iterator_range(Container &&c)
* (A & B) == B => A & B   if B is a power of 2
* replace null sentinel in constants with makeArrayRef

llvm-svn: 359416
2019-04-29 05:38:22 +00:00
Lang Hames 15c072a161 [lli] Fix a typo in a cl::opt description.
Patch by Wasiher. Thanks Wasiher!

Differential Revision: https://reviews.llvm.org/D61135

llvm-svn: 359384
2019-04-27 16:13:53 +00:00
Fangrui Song 763a2e1f36 [llvm-nm][llvm-readelf] Avoid single-dash -long-option in tests
llvm-svn: 359383
2019-04-27 16:12:14 +00:00
Fangrui Song 0969af6962 [llvm-nm] Support section type 'u': STB_GNU_UNIQUE
llvm-svn: 359380
2019-04-27 15:32:53 +00:00
Lang Hames a9fdf375b3 [ORC] Add a 'plugin' interface to ObjectLinkingLayer for events/configuration.
ObjectLinkingLayer::Plugin provides event notifications when objects are loaded,
emitted, and removed. It also provides a modifyPassConfig callback that allows
plugins to modify the JITLink pass configuration.

This patch moves eh-frame registration into its own plugin, and teaches
llvm-jitlink to only add that plugin when performing execution runs on
non-Windows platforms. This should allow us to re-enable the test case that was
removed in r359198.

llvm-svn: 359357
2019-04-26 22:58:39 +00:00
Fangrui Song 283bc74054 [llvm-nm] Revert inadvertently committed 'i' change in r359314
llvm-svn: 359315
2019-04-26 16:27:11 +00:00
Fangrui Song 5015aa854d [ThinLTO] Fix X86/strong_non_prevailing.ll after llvm-nm 'r' change
llvm-svn: 359314
2019-04-26 16:21:51 +00:00
Fangrui Song 5f184f1780 [llvm-nm] Generalize symbol types 'N', 'n' and '?'
llvm-svn: 359312
2019-04-26 16:03:31 +00:00
Fangrui Song 0bf06a8f59 [llvm-nm] Fix handling of symbol types 't' 'd' 'r'
In addition, fix and convert the two tests to yaml2obj based. This
allows us to delete two executables.

X86/weak.test: 'v' was not tested
X86/init-fini.test: symbol types of __bss_start _edata _end were wrong
  GNU nm reports __init_array_start as 't', and __preinit_array_start as 'd'.
  __init_array_start is 't' just because its section ".init_array" starts with ".init"

  'd' makes more sense and allows us to drop the weird SHT_INIT_ARRAY rule.
  So, change __init_array_start to 'd' instead.

llvm-svn: 359311
2019-04-26 16:01:48 +00:00
George Rimar 5fcdebe75f [yaml2obj] - Make implicitSectionNames() return std::vector<StringRef>. NFCI.
No need to use SmallVector of char* here.
This simplifies the code.

llvm-svn: 359301
2019-04-26 13:09:11 +00:00
George Rimar c1da14941f [yaml2obj] - Remove excessive variable. NFC.
`auto &Strtab` was used only once.

llvm-svn: 359300
2019-04-26 12:45:54 +00:00
George Rimar fb7780a41f [yaml2obj] - Make the code to match the LLVM style. NFCI.
This renames the variables to uppercase and
removes use of `auto` for unobvious type.

llvm-svn: 359298
2019-04-26 12:20:51 +00:00
George Rimar da1b3abad6 [yaml2elf] - Cleanup the initSectionHeaders(). NFCI.
This encapsulates the section specific code inside the
corresponding writeSectionContent methods.
Making the code a bit more consistent.

llvm-svn: 359297
2019-04-26 12:15:32 +00:00
Nico Weber ae73e1fcfb Minor formatting tweak, no behavior change
llvm-svn: 359295
2019-04-26 11:44:10 +00:00
Fangrui Song 2aa0bdeb25 Fix typos: (re)?sor?uce -> (re)?source
Closes: https://github.com/llvm/llvm-project/pull/10

In-collaboration-with:	Olivier Cochard-Labbé <olivier@FreeBSD.org>
Signed-off-by: Enji Cooper <yaneurabeya@gmail.com>

Differential Revision: https://reviews.llvm.org/D61021

llvm-svn: 359277
2019-04-26 05:56:23 +00:00
Fangrui Song 2db79e9d2c [llvm-objcopy] Accept --long-option but not -long-option
Summary:

llvm-{objcopy,strip} (and many other LLVM binary utilities) accept
cl::opt style -long-option as well as many short options (e.g. -p -S
-x). People who use them as replacement of GNU binutils often use the
grouped option syntax (POSIX Utility Conventions), e.g. -Sx => -S -x,
-Wd => -W -d, -sj.text => -s -j.text

There is ambiguity if a long option starts with the character used by a
short option. Drop the support for -long-option to resolve the ambiguity.

This divergence from other utilities is accepted (other utilities
continue supporting -long-option).
https://lists.llvm.org/pipermail/llvm-dev/2019-April/131786.html

Reviewers: alexshap, jakehehrlich, jhenderson, rupprecht, espindola

Reviewed By: jakehehrlich, jhenderson, rupprecht

Subscribers: grimar, emaste, arichardson, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60439

llvm-svn: 359265
2019-04-26 02:10:10 +00:00
George Rimar 45d042ed96 [yaml2obj] - Don't crash on invalid inputs.
yaml2obj might crash on invalid input when unable to parse the YAML.

Recently a crash with a very similar nature was fixed for an empty files. 
This patch revisits the fix and does it in yaml::Input instead.
It seems to be more correct way to handle such situation.

With that crash for invalid inputs is also fixed now.

Differential revision: https://reviews.llvm.org/D61059

llvm-svn: 359178
2019-04-25 09:59:55 +00:00
Lang Hames cf49aa3908 [llvm-rtdyld] Add support for passing command line arguments to rtdyld-run code.
The --args option can now be used to pass arguments to code linked with
llvm-rtdyld. E.g.

$ llvm-rtdyld file1.o file2.o --args a b c

is equivalent to:

$ ld -o program file1.o file2.o
$ ./program a b c

This is the rtdyld counterpart to the jitlink change in r359115, and makes
benchmarking and comparison between the tools easier.

llvm-svn: 359168
2019-04-25 05:02:10 +00:00
Alina Sbirlea 733c8c40c8 Enable LoopVectorization by default.
Summary:
When refactoring vectorization flags, vectorization was disabled by default in the new pass manager.
This patch re-enables is for both managers, and changes the assumptions opt makes, based on the new defaults.
Comments in opt.cpp should clarify the intended use of all flags to enable/disable vectorization.

Reviewers: chandlerc, jgorbe

Subscribers: jlebar, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D61091

llvm-svn: 359167
2019-04-25 04:49:48 +00:00
Fangrui Song 3458ff361a [llvm-objdump] errorToErrorCode+message -> toString
For test/Object/elf-invalid-phdr.test, the intended error message got lost due to errorToErrorCode().

llvm-svn: 359166
2019-04-25 04:31:26 +00:00
Nico Weber 23cb79ff93 llvm-cvtres: Make new dupe resource error a bit friendlier
For well-known type IDs, include the name of the type.

To not duplicate the ID->name map, make llvm-readobj call this new
function as well.  It has slightly different output, so this also
requires updating a few tests.

Differential Revision: https://reviews.llvm.org/D61086

llvm-svn: 359153
2019-04-24 23:26:30 +00:00
Nico Weber 1591693c7c llvm-cvtres: Remove a default argument. No behavior change.
llvm-svn: 359128
2019-04-24 19:13:38 +00:00
Lang Hames d959a609a4 [JITLink] Add support for passing arguments to jit-linked code.
The --args option can now be used to pass arguments to code linked with
llvm-jitlink. E.g.

$ llvm-jitlink file1.o file2.o --args a b c

is equivalent to:

$ ld -o program file1.o file2.o
$ ./program a b c

llvm-svn: 359115
2019-04-24 17:23:05 +00:00
Stanislav Mekhanoshin cee607e414 [AMDGPU] Add gfx1010 target definitions
Differential Revision: https://reviews.llvm.org/D61041

llvm-svn: 359113
2019-04-24 17:03:15 +00:00
JF Bastien 46d67fa6c5 Revert "[llvm-objdump] errorToErrorCode+message -> toString"
Revert r359100

It breaks llvm/test/Object/elf-invalid-phdr.test

llvm-svn: 359110
2019-04-24 16:49:30 +00:00
Lang Hames b1ba4d8a8a [JITLink] Refer to FDE's CIE (not the most recent CIE) when parsing eh-frame.
Frame Descriptor Entries (FDEs) have a pointer back to a Common Information
Entry (CIE) that describes how the rest FDE should be parsed. JITLink had been
assuming that FDEs always referred to the most recent CIE encountered, but the
spec allows them to point back to any previously encountered CIE. This patch
fixes JITLink to look up the correct CIE for the FDE.

The testcase is a MachO binary with an FDE that refers to a CIE that is not the
one immediately proceeding it (the layout can be viewed wit
'dwarfdump --eh-frame <testcase>'. This test case had to be a binary as llvm-mc
now sorts FDEs (as of r356216) to ensure FDEs *do* point to the most recent CIE.

llvm-svn: 359105
2019-04-24 15:15:55 +00:00
Fangrui Song aaecb8f799 [llvm-objdump] Delete redundant check
llvm-svn: 359102
2019-04-24 15:09:23 +00:00
George Rimar 93a47a6291 [obj2yamp] - Simplify and cleanup the code in ELFDumper<ELFT>::dumpGroup a bit. NFC.
This makes the variables naming to match LLVM style,
simplifies the code used to extract the group members,
simplifies the loop and reorders the code around a bit.

llvm-svn: 359101
2019-04-24 15:03:53 +00:00
Fangrui Song a5f8dcb63f [llvm-objdump] errorToErrorCode+message -> toString
llvm-svn: 359100
2019-04-24 15:03:46 +00:00
Fangrui Song de0462a500 [yaml2obj] Replace num_zeros with write_zeros
llvm-svn: 359091
2019-04-24 13:23:15 +00:00
George Rimar b49e192a37 [yaml2elf] - Replace a loop with write_zeros(). NFC.
And apply clang-format to the method changed.

llvm-svn: 359090
2019-04-24 13:02:15 +00:00
Fangrui Song b5f3984541 [CommandLine] Provide parser<unsigned long> instantiation to allow cl::opt<uint64_t> on LP64 platforms
Summary:
And migrate opt<unsigned long long> to opt<uint64_t>

Fixes PR19665

Differential Revision: https://reviews.llvm.org/D60933

llvm-svn: 359068
2019-04-24 02:40:20 +00:00
Nico Weber 39a2d20a0f llvm-cvtres: Accept /? as help flag, like cvtres.exe
llvm-svn: 359064
2019-04-24 02:11:24 +00:00
Adrian Prantl 2351d6102f [dsymutil] Put Swift interface files into a per-arch subdirectory.
This was meant to be part of the original commit r358921, but somehow
got lost.

<rdar://problem/49751748>

llvm-svn: 359010
2019-04-23 16:42:35 +00:00
Adrian Prantl 53bd7ce42e [dsymutil] Fix use-after-free when sys::path::append grows the buffer.
<rdar://problem/50117620>

llvm-svn: 359003
2019-04-23 15:44:22 +00:00
Adrian Prantl c7bde29cfe Revert "[dsymutil] Fix use-after-free when sys::path::append grows the buffer."
llvm-svn: 359002
2019-04-23 15:44:19 +00:00
Adrian Prantl 03e906d9d5 [dsymutil] Fix use-after-free when sys::path::append grows the buffer.
<rdar://problem/50117620>

llvm-svn: 359001
2019-04-23 15:39:13 +00:00
Adrian Prantl 05aad1567b Fully qualify llvm::Optional, some compilers complain otherwise.
llvm-svn: 358933
2019-04-22 22:51:34 +00:00
Adrian Prantl e495ec23fe Try to work around compile errors with older versions of GCC.
llvm-svn: 358927
2019-04-22 22:40:37 +00:00
Adrian Prantl 0d809aa218 [dsymutil] Collect parseable Swift interfaces in the .dSYM bundle.
When a Swift module built with debug info imports a library without
debug info from a textual interface, the textual interface is
necessary to reconstruct types defined in the library's interface. By
recording the Swift interface files in DWARF dsymutil can collect them
and LLDB can find them.

This patch teaches dsymutil to look for DW_TAG_imported_modules and
records all references to parseable Swift ingterfrace files and copies
them to

  a.out.dSYM/Contents/Resources/<Arch>/<ModuleName>.swiftinterface

<rdar://problem/49751748>

llvm-svn: 358921
2019-04-22 21:33:22 +00:00
Fangrui Song a5355a5ed1 Use llvm::stable_sort. NFC
llvm-svn: 358897
2019-04-22 15:53:43 +00:00
Nico Weber 405e62b805 Attemp get llvm-jitlink building on Windows
By removing an include of dlfcn.h that looks unused.

And clang-format a too-long line while here.

llvm-svn: 358864
2019-04-21 23:50:24 +00:00
Lang Hames bc76bbcaa0 [JITLink] Add an option to dump relocated section content.
The -dump-relocated-section-content option will dump the contents of each
section after relocations are applied, and before any checks are run or
code executed.

llvm-svn: 358863
2019-04-21 20:34:19 +00:00
Lang Hames daed9b10f1 [JITLink] Add BinaryFormat to JITLink's dependencies.
Hopefully this will fix the missing dependence on llvm::identify_magic that is
showing up on some PPC bots. E.g.

http://lab.llvm.org:8011/builders/clang-ppc64le-linux-multistage/builds/9617

llvm-svn: 358827
2019-04-20 19:48:45 +00:00
Lang Hames d9a7a7d3d0 [JITLink] Add llvm-jitlink subdirectory to tools/LLVMBuild.txt
llvm-svn: 358823
2019-04-20 17:58:29 +00:00
Lang Hames 11c8dfa583 Initial implementation of JITLink - A replacement for RuntimeDyld.
Summary:

JITLink is a jit-linker that performs the same high-level task as RuntimeDyld:
it parses relocatable object files and makes their contents runnable in a target
process.

JITLink aims to improve on RuntimeDyld in several ways:

(1) A clear design intended to maximize code-sharing while minimizing coupling.

RuntimeDyld has been developed in an ad-hoc fashion for a number of years and
this had led to intermingling of code for multiple architectures (e.g. in
RuntimeDyldELF::processRelocationRef) in a way that makes the code more
difficult to read, reason about, extend. JITLink is designed to isolate
format and architecture specific code, while still sharing generic code.

(2) Support for native code models.

RuntimeDyld required the use of large code models (where calls to external
functions are made indirectly via registers) for many of platforms due to its
restrictive model for stub generation (one "stub" per symbol). JITLink allows
arbitrary mutation of the atom graph, allowing both GOT and PLT atoms to be
added naturally.

(3) Native support for asynchronous linking.

JITLink uses asynchronous calls for symbol resolution and finalization: these
callbacks are passed a continuation function that they must call to complete the
linker's work. This allows for cleaner interoperation with the new concurrent
ORC JIT APIs, while still being easily implementable in synchronous style if
asynchrony is not needed.

To maximise sharing, the design has a hierarchy of common code:

(1) Generic atom-graph data structure and algorithms (e.g. dead stripping and
 |  memory allocation) that are intended to be shared by all architectures.
 |
 + -- (2) Shared per-format code that utilizes (1), e.g. Generic MachO to
       |  atom-graph parsing.
       |
       + -- (3) Architecture specific code that uses (1) and (2). E.g.
                JITLinkerMachO_x86_64, which adds x86-64 specific relocation
                support to (2) to build and patch up the atom graph.

To support asynchronous symbol resolution and finalization, the callbacks for
these operations take continuations as arguments:

  using JITLinkAsyncLookupContinuation =
      std::function<void(Expected<AsyncLookupResult> LR)>;

  using JITLinkAsyncLookupFunction =
      std::function<void(const DenseSet<StringRef> &Symbols,
                         JITLinkAsyncLookupContinuation LookupContinuation)>;

  using FinalizeContinuation = std::function<void(Error)>;

  virtual void finalizeAsync(FinalizeContinuation OnFinalize);

In addition to its headline features, JITLink also makes other improvements:

  - Dead stripping support: symbols that are not used (e.g. redundant ODR
    definitions) are discarded, and take up no memory in the target process
    (In contrast, RuntimeDyld supported pointer equality for weak definitions,
    but the redundant definitions stayed resident in memory).

  - Improved exception handling support. JITLink provides a much more extensive
    eh-frame parser than RuntimeDyld, and is able to correctly fix up many
    eh-frame sections that RuntimeDyld currently (silently) fails on.

  - More extensive validation and error handling throughout.

This initial patch supports linking MachO/x86-64 only. Work on support for
other architectures and formats will happen in-tree.

Differential Revision: https://reviews.llvm.org/D58704

llvm-svn: 358818
2019-04-20 17:10:34 +00:00
Fangrui Song b48e41be96 [llvm-objdump] Fix End in disassemblyObject after rL358806
llvm-svn: 358809
2019-04-20 07:48:41 +00:00
Fangrui Song ce12ea8dfc [llvm-objdump] Don't disassemble symbols before SectionAddr
This was caught by UBSAN

tools/llvm-objdump/X86/macho-disassembly-g-dsym.test
tools/llvm-objdump/X86/hex-displacement.test

llvm-svn: 358806
2019-04-20 07:19:24 +00:00
Fangrui Song 8f28f7a488 [llvm-objdump] Simplify --{start,stop}-address
llvm-svn: 358803
2019-04-20 02:10:48 +00:00
Alina Sbirlea 0499a2f961 [NewPassManager] Adding pass tuning options: loop vectorize.
Summary:
Trying to add the plumbing necessary to add tuning options to the new pass manager.
Testing with the flags for loop vectorize.

Reviewers: chandlerc

Subscribers: sanjoy, mehdi_amini, jlebar, steven_wu, dexonsmith, dang, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59723

llvm-svn: 358763
2019-04-19 16:11:59 +00:00
Fangrui Song 51873d3150 [dsymutil] DwarfLinker: delete unused parameter
llvm-svn: 358762
2019-04-19 15:45:25 +00:00
Igor Kudrin 99f641ccad [llvm-symbolizer] Add llvm-addr2line
This adds an alias for llvm-symbolizer with different defaults so that
it can be used as a drop-in replacement for GNU's addr2line.

If a substring "addr2line" is found in the tool's name:
  * it defaults "-i", "-f" and "-C" to OFF;
  * it uses "--output-style=GNU" by default.

Differential Revision: https://reviews.llvm.org/D60067

llvm-svn: 358749
2019-04-19 10:17:52 +00:00
Igor Kudrin 1b71b7f3b8 [llvm-symbolizer] Unhide and document the "-output-style" option
With the latest changes, the option gets useful for users of
llvm-symbolizer, not only for the upcoming llvm-addr2line.

Differential Revision: https://reviews.llvm.org/D60816

llvm-svn: 358748
2019-04-19 10:14:18 +00:00
Igor Kudrin 4bc29cbf6b [llvm-symbolizer] Make the output with -output-style=GNU closer to addr2line's
This patch addresses two differences in the output of llvm-symbolizer
and GNU's addr2line:

* llvm-symbolizer prints an empty line after the report for an address.

* With "-f -i=0", llvm-symbolizer replaces the name of an inlined
  function with the name from the symbol table, i. e., the top caller
  function in the inlining chain. addr2line preserves the name of the
  inlined function.

Differential Revision: https://reviews.llvm.org/D60770

llvm-svn: 358747
2019-04-19 10:12:56 +00:00
Jordan Rupprecht 2b32902a88 [llvm-objcopy] Add -B mips
llvm-svn: 358667
2019-04-18 14:22:37 +00:00
George Rimar a630b34057 [yaml2elf/obj2yaml] - Allow normal parsing/dumping of the .rela.dyn section
.rela.dyn is a section that has sh_info normally
set to zero. And Info is an optional field in the description
of the relocation section in YAML.

But currently, yaml2obj would fail to produce the object when
Info is not explicitly listed.

The patch fixes the issue.

Differential revision: https://reviews.llvm.org/D60820

llvm-svn: 358656
2019-04-18 11:02:07 +00:00
James Henderson 66a9d0f8c6 [llvm-objcopy][llvm-strip] Add switch to allow removing referenced sections
llvm-objcopy currently emits an error if a section to be removed is
referenced by another section. This is a reasonable thing to do, but is
different to GNU objcopy. We should allow users who know what they are
doing to have a way to produce the invalid ELF. This change adds a new
switch --allow-broken-links to both llvm-strip and llvm-objcopy to do
precisely that. The corresponding sh_link field is then set to 0 instead
of an error being emitted.

I cannot use llvm-readelf/readobj to test the link fields because they
emit an error if any sections, like the .dynsym, cannot be properly
loaded.

Reviewed by: rupprecht, grimar

Differential Revision: https://reviews.llvm.org/D60324

llvm-svn: 358649
2019-04-18 09:13:30 +00:00
Steven Wu 05a358cdcd [ThinLTO] Fix ThinLTOCodegenerator to export llvm.used symbols
Summary:
Reapply r357931 with fixes to ThinLTO testcases and llvm-lto tool.

ThinLTOCodeGenerator currently does not preserve llvm.used symbols and
it can internalize them. In order to pass the necessary information to the
legacy ThinLTOCodeGenerator, the input to the code generator is
rewritten to be based on lto::InputFile.

Now ThinLTO using the legacy LTO API will requires data layout in
Module.

"internalize" thinlto action in llvm-lto is updated to run both
"promote" and "internalize" with the same configuration as
ThinLTOCodeGenerator. The old "promote" + "internalize" option does not
produce the same output as ThinLTOCodeGenerator.

This fixes: PR41236
rdar://problem/49293439

Reviewers: tejohnson, pcc, kromanova, dexonsmith

Reviewed By: tejohnson

Subscribers: ormris, bd1976llvm, mehdi_amini, inglorion, eraman, hiraditya, jkorous, dexonsmith, arphaman, dang, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60421

llvm-svn: 358601
2019-04-17 17:38:09 +00:00
Fangrui Song c82e92bca8 Change some llvm::{lower,upper}_bound to llvm::bsearch. NFC
llvm-svn: 358564
2019-04-17 07:58:05 +00:00
Jordan Rupprecht b0b65cae59 [llvm-objcopy] Support full list of bfd targets that lld uses.
Summary:
This change takes the full list of bfd targets that lld supports (see `ScriptParser.cpp`), including generic handling for `*-freebsd` targets (which uses the same settings but with a FreeBSD OSABI). In particular this adds mips support for `--output-target` (but not yet via `--binary-architecture`).

lld and llvm-objcopy use their own different custom data structures, so I'd prefer to check this in as-is (add support directly in llvm-objcopy, including all the test coverage) and do a separate NFC patch(s) that consolidate the two by putting this mapping into libobject.

See [[ https://bugs.llvm.org/show_bug.cgi?id=41462 | PR41462 ]].

Reviewers: jhenderson, jakehehrlich, espindola, alexshap, arichardson

Reviewed By: arichardson

Subscribers: fedor.sergeev, emaste, sdardis, krytarowski, atanasyan, llvm-commits, MaskRay, arichardson

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60773

llvm-svn: 358562
2019-04-17 07:42:31 +00:00
Andrea Di Biagio 57cef58672 [MCA] Moved the bottleneck analysis to its own file. NFCI
llvm-svn: 358554
2019-04-17 06:02:05 +00:00
Nico Weber 3bfa6a6dad llvm-undname: Add a -raw-file flag to pass a raw buffer to microsoftDemangle
The default handling splits input into lines. Since
llvm-microsoft-demangle-fuzzer doesn't do this, oss-fuzz produces inputs
that only trigger crashes if the input isn't split into lines. This adds
a hidden flag -raw-file which passes file contents to microsoftDemangle() in
the same way the fuzzer does, for reproducing oss-fuzz reports.

Also change llvm-undname to have a non-0 exit code for invalid symbols.

Differential Revision: https://reviews.llvm.org/D60771

llvm-svn: 358485
2019-04-16 12:51:40 +00:00
Fangrui Song fa860ff733 [llvm-objdump] Align instructions to a tab stop in disassembly output
This relands D60376/rL358405, with the difference: sed 'y/\t/ /' -> tr '\t' ' '
BSD sed doesn't support escape characters for the 'y' command.
I didn't use it in rL358405 because it was not listed at
https://llvm.org/docs/GettingStarted.html#software but it
should be available.

Original description:

In GNU objdump, -w/--wide aligns instructions in the disassembly output.
This patch does the same to llvm-objdump. However, we always use the
wide format (-w/--wide is ignored), because the narrow format
(instructions are misaligned) is probably not very useful.

In llvm-readobj, we made a similar decision: always use the wide format,
accept but ignore -W/--wide.

To save some columns, we change the tab before hex bytes (controlled by
--[no-]show-raw-insn) to a space.

llvm-svn: 358474
2019-04-16 03:56:55 +00:00
Fangrui Song 051a699ed6 [llvm-objdump] Simplify PrintHelpMessage() logic
This relands rL358418. It missed one test that should also use -macho
Note, all the other -private-header -exports-trie tests are used
together with -macho.

llvm-svn: 358472
2019-04-16 02:37:29 +00:00
Alex Lorenz d9d0c3e138 Revert r358405: "[llvm-objdump] Align instructions to a tab stop in disassembly output"
The test fails on darwin due to a sed error:

sed: 1: "y/\t/ /": transform strings are not the same length
llvm-svn: 358459
2019-04-15 22:36:12 +00:00
Alex Lorenz 16256123d0 Revert r358418: "[llvm-objdump] Simplify PrintHelpMessage() logic"
This reverts commit r358418 as it broke `test/Object/objdump-export-list`
on Darwin.

llvm-svn: 358443
2019-04-15 20:16:19 +00:00
Don Hinton b85f74a283 [CommandLineParser] Add DefaultOption flag
Summary: Add DefaultOption flag to CommandLineParser which provides a
default option or alias, but allows users to override it for some
other purpose as needed.

Also, add `-h` as a default alias to `-help`, which can be seamlessly
overridden by applications like llvm-objdump and llvm-readobj which
use `-h` as an alias for other options.

(relanding after revert, r358414)
Added DefaultOptions.clear() to reset().

Reviewers: alexfh, klimek

Reviewed By: klimek

Subscribers: kristina, MaskRay, mehdi_amini, inglorion, dexonsmith, hiraditya, llvm-commits, jhenderson, arphaman, cfe-commits

Tags: #clang, #llvm

Differential Revision: https://reviews.llvm.org/D59746

llvm-svn: 358428
2019-04-15 17:18:10 +00:00
Fangrui Song 204339a234 [llvm-objdump] Simplify PrintHelpMessage() logic
llvm-svn: 358418
2019-04-15 15:52:32 +00:00
Fangrui Song 523758ef5b [llvm-objdump] Wrap things in namespace llvm
llvm-svn: 358417
2019-04-15 15:31:42 +00:00
Fangrui Song 8c1353dd14 [llvm-objdump] Delete unused forward declarations
llvm-svn: 358416
2019-04-15 15:08:01 +00:00
Fangrui Song 9d812f4499 [llvm-objdump] Reorganize cl::opt variables and move Mach-O specifics to MachODump.cpp
llvm-svn: 358415
2019-04-15 15:00:10 +00:00
Ilya Biryukov 70921d4a86 Revert r358337: "[CommandLineParser] Add DefaultOption flag"
The change causes test failures under asan. Reverting to unbreak our
integrate.

llvm-svn: 358414
2019-04-15 14:43:50 +00:00
Fangrui Song b688a200e4 [llvm-objdump] Align instructions to a tab stop in disassembly output
Summary:
In GNU objdump, -w/--wide aligns instructions in the disassembly output.
This patch does the same to llvm-objdump. However, we always use the
wide format (-w/--wide is ignored), because the narrow format
(instructions are misaligned) is probably not very useful.

In llvm-readobj, we made a similar decision: always use the wide format,
accept but ignore -W/--wide.

To save some columns, we change the tab before hex bytes (controlled by
--[no-]show-raw-insn) to a space.

Reviewers: rupprecht, jhenderson, grimar

Reviewed By: jhenderson

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60376

llvm-svn: 358405
2019-04-15 13:32:41 +00:00
Eugene Leviant 4918738c07 [llvm-readelf] Correctly dump symbols whose section id is SHN_XINDEX
Differential revision: https://reviews.llvm.org/D60614

llvm-svn: 358396
2019-04-15 11:21:47 +00:00
Stephen Tozer 19bb1d5739 [llvm-readobj] Reapply: Improve error message for --string-dump
This is a resubmission of a previous patch that caused test failures,
with the fixes for the relevant tests included.

Fixes bug 40630: https://bugs.llvm.org/show_bug.cgi?id=40630

This patch changes the error message when the section specified by
--string-dump cannot be found by including the name of the section in
the error message and changing the prefix text to not imply that the
file itself was invalid. As part of this change some uses of
std::error_code have been replaced with the llvm Error class to better
encapsulate the error info (rather than passing File strings around),
and the WithColor class replaces string literal error prefixes.

llvm-svn: 358395
2019-04-15 11:17:48 +00:00
Don Hinton 7d2021defc [CommandLineParser] Add DefaultOption flag
Summary: Add DefaultOption flag to CommandLineParser which provides a
default option or alias, but allows users to override it for some
other purpose as needed.

Also, add `-h` as a default alias to `-help`, which can be seamlessly
overridden by applications like llvm-objdump and llvm-readobj which
use `-h` as an alias for other options.

Reviewers: alexfh, klimek

Reviewed By: klimek

Subscribers: MaskRay, mehdi_amini, inglorion, dexonsmith, hiraditya, llvm-commits, jhenderson, arphaman, cfe-commits

Tags: #clang, #llvm

Differential Revision: https://reviews.llvm.org/D59746

llvm-svn: 358337
2019-04-13 16:55:28 +00:00
Philip Reames eea989a909 [StackMaps] Add location size to llvm-readobj -stackmap output
The size field of a location can be different for each entry, so it is useful to have this displayed in the output of llvm-readobj -stackmap. Below is an example of how the output would look:

Record ID: 2882400000, instruction offset: 16
   3 locations:
     #1: Constant 1, size: 8
     #2: Constant 2, size: 8
     #3: Constant 3, size: 8
   0 live-outs: [ ]

Patch By: jacob.hughes@kcl.ac.uk (with heavy modification by me)
Differential Revision: https://reviews.llvm.org/D59169

llvm-svn: 358324
2019-04-13 03:08:45 +00:00
Philip Reames f7acef9c88 [llvm-readobj] Minor style tweak for consistency sake [NFC]
llvm-svn: 358323
2019-04-13 02:23:08 +00:00
Philip Reames 377f507a9f [StackMaps] Remove format version from the class name [NFC]
Motivation is to reduce silly diffs when we change the format.  For instance, this causes most of D59020 to disappear.

llvm-svn: 358322
2019-04-13 02:02:56 +00:00
Thomas Lively fef8de66a6 [WebAssembly] Add DataCount section to object files
Summary:
This ensures that object files will continue to validate as
WebAssembly modules in the presence of bulk memory operations. Engines
that don't support bulk memory operations will not recognize the
DataCount section and will report validation errors, but that's ok
because object files aren't supposed to be run directly anyway.

Reviewers: aheejin, dschuff, sbc100

Subscribers: jgravelle-google, hiraditya, sunfish, rupprecht, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60623

llvm-svn: 358315
2019-04-12 22:27:48 +00:00
Lang Hames c7c1f21525 Simplify decoupling between RuntimeDyld/RuntimeDyldChecker, add 'got_addr' util.
This patch reduces the number of functions in the interface between RuntimeDyld
and RuntimeDyldChecker by combining "GetXAddress" and "GetXContent" functions
into "GetXInfo" functions that return a struct describing both the address and
content. The GetStubOffset function is also replaced with a pair of utilities,
GetStubInfo and GetGOTInfo, that fit the new scheme. For RuntimeDyld both of
these functions will return the same result, but for the new JITLink linker
(https://reviews.llvm.org/D58704) these will provide the addresses of PLT stubs
and GOT entries respectively.

For JITLink's use, a 'got_addr' utility has been added to the rtdyld-check
language, and the syntax of 'got_addr' and 'stub_addr' has been changed: both
functions now take two arguments, a 'stub container name' and a target symbol
name. For llvm-rtdyld/RuntimeDyld the stub container name is the object file
name and section name, separated by a slash. E.g.:

rtdyld-check: *{8}(stub_addr(foo.o/__text, y)) = y

For the upcoming llvm-jitlink utility, which creates stubs on a per-file basis
rather than a per-section basis, the container name is just the file name. E.g.:

jitlink-check: *{8}(got_addr(foo.o, y)) = y
llvm-svn: 358295
2019-04-12 18:07:28 +00:00
Eugene Leviant 88089fed9c [llvm-objcopy] Fill .symtab_shndx section correctly
Differential revision: https://reviews.llvm.org/D60555

llvm-svn: 358278
2019-04-12 11:59:30 +00:00
Xing GUO e855e2ebfc [llvm-readobj] Change variables' name to match LLVM-style. NFC.
llvm-svn: 358259
2019-04-12 07:09:41 +00:00
Ali Tamur 7822b46188 Revert "Use llvm::lower_bound. NFC"
This reverts commit rL358161.

This patch have broken the test:
llvm/test/tools/llvm-exegesis/X86/uops-CMOV16rm-noreg.s

llvm-svn: 358199
2019-04-11 17:35:20 +00:00
Serge Guelton 3742bb89f8 Make llvm-nm -help great again
Only display help from the llvm-nm category instead of all llvm options, which make it much more usable.
There's still an issue with -s, which is probably a bug in llvm::cl and worth another commit.

Differential Revision: https://reviews.llvm.org/D60411

llvm-svn: 358185
2019-04-11 15:22:48 +00:00
Fangrui Song 71cce580b9 Use llvm::lower_bound. NFC
llvm-svn: 358161
2019-04-11 10:25:41 +00:00
Roman Lebedev fbb823891d [llvm-exegesis] Fix serialization/deserialization of special NoRegister register (PR41448)
Summary:
A *lot* of instructions have this special register.
It seems this never really worked, but i finally noticed it only
because it happened to break for `CMOV16rm` instruction.

We serialized that register as "" (empty string), which is naturally
'ignored' during deserialization, so we re-create a `MCInst` with
too few operands.

And when we then happened to try to resolve variant sched class
for this mis-serialized instruction, and the variant predicate
tried to read an operand that was out of bounds since we got less operands,
we crashed.

Fixes [[ https://bugs.llvm.org/show_bug.cgi?id=41448 | PR41448 ]].

Reviewers: craig.topper, courbet

Reviewed By: courbet

Subscribers: tschuett, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60517

llvm-svn: 358153
2019-04-11 07:20:50 +00:00
Roman Lebedev 1992e8f38e [llvm-exegesis] Pacify bots - don't std::move() - prevents copy elision
llvm-svn: 358079
2019-04-10 12:47:47 +00:00
Xing GUO 8ab7414580 [llvm-readobj] Should declare `ListScope` for `verneed` entries.
Summary: YAML mappings require keys to be unique. See: https://yaml.org/spec/1.2/spec.html#id2764652

Reviewers: jhenderson, grimar, rupprecht, espindola, ruiu

Reviewed By: ruiu

Subscribers: ruiu, emaste, arichardson, MaskRay, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60438

llvm-svn: 358078
2019-04-10 12:47:21 +00:00
Roman Lebedev 41bdeb7b12 [llvm-exegesis] YamlContext: fix some missing spaces/quotes/newlines in error strings
llvm-svn: 358077
2019-04-10 12:20:14 +00:00
Roman Lebedev 628f1ae504 [llvm-exegesis] Fix error propagation from yaml writing (from serialization)
Investigating https://bugs.llvm.org/show_bug.cgi?id=41448

llvm-svn: 358076
2019-04-10 12:19:57 +00:00
Fangrui Song 7d4ad14371 [llvm-objdump] Don't print trailing space in dumpBytes
In disassembly output, dumpBytes prints a space, followed by a tab
printed by printInstr. Remove the extra space.

llvm-svn: 358045
2019-04-10 05:31:21 +00:00
Fangrui Song 5f2b5cd85e [llvm-objdump] Accept and ignore --wide/-w
This is similar to what we do for llvm-readobj (--wide/-W is for GNU
readelf compatibility).

The test will be added in D60376.

llvm-svn: 358043
2019-04-10 04:46:01 +00:00
Robert Widmann d1ba3b13f8 [LLVM-C] Add Section and Symbol Iterator Accessors for Object File Binaries
Summary: This brings us to full feature parity with the old API, so I've deprecated it and updated the tests.  I'll do a follow-up patch to do some more cleanup and documentation work in this header.

Reviewers: whitequark, deadalnix

Reviewed By: whitequark

Subscribers: hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60407

llvm-svn: 358037
2019-04-09 21:53:31 +00:00
Eric Christopher 202c9b99e0 Remove the unit at a time option
Removes the code from opt and the pass manager builder.
The code was unused - even by the C library code that was supposed to set
it and had been removed previously.

llvm-svn: 358024
2019-04-09 18:29:22 +00:00
Simon Pilgrim 563f35ab2d [llvm-rtdyld] Fix missing include on MSVC builds.
llvm-svn: 357990
2019-04-09 10:15:10 +00:00
Fangrui Song 3f2096833a [llvm-objdump] Migrate some functions from std::error_code to Error
llvm-svn: 357965
2019-04-09 05:41:24 +00:00
Lang Hames 941f247d30 [RuntimeDyld] Decouple RuntimeDyldChecker from RuntimeDyld.
This will allow RuntimeDyldChecker (and rtdyld-check tests) to test a new JIT
linker: JITLink (https://reviews.llvm.org/D58704).

llvm-svn: 357947
2019-04-08 21:50:48 +00:00
Shoaib Meenai 867131a96c [BinaryFormat] Update Mach-O ARM64E CPU subtype and dumping
The new value is taken from <mach/machine.h> in the MacOSX10.14 SDK from
Xcode 10.1. Update llvm-objdump and llvm-readobj accordingly.

Differential Revision: https://reviews.llvm.org/D58636

llvm-svn: 357945
2019-04-08 21:37:08 +00:00
Steven Wu f41e70d6eb Revert [ThinLTO] Fix ThinLTOCodegenerator to export llvm.used symbols
This reverts r357931 (git commit 8b70a5c11e)

llvm-svn: 357932
2019-04-08 18:53:21 +00:00
Steven Wu 8b70a5c11e [ThinLTO] Fix ThinLTOCodegenerator to export llvm.used symbols
Summary:
ThinLTOCodeGenerator currently does not preserve llvm.used symbols and
it can internalize them. In order to pass the necessary information to the
legacy ThinLTOCodeGenerator, the input to the code generator is
rewritten to be based on lto::InputFile.

This fixes: PR41236
rdar://problem/49293439

Reviewers: tejohnson, pcc, dexonsmith

Reviewed By: tejohnson

Subscribers: mehdi_amini, inglorion, eraman, hiraditya, jkorous, dang, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60226

llvm-svn: 357931
2019-04-08 18:24:10 +00:00
Fangrui Song f67de6c940 [llvm-objdump] Migrate relocation handling functions from error_code to Error
llvm-svn: 357920
2019-04-08 16:24:08 +00:00
Andrea Di Biagio f6a60f1f80 [llvm-mca][scheduler-stats] Print issued micro opcodes per cycle. NFCI
It makes more sense to print out the number of micro opcodes that are issued
every cycle rather than the number of instructions issued per cycle.
This behavior is also consistent with the dispatch-stats: numbers from the two
views can now be easily compared.

llvm-svn: 357919
2019-04-08 16:05:54 +00:00
Xing GUO 0df95d2d31 [llvm-readobj] Use `reinterpret_cast` instead of C-style casting. NFC.
llvm-svn: 357903
2019-04-08 11:48:36 +00:00
Roman Lebedev eb1a156d7f [llvm-exegesis] benchmarkMain(): less cryptic error if built w/o libpfm
Wanted to check if inablility to measure latency of CMOV32rm
is a regression from D60041 / D60138, but unable to do that
because the llvm-exegesis-{8,9} from debian sid fails
with that cryptic, unhelpful error.

I suspect this will be a better error.

llvm-svn: 357900
2019-04-08 10:50:31 +00:00
Roman Lebedev a82235843b [llvm-exegesis][X86] Randomize CMOVcc/SETcc OPERAND_COND_CODE CondCodes
Reviewers: courbet, gchatelet

Reviewed By: gchatelet

Subscribers: tschuett, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60066

llvm-svn: 357898
2019-04-08 10:11:00 +00:00
Fangrui Song 996b90932a [llvm-objdump] Fix MC/ARM/arm-macho-calls.s
llvm-svn: 357880
2019-04-08 01:22:38 +00:00
Fangrui Song 32087b65e7 [llvm-objdump] Split disassembleObject and simplify --{start,stop}-address handling
The main disassembly loop is hard to read due to special handling of ARM
ELF data & ELF data. Split off the logic into two functions
dumpARMELFData and dumpELFData. Hoist some checks outside of the loop.

--start-address --stop-address have redundant checks and minor off-by-1
issues. Fix them.

llvm-svn: 357869
2019-04-07 16:33:24 +00:00
Fangrui Song 47a7662e29 [llvm-objdump] Fix split of source lines; don't ltrim source lines
If the file does not end with a newline, it may be dropped. Fix the
splitting algorithm.

Also delete an unnecessary SourceCache lookup.

llvm-svn: 357858
2019-04-07 10:16:46 +00:00
Fangrui Song af7314b317 [llvm-objdump] Simplify some ELF typename: ELFFile<ELFT>::Elf_xxx -> ELFT::xxx
llvm-svn: 357857
2019-04-07 08:29:04 +00:00
Fangrui Song 454a7bb372 .
llvm-svn: 357856
2019-04-07 08:28:56 +00:00
Fangrui Song e7834bd159 [llvm-objdump] Simplify Expected<T> handling with unwrapOrError
llvm-svn: 357855
2019-04-07 08:19:55 +00:00
Fangrui Song 545ed223a6 [llvm-objdump] Simplify disassembleObject
* Use std::binary_search to replace some std::lower_bound
* Use llvm::upper_bound to replace some std::upper_bound
* Use format_hex and support::endian::read{16,32}

llvm-svn: 357853
2019-04-07 05:32:16 +00:00
Fangrui Song 6a0746a92f Change some StringRef::data() reinterpret_cast to bytes_begin() or arrayRefFromStringRef()
llvm-svn: 357852
2019-04-07 03:58:42 +00:00
Roman Lebedev 404bdb1c9e [llvm-exegesis][X86] Handle CMOVcc/SETcc OPERAND_COND_CODE OperandType
Summary:
D60041 / D60138 refactoring changed how CMOV/SETcc opcodes
are handled. concode is now an immediate, with it's own operand type.

This at least allows to not crash on the opcode.
However, this still won't generate all the snippets
with all the condcode enumerators. D60066 does that.

Reviewers: courbet, gchatelet

Reviewed By: gchatelet

Subscribers: tschuett, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60057

llvm-svn: 357841
2019-04-06 14:16:26 +00:00
Craig Topper 80aa2290fb [X86] Merge the different Jcc instructions for each condition code into single instructions that store the condition code as an operand.
Summary:
This avoids needing an isel pattern for each condition code. And it removes translation switches for converting between Jcc instructions and condition codes.

Now the printer, encoder and disassembler take care of converting the immediate. We use InstAliases to handle the assembly matching. But we print using the asm string in the instruction definition. The instruction itself is marked IsCodeGenOnly=1 to hide it from the assembly parser.

Reviewers: spatel, lebedev.ri, courbet, gchatelet, RKSimon

Reviewed By: RKSimon

Subscribers: MatzeB, qcolombet, eraman, hiraditya, arphaman, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60228

llvm-svn: 357802
2019-04-05 19:28:09 +00:00
Craig Topper 7323c2bf85 [X86] Merge the different SETcc instructions for each condition code into single instructions that store the condition code as an operand.
Summary:
This avoids needing an isel pattern for each condition code. And it removes translation switches for converting between SETcc instructions and condition codes.

Now the printer, encoder and disassembler take care of converting the immediate. We use InstAliases to handle the assembly matching. But we print using the asm string in the instruction definition. The instruction itself is marked IsCodeGenOnly=1 to hide it from the assembly parser.

Reviewers: andreadb, courbet, RKSimon, spatel, lebedev.ri

Reviewed By: andreadb

Subscribers: hiraditya, lebedev.ri, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60138

llvm-svn: 357801
2019-04-05 19:27:49 +00:00
Craig Topper e0bfeb5f24 [X86] Merge the different CMOV instructions for each condition code into single instructions that store the condition code as an immediate.
Summary:
Reorder the condition code enum to match their encodings. Move it to MC layer so it can be used by the scheduler models.

This avoids needing an isel pattern for each condition code. And it removes
translation switches for converting between CMOV instructions and condition
codes.

Now the printer, encoder and disassembler take care of converting the immediate.
We use InstAliases to handle the assembly matching. But we print using the
asm string in the instruction definition. The instruction itself is marked
IsCodeGenOnly=1 to hide it from the assembly parser.

This does complicate the scheduler models a little since we can't assign the
A and BE instructions to a separate class now.

I plan to make similar changes for SETcc and Jcc.

Reviewers: RKSimon, spatel, lebedev.ri, andreadb, courbet

Reviewed By: RKSimon

Subscribers: gchatelet, hiraditya, kristina, lebedev.ri, jdoerfert, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60041

llvm-svn: 357800
2019-04-05 19:27:41 +00:00
Stephen Tozer bbeca849d7 Revert "[llvm-readobj] Improve error message for --string-dump"
This reverts commit 681b0798db.

Reverted due to causing build failures: llvm-svn: 357772

llvm-svn: 357774
2019-04-05 16:32:25 +00:00
Stephen Tozer 681b0798db [llvm-readobj] Improve error message for --string-dump
Fixes bug 40630: https://bugs.llvm.org/show_bug.cgi?id=40630

This patch changes the error message when the section specified by
--string-dump cannot be found by including the name of the section in
the error message and changing the prefix text to not imply that the
file itself was invalid. As part of this change some uses of
std::error_code have been replaced with the llvm Error class to better
encapsulate the error info (rather than passing File strings around),
and the WithColor class replaces string literal error prefixes.

Differential Revision: https://reviews.llvm.org/D59946

llvm-svn: 357772
2019-04-05 16:15:50 +00:00
Stephen Tozer ed7f8e6509 [format] Add correct punctuation to comment
Test commit that adds a grammatically correct full stop to a single
comment.

llvm-svn: 357771
2019-04-05 15:59:07 +00:00
Guillaume Chatelet 848df5b509 Add an option do not dump the generated object on disk
Reviewers: courbet

Subscribers: llvm-commits, bdb

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60317

llvm-svn: 357769
2019-04-05 15:18:59 +00:00
David Callahan f498bdcebf Include invoke'd functions for recursive extract
Summary: When recursively extracting a function from a bit code file, include functions mentioned in InvokeInst as well as CallInst

Reviewers: loladiro, espindola, volkan

Reviewed By: loladiro

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60231

llvm-svn: 357735
2019-04-04 23:30:47 +00:00
Adrian Prantl ce2d45e7ba llvm-dwarfdump: Support alternative architecture names in the -arch filter
<rdar://problem/47918606>

llvm-svn: 357706
2019-04-04 15:48:40 +00:00
Igor Kudrin 734a2bc737 [llvm-symbolizer] Allow more flexible usage of -e.
addr2line allows -e to be grouped with other options; it also allows it
to prefix the value. Thus, all the following usages are possible:

  * addr2line -f -e <bin> <addr>
  * addr2line -fe <bin> <addr>
  * addr2line -f e<bin> <addr>
  * addr2line -fe<bin> <addr>

This patch adds the same for llvm-symbolizer.

Differential Revision: https://reviews.llvm.org/D60196

llvm-svn: 357676
2019-04-04 08:45:06 +00:00
Igor Kudrin 0fed7b0564 [llvm-symbolizer] Add `--output-style` switch.
In general, llvm-symbolizer follows the output style of GNU's addr2line.
However, there are still some differences; in particular, for a requested
address, llvm-symbolizer prints line and column, while addr2line prints
only the line number.

This patch adds a new switch to select the preferred style.

Differential Revision: https://reviews.llvm.org/D60190

llvm-svn: 357675
2019-04-04 08:39:40 +00:00
Hubert Tong ab2eb2bfac [XCOFF] Add functionality for parsing AIX XCOFF object file headers
Summary:
1. Add functionality for parsing AIX XCOFF object files headers.
2. Only support 32-bit AIX XCOFF object files in this patch.
3. Print out the AIX XCOFF object file header in YAML format.

Reviewers: sfertile, hubert.reinterpretcast, jasonliu, mstorsjo, zturner, rnk

Reviewed By: sfertile, hubert.reinterpretcast

Subscribers: jsji, mgorny, hiraditya, jdoerfert, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59419

Patch by Digger Lin

llvm-svn: 357663
2019-04-04 00:53:21 +00:00
Reid Kleckner e10d00419a [codeview] Remove Type member from CVRecord
Summary:
Now CVType and CVSymbol are effectively type-safe wrappers around
ArrayRef<uint8_t>. Make the kind() accessor load it from the
RecordPrefix, which is the same for types and symbols.

Reviewers: zturner, aganea

Subscribers: hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60018

llvm-svn: 357658
2019-04-04 00:28:48 +00:00
George Rimar 6da44ad75d [yaml2obj][obj2yaml] - Change how symbol's binding is descibed when parsing/dumping.
Currently, YAML has the following syntax for describing the symbols:

Symbols:
  Local:
    LocalSymbol1:
    ...
    LocalSymbol2:
    ...
  ...
  Global:
    GlobalSymbol1:
  ...
  Weak:
  ...
  GNUUnique:

I.e. symbols are grouped by their bindings. That is not very convenient,
because:

It does not allow to set a custom binding, what can be useful for producing
broken/special outputs for test cases. Adding a new binding would require to
change a syntax (what we observed when added GNUUnique recently).

It does not allow to change the order of the symbols in .symtab/.dynsym,
i.e. currently all Local symbols are placed first, then Global, Weak and GNUUnique
are following, but we are not able to change the order.

It is not consistent. Binding is just one of the properties of the symbol,
we do not group them by other properties.

It makes the code more complex that it can be. This patch shows it can be simplified
with the change performed.

The patch changes the syntax to just:

Symbols:
  Symbol1:
  ...
  Symbol2:
  ...
...

With that, we are able to work with the binding field just like with any other symbol property.

Differential revision: https://reviews.llvm.org/D60122

llvm-svn: 357595
2019-04-03 14:53:42 +00:00
James Henderson d931cf3e46 [llvm-objcopy] Make section rename/set flags case-insensitive
This fixes https://bugs.llvm.org/show_bug.cgi?id=41305. GNU objcopy
--set-section-flags/--rename-section flags are case-insensitive, so this
patch updates llvm-objcopy to match.

Reviewed by: grimar

Differential Revision: https://reviews.llvm.org/D60200

llvm-svn: 357590
2019-04-03 14:40:27 +00:00
Xing GUO 8f6166a72e [llvm-readobj] Add GNU style dumper for .gnu.version section
Summary: Currently, `llvm-readobj` do not support GNU style dumper for symbol versioning sections. In this patch, I would like to implement dumper for `.gnu.version` section

Reviewers: jhenderson, rupprecht, grimar

Reviewed By: jhenderson, rupprecht

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59877

llvm-svn: 357578
2019-04-03 13:32:49 +00:00
James Henderson ef93be84d3 [llvm-nm]Add support for --no-demangle
GNU nm has --no-demangle, so llvm-nm should too. It disables the
--demangle switch. The patch also allows --demangle to be specified
multiple times (the last of all --no-demangle/--demangle switches
takes precedence).

Reviewed by: grimar, rupprecht, mattd

Differential Revision: https://reviews.llvm.org/D60134

llvm-svn: 357575
2019-04-03 12:57:46 +00:00
Jordan Rupprecht 017deaf1ae [llvm-objcopy] Change SHT_NOBITS to SHT_PROBITS for some --set-section-flags
Summary:
Some flags accepted by --set-section-flags and --rename-section can change a SHT_NOBITS section to a SHT_PROGBITS section. Note that none of them can change a SHT_PROGBITS to SHT_NOBITS.

The full list (found via experimentation of individually setting each flag) that does this is: contents, load, noload, code, data, rom, and debug.

This was found by testing llvm-objcopy with the gnu binutils test suite, specifically this test case: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=blob;f=binutils/testsuite/binutils-all/copy-1.d;h=f2b0d9e90df738c2891b4d5c7b62f62894b556ca;hb=HEAD

Reviewers: jhenderson, grimar, jakehehrlich, alexshap, espindola

Reviewed By: jhenderson

Subscribers: emaste, arichardson, MaskRay, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59958

llvm-svn: 357492
2019-04-02 16:49:56 +00:00
Shoaib Meenai 7c16c5b6ae [llvm-shlib] Fix cross-compilation for LLVM-C
When we're cross-compiling, build and use a native llvm-nm instead of
attempting to use the one from the target's build tree.

A nice follow-up would be to add a cache variable to allow specifying a
path to an external native llvm-nm instead of building one ourselves,
similar to LLVM_TABLEGEN and LLVM_CONFIG_PATH.

Differential Revision: https://reviews.llvm.org/D60025

llvm-svn: 357487
2019-04-02 15:58:05 +00:00
Shoaib Meenai f92acf1299 [cmake] Add function for building native tool
Instead of duplicating functionality for building native versions of
tblgen and llvm-config, add a function to set up a native tool build.
This will also be used for llvm-nm in a follow-up.

This should be NFC for tblgen, besides the slightly different COMMENT
for the custom command (it'll display the tablegen target name instead
of always saying TableGen).  For the native llvm-config, it's a behavior
change in that we'll use llvm_ExternalProject_BuildCmd instead of
constructing the build command manually, always build in Release, and
reference the correct binary path for multi-config generators. I believe
all of these changes to be bug fixes.

Differential Revision: https://reviews.llvm.org/D60024

llvm-svn: 357486
2019-04-02 15:58:03 +00:00
James Henderson 38cb238f75 [llvm-objcopy]Allow llvm-objcopy to be used on an ELF file with no section headers
This patch fixes https://bugs.llvm.org/show_bug.cgi?id=41293 and
https://bugs.llvm.org/show_bug.cgi?id=41045. llvm-objcopy assumed that
it could always read a section header string table. This isn't the case
when the sections were previously all stripped, and the e_shstrndx field
was set to 0. This patch fixes this. It also fixes a double space in an
error message relating to this issue, and prevents llvm-objcopy from
adding extra space for non-existent section headers, meaning that
--strip-sections on the output of a previous --strip-sections run
produces identical output, simplifying the test.

Reviewed by: rupprecht, grimar

Differential Revision: https://reviews.llvm.org/D59989

llvm-svn: 357475
2019-04-02 14:11:13 +00:00
Pavel Labath 3cee663e71 Add minidump support to obj2yaml
Summary:
This patch adds the code needed to parse a minidump file into the
MinidumpYAML model, and the necessary glue code so that obj2yaml can
recognise the minidump files and process them.

Reviewers: jhenderson, zturner, clayborg

Subscribers: mgorny, lldb-commits, amccarth, markmentovai, aprantl, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59634

llvm-svn: 357469
2019-04-02 11:58:37 +00:00
Yi Kong f2baddb0fc [llvm-objcopy] Add --keep-symbols option
Differential Revision: https://reviews.llvm.org/D60054

llvm-svn: 357418
2019-04-01 18:12:43 +00:00
Liang Zou 9f4a4d3974 fix typo: "\t" => " "
Reviewers: llvm.org, Jim

Reviewed By: Jim

Subscribers: arsenm, jvesely, nhaehnle, rupprecht, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59983

llvm-svn: 357365
2019-03-31 14:49:00 +00:00
Fangrui Song 7dd1c36cd7 [cmake] Change deprecated $<CONFIG> to $<CONFIGURATION>. NFC
See rL357338 for a similar change. The informational expression
$<CONFIGURATION> has been deprecated since CMake 3.0

llvm-svn: 357348
2019-03-30 14:38:51 +00:00
Fangrui Song 82b01e002e [llvm-objcopy] Replace the size() helper with SectionTableRef::size
Summary:
BTW, STLExtras.h provides llvm::size() which is similar to std::size()
for random access iterators. However, if we prefer qualified
llvm::size(), the member function .size() will be more convenient.

Reviewers: jhenderson, jakehehrlich, rupprecht, grimar, alexshap, espindola

Reviewed By: grimar

Subscribers: emaste, arichardson, jdoerfert, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60028

llvm-svn: 357347
2019-03-30 14:08:59 +00:00
Jordan Rupprecht 871baa2551 [llvm-readobj] Add some generic notes (e.g. NT_VERSION)
Summary: Support reading notes that don't have a standard note name.

Reviewers: MaskRay

Reviewed By: MaskRay

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59969

llvm-svn: 357271
2019-03-29 16:48:19 +00:00
Jordan Rupprecht 342aaa14b1 [llvm-readelf] Allow prefix flags for -p and -x
Summary: This allows syntax like `llvm-readelf -p.data1 -x.data2`.

Reviewers: jhenderson

Reviewed By: jhenderson

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59965

llvm-svn: 357270
2019-03-29 16:43:13 +00:00
Fangrui Song 8da6a6cec3 [llvm-objcopy] Fix case style of LayoutSegments. NFC
llvm-svn: 357265
2019-03-29 15:27:58 +00:00
Roman Lebedev 4d81e87765 [NFC][llvm-exegesis] Also promote getSchedClassPoint() into ResolvedSchedClass.
Summary:
It doesn't need anything from Analysis::SchedClassCluster class,
and takes ResolvedSchedClass as param, so this seems rather fitting.

Reviewers: courbet, gchatelet

Reviewed By: courbet

Subscribers: tschuett, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59994

llvm-svn: 357263
2019-03-29 14:58:01 +00:00
Roman Lebedev 1d1330c546 [NFC][llvm-exegesis] Refactor ResolvedSchedClass & friends
Summary:
`ResolvedSchedClass` will need to be used outside of `Analysis`
(before `InstructionBenchmarkClustering` even), therefore promote
it into a non-private top-level class, and while there also
move all of the functions that are only called by `ResolvedSchedClass`
into that same new file.

Reviewers: courbet, gchatelet

Reviewed By: courbet

Subscribers: mgorny, tschuett, mgrang, jdoerfert, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59993

llvm-svn: 357259
2019-03-29 14:24:27 +00:00
Andrea Di Biagio e074ac60b4 [MCA] Add an experimental MicroOpQueue stage.
This patch adds an experimental stage named MicroOpQueueStage.
MicroOpQueueStage can be used to simulate a hardware micro-op queue (basically,
a decoupling queue between 'decode' and 'dispatch').  Users can specify a queue
size, as well as a optional MaxIPC (which - in the absence of a "Decoders" stage
- can be used to simulate a different throughput from the decoders).

This stage is added to the default pipeline between the EntryStage and the
DispatchStage only if PipelineOption::MicroOpQueue is different than zero. By
default, llvm-mca sets PipelineOption::MicroOpQueue to the value of hidden flag
-micro-op-queue-size.

Throughput from the decoder can be simulated via another hidden flag named
-decoder-throughput.  That flag allows us to quickly experiment with different
frontend throughputs.  For targets that declare a loop buffer, flag
-decoder-throughput allows users to do multiple runs, each time simulating a
different throughput from the decoders.

This stage can/will be extended in future. For example, we could add a "buffer
full" event to notify bottlenecks caused by backpressure. flag
-decoder-throughput would probably go away if in future we delegate to another
stage (DecoderStage?) the simulation of a (potentially variable) throughput from
the decoders. For now, flag -decoder-throughput is "good enough" to run some
simple experiments.

Differential Revision: https://reviews.llvm.org/D59928

llvm-svn: 357248
2019-03-29 12:15:37 +00:00
James Henderson 814ab373ac [llvm-readelf]Merge dynamic and static relocation printing to avoid code duplication
The majority of the printRelocation and printDynamicRelocation functions
were identical. This patch factors this all out into a new function.
There are a couple of minor differences to do with printing of symbols
without names, but I think these are harmless, and in some cases a small
improvement.

Reviewed by: grimar, rupprecht, Higuoxing

Differential Revision: https://reviews.llvm.org/D59823

llvm-svn: 357246
2019-03-29 11:47:19 +00:00
Roman Lebedev b8fb15d412 [NFC][llvm-exegesis] Refactor Analysis::SchedClassCluster::measurementsMatch()
Summary:
The diff looks scary but it really isn't:
1. I moved the check for the number of measurements into `SchedClassClusterCentroid::validate()`
2. While there, added a check that we can only have a single inverse throughput measurement. I missed that when adding it initially.
3. In `Analysis::SchedClassCluster::measurementsMatch()` is called with the current LLVM values from schedule class and the values from Centroid.
3.1. The values from centroid we can already get from `SchedClassClusterCentroid::getAsPoint()`.
     This isn't 100% a NFC, because previously for inverse throughput we used `min()`. I have asked whether i have done that correctly in
     https://reviews.llvm.org/D57647?id=184939#inline-510384 but did not hear back. I think `avg()` should be used too, thus it is a fix.
3.2. Finally, refactor the computation of the LLVM-specified values into `Analysis::SchedClassCluster::getSchedClassPoint()`
     I will need that function for [[ https://bugs.llvm.org/show_bug.cgi?id=41275 | PR41275 ]]

Reviewers: courbet, gchatelet

Reviewed By: courbet

Subscribers: tschuett, jdoerfert, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59951

llvm-svn: 357245
2019-03-29 11:36:08 +00:00
Fangrui Song 5ed0a8bf13 [llvm-objcopy] Delete two redundant reinterpret_cast. NFC
llvm-svn: 357238
2019-03-29 08:08:20 +00:00
Xing GUO 09a77fec75 [llvm-readobj] Change variable names to match LLVM-style. NFC.
Summary: This patch helps change variable names to match LLVM-style

Reviewers: jhenderson, Higuoxing

Reviewed By: Higuoxing

Subscribers: rupprecht, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59931

llvm-svn: 357230
2019-03-29 01:26:36 +00:00
Jordan Rupprecht 1dc28b6d2b [llvm-readobj] Fix formatting of unknown note types
llvm-svn: 357221
2019-03-28 23:08:06 +00:00
Puyan Lotfi 6c82695753 [yaml2obj] Fixing opening empty yaml files.
Essentially echo "" | yaml2obj crashes. This patch attempts to trim whitespace
and determine if the yaml string in the file is empty or not. If the input is
empty then it will not properly print out an error message and return an error
code.

Differential Revision: https://reviews.llvm.org/D59964

A    test/tools/yaml2obj/empty.yaml
M    tools/yaml2obj/yaml2obj.cpp

llvm-svn: 357219
2019-03-28 22:55:08 +00:00
Jordan Rupprecht bd95a9f46d [llvm-objcopy][NFC] Move ELF-specific logic into /ELF/ directory
llvm-svn: 357199
2019-03-28 18:27:00 +00:00
Xing GUO 12632c761b - Addressed comments
llvm-svn: 357166
2019-03-28 12:51:56 +00:00
Xing GUO 7ffd91124b - Addressed @jhenderson 's comments
- Format patch

llvm-svn: 357165
2019-03-28 12:51:46 +00:00
Xing GUO 137315e7d4 [llvm-readobj] Add new helper function `getSymbolVersionByIndex()`
Summary: When implementing `GNU style` dumper for `.gnu.version` section, we should find symbol version name by `vs_index`.

Reviewers: jhenderson, rupprecht

Reviewed By: rupprecht

Subscribers: arphaman, rupprecht, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59545

llvm-svn: 357164
2019-03-28 12:51:35 +00:00
George Rimar 4111299584 [yaml2obj][obj2yaml] - Teach yaml2obj/obj2yaml tools about STB_GNU_UNIQUE symbols.
yaml2obj/obj2yaml does not support the symbols with STB_GNU_UNIQUE yet.
Currently, obj2yaml fails with llvm_unreachable when met such a symbol.

I faced it when investigated the https://bugs.llvm.org/show_bug.cgi?id=41196.

Differential revision: https://reviews.llvm.org/D59875

llvm-svn: 357158
2019-03-28 10:52:14 +00:00
Roman Lebedev c2423fe689 [llvm-exegesis] Introduce a 'naive' clustering algorithm (PR40880)
Summary:
This is an alternative to D59539.

Let's suppose we have measured 4 different opcodes, and got: `0.5`, `1.0`, `1.5`, `2.0`.
Let's suppose we are using `-analysis-clustering-epsilon=0.5`.
By default now we will start processing the `0.5` point, find that `1.0` is it's neighbor, add them to a new cluster.
Then we will notice that `1.5` is a neighbor of `1.0` and add it to that same cluster.
Then we will notice that `2.0` is a neighbor of `1.5` and add it to that same cluster.
So all these points ended up in the same cluster.
This may or may not be a correct implementation of dbscan clustering algorithm.

But this is rather horribly broken for the reasons of comparing the clusters with the LLVM sched data.
Let's suppose all those opcodes are currently in the same sched cluster.
If i specify `-analysis-inconsistency-epsilon=0.5`, then no matter
the LLVM values this cluster will **never** match the LLVM values,
and thus this cluster will **always** be displayed as inconsistent.

The solution is obviously to split off some of these opcodes into different sched cluster.
But how do i do that? Out of 4 opcodes displayed in the inconsistency report,
which ones are the "bad ones"? Which ones are the most different from the checked-in data?
I'd need to go in to the `.yaml` and look it up manually.

The trivial solution is to, when creating clusters, don't use the full dbscan algorithm,
but instead "pick some unclustered point, pick all unclustered points that are it's neighbor,
put them all into a new cluster, repeat". And just so as it happens, we can arrive
at that algorithm by not performing the "add neighbors of a neighbor to the cluster" step.

But that won't work well once we teach analyze mode to operate in on-1D mode
(i.e. on more than a single measurement type at a time), because the clustering would
depend on the order of the measurements.

Instead, let's just create a single cluster per opcode, and put all the points of that opcode into said cluster.
And simultaneously check that every point in that cluster is a neighbor of every other point in the cluster,
and if they are not, the cluster (==opcode) is unstable.

This is //yet another// step to bring me closer to being able to continue cleanup of bdver2 sched model..

Fixes [[ https://bugs.llvm.org/show_bug.cgi?id=40880 | PR40880 ]].

Reviewers: courbet, gchatelet

Reviewed By: courbet

Subscribers: tschuett, jdoerfert, RKSimon, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59820

llvm-svn: 357152
2019-03-28 08:55:01 +00:00
Eric Christopher 1857edb256 Fix typoed variable name.
NFCI.

llvm-svn: 357138
2019-03-28 01:12:13 +00:00
Jessica Paquette eaf4df4782 [opt-viewer] Teach optrecord.py about !Failure tags
WarnMissedTransforms.cpp produces remarks that use !Failure tags.

These weren't supported in optrecord.py, so if you encountered one in any of
the tools, the tool would crash.

Add them as a type of missed optimization.

Differential Revision: https://reviews.llvm.org/D59895

llvm-svn: 357110
2019-03-27 18:35:04 +00:00
Jessica Paquette beda859a15 [opt-viewer] Make filter_=None by default in get_remarks and gather_results
Right now, if you try to use optdiff.py on any opt records, it will fail because
its calls to gather_results weren't updated to support filtering.

Since filters are supposed to be optional, this makes them None by default in
get_remarks and in gather_results. This allows other tools that don't support
filtering to still use the functions as is.

Differential Revision: https://reviews.llvm.org/D59894

llvm-svn: 357106
2019-03-27 18:14:32 +00:00
Fangrui Song 95db95729c [llvm-dwarfdump] Simplify -o handling
ToolOutputFile handles '-' so no need to specialize here.
Also, we neither reassign the variable nor pass it around, thus no need
to use std::unique_ptr<ToolOutputFile>.

exit(1) -> return 1;  to call the destructor of raw_fd_stream

llvm-svn: 357051
2019-03-27 08:19:36 +00:00
George Rimar 279898b315 [llvm-objcopy] - Strip sections before symbols.
This is a fix for https://bugs.llvm.org/show_bug.cgi?id=40007.

Idea is to swap the order of stripping. So that we strip sections before
symbols what allows us to strip relocation sections without emitting
the error about relative symbols.

Differential revision: https://reviews.llvm.org/D59763

llvm-svn: 357017
2019-03-26 18:42:15 +00:00
Clement Courbet 52da938cd0 [llvm-exegesis] Allow the target to disable the selection of some registers.
Summary:
This prevents "Cannot encode high byte register in REX-prefixed instruction"
from happening on instructions that require REX encoding when AH & co
get selected.
On the down side, these 4 registers can no longer be selected
automatically, but this avoids having to expose all the X86 encoding
complexity.

Reviewers: gchatelet

Subscribers: tschuett, jdoerfert, llvm-commits, bdb

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59821

llvm-svn: 357003
2019-03-26 15:44:57 +00:00
Andrea Di Biagio ddce32e2f3 [MCA] Correctly update the UsedResourceGroups mask in the InstrBuilder.
Found by inspection when looking at the debug output of MCA.
This problem was latent, and none of the upstream models were affected by it.
No functional change intended.

llvm-svn: 357000
2019-03-26 15:38:37 +00:00
James Henderson 1f44814952 [llvm-objcopy]Preserve data in segments not covered by sections
llvm-objcopy previously knew nothing about data in segments that wasn't
covered by section headers, meaning that it wrote zeroes instead of what
was there. As it is possible for this data to be useful to the loader,
this patch causes llvm-objcopy to start preserving this data. Data in
sections that are explicitly removed continues to be written as zeroes.

This fixes https://bugs.llvm.org/show_bug.cgi?id=41005.

Reviewed by: jakehehrlich, rupprecht

Differential Revision: https://reviews.llvm.org/D59483

llvm-svn: 356919
2019-03-25 16:36:26 +00:00
Serge Guelton dcb128f73b Revert 356905
Commited from wrong directory...

llvm-svn: 356907
2019-03-25 15:18:55 +00:00
Serge Guelton 2dea7c4113 Python 2/3 compat: queue vs Queue
Differential Revision: https://reviews.llvm.org/D59590

llvm-svn: 356905
2019-03-25 15:14:15 +00:00
George Rimar e6963be317 [llvm-objcopy] - Refactor the code. NFC.
The idea of the patch is about to move out the code to a new
helper static functions (to reduce the size of 'handleArgs' and to
isolate the parts of it's logic).

Differential revision: https://reviews.llvm.org/D59762

llvm-svn: 356889
2019-03-25 12:34:25 +00:00
Xing GUO ea16be1ca7 [llvm-readobj] Separate `Symbol Version` dumpers into `LLVM style` and `GNU style`
Summary:
Currently, llvm-readobj can dump symbol version sections only in LLVM style. In this patch, I would like to separate these dumpers into GNU style and 
LLVM style for future implementation.

Reviewers: grimar, jhenderson, mattd, rupprecht

Reviewed By: jhenderson, rupprecht

Subscribers: ormris, dyung, RKSimon, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59186

llvm-svn: 356881
2019-03-25 11:02:49 +00:00
George Rimar 272571718c Recommit r356738 "[llvm-objcopy] - Implement replaceSectionReferences for GroupSection class."
Fix: r356853 + set AddressAlign to 4 in 
Inputs/compress-debug-sections.yaml for the new group section introduced.

Original commit message:

Currently, llvm-objcopy incorrectly handles compression and decompression of the
sections from COMDAT groups, because we do not implement the
replaceSectionReferences for this type of the sections.

The patch does that.

Differential revision: https://reviews.llvm.org/D59638

llvm-svn: 356856
2019-03-24 14:41:45 +00:00
George Rimar 0a5d4b8472 [llvm-objcopy] - Report SHT_GROUP sections with invalid alignment.
This patch fixes the reason of ubsan failure (UB detected) 
happened after landing the D59638 (I had to revert it).
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap-ubsan/builds/11760/steps/check-llvm%20ubsan/logs/stdio)

Problem is the following. Our implementation of GroupSection assumes that
its address is 4 bytes aligned when writes it:

template <class ELFT>
void ELFSectionWriter<ELFT>::visit(const GroupSection &Sec) {
  ELF::Elf32_Word *Buf =
      reinterpret_cast<ELF::Elf32_Word *>(Out.getBufferStart() + Sec.Offset);
...

But the test case for D59638 did not set AddressAlign in YAML. So address was
not 4 bytes aligned since Sec.Offset was odd. That triggered the issue.

This patch teaches llvm-objcopy to report an error for such sections (which should
not met in reality), what is better than having UB.

Differential revision: https://reviews.llvm.org/D59695

llvm-svn: 356853
2019-03-24 13:31:08 +00:00
Alexey Lapshin b2c4b8bded [DebugInfo] follow up for "add SectionedAddress to DebugInfo interfaces"
[Symbolizer] Add getModuleSectionIndexForAddress() helper routine

  The https://reviews.llvm.org/D58194 patch changed symbolizer interface.
  Particularily it requires not only Address but SectionIndex also.
  Note object::SectionedAddress parameter:

  Expected<DILineInfo> symbolizeCode(const std::string &ModuleName,
                                   object::SectionedAddress ModuleOffset,
                                   StringRef DWPName = "");

  There are callers of symbolizer which do not know particular section index.
  That patch creates getModuleSectionIndexForAddress() routine which
  will detect section index for the specified address. Thus if caller
  set ModuleOffset.SectionIndex into object::SectionedAddress::UndefSection
  state then symbolizer would detect section index using
  getModuleSectionIndexForAddress routine.

  Differential Revision: https://reviews.llvm.org/D58848

llvm-svn: 356829
2019-03-23 08:08:40 +00:00
Reid Kleckner e6a81b9bec [pdb] Add -type-stats and sort stats by descending size
Summary:
It prints this on chromium browser_tests.exe.pdb:

  Types
           Total: 5647475 entries ( 371,897,512 bytes,   65.85 avg)
  --------------------------------------------------------------------------
        LF_CLASS:  397894 entries ( 119,537,780 bytes,  300.43 avg)
    LF_STRUCTURE:  236351 entries (  83,208,084 bytes,  352.05 avg)
    LF_FIELDLIST:  291003 entries (  66,087,920 bytes,  227.10 avg)
    LF_MFUNCTION: 1884176 entries (  52,756,928 bytes,   28.00 avg)
      LF_POINTER: 1149030 entries (  13,877,344 bytes,   12.08 avg)
      LF_ARGLIST:  789980 entries (  12,436,752 bytes,   15.74 avg)
   LF_METHODLIST:  361498 entries (   8,351,008 bytes,   23.10 avg)
         LF_ENUM:   16069 entries (   6,108,340 bytes,  380.13 avg)
    LF_PROCEDURE:  269374 entries (   4,309,984 bytes,   16.00 avg)
     LF_MODIFIER:  235602 entries (   2,827,224 bytes,   12.00 avg)
        LF_UNION:    9131 entries (   2,072,168 bytes,  226.94 avg)
      LF_VFTABLE:     323 entries (     207,784 bytes,  643.29 avg)
        LF_ARRAY:    6639 entries (     106,380 bytes,   16.02 avg)
      LF_VTSHAPE:     126 entries (       6,472 bytes,   51.37 avg)
     LF_BITFIELD:     278 entries (       3,336 bytes,   12.00 avg)
        LF_LABEL:       1 entries (           8 bytes,    8.00 avg)

The PDB is overall 1.9GB, so the LF_CLASS and LF_STRUCTURE declarations
account for about 10% of the overall file size. I was surprised to find
that on average LF_FIELDLIST records are short. Maybe this is because
there are many more types with short member lists than there are
instantiations with lots of members, like std::vector.

Reviewers: aganea, zturner

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59672

llvm-svn: 356813
2019-03-22 21:22:13 +00:00
Xing GUO 4218d45f65 [llvm-readobj] Revert bad changes
llvm-svn: 356777
2019-03-22 16:20:54 +00:00
Xing GUO 94a0cffe25 [llvm-readobj] Separate `Symbol Version` dumpers into `LLVM style` and `GNU style`
Summary:
Currently, llvm-readobj can dump symbol version sections only in LLVM style. In this patch, I would like to separate these dumpers into GNU style and 
LLVM style for future implementation.

Reviewers: grimar, jhenderson, mattd, rupprecht

Reviewed By: rupprecht

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59186

llvm-svn: 356764
2019-03-22 15:42:13 +00:00
Pavel Labath 69de7a955e [ObjectYAML] Add basic minidump generation support
Summary:
This patch adds the ability to read a yaml form of a minidump file and
write it out as binary. Apart from the minidump header and the stream
directory, only three basic stream kinds are supported:
- Text: This kind is used for streams which contain textual data. This
  is typically the contents of a /proc file on linux (e.g.
  /proc/PID/maps). In this case, we just put the raw stream contents
  into the yaml.
- SystemInfo: This stream contains various bits of information about the
  host system in binary form. We expose the data in a structured form.
- Raw: This kind is used as a fallback when we don't have any special
  knowledge about the stream. In this case, we just print the stream
  contents in hex.

For this code to be really useful, more stream kinds will need to be
added (particularly for things like lists of memory regions and loaded
modules). However, these can be added incrementally.

Reviewers: jhenderson, zturner, clayborg, aprantl

Subscribers: mgorny, lemo, llvm-commits, lldb-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59482

llvm-svn: 356753
2019-03-22 14:47:26 +00:00
George Rimar d822018dbe Revert r356738 "[llvm-objcopy] - Implement replaceSectionReferences for GroupSection class."
Seems this broke ubsan bot:
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap-ubsan/builds/11760

llvm-svn: 356745
2019-03-22 12:14:04 +00:00
George Rimar 73e1c4a030 [llvm-objcopy] - Implement replaceSectionReferences for GroupSection class.
Currently, llvm-objcopy incorrectly handles compression and decompression of the
sections from COMDAT groups, because we do not implement the
replaceSectionReferences for this type of the sections.

The patch does that.

Differential revision: https://reviews.llvm.org/D59638

llvm-svn: 356738
2019-03-22 10:24:37 +00:00
James Henderson c040d5de25 [llvm-objcopy]Add support for *-freebsd output formats
GNU objcopy can support output formats like elf32-i386-freebsd and
elf64-x86-64-freebsd. The only difference from their regular non-freebsd
counterparts that I have observed is that the freebsd versions set the
OS/ABI field to ELFOSABI_FREEBSD. This patch sets the OS/ABI field
according based on the format whenever --output-format is specified.

Reviewed by: rupprecht, grimar

Differential Revision: https://reviews.llvm.org/D59645

llvm-svn: 356737
2019-03-22 10:21:09 +00:00
Jordan Rupprecht ae672b482f [llvm-objdump] Support arg grouping for -j and -M (e.g. llvm-objdump -sj.foo -dMreg-names-raw)
Summary:
r354375 added support for most objdump groupings, but didn't add support for -j|--sections, because that wasn't possible.
r354870 added --disassembler options, but grouping still wasn't available.
r355185 supported values for grouped options.

This just puts the three of them together. This supports -j in modes like `-s -j .foo`, `-sj .foo`, `-sj=.foo`, or `-sj.foo`, and similar for `-M`.

Reviewers: ormris, jhenderson, ikudrin

Reviewed By: jhenderson, ikudrin

Subscribers: javed.absar, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59618

llvm-svn: 356697
2019-03-21 18:45:44 +00:00
Reid Kleckner cda7ff9ddc [llvm-pdbutil] Add -type-ref-stats to help find unused type info
Summary:
This considers module symbol streams and the global symbol stream to be
roots. Most types that this considers "unreferenced" are referenced by
LF_UDT_MOD_SRC_LINE id records, which VC seems to always include.
Essentially, they are types that the user can only find in the debugger
if they call them by name, they cannot be found by traversing a symbol.

In practice, around 80% of type information in a PDB is referenced by a
symbol. That seems like a reasonable number.

I don't really plan to do anything with this tool. It mostly just exists
for informational purposes, and to confirm that we probably don't need
to implement type reference tracking in LLD. We can continue to merge
all types as we do today without wasting space.

Reviewers: zturner, aganea

Subscribers: mgorny, hiraditya, arphaman, jdoerfert, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59620

llvm-svn: 356692
2019-03-21 18:02:34 +00:00
Xing GUO 3e9e55491e [llvm-readobj] Format codes. NFC.
llvm-svn: 356664
2019-03-21 13:42:06 +00:00
Thomas Lively f6f4f84378 [WebAssembly] Target features section
Summary:
Implements a new target features section in assembly and object files
that records what features are used, required, and disallowed in
WebAssembly objects. The linker uses this information to ensure that
all objects participating in a link are feature-compatible and records
the set of used features in the output binary for use by optimizers
and other tools later in the toolchain.

The "atomics" feature is always required or disallowed to prevent
linking code with stripped atomics into multithreaded binaries. Other
features are marked used if they are enabled globally or on any
function in a module.

Future CLs will add linker flags for ignoring feature compatibility
checks and for specifying the set of allowed features, implement using
the presence of the "atomics" feature to control the type of memory
and segments in the linked binary, and add front-end flags for
relaxing the linkage policy for atomics.

Reviewers: aheejin, sbc100, dschuff

Subscribers: jgravelle-google, hiraditya, sunfish, mgrang, jfb, jdoerfert, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59173

llvm-svn: 356610
2019-03-20 20:26:45 +00:00
Tim Renouf d737b551e9 [AMDGPU] Factored PAL metadata handling out into its own class
Summary:
This commit introduces a new AMDGPUPALMetadata class that:
* is inside the AMDGPU target;
* keeps an in-memory representation of PAL metadata;
* provides a method to read the frontend-supplied metadata from LLVM IR;
* provides methods for the asm printer to set metadata items;
* provides methods to write the metadata as a binary blob to put in a
  .note record or as an asm directive;
* provides a method to read the metadata as a binary blob from a .note
  record.

Because llvm-readobj cannot call directly into a target, I had to remove
llvm-readobj's ability to dump PAL metadata, pending a resolution to
https://reviews.llvm.org/D52821

Differential Revision: https://reviews.llvm.org/D57027

Change-Id: I756dc830894fcb6850324cdcfa87c0120eb2cf64
llvm-svn: 356582
2019-03-20 17:42:00 +00:00
George Rimar 0373bedb41 [llvm-objcopy] - Use replaceSectionReferences to update the sections for symbols in symbol table.
If the compression was used and we had a symbol not involved in relocation,
we never updated its section and it was silently removed from the output.

Differential revision: https://reviews.llvm.org/D59542

llvm-svn: 356554
2019-03-20 13:57:47 +00:00
Francis Visoiu Mistrih 5a05cc0eeb Reland "[Remarks] Add a new Remark / RemarkParser abstraction"
This adds a Remark class that allows us to share code when working with
remarks.

The C API has been updated to reflect this. Instead of the parser
generating C structs, it's now using a C++ object that is used through
opaque pointers in C. This gives us much more flexibility on what
changes we can make to the internal state of the object and interacts
much better with scenarios where the library is used through dlopen.

* C API updates:
  * move from C structs to opaque pointers and functions
  * the remark type is now an enum instead of a string
* unit tests updates:
  * use mostly the C++ API
  * keep one test for the C API
  * rename to YAMLRemarksParsingTest
* a typo was fixed: AnalysisFPCompute -> AnalysisFPCommute.
* a new error message was added: "expected a remark tag."
* llvm-opt-report has been updated to use the C++ parser instead of the
C API

Differential Revision: https://reviews.llvm.org/D59049

Original llvm-svn: 356491

llvm-svn: 356519
2019-03-19 21:11:07 +00:00
Francis Visoiu Mistrih 064774f753 Revert "[Remarks] Add a new Remark / RemarkParser abstraction"
This reverts commit 51dc6a8c84cd6a58562e320e1828a0158dbbf750.

Breaks
http://lab.llvm.org:8011/builders/clang-cmake-x86_64-sde-avx512-linux/builds/20034/steps/build%20stage%201/logs/stdio.

llvm-svn: 356492
2019-03-19 18:21:43 +00:00
Francis Visoiu Mistrih 9ef60a2539 [Remarks] Add a new Remark / RemarkParser abstraction
This adds a Remark class that allows us to share code when working with
remarks.

The C API has been updated to reflect this. Instead of the parser
generating C structs, it's now using a C++ object that is used through
opaque pointers in C. This gives us much more flexibility on what
changes we can make to the internal state of the object and interacts
much better with scenarios where the library is used through dlopen.

* C API updates:
  * move from C structs to opaque pointers and functions
  * the remark type is now an enum instead of a string
* unit tests updates:
  * use mostly the C++ API
  * keep one test for the C API
  * rename to YAMLRemarksParsingTest
* a typo was fixed: AnalysisFPCompute -> AnalysisFPCommute.
* a new error message was added: "expected a remark tag."
* llvm-opt-report has been updated to use the C++ parser instead of the
C API

Differential Revision: https://reviews.llvm.org/D59049

llvm-svn: 356491
2019-03-19 18:09:51 +00:00
Jordan Rupprecht 4a6b9f2316 [llvm-ar] Support N [count] modifier
Summary:
GNU ar supports the 'N' count modifier for the extract (x) and delete (d) operations. When an archive contains multiple members with the same name, this can be used to extract (or delete) them individually. For example:

```
$ llvm-ar t archive.a
foo
foo
$ llvm-ar x archive.a
-> Writes foo twice, overwriting it the second time :( :(
$ llvm-ar xN 1 archive.a foo && mv foo foo.1
$ llvm-ar xN 2 archive.a foo && mv foo foo.2
-> Write foo twice, renaming it in between invocations to preserve all versions
```

Reviewers: ruiu, MaskRay

Reviewed By: ruiu, MaskRay

Subscribers: jdoerfert, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59503

llvm-svn: 356466
2019-03-19 16:09:54 +00:00
Serge Guelton d2f2f33ef2 Use response file when generating LLVM-C.dll
As discovered in D56774 the command line gets to long, so use a response file
to give the script the libs. This change has been tested and is confirmed
working for me.

Commited on behalf of Jakob Bornecrantz.
Differential Revision: https://reviews.llvm.org/D56781

llvm-svn: 356443
2019-03-19 09:14:09 +00:00
Jake Ehrlich 5049c3422d [llvm-objcopy] Make .build-id linking atomic
This change makes linking into .build-id atomic and safe to use.
Some users under particular workflows are reporting that this races
more than half the time under particular conditions.

llvm-svn: 356404
2019-03-18 20:35:18 +00:00
George Rimar faf308b11a [llvm-objcopy] - Calculate the string table section sizes correctly.
This fixes the https://bugs.llvm.org/show_bug.cgi?id=40980.

Previously if string optimization occurred as a result of
StringTableBuilder's finalize() method, the size wasn't updated.

This hopefully also makes the interaction between sections during finalization
processes a bit more clear.

Differential revision: https://reviews.llvm.org/D59488

llvm-svn: 356371
2019-03-18 14:27:41 +00:00
Roman Lebedev 23629385f1 [llvm-exegesis] Separate tool options into three categories.
Results in much nicer -help output:
```
$ ./bin/llvm-exegesis -help
USAGE: llvm-exegesis [options]

OPTIONS:

Color Options:

  -color                                         - Use colors in output (default=autodetect)

General options:

  -enable-cse-in-irtranslator                    - Should enable CSE in irtranslator
  -enable-cse-in-legalizer                       - Should enable CSE in Legalizer

Generic Options:

  -help                                          - Display available options (-help-hidden for more)
  -help-list                                     - Display list of available options (-help-list-hidden for more)
  -version                                       - Display the version of this program

llvm-exegesis analysis options:

  -analysis-clustering-epsilon=<number>          - dbscan epsilon for benchmark point clustering
  -analysis-clusters-output-file=<string>        -
  -analysis-display-unstable-clusters            - if there is more than one benchmark for an opcode, said benchmarks may end up not being clustered into the same cluster if the measured performance characteristics are different. by default all such opcodes are filtered out. this flag will instead show only such unstable opcodes
  -analysis-inconsistencies-output-file=<string> -
  -analysis-inconsistency-epsilon=<number>       - epsilon for detection of when the cluster is different from the LLVM schedule profile values
  -analysis-numpoints=<uint>                     - minimum number of points in an analysis cluster

llvm-exegesis benchmark options:

  -ignore-invalid-sched-class                    - ignore instructions that do not define a sched class
  -mode=<value>                                  - the mode to run
    =latency                                     -   Instruction Latency
    =inverse_throughput                          -   Instruction Inverse Throughput
    =uops                                        -   Uop Decomposition
    =analysis                                    -   Analysis
  -num-repetitions=<uint>                        - number of time to repeat the asm snippet
  -opcode-index=<int>                            - opcode to measure, by index
  -opcode-name=<string>                          - comma-separated list of opcodes to measure, by name
  -snippets-file=<string>                        - code snippets to measure

llvm-exegesis options:

  -benchmarks-file=<string>                      - File to read (analysis mode) or write (latency/uops/inverse_throughput modes) benchmark results. “-” uses stdin/stdout.
  -mcpu=<string>                                 - cpu name to use for pfm counters, leave empty to autodetect
```

llvm-svn: 356364
2019-03-18 11:32:37 +00:00
Fangrui Song 8b0a15b0ef [llvm-profdata] Deleted unused Cutoffs added by D16005
llvm-svn: 356248
2019-03-15 10:43:51 +00:00
James Henderson b10f48bbb4 [yaml2obj]Allow explicit setting of p_filesz, p_memsz, and p_offset
yaml2obj currently derives the p_filesz, p_memsz, and p_offset values of
program headers from their sections. This makes writing tests for
certain formats more complex, and sometimes impossible. This patch
allows setting these fields explicitly, overriding the default value,
when relevant.

Reviewed by: jakehehrlich, Higuoxing

Differential Revision: https://reviews.llvm.org/D59372

llvm-svn: 356247
2019-03-15 10:35:27 +00:00
Fangrui Song 53dcf2d499 [llvm-readobj] Delete unused variable. NFC
llvm-svn: 356246
2019-03-15 10:34:57 +00:00
Fangrui Song 3dfc3fb02b [llvm-objcopy] Delete unused parameter from replaceDebugSections. NFC
llvm-svn: 356245
2019-03-15 10:27:28 +00:00
Fangrui Song a85bf8747a [llvm-objcopy] Don't use {}; NFC
llvm-svn: 356244
2019-03-15 10:20:51 +00:00
Jordan Rupprecht 12ed01dcf9 [llvm-strip] Hook up (unimplemented) --only-keep-debug
For ELF, we accept but ignore --only-keep-debug. Do the same for llvm-strip.

COFF does implement this, so update the test that it is supported.

llvm-svn: 356207
2019-03-14 21:51:42 +00:00
Max Moroz a80d9ce5cf Speeding up llvm-cov export with multithreaded renderFiles implementation.
Summary:
CoverageExporterJson::renderFiles accounts for most of the execution time given a large profdata file with multiple binaries.

Proposed solution is to generate JSON for each file in parallel and sort at the end to preserve deterministic output. Also added flags to skip generating parts of the output to trim the output size.

Patch by Sajjad Mirza (@sajjadm).

Reviewers: Dor1s, vsk

Reviewed By: Dor1s, vsk

Subscribers: liaoyuke, mgrang, jdoerfert, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59277

llvm-svn: 356178
2019-03-14 17:49:27 +00:00
James Henderson b5de5e25de [llvm-objcopy]Don't implicitly strip sections in segments
This patch changes llvm-objcopy's behaviour to not strip sections that
are in segments, if they otherwise would be due to a stripping operation
(--strip-all, --strip-sections, --strip-non-alloc). This preserves the
segment contents. It does not change the behaviour of --strip-all-gnu
(although we could choose to do so), because GNU objcopy's behaviour in
this case seems to be to strip the section, nor does it prevent removing
of sections in segments with --remove-section (if a user REALLY wants to
remove a section, we should probably let them, although I could be
persuaded that warning might be appropriate). Tests have been added to
show this latter behaviour.

This fixes https://bugs.llvm.org/show_bug.cgi?id=41006.

Reviewed by: grimar, rupprecht, jakehehrlich

Differential Revision: https://reviews.llvm.org/D59293

This is a reland of r356129, attempting to fix greendragon failures
due to a suspected compatibility issue with od on the greendragon bots
versus other versions.

llvm-svn: 356136
2019-03-14 11:47:41 +00:00
James Henderson e81f5f91b4 Revert r356129 due to greendragon bot failures
llvm-svn: 356133
2019-03-14 11:23:04 +00:00
James Henderson c03a95d465 [llvm-objcopy]Don't implicitly strip sections in segments
This patch changes llvm-objcopy's behaviour to not strip sections that
are in segments, if they otherwise would be due to a stripping operation
(--strip-all, --strip-sections, --strip-non-alloc). This preserves the
segment contents. It does not change the behaviour of --strip-all-gnu
(although we could choose to do so), because GNU objcopy's behaviour in
this case seems to be to strip the section, nor does it prevent removing
of sections in segments with --remove-section (if a user REALLY wants to
remove a section, we should probably let them, although I could be
persuaded that warning might be appropriate). Tests have been added to
show this latter behaviour.

This fixes https://bugs.llvm.org/show_bug.cgi?id=41006.

Reviewed by: grimar, rupprecht, jakehehrlich

Differential Revision: https://reviews.llvm.org/D59293

llvm-svn: 356129
2019-03-14 10:20:27 +00:00
Jordan Rupprecht e4aa3f8264 [llvm-objcopy][NFC] Remove unnecessary llvm-objcopy.h #includes
llvm-svn: 356109
2019-03-13 23:40:16 +00:00
Jordan Rupprecht 42bc1e241c [llvm-objcopy] Cleanup errors from CopyConfig and remove llvm-objcopy.h dependency
error() was previously cleaned up from CopyConfig, but new uses were introduced.

This also tweaks the error message for --add-symbol to report all invalid flags.

llvm-svn: 356105
2019-03-13 22:26:01 +00:00
Tim Renouf ed0b9af997 [AMDGPU] Switched HSA metadata to use MsgPackDocument
Summary:
MsgPackDocument is the lighter-weight replacement for MsgPackTypes. This
commit switches AMDGPU HSA metadata processing to use MsgPackDocument
instead of MsgPackTypes.

Differential Revision: https://reviews.llvm.org/D57024

Change-Id: I0751668013abe8c87db01db1170831a76079b3a6
llvm-svn: 356081
2019-03-13 18:55:50 +00:00
Francis Visoiu Mistrih dd42236c6c Reland "[Remarks] Add -foptimization-record-passes to filter remark emission"
Currently we have -Rpass for filtering the remarks that are displayed as
diagnostics, but when using -fsave-optimization-record, there is no way
to filter the remarks while generating them.

This adds support for filtering remarks by passes using a regex.
Ex: `clang -fsave-optimization-record -foptimization-record-passes=inline`

will only emit the remarks coming from the pass `inline`.

This adds:

* `-fsave-optimization-record` to the driver
* `-opt-record-passes` to cc1
* `-lto-pass-remarks-filter` to the LTOCodeGenerator
* `--opt-remarks-passes` to lld
* `-pass-remarks-filter` to llc, opt, llvm-lto, llvm-lto2
* `-opt-remarks-passes` to gold-plugin

Differential Revision: https://reviews.llvm.org/D59268

Original llvm-svn: 355964

llvm-svn: 355984
2019-03-12 21:22:27 +00:00
Francis Visoiu Mistrih 1d6c47ad2b Revert "[Remarks] Add -foptimization-record-passes to filter remark emission"
This reverts commit 20fff32b7d.

llvm-svn: 355976
2019-03-12 20:54:18 +00:00
Francis Visoiu Mistrih 20fff32b7d [Remarks] Add -foptimization-record-passes to filter remark emission
Currently we have -Rpass for filtering the remarks that are displayed as
diagnostics, but when using -fsave-optimization-record, there is no way
to filter the remarks while generating them.

This adds support for filtering remarks by passes using a regex.
Ex: `clang -fsave-optimization-record -foptimization-record-passes=inline`

will only emit the remarks coming from the pass `inline`.

This adds:

* `-fsave-optimization-record` to the driver
* `-opt-record-passes` to cc1
* `-lto-pass-remarks-filter` to the LTOCodeGenerator
* `--opt-remarks-passes` to lld
* `-pass-remarks-filter` to llc, opt, llvm-lto, llvm-lto2
* `-opt-remarks-passes` to gold-plugin

Differential Revision: https://reviews.llvm.org/D59268

llvm-svn: 355964
2019-03-12 20:28:50 +00:00
James Henderson 9bc817a0ae [yaml2obj]Allow explicit symbol indexes in relocations and emit error for bad names
Prior to this change, the "Symbol" field of a relocation would always be
assumed to be a symbol name, and if no such symbol existed, the
relocation would reference index 0. This confused me when I tried to use
a literal symbol index in the field: since "0x1" was not a known symbol
name, the symbol index was set as 0. This change falls back to treating
unknown symbol names as integers, and emits an error if the name is not
found and the string is not an integer.

Note that the Symbol field is optional, so if a relocation doesn't
reference a symbol, it shouldn't be specified. The new error required a
number of test updates.

Reviewed by: grimar, ruiu
Differential Revision: https://reviews.llvm.org/D58510

llvm-svn: 355938
2019-03-12 17:00:25 +00:00
Xing GUO eec3206a41 [llvm-readobj] Print symbol version when dumping relocations (PR31564)
Summary: This helps resolve https://bugs.llvm.org/show_bug.cgi?id=31564

Reviewers: jhenderson, grimar

Reviewed By: jhenderson

Subscribers: rupprecht, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59175

llvm-svn: 355922
2019-03-12 14:30:13 +00:00
Eugene Leviant c76671b231 [llvm-objcopy] Remove unneeded checks. NFC
Differential revision: https://reviews.llvm.org/D59081

llvm-svn: 355914
2019-03-12 12:41:06 +00:00
Peter Collingbourne bf92b3f480 llvm-objcopy: Remove unused field. NFCI.
Differential Revision: https://reviews.llvm.org/D59126

llvm-svn: 355892
2019-03-12 02:17:01 +00:00
Nathan Lanza cc51dc649a Add Swift enumerator value for CodeView::SourceLanguage
Summary:
Swift now generates PDBs for debugging on Windows. llvm and lldb
need a language enumerator value too properly handle the output
emitted by swiftc.

Subscribers: jdoerfert, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59231

llvm-svn: 355882
2019-03-11 23:27:59 +00:00
George Rimar 33e498b785 [yaml2obj] - Simplify. NFC.
llvm-svn: 355832
2019-03-11 16:10:02 +00:00
George Rimar d8a5c6cf19 [llvm-objcopy] - Fix --compress-debug-sections when there are relocations.
When --compress-debug-sections is given,
llvm-objcopy removes the uncompressed sections and adds compressed to the section list.
This makes all the pointers to old sections to be outdated.

Currently, code already has logic for replacing the target sections of the relocation
sections. But we also have to update the relocations by themselves.

This fixes https://bugs.llvm.org/show_bug.cgi?id=40885.

Differential revision: https://reviews.llvm.org/D58960

llvm-svn: 355821
2019-03-11 11:01:24 +00:00
Sunil Srivastava ae8fe4e093 Improve "llvm-nm -f sysv" output for Elf files
Specifically, compute and Print Type and Section columns.

This is a re-commit of rL354833, after fixing the Asan problem found a a buildbot.

Differential Revision: https://reviews.llvm.org/D59060

llvm-svn: 355742
2019-03-08 22:00:50 +00:00
James Henderson b41130bedc [llvm-readelf]Don't lose negative-ness of negative addends for no symbol relocations
llvm-readelf prints relocation addends as:

  <symbol value>[+-]<absolute addend>

where [+-] is determined from whether addend is less than zero or not.
However, it does not print the +/- if there is no symbol, which meant
that negative addends became their positive value with no indication
that this had happened. This patch stops the absolute conversion when
addends are negative and there is no associated symbol.

Reviewed by: Higuoxing, mattd, MaskRay

Differential Revision: https://reviews.llvm.org/D59095

llvm-svn: 355696
2019-03-08 13:22:05 +00:00
Matt Davis 6c5a49ccb9 [llvm-mca] Emit a message when no bottlenecks are identified.
Summary:
Since bottleneck hints are enabled via user request, it can be
confusing if no bottleneck information is presented.  Such is the
case when no bottlenecks are identified.  This patch emits a message
in that case.

Reviewers: andreadb

Reviewed By: andreadb

Subscribers: tschuett, gbedwell, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59098

llvm-svn: 355628
2019-03-07 19:34:44 +00:00
Xing GUO eee6226c21 [llvm-readobj] Dump DT_USED value as string like GNU readelf does
Reviewers: jhenderson

Reviewed By: jhenderson

Subscribers: rupprecht, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59089

llvm-svn: 355600
2019-03-07 14:53:10 +00:00
George Rimar a5a0a0f049 [yaml2obj] - Allow producing ELFDATANONE ELFs
I need this to remove a binary from LLD test suite.
The patch also simplifies the code a bit.

Differential revision: https://reviews.llvm.org/D59082

llvm-svn: 355591
2019-03-07 12:09:19 +00:00
Francis Visoiu Mistrih b8a847c0a3 Reland "[Remarks] Refactor remark diagnostic emission in a RemarkStreamer"
This allows us to store more info about where we're emitting the remarks
without cluttering LLVMContext. This is needed for future support for
the remark section.

Differential Revision: https://reviews.llvm.org/D58996

Original llvm-svn: 355507

llvm-svn: 355514
2019-03-06 15:20:13 +00:00
Francis Visoiu Mistrih 6b622ebea0 Revert "[Remarks] Refactor remark diagnostic emission in a RemarkStreamer"
This reverts commit 2e8c4997a2089f8228c843fd81b148d903472e02.

Breaks bots.

llvm-svn: 355511
2019-03-06 14:52:37 +00:00
Francis Visoiu Mistrih 9052f50cb4 [Remarks] Refactor remark diagnostic emission in a RemarkStreamer
This allows us to store more info about where we're emitting the remarks
without cluttering LLVMContext. This is needed for future support for
the remark section.

Differential Revision: https://reviews.llvm.org/D58996

llvm-svn: 355507
2019-03-06 14:32:08 +00:00
George Rimar 281a5beefa [llvm-objcopy] - Remove dead code. NFCI.
DecompressedSection can only be created if --decompress-debug-sections is specified.
https://github.com/llvm-mirror/llvm/blob/master/tools/llvm-objcopy/ELF/ELFObjcopy.cpp#L492

If it is specified when !zlib::isAvailable(), we error out early when parsing the options:
https://github.com/llvm-mirror/llvm/blob/master/tools/llvm-objcopy/CopyConfig.cpp#L657

What means the code I am removing in this patch is dead.

Differential revision: https://reviews.llvm.org/D59017

llvm-svn: 355505
2019-03-06 14:12:18 +00:00
George Rimar 5f0b7d2f46 [llvm-objcopy] - Remove an excessive zlib::isAvailable() check and dead code.
There are 2 places where llvm-objcopy creates CompressedSection:

For --compress-debug-sections. It might create the compressed section from
regular here:
https://github.com/llvm-mirror/llvm/blob/master/tools/llvm-objcopy/ELF/ELFObjcopy.cpp#L486

All initially compressed sections are created as CompressedSection during reading the sections
from an object:
https://github.com/llvm-mirror/llvm/blob/master/tools/llvm-objcopy/ELF/Object.cpp#L1118
Those have DebugCompressionType::None type and a different constructor.

Case 1 has the following code in its constructor:

if (!zlib::isAvailable()) {
  CompressionType = DebugCompressionType::None;
  return;
}
(https://github.com/llvm-mirror/llvm/blob/master/tools/llvm-objcopy/ELF/Object.cpp#L267)

We can never reach that code with because would report an error much earlier:
https://github.com/llvm-mirror/llvm/blob/master/tools/llvm-objcopy/CopyConfig.cpp#L480

So the code I am removing is dead. Landing this will address the issue mentioned in https://bugs.llvm.org/show_bug.cgi?id=40886.

Differential revision: https://reviews.llvm.org/D59019

llvm-svn: 355503
2019-03-06 14:08:27 +00:00
George Rimar f2eb8caa3f [llvm-objcopy] - Fix incorrect CompressedSection creation.
We should create CompressedSection only if the section has SHF_COMPRESSED flag
or it's name starts from '.zdebug'.
Currently, we create it if section's data starts from ZLIB signature.

Differential revision: https://reviews.llvm.org/D59018

llvm-svn: 355501
2019-03-06 14:01:54 +00:00
Florian Hahn 74fea3665f [opt] Report if the provided architecture is invalid.
Partly addresses PR15026.

There are a few tests that passed in invalid architectures, which are fixed in: rL355349 and D58931

Reviewers: echristo, efriedma, rengolin, atrick

Reviewed By: efriedma

Differential Revision: https://reviews.llvm.org/D58933

llvm-svn: 355455
2019-03-05 23:10:28 +00:00
Francis Visoiu Mistrih 1c4bab3ba4 [OptRemarks] Make OptRemarks more generic: rename OptRemarks to Remarks
Getting rid of the name "optimization remarks" for anything that
involves handling remarks on the client side.

It's safer to do this now, before we get stuck with that name in all the
APIs and public interfaces we decide to export to users in the future.

This renames llvm/tools/opt-remarks to llvm/tools/remarks-shlib, and now
generates `libRemarks.dylib` instead of `libOptRemarks.dylib`.

Differential Revision: https://reviews.llvm.org/D58535

llvm-svn: 355439
2019-03-05 20:45:17 +00:00
George Rimar ade3c70537 [llvm-objcopy] - Simplify `isCompressable` and fix the issue relative.
When --compress-debug-sections is given, llvm-objcopy do not compress
sections that have "ZLIB" header in data. Normally this signature is used
in zlib-gnu compression format. But if zlib-gnu used then the name of the compressed
section should start from .z* (e.g .zdebug_info). If it does not, then it is not
a zlib-gnu format and section should be treated as a normal uncompressed section.

Differential revision: https://reviews.llvm.org/D58908

llvm-svn: 355399
2019-03-05 13:07:43 +00:00
George Rimar 1e93080ca8 [llvm-objcopy] - Report "no zlib available" error properly when --compress-debug-sections is used.
If zlib is not available, and --compress-debug-sections is passed, 
we want to report an error. Currently, it is only reported for
--compress_debug_sections= form of the option.

Fixes the https://bugs.llvm.org/show_bug.cgi?id=40886.

I do not think there is a way to write a test for this.

Differential revision: https://reviews.llvm.org/D58909

llvm-svn: 355391
2019-03-05 11:32:14 +00:00
Evgeniy Stepanov e800a32b93 Fix wrong enum value in switch.
llvm-svn: 355338
2019-03-04 21:00:28 +00:00
Rong Xu db29a3a438 [PGO] Context sensitive PGO (part 3)
Part 3 of CSPGO changes (mostly related to PassMananger).

Differential Revision: https://reviews.llvm.org/D54175

llvm-svn: 355330
2019-03-04 20:21:27 +00:00
Andrea Di Biagio 9735d9011a [MCA] Correctly initialize struct SummaryView::BackPressureInfo.
This should appease the buildbots.

llvm-svn: 355309
2019-03-04 12:23:05 +00:00
Andrea Di Biagio be3281a281 [MCA] Highlight kernel bottlenecks in the summary view.
This patch adds a new flag named -bottleneck-analysis to print out information
about throughput bottlenecks.

MCA knows how to identify and classify dynamic dispatch stalls. However, it
doesn't know how to analyze and highlight kernel bottlenecks.  The goal of this
patch is to teach MCA how to correlate increases in backend pressure to backend
stalls (and therefore, the loss of throughput).

From a Scheduler point of view, backend pressure is a function of the scheduler
buffer usage (i.e. how the number of uOps in the scheduler buffers changes over
time). Backend pressure increases (or decreases) when there is a mismatch
between the number of opcodes dispatched, and the number of opcodes issued in
the same cycle.  Since buffer resources are limited, continuous increases in
backend pressure would eventually leads to dispatch stalls. So, there is a
strong correlation between dispatch stalls, and how backpressure changed over
time.

This patch teaches how to identify situations where backend pressure increases
due to:
 - unavailable pipeline resources.
 - data dependencies.

Data dependencies may delay execution of instructions and therefore increase the
time that uOps have to spend in the scheduler buffers. That often translates to
an increase in backend pressure which may eventually lead to a bottleneck.
Contention on pipeline resources may also delay execution of instructions, and
lead to a temporary increase in backend pressure.

Internally, the Scheduler classifies instructions based on whether register /
memory operands are available or not.

An instruction is marked as "ready to execute" only if data dependencies are
fully resolved.
Every cycle, the Scheduler attempts to execute all instructions that are ready
to execute. If an instruction cannot execute because of unavailable pipeline
resources, then the Scheduler internally updates a BusyResourceUnits mask with
the ID of each unavailable resource.

ExecuteStage is responsible for tracking changes in backend pressure. If backend
pressure increases during a cycle because of contention on pipeline resources,
then ExecuteStage sends a "backend pressure" event to the listeners.
That event would contain information about instructions delayed by resource
pressure, as well as the BusyResourceUnits mask.

Note that ExecuteStage also knows how to identify situations where backpressure
increased because of delays introduced by data dependencies.

The SummaryView observes "backend pressure" events and prints out a "bottleneck
report".

Example of bottleneck report:

```
Cycles with backend pressure increase [ 99.89% ]
Throughput Bottlenecks:
  Resource Pressure       [ 0.00% ]
  Data Dependencies:      [ 99.89% ]
   - Register Dependencies [ 0.00% ]
   - Memory Dependencies   [ 99.89% ]
```

A bottleneck report is printed out only if increases in backend pressure
eventually caused backend stalls.

About the time complexity:

Time complexity is linear in the number of instructions in the
Scheduler::PendingSet.

The average slowdown tends to be in the range of ~5-6%.
For memory intensive kernels, the slowdown can be significant if flag
-noalias=false is specified. In the worst case scenario I have observed a
slowdown of ~30% when flag -noalias=false was specified.

We can definitely recover part of that slowdown if we optimize class LSUnit (by
doing extra bookkeeping to speedup queries). For now, this new analysis is
disabled by default, and it can be enabled via flag -bottleneck-analysis. Users
of MCA as a library can enable the generation of pressure events through the
constructor of ExecuteStage.

This patch partially addresses https://bugs.llvm.org/show_bug.cgi?id=37494

Differential Revision: https://reviews.llvm.org/D58728

llvm-svn: 355308
2019-03-04 11:52:34 +00:00
Xing GUO b285878907 [llvm-objdump] Should print unknown d_tag in hex format
Summary:
Currently, `llvm-objdump` prints "unknown" instead of d_tag value in hex format. Because getDynamicTagAsString returns "unknown" rather than empty 
string.

Reviewers: grimar, jhenderson

Reviewed By: jhenderson

Subscribers: rupprecht, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D58763

llvm-svn: 355262
2019-03-02 04:20:28 +00:00
Vlad Tsyrklevich 53a9f1d367 Revert "[DWARFFormValue] Cleanup DWARFFormValue interface. (2/2) (NFC)"
This reverts commit r355233, it was causing UBSan failures.

llvm-svn: 355255
2019-03-02 01:10:00 +00:00
Caroline Tice bcdb1f3d04 llvm-dwarfdump: Add new variable, parameter and inlining statistics; also function source location statistics.
Add statistics for abstract origins, function, variable and parameter
locations; break the 'variable' counts down into variables and
parameters. Also update call site counting to check for
DW_AT_call_{file,line} in addition to DW_TAG_call_site.

Differential revision: https://reviews.llvm.org/D58849

llvm-svn: 355243
2019-03-01 23:51:54 +00:00
Jonas Devlieghere 2dc2baa8cc [DWARFFormValue] Cleanup DWARFFormValue interface. (2/2) (NFC)
Continues the work started in r354941. Changes (all but one) uses of the
extractValue to static createFromData.

llvm-svn: 355233
2019-03-01 22:14:24 +00:00
Paul Robinson 1ca25763f0 [DWARF] Make -g with empty assembler source work better.
This was sometimes causing clang or llvm-mc to crash, and in other
cases could emit a bogus DWARF line-table header. I did an interim
patch in r352541; this patch should be a cleaner and more complete
fix, and retains the test.

Addresses PR40538.

Differential Revision: https://reviews.llvm.org/D58750

llvm-svn: 355226
2019-03-01 20:58:04 +00:00
Matt Davis 8a6f11f45f [llvm-readobj] Display section names for STT_SECTION symbols.
Summary:
This patch will obtain the section name for symbols that refer to a section.  Prior to this patch the Name field for STT_SECTIONs was blank, now it is populated.

Before:
```
Symbol table '.symtab' contains 6 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND
     1: 0000000000000000     0 SECTION LOCAL  DEFAULT    1
     2: 0000000000000000     0 SECTION LOCAL  DEFAULT    3
     3: 0000000000000000     0 SECTION LOCAL  DEFAULT    4
     4: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND _GLOBAL_OFFSET_TABLE_
     5: 0000000000000000     0 TLS     GLOBAL DEFAULT  UND sym
```

With this patch:
```
Symbol table '.symtab' contains 6 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND
     1: 0000000000000000     0 SECTION LOCAL  DEFAULT    1 .text
     2: 0000000000000000     0 SECTION LOCAL  DEFAULT    3 .data
     3: 0000000000000000     0 SECTION LOCAL  DEFAULT    4 .bss
     4: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND _GLOBAL_OFFSET_TABLE_
     5: 0000000000000000     0 TLS     GLOBAL DEFAULT  UND sym
```

This fixes PR40788

Reviewers: jhenderson, rupprecht, espindola

Reviewed By: rupprecht

Subscribers: emaste, javed.absar, arichardson, MaskRay, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D58796

llvm-svn: 355207
2019-03-01 17:31:32 +00:00
George Rimar a7ba1a0f81 [yaml2obj] - Allow setting custom sh_info for RawContentSection sections.
This is for tweaking SHT_SYMTAB sections.
Their sh_info contains the (number of symbols + 1) usually.
But for creating invalid inputs for test cases it would be convenient
to allow explicitly override this field from YAML.

Differential revision: https://reviews.llvm.org/D58779

llvm-svn: 355193
2019-03-01 10:18:16 +00:00
Alexey Lapshin b0224b1f1d Attempt to fix buildbot after r354972 [#2]. NFCI.
llvm-svn: 355192
2019-03-01 10:15:18 +00:00
Igor Kudrin a38432cefb [CommandLine] Allow grouping options which can have values.
This patch allows all forms of values for options to be used at the end
of a group. With the fix, it is possible to follow the way GNU binutils
tools handle grouping options better. For example, the -j option can be
used with objdump in any of the following ways:

$ objdump -d -j .text a.o
$ objdump -d -j.text a.o
$ objdump -dj .text a.o
$ objdump -dj.text a.o

Differential Revision: https://reviews.llvm.org/D58711

llvm-svn: 355185
2019-03-01 09:22:42 +00:00
Peter Collingbourne ac136cd18c llvm-readobj: Try the DWARF CFI dumper on all machines.
There's no reason to limit the DWARF CFI dumper to EM_386 and EM_X86_64;
ELF files could contain DWARF CFI on almost any platform (even 32-bit ARM;
NetBSD uses DWARF CFI on that platform). So start using the DWARF CFI dumper
unconditionally so that we can dump .eh_frame sections on the remaining ELF
platforms as well as in NetBSD binaries.

Differential Revision: https://reviews.llvm.org/D58761

llvm-svn: 355151
2019-02-28 22:42:55 +00:00
Adrian Prantl fa37a00044 dsymutil support for DW_OP_convert
Add support for cloning DWARF expressions that contain base type DIE
references in dsymutil.

<rdar://problem/48167812>

Differential Revision: https://reviews.llvm.org/D58534

llvm-svn: 355148
2019-02-28 22:12:32 +00:00
Rong Xu a6ff69f6dd [PGO] Context sensitive PGO (part 2)
Part 2 of CSPGO changes (mostly related to ProfileSummary).
Note that I use a default parameter in setProfileSummary() and getSummary().
This is to break the dependency in clang. I will make the parameter explicit
after changing clang in a separated patch.

Differential Revision: https://reviews.llvm.org/D54175

llvm-svn: 355131
2019-02-28 19:55:07 +00:00
Jonas Devlieghere f9b0b9a504 [dsymutil] Use rfind for paths with parentheses
Dsymutil gets library member information is through the ambiguous
/path/to/archive.a(member.o). The current logic we use would get
confused by additional parentheses. Using rfind mitigates this issue.

llvm-svn: 355114
2019-02-28 18:46:04 +00:00
Tom Stellard d20073f47b llvm-config: Include -stdlib= in --cxxflags
Summary:
This was removed in r349068, but it is needed when llvm is compiled
using the non-default c++ standard library on a platform.

Reviewers: sylvestre.ledru, infinity0, mgorny, cuviper

Reviewed By: sylvestre.ledru

Subscribers: jdoerfert, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D57859

llvm-svn: 355107
2019-02-28 17:31:59 +00:00
George Rimar bf447a5bb5 [llvm-objdump] - Improve the error message for "removing a section that is used by relocation" case.
This refines/improves the error message introduced in D58625

Differential revision: https://reviews.llvm.org/D58709

llvm-svn: 355074
2019-02-28 08:21:50 +00:00
George Rimar 7b4fce12b3 [llvm-readobj] - Fix the invalid dumping of the dynamic sections without terminating DT_NULL entry.
This is https://bugs.llvm.org/show_bug.cgi?id=40861,

Previously llvm-readobj would print the DT_NULL sometimes
for the dynamic section that has no terminator entry.

The logic of printDynamicTable was a bit overcomplicated.
I rewrote it slightly to fix the issue and commented.

Differential revision: https://reviews.llvm.org/D58716

llvm-svn: 355073
2019-02-28 08:15:59 +00:00
Matt Davis 1d5c23523e [llvm-cxxfilt] Re-enable split and demangle stdin input on certain non-alphanumerics.
This restores the patch that splits demangled stdin input on
non-alphanumerics.  I had reverted this patch earlier because it broke
Windows build-bots.  I have updated the test so that it passes on
Windows.

I was running the test from powershell and never saw the issue until I
switched to the mingw shell.

This reverts commit 628ab5c682.

llvm-svn: 355031
2019-02-27 21:39:11 +00:00
Matt Davis 628ab5c682 Revert "[llvm-cxxfilt] Split and demangle stdin input on certain non-alphanumerics."
This reverts commit 5cd5f8f256.
The test passes on linux, but fails on the windows build-bots.

This test failure seems to be a quoting issue between my test and
FileCheck on Windows.  I'm reverting this patch until I can replicate
and fix in my Windows environment.

llvm-svn: 355021
2019-02-27 19:52:02 +00:00
Matt Davis 7a24dbdfd3 [llvm-readobj] Print section type values for unknown sections.
Summary:
This patch displays a hexadecimal section value (Elf_Shdr::sh_type) or section-relative offset when printing unknown sections.

Here is a subset of the output (ignoring the fields following "Type" when dumping an ELF's GNU `--section-headers` table).
Section Headers:
```
  [Nr] Name              Type
  [16] android_rel       LOOS+0x1
  [17] android_rela      LOOS+0x2
  [27] unknown           0x1000: <unknown>
  [28] loos              LOOS+0
  [30] hios              VERSYM
  [31] loproc            LOPROC+0
  [33] hiproc            LOPROC+0xFFFFFFF
  [34] louser            LOUSER+0
  [36] hiuser            LOUSER+0x7FFFFFFF
```

As a comparison, the previous output looked something like the above, but with a blank "Type" field:
```
  [Nr] Name              Type
  [27] unknown
  [28] loos
  [30] hios              VERSYM
  [31] loproc
  [33] hiproc
  [34] louser
  [36] hiuser
```

This fixes PR40773

Reviewers: jhenderson, rupprecht, Bigcheese

Reviewed By: jhenderson, rupprecht, Bigcheese

Subscribers: MaskRay, Bigcheese, srhines, jdoerfert, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D58701

llvm-svn: 355014
2019-02-27 18:39:17 +00:00
Rong Xu 6cdf3d8086 Recommit r354930 "[PGO] Context sensitive PGO (part 1)"
Fixed UBSan failures.

llvm-svn: 355005
2019-02-27 17:24:33 +00:00
Xing GUO d78164a8ab [llvm-objdump] Should print strings when dumping DT_RPATH, DT_RUNPATH, DT_SONAME, DT_AUXILIARY and DT_FILTER tags in dynamic section.
Summary:
Before:
```
Dynamic Section:
  NEEDED               libpthread.so.0
  ...
  NEEDED               ld-linux-x86-64.so.2
  RPATH                0x00000000001c2e61
```

After:
```
Dynamic Section:
  NEEDED               libpthread.so.0
  ...
  NEEDED               ld-linux-x86-64.so.2
  RPATH                $ORIGIN/../lib
```

Only a small problem here, I have no idea on choosing test case. I see there's a test 
file(test/tools/llvm-objdump/private-headers-dynamic-section.test). But it has no DT_RPATH and DT_RUNPATH tags. Shall I replace the ELF file in the 
Inputs dir by a new one?

Reviewers: jhenderson, grimar

Reviewed By: jhenderson

Subscribers: srhines, rupprecht, jfb, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D58707

llvm-svn: 355001
2019-02-27 16:37:15 +00:00
Matt Davis 5cd5f8f256 [llvm-cxxfilt] Split and demangle stdin input on certain non-alphanumerics.
Summary:
This patch attempts to replicate GNU c++-filt behavior when splitting stdin input for demangling.

Previously, cxx-filt would split input only on spaces.  Each delimited item is then demangled.
From what I have tested, GNU c++filt also splits input on any character that does not make
up the mangled name (notably commas, but also a large set of non-alphanumeric characters).

This patch splits stdin input on any character that does not belong to the Itanium mangling
format (since Itanium is currently the only supported format in llvm-cxxfilt).

This is an update to PR39990

Reviewers: jhenderson, tejohnson, compnerd

Reviewed By: compnerd

Subscribers: erik.pilkington, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D58416

llvm-svn: 354998
2019-02-27 16:29:50 +00:00
Alexey Lapshin 77fc1f6049 [DebugInfo] add SectionedAddress to DebugInfo interfaces.
That patch is the fix for https://bugs.llvm.org/show_bug.cgi?id=40703
   "wrong line number info for obj file compiled with -ffunction-sections"
   bug. The problem happened with only .o files. If object file contains
   several .text sections then line number information showed incorrectly.
   The reason for this is that DwarfLineTable could not detect section which
   corresponds to specified address(because address is the local to the
   section). And as the result it could not select proper sequence in the
   line table. The fix is to pass SectionIndex with the address. So that it
   would be possible to differentiate addresses from various sections. With
   this fix llvm-objdump shows correct line numbers for disassembled code.

   Differential review: https://reviews.llvm.org/D58194

llvm-svn: 354972
2019-02-27 13:17:36 +00:00
George Rimar 79fb858053 [llvm-objcopy] - Check for invalidated relocations when removing a section.
This is https://bugs.llvm.org/show_bug.cgi?id=40818

Removing a section that is used by relocation is an error
we did not report. The patch fixes that.

Differential revision: https://reviews.llvm.org/D58625

llvm-svn: 354962
2019-02-27 11:18:27 +00:00
James Henderson 5b27402bee [llvm-readobj]Fix error messages for bad archive members and add testing for archive handling
llvm-readobj's error messages were broken for bad archive members. This
patch fixes them, and also adds testing for archive and thin archive
handling within llvm-readobj.

Reviewed by: rupprecht, grimar, higuoxing

Differential Revision: https://reviews.llvm.org/D58681

llvm-svn: 354960
2019-02-27 11:07:08 +00:00
Simon Pilgrim 65706cf715 Fix Wenum-compare gcc7 warning. NFCI.
llvm-svn: 354958
2019-02-27 10:19:53 +00:00
Fangrui Song 73f16996de [llvm-readobj] Print DF_1_DISPRELPND
The test will be added by D58677.

llvm-svn: 354955
2019-02-27 05:37:11 +00:00
Vlad Tsyrklevich c01643087e Revert "[PGO] Context sensitive PGO (part 1)"
This reverts commit r354930, it was causing UBSan failures.

llvm-svn: 354953
2019-02-27 03:45:28 +00:00
Rong Xu 35d2d51369 [PGO] Context sensitive PGO (part 1)
Current PGO profile counts are not context sensitive. The branch probabilities
for the inlined functions are kept the same for all call-sites, and they might
be very different from the actual branch probabilities. These suboptimal
profiles can greatly affect some downstream optimizations, in particular for
the machine basic block placement optimization.

In this patch, we propose to have a post-inline PGO instrumentation/use pass,
which we called Context Sensitive PGO (CSPGO). For the users who want the best
possible performance, they can perform a second round of PGO instrument/use on
the top of the regular PGO. They will have two sets of profile counts. The
first pass profile will be manly for inline, indirect-call promotion, and
CGSCC simplification pass optimizations. The second pass profile is for
post-inline optimizations and code-gen optimizations.

A typical usage:
// Regular PGO instrumentation and generate pass1 profile.
> clang -O2 -fprofile-generate source.c -o gen
> ./gen
> llvm-profdata merge default.*profraw -o pass1.profdata
// CSPGO instrumentation.
> clang -O2 -fprofile-use=pass1.profdata -fcs-profile-generate -o gen2
> ./gen2
// Merge two sets of profiles
> llvm-profdata merge default.*profraw pass1.profdata -o profile.profdata
// Use the combined profile. Pass manager will invoke two PGO use passes.
> clang -O2 -fprofile-use=profile.profdata -o use

This change touches many components in the compiler. The reviewed patch
(D54175) will committed in phrases.

Differential Revision: https://reviews.llvm.org/D54175

llvm-svn: 354930
2019-02-26 22:37:46 +00:00
Xing GUO 85b50a7679 [llvm-objdump] Add `Version Definitions` dumper
Summary: `llvm-objdump` needs a `Version Definitions` dumper.

Reviewers: grimar, jhenderson

Reviewed By: grimar, jhenderson

Subscribers: rupprecht, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D58615

llvm-svn: 354871
2019-02-26 13:06:16 +00:00
Igor Kudrin 2d3faad706 [llvm-objdump] Implement -Mreg-names-raw/-std options.
The --disassembler-options, or -M, are used to customize
the disassembler and affect its output.

The two implemented options allow selecting register names on ARM:
* With -Mreg-names-raw, the disassembler uses rNN for all registers.
* With -Mreg-names-std it prints sp, lr and pc for r13, r14 and r15,
  which is the default behavior of llvm-objdump.

Differential Revision: https://reviews.llvm.org/D57680

llvm-svn: 354870
2019-02-26 12:15:14 +00:00
Clement Courbet 0ddf81c43d [llvm-exegesis] Teach llvm-exegesis to handle instructions with multiple tied variables.
Reviewers: gchatelet

Subscribers: tschuett, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D58285

llvm-svn: 354862
2019-02-26 10:54:45 +00:00
Eugene Leviant 53350d0411 [llvm-objcopy] Add --set-start, --change-start and --adjust-start
Differential revision: https://reviews.llvm.org/D58173

llvm-svn: 354854
2019-02-26 09:24:22 +00:00
Vlad Tsyrklevich c6d54ae9da Revert "Improve "llvm-nm -f sysv" output for Elf files"
This reverts commit r354833, it was causing ASan test failures on
sanitizer-x86_64-linux-fast.

llvm-svn: 354849
2019-02-26 07:04:56 +00:00
Sunil Srivastava d72d16f444 Improve "llvm-nm -f sysv" output for Elf files
Specifically, compute and Print Type and Section columns.

Differential Revision: https://reviews.llvm.org/D58263

llvm-svn: 354833
2019-02-26 00:19:39 +00:00
Eugene Leviant 51c1f640aa [llvm-objcopy] Add --add-symbol
Differential revision: https://reviews.llvm.org/D58234

llvm-svn: 354787
2019-02-25 14:12:41 +00:00
Xing GUO 56d651db0f [llvm-objdump] Add `Version References` dumper
Summary: Add symbol version dumper for [#30241](https://bugs.llvm.org/show_bug.cgi?id=30241)

Reviewers: jhenderson, MaskRay, kristina, emaste, grimar

Reviewed By: jhenderson, grimar

Subscribers: grimar, rupprecht, jakehehrlich, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D54697

llvm-svn: 354782
2019-02-25 13:13:19 +00:00