Commit Graph

4653 Commits

Author SHA1 Message Date
Eric Astor 6b70a83d9c [ms] [llvm-ml] Add support for .radix directive, and accept all radix specifiers
Add support for .radix directive, and radix specifiers [yY] (binary), [oOqQ] (octal), and [tT] (decimal).

Also, when lexing MASM integers, require radix specifier; MASM requires that all literals without a radix specifier be treated as in the default radix. (e.g., 0100 = 100)

Relanding D87400, now with fewer ms-inline-asm tests broken!

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D88337
2020-09-29 16:55:51 -04:00
Georgii Rymar 4ba00619ee [llvm-readobj/elf] - Fix the PREL31 relocation computation used for dumping arm32 unwind info (-u).
This is a part of https://bugs.llvm.org/show_bug.cgi?id=47581.

We have the following computation:
```
(1) uint64_t Location = Address & 0x7fffffff;
(2) if (Location & 0x04000000)
(3)   Location |= (uint64_t) ~0x7fffffff;
(4) return Location + Place;
```

At line 2 there is a mistype. The constant should be `0x40000000`,
not `0x04000000`, because the intention here is to sign extend the `Location`,
which is the 31 bit signed value.

Differential revision: https://reviews.llvm.org/D88407
2020-09-28 16:22:56 +03:00
Georgii Rymar dab9917164 [yaml2obj][obj2yaml] - Add a support for SHT_ARM_EXIDX section.
This adds the support for SHT_ARM_EXIDX sections to obj2yaml/yaml2obj tools.

SHT_ARM_EXIDX is a ARM specific index table filled with entries.
Each entry consists of two 4-bytes values (words).
(https://developer.arm.com/documentation/ihi0038/c/?lang=en#index-table-entries)

Differential revision: https://reviews.llvm.org/D88228
2020-09-28 11:45:49 +03:00
Georgii Rymar ea0f66e848 [obj2yaml][yaml2obj] - Stop recognizing SHT_MIPS_ABIFLAGS on non-MIPS targets.
Currently we are always recognizing the `SHT_MIPS_ABIFLAGS` section,
even on non-MIPS targets.

The problem of doing this is briefly discussed in D88228 which does the same for `SHT_ARM_EXIDX`:

"The problem is that `SHT_ARM_EXIDX` shares the value with `SHT_X86_64_UNWIND (0x70000001U)`.
We might have other machine specific conflicts, e.g.
`SHT_ARM_ATTRIBUTES` vs `SHT_MSP430_ATTRIBUTES` vs `SHT_RISCV_ATTRIBUTES (0x70000003U)`."

I think we should only recognize target specific sections when the machine type
matches. I.e. `SHT_MIPS_*` should be recognized only on `MIPS`, `SHT_ARM_*`
only on `ARM` etc.

This patch stops recognizing `SHT_MIPS_ABIFLAGS` on `non-MIPS` targets.

Note: I had to update `ScalarEnumerationTraits<ELFYAML::MIPS_ISA>::enumeration`, because
otherwise test crashes, calling `llvm_unreachable`.

Differential revision: https://reviews.llvm.org/D88294
2020-09-28 11:28:53 +03:00
Andrew Litteken 69c6f6be07 Revert "[IRSim] Adding basic implementation of llvm-sim."
This reverts commit 15645d044b.
2020-09-25 16:18:48 -05:00
Andrew Litteken 15645d044b [IRSim] Adding basic implementation of llvm-sim.
This is a similarity visualization tool that accepts a Module and
passes it to the IRSimilarityIdentifier.  The resulting SimilarityGroups
are output in a JSON file.

Tests are found in test/tools/llvm-sim and check for the file not found,
a bad module, and that the JSON is created correctly.

Reviewers: paquette, jroelofs

Differential Revision: https://reviews.llvm.org/D86974
2020-09-25 15:12:34 -05:00
Georgii Rymar 64f878db50 [Object/yaml2obj/obj2yaml][test] - Split, cleanup and move MIPS abi-flags.yaml test. NFCI.
We have the `Object/Mips/abi-flags.yaml` which tests how yaml2obj/obj2yaml
handle `SHT_MIPS_ABIFLAGS` sections.

This patch splits it into two tests: one for obj2yaml and one for yaml2obj
and moves the result to right places.

Differential revision: https://reviews.llvm.org/D88231
2020-09-25 12:04:55 +03:00
Georgii Rymar bb0597067d [llvm-readelf/obj] - Stop printing wrong addresses for arm32 unwind info for non-relocatable objects.
This is the first patch for https://bugs.llvm.org/show_bug.cgi?id=47581.

Currently -u does not compute function addresses correctly and
dumps broken addresses for non-relocatable objects.

ARM spec says:
"An index table entry consists of 2 words.
The first word contains a prel31 offset (see Relocations) to the start of a function, with bit 31 clear."
...
"The relocated 31 bits form a place-relative signed offset to the referenced entity.
For brevity, this document will refer to the results of these relocations as "prel31 offsets"."

(https://developer.arm.com/documentation/ihi0038/c/?lang=en#index-table-entries)
(https://developer.arm.com/documentation/ihi0038/c/?lang=en#relocations)

Currently we use an address of the SHT_ARM_EXIDX section instead of an address of an entry
in computations. As a result we compute an offset that is not really "place-relative",
but section relative, what is wrong.

The patch fixes this issue.

Differential revision: https://reviews.llvm.org/D88076
2020-09-24 13:21:00 +03:00
Alexander Shaposhnikov e60a760b7d [llvm-objcopy][MachO] Fix --add-section
This diff fixes --add-section functionality and simplifies the tests organization.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D87497
2020-09-24 01:51:10 -07:00
Xing GUO 505ac22f1a [DWARFYAML] Make the ExtLen field of extended opcodes optional.
This patch makes the 'ExtLen' field of extended opcodes optional. We
don't need to manually calculate it in the future.

Reviewed By: jhenderson, MaskRay

Differential Revision: https://reviews.llvm.org/D88136
2020-09-24 14:13:26 +08:00
Jonas Devlieghere e1ef7183c6 [dwarfdump] Warn for tags with DW_CHILDREN_yes but no children.
Flag DIEs that have DW_CHILDREN_yes set in their abbreviation but don't
actually have any children.

rdar://59809554

Differential revision: https://reviews.llvm.org/D88048
2020-09-23 22:12:04 -07:00
Eric Astor b901b6ab17 Revert "[ms] [llvm-ml] Add support for .radix directive, and accept all radix specifiers"
This reverts commit 5dd1b6d612.
2020-09-23 13:59:34 -04:00
Eric Astor 5dd1b6d612 [ms] [llvm-ml] Add support for .radix directive, and accept all radix specifiers
Add support for .radix directive, and radix specifiers [yY] (binary), [oOqQ] (octal), and [tT] (decimal).

Also, when lexing MASM integers, require radix specifier; MASM requires that all literals without a radix specifier be treated as in the default radix. (e.g., 0100 = 100)

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D87400
2020-09-23 13:45:58 -04:00
David Greene b877933784 [UpdateTestChecks] Remove bug-exposing test
Remove RISCV codegen tests for --include-generated-funcs because apparently
MachineOutliner has a bug on that target that is exposed by expensive-checks.
2020-09-23 11:28:28 -05:00
Sebastian Neubauer a343b9b032 Revert "[AMDGPU] Insert waitcnt after returning from call"
This reverts commit ca907bfb57.

According to michel.daenzer,
> This completely broke the Mesa radeonsi driver on Navi 14. Xorg +
> xterm come up with major corruption & psychedelic colours.
2020-09-23 17:16:39 +02:00
Georgii Rymar bd99fb4e0b [llvm-readelf/obj] - Fix extended section symbol indices printed in warnings for MIPS GOT/PLT entries.
Recent refactoring introduced a symbol index argument for `getFullSymbolName` method,
which is only used for reporting error messages about invalid extended symbol indexes.

There are few issues in the implementation and we don't report correct symbol indices
when dumping MIPS GOT/PLT entries currently.

This patch adds test cases and fixes the issue.

Differential revision: https://reviews.llvm.org/D88089
2020-09-23 13:42:56 +03:00
Georgii Rymar d4035af253 [llvm-readelf/obj] - Print section symbol names properly when dumping relocations.
Currently `--relocations` ignores section symbol names and always prints
section names for them. This is inconsistent with GNU readelf and with `--symbols`.

We have a code in `getFullSymbolName` (which is used for `--symbols`) which can be
reused for `getRelocationTarget` (used for `--relocations`).
With that the issue described is fixed and code becomes a bit shorter.
Also with this change we start to print more relocations (in situations when we just
showed warnings instead before) and also start to report more diagnostic warnings
(see reloc-zero-name-or-value.test).

Differential revision: https://reviews.llvm.org/D87613
2020-09-23 13:25:39 +03:00
Sebastian Neubauer ca907bfb57 [AMDGPU] Insert waitcnt after returning from call
When memory operations are outstanding on function calls, either the
caller or the callee can insert a waitcnt to ensure that all reads are
finished.
Calls need some time to be executed, so if the callee inserts the
waitcnt, filling the instruction buffer and waiting for memory will be
interleaved, hiding some latency. This comes at the cost of having a
waitcnt inside functions that may not be needed as no memory operations
are outstanding.

For function calls, this is already implemented. The same principal
applies to returns: If the caller inserts a waitcnt after the call, the
callee does not have to wait and the return and memory operation can be
run in parallel.

This commit implements waiting in the caller after returning from a
function call.

Differential Revision: https://reviews.llvm.org/D87674
2020-09-23 12:17:59 +02:00
Georgii Rymar 28b84dd138 [llvm-readobj/elf] - Stop reporting invalid extended indexes in warnings for unnamed section symbols.
We have an issue with `getFullSymbolName`: it assumes that the symbol passed is
always in the `.symtab`, what is wrong. We might calculate and report a wrong index currently.
I've added a test case revealing that.

This patch adds the "symbol index" argument to `getFullSymbolName` signature,
what fixes the issue.

Differential revision: https://reviews.llvm.org/D87899
2020-09-22 11:55:15 +03:00
Georgii Rymar 095f6fbbd7 [llvm-readelf/obj] - Stop printing invalid names for unnamed section symbols.
We have an issue with `ELFDumper<ELFT>::getSymbolSectionName`:
1) It is used deeply for both LLVM/GNU styles and might return LLVM-style only
   values to describe symbols: "Undefined", "Processor Specific", "Absolute", etc.

2) `getSymbolSectionName` is used by `getFullSymbolName` and these special values
   might appear instead of symbol names in many places.
   This occurs for unnamed section symbols currently.

This patch extracts the LLVM specific logic to `LLVMStyle<ELFT>::printSymbolSection`,
which seems to be the only place where we want to print the special values mentioned.
It also adds a meaningful new warning that is reported when we are unable to get
a section index for a section symbol.

Differential revision: https://reviews.llvm.org/D87764
2020-09-21 13:05:46 +03:00
wlei a8b8a9374a [llvm-profdata]Fix llvm-profdata crash on compact binary profile
llvm-profdata `show` and `overlap` will crash in `getFuncName` on compact binary profile. This change fixed this by switching to use `getName`.

 `getFuncName` is misused in llvm-profdata. As showed below, `GUIDToFuncNameMap` is only supported in compilation mode, there is no initialization in llvm-profdata. Compact profile whose MD5 is true would try to query `GUIDToFuncNameMap` then caused the crash. So fix this by switching to `getName`

Reviewed By: MaskRay, wmi, wenlei, weihe, hoy

Differential Revision: https://reviews.llvm.org/D87740
2020-09-20 16:58:34 -07:00
David Blaikie ad68a8b952 DebugInfo: Cleanup RLE dumping, using a length-constrained DataExtractor rather than carrying the end offset separately 2020-09-18 19:32:38 -07:00
Alexander Shaposhnikov 5495b69164 [llvm-objcopy][MachO] Add llvm-bitcode-strip driver
This diff adds llvm-bitcode-strip driver to llvm-objcopy.
In the future this will enable us to build a replacement for the tool bitcode_strip.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D87212
2020-09-18 18:13:05 -07:00
Vedant Kumar 3c731ba5f1 [llvm-cov] Allow commas in filenames passed to `-object` flag
Currently, -object takes a comma separated list of objects as an
argument, which prevents it working with path names that contain a
comma. Drop comma-separated support, which requires to set pass the
-object flag multiple times to set multiple objects.

Patch by Andrew Gallagher!

Differential Revision: https://reviews.llvm.org/D87003
2020-09-18 13:46:29 -07:00
Xing GUO 2d35092cd2 [DWARFYAML] Make the include_directories, file_names and opcodes fields of the line table optional.
This patch makes the include_directories, file_names and opcodes fields
of the line table optional. This helps us simplify some tests.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D87878
2020-09-18 20:21:11 +08:00
David Greene 7c8bb409f3 [UpdateCCTestChecks] Include generated functions if asked
Add the --include-generated-funcs option to update_cc_test_checks.py so that any
functions created by the compiler that don't exist in the source will also be
checked.

We need to maintain the output order of generated function checks so that
CHECK-LABEL works properly.  To do so, maintain a list of functions output for
each prefix in the order they are output.  Use this list to output checks for
generated functions in the proper order.

Differential Revision: https://reviews.llvm.org/D83004
2020-09-18 06:34:59 -05:00
Georgii Rymar 279943edf8 [obj2yaml] - Don't emit EM_NONE.
When ELF header's `e_machine == 0`, we emit:

```
Machine: EM_NONE
```

We can avoid doing this, because yaml2obj sets the
`e_machine` field to `EM_NONE` by default.

Differential revision: https://reviews.llvm.org/D87829
2020-09-17 15:58:44 +03:00
Georgii Rymar 0dca1ac617 [llvm-readelf/obj][test] - Document what we print in various places for unnamed section symbols.
We have an issue with `ELFDumper<ELFT>::getSymbolSectionName`:
1) It is used deeply for both LLVM/GNU styles and might return LLVM-style only
   values to describe symbols: "Undefined", "Processor Specific", "Absolute", etc.

2) `getSymbolSectionName` is used by `getFullSymbolName` and these special values
   might appear in instead of symbol names in many places.
   This occurs for unnamed section symbols.

It was not noticed because for most cases I've found it is unexpected to have an
unnamed section symbol. This patch documents the existent behavior, adds tests and FIXMEs.

Differential revision: https://reviews.llvm.org/D87763
2020-09-17 15:56:51 +03:00
Fangrui Song c16417f65f [llvm-cov gcov] Add --demangled-names (-m)
gcov 4.9 introduced the option.
2020-09-16 23:18:50 -07:00
Fangrui Song e69092be52 [llvm-cov gcov][test] Move tests to gcov/
And rename llvm-cov.test (misnomer) to basic.test
2020-09-16 22:42:49 -07:00
Rahman Lavaee b1cb9d6271 [obj2yaml] - Match ".stack_size" with the original section name, and not the uniquified name.
Without this patch, obj2yaml decodes the content of only one ".stack_size" section. Other sections are dumped with their full contents.

Reviewed By: grimar, MaskRay

Differential Revision: https://reviews.llvm.org/D87727
2020-09-16 14:17:29 -07:00
David Greene ce0eb81c72 [UpdateTestChecks] Allow $ in function names
Some compilers generation functions with '$' in their names, so recognize those
functions.

This also requires recognizing function names inside quotes in some contexts in
order to escape certain characters.

Differential Revision: https://reviews.llvm.org/D82995
2020-09-16 14:34:18 -05:00
Rahman Lavaee f80f2516a2 Revert "[obj2yaml] - Match ".stack_size" with the original section name, and not the uniquified name."
This reverts commit 14e55f8298.
2020-09-16 11:42:37 -07:00
Rahman Lavaee 14e55f8298 [obj2yaml] - Match ".stack_size" with the original section name, and not the uniquified name.
Without this patch, obj2yaml decodes the content of only one ".stack_size" section. Other sections are dumped with their full contents.

Reviewed By: grimar, MaskRay

Differential Revision: https://reviews.llvm.org/D87727
2020-09-16 11:33:20 -07:00
Georgii Rymar a8d02015fc [llvm-readobj][test] - Improve section-symbols.test
`section-symbols.test` tests how we print section symbols in
different situations.

We might have 2 different cases:
1) A named STT_SECTION symbol.
2) An unnamed STT_SECTION symbol.

Usually section symbols have no name and then `--symbols` uses their
section names when prints them. If symbol has a name, then it is used.

For `--relocations` we also want to have this logic probably,
but currently we always ignore symbol names and always use section names.
It is not consistent with GNU readelf and with our logic for `--symbols`.

This patch refines testing to document the existent behavior and improve
coverage.

Differential revision: https://reviews.llvm.org/D87612
2020-09-16 12:36:09 +03:00
Georgii Rymar ef4851742d [llvm-readobj][test] - Address a forgotten review comment for D86923.
Seems I've forgot to address this bit and this looks like a reason
of a failture on mac (http://45.33.8.238/mac/20491/step_11.txt).
2020-09-16 11:51:26 +03:00
Xing GUO 4171d5c30a [obj2yaml] Add support for dumping the .debug_addr(v5) section.
This patch adds support for dumping the .debug_addr(v5) section to
obj2yaml.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D87601
2020-09-16 14:48:03 +08:00
Martin Storsjö 794467b916 [llvm-rc] Allow omitting components from VERSIONINFO versions
MS rc.exe doesn't require specifying all 4 components.

Differential Revision: https://reviews.llvm.org/D87570
2020-09-16 09:34:26 +03:00
Petr Hosek 9c73e55510 Revert "[DebugInfo] Remove dots from getFilenameByIndex return value"
This is failing on Windows bots due to path separator normalization.

This reverts commit 042c235068.
2020-09-15 10:06:47 -07:00
Georgii Rymar 7df873f9c6 [llvm-readobj/elf] - Don't crash when the size of s dynamic symbol table, inferred from the hash table, is broken.
Currently we might derive the dynamic symbol table size from the DT_HASH hash table (using its `nchain` field).
It is possible to crash dumpers with a broken relocation that refers to a symbol with an index
that is too large. To trigger it, the inferred size of the dynamic symbol table should go past the end of the object.

This patch adds a size validation + warning.

Differential revision: https://reviews.llvm.org/D86923
2020-09-15 17:44:16 +03:00
Martin Storsjö 7b416c5e36 [llvm-readobj] [ARMWinEH] Print ARM64 packed unwind info
In addition to printing the individual fields, synthesize and
print the corresponding prolog for the unwind info (in reverse
order, to match how it's printed for non-packed unwind info).

Differential Revision: https://reviews.llvm.org/D87370
2020-09-15 08:50:02 +03:00
Petr Hosek 042c235068 [DebugInfo] Remove dots from getFilenameByIndex return value
When concatenating directory with filename in getFilenameByIndex, we
might end up with a path that contains extra dots. For example, if the
input is /path and ./example, we would return /path/./example. Run
sys::path::remove_dots on the output to eliminate unnecessary dots.

Differential Revision: https://reviews.llvm.org/D87657
2020-09-14 20:19:06 -07:00
Eric Astor 23a2b03221 [ms] [llvm-ml] Add basic support for SEH, including PROC FRAME
Add basic support for SEH, including PROC FRAME

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D86948
2020-09-14 14:32:55 -04:00
Eric Astor 20201dc76a [ms] [llvm-ml] Add support for size queries in MASM
Add support for size inference, sizeof, typeof, and lengthof.

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D86947
2020-09-14 14:27:06 -04:00
Eric Astor 7c44ee8e19 [ms] [llvm-ml] Fix struct padding logic
MASM structs are end-padded to have size a multiple of the smaller of the requested alignment and the size of their largest field (taken recursively, if they have a field of STRUCT type).

This matches the behavior of ml.exe and ml64.exe. Our original implementation followed the MASM 6.0 documentation, which instead specified that MASM structs were padded to a multiple of their requested alignment.

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D87248
2020-09-14 14:12:20 -04:00
Eric Astor da17e0d5c1 [ms] [llvm-ml] Add missing built-in type aliases
Add signed aliases for integral types, as well as the "DF" abbreviation for the FWORD type.

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D87246
2020-09-14 14:09:24 -04:00
Georgii Rymar 7448e64a79 [llvm-readobj/elf] - Don't use unwrapOrError when reporting errors about SHT_DYNAMIC sections.
This changes messages reported to stop using dynamic section names (use `describe()` instead).
This allows to avoid `unwrapOrError` and improves diagnostics.

Differential revision: https://reviews.llvm.org/D87503
2020-09-14 14:55:16 +03:00
Georgii Rymar 14e191a0e7 [llvm-readobj] - Cleanup implementation LLVMStyle<ELFT>::printAddrsig().
It has following issues:
1) `getStaticSymbolName` returns `std::string`, but the code
   assigns a result to `Expected<std::string>`.
2) The code uses `unwrapOrError` and never tests the error reported.

This patch fixes these issues.

Differential revision: https://reviews.llvm.org/D87507
2020-09-14 14:52:06 +03:00
Georgii Rymar e9c314611b [llvm-readelf/obj] - Refine and generalize the code that is used to dump notes.
There is some code that can be shared between GNU/LLVM styles.
Also, this fixes 2 inconsistencies related to dumping unknown note types:
1) For GNU style we printed "Unknown note type: (0x00000003)" in some cases, and
   "Unknown note type (0x00000003)" (no colon) in other cases.
   GNU readelf always prints `:`. This patch removes the related code
   duplication and does the same.
2) For LLVM style in some cases we printed "Unknown note type (0x00000003)",
   but sometimes just "Unknown (0x00000003)". The latter is the right form, which
   is consistent with other unknowns that are printed in LLVM style.

Rebased on top of D87453.

Differential revision: https://reviews.llvm.org/D87454
2020-09-14 14:31:50 +03:00
Georgii Rymar f4eb94e1db [llvm-readobj/elf][test] - Test all core note types properly.
Currently we don't test all core note types that are defined in
`getCoreNoteTypeName` in ELFDumper.cpp.

Also we don't have a test for an unknown core note type.

This patch fixes it.

Differential revision: https://reviews.llvm.org/D87453
2020-09-14 14:27:12 +03:00
Fangrui Song 4d7b194543 [llvm-cov gcov] Refactor counting and reporting
The current organization of FileInfo and its referenced utility functions of
(GCOVFile, GCOVFunction, GCOVBlock) is messy. Some members of FileInfo are just
copied from GCOVFile. FileInfo::print (.gcov output and --intermediate output)
is interleaved with branch statistics and computation of line execution counts.
--intermediate has to do redundant .gcov output to gather branch statistics.

This patch deletes lots of code and introduces a clearer work flow:

```
fn collectFunction
  for each block b
    for each line lineNum
      let line be LineInfo of the file on lineNum
      line.exists = 1
      increment function's lines & linesExec if necessary
      increment line.count
      line.blocks.push_back(&b)

fn collectSourceLine
  compute cycle counts
  count = incoming_counts + cycle_counts
  if line.exists
    ++summary->lines
    if line.count
      ++summary->linesExec

fn collectSource
  for each line
    call collectSourceLine

fn main
  for each function
    call collectFunction
    print function summary
  for each source file
    call collectSource
    print file summary
    annotate the source file with line execution counts
  if -i
    print intermediate file
```

The output order of functions and files now follows the original order in
.gcno files.
2020-09-13 23:00:59 -07:00
Fangrui Song 44664a5448 [llvm-cov gcov][test] Unsupport Windows 2020-09-13 15:17:14 -07:00
Fangrui Song b2c32c90ba [llvm-cov gcov] Add -r (--relative-only) && -s (--source-prefix)
gcov 4.7 introduced the two options.
https://sourceware.org/pipermail/gcc-patches/2011-November/328782.html

-r only dumps files with relative paths or absolute paths with the prefix
specified by -s. The two options are useful filtering out system header files.
2020-09-13 14:54:20 -07:00
Fangrui Song 8cf1ac97ce [llvm-cov gcov] Improve accuracy when some edges are not measured
Also guard against infinite recursion if GCOV_ARC_ON_TREE edges contain a cycle.
2020-09-12 22:33:41 -07:00
Evgeny Leviant 2e61cd1295 [MachineScheduler] Fix operand scheduling for pre/post-increment loads
Differential revision: https://reviews.llvm.org/D87557
2020-09-12 16:53:12 +03:00
Xing GUO 0e47a8d17f [obj2yaml] Add support for dumping the .debug_ranges section.
This patch adds support for dumping the .debug_ranges section to
elf2yaml.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D87429
2020-09-11 08:47:01 +08:00
Martin Storsjö 6313f55619 [llvm-readobj] [ARMWinEH] Fix printing of exception handlers with packed epilogues
If there's a packed epilogue (indicated by the flag E), the EpilogueCount()
field actually should be interpreted as EpilogueOffset.

Differential Revision: https://reviews.llvm.org/D87365
2020-09-10 11:26:43 +03:00
Martin Storsjö 8060283ff8 [llvm-readobj] [ARMWinEH] Print set_fp/add_fp differently in epilogues
This matches how e.g. stp/ldp and other opcodes are printed differently
for epilogues.

Also add a missing --strict-whitespace in an existing test that
was added explicitly for testing vertical alignment, and change to
using temp files for the generated object files.

Differential Revision: https://reviews.llvm.org/D87363
2020-09-10 11:26:43 +03:00
Alexander Shaposhnikov b897729a39 [llvm-install-name-tool] Add -V flag
This diff adds -V alias for --version to make llvm-install-name-tool
consistent with other tools (llvm-objcopy, llvm-strip, etc).

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D87264
2020-09-09 17:11:08 -07:00
Ronak Chauhan f078577f31 Revert "[AMDGPU] Support disassembly for AMDGPU kernel descriptors"
This reverts commit 487a805310.

Tests fail on big endian machines.
2020-09-09 18:01:28 +05:30
Xing GUO 25f3cc0ced [elf2yaml] Fix dumping a debug section whose name is not recognized.
If the debug section's name isn't recognized, it should be
dumped as a raw content section.

Reviewed By: jhenderson, grimar

Differential Revision: https://reviews.llvm.org/D87346
2020-09-09 20:07:05 +08:00
Fangrui Song b9d086693b [llvm-cov gcov] Compute unmeasured arc counts by Kirchhoff's circuit law
For a CFG G=(V,E), Knuth describes that by Kirchoff's circuit law, the minimum
number of counters necessary is |E|-(|V|-1). The emitted edges form a spanning
tree. libgcov emitted .gcda files leverages this optimization while clang
--coverage's doesn't.

Propagate counts by Kirchhoff's circuit law so that llvm-cov gcov can
correctly print line counts of gcc --coverage emitted files and enable
the future improvement of clang --coverage.
2020-09-08 18:45:11 -07:00
David Blaikie 69da27c749 llvm-symbolizer: Add optional "start file" to match "start line"
Since a function might have portions of its code coming from multiple
different files, "start line" is ambiguous (it can't just be resolved
relative to the file/line specified). Add start file to disambiguate it.
2020-09-08 15:40:58 -07:00
Alexander Shaposhnikov ce49b7d9ca [llvm-install-name-tool] Add a test with multiple input files
This diff adds a test which checks the error-message when multiple input files
are passed to llvm-install-name-tool.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D87268
2020-09-08 11:31:28 -07:00
Ronak Chauhan 487a805310 [AMDGPU] Support disassembly for AMDGPU kernel descriptors
Decode AMDGPU Kernel descriptors as assembler directives.

Reviewed By: scott.linder, jhenderson, kzhuravl

Differential Revision: https://reviews.llvm.org/D80713
2020-09-08 21:26:11 +05:30
Xing GUO 25c3fa3f13 [DWARFYAML] Make the debug_ranges section optional.
This patch makes the debug_ranges section optional. When we specify an
empty debug_ranges section, yaml2obj only emits the section header.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D87263
2020-09-08 19:55:47 +08:00
Xing GUO 3cda698723 [obj2yaml] Stop parsing the debug_str section when it encounters a string without the null terminator.
When obj2yaml encounters a string without the null terminator, it should
stop parsing the debug_str section. This patch addresses comments in
[D86867](https://reviews.llvm.org/D86867#inline-803291).

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D87261
2020-09-08 16:09:36 +08:00
Alexander Shaposhnikov 10af5bad44 [llvm-objcopy] Consolidate and unify version tests
In this diff the tests which verify version printing functionality are refactored.
Since they are not specific to a particular format we move them into tool-version.test
and slightly unify (similarly to tool-name.test and tool-help-message.test).

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D87211
2020-09-07 18:44:32 -07:00
Eric Astor a3ec4a3158 [ms] [llvm-ml] Allow use of locally-defined variables in expressions
MASM allows variables defined by equate statements to be used in expressions.

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D86946
2020-09-07 14:00:14 -04:00
Eric Astor 2feb6e9b84 [ms] [llvm-ml] Fix STRUCT field alignment
MASM aligns fields to the _minimum_ of the STRUCT alignment value and the size of the next field.

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D86945
2020-09-07 13:58:59 -04:00
Eric Astor e52e7ad54d [ms] [llvm-ml] Add support for bitwise named operators (AND, NOT, OR) in MASM
Add support for expressions of the form '1 or 2', etc.

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D86944
2020-09-07 13:57:54 -04:00
Xing GUO 3097427f93 [obj2yaml] Add support for dumping the .debug_str section.
This patch adds support for dumping the .debug_str section to obj2yaml.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D86867
2020-09-07 19:46:56 +08:00
Xing GUO 40f4131fce [DWARFYAML] Make the debug_addr section optional.
This patch makes the debug_addr section optional. When an empty
debug_addr section is specified, yaml2obj only emits a section header
for it.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D87205
2020-09-07 16:17:18 +08:00
Daniel Sanders 3f1a9b7eca [objdump][macho] Emit segment names along with section names
I recently came across a MachO with multiple sections of the same name but
different segments. We should emit the segment name alongside the section name
for MachO's.

Differential Revision: https://reviews.llvm.org/D87119
2020-09-04 09:57:02 -07:00
Xing GUO 2966d6e80b [MachOYAML] Allow handcrafting custom contents for DWARF sections.
This patch enables users to handcraft custom contents for DWARF
sections. If we specify the contents of DWARF sections both in the
'DWARF' entry and the 'content', yaml2obj will emit an error message.
In addition, this patch helps remove the restriction that only the
content of sections whose segname are __DWARF can be specified in the
"DWARF" entry.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D87126
2020-09-04 22:42:45 +08:00
Georgii Rymar b7c1810986 [llvm-readobj/elf] - Refine signature of print*Reloc methods.
This makes the interface cleaner and slightly improves messages
reported.

Differential revision: https://reviews.llvm.org/D87086
2020-09-04 12:53:29 +03:00
Fangrui Song 3d54976a70 [llvm-symbolizer] Add back --use-symbol-table=true
It is used by clusterfuzz (https://github.com/google/clusterfuzz/pull/2009/)
and having this compatibility option for a while can help they do bisection
with the latest llvm-symbolizer.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D87067
2020-09-03 14:27:06 -07:00
Georgii Rymar 56ee83e36c [llvm-readobj/elf] - Improve warning messages, reported for .stack_sizes sections.
Instead of referring to stack sizes sections only by name, we can add
section indexes and types to warnings reported.

Differential revision: https://reviews.llvm.org/D86934
2020-09-03 13:17:07 +03:00
Xing GUO 67ce11405b [llvm-dwarfdump] Warn user when it encounters no null terminated strings.
When llvm-dwarfdump encounters no null terminated strings, we should
warn user about it rather than ignore it and print nothing.

Before this patch, when llvm-dwarfdump dumps a .debug_str section whose
content is "abc", it prints:

```
.debug_str contents:
```

After this patch:

```
.debug_str contents:
warning: no null terminated string at offset 0x0
```

Reviewed By: jhenderson, MaskRay

Differential Revision: https://reviews.llvm.org/D86998
2020-09-03 08:49:57 +08:00
Eric Astor ddd48cdba6 [ms] [llvm-ml] Add support for line continuations in MASM
Add support for line continuations (the "backslash operator") in MASM by modifying the Parser's Lex method.

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D83347
2020-09-02 12:12:04 -04:00
Xing GUO 369f9169a5 [DebugInfo] Simplify string table dumpers.
This patch adds a helper function DumpStrSection to simplify codes.
Besides, nonprintable chars in debug_str and debug_str.dwo sections
are printed as escaped chars.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D86918
2020-09-02 08:41:10 +08:00
Georgii Rymar 3d90a61cf2 [llvm-readobj] - Remove Error.cpp,.h and drop dependencies in the code.
We have Error.cpp/.h which contains some code for working with error codes.
In fact we use Error/Expected<> almost everywhere already and we can get rid
of these files.

Note: a few places in the code used readobj specific error codes,
e.g. `return readobj_error::unknown_symbol`. But these codes are never really used,
i.e. the code checks the fact of a success/error call only.
So I've changes them to `return inconvertibleErrorCode()` for now.
It seems that these places probably should be converted to use `Error`/`Expected<>`.

Differential revision: https://reviews.llvm.org/D86772
2020-09-01 16:46:17 +03:00
Xing GUO ea1f5133cc [llvm-dwarfdump][test] Add tests for dumping the .debug_line_str section.
This patch adds tests for dumping the .debug_line_str section. The
dumper of the debug_line_str section is able to write nonprintable
strings as escaped chars while the dumper of the debug_str and
debug_str.dwo sections cannot. I will fix them and add tests for them
in a follow-up patch.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D86916
2020-09-01 20:22:01 +08:00
Georgii Rymar ca233be2f0 [llvm-readobj][test] - Remove amdgpu-elf-definitions.test
`amdgpu-elf-definitions.test` is a file that tests
`Format`, 'OS/ABI' and 'Machine' fields printed for EM_AMDGPU.
It uses a precompiled object.

It is not needed to have it, because:
1) We already test formats in `file-header-format.test` + `llvm/unittests/Object/ELFObjectFileTest.cpp`.
2) We test 'OS/ABI' in `file-header-os-abi.test` for all possible values.
3) We test machine types in `file-header-machine-types.test`

In addition we have `amdgpu-elf-headers.test` which tests EM_AMDGPU specific flags. I.e.
it also tests the fact that llvm-readobj/elf are able to dump such objects.

This patch removes the test case and the corresponding binary.

Differential revision: https://reviews.llvm.org/D86774
2020-09-01 14:04:14 +03:00
Georgii Rymar 6c6984ecd5 [llvm-readobj][test] - Update the comment in archive.test
Addresses post commit review comment for D86771.
2020-09-01 14:02:13 +03:00
Georgii Rymar 7de090a324 [llvm-readobj/elf] - Report warnings instead of errors when dumping broken stack sizes sections.
This replaces `reportError` calls with `reportUniqueWarning` and improves testing
for the code that is related to stack sizes dumping.

Differential revision: https://reviews.llvm.org/D86783
2020-09-01 13:46:54 +03:00
Xing GUO 428b2ffad4 [DWARFYAML] Make the debug_str section optional.
This patch makes the debug_str section optional. When the debug_str
section exists but doesn't contain anything, yaml2obj will emit a
section header for it.

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D86860
2020-09-01 10:02:09 +08:00
Georgii Rymar 506f89ce91 [llvm-readobj/elf] - Don't fail when dumping an archive with a member that can't be recognized.
Imagine we have an archive that has 3 objects in the following order:
<valid known object>,<unknown object> and <valid known object>.

Currently llvm-readelf/obj report an error and stops dumping in the middle.
This patch changes the error reported to warning.

Differential revision: https://reviews.llvm.org/D86771
2020-08-31 13:16:15 +03:00
Xing GUO 1d01fc100b [Test] Simplify DWARF test cases. NFC.
The Length, AbbrOffset and Values fields of the debug_info section are
optional. This patch helps remove them and simplify test cases.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D86857
2020-08-31 14:03:48 +08:00
Xing GUO 12e832cbcb [DWARFYAML] Make the debug_abbrev_offset field optional.
This patch helps make the debug_abbrev_offset field optional. We don't
need to calculate the value of this field in the future.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D86614
2020-08-29 14:54:52 +08:00
sstefan1 7128e64706 [UpdateTestChecks] include { in function signature check line
After D85099, if we have attribute group in the function signature that hasn't
been seen before, and later a callsite with the same attribute group, filecheck will evaluate
the first attribute group to for example '#0 {'. We now include { in the args_and_sig group to avoid this.

Differential Revision: https://reviews.llvm.org/D86769
2020-08-28 23:44:03 +02:00
Xing GUO f20e6c7253 [DWARFYAML] Abbrev codes in a new abbrev table should start from 1 (by default).
The abbrev codes in a new abbrev table should start from 1 (by default),
rather than inherit the value from the code in the previous table.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D86545
2020-08-28 21:18:11 +08:00
Georgii Rymar 4cd14256b2 [llvm-readobj][test] - Test "Format" values.
This adds testing for the "Format" field printed with `--file-headers`.

llvm-readelf doesn't use them, so only llvm-readobj needs to be tested.

All possible values are defined and tested in `ELFObjectFile<ELFT>::getFileFormatName()`.
Here we test just a few arbitrary ones.

Differential revision: https://reviews.llvm.org/D86350
2020-08-28 11:31:25 +03:00
Russell Gallop c9455d3c57 [Test] Tidy up loose ends from LLVM_HAS_GLOBAL_ISEL
This hasn't been allowed as a build option since r309990

Remove leftover REQUIRES: global-isel

Differential Revision: https://reviews.llvm.org/D86714
2020-08-27 16:36:27 +01:00
Xing GUO 8daa3264a3 [DWARFYAML] Make the unit_length and header_length fields optional.
This patch makes the unit_length and header_length fields of line tables
optional. yaml2obj is able to infer them for us.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D86590
2020-08-26 20:35:10 +08:00
Georgii Rymar 4b5460762a [llvm-readobj][test] - Commit trivial.obj.elf-amdhsa-gfx803 binary back.
It was removed in rGcbedbd12e9837e049f0a936636a82ff39b75692b by mistake.
2020-08-26 14:56:40 +03:00
Georgii Rymar cbedbd12e9 [llvm-readobj/elf][test] - Add testing for EM_* specific OS/ABI values.
We have no tests for OS/ABI values specific to
EM_TI_C6000, ELFOSABI_AMDGPU_MESA3D and ELFOSABI_ARM machines.

Also, related arrays in the code are not grouped together.
(That is why such testing was missed I guess).

The patch fixes that all.

Differential revision: https://reviews.llvm.org/D86341
2020-08-26 14:41:16 +03:00
Georgii Rymar 7749e2de20 [llvm-readobj] - Don`t crash when --section-symbols is requested for an object w/o .symtab.
llvm-readobj crashes when `-S --section-symbols` is used
on an object that has no symbol table.

The patch fixes it.

Differential revision: https://reviews.llvm.org/D86520
2020-08-26 14:13:05 +03:00
Georgii Rymar f65ab626e5 [llvm-readelf][test] - Refine the sections-ext.test
The `sections-ext.test` is a test for ELF that is used
to test `--st`, `--sr` and `--sd` extension options for `-S`.

There are 2 problems with it:
1) It is broken, because for CHECK lines it contains there is
no corresponding `FileCheck` call.

2) It uses the precompiled object: `trivial.obj.elf-i386`.
This is the last ELF test where `trivial.obj.elf-i386` is used so we can get
rid of the binary and use an YAML description.

Also, there is a `Inputs/trivial.ll` file that describes how `trivial*` objects
in `Inputs` folders are created. I've removed it from `ELF`, because it is not
actual anymore (we have no more input binaries created with the use of trivial.ll there)
and copied the refined versions of it to `COFF`, `MachO` and `wasm` Input folders.

Differential revision: https://reviews.llvm.org/D86462
2020-08-26 14:01:31 +03:00
Martin Storsjö db259fe38b [llvm-readobj] Fix arm64 unwind opcode disassembly printing
Add a missing minus, fix vertical alignment of instructions for one opcode.

Differential Revision: https://reviews.llvm.org/D86523
2020-08-26 09:38:11 +03:00
Adrien Guinet c6f7ac0071 [llvm-lipo] Add support for bitcode files
A Mach-O universal binary may contain bitcode as a slice.
This diff adds proper handling of such binaries to llvm-lipo.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D85740
2020-08-25 21:11:18 -07:00
Shoaib Meenai 22cd6bee4a [llvm-libtool-darwin] Address post-commit feedback
Address James Henderson's comments on https://reviews.llvm.org/D86359.
2020-08-25 15:04:23 -07:00
Xing GUO 1dc57ada0c [DWARFYAML] Make the 'Attributes' field optional.
This patch makes the 'Attributes' field optional. We don't need to
explicitly specify the 'Attributes' field in the future.

Reviewed By: jhenderson, grimar

Differential Revision: https://reviews.llvm.org/D86537
2020-08-25 22:37:43 +08:00
Georgii Rymar 41726f8d5b [llvm-readobj] - Print "Unknown" when a program header is unknown.
Currently, when a program header type is unknown, we dont print anything:

```
ProgramHeader {
  Type:  (0x60000000)
```

With this patch the output will be:

```
ProgramHeader {
  Type: Unknown (0x60000000)
```

It was discussed in D85526 and consistent with what we print for
'--sections' already, e.g.:

```
Section {
  Name: .sec
  Type: Unknown (0x7FFFFFFF)
}
```

Differential revision: https://reviews.llvm.org/D86213
2020-08-25 13:05:17 +03:00
Georgii Rymar 275bed899e [llvm-readelf/obj] - Change the return type of the `createDRI(...)` to `Expected<>`
This allows to get rid of "Invalid data was encountered while parsing the file"
error reported in cases when sh_size/sh_offset of sections are broken.

Differential revision: https://reviews.llvm.org/D86451
2020-08-25 12:11:26 +03:00
OCHyams 3ff4d75c9c [llvm-dwarfdump] Fix misleading scope byte coverage statistics
Fixes PR46575.

Bump statistics version to 6.

Without this patch, for a variable described with a location list the stat
'sum_all_variables(#bytes in parent scope covered by DW_AT_location)' is
calculated by summing all bytes covered by the location ranges in the list and
capping the result to the number of bytes in the parent scope. With the patch,
only bytes which overlap with the parent DIE scope address ranges contribute to
the stat. A new stat 'sum_all_variables(#bytes in any scope covered by
DW_AT_location)' has been added which displays the total bytes covered when
ignoring scopes.
2020-08-25 06:40:11 +01:00
Alexandre Ganea 1f04678f87 Disable 'not' test on Windows because 'env' from GnuWin32 cannot be used without arguments. 2020-08-24 21:55:34 -04:00
Fangrui Song a5e65ad3a3 [not][test] Fix disable-symbolization.test when 'printenv' is not available
On Windows, 'env' or 'printenv' may not exist.

Also switch back to 'env' which is specified by POSIX.1-2017. 'printenv' is not
standard (I picked it because 'printenv' exists on GnuWin32 but 'env' does not).

Reviewed By: zequanwu

Differential Revision: https://reviews.llvm.org/D86496
2020-08-24 17:27:34 -07:00
Craig Topper f7c87b7e37 [X86] Copy the tuning features and scheduler model from pentium4/x86-64 to generic
This is preparation for making clang default to -mtune=generic when no -march is specified. This will allow the default tuning to be "generic" even though our default march is "pentium4" or "x86-64".

To avoid llc lit test regressions, if no mcpu is specified, I've defaulted tune to use i586 to match the old tuning settings of no CPU. Some tests explicitly used -mcpu=generic which I've removed so they instead get this default of architecture features from generic and tune from i586.

I updated one llvm-mca test to check a different CPU since generic has a scheduler model now

Differential Revision: https://reviews.llvm.org/D86312
2020-08-24 14:47:10 -07:00
Shoaib Meenai 68bae34c65 [llvm-libtool-darwin] Add support for -V option
The -V option in cctools' libtool prints out the version number and
performs any specified operation. Add this option to LLVM's version.
cctools is more forgiving of invalid command lines when -V is specified,
but I think it's better to give errors instead of silently producing no
output.

Unfortunately, when -V is present, options that would otherwise be
required aren't anymore, so we need to perform some manual argument
validation.

Reviewed By: alexshap

Differential Revision: https://reviews.llvm.org/D86359
2020-08-24 13:48:23 -07:00
Fangrui Song 481c3b8c94 [llvm-dwarfdump] --statistics: break lines and indent by 2
so that the user does not have to pipe the output to `jq` or `python -m json.tool`.
This change makes testing more convenient because `-NEXT` patterns can be used.
The "prettify by default" is a good tradeoff to make. The output size increases a bit.

Differential Revision: https://reviews.llvm.org/D86318
2020-08-22 13:58:18 -07:00
Azharuddin Mohammed 6a64079699 Fix llvm/test/tools/lto/hide-linkonce-odr.ll
Remove unnecessary dependency on libSystem.
2020-08-21 18:32:35 -07:00
Fangrui Song 06cad825cd PrintStackTrace: don't symbolize if LLVM_DISABLE_SYMBOLIZATION is set
See http://lists.llvm.org/pipermail/llvm-dev/2017-June/113975.html for a related previous discussion.
Many tools install signal handlers to print stack traces and optionally
symbolize the addresses with an external program 'llvm-symbolizer' (when
searching for 'llvm-symbolizer', the directory containg the executable
is preferred over PATH).

'llvm-symbolizer' can be slow if the executable is large and/or if
llvm-symbolizer' itself is under-optimized. For example, my 'llvm-lto2' from a
-DCMAKE_BUILD_TYPE=Debug build is 443MiB. The 'llvm-symbolizer' from the same
build takes ~2s to symbolize it. (An optimized 'llvm-symbolizer' takes 0.34s).
A crashed clang may take more than 5s to symbolize a stack trace.

If a test file has several `not --crash` RUN lines. It can be very slow in a Debug build.
This patch makes `not --crash` set an environment variable to suppress symbolization.
This is similar to D33804 which uses a command line option.
I pick 'symbolization' instead of 'symbolication' because the former is
used much more commonly and its stem matches 'llvm-symbolizer'.

Also set LLVM_DISABLE_CRASH_REPORT=1, which is currently only applicable on
`__APPLE__`.

Reviewed By: dblaikie, aganea

Differential Revision: https://reviews.llvm.org/D86170
2020-08-21 11:27:13 -07:00
Georgii Rymar 88dd7c8549 [llvm-readelf][test] - Refine the merged.test
The testing in merged.test is not ideal.
It uses the following set of flags: `-aeWhSrnudlVgIs`
and the precompiled object `trivial.obj.elf-i386`.

In fact, this object, for example, does not contain versioning stuff,
so specifying `-V` does not make much sense for it, so it is not really tested.
Also, we want to avoid using of precompiled objects, ideally we
want to use a YAML that triggers an output for each of the short options in use:

```
-a: --all
-e: --headers
-W: --wide
-h: --file-headers, --file-header
-S: --section-headers, --sections
-r: --relocations
-n: --notes
-u: --unwind
-d: --dynamic-table
-l: --program-headers
-V: --version-info
-g: --elf-section-groups
-I: --elf-hash-histogram
-s: --symbols
```

Note that, for example, we do not need to have groups to test `-g`, because for a
object with no groups llvm-readelf still prints "There are no section groups in this file.",
but that is not always the case. E.g. for `-d` we don't print anything when we have
no dynamic table, so we have to describe it in a YAML to test `-d` properly.

Also, we probably want to test cases with and without `-a`(-all) option separately
to be sure that we handle all options and not only the first one in the sequence.

Differential revision: https://reviews.llvm.org/D86283
2020-08-21 15:07:37 +03:00
Georgii Rymar aa456a6df4 [llvm-readobj/elf] - Refine the code for broken PT_DYNAMIC segment diagnostic.
The code that reports "PT_DYNAMIC segment offset + size exceeds the size of the file"
has an issue: it is possible to bypass the validation by overflowing the size + offset result.

Differential revision: https://reviews.llvm.org/D85519
2020-08-21 15:05:46 +03:00
Xing GUO f5643dc3dc Recommit: [DWARFYAML] Add support for referencing different abbrev tables.
The original commit (7ff0ace96db9164dcde232c36cab6519ea4fce8) was causing
build failure and was reverted in 6d242a7326

==================== Original Commit Message ====================
This patch adds support for referencing different abbrev tables. We use
'ID' to distinguish abbrev tables and use 'AbbrevTableID' to explicitly
assign an abbrev table to compilation units.

The syntax is:
```
debug_abbrev:
  - ID: 0
    Table:
      ...
  - ID: 1
    Table:
      ...
debug_info:
  - ...
    AbbrevTableID: 1 ## Reference the second abbrev table.
  - ...
    AbbrevTableID: 0 ## Reference the first abbrev table.
```

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D83116
2020-08-21 19:02:10 +08:00
Georgii Rymar c66b82f14c [llvm-readelf] - Start recognizing 'PT_OPENBSD_*' segment types.
Its a follow-up for D85830, it stops ignoring 'PT_OPENBSD_*' segment types.
Now them are recognized properly.

Note: GNU readelf does not recognize them, though perhaps it shouldn't.
Anyways, it was reported to binutils: https://sourceware.org/bugzilla/show_bug.cgi?id=26405#c0

Differential revision: https://reviews.llvm.org/D86208
2020-08-21 13:13:05 +03:00
Xing GUO 6d242a7326 Revert "[DWARFYAML] Add support for referencing different abbrev tables."
This reverts commit f7ff0ace96.

This change is causing build failure.

http://lab.llvm.org:8011/builders/clang-cmake-armv7-global-isel/builds/10400
2020-08-21 12:15:54 +08:00
Xing GUO f7ff0ace96 [DWARFYAML] Add support for referencing different abbrev tables.
This patch adds support for referencing different abbrev tables. We use
'ID' to distinguish abbrev tables and use 'AbbrevTableID' to explicitly
assign an abbrev table to compilation units.

The syntax is:
```
debug_abbrev:
  - ID: 0
    Table:
      ...
  - ID: 1
    Table:
      ...
debug_info:
  - ...
    AbbrevTableID: 1 ## Reference the second abbrev table.
  - ...
    AbbrevTableID: 0 ## Reference the first abbrev table.
```

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D83116
2020-08-21 11:44:25 +08:00
Xing GUO e151acc073 [DWARFYAML] Add support for emitting multiple abbrev tables.
This patch adds support for emitting multiple abbrev tables. Currently,
compilation units will always reference the first abbrev table.

Reviewed By: jhenderson, labath

Differential Revision: https://reviews.llvm.org/D86194
2020-08-21 11:44:25 +08:00
Xing GUO 290e399f96 [DWARFYAML] Add support for emitting multiple abbrev tables.
This patch adds support for emitting multiple abbrev tables. Currently,
compilation units will always reference the first abbrev table.

Reviewed By: jhenderson, labath

Differential Revision: https://reviews.llvm.org/D86194
2020-08-21 10:12:08 +08:00
Fangrui Song 2b0f40b049 [llvm-dwarfdump] Fix a typo: witin -> within 2020-08-20 14:12:37 -07:00
Georgii Rymar 61152a71a1 Revert "[llvm-readobj/elf] - Refine the code for broken PT_DYNAMIC segment diagnostic."
This reverts commit 455d5a8a06.

It broke UBSan:
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap-ubsan/builds/21386/steps/check-llvm%20ubsan/logs/stdio

/b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/test/tools/llvm-readobj/ELF/malformed-pt-dynamic.test:62:10: error: WARN3: expected string not found in input
# WARN3: error: '[[FILE]]': Invalid data was encountered while parsing the file
         ^
<stdin>:2:1: note: scanning from here
/b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/tools/llvm-readobj/ELFDumper.cpp:1956:46: runtime error: addition of unsigned offset to 0x0000020c5b30 overflowed to 0x0000020c5b2f
^
<stdin>:2:1: note: with "FILE" equal to "/b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm_build_ubsan/test/tools/llvm-readobj/ELF/Output/malformed-pt-dynamic\\.test\\.tmp3"
/b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/tools/llvm-readobj/ELFDumper.cpp:1956:46: runtime error: addition of unsigned offset to 0x0000020c5b30 overflowed to 0x0000020c5b2f
^
<stdin>:2:117: note: possible intended match here
/b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/tools/llvm-readobj/ELFDumper.cpp:1956:46: runtime error: addition of unsigned offset to 0x0000020c5b30 overflowed to 0x0000020c5b2f
                                                                                                                    ^
Input file: <stdin>
Check file: /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/test/tools/llvm-readobj/ELF/malformed-pt-dynamic.test
2020-08-20 14:04:30 +03:00
Georgii Rymar 455d5a8a06 [llvm-readobj/elf] - Refine the code for broken PT_DYNAMIC segment diagnostic.
The code that reports "PT_DYNAMIC segment offset + size exceeds the size of the file"
has an issue: it is possible to bypass the validation by overflowing the size + offset result.

Differential revision: https://reviews.llvm.org/D85519
2020-08-20 12:28:34 +03:00
Georgii Rymar a6436b0b3a [yaml2obj] - Make the 'Machine' key optional.
Currently we have to set 'Machine' to something in our
YAML descriptions. Usually we use 'EM_X86_64' for 64-bit targets
and 'EM_386' for 32-bit targets. At the same time, in fact, in most
cases our tests do not need a machine type and we can use
'EM_NONE'.

This is cleaner, because avoids the need of using a particular machine.

In this patch I've made the 'Machine' key optional (the default value,
when it is not specified is `EM_NONE`) and removed it (where possible)
from yaml2obj, obj2yaml and llvm-readobj tests.

There are few tests left where I decided not to remove it, because
I didn't want to touch CHECK lines or doing anything more complex
than a removing a "Machine: *" line and formatting lines around.

Differential revision: https://reviews.llvm.org/D86202
2020-08-20 11:40:51 +03:00
Ronak Chauhan fdf71d486c Revert "[AMDGPU] Support disassembly for AMDGPU kernel descriptors"
This reverts commit cacfb02d28.

Reverting due to buildbot failures.
2020-08-19 13:12:29 +05:30
Ronak Chauhan cacfb02d28 [AMDGPU] Support disassembly for AMDGPU kernel descriptors
Decode AMDGPU Kernel descriptors as assembler directives.

Reviewed By: scott.linder

Differential Revision: https://reviews.llvm.org/D80713
2020-08-19 08:49:07 +05:30
Fangrui Song aa48a480b8 [llvm-dwarfdump][test] Add a --statistics test for a DW_AT_artificial variable
There is an untested but useful case: `this` (even if not written) is counted as a
source variable.

Reviewed By: dblaikie

Differential Revision: https://reviews.llvm.org/D86044
2020-08-18 09:08:38 -07:00
Georgii Rymar 740332b6cc [llvm-readobj/elf] - Refine testing of broken Android's packed relocation sections.
This uses modern `split-file` tool to merge 5 `packed-relocs-error*.s` tests to a
new `packed-relocs-errors.s` and adds testing for GNU style.

Differential revision: https://reviews.llvm.org/D85835
2020-08-18 16:23:41 +03:00
Georgii Rymar bd7daf5ceb [yaml2obj] - Don't crash when `FileHeader` declares an empty `Flags` key in specific situations.
We currently call the `llvm_unreachable` for the following YAML:

```
--- !ELF
FileHeader:
  Class:   ELFCLASS32
  Data:    ELFDATA2LSB
  Type:    ET_REL
  Machine: EM_NONE
  Flags:   [ ]
```

it happens because the `Flags` key is present, though `EM_NONE` is a
machine type that has no known `EF_*` values and we call `llvm_unreachable` by mistake.

Differential revision: https://reviews.llvm.org/D86138
2020-08-18 16:09:28 +03:00
Ronak Chauhan e760e85680 [llvm-objdump][AMDGPU] Detect CPU string
AMDGPU ISA isn't backwards compatible and hence -mcpu must always be specified during disassembly.
However, the AMDGPU target CPU is stored in e_flags in the ELF object.

This patch allows targets to implement CPU string detection, and also implements it for AMDGPU by looking at e_flags.

Reviewed By: scott.linder

Differential Revision: https://reviews.llvm.org/D84519
2020-08-18 17:43:16 +05:30
Georgii Rymar 6786b3e307 [llvm-readobj/elf] - Refine the malformed-pt-dynamic.test.
This is splitted out from D85519, but significantly reworked.

Changes:
1) This test was changed to stop using python.
2) Use NoHeaders: true instead of `llvm-objcopy --strip-sections`.
3) Test llvm-readelf too (not just llvm-readobj).
4) Simplify the YAML used a bit (e.g. remove PT_LOAD).
5) Test 2 different cases: objects with section header table and without.

Differential revision: https://reviews.llvm.org/D86073
2020-08-18 11:51:03 +03:00
Georgii Rymar b475eca1ed [llvm-readobj/elf] - Merge mips-got-overlapped.test to mips-got.test and refine testing.
The `mips-got-overlapped.test` was introduced in D16968 and its intention is
to check that when there is an empty section at the same address as `.got`,
then we are able to locate `.got` and dump it.

The issue is that this test does not test llvm-readelf and uses a precompiled
object. This path starts using YAML instead and merges
mips-got-overlapped.test to mips-got.test.

Differential revision: https://reviews.llvm.org/D86080
2020-08-18 11:37:34 +03:00
Hongtao Yu 819b2d9c79 [llvm-objdump] Symbolize binary addresses for low-noisy asm diff.
When diffing disassembly dump of two binaries, I see lots of noises from mismatched jump target addresses and global data references, which unnecessarily causes diffs on every function, making it impractical. I'm trying to symbolize the raw binary addresses to minimize the diff noise.
In this change, a local branch target is modeled as a label and the branch target operand will simply be printed as a label. Local labels are collected by a separate pre-decoding pass beforehand. A global data memory operand will be printed as a global symbol instead of the raw data address. Unfortunately, due to the way the disassembler is set up and to be less intrusive, a global symbol is always printed as the last operand of a memory access instruction. This is less than ideal but is probably acceptable from checking code quality point of view since on most targets an instruction can have at most one memory operand.

So far only the X86 disassemblers are supported.

Test Plan:

llvm-objdump -d  --x86-asm-syntax=intel --no-show-raw-insn --no-leading-addr :
```
Disassembly of section .text:

<_start>:
               	push	rax
               	mov	dword ptr [rsp + 4], 0
               	mov	dword ptr [rsp], 0
               	mov	eax, dword ptr [rsp]
               	cmp	eax, dword ptr [rip + 4112]  # 202182 <g>
               	jge	0x20117e <_start+0x25>
               	call	0x201158 <foo>
               	inc	dword ptr [rsp]
               	jmp	0x201169 <_start+0x10>
               	xor	eax, eax
               	pop	rcx
               	ret
```

llvm-objdump -d  **--symbolize-operands** --x86-asm-syntax=intel --no-show-raw-insn --no-leading-addr :
```
Disassembly of section .text:

<_start>:
               	push	rax
               	mov	dword ptr [rsp + 4], 0
               	mov	dword ptr [rsp], 0
<L1>:
               	mov	eax, dword ptr [rsp]
               	cmp	eax, dword ptr  <g>
               	jge	 <L0>
               	call	 <foo>
               	inc	dword ptr [rsp]
               	jmp	 <L1>
<L0>:
               	xor	eax, eax
               	pop	rcx
               	ret
```

Note that the jump instructions like `jge 0x20117e <_start+0x25>` without this work is printed as a real target address and an offset from the leading symbol. With a change in the optimizer that adds/deletes an instruction, the address and offset may shift for targets placed after the instruction. This will be a problem when diffing the disassembly from two optimizers where there are unnecessary false positives due to such branch target address changes. With `--symbolize-operand`, a label is printed for a branch target instead to reduce the false positives. Similarly, the disassemble of PC-relative global variable references is also prone to instruction insertion/deletion.

Reviewed By: jhenderson, MaskRay

Differential Revision: https://reviews.llvm.org/D84191
2020-08-17 16:55:12 -07:00
Georgii Rymar bc902191d3 [llvm-readobj] - Remove unwrapOrError calls from GNUStyle<ELFT>::printRelocations.
This fixes existent FIXMEs: we should not error out when unable to
find the number of relocations.

Differential revision: https://reviews.llvm.org/D85891
2020-08-17 15:16:36 +03:00
Georgii Rymar 6567f82216 [llvm-readobj/elf] - Refine the warning about the broken PT_DYNAMIC segment.
Splitted out from D85519.

Currently we report "PT_DYNAMIC segment offset + size exceeds the size of the file",
this changes it to
"PT_DYNAMIC segment offset (0x1234) + file size (0x5678) exceeds the size of the file (0x68ab)"

Differential revision: https://reviews.llvm.org/D85654
2020-08-17 14:57:19 +03:00
Shoaib Meenai 402b063c80 [llvm-libtool-darwin] Fix test on all host architectures
By default, if a universal binary has a slice matching the host
architecture, llvm-objdump will only print that slice, otherwise it'll
print all architectures. Explicitly pass `--arch all` to force it to
always print all architectures, as we want for this test.
2020-08-16 00:18:03 -07:00
Shoaib Meenai 12b4df9919 [llvm-libtool-darwin] Speculative buildbot fix
http://lab.llvm.org:8011/builders/llvm-clang-win-x-armv7l is failing
this test. Attempt to explicitly use the Mach-O dump format as a
speculative fix.
2020-08-15 21:32:09 -07:00
Sameer Arora 1aed1e72e8 [llvm-libtool-darwin] Add support for -l and -L
Add support for passing in libraries via `-l` and `-L` options to
`llvm-libtool-darwin`.

Reviewed by jhenderson, smeenai

Differential Revision: https://reviews.llvm.org/D85540
2020-08-14 11:44:17 -07:00
Sameer Arora 8f6f6f407a [llvm-libtool-darwin] Support universal outputs
Add support for producing universal binaries containing archives when
`llvm-libtool-darwin` is given inputs of multiple architectures.

Reviewed by jhenderson, smeenai

Differential Revision: https://reviews.llvm.org/D85334
2020-08-14 11:32:07 -07:00
Sameer Arora bd2853f799 [llvm-libtool-darwin] Add support for -arch_only
Add support for -arch_only option for llvm-libtool-darwin. This diff
also adds support for accepting universal files as input and flattening
them to create the required static library. Supports input universal
files contaning both Mach-O object files or archives.

Differences from cctools' libtool:
- `-arch_only` can be specified multiple times
- archives containing universal files are considered invalid (libtool
allows such archives)

Reviewed by jhenderson, smeenai

Differential Revision: https://reviews.llvm.org/D84770
2020-08-13 11:08:46 -07:00
Fangrui Song 7f8c49b016 [llvm-objdump] Change symbol name/PLT decoding errors to warnings
If the referenced symbol of a J[U]MP_SLOT is invalid (e.g. symbol index 0), llvm-objdump -d will bail out:

```
error: 'a': st_name (0x326600) is past the end of the string table of size 0x7
```

where 0x326600 is the st_name field of the first entry past the end of .symtab

Change it to a warning to continue dumping.
`X86/plt.test` uses a prebuilt executable, so I pick `ELF/AArch64/plt.test`
which has a YAML input and can be easily modified.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D85623
2020-08-13 08:13:42 -07:00
Georgii Rymar 936ba7eec3 [llvm-readobj/elf][test] - Refine --headers testing and the related code comment.
Specifying --headers is equivalent to setting --file-headers,
--program-headers and --section-headers at the same time.

The existent test case uses a precompiled object and doesn't test the
output properly. This patch fixes it.

Differential revision: https://reviews.llvm.org/D85832
2020-08-13 13:52:49 +03:00
Xing GUO b7d5d1ec64 [DWARFYAML] Replace InitialLength with Format and Length. NFC.
This change replaces the InitialLength of pub-tables with Format and
Length. All the InitialLength fields have been removed.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D85880
2020-08-13 18:39:03 +08:00
Stanislav Mekhanoshin 839762b044 [AMDGPU][test] Add dedicated llvm-readobj test.
Differential Revision: https://reviews.llvm.org/D85683
2020-08-12 09:11:36 -07:00
Xing GUO e891b6a75d [DWARFYAML] Make the address size of compilation units optional.
This patch makes the 'AddrSize' field optional. If the address size is
missing, yaml2obj will infer it from the object file.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D85805
2020-08-12 21:47:32 +08:00
Georgii Rymar 3b0a4e9584 [llvm-readobj] - Refine logic of the symbol table locating in printRelocationsHelper().
This removes the last `unwrapOrError` call from the `printRelocationsHelper`.

There is a little additional complexity because of `SHT_RELR/SHT_ANDROID_RELR` sections.
Such sections contains only relative relocations and they do not have a
symbol table associated with them, hence we should not try to treat
their `sh_link` field as a reference to a symbol table.

Differential revision: https://reviews.llvm.org/D85430
2020-08-12 14:03:56 +03:00
Igor Kudrin 9ceb192e14 [llvm-dwarfdump] Avoid crashing if an abbreviation offset is invalid.
Note that DWARFUnit::getAbbreviations() returns nullptr if the
abbreviations could not be read, but callers used the returned
pointer without checking.

Differential Revision: https://reviews.llvm.org/D85738
2020-08-12 16:01:53 +07:00
Johannes Doerfert 97ce7fd89f [UpdateTestChecks] Match unnamed values like "@[0-9]+" and "![0-9]+"
With this patch we will match most *uses* of "temporary" named things in
the IR via regular expressions, not their name at creation time. The new
"values" we match are:
  - "unnamed" globals: `@[0-9]+`
  - debug metadata: `!dbg ![0-9]+`
  - loop metadata: `!loop ![0-9]+`
  - tbaa metadata: `!tbaa ![0-9]+`
  - range metadata: `!range ![0-9]+`
  - generic metadata: `metadata ![0-9]+`
  - attributes groups: `#[0-9]`

We still don't match the declarations but that can be done later. This
patch can introduce churn when existing check lines contain the old
hardcoded versions of the above "values". We can add a flag to opt-out,
or opt-in, if necessary.

Reviewed By: arichardson, MaskRay

Differential Revision: https://reviews.llvm.org/D85099
2020-08-12 01:04:16 -05:00
jasonliu 0dc5e0cd39 [XCOFF][llvm-readobj] Move XCOFF test to XCOFF directory
Summary:
COFF and XCOFF in llvm are very different and serves different platform.
Since we have different Dumper.cpp file in llvm-readobj's
implementation, we should have separate testing directory for them too.

Reviewed By: jhenderson, DiggerLin

Differential Revision: https://reviews.llvm.org/D85675
2020-08-11 17:31:59 +00:00
Xing GUO 45a4f4c806 [DWARFYAML] Teach yaml2obj emit the correct line table program.
The following issues are addressed in this patch.

1. The operands of DW_LNE_set_discriminator should be an ULEB128 number
   rather than an address.
2. Test the emitted opcodes.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D85717
2020-08-12 00:18:54 +08:00
Xing GUO 1d4bc08ce4 [DWARFYAML] Let the address size of line tables inferred from the object file.
Currently, the line table uses the first compilation unit's address size
as its address size. It's not the right behavior. The address size should be
inferred from the target machine.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D85707
2020-08-11 22:45:55 +08:00
Georgii Rymar 034b95e283 [llvm-readobj/elf] - Remove `unwrapOrError` calls from DumpStyle<ELFT>::printRelocationsHelper.
It removes all `unwrapOrError` calls except the first one, which
is is bit different and can be removed separately.

Differential revision: https://reviews.llvm.org/D85303
2020-08-11 16:49:37 +03:00
Georgii Rymar d2e2ad2921 [llvm-readobj][test] - Rename gnu-phdrs.test to program-headers.test, NFCI.
program-headers.test was merged to gnu-phdrs.test and removed in D85526.
Now, gnu-phdrs.test contains tests for both GNU and LLVM style.

It was decided that this renaming should be done separately for a better
commit history and diffing purposes.
2020-08-11 16:46:05 +03:00
Georgii Rymar d212b63a79 [llvm-readobj] - Refine program headers testing (`-l`, `--program-headers` and `--segment`).
We have `gnu-phdrs.test` that tests
`-l`, `--program-headers` and `--segment` options for `llvm-readelf`.

We also have:
1) `program-headers.test` that tests `--program-headers` and `-l`
   for `llvm-readobj`. It doesn't test `--segment` and also uses
   lots of precompiled objects. It is very incomplete in compare
   with the `gnu-phdrs.test`.

2) `pt-gnu-property.test` that contains a simple test for the
   `PT_GNU_PROPERTY` segment. There is no reason to have it in
   a separate file.

This patch:
1) Merges `program-headers.test` and `pt-gnu-property.test` to `gnu-phdrs.test`.
2) Removes 2 precompiled binaries used by `program-headers.test`
   (other ones are still used by another tests).

Differential revision: https://reviews.llvm.org/D85526
2020-08-11 16:38:11 +03:00
Xing GUO 02af25df2b [macho2yaml][test] Specify the object file's endianness. NFC.
This patch is intended to fix testing failure on big endian machine.

Failed testing: http://lab.llvm.org:8011/builders/clang-ppc64be-linux/builds/52450
2020-08-11 11:46:50 +08:00
Xing GUO 655923de3d [macho2yaml][test] Specify the object file's endianness. NFC.
This change specifies the endianness of the generated object file to fix
the testing failure.

Testing failure: http://lab.llvm.org:8011/builders/clang-ppc64be-linux/builds/52449
2020-08-11 11:14:54 +08:00
Xing GUO 3c5758964c [macho2yaml] Refactor the DWARF section dumpers.
This patch refactors the DWARF section dumpers. When dumping a DWARF
section, if the DWARF parser fails to parse the section, we will dump it
as a raw content section. This patch also fixes a bug in
DWARFYAML::Data::isEmpty(). Finally, a test case that tests dumping the
__debug_aranges section is added.

Reviewed By: jhenderson, grimar

Differential Revision: https://reviews.llvm.org/D85506
2020-08-11 10:18:34 +08:00
Alexandre Ganea a680ea2c72 Fix "last accessed time" test failing on Windows
Before this patch, the tests in llvm/test/tools/llvm-objcopy/ELF/strip-preserve-atime.test used to fail on my machine, because the "last accessed time" is disabled in the OS by default since Windows XP. One needs to explicitly enable it for the feature to work. Otherwise the last access time is the last write time. Please see: https://superuser.com/questions/251263/the-last-access-date-is-not-changed-even-after-reading-the-file-on-windows-7

    Differential Revision: https://reviews.llvm.org/D85669
2020-08-10 16:03:14 -04:00
Fangrui Song 0b7f125219 [llvm-symbolizer] Add back --version and add a -v alias
The switch from llvm::cl to OptTable (D83530) dropped --version, which
is needed by some users.

This patch also adds a -v alias, which is available in GNU addr2line.

The version dumping is similar to llvm-objcopy --version (exotic):

```
llvm-symbolizer
LLVM (http://llvm.org/):
  LLVM version 12.0.0git
  Optimized build with assertions.
  Default target: x86_64-unknown-linux-gnu
  Host CPU: skylake-avx512
```

Reviewed By: dyung, jhenderson

Differential Revision: https://reviews.llvm.org/D85624
2020-08-10 08:21:43 -07:00
James Henderson ca05601cd2 [DebugInfo] Don't error for zero-length arange entries
Although the DWARF specification states that .debug_aranges entries
can't have length zero, these can occur in the wild. There's no
particular reason to enforce this part of the spec, since functionally
they have no impact. The patch removes the error and introduces a new
warning for premature terminator entries which does not stop parsing.

This is a relanding of cb3a598c87, adding the missing obj2yaml part
that was needed.

Fixes https://bugs.llvm.org/show_bug.cgi?id=46805. See also
https://reviews.llvm.org/D71932 which originally introduced the error.

Reviewed by: ikudrin, dblaikie, Higuoxing

Differential Revision: https://reviews.llvm.org/D85313
2020-08-10 14:57:52 +01:00
Georgii Rymar c990455db9 [llvm-readelf/obj] - Refine the implementation of printMipsReginfo().
It adds the proper warnings reporting and updates the mips-reginfo.test to
remove using of the precompiled binary.

Differential revision: https://reviews.llvm.org/D85511
2020-08-09 11:10:12 +03:00
Georgii Rymar cfd2ee29e0 [llvm-readobj] - Remove 3 excessive test cases.
This patch does the following:

1) Removes mips-options.test and the corresponding Inputs/mips-options.elf-mips64el binary:
This is a test that checks that --dynamic-table is able to print the DT_MIPS_OPTIONS tag.
We are testing it in dynamic-tags-machine-specific.test already.
(https://github.com/llvm/llvm-project/blob/master/llvm/test/tools/llvm-readobj/ELF/dynamic-tags-machine-specific.test#L235)

2) Removes mips-rld-map-rel.test and the corresponding Inputs/mips-rld-map-rel.elf-mipsel binary.
This is a test that checks that --dynamic-table is able to print the DT_MIPS_RLD_MAP_REL tag.
We are testing it in dynamic-tags-machine-specific.test already.
(https://github.com/llvm/llvm-project/blob/master/llvm/test/tools/llvm-readobj/ELF/dynamic-tags-machine-specific.test#L257)

3) Removes ppc64-glink.test test and the corresponding Inputs/ppc64.exe binary.
This is a test that checks that --dynamic-table is able to print the DT_PPC64_GLINK tag.
We are testing it in dynamic-tags-machine-specific.test already.
(https://github.com/llvm/llvm-project/blob/master/llvm/test/tools/llvm-readobj/ELF/dynamic-tags-machine-specific.test#L337)

Differential revision: https://reviews.llvm.org/D85515
2020-08-09 11:00:49 +03:00
weihe 540489de68 [llvm-profdata] Implement llvm-profdata overlap for sample profiles
Implemented the `llvm-profdata overlap` feature for sample profiles. It reports weighted //similarity// and unweighted //overlap// metrics at program and function level for two input profiles. Similarity metrics are symmetric with regards to the order of two input profiles. By default, the tool only reports program-level summary. Users can look into function-level details via additional options `--function`, `--similarity-cutoff`, and `--value-cutoff`.

The similarity metrics are designed as follows:
* Program-level summary
    * Whole program profile similarity is an aggregate over function-level similarity `FS`: `PS = sum(FS(A) * avg_weight(A))` for all function `A`.
    * Whole program sample overlap: `PSO = common_samples / total_samples`.
    * Function overlap: `FO = #common_function / #total_function`.
    * Hot-function overlap: `HFO = #common_hot_function / #total_hot_function`.
    * Hot-block overlap: `HBO = #common_hot_block / #total_hot_block`.
* Function-level details
    * Function-level similarity is an aggregate over line/block-level similarities `BS` of all sample lines/blocks in the function, weighted by the closeness of the function's weights in two profiles: `FS = sum(BS(i)) * (1 - weight_distance(A))`.
    * Function-level sample overlap: `FSO = common_samples / total_samples` for samples in the function.

Reviewed By: wenlei, hoyFB, wmi

Differential Revision: https://reviews.llvm.org/D83852
2020-08-08 17:49:48 -07:00
Sameer Arora 71a1f135e4 [llvm-libtool-darwin] Add support for -D and -U options
Add support for `-D` and `-U` options for llvm-libtool-darwin. `-D`
allows for using zero for timestamps and UIDs/GIDs. `-U` allows for
using actual timestamps and UIDs/GIDs.

Reviewed by jhenderson, smeenai

Differential Revision: https://reviews.llvm.org/D84209
2020-08-07 14:44:32 -07:00
Sameer Arora d9a9192984 [llvm-libtool-darwin] Add support for -filelist option
Add support for `-filelist` option for llvm-libtool-darwin. `-filelist`
option allows for passing in a file containing a list of filenames.

Reviewed by jhenderson, smeenai

Differential Revision: https://reviews.llvm.org/D84206
2020-08-07 14:29:24 -07:00
Sameer Arora 645de3664a [llvm-libtool-darwin] Add constant CPU_SUBTYPE_ARM64_V8
Add support for constant MachO::CPU_SUBTYPE_ARM64_V8. This constant is
needed so as to match `llvm-libtool-darwin`'s behavior to that of
cctools' libtool when `-arch_only` flag is passed in on command line.

Reviewed by jhenderson, alexshap, smeenai

Differential Revision: https://reviews.llvm.org/D85041
2020-08-07 14:09:27 -07:00
Xing GUO 40506d5e2f [DWARFYAML][debug_info] Make the 'Values' field optional.
This patch makes the 'Values' field optional. This is useful when we
handcraft the terminating entry of DIEs.

```
debug_info:
  - Version:  4
    ...
    Entries:
      - AbbrCode: 1
        Values:
          - Value: 0x1234
      - AbbrCode: 0 ## Termination
```

Reviewed By: jhenderson, grimar

Differential Revision: https://reviews.llvm.org/D85397
2020-08-06 20:43:52 +08:00
Xing GUO 6616d84cee [obj2yaml] Test dumping an empty .debug_aranges section.
This patch adds one test case that tests dumping an empty .debug_aranges
section.

Reviewed By: jhenderson, grimar

Differential Revision: https://reviews.llvm.org/D85405
2020-08-06 20:40:12 +08:00
Alexey Lapshin c6eb76093c [dsymutil] Disable dsymutil/X86/reproducer.test on windows.
The dsymutil/X86/reproducer.test test could create paths
longer than MAX_PATH:

C:\ps4-buildslave2\llvm-clang-x86_64-expensive-checks-win
\build\test\tools\dsymutil\X86\Output\reproducer.test.tmp.repro
\ps4-buildslave2\llvm-clang-x86_64-expensive-checks-win\build
\test\tools\dsymutil\X86\Output\reproducer.test.tmp\Inputs\
basic1.macho.x86_64.o

Disable this test on windows.

Reviewed By: JDevlieghere

Differential Revision: https://reviews.llvm.org/D85294
2020-08-06 12:49:35 +03:00
Greg Clayton e1de85f9f4 Add verification for DW_AT_decl_file and DW_AT_call_file.
LTO builds have been creating invalid DWARF and one of the errors was a file index that was out of bounds. "llvm-dwarfdump --verify" will check all file indexes for line tables already, but there are no checks for the validity of file indexes in attributes.

The verification will verify if there is a DW_AT_decl_file/DW_AT_call_file that:
- there is a line table for the compile unit
- the file index is valid
- the encoding is appropriate

Tests are added that test all of the above conditions.

Differential Revision: https://reviews.llvm.org/D84817
2020-08-05 15:30:13 -07:00
Xing GUO bd7f3f8a3e [obj2yaml] Add support for dumping the .debug_aranges section.
This patch adds support for dumping DWARF sections to obj2yaml. The
.debug_aranges section is used to illustrate the basic idea.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D85094
2020-08-05 19:19:05 +08:00
Georgii Rymar f97019ad6e [llvm-readobj/elf] - Add a testing for --stackmap and refine the implementation.
Currently, we only test the `--stackmap` option here:
https://github.com/llvm/llvm-project/blob/master/llvm/test/Object/stackmap-dump.test
it uses a precompiled MachO binary currently and I've found no tests for this option for ELF.

The implementation also has issues. For example, it might assert on a wrong version
of the .llvm-stackmaps section. Or it might crash on an empty or truncated section.

This patch introduces a new tools/llvm-readobj/ELF test file as well as implements a few
basic checks to catch simple crashes/issues

It also eliminates `unwrapOrError` calls in `printStackMap()`.

Differential revision: https://reviews.llvm.org/D85208
2020-08-05 13:09:04 +03:00
Benjamin Kramer c558c22cab [llvm-symbolizer] Add legacy aliases -demangle=true and -demangle=false.
This is used in the wild, don't break compatibility for no good reason.
https://github.com/google/pprof/blob/master/internal/binutils/addr2liner_llvm.go
2020-08-05 12:07:46 +02:00
Martin Storsjö b989fcbae6 [llvm-rc] Allow string table values split into multiple string literals
This can practically easily be a product of combining strings with
macros in resource files.

This fixes https://github.com/mstorsjo/llvm-mingw/issues/140.

As string literals within llvm-rc are handled as StringRefs, each
referencing an uninterpreted slice of the input file, with actual
interpretation of the input string (codepage handling, unescaping etc)
done only right before writing them out to disk, it's hard to
concatenate them other than just bundling them up in a vector,
without rearchitecting a large part of llvm-rc.

This matches how the same already is supported in VersionInfoValue,
with a std::vector<IntOrString> Values.

MS rc.exe only supports concatenated string literals in version info
values (already supported), string tables (implemented in this patch)
and user data resources (easily implemented in a separate patch, but
hasn't been requested by any end user yet), while GNU windres supports
string immediates split into multiple strings anywhere (e.g. like
(100 ICON "myicon" ".ico"). Not sure if concatenation in other
statements actually is used in the wild though, in resource files
normally built by GNU windres.

Differential Revision: https://reviews.llvm.org/D85183
2020-08-05 08:59:32 +03:00
Zequan Wu e3df947175 [llvm-cov] reset executation count to 0 after wrapped segment
Fix the bug: https://bugs.llvm.org/show_bug.cgi?id=36979. It also fixes this bug: https://bugs.llvm.org/show_bug.cgi?id=35404, which I think is caused by the same problem.

Differential Revision: https://reviews.llvm.org/D85036
2020-08-04 18:38:44 -07:00
Fangrui Song 0729a77280 [llvm-symbolizer][test] Fix pdb/pdb.test after D83530
This is a Windows only test which requires HAVE_DIA_SDK, so I failed to notice it.
2020-08-04 12:23:39 -07:00
Fangrui Song 12cb400fd2 [llvm-symbolizer] Add compatibility aliases for --inlining={true,false}
D83530 removed --inlining={true,false} which were used by old asan_symbolize.py script.
Add compatibility aliases so that old asan_symbolize.py and sanitizer
binaries can work with new llvm-symbolizer.

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D85228
2020-08-04 11:32:05 -07:00
Fangrui Song 593e196297 [llvm-symbolizer] Switch command line parsing from llvm::cl to OptTable
for the advantage outlined by D83639 ([OptTable] Support grouped short options)

Some behavior changes:

* -i={0,false} is removed. Use --no-inlines instead.
* --demangle={0,false} is removed. Use --no-demangle instead
* -untag-addresses={0,false} is removed. Use --no-untag-addresses instead

Added a higher level API OptTable::parseArgs which handles optional
initial options populated from an environment variable, expands response
files recursively, and parses options.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D83530
2020-08-04 08:53:15 -07:00
Xing GUO 79b44a4d47 [YAMLTraits] Fix mapping <none> value that followed by comments.
When mapping an optional value, if the value is <none> and followed
by comments, there will be a parsing error. This patch helps fix this
issue.

e.g.,

When mapping the following YAML,

```
Sections:
  - Name:  blah
    Type:  SHT_foo
    Flags: [[FLAGS=<none>]] ## some comments.
```

the raw value of `ScalarNode` is "<none> " rather than "<none>". We need
to remove the spaces.

Differential Revision: https://reviews.llvm.org/D85180
2020-08-04 18:36:05 +08:00
Georgii Rymar 9486201837 [llvm-readobj/readelf] - Refine the implementation of printMipsOptions().
`printMipsOptions()` and the test related has the following issues currently:

1) It does not check the value of Elf_Mips_Options<ELFT>::size field.
2) For ODK_REGINFO options it is possible to read past the end of buffer,
   because there is no check against the `sizeof(Elf_Mips_RegInfo<ELFT>)`.
3) The error about the broken size is just printed to the standard output.
4) The binary input is used for the test.
5) There is no testing for multiple options in the .MIPS.options section,
   though the code supports it.
6) Only llvm-readobj is tested, but not llvm-readelf.
7) "Unsupported MIPS options tag" message does not reveal the tag ID/name.

This patch fixes all of these points.

Differential revision: https://reviews.llvm.org/D84854
2020-08-04 11:40:02 +03:00
Fangrui Song bcea3a7a28 Add test utility 'split-file'
See https://lists.llvm.org/pipermail/llvm-dev/2020-July/143373.html
"[llvm-dev] Multiple documents in one test file" for some discussions.

This patch has explored several alternatives. The current semantics are similar to
what @dblaikie proposed.
`split-file filename output` splits the input file into multiple parts separated by
regex `^(.|//)--- filename` and write each part to the file `output/filename`
(`filename` can include path separators).

Use case A (organizing input of different formats (e.g. linker
script+assembly) in one file).

```
# RUN: split-file %s %t
# RUN: llvm-mc %t/asm -o %t.o
# RUN: ld.lld -T %t/lds %t.o -o %t
This is sometimes better than the %S/Inputs/ approach because the user
can see the auxiliary files immediately and don't have to open another file.

# asm
...
# lds
...
```

Use case B (for utilities which don't have built-in input splitting
feature):

```
// RUN: split-file %s %t
// RUN: llc < %t/1.ll | FileCheck %s --check-prefix=CASE1
// RUN: llc < %t/2.ll | FileCheck %s --check-prefix=CASE2
Combing tests prudently can improve readability.
For example, when testing parsing errors if the recovery mechanism isn't possible,
grouping the tests in one file can more readily see test coverage/strategy.

//--- 1.ll
...
//--- 2.ll
...
```

Since this is a new utility, there is no git history concerns for
UpperCase variable names. I use lowerCase variable names like mlir/lld.

Reviewed By: jhenderson, lattner

Differential Revision: https://reviews.llvm.org/D83834
2020-08-03 20:42:09 -07:00
Xing GUO 08649d4321 [DWARFYAML] Implement the .debug_loclists section.
This patch implements the .debug_loclists section. There are only two
DWARF expressions are implemented in this patch (DW_OP_consts,
DW_OP_stack_value). We will implement more in the future.

The YAML description of the .debug_loclists section is:

```
debug_loclists:
  - Format:              DWARF32 ## Optional
    Length:              0x1234  ## Optional
    Version:             5       ## Optional (5 by default)
    AddressSize:         8       ## Optional
    SegmentSelectorSize: 0       ## Optional (0 by default)
    OffsetEntryCount:    1       ## Optional
    Offsets:             [ 1 ]   ## Optional
    Lists:
      - Entries:
          - Operator:          DW_LLE_startx_endx
            Values:            [ 0x1234, 0x4321 ]
            DescriptorsLength: 0x1234             ## Optional
            Descriptors:
              - Operator: DW_OP_consts
                Values:   [ 0x1234 ]
```

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D84234
2020-08-03 23:20:15 +08:00
Xing GUO 2d8ca4ae2b [DWARFYAML] Offsets should be omitted when the OffsetEntryCount is 0.
The offsets field should be omitted when the 'OffsetEntryCount' entry is
specified to be 0.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D85006
2020-08-03 22:06:08 +08:00
Georgii Rymar 3de9323001 [llvm-readobj] - Don't stop dumping when the name of a relocation section can't be read.
This removes undesired `unwrapOrError` calls from printRelocations() methods.

Differential revision: https://reviews.llvm.org/D84408
2020-08-03 14:49:09 +03:00
Georgii Rymar 2bca784ab8 [llvm-readobj] - Massive test cases cleanup.
This patch does the following:
1) Starts using YAML macro to reduce the number of YAML documents in tests.
2) Adds `#` before 'RUN'/`CHECK` lines in a few tests where it is missing.
3) Removes unused YAML keys.
4) Starts using `ENTSIZE=<none>` to simplify tests (see D84526).
5) Removes trailing white spaces in a few places.

Differential revision: https://reviews.llvm.org/D85013
2020-08-03 14:07:06 +03:00
Georgii Rymar 7e32797552 [llvm-readobj] - Don't call `unwrapOrErr` in `findSectionByName`.
We have a `findSectionByName` helper that tries to find a section
by it name. It is used in a few places, but never tested.

I'd like to reuse this helper for a different place.
For this, I've changed it to return Expected<> and now it
doesn't use `unwrapOrErr` anymore. It also now a member of
Dumper class and might report warnings.

Differential revision: https://reviews.llvm.org/D84651
2020-08-03 12:41:00 +03:00
Georgii Rymar d919ae9df8 [yaml2obj] - Add a support for "<none>" value for all optional fields.
It implements an approach suggested in the D84398 thread.

With it the following:

```
Sections:
  - Name:   .bar
    Type:   SHT_PROGBITS
    Offset: [[MACRO=<none>]]
```

works just like the `Offset` key was not specified.
It is useful for tests that want to have a default value for a field and to
have a way to override it at the same time.

Differential revision: https://reviews.llvm.org/D84526
2020-08-03 12:27:39 +03:00
Xing GUO 74b02d73e3 [DWARFYAML] Make the debug_aranges entry optional.
This patch makes the 'debug_aranges' entry optional. If the entry is
empty, yaml2obj will only emit the header for it.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D84921
2020-07-31 20:18:53 +08:00
Xing GUO 3da6a974db [DWARFYAML] Make the 'Length' field of the address range table optional.
This patch makes the 'Length' field of the address range table optional.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D84911
2020-07-30 17:42:18 +08:00
Xing GUO 006f6f8ac6 [DWARFYAML] Make the 'AddressSize', 'SegmentSelectorSize' fields optional.
This patch makes the 'AddressSize' and 'SegmentSelectorSize' fields of
address range table optional.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D84907
2020-07-30 17:39:58 +08:00
Xing GUO bfa140376d [DWARFYAML] Make the field names consistent with the DWARF spec. NFC.
This patch replaces 'AddrSize'/'SegSize' with
'AddressSize'/'SegmentSelectorSize'. NFC.
2020-07-29 23:10:08 +08:00
Xing GUO c12394fef3 [llvm-readelf][test] Improve wording in the comments. NFC.
This patch addresses comments in D84640 (https://reviews.llvm.org/D84640#2178475).
2020-07-29 09:59:28 +08:00
Fangrui Song dd405f1a53 Revert D83834 "Add test utility 'extract'"
This reverts commit d054c7ee2e.

There are discussions about the utility name, its functionality and user interface.
Revert before we reach consensus.
2020-07-28 13:26:33 -07:00
Xing GUO 22ec861d28 [DWARFYAML] Add support for emitting custom range list content.
This patch adds support for emitting custom range list content.

We are able to handcraft a custom range list via the following syntax.

```
debug_rnglists:
  - Lists:
      - Entries:
          - Operator: DW_RLE_startx_endx
            Values:   [ 0x1234, 0x1234 ]
      - Content: '1234567890abcdef'
      - Content: 'abcdef1234567890'
```

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D84618
2020-07-28 22:11:16 +08:00
Georgii Rymar bd93f5ce07 [yaml2obj] - Add a way to override sh_type section field.
This adds the `ShType` key similar to others `Sh*` keys we have.

My use case is the following. Imagine we have a `SHT_SYMTAB_SHNDX`
section and want to hide it from a dumper. The natural way would be to
do something like:

```
  - Name:    .symtab_shndx
    Type:    [[TYPE=SHT_SYMTAB_SHNDX]]
    Entries: [ 0, 1 ]

```

and then change the TYPE from `SHT_SYMTAB_SHNDX` to something else,
for example to `SHT_PROGBITS`.

But we have a problem: regular sections does not have `Entries` key,
so yaml2obj will be unable to produce a section.

The solution is to introduce a `ShType` key to override the final type.

This is not the first time I am facing the need to change the type. I
was able to invent workarounds or solved issues differently in the past,
but finally came to conclusion that we just should support the `ShType`.

Differential revision: https://reviews.llvm.org/D84738
2020-07-28 16:16:42 +03:00
Georgii Rymar 93b7915504 [llvm-readobj] - Add comments and formatting to mips-options-sec.test and mips-reginfo.test. NFCI.
This will allow to extend them (needed for D84651).
2020-07-28 13:32:09 +03:00
Mikhail Kalashnikov 67070d98fa [llvm-readelf] Symbol index in symbol table printing is not reset
Stop using static variables for keeping track of symbol indices.

Bugfix for: https://bugs.llvm.org/show_bug.cgi?id=46777

Differential revision: https://reviews.llvm.org/D84606
2020-07-28 13:15:53 +03:00
Georgii Rymar ee068aafbc [llvm-readelf] - Do not treat SHT_ANDROID_RELR sections the same as SHT_RELR.
Currently, when dumping section headers, llvm-readelf
prints "RELR" for SHT_ANDROID_RELR/SHT_RELR sections.
The behavior was introduced in D47919 and revealed in D84330.

But "SHT_ANDROID_RELR" has a different value from "SHT_RELR".
Also, "SHT_ANDROID_REL/SHT_ANDROID_RELA" are printed as "ANDROID_REL/ANDROID_RELA",
what makes the handling of the "SHT_ANDROID_RELR" inconsistent.

This patch makes llvm-readelf to print "ANDROID_RELR" instead of "RELR".

Differential revision: https://reviews.llvm.org/D84393
2020-07-28 11:24:58 +03:00
Wei Mi a23f62343c Supplement instr profile with sample profile.
PGO profile is usually more precise than sample profile. However, PGO profile
needs to be collected from loadtest and loadtest may not be representative
enough to the production workload. Sample profile collected from production
can be used as a supplement -- for functions cold in loadtest but warm/hot
in production, we can scale up the related function in PGO profile if the
function is warm or hot in sample profile.

The implementation contains changes in compiler side and llvm-profdata side.
Given an instr profile and a sample profile, for a function cold in PGO
profile but warm/hot in sample profile, llvm-profdata will either mark
all the counters in the profile to be -1 or scale up the max count in the
function to be above hot threshold, depending on the zero counter ratio in
the profile. The assumption is if there are too many counters being zero
in the function profile, the profile is more likely to cause harm than good,
then llvm-profdata will mark all the counters to be -1 indicating the
function is hot but the profile is unaccountable. In compiler side, if a
function profile with all -1 counters is seen, the function entry count will
be set to be above hot threshold but its internal profile will be dropped.

In the long run, it may be useful to let compiler support using PGO profile
and sample profile at the same time, but that requires more careful design
and more substantial changes to make two profiles work seamlessly. The patch
here serves as a simple intermediate solution.

Differential Revision: https://reviews.llvm.org/D81981
2020-07-27 20:17:40 -07:00
Xing GUO 6bf989b947 [llvm-readelf] Fix emitting incorrect number of spaces in '--hex-dump'.
This patch helps teach llvm-readelf to emit a correct number spaces when
dumping in hex format.

Before this patch, when the hex data doesn't fill the 4th column, some
spaces are missing.

```
Hex dump of section '.sec':
0x00000000 00000000 00000000 00000000 00000000 ................
0x00000010 00000000 00000000 00000000 0000 ..............
```

After this patch:

```
Hex dump of section '.sec':
0x00000000 00000000 00000000 00000000 00000000 ................
0x00000010 00000000 00000000 00000000 0000     ..............
```

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D84640
2020-07-28 09:54:33 +08:00
Vy Nguyen ee7caa7593 Reland [llvm-exegesis] Add benchmark latency option on X86 that uses LBR for more precise measurements.
Starting with Skylake, the LBR contains the precise number of cycles between the two
        consecutive branches.
        Making use of this will hopefully make the measurements more precise than the
        existing methods of using RDTSC.

                Differential Revision: https://reviews.llvm.org/D77422

New change: check for existence of field `cycles` in perf_branch_entry before enabling this mode.
This should prevent compilation errors when building for older kernel whose headers don't support it.
2020-07-27 12:38:05 -04:00
Roman Lebedev f2ab2134c7
[XRay] Account: recursion detection
Summary:
Recursion detection can be non-trivial. Currently, the state-of-the-art for LLVM,
as far as i'm concerned, is D72362 `[clang-tidy] misc-no-recursion: a new check`.
However, it is quite limited:
* It does very basic call-graph based analysis, in the sense it will report even dynamically-unreachable recursion.
* It is inherently limited to a single TU
* It is hard to gauge how problematic each recursion is in practice.

Some of that can be addressed by adding clang analyzer-based check,
then it would at least support multiple TU's.

However, we can approach this problem from another angle - dynamic run-time analysis.
We already have means to capture a run-time callgraph (XRay, duh),
and there are already means to reconstruct it within `llvm-xray` tool.

This proposes to add a `-recursive-calls-only` switch to the `account` tool.
When the switch is on, when re-constructing callgraph for latency reconstruction,
each time we enter/leave some function, we increment/decrement an entry for the function
in a "recursion depth" map. If, when we leave the function, said entry was at `1`,
then that means the function didn't call itself, however if it is at `2` or more,
then that means the function (possibly indirectly) called itself.

If the depth is 1, we don't account the time spent there,
unless within this call stack the function already recursed into itself.
Note that we don't pay for recursion depth tracking when `recursive-calls-only` is not on,
and the perf impact is insignificant (+0.3% regression)

The overhead of the option is actually negative, around -5.26% user time on a medium-sized (3.5G) XRay log.
As a practical example, that 3.5G log is a capture of the entire middle-end opt pipeline
at `-O3` for RawSpeed unity build. There are total of `5500` functions in the log,
however `-recursive-calls-only` says that `269`, or 5%, are recursive.

Having this functionality could be helpful for recursion eradication.

Reviewers: dberris, mboerger

Reviewed By: dberris

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D84582
2020-07-27 10:15:44 +03:00
Martin Storsjö 4d09ed953b [llvm-lib] Support adding short import library objects with llvm-lib
This fixes PR 42837.

Differential Revision: https://reviews.llvm.org/D84465
2020-07-24 22:15:08 +03:00
Georgii Rymar 6a0f0746ac [obj2yaml][yaml2obj] - Add note-section.yaml tests.
They were a part of D68983, but were lost in the last
diff and were not committed for unknown reason.

I've renamed (from elf-sht-note.yaml) them and fixed
broken formating a few places. Everything else remained
untouched.
2020-07-24 14:27:54 +03:00
Fangrui Song d054c7ee2e Add test utility 'extract'
See https://lists.llvm.org/pipermail/llvm-dev/2020-July/143373.html
"[llvm-dev] Multiple documents in one test file" for some discussions.

`extract part filename` splits the input file into multiple parts separated by
regex `^(.|//)--- ` and extract the specified part to stdout or the
output file (if specified).

Use case A (organizing input of different formats (e.g. linker
script+assembly) in one file).

```
// RUN: extract lds %s -o %t.lds
// RUN: extract asm %s -o %t.s
// RUN: llvm-mc %t.s -o %t.o
// RUN: ld.lld -T %t.lds %t.o -o %t
This is sometimes better than the %S/Inputs/ approach because the user
can see the auxiliary files immediately and don't have to open another file.
```

Use case B (for utilities which don't have built-in input splitting
feature):

```
// RUN: extract case1 %s | llc | FileCheck %s --check-prefix=CASE1
// RUN: extract case2 %s | llc | FileCheck %s --check-prefix=CASE2
Combing tests prudently can improve readability.
This is sometimes better than having multiple test files.
```

Since this is a new utility, there is no git history concerns for
UpperCase variable names. I use lowerCase variable names like mlir/lld.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D83834
2020-07-23 19:15:35 -07:00
Xing GUO 92874d2866 [DWARFYAML] Refactor emitDebugInfo() to make the length be inferred.
This patch refactors `emitDebugInfo()` to make the length field be
inferred from its content. Besides, the `Visitor` class is removed in
this patch. The original `Visitor` class helps us determine an
appropriate length and emit the .debug_info section. These two
processes can be merged into one process. Besides, the length field
should be inferred when it's missing rather than when it's zero.

Reviewed By: jhenderson, labath

Differential Revision: https://reviews.llvm.org/D84008
2020-07-23 23:00:19 +08:00
Rong Xu 50da55a585 [PGO] Supporting code for always instrumenting entry block
This patch includes the supporting code that enables always
instrumenting the function entry block by default.

This patch will NOT the default behavior.

It adds a variant bit in the profile version, adds new directives in
text profile format, and changes llvm-profdata tool accordingly.

This patch is a split of D83024 (https://reviews.llvm.org/D83024)
Many test changes from D83024 are also included.

Differential Revision: https://reviews.llvm.org/D84261
2020-07-22 15:01:53 -07:00
Georgii Rymar e2529e2dfa [llvm-readobj] - Don't get the name of the symbol table in ELFDumper<ELFT>::printSymbolsHelper.
It was requested in D84173 thread to not do it, because otherwise we extract and
check the name of the symbol table in LLVM style, but do not use it and
might report a warning which perhaps might be confusing.

Differential revision: https://reviews.llvm.org/D84231
2020-07-22 17:28:20 +03:00
Georgii Rymar 3520297039 [llvm-readelf] - Introduce describe() helper functions.
These functions can be used to generate strings like
"SHT_?? section with index ?" to describe sections in error/warning messages,
what helps to simplify and generalize them.

Also this allows to isolate the following common code pattern:
`&Sec - &cantFail(Obj->sections()).front();`

Differential revision: https://reviews.llvm.org/D84240
2020-07-22 14:03:17 +03:00
Georgii Rymar 066e209c9d [llvm-readelf/readobj] - Fix the behavior when a sections is included in two groups at the same time.
The current behavior was introduced by me in D37567 and it is a bit strange. It prints the
"Error: ...." message to the errs() manually and stops dumping the group section which has this error.
This behavior is consistent with GNU though, but it is very inconsistent with what the regular llvm-readelf
code usually does/prints, so I suggest to change the implementation:

1) Instead of printing "Error: ...." to errs() - just report a warning.
2) Try to continue dumping the section.
3) Merge broken-group.test to group.text.

This is what this patch does.

Differential revision: https://reviews.llvm.org/D84170
2020-07-22 13:29:54 +03:00
Georgii Rymar 54ef74f738 [llvm-readobj/readelf] - Don't fail dumping when unable to read the name of the SHT_DYNSYM section.
We have an issue currently: we are trying to read the name of the SHT_DYNSYM section
very early and using `unwrapOrError` call for that.

The name is needed only for the GNU output. Because of the current logic, the tool
fails to dump the whole object when something is wrong with the name of the .dynsym section.

This patch delays reading the name and also allows it to be broken.

Differential revision: https://reviews.llvm.org/D84173
2020-07-22 13:11:46 +03:00
Amy Huang 0881d0bed3 [PDB][NativeSession] Clean up some things in NativeSession.
-Use the actual sect/offset to keep track of symbols in the cache so they don't get created multiple times with different addresses.
-Remove getSymTag from PDBFunctionSymbol/PDBPublicSymbol because it's already implemented in the base class
-Merge the symbolizer test files for DIA and native, since the tests are the same.
-Implement getCompilandId for NativeLineNumber

Reviewed By: amccarth

Differential Revision: https://reviews.llvm.org/D84208
2020-07-21 16:54:52 -07:00
Sameer Arora 9e783716a2 [llvm-libtool-darwin] Allow flattening archives
Add support for flattening archives while creating static libraries.
Hence, can now pass archives as input in addition to Mach-O binaries.
Furthermore, archives themselves must only conatain Mach-O binaries. As
per cctools' libtool's behavior, llvm-libtool-darwin does not flatten
archives recursively.

Reviewed by alexshap, smeenai, jhenderson

Differential Revision: https://reviews.llvm.org/D83520
2020-07-21 13:53:15 -07:00
Sameer Arora 303a7f7a26 [llvm-libtool-darwin] Add support for -static option
Add support for creating static libraries when the input includes only
Mach-O binaries (and not libraries/archives themselves).

Reviewed by alexshap, Ktwu, smeenai, jhenderson, MaskRay, mtrent

Differential Revision: https://reviews.llvm.org/D83002
2020-07-21 13:08:49 -07:00
Fangrui Song aa830e9768 [LLVMgold.so][test] Fix tests after D84132/55fa315b0352 2020-07-21 10:18:32 -07:00
Fangrui Song 8c6d48baf6 [llvm-readobj] Construct relocation-aware DWARFDataExtractor to decode .eh_frame addresses correctly
In an object file, a "PC Begin" field in a FDE is usually relocated by a
PC-relative relocation. Use a relocation-aware DWARFDataExtractor overload (with
DWARFContext and a reference to its internal .eh_frame representation) to decode
addresses correctly. In an object file, most sections have addresses of zero. So
the displayed addresses are almost always offsets relative to the start of the
associated text section.

DWARFContext::create handles .eh_frame and .rela.eh_frame by itself, so if there
are more than one .eh_frame (technically possible, but almost always erronerous
in practice), this will only handle the first one.  Supporting multiple
.eh_frame is beyond the scope of this patch.

Reviewed By: grimar, jhenderson

Differential Revision: https://reviews.llvm.org/D84106
2020-07-21 08:33:19 -07:00
Fangrui Song 55fa315b03 [LLVMgold.so] -plugin-opt=save-temps: save combined module to .lto.o instead of .o
This matches LLD and fixes https://sourceware.org/bugzilla/show_bug.cgi?id=26262#c1

.o is a bad choice for save-temps output because it is easy to override the bitcode file (*.o)

```
 # Use bfd for the example, -fuse-ld=gold is similar.
clang -flto -c a.c  # generate bitcode file a.o
clang -fuse-ld=bfd -flto a.o -o a -Wl,-plugin-opt=save-temps  # override a.o

 # The user repeats the command but get surprised, because a.o is now a combined module.
clang -fuse-ld=bfd -flto a.o -o a -Wl,-plugin-opt=save-temps
```

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D84132
2020-07-20 10:02:56 -07:00
Elvina Yakubova df952cb914 [llvm-readobj] Print error when executed with no input files
This patch changes llvm-readelf (and llvm-readobj for consistency)
behavior to print an error when executed with no input files.

Reading from stdin can be achieved via a '-' for the input
object.

Fixes https://bugs.llvm.org/show_bug.cgi?id=46400

Differential Revision: https://reviews.llvm.org/D83704

Reviewed by: jhenderson, MaskRay, sbc, jyknight
2020-07-20 10:39:05 +01:00
Elvina Yakubova b36a3e6140 [llvm-readobj] Update tests because of changes in llvm-readobj behavior
This patch updates tests using llvm-readobj and llvm-readelf, because
soon reading from stdin will be achievable only via a '-' as described
here: https://bugs.llvm.org/show_bug.cgi?id=46400. Patch with changes to
llvm-readobj behavior is here: https://reviews.llvm.org/D83704

Differential Revision: https://reviews.llvm.org/D83912

Reviewed by: jhenderson, MaskRay, grimar
2020-07-20 10:39:04 +01:00
Georgii Rymar 256aea816d [llvm-readelf/readobj] - Refine the error reporting in printMipsABIFlags() methods.
It fixes/improves the following:
1) Some code was duplicated.
2) A "The .MIPS.abiflags section has a wrong size" error was not reported as a warning,
   but was printed to stdout for the LLVM style. Also, it was reported as an error for the GNU style.
   This patch changes the behavior to be consistent and to report warnings.
3) `unwrapOrError()` was used before, now a warning is reported instead.

Differential revision: https://reviews.llvm.org/D84033
2020-07-20 11:30:17 +03:00
Xing GUO 1ab3d6c819 [DWARFYAML] Implement the .debug_rnglists section.
This patch implements the .debug_rnglists section. We are able to
produce the .debug_rnglists section by the following syntax.

```
debug_rnglists:
  - Format:              DWARF32 ## Optional
    Length:              0x1234  ## Optional
    Version:             5       ## Optional
    AddressSize:         0x08    ## Optional
    SegmentSelectorSize: 0x00    ## Optional
    OffsetEntryCount:    2       ## Optional
    Offsets:             [1, 2]  ## Optional
    Lists:
      - Entries:
          - Operator: DW_RLE_base_address
            Values:   [ 0x1234 ]
```

The generated .debug_rnglists is verified by llvm-dwarfdump, except for
the operator DW_RLE_startx_endx, since llvm-dwarfdump doesn't support
it.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D83624
2020-07-20 10:42:27 +08:00
sstefan1 cf11050696 [Utils][Fix] remove unnecessary ; at the end 2020-07-19 20:48:28 +02:00
sstefan1 937bad3594 [Utils] Check function attributes in update_test_checks
Summary:
This introduces new flag to the update_test_checks and
update_cc_test_checks that allows for function attributes
to be checked in a check-line. If the flag is not set,
the behavior should remain the same.

Reviewers: jdoerfert

Subscribers: arichardson, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D83629
2020-07-19 20:07:24 +02:00
Fangrui Song 3ab0f53ef3 [DebugInfo] Respect relocations when decoding DW_EH_PE_sdata4 & DW_EH_PE_sdata8 and support R_ARM_REL32
The addresses in llvm-dwarfdump --eh-frame output for object files are closer to readelf -wf output now.
2020-07-18 09:00:50 -07:00
Sameer Arora 6c43ed608d Introducing llvm-libtool-darwin
This diff starts the implementation of llvm-libtool-darwin
(an llvm based replacement of cctool's libtool).
Libtool is used for creating static and dynamic libraries
from a bunch of object files given as input.

Reviewed by alexshap, smeenai, jhenderson, MaskRay

Differential Revision: https://reviews.llvm.org/D82923
2020-07-17 08:07:02 -07:00
Clement Courbet 6bddd099ac Revert "[llvm-exegesis] Add benchmark latency option on X86 that uses LBR for more precise measurements."
From @erichkeane:
```
This patch doesn't seem to build for me:
/iusers/ekeane1/workspaces/llvm-project/llvm/tools/llvm-exegesis/lib/X86/X86Counter.cpp: In function ‘llvm::Error llvm::exegesis::parseDataBuffer(const char*, size_t, const void*, const void*, llvm::SmallVector<long int, 4>*)’:
/iusers/ekeane1/workspaces/llvm-project/llvm/tools/llvm-exegesis/lib/X86/X86Counter.cpp:99:37: error: ‘struct perf_branch_entry’ has no member named ‘cycles’

CycleArray->push_back(Entry.cycles);
I'm on RHEL7, so I have kernel 3.10, so it doesn't have 'cycles'.

According ot this: https://elixir.bootlin.com/linux/v4.3/source/include/uapi/linux/perf_event.h#L963 kernel 4.3 is the first time that 'cycles' appeared in this structure.
```
2020-07-17 16:55:17 +02:00
Georgii Rymar 6227f04a09 [llvm-readobj] - Add proper testing for the SHT_MIPS_ABIFLAGS section.
This rewrites the mips-abiflags.test to stop using recompiled objects,
adds testing for all missed bits and also adds two missing enum values
to lib/ObjectYAML, which are used in the new test.

Differential revision: https://reviews.llvm.org/D83954
2020-07-17 15:24:39 +03:00
Vy Nguyen 1360e140cc [llvm-exegesis] Add benchmark latency option on X86 that uses LBR for more precise measurements.
Starting with Skylake, the LBR contains the precise number of cycles between the two
    consecutive branches.
    Making use of this will hopefully make the measurements more precise than the
    existing methods of using RDTSC.

            Differential Revision: https://reviews.llvm.org/D77422
2020-07-16 12:12:46 -04:00
Xing GUO 10478c9c64 [DWARFYAML] Implement the .debug_str_offsets section.
This patch helps add support for emitting the .debug_str_offsets section
to yaml2elf.

Reviewed By: jhenderson, MaskRay

Differential Revision: https://reviews.llvm.org/D83853
2020-07-16 22:33:13 +08:00
Georgii Rymar ed605b7b96 [yaml2obj] - Fix an issue with NoHeaders key.
When setting the NoHeaders to false,
the e_shnum field wasn't set correctly.

This patch fixes this bug.

Differential revision: https://reviews.llvm.org/D83941
2020-07-16 14:22:36 +03:00
Georgii Rymar 7a587ca932 [yaml2obj] - Rename FileHeader::SH* fields.
In D83482 we agreed to name e_* fields that are used for overriding
values (like e_phoff) as EPh* (e.g. EPhOff).

Currently we have a set of e_sh* fields that are named inconsistently
with this rule. This patch renames all of them.

Differential revision: https://reviews.llvm.org/D83766
2020-07-15 12:47:31 +03:00
Georgii Rymar 327c445035 [llvm-readobj] - Verify the location of program headers better.
This improves condition in the ELFFile::program_headers().
Previously if was possible to read the headers from the wrong place when
the value of e_phoff was so large that computation overflowed.

Differential revision: https://reviews.llvm.org/D83774
2020-07-15 12:37:23 +03:00
Richard Smith f49edafd9a Fix test that was accidentally adding the llvm-as binary into an
IR archive.
2020-07-14 17:07:40 -07:00
Jan Sjodin 66b409582a llvm-link: Add support for archive files as inputs
This patch adds support for archive files as inputs to llvm-link. One
of the use-cases is for OpenMP, where device specific libraries need
to be extracted from libraries containing bundled object files. The
clang-offload-bundler will support extracting these archives, which
will be passed into llvm-link, see https://reviews.llvm.org/D80816.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D81109
2020-07-14 15:30:59 -04:00
Shuhong Liu 85bed2f381 [AIX] Remove diff -a option on llvm-cov.test
Summary:
llvm-cov.test generates .gcov files and compared with target
sample files. Since the files do not contain any binary data
(files are plain ASCII texts), remove -a from diff. And this
fix will the error on AIX since the default diff tool on AIX
does not support -a option.

Reviewers: hubert.reinterpretcast, daltenty, stevewan

Subscribers: llvm-commits

Tags: #LLVM

Differential Revision: https://reviews.llvm.org/D83711
2020-07-14 12:24:36 -04:00
Georgii Rymar 90e34b563a [yaml2obj] - Refine handling of the NoHeaders key.
Imagine we have an YAML description for some object and we want to
produce 2 outputs: with and without the section header.
A natural way to do it would look like:

```
--- !ELF
FileHeader:
  Class:   ELFCLASS64
  Data:    ELFDATA2LSB
  Type:    ET_REL
  Machine: EM_X86_64
Sections:
...
SectionHeaderTable:
  NoHeaders: [[NOHEADERS]]

```
But currently, we do not distinguish between no `NoHeaders` key case
and `NoHeaders == false`. Because of this we can't simply specify
`NOHEADERS = false`, as tool starts to complain.

With this patch the behavior changed. When we have:

```
SectionHeaderTable:
  NoHeaders: false

```
it is the same as we have no `SectionHeaderTable` at all.
(`NoHeaders` key still can't be used with `Sections/Excluded` keys)

Differential revision: https://reviews.llvm.org/D83672
2020-07-14 15:09:06 +03:00
Georgii Rymar 84a1bc7f2c [test/Object][llvm-objdump] - llvm-objdump: don't abort() when the e_phoff field is invalid and refine testing.
llvm-objdump currently calls report_fatal_error() when the e_phoff field is invalid.

This is tested by elf-invalid-phdr.test which has the following issues:
1) It uses a precompiled object.
2) it could be a part of invalid.test.
3) It tests the Object lib, but we have no separate test for llvm-objdump.

This patch addresses issues mentioned.

Differential revision: https://reviews.llvm.org/D83559
2020-07-14 14:45:18 +03:00
Georgii Rymar 7ef17638d5 [llvm-readobj] - Stop using unwrapOrError() for all program_headers() calls.
program_headers() returns the list of program headers. This change allows
to continue attempt of dumping when something is wrong with program headers.

Differential revision: https://reviews.llvm.org/D83554
2020-07-14 14:16:57 +03:00
Xing GUO b539f01221 [DWARFYAML] Add support for emitting value forms of strx, addrx, etc.
This patch adds support for emitting value forms of DW_FORM_strx,
DW_FORM_addrx, DW_FORM_loclistx and DW_FORM_rnglistx.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D83749
2020-07-14 19:10:15 +08:00
Georgii Rymar 3001569323 [yaml2obj] - Add a syntax to override e_phoff, e_phentsize and e_phnum fields.
This adds `EPhOff`, `EPhEntSize` and `EPhNum` keys.
Will be useful for creating broken objects for testing llvm-readelf.

Differential revision: https://reviews.llvm.org/D83482
2020-07-14 13:16:19 +03:00
stevewan a7e9c5a39f [llvm-ar][test][AIX] Unsupport error-opening-directory.test on AIX
Summary:
The test fails on AIX as it allows open() and read() on a directory. This patch adds `# UNSUPPORTED:
system-aix` to the test to prevent it from running on AIX.

Reviewers: sameerarora101, daltenty, ShuhongL, hubert.reinterpretcast, MaskRay, smeenai, alexshap

Reviewed By: sameerarora101, hubert.reinterpretcast, MaskRay, smeenai

Subscribers: MaskRay, rupprecht, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D83579
2020-07-13 12:00:07 -04:00
Eric Astor 3aabfa2808 [ms] [llvm-ml] Restore omitted changes requested by reviewer 2020-07-13 10:49:19 -04:00
Eric Astor f08e8b6d7c [ms] [llvm-ml] Add support for MASM STRUCT casting field accessors: (<TYPE> PTR <value>).<field>
Summary:
Add support for MASM STRUCT casting field accessors: (<TYPE> PTR <value>).<field>

Since these are operands, we add them to X86AsmParser. If/when we extend MASM support to other architectures (e.g., ARM), we will need similar changes there as well.

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D83346
2020-07-13 10:40:47 -04:00
Eric Astor 7f85e98082 [ms] [llvm-ml] Fix MASM support for nested unnamed STRUCTs and UNIONs
Summary: Fix MASM support for nested unnamed STRUCTs and UNIONs

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D83345
2020-07-13 10:36:56 -04:00
Eric Astor 4cdea5faf9 [ms] [llvm-ml] Improve MASM STRUCT field accessor support
Summary:
Adds support for several accessors:
- `[<identifier>.<struct name>].<field>`
- `[<identifier>.<struct name>.<field>].<subfield>` (where `field` has already-defined STRUCT type)
- `[<variable>.<field>].<subfield>` (where `field` has already-defined STRUCT type)

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D83344
2020-07-13 10:34:30 -04:00
Georgii Rymar 725412f1f5 [yaml2obj] - Refactor header-sh-fields.yaml test.
This refines the test to use macros. It is needed for
a follow-up change that adds a functionality to
override more fields.

Also, it is just cleaner to test each key separately.

Differential revision: https://reviews.llvm.org/D83481
2020-07-13 16:12:22 +03:00
Georgii Rymar 84a170178c [llvm-readobj] - Add a generic test for --dyn-relocations and fix an issue.
We have an issue currently: --dyn-relocations always prints the following
relocation header when dumping `DynPLTRelRegion`:

"Offset  Info  Type Symbol's Value  Symbol's Name + Addend"

I.e. even for an empty object, --dyn-relocations still prints this.
It is a easy to fix bug, but we have no dedicated test case for this option.
(we have a dynamic-reloc-no-section-headers.test, which has a slightly different purpose).

This patch adds a test and fixes the behavior.

Differential revision: https://reviews.llvm.org/D83387
2020-07-13 14:22:03 +03:00
Fangrui Song d1bcddb5c1 [llvm-objdump][test] Move tests after dc4a6f5db4
Move RISCV/ to ELF/RISCV/ as well.
2020-07-11 16:45:05 -07:00
Vitaly Buka c06417b24d Fix check-all with -DLLVM_USE_SANITIZER=Address 2020-07-10 01:47:51 -07:00