Commit Graph

419 Commits

Author SHA1 Message Date
Sam Clegg 9a72d3e3e4 [WebAssembly] Add support for named data sections in wasm binaries
Followup to https://reviews.llvm.org/D91769 which added support
for names globals.

Differential Revision: https://reviews.llvm.org/D92909
2020-12-09 12:57:07 -08:00
Georgii Rymar abae3c1196 [obj2yaml] - Support dumping objects that have multiple SHT_SYMTAB_SHNDX sections.
It is allowed to have multiple `SHT_SYMTAB_SHNDX` sections, though
we currently don't implement it.

The current implementation assumes that there is a maximum of one SHT_SYMTAB_SHNDX
section and that it is always linked with .symtab section.

This patch drops this limitations.

Differential revision: https://reviews.llvm.org/D92644
2020-12-09 12:14:58 +03:00
Derek Schuff 0a391060f1 [WebAssembly] Add Object and ObjectWriter support for wasm COMDAT sections
Allow sections to be placed into COMDAT groups, in addtion to functions and data
segments.

Also make section symbols unnamed, which allows sections with identical names
(section names are independent of their section symbols, but previously we
gave the symbols the same name as their sections, which results in collisions
when sections are identically-named).

Differential Revision: https://reviews.llvm.org/D92691
2020-12-07 12:12:44 -08:00
Georgii Rymar ffbce65f95 [lib/Object, tools] - Make ELFObjectFile::getELFFile return reference.
We always have an object, so we don't have to return a pointer.

Differential revision: https://reviews.llvm.org/D92560
2020-12-04 16:02:29 +03:00
Georgii Rymar ea8c8a5097 [obj2yaml] - Teach tool to emit the "SectionHeaderTable" key and sort sections by file offset.
Currently when we dump sections, we dump them in the order,
which is specified in the sections header table.

With that the order in the output might not match the order in the file.
This patch starts sorting them by by file offsets when dumping.

When the order in the section header table doesn't match the order
in the file, we should emit the "SectionHeaderTable" key. This patch does it.

Differential revision: https://reviews.llvm.org/D91249
2020-12-01 12:59:15 +03:00
Georgii Rymar ee9ffc7345 [obj2yaml] - Dump the `EShNum` key in some cases.
This patch starts emitting the `EShNum` key, when the `e_shnum = 0`
and the section header table exists.

`e_shnum` might be 0, when the the number of entries in the section header
table is larger than or equal to SHN_LORESERVE (0xff00).
In this case the real number of entries
in the section header table is held in the `sh_size`
member of the initial entry in section header table.

Currently, obj2yaml crashes when an object has `e_shoff != 0` and the `sh_size`
member of the initial entry in section header table is `0`.
This patch fixes it.

Differential revision: https://reviews.llvm.org/D92098
2020-11-27 15:56:10 +03:00
Georgii Rymar c2090ff594 [obj2yaml] - Don't assert when trying to calculate the expected section offset.
The following line asserts when `sh_addralign > MAX_UINT32 && (uint32_t)sh_addralign == 0`:

```
    ExpectedOffset = alignTo(ExpectedOffset,
                             SecHdr.sh_addralign ? SecHdr.sh_addralign : 1);
```

it happens because `sh_addralign` is truncated to 32-bit value, but `alignTo`
doesn't accept `Align == 0`. We should change `1` to `1uLL`.

Differential revision: https://reviews.llvm.org/D92163
2020-11-27 15:38:22 +03:00
Andy Wingo 1933c9d41a [WebAssembly] Factor out WasmTableType in binary format
This commit factors out a WasmTableType definition from WasmTable, as is
the case for WasmGlobal and other data types.  Also add support for
extracting the SymbolName for a table from the linking section's symbol
table.

Differential Revision: https://reviews.llvm.org/D91849
2020-11-25 08:00:08 -08:00
Georgii Rymar 5edb90c927 [obj2yaml] - Dump section offsets in some cases.
Currently we never dump the `sh_offset` key.
Though it sometimes an important information.

To reduce the noise this patch implements the following logic:
1) The "Offset" key for the first section is always emitted.
2) If we can derive the offset for a next section naturally,
   then the "Offset" key is omitted.

By "naturally" I mean that section[X] offset is expected to be:
```
offsetOf(section[X]) == alignTo(section[X - 1].sh_offset + section[X - 1].sh_size, section[X].sh_addralign)
```

So, when it has the expected value, we omit it from the output.

Differential revision: https://reviews.llvm.org/D91152
2020-11-25 12:41:01 +03:00
Sam Clegg 1827005cfc [WebAssembly] Add support for named globals in the object format.
Differential Revision: https://reviews.llvm.org/D91769
2020-11-19 00:17:22 -08:00
Georgii Rymar a7a447be0f [yaml2obj] - ProgramHeaders: introduce FirstSec/LastSec instead of Sections list.
Imagine we have a YAML declaration of few sections: `foo1`, `<unnamed 2>`, `foo3`, `foo4`.

To put them into segment we can do (1*):

```
Sections:
 - Section: foo1
 - Section: foo4
```

or we can use (2*):

```
Sections:
 - Section: foo1
 - Section: foo3
 - Section: foo4
```

or (3*) :

```
Sections:
 - Section: foo1
## "(index 2)" here is a name that we automatically created for a unnamed section.
 - Section: (index 2)
 - Section: foo3
 - Section: foo4
```

It looks really confusing that we don't have to list all of sections.

At first I've tried to make this rule stricter and report an error when there is a gap
(i.e. when a section is included into segment, but not listed explicitly).
This did not work perfect, because such approach conflicts with unnamed sections/fills (see (3*)).

This patch drops "Sections" key and introduces 2 keys instead: `FirstSec` and `LastSec`.
Both are optional.

Differential revision: https://reviews.llvm.org/D90458
2020-11-09 13:00:50 +03:00
António Afonso 32d239a758 Fix debug_abbrev emitter to only assign table id once
While generating yamls for my tests I noticed that the new debug_abbrev format (with multiple table support) was incorrectly assigning id's to the table because it was generating one per abbrev entry in the table. For instance, the first table would get id 4 when 5 abbrev entries existed in the table. By itself this is not a problem but the corresponding debug_info sections were still referencing id 0. This was introduced here: https://reviews.llvm.org/D83116.

Maybe a better fix is to actually correctly calculate the table id when emitting debug info? From a quick glance it seems to me the ID is just being calculated as the distance between the first DWARFAbbreviationDeclarationSet and the one the debug info entry points to, which means it's just its index and not the actual table id that was generated when emitting the debug_abbrev tables. With my fix I guess this is fine but on the diff that introduced this Pavel mentioned that he would like to have some sort of unique id between them but not necessarily +1 increasing, but for that to work we need to actually find the table ID, I guess by going directly to Y.DebugAbbrev but to honest I have no idea how to link the DWARFAbbreviationDeclarationSet and the Y.DebugAbbrev, so I just did this simple fix.

I also realized there's barely any tests for MachO so it might useful to invest on that if the tool is being reworked on.

Reviewed By: Higuoxing, jhenderson

Differential Revision: https://reviews.llvm.org/D87179
2020-11-08 18:11:50 -08:00
Rahman Lavaee 82e7c4ce45 [obj2yaml] [yaml2obj] Add yaml support for SHT_LLVM_BB_ADDR_MAP section.
YAML support allows us to better test the feature in the subsequent patches. The implementation is quite similar to the .stack_sizes section.

Reviewed By: jhenderson, grimar

Differential Revision: https://reviews.llvm.org/D88717
2020-11-06 12:44:42 -08:00
Georgii Rymar 2bfaf19516 [yaml2obj] - Make `Section::Link` field to be `Optional<>`.
`Link` is not an optional field currently.
Because of this it is not convenient to write macros.

This makes it optional and fixes corresponding test cases.

Differential revision: https://reviews.llvm.org/D90390
2020-10-30 16:18:53 +03:00
Georgii Rymar 47369e194a [yaml2obj][obj2yaml] - Teach tools to work with regular archives.
This teaches obj2yaml to dump valid regular (not thin) archives.
This also teaches yaml2obj to recognize archives YAML descriptions,
what allows to craft all different kinds of archives (valid and broken ones).

Differential revision: https://reviews.llvm.org/D89949
2020-10-28 15:27:11 +03:00
Jonas Devlieghere 97b8e2c1f0 [llvm] Make obj2yaml and yaml2obj LLVM utilities instead of tools
For testing purposes I need a way to build and install FileCheck and
yaml2obj. I had to choose between making FileCheck an LLVM tool and
making obj2yaml and yaml2obj utilities. I think the distinction is
rather arbitrary but my understanding is that tools are things meant for
the toolchain while utilities are more used for things like testing,
which is the case here.

The functional difference is that these tools now end up in the
${LLVM_UTILS_INSTALL_DIR}, which defaults to the ${LLVM_TOOLS_INSTALL_DIR}.
Unless you specified a different value or you added obj2yaml and
yaml2obj to ${LLVM_TOOLCHAIN_TOOLS}, this patch shouldn't change
anything.

Differential revision: https://reviews.llvm.org/D89357
2020-10-19 10:21:21 -07:00
Georgii Rymar 400103f3d5 [yaml2obj/obj2yaml] - Add support of 'Size' and 'Content' keys for all sections.
Many sections either do not have a support of `Size`/`Content` or support just a
one of them, e.g only `Content`.

`Section` is the base class for sections. This patch adds `Content` and `Size` members
to it and removes similar members from derived classes. This allows to cleanup and
generalize the code and adds a support of these keys for all sections (`SHT_MIPS_ABIFLAGS`
is a only exception, it requires unrelated specific changes to be done).

I had to update/add many tests to test the new functionality properly.

Differential revision: https://reviews.llvm.org/D89039
2020-10-15 11:11:41 +03:00
Paulo Matos 388fb67b0d [WebAssembly] Added .tabletype to asm and multiple table support in obj files
Adds more testing in basic-assembly.s and a new test tables.s.
Adds support to yaml reading and writing of tables as well.

Differential Revision: https://reviews.llvm.org/D88815
2020-10-13 07:52:23 -07:00
Xing GUO dd55499472 [DWARFYAML] Make the opcode_base and the standard_opcode_lengths fields optional.
This patch makes the opcode_base and the standard_opcode_lengths fields
of the line table optional. When both of them are not specified,
yaml2obj emits them according to the line table's version.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D88355
2020-10-09 11:10:03 +08:00
Georgii Rymar 82311766d9 [obj2yaml] - Rename `Group` to `GroupSection`. NFC.
The `Group` class represents a group section and it is
named inconsistently with other sections which all has
the "Section" suffix. It is sometimes confusing,
this patch addresses the issue.

Differential revision: https://reviews.llvm.org/D88892
2020-10-07 17:04:15 +03:00
Georgii Rymar 5829dc9250 [yaml2obj][elf2yaml] - Add a support for the `EntSize` field for `SHT_HASH` sections.
Specification  for SHT_HASH table says (https://refspecs.linuxbase.org/elf/gabi4+/ch5.dynamic.html#hash)
that it contains Elf32_Word entries for both 32/64 bit objects.

Currently both GNU linkers and LLD sets the `sh_entsize` field to `4`.

At the same time, `yaml2obj` ignores the `EntSize` field for SHT_HASH sections.
This patch fixes this and also adds a support for obj2yaml: it will not
dump this field when the `sh_entsize` contains the default value (`4`).

Differential revision: https://reviews.llvm.org/D88652
2020-10-02 12:01:50 +03:00
Georgii Rymar dab9917164 [yaml2obj][obj2yaml] - Add a support for SHT_ARM_EXIDX section.
This adds the support for SHT_ARM_EXIDX sections to obj2yaml/yaml2obj tools.

SHT_ARM_EXIDX is a ARM specific index table filled with entries.
Each entry consists of two 4-bytes values (words).
(https://developer.arm.com/documentation/ihi0038/c/?lang=en#index-table-entries)

Differential revision: https://reviews.llvm.org/D88228
2020-09-28 11:45:49 +03:00
Georgii Rymar ea0f66e848 [obj2yaml][yaml2obj] - Stop recognizing SHT_MIPS_ABIFLAGS on non-MIPS targets.
Currently we are always recognizing the `SHT_MIPS_ABIFLAGS` section,
even on non-MIPS targets.

The problem of doing this is briefly discussed in D88228 which does the same for `SHT_ARM_EXIDX`:

"The problem is that `SHT_ARM_EXIDX` shares the value with `SHT_X86_64_UNWIND (0x70000001U)`.
We might have other machine specific conflicts, e.g.
`SHT_ARM_ATTRIBUTES` vs `SHT_MSP430_ATTRIBUTES` vs `SHT_RISCV_ATTRIBUTES (0x70000003U)`."

I think we should only recognize target specific sections when the machine type
matches. I.e. `SHT_MIPS_*` should be recognized only on `MIPS`, `SHT_ARM_*`
only on `ARM` etc.

This patch stops recognizing `SHT_MIPS_ABIFLAGS` on `non-MIPS` targets.

Note: I had to update `ScalarEnumerationTraits<ELFYAML::MIPS_ISA>::enumeration`, because
otherwise test crashes, calling `llvm_unreachable`.

Differential revision: https://reviews.llvm.org/D88294
2020-09-28 11:28:53 +03:00
Xing GUO 505ac22f1a [DWARFYAML] Make the ExtLen field of extended opcodes optional.
This patch makes the 'ExtLen' field of extended opcodes optional. We
don't need to manually calculate it in the future.

Reviewed By: jhenderson, MaskRay

Differential Revision: https://reviews.llvm.org/D88136
2020-09-24 14:13:26 +08:00
Georgii Rymar 279943edf8 [obj2yaml] - Don't emit EM_NONE.
When ELF header's `e_machine == 0`, we emit:

```
Machine: EM_NONE
```

We can avoid doing this, because yaml2obj sets the
`e_machine` field to `EM_NONE` by default.

Differential revision: https://reviews.llvm.org/D87829
2020-09-17 15:58:44 +03:00
Rahman Lavaee b1cb9d6271 [obj2yaml] - Match ".stack_size" with the original section name, and not the uniquified name.
Without this patch, obj2yaml decodes the content of only one ".stack_size" section. Other sections are dumped with their full contents.

Reviewed By: grimar, MaskRay

Differential Revision: https://reviews.llvm.org/D87727
2020-09-16 14:17:29 -07:00
Rahman Lavaee f80f2516a2 Revert "[obj2yaml] - Match ".stack_size" with the original section name, and not the uniquified name."
This reverts commit 14e55f8298.
2020-09-16 11:42:37 -07:00
Rahman Lavaee 14e55f8298 [obj2yaml] - Match ".stack_size" with the original section name, and not the uniquified name.
Without this patch, obj2yaml decodes the content of only one ".stack_size" section. Other sections are dumped with their full contents.

Reviewed By: grimar, MaskRay

Differential Revision: https://reviews.llvm.org/D87727
2020-09-16 11:33:20 -07:00
Xing GUO 4171d5c30a [obj2yaml] Add support for dumping the .debug_addr(v5) section.
This patch adds support for dumping the .debug_addr(v5) section to
obj2yaml.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D87601
2020-09-16 14:48:03 +08:00
Georgii Rymar 4845531fa8 [lib/Object] - Refine interface of ELFFile<ELFT>. NFCI.
`ELFFile<ELFT>` has many methods that take pointers,
though they assume that arguments are never null and
hence could take references instead.

This patch performs such clean-up.

Differential revision: https://reviews.llvm.org/D87385
2020-09-15 11:38:31 +03:00
Xing GUO 0e47a8d17f [obj2yaml] Add support for dumping the .debug_ranges section.
This patch adds support for dumping the .debug_ranges section to
elf2yaml.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D87429
2020-09-11 08:47:01 +08:00
Xing GUO 25f3cc0ced [elf2yaml] Fix dumping a debug section whose name is not recognized.
If the debug section's name isn't recognized, it should be
dumped as a raw content section.

Reviewed By: jhenderson, grimar

Differential Revision: https://reviews.llvm.org/D87346
2020-09-09 20:07:05 +08:00
Xing GUO 25c3fa3f13 [DWARFYAML] Make the debug_ranges section optional.
This patch makes the debug_ranges section optional. When we specify an
empty debug_ranges section, yaml2obj only emits the section header.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D87263
2020-09-08 19:55:47 +08:00
Xing GUO 3cda698723 [obj2yaml] Stop parsing the debug_str section when it encounters a string without the null terminator.
When obj2yaml encounters a string without the null terminator, it should
stop parsing the debug_str section. This patch addresses comments in
[D86867](https://reviews.llvm.org/D86867#inline-803291).

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D87261
2020-09-08 16:09:36 +08:00
Xing GUO 3097427f93 [obj2yaml] Add support for dumping the .debug_str section.
This patch adds support for dumping the .debug_str section to obj2yaml.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D86867
2020-09-07 19:46:56 +08:00
Xing GUO 428b2ffad4 [DWARFYAML] Make the debug_str section optional.
This patch makes the debug_str section optional. When the debug_str
section exists but doesn't contain anything, yaml2obj will emit a
section header for it.

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D86860
2020-09-01 10:02:09 +08:00
Xing GUO 8daa3264a3 [DWARFYAML] Make the unit_length and header_length fields optional.
This patch makes the unit_length and header_length fields of line tables
optional. yaml2obj is able to infer them for us.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D86590
2020-08-26 20:35:10 +08:00
Georgii Rymar 357dc1ed12 [obj2yaml] - Cleanup error reporting (remove Error.cpp/.h files)
This removes Error.cpp/.h files from obj2yaml.
These files are not needed because we are
using `Error`s instead of error codes widely and do
not need a logic related to obj2yaml specific
error codes anymore.

I had to adjust just a few lines of tool's code
to remove remaining dependencies.

Differential revision: https://reviews.llvm.org/D86536
2020-08-26 14:25:11 +03:00
Xing GUO f5643dc3dc Recommit: [DWARFYAML] Add support for referencing different abbrev tables.
The original commit (7ff0ace96db9164dcde232c36cab6519ea4fce8) was causing
build failure and was reverted in 6d242a7326

==================== Original Commit Message ====================
This patch adds support for referencing different abbrev tables. We use
'ID' to distinguish abbrev tables and use 'AbbrevTableID' to explicitly
assign an abbrev table to compilation units.

The syntax is:
```
debug_abbrev:
  - ID: 0
    Table:
      ...
  - ID: 1
    Table:
      ...
debug_info:
  - ...
    AbbrevTableID: 1 ## Reference the second abbrev table.
  - ...
    AbbrevTableID: 0 ## Reference the first abbrev table.
```

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D83116
2020-08-21 19:02:10 +08:00
Xing GUO 6d242a7326 Revert "[DWARFYAML] Add support for referencing different abbrev tables."
This reverts commit f7ff0ace96.

This change is causing build failure.

http://lab.llvm.org:8011/builders/clang-cmake-armv7-global-isel/builds/10400
2020-08-21 12:15:54 +08:00
Xing GUO f7ff0ace96 [DWARFYAML] Add support for referencing different abbrev tables.
This patch adds support for referencing different abbrev tables. We use
'ID' to distinguish abbrev tables and use 'AbbrevTableID' to explicitly
assign an abbrev table to compilation units.

The syntax is:
```
debug_abbrev:
  - ID: 0
    Table:
      ...
  - ID: 1
    Table:
      ...
debug_info:
  - ...
    AbbrevTableID: 1 ## Reference the second abbrev table.
  - ...
    AbbrevTableID: 0 ## Reference the first abbrev table.
```

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D83116
2020-08-21 11:44:25 +08:00
Xing GUO 290e399f96 [DWARFYAML] Add support for emitting multiple abbrev tables.
This patch adds support for emitting multiple abbrev tables. Currently,
compilation units will always reference the first abbrev table.

Reviewed By: jhenderson, labath

Differential Revision: https://reviews.llvm.org/D86194
2020-08-21 10:12:08 +08:00
Georgii Rymar a6436b0b3a [yaml2obj] - Make the 'Machine' key optional.
Currently we have to set 'Machine' to something in our
YAML descriptions. Usually we use 'EM_X86_64' for 64-bit targets
and 'EM_386' for 32-bit targets. At the same time, in fact, in most
cases our tests do not need a machine type and we can use
'EM_NONE'.

This is cleaner, because avoids the need of using a particular machine.

In this patch I've made the 'Machine' key optional (the default value,
when it is not specified is `EM_NONE`) and removed it (where possible)
from yaml2obj, obj2yaml and llvm-readobj tests.

There are few tests left where I decided not to remove it, because
I didn't want to touch CHECK lines or doing anything more complex
than a removing a "Machine: *" line and formatting lines around.

Differential revision: https://reviews.llvm.org/D86202
2020-08-20 11:40:51 +03:00
Simon Pilgrim 90a1e6509f Fix MSVC implicit truncation narrowing conversion warning. 2020-08-19 14:41:40 +01:00
Xing GUO 419326a445 [obj2yaml] Refactor the .debug_pub* sections dumper.
It's good to reuse the DWARF parser in lib/DebugInfo so that we don't
need to maintain a separate parser in client side (obj2yaml). Besides,
A test case is added whose length field is a very huge value which makes
obj2yaml stuck when parsing the section.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D86192
2020-08-19 21:13:52 +08:00
Xing GUO b7d5d1ec64 [DWARFYAML] Replace InitialLength with Format and Length. NFC.
This change replaces the InitialLength of pub-tables with Format and
Length. All the InitialLength fields have been removed.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D85880
2020-08-13 18:39:03 +08:00
Xing GUO 449b4d2a8b [macho2yaml] Remove an unused variable. NFC. 2020-08-13 11:14:31 +08:00
Xing GUO 3c5758964c [macho2yaml] Refactor the DWARF section dumpers.
This patch refactors the DWARF section dumpers. When dumping a DWARF
section, if the DWARF parser fails to parse the section, we will dump it
as a raw content section. This patch also fixes a bug in
DWARFYAML::Data::isEmpty(). Finally, a test case that tests dumping the
__debug_aranges section is added.

Reviewed By: jhenderson, grimar

Differential Revision: https://reviews.llvm.org/D85506
2020-08-11 10:18:34 +08:00
James Henderson ca05601cd2 [DebugInfo] Don't error for zero-length arange entries
Although the DWARF specification states that .debug_aranges entries
can't have length zero, these can occur in the wild. There's no
particular reason to enforce this part of the spec, since functionally
they have no impact. The patch removes the error and introduces a new
warning for premature terminator entries which does not stop parsing.

This is a relanding of cb3a598c87, adding the missing obj2yaml part
that was needed.

Fixes https://bugs.llvm.org/show_bug.cgi?id=46805. See also
https://reviews.llvm.org/D71932 which originally introduced the error.

Reviewed by: ikudrin, dblaikie, Higuoxing

Differential Revision: https://reviews.llvm.org/D85313
2020-08-10 14:57:52 +01:00
Xing GUO 100e1f911c [macho2yaml] Remove unused functions. NFC.
dumpDebugStrings() and dumpDebugAbbrev() are no longer used in
macho2yaml. This patch helps remove them.

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D85496
2020-08-07 20:34:15 +08:00
Xing GUO 4357986b41 [DWARFYAML][debug_info] Pull out dwarf::FormParams from DWARFYAML::Unit.
Unit.Format, Unit.Version and Unit.AddrSize are replaced with
dwarf::FormParams in D84496 to get rid of unnecessary functions
getOffsetSize() and getRefSize(). However, that change makes it
difficult to make AddrSize optional (Optional<uint8_t>). This change
pulls out dwarf::FormParams from DWARFYAML::Unit and use it as a helper
struct in DWARFYAML::emitDebugInfo().

Reviewed By: jhenderson, MaskRay

Differential Revision: https://reviews.llvm.org/D85296
2020-08-06 16:39:00 +08:00
Xing GUO bd7f3f8a3e [obj2yaml] Add support for dumping the .debug_aranges section.
This patch adds support for dumping DWARF sections to obj2yaml. The
.debug_aranges section is used to illustrate the basic idea.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D85094
2020-08-05 19:19:05 +08:00
Xing GUO 74b02d73e3 [DWARFYAML] Make the debug_aranges entry optional.
This patch makes the 'debug_aranges' entry optional. If the entry is
empty, yaml2obj will only emit the header for it.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D84921
2020-07-31 20:18:53 +08:00
Xing GUO bbb057c49a [DWARFYAML] Replace 'Format', 'Version', etc with 'FormParams'. NFC.
This patch replaces 'Format', 'Version' fields, etc with 'FormParams' to
simplify codes.

Reviewed By: labath

Differential Revision: https://reviews.llvm.org/D84496
2020-07-24 16:54:51 +08:00
Georgii Rymar 7a587ca932 [yaml2obj] - Rename FileHeader::SH* fields.
In D83482 we agreed to name e_* fields that are used for overriding
values (like e_phoff) as EPh* (e.g. EPhOff).

Currently we have a set of e_sh* fields that are named inconsistently
with this rule. This patch renames all of them.

Differential revision: https://reviews.llvm.org/D83766
2020-07-15 12:47:31 +03:00
Georgii Rymar 3001569323 [yaml2obj] - Add a syntax to override e_phoff, e_phentsize and e_phnum fields.
This adds `EPhOff`, `EPhEntSize` and `EPhNum` keys.
Will be useful for creating broken objects for testing llvm-readelf.

Differential revision: https://reviews.llvm.org/D83482
2020-07-14 13:16:19 +03:00
Xing GUO b6f08b7cfd [DWARFYAML][debug_gnu_*] Add the missing context `IsGNUStyle`. NFC.
This patch helps add the missing context `IsGNUStyle`. Before this patch, yaml2obj cannot parse the YAML description of 'debug_gnu_pubnames' and 'debug_gnu_pubtypes' correctly due to the missing context.

In other words, if we have

```
DWARF:
  debug_gnu_pubtypes:
    Length:
      TotalLength: 0x1234
    Version:    2
    UnitOffset: 0x1234
    UnitSize:   0x4321
    Entries:
      - DieOffset:  0x12345678
        Name:       abc
        Descriptor: 0x00      ## Descriptor can never be mapped into Entry.Descriptor
```

yaml2obj will complain that "error: unknown key 'Descriptor'".

This patch helps resolve this problem.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D82435
2020-07-03 18:12:58 +08:00
Xing GUO fe08ab542b [DWARFYAML][debug_info] Replace 'InitialLength' with 'Format' and 'Length'.
'InitialLength' is replaced with 'Format' (DWARF32 by default) and 'Length' in this patch.
Besides, test cases for DWARFv4 and DWARFv5, DWARF32 and DWARF64 is
added.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D82622
2020-06-30 16:28:39 +08:00
Simon Pilgrim 6551b7a9d8 Add explicit Twine.h include to try and fix ICE on clang-ppc64be-linux 2020-06-26 14:14:09 +01:00
Xing GUO 0431e4bcb2 Recommit "[DWARFYAML][debug_line] Replace `InitialLength` with `Format` and `Length`."
This recommits fcc0c186e9
2020-06-13 23:39:11 +08:00
Xing GUO 325f7607b0 Revert "[DWARFYAML][debug_line] Replace `InitialLength` with `Format` and `Length`."
This reverts commit fcc0c186e9.
2020-06-13 17:57:02 +08:00
Xing GUO fcc0c186e9 [DWARFYAML][debug_line] Replace `InitialLength` with `Format` and `Length`. 2020-06-13 17:47:06 +08:00
Reid Kleckner 1c03389c29 Re-land "Migrate the rest of COFFObjectFile to Error"
This reverts commit 101fbc0138.

Remove leftover debugging attribute.

Update LLDB as well, which was missed before.
2020-06-11 14:46:16 -07:00
Nico Weber 101fbc0138 Revert "Migrate the rest of COFFObjectFile to Error"
This reverts commit b5289656b8.
__attribute__((optnone)) doesn't build with msvc, see
http://lab.llvm.org:8011/builders/clang-x64-windows-msvc/builds/16326
2020-06-05 21:20:11 -04:00
Reid Kleckner b5289656b8 Migrate the rest of COFFObjectFile to Error 2020-06-05 16:29:05 -07:00
Xing GUO 929edd8bd2 [DWARFYAML][debug_aranges] Replace InitialLength with Format and Length.
This patch addresses the comment in [D80972](https://reviews.llvm.org/D80972#inline-744217).

Before this patch, the initial length field of .debug_aranges section should be declared as:

```
## 32-bit DWARF
debug_aranges:
  - Length:
      TotalLength: 0x20
    Version: 2
    ...

## 64-bit DWARF
debug_aranges:
  - Length:
      TotalLength:   0xffffffff
      TotalLength64: 0x20
    Version: 2
    ...
```

After this patch:

```
## 32-bit DWARF
debug_aranges:
  - [[Format:  DWARF32]] ## Optional
    Length:  0x20
    Version: 2
    ...

## 64-bit DWARF
debug_aranges:
  - Format:  DWARF64
    Length:  0x20
    Version: 2
```

Current implementation of generating DWARF64 .debug_aranges section is buggy. A follow-up patch will improve it and add test cases for DWARF64.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D81063
2020-06-05 12:16:44 +08:00
Xing GUO d3f49b8d37 [ObjectYAML][DWARF] Let `dumpPubSection` return `DWARFYAML::PubSection`.
Summary: This patch addresses comments in [D80722](https://reviews.llvm.org/D80722#inline-742353)

Reviewers: grimar, jhenderson

Reviewed By: grimar, jhenderson

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80861
2020-06-02 14:38:26 +08:00
Xing GUO ea7db621d2 [ObjectYAML][DWARF] Make the `PubSection` optional.
This patch helps make the `PubSection` optional in the DWARF structure.

Reviewed By: jhenderson, aprantl

Differential Revision: https://reviews.llvm.org/D80722
2020-05-29 20:11:53 +08:00
Georgii Rymar e2b134b01a [yaml2obj] - Stop using square brackets for unique suffixes.
For describing section/symbol names we can use unique suffixes,
e.g:

```
- Name: '.foo [1]`
- Name: '.foo [2]`
```

It can be a problem (see https://reviews.llvm.org/D79984#inline-734829),
because `[]` are sometimes used to describe a macros:

```
- Name: "[[a0]]"
```

Seems the better approach is to use something else, like "()".
This patch does it and refactors the code related.

Differential revision: https://reviews.llvm.org/D80123
2020-05-19 12:59:13 +03:00
Greg Clayton 6e73f12a64 Fix buildbots errors after comitting D78782.
Rename "Ranges" variables to "DebugRanges" to avoid warnings/errors on machines that have extra settings enabled.

https://reviews.llvm.org/D78782
2020-05-13 22:01:57 -07:00
Greg Clayton 6025fc2243 Add .debug_ranges support to the DWARF YAML.
Summary: This allows DIEs with DW_AT_ranges to be encoded and decoded _and_ actually have their address ranges be included instead of having DW_AT_ranges with a section offset value for a section that doesn't exist.

Reviewers: labath, aprantl, JDevlieghere, dblaikie, probinson

Subscribers: hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D78782
2020-05-13 16:21:45 -07:00
Reid Kleckner 77ecf90c52 [COFF] Migrate COFFObjectFile to Expected<T>
I noticed that std::error_code() does one-time initialization. Avoid
that overhead with Expected<T> and llvm::Error. Also, it is consistent
with the virtual interface and ELF, and generally cleaner.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D79643
2020-05-08 14:01:39 -07:00
Alexander Shaposhnikov c19c3293d3 [ObjectYAML][MachO] Add support for relocations
Add support for relocations for MachO to ObjectYAML / yaml2obj / obj2yaml.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D77844
2020-04-22 11:50:55 -07:00
Georgii Rymar 317c4913c6 [obj2yaml] - Fix the issue with dumping empty sections when dumping program headers.
Imagine we have:

```
ProgramHeaders:
  - Type:  PT_LOAD
    Flags: [ PF_W, PF_R ]
    Sections:
      - Section: .bar
    VAddr: 0x2000
Sections:
  - Name:    .foo
    Type:    SHT_PROGBITS
    Flags:   [ SHF_ALLOC, SHF_EXECINSTR ]
    Address: 0x1000
  - Name:    .bar
    Type:    SHT_PROGBITS
    Flags:   [ SHF_ALLOC, SHF_EXECINSTR ]
    Address: 0x2000
```

Both `.foo` and `.bar` share the same starting file offset,
but `VA(.foo)` < `VA(PT_LOAD)`, we should not include it into segment.

This patch fixes the issue.

Differential revision: https://reviews.llvm.org/D77652
2020-04-22 12:36:00 +03:00
Georgii Rymar 1c57752ff5 [obj2yaml] - Teach obj2yaml to dump SHT_NOBITS sections when dumping program headers.
SHT_NOBITS are a bit special because occupy no physical space.
This patch adds support for them.

Differential revision: https://reviews.llvm.org/D77805
2020-04-20 14:35:28 +03:00
Georgii Rymar 56a8150428 [obj2yaml] - Do not dump the segment's "Align" field when it is equal to 1.
yaml2obj sets the `Align` to 1 by default, hence we can stop
dumping it to reduce the output.

Differential revision: https://reviews.llvm.org/D77716
2020-04-09 14:20:36 +03:00
Georgii Rymar b3f13bc165 [obj2yaml] - Teach tool to dump program headers.
Currently obj2yaml does not dump program headers,
this patch teaches it to do that.

Differential revision: https://reviews.llvm.org/D75342
2020-03-31 18:10:19 +03:00
Georgii Rymar aefec9ed77 [obj2yaml] - Refactor how we dump sections. NFCI.
This is a NFC splitted from D75342.

Previously obj2yaml never dumped a normal SHT_NULL section (i.e. when it is just zeroed)
or non-allocatable SHT_STRTAB/SHT_SYMTAB/SHT_DYNSYM sections.

This patch does not change the output, but it changes the logic so that we now dump these
sections, and them remove them later. It allows us to create and work with our internal representation
of sections, i.e. to work with the vector of Chunks, what looks cleaner.

It is used by D75342 and also should help us to support dumping a content that does not
belong to a section (i.e. to dump some data as `Fill` chunks).

Differential revision: https://reviews.llvm.org/D76684
2020-03-26 14:04:07 +03:00
Heejin Ahn f93426c5b9 [WebAssembly] Move event section before global section
Summary:
https://github.com/WebAssembly/exception-handling/issues/98

Also this moves many parts of code to make code align with the section
order, even if they don't affect the output.

Reviewers: tlively

Subscribers: dschuff, sbc100, hiraditya, sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D76752
2020-03-25 11:49:03 -07:00
Georgii Rymar 601d25cb73 [obj2yaml] - Simplify and reduce `ELFDumper<ELFT>::dumpSections`. NFCI.
This method it a bit too large.
It is becoming inconvenient to update it.
This patch suggests a way to reduce and cleanup it.

Differential revision: https://reviews.llvm.org/D76499
2020-03-21 18:15:26 +03:00
Georgii Rymar a02b38698b [obj2yaml] - SHT_DYNAMIC and SHT_REL* sections: stop dumping sh_entsize field when it has the default value.
Currently obj2yaml always emits the `EntSize` property when `sh_entsize != 0`.
It is not correct. For example, for `SHT_DYNAMIC` section, `EntSize == 0`
is abnormal, while `sizeof(ELFT::Dyn)` is the expected default.

To reduce the output produces we should not dump default values.

yaml2obj tests that shows `sh_entsize` values produced are:
1) For `SHT_REL*` sections: `yaml2obj\ELF\reloc-sec-entry-size.yaml`
2) For `SHT_DYNAMIC`: `yaml2obj\ELF\dynamic-section.yaml`

Differential revision: https://reviews.llvm.org/D76227
2020-03-19 17:25:53 +03:00
Georgii Rymar e26e9ba288 [obj2yaml] - Stop dumping an empty sh_info field for SHT_RELA/SHT_REL sections.
`.rela.dyn` is a dynamic relocation section that normally has
no value in `sh_info` field.

The existent `elf-reladyn-section-shinfo.yaml` which tests this piece has issues:

1) It does not check the case when we have more than one `SHT_REL[A]`
   section with `sh_info == 0` in the object. Because of this it did not catch the issue.
   Currently we print an excessive "Info" field:

```
  - Name:            .rela.dyn
    Type:            SHT_RELA
    EntSize:         0x0000000000000018
  - Name:            .rel.dyn
    Type:            SHT_REL
    EntSize:         0x0000000000000010
    Info:            ' [1]'
```

2) It seems can be more generic. I've added a `rel-rela-section.yaml` instead.

Differential revision: https://reviews.llvm.org/D76281
2020-03-19 14:00:21 +03:00
Pavel Labath eb2b17eea7 Use DWARFDataExtractor::getInitialLength in debug_aranges
Summary:
getInitialLength is a *DWARF*DataExtractor method so I had to "upgrade"
some DataExtractors to be able to make use of it.

Reviewers: ikudrin, jhenderson, probinson

Subscribers: aprantl, hiraditya, llvm-commits, dblaikie

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D75535
2020-03-04 13:01:07 +01:00
Georgii Rymar d58e383f23 [obj2yaml] - Dump allocatable SHT_STRTAB, SHT_SYMTAB and SHT_DYNSYM sections.
Sometimes we need to dump an object and build it again from a YAML
description produced. The problem is that obj2yaml does not dump some
of sections, like string tables and symbol tables.

Because of that yaml2obj implicitly creates them and sections created
are not placed at their original locations. They are added to the end of a section list.
That makes a preparing test cases task harder than it can be.

This patch teaches obj2yaml to dump parts of allocatable SHT_STRTAB, SHT_SYMTAB
and SHT_DYNSYM sections to print placeholders for them.
This also allows to preserve usefull parameters, like virtual address.

Differential revision: https://reviews.llvm.org/D74955
2020-03-03 11:32:49 +03:00
Georgii Rymar e93f7e6d1a [obj2yaml] - Split sections dumping to a new ELFDumper<ELFT>::dumpSections() method.
ELFDumper<ELFT>::dump() is too large and deserves splitting.

Differential revision: https://reviews.llvm.org/D75172
2020-03-03 11:11:21 +03:00
Georgii Rymar 31f2ad9c36 [yaml2obj] - Automatically assign sh_addr for allocatable sections.
I've noticed that it is not convenient to create YAMLs from
binaries (using obj2yaml) that have to be test cases for obj2yaml
later (after applying yaml2obj).

The problem, for example is that obj2yaml emits "DynamicSymbols:"
key instead of .dynsym. It also does not create .dynstr.
And when a YAML document without explicitly defined .dynsym/.dynstr
is given to yaml2obj, we have issues:

1) These sections are placed after non-allocatable sections (I've fixed it in D74756).
2) They have VA == 0. User needs create descriptions for such sections explicitly manually
    to set a VA.

This patch addresses (2). I suggest to let yaml2obj assign virtual addresses by itself.
It makes an output binary to be much closer to "normal" ELF.
(It is still possible to use "Address: 0x0" for a section to get the original behavior
if it is needed)

Differential revision: https://reviews.llvm.org/D74764
2020-02-22 14:43:54 +03:00
Georgii Rymar b7c8a8e834 [obj2yaml] - Fix a -Wsign-compare warning gived by GCC 9.2
I was reported that with commit:
https://github.com/llvm/llvm-project/commit/d3963051c490

gcc-9.2 is giving the warning below.
This should help (I have no gcc 9.2 to test).

[ 57%] Building CXX object tools/obj2yaml/CMakeFiles/obj2yaml.dir/elf2yaml.cpp.o
/llvm/tools/obj2yaml/elf2yaml.cpp: In instantiation of ‘llvm::Expected<llvm::ELFYAML::Object*>
{anonymous}::ELFDumper<ELFT>::dump() [with ELFT = llvm::object::ELFType<llvm::support::little, false>]’:
/llvm/tools/obj2yaml/elf2yaml.cpp:1218:31:   required from ‘llvm::Error elf2yaml(llvm::raw_ostream&,
const llvm::object::ELFFile<ELFT>&) [with ELFT = llvm::object::ELFType<llvm::support::little, false>]’
/llvm/tools/obj2yaml/elf2yaml.cpp:1231:47:   required from here
/llvm/tools/obj2yaml/elf2yaml.cpp:207:41: warning: comparison of integer expressions of different
signedness: ‘llvm::support::detail::packed_endian_specific_integral<unsigned int, llvm::support::little, 1>::value_type’ {aka ‘unsigned int’} and ‘int’ [-Wsign-compare]
  207 |     if (!SymTab || SymTabShndx->sh_link != SymTab - Sections.begin())
/llvm/tools/obj2yaml/elf2yaml.cpp: In instantiation of ‘llvm::Expected<llvm::ELFYAML::Object*>
{anonymous}::ELFDumper<ELFT>::dump() [with ELFT = llvm::object::ELFType<llvm::support::big, false>]’:
...
2020-02-15 18:35:57 +03:00
Bill Wendling c55cf4afa9 Revert "Remove redundant "std::move"s in return statements"
The build failed with

  error: call to deleted constructor of 'llvm::Error'

errors.

This reverts commit 1c2241a793.
2020-02-10 07:07:40 -08:00
Bill Wendling 1c2241a793 Remove redundant "std::move"s in return statements 2020-02-10 06:39:44 -08:00
Georgii Rymar 60f161eb62 [yaml2obj][obj2yaml] - Simplify format of the SHT_LLVM_ADDRSIG section.
Previously the description allowed to describe symbols with use of
`Name` and `Index` keys. This patch removes them and now it is still
possible to use either names or symbol indexes, but the code is simpler
and the format is slightly different.

Such a change will be useful for another patches, e.g:
https://reviews.llvm.org/D73788#inline-671077

Differential revision: https://reviews.llvm.org/D73888
2020-02-05 12:33:14 +03:00
Georgii Rymar bec54e464e [yaml2obj/obj2yaml] - Add support for the SHT_LLVM_CALL_GRAPH_PROFILE section.
This is a LLVM specific section that is well described here:
https://llvm.org/docs/Extensions.html#sht-llvm-call-graph-profile-section-call-graph-profile

This patch teaches yaml2obj and obj2yaml about how to work with it.

Differential revision: https://reviews.llvm.org/D73788
2020-02-04 15:13:20 +03:00
Benjamin Kramer adcd026838 Make llvm::StringRef to std::string conversions explicit.
This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.

This doesn't actually modify StringRef yet, I'll do that in a follow-up.
2020-01-28 23:25:25 +01:00
Igor Kudrin dcff3961c2 [DWARF] Return Error from DWARFDebugArangeSet::extract().
This helps to detect and report parsing errors better.
The patch follows the ideas of LLDB's patches D59370 and D59381.

It adds tests for valid and some invalid cases. More checks and
tests to come. Note that the patch fixes validation of the Length
field because the value does not include the field itself.

The existing users are updated to show the error messages.

Differential Revision: https://reviews.llvm.org/D71875
2020-01-23 12:41:05 +07:00
Georgii Rymar 7570d387c2 [yaml2obj/obj2yaml] - Add support for SHT_RELR sections.
Note: this is a reland with a trivial 2 lines fix in ELFState<ELFT>::writeSectionContent.
      It adds a check similar to ones we already have for other sections to fix the case revealed
      by bots, like http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-ubuntu-fast/builds/60744.

The encoded sequence of Elf*_Relr entries in a SHT_RELR section looks
like [ AAAAAAAA BBBBBBB1 BBBBBBB1 ... AAAAAAAA BBBBBB1 ... ]
i.e. start with an address, followed by any number of bitmaps. The address
entry encodes 1 relocation. The subsequent bitmap entries encode up to 63(31)
relocations each, at subsequent offsets following the last address entry.

More information is here:
https://github.com/llvm-mirror/llvm/blob/master/lib/Object/ELF.cpp#L272

This patch adds a support for these sections.

Differential revision: https://reviews.llvm.org/D71872
2020-01-15 15:15:24 +03:00
Georgii Rymar ca6f616532 Revert "[yaml2obj/obj2yaml] - Add support for SHT_RELR sections."
This reverts commit 46d11e30ee.

It broke bots. E.g. http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-ubuntu-fast/builds/60744
2020-01-15 14:19:00 +03:00
Georgii Rymar 46d11e30ee [yaml2obj/obj2yaml] - Add support for SHT_RELR sections.
The encoded sequence of Elf*_Relr entries in a SHT_RELR section looks
like [ AAAAAAAA BBBBBBB1 BBBBBBB1 ... AAAAAAAA BBBBBB1 ... ]
i.e. start with an address, followed by any number of bitmaps. The address
entry encodes 1 relocation. The subsequent bitmap entries encode up to 63(31)
relocations each, at subsequent offsets following the last address entry.

More information is here:
https://github.com/llvm-mirror/llvm/blob/master/lib/Object/ELF.cpp#L272

This patch adds a support for these sections.

Differential revision: https://reviews.llvm.org/D71872
2020-01-15 13:54:08 +03:00
Georgii Rymar daff7b8589 [yaml2obj] - Make DynamicSymbols to be Optional<> too.
We already have Symbols property to list regular symbols and
it is currently Optional<>. This patch makes DynamicSymbols to be optional
too. With this there is no need to define a dummy symbol anymore to trigger
creation of the .dynsym and it is now possible to define an empty .dynsym using
just the following line:

DynamicSymbols: []
(it is important to have when you do not want to have dynamic symbols,
but want to have a .dynsym)

Now the code is consistent and it helped to fix a bug: previously we
did not report an error when both Content/Size and an empty
Symbols/DynamicSymbols list were specified.

Differential revision: https://reviews.llvm.org/D70956
2019-12-04 13:12:02 +03:00
Georgii Rymar 13cbcf1c1a [yaml2obj] - Add a way to describe content of the SHT_GNU_verneed section with "Content".
There is no way to set raw content for SHT_GNU_verneed section.
This patch implements it.

Differential revision: https://reviews.llvm.org/D70816
2019-11-29 10:50:00 +03:00
Georgii Rymar f69ac55d60 [yaml2obj] - Teach tool to describe SHT_GNU_verdef section with a "Content" property.
There is no way to set raw content for SHT_GNU_verdef section.
This patch implements it.

Differential revision: https://reviews.llvm.org/D70710
2019-11-26 15:35:05 +03:00
Georgii Rymar 9659464d7e [yaml2obj/obj2yaml] - Add support for SHT_LLVM_DEPENDENT_LIBRARIES sections.
This section contains strings specifying libraries to be added to the link by the linker.
The strings are encoded as standard null-terminated UTF-8 strings.

This patch adds a way to describe and dump SHT_LLVM_DEPENDENT_LIBRARIES sections.

I introduced a new YAMLFlowString type here. That used to teach obj2yaml to dump
them like:

```
Libraries: [ foo, bar ]
```

instead of the following (if StringRef would be used):

```
Libraries:
  - foo
  - bar
```

Differential revision: https://reviews.llvm.org/D70598
2019-11-25 12:57:53 +03:00