Commit Graph

96 Commits

Author SHA1 Message Date
Xing GUO f634395795 [NFC] mv llvm/test/tools/obj2yaml/macho-DWARF-debug-ranges.yaml llvm/test/ObjectYAML/MachO/DWARF-debug_ranges.yaml 2020-06-14 16:39:15 +08:00
Xing GUO ff9c1ae213 [ObjectYAML][DWARF] Let the target address size be inferred from FileHeader.
This patch adds a new field `bool Is64bit` in `DWARFYAML::Data` to indicate the address size of target. It's helpful for inferring the `AddrSize` in some DWARF sections.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D81709
2020-06-14 12:42:20 +08:00
Georgii Rymar 818ab3d654 [yaml2obj] - Allocate the file space for SHT_NOBITS sections in some cases.
This teaches yaml2obj to allocate file space for a no-bits section
when there is a non-nobits section in the same segment that follows it.

It was discussed in D78005 thread and matches GNU linkers and LLD behavior.

Differential revision: https://reviews.llvm.org/D80629
2020-06-11 12:54:53 +03:00
Xing GUO 929edd8bd2 [DWARFYAML][debug_aranges] Replace InitialLength with Format and Length.
This patch addresses the comment in [D80972](https://reviews.llvm.org/D80972#inline-744217).

Before this patch, the initial length field of .debug_aranges section should be declared as:

```
## 32-bit DWARF
debug_aranges:
  - Length:
      TotalLength: 0x20
    Version: 2
    ...

## 64-bit DWARF
debug_aranges:
  - Length:
      TotalLength:   0xffffffff
      TotalLength64: 0x20
    Version: 2
    ...
```

After this patch:

```
## 32-bit DWARF
debug_aranges:
  - [[Format:  DWARF32]] ## Optional
    Length:  0x20
    Version: 2
    ...

## 64-bit DWARF
debug_aranges:
  - Format:  DWARF64
    Length:  0x20
    Version: 2
```

Current implementation of generating DWARF64 .debug_aranges section is buggy. A follow-up patch will improve it and add test cases for DWARF64.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D81063
2020-06-05 12:16:44 +08:00
Georgii Rymar e2b134b01a [yaml2obj] - Stop using square brackets for unique suffixes.
For describing section/symbol names we can use unique suffixes,
e.g:

```
- Name: '.foo [1]`
- Name: '.foo [2]`
```

It can be a problem (see https://reviews.llvm.org/D79984#inline-734829),
because `[]` are sometimes used to describe a macros:

```
- Name: "[[a0]]"
```

Seems the better approach is to use something else, like "()".
This patch does it and refactors the code related.

Differential revision: https://reviews.llvm.org/D80123
2020-05-19 12:59:13 +03:00
Greg Clayton 6025fc2243 Add .debug_ranges support to the DWARF YAML.
Summary: This allows DIEs with DW_AT_ranges to be encoded and decoded _and_ actually have their address ranges be included instead of having DW_AT_ranges with a section offset value for a section that doesn't exist.

Reviewers: labath, aprantl, JDevlieghere, dblaikie, probinson

Subscribers: hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D78782
2020-05-13 16:21:45 -07:00
Fangrui Song bd3058e82d [obj2yaml][test] Move tests to binary format specific subdirectories
Similar to D70264.
Create COFF/, ELF/, MachO/, Minidump/, XCOFF/, and move tests there.

Reviewed By: grimar, jhenderson, labath

Differential Revision: https://reviews.llvm.org/D79154
2020-04-30 08:42:11 -07:00
Georgii Rymar 317c4913c6 [obj2yaml] - Fix the issue with dumping empty sections when dumping program headers.
Imagine we have:

```
ProgramHeaders:
  - Type:  PT_LOAD
    Flags: [ PF_W, PF_R ]
    Sections:
      - Section: .bar
    VAddr: 0x2000
Sections:
  - Name:    .foo
    Type:    SHT_PROGBITS
    Flags:   [ SHF_ALLOC, SHF_EXECINSTR ]
    Address: 0x1000
  - Name:    .bar
    Type:    SHT_PROGBITS
    Flags:   [ SHF_ALLOC, SHF_EXECINSTR ]
    Address: 0x2000
```

Both `.foo` and `.bar` share the same starting file offset,
but `VA(.foo)` < `VA(PT_LOAD)`, we should not include it into segment.

This patch fixes the issue.

Differential revision: https://reviews.llvm.org/D77652
2020-04-22 12:36:00 +03:00
Georgii Rymar 3471ae9dad [yaml2obj] - Verify that sections are sorted by their file offsets when creating segments.
This validates that sections listed for a segment in the YAML
declaration are ordered by their file offsets.

It might help to simplify the file size computation, but also
is useful by itself as helps to avoid issues in test cases and
to maintain their readability.

Differential revision: https://reviews.llvm.org/D78361
2020-04-21 15:50:42 +03:00
Georgii Rymar 1c57752ff5 [obj2yaml] - Teach obj2yaml to dump SHT_NOBITS sections when dumping program headers.
SHT_NOBITS are a bit special because occupy no physical space.
This patch adds support for them.

Differential revision: https://reviews.llvm.org/D77805
2020-04-20 14:35:28 +03:00
Georgii Rymar 56a8150428 [obj2yaml] - Do not dump the segment's "Align" field when it is equal to 1.
yaml2obj sets the `Align` to 1 by default, hence we can stop
dumping it to reduce the output.

Differential revision: https://reviews.llvm.org/D77716
2020-04-09 14:20:36 +03:00
Georgii Rymar b3f13bc165 [obj2yaml] - Teach tool to dump program headers.
Currently obj2yaml does not dump program headers,
this patch teaches it to do that.

Differential revision: https://reviews.llvm.org/D75342
2020-03-31 18:10:19 +03:00
Georgii Rymar fecce903db [obj2yaml][test] - Update test after output change.
D76227 changed the output. This test was forgotten because
belonged to a different patch.
2020-03-19 17:42:36 +03:00
Georgii Rymar a02b38698b [obj2yaml] - SHT_DYNAMIC and SHT_REL* sections: stop dumping sh_entsize field when it has the default value.
Currently obj2yaml always emits the `EntSize` property when `sh_entsize != 0`.
It is not correct. For example, for `SHT_DYNAMIC` section, `EntSize == 0`
is abnormal, while `sizeof(ELFT::Dyn)` is the expected default.

To reduce the output produces we should not dump default values.

yaml2obj tests that shows `sh_entsize` values produced are:
1) For `SHT_REL*` sections: `yaml2obj\ELF\reloc-sec-entry-size.yaml`
2) For `SHT_DYNAMIC`: `yaml2obj\ELF\dynamic-section.yaml`

Differential revision: https://reviews.llvm.org/D76227
2020-03-19 17:25:53 +03:00
Georgii Rymar 9c69cc109b [obj2yaml] - SHT_REL*, SHT_DYNAMIC sections: add tests to document the behavior when sh_entsize is broken.
We do not have tests that shows the current behavior.
It is needed for D76227 which changes the logic of dumping of `EntSize` fields.

Differential revision: https://reviews.llvm.org/D76282
2020-03-19 16:43:40 +03:00
Georgii Rymar e26e9ba288 [obj2yaml] - Stop dumping an empty sh_info field for SHT_RELA/SHT_REL sections.
`.rela.dyn` is a dynamic relocation section that normally has
no value in `sh_info` field.

The existent `elf-reladyn-section-shinfo.yaml` which tests this piece has issues:

1) It does not check the case when we have more than one `SHT_REL[A]`
   section with `sh_info == 0` in the object. Because of this it did not catch the issue.
   Currently we print an excessive "Info" field:

```
  - Name:            .rela.dyn
    Type:            SHT_RELA
    EntSize:         0x0000000000000018
  - Name:            .rel.dyn
    Type:            SHT_REL
    EntSize:         0x0000000000000010
    Info:            ' [1]'
```

2) It seems can be more generic. I've added a `rel-rela-section.yaml` instead.

Differential revision: https://reviews.llvm.org/D76281
2020-03-19 14:00:21 +03:00
Georgii Rymar 0095200035 [obj2yaml][test] - Remove excessive missing_symtab.test test.
This test uses a precompiled object and duplicates
the functionality of a modern elf-no-symtab.yaml test that
uses yaml2obj for producing inputs.

Differential revision: https://reviews.llvm.org/D76217
2020-03-17 12:42:52 +03:00
Georgii Rymar 2005c60a6b [obj2yaml][test] - Simplify call-graph-profile-section.yaml. NFCI.
Use yaml2obj -D to simplify.
We started to use it for other tests recently.
2020-03-16 11:47:10 +03:00
Georgii Rymar 6f3de2e53d [yaml2obj][obj2yaml][test] - Add base tests for relocation addends.
We had no test for `Addend` field of a relocation. Though the
current behavior is not ideal and might need to be fixed.

This patch adds 2 test cases to document the current
behavior and add a few FIXMEs. These FIXME are fixed in the
follow-up: https://reviews.llvm.org/D75527

Differential revision: https://reviews.llvm.org/D75528
2020-03-13 13:07:46 +03:00
Georgii Rymar e4ceb8f421 [lib/ObjectYAML] - Make `ELFYAML::Relocation::Offset` optional.
Currently `yaml2obj` require `Offset` field in a relocation description.
There are many cases when `Offset` is insignificant in a context of a test case.

Making `Offset` optional allows to simplify our test cases.
This is what this patch does.

Also, with this patch `obj2yaml` does not dump a zero offset of a relocation.

Differential revision: https://reviews.llvm.org/D75608
2020-03-06 13:59:58 +03:00
Georgii Rymar 7391885d5c [yaml2obj][obj2yaml][Object][test] - Improve testing of relocation types.
The intention was to remove the `Object/X86/yaml-elf-x86-rel-broken.yaml test`,
This test is at the wrong place.

`yaml-elf-x86-rel-broken.yaml` was introduced in
rG892c6c86ea25dc97668ff1f1b7bf1108e85fa5ec to check that
yaml2obj can use an arbitrary `Hex32` value as a relocation type.
We have tests that check the similar functionality.

I've improved them and removed the `yaml-elf-x86-rel-broken.yaml`

Differential revision: https://reviews.llvm.org/D75679
2020-03-06 13:38:01 +03:00
Georgii Rymar d58e383f23 [obj2yaml] - Dump allocatable SHT_STRTAB, SHT_SYMTAB and SHT_DYNSYM sections.
Sometimes we need to dump an object and build it again from a YAML
description produced. The problem is that obj2yaml does not dump some
of sections, like string tables and symbol tables.

Because of that yaml2obj implicitly creates them and sections created
are not placed at their original locations. They are added to the end of a section list.
That makes a preparing test cases task harder than it can be.

This patch teaches obj2yaml to dump parts of allocatable SHT_STRTAB, SHT_SYMTAB
and SHT_DYNSYM sections to print placeholders for them.
This also allows to preserve usefull parameters, like virtual address.

Differential revision: https://reviews.llvm.org/D74955
2020-03-03 11:32:49 +03:00
Georgii Rymar 31f2ad9c36 [yaml2obj] - Automatically assign sh_addr for allocatable sections.
I've noticed that it is not convenient to create YAMLs from
binaries (using obj2yaml) that have to be test cases for obj2yaml
later (after applying yaml2obj).

The problem, for example is that obj2yaml emits "DynamicSymbols:"
key instead of .dynsym. It also does not create .dynstr.
And when a YAML document without explicitly defined .dynsym/.dynstr
is given to yaml2obj, we have issues:

1) These sections are placed after non-allocatable sections (I've fixed it in D74756).
2) They have VA == 0. User needs create descriptions for such sections explicitly manually
    to set a VA.

This patch addresses (2). I suggest to let yaml2obj assign virtual addresses by itself.
It makes an output binary to be much closer to "normal" ELF.
(It is still possible to use "Address: 0x0" for a section to get the original behavior
if it is needed)

Differential revision: https://reviews.llvm.org/D74764
2020-02-22 14:43:54 +03:00
Fangrui Song e2d7c5b2b6 [yaml2obj][test] Simplify some e_machine EI_CLASS EI_DATA tests
When both little-endian and big-endian are tested, or both 32-bit and 64-bit are tested, use a template like the following with `-D BITS=32 -D ENCODE=LSB`

```
--- !ELF
FileHeader:
 Class:   ELFCLASS[[BITS]]
 Data:    ELFDATA2[[ENCODE]]
 Type:    ET_DYN
 Machine: EM_X86_64
```

Reviewed By: grimar, jhenderson

Differential Revision: https://reviews.llvm.org/D73828
2020-02-07 09:35:26 -08:00
Georgii Rymar 60f161eb62 [yaml2obj][obj2yaml] - Simplify format of the SHT_LLVM_ADDRSIG section.
Previously the description allowed to describe symbols with use of
`Name` and `Index` keys. This patch removes them and now it is still
possible to use either names or symbol indexes, but the code is simpler
and the format is slightly different.

Such a change will be useful for another patches, e.g:
https://reviews.llvm.org/D73788#inline-671077

Differential revision: https://reviews.llvm.org/D73888
2020-02-05 12:33:14 +03:00
Fangrui Song 531fad736e [test] yaml2obj -docnum => --docnum=
Make usage more consistent, and make it possible to enable LongOptionsUseDoubleDash.
2020-02-04 10:33:21 -08:00
Georgii Rymar bec54e464e [yaml2obj/obj2yaml] - Add support for the SHT_LLVM_CALL_GRAPH_PROFILE section.
This is a LLVM specific section that is well described here:
https://llvm.org/docs/Extensions.html#sht-llvm-call-graph-profile-section-call-graph-profile

This patch teaches yaml2obj and obj2yaml about how to work with it.

Differential revision: https://reviews.llvm.org/D73788
2020-02-04 15:13:20 +03:00
Georgii Rymar e6b55cbcdc [yaml2obj][obj2yaml] - Add lost test cases.
It is a part of https://reviews.llvm.org/D71872 which
was lost somehow during relanding after being reverted:

https://reviews.llvm.org/rG7570d387c21935b58afa67cb9ee17250e38721fa
2020-01-29 15:40:35 +03:00
Igor Kudrin dcff3961c2 [DWARF] Return Error from DWARFDebugArangeSet::extract().
This helps to detect and report parsing errors better.
The patch follows the ideas of LLDB's patches D59370 and D59381.

It adds tests for valid and some invalid cases. More checks and
tests to come. Note that the patch fixes validation of the Length
field because the value does not include the field itself.

The existing users are updated to show the error messages.

Differential Revision: https://reviews.llvm.org/D71875
2020-01-23 12:41:05 +07:00
Fangrui Song 55c81d4282 [test] Use yaml2obj -o %t instead of > %t
To improve consistency and avoid unneeded shell feature (output
redirection).

While here, make other changes to improve consistency

--docnum 1 => --docnum=1
-docnum=x => --docnum=x
2020-01-21 17:20:18 -08:00
Georgii Rymar ca6f616532 Revert "[yaml2obj/obj2yaml] - Add support for SHT_RELR sections."
This reverts commit 46d11e30ee.

It broke bots. E.g. http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-ubuntu-fast/builds/60744
2020-01-15 14:19:00 +03:00
Georgii Rymar 46d11e30ee [yaml2obj/obj2yaml] - Add support for SHT_RELR sections.
The encoded sequence of Elf*_Relr entries in a SHT_RELR section looks
like [ AAAAAAAA BBBBBBB1 BBBBBBB1 ... AAAAAAAA BBBBBB1 ... ]
i.e. start with an address, followed by any number of bitmaps. The address
entry encodes 1 relocation. The subsequent bitmap entries encode up to 63(31)
relocations each, at subsequent offsets following the last address entry.

More information is here:
https://github.com/llvm-mirror/llvm/blob/master/lib/Object/ELF.cpp#L272

This patch adds a support for these sections.

Differential revision: https://reviews.llvm.org/D71872
2020-01-15 13:54:08 +03:00
Georgii Rymar 9659464d7e [yaml2obj/obj2yaml] - Add support for SHT_LLVM_DEPENDENT_LIBRARIES sections.
This section contains strings specifying libraries to be added to the link by the linker.
The strings are encoded as standard null-terminated UTF-8 strings.

This patch adds a way to describe and dump SHT_LLVM_DEPENDENT_LIBRARIES sections.

I introduced a new YAMLFlowString type here. That used to teach obj2yaml to dump
them like:

```
Libraries: [ foo, bar ]
```

instead of the following (if StringRef would be used):

```
Libraries:
  - foo
  - bar
```

Differential revision: https://reviews.llvm.org/D70598
2019-11-25 12:57:53 +03:00
Georgii Rymar dd101539da [yaml2obj/obj2yaml] - Add support for SHT_LLVM_LINKER_OPTIONS sections.
SHT_LLVM_LINKER_OPTIONS section contains pairs of null-terminated strings.
This patch adds support for them.

Differential revision: https://reviews.llvm.org/D69895
2019-11-12 09:55:20 +03:00
georgerim a7aee6c47a [yaml2obj/obj2yaml] - Add support for SHT_GNU_HASH section.
This adds parsing and dumping support for GNU hash sections.
They are described nicely here: https://blogs.oracle.com/solaris/gnu-hash-elf-sections-v2

Differential revision: https://reviews.llvm.org/D69399
2019-10-31 12:25:37 +03:00
Pavel Labath 83a55c6a57 minidump: Rename some architecture constants
The architecture enum contains two kinds of contstants: the "official" ones
defined by Microsoft, and unofficial constants added by breakpad to cover the
architectures not described by the first ones.

Up until now, there was no big need to differentiate between the two. However,
now that Microsoft has defined
https://docs.microsoft.com/en-us/windows/win32/api/sysinfoapi/ns-sysinfoapi-system_info
a constant for ARM64, we have a name clash.

This patch renames all breakpad-defined constants with to include the prefix
"BP_". This frees up the name "ARM64", which I'll re-introduce with the new
"official" value in a follow-up patch.

Reviewers: amccarth, clayborg

Subscribers: lldb-commits, llvm-commits

Differential Revision: https://reviews.llvm.org/D69285
2019-10-30 14:46:00 +01:00
Georgii Rymar 12c9ffd108 [obj2yaml] - Update a comment in a test case. NFC.
Addresses post-commit comments for D69160.
2019-10-29 11:58:19 +03:00
Georgii Rymar 3fe7f1dcf4 [yaml2obj] - Make .symtab to be not mandatory section for SHT_REL[A] section.
Before this change .symtab section was required for SHT_REL[A] section
declarations. yaml2obj automatically defined it in case when YAML document
did not have it.

With this change it is now possible to produce an object that
has a relocation section, but has no symbol table.

It simplifies the code and also it is inline with how we handle Link fields
for another special sections.

Differential revision: https://reviews.llvm.org/D69260
2019-10-29 11:43:12 +03:00
georgerim 9acaf54277 [obj2yaml] - Better dumping for relocations without symbols associated.
This just reorders the code and removes an assignment
of an empty string for the case when a relocation has
no symbol associated. With this our output becomes
cleaner and shorter.

Differential revision: https://reviews.llvm.org/D69255
2019-10-25 12:29:31 +03:00
georgerim 53c164e221 [llvm/Object] - Fix the error message reported for a broken SHT_SYMTAB_SHNDX section.
SHT_SYMTAB_SHNDX should have the same number of entries as the symbol table
associated (https://www.sco.com/developers/gabi/latest/ch4.sheader.html)

We currently can report the following message:
"SHT_SYMTAB_SHNDX section has sh_size (24) which is not equal to the number of symbols (2)"

It is just broken. This patch refines/fixes it.

Differential revision: https://reviews.llvm.org/D69305
2019-10-25 12:19:46 +03:00
George Rimar 6fc2891916 [obj2yaml] - Fix a comment. NFC.
I forgot to address this nit before committing..

llvm-svn: 375405
2019-10-21 10:40:43 +00:00
George Rimar 4ec0b08438 [obj2yaml] - Stop triggering UB when dumping corrupted strings.
We have a following code to find quote type:

if (isspace(S.front()) || isspace(S.back()))
...

Problem is that:

"int isspace( int ch ): The behavior is undefined if the value of
ch is not representable as unsigned char and is not equal to EOF."
(https://en.cppreference.com/w/cpp/string/byte/isspace)

This patch shows how this UB can be triggered and fixes an issue.

Differential revision: https://reviews.llvm.org/D69160

llvm-svn: 375404
2019-10-21 10:38:03 +00:00
George Rimar 2779987d0e [yaml2obj][obj2yaml] - Do not create a symbol table by default.
This patch tries to resolve problems faced in D68943
and uses some of the code written by Konrad Wilhelm Kleine
in that patch.

Previously, yaml2obj tool always created a .symtab section.
This patch changes that. With it we only create it when
have a "Symbols:" tag in the YAML document or when
we need to create it because it is used by another section(s).

obj2yaml follows the new behavior and does not print "Symbols:"
anymore when there is no symbol table.

Differential revision: https://reviews.llvm.org/D69041

llvm-svn: 375361
2019-10-20 14:47:17 +00:00
Joseph Tremoulet a50272f826 Update MinidumpYAML to use minidump::Exception for exception stream
Reviewers: labath, jhenderson, clayborg, MaskRay, grimar

Reviewed By: grimar

Subscribers: lldb-commits, grimar, MaskRay, hiraditya, llvm-commits

Tags: #llvm, #lldb

Differential Revision: https://reviews.llvm.org/D68657

llvm-svn: 375242
2019-10-18 14:56:19 +00:00
Pavel Labath 3aa7e76677 MinidumpYAML: Add support for the memory info list stream
Summary:
The implementation is fairly straight-forward and uses the same patterns
as the existing streams. The yaml form does not attempt to preserve the
data in the "gaps" that can be created by setting a larger-than-required
header or entry size in the stream header, because the existing consumer
(lldb) does not make use of the information in the gap in any way, and
attempting to preserve that would make the implementation more
complicated.

Reviewers: amccarth, jhenderson, clayborg

Subscribers: llvm-commits, lldb-commits, markmentovai, zturner, JosephTremoulet

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D68645

llvm-svn: 374337
2019-10-10 13:05:46 +00:00
George Rimar fc9104d42a Recommit r373598 "[yaml2obj/obj2yaml] - Add support for SHT_LLVM_ADDRSIG sections."
Fix: call `consumeError()` for a case missed.

Original commit message:

SHT_LLVM_ADDRSIG is described here:
https://llvm.org/docs/Extensions.html#sht-llvm-addrsig-section-address-significance-table

This patch teaches tools to dump them and to parse the YAML declarations of such sections.

Differential revision: https://reviews.llvm.org/D68333

llvm-svn: 373606
2019-10-03 14:52:33 +00:00
George Rimar 9f6cf2a081 Revert r373598 "[yaml2obj/obj2yaml] - Add support for SHT_LLVM_ADDRSIG sections."
It broke BB:
http://lab.llvm.org:8011/builders/clang-x86_64-debian-fast/builds/18655/steps/test/logs/stdio

llvm-svn: 373599
2019-10-03 14:04:47 +00:00
George Rimar 32cbabfecb [yaml2obj/obj2yaml] - Add support for SHT_LLVM_ADDRSIG sections.
SHT_LLVM_ADDRSIG is described here:
https://llvm.org/docs/Extensions.html#sht-llvm-addrsig-section-address-significance-table

This patch teaches tools to dump them and to parse the YAML declarations of such sections.

Differential revision: https://reviews.llvm.org/D68333

llvm-svn: 373598
2019-10-03 13:57:08 +00:00
George Rimar e5163ebf8d [yaml2obj/obj2yaml] - Add support for SHT_HASH sections.
SHT_HASH specification is:
http://www.sco.com/developers/gabi/latest/ch5.dynamic.html#hash

In short the format is the following: it has 2 uint32 fields
in its header: nbucket and nchain followed by (nbucket + nchain)
uint32 values.

This patch allows dumping and parsing such sections.

Differential revision: https://reviews.llvm.org/D68085

llvm-svn: 373315
2019-10-01 09:45:59 +00:00
George Rimar 1a219aa8df [yaml2obj/obj2yaml] - Add support for .stack_sizes sections.
.stack_sizes is a SHT_PROGBITS section that contains pairs of
<address (4/8 bytes), stack size (uleb128)>.

This patch teach tools to parse and dump it.

Differential revision: https://reviews.llvm.org/D67757

llvm-svn: 372762
2019-09-24 14:22:37 +00:00