Commit Graph

330 Commits

Author SHA1 Message Date
Fangrui Song 9d2504b6d8 [llvm-readobj][yaml2obj] Support SHT_LLVM_SYMPART, SHT_LLVM_PART_EHDR and SHT_LLVM_PART_PHDR
See http://lists.llvm.org/pipermail/llvm-dev/2019-February/130583.html
and D60242 for the lld partition feature.

This patch:

* Teaches yaml2obj to parse the 3 section types.
* Teaches llvm-readobj/llvm-readelf to dump the 3 section types.

There is no test for SHT_LLVM_DEPENDENT_LIBRARIES in llvm-readobj. Add
it as well.

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D67228

llvm-svn: 371157
2019-09-06 00:53:28 +00:00
Fangrui Song c3bc697974 [yaml2obj] Write the section header table after section contents
Linkers (ld.bfd/gold/lld) place the section header table at the very
end. This allows tools to strip it, which is optional in executable/shared objects.
In addition, if we add or section, the size of the section header table
will change. Placing the section header table in the end keeps section
offsets unchanged.

yaml2obj currently places the section header table immediately after the
program header. Follow what linkers do to make offset updating easier.

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D67221

llvm-svn: 371074
2019-09-05 14:25:57 +00:00
George Rimar 33b1a0eb5e [lib/ObjectYAML] - Cleanup the private interface of ELFState<ELFT>. NFCI.
In D62809 I accidentally added "ELFState<ELFT> &State" as the
first parameter to two methods. There is no reason for having that.
I removed this argument and also moved finalizeStrings declaration to
remove an excessive 'private:' tag.

Differential revision: https://reviews.llvm.org/D67157

llvm-svn: 371033
2019-09-05 08:59:28 +00:00
George Rimar e7b4d20998 Recommit r371023 "[lib/ObjectYAML] - Stop calling error(1) when mapping the st_other field of a symbol."
Fix: added missing return "return 0;"

Original commit message:
This eliminates one of the error(1) call in this lib.
It is different from the others because happens on a fields mapping stage
and can be easily fixed.

Differential revision: https://reviews.llvm.org/D67150

llvm-svn: 371030
2019-09-05 08:52:26 +00:00
George Rimar faa2a08987 Revert r371023 "[lib/ObjectYAML] - Stop calling error(1) when mapping the st_other field of a symbol." (2)
Forgot to revert the cpp file.

llvm-svn: 371025
2019-09-05 08:39:44 +00:00
George Rimar 2c9c432256 [lib/ObjectYAML] - Stop calling error(1) when mapping the st_other field of a symbol.
This eliminates one of the error(1) call in this lib.
It is different from the others because happens on a fields mapping stage
and can be easily fixed.

Differential revision: https://reviews.llvm.org/D67150

llvm-svn: 371023
2019-09-05 08:28:43 +00:00
Fangrui Song 441d450115 [yaml2obj] Support PT_GNU_STACK and PT_GNU_RELRO
PT_GNU_STACK is used in an llvm-objcopy test.

I plan to use PT_GNU_RELRO in a patch to improve nested segment
processing in llvm-objcopy (PR42963).

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D67146

llvm-svn: 370857
2019-09-04 09:19:31 +00:00
George Rimar 86cc736df1 [yaml2obj] - Allow overriding sh_name fields of the sections.
This is in line with the previous changes which allowed to
override the sh_offset/sh_size and useful for writing test cases.

Differential revision: https://reviews.llvm.org/D66998

llvm-svn: 370633
2019-09-02 09:47:17 +00:00
George Rimar 4e71702cd4 [yaml2obj][obj2yaml] - Use a single "Other" field instead of "Other", "Visibility" and "StOther".
Currenly we can encode the 'st_other' field of symbol using 3 fields.
'Visibility' is used to encode STV_* values.
'Other' is used to encode everything except the visibility, but it can't handle arbitrary values.
'StOther' is used to encode arbitrary values when 'Visibility'/'Other' are not helpfull enough.

'st_other' field is used to encode symbol visibility and platform-dependent
flags and values. Problem to encode it is that it consists of Visibility part (STV_* values)
which are enumeration values and the Other part, which is different and inconsistent.

For MIPS the Other part contains flags for all STO_MIPS_* values except STO_MIPS_MIPS16.
(Like comment in ELFDumper says: "Someones in their infinite wisdom decided to make
STO_MIPS_MIPS16 flag overlapped with other ST_MIPS_xxx flags."...)

And for PPC64 the Other part might actually encode any value.

This patch implements custom logic for handling the st_other and removes
'Visibility' and 'StOther' fields.

Here is an example of a new YAML style this patch allows:

- Name:  foo
  Other: [ 0x4 ]
- Name:  bar
  Other: [ STV_PROTECTED, 4 ]
- Name:  zed
  Other: [ STV_PROTECTED, STO_MIPS_OPTIONAL, 0xf8 ]

Differential revision: https://reviews.llvm.org/D66886

llvm-svn: 370472
2019-08-30 13:39:22 +00:00
Dan Gohman da84b688f9 [WebAssembly] Make __attribute__((used)) not imply export.
Add an WASM_SYMBOL_NO_STRIP flag, so that __attribute__((used)) doesn't
need to imply exporting. When targeting Emscripten, have
WASM_SYMBOL_NO_STRIP imply exporting.

Differential Revision: https://reviews.llvm.org/D62542

llvm-svn: 370415
2019-08-29 22:40:00 +00:00
George Rimar de0bc44883 [yaml2obj] - Allow placing local symbols after globals.
This allows us to produce broken binaries with local
symbols placed after global in '.dynsym'/'.symtab'

Also, simplifies the code.

Differential revision: https://reviews.llvm.org/D66799

llvm-svn: 370331
2019-08-29 10:58:47 +00:00
Vlad Tsyrklevich b8a96f4bf5 Reland "[yaml2obj] - Don't allow setting StOther and Other/Visibility at the same time."
This relands this commit, I mistakenly reverted the original change
thinking it was the cause of the observed MSan failures but it was not.

llvm-svn: 370206
2019-08-28 14:04:09 +00:00
Vlad Tsyrklevich aba62e9c00 Revert "[yaml2obj] - Don't allow setting StOther and Other/Visibility at the same time."
This reverts commit r370032, it was causing check-llvm failures on
sanitizer-x86_64-linux-bootstrap-msan

llvm-svn: 370198
2019-08-28 13:15:08 +00:00
George Rimar 7a2e21d9f4 [yaml2obj] - Don't allow setting StOther and Other/Visibility at the same time.
This is a follow up discussed in the comments of D66583.

Currently, if for example, we have both StOther and Other set in YAML document for a symbol,
then yaml2obj reports an "unknown key 'Other'" error.
It happens because 'mapOptional()' is never called for 'Other/Visibility' in this case,
leaving those unhandled.

This message does not describe the reason of the error well. This patch fixes it.

Differential revision: https://reviews.llvm.org/D66642

llvm-svn: 370032
2019-08-27 09:58:39 +00:00
George Rimar 668b11b2c8 [yaml2obj] - Allow setting the symbol st_other field to any integer.
st_other field of a symbol usually contains its visibility.
Other bits are usually 0, though some targets, like
MIPS can set them using the named bit field values.

Problem is that there is no way to set an arbitrary value now,
though that might be useful for our test cases.

In this patch I introduced a way to set st_other to any numeric
value using the new StOther field.
I added a test and simplified the existent one to show the effect/benefit

Differential revision: https://reviews.llvm.org/D66583

llvm-svn: 369742
2019-08-23 09:31:07 +00:00
George Rimar 91208447d0 [yaml2obj] - Lookup relocation symbols in dynamic symbol when .dynsym referenced.
This fixes https://bugs.llvm.org/show_bug.cgi?id=40337.

Previously, it was always assumed that relocations referenced symbols in the static symbol table. 
Now, if the Link field references a section called ".dynsym" it will look up these symbols
in the dynamic symbol table.

This patch is heavily based on D59097 by James Henderson

Differential revision: https://reviews.llvm.org/D66532

llvm-svn: 369645
2019-08-22 12:39:56 +00:00
Pavel Labath 82275ec51d MinidumpYAML: move serialization code to MinidumpEmitter.cpp
Summary:
The code for serializing minidumps was living in MinidumpYAML.cpp
so that it would be accessible from unit tests. While this had its
advantages, it was also unfortunate because it broke symmetry with all
other yaml2obj serializers.

Fortunately, nowadays all of yaml2obj is a library, so we don't need to
do anything special. This patch improves the code consistency by moving
the serialization code to MinidumpEmitter.cpp to match the style used in
other backends. It also removes the writeAsBinary entry point in favor
of the more general convertYAML interface.

This patch is just massaging the code a bit. There shouldn't be any
functional change here.

Reviewers: jhenderson, abrachet

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D66474

llvm-svn: 369517
2019-08-21 11:30:48 +00:00
Seiya Nuta 522377494b [yaml2obj/obj2yaml][MachO] Allow setting custom section data
Reviewers: alexshap, jhenderson, rupprecht

Reviewed By: alexshap, jhenderson

Subscribers: abrachet, hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D65799

llvm-svn: 369348
2019-08-20 08:49:07 +00:00
Jonas Devlieghere 0eaee545ee [llvm] Migrate llvm::make_unique to std::make_unique
Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.

llvm-svn: 369013
2019-08-15 15:54:37 +00:00
George Rimar d3963051c4 [yaml2obj/obj2yaml] - Add a basic support for extended section indexes.
In some cases a symbol might have section index == SHN_XINDEX.
This is an escape value indicating that the actual section header index
is too large to fit in the containing field.
Then the SHT_SYMTAB_SHNDX section is used. It contains the 32bit values
that stores section indexes.

ELF gABI says that there can be multiple SHT_SYMTAB_SHNDX sections,
i.e. for example one for .symtab and one for .dynsym
(1) https://groups.google.com/forum/#!topic/generic-abi/-XJAV5d8PRg
(2) DT_SYMTAB_SHNDX: http://www.sco.com/developers/gabi/latest/ch5.dynamic.html

In this patch I am only supporting a single SHT_SYMTAB_SHNDX associated
with a .symtab. This is a more or less common case which is used a few tests I saw in LLVM.

I decided not to create the SHT_SYMTAB_SHNDX section as "implicit",
but implement is like a kind of regular section for now.
i.e. tools do not recreate this section or its content, like they do for
symbol table sections, for example. That should allow to write all kind of
possible broken test cases for our needs and keep the output closer to requested.

Differential revision: https://reviews.llvm.org/D65446

llvm-svn: 368272
2019-08-08 09:49:05 +00:00
Alex Brachet c22d9666fc [yaml2obj] Move core yaml2obj code into lib and include for use in unit tests
Reviewers: jhenderson, rupprecht, MaskRay, grimar, labath

Reviewed By: rupprecht

Subscribers: gribozavr, mgrang, seiya, mgorny, sbc100, hiraditya, aheejin, jakehehrlich, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D65255

llvm-svn: 368119
2019-08-07 02:44:49 +00:00
Dmitri Gribenko fc21bb661f Revert "[yaml2obj] Move core yaml2obj code into lib and include for use in unit tests"
This reverts commit r368021, it broke tests.

llvm-svn: 368035
2019-08-06 13:39:50 +00:00
Alex Brachet 3cfeaa4d2c [yaml2obj] Move core yaml2obj code into lib and include for use in unit tests
Reviewers: jhenderson, rupprecht, MaskRay, grimar, labath

Reviewed By: rupprecht

Subscribers: seiya, mgorny, sbc100, hiraditya, aheejin, jakehehrlich, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D65255

llvm-svn: 368021
2019-08-06 12:15:18 +00:00
Nilanjana Basu da60fc813c Changing representation of .cv_def_range directives in Codeview debug info assembly format for better readability
llvm-svn: 367867
2019-08-05 14:16:58 +00:00
Nilanjana Basu b5e4d7de17 Revert "Changing representation of .cv_def_range directives in Codeview debug info assembly format for better readability"
This reverts commit a885afa9fa.

llvm-svn: 367861
2019-08-05 13:55:21 +00:00
Nilanjana Basu a885afa9fa Changing representation of .cv_def_range directives in Codeview debug info assembly format for better readability
llvm-svn: 367850
2019-08-05 13:11:51 +00:00
Eric Christopher 5fb56b1966 Temporarily Revert "Changing representation of cv_def_range directives in Codeview debug info assembly format for better readability"
This is breaking bots and the author asked me to revert.

This reverts commit 367704.

llvm-svn: 367707
2019-08-02 19:10:37 +00:00
Nilanjana Basu 1c67521591 Changing representation of cv_def_range directives in Codeview debug info assembly format for better readability
llvm-svn: 367704
2019-08-02 18:44:39 +00:00
George Rimar eb41f7f081 [yaml2obj] - Allow overriding the sh_size field.
There is no way to set broken sh_size field currently
for sections. It can be usefull for writing the
test cases. 

Differential revision: https://reviews.llvm.org/D64401

llvm-svn: 365766
2019-07-11 12:59:29 +00:00
Stanislav Mekhanoshin 22b2c3d651 [AMDGPU] gfx908 target
Differential Revision: https://reviews.llvm.org/D64429

llvm-svn: 365525
2019-07-09 18:10:06 +00:00
George Rimar 9df825f429 [yaml2obj] - Allow overriding sh_offset field from the YAML.
Some of our test cases are using objects which
has sections with a broken sh_offset field.

There was no way to set it from YAML until this patch.

Differential revision: https://reviews.llvm.org/D63879

llvm-svn: 364898
2019-07-02 10:20:12 +00:00
George Rimar 687d47c2b0 [yaml2obj] - Allow overriding e_shentsize, e_shoff, e_shnum and e_shstrndx fields in the YAML.
This allows setting different values for e_shentsize, e_shoff, e_shnum
and e_shstrndx fields and is useful for producing broken inputs for various
test cases.

Differential revision: https://reviews.llvm.org/D63771

llvm-svn: 364517
2019-06-27 11:08:42 +00:00
Nicolai Haehnle 08e8cb5760 AMDGPU/MC: Add .amdgpu_lds directive
Summary:
The directive defines a symbol as an group/local memory (LDS) symbol.
LDS symbols behave similar to common symbols for the purposes of ELF,
using the processor-specific SHN_AMDGPU_LDS as section index.

It is the linker and/or runtime loader's job to "instantiate" LDS symbols
and resolve relocations that reference them.

It is not possible to initialize LDS memory (not even zero-initialize
as for .bss).

We want to be able to link together objects -- starting with relocatable
objects, but possible expanding to shared objects in the future -- that
access LDS memory in a flexible way.

LDS memory is in an address space that is entirely separate from the
address space that contains the program image (code and normal data),
so having program segments for it doesn't really make sense.

Furthermore, we want to be able to compile multiple kernels in a
compilation unit which have disjoint use of LDS memory. In that case,
we may want to place LDS symbols differently for different kernels
to save memory (LDS memory is very limited and physically private to
each kernel invocation), so we can't simply place LDS symbols in a
.lds section.

Hence this solution where LDS symbols always stay undefined.

Change-Id: I08cbc37a7c0c32f53f7b6123aa0afc91dbc1748f

Reviewers: arsenm, rampitec, t-tye, b-sumner, jsjodin

Subscribers: kzhuravl, jvesely, wdng, yaxunl, dstuttard, tpr, rupprecht, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D61493

llvm-svn: 364296
2019-06-25 11:51:35 +00:00
George Rimar b6e20937b3 [yaml2obj/obj2yaml] - Make RawContentSection::Info Optional<>
This allows to customize this field for "implicit" sections properly.

Differential revision: https://reviews.llvm.org/D63487

llvm-svn: 363777
2019-06-19 08:57:38 +00:00
George Rimar cfa1a62a4c [yaml2obj] - Allow setting cutom Flags for implicit sections.
With this patch we get ability to set any flags we want
for implicit sections defined in YAML.

Differential revision: https://reviews.llvm.org/D63136

llvm-svn: 363367
2019-06-14 11:01:14 +00:00
Stanislav Mekhanoshin c43e67bfff [AMDGPU] gfx1011/gfx1012 targets
Differential Revision: https://reviews.llvm.org/D63307

llvm-svn: 363344
2019-06-14 00:33:31 +00:00
George Rimar 1e41007aeb [yaml2obj/obj2yaml] - Make RawContentSection::Content and RawContentSection::Size optional
This is a follow-up for D62809.

Content and Size fields should be optional as was discussed in comments
of the D62809's thread. With that, we can describe a specific string table and
symbol table sections in a more correct way and also show appropriate errors.

The patch adds lots of test cases where the behavior is described in details.

Differential revision: https://reviews.llvm.org/D62957

llvm-svn: 362931
2019-06-10 12:43:18 +00:00
Fangrui Song f090e6f7b6 [llvm-objdump/llvm-readobj/obj2yaml/yaml2obj] Support DT_PPC_GOT and DT_PPC_OPT
In glibc, DT_PPC_GOT indicates that PowerPC32 Secure PLT ABI is used.
I plan to use it in D62464.

DT_PPC_OPT currently indicates if a TLSDESC inspired TLS optimization is
enabled.

Reviewed By: grimar, jhenderson, rupprecht

Differential Revision: https://reviews.llvm.org/D62851

llvm-svn: 362569
2019-06-05 01:36:48 +00:00
Peter Smith 49d7221f71 [AArch64][ELF][llvm-readobj] Add support for BTI and PAC dynamic tags
ELF for the 64-bit Arm Architecture defines two processor-specific dynamic
tags:
DT_AARCH64_BTI_PLT 0x70000001, d_val
DT_AARCH64_PAC_PLT 0x70000003, d_val

These presence of these tags indicate that PLT sequences have been
protected using Branch Target Identification and Pointer Authentication
respectively. The presence of both indicates that the PLT sequences have
been protected with both Branch Target Identification and Pointer
Authentication.

This patch adds the tags and tests for llvm-readobj and yaml2obj.

As some of the processor specific dynamic tags overlap, this patch splits
them up, keeping their original default value if they were not previously
mentioned explicitly in a switch case.

Differential Revision: https://reviews.llvm.org/D62596

llvm-svn: 362493
2019-06-04 11:44:33 +00:00
Tom Tan eb4d6142dc [COFF, ARM64] Add CodeView register mapping
CodeView has its own register map which is defined in cvconst.h. Missing this
mapping before saving register to CodeView causes debugger to show incorrect
value for all register based variables, like variables in register and local
variables addressed by register (stack pointer + offset).

This change added mapping between LLVM register and CodeView register so the
correct register number will be stored to CodeView/PDB, it aso fixed the
mapping from CodeView register number to register name based on current
CPUType but print PDB to yaml still assumes X86 CPU and needs to be fixed.

Differential Revision: https://reviews.llvm.org/D62608

llvm-svn: 362280
2019-05-31 23:43:31 +00:00
Jason Liu 9212206d25 [XCOFF] Implement parsing symbol table for xcoffobjfile and output as yaml format
Summary:
This patch implement parsing symbol table for xcoffobjfile and
output as yaml format. Parsing auxiliary entries of a symbol
will be in a separate patch.

The XCOFF object file (aix_xcoff.o) used in the test comes from
-bash-4.2$ cat test.c
extern int i;
extern int TestforXcoff;
int main()
{
i++;
TestforXcoff--;
}

Patch by DiggerLin

Reviewers: sfertile, hubert.reinterpretcast, MaskRay, daltenty

Differential Revision: https://reviews.llvm.org/D61532

llvm-svn: 361832
2019-05-28 14:37:59 +00:00
Ben Dunbobbin 1d16515fb4 [ELF] Implement Dependent Libraries Feature
This patch implements a limited form of autolinking primarily designed to allow
either the --dependent-library compiler option, or "comment lib" pragmas (
https://docs.microsoft.com/en-us/cpp/preprocessor/comment-c-cpp?view=vs-2017) in
C/C++ e.g. #pragma comment(lib, "foo"), to cause an ELF linker to automatically
add the specified library to the link when processing the input file generated
by the compiler.

Currently this extension is unique to LLVM and LLD. However, care has been taken
to design this feature so that it could be supported by other ELF linkers.

The design goals were to provide:

- A simple linking model for developers to reason about.
- The ability to to override autolinking from the linker command line.
- Source code compatibility, where possible, with "comment lib" pragmas in other
  environments (MSVC in particular).

Dependent library support is implemented differently for ELF platforms than on
the other platforms. Primarily this difference is that on ELF we pass the
dependent library specifiers directly to the linker without manipulating them.
This is in contrast to other platforms where they are mapped to a specific
linker option by the compiler. This difference is a result of the greater
variety of ELF linkers and the fact that ELF linkers tend to handle libraries in
a more complicated fashion than on other platforms. This forces us to defer
handling the specifiers to the linker.

In order to achieve a level of source code compatibility with other platforms
we have restricted this feature to work with libraries that meet the following
"reasonable" requirements:

1. There are no competing defined symbols in a given set of libraries, or
   if they exist, the program owner doesn't care which is linked to their
   program.
2. There may be circular dependencies between libraries.

The binary representation is a mergeable string section (SHF_MERGE,
SHF_STRINGS), called .deplibs, with custom type SHT_LLVM_DEPENDENT_LIBRARIES
(0x6fff4c04). The compiler forms this section by concatenating the arguments of
the "comment lib" pragmas and --dependent-library options in the order they are
encountered. Partial (-r, -Ur) links are handled by concatenating .deplibs
sections with the normal mergeable string section rules. As an example, #pragma
comment(lib, "foo") would result in:

.section ".deplibs","MS",@llvm_dependent_libraries,1
         .asciz "foo"

For LTO, equivalent information to the contents of a the .deplibs section can be
retrieved by the LLD for bitcode input files.

LLD processes the dependent library specifiers in the following way:

1. Dependent libraries which are found from the specifiers in .deplibs sections
   of relocatable object files are added when the linker decides to include that
   file (which could itself be in a library) in the link. Dependent libraries
   behave as if they were appended to the command line after all other options. As
   a consequence the set of dependent libraries are searched last to resolve
   symbols.
2. It is an error if a file cannot be found for a given specifier.
3. Any command line options in effect at the end of the command line parsing apply
   to the dependent libraries, e.g. --whole-archive.
4. The linker tries to add a library or relocatable object file from each of the
   strings in a .deplibs section by; first, handling the string as if it was
   specified on the command line; second, by looking for the string in each of the
   library search paths in turn; third, by looking for a lib<string>.a or
   lib<string>.so (depending on the current mode of the linker) in each of the
   library search paths.
5. A new command line option --no-dependent-libraries tells LLD to ignore the
   dependent libraries.

Rationale for the above points:

1. Adding the dependent libraries last makes the process simple to understand
   from a developers perspective. All linkers are able to implement this scheme.
2. Error-ing for libraries that are not found seems like better behavior than
   failing the link during symbol resolution.
3. It seems useful for the user to be able to apply command line options which
   will affect all of the dependent libraries. There is a potential problem of
   surprise for developers, who might not realize that these options would apply
   to these "invisible" input files; however, despite the potential for surprise,
   this is easy for developers to reason about and gives developers the control
   that they may require.
4. This algorithm takes into account all of the different ways that ELF linkers
   find input files. The different search methods are tried by the linker in most
   obvious to least obvious order.
5. I considered adding finer grained control over which dependent libraries were
   ignored (e.g. MSVC has /nodefaultlib:<library>); however, I concluded that this
   is not necessary: if finer control is required developers can fall back to using
   the command line directly.

RFC thread: http://lists.llvm.org/pipermail/llvm-dev/2019-March/131004.html.

Differential Revision: https://reviews.llvm.org/D60274

llvm-svn: 360984
2019-05-17 03:44:15 +00:00
Pavel Labath 2d29e16c30 Minidump: Add support for the MemoryList stream
Summary:
the stream format is exactly the same as for ThreadList and ModuleList
streams, only the entry types are slightly different, so the changes in
this patch are just straight-forward applications of established
patterns.

Reviewers: amccarth, jhenderson, clayborg

Subscribers: markmentovai, lldb-commits, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D61885

llvm-svn: 360908
2019-05-16 15:17:30 +00:00
Reid Kleckner 7c438c5b07 [codeview] Finish support for reading and writing S_ANNOTATION records
Implement dumping via llvm-pdbutil and llvm-readobj.

llvm-svn: 360813
2019-05-15 20:53:39 +00:00
Reid Kleckner 7eb6b5ffc3 [COFF] Fix .bss section size bug in obj2yaml / yaml2obj
We need to serialize SizeOfRawData through even when there is no data,
as in a .bss section.

Fixes PR41836

llvm-svn: 360473
2019-05-10 21:53:44 +00:00
Pavel Labath dcdb3c6650 MinidumpYAML: add support for the ThreadList stream
Summary:
The implementation is a pretty straightforward extension of the pattern
used for (de)serializing the ModuleList stream. Since there are other
streams which use the same format (MemoryList and MemoryList64, at
least). I tried to generalize the code a bit so that adding future
streams of this type can be done with less code.

Reviewers: amccarth, jhenderson, clayborg

Subscribers: markmentovai, lldb-commits, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D61423

llvm-svn: 360350
2019-05-09 15:13:53 +00:00
George Rimar 0974688a42 [yaml2obj] - Allow setting st_value explicitly for Symbol.
In some cases it is useful to explicitly set symbol's st_name value.
For example, I am using it in a patch for LLD to remove the broken
binary from a test case and replace it with a YAML test.

Differential revision: https://reviews.llvm.org/D61180

llvm-svn: 360137
2019-05-07 12:10:51 +00:00
Sam Clegg 5f8c2edef3 [WebAssembly] Add more test coverage for reloctions against section symbols
The only known user of this relocation type and symbol type is
the debug info sections, but we were not testing the `--relocatable`
output path.

This change adds a minimal test case to cover relocations against
section symbols includes `--relocatable` output.

Differential Revision: https://reviews.llvm.org/D61623

llvm-svn: 360110
2019-05-07 03:53:16 +00:00
Dan Gohman 3b5b9d0e72 [WebAssembly] Support EXPLICIT_NAME symbols in llvm-readobj
Teach llvm-readobj about WASM_SYMBOL_EXPLICIT_NAME.

Differential Revision: https://reviews.llvm.org/D61323

Reviewer: sbc100
llvm-svn: 359602
2019-04-30 19:30:24 +00:00
George Rimar 45d042ed96 [yaml2obj] - Don't crash on invalid inputs.
yaml2obj might crash on invalid input when unable to parse the YAML.

Recently a crash with a very similar nature was fixed for an empty files. 
This patch revisits the fix and does it in yaml::Input instead.
It seems to be more correct way to handle such situation.

With that crash for invalid inputs is also fixed now.

Differential revision: https://reviews.llvm.org/D61059

llvm-svn: 359178
2019-04-25 09:59:55 +00:00
Stanislav Mekhanoshin cee607e414 [AMDGPU] Add gfx1010 target definitions
Differential Revision: https://reviews.llvm.org/D61041

llvm-svn: 359113
2019-04-24 17:03:15 +00:00
Benjamin Kramer 7085795284 MinidumpYAML: Fix ambiguity between std::make_unique and llvm::make_unique
llvm-svn: 358673
2019-04-18 15:06:03 +00:00
Pavel Labath 7429d86f36 MinidumpYAML: Add support for ModuleList stream
Summary:
This patch adds support for yaml (de)serialization of the minidump
ModuleList stream. It's a fairly straight forward-application of the
existing patterns to the ModuleList structures defined in previous
patches.

One thing, which may be interesting to call out explicitly is the
addition of "new" allocation functions to the helper BlobAllocator
class. The reason for this was, that there was an emerging pattern of a
need to allocate space for entities, which do not have a suitable
lifetime for use with the existing allocation functions. A typical
example of that was the "size" of various lists, which is only available
as a temporary returned by the .size() method of some container. For
these cases, one can use the new set of allocation functions, which
will take a temporary object, and store it in an allocator-managed
buffer until it is written to disk.

Reviewers: amccarth, jhenderson, clayborg, zturner

Subscribers: lldb-commits, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60405

llvm-svn: 358672
2019-04-18 14:57:31 +00:00
Thomas Lively fef8de66a6 [WebAssembly] Add DataCount section to object files
Summary:
This ensures that object files will continue to validate as
WebAssembly modules in the presence of bulk memory operations. Engines
that don't support bulk memory operations will not recognize the
DataCount section and will report validation errors, but that's ok
because object files aren't supposed to be run directly anyway.

Reviewers: aheejin, dschuff, sbc100

Subscribers: jgravelle-google, hiraditya, sunfish, rupprecht, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60623

llvm-svn: 358315
2019-04-12 22:27:48 +00:00
Pavel Labath 4a6dd1e683 Fix MSVC build for r357749
MSVC found the bare "make_unique" invocation ambiguous (between std::
and llvm:: versions). Explicitly qualifying the call with llvm:: should
hopefully fix it.

llvm-svn: 357750
2019-04-05 08:26:58 +00:00
Pavel Labath 51d9fa0a22 Minidump: Add support for reading/writing strings
Summary:
Strings in minidump files are stored as a 32-bit length field, giving
the length of the string in *bytes*, which is followed by the
appropriate number of UTF16 code units. The string is also supposed to
be null-terminated, and the null-terminator is not a part of the length
field. This patch:
- adds support for reading these strings out of the minidump file (this
  implementation does not depend on proper null-termination)
- adds support for writing them to a minidump file
- using the previous two pieces implements proper (de)serialization of
  the CSDVersion field of the SystemInfo stream. Previously, this was
  only read/written as hex, and no attempt was made to access the
  referenced string -- now this string is read and written correctly.

The changes are tested via yaml2obj|obj2yaml round-trip as well as a
unit test which checks the corner cases of the string deserialization
logic.

Reviewers: jhenderson, zturner, clayborg

Subscribers: llvm-commits, aprantl, markmentovai, amccarth, lldb-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59775

llvm-svn: 357749
2019-04-05 08:06:26 +00:00
Hubert Tong ab2eb2bfac [XCOFF] Add functionality for parsing AIX XCOFF object file headers
Summary:
1. Add functionality for parsing AIX XCOFF object files headers.
2. Only support 32-bit AIX XCOFF object files in this patch.
3. Print out the AIX XCOFF object file header in YAML format.

Reviewers: sfertile, hubert.reinterpretcast, jasonliu, mstorsjo, zturner, rnk

Reviewed By: sfertile, hubert.reinterpretcast

Subscribers: jsji, mgorny, hiraditya, jdoerfert, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59419

Patch by Digger Lin

llvm-svn: 357663
2019-04-04 00:53:21 +00:00
Reid Kleckner e10d00419a [codeview] Remove Type member from CVRecord
Summary:
Now CVType and CVSymbol are effectively type-safe wrappers around
ArrayRef<uint8_t>. Make the kind() accessor load it from the
RecordPrefix, which is the same for types and symbols.

Reviewers: zturner, aganea

Subscribers: hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60018

llvm-svn: 357658
2019-04-04 00:28:48 +00:00
George Rimar 6da44ad75d [yaml2obj][obj2yaml] - Change how symbol's binding is descibed when parsing/dumping.
Currently, YAML has the following syntax for describing the symbols:

Symbols:
  Local:
    LocalSymbol1:
    ...
    LocalSymbol2:
    ...
  ...
  Global:
    GlobalSymbol1:
  ...
  Weak:
  ...
  GNUUnique:

I.e. symbols are grouped by their bindings. That is not very convenient,
because:

It does not allow to set a custom binding, what can be useful for producing
broken/special outputs for test cases. Adding a new binding would require to
change a syntax (what we observed when added GNUUnique recently).

It does not allow to change the order of the symbols in .symtab/.dynsym,
i.e. currently all Local symbols are placed first, then Global, Weak and GNUUnique
are following, but we are not able to change the order.

It is not consistent. Binding is just one of the properties of the symbol,
we do not group them by other properties.

It makes the code more complex that it can be. This patch shows it can be simplified
with the change performed.

The patch changes the syntax to just:

Symbols:
  Symbol1:
  ...
  Symbol2:
  ...
...

With that, we are able to work with the binding field just like with any other symbol property.

Differential revision: https://reviews.llvm.org/D60122

llvm-svn: 357595
2019-04-03 14:53:42 +00:00
Michael Liao deef72091a [ObjectYAML] Fix build issue
- ObjectYAML depends on Object as minidump support adds additional
  dependency.

llvm-svn: 357471
2019-04-02 13:01:45 +00:00
Pavel Labath 3cee663e71 Add minidump support to obj2yaml
Summary:
This patch adds the code needed to parse a minidump file into the
MinidumpYAML model, and the necessary glue code so that obj2yaml can
recognise the minidump files and process them.

Reviewers: jhenderson, zturner, clayborg

Subscribers: mgorny, lldb-commits, amccarth, markmentovai, aprantl, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59634

llvm-svn: 357469
2019-04-02 11:58:37 +00:00
George Rimar 4111299584 [yaml2obj][obj2yaml] - Teach yaml2obj/obj2yaml tools about STB_GNU_UNIQUE symbols.
yaml2obj/obj2yaml does not support the symbols with STB_GNU_UNIQUE yet.
Currently, obj2yaml fails with llvm_unreachable when met such a symbol.

I faced it when investigated the https://bugs.llvm.org/show_bug.cgi?id=41196.

Differential revision: https://reviews.llvm.org/D59875

llvm-svn: 357158
2019-03-28 10:52:14 +00:00
Pavel Labath c5f7905e93 MinidumpYAML.cpp: Fix some code standard violations missed during review
functions should begin with lower case letters. NFC.

llvm-svn: 356901
2019-03-25 14:45:31 +00:00
Pavel Labath 69de7a955e [ObjectYAML] Add basic minidump generation support
Summary:
This patch adds the ability to read a yaml form of a minidump file and
write it out as binary. Apart from the minidump header and the stream
directory, only three basic stream kinds are supported:
- Text: This kind is used for streams which contain textual data. This
  is typically the contents of a /proc file on linux (e.g.
  /proc/PID/maps). In this case, we just put the raw stream contents
  into the yaml.
- SystemInfo: This stream contains various bits of information about the
  host system in binary form. We expose the data in a structured form.
- Raw: This kind is used as a fallback when we don't have any special
  knowledge about the stream. In this case, we just print the stream
  contents in hex.

For this code to be really useful, more stream kinds will need to be
added (particularly for things like lists of memory regions and loaded
modules). However, these can be added incrementally.

Reviewers: jhenderson, zturner, clayborg, aprantl

Subscribers: mgorny, lemo, llvm-commits, lldb-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59482

llvm-svn: 356753
2019-03-22 14:47:26 +00:00
Thomas Lively f6f4f84378 [WebAssembly] Target features section
Summary:
Implements a new target features section in assembly and object files
that records what features are used, required, and disallowed in
WebAssembly objects. The linker uses this information to ensure that
all objects participating in a link are feature-compatible and records
the set of used features in the output binary for use by optimizers
and other tools later in the toolchain.

The "atomics" feature is always required or disallowed to prevent
linking code with stripped atomics into multithreaded binaries. Other
features are marked used if they are enabled globally or on any
function in a module.

Future CLs will add linker flags for ignoring feature compatibility
checks and for specifying the set of allowed features, implement using
the presence of the "atomics" feature to control the type of memory
and segments in the linked binary, and add front-end flags for
relaxing the linkage policy for atomics.

Reviewers: aheejin, sbc100, dschuff

Subscribers: jgravelle-google, hiraditya, sunfish, mgrang, jfb, jdoerfert, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59173

llvm-svn: 356610
2019-03-20 20:26:45 +00:00
James Henderson b10f48bbb4 [yaml2obj]Allow explicit setting of p_filesz, p_memsz, and p_offset
yaml2obj currently derives the p_filesz, p_memsz, and p_offset values of
program headers from their sections. This makes writing tests for
certain formats more complex, and sometimes impossible. This patch
allows setting these fields explicitly, overriding the default value,
when relevant.

Reviewed by: jakehehrlich, Higuoxing

Differential Revision: https://reviews.llvm.org/D59372

llvm-svn: 356247
2019-03-15 10:35:27 +00:00
George Rimar a5a0a0f049 [yaml2obj] - Allow producing ELFDATANONE ELFs
I need this to remove a binary from LLD test suite.
The patch also simplifies the code a bit.

Differential revision: https://reviews.llvm.org/D59082

llvm-svn: 355591
2019-03-07 12:09:19 +00:00
George Rimar a7ba1a0f81 [yaml2obj] - Allow setting custom sh_info for RawContentSection sections.
This is for tweaking SHT_SYMTAB sections.
Their sh_info contains the (number of symbols + 1) usually.
But for creating invalid inputs for test cases it would be convenient
to allow explicitly override this field from YAML.

Differential revision: https://reviews.llvm.org/D58779

llvm-svn: 355193
2019-03-01 10:18:16 +00:00
George Rimar b75bf8784e [yaml2obj][obj2yaml] - Add support for the architecture specific dynamic tags.
This allows tools to parse/dump the architecture specific tags
like DT_MIPS_*, DT_PPC64_* and DT_HEXAGON_*

Also fixes a bug in DynamicTags.def which was revealed in this patch.

Differential revision: https://reviews.llvm.org/D58667

llvm-svn: 354876
2019-02-26 14:14:49 +00:00
James Henderson fd99780c09 [yaml2obj]Re-allow dynamic sections to have raw content
Recently, support was added to yaml2obj to allow dynamic sections to
have a list of entries, to make it easier to write tests with dynamic
sections. However, this change also removed the ability to provide
custom contents to the dynamic section, making it hard to test
malformed contents (e.g. because the section is not a valid size to
contain an array of entries). This change reinstates this. An error is
emitted if raw content and dynamic entries are both specified.

Reviewed by: grimar, ruiu

Differential Review: https://reviews.llvm.org/D58543

llvm-svn: 354770
2019-02-25 11:02:24 +00:00
George Rimar 623ae72ad4 [yaml2obj][obj2yaml] - Support SHT_GNU_verdef (.gnu.version_d) section.
This patch adds support for parsing/dumping the .gnu.version section.

Description of the section is: https://refspecs.linuxfoundation.org/LSB_1.3.0/gLSB/gLSB/symverdefs.html

Differential revision: https://reviews.llvm.org/D58437

llvm-svn: 354574
2019-02-21 12:21:43 +00:00
James Henderson 67bdfb0a59 [yaml2obj]Allow symbol Index field to take values lower than SHN_LORESERVE
In order to test tool handling of invalid section indexes, I need to
create an object containing such an invalid section index. I could
create a hex-edited binary, but having the ability to use yaml2obj is
preferable. Prior to this change, yaml2obj would reject any explicit
section indexes less than SHN_LORESERVE. This patch changes it to allow
any value.

I had to change the test to use llvm-readelf instead of llvm-readobj,
because llvm-readobj does not like invalid section indexes. I've also
expanded the test to show that the most common SHN_* values are accepted
(SHN_UNDEF, SHN_ABS, SHN_COMMON).

Reviewed by: grimar, jakehehrlich

Differential Revision: https://reviews.llvm.org/D58445

llvm-svn: 354566
2019-02-21 10:57:15 +00:00
Fangrui Song 61cd368cdc [ObjectYAML] Support SHT_MIPS_DWARF section type flag
Also reorder SHT_MIPS_DWARF and SHT_MIPS_ABIFLAGS in Object/ELF.cpp.
The test will be added by D58457.

llvm-svn: 354563
2019-02-21 10:19:08 +00:00
Thomas Lively 2e1504091e [WebAssembly] Update MC for bulk memory
Summary:
Rename MemoryIndex to InitFlags and implement logic for determining
data segment layout in ObjectYAML and MC. Also adds a "passive" flag
for the .section assembler directive although this cannot be assembled
yet because the assembler does not support data sections.

Reviewers: sbc100, aardappel, aheejin, dschuff

Subscribers: jgravelle-google, hiraditya, sunfish, rupprecht, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D57938

llvm-svn: 354397
2019-02-19 22:56:19 +00:00
James Henderson d82914c8d2 [yaml2obj][obj2yaml] Remove section type range markers from allowed mappings and support hex values
yaml2obj/obj2yaml previously supported SHT_LOOS, SHT_HIOS, and
SHT_LOPROC for section types. These are simply values that delineate a
range and don't really make sense as valid values. For example if a
section has type value 0x70000000, obj2yaml shouldn't print this value
as SHT_LOPROC. Additionally, this was missing the three other range
markers (SHT_HIPROC, SHT_LOUSER and SHT_HIUSER).

This change removes these three range markers. It also adds support for
specifying the type as an integer, to allow section types that LLVM
doesn't know about.

Reviewed by: grimar

Differential Revision: https://reviews.llvm.org/D58383

llvm-svn: 354344
2019-02-19 16:22:21 +00:00
George Rimar 646af08e2b [yaml2obj][obj2yaml] - Support SHT_GNU_versym (.gnu.version) section.
This patch adds support for parsing dumping the .gnu.version section.
Description of the section is: https://refspecs.linuxfoundation.org/LSB_1.3.0/gLSB/gLSB/symversion.html#SYMVERTBL

Differential revision: https://reviews.llvm.org/D58280

llvm-svn: 354338
2019-02-19 15:29:07 +00:00
George Rimar 0621b79587 Recommit r354328, r354329 "[obj2yaml][yaml2obj] - Add support of parsing/dumping of the .gnu.version_r section."
Fix:
Replace
assert(!IO.getContext() && "The IO context is initialized already");
with
assert(IO.getContext() && "The IO context is not initialized");
(this was introduced in r354329, where I tried to quickfix the darwin BB
and seems copypasted the assert from the wrong place).

Original commit message:

The section is described here:
https://refspecs.linuxfoundation.org/LSB_1.3.0/gLSB/gLSB/symverrqmts.html

Patch just teaches obj2yaml/yaml2obj to dump and parse such sections.

We did the finalization of string tables very late,
and I had to move the logic to make it a bit earlier.
That was needed in this patch since .gnu.version_r adds strings to .dynstr.
This might also be useful for implementing other special sections.

Everything else changed in this patch seems to be straightforward.

Differential revision: https://reviews.llvm.org/D58119

llvm-svn: 354335
2019-02-19 14:53:48 +00:00
George Rimar aa735de65f Revert r354328, r354329 "[obj2yaml][yaml2obj] - Add support of parsing/dumping of the .gnu.version_r section."
Something went wrong. Bots are unhappy:
http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-ubuntu-fast/builds/44113/steps/test/logs/stdio

llvm-svn: 354332
2019-02-19 14:38:25 +00:00
George Rimar 12b283dfa6 Fix BB after r354328.
Bot:
http://lab.llvm.org:8011/builders/lld-x86_64-darwin13/builds/30188/steps/build_Lld/logs/stdio

Error:
/Users/buildslave/as-bldslv9_new/lld-x86_64-darwin13/llvm.src/lib/ObjectYAML/ELFYAML.cpp:1013:15: error: unused variable 'Object' [-Werror,-Wunused-variable]
  const auto *Object = static_cast<ELFYAML::Object *>(IO.getContext());
              ^
/Users/buildslave/as-bldslv9_new/lld-x86_64-darwin13/llvm.src/lib/ObjectYAML/ELFYAML.cpp:1023:15: error: unused variable 'Object' [-Werror,-Wunused-variable]
  const auto *Object = static_cast<ELFYAML::Object *>(IO.getContext());

Fix:
change 
  const auto *Object = static_cast<ELFYAML::Object *>(IO.getContext());
  assert(Object && "The IO context is not initialized");
to
  assert(!IO.getContext() && "The IO context is initialized already");

llvm-svn: 354329
2019-02-19 14:22:10 +00:00
George Rimar c09f2cd0cb [obj2yaml][yaml2obj] - Add support of parsing/dumping of the .gnu.version_r section.
The section is described here:
https://refspecs.linuxfoundation.org/LSB_1.3.0/gLSB/gLSB/symverrqmts.html

Patch just teaches obj2yaml/yaml2obj to dump and parse such sections.

We did the finalization of string tables very late,
and I had to move the logic to make it a bit earlier.
That was needed in this patch since .gnu.version_r adds strings to .dynstr.
This might also be useful for implementing other special sections.

Everything else changed in this patch seems to be straightforward.

Differential revision: https://reviews.llvm.org/D58119

llvm-svn: 354328
2019-02-19 14:03:14 +00:00
George Rimar b87ea73706 [yaml2obj/obj2yaml] - Move `Info` field out from `Section` class.
ELFYAML.h contains a `Section` class which is a base for a few other
sections classes that are used for mapping different section types.
`Section` has a `StringRef Info` field used for storing sh_info.

At the same time, sh_info has very different meanings for sections and
cannot be processed in a similar way generally,
for example ELFDumper does not handle it in `dumpCommonSection`
but do that in `dumpGroup` and `dumpCommonRelocationSection` respectively.

At this moment, we have and handle it as a string, because that was possible for
the current use case. But also it can simply be a number:
For SHT_GNU_verdef is "The number of version definitions within the section."

The patch moves `Info` field out to be able to have it as a number. 
With that change, each class will be able to decide what type and purpose
of the sh_info field it wants to use.

I also had to edit 2 test cases. This is because patch fixes a bug. Previously we 
accepted yaml files with Info fields for all sections (for example, for SHT_DYNSYM too).
But we do not handle it and the resulting objects had zero sh_info fields set for
such sections. Now it is accepted only for sections that supports it.

Differential revision: https://reviews.llvm.org/D58054

llvm-svn: 353810
2019-02-12 09:08:59 +00:00
George Rimar 0745ca7830 [lib/ObjectYAML] - Fix BB after r353607 [2]. NFC.
The second and the last place it seems.

Error was:
[  4%] Building CXX object lib/Support/CMakeFiles/LLVMSupport.dir/Error.cpp.o
/Users/buildslave/as-bldslv9_new/lld-x86_64-darwin13/llvm.src/lib/ObjectYAML/ELFYAML.cpp:993:15: error: unused variable 'Object' [-Werror,-Wunused-variable]
  const auto *Object = static_cast<ELFYAML::Object *>(IO.getContext());

llvm-svn: 353609
2019-02-09 12:14:20 +00:00
George Rimar cc22d887ac [lib/ObjectYAML] - Fix BB after r353607. NFC.
Error was:
[  4%] Building CXX object lib/Support/CMakeFiles/LLVMSupport.dir/DAGDeltaAlgorithm.cpp.o
/Users/buildslave/as-bldslv9_new/lld-x86_64-darwin13/llvm.src/lib/ObjectYAML/ELFYAML.cpp:666:15: error: unused variable 'Object' [-Werror,-Wunused-variable]
  const auto *Object = static_cast<ELFYAML::Object *>(IO.getContext());
(http://lab.llvm.org:8011/builders/lld-x86_64-darwin13/builds/29920)

llvm-svn: 353608
2019-02-09 12:04:39 +00:00
George Rimar 0e7ed91264 [yaml2obj][obj2yaml] - Add support for dumping/parsing .dynamic sections.
This teaches the tools to parse and dump
the .dynamic section and its dynamic tags.

Differential revision: https://reviews.llvm.org/D57691

llvm-svn: 353606
2019-02-09 11:34:28 +00:00
Sam Clegg d6ef8da317 [WebAssembly] Add symbol flag to the binary format llvm.used
Summary:
Rather than add a new attribute
See https://github.com/WebAssembly/tool-conventions/issues/64

Subscribers: dschuff, jgravelle-google, aheejin, sunfish, rupprecht, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D57864

llvm-svn: 353360
2019-02-07 01:24:44 +00:00
James Henderson c836e48841 [yaml2obj]Allow number for ELF symbol type
yaml2obj previously only recognised standard STT_* names, and didn't
allow arbitrary numbers. This change allows the user to specify a number
for the type instead. It also adds a test to verify the existing
behaviour for obj2yaml for unkown symbol types.

Reviewed by: grimar

Differential Revision: https://reviews.llvm.org/D57822

llvm-svn: 353315
2019-02-06 17:16:33 +00:00
Roman Lebedev 418280100b [yaml::BinaryRef] Slight perf tuning (for llvm-exegesis analysis mode)
Summary:
llvm-exegesis uses this functionality to read it's benchmark dumps.
This reading of `.yaml`s takes ~60% of runtime for 14656 benchmark points (i.e. one sweep over all x86 instructions),
but only 30% of time for 3x as much benchmark points.

In particular, this `BinaryRef` appears to be an obvious pain point.
Without patch:
```
$ perf stat -r 25 ./bin/llvm-exegesis -mode=analysis -analysis-epsilon=1.0 -benchmarks-file=/tmp/benchmarks-inverse_throughput-onefull.yaml -analysis-clusters-output-file="" -analysis-inconsistencies-output-file=/tmp/clusters-orig.html
no exegesis target for x86_64-unknown-linux-gnu, using default
Parsed 14656 benchmark points
Printing sched class consistency analysis results to file '/tmp/clusters-orig.html'
...
no exegesis target for x86_64-unknown-linux-gnu, using default
Parsed 14656 benchmark points
Printing sched class consistency analysis results to file '/tmp/clusters-orig.html'

 Performance counter stats for './bin/llvm-exegesis -mode=analysis -analysis-epsilon=1.0 -benchmarks-file=/tmp/benchmarks-inverse_throughput-onefull.yaml -analysis-clusters-output-file= -analysis-inconsistencies-output-file=/tmp/clusters-orig.html' (25 runs):

            972.86 msec task-clock                #    0.994 CPUs utilized            ( +-  0.25% )
                30      context-switches          #   30.774 M/sec                    ( +- 21.74% )
                 0      cpu-migrations            #    0.370 M/sec                    ( +- 67.81% )
             11873      page-faults               # 12211.512 M/sec                   ( +-  0.00% )
        3898373408      cycles                    # 4009682.186 GHz                   ( +-  0.25% )  (83.12%)
         360399748      stalled-cycles-frontend   #    9.24% frontend cycles idle     ( +-  0.54% )  (83.24%)
        1099450483      stalled-cycles-backend    #   28.20% backend cycles idle      ( +-  0.59% )  (33.63%)
        4910528820      instructions              #    1.26  insn per cycle
                                                  #    0.22  stalled cycles per insn  ( +-  0.13% )  (50.21%)
        1111976775      branches                  # 1143726625.854 M/sec              ( +-  0.10% )  (66.77%)
          23248474      branch-misses             #    2.09% of all branches          ( +-  0.19% )  (83.29%)

           0.97850 +- 0.00647 seconds time elapsed  ( +-  0.66% )
```
With the patch:
```
$ perf stat -r 25 ./bin/llvm-exegesis -mode=analysis -analysis-epsilon=1.0 -benchmarks-file=/tmp/benchmarks-inverse_throughput-onefull.yaml -analysis-clusters-output-file="" -analysis-inconsistencies-output-file=/tmp/clusters-new.html
no exegesis target for x86_64-unknown-linux-gnu, using default
Parsed 14656 benchmark points
Printing sched class consistency analysis results to file '/tmp/clusters-new.html'
...
no exegesis target for x86_64-unknown-linux-gnu, using default
Parsed 14656 benchmark points
Printing sched class consistency analysis results to file '/tmp/clusters-new.html'

 Performance counter stats for './bin/llvm-exegesis -mode=analysis -analysis-epsilon=1.0 -benchmarks-file=/tmp/benchmarks-inverse_throughput-onefull.yaml -analysis-clusters-output-file= -analysis-inconsistencies-output-file=/tmp/clusters-new.html' (25 runs):

            905.29 msec task-clock                #    0.999 CPUs utilized            ( +-  0.11% )
                15      context-switches          #   16.533 M/sec                    ( +- 32.27% )
                 0      cpu-migrations            #    0.000 K/sec
             11873      page-faults               # 13121.789 M/sec                   ( +-  0.00% )
        3627759720      cycles                    # 4009283.100 GHz                   ( +-  0.11% )  (83.19%)
         370401480      stalled-cycles-frontend   #   10.21% frontend cycles idle     ( +-  0.22% )  (83.19%)
        1007114438      stalled-cycles-backend    #   27.76% backend cycles idle      ( +-  0.34% )  (33.62%)
        4414014304      instructions              #    1.22  insn per cycle
                                                  #    0.23  stalled cycles per insn  ( +-  0.08% )  (50.36%)
        1003751700      branches                  # 1109314021.971 M/sec              ( +-  0.07% )  (66.97%)
          24611010      branch-misses             #    2.45% of all branches          ( +-  0.10% )  (83.41%)

           0.90593 +- 0.00105 seconds time elapsed  ( +-  0.12% )
```
So this decreases the overall run time of llvm-exegesis analysis mode (on one sweep) by roughly -7%.

To be noted, `BinaryRef::writeAsBinary()` change is the reason for the perf changes,
usage of `llvm::isHexDigit()` instead of `isxdigit()` does not appear to have any perf impact,
i have only changed it "for symmetry".

`writeAsBinary()` change is correct, it produces identical de-hex-ified buffer, and the final output is thus identical:
```
$ sha512sum /tmp/clusters-*
db4bbd904fe8840853b589b032c5041bc060b91bcd9c27b914b56581fbc473550eea74b852238c79963b5adf2419f379e9f5db76784048b48e3937f9f3e732bf  /tmp/clusters-new.html
db4bbd904fe8840853b589b032c5041bc060b91bcd9c27b914b56581fbc473550eea74b852238c79963b5adf2419f379e9f5db76784048b48e3937f9f3e732bf  /tmp/clusters-orig.html
```

Reviewers: silvas, espindola, sbc100, zturner, courbet, gchatelet

Reviewed By: gchatelet

Subscribers: tschuett, RKSimon, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D57699

llvm-svn: 353282
2019-02-06 08:57:01 +00:00
Martin Storsjo e5eb6fb950 [COFF] Add new relocation types.
Differential Revision: https://reviews.llvm.org/D57291

llvm-svn: 352324
2019-01-27 19:53:36 +00:00
Chandler Carruth 2946cd7010 Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636
2019-01-19 08:50:56 +00:00
Thomas Lively cbda16eb8e [WebAssembly] Parse llvm.ident into producers section
llvm-svn: 351413
2019-01-17 02:29:55 +00:00
Thomas Lively 3cfcc94c09 Revert "[WebAssembly] Parse llvm.ident into producers section"
This reverts commit eccdbba3a02a33e13b5262e92200a33e2ead873d.

llvm-svn: 351410
2019-01-17 00:39:49 +00:00
Thomas Lively a56c23c5ba [WebAssembly] Parse llvm.ident into producers section
Summary:
Everything before the word "version" is the tool, and everything after
the word "version" is the version.

Reviewers: aheejin, dschuff

Subscribers: sbc100, jgravelle-google, sunfish, llvm-commits

Differential Revision: https://reviews.llvm.org/D56742

llvm-svn: 351399
2019-01-16 23:46:14 +00:00
Thomas Lively 6a87ddac9a [WebAssembly] Massive instruction renaming
Summary:
An automated renaming of all the instructions listed at
https://github.com/WebAssembly/spec/issues/884#issuecomment-426433329
as well as some similarly-named identifiers.

Reviewers: aheejin, dschuff, aardappel

Subscribers: sbc100, jgravelle-google, eraman, sunfish, jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D56338

llvm-svn: 350609
2019-01-08 06:25:55 +00:00
Martin Storsjo 93a7137c0a [ObjectYAML] [COFF] Support multiple symbols with the same name
Differential Revision: https://reviews.llvm.org/D56294

llvm-svn: 350566
2019-01-07 20:55:33 +00:00
George Rimar 6367d7a6d1 [yaml2obj/obj2yaml] - Support dumping/parsing ABI version.
These tools were assuming ABI version is 0,
that is not always true.

Patch teaches them to work with that field.

Differential revision: https://reviews.llvm.org/D55884

llvm-svn: 349737
2018-12-20 10:43:49 +00:00
Sam Clegg 03801256d8 [WebAssembly] Update dylink section parsing
This updates the format of the dylink section in accordance with
recent "spec" change:
  https://github.com/WebAssembly/tool-conventions/pull/77

Differential Revision: https://reviews.llvm.org/D55609

llvm-svn: 348989
2018-12-12 23:40:58 +00:00
Krzysztof Parzyszek bc86027519 [Hexagon] Add missing flags to ELF YAMLIO
llvm-svn: 347768
2018-11-28 16:25:47 +00:00
Sam Clegg e4afbc6804 [WebAssembly] Add support for dylink section in object format
See https://github.com/WebAssembly/tool-conventions/blob/master/DynamicLinking.md.

Differential Revision: https://reviews.llvm.org/D54490

llvm-svn: 346880
2018-11-14 18:36:24 +00:00
Heejin Ahn da419bdb5e [WebAssembly] Add support for the event section
Summary:
This adds support for the 'event section' specified in the exception
handling proposal. (This was named 'exception section' first, but later
renamed to 'event section' to take possibilities of other kinds of
events into consideration. But currently we only store exception info in
this section.)

The event section is added between the global section and the export
section. This is for ease of validation per request of the V8 team.

This patch:
- Creates the event symbol type, which is a weak symbol
- Makes 'throw' instruction take the event symbol '__cpp_exception'
- Adds relocation support for events
- Adds WasmObjectWriter / WasmObjectFile (Reader) support
- Adds obj2yaml / yaml2obj support
- Adds '.eventtype' printing support

Reviewers: dschuff, sbc100, aardappel

Subscribers: jgravelle-google, sunfish, llvm-commits

Differential Revision: https://reviews.llvm.org/D54096

llvm-svn: 346825
2018-11-14 02:46:21 +00:00
Derek Schuff 6881806241 [WebAssembly] Add shared memory support to limits field
Support the IS_SHARED bit in the memory limits flag word.
The compiler does not create object files with memory definitions,
but the field is used by the linker.

Differential Revision: https://reviews.llvm.org/D54131

llvm-svn: 346246
2018-11-06 17:27:25 +00:00