Commit Graph

475 Commits

Author SHA1 Message Date
Fangrui Song 814dffa4b7 [llvm-objcopy][MachO] Support LC_LINKER_OPTIMIZATION_HINT load command
The load command is currently specific to arm64 and holds information
for instruction rewriting, e.g.  converting a GOT load to an ADR to
compute a local address.
(On ELF the information is usually conveyed by relocations, e.g.
R_X86_64_REX_GOTPCRELX, R_PPC64_TOC16_HA)

Reviewed By: alexander-shaposhnikov

Differential Revision: https://reviews.llvm.org/D104968
2021-06-29 18:47:55 -07:00
Fangrui Song 69937a8080 [llvm-objcopy][MachO] Support ARM64_RELOC_ADDEND
An ARM64_RELOC_ADDEND relocation reuses the symbol field for the addend value.
We should pass through such relocations.

Reviewed By: alexander-shaposhnikov

Differential Revision: https://reviews.llvm.org/D104967
2021-06-29 11:23:30 -07:00
Fangrui Song d619cf5ac5 [llvm-objcopy][MachO] Copy LC_LINKER_OPTIMIZATION_HINT
This fixes `error: unsupported load command (cmd=0x2e)`
2021-06-16 12:09:50 -07:00
Fangrui Song 1de18ad8d7 [llvm-objcopy] Make ihex writer similar to binary writer
There is no need to differentiate whether `UseSegments` is true or
false. Unifying the cases makes the behavior closer to BinaryWriter.

This improves compatibility with objcopy because SHF_ALLOC sections not in
a PT_LOAD will not be skipped. Such cases are usually erroneous input, though.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D104186
2021-06-16 10:08:20 -07:00
Ian McIntyre 5899278758 [llvm-objcopy] Exclude empty sections in IHexWriter output
IHexWriter was evaluating a section's physical address when deciding if
that section should be written to an output. This approach does not
account for a zero-sized section that has the same physical address as a
sized section. The behavior varies from GNU objcopy, and may result in a
HEX file that does not include all program sections.

The IHexWriter now excludes zero-sized sections when deciding what
should be written to the output. This affects the contents of the
writer's `Sections` collection; we will not try to insert multiple
sections that could have the same physical address. The behavior seems
consistent with GNU objcopy, which always excludes empty sections,
no matter the address.

The new test case evaluates the IHexWriter behavior when provided a
variety of empty sections that overlap or append a filled section. See
the input file's comments for more information. Given that test input,
and the change to the IHexWriter, GNU objcopy and llvm-objcopy produce
the same output.

Reviewed By: jhenderson, MaskRay, evgeny777

Differential Revision: https://reviews.llvm.org/D101332
2021-06-12 12:23:07 -07:00
Alexander Shaposhnikov 0276cc742b [llvm-objcopy][MachO] Do not strip symbols with the flag REFERENCED_DYNAMICALLY set
Do not strip symbols having the flag REFERENCED_DYNAMICALLY set.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D104092
2021-06-11 16:34:59 -07:00
Sergey Dmitriev 1fb5278882 [llvm-strip] Add support for '--' for delimiting options from input files
This will allow to use llvm-strip with file names that begin with dashes.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D102825
2021-05-20 03:33:51 -07:00
Sergey Dmitriev 0b12963b74 [llvm-objcopy] Update LIT test to resolve bot failure [NFC]
Reviewed By: hubert.reinterpretcast

Differential Revision: https://reviews.llvm.org/D102823
2021-05-19 19:56:35 -07:00
Sergey Dmitriev f24f140290 [llvm-objcopy] Add support for '--' for delimiting options from input/output files
This will allow to use llvm-objcopy with file names that begin with dashes.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D102665
2021-05-19 01:56:46 -07:00
Alex Orlov d8e65585f7 Fixed llvm-objcopy to add correct symbol table for ELF with program headers.
This fixes the following bugs:
https://bugs.llvm.org/show_bug.cgi?id=43935

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D102258
2021-05-12 12:39:30 +04:00
Fangrui Song b3336bfa2e [llvm-objcopy][ELF] --only-keep-debug: set offset/size of segments with no sections to zero
PR50160: we currently ignore non-PT_PHDR segments with no sections, not
accounting for its p_offset and p_filesz: this can cause an out-of-bounds write
in `writeSegmentData` if the p_offset+p_filesz is larger than the total file
size.

This can be fixed by setting p_offset=p_filesz=0. The logic nicely unifies with
the logic added in D90897.

Reviewed By: jhenderson, rupprecht

Differential Revision: https://reviews.llvm.org/D101560
2021-05-05 10:26:57 -07:00
Fangrui Song 96f3a63076 [llvm-objcopy] --dump-section: error if '=' is missing or filename is empty
Fix PR45416: the diagnostic when '=' is missing is misleading.
`FileOutputBuffer::create` returns successfully when the filename is empty
(the temporary file is `.tmp%%%%%%%`), but `FileOutputBuffer::commit` will error when
renaming `.tmp%%%%%%%` to the empty name).

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D101697
2021-05-04 17:30:57 -07:00
Jez Ng c00fc180ec [llvm-readobj] Recognize N_THUMB_DEF as a symbol flag
The right symbol flag mask is ~0x7, not ~0xf.

Also emit string names for the other flags (we were missing some).

Reviewed By: #lld-macho, gkm

Differential Revision: https://reviews.llvm.org/D101548
2021-04-30 17:39:56 -04:00
Alexander Shaposhnikov 86f291ebb2 [llvm-objcopy][MachO] Add support for LC_THREAD/LC_UNIXTHREAD
Add support for LC_THREAD/LC_UNIXTHREAD
(these load commands can be copied over without any modifications).

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D101384
2021-04-28 16:29:33 -07:00
Alexander Shaposhnikov 412437aec0 Revert "[llvm-objcopy][MachO] Add support for LC_THREAD/LC_UNIXTHREAD"
This reverts commit 4dfddf715b
since it breaks some build bots (e.g. clang-ppc64be-linux)
2021-04-27 16:19:59 -07:00
Alexander Shaposhnikov 4dfddf715b [llvm-objcopy][MachO] Add support for LC_THREAD/LC_UNIXTHREAD
Add support for LC_THREAD/LC_UNIXTHREAD
(these load commands can be copied over without any modifications).

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D101384
2021-04-27 15:54:51 -07:00
Alexander Shaposhnikov ede56e5127 [llvm-objcopy][MachO] Add support for --keep-undefined
This diff introduces --keep-undefined in llvm-objcopy/llvm-strip for Mach-O
which makes the tools preserve undefined symbols.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D97040
2021-03-08 18:57:25 -08:00
James Henderson 076698154a [llvm-objcopy] Fix crash for binary input files with non-ascii names
The code was using the standard isalnum function which doesn't handle
values outside the non-ascii range. Switching to using llvm::isAlnum
instead ensures we don't provoke undefined behaviour, which can in some
cases result in crashes.

Reviewed by: MaskRay

Differential Revision: https://reviews.llvm.org/D97663
2021-03-05 08:57:40 +00:00
James Henderson 47c343d768 [llvm-objcopy][test] Fix test that could have passed spuriously
The test was showing that when --strip-unneeded is specified for an
executable, all the symbols are stripped. However, the set of symbols
used in the test would be stripped by --strip-unneeded for an ET_REL
object too. Fix this by adding additional symbols that aren't normally
stripped by --strip-unneeded.

Reviewed by: MaskRay

Differential Revision: https://reviews.llvm.org/D97664
2021-03-05 08:57:39 +00:00
James Henderson 1562e4552c [llvm-objcopy][llvm-strip][test] Improve testing
This patch adds a number of new test cases that cover various
llvm-objcopy and llvm-strip features that had missing test coverage of
various descriptions:
* --add-section - checked the shdr properties, not just the content.
* Dedicated test case for --add-symbol when there are many sections.
* Show that --change-start accepts negative values without overflow.
  This was previously present but got lost between review versions.
* --dump-section - show that multiple sections can be dumped
  simultaneously to different files, and that an error is reported when
  a section cannot be found.
* --globalize-symbol(s) - show that symbols that are not mentioned are
  not globalized, if they would otherwise be, and that missing symbols
  from the list do not cause problems.
* --keep-global-symbol - show that the --regex option can be used in
  conjunction with this option.
* --keep-symbol - show that the --regex option can be used in
  conjunction with this option.
* --localize-symbol(s) - show that symbols that are not mentioned are
  not localized, if they would otherwise be, and that missing symbols
  from the list do not cause problems.
* --prefix-alloc-sections - show the behaviour of an empty string
  argument and multiple arguments.
* --prefix-symbols - show the behaviour of an empty string argument and
  multiple arguments. Also show the option applies to undefined symbols.
* --redefine-symbol - show that symbols with no name can be renamed,
  that it is not an error if a symbol is not specified, and that the
  option doesn't chain (i.e. --redefine-sym a=b --redefine-sym b=c does
  not redefine a as c).
* --rename-section - show that all section flags are preserved if none
  are specified. Also show that the option does not chain.
* --set-section-alignment - show that only specified sections have
  their alignments changed.
* --set-section-flags - show which section flags are preserved when this
  option is used. Also show that unspecified sections are not affected.
* --preserve-dates - show that -p is an alias of --preserve-dates.
* --strip-symbol - show that --regex works with this option for
  llvm-objcopy as well as llvm-strip.
* --strip-unneeded-symbol(s) - show more clearly that needed symbols are
  not stripped even if requested by this option.
* --allow-broken-links - show the sh_link of a symbol table is set to 0
  when its string table has been removed when this option is specified.
* --weaken-symbol(s) - show that symbols that are not mentioned are not
  weakened, if they would otherwise be, and that missing symbols from
  the list do not cause problems.
* --wildcard - show the wildcard behaviour for several options that were
  previously unchecked.

Reviewed by: alexshap

Differential Revision: https://reviews.llvm.org/D97666
2021-03-04 11:32:27 +00:00
James Henderson 8bb74d16ef [llvm-objcopy/strip] Fix off-by-one error in SYMTAB_SHNDX need check
The check for whether an extended symbol index table was required
dropped the first SHN_LORESERVE sections from the sections array before
checking whether the remaining sections had symbols. Unfortunately, the
null section header is not present in this list, so the check was
skipping the first section that might be important. If that section
contained a symbol, and no subsequent ones did, the .symtab_shndx
section would not be emitted, leading to a corrupt object.

Also consolidate and expand test coverage in the area to cover this bug
and other aspects of the SYMTAB_SHNDX section.

Reviewed by: alexshap, MaskRay

Differential Revision: https://reviews.llvm.org/D97661
2021-03-04 10:23:45 +00:00
James Henderson 49c91a64fd [llvm-objcopy][test] Improve many-sections object and test case
Additionally do some test tidy-ups and improve coverage of symbol
section indexes where the logical section index >= SHN_LORESERVE.

The symbol and section names in the many-section input object were
mostly shared. This patch changes them to be distinct, enabling
different operations such as --add-symbol, to be more targeted, when
using the object. It also makes the test less confusing and removes some
oddness in the symbol table order, presumably caused by the duplicate
names.

The input object was built from assembly that was of the form:
.section s1
sym1:
.section s2
sym2:
...
with a total of 65536 such occurrences. llvm-objcopy was then used to
remove the empty .text section automatically generated by MC, and
incidentally to move .strtab to the end of the object. This ensured that
the section/symbol indexes matched their name (i.e. section index 1 was
s1, section index 2 was s2 etc, and sym1 was in s1, sym2 in s2 etc).

Reviewed by: MaskRay

Differential Revision: https://reviews.llvm.org/D97660
2021-03-04 09:42:43 +00:00
Fangrui Song 17b4e695ce [llvm-objcopy] If input=output, preserve umask bits, otherwise drop S_ISUID/S_ISGID bits
This makes the behavior similar to cp

```
chmod u+s,g+s,o+x a
sudo llvm-strip a -o b
// With this patch, b drops set-user-ID and set-group-ID bits.
// sudo cp a b => b does not have set-user-ID or set-group-ID bits.
```

This also changes the behavior for the following case:

```
chmod u+s,g+s,o+x a
llvm-strip a
// a preserves set-user-ID and set-group-ID bits.
// This matches binutils<2.36 and probably >=2.37.  2.36 and 2.36.1 have some compatibility issues.
```

Differential Revision: https://reviews.llvm.org/D97253
2021-02-24 11:10:09 -08:00
Fangrui Song c465429f28 [llvm-objcopy] Delete --build-id-link-{dir,input,output}
The few options are niche. They solved a problem which was traditionally solved
with more shell commands (`llvm-readelf -n` fetches the Build ID. Then
`ln` is used to hard link the file to a directory derived from the Build ID.)

Due to limitation, they are no longer used by Fuchsia and they don't appear to
be used elsewhere (checked with Google Search and Debian Code Search). So delete
them without a transition period.

Announcement: https://lists.llvm.org/pipermail/llvm-dev/2021-February/148446.html

Differential Revision: https://reviews.llvm.org/D96310
2021-02-15 11:17:32 -08:00
Fangrui Song 3e837e1735 [llvm-objcopy][test] Stablize build-id-link-dir.test 2021-02-08 17:22:22 -08:00
Patrick Oppenlander 93345e825a [llvm-objcopy] -O binary: consider SHT_NOBITS sections to be empty
This is consistent with BFD objcopy.

Previously llvm objcopy would allocate space for SHT_NOBITS sections
often resulting in enormous binary files.

New test case (binary-paddr.test %t6).

Reviewed By: jhenderson, MaskRay

Differential Revision: https://reviews.llvm.org/D95569
2021-02-01 15:01:25 -08:00
Abhina Sreeskantharajan 42a21778f6 [test] Use host platform specific error message substitution in lit tests
On z/OS, the following error message is not matched correctly in lit tests.

```
EDC5129I No such file or directory.
```

This patch uses a lit config substitution to check for platform specific error messages.

Reviewed By: muiez, jhenderson

Differential Revision: https://reviews.llvm.org/D95246
2021-01-29 07:16:30 -05:00
Georgii Rymar d5e48f1347 [yaml2obj][obj2yaml] - Improve how we set/dump the sh_entsize field.
We already set the `sh_entsize` field in a single place
for all non-implicit sections.

This patch reorders the logic slightly and with it
we finally have the only one place where the `sh_entsize` is set.

obj2yaml will not dump the `EntSize` key for `SHT_DYNSYM/SHT_SYMTAB` sections anymore,
when the value of `sh_entsize` is equal to `sizeof(Elf_Sym)`

Note that this also seems revealed an issue in llvm-objcopy:
Previously yaml2obj set the `sh_entsize` for the `.symtab` section to 0x18,
now we it sets it for `SHT_SYMTAB` sections, i.e. by type.
But the `llvm-objcopy/ELF/only-keep-debug.test` has a `.symtab` section of type `SHT_STRTAB`,
and now yaml2obj sets the `sh_entsize` to 0 for it.
I had to update the corresponding check lines for `ES`, but the behavior of
`llvm-objcopy` should be fixed instead I think.
I've added a TODO and a comment.

Differential revision: https://reviews.llvm.org/D95364
2021-01-26 13:33:02 +03:00
Abhina Sreeskantharajan 978444d531 Revert "[SystemZ][z/OS] Fix No such file or directory expression error"
This reverts commit 06f8a49693.
2021-01-25 08:29:38 -05:00
Abhina Sreeskantharajan 84851a274e Revert "[SystemZ][z/OS] Fix No such file or directory expression error matching in lit tests - continued"
This reverts commit 520b5ecf85.
2021-01-25 08:29:38 -05:00
Abhina Sreeskantharajan 520b5ecf85 [SystemZ][z/OS] Fix No such file or directory expression error matching in lit tests - continued
This is a continuation of https://reviews.llvm.org/D94239. I missed some other spellings of the same error.

Reviewed By: muiez

Differential Revision: https://reviews.llvm.org/D95246
2021-01-22 13:54:25 -05:00
Abhina Sreeskantharajan 689aaba7ac [SystemZ][z/OS] Fix No such file or directory expression error matching in lit tests
On z/OS, the following error message is not matched correctly in lit tests. This patch updates the CHECK expression to match successfully.
```
EDC5129I No such file or directory.
```

Reviewed By: muiez

Differential Revision: https://reviews.llvm.org/D94239
2021-01-18 07:14:37 -05:00
Brandon Bergren 696bd3073f [PowerPC] Support powerpcle target in LLVMObject [2/5]
Add object file handling for powerpcle-*-*.

Adjust tests.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D93916
2021-01-02 12:17:39 -06:00
Georgii Rymar 98a4289810 [llvm-readobj] - For SHT_REL relocations, don't display an addend.
This is https://bugs.llvm.org/show_bug.cgi?id=44257.

In LLVM style we always print `0` as addend when dumping
SHT_REL relocations. It is confusing, this patch stops
printing it as the first comment on the bug page suggests.

Differential revision: https://reviews.llvm.org/D93033
2020-12-14 12:03:00 +03:00
Fangrui Song 20de182246 [llvm-objcopy] --only-keep-debug: place zero-size segment according to its parent segment
Alternative to D74755. sectionWithinSegment() treats an empty section as having
a size of 1. Due to the rule, an empty .tdata will not be attributed to an
empty PT_TLS. (The empty p_align=64 PT_TLS is for Android Bionic's TCB
compatibility (ELF-TLS). See https://reviews.llvm.org/D62055#1507426)

Currently --only-keep-debug will not layout a segment with no section
(layoutSegmentsForOnlyKeepDebug()), thus p_offset of PT_TLS can go past the end
of the file. The strange p_offset can trigger validation errors for subsequent
tools, e.g. llvm-objcopy errors when reading back the separate debug file
(readProgramHeaders()).

This patch places such an empty segment according to its parent segment.  This
special cases works for the empty PT_TLS used in Android. For a non-empty
segment, it should have at least one non-empty section and will be handled by
the normal code. Note, p_memsz PT_LOAD is rejected by both Linux and FreeBSD.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D90897
2020-11-11 09:21:10 -08:00
Georgii Rymar a7a447be0f [yaml2obj] - ProgramHeaders: introduce FirstSec/LastSec instead of Sections list.
Imagine we have a YAML declaration of few sections: `foo1`, `<unnamed 2>`, `foo3`, `foo4`.

To put them into segment we can do (1*):

```
Sections:
 - Section: foo1
 - Section: foo4
```

or we can use (2*):

```
Sections:
 - Section: foo1
 - Section: foo3
 - Section: foo4
```

or (3*) :

```
Sections:
 - Section: foo1
## "(index 2)" here is a name that we automatically created for a unnamed section.
 - Section: (index 2)
 - Section: foo3
 - Section: foo4
```

It looks really confusing that we don't have to list all of sections.

At first I've tried to make this rule stricter and report an error when there is a gap
(i.e. when a section is included into segment, but not listed explicitly).
This did not work perfect, because such approach conflicts with unnamed sections/fills (see (3*)).

This patch drops "Sections" key and introduces 2 keys instead: `FirstSec` and `LastSec`.
Both are optional.

Differential revision: https://reviews.llvm.org/D90458
2020-11-09 13:00:50 +03:00
Cornelius Aschermann 4d54c6fc5a [llvm-objcopy][MachO] Fix adding multiple sections
This diff fixes missing fields initialization (Size, VMSize).
Previously this resulted in broken binaries when multiple sections
were added in one tool's invocatation.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D90690
2020-11-07 18:16:06 -08:00
Alexander Shaposhnikov f147f59cd3 [llvm-objcopy][MachO] Skip sections with zero offset
Some binaries can contain regular sections with zero offset and zero size.
This diff makes llvm-objcopy's handling of such sections consistent with
cctools's strip (which doesn't modify them),
previously the tool would allocate file space for them.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D90796
2020-11-06 13:29:43 -08:00
Fangrui Song ee142c4988 [llvm-objcopy] Make --set-section-flags work with --add-section
This matches behavior GNU objcopy and can simplify clang-offload-bundler
(which currently works around the issue by invoking llvm-objcopy twice).

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D90438
2020-11-04 09:39:14 -08:00
Keith Smiley 51f8d46491 [llvm-install-name-tool] Quote passed rpath args in error messages
This diff refactors error reporting to make it more clear
what arguments were passed to llvm-install-name-tool.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D90080
2020-10-28 15:22:33 -07:00
Keith Smiley 77cbf25953 [llvm-install-name-tool] Add -prepend_rpath option
This diff adds the option -prepend_rpath which inserts an rpath as
the first rpath in the binary.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D89605
2020-10-23 15:01:03 -07:00
Alexander Shaposhnikov 27e11d7120 [MC] Adjust StringTableBuilder for linked Mach-O binaries
LD64 emits string tables which start with a space and a zero byte.
This diff adjusts StringTableBuilder for linked Mach-O binaries to match LD64's behavior.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D89561
2020-10-22 19:19:41 -07:00
Alexander Shaposhnikov 861429160c [llvm-objcopy][MachO] Fix the calculation of the output size
Virtual sections do not contribute to the final output size.
This diff fixes the corresponding calculations in the method MachOWriter::totalSize.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D89661
2020-10-22 13:58:11 -07:00
Tobias Hieta 61133e0b11 [llvm-install-name-tool] Add -delete_all_rpaths option
This diff adds an option to remove all rpaths from a Mach-O binary.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D88674
2020-10-13 00:45:57 -07:00
Jez Ng 84cc39c329 [llvm-readobj] Don't print out section names for STABS symbols
This diff is similar to what D71394 did for `llvm-objdump` -- it avoids
trying to look up a section name for STABS symbols, since some STABS
symbol types (like `N_OSO`) use the `n_sect` field to store other data
instead of a section index.

Differential Revision: https://reviews.llvm.org/D88468
2020-10-12 18:55:40 -07:00
Alexander Shaposhnikov 315970de1d [llvm-objcopy][MachO] Add support for universal binaries
This diff adds support for universal binaries to llvm-objcopy.
This is a recommit of 32c8435ef7 with the asan issue fixed.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D88400
2020-10-06 04:01:40 -07:00
Dmitri Gribenko 80f66ac0d5 Revert "[llvm-objcopy][MachO] Add support for universal binaries"
This reverts commit 32c8435ef7. It fails
ASan, details in https://reviews.llvm.org/D88400.
2020-10-06 11:29:24 +02:00
Alexander Shaposhnikov 32c8435ef7 [llvm-objcopy][MachO] Add support for universal binaries
This diff adds support for universal binaries to llvm-objcopy.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D88400
2020-10-05 10:44:28 -07:00
Alexander Shaposhnikov e60a760b7d [llvm-objcopy][MachO] Fix --add-section
This diff fixes --add-section functionality and simplifies the tests organization.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D87497
2020-09-24 01:51:10 -07:00
Georgii Rymar d4035af253 [llvm-readelf/obj] - Print section symbol names properly when dumping relocations.
Currently `--relocations` ignores section symbol names and always prints
section names for them. This is inconsistent with GNU readelf and with `--symbols`.

We have a code in `getFullSymbolName` (which is used for `--symbols`) which can be
reused for `getRelocationTarget` (used for `--relocations`).
With that the issue described is fixed and code becomes a bit shorter.
Also with this change we start to print more relocations (in situations when we just
showed warnings instead before) and also start to report more diagnostic warnings
(see reloc-zero-name-or-value.test).

Differential revision: https://reviews.llvm.org/D87613
2020-09-23 13:25:39 +03:00