This patch is follow-up for D91028. It implements direct writing into the
output stream for wasm.
Depends on D91028
Differential Revision: https://reviews.llvm.org/D95478
This patch removes creation of the resulting file from the
executeObjcopyOnBinary() function. For the most use cases, the
executeObjcopyOnBinary receives output file as a parameter
- raw_ostream &Out. The splitting .dwo file is implemented differently:
file containg .dwo tables is created inside executeObjcopyOnBinary().
When objcopy functionality would be moved into separate library,
current implementation will become inconvenient. The goal of that
refactoring is to separate concerns: It might be convenient to
to do dwo tables splitting but to create resulting file differently.
Differential Revision: https://reviews.llvm.org/D98582
The D93881 added functionality which preserve ownership for output file
if llvm-objcopy is called under root. That code was added into the place
where output file is created. The llvm-objcopy already has a function which
sets/restores rights/permissions for the output file.
That is the restoreStatOnFile() function. This patch moves code
(preserving ownershipping) into the restoreStatOnFile() function.
Differential Revision: https://reviews.llvm.org/D98511
During D88827 it was requested to remove the local implementation
of Memory/File Buffers:
// TODO: refactor the buffer classes in LLVM to enable us to use them here
// directly.
This patch uses raw_ostream instead of Buffers. Generally, using streams
could allow us to reduce memory usages. No need to load all data into the
memory - the data could be streamed through a smaller buffer.
Thus, this patch uses raw_ostream as an interface for output data:
Error executeObjcopyOnBinary(CopyConfig &Config,
object::Binary &In,
raw_ostream &Out);
Note 1. This patch does not change the implementation of Writers
so that data would be directly stored into raw_ostream.
This is assumed to be done later.
Note 2. It would be better if Writers would be implemented in a such way
that data could be streamed without seeking/updating. If that would be
inconvenient then raw_ostream could be replaced with raw_pwrite_stream
to have a possibility to seek back and update file headers.
This is assumed to be done later if necessary.
Note 3. Current FileOutputBuffer allows using a memory-mapped file.
The raw_fd_ostream (which could be used if data should be stored in the file)
does not allow us to use a memory-mapped file. Memory map functionality
could be implemented for raw_fd_ostream:
It is possible to add resize() method into raw_ostream.
class raw_ostream {
void resize(uint64_t size);
}
That method, implemented for raw_fd_ostream, could create a memory-mapped file.
The streamed data would be written into that memory file then.
Thus we would be able to use memory-mapped files with raw_fd_ostream.
This is assumed to be done later if necessary.
Differential Revision: https://reviews.llvm.org/D91028
This diff introduces --keep-undefined in llvm-objcopy/llvm-strip for Mach-O
which makes the tools preserve undefined symbols.
Test plan: make check-all
Differential revision: https://reviews.llvm.org/D97040
The code was using the standard isalnum function which doesn't handle
values outside the non-ascii range. Switching to using llvm::isAlnum
instead ensures we don't provoke undefined behaviour, which can in some
cases result in crashes.
Reviewed by: MaskRay
Differential Revision: https://reviews.llvm.org/D97663
The help text and documentation for the --discard-all option failed to
mention that the option also causes the removal of debug sections. This
change fixes both for both llvm-objcopy and llvm-strip.
Reviewed by: MaskRay
Differential Revision: https://reviews.llvm.org/D97662
The check for whether an extended symbol index table was required
dropped the first SHN_LORESERVE sections from the sections array before
checking whether the remaining sections had symbols. Unfortunately, the
null section header is not present in this list, so the check was
skipping the first section that might be important. If that section
contained a symbol, and no subsequent ones did, the .symtab_shndx
section would not be emitted, leading to a corrupt object.
Also consolidate and expand test coverage in the area to cover this bug
and other aspects of the SYMTAB_SHNDX section.
Reviewed by: alexshap, MaskRay
Differential Revision: https://reviews.llvm.org/D97661
This makes the behavior similar to cp
```
chmod u+s,g+s,o+x a
sudo llvm-strip a -o b
// With this patch, b drops set-user-ID and set-group-ID bits.
// sudo cp a b => b does not have set-user-ID or set-group-ID bits.
```
This also changes the behavior for the following case:
```
chmod u+s,g+s,o+x a
llvm-strip a
// a preserves set-user-ID and set-group-ID bits.
// This matches binutils<2.36 and probably >=2.37. 2.36 and 2.36.1 have some compatibility issues.
```
Differential Revision: https://reviews.llvm.org/D97253
The few options are niche. They solved a problem which was traditionally solved
with more shell commands (`llvm-readelf -n` fetches the Build ID. Then
`ln` is used to hard link the file to a directory derived from the Build ID.)
Due to limitation, they are no longer used by Fuchsia and they don't appear to
be used elsewhere (checked with Google Search and Debian Code Search). So delete
them without a transition period.
Announcement: https://lists.llvm.org/pipermail/llvm-dev/2021-February/148446.html
Differential Revision: https://reviews.llvm.org/D96310
As of binutils 2.36, GNU strip calls chown(2) for "sudo strip foo" and
"sudo strip foo -o foo", but no "sudo strip foo -o bar" or "sudo strip
foo -o ./foo". In other words, while "sudo strip foo -o bar" creates a
new file bar with root access, "sudo strip foo" will keep the owner and
group of foo unchanged. Currently llvm-objcopy and llvm-strip behave
differently, always changing the owner and gropu to root. The
discrepancy prevents Chrome OS from migrating to llvm-objcopy and
llvm-strip as they change file ownership and cause intended users/groups
to lose access when invoked by sudo with the following sequence
(recommended in man page of GNU strip).
1.<Link the executable as normal.>
1.<Copy "foo" to "foo.full">
1.<Run "strip --strip-debug foo">
1.<Run "objcopy --add-gnu-debuglink=foo.full foo">
This patch makes llvm-objcopy and llvm-strip follow GNU's behavior.
Link: crbug.com/1108880
This is consistent with BFD objcopy.
Previously llvm objcopy would allocate space for SHT_NOBITS sections
often resulting in enormous binary files.
New test case (binary-paddr.test %t6).
Reviewed By: jhenderson, MaskRay
Differential Revision: https://reviews.llvm.org/D95569
No longer rely on an external tool to build the llvm component layout.
Instead, leverage the existing `add_llvm_componentlibrary` cmake function and
introduce `add_llvm_component_group` to accurately describe component behavior.
These function store extra properties in the created targets. These properties
are processed once all components are defined to resolve library dependencies
and produce the header expected by llvm-config.
Differential Revision: https://reviews.llvm.org/D90848
Alternative to D74755. sectionWithinSegment() treats an empty section as having
a size of 1. Due to the rule, an empty .tdata will not be attributed to an
empty PT_TLS. (The empty p_align=64 PT_TLS is for Android Bionic's TCB
compatibility (ELF-TLS). See https://reviews.llvm.org/D62055#1507426)
Currently --only-keep-debug will not layout a segment with no section
(layoutSegmentsForOnlyKeepDebug()), thus p_offset of PT_TLS can go past the end
of the file. The strange p_offset can trigger validation errors for subsequent
tools, e.g. llvm-objcopy errors when reading back the separate debug file
(readProgramHeaders()).
This patch places such an empty segment according to its parent segment. This
special cases works for the empty PT_TLS used in Android. For a non-empty
segment, it should have at least one non-empty section and will be handled by
the normal code. Note, p_memsz PT_LOAD is rejected by both Linux and FreeBSD.
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D90897
This diff fixes missing fields initialization (Size, VMSize).
Previously this resulted in broken binaries when multiple sections
were added in one tool's invocatation.
Test plan: make check-all
Differential revision: https://reviews.llvm.org/D90690
Some binaries can contain regular sections with zero offset and zero size.
This diff makes llvm-objcopy's handling of such sections consistent with
cctools's strip (which doesn't modify them),
previously the tool would allocate file space for them.
Test plan: make check-all
Differential revision: https://reviews.llvm.org/D90796
This matches behavior GNU objcopy and can simplify clang-offload-bundler
(which currently works around the issue by invoking llvm-objcopy twice).
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D90438
This diff refactors error reporting to make it more clear
what arguments were passed to llvm-install-name-tool.
Test plan: make check-all
Differential revision: https://reviews.llvm.org/D90080
This diff adds the option -prepend_rpath which inserts an rpath as
the first rpath in the binary.
Test plan: make check-all
Differential revision: https://reviews.llvm.org/D89605
This diff refactors the code which determines the tool type based on
how llvm-objcopy is invoked (objcopy vs strip vs bitcode-strip vs install-name-tool).
NFC.
Test plan: make check-all
Differential revision: https://reviews.llvm.org/D89713
LD64 emits string tables which start with a space and a zero byte.
This diff adjusts StringTableBuilder for linked Mach-O binaries to match LD64's behavior.
Test plan: make check-all
Differential revision: https://reviews.llvm.org/D89561
Virtual sections do not contribute to the final output size.
This diff fixes the corresponding calculations in the method MachOWriter::totalSize.
Test plan: make check-all
Differential revision: https://reviews.llvm.org/D89661
This diff adds support for universal binaries to llvm-objcopy.
This is a recommit of 32c8435ef7 with the asan issue fixed.
Test plan: make check-all
Differential revision: https://reviews.llvm.org/D88400
Remove usages of special error reporting functions(error(),
reportError()). Errors are reported as Expected<>/Error returning
values. This part is for ELF subfolder of llvm-objcopy.
Testing: check-all.
Differential Revision: https://reviews.llvm.org/D87987
Remove usages of special error reporting functions(error(),
reportError()). This patch is extracted from D87987.
Errors are reported as Expected<>/Error returning values.
This part is for COFF subfolder of llvm-objcopy.
Testing: check-all.
Differential Revision: https://reviews.llvm.org/D88213
Remove usages of special error reporting functions(error(),
reportError()). This patch is extracted from D87987.
Errors are reported as Expected<>/Error returning values.
This part is for MachO subfolder of llvm-objcopy.
Testing: check-all.
Reviewed By: jhenderson, alexshap
Differential Revision: https://reviews.llvm.org/D88113
This diff fixes --add-section functionality and simplifies the tests organization.
Test plan: make check-all
Differential revision: https://reviews.llvm.org/D87497
This diff adds llvm-bitcode-strip driver to llvm-objcopy.
In the future this will enable us to build a replacement for the tool bitcode_strip.
Test plan: make check-all
Differential revision: https://reviews.llvm.org/D87212
The code which validates the value of -id is moved into the function parseInstallNameToolOptions.
Test plan: make check-all
Differential revision: https://reviews.llvm.org/D87855
`ELFFile<ELFT>` has many methods that take pointers,
though they assume that arguments are never null and
hence could take references instead.
This patch performs such clean-up.
Differential revision: https://reviews.llvm.org/D87385
This diff adds -V alias for --version to make llvm-install-name-tool
consistent with other tools (llvm-objcopy, llvm-strip, etc).
Test plan: make check-all
Differential revision: https://reviews.llvm.org/D87264
Add symlinks for `llvm-libtool-darwin` and
`llvm-install-name-tool`.
Reviewed by jhenderson, smeenai
Differential Revision: https://reviews.llvm.org/D85054
This diff merges all options for llvm-install-name-tool under a single
function processLoadCommands. Also adds another test case for -add_rpath
option.
Test plan: make check-all
Reviewed by: jhenderson, alexshap, smeenai, Ktwu
Differential Revision: https://reviews.llvm.org/D82812
This diff merges all options for llvm-install-name-tool under a single
function processLoadCommands. Also adds another test case for -add_rpath
option.
Test plan: make check-all
Reviewed by: jhenderson, alexshap, smeenai, Ktwu
Differential Revision: https://reviews.llvm.org/D82812
This patch resolves crash that occurs when user wanted to remove all
symbols and add a brand new one using:
```
llvm-objcopy -R .symtab --add-symbol foo=1234 in.o out.o
```
Before these changes the symbol table internally being null when adding
new symbols. For now we will regenerate symtab in this case.
This fixes: https://bugs.llvm.org/show_bug.cgi?id=43930
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D82935
Implement `-change` option for install-name-tool. The behavior exactly
matches that of cctools. Depends on D82410.
Reviewed By: jhenderson, smeenai
Differential Revision: https://reviews.llvm.org/D82613
Implement `-id` option for install-name-tool. Differences from cctool's
behavior:
- Does **NOT** throw an error if multiple -id options are specified.
Instead, picks the last one.
- Throws an error in case empty id is specified.
Reviewed By: jhenderson, smeenai
Differential Revision: https://reviews.llvm.org/D82410
This diff addresses the old TODO in MachOObjcopy.cpp and
correctly sets the page size used for alignment of segments.
In particular, now llvm-objcopy's output is consistent
with the input (the alignment of vmsize doesn't change).
Test plan:
1. make check-all
2. verify that a binary copied via llvm-objcopy now correctly works on iOS.
Differential revision: https://reviews.llvm.org/D82503
SHT_GROUP sections contain a reference to a symbol indicating their
"signature" symbol. The symbol table containing this symbol is referred
to by the group section's sh_link field. If llvm-objcopy is instructed
to remove the symbol table, it will emit an error.
This fixes https://bugs.llvm.org/show_bug.cgi?id=46153.
Reviewed By: jhenderson, Higuoxing
Differential Revision: https://reviews.llvm.org/D82274
This diff implements -rpath option for llvm-install-name-tool
which replaces the rpath value in the specified Mach-O binary.
Patch by Sameer Arora!
Test plan: make check-all
Differential revision: https://reviews.llvm.org/D82051
This diff updates the help messages for llvm-objcopy, llvm-strip and
llvm-install-name-tool.
Patch by Sameer Arora!
Test plan: make check-all
Differential revision: https://reviews.llvm.org/D81907
This diff adds support for deleting an rpath from a Mach-O binary.
Patch by Sameer Arora!
Test plan: make check-all
Differential revision: https://reviews.llvm.org/D81527
This diff adds support for copying binaries
containing a LC_CODE_SIGNATURE load command.
Test plan: make check-all
Differential revision: https://reviews.llvm.org/D81768
Fix the calculation of the field cmdsize (in the function buildRPathLoadCommand)
to account for the null byte terminator.
Patch by Sameer Arora!
Test plan: make check-all
Differential revision: https://reviews.llvm.org/D81575
Reorder `DumpSection` under `handleArgs` in file `MachOObjcopy.cpp`. The
operation to dump a section is now performed before both add and remove
section operations for MachO file format.
Change for the ELF format at D81097. Together fixes
https://bugs.llvm.org/show_bug.cgi?id=44283
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D81123
Reorder `DumpSection` under `handleArgs` in file `ELFObjcopy.cpp`.
`DumpSection` is placed before `replaceAndRemoveSections` and is
therefore now the first operation under `handleArgs`. Thus, it is now
performed before both `add` and `remove` section operations.
Change for the MachO format at D81123. Together fixes https://bugs.llvm.org/show_bug.cgi?id=44283.
Reviewed By: alexshap, jhenderson, MaskRay
Differential Revision: https://reviews.llvm.org/D81097
When a group member is removed, the corresponding record in the
SHT_GROUP section has to be deleted.
This fixes PR46064.
Differential Revision: https://reviews.llvm.org/D80568
When a SHT_GROUP section is removed, but other sections of the group are
kept, the SHF_GROUP flag of these sections should be dropped, otherwise
the resulting ELF file will be malformed.
Differential Revision: https://reviews.llvm.org/D80511
cctools strip has the option "-T" which removes Swift symbols.
This diff implements this option in llvm-strip for MachO.
Test plan: make check-all
Differential revision: https://reviews.llvm.org/D80099
I noticed that std::error_code() does one-time initialization. Avoid
that overhead with Expected<T> and llvm::Error. Also, it is consistent
with the virtual interface and ELF, and generally cleaner.
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D79643
To avoid undefined behavior caught by -fsanitize=undefined on binary-paddr.test
void SectionWriter::visit(const Section &Sec) {
if (Sec.Type != SHT_NOBITS)
// Sec.Contents is empty while Sec.Offset may be out of bound
llvm::copy(Sec.Contents, Out.getBufferStart() + Sec.Offset);
}
N_PEXT bit should not affect whether a symbol is considered to be external or not.
This also fixes the construction of the symbol table since it relies on the correct
ordering of symbols.
Test plan: make check-all
Differential revision: https://reviews.llvm.org/D78888
After SHF_ALLOC sections are ordered by LMA:
* If initial sections are empty, GNU objcopy skips their contents while we
emit leading zeros. (binary-paddr.test %t4)
* If trailing sections are empty, GNU objcopy skips their contents while we
emit trailing zeros. (binary-paddr.test %t5)
This patch matches GNU objcopy's behavior. Linkers don't keep p_memsz
PT_LOAD segments. Such empty sections would not have a containing
PT_LOAD and `Section::ParentSegment` might be null if linkers fail to
optimize the file offsets (lld D79254).
In particular, without D79254, the arm Linux kernel's multi_v5_defconfig
depends on this behavior: in `vmlinux`, an empty .text_itcm is mapped at
a very high address (0xfffe0000) but the kernel does not expect
`objcopy -O binary` to create a very large `arch/arm/boot/Image`
(0xfffe0000-0xc0000000 ~= 1GiB). See https://bugs.llvm.org/show_bug.cgi?id=45632
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D79229
The debug directory payload is not located directly after the
debug directory entry itself, but can essentially be located anywhere
in the binary (even outside of mapped sections, although we don't
handle that case).
Differential Revision: https://reviews.llvm.org/D78921
Fix handling of relocations with r_extern == 0.
If r_extern == 0 then r_symbolnum is an index of a section rather than a symbol index.
Patch by Seiya Nuta and Alexander Shaposhnikov.
Test plan: make check-all
Differential revision: https://reviews.llvm.org/D78946
This diff fixes the calculation of the field vmsize
in LC_SEGMENT/LC_SEGMENT_64 load commands.
Test plan: make check-all
Differential revision: https://reviews.llvm.org/D78799
LC_LOAD_WEAK_DYLIB is analogous to LC_LOAD_DYLIB and doesn't require any special handling.
Test plan: make check-all
Differential revision: https://reviews.llvm.org/D78602
Make --remove-section clean up dead symbols, return an Error if it can't be safely done.
Test plan: make check-all
Differential revision: https://reviews.llvm.org/D78474
Don't error on Config.KeepFileSymbols for COFF and Mach-O.
Original description:
GNU objcopy removes STT_FILE symbols for strip-debug operations, and
keeps them for --discard-all operation. Match their behaviour for
llvm-objcopy.
Bug: https://github.com/android/ndk/issues/1212
Differential Revision: https://reviews.llvm.org/D76675
The header magic value is held in the native endianness, so the method used in
cc0ec3fdb9. Use MachOReader / MachOWriter's existing endianness tests instead.
Use shift/mask operations to access r_symbolnum rather than relying on
MachO::relocation_info. This should fix the big-endian bot failures that were
caused by 386f1c114d.
It can be used to avoid passing the begin and end of a range.
This makes the code shorter and it is consistent with another
wrappers we already have.
Differential revision: https://reviews.llvm.org/D78016
Alexey Lapshin