The Mach-O object file format is limited to 4GB because its used of
32-bit offsets in the header. It is possible for dsymutil to (silently)
emit an invalid binary. Instead of having consumers deal with this, emit
an error instead.
Add a flag to change dsymutil's behavior and force a static variable to
keep its enclosing function. The test shows a situation where that could
be useful. I'm not convinced this behavior makes sense as a default,
which is why it's behind a flag.
rdar://74918374
Differential revision: https://reviews.llvm.org/D101337
Consider the .debug_pubnames and .debug_pubtypes their own kind of
accelerator and stop emitting them together with the Apple-style
accelerator tables. The only reason we were still emitting both was for
(byte-for-byte) compatibility with dsymutil-classic.
- This patch adds a new accelerator table kind "Pub" which can be
specified with --accelerator=Pub.
- This patch removes the ability to emit both pubnames/types and apple
style accelerator tables. I don't think anyone is relying on that but
it's worth pointing out.
- This patch removes the --minimize option and makes this behavior the
default. Specifying the flag will result in a warning but won't abort
the program.
Differential revision: https://reviews.llvm.org/D99907
Problem:
On SystemZ we need to open text files in text mode. On Windows, files opened in text mode adds a CRLF '\r\n' which may not be desirable.
Solution:
This patch adds two new flags
- OF_CRLF which indicates that CRLF translation is used.
- OF_TextWithCRLF = OF_Text | OF_CRLF indicates that the file is text and uses CRLF translation.
Developers should now use either the OF_Text or OF_TextWithCRLF for text files and OF_None for binary files. If the developer doesn't want carriage returns on Windows, they should use OF_Text, if they do want carriage returns on Windows, they should use OF_TextWithCRLF.
So this is the behaviour per platform with my patch:
z/OS:
OF_None: open in binary mode
OF_Text : open in text mode
OF_TextWithCRLF: open in text mode
Windows:
OF_None: open file with no carriage return
OF_Text: open file with no carriage return
OF_TextWithCRLF: open file with carriage return
The Major change is in llvm/lib/Support/Windows/Path.inc to only set text mode if the OF_CRLF is set.
```
if (Flags & OF_CRLF)
CrtOpenFlags |= _O_TEXT;
```
These following files are the ones that still use OF_Text which I left unchanged. I modified all these except raw_ostream.cpp in recent patches so I know these were previously in Binary mode on Windows.
./llvm/lib/Support/raw_ostream.cpp
./llvm/lib/TableGen/Main.cpp
./llvm/tools/dsymutil/DwarfLinkerForBinary.cpp
./llvm/unittests/Support/Path.cpp
./clang/lib/StaticAnalyzer/Core/HTMLDiagnostics.cpp
./clang/lib/Frontend/CompilerInstance.cpp
./clang/lib/Driver/Driver.cpp
./clang/lib/Driver/ToolChains/Clang.cpp
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D99426
Fix spurious warnings for missing symbols with thinLTO. The latter
appends a unique suffix to avoid collisions for exported private
symbols, resulting in dsymutil complaining it couldn't find the symbol
in the object file.
rdar://75434058
Differential revision: https://reviews.llvm.org/D99125
Current dsymutil implementation of hasLiveMemoryLocation()/hasLiveAddressRange()
and applyValidRelocs() assume that calls should be done in certain order
(from first Dies to last). Multi-thread implementation might call these methods
in other order(it might process compilation units in order other than they are physically
located), so we remove restriction that searching for relocations should be done
in ascending order. This change does not introduce noticable performance degradation.
The testing results for clang binary:
golden-dsymutil/dsymutil 23787992
clang MD5: 5efa8fd9355ebf81b65f24db5375caa2
elapsed time=91sec
build-Release/bin/dsymutil 23855616
clang MD5: 5efa8fd9355ebf81b65f24db5375caa2
elapsed time=91sec
Differential Revision: https://reviews.llvm.org/D93106
The modification time in the debug map is expressed using second
precision, while the modification time returned by the filesystem could
be more precise. Avoid spurious warnings about timestamp mismatches by
truncating the modification time reported by the system to seconds.
This re-lands e5553b9a6a with two small fixes to the tests:
- Don't touch the source directory in debug-map-parsing.test but
instead copy everything over in a temporary directory in
timestamp-mismatch.test.
- Don't redirect stderr to stdout to avoid the output getting
intertwined in extern-alias.test.
This reverts commit e5553b9a6a.
Tests are not allowed to modify the source. Please figure out a way to
use %t rather than dynamically modifying the inputs.
Currently dsymutil will silently fail when processing binaries with
Dwarf 5 debug info. This patch adds rudimentary support for Dwarf 5 in
dsymutil.
- Recognize relocations in the debug_addr section.
- Recognize (a subset of) Dwarf 5 form values.
- Emits valid Dwarf 5 compile unit header chains.
To simplify things (and avoid having to emit indexed sections) I decided
to emit the relocated addresses directly in the debug info section.
- DW_FORM_strx gets relocated and rewritten to DW_FORM_strp
- DW_FORM_addrx gets relocated and rewritten to DW_FORM_addr
Obviously there's a lot of work left, but this should be a step in the
right direction.
rdar://62345491
Differential revision: https://reviews.llvm.org/D94323
Copy over the __eh_frame from the binary into the dSYM. This helps
kernel developers that are working with only dSYMs (i.e. no binaries)
when debugging a core file. This only kicks in when the __eh_frame
exists in the linked binary. Most of the time ld64 will remove the
section in favor of compact unwind info. When it is emitted, it's
generally small enough and should not bloat the dSYM.
rdar://69774935
Differential revision: https://reviews.llvm.org/D94460
Add a warning when the timestmap doesn't match between the object file
and the debug map entry. We were already emitting such warnings for
archive members and swift interface files. This patch also unifies the
warning across all three.
rdar://65614640
Differential revision: https://reviews.llvm.org/D94536
Current interface of AddressMap assumes that relocations exist.
That is correct for not-linked object file but is not correct
for linked executable. This patch changes interface in such way
that AddressMap could be used not only with not-linked object files:
hasValidRelocationAt()
replaced with:
hasLiveMemoryLocation()
hasLiveAddressRange()
Differential Revision: https://reviews.llvm.org/D87723
No longer rely on an external tool to build the llvm component layout.
Instead, leverage the existing `add_llvm_componentlibrary` cmake function and
introduce `add_llvm_component_group` to accurately describe component behavior.
These function store extra properties in the created targets. These properties
are processed once all components are defined to resolve library dependencies
and produce the header expected by llvm-config.
Differential Revision: https://reviews.llvm.org/D90848
Make these types conform to the LLVM Coding Standards:
> Type names (including classes, structs, enums, typedefs, etc) should
> be nouns and start with an upper-case letter.
dsymutil was incorrectly ignoring aliases to private extern symbols in
the MachODebugMapParser. This resulted in spurious warnings about not
being able to find symbols.
rdar://49652389
Differential revision: https://reviews.llvm.org/D89444
Rename the DwarfFile class in DWARFLinker to DWARFFile. This is
consistent with the other DWARF classes and avoids a ODR violation with
the DwarfFile class in AsmPrinter.
- Fix a memory leak accidentally introduced yesterday by using CodeGen's
existing mangling context instead of creating a new context afresh.
- Move GNU-runtime ObjC method mangling into the AST mangler; this will
eventually be necessary to support direct methods there, but is also
just the right architecture.
- Make the Apple-runtime method mangling work properly when given an
interface declaration, fixing a bug (which had solidified into a test)
where mangling a category method from the interface could cause it to
be mangled as if the category name was a class name. (Category names
are namespaced within their class and have no global meaning.)
- Fix a code cross-reference in dsymutil.
Based on a patch by Ellis Hoag.
Treat N_AST symbol table entries like other debug entries and don't emit
them in the linked binary.
Differential revision: https://reviews.llvm.org/D81205
Reproducer.cpp:70:12: error: could not convert ‘Repro’ from
‘std::unique_ptr<llvm::dsymutil::ReproducerGenerate,
std::default_delete<llvm::dsymutil::ReproducerGenerate> >’ to
‘llvm::Expected<std::unique_ptr<llvm::dsymutil::Reproducer> >’
Add support for generating a dsymutil reproducer. The result is a folder
containing all the object files for linking.
When --gen-reproducer is passed, dsymutil uses a FileCollectorFileSystem
which keeps track of all the files used by dsymutil. These files are
copied into a temporary directory when dsymutil exists.
When this path is passed to --use-reproducer, dsymutil uses a
RedirectingFileSystem that will use the files from the reproducer
directory instead of the actual paths. This means you don't need to mess
with the OSO path prefix.
Differential revision: https://reviews.llvm.org/D79398
This patch adds statistics about the contribution of each object file to
the linked debug info. When --statistics is passed to dsymutil, it
prints a table after linking as illustrated below.
It lists the object file name, the size of the debug info in the object
file in bytes, and the absolute size contribution to the linked dSYM and
the percentage difference. The table is sorted by the output size, so
the object files contributing the most to the link are listed first.
.debug_info section size (in bytes)
-------------------------------------------------------------------------------
Filename Object dSYM Change
-------------------------------------------------------------------------------
basic2.macho.x86_64.o 210b 165b -24.00%
basic3.macho.x86_64.o 177b 150b -16.51%
basic1.macho.x86_64.o 125b 129b 3.15%
-------------------------------------------------------------------------------
Total 512b 444b -14.23%
-------------------------------------------------------------------------------
Differential revision: https://reviews.llvm.org/D79513
This patch threads the virtual file system through dsymutil.
Currently there is no good way to find out exactly what files are
necessary in order to reproduce a dsymutil link, at least not without
knowledge of how dsymutil's internals. My motivation for this change is
to add lightweight "reproducers" that automatically gather the input
object files through the FileCollectorFileSystem. The files together
with the YAML mapping will allow us to transparently reproduce a
dsymutil link, even without having to mess with the OSO path prefix.
Differential revision: https://reviews.llvm.org/D79376
Summary:
In D77860, we have changed `getSymbolFlags()` return type to `Expected<uint32_t>`.
This change helps bubble the error further up the stack.
Reviewers: jhenderson, grimar, JDevlieghere, MaskRay
Reviewed By: jhenderson
Subscribers: hiraditya, MaskRay, rupprecht, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D79075
As reported here: https://reviews.llvm.org/D75153#1987272
Before, each instance of llvm-cov was creating one thread per hardware core, which wasn't needed probably because the number of inputs were small. This was probably causing a thread rlimit issue on large core count systems.
After this patch, the previous behavior is restored (to what was before rG8404aeb5):
If --num-threads is not specified, we create one thread per input, up to num.cores.
When specified, --num-threads indicates any number of threads, with no upper limit.
Differential Revision: https://reviews.llvm.org/D78408
For implementing "remove obsolete debug info in lld", it is neccesary
to have DWARF generation code implementation. dsymutil uses DwarfStreamer
for that purpose. DwarfStreamer uses AsmPrinter. It is considered OK
to use AsmPrinter based code in lld(D74169). This patch moves
DwarfStreamer implementation into DWARFLinker, so that it could be reused
from lld.
Generally, a better place for such a common DWARF generation code would be
not DWARFLinker but an additional separate library. Such a library could
contain a single version of DWARF generation routines and could also
be independent of AsmPrinter. At the current moment, DwarfStreamer
does not pretend to be such a general implementation of DWARF generation.
So I decided to put it into DWARFLinker since it is the only user
of DwarfStreamer.
Testing: it passes "check-all" lit testing. MD5 checksum for clang .dSYM
bundle matches for the dsymutil with/without that patch.
Reviewed By: JDevlieghere
Differential revision: https://reviews.llvm.org/D77169
to remap object file paths (but no source paths) before
processing. This is meant to be used for Clang objects where the
module cache location was remapped using ``-fdebug-prefix-map``; to
help dsymutil find the Clang module cache.
<rdar://problem/55685132>
Differential Revision: https://reviews.llvm.org/D76391
MCTargetOptionsCommandFlags.inc and CommandFlags.inc are headers which contain
cl::opt with static storage.
These headers are meant to be incuded by tools to make it easier to parametrize
codegen/mc.
However, these headers are also included in at least two libraries: lldCommon
and handle-llvm. As a result, when creating DYLIB, clang-cpp holds a reference
to the options, and lldCommon holds another reference. Linking the two in a
single executable, as zig does[0], results in a double registration.
This patch explores an other approach: the .inc files are moved to regular
files, and the registration happens on-demand through static declaration of
options in the constructor of a static object.
[0] https://bugzilla.redhat.com/show_bug.cgi?id=1756977#c5
Differential Revision: https://reviews.llvm.org/D75579
Summary: The wrong variable was being checked for an error, which mean a llvm::Error went unchecked and crashes dsymutil. Discovered this when trying to feed an ELF file to "dsymutil --update" and running into the crash.
Reviewers: aprantl, JDevlieghere
Subscribers: llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D75777
Fangrui Song