This patch adds statistics about the contribution of each object file to
the linked debug info. When --statistics is passed to dsymutil, it
prints a table after linking as illustrated below.
It lists the object file name, the size of the debug info in the object
file in bytes, and the absolute size contribution to the linked dSYM and
the percentage difference. The table is sorted by the output size, so
the object files contributing the most to the link are listed first.
.debug_info section size (in bytes)
-------------------------------------------------------------------------------
Filename Object dSYM Change
-------------------------------------------------------------------------------
basic2.macho.x86_64.o 210b 165b -24.00%
basic3.macho.x86_64.o 177b 150b -16.51%
basic1.macho.x86_64.o 125b 129b 3.15%
-------------------------------------------------------------------------------
Total 512b 444b -14.23%
-------------------------------------------------------------------------------
Differential revision: https://reviews.llvm.org/D79513
Summary:
Current implementation of DWARFDie::getName(DINameKind Kind) could
lead to double call to DWARFDie::find(DW_AT_name) in following
scenario:
getName(LinkageName);
getName(ShortName);
getName(LinkageName) calls find(DW_AT_name) if linkage name is not
found. Then, it is called again in getName(ShortName). This patch
alows to request LinkageName and ShortName separately
to avoid extra call to find(DW_AT_name).
It helps D74169 to parse clang debuginfo faster(~1%).
Reviewers: clayborg, dblaikie
Differential Revision: https://reviews.llvm.org/D79173
to remap object file paths (but no source paths) before
processing. This is meant to be used for Clang objects where the
module cache location was remapped using ``-fdebug-prefix-map``; to
help dsymutil find the Clang module cache.
<rdar://problem/55685132>
Differential Revision: https://reviews.llvm.org/D76391
Record the address of a tail-calling branch instruction within its call
site entry using DW_AT_call_pc. This allows a debugger to determine the
address to use when creating aritificial frames.
This creates an extra attribute + relocation at tail call sites, which
constitute 3-5% of all call sites in xnu/clang respectively.
rdar://60307600
Differential Revision: https://reviews.llvm.org/D76336
Summary:
This is a preparatory change for allowing LLVM to emit DW_OP_convert
operations converting to the generic type.
If DW_OP_convert's operand is 0, it converts the top of the stack to the
generic type, as specified by DWARFv5 section 2.5.1.6:
"[...] takes one operand, which is an unsigned LEB128 integer that
represents the offset of a debugging information entry in the current
compilation unit, or value 0 which represents the generic type."
This adds support for such operations to dsymutil.
Reviewers: aprantl, markus, friss, JDevlieghere
Reviewed By: aprantl, JDevlieghere
Subscribers: hiraditya, llvm-commits
Tags: #debug-info, #llvm
Differential Revision: https://reviews.llvm.org/D76142
Currently dsymutil when generating accelerator tables will attempt to strip the template parameters from names for subroutines.
For some overload operators which contain < in their names e.g. operator< the current method ends up stripping the operator name as well,
we just end up with the name operator in the table for each case.
Differential Revision: https://reviews.llvm.org/D75545
Summary:
DWARFContext has all the required information to access source debug info.
It is not necessary to use "const object::ObjectFile" to create DWARFContext.
Thus this patch removes all usages of "const object::ObjectFile"
from DWARFLinker. Instead, already created DWARFContext is passed
to DWARFLinker. The purpose is to not depend on "const object::ObjectFile".
The patch looks big, but most of changes are renamings and movements.
Testing: it passes "check-all" lit testing. MD5 checksum for clang .dSYM bundle
matches for the dsymutil with/without that patch.
Reviewers: JDevlieghere, friss, dblaikie, aprantl
Reviewed By: JDevlieghere
Subscribers: hiraditya, llvm-commits
Tags: #llvm, #debug-info
Differential Revision: https://reviews.llvm.org/D75029
This patch fixes a bug that would cause dsymutil to collect
.swiftinterface files for the Swift stdlib and other SDK
modules. There is no advantage in copying these since they should be
loaded from the ones bundled with LLDB's embedded Swift compiler
instead and copying them will cause LLDB to recompile them from source
instead of loading their prebuilt cached counterparts in the SDK.
rdar://problem/57463247
Differential Revisions: https://reviews.llvm.org/D75196
Summary:
this review is extracted from D74308.
It creates two error handlers which allow to redefine error
reporting routine and should be used for all places
where errors are reported:
std::function<void(Error)> RecoverableErrorHandler = defaultErrorHandler;
std::function<void(Error)> WarningHandler = defaultWarningHandler;
It also creates accessors to above handlers which should be used to
report errors.
function_ref<void(Error)> getRecoverableErrorHandler() {
return RecoverableErrorHandler;
}
function_ref<void(Error)> getWarningHandler() { return WarningHandler; }
It patches all error reporting places inside DWARFContext and DWARLinker.
Reviewers: jhenderson, dblaikie, probinson, aprantl, JDevlieghere
Reviewed By: jhenderson, JDevlieghere
Subscribers: hiraditya, llvm-commits
Tags: #llvm, #debug-info
Differential Revision: https://reviews.llvm.org/D74481
The goal of this patch is to maximize CPU utilization on multi-socket or high core count systems, so that parallel computations such as LLD/ThinLTO can use all hardware threads in the system. Before this patch, on Windows, a maximum of 64 hardware threads could be used at most, in some cases dispatched only on one CPU socket.
== Background ==
Windows doesn't have a flat cpu_set_t like Linux. Instead, it projects hardware CPUs (or NUMA nodes) to applications through a concept of "processor groups". A "processor" is the smallest unit of execution on a CPU, that is, an hyper-thread if SMT is active; a core otherwise. There's a limit of 32-bit processors on older 32-bit versions of Windows, which later was raised to 64-processors with 64-bit versions of Windows. This limit comes from the affinity mask, which historically is represented by the sizeof(void*). Consequently, the concept of "processor groups" was introduced for dealing with systems with more than 64 hyper-threads.
By default, the Windows OS assigns only one "processor group" to each starting application, in a round-robin manner. If the application wants to use more processors, it needs to programmatically enable it, by assigning threads to other "processor groups". This also means that affinity cannot cross "processor group" boundaries; one can only specify a "preferred" group on start-up, but the application is free to allocate more groups if it wants to.
This creates a peculiar situation, where newer CPUs like the AMD EPYC 7702P (64-cores, 128-hyperthreads) are projected by the OS as two (2) "processor groups". This means that by default, an application can only use half of the cores. This situation could only get worse in the years to come, as dies with more cores will appear on the market.
== The problem ==
The heavyweight_hardware_concurrency() API was introduced so that only *one hardware thread per core* was used. Once that API returns, that original intention is lost, only the number of threads is retained. Consider a situation, on Windows, where the system has 2 CPU sockets, 18 cores each, each core having 2 hyper-threads, for a total of 72 hyper-threads. Both heavyweight_hardware_concurrency() and hardware_concurrency() currently return 36, because on Windows they are simply wrappers over std:🧵:hardware_concurrency() -- which can only return processors from the current "processor group".
== The changes in this patch ==
To solve this situation, we capture (and retain) the initial intention until the point of usage, through a new ThreadPoolStrategy class. The number of threads to use is deferred as late as possible, until the moment where the std::threads are created (ThreadPool in the case of ThinLTO).
When using hardware_concurrency(), setting ThreadCount to 0 now means to use all the possible hardware CPU (SMT) threads. Providing a ThreadCount above to the maximum number of threads will have no effect, the maximum will be used instead.
The heavyweight_hardware_concurrency() is similar to hardware_concurrency(), except that only one thread per hardware *core* will be used.
When LLVM_ENABLE_THREADS is OFF, the threading APIs will always return 1, to ensure any caller loops will be exercised at least once.
Differential Revision: https://reviews.llvm.org/D71775
When the DW_AT_call_return_pc matches a relocation, the call return pc
would get relocated twice, once because of the relocation in the object
file and once because of dsymutil. The same problem exists for the low
and high PC and the fix is the same. We remember the low, high and
return pc of the original DIE and relocate that, rather than the
potentially already relocated value.
Reviewed offline by Fred Riss.
This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.
This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.
This doesn't actually modify StringRef yet, I'll do that in a follow-up.
The Version was used only to determine the size of an operand of
DW_OP_call_ref. The size was 4 for all versions apart from 2, but
the DW_OP_call_ref operation was introduced only in DWARF3. Thus,
the code may be simplified and using of Version may be eliminated.
Differential Revision: https://reviews.llvm.org/D73264
Summary:
The primary goal of this refactoring is to separate DWARF optimizing part.
So that it could be reused by linker or by any other client.
There was a thread on llvm-dev discussing the necessity of such a refactoring:
http://lists.llvm.org/pipermail/llvm-dev/2019-September/135068.html.
This is a final part from series of patches for dsymutil.
Previous patches : D71068, D71839, D72476. This patch:
1. Creates lib/DWARFLinker interface :
void addObjectFile(DwarfLinkerObjFile &ObjFile);
bool link();
void setOptions;
1. Moves all linking logic from tools/dsymutil/DwarfLinkerForBinary
into lib/DWARFLinker.
2. Renames RelocationManager into AddressesManager.
3. Remarks creation logic moved from separate parallel execution
into object file loading routine.
Testing: it passes "check-all" lit testing. MD5 checksum for clang .dSYM bundle
matches for the dsymutil with/without that patch.
Reviewers: JDevlieghere, friss, dblaikie, aprantl, jdoerfert
Reviewed By: JDevlieghere
Subscribers: merge_guards_bot, hiraditya, jfb, llvm-commits, probinson, thegameg
Tags: #llvm, #debug-info
Differential Revision: https://reviews.llvm.org/D72915
Summary:
This is the next portion of patches for dsymutil.
Create DwarfEmitter interface to generate all debug info tables.
Put DwarfEmitter into DwarfLinker library and make tools/dsymutil/DwarfStreamer
to be child of DwarfEmitter.
It passes check-all testing. MD5 checksum for clang .dSYM bundle matches
for the dsymutil with/without that patch.
Reviewers: JDevlieghere, friss, dblaikie, aprantl
Reviewed By: JDevlieghere
Subscribers: merge_guards_bot, hiraditya, thegameg, probinson, llvm-commits
Tags: #llvm, #debug-info
Differential Revision: https://reviews.llvm.org/D72476
Summary:
This patch relands D71271. The problem with D71271 is that it has cyclic dependency:
CodeGen->AsmPrinter->DebugInfoDWARF->CodeGen. To avoid cyclic dependency this patch
puts implementation for DWARFOptimizer into separate library: lib/DWARFLinker.
Thus the difference between this patch and D71271 is in that DWARFOptimizer renamed into
DWARFLinker and it`s files are put into lib/DWARFLinker.
Reviewers: JDevlieghere, friss, dblaikie, aprantl
Reviewed By: JDevlieghere
Subscribers: thegameg, merge_guards_bot, probinson, mgorny, hiraditya, llvm-commits
Tags: #llvm, #debug-info
Differential Revision: https://reviews.llvm.org/D71839
Jonas Devlieghere