On macOS Catalina, calling objc_debug_class_getNameRaw on some of the
ISA pointers returns NULL, causing us to crash and unwind before reading
all the Objective-C classes. This does not happen on macOS Big Sur.
Account for that possibility and skip the class when that happens.
Fill out ProcessMachCore::DoLoadCore to handle LC_NOTE hints with
a UUID or with a UUID+address, and load the binary at the specified
offset correctly. Add tests for all four combinations. Change
DynamicLoaderStatic to not re-set a Section's load address in the
Target if it's already been specified.
Differential Revision: https://reviews.llvm.org/D99571
rdar://51490545
Inline callstacks were being incorrectly displayed in the results of "image lookup --address". The deepest frame wasn't displaying the line table line entry, it was always showing the inline information's call file and line on the previous frame. This is now fixed and has tests to make sure it doesn't regress.
Differential Revision: https://reviews.llvm.org/D98761
Since quite a while Apple's LLDB fork (that contains the Swift debugging
support) is randomly crashing in `CommandLineParser::addOption` with an error
such as `CommandLine Error: Option 'h' registered more than once!`
The backtrace of the crashing thread is shown below. There are also usually many
other threads also performing similar clang::FrontendActions which are all
trying to generate (usually outdated) Clang modules which are used by Swift for
various reasons.
```
[ 6] LLDB`CommandLineParser::addOption(llvm:🆑:Option*, llvm:🆑:SubCommand*) + 856
[ 7] LLDB`CommandLineParser::addOption(llvm:🆑:Option*, llvm:🆑:SubCommand*) + 733
[ 8] LLDB`CommandLineParser::addOption(llvm:🆑:Option*, bool) + 184
[ 9] LLDB`llvm:🆑:ParseCommandLineOptions(...) [inlined] ::CommandLineParser::ParseCommandLineOptions(... + 1279
[ 9] LLDB`llvm:🆑:ParseCommandLineOptions(...) + 497
[ 10] LLDB`setCommandLineOpts(clang::CodeGenOptions const&) + 416
[ 11] LLDB`EmitAssemblyHelper::EmitAssemblyWithNewPassManager(...) + 98
[ 12] LLDB`clang::EmitBackendOutput(...) + 4580
[ 13] LLDB`PCHContainerGenerator::HandleTranslationUnit(clang::ASTContext&) + 871
[ 14] LLDB`clang::MultiplexConsumer::HandleTranslationUnit(clang::ASTContext&) + 43
[ 15] LLDB`clang::ParseAST(clang::Sema&, bool, bool) + 579
[ 16] LLDB`clang::FrontendAction::Execute() + 74
[ 17] LLDB`clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) + 1808
```
The underlying reason for the crash is that the CommandLine code in LLVM isn't
thread-safe and will never be thread-safe with its current architecture. The way
LLVM's CommandLine logic works is that all parts of the LLVM can provide command
line arguments by defining `cl::opt` global variables and their constructors
(which are invoked during static initialisation) register the variable in LLVM's
CommandLineParser (which is also just a global variable). At some later point
after static initialization we actually try to parse command line arguments and
we ask the CommandLineParser to parse our `argv`. The CommandLineParser then
lazily constructs it's internal parsing state in a non-thread-safe way (this is
where the crash happens), parses the provided command line and then goes back to
the respective `cl::opt` global variables and sets their values according to the
parse result.
As all of this is based on global state, this whole mechanism isn't thread-safe
so the only time to ever use it is when we know we only have one active thread
dealing with LLVM logic. That's why nearly all callers of
`llvm:🆑:ParseCommandLineOptions` are at the top of the `main` function of the
some LLVM-based tool. One of the few exceptions to this rule is in the
`setCommandLineOpts` function in `BackendUtil.cpp` which is in our backtrace:
```
static void setCommandLineOpts(const CodeGenOptions &CodeGenOpts) {
SmallVector<const char *, 16> BackendArgs;
BackendArgs.push_back("clang"); // Fake program name.
if (!CodeGenOpts.DebugPass.empty()) {
BackendArgs.push_back("-debug-pass");
BackendArgs.push_back(CodeGenOpts.DebugPass.c_str());
}
if (!CodeGenOpts.LimitFloatPrecision.empty()) {
BackendArgs.push_back("-limit-float-precision");
BackendArgs.push_back(CodeGenOpts.LimitFloatPrecision.c_str());
}
BackendArgs.push_back(nullptr);
llvm:🆑:ParseCommandLineOptions(BackendArgs.size() - 1,
BackendArgs.data());
}
```
This is trying to set `cl::opt` variables in the LLVM backend to their right
value as the passed via CodeGenOptions by invoking the CommandLine parser. As
this is just in some generic Clang CodeGen code (where we allow having multiple
threads) this is code is clearly wrong. If we're unlucky it either overwrites
the value of the global variables or it causes the CommandLine parser to crash.
So the next question is why is this only crashing in LLDB? The main reason seems
to be that easiest way to crash this code is to concurrently enter the initial
CommandLineParser construction where it tries to collect all the registered
`cl::opt` options and checks for sanity:
```
// If it's a DefaultOption, check to make sure it isn't already there.
if (O->isDefaultOption() &&
SC->OptionsMap.find(O->ArgStr) != SC->OptionsMap.end())
return;
// Add argument to the argument map!
if (!SC->OptionsMap.insert(std::make_pair(O->ArgStr, O)).second) {
errs() << ProgramName << ": CommandLine Error: Option '" << O->ArgStr
<< "' registered more than once!\n";
HadErrors = true;
}
```
The `OptionsMap` here is global variable and if we end up in this code with two
threads at once then two threads at the same time can register an option (such
as 'h') when they pass the first `if` and then we fail with the sanity check in
the second `if`.
After this sanity check and initial setup code the only remaining work is just
parsing the provided CommandLine which isn't thread-safe but at least doesn't
crash in all my attempts at breaking it (as it's usually just reading from the
already generated parser state but not further modifying it). The exception to
this is probably that once people actually specify the options in the code
snippet above we might run into some new interesting ways to crash everything.
To go back to why it's only affecting LLDB: Nearly all LLVM tools I could find
(even if they are using threads) seem to call the CommandLine parser at the
start so they all execute the initial parser setup at a point where there is
only one thread. So once the code above is executed they are mostly safe from
the sanity check crashes. We even have some shady code for the gtest `main` in
`TestMain.cpp` which is why this also doesn't affect unit tests.
The only exception to this rule is ... *drum roll* ... LLDB! it's not using that
CommandLine library for parsing options so it also never ends up calling it in
`main`. So when we end up in the `FrontendAction` code from the backtrace we are
already very deep in some LLDB logic and usually already have several threads.
In a situation where Swift decides to compile a large amount of Clang modules in
parallel we then end up entering this code via several threads. If several
threads reach this code at the same time we end up in the situation where the
sanity-checking code of CommandLine crashes. I have a very reliable way of
demonstrating the whole thing in D99650 (just run the unit test several times,
it usually crashes after 3-4 attempts).
We have several ways to fix this:
1. Make the whole CommandLine mechanism in LLVM thread-safe.
2. Get rid of `setCommandLineOpts` in `BackendUtil.cpp` and other callers of the
command line parsing in generic Clang code.
3. Initialise the CommandLine library in a safe point in LLDB.
Option 1 is just a lot of work and I'm not even sure where to start. The whole
mechanism is based on global variables and global state and this seems like a
humongous task.
Option 2 is probably the best thing we can do in the near future. There are only
two callers of the command line parser in generic Clang code. The one in
`BackendUtils.cpp` looks like it can be replaced with some reasonable
refactoring (as it only deals with two specific options). There is another one
in `ExecuteCompilerInvocation` which deals with forwarding the generic `-mllvm`
options to the backend which seems like it will just end up requiring us to do
Option 1.
Option 3 is what this patch is doing. We just parse some dummy command line
invocation in a point of the LLDB execution where we only have one thread that
is dealing with LLVM/Clang stuff. This way we are at least prevent the frequent
crashes for users as parsing the dummy command line invocation will set up the
initial parser state safely.
Fixes rdar://70989856
Reviewed By: mib, JDevlieghere
Differential Revision: https://reviews.llvm.org/D99652
The ObjC runtime offers both signed & unsigned tagged pointer value
accessors to tagged pointer providers, but lldb's tagged pointer
code only implemented the unsigned one. This patch adds an
emulation of the signed one.
The motivation for doing this is that NSNumbers use the signed
accessor (they are always signed) and we need to follow that in our
summary provider or we will get incorrect values for negative
NSNumbers.
The data-formatter-objc test file had NSNumber examples (along with lots of other
goodies) but the NSNumber values weren't tested. So I also added
checks for those values to the test.
I also did a quick audit of the other types in that main.m file, and
it looks like pretty much all the other values are either intermediates
or are tested.
Differential Revision: https://reviews.llvm.org/D99694
Respect --apple-sdk <path> if it's specified. If the SDK is simply
mounted from some disk image, and not actually installed, this is the
only way to use it.
Differential Revision: https://reviews.llvm.org/D99746
Debugging tests sometimes involves debugging the Python source. This adds a paragraph to
the "Debugging Test Failures" section about using `pdb`, and also describes how to run
lldb commands from pdb.
Differential Revision: https://reviews.llvm.org/D99744
and probably other posix oses. Use extra_images to ensure
LD_LIBRARY_PATH is set correctly.
Also take the opportunity to remove hand-rolled library extension
management code in favor of the existing one.
The test uses debug info from one binary to debug a different one. This
does not work on macos, and its pure luck that it works elsewhere (the
variable that it inspects happens to have the same address in both).
The purpose of this test is to verify that lldb has not overwritten the
target executable. That can be more easily achieved by checking the exit
code of the binary, so change the test to do that.
Also remove the llgs_test decorator, as it's preventing the test from
running on macos. All the test needs is the platform functionality of
lldb-server, which is available everywhere.
This fixes flakiness in TestVSCode_launch.test_progress_events
vscode.progress_events some times failed to populate in time for
follow up iterations.
Adding a minor delay before the the for the loop fixes the issue.
Reviewed By: clayborg
Differential Revision: https://reviews.llvm.org/D99497
This reverts commit 71b648f715.
There was a typo in the last commit which was causing LLDB AArch64 Linux
buildbot testsuite failures. Now fixed in current version.
This fixes (works around) two errors with gcc-6.5.
- in the RegisterContext_x86 files, gcc is unable to synthesize a
default constructor -- it thinks it needs to initialize the virtual
base class, even though said classes are abstract. I fix that by
providing a dummy constructor.
- In ReproducerInstrumentationTest, it is not able to deduce that the
TestingRegistry class is movable (it contains a map of unique
pointers). I change the type from Optional<TestingRegistry> to
unique_ptr<TestingRegistry), so that moving is not required
(copying/moving a polymorphic type is not a very good idea in any
case).
Remove the remaining references to LLDB_CAPTURE_REPRODUCER. I removed
the functionality in an earlier commit but forgot that there was a
corresponding test and logic to unset it in our test suite.
Consistently use return with EXIT_SUCCESS or EXIT_FAILURE instead of
mix-and-matching return, exit 0, 1 etc.
Differential revision: https://reviews.llvm.org/D99701
Remove the LLDB_CAPTURE_REPRODUCER as it is inherently dangerous. The
reproducers require careful initialization which cannot be guaranteed by
overwriting the reproducer mode at this level.
If we want to provide this functionality, we should do it in the driver
instead. It was originally added to enable capture in CI, but we now
have a dedicated CI job that captures and replays the test suite.
This implements the interactive trace start and stop methods.
This diff ended up being much larger than I anticipated because, by doing it, I found that I had implemented in the beginning many things in a non optimal way. In any case, the code is much better now.
There's a lot of boilerplate code due to the gdb-remote protocol, but the main changes are:
- New tracing packets: jLLDBTraceStop, jLLDBTraceStart, jLLDBTraceGetBinaryData. The gdb-remote packet definitions are quite comprehensive.
- Implementation of the "process trace start|stop" and "thread trace start|stop" commands.
- Implementaiton of an API in Trace.h to interact with live traces.
- Created an IntelPTDecoder for live threads, that use the debugger's stop id as checkpoint for its internal cache.
- Added a functionality to stop the process in case "process tracing" is enabled and a new thread can't traced.
- Added tests
I have some ideas to unify the code paths for post mortem and live threads, but I'll do that in another diff.
Differential Revision: https://reviews.llvm.org/D91679
This patch adds a test case to test AArch64 dynamic register sets.
This tests for the availability of certain register sets and query
their registers accordingly.
Reviewed By: labath, DavidSpickett
Differential Revision: https://reviews.llvm.org/D96463
This patch adds two new dynamic register sets for AArch64 MTE and
Pointer Authentication features. These register sets are dynamic and
will only be available if underlying hardware support either of these
features. LLDB will pull in Aux vector information and create register
infos based on that information.
A follow up patch will add a test case to test these feature registers.
Reviewed By: labath, DavidSpickett
Differential Revision: https://reviews.llvm.org/D96460
This is patch adds support for adding dynamic register sets for
AArch64 dynamic features in LLDB. AArch64 has optional features like
SVE, Pointer Authentication and MTE which means LLDB needs to decide
at run time which registers it needs to pull in for the current
executable based on underlying support for a certain feature.
This patch makes necessary adjustments to make way for dynamic
register infos and dynamic register sets.
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D96458
This patch fixes an issue, where if the thread has a signal blocked when
we try to inject it into the process (via vCont), then instead of
executing straight away, the injected signal will trigger another stop
when the thread unblocks the signal.
As (linux) threads start their life with SIGUSR1 (among others)
disabled, and only enable it during initialization, injecting the signal
during this window did not behave as expected. The fix is to change the
test to ensure the signal gets injected with the signal unblocked.
The simplest way to do this was to write a dedicated inferior for this
test. I also created a new header to factor out the function retrieving
the (os-specific) thread id.
As discussed on lldb-dev
<https://lists.llvm.org/pipermail/lldb-dev/2021-March/016777.html> the
mips code is unmaintained and untested. It also carries a lot of
technical debt which is not limited to mips-specific code.
Generic mips support remains (and is going to be used by the upcoming
freebsd code). Resurrecting mips support should be a matter of re-adding
the relevant register context files (while avoiding reintroducing the
debt).
Add a minimal support for the multiprocess extension in lldb-server.
The server indicates support for it via qSupported, and accepts
thread-ids containing a PID. However, it still does not support
debugging more than one inferior, so any other PID value results
in an error.
Differential Revision: https://reviews.llvm.org/D98482
TestVSCode_disconnect.test_launch fails with clean up error because
disconnect gets called twice once from the test case and once from
the tear down hook.
This patch disables disconnect after its been called from test_launch
Reviewed By: clayborg
Differential Revision: https://reviews.llvm.org/D99491
This is apparently allowed in Objective-C so we should test this in LLDB.
Reviewed By: teemperor
Differential Revision: https://reviews.llvm.org/D99513
This test is not useful as the functions it's testing are just returning
a constant. It also fails in unoptimized builds as it's comparing
character strings by address.
Remove the "depth" variable, as the same information can be obtained
through die_index_stack.size().
Also add a test case for a one tricky case I noticed -- a unit
containing only a null unit die.
When LLVM error handling was introduced to the parsing of the .debug_aranges it would cause major issues if any DWARFDebugArangeSet::extract() calls returned any errors. The code in DWARFDebugInfo::GetCompileUnitAranges() would end up calling DWARFDebugAranges::extract() which would return an error if _any_ DWARFDebugArangeSet had any errors, but it default constructed a DWARFDebugAranges object into DWARFDebugInfo::m_cu_aranges_up and populated it partially, and returned an error prior to finishing much needed functionality in the DWARFDebugInfo::GetCompileUnitAranges() function. Subsequent callers to this function would see that the DWARFDebugInfo::m_cu_aranges_up was actually valid and return this partially populated DWARFDebugAranges reference _and_ it would not be sorted or minimized.
This above bugs would cause an incomplete .debug_aranges parsing, it would skip manually parsing any compile units for ranges, and would not sort the DWARFDebugAranges in m_cu_aranges_up.
This bug would also cause breakpoints set by file and line to fail to set correctly if a symbol context for an address could not be resolved properly, which the incomplete and unsorted DWARFDebugAranges object that DWARFDebugInfo::GetCompileUnitAranges() returned would cause symbol context lookups resolved by address (breakpoint address) to fail to find any DWARF debug info for a given address.
This patch fixes all of the issues that I found:
- DWARFDebugInfo::GetCompileUnitAranges() no longer returns a "llvm::Expected<DWARFDebugAranges &>", but just returns a "const DWARFDebugAranges &". Why? Because this code contained a fallback that would parse all of the valid DWARFDebugArangeSet objects, and would check which compile units had valid .debug_aranges set entries, and manually build an address ranges table using DWARFUnit::BuildAddressRangeTable(). If we return an error because any DWARFDebugArangeSet has any errors, then we don't do any of this code. Now we parse all DWARFDebugArangeSet objects that have no errors, if any calls to DWARFDebugArangeSet::extract() return errors, we skip that DWARFDebugArangeSet so that we can use the fallback call to DWARFUnit::BuildAddressRangeTable(). Since DWARFDebugInfo::GetCompileUnitAranges() needs to parse what it can from the .debug_aranges and build address ranges tables for any compile units that don't have any .debug_aranges sets, everything now works as expected.
- Fix an issue where a DWARFDebugArangeSet contains multiple terminator entries. The LLVM parser and llvm-dwarfdump properly warn about this because it happens with linux compilers and linkers and was the original cause of the bug I am fixing here. We now correctly warn about this issue if "log enable dwarf info" is enabled, but we continue to parse the DWARFDebugArangeSet correctly so we don't lose data that is contained in the .debug_aranges section.
- DWARFDebugAranges::extract() no longer returns a llvm::Error because we need to be able to parse all of the valid DWARFDebugArangeSet objects. It also will correctly skip a DWARFDebugArangeSet object that has errors in the middle of the stream by setting the start offsets of each DWARFDebugArangeSet to be calculated by the previous DWARFDebugArangeSet::extract() calculated offset that uses the header which contains the length of the DWARFDebugArangeSet. This means if do we run into real errors while parsing individual DWARFDebugArangeSet objects, we can continue to parse the rest of the validly encoded DWARFDebugArangeSet objects in the .debug_aranges section. This will allow LLDB to parse DWARF that contains a possibly newer .debug_aranges set format than LLDB currently supports because we will error out for the parsing of the DWARFDebugArangeSet, but be able to skip to the next DWARFDebugArangeSet object using the "DWARFDebugArangeSet.m_header.length" field to calculate the next starting offset.
Tests were added to cover all new functionality.
Differential Revision: https://reviews.llvm.org/D99401
Print LLVM's pretty stack trace when lldb-vscode crashes. Also removes
the unnecessary call to PrintStackTraceOnErrorSignal in lldb-server as
it's already part of InitLLVM.
Differential revision: https://reviews.llvm.org/D99535
Muhammad Omair Javaid