Add `IsAggregateType` to the SB API.
I'd like to use this from tests, and there are numerous other `Is<X>Type`
predicates on `SBType`.
Differential Revision: https://reviews.llvm.org/D121252
Our SIGTSTP handler was working, but that was mostly accidental.
The reason it worked is because lldb is multithreaded for most of its
lifetime and the OS is reasonably fast at responding to signals. So,
what happened was that the kill(SIGTSTP) which we sent from inside the
handler was delivered to another thread while the handler was still set
to SIG_DFL (which then correctly put the entire process to sleep).
Sometimes it happened that the other thread got the second signal after
the first thread had already restored the handler, in which case the
signal handler would run again, and it would again attempt to send the
SIGTSTP signal back to itself.
Normally it didn't take many iterations for the signal to be delivered
quickly enough. However, if you were unlucky (or were playing around
with pexpect) you could get SIGTSTP while lldb was single-threaded, and
in that case, lldb would go into an endless loop because the second
SIGTSTP could only be handled on the main thread, and only after the
handler for the first signal returned (and re-installed itself). In that
situation the handler would keep re-sending the signal to itself.
This patch fixes the issue by implementing the handler the way it
supposed to be done:
- before sending the second SIGTSTP, we unblock the signal (it gets
automatically blocked upon entering the handler)
- we use raise to send the signal, which makes sure it gets delivered to
the thread which is running the handler
This also means we don't need the SIGCONT handler, as our TSTP handler
resumes right after the entire process is continued, and we can do the
required work there.
I also include a test case for the SIGTSTP flow. It uses pexpect, but it
includes a couple of extra twists. Specifically, I needed to create an
extra process on top of lldb, which will run lldb in a separate process
group and simulate the role of the shell. This is needed because SIGTSTP
is not effective on a session leader (the signal gets delivered, but it
does not cause a stop) -- normally there isn't anyone to notice the
stop.
Differential Revision: https://reviews.llvm.org/D120320
The race is between these two pieces of code that are executed in two separate
lldb-vscode threads (the first is in the main thread and another is in the
event-handling thread):
```
// lldb-vscode.cpp
g_vsc.debugger.SetAsync(false);
g_vsc.target.Launch(launch_info, error);
g_vsc.debugger.SetAsync(true);
```
```
// Target.cpp
bool old_async = debugger.GetAsyncExecution();
debugger.SetAsyncExecution(true);
debugger.GetCommandInterpreter().HandleCommands(GetCommands(), exc_ctx,
options, result);
debugger.SetAsyncExecution(old_async);
```
The sequence that leads to the bug is this one:
1. Main thread enables synchronous mode and launches the process.
2. When the process is launched, it generates the first stop event.
3. This stop event is catched by the event-handling thread and DoOnRemoval
is invoked.
4. Inside DoOnRemoval, this thread runs stop hooks. And before running stop
hooks, the current synchronization mode is stored into old_async (and
right now it is equal to "false").
5. The main thread finishes the launch and returns to lldb-vscode, the
synchronization mode is restored to asynchronous by lldb-vscode.
6. Event-handling thread finishes stop hooks processing and restores the
synchronization mode according to old_async (i.e. makes the mode synchronous)
7. And now the mode is synchronous while lldb-vscode expects it to be
asynchronous. Synchronous mode forbids the process to broadcast public stop
events, so, VS Code just hangs because lldb-vscode doesn't notify it about
stops.
So, this diff makes the target intercept the first stop event if the process is
launched in the synchronous mode, thus preventing stop hooks execution.
The bug is only present on Windows because other platforms already
intercept this event using their own hijacking listeners.
So, this diff also fixes some problems with lldb-vscode tests on Windows to make
it possible to run the related test. Other tests still can't be enabled because
the debugged program prints something into stdout and LLDB can't intercept this
output and redirect it to lldb-vscode properly.
Reviewed By: jingham
Differential Revision: https://reviews.llvm.org/D119548
We discovered that when using "launchCommands" or "attachCommands" that there was an issue where these commands were not being run synchronously. There were further problems in this case where we would get thread events for the process that was just launched or attached before the IDE was ready, which is after "configurationDone" was sent to lldb-vscode.
This fix introduces the ability to wait for the process to stop after the run or attach to ensure that we have a stopped process at the entry point that is ready for the debug session to proceed. This also allows us to run the normal launch or attach without needing to play with the async flag the debugger. We spin up the thread that listens for process events before we start the launch or attach, but we stop the first eStateStopped (with stop ID of zero) event from being delivered through the DAP protocol because the "configurationDone" request handler will deliver it manually as the IDE expects a stop after configuration done. The request_configurationDone will also only deliver the stop packet if the "stopOnEntry" is False in the launch configuration.
Also added a new "timeout" to the launch and attach launch configuration arguments that can be set and defaults to 30 seconds. Since we now poll to detect when the process is stopped, we need a timeout that can be changed in case certain workflows take longer that 30 seconds to attach. If the process is not stopped by the timeout, an error will be retured for the launch or attach.
Added a flag to the vscode.py protocol classes that detects and ensures that no "stopped" events are sent prior to "configurationDone" has been sent and will raise an error if it does happen.
This should make our launching and attaching more reliable and avoid some deadlocks that were being seen (https://reviews.llvm.org/D119548).
Differential Revision: https://reviews.llvm.org/D119797
Instead of using sleeps, have the inferior notify us (via a trap opcode) that
the requested number of threads have been created.
This allows us to get rid of some fairly dodgy test utility code --
wait_for_thread_count seemed like it was waiting for the threads to
appear, but it never actually let the inferior run, so it only succeeded
if the threads were already started when the function was called. Since
the function was called after a fairly small delay (1s, usually), this
is probably the reason why the tests were failing on some bots.
Differential Revision: https://reviews.llvm.org/D119167
This fixes TestGdbRemoteSingleStep.py and TestGdbRemote_vCont.py. This
patch updates the test to account for the possibility that the constants
are already materialized. This appears to behave differently between
embedded arm64 devices and Apple Silicon.
Implement the qXfer:siginfo:read that is used to read the siginfo_t
(extended signal information) for the current thread. This is currently
implemented on FreeBSD and Linux.
Differential Revision: https://reviews.llvm.org/D117113
Previously we would persist the flags indicating whether the remote side
supports a particular feature across reconnects, which is obviously not
a good idea.
I implement the clearing by nuking (its the only way to be sure :) the
entire GDBRemoteCommunication object in the disconnect operation and
creating a new one upon connection. This allows us to maintain a nice
invariant that the GDBRemoteCommunication object (which is now a
pointer) exists only if it is connected. The downside to that is that a
lot of functions now needs to check the validity of the pointer instead
of blindly accessing the object.
The process communication does not suffer from the same issue because we
always destroy the entire Process object for a relaunch.
Differential Revision: https://reviews.llvm.org/D116539
It was being used only in some very old tests (which pass even without
it) and its implementation is highly questionable.
These days we have different mechanisms for requesting a build with a
particular kind of c++ library (USE_LIB(STD)CPP in the makefile).
This is an updated version of the https://reviews.llvm.org/D113789 patch with the following changes:
- We no longer modify modification times of the cache files
- Use LLVM caching and cache pruning instead of making a new cache mechanism (See DataFileCache.h/.cpp)
- Add signature to start of each file since we are not using modification times so we can tell when caches are stale and remove and re-create the cache file as files are changed
- Add settings to control the cache size, disk percentage and expiration in days to keep cache size under control
This patch enables symbol tables to be cached in the LLDB index cache directory. All cache files are in a single directory and the files use unique names to ensure that files from the same path will re-use the same file as files get modified. This means as files change, their cache files will be deleted and updated. The modification time of each of the cache files is not modified so that access based pruning of the cache can be implemented.
The symbol table cache files start with a signature that uniquely identifies a file on disk and contains one or more of the following items:
- object file UUID if available
- object file mod time if available
- object name for BSD archive .o files that are in .a files if available
If none of these signature items are available, then the file will not be cached. This keeps temporary object files from expressions from being cached.
When the cache files are loaded on subsequent debug sessions, the signature is compare and if the file has been modified (uuid changes, mod time changes, or object file mod time changes) then the cache file is deleted and re-created.
Module caching must be enabled by the user before this can be used:
symbols.enable-lldb-index-cache (boolean) = false
(lldb) settings set symbols.enable-lldb-index-cache true
There is also a setting that allows the user to specify a module cache directory that defaults to a directory that defaults to being next to the symbols.clang-modules-cache-path directory in a temp directory:
(lldb) settings show symbols.lldb-index-cache-path
/var/folders/9p/472sr0c55l9b20x2zg36b91h0000gn/C/lldb/IndexCache
If this setting is enabled, the finalized symbol tables will be serialized and saved to disc so they can be quickly loaded next time you debug.
Each module can cache one or more files in the index cache directory. The cache file names must be unique to a file on disk and its architecture and object name for .o files in BSD archives. This allows universal mach-o files to support caching multuple architectures in the same module cache directory. Making the file based on the this info allows this cache file to be deleted and replaced when the file gets updated on disk. This keeps the cache from growing over time during the compile/edit/debug cycle and prevents out of space issues.
If the cache is enabled, the symbol table will be loaded from the cache the next time you debug if the module has not changed.
The cache also has settings to control the size of the cache on disk. Each time LLDB starts up with the index cache enable, the cache will be pruned to ensure it stays within the user defined settings:
(lldb) settings set symbols.lldb-index-cache-expiration-days <days>
A value of zero will disable cache files from expiring when the cache is pruned. The default value is 7 currently.
(lldb) settings set symbols.lldb-index-cache-max-byte-size <size>
A value of zero will disable pruning based on a total byte size. The default value is zero currently.
(lldb) settings set symbols.lldb-index-cache-max-percent <percentage-of-disk-space>
A value of 100 will allow the disc to be filled to the max, a value of zero will disable percentage pruning. The default value is zero.
Reviewed By: labath, wallace
Differential Revision: https://reviews.llvm.org/D115324
Introduce a FreeBSDKernel plugin that provides the ability to read
FreeBSD kernel core dumps. The plugin utilizes libfbsdvmcore to provide
support for both "full memory dump" and minidump formats across variety
of architectures supported by FreeBSD. It provides the ability to read
kernel memory, as well as the crashed thread status with registers
on arm64, i386 and x86_64.
Differential Revision: https://reviews.llvm.org/D114911
Introduce a FreeBSDKernel plugin that provides the ability to read
FreeBSD kernel core dumps. The plugin utilizes libfbsdvmcore to provide
support for both "full memory dump" and minidump formats across variety
of architectures supported by FreeBSD. It provides the ability to read
kernel memory, as well as the crashed thread status with registers
on arm64, i386 and x86_64.
Differential Revision: https://reviews.llvm.org/D114911
727bd89b60 broke the UBSan decorator. The decorator compiles a custom
source code snippet that exposes UB and verifies the presence of a UBSan
symbol in the generated binary. The aforementioned commit broke both by
compiling a snippet without UB and discarding the result.
Make sure to add the PrivateFrameworks directory to the frameworks path
when using an internal SDK. This is necessary for the "on-device" test
suite.
rdar://84519268
Differential revision: https://reviews.llvm.org/D114742
This adds a new platform class, whose job is to enable running
(debugging) executables under qemu.
(For general information about qemu, I recommend reading the RFC thread
on lldb-dev
<https://lists.llvm.org/pipermail/lldb-dev/2021-October/017106.html>.)
This initial patch implements the necessary boilerplate as well as the
minimal amount of functionality needed to actually be able to do
something useful (which, in this case means debugging a fully statically
linked executable).
The knobs necessary to emulate dynamically linked programs, as well as
to control other aspects of qemu operation (the emulated cpu, for
instance) will be added in subsequent patches. Same goes for the ability
to automatically bind to the executables of the emulated architecture.
Currently only two settings are available:
- architecture: the architecture that we should emulate
- emulator-path: the path to the emulator
Even though this patch is relatively small, it doesn't lack subtleties
that are worth calling out explicitly:
- named sockets: qemu supports tcp and unix socket connections, both of
them in the "forward connect" mode (qemu listening, lldb connecting).
Forward TCP connections are impossible to realise in a race-free way.
This is the reason why I chose unix sockets as they have larger, more
structured names, which can guarantee that there are no collisions
between concurrent connection attempts.
- the above means that this code will not work on windows. I don't think
that's an issue since user mode qemu does not support windows anyway.
- Right now, I am leaving the code enabled for windows, but maybe it
would be better to disable it (otoh, disabling it means windows
developers can't check they don't break it)
- qemu-user also does not support macOS, so one could contemplate
disabling it there too. However, macOS does support named sockets, so
one can even run the (mock) qemu tests there, and I think it'd be a
shame to lose that.
Differential Revision: https://reviews.llvm.org/D114509
This is a preparatory commit to enable mocking of qemu startup. That
will involve running the mock server in a separate process, so there's
no need for multithreading.
Initialization is moved from the start function into the constructor
(which can then take an actual socket instead of a class), and the run
method is made public.
Depends on D114156.
Differential Revision: https://reviews.llvm.org/D114157
We were using the client socket close as a way to terminate the handler
thread. But this kind of concurrent access to the same socket is not
safe. It also complicates running the handler without a dedicated thread
(next patch).
Instead, here I add an explicit way for a packet handler to request
termination. Waiting for lldb to terminate the connection would almost
be sufficient, but in the pty test we want to keep the pty open so we
can examine its state. Ability to disconnect at an arbitrary point may
be useful for testing other aspects of lldb functionality as well.
The way this works is that now each packet handler can optionally return
a list of responses (instead of just one). One of those responses (it
only makes sense for it to be the last one) can be a special
RESPONSE_DISCONNECT object, which triggers a disconnection (via a new
TerminateConnectionException).
As the mock server now cleans up the connection whenever it disconnects,
the pty test needs to explicitly dup(2) the descriptors in order to
inspect the post-disconnect state.
Differential Revision: https://reviews.llvm.org/D114156
This infrastructure has proven proven its worth, so give it a more
prominent place.
My immediate motivation for this is the desire to reuse this
infrastructure for qemu platform testing, but I believe this move makes
sense independently of that. Moving this code to the packages tree will
allow as to add more structure to the gdb client tests -- currently they
are all crammed into the same test folder as that was the only way they
could access this code.
I'm splitting the code into two parts while moving it. The first once
contains just the generic gdb protocol wrappers, while the other one
contains the unit test glue. The reason for that is that for qemu
testing, I need to run the gdb code in a separate process, so I will
only be using the first part there.
Differential Revision: https://reviews.llvm.org/D113893
GDB and LLDB use different signal models. GDB uses a predefined set
of signal codes, and maps platform's signos to them. On the other hand,
LLDB has historically simply passed native signos.
In order to improve compatibility between LLDB and gdbserver, the GDB
signal model should be used. However, GDB does not provide a mapping
for all existing signals on Linux and unsupported signals are passed
as 'unknown'. Limiting LLDB to this behavior could be considered
a regression.
To get the best of both worlds, use the LLDB signal model when talking
to lldb-server, and the GDB signal model otherwise. For this purpose,
new versions of lldb-server indicate "native-signals+" via qSupported.
At the same time, we also detect older versions of lldb-server
via QThreadSuffixSupported for backwards compatibility. If neither test
succeeds, we assume gdbserver or another implementation using GDB model.
Differential Revision: https://reviews.llvm.org/D108078
This patch fixes an amusing bug where a Platform::Kill operation would
happily terminate a proces on a completely different platform, as long
as they have the same process ID. This was due to the fact that the
implementation was iterating through all known (debugged) processes in
order terminate them directly.
This patch just deletes that logic, and makes everything go through the
OS process termination APIs. While it would be possible to fix the logic
to check for a platform match, it seemed to me that the implementation
was being too smart for its own good -- accessing random Process
objects without knowing anything about their state is risky at best.
Going through the os ensures we avoid any races.
I also "upgrade" the termination signal to a SIGKILL to ensure the
process really dies after this operation.
Differential Revision: https://reviews.llvm.org/D113184
SetSourceMapFromArguments is called after the core is loaded. This means
that the source file for the crashing code won't have the source map applied.
Move the call to SetSourceMapFromArguments in request_attach to just after
the call to RunInitCommands, matching request_launch behavior.
Reviewed By: clayborg, wallace
Differential Revision: https://reviews.llvm.org/D112834
join is only available since python-3.8, but the all the interesting
magic happens in shlex.quote, which has been around since 3.3.
Use shlex.quote, and instead provide a home-grown helper function to
handle the joining.
Differential Revision: https://reviews.llvm.org/D112802
Running tests with -t prints all lldb commands being run. It makes sense
to print all the build commands as well.
Differential Revision: https://reviews.llvm.org/D112212
Create a valid triple in the Darwin builder. Currently it was
incorrectly treating the os and version as two separate components in
the triple.
Differential revision: https://reviews.llvm.org/D112676
When targeting iOS, the default dwarf version is 2 and not 4. Currently,
the test suite does not pick up on that because it invokes the test
compiler without a target triple. This patch fixes that and now
correctly skips tests that have a dwarf version specified in a skipIf
decorator.
rdar://84530477
Differential revision: https://reviews.llvm.org/D112325
This patch is a smaller version of a previous patch https://reviews.llvm.org/D110804.
This patch modifies the output of "statistics dump" to be able to get stats from the current target. It adds 3 new stats as well. The output of "statistics dump" is now emitted as JSON so that it can be used to track performance and statistics and the output could be used to populate a database that tracks performance. Sample output looks like:
(lldb) statistics dump
{
"expressionEvaluation": {
"failures": 0,
"successes": 0
},
"firstStopTime": 0.34164492800000001,
"frameVariable": {
"failures": 0,
"successes": 0
},
"launchOrAttachTime": 0.31969605400000001,
"targetCreateTime": 0.0040863039999999998
}
The top level keys are:
"expressionEvaluation" which replaces the previous stats that were emitted as plain text. This dictionary contains the success and fail counts.
"frameVariable" which replaces the previous stats for "frame variable" that were emitted as plain text. This dictionary contains the success and fail counts.
"targetCreateTime" contains the number of seconds it took to create the target and load dependent libraries (if they were enabled) and also will contain symbol preloading times if that setting is enabled.
"launchOrAttachTime" is the time it takes from when the launch/attach is initiated to when the first private stop occurs.
"firstStopTime" is the time in seconds that it takes to stop at the first stop that is presented to the user via the LLDB interface. This value will only have meaning if you set a known breakpoint or stop location in your code that you want to measure as a performance test.
This diff is also meant as a place to discuess what we want out of the "statistics dump" command before adding more funcionality. It is also meant to clean up the previous code that was storting statistics in a vector of numbers within the lldb_private::Target class.
Differential Revision: https://reviews.llvm.org/D111686
None of the commands we run really rely on shell features. Running them
with shell=False, simplifies the code as there is no need for elaborate
quoting.
Differential Revision: https://reviews.llvm.org/D111990
We had two sets of build<flavour> methods, whose bodies were largely
identical. This makes any kind of modification in their vicinity
repetitive and error-prone.
Replace each set with a single method taking an optional debug_info
parameter.
Differential Revision: https://reviews.llvm.org/D111989
I added some tests for the case where the breakpoints take immediately
to the extant test case, and made a new test case for when the source
regex breakpoint will be set in a dlopen-ed library.
I also noticed when doing this that "lldbutil.run_to_source_breakpoint
can't handle the case where the breakpoint will be in a dlopen-ed
library, since it requires the breakpoint to have at least 1 location
before run. I fixed that by adding a parameter to say whether a
before run location is expected.
Differential Revision: https://reviews.llvm.org/D111920
We only had that ability for regular debugger launches. This meant that
it was not possible to use the normal dlopen patterns in attach tests.
This fixes that.
This addresses the flakyness of (at least) TestMultilineNavigation,
which was failing when the editline history of a concurrently executing
test made leaked in. Using a test-specific home directory ensures the
tests are independent.
Regex is matching a binary so it needs `re.DOTALL`.
vFile:fstat packet data ATTACHMENT is not hex-encoded but it is only
escaped.
It is difficult to reproduce, it came from buildbot:
https://lab.llvm.org/staging/#/builders/16/builds/10597
******************** TEST 'lldb-api :: tools/lldb-server/TestGdbRemotePlatformFile.py' FAILED ********************
FAIL: test_platform_file_fstat_llgs (TestGdbRemotePlatformFile.TestGdbRemotePlatformFile)
----------------------------------------------------------------------
Traceback (most recent call last):
File "lldb/packages/Python/lldbsuite/test/tools/lldb-server/gdbremote_testcase.py", line 52, in test_method
return attrvalue(self)
File "lldb/test/API/tools/lldb-server/TestGdbRemotePlatformFile.py", line 259, in test_platform_file_fstat
context = self.expect_gdbremote_sequence()
File "lldb/packages/Python/lldbsuite/test/tools/lldb-server/gdbremote_testcase.py", line 621, in expect_gdbremote_sequence
return expect_lldb_gdbserver_replay(
File "lldb/packages/Python/lldbsuite/test/tools/lldb-server/lldbgdbserverutils.py", line 198, in expect_lldb_gdbserver_replay
context = sequence_entry.assert_match(
File "lldb/packages/Python/lldbsuite/test/tools/lldb-server/lldbgdbserverutils.py", line 479, in assert_match
return self._assert_regex_match(asserter, actual_packet, context)
File "lldb/packages/Python/lldbsuite/test/tools/lldb-server/lldbgdbserverutils.py", line 446, in _assert_regex_match
asserter.fail(
AssertionError: regex '^\$F([0-9a-fA-F]+);(.*)#[0-9a-fA-F]{2}$' failed to match against content '$F40;^@^@^@ ^@
^C<8d>^@^@<81><80>^@^@^@^A^@^@^CX^@^@^CX^@^@^@^@^@^@^@^@^@^@^@^W^@^@^@^@^@^@^P^@^@^@^@^@^@^@^@
a=XXa=XXa=XX#6b'
Dave Lee