Instead of using sleeps, have the inferior notify us (via a trap opcode) that
the requested number of threads have been created.
This allows us to get rid of some fairly dodgy test utility code --
wait_for_thread_count seemed like it was waiting for the threads to
appear, but it never actually let the inferior run, so it only succeeded
if the threads were already started when the function was called. Since
the function was called after a fairly small delay (1s, usually), this
is probably the reason why the tests were failing on some bots.
Differential Revision: https://reviews.llvm.org/D119167
This fixes TestGdbRemoteSingleStep.py and TestGdbRemote_vCont.py. This
patch updates the test to account for the possibility that the constants
are already materialized. This appears to behave differently between
embedded arm64 devices and Apple Silicon.
Implement the qXfer:siginfo:read that is used to read the siginfo_t
(extended signal information) for the current thread. This is currently
implemented on FreeBSD and Linux.
Differential Revision: https://reviews.llvm.org/D117113
Previously we would persist the flags indicating whether the remote side
supports a particular feature across reconnects, which is obviously not
a good idea.
I implement the clearing by nuking (its the only way to be sure :) the
entire GDBRemoteCommunication object in the disconnect operation and
creating a new one upon connection. This allows us to maintain a nice
invariant that the GDBRemoteCommunication object (which is now a
pointer) exists only if it is connected. The downside to that is that a
lot of functions now needs to check the validity of the pointer instead
of blindly accessing the object.
The process communication does not suffer from the same issue because we
always destroy the entire Process object for a relaunch.
Differential Revision: https://reviews.llvm.org/D116539
It was being used only in some very old tests (which pass even without
it) and its implementation is highly questionable.
These days we have different mechanisms for requesting a build with a
particular kind of c++ library (USE_LIB(STD)CPP in the makefile).
This is an updated version of the https://reviews.llvm.org/D113789 patch with the following changes:
- We no longer modify modification times of the cache files
- Use LLVM caching and cache pruning instead of making a new cache mechanism (See DataFileCache.h/.cpp)
- Add signature to start of each file since we are not using modification times so we can tell when caches are stale and remove and re-create the cache file as files are changed
- Add settings to control the cache size, disk percentage and expiration in days to keep cache size under control
This patch enables symbol tables to be cached in the LLDB index cache directory. All cache files are in a single directory and the files use unique names to ensure that files from the same path will re-use the same file as files get modified. This means as files change, their cache files will be deleted and updated. The modification time of each of the cache files is not modified so that access based pruning of the cache can be implemented.
The symbol table cache files start with a signature that uniquely identifies a file on disk and contains one or more of the following items:
- object file UUID if available
- object file mod time if available
- object name for BSD archive .o files that are in .a files if available
If none of these signature items are available, then the file will not be cached. This keeps temporary object files from expressions from being cached.
When the cache files are loaded on subsequent debug sessions, the signature is compare and if the file has been modified (uuid changes, mod time changes, or object file mod time changes) then the cache file is deleted and re-created.
Module caching must be enabled by the user before this can be used:
symbols.enable-lldb-index-cache (boolean) = false
(lldb) settings set symbols.enable-lldb-index-cache true
There is also a setting that allows the user to specify a module cache directory that defaults to a directory that defaults to being next to the symbols.clang-modules-cache-path directory in a temp directory:
(lldb) settings show symbols.lldb-index-cache-path
/var/folders/9p/472sr0c55l9b20x2zg36b91h0000gn/C/lldb/IndexCache
If this setting is enabled, the finalized symbol tables will be serialized and saved to disc so they can be quickly loaded next time you debug.
Each module can cache one or more files in the index cache directory. The cache file names must be unique to a file on disk and its architecture and object name for .o files in BSD archives. This allows universal mach-o files to support caching multuple architectures in the same module cache directory. Making the file based on the this info allows this cache file to be deleted and replaced when the file gets updated on disk. This keeps the cache from growing over time during the compile/edit/debug cycle and prevents out of space issues.
If the cache is enabled, the symbol table will be loaded from the cache the next time you debug if the module has not changed.
The cache also has settings to control the size of the cache on disk. Each time LLDB starts up with the index cache enable, the cache will be pruned to ensure it stays within the user defined settings:
(lldb) settings set symbols.lldb-index-cache-expiration-days <days>
A value of zero will disable cache files from expiring when the cache is pruned. The default value is 7 currently.
(lldb) settings set symbols.lldb-index-cache-max-byte-size <size>
A value of zero will disable pruning based on a total byte size. The default value is zero currently.
(lldb) settings set symbols.lldb-index-cache-max-percent <percentage-of-disk-space>
A value of 100 will allow the disc to be filled to the max, a value of zero will disable percentage pruning. The default value is zero.
Reviewed By: labath, wallace
Differential Revision: https://reviews.llvm.org/D115324
Introduce a FreeBSDKernel plugin that provides the ability to read
FreeBSD kernel core dumps. The plugin utilizes libfbsdvmcore to provide
support for both "full memory dump" and minidump formats across variety
of architectures supported by FreeBSD. It provides the ability to read
kernel memory, as well as the crashed thread status with registers
on arm64, i386 and x86_64.
Differential Revision: https://reviews.llvm.org/D114911
Introduce a FreeBSDKernel plugin that provides the ability to read
FreeBSD kernel core dumps. The plugin utilizes libfbsdvmcore to provide
support for both "full memory dump" and minidump formats across variety
of architectures supported by FreeBSD. It provides the ability to read
kernel memory, as well as the crashed thread status with registers
on arm64, i386 and x86_64.
Differential Revision: https://reviews.llvm.org/D114911
727bd89b60 broke the UBSan decorator. The decorator compiles a custom
source code snippet that exposes UB and verifies the presence of a UBSan
symbol in the generated binary. The aforementioned commit broke both by
compiling a snippet without UB and discarding the result.
Make sure to add the PrivateFrameworks directory to the frameworks path
when using an internal SDK. This is necessary for the "on-device" test
suite.
rdar://84519268
Differential revision: https://reviews.llvm.org/D114742
This adds a new platform class, whose job is to enable running
(debugging) executables under qemu.
(For general information about qemu, I recommend reading the RFC thread
on lldb-dev
<https://lists.llvm.org/pipermail/lldb-dev/2021-October/017106.html>.)
This initial patch implements the necessary boilerplate as well as the
minimal amount of functionality needed to actually be able to do
something useful (which, in this case means debugging a fully statically
linked executable).
The knobs necessary to emulate dynamically linked programs, as well as
to control other aspects of qemu operation (the emulated cpu, for
instance) will be added in subsequent patches. Same goes for the ability
to automatically bind to the executables of the emulated architecture.
Currently only two settings are available:
- architecture: the architecture that we should emulate
- emulator-path: the path to the emulator
Even though this patch is relatively small, it doesn't lack subtleties
that are worth calling out explicitly:
- named sockets: qemu supports tcp and unix socket connections, both of
them in the "forward connect" mode (qemu listening, lldb connecting).
Forward TCP connections are impossible to realise in a race-free way.
This is the reason why I chose unix sockets as they have larger, more
structured names, which can guarantee that there are no collisions
between concurrent connection attempts.
- the above means that this code will not work on windows. I don't think
that's an issue since user mode qemu does not support windows anyway.
- Right now, I am leaving the code enabled for windows, but maybe it
would be better to disable it (otoh, disabling it means windows
developers can't check they don't break it)
- qemu-user also does not support macOS, so one could contemplate
disabling it there too. However, macOS does support named sockets, so
one can even run the (mock) qemu tests there, and I think it'd be a
shame to lose that.
Differential Revision: https://reviews.llvm.org/D114509
This is a preparatory commit to enable mocking of qemu startup. That
will involve running the mock server in a separate process, so there's
no need for multithreading.
Initialization is moved from the start function into the constructor
(which can then take an actual socket instead of a class), and the run
method is made public.
Depends on D114156.
Differential Revision: https://reviews.llvm.org/D114157
We were using the client socket close as a way to terminate the handler
thread. But this kind of concurrent access to the same socket is not
safe. It also complicates running the handler without a dedicated thread
(next patch).
Instead, here I add an explicit way for a packet handler to request
termination. Waiting for lldb to terminate the connection would almost
be sufficient, but in the pty test we want to keep the pty open so we
can examine its state. Ability to disconnect at an arbitrary point may
be useful for testing other aspects of lldb functionality as well.
The way this works is that now each packet handler can optionally return
a list of responses (instead of just one). One of those responses (it
only makes sense for it to be the last one) can be a special
RESPONSE_DISCONNECT object, which triggers a disconnection (via a new
TerminateConnectionException).
As the mock server now cleans up the connection whenever it disconnects,
the pty test needs to explicitly dup(2) the descriptors in order to
inspect the post-disconnect state.
Differential Revision: https://reviews.llvm.org/D114156
This infrastructure has proven proven its worth, so give it a more
prominent place.
My immediate motivation for this is the desire to reuse this
infrastructure for qemu platform testing, but I believe this move makes
sense independently of that. Moving this code to the packages tree will
allow as to add more structure to the gdb client tests -- currently they
are all crammed into the same test folder as that was the only way they
could access this code.
I'm splitting the code into two parts while moving it. The first once
contains just the generic gdb protocol wrappers, while the other one
contains the unit test glue. The reason for that is that for qemu
testing, I need to run the gdb code in a separate process, so I will
only be using the first part there.
Differential Revision: https://reviews.llvm.org/D113893
GDB and LLDB use different signal models. GDB uses a predefined set
of signal codes, and maps platform's signos to them. On the other hand,
LLDB has historically simply passed native signos.
In order to improve compatibility between LLDB and gdbserver, the GDB
signal model should be used. However, GDB does not provide a mapping
for all existing signals on Linux and unsupported signals are passed
as 'unknown'. Limiting LLDB to this behavior could be considered
a regression.
To get the best of both worlds, use the LLDB signal model when talking
to lldb-server, and the GDB signal model otherwise. For this purpose,
new versions of lldb-server indicate "native-signals+" via qSupported.
At the same time, we also detect older versions of lldb-server
via QThreadSuffixSupported for backwards compatibility. If neither test
succeeds, we assume gdbserver or another implementation using GDB model.
Differential Revision: https://reviews.llvm.org/D108078
This patch fixes an amusing bug where a Platform::Kill operation would
happily terminate a proces on a completely different platform, as long
as they have the same process ID. This was due to the fact that the
implementation was iterating through all known (debugged) processes in
order terminate them directly.
This patch just deletes that logic, and makes everything go through the
OS process termination APIs. While it would be possible to fix the logic
to check for a platform match, it seemed to me that the implementation
was being too smart for its own good -- accessing random Process
objects without knowing anything about their state is risky at best.
Going through the os ensures we avoid any races.
I also "upgrade" the termination signal to a SIGKILL to ensure the
process really dies after this operation.
Differential Revision: https://reviews.llvm.org/D113184
SetSourceMapFromArguments is called after the core is loaded. This means
that the source file for the crashing code won't have the source map applied.
Move the call to SetSourceMapFromArguments in request_attach to just after
the call to RunInitCommands, matching request_launch behavior.
Reviewed By: clayborg, wallace
Differential Revision: https://reviews.llvm.org/D112834
join is only available since python-3.8, but the all the interesting
magic happens in shlex.quote, which has been around since 3.3.
Use shlex.quote, and instead provide a home-grown helper function to
handle the joining.
Differential Revision: https://reviews.llvm.org/D112802
Running tests with -t prints all lldb commands being run. It makes sense
to print all the build commands as well.
Differential Revision: https://reviews.llvm.org/D112212
Create a valid triple in the Darwin builder. Currently it was
incorrectly treating the os and version as two separate components in
the triple.
Differential revision: https://reviews.llvm.org/D112676
When targeting iOS, the default dwarf version is 2 and not 4. Currently,
the test suite does not pick up on that because it invokes the test
compiler without a target triple. This patch fixes that and now
correctly skips tests that have a dwarf version specified in a skipIf
decorator.
rdar://84530477
Differential revision: https://reviews.llvm.org/D112325
This patch is a smaller version of a previous patch https://reviews.llvm.org/D110804.
This patch modifies the output of "statistics dump" to be able to get stats from the current target. It adds 3 new stats as well. The output of "statistics dump" is now emitted as JSON so that it can be used to track performance and statistics and the output could be used to populate a database that tracks performance. Sample output looks like:
(lldb) statistics dump
{
"expressionEvaluation": {
"failures": 0,
"successes": 0
},
"firstStopTime": 0.34164492800000001,
"frameVariable": {
"failures": 0,
"successes": 0
},
"launchOrAttachTime": 0.31969605400000001,
"targetCreateTime": 0.0040863039999999998
}
The top level keys are:
"expressionEvaluation" which replaces the previous stats that were emitted as plain text. This dictionary contains the success and fail counts.
"frameVariable" which replaces the previous stats for "frame variable" that were emitted as plain text. This dictionary contains the success and fail counts.
"targetCreateTime" contains the number of seconds it took to create the target and load dependent libraries (if they were enabled) and also will contain symbol preloading times if that setting is enabled.
"launchOrAttachTime" is the time it takes from when the launch/attach is initiated to when the first private stop occurs.
"firstStopTime" is the time in seconds that it takes to stop at the first stop that is presented to the user via the LLDB interface. This value will only have meaning if you set a known breakpoint or stop location in your code that you want to measure as a performance test.
This diff is also meant as a place to discuess what we want out of the "statistics dump" command before adding more funcionality. It is also meant to clean up the previous code that was storting statistics in a vector of numbers within the lldb_private::Target class.
Differential Revision: https://reviews.llvm.org/D111686
None of the commands we run really rely on shell features. Running them
with shell=False, simplifies the code as there is no need for elaborate
quoting.
Differential Revision: https://reviews.llvm.org/D111990
We had two sets of build<flavour> methods, whose bodies were largely
identical. This makes any kind of modification in their vicinity
repetitive and error-prone.
Replace each set with a single method taking an optional debug_info
parameter.
Differential Revision: https://reviews.llvm.org/D111989
I added some tests for the case where the breakpoints take immediately
to the extant test case, and made a new test case for when the source
regex breakpoint will be set in a dlopen-ed library.
I also noticed when doing this that "lldbutil.run_to_source_breakpoint
can't handle the case where the breakpoint will be in a dlopen-ed
library, since it requires the breakpoint to have at least 1 location
before run. I fixed that by adding a parameter to say whether a
before run location is expected.
Differential Revision: https://reviews.llvm.org/D111920
We only had that ability for regular debugger launches. This meant that
it was not possible to use the normal dlopen patterns in attach tests.
This fixes that.
This addresses the flakyness of (at least) TestMultilineNavigation,
which was failing when the editline history of a concurrently executing
test made leaked in. Using a test-specific home directory ensures the
tests are independent.
Regex is matching a binary so it needs `re.DOTALL`.
vFile:fstat packet data ATTACHMENT is not hex-encoded but it is only
escaped.
It is difficult to reproduce, it came from buildbot:
https://lab.llvm.org/staging/#/builders/16/builds/10597
******************** TEST 'lldb-api :: tools/lldb-server/TestGdbRemotePlatformFile.py' FAILED ********************
FAIL: test_platform_file_fstat_llgs (TestGdbRemotePlatformFile.TestGdbRemotePlatformFile)
----------------------------------------------------------------------
Traceback (most recent call last):
File "lldb/packages/Python/lldbsuite/test/tools/lldb-server/gdbremote_testcase.py", line 52, in test_method
return attrvalue(self)
File "lldb/test/API/tools/lldb-server/TestGdbRemotePlatformFile.py", line 259, in test_platform_file_fstat
context = self.expect_gdbremote_sequence()
File "lldb/packages/Python/lldbsuite/test/tools/lldb-server/gdbremote_testcase.py", line 621, in expect_gdbremote_sequence
return expect_lldb_gdbserver_replay(
File "lldb/packages/Python/lldbsuite/test/tools/lldb-server/lldbgdbserverutils.py", line 198, in expect_lldb_gdbserver_replay
context = sequence_entry.assert_match(
File "lldb/packages/Python/lldbsuite/test/tools/lldb-server/lldbgdbserverutils.py", line 479, in assert_match
return self._assert_regex_match(asserter, actual_packet, context)
File "lldb/packages/Python/lldbsuite/test/tools/lldb-server/lldbgdbserverutils.py", line 446, in _assert_regex_match
asserter.fail(
AssertionError: regex '^\$F([0-9a-fA-F]+);(.*)#[0-9a-fA-F]{2}$' failed to match against content '$F40;^@^@^@ ^@
^C<8d>^@^@<81><80>^@^@^@^A^@^@^CX^@^@^CX^@^@^@^@^@^@^@^@^@^@^@^W^@^@^@^@^@^@^P^@^@^@^@^@^@^@^@
a=XXa=XXa=XX#6b'
Add a new SaveCore() process method that can be used to request a core
dump. This is currently implemented on NetBSD via the PT_DUMPCORE
ptrace(2) request, and enabled via 'savecore' extension.
Protocol-wise, a new qSaveCore packet is introduced. It accepts zero
or more semicolon-separated key:value options, invokes the core dump
and returns a key:value response. Currently the only option supported
is "path-hint", and the return value contains the "path" actually used.
The support for the feature is exposed via qSaveCore qSupported feature.
Differential Revision: https://reviews.llvm.org/D101285
Test dependent on pexpect fail randomly with timeouts on Arm/AArch64 Linux
buildbots. I am setting pexpect timeout from 30 to 60.
I will revert this back if this doesnt improve random failures.
Always codesign binaries on macOS. Apple Silicon has stricter
codesigning requirements, for example requiring macCatalyst binaries to
be signed. Ad-hoc sign everything like we do for other Darwin platforms.
Adrian Prantl