Both serve the same purpose (finding shared libraries) and allow one to
launch a dynamically linked executable by just specifying the platform
sysroot.
The lldb-server code is currently set up in a way that each
NativeProcess instance does its own waitpid handling. This works fine
for BSDs, where the code can do a waitpid(process_id), and get
information for all threads in that process.
The situation is trickier on linux, because waitpid(pid) will only
return information for the main thread of the process (one whose tid ==
pid). For this reason the linux code does a waitpid(-1), to get
information for all threads. This was fine while we were supporting just
a single process, but becomes a problem when we have multiple processes
as they end up stealing each others events.
There are two possible solutions to this problem:
- call waitpid(-1) centrally, and then dispatch the events to the
appropriate process
- have each process call waitpid(tid) for all the threads it manages
This patch implements the second approach. Besides fitting better into
the existing design, it also has the added benefit of ensuring
predictable ordering for thread/process creation events (which come in
pairs -- one for the parent and one for the child). The first approach
OTOH, would make this ordering even more complicated since we would
have to keep the half-threads hanging in mid-air until we find the
process we should attach them to.
The downside to this approach is an increased number of syscalls (one
waitpid for each thread), but I think we're pretty far from optimizing
things like this, and so the cleanliness of the design is worth it.
The included test reproduces the circumstances which should demonstrate
the bug (which manifests as a hung test), but I have not been able to
get it to fail. The only place I've seen this failure modes are very
rare hangs in the thread sanitizer tests (tsan forks an addr2line
process to produce its error messages).
Differential Revision: https://reviews.llvm.org/D116372
This small patch adds two useful improvements:
- allows one to specify the emulator path as a bare filename, and have
it be looked up in the PATH
- allows one to leave the path empty and have the filename be derived
from the architecture.
It was being used only in some very old tests (which pass even without
it) and its implementation is highly questionable.
These days we have different mechanisms for requesting a build with a
particular kind of c++ library (USE_LIB(STD)CPP in the makefile).
Now that we are caching the dwarf index as well, we will always have
more than one cache file (when not using accelerator tables). I have
adjusted the test to check for the presence of one _symtab_ index.
When we switched options over to use the Options.td file, a bug was introduced that caused the "-g" option for "settings set" to require a filename arguemnt. This patch fixes this issue and adds a test so this doesn't regress.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D116012
This patch add the ability to cache the manual DWARF indexing results to disk for faster subsequent debug sessions. Manual DWARF indexing is time consuming and causes all DWARF to be fully parsed and indexed each time you debug a binary that doesn't have an acceptable accelerator table. Acceptable accelerator tables include .debug_names in DWARF5 or Apple accelerator tables.
This patch breaks up testing by testing all of the encoding and decoding of required C++ objects in a gtest unit test, and then has a test to verify the debug info cache is generated correctly.
This patch also adds the ability to track when a symbol table or DWARF index is loaded or saved to the cache in the "statistics dump" command. This is essential to know in statistics as it can help explain why a debug session was slower or faster than expected.
Reviewed By: labath, wallace
Differential Revision: https://reviews.llvm.org/D115951
Introduce initial support for using libkvm on FreeBSD. The library
can be used as an alternate implementation for processing kernel
coredumps but it can also be used to access live kernel memory through
specifying "/dev/mem" as the core file, i.e.:
lldb --core /dev/mem /boot/kernel/kernel
Differential Revision: https://reviews.llvm.org/D116005
Version 2 of 'main bin spec' LC_NOTE allows for the specification
of a slide of where the binary is loaded in the corefile virtual
address space. It also adds a (currently unused) platform field
for the main binary.
Some corefile creators will only have a UUID and an offset to be
applied to the binary.
Changed TestFirmwareCorefiles.py to test this new form of
'main bin spec' with a slide, and also to run on both x86_64
and arm64 macOS systems.
Differential Revision: https://reviews.llvm.org/D116094
rdar://85938455
This starts to fix the other half of the lifetime problems in this code
-- dangling references. SB objects created on the stack will go away
when the function returns, which is a problem if the python code they
were meant for stashes a reference to them somewhere. Most of the time
this goes by unnoticed, as the code rarely has a reason to store these,
but in case it does, we shouldn't respond by crashing.
This patch fixes the management for a couple of SB objects (Debugger,
Frame, Thread). The SB objects are now created on the heap, and
their ownership is immediately passed on to SWIG, which will ensure they
are destroyed when the last python reference goes away. I will handle
the other objects in separate patches.
I include one test which demonstrates the lifetime issue for SBDebugger.
Strictly speaking, one should create a test case for each of these
objects and each of the contexts they are being used. That would require
figuring out how to persist (and later access) each of these objects.
Some of those may involve a lot of hoop-jumping (we can run python code
from within a frame-format string). I don't think that is
necessary/worth it since the new wrapper functions make it very hard to
get this wrong.
Differential Revision: https://reviews.llvm.org/D115925
The test was attempting to make a universal x86_64/arm64 binary, but some older bots don't have a macOS SDK that can handle this. Switching over to using a yaml file instead should solve the problem.
This setting is for variables we want to pass to the emulator only --
then will be automatically removed from the target environment by our
environment diffing code. This variable can be used to pass various
QEMU_*** variables (although most of these can be passed through
emulator-args as well), as well as any other variables that can affect
the operation of the emulator (e.g. LD_LIBRARY_PATH).
They were being applied too narrowly (they didn't cover signed char *,
for instance), and too broadly (they covered SomeTemplate<char[6]>) at
the same time.
Differential Revision: https://reviews.llvm.org/D112709
This is an updated version of the https://reviews.llvm.org/D113789 patch with the following changes:
- We no longer modify modification times of the cache files
- Use LLVM caching and cache pruning instead of making a new cache mechanism (See DataFileCache.h/.cpp)
- Add signature to start of each file since we are not using modification times so we can tell when caches are stale and remove and re-create the cache file as files are changed
- Add settings to control the cache size, disk percentage and expiration in days to keep cache size under control
This patch enables symbol tables to be cached in the LLDB index cache directory. All cache files are in a single directory and the files use unique names to ensure that files from the same path will re-use the same file as files get modified. This means as files change, their cache files will be deleted and updated. The modification time of each of the cache files is not modified so that access based pruning of the cache can be implemented.
The symbol table cache files start with a signature that uniquely identifies a file on disk and contains one or more of the following items:
- object file UUID if available
- object file mod time if available
- object name for BSD archive .o files that are in .a files if available
If none of these signature items are available, then the file will not be cached. This keeps temporary object files from expressions from being cached.
When the cache files are loaded on subsequent debug sessions, the signature is compare and if the file has been modified (uuid changes, mod time changes, or object file mod time changes) then the cache file is deleted and re-created.
Module caching must be enabled by the user before this can be used:
symbols.enable-lldb-index-cache (boolean) = false
(lldb) settings set symbols.enable-lldb-index-cache true
There is also a setting that allows the user to specify a module cache directory that defaults to a directory that defaults to being next to the symbols.clang-modules-cache-path directory in a temp directory:
(lldb) settings show symbols.lldb-index-cache-path
/var/folders/9p/472sr0c55l9b20x2zg36b91h0000gn/C/lldb/IndexCache
If this setting is enabled, the finalized symbol tables will be serialized and saved to disc so they can be quickly loaded next time you debug.
Each module can cache one or more files in the index cache directory. The cache file names must be unique to a file on disk and its architecture and object name for .o files in BSD archives. This allows universal mach-o files to support caching multuple architectures in the same module cache directory. Making the file based on the this info allows this cache file to be deleted and replaced when the file gets updated on disk. This keeps the cache from growing over time during the compile/edit/debug cycle and prevents out of space issues.
If the cache is enabled, the symbol table will be loaded from the cache the next time you debug if the module has not changed.
The cache also has settings to control the size of the cache on disk. Each time LLDB starts up with the index cache enable, the cache will be pruned to ensure it stays within the user defined settings:
(lldb) settings set symbols.lldb-index-cache-expiration-days <days>
A value of zero will disable cache files from expiring when the cache is pruned. The default value is 7 currently.
(lldb) settings set symbols.lldb-index-cache-max-byte-size <size>
A value of zero will disable pruning based on a total byte size. The default value is zero currently.
(lldb) settings set symbols.lldb-index-cache-max-percent <percentage-of-disk-space>
A value of 100 will allow the disc to be filled to the max, a value of zero will disable percentage pruning. The default value is zero.
Reviewed By: labath, wallace
Differential Revision: https://reviews.llvm.org/D115324
Introduce a FreeBSDKernel plugin that provides the ability to read
FreeBSD kernel core dumps. The plugin utilizes libfbsdvmcore to provide
support for both "full memory dump" and minidump formats across variety
of architectures supported by FreeBSD. It provides the ability to read
kernel memory, as well as the crashed thread status with registers
on arm64, i386 and x86_64.
Differential Revision: https://reviews.llvm.org/D114911
The pdb lldb tests do not work correctly with both the VS2019 and VS2017 toolsets at the moment. This change updates several of the tests to work with both toolsets. Unfortunately, this makes the tests suboptimal for both toolsets, but we can update them to be better for VS2019 once we officially drop VS2017. This change is meant to bridge the gap until the update happens, so that the buildbots can work with either toolset.
Differential Revision: https://reviews.llvm.org/D115482
Currently, we'll try to instantiate a ClangREPL for every known
language. The plugin manager already knows what languages it supports,
so rely on that to only instantiate a REPL when we know the requested
language is supported.
rdar://86439474
Differential revision: https://reviews.llvm.org/D115698
Introduce a FreeBSDKernel plugin that provides the ability to read
FreeBSD kernel core dumps. The plugin utilizes libfbsdvmcore to provide
support for both "full memory dump" and minidump formats across variety
of architectures supported by FreeBSD. It provides the ability to read
kernel memory, as well as the crashed thread status with registers
on arm64, i386 and x86_64.
Differential Revision: https://reviews.llvm.org/D114911
Add lldb support for a Mach-O "load binary" LC_NOTE which provides
a UUID, load address/slide, and possibly a name of a binary that
should be loaded when examining the core.
struct load_binary
{
uint32_t version; // currently 1
uuid_t uuid; // all zeroes if uuid not specified
uint64_t load_address; // virtual address where the macho is loaded, UINT64_MAX if unavail
uint64_t slide; // slide to be applied to file address to get load address, 0 if unavail
char name_cstring[]; // must be nul-byte terminated c-string, '\0' alone if name unavail
} __attribute__((packed));
Differential Revision: https://reviews.llvm.org/D115494
rdar://85069250
This commit should fix a heap-use-after-free bug that was caught by the
sanitizer bot.
The issue is that we were reading memory from a second target into a
`SBData` object in Python, that was passed to lldb's internal
`ScriptedProcess::DoReadMemory` C++ method.
The ScriptedPythonInterface then extracts the underlying `DataExtractor`
from the `SBData` object, and is used to read the memory with the
appropriate address size and byte order.
Unfortunately, it seems that even though the DataExtractor object was
still valid, it pointed to invalid, possibly garbage-collected memory
from Python.
To mitigate this, the patch uses `SBData::SetDataWithOwnership` to copy
the pointed buffer to lldb's heap memory which prevents the
use-after-free error.
rdar://84511405
Differential Revision: https://reviews.llvm.org/D115654
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch introduces a new method to SBData: SetDataWithOwnership.
Instead of referencing the pointer to the data, this method copies the
data buffer into lldb's heap memory.
This can prevent having the underlying DataExtractor object point to
freed/garbage-collected memory.
Differential Revision: https://reviews.llvm.org/D115652
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
With arm64e ARMv8.3 pointer authentication, lldb needs to know how
many bits are used for addressing and how many are used for pointer
auth signing. This should be determined dynamically from the inferior
system / corefile, but there are some workflows where it still isn't
recorded and we fall back on a default value that is correct on some
Darwin environments.
This patch also explicitly sets the vendor of mach-o binaries to
Apple, so we select an Apple ABI instead of a random other ABI.
It adds a function pointer formatter for systems where pointer
authentication is in use, and we can strip the ptrauth bits off
of the function pointer address and get a different value that
points to an actual symbol.
Differential Revision: https://reviews.llvm.org/D115431
rdar://84644661
This is a post-review update for D115313, to rephrase source display
warning messages for artificial locations, making them more
understandable for the end-user.
Differential Revision: https://reviews.llvm.org/D115461
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
It can happen that a line entry reports that some source code is located
at line 0. In DWARF, line 0 is a special location which indicates that
code has no 1-1 mapping with source.
When stopping in one of those artificial locations, lldb doesn't know which
line to display and shows the beginning of the file instead.
This patch mitigates this behaviour by checking if the current symbol context
of the line entry has a matching function, in which case, it slides the
source listing to the start of that function.
This patch also shows the user a warning explaining why lldb couldn't
show sources at that location.
rdar://83118425
Differential Revision: https://reviews.llvm.org/D115313
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
Test is using "next" commands to make progress in the process. D115137
added an additional statement to the program, without adding a command
to step over it. This only seemed to matter for the libc++ flavour of
the test, possibly because libstdc++ list is "empty" in its
uninitialized state.
Since moving with step commands is a treacherous, this patch adds a
run-to-breakpoint command to the test. It only does this for the
affected step, but one may consider doing it elsewhere too.
Qemu normally forwards its (host) environment variables to the emulated
process. While this works fine for most variables, there are some (few, but
fairly important) variables where this is not possible. LD_LIBRARY_PATH
is the probably the most important of those -- we don't want the library
search path for the emulated libraries to interfere with the libraries
that the emulator itself needs.
For this reason, qemu provides a mechanism (QEMU_SET_ENV,
QEMU_UNSET_ENV) to set variables only for the emulated process. This
patch makes use of that functionality to pass any user-provided
variables to the emulated process. Since we're piggy-backing on the
normal lldb environment-handling mechanism, all the usual mechanism to
provide environment (target.env-vars setting, SBLaunchInfo, etc.) work
out-of-the-box, and the only thing we need to do is to properly
construct the qemu environment variables.
This patch also adds a new setting -- target-env-vars, which represents
environment variables which are added (on top of the host environment)
to the default launch environments of all (qemu) targets. The reason for
its existence is to enable the configuration (e.g., from a startup
script) of the default launch environment, before any target is created.
The idea is that this would contain the variables (like the
aforementioned LD_LIBRARY_PATH) common to all targets being debugged on
the given system. The user is, of course, free to customize the
environment for a particular target in the usual manner.
The reason I do not want to use/recommend the "global" version of the
target.env-vars setting for this purpose is that the setting would apply
to all targets, whereas the settings (their values) I have mentioned
would be specific to the given platform.
Differential Revision: https://reviews.llvm.org/D115246
Also add tests to check that we print the warning in the right
circumstances.
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D114877
This is a split of D113724. Calling `TypeSystemClang::AddMethodToCXXRecordType`
to create function decls for class methods.
Differential Revision: https://reviews.llvm.org/D113930
The test was flaky because it was trying to read from the (redirected)
stdout file before the data was been flushed to it. This would not be a
problem for a "normal" debug session, but since here the emulator and
the target binary coexist in the same process (and this is true both for
real qemu and our fake implementation), there
is a window of time between the stub returning an exit packet (which is
the event that the test is waiting for) and the process really exiting
(which is when the normal flushing happens).
This patch adds an explicit flush to work around this. Theoretically,
it's possible that real code could run into this issue as well, but such
a use case is not very likely. If we wanted to fix this for real, we
could add some code which waits for the host process to terminate (in
addition to receiving the termination packet), but this is somewhat
complicated by the fact that this code lives in the gdb-remote process
plugin.
This setting allows the user to pass additional arguments to the qemu instance.
While we may want to introduce dedicated settings for the most common qemu
arguments (-cpu, for one), having this setting allows us to avoid creating a
setting for every possible argument.
Differential Revision: https://reviews.llvm.org/D115151
This patch should fix a Windows test failure for the
InvalidScriptedThread test:
https://lab.llvm.org/buildbot/#/builders/83/builds/12571
This refactors the test to stop using python `tempfile` since it's not
supported on Windows and creates a logfile at runtime in the test folder.
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch adds support for arm64(e) targets to ScriptedProcess, by
providing the `DynamicRegisterInfo` to the base `lldb.ScriptedThread` class.
This allows create and debugging ScriptedProcess on Apple Silicon
hardware as well as Apple mobile devices.
It also replace the C++ asserts on `ScriptedThread::GetDynamicRegisterInfo`
by some error logging, re-enables `TestScriptedProcess` for arm64
Darwin platforms and adds a new invalid Scripted Thread test.
rdar://85892451
Differential Revision: https://reviews.llvm.org/D114923
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This adds extra tests for libstdcpp and libcxx list and forward_list formatters to check whether formatter behaves correctly when applied on pointer and reference values.
Reviewed By: wallace
Differential Revision: https://reviews.llvm.org/D115137
This adds the formatters for libstdcpp's deque as a python
implementation. It adds comprehensive tests for the two different
storage strategies deque uses. Besides that, this fixes a couple of bugs
in the libcxx implementation. Finally, both implementation run against
the same tests.
This is a minor improvement on top of Danil Stefaniuc's formatter.
Lldb uses a pty to read/write to the standard input and output of the
debugged process. For host processes this would be automatically set up
by Target::FinalizeFileActions. The Qemu platform is in a unique
position of not really being a host platform, but not being remote
either. It reports IsHost() = false, but it is sufficiently host-like
that we can use the usual pty mechanism.
This patch adds the necessary glue code to enable pty redirection. It
includes a small refactor of Target::FinalizeFileActions and
ProcessLaunchInfo::SetUpPtyRedirection to reduce the amount of
boilerplate that would need to be copied.
I will note that qemu is not able to separate output from the emulated
program from the output of the emulator itself, so the two will arrive
intertwined. Normally this should not be a problem since qemu should not
produce any output during regular operation, but some output can slip
through in case of errors. This situation should be pretty obvious (to a
human), and it is the best we can do anyway.
For testing purposes, and inspired by lldb-server tests, I have extended
the mock emulator with the ability "program" the behavior of the
"emulated" program via command-line arguments.
Differential Revision: https://reviews.llvm.org/D114796
This implements `DoLoadImage` and `UnloadImage` in the Windows platform
plugin modelled after the POSIX platform plugin. This was previously
unimplemented and resulted in a difficult to decipher error without any
logging.
This implementation is intended to support enables the use of LLDB's
Swift REPL on Windows.
Paths which are added to the library search path are persistent and
applied to all subsequent loads. This can be adjusted in the future by
storing all the cookies and restoring the path prior to returning from
the helper. However, the dynamic path count makes this a bit more
challenging.
Reviewed By: @JDevlieghere
Differential Revision: https://reviews.llvm.org/D77287
minidebuginfo-set-and-hit-breakpoint.test is failing on Arm/Linux most
probably due to an ill formed binary after removal of certain sections
from executable. I am marking it as XFAIL for further investigation.
Only lldb-arm-ubuntu is failing after https://reviews.llvm.org/D114288 and there isn't enough input context to see why this is failing. It works on x86_64 linux just fine.
Pavel Labath