Previously, the ScriptedThread used the thread index as the thread id.
This patch parses the crashlog json to extract the actual thread "id" value,
and passes this information to the Crashlog ScriptedProcess blueprint,
to create a higher fidelity ScriptedThreaad.
It also updates the blueprint to show the thread name and thread queue.
Finally, this patch updates the interactive crashlog test to reflect
these changes.
rdar://90327854
Differential Revision: https://reviews.llvm.org/D122422
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch pipes down the `-a|--load-all` crashlog command option to the
Scripted Process initializer to load all the images used by crashed
process instead of only loading the images related to the crashed
thread.
This allows us to recreate artificial frames also for the non-crashed
scripted threads.
rdar://90396265
Differential Revision: https://reviews.llvm.org/D121826
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch makes the crashlog interactive mode show the scripted process
status with the crashed scripted thread backtrace after launching it.
rdar://89634338
Differential Revision: https://reviews.llvm.org/D121038
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch introduces a new way to load modules programatically with
Scripted Processes. To do so, the scripted process blueprint holds a
list of dictionary describing the modules to load, which their path or
uuid, load address and eventually a slide offset.
LLDB will fetch that list after launching the ScriptedProcess, and
iterate over each entry to create the module that will be loaded in the
Scripted Process' target.
The patch also refactors the StackCoreScriptedProcess test to stop
inside the `libbaz` module and make sure it's loaded correctly and that
we can fetch some variables from it.
rdar://74520238
Differential Revision: https://reviews.llvm.org/D120969
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch should fix the test failure on scripted_crashlog_json.test.
The failure is happening because crash reporter will obfuscate the
executable path in the crashlog, if it is located inside the user's
home directory and replace it with `/USER/*/` as a placeholder.
To fix that, we can patch the placeholder with the executable path
before loading the crashlog in lldb.
This also fixes a bug where we would create another target when loading
the crashlog in a scripted process, even if lldb already had a target
for it. Now, crashlog will only create a target if there is none in lldb.
Differential Revision: https://reviews.llvm.org/D120598
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
In some cases, it can happen that crashlogs don't have any signal in
the exception, which causes the parser to crash.
This fixes the parsing by checking if the `signal` field is in the
`exception` dictionary before trying to access it.
rdar://84552251
Differential Revision: https://reviews.llvm.org/D119504
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch introduces a new type of ScriptedProcess: CrashLogScriptedProcess.
It takes advantage of lldb's crashlog parsers and Scripted Processes to
reconstruct a static debugging session with symbolicated stackframes, instead
of just dumping out everything in the user's terminal.
The crashlog command also has an interactive mode that only provide a
very limited experience. This is why this patch removes all the logic
for this interactive mode and creates CrashLogScriptedProcess instead.
This will fetch and load all the libraries that were used by the crashed
thread and re-create all the frames artificially.
rdar://88721117
Differential Revision: https://reviews.llvm.org/D119501
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch adds the ability for ScriptedThread to load artificial stack
frames. To do so, the interpreter instance can create a list that will
contain the frame index and its pc address.
Then, when the Scripted Process plugin stops, it will refresh its
Scripted Threads state by invalidating their register context and load
to list from the interpreter object and reconstruct each frame.
This patch also removes all of the default implementation for
`get_stackframes` from the derived ScriptedThread classes, and add the
interface code for the Scripted Thread Interface.
rdar://88721095
Differential Revision: https://reviews.llvm.org/D119388
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch removes the `my_scripted_process.py` blueprint since it's not
used anymore.
The patch also updates the base ScriptedProcess and ScriptedThread
initializers to automatically initialize convinience variables, to
access debugger from the ScriptedProcess, access the SBProcess and
ScriptedProcess object from a ScriptedThread instance.
Differential Revision: https://reviews.llvm.org/D119386
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
The symbolicator assumes that the first image in the image list is the
main image. That isn't always the case. For JSON crashlogs we can use
the procName to move the main image to the front of the list.
rdar://83907760
Instead trying to pro-actively determine if the first line in a
crashlog contains meta data, change the heuristic to do the following:
1. To trying to parse the whole file. If that fails, then:
2. Strip the first line and try parsing the remainder of the file. If
that fails, then:
3. Fall back to the textual crashlog parser.
rdar://88580543
Differential revision: https://reviews.llvm.org/D119755
This patch fixes the register parser for arm64 crashlogs.
Compared to x86_64 crashlogs, the arm64 crashlogs nests the general
purpose registers into a separate dictionary within `thread_state`
dictionary. It uses the dictionary key as the the register number.
Differential Revision: https://reviews.llvm.org/D119168
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch fixes a timeout issue on the ScriptedProcess test that was
happening on intel platforms. The timeout was due to a misreporting of
the StopInfo in the ScriptedThread that caused the ScriptedProcess to
never stop.
To solve this, this patch changes the way a ScriptedThread reports its
stop reason by making it more architecture specific. In order to do so,
this patch also refactors the ScriptedProcess & ScriptedThread
initializer methods to provide an easy access to the target architecture.
Differential Revision: https://reviews.llvm.org/D118484
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
Previously, importing `crashlog` resulted in a message being printed. The
message was about other commands (those in heap.py), not `crashlog`. The
changes in D117237 made it so that the heap.py messages were printed only when
importing `lldb.macosx.heap`, not when importing `lldb.macosx.crashlog`. Some
users may see no output and think `crashlog` wasn't successfully loaded. This
ensures users see that `crashlog` is loaded.
rdar://88283132
Differential Revision: https://reviews.llvm.org/D119155
Rosetta crashlogs can have their own thread register state. Unlike the
other registers which ware directly listed under "threadState", the
Rosetta registers are nested under their own key in the JSON, as
illustrated below:
{
"threadState":
{
"rosetta":
{
"tmp2":
{
"value": 4935057216
},
"tmp1":
{
"value": 4365863188
},
"tmp0":
{
"value": 18446744073709551615
}
}
}
}
This patch adds support of multiple Scripted Threads in a ScriptedProcess.
This is done by fetching the Scripted Threads info dictionary at every
ScriptedProcess::DoUpdateThreadList and iterate over each element to
create a new ScriptedThread using the object instance, if it was not
already available.
This patch also adds the ability to pass a pointer of a script interpreter
object instance to initialize a ScriptedInterface instead of having to call
the script object initializer in the ScriptedInterface constructor.
This is used to instantiate the ScriptedThreadInterface from the
ScriptedThread constructor, to be able to perform call on that script
interpreter object instance.
Finally, the patch also updates the scripted process test to check for
multiple threads.
rdar://84507704
Differential Revision: https://reviews.llvm.org/D117071
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch adds a new method to the Scripted Process interface to
retrive a dictionary of Scripted Threads. It uses the thread ID as a key
and the Scripted Thread instance as the value.
This dictionary will be used to create Scripted Threads in lldb and
perform calls to the python scripted thread object.
rdar://87427126
Differential Revision: https://reviews.llvm.org/D117068
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
Update examples and docs to demonstrate using `__lldb_init_module` instead of
the idiom that checks for `lldb.debugger` at the top-level.
```
if __name__ == '__main__':
...
elif lldb.debugger:
...
```
Is replaced with:
```
if __name__ == '__main__':
...
def __lldb_init_module(debugger, internal_dict):
...
```
This change is for two reasons. First, it's generally encouraged not to only
use the convenience singletons (`lldb.{debugger,process,target,etc}`)
interactively from the `script` command. Second, there's a bug where
registering a python class as a command (using `command script add -c ...`),
result in the command not being runnable. Note that registering function-backed
commands does not have this bug.
Differential Revision: https://reviews.llvm.org/D117237
Convert the `crashlog` command to be implemented as a class. The `Symbolicate`
function is switched to a class, to implement `get_long_help`. The text for the
long help comes from the help output generated by `OptionParser`. That is, the
output of `help crashlog` is the same as `crashlog --help`.
Differential Revision: https://reviews.llvm.org/D117165
This patch adds support for arm64(e) targets to ScriptedProcess, by
providing the `DynamicRegisterInfo` to the base `lldb.ScriptedThread` class.
This allows create and debugging ScriptedProcess on Apple Silicon
hardware as well as Apple mobile devices.
It also replace the C++ asserts on `ScriptedThread::GetDynamicRegisterInfo`
by some error logging, re-enables `TestScriptedProcess` for arm64
Darwin platforms and adds a new invalid Scripted Thread test.
rdar://85892451
Differential Revision: https://reviews.llvm.org/D114923
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This adds the formatters for libstdcpp's deque as a python
implementation. It adds comprehensive tests for the two different
storage strategies deque uses. Besides that, this fixes a couple of bugs
in the libcxx implementation. Finally, both implementation run against
the same tests.
This is a minor improvement on top of Danil Stefaniuc's formatter.
This diff is adding the capping_size determination for the list and forward list, to limit the number of children to be displayed. Also it modifies and unifies tests for libcxx and libstdcpp list data formatter.
Reviewed By: wallace
Differential Revision: https://reviews.llvm.org/D114433
We need to add checks that ensure that some core variables are valid, so
that we avoid printing out garbage data. The worst that could happen is
that an non-initialized variable is being printed as something with
123123432 children instead of 0.
Differential Revision: https://reviews.llvm.org/D114458
As suggested by @labath in https://reviews.llvm.org/D114403, we should
make the formatter more resilient to corrupted data. The Libcxx version
explicitly checks for engaged = 1, so we can do that as well for safety.
Differential Revision: https://reviews.llvm.org/D114450
This diff adds a data formatter and tests for libstdcpp's unordered_map, unordered_set, unordered_multimap, unordered_multiset
Reviewed By: wallace
Differential Revision: https://reviews.llvm.org/D113760
This patch changes the ScriptedProcess test to use a stack-only skinny
corefile as a backing store.
The corefile is saved as a temporary file at the beginning of the test,
and a second target is created for the ScriptedProcess. To do so, we use
the SBAPI from the ScriptedProcess' python script to interact with the
corefile process.
This patch also makes some small adjustments to the other ScriptedProcess
scripts to resolve some inconsistencies and removes the raw memory dump
that was previously checked in.
Differential Revision: https://reviews.llvm.org/D112047
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch changes the `ScriptedThread` initializer in couple of ways:
- It replaces the `SBTarget` parameter by a `SBProcess` (pointing to the
`ScriptedProcess` that "owns" the `ScriptedThread`).
- It adds a reference to the `ScriptedProcessInfo` Dictionary, to pass
arbitrary user-input to the `ScriptedThread`.
This patch also fixes the SWIG bindings methods that call the
`ScriptedProcess` and `ScriptedThread` initializers by passing all the
arguments to the appropriate `PythonCallable` object.
Differential Revision: https://reviews.llvm.org/D112046
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This diff adds a data formatter for libstdcpp's forward_list. Besides, it refactors the existing code by extracting the common functionality between libstdcpp forward_list and list formatters into the AbstractListSynthProvider class.
Reviewed By: wallace
Differential Revision: https://reviews.llvm.org/D113362
This diff adds a data formatter for libstdcpp's multiset. Besides, it improves and unifies the tests for multiset for libcxx and libstdcpp for maintainability.
Reviewed By: wallace
Differential Revision: https://reviews.llvm.org/D112785
This diff adds a data formatter for libstdcpp's multimap. Besides, it improves and unifies the tests for multimap for libcxx and libstdcpp for maintainability.
Reviewed By: wallace
Differential Revision: https://reviews.llvm.org/D112752
This diff adds a data formatter for libstdcpp's set. Besides, it unifies the tests for set for libcxx and libstdcpp for maintainability.
Reviewed By: wallace
Differential Revision: https://reviews.llvm.org/D112537
heap.py has a lot of large hand written expressions and each name in the
expression will be looked up by clang during expression parsing. For
function parameters this will be in Sema::ActOnParamDeclarator(...) in order to
catch redeclarations of parameters. The names are not needed and we have seen
some rare cases where since we don't have symbols we end up in
SymbolContext::FindBestGlobalDataSymbol(...) which may conflict with other global
symbols.
There may be a way to make this lookup smarter to avoid these cases but it is
not clear how well tested this path is and how much work it would be to fix it.
So we will go with this fix while we investigate more.
This is a second try at getting all the cases we care about.
Ref: rdar://78265641
This patch adds support for memory regions in Scripted Processes.
This is necessary to read the stack memory region in order to
reconstruct each stackframe of the program.
In order to do so, this patch makes some changes to the SBAPI, namely:
- Add a new constructor for `SBMemoryRegionInfo` that takes arguments
such as the memory region name, address range, permissions ...
This is used when reading memory at some address to compute the offset
in the binary blob provided by the user.
- Add a `GetMemoryRegionContainingAddress` method to `SBMemoryRegionInfoList`
to simplify the access to a specific memory region.
With these changes, lldb is now able to unwind the stack and reconstruct
each frame. On top of that, reloading the target module at offset 0 allows
lldb to symbolicate the `ScriptedProcess` using debug info, similarly to an
ordinary Process.
To test this, I wrote a simple program with multiple function calls, ran it in
lldb, stopped at a leaf function and read the registers values and copied
the stack memory into a binary file. These are then used in the python script.
Differential Revision: https://reviews.llvm.org/D108953
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch introduces the `ScriptedThread` class with its python
interface.
When used with `ScriptedProcess`, `ScriptedThreaad` can provide various
information such as the thread state, stop reason or even its register
context.
This can be used to reconstruct the program stack frames using lldb's unwinder.
rdar://74503836
Differential Revision: https://reviews.llvm.org/D107585
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
JSON crashlogs have an optional field named reportNotes that contains
any potential errors encountered by the crash reporter when generating
the crashlog. Parse and display them in LLDB.
Differential revision: https://reviews.llvm.org/D111339
Gracefully deal with JSON crashlogs that don't have thread state
available and print an error saying as much: "No thread state (register
information) available".
rdar://83955858
Differential revision: https://reviews.llvm.org/D111341
JSON crashlogs normally start with a single line of meta data that we
strip unconditionally. Some producers started omitting the meta data
which tripped up crashlog. Be more resilient by only removing the first
line when we know it really is meta data.
rdar://82641662
When adding an image to a target for crashlog purposes, avoid specifying
the architecture of the image.
This has the effect of making SBTarget::AddModule infer the ArchSpec for
the image based on the SBTarget's architecture, which LLDB puts serious
effort into calculating correctly (in TargetList::CreateTargetInternal).
The status quo is that LLDB randomly guesses the ArchSpec for a module
if its architecture is specified, via:
```
SBTarget::AddModule -> Platform::GetAugmentedArchSpec -> Platform::IsCompatibleArchitecture ->
GetSupportedArchitectureAtIndex -> {ARM,x86}GetSupportedArchitectureAtIndex
```
... which means that the same crashlog can fail to load on an Apple
Silicon Mac (due to the random guess of arm64e-apple-macosx for the
module's ArchSpec not being compatible with the SBTarget's (correct)
ArchSpec), while loading just fine on an Intel Mac.
I'm not sure how to add a test for this (it doesn't look like there's
test coverage of this path in-tree). It seems like it would be pretty
complicated to regression test: the host LLDB would need to be built for
arm64e, we'd need a hand-crafted arm64e iOS crashlog, and we'd need a
binary with an iOS deployment target. I'm open to other / simpler
options.
rdar://82679400
Differential Revision: https://reviews.llvm.org/D110013
mdfind can return multiple results, some of which are not even dSYM
bundles, but Xcode archives (.xcrachive).
Currently, we end up concatenating the paths, which is obviously bogus.
This patch not only fixes that, but now also skips paths that don't have
a Contents/Resources/DWARF subdirectory.
rdar://81270312
Differential revision: https://reviews.llvm.org/D109263
This patch introduces Scripted Processes to lldb.
The goal, here, is to be able to attach in the debugger to fake processes
that are backed by script files (in Python, Lua, Swift, etc ...) and
inspect them statically.
Scripted Processes can be used in cooperative multithreading environments
like the XNU Kernel or other real-time operating systems, but it can
also help us improve the debugger testing infrastructure by writting
synthetic tests that simulates hard-to-reproduce process/thread states.
Although ScriptedProcess is not feature-complete at the moment, it has
basic execution capabilities and will improve in the following patches.
rdar://65508855
Differential Revision: https://reviews.llvm.org/D100384
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
heap.py has a lot of large hand written expressions and each name in the
expression will be looked up by clang during expression parsing. For
function parameters this will be in Sema::ActOnParamDeclarator(...) in order to
catch redeclarations of parameters. The names are not needed and we have seen
some rare cases where since we don't have symbols we end up in
SymbolContext::FindBestGlobalDataSymbol(...) which may conflict with other global
symbols.
There may be a way to make this lookup smarter to avoid these cases but it is
not clear how well tested this path is and how much work it would be to fix it.
So we will go with this fix while we investigate more.
Ref: rdar://78265641
Med Ismail Bennani