This patch introduces Scripted Processes to lldb.
The goal, here, is to be able to attach in the debugger to fake processes
that are backed by script files (in Python, Lua, Swift, etc ...) and
inspect them statically.
Scripted Processes can be used in cooperative multithreading environments
like the XNU Kernel or other real-time operating systems, but it can
also help us improve the debugger testing infrastructure by writting
synthetic tests that simulates hard-to-reproduce process/thread states.
Although ScriptedProcess is not feature-complete at the moment, it has
basic execution capabilities and will improve in the following patches.
rdar://65508855
Differential Revision: https://reviews.llvm.org/D95713
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch introduces Scripted Processes to lldb.
The goal, here, is to be able to attach in the debugger to fake processes
that are backed by script files (in Python, Lua, Swift, etc ...) and
inspect them statically.
Scripted Processes can be used in cooperative multithreading environments
like the XNU Kernel or other real-time operating systems, but it can
also help us improve the debugger testing infrastructure by writting
synthetic tests that simulates hard-to-reproduce process/thread states.
Although ScriptedProcess is not feature-complete at the moment, it has
basic execution capabilities and will improve in the following patches.
rdar://65508855
Differential Revision: https://reviews.llvm.org/D95713
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
In order to facilitate the writting of Scripted Processes, this patch
introduces a `ScriptedProcess` python base class in the lldb module.
The base class holds the python interface with all the - abstract -
methods that need to be implemented by the inherited class but also some
methods that can be overwritten.
This patch also provides an example of a Scripted Process with the
`MyScriptedProcess` class.
rdar://65508855
Differential Revision: https://reviews.llvm.org/D95712
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch adds a ScriptedProcess interface to the ScriptInterpreter and
more specifically, to the ScriptInterpreterPython.
This interface will be used in the C++ `ScriptProcess` Process Plugin to
call the script methods.
At the moment, not all methods are implemented, they will upstreamed in
upcoming patches.
This patch also adds helper methods to the ScriptInterpreter to
convert `SBAPI` Types (SBData & SBError) to `lldb_private` types
(DataExtractor & Status).
rdar://65508855
Differential Revision: https://reviews.llvm.org/D95711
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
Ignore `-Wreturn-type-c-linkage` diagnostics for `LLDBSwigPythonBreakpointCallbackFunction`.
The function is defined in `python-wrapper.swig` which uses `extern "C" { ... }` blocks.
The declaration of this function in `ScriptInterpreterPython.cpp` already uses these
same pragmas to silence the warning there.
This prevents `-Werror` builds from failing.
Differential Revision: https://reviews.llvm.org/D98368
This patch introduces Scripted Processes to lldb.
The goal, here, is to be able to attach in the debugger to fake processes
that are backed by script files (in Python, Lua, Swift, etc ...) and
inspect them statically.
Scripted Processes can be used in cooperative multithreading environments
like the XNU Kernel or other real-time operating systems, but it can
also help us improve the debugger testing infrastructure by writting
synthetic tests that simulates hard-to-reproduce process/thread states.
Although ScriptedProcess is not feature-complete at the moment, it has
basic execution capabilities and will improve in the following patches.
rdar://65508855
Differential Revision: https://reviews.llvm.org/D95713
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
In order to facilitate the writting of Scripted Processes, this patch
introduces a `ScriptedProcess` python base class in the lldb module.
The base class holds the python interface with all the - abstract -
methods that need to be implemented by the inherited class but also some
methods that can be overwritten.
This patch also provides an example of a Scripted Process with the
`MyScriptedProcess` class.
rdar://65508855
Differential Revision: https://reviews.llvm.org/D95712
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch adds a ScriptedProcess interface to the ScriptInterpreter and
more specifically, to the ScriptInterpreterPython.
This interface will be used in the C++ `ScriptProcess` Process Plugin to
call the script methods.
At the moment, not all methods are implemented, they will upstreamed in
upcoming patches.
This patch also adds helper methods to the ScriptInterpreter to
convert `SBAPI` Types (SBData & SBError) to `lldb_private` types
(DataExtractor & Status).
rdar://65508855
Differential Revision: https://reviews.llvm.org/D95711
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
The Python code generated by SWIG is compatible with both Python 2 and
Python 3. The -py3 option enables Python 2 incompatible features such as
function annotations and abstract base classes.
Differential revision: https://reviews.llvm.org/D96096
Enums and constants are currently missing in the new LLDB Python API docs.
In theory we could just let them be autogenerated like the SB API classes, but sadly the generated documentation
suffers from a bunch of problems. Most of these problems come from the way SWIG is representing enums, which is
done by translating every single enum case into its own constant. This has a bunch of nasty effects:
* Because SWIG throws away the enum types, we can't actually reference the enum type itself in the API. Also because automodapi is impossible to script, this can't be fixed in post (at least without running like sed over the output files).
* The lack of enum types also causes that every enum *case* has its own full doc page. Having a full doc page that just shows a single enum case is pointless and it really slows down sphinx.
* There is no SWIG code for the enums, so there is also no place to write documentation strings for them. Also there is no support for copying the doxygen strings (which would be in the wrong format, but better than nothing) for enums (let alone our defines), so we can't really document all this code.
* Because the enum cases are just forwards to the native lldb module (which we mock), automodapi actually takes the `Mock` docstrings and adds it to every single enum case.
I don't see any way to solve this via automodapi or SWIG. The most reasonable way to solve this is IMHO to write a simple Clang tool
that just parses our enum/constant headers and emits an *.rst file that we check in. This way we can do all the LLDB-specific enum case and constant
grouping that we need to make a readable documentation page.
As we're without any real documentation until I get around to write that tool, I wrote a doc page for the enums/constants as a stop gap measure.
Most of this is done by just grepping our enum header and then manually cleaning up all the artifacts and copying the few doc strings we have.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D94959
The first line of the doc string ends up on the SB API class summary at
the root page of the Python API web page of LLDB. Currently many of the
descriptions are missing or are several lines which makes the table really
hard to read.
This just adds the missing docstrings where possible and fixes the formatting
where necessary.
This translates most of the old ASCII art in our documentation to the
equivalent in restructured text (which the new version of the LLDB docs
is using).
This reverts commit f775fe5964.
I fixed a return type error in the original patch that was causing a test failure.
Also added a REQUIRES: python to the shell test so we'll skip this for
people who build lldb w/o Python.
Also added another test for the error printing.
`struct Py_buffer_RAII` definition uses explicit deleted functions which are not supported by SWIG 2 (only 3).
To get around this I moved this struct to an .h file that is included to avoid being parsed by swig.
Reviewed By: lawrence_danna
Differential Revision: https://reviews.llvm.org/D86381
LLVM install component targets needs to be in the form of: install-{target}[-stripped]
I tested with:
```
cmake ... -DLLVM_ENABLE_PROJECTS="clang;lldb" -DLLVM_DISTRIBUTION_COMPONENTS="lldb;liblldb;lldb-python-scripts;" ...
DESTDIR=... ninja install-distribution
```
@JDevlieghere `finish_swig_python_scripts` is a really weird name for a distribution component, any reason that it has to be this way?
Differential Revision: https://reviews.llvm.org/D86235
This patch is a big sed to rename the following variables:
s/PYTHON_LIBRARIES/Python3_LIBRARIES/g
s/PYTHON_INCLUDE_DIRS/Python3_INCLUDE_DIRS/g
s/PYTHON_EXECUTABLE/Python3_EXECUTABLE/g
s/PYTHON_RPATH/Python3_RPATH/g
I've also renamed the CMake module to better express its purpose and for
consistency with FindLuaAndSwig.
Differential revision: https://reviews.llvm.org/D85976
After moving python.swig and lua.swig into their respective
subdirectories, the relative paths in these files were out of date. This
fixes that and ensures the appropriate include paths are set in the SWIG
invocation.
Differential revision: https://reviews.llvm.org/D85859
Separate the CMake logic for Lua and Python to clearly distinguish
between code specific to either scripting language and the code shared
by both.
What this patch does is:
- Move Python specific code into the bindings/python subdirectory.
- Move the Lua specific code into the bindings/lua subdirectory.
- Add the _python suffix to Python specific functions/targets.
- Fix a dependency issue that would check the binding instead of
whether the scripting language is enabled.
Note that this patch also changes where the bindings are generated,
which might affect downstream projects that check them in.
Differential revision: https://reviews.llvm.org/D85708
This was reverted due to a python2-specific bug. Re-landing with a fix
for python2.
Summary:
One small step in my long running quest to improve python exception handling in
LLDB. Replace GetInteger() which just returns an int with As<long long> and
friends, which return Expected types that can track python exceptions
Reviewers: labath, jasonmolenda, JDevlieghere, vadimcn, omjavaid
Reviewed By: labath, omjavaid
Subscribers: omjavaid, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D78462
Summary:
One small step in my long running quest to improve python exception handling in
LLDB. Replace GetInteger() which just returns an int with As<long long> and
friends, which return Expected types that can track python exceptions
Reviewers: labath, jasonmolenda, JDevlieghere, vadimcn
Reviewed By: labath
Subscribers: lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D78462
Summary:
The buffer protocol does not allow us to just call PyBuffer_Release
and assume the buffer will still be there. Most things that implement the
buffer protocol will let us get away with that, but not all. We need
to release it at the end of the SWIG wrapper.
Reviewers: labath, jasonmolenda, JDevlieghere, vadimcn
Reviewed By: labath
Subscribers: lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D77480
Summary:
`SBThread.GetStopDescription` is a curious API as it takes a buffer length as a parameter that specifies
how many bytes the buffer we pass has. Then we fill the buffer until the specified length (or the length
of the stop description string) and return the string length. If the buffer is a nullptr however, we instead
return how many bytes we would have written to the buffer so that the user can allocate a buffer with
the right size and pass that size to a subsequent `SBThread.GetStopDescription` call.
Funnily enough, it is not possible to pass a nullptr via the Python SWIG bindings, so that might be the
first API in LLDB that is not only hard to use correctly but impossible to use correctly. The only way to
call this function via Python is to throw in a large size limit that is hopefully large enough to contain the
stop description (otherwise we only get the truncated stop description).
Currently passing a size limit that is smaller than the returned stop description doesn't cause the
Python bindings to return the stop description but instead the truncated stop description + uninitialized characters
at the end of the string. The reason for this is that we return the result of `snprintf` from the method
which returns the amount of bytes that *would* have been written (which is larger than the buffer).
This causes our Python bindings to return a string that is as large as full stop description but the
buffer that has been filled is only as large as the passed in buffer size.
This patch fixes this issue by just recalculating the string length in our buffer instead of relying on the wrong
return value. We also have to do this in a new type map as the old type map is also used for all methods
with the given argument pair `char *dst, size_t dst_len` (e.g. SBProcess.GetSTDOUT`). These methods have
different semantics for these arguments and don't null-terminate the returned buffer (they instead return the
size in bytes) so we can't change the existing typemap without breaking them.
Reviewers: labath, jingham
Reviewed By: labath
Subscribers: clayborg, shafik, abidh, JDevlieghere, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D72086
All the code required to generate the language bindings for Python and
Lua lives under scripts, even though the majority of this code aren't
scripts at all, and surrounded by scripts that are totally unrelated.
I've reorganized these files and moved everything related to the
language bindings into a new top-level directory named bindings. This
makes the corresponding files self contained and much more discoverable.
Differential revision: https://reviews.llvm.org/D72437
Med Ismail Bennani