The current LLDB Python docs are missing documentation for all the special
members such as conversion functions (`__int__`) and other special functions
(`__len__`).
The reason for that is that the `automodapi` plugin we're using to generate the
*.rst files simply doesn't emit them. There doesn't seem to be any config option
to enable those in `automodapi` and it's not even clear why they are filtered. I
assume the leading underscore in their names makes them look like private
methods.
This patch just forcibly adds a few selected special members functions to the
list of functions that sphinx should always document. This will cause sphinx to
warn if a class doesn't have one of those functions but it's better than not
having them documented.
The main motivation here is that since `SBAddress.__int__` is one of the few
functions that is only available in the embedded Python REPL which would be good
to have in the public documentation.
Fixes rdar://64647665
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D105480
References with a single '`' around them are interpreted as references instead
of text with monospaced font since the introduction of the new Python API
generator. This meant that all the single-quoted code in this document that
doesn't reference any Python class was throwing sphinx errors. This just adds
the neede extra ` around this code and fixed up the legitimate typos
(e.g. `SBframe` -> `SBFrame`).
Add a new feature to process save-core on Darwin systems -- for
lldb to create a user process corefile with only the dirty (modified
memory) pages included. All of the binaries that were used in the
corefile are assumed to still exist on the system for the duration
of the use of the corefile. A new --style option to process save-core
is added, so a full corefile can be requested if portability across
systems, or across time, is needed for this corefile.
debugserver can now identify the dirty pages in a memory region
when queried with qMemoryRegionInfo, and the size of vm pages is
given in qHostInfo.
Create a new "all image infos" LC_NOTE for Mach-O which allows us
to describe all of the binaries that were loaded in the process --
load address, UUID, file path, segment load addresses, and optionally
whether code from the binary was executing on any thread. The old
"read dyld_all_image_infos and then the in-memory Mach-O load
commands to get segment load addresses" no longer works when we
only have dirty memory.
rdar://69670807
Differential Revision: https://reviews.llvm.org/D88387
This adds a basic SB API for creating and stopping traces.
Note: This doesn't add any APIs for inspecting individual instructions. That'd be a more complicated change and it might be better to enhande the dump functionality to output the data in binary format. I'll leave that for a later diff.
This also enhances the existing tests so that they test the same flow using both the command interface and the SB API.
I also did some cleanup of legacy code.
Differential Revision: https://reviews.llvm.org/D103500
This patch specifies a few guidelines that our API tests should follow.
The motivations for this are twofold:
1. API tests have unexpected pitfalls that especially new contributors run into
when writing tests. To prevent the frustration of letting people figure those
pitfalls out by trial-and-error, let's just document them briefly in one place.
2. It prevents some arguing about what is the right way to write tests. I really
like to have fast and reliable API test suite, but I also don't want to be the
bogeyman that has to insist in every review that the test should be rewritten to
not launch a process for no good reason. It's much easier to just point to a
policy document.
I omitted some guidelines that I think could be controversial (e.g., the whole
"should assert message describe failure or success").
Reviewed By: shafik
Differential Revision: https://reviews.llvm.org/D101153
Introduce three new stop reasons for fork, vfork and vforkdone events.
This includes server support for serializing fork/vfork events into
gdb-remote protocol. The stop infos for the two base events take a pair
of PID and TID for the newly forked process.
Differential Revision: https://reviews.llvm.org/D100196
When looking up user specified formatters qualifiers are removed from types before matching,
I have added a clarifying example to the document and added an example to a relevant test to demonstrate this behavior.
Differential Revision: https://reviews.llvm.org/D99827
Debugging tests sometimes involves debugging the Python source. This adds a paragraph to
the "Debugging Test Failures" section about using `pdb`, and also describes how to run
lldb commands from pdb.
Differential Revision: https://reviews.llvm.org/D99744
Remove the remaining references to LLDB_CAPTURE_REPRODUCER. I removed
the functionality in an earlier commit but forgot that there was a
corresponding test and logic to unset it in our test suite.
This implements the interactive trace start and stop methods.
This diff ended up being much larger than I anticipated because, by doing it, I found that I had implemented in the beginning many things in a non optimal way. In any case, the code is much better now.
There's a lot of boilerplate code due to the gdb-remote protocol, but the main changes are:
- New tracing packets: jLLDBTraceStop, jLLDBTraceStart, jLLDBTraceGetBinaryData. The gdb-remote packet definitions are quite comprehensive.
- Implementation of the "process trace start|stop" and "thread trace start|stop" commands.
- Implementaiton of an API in Trace.h to interact with live traces.
- Created an IntelPTDecoder for live threads, that use the debugger's stop id as checkpoint for its internal cache.
- Added a functionality to stop the process in case "process tracing" is enabled and a new thread can't traced.
- Added tests
I have some ideas to unify the code paths for post mortem and live threads, but I'll do that in another diff.
Differential Revision: https://reviews.llvm.org/D91679
In D94489 we changed the way we build the docs and now have some additional
dependencies to generate the Python API docs. As the same sphinx project is
generating the man pages for LLDB it should have in theory the same setup code
that sets up the mocked LLDB module.
However, as we don't have that setup code the man page generation just fails as
there is no mocked LLDB module and the Python API generation errors out.
The man page anyway doesn't cover the Python API so I don't think there is any
point of going through the whole process (and requiring the sphinx plugins) just
to generate the (eventually unused) Python docs.
This patch just skips the relevant Python API generation when we are building
the man page.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D98441
Add a column to the table of convenience variables with the equivalent
API to get to the current debugger, target, process, etc.
We often get asked to make convenience variables available outside of
the interactive interpreter. After explaining why that's not possible, a
common complaint is that it's hard to find out how to get to these
variables in a non-interactive context, for example how to get to the
current frame when given a thread. This patch aims to alleviate that by
including the APIs to navigate between these instances in the table.
Differential revision: https://reviews.llvm.org/D97778
This patch changes the short option used in `CommandOptionsProcessLaunch`
for the `-v|--environment` command option to `-E|--environment`.
The reason for that is, that it collides with the `-v|--structured-data-value`
command option generated by `OptionGroupPythonClassWithDict` that
I'm using in an upcoming patch for the `process launch` command.
The long option `--environment` remains the same.
Differential Review: https://reviews.llvm.org/D95100
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
Allow users to use a non-system version of perl, python and awk, which is useful
in certain package managers.
Reviewed By: JDevlieghere, MaskRay
Differential Revision: https://reviews.llvm.org/D95119
For unknown reasons the alabaster theme on the docs server is always generating
a GitHub link in the side bar. Beside the privacy problems of having an iframe
to some third-party service, we never configured any GitHub integration so
this button just links to the GitHub main site.
The button generation should be disabled by default, but as that's apparently
not true in the alabaster theme on the server, this patch tries working around
the issue by just explicitly turning off the GitHub integration.
Apparently the sphinx version on the server doesn't place <p> tags in the
table cells, so the previous fix from commit 7fce3b240b
didn't fix the bug for that sphinx version. Just expand the CSS workaround
to all <td> tags.
This patch implements a filter that post-processes some of the generated RST sources
of the Python API docs. I mainly want to avoid two things:
1. Filter out all the inheritance boilerplate that just keeps mentioning for
every class that it inherits from the builtin 'object'. There is no inheritance
in the SB API.
2. More importantly, removes the SWIG generated `thisown` attribute from the
public documentation. I don't think we want users to mess with that attribute
and this is probably causing more confusion than it would help anyone. It also
makes the documentation for some smaller classes more verbose than necessary.
This patch just uses the sphinx event for reading source and removes the parts
that we don't want in documentation.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D94967
The tables in the new LLDB documentation currently are less wide than their
contents. The reason for that seems to be the `-webkit-hyphens: auto` property
that sphinx is setting for all `p` tags. The `p` tags in the generated Python
documentation seem to trigger some Safari layout issue, so Safari is calculating
the cell width to be smaller than it should be (which ends up looking like this
{F15104344} ).
This patch just sets that property back to the browser default `manual`. Not
sure if that's the proper workaround, but I clicked around on the website with
the changed CSS and nothing looked funny (which is I believe how webdev unit
testing works).
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D94991
This is mostly SEO so that the new API can take over the old API when people
search for the different SB* classes. Sadly epydoc decided to throw in a -class
prefix behind all the class file names, so we can't just overwrite the old files
with the newly generated ones.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D94900
Enums and constants are currently missing in the new LLDB Python API docs.
In theory we could just let them be autogenerated like the SB API classes, but sadly the generated documentation
suffers from a bunch of problems. Most of these problems come from the way SWIG is representing enums, which is
done by translating every single enum case into its own constant. This has a bunch of nasty effects:
* Because SWIG throws away the enum types, we can't actually reference the enum type itself in the API. Also because automodapi is impossible to script, this can't be fixed in post (at least without running like sed over the output files).
* The lack of enum types also causes that every enum *case* has its own full doc page. Having a full doc page that just shows a single enum case is pointless and it really slows down sphinx.
* There is no SWIG code for the enums, so there is also no place to write documentation strings for them. Also there is no support for copying the doxygen strings (which would be in the wrong format, but better than nothing) for enums (let alone our defines), so we can't really document all this code.
* Because the enum cases are just forwards to the native lldb module (which we mock), automodapi actually takes the `Mock` docstrings and adds it to every single enum case.
I don't see any way to solve this via automodapi or SWIG. The most reasonable way to solve this is IMHO to write a simple Clang tool
that just parses our enum/constant headers and emits an *.rst file that we check in. This way we can do all the LLDB-specific enum case and constant
grouping that we need to make a readable documentation page.
As we're without any real documentation until I get around to write that tool, I wrote a doc page for the enums/constants as a stop gap measure.
Most of this is done by just grepping our enum header and then manually cleaning up all the artifacts and copying the few doc strings we have.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D94959
sphinx processes text in backticks depending on what 'role' it has (e.g.,
`:code:\`blub\`` -> role is `code`). If no role is provided, the default role is
taken which is right now using the default value of `content`. `content` only
really makes the text cursive which isn't really useful for anything right now.
Sphinx recommends using the `any` role by default [1] as that turns text in
backticks without an explicit roles into some kind of smart reference. If we did
this in LLDB, then we could just reference SB API classes by doing `\`SBValue\``
instead of typing out the rather verbose `:py:class:`/`:py:func:`/... role
before each reference. This would be especially nice when writing the SB API
docs itself as we constantly have to reference other classes.
[1] https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html#role-any
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D94899
Right now we're using the 'content' role as default which will just render
these things as cursive (which isn't really useful for code examples). It also
prevents us from assigning a more useful default role in the future.
The build server should now have the missing dependencies.
Original summary:
Currently LLDB uses epydoc to generate the Python API reference for the website.
epydoc however is unmaintained since more than a decade and no longer works with
Python 3. Also whatever setup we had once for generating the documentation on
the website server no longer seems to work, so the current website documentation
has been stale since more than a year.
This patch replaces epydoc with sphinx and its automodapi plugin that can
generate Python API references. LLVM already uses sphinx for the rest of the
documentation, so this way we are more consistent with the rest of LLVM. The
only new dependency is the automodapi plugin for sphinx.
This patch effectively does the following things:
* Remove the epydoc code.
* Make a new dummy Python API page in our website that just calls the Sphinx
command for generated the API documentation.
* Add a mock _lldb module that is only used when generating the Python API.
This way we don't have to build all of LLDB to generate the API reference.
Some notes:
* The long list of skips is necessary due to boilerplate functions that SWIG
is generating. Sadly automodapi is not really scriptable from what I can see,
so we have to blacklist this stuff manually.
* The .gitignore change because automodapi wants a subfolder of our
documentation directory to place generated documentation files there. The path
is also what is used on the website, so we can't really workaround this
(without copying the whole `docs` dir somewhere else when we build).
* We have to use environment variables to pass our build path to our sphinx
configuration. Sphinx doesn't support passing variables onto that script.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D94489
Currently LLDB uses epydoc to generate the Python API reference for the website.
epydoc however is unmaintained since more than a decade and no longer works with
Python 3. Also whatever setup we had once for generating the documentation on
the website server no longer seems to work, so the current website documentation
has been stale since more than a year.
This patch replaces epydoc with sphinx and its automodapi plugin that can
generate Python API references. LLVM already uses sphinx for the rest of the
documentation, so this way we are more consistent with the rest of LLVM. The
only new dependency is the automodapi plugin for sphinx.
This patch effectively does the following things:
* Remove the epydoc code.
* Make a new dummy Python API page in our website that just calls the Sphinx
command for generated the API documentation.
* Add a mock _lldb module that is only used when generating the Python API.
This way we don't have to build all of LLDB to generate the API reference.
Some notes:
* The long list of skips is necessary due to boilerplate functions that SWIG
is generating. Sadly automodapi is not really scriptable from what I can see,
so we have to blacklist this stuff manually.
* The .gitignore change because automodapi wants a subfolder of our
documentation directory to place generated documentation files there. The path
is also what is used on the website, so we can't really workaround this
(without copying the whole `docs` dir somewhere else when we build).
* We have to use environment variables to pass our build path to our sphinx
configuration. Sphinx doesn't support passing variables onto that script.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D94489
Bump the required SWIG version to 3. If my memory serves me well we last
bumped the required SWIG version to 2 for Python 3. At that time SWIG 3
had already been around for a while so everyone I know was already using
that.
It appears that SWIG 3 is the only version that officially supports
C++11 which we're using in the typemap. SWIG 3 was released in 2014 so I
think it's reasonable to make that the minimum required version.
https://bugs.llvm.org/show_bug.cgi?id=48685
Differential revision: https://reviews.llvm.org/D94244
This extends the "memory region" command to
show tagged regions on AArch64 Linux when the MTE
extension is enabled.
(lldb) memory region the_page
[0x0000fffff7ff8000-0x0000fffff7ff9000) rw-
memory tagging: enabled
This is done by adding an optional "flags" field to
the qMemoryRegion packet. The only supported flag is
"mt" but this can be extended.
This "mt" flag is read from /proc/{pid}/smaps on Linux,
other platforms will leave out the "flags" field.
Where this "mt" flag is received "memory region" will
show that it is enabled. If it is not or the target
doesn't support memory tagging, the line is not shown.
(since majority of the time tagging will not be enabled)
Testing is added for the existing /proc/{pid}/maps
parsing and the new smaps parsing.
Minidump parsing has been updated where needed,
though it only uses maps not smaps.
Target specific tests can be run with QEMU and I have
added MTE flags to the existing helper scripts.
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D87442
Depends on D89283.
The goal of this packet (jTraceGetSupportedType) is to be able to query the gdb-server for the tracing technology that can work for the current debuggeer, which can make the user experience simpler but allowing the user to simply type
thread trace start
to start tracing the current thread without even telling the debugger to use "intel-pt", for example. Similarly, `thread trace start [args...]` would accept args beloging to the working trace type.
Also, if the user typed
help thread trace start
We could directly show the help information of the trace type that is supported for the target, or mention instead that no tracing is supported, if that's the case.
I added some simple tests, besides, when I ran this on my machine with intel-pt support, I got
$ process plugin packet send "jTraceSupportedType"
packet: jTraceSupportedType
response: {"description":"Intel Processor Trace","pluginName":"intel-pt"}
On a machine without intel-pt support, I got
$ process plugin packet send "jTraceSupportedType"
packet: jTraceSupportedType
response: E00;
Reviewed By: clayborg, labath
Differential Revision: https://reviews.llvm.org/D90490
Changed two references to developers as "he" or "him" to the more neutral "they".
Reviewed By: JDevlieghere, sylvestre.ledru
Differential Revision: https://reviews.llvm.org/D78807
The statement that lldb-server can handle
decimal and hex numbers is misleading.
(it can only handle hex with 0x prefix)
Mentioning non decimal numbers at all
is just creating more confusion for anyone
who tries to use them with lldb-server.
Differential Revision: https://reviews.llvm.org/D89383
In a discussion with Jim last week we came to the realization that often
we get asked about things that might not be documented on the website,
but that have been pretty well explained elsewhere. In those situations
it's often easier to quickly answer the question than searching for that
presentation you gave 3 years ago if you remember at all.
This often results in us having to answer the same questions over and
over again. We could add the questions and their answer to the website,
but that means we (1) have to duplicate the work and (2) now have to
maintain it.
A more efficient solution is to add a page with external resources with
the caveat that they might be outdated. That's exactly the purpose of
this patch.
I've added a few links that came to mind, but I don't want to be the
arbiter of what should and should not be included. I'd hope that over
time the community can crowd-source the best resources.
Differential revision: https://reviews.llvm.org/D89215
This patch adds a HowTo document to lldb docs which gives instruction for
setting up a virtual environment based on QEMU emulator for LLDB testing.
Instruction in this document are tested on Arm and AArch64 targets but
can easily be duplicated for other targets supported by QEMU.
This helps test LLDB in absence for modern AArch64 features not released
in publicly available hardware till date.
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D82064
Create a "Design" section for the LLDB documentation. The goal is to
have design documents that describe how the LLDB internals work.
Currently similar pages are mixed together under the "Development". The
existing pages describing the architecture, the reproducers, the
structured data plugins, and the SB API could be housed here. I hope
we'd see more pages being added here in the future.
Differential revision: https://reviews.llvm.org/D88516
The `macos-setup-codesign.sh` script has been in place for over two years. If there are no known issues, it's a good time to drop the manual steps from the docs.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D88257
This reverts commit f775fe5964.
I fixed a return type error in the original patch that was causing a test failure.
Also added a REQUIRES: python to the shell test so we'll skip this for
people who build lldb w/o Python.
Also added another test for the error printing.
This enables support for writing LLDB documentation in markdown in
addition to reStructured text. We already had documentation written in
markdown (StructuredDataPlugins and DarwinLog) which will now also be
available on the website.
This patch changes the command interpreter sourcing logic for the REPL
init file. Instead of looking for a arbitrary file name, it standardizes
the REPL init file name to match to following scheme:
`.lldbinit-<language>-repl`
This will make the naming more homogenous and the sourcing logic future-proof.
rdar://65836048
Differential Revision: https://reviews.llvm.org/D86987
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
1. Extended the gdb-remote communication related classes with disk file/directory
completion functions;
2. Added two common completion functions RemoteDiskFiles and
RemoteDiskDirectories based on the functions above;
3. Added completion for these commands:
A. platform get-file <remote-file> <local-file>;
B. platform put-file <local-file> <remote-file>;
C. platform get-size <remote-file>;
D. platform settings -w <remote-dir>;
E. platform open file <remote-file>.
4. Added related tests for client and server;
5. Updated docs/lldb-platform-packets.txt.
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D85284
This patch adds the infrastructure to have language specific REPL init
files. It's the foundation work to a following patch that will introduce
Swift REPL init file.
When lldb is launched with the `--repl` option, it will look for a REPL
init file in the home directory and source it. This overrides the
default `~/.lldbinit`, which content might make the REPL behave
unexpectedly. If the REPL init file doesn't exists, lldb will fall back
to the default init file.
rdar://65836048
Differential Revision: https://reviews.llvm.org/D86242
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
Separate the CMake logic for Lua and Python to clearly distinguish
between code specific to either scripting language and the code shared
by both.
What this patch does is:
- Move Python specific code into the bindings/python subdirectory.
- Move the Lua specific code into the bindings/lua subdirectory.
- Add the _python suffix to Python specific functions/targets.
- Fix a dependency issue that would check the binding instead of
whether the scripting language is enabled.
Note that this patch also changes where the bindings are generated,
which might affect downstream projects that check them in.
Differential revision: https://reviews.llvm.org/D85708
Although this issue is not specific to macOS, Python (in)compatibility
comes up quite often and we've been linking users to this page. This
just adds more details for this particular scenario.
Differential revision: https://reviews.llvm.org/D82507
Jim pointed out that "every time somebody has touched the documentation
on startup files they have stated that we source the application one and
then the global one, even though in actual fact we’ve never done that."
Indeed, when we read the application specific .lldbinit file, the global
one is not read. This patch updates the man page to reflect that.
This adds a new target `check-lldb-reproducers` that replaces the old
`check-lldb-repro`. The latter would only run the shell tests, while
`check-lldb-reproducers` includes the API tests as well. The new target
will be used on GreenDragon.
It's still possible to run just the shell tests with reproducers,
although now that requires crafting the lit invocation yourself. The
parameters haven't changed and are the shame for the API and shell
tests:
--param lldb-run-with-repro=capture
--param lldb-run-with-repro=replay
This patch also updates the reproducer documentation.
Before the transition to libOption it was possible to specify arguments
for the inferior without -- as long as they didn't start with a dash.
For example, the following invocations should all behave the same:
$ lldb inferior inferior-arg
$ lldb inferior -- inferior-arg
$ lldb -- inferior inferior-arg
This patch fixes that behavior, documents it and adds a test to cover
the different combinations.
Differential revision: https://reviews.llvm.org/D80165
Expand on the structure of the LLDB test suite. So far this information
has been mostly "tribal knowledge". By writing it down I hope to make it
easier to understand our test suite for anyone that's new to the
project.
Haibo told me he didn't have any issues with Python 3.8 and I was able
to confirm that. Even though we don't have bot running with 3.8, I think
it safe to mark it as supported in the docs.
Summary: The logic of the sentence made more sense when "with" is replaced with "without".
Reviewers: labath
Subscribers: lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D74895
Remove all beginning > from the sample commands as my accidental
copy-paste (multiple times...) will discard ./bin/llvm-lit which is
difficult to rebuild (I have to rm -rf and cmake it all again).
Differential Revision: https://reviews.llvm.org/D74296
When a thread stops, this checks depending on the platform if the top frame is
an abort stack frame. If so, it looks for an assert stack frame in the upper
frames and set it as the most relavant frame when found.
To do so, the StackFrameRecognizer class holds a "Most Relevant Frame" and a
"cooked" stop reason description. When the thread is about to stop, it checks
if the current frame is recognized, and if so, it fetches the recognized frame's
attributes and applies them.
rdar://58528686
Differential Revision: https://reviews.llvm.org/D73303
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
Raphael Isemann