Add a new feature to process save-core on Darwin systems -- for
lldb to create a user process corefile with only the dirty (modified
memory) pages included. All of the binaries that were used in the
corefile are assumed to still exist on the system for the duration
of the use of the corefile. A new --style option to process save-core
is added, so a full corefile can be requested if portability across
systems, or across time, is needed for this corefile.
debugserver can now identify the dirty pages in a memory region
when queried with qMemoryRegionInfo, and the size of vm pages is
given in qHostInfo.
Create a new "all image infos" LC_NOTE for Mach-O which allows us
to describe all of the binaries that were loaded in the process --
load address, UUID, file path, segment load addresses, and optionally
whether code from the binary was executing on any thread. The old
"read dyld_all_image_infos and then the in-memory Mach-O load
commands to get segment load addresses" no longer works when we
only have dirty memory.
rdar://69670807
Differential Revision: https://reviews.llvm.org/D88387
This adds a basic SB API for creating and stopping traces.
Note: This doesn't add any APIs for inspecting individual instructions. That'd be a more complicated change and it might be better to enhande the dump functionality to output the data in binary format. I'll leave that for a later diff.
This also enhances the existing tests so that they test the same flow using both the command interface and the SB API.
I also did some cleanup of legacy code.
Differential Revision: https://reviews.llvm.org/D103500
This patch specifies a few guidelines that our API tests should follow.
The motivations for this are twofold:
1. API tests have unexpected pitfalls that especially new contributors run into
when writing tests. To prevent the frustration of letting people figure those
pitfalls out by trial-and-error, let's just document them briefly in one place.
2. It prevents some arguing about what is the right way to write tests. I really
like to have fast and reliable API test suite, but I also don't want to be the
bogeyman that has to insist in every review that the test should be rewritten to
not launch a process for no good reason. It's much easier to just point to a
policy document.
I omitted some guidelines that I think could be controversial (e.g., the whole
"should assert message describe failure or success").
Reviewed By: shafik
Differential Revision: https://reviews.llvm.org/D101153
Introduce three new stop reasons for fork, vfork and vforkdone events.
This includes server support for serializing fork/vfork events into
gdb-remote protocol. The stop infos for the two base events take a pair
of PID and TID for the newly forked process.
Differential Revision: https://reviews.llvm.org/D100196
When looking up user specified formatters qualifiers are removed from types before matching,
I have added a clarifying example to the document and added an example to a relevant test to demonstrate this behavior.
Differential Revision: https://reviews.llvm.org/D99827
Debugging tests sometimes involves debugging the Python source. This adds a paragraph to
the "Debugging Test Failures" section about using `pdb`, and also describes how to run
lldb commands from pdb.
Differential Revision: https://reviews.llvm.org/D99744
Remove the remaining references to LLDB_CAPTURE_REPRODUCER. I removed
the functionality in an earlier commit but forgot that there was a
corresponding test and logic to unset it in our test suite.
This implements the interactive trace start and stop methods.
This diff ended up being much larger than I anticipated because, by doing it, I found that I had implemented in the beginning many things in a non optimal way. In any case, the code is much better now.
There's a lot of boilerplate code due to the gdb-remote protocol, but the main changes are:
- New tracing packets: jLLDBTraceStop, jLLDBTraceStart, jLLDBTraceGetBinaryData. The gdb-remote packet definitions are quite comprehensive.
- Implementation of the "process trace start|stop" and "thread trace start|stop" commands.
- Implementaiton of an API in Trace.h to interact with live traces.
- Created an IntelPTDecoder for live threads, that use the debugger's stop id as checkpoint for its internal cache.
- Added a functionality to stop the process in case "process tracing" is enabled and a new thread can't traced.
- Added tests
I have some ideas to unify the code paths for post mortem and live threads, but I'll do that in another diff.
Differential Revision: https://reviews.llvm.org/D91679
In D94489 we changed the way we build the docs and now have some additional
dependencies to generate the Python API docs. As the same sphinx project is
generating the man pages for LLDB it should have in theory the same setup code
that sets up the mocked LLDB module.
However, as we don't have that setup code the man page generation just fails as
there is no mocked LLDB module and the Python API generation errors out.
The man page anyway doesn't cover the Python API so I don't think there is any
point of going through the whole process (and requiring the sphinx plugins) just
to generate the (eventually unused) Python docs.
This patch just skips the relevant Python API generation when we are building
the man page.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D98441
Add a column to the table of convenience variables with the equivalent
API to get to the current debugger, target, process, etc.
We often get asked to make convenience variables available outside of
the interactive interpreter. After explaining why that's not possible, a
common complaint is that it's hard to find out how to get to these
variables in a non-interactive context, for example how to get to the
current frame when given a thread. This patch aims to alleviate that by
including the APIs to navigate between these instances in the table.
Differential revision: https://reviews.llvm.org/D97778
This patch changes the short option used in `CommandOptionsProcessLaunch`
for the `-v|--environment` command option to `-E|--environment`.
The reason for that is, that it collides with the `-v|--structured-data-value`
command option generated by `OptionGroupPythonClassWithDict` that
I'm using in an upcoming patch for the `process launch` command.
The long option `--environment` remains the same.
Differential Review: https://reviews.llvm.org/D95100
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
Allow users to use a non-system version of perl, python and awk, which is useful
in certain package managers.
Reviewed By: JDevlieghere, MaskRay
Differential Revision: https://reviews.llvm.org/D95119
For unknown reasons the alabaster theme on the docs server is always generating
a GitHub link in the side bar. Beside the privacy problems of having an iframe
to some third-party service, we never configured any GitHub integration so
this button just links to the GitHub main site.
The button generation should be disabled by default, but as that's apparently
not true in the alabaster theme on the server, this patch tries working around
the issue by just explicitly turning off the GitHub integration.
Apparently the sphinx version on the server doesn't place <p> tags in the
table cells, so the previous fix from commit 7fce3b240b
didn't fix the bug for that sphinx version. Just expand the CSS workaround
to all <td> tags.
This patch implements a filter that post-processes some of the generated RST sources
of the Python API docs. I mainly want to avoid two things:
1. Filter out all the inheritance boilerplate that just keeps mentioning for
every class that it inherits from the builtin 'object'. There is no inheritance
in the SB API.
2. More importantly, removes the SWIG generated `thisown` attribute from the
public documentation. I don't think we want users to mess with that attribute
and this is probably causing more confusion than it would help anyone. It also
makes the documentation for some smaller classes more verbose than necessary.
This patch just uses the sphinx event for reading source and removes the parts
that we don't want in documentation.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D94967
The tables in the new LLDB documentation currently are less wide than their
contents. The reason for that seems to be the `-webkit-hyphens: auto` property
that sphinx is setting for all `p` tags. The `p` tags in the generated Python
documentation seem to trigger some Safari layout issue, so Safari is calculating
the cell width to be smaller than it should be (which ends up looking like this
{F15104344} ).
This patch just sets that property back to the browser default `manual`. Not
sure if that's the proper workaround, but I clicked around on the website with
the changed CSS and nothing looked funny (which is I believe how webdev unit
testing works).
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D94991
This is mostly SEO so that the new API can take over the old API when people
search for the different SB* classes. Sadly epydoc decided to throw in a -class
prefix behind all the class file names, so we can't just overwrite the old files
with the newly generated ones.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D94900
Enums and constants are currently missing in the new LLDB Python API docs.
In theory we could just let them be autogenerated like the SB API classes, but sadly the generated documentation
suffers from a bunch of problems. Most of these problems come from the way SWIG is representing enums, which is
done by translating every single enum case into its own constant. This has a bunch of nasty effects:
* Because SWIG throws away the enum types, we can't actually reference the enum type itself in the API. Also because automodapi is impossible to script, this can't be fixed in post (at least without running like sed over the output files).
* The lack of enum types also causes that every enum *case* has its own full doc page. Having a full doc page that just shows a single enum case is pointless and it really slows down sphinx.
* There is no SWIG code for the enums, so there is also no place to write documentation strings for them. Also there is no support for copying the doxygen strings (which would be in the wrong format, but better than nothing) for enums (let alone our defines), so we can't really document all this code.
* Because the enum cases are just forwards to the native lldb module (which we mock), automodapi actually takes the `Mock` docstrings and adds it to every single enum case.
I don't see any way to solve this via automodapi or SWIG. The most reasonable way to solve this is IMHO to write a simple Clang tool
that just parses our enum/constant headers and emits an *.rst file that we check in. This way we can do all the LLDB-specific enum case and constant
grouping that we need to make a readable documentation page.
As we're without any real documentation until I get around to write that tool, I wrote a doc page for the enums/constants as a stop gap measure.
Most of this is done by just grepping our enum header and then manually cleaning up all the artifacts and copying the few doc strings we have.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D94959
sphinx processes text in backticks depending on what 'role' it has (e.g.,
`:code:\`blub\`` -> role is `code`). If no role is provided, the default role is
taken which is right now using the default value of `content`. `content` only
really makes the text cursive which isn't really useful for anything right now.
Sphinx recommends using the `any` role by default [1] as that turns text in
backticks without an explicit roles into some kind of smart reference. If we did
this in LLDB, then we could just reference SB API classes by doing `\`SBValue\``
instead of typing out the rather verbose `:py:class:`/`:py:func:`/... role
before each reference. This would be especially nice when writing the SB API
docs itself as we constantly have to reference other classes.
[1] https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html#role-any
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D94899
Right now we're using the 'content' role as default which will just render
these things as cursive (which isn't really useful for code examples). It also
prevents us from assigning a more useful default role in the future.
The build server should now have the missing dependencies.
Original summary:
Currently LLDB uses epydoc to generate the Python API reference for the website.
epydoc however is unmaintained since more than a decade and no longer works with
Python 3. Also whatever setup we had once for generating the documentation on
the website server no longer seems to work, so the current website documentation
has been stale since more than a year.
This patch replaces epydoc with sphinx and its automodapi plugin that can
generate Python API references. LLVM already uses sphinx for the rest of the
documentation, so this way we are more consistent with the rest of LLVM. The
only new dependency is the automodapi plugin for sphinx.
This patch effectively does the following things:
* Remove the epydoc code.
* Make a new dummy Python API page in our website that just calls the Sphinx
command for generated the API documentation.
* Add a mock _lldb module that is only used when generating the Python API.
This way we don't have to build all of LLDB to generate the API reference.
Some notes:
* The long list of skips is necessary due to boilerplate functions that SWIG
is generating. Sadly automodapi is not really scriptable from what I can see,
so we have to blacklist this stuff manually.
* The .gitignore change because automodapi wants a subfolder of our
documentation directory to place generated documentation files there. The path
is also what is used on the website, so we can't really workaround this
(without copying the whole `docs` dir somewhere else when we build).
* We have to use environment variables to pass our build path to our sphinx
configuration. Sphinx doesn't support passing variables onto that script.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D94489
Currently LLDB uses epydoc to generate the Python API reference for the website.
epydoc however is unmaintained since more than a decade and no longer works with
Python 3. Also whatever setup we had once for generating the documentation on
the website server no longer seems to work, so the current website documentation
has been stale since more than a year.
This patch replaces epydoc with sphinx and its automodapi plugin that can
generate Python API references. LLVM already uses sphinx for the rest of the
documentation, so this way we are more consistent with the rest of LLVM. The
only new dependency is the automodapi plugin for sphinx.
This patch effectively does the following things:
* Remove the epydoc code.
* Make a new dummy Python API page in our website that just calls the Sphinx
command for generated the API documentation.
* Add a mock _lldb module that is only used when generating the Python API.
This way we don't have to build all of LLDB to generate the API reference.
Some notes:
* The long list of skips is necessary due to boilerplate functions that SWIG
is generating. Sadly automodapi is not really scriptable from what I can see,
so we have to blacklist this stuff manually.
* The .gitignore change because automodapi wants a subfolder of our
documentation directory to place generated documentation files there. The path
is also what is used on the website, so we can't really workaround this
(without copying the whole `docs` dir somewhere else when we build).
* We have to use environment variables to pass our build path to our sphinx
configuration. Sphinx doesn't support passing variables onto that script.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D94489
Bump the required SWIG version to 3. If my memory serves me well we last
bumped the required SWIG version to 2 for Python 3. At that time SWIG 3
had already been around for a while so everyone I know was already using
that.
It appears that SWIG 3 is the only version that officially supports
C++11 which we're using in the typemap. SWIG 3 was released in 2014 so I
think it's reasonable to make that the minimum required version.
https://bugs.llvm.org/show_bug.cgi?id=48685
Differential revision: https://reviews.llvm.org/D94244
This extends the "memory region" command to
show tagged regions on AArch64 Linux when the MTE
extension is enabled.
(lldb) memory region the_page
[0x0000fffff7ff8000-0x0000fffff7ff9000) rw-
memory tagging: enabled
This is done by adding an optional "flags" field to
the qMemoryRegion packet. The only supported flag is
"mt" but this can be extended.
This "mt" flag is read from /proc/{pid}/smaps on Linux,
other platforms will leave out the "flags" field.
Where this "mt" flag is received "memory region" will
show that it is enabled. If it is not or the target
doesn't support memory tagging, the line is not shown.
(since majority of the time tagging will not be enabled)
Testing is added for the existing /proc/{pid}/maps
parsing and the new smaps parsing.
Minidump parsing has been updated where needed,
though it only uses maps not smaps.
Target specific tests can be run with QEMU and I have
added MTE flags to the existing helper scripts.
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D87442
Depends on D89283.
The goal of this packet (jTraceGetSupportedType) is to be able to query the gdb-server for the tracing technology that can work for the current debuggeer, which can make the user experience simpler but allowing the user to simply type
thread trace start
to start tracing the current thread without even telling the debugger to use "intel-pt", for example. Similarly, `thread trace start [args...]` would accept args beloging to the working trace type.
Also, if the user typed
help thread trace start
We could directly show the help information of the trace type that is supported for the target, or mention instead that no tracing is supported, if that's the case.
I added some simple tests, besides, when I ran this on my machine with intel-pt support, I got
$ process plugin packet send "jTraceSupportedType"
packet: jTraceSupportedType
response: {"description":"Intel Processor Trace","pluginName":"intel-pt"}
On a machine without intel-pt support, I got
$ process plugin packet send "jTraceSupportedType"
packet: jTraceSupportedType
response: E00;
Reviewed By: clayborg, labath
Differential Revision: https://reviews.llvm.org/D90490
Changed two references to developers as "he" or "him" to the more neutral "they".
Reviewed By: JDevlieghere, sylvestre.ledru
Differential Revision: https://reviews.llvm.org/D78807
The statement that lldb-server can handle
decimal and hex numbers is misleading.
(it can only handle hex with 0x prefix)
Mentioning non decimal numbers at all
is just creating more confusion for anyone
who tries to use them with lldb-server.
Differential Revision: https://reviews.llvm.org/D89383
In a discussion with Jim last week we came to the realization that often
we get asked about things that might not be documented on the website,
but that have been pretty well explained elsewhere. In those situations
it's often easier to quickly answer the question than searching for that
presentation you gave 3 years ago if you remember at all.
This often results in us having to answer the same questions over and
over again. We could add the questions and their answer to the website,
but that means we (1) have to duplicate the work and (2) now have to
maintain it.
A more efficient solution is to add a page with external resources with
the caveat that they might be outdated. That's exactly the purpose of
this patch.
I've added a few links that came to mind, but I don't want to be the
arbiter of what should and should not be included. I'd hope that over
time the community can crowd-source the best resources.
Differential revision: https://reviews.llvm.org/D89215
Jason Molenda