The sphinx-automodapi extension requires that the generated RST files
live next to the index file. This means that we generate them in the
source directory rather than the build directory. This patch ensures
these files are removed again when sphinx finishes its build.
The proper solution to this problem would be to move everything in the
doc folder from the source directory to the build directory before
generating the docs.
I believe that old RST files being kept around is the reason that the
Python API references on the website isn't getting updated. This patch
is meant as a speculative fix and a way to confirm that.
llvm's JSON parser supports 64 bit integers, but other tools like the
ones written in JS don't support numbers that big, so we need to
represent these possibly big numbers as a string. This diff uses that to
represent addresses and tsc zero. The former is printed in hex for and
the latter in decimal string form. The schema was updated mentioning
that.
Besides that, I fixed some remaining issues and now all test pass. Before I wasn't running all tests because for some reason my computer reverted perf_paranoid to 1.
Differential Revision: https://reviews.llvm.org/D127819
As discusses offline with @jj10305, we are updating some naming used throughout the code, specially in the json schema
- traceBuffer -> iptTrace
- core -> cpu
Differential Revision: https://reviews.llvm.org/D127817
This applies the changes requested for diff 12.
- use DenseMap<ConstString, _> instead of std::unordered_map<ConstString, _>, which is more idiomatic and possibly performant.
- deduplicate some code in Trace.cpp by using helper functions for fetching in maps
- stop using size and offset when fetching binary data, because we in fact read the entire buffers all the time. If we ever need streaming, we can implement it then. Now, the size is used only to check that we are getting the correct amount of data. This is useful because in some cases determining the size doesn't involve fetching the actual data.
- added back the x86_64 macro to the perf tests
- added more documentation
- simplified some file handling
- fixed some comments
Differential Revision: https://reviews.llvm.org/D127752
:q!
This diff is massive, but it's because it connects the client with lldb-server
and also ensures that the postmortem case works.
- Flatten the postmortem trace schema. The reason is that the schema has become quite complex due to the new multicore case, which defeats the original purpose of having a schema that could work for every trace plug-in. At this point, it's better that each trace plug-in defines it's own full schema. This means that the only common field is "type".
-- Because of this new approach, I merged the "common" trace load and saving functionalities into the IntelPT one. This simplified the code quite a bit. If we eventually implement another trace plug-in, we can see then what we could reuse.
-- The new schema, which is flattened, has now better comments and is parsed better. A change I did was to disallow hex addresses, because they are a bit error prone. I'm asking now to print the address in decimal.
-- Renamed "intel" to "GenuineIntel" in the schema because that's what you see in /proc/cpuinfo.
- Implemented reading the context switch trace data buffer. I had to do
some refactors to do that cleanly.
-- A major change that I did here was to simplify the perf_event circular buffer reading logic. It was too complex. Maybe the original Intel author had something different in mind.
- Implemented all the necessary bits to read trace.json files with per-core data.
- Implemented all the necessary bits to save to disk per-core trace session.
- Added a test that ensures that parsing and saving to disk works.
Differential Revision: https://reviews.llvm.org/D126015
- Add a warnings field in the jLLDBGetState response, for warnings to be delivered to the client for troubleshooting. This removes the need to silently log lldb-server's llvm::Errors and not expose them easily to the user
- Simplify the tscPerfZeroConversion struct and schema. It used to extend a base abstract class, but I'm doubting that we'll ever add other conversion mechanisms because all modern kernels support perf zero. It is also the one who is supposed to work with the timestamps produced by the context switch trace, so expecting it is imperative.
- Force tsc collection for cpu tracing.
- Add a test checking that tscPerfZeroConversion is returned by the GetState request
- Add a pre-check for cpu tracing that makes sure that perf zero values are available.
Differential Revision: https://reviews.llvm.org/D125932
- Add collection of context switches per cpu grouped with the per-cpu intel pt traces.
- Move the state handling from the interl pt trace class to the PerfEvent one.
- Add support for stopping and enabling perf event groups.
- Return context switch entries as part of the jLLDBTraceGetState response.
- Move the triggers of whenever the process stopped or resumed. Now the will-resume notification is in a better location, which will ensure that we'll capture the instructions that will be executed.
- Remove IntelPTSingleBufferTraceUP. The unique pointer was useless.
- Add unit tests
Differential Revision: https://reviews.llvm.org/D125897
D120690 introduced `eBasicTypeChar8` but missed proper documentation order. This also introduces the missing bindings data on Swig, which should correspond with the documented information.
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D116136
Tested by building the docs-lldb-html target and
confirming the code-block renders properly with
the fix.
Patch by Michael Buch!
Differential Revision: https://reviews.llvm.org/D127437
Include a reference to the documentation for "on demand symbols" in the
documentation index. This will ensure the page shows up in the side bar
on the website.
This diffs implements per-core tracing on lldb-server. It also includes tests that ensure that tracing can be initiated from the client and that the jLLDBGetState ppacket returns the list of trace buffers per core.
This doesn't include any decoder changes.
Finally, this makes some little changes here and there improving the existing code.
A specific piece of code that can't reliably be tested is when tracing
per core fails due to permissions. In this case we add a
troubleshooting message and this is the manual test:
```
/proc/sys/kernel/perf_event_paranoid set to 1
(lldb) process trace start --per-core-tracing error: perf event syscall failed: Permission denied
You might need that /proc/sys/kernel/perf_event_paranoid has a value of 0 or -1.
``
Differential Revision: https://reviews.llvm.org/D124858
I'm refactoring IntelPTThreadTrace into IntelPTSingleBufferTrace so that it can
both single threads or single cores. In this diff I'm basically renaming the
class, moving it to its own file, and removing all the pieces that are not used
along with some basic cleanup.
Differential Revision: https://reviews.llvm.org/D124648
This updates the documentation of the gdb-remote protocol, as well as the help messages, to include the new --per-core-tracing option.
Differential Revision: https://reviews.llvm.org/D124640
In order to open perf events per core, we need to first get the list of
core ids available in the system. So I'm adding a function that does
that by parsing /proc/cpuinfo. That seems to be the simplest and most
portable way to do that.
Besides that, I made a few refactors and renames to reflect better that
the cpu info that we use in lldb-server comes from procfs.
Differential Revision: https://reviews.llvm.org/D124573
This diff introduces a new symbol on-demand which skips
loading a module's debug info unless explicitly asked on
demand. This provides significant performance improvement
for application with dynamic linking mode which has large
number of modules.
The feature can be turned on with:
"settings set symbols.load-on-demand true"
The feature works by creating a new SymbolFileOnDemand class for
each module which wraps the actual SymbolFIle subclass as member
variable. By default, most virtual methods on SymbolFileOnDemand are
skipped so that it looks like there is no debug info for that module.
But once the module's debug info is explicitly requested to
be enabled (in the conditions mentioned below) SymbolFileOnDemand
will allow all methods to pass through and forward to the actual SymbolFile
which would hydrate module's debug info on-demand.
In an internal benchmark, we are seeing more than 95% improvement
for a 3000 modules application.
Currently we are providing several ways to on demand hydrate
a module's debug info:
* Source line breakpoint: matching in supported files
* Stack trace: resolving symbol context for an address
* Symbolic breakpoint: symbol table match guided promotion
* Global variable: symbol table match guided promotion
In all above situations the module's debug info will be on-demand
parsed and indexed.
Some follow-ups for this feature:
* Add a command that allows users to load debug info explicitly while using a
new or existing command when this feature is enabled
* Add settings for "never load any of these executables in Symbols On Demand"
that takes a list of globs
* Add settings for "always load the the debug info for executables in Symbols
On Demand" that takes a list of globs
* Add a new column in "image list" that shows up by default when Symbols On
Demand is enable to show the status for each shlib like "not enabled for
this", "debug info off" and "debug info on" (with a single character to
short string, not the ones I just typed)
Differential Revision: https://reviews.llvm.org/D121631
Update the response schema of the TraceGetState packet and add
Intel PT specific response structure that contains the TSC conversion,
if it exists. The IntelPTCollector loads the TSC conversion and caches
it to prevent unnecessary calls to perf_event_open. Move the TSC conversion
calculation from Perf.h to TraceIntelPTGDBRemotePackets.h to remove
dependency on Linux specific headers.
Differential Revision: https://reviews.llvm.org/D122246
Reviewing some recent fixes to the platform packet implementations
in lldb, I saw the docs were out of sync in a few spots. Updated them.
Differential Revision: https://reviews.llvm.org/D118842
As raised here: https://lists.llvm.org/pipermail/llvm-dev/2021-November/153881.html
Now that VS2022 is on general release, LLVM is expected to build on VS2017, VS2019 and VS2022, which is proving hazardous to maintain due to changes in behaviour including preprocessor and constexpr changes. Plus of the few developers that work with VS, many have already moved to VS2019/22.
This patch proposes to raise the minimum supported version to VS2019 (16.x) - I've made the hard limit 16.0 or later, with the soft limit VS2019 16.7 - older versions of VS2019 are "allowed" (at your own risk) via the LLVM_FORCE_USE_OLD_TOOLCHAIN cmake flag.
Differential Revision: https://reviews.llvm.org/D114639
The existing instructions for lldb on Windows can be more explicit. This adds a few details on how to install various components and the easiest way to get to a working build.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D118425
Update examples and docs to demonstrate using `__lldb_init_module` instead of
the idiom that checks for `lldb.debugger` at the top-level.
```
if __name__ == '__main__':
...
elif lldb.debugger:
...
```
Is replaced with:
```
if __name__ == '__main__':
...
def __lldb_init_module(debugger, internal_dict):
...
```
This change is for two reasons. First, it's generally encouraged not to only
use the convenience singletons (`lldb.{debugger,process,target,etc}`)
interactively from the `script` command. Second, there's a bug where
registering a python class as a command (using `command script add -c ...`),
result in the command not being runnable. Note that registering function-backed
commands does not have this bug.
Differential Revision: https://reviews.llvm.org/D117237
Although we moved to Github Issues. The bug report message refers to
Bugzilla still. This patch tries to update these URLs.
Reviewed By: MaskRay, Quuxplusone, jhenderson, libunwind, libc++
Differential Revision: https://reviews.llvm.org/D116351
Support three new keys in the qProcessInfo response from the remote
gdb stub to handle the case of attaching to a core running some type
of standalone/firmware code and the stub knows the UUID and load
address-or-slide for the binary. There will be no proper DynamicLoader
plugin in this scenario, but we can try to locate and load the binary
into lldb at the correct offset.
Differential Revision: https://reviews.llvm.org/D116211
rdar://75191077
[NFC] As part of using inclusive language within the llvm project, this patch
renames master plan to controlling plan in lldb.
Reviewed By: jingham
Differential Revision: https://reviews.llvm.org/D113019
It is surprisingly difficult to write a simple python script that
can reliably `import lldb` without failing, or crashing. I'm
currently resorting to convolutions like this:
def find_lldb(may_reexec=False):
if prefix := os.environ.get('LLDB_PYTHON_PREFIX'):
if os.path.realpath(prefix) != os.path.realpath(sys.prefix):
raise Exception("cannot import lldb.\n"
f" sys.prefix should be: {prefix}\n"
f" but it is: {sys.prefix}")
else:
line1, line2 = subprocess.run(
['lldb', '-x', '-b', '-o', 'script print(sys.prefix)'],
encoding='utf8', stdout=subprocess.PIPE,
check=True).stdout.strip().splitlines()
assert line1.strip() == '(lldb) script print(sys.prefix)'
prefix = line2.strip()
os.environ['LLDB_PYTHON_PREFIX'] = prefix
if sys.prefix != prefix:
if not may_reexec:
raise Exception(
"cannot import lldb.\n" +
f" This python, at {sys.prefix}\n"
f" does not math LLDB's python at {prefix}")
os.environ['LLDB_PYTHON_PREFIX'] = prefix
python_exe = os.path.join(prefix, 'bin', 'python3')
os.execl(python_exe, python_exe, *sys.argv)
lldb_path = subprocess.run(['lldb', '-P'],
check=True, stdout=subprocess.PIPE,
encoding='utf8').stdout.strip()
sys.path = [lldb_path] + sys.path
This patch aims to replace all that with:
#!/usr/bin/env lldb-python
import lldb
...
... by adding the following features:
* new command line option: --print-script-interpreter-info. This
prints language-specific information about the script interpreter
in JSON format.
* new tool (unix only): lldb-python which finds python and exec's it.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D112973
[NFC] As part of using inclusive language within the llvm project,
this patch replaces master with main when referring to `.chm` files.
Reviewed By: teemperor
Differential Revision: https://reviews.llvm.org/D113299
Jim says:
lldb has a -Q or --source-quietly option, which supposedly does:
--source-quietly Tells the debugger to execute this one-line lldb command before any file has been loaded.
That seems like a weird description, since we don't generally use source for one line entries, but anyway, let's try it:
> $LLDB_LLVM/clean-mono/build/Debug/bin/lldb -Q "script print('I should be quiet')" a.out -O "script print('I should be before')" -o "script print('I should be after')"
(lldb) script print('I should be before')
I should be before
(lldb) target create "script print('I should be quiet')"
error: unable to find executable for 'script print('I should be quiet')'
That was weird. The first real -O gets sourced but not quietly, then the argument to the -Q gets treated as the target.
> $LLDB_LLVM/clean-mono/build/Debug/bin/lldb -Q a.out -O "script print('I should be before')" -o "script print('I should be after')"
(lldb) script print('I should be before')
I should be before
(lldb) target create "a.out"
Current executable set to '/tmp/a.out' (x86_64).
(lldb) script print('I should be after')
I should be after
Well, that's a little better, but the -Q option seems to have done nothing.
---
This fixes the description of --source-quietly, as well as causing it
to actually suppress echoing while executing the initialization
commands.
Reviewed By: jingham
Differential Revision: https://reviews.llvm.org/D112988
The recommonmark package is no longer required since all the documents
have been converted to .rst. Remove the related support code from
docs/conf.py.
Differential Revision: https://reviews.llvm.org/D112612
We had two sets of build<flavour> methods, whose bodies were largely
identical. This makes any kind of modification in their vicinity
repetitive and error-prone.
Replace each set with a single method taking an optional debug_info
parameter.
Differential Revision: https://reviews.llvm.org/D111989
Jason Molenda