forked from OSchip/llvm-project
505 lines
41 KiB
HTML
Executable File
505 lines
41 KiB
HTML
Executable File
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
|
||
<html xmlns="http://www.w3.org/1999/xhtml">
|
||
<head>
|
||
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
|
||
<link href="style.css" rel="stylesheet" type="text/css" />
|
||
<title>LLDB Projects</title>
|
||
</head>
|
||
|
||
<body>
|
||
<div class="www_title">
|
||
Project ideas for the <strong>LLDB</strong> Debugger
|
||
</div>
|
||
|
||
<div id="container">
|
||
<div id="content">
|
||
<!--#include virtual="sidebar.incl"-->
|
||
<div id="middle">
|
||
<div class="post">
|
||
<h1 class ="postheader">Projects</h1>
|
||
<div class="postcontent">
|
||
|
||
<p>The following is a mostly unordered set of the ideas for improvements
|
||
to the LLDB debugger. Some are fairly deep, some would require less
|
||
effort.
|
||
</p>
|
||
|
||
<ol>
|
||
<li>
|
||
Speed up type realization in lldb.
|
||
<p>
|
||
The type of problem I'm addressing here is the situation where you are
|
||
debugging a large program (lldb built with debug clang/swift will do)
|
||
and you go to print a simple expression, and lldb goes away for 30 seconds.
|
||
When you sample it, it is always busily churning through all the CU's in the
|
||
world looking for something. The problem isn't that looking for something in
|
||
particular is slow, but rather that we somehow turned an bounded search
|
||
(maybe a subtype of "std::string" into an unbounded search (all things with the
|
||
name of that subtype.) Or didn't stop when we got a reasonable answer
|
||
proximate to the context of the search, but let the search leak out globally.
|
||
And quite likely there are other issues that I haven't guessed yet.
|
||
But if you end up churning though 3 or 4 Gig of debug info, that's going to be slow
|
||
no matter how well written your debug reader is...
|
||
</p>
|
||
<p>
|
||
My guess is the work will be more in the general symbol lookup than in the DWARF
|
||
parser in particular, but it may be a combination of both.
|
||
</p>
|
||
<p>
|
||
As a user debugging a largish program, this is the most obvious lameness of lldb.
|
||
</p>
|
||
</li>
|
||
|
||
<li>
|
||
Symbol name completion in the expression parser.
|
||
<p>
|
||
This is the other obvious lameness of lldb. You can do:
|
||
</p>
|
||
<code><pre><tt>
|
||
(lldb) frame var foo.b<TAB>
|
||
</tt></pre></code>
|
||
<p>
|
||
and we will tell you it is "foo.bar". But you can't do that in the expression parser.
|
||
This will require collaboration with the clang/swift folks to get the right extension
|
||
points in the compiler. And whatever they are, lldb will need use them to tell the
|
||
compiler about what names are available. It will be important to avoid the pitfalls
|
||
of #1 where we wander into the entire DWARF world.
|
||
</p>
|
||
|
||
</li>
|
||
<li>
|
||
Make a high speed asynchronous communication channel to replace the gdb-remote protocol.
|
||
<p>
|
||
All lldb debugging nowadays is done by talking to a debug agent. We used the
|
||
gdb-remote protocol because that is universal, and good enough, and you have
|
||
to support it anyway since so many little devices & JTAG's and VM's etc
|
||
support it. But it is really old, not terribly high performance, and
|
||
can't really handle sending or receiving messages while the process is
|
||
supposedly running. It should have compression built in, remove the hand-built
|
||
checksums and rely on the robust communication protocols we always have nowadays,
|
||
allow for out-of-order requests/replies, allow for reconnecting to a temporarily
|
||
disconnected debug session, regularize all of the packet formatting into JSON or
|
||
BSON or whatever while including a way to do large binary transfers. It must be
|
||
possible to come up with something faster, and better tunable for the many
|
||
communications pathways we end up supporting.
|
||
</p>
|
||
</li>
|
||
|
||
<li>
|
||
Fix local variable lookup in the lldb expression parser.
|
||
|
||
<p>
|
||
The injection of local variables into the clang expression parser is
|
||
currently done incorrectly - it happens too late in the lookup. This results
|
||
in namespace variables & functions, same named types and ivars shadowing
|
||
locals when it should be the other way around. An attempt was made to fix
|
||
this by manually inserting all the visible local variables into wrapper function
|
||
in the expression text. This mostly gets the job done but that method
|
||
means you have to realize all the types
|
||
and locations of all local variables for even the simplest of expressions, and
|
||
when run on large programs (e.g. lldb) it would cause unacceptable delays. And
|
||
it was very fragile since an error in realizing any of the locals would cause
|
||
all expressions run in that context to fail. We need to fix this by adjusting
|
||
the points where name lookup calls out to lldb in clang.
|
||
</p>
|
||
</li>
|
||
<li>
|
||
Fix the event handling/process control machinery to support calling SB & Commands
|
||
everywhere, and to support non-stop debugging
|
||
<p>
|
||
There is a fairly ad-hoc system to handle when it is safe to run SB API's and
|
||
command line commands. This is actually a bit of a tricky problem, since we
|
||
allow access to the command line and SB API from some funky places in lldb.
|
||
The Operating System plugins are the most obvious instance, since they get
|
||
run right after lldb is told by debugserver that the process has stopped, but
|
||
before it has finished collating the information from the stop for presentation
|
||
to the higher levels. But breakpoint callbacks have some of the same problems,
|
||
and other things like the scripted stepping operations and any fancier extension
|
||
points we want to add to the debugger are going to be hard to implement robustly
|
||
till we work on a finer-grained and more explicit control over who gets to control
|
||
the process state.
|
||
<p>
|
||
We also won't have any chance of supporting non-stop debugging - which is a useful
|
||
mode for programs that have a lot of high-priority or real-time worker threads -
|
||
until we get this sorted out.
|
||
</p>
|
||
</li>
|
||
|
||
<li>
|
||
Finish the language abstraction and remove all the unnecessary C++/clang-based API's
|
||
<p>
|
||
An important part of making lldb a more useful "debugger toolkit" as opposed to a
|
||
C/C++/ObjC/Swift debugger is to have a clean abstraction for language support.
|
||
We did most, but not all, of the physical separation. We need to finish that.
|
||
And then by force of necessity the API's really look like the interface to a C++
|
||
type system with a few swift bits added on. How you would go about adding a new
|
||
language is unclear and much more trouble than it is worth at present. But if
|
||
we made this nice, we could add a lot of value to other language projects.
|
||
</p>
|
||
</li>
|
||
|
||
<li>
|
||
Add some syntax to generate data formatters from type definitions
|
||
<p>
|
||
Uses of the data formatters fall into two types. There are data formatters for
|
||
types where the structure elements pretty much tell you how to present the data,
|
||
you just need a little expression language to express how to turn them into what
|
||
the user expects to see. Then there are the ones (like pretty much all our
|
||
Foundation/AppKit/UIKit formatters) that use deep magic to figure out how the type
|
||
is actually laid out. The latter are pretty much always going to have to be done by hand.
|
||
</p>
|
||
<p>
|
||
But for the ones where the information is expressed in the fields, it would be great
|
||
to have a way to express the instructions to produce summaries and children in some
|
||
form you could embed next to the types and have the compiler produce a byte code
|
||
form of the instructions and then make that available to lldb along with the library.
|
||
This isn't as simple as having clang run over the headers and produce something from the
|
||
types directly. After all, clang has no way of knowing that the interesting thing about
|
||
a std::vector is the elements that you get by calling size (for the summary) and [] for
|
||
the elements. But it shouldn't be hard to come up with a generic markup to express this.
|
||
</p>
|
||
</li>
|
||
|
||
<li>
|
||
Allow the expression parser to access dynamic type/data formatter information
|
||
<p>
|
||
This seems like a smaller one. The symptom is your object is Foo child of Bar, and in
|
||
the Locals view you see all the fields of Foo, but because the static type of the object
|
||
is Bar, you can't see any of the fields of Foo. But if you could get this working,
|
||
you could hijack the mechanism to make the results of the value object summaries/synthetic
|
||
children available to expressions. And if you can do that, you could add other properties
|
||
to an object externally (through Python or some other extension point) and then have
|
||
these also available in the expression parser. You could use this to express invariants
|
||
for data structures, or other more advanced uses of types in the debugger.
|
||
</p>
|
||
<p>
|
||
Another version of this is to allow access to synthetic children in the expression
|
||
parser. Otherwise you end up in situations like:
|
||
</p>
|
||
<code><pre><tt>
|
||
(lldb) print return_a_foo()
|
||
(SomeVectorLikeType) $1 = {
|
||
[0] = 0
|
||
[1] = 1
|
||
[2] = 2
|
||
[3] = 3
|
||
[4] = 4
|
||
}
|
||
</tt></pre></code>
|
||
|
||
<p>
|
||
That's good but:
|
||
</p>
|
||
<code><pre><tt>
|
||
(lldb) print return_a_foo()[2]
|
||
</tt></pre></code>
|
||
<p>
|
||
fails because the expression parser doesn't know anything about the array-like nature of
|
||
SomeVectorLikeType that it gets from the synthetic children.
|
||
</p>
|
||
</li>
|
||
|
||
<li>
|
||
Recover thread information lazily
|
||
<p>
|
||
LLDB stores all the user intentions for a thread in the ThreadPlans stored in
|
||
the Thread class. That allows us to reliably implement a very natural model for
|
||
users moving through a debug session. For example, if step-over stops at a breakpoint
|
||
in an function in a younger region of the stack, continue will complete the step-over
|
||
rather than having to manually step out. But that means that it is important that
|
||
the Thread objects live as long as the Threads they represent. For programs with many
|
||
threads, but only one that you are debugging, that makes stepping less efficient,
|
||
since now you have to fetch the thread list on every step or stepping doesn't work
|
||
correctly. This is especially an issue when the threads are provided by an
|
||
Operating System plugin, where it may take non-trivial work to reconstruct the
|
||
thread list. It would be better to fetch threads lazily but keep "unseen" threads in
|
||
a holding area, and only retire them when we know we've fetched the whole thread list
|
||
and ensured they are no longer alive.
|
||
</p>
|
||
</li>
|
||
<li>
|
||
Add an extension point in the breakpoint search machinery.
|
||
<p>
|
||
This would allow highly customizable, algorithmic breakpoint types, like "break on every
|
||
use of some particular instruction, or instruction pattern, etc."
|
||
</p>
|
||
</li>
|
||
|
||
<li>
|
||
Make Python-backed commands first class citizens
|
||
<p>
|
||
As it stands, Python commands have no way to advertise their options. They are
|
||
required to parse their arguments by hand. That leads to inconsistency, and more
|
||
importantly means they can't take advantage of auto-generated help and command
|
||
completion. This leaves python-backed commands feeling worse than built-in ones.
|
||
</p>
|
||
<p>
|
||
As part of this job, it would also be great to hook automatically hook the "type" of an option value
|
||
or argument (e.g. eArgTypeShlibName) to sensible default completers. You need to be able to
|
||
over-ride this in more complicated scenarios (like in "break set" where the presence of
|
||
a "-s" option limits the search for completion of a "-n" option.) But in common cases it is
|
||
unnecessary busy-work to have to supply the completer AND the type. If this worked, then
|
||
it would be easier for Python commands to also get correct completers.
|
||
</p>
|
||
</li>
|
||
|
||
<li>
|
||
Reimplement the command interpreter commands using the SB API
|
||
<p>
|
||
Currently, all the CommandObject::DoExecute methods are implemented
|
||
using the lldb_private API's. That generally means that there's code
|
||
that gets duplicated between the CommandObject and the SB API that does
|
||
roughly the same thing. We would reduce this code duplication, present a
|
||
single coherent face to the users of lldb, and keep
|
||
ourselves more honest about what we need in the SB API's if we implemented
|
||
the CommandObjects::DoExecute methods using the SB API's.
|
||
</p>
|
||
<p>
|
||
BTW, it is only the way it was much easier to develop lldb if it had a functioning
|
||
command-line early on. So we did that first, and developed the SB API's when lldb
|
||
was more mature. There's no good technical reason to have the commands use the
|
||
lldb_private API's.
|
||
</p>
|
||
</li>
|
||
|
||
<li>
|
||
Documentation and better examples
|
||
|
||
<p>
|
||
We need to put the lldb syntax docs in the tutorial somewhere that is more easily accessible.
|
||
On suggestion is to add non-command based help to the help system, and then have a "help lldb"
|
||
or "help syntax" type command with this info. Be nice if the non-command based help could
|
||
be hierarchical so you could make topics.
|
||
</p>
|
||
|
||
<p>
|
||
There's a fair bit of docs about the SB API's, but it is spotty. Some classes are
|
||
well documented in the Python "help (lldb.SBWhatever)" and some are not.
|
||
</p>
|
||
|
||
|
||
<p>
|
||
We need more conceptual docs. And we need more examples. And we could provide a
|
||
clean pluggable example for using LLDB standalone from Python. The process_events.py
|
||
is a start of this, but it just handles process events, and it is really a quick sketch
|
||
not a polished expandable proto-tool.
|
||
</p>
|
||
|
||
</li>
|
||
|
||
<li>
|
||
Make a more accessible plugin architecture for lldb.
|
||
|
||
<p>
|
||
Right now, you can only use the Python or SB API's to extend an extant lldb. You can't
|
||
implement any of the actual lldb Plugins as plugins. That means anybody that wants to
|
||
add new Object file/Process/Language etc support has to build and distribute their own
|
||
lldb. This is tricky because the API's the plugins use are currently not stable (and
|
||
recently have been changing quite a lot.) We would have to define a subset of lldb_private
|
||
that you could use, and some way of telling whether the plugins were compatible with
|
||
the lldb. But long-term, making this sort of extension possible will make lldb more
|
||
appealing for research and 3rd party uses.
|
||
</p>
|
||
</li>
|
||
|
||
<li>
|
||
Use instruction emulation to avoid the overhead of swapping trap and instruction for breakpoints
|
||
|
||
<p>
|
||
At present, breakpoints are implemented by inserting a trap instruction, then when the trap
|
||
is hit, replace the trap with the actual instruction and single step. Then swap back and
|
||
continue. This causes problems for read only text, and also means that no-stop debugging
|
||
ust either stop all threads briefly to handle this two-step or risk missing some
|
||
breakpoint hits. If you emulated the instruction and wrote back the results, you wouldn't
|
||
have these problems, and it would also save a stop per breakpoint hit. Since we use breakpoints
|
||
to implement stepping, this savings could be significant on slow connections.
|
||
</p>
|
||
</li>
|
||
<li>
|
||
Use the JIT to speed up conditional breakpoint evaluation
|
||
|
||
<p>
|
||
We already JIT and cache the conditional expressions for breakpoints for the C family of
|
||
languages, so we aren't re-compiling every time you hit the breakpoint. And if we couldn't
|
||
IR interpret the expression, we leave the JIT'ed code in place for reuse. But it would
|
||
be even better if we could also insert the "stop or not" decision into the code at the
|
||
breakpoint, so you would only actually stop the process when the condition was true.
|
||
Greg's idea was that if you had a conditional breakpoint set when you started the
|
||
debug session, Xcode could rebuild and insert enough no-ops that we could instrument
|
||
the breakpoint site and call the conditional expression, and only trap if the conditional was true.
|
||
</p>
|
||
</li>
|
||
|
||
<li>
|
||
Broaden the idea in "target stop-hook" to cover more events in the debugger
|
||
|
||
<p>
|
||
Shared library loads, command execution, User directed memory/register reads and writes
|
||
are all places where you would reasonably want to hook into the debugger.
|
||
</p>
|
||
</li>
|
||
|
||
<li>
|
||
Mock classes for testing
|
||
<p>
|
||
We need "ProcessMock" and "ObjectFileMock" and the like. These would be real
|
||
plugin implementations for their underlying lldb classes, with the addition
|
||
that you can prime them from some sort of text based input files. For classes
|
||
that manage changes over time (like process) you would need to program the state
|
||
at StopPoint 0, StopPoint 1, etc. These could then be used for testing reactions
|
||
to complex threading
|
||
problems & the like, and also for simulating hard-to-test environments (like bare
|
||
board debugging).
|
||
</p>
|
||
</li>
|
||
|
||
<li>
|
||
A Bug-Trapper infrastructure
|
||
<p>
|
||
We very often have bugs that can't be reproduced locally. So having a bug-report-trapper
|
||
that can gather enough information from the
|
||
surroundings of a bug so that we can replay the session locally would be a big help
|
||
tracking down issues in this situation. This is tricky because you
|
||
can't necessarily require folks to leak information about their code in order to
|
||
file bug reports. So not only will you have to figure out what state to gather,
|
||
you're also going to have to anonymize it somehow. But we very often have bugs
|
||
from people that can't reduce the problem to a simple test case and can't give us
|
||
our code, and we often just can't help them as things stand now. Note that adding
|
||
the ProcessMock would be a good first stage towards this, since you could make a ProcessMock
|
||
creator/serializer from the current lldb state.
|
||
</p>
|
||
</li>
|
||
|
||
<li>
|
||
Expression parser needs syntax for "{symbol,type} A in CU B.cpp" etc.
|
||
|
||
<p>
|
||
Sometimes you need to specify non-visible or ambiguous types to the expression parser.
|
||
We were planning to do $b_dot_cpp$A or something like. You might want to specify a
|
||
static in a function, in a source file, or in a shared library. So the syntax should
|
||
support all these.
|
||
</p>
|
||
</li>
|
||
|
||
<li>
|
||
Add a "testButDontAbort" style test to the UnitTest framework.
|
||
|
||
<p>
|
||
The way we use unittest now (maybe this is the only way it can work, I don't know)
|
||
you can't report a real failure and continue with the test. That is appropriate
|
||
in some cases: if I'm supposed to hit breakpoint A before I evaluate an expression,
|
||
and don't hit breakpoint A, the test should fail. But it means that if I want to
|
||
test five different expressions, I can either do it in one test, which is good
|
||
because it means I only have to fire up one process, attach to it, and get it to
|
||
a certain point. But it also means if the first test fails, the other four don't
|
||
even get run. So though at first we wrote a bunch of test like this, as time went
|
||
on we switched more to writing "one at a time" tests because they were more robust
|
||
against a single failure. That makes the test suite run much more slowly. It
|
||
would be great to add a "test_but_dont_abort" variant of the tests, then we could
|
||
gang tests that all drive to the same place and do similar things. As an added
|
||
benefit, it would allow us to be more thorough in writing tests, since each
|
||
test would have lower costs.
|
||
</p>
|
||
</li>
|
||
|
||
<li>
|
||
Convert the dotest style tests to use lldbutil.run_to_source_breakpoint.
|
||
|
||
<p>
|
||
run_to_source_breakpoint & run_to_name_breakpoint provide a compact API that
|
||
does in one line what the first 10 or 20 lines of most of the old tests now do by
|
||
hand. Using these functions makes tests much more readable, and by centralizing
|
||
common functionality will make maintaining the testsuites easier in the future.
|
||
This is more of a finger exercise, and perhaps best implemented by a rule like:
|
||
"If you touch a test case, and it isn't using run_to_source_breakpoint, please
|
||
make it do so".
|
||
</p>
|
||
</li>
|
||
|
||
<li>
|
||
Unify Watchpoint's & Breakpoints.
|
||
<p>
|
||
Option handling isn't shared, and more importantly the PerformAction's have a lot
|
||
of duplicated common code, most of which works less well on the Watchpoint side.
|
||
</p>
|
||
</li>
|
||
|
||
<li>
|
||
Reverse debugging.
|
||
<p>
|
||
This is kind of a holy grail, it's hard to support for complex apps (many
|
||
threads, shared memory, etc.) But it would be SO nice to have...
|
||
</p>
|
||
</li>
|
||
|
||
<li>
|
||
Non-stop debugging.
|
||
|
||
<p>
|
||
By this I mean allowing some threads in the target program to run while stopping
|
||
other threads. This is supported in name in lldb at present, but lldb makes the
|
||
assumption "If I get a stop, I won't get another stop unless I actually run the
|
||
program." in a bunch of places so getting it to work reliably will be some a
|
||
good bit of work. And figuring out how to present this in the UI will also be tricky.
|
||
</p>
|
||
</li>
|
||
|
||
<li>
|
||
Fix and continue.
|
||
<p>
|
||
We did this in gdb without a real JIT. The implementation shouldn't be that hard,
|
||
especially if you can build the executable for fix and continue. The tricky part is
|
||
how to verify that the user can only do the kinds of fixes that are safe to do.
|
||
No changing object sizes is easy to detect, but there were many more subtle changes
|
||
(function you are fixing is on the stack...) that take more work to prevent.
|
||
And then you have to explain these conditions the user in some helpful way.
|
||
</p>
|
||
</li>
|
||
<li>
|
||
Unified IR interpreter.
|
||
<p>
|
||
Currently IRInterpreter implements a portion of the LLVM IR, but it doesn't handle
|
||
vector data types and there are plenty of instructions it also doesn't support.
|
||
Conversely, lli supports most of LLVM's IR but it doesn't handle remote memory and
|
||
its function calling support is very rudimentary. It would be useful to unify these
|
||
and make the IR interpreter -- both for LLVM and LLDB -- better. An alternate strategy
|
||
would be simply to JIT into the current process but have callbacks for non-stack memory
|
||
access.
|
||
</p>
|
||
</li>
|
||
<li>
|
||
Teach lldb to predict exception propagation at the throw site
|
||
|
||
<p>
|
||
There are a bunch of places in lldb where we need to know at the point where an
|
||
exception is thrown, what frame will catch the exception.
|
||
</p>
|
||
<p>
|
||
For instance, if an expression throws an exception, we need to know whether the
|
||
exception will be caught in the course of the expression evaluation. If so it
|
||
would be safe to let the expression continue. But since we would destroy the
|
||
state of the thread if we let the exception escape the expression, we currently
|
||
stop the expression evaluation if we see a throw. If we knew where it would be
|
||
caught we could distinguish these two cases.
|
||
</p>
|
||
<p>
|
||
Similarly, when you step over a call that throws, you want to stop at the throw
|
||
point if you know the exception will unwind past the frame you were stepping in,
|
||
but it would annoying to have the step abort every time an exception was thrown.
|
||
If we could predict the catching frame, we could do this right.
|
||
</p>
|
||
<p>
|
||
And of course, this would be a useful piece of information to display when stopped
|
||
at a throw point.
|
||
</p>
|
||
</li>
|
||
</ol>
|
||
</div>
|
||
<div class="postfooter"></div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</body>
|
||
</html>
|