forked from OSchip/llvm-project
324 lines
19 KiB
HTML
324 lines
19 KiB
HTML
|
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
|
||
|
|
<html xmlns="http://www.w3.org/1999/xhtml">
|
||
|
|
<head>
|
||
|
|
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
|
||
|
|
<link href="style.css" rel="stylesheet" type="text/css" />
|
||
|
|
<title>Symbolicating with LLDB</title>
|
||
|
|
</head>
|
||
|
|
|
||
|
|
<body>
|
||
|
|
<div class="www_title">
|
||
|
|
The <strong>LLDB</strong> Debugger
|
||
|
|
</div>
|
||
|
|
|
||
|
|
<div id="container">
|
||
|
|
<div id="content">
|
||
|
|
|
||
|
|
<!--#include virtual="sidebar.incl"-->
|
||
|
|
|
||
|
|
<div id="middle">
|
||
|
|
<div class="post">
|
||
|
|
<h1 class="postheader">Manual Symbolication with LLDB</h1>
|
||
|
|
<div class="postcontent">
|
||
|
|
<p>LLDB is separated into a shared library that contains the core of the debugger,
|
||
|
|
and a driver that implements debugging and a command interpreter. LLDB can be
|
||
|
|
used to symbolicate your crash logs and can often provide more information than
|
||
|
|
other symbolication programs:
|
||
|
|
</p>
|
||
|
|
<ul>
|
||
|
|
<li>Inlined functions</li>
|
||
|
|
<li>Variables that are in scope for an address along with their locations</li>
|
||
|
|
</ul>
|
||
|
|
<p>The simplest form of symbolication is to just load an executable:</p>
|
||
|
|
<code><pre><tt><b>(lldb)</b> target create --no-dependents --arch x86_64 /tmp/a.out
|
||
|
|
</tt></pre></code>
|
||
|
|
<p>We use the "--no-dependents" flag with the "target create" command so
|
||
|
|
that we don't load all of the dependent shared libraries from the current
|
||
|
|
OS. When we symbolicate, we are often symbolicating a binary that
|
||
|
|
was running on another system, and even though the main executable might
|
||
|
|
reference shared libraries in "/usr/lib", we often don't want to load
|
||
|
|
the versions on the current computer.</p>
|
||
|
|
<p>Using the "image list" command will show us a list of all shared libraries
|
||
|
|
associated with the current target. As expected, we currently only have a single
|
||
|
|
binary:
|
||
|
|
</p>
|
||
|
|
<code><pre><tt><b>(lldb)</b> image list
|
||
|
|
[ 0] 73431214-6B76-3489-9557-5075F03E36B4 0x0000000100000000 /tmp/a.out
|
||
|
|
/tmp/a.out.dSYM/Contents/Resources/DWARF/a.out
|
||
|
|
</tt></pre></code>
|
||
|
|
|
||
|
|
<p>Now we can lookup an address:</p>
|
||
|
|
<code><pre><tt><b>(lldb)</b> image lookup --address 0x100000aa3
|
||
|
|
Address: a.out[0x0000000100000aa3] (a.out.__TEXT.__text + 131)
|
||
|
|
Summary: a.out`main + 67 at main.c:13
|
||
|
|
</tt></pre></code>
|
||
|
|
<p>Since we haven't specified a slide or any load addresses for individual sections
|
||
|
|
in the binary, the address that we use here is a <b>file</b> address. A <b>file</b>
|
||
|
|
address refers to a virtual address as defined by each object file.
|
||
|
|
</p>
|
||
|
|
<p>If we didn't use the "--no-dependents" option with "target create", we would
|
||
|
|
have loaded all dependent shared libraries:<p>
|
||
|
|
<code><pre><tt><b>(lldb)</b> image list
|
||
|
|
[ 0] 73431214-6B76-3489-9557-5075F03E36B4 0x0000000100000000 /tmp/a.out
|
||
|
|
/tmp/a.out.dSYM/Contents/Resources/DWARF/a.out
|
||
|
|
[ 1] 8CBCF9B9-EBB7-365E-A3FF-2F3850763C6B 0x0000000000000000 /usr/lib/system/libsystem_c.dylib
|
||
|
|
[ 2] 62AA0B84-188A-348B-8F9E-3E2DB08DB93C 0x0000000000000000 /usr/lib/system/libsystem_dnssd.dylib
|
||
|
|
[ 3] C0535565-35D1-31A7-A744-63D9F10F12A4 0x0000000000000000 /usr/lib/system/libsystem_kernel.dylib
|
||
|
|
...
|
||
|
|
</tt></pre></code>
|
||
|
|
|
||
|
|
|
||
|
|
<p>Now if we do a lookup using a <b>file</b> address, this can result in multiple
|
||
|
|
matches since most shared libraries have a virtual address space that starts at zero:</p>
|
||
|
|
<code><pre><tt><b>(lldb)</b> image lookup -a 0x1000
|
||
|
|
Address: a.out[0x0000000000001000] (a.out.__PAGEZERO + 4096)
|
||
|
|
|
||
|
|
Address: libsystem_c.dylib[0x0000000000001000] (libsystem_c.dylib.__TEXT.__text + 928)
|
||
|
|
Summary: libsystem_c.dylib`mcount + 9
|
||
|
|
|
||
|
|
Address: libsystem_dnssd.dylib[0x0000000000001000] (libsystem_dnssd.dylib.__TEXT.__text + 456)
|
||
|
|
Summary: libsystem_dnssd.dylib`ConvertHeaderBytes + 38
|
||
|
|
|
||
|
|
Address: libsystem_kernel.dylib[0x0000000000001000] (libsystem_kernel.dylib.__TEXT.__text + 1116)
|
||
|
|
Summary: libsystem_kernel.dylib`clock_get_time + 102
|
||
|
|
...
|
||
|
|
</tt></pre></code>
|
||
|
|
<p>To avoid getting multiple file address matches, you can specify the
|
||
|
|
<b>name</b> of the shared library to limit the search:</p>
|
||
|
|
<code><pre><tt><b>(lldb)</b> image lookup -a 0x1000 <b>a.out</b>
|
||
|
|
Address: a.out[0x0000000000001000] (a.out.__PAGEZERO + 4096)
|
||
|
|
</tt></pre></code>
|
||
|
|
</div>
|
||
|
|
<div class="postfooter"></div>
|
||
|
|
</div>
|
||
|
|
<div class="post">
|
||
|
|
<h1 class="postheader">Defining Load Addresses for Sections</h1>
|
||
|
|
<div class="postcontent">
|
||
|
|
<p>When symbolicating your crash logs, it can be tedious if you always have to
|
||
|
|
fixup your addresses into file specified addresses. To avoid having to do any
|
||
|
|
conversion, you can set the load address for the sections of the modules in your target.
|
||
|
|
Once you set any section load address locations, lookups will switch to using
|
||
|
|
<b>load</b> addresses. You can slide all sections in the executable by the same amount,
|
||
|
|
or set the <b>load</b> address if individual sections. The
|
||
|
|
"target moodules load" command allows us to set the <b>load</b> address for
|
||
|
|
the sections.
|
||
|
|
<p>Below is an example of sliding all sections in <b>a.out</b> by adding 0x123000 to each section's <b>file</b> address:</p>
|
||
|
|
<code><pre><tt><b>(lldb)</b> target create --no-dependents --arch x86_64 /tmp/a.out
|
||
|
|
<b>(lldb)</b> target modules load --file a.out --slide 0x123000
|
||
|
|
</tt></pre></code>
|
||
|
|
<p>It is often much easier to just specify the actual load location of each section by name.
|
||
|
|
Crash logs on MacOSX actuall have a <b>Binary Images</b> section that specifies
|
||
|
|
that address of the __TEXT segments for each binary. Computing the slide can be tricky
|
||
|
|
for executables and you need to be careful to calculate the slide correctly. If you specify the
|
||
|
|
address of the __TEXT segment, you don't need to do any calculations. To specify
|
||
|
|
the load addresses of sections we can specify one or more section name + address pairs
|
||
|
|
in the "target modules load" command:</p>
|
||
|
|
<code><pre><tt><b>(lldb)</b> target create --no-dependents --arch x86_64 /tmp/a.out
|
||
|
|
<b>(lldb)</b> target modules load --file a.out __TEXT 0x100123000
|
||
|
|
</tt></pre></code>
|
||
|
|
<p>We specified that the <b>__TEXT</b> section is loaded at 0x100123000.
|
||
|
|
Now that we have defined where sections have been loaded in our target,
|
||
|
|
any lookups we do will now use <b>load</b> addresses so we don't have to
|
||
|
|
do any math on the addresses in the stack backtraces, we can just use the
|
||
|
|
raw addresses:</p>
|
||
|
|
<code><pre><tt><b>(lldb)</b> image lookup --address 0x100123aa3
|
||
|
|
Address: a.out[0x0000000100000aa3] (a.out.__TEXT.__text + 131)
|
||
|
|
Summary: a.out`main + 67 at main.c:13
|
||
|
|
</tt></pre></code>
|
||
|
|
</div>
|
||
|
|
<div class="postfooter"></div>
|
||
|
|
</div>
|
||
|
|
<div class="post">
|
||
|
|
<h1 class="postheader">Loading Multiple Executables</h1>
|
||
|
|
<div class="postcontent">
|
||
|
|
<p>You often have more than one executable involved when you need to symbolicate
|
||
|
|
a crash log. When this happens, you create a target for the main executable
|
||
|
|
or one of the shared libraries, then add more modules to the target using the
|
||
|
|
"target modules add" command.<p>
|
||
|
|
<p>Lets say we have a Darwin crash log that contains the following images:
|
||
|
|
<code><pre><tt>Binary Images:
|
||
|
|
<font color=blue>0x100000000</font> - 0x100000ff7 <A866975B-CA1E-3649-98D0-6C5FAA444ECF> /tmp/a.out
|
||
|
|
<font color=green>0x7fff83f32000</font> - 0x7fff83ffefe7 <8CBCF9B9-EBB7-365E-A3FF-2F3850763C6B> /usr/lib/system/libsystem_c.dylib
|
||
|
|
<font color=red>0x7fff883db000</font> - 0x7fff883e3ff7 <62AA0B84-188A-348B-8F9E-3E2DB08DB93C> /usr/lib/system/libsystem_dnssd.dylib
|
||
|
|
<font color=purple>0x7fff8c0dc000</font> - 0x7fff8c0f7ff7 <C0535565-35D1-31A7-A744-63D9F10F12A4> /usr/lib/system/libsystem_kernel.dylib
|
||
|
|
</tt></pre></code>
|
||
|
|
|
||
|
|
<p>First we create the target using the main executable and then add any extra shared libraries we want:</p>
|
||
|
|
<code><pre><tt><b>(lldb)</b> target create --no-dependents --arch x86_64 /tmp/a.out
|
||
|
|
<b>(lldb)</b> target modules add /usr/lib/system/libsystem_c.dylib
|
||
|
|
<b>(lldb)</b> target modules add /usr/lib/system/libsystem_dnssd.dylib
|
||
|
|
<b>(lldb)</b> target modules add /usr/lib/system/libsystem_kernel.dylib
|
||
|
|
</tt></pre></code>
|
||
|
|
<p>If you have debug symbols in stand alone files, such as dSYM files on MacOSX, you can specify their paths using the <b>--symfile</b> option for the "target create" (recent LLDB releases only) and "target modules add" commands:</p>
|
||
|
|
<code><pre><tt><b>(lldb)</b> target create --no-dependents --arch x86_64 /tmp/a.out <b>--symfile /tmp/a.out.dSYM</b>
|
||
|
|
<b>(lldb)</b> target modules add /usr/lib/system/libsystem_c.dylib <b>--symfile /build/server/a/libsystem_c.dylib.dSYM</b>
|
||
|
|
<b>(lldb)</b> target modules add /usr/lib/system/libsystem_dnssd.dylib <b>--symfile /build/server/b/libsystem_dnssd.dylib.dSYM</b>
|
||
|
|
<b>(lldb)</b> target modules add /usr/lib/system/libsystem_kernel.dylib <b>--symfile /build/server/c/libsystem_kernel.dylib.dSYM</b>
|
||
|
|
</tt></pre></code>
|
||
|
|
<p>Then we set the load addresses for each __TEXT section using (note the colors of the load addresses above and below) the first address from the Binary Images section for each image:</p>
|
||
|
|
<code><pre><tt><b>(lldb)</b> target modules load --file a.out <font color=blue>0x100000000</font>
|
||
|
|
<b>(lldb)</b> target modules load --file libsystem_c.dylib <font color=green>0x7fff83f32000</font>
|
||
|
|
<b>(lldb)</b> target modules load --file libsystem_dnssd.dylib <font color=red>0x7fff883db000</font>
|
||
|
|
<b>(lldb)</b> target modules load --file libsystem_kernel.dylib <font color=purple>0x7fff8c0dc000</font>
|
||
|
|
</tt></pre></code>
|
||
|
|
<p>Now any stack backtraces that haven't been symbolicated can be symbolicated using "image lookup"
|
||
|
|
with the raw backtrace addresses.</p>
|
||
|
|
<p>Given the following raw backtrace:</p>
|
||
|
|
<code><pre><tt>Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
|
||
|
|
0 libsystem_kernel.dylib 0x00007fff8a1e6d46 __kill + 10
|
||
|
|
1 libsystem_c.dylib 0x00007fff84597df0 abort + 177
|
||
|
|
2 libsystem_c.dylib 0x00007fff84598e2a __assert_rtn + 146
|
||
|
|
3 a.out 0x0000000100000f46 main + 70
|
||
|
|
4 libdyld.dylib 0x00007fff8c4197e1 start + 1
|
||
|
|
</tt></pre></code>
|
||
|
|
<p>We can now symbolicate the <b>load</b> addresses:<p>
|
||
|
|
<code><pre><tt><b>(lldb)</b> image lookup -a 0x00007fff8a1e6d46
|
||
|
|
<b>(lldb)</b> image lookup -a 0x00007fff84597df0
|
||
|
|
<b>(lldb)</b> image lookup -a 0x00007fff84598e2a
|
||
|
|
<b>(lldb)</b> image lookup -a 0x0000000100000f46
|
||
|
|
</tt></pre></code>
|
||
|
|
</div>
|
||
|
|
<div class="postfooter"></div>
|
||
|
|
</div>
|
||
|
|
<div class="post">
|
||
|
|
<h1 class="postheader">Getting Variable Information</h1>
|
||
|
|
<div class="postcontent">
|
||
|
|
<p>If you add the --verbose flag to the "image lookup --address" command,
|
||
|
|
you can get verbose information which can often include the locations
|
||
|
|
of some of your local variables:
|
||
|
|
<code><pre><tt>><b>(lldb)</b> image lookup --address 0x100123aa3 --verbose
|
||
|
|
Address: a.out[0x0000000100000aa3] (a.out.__TEXT.__text + 110)
|
||
|
|
Summary: a.out`main + 50 at main.c:13
|
||
|
|
Module: file = "/tmp/a.out", arch = "x86_64"
|
||
|
|
CompileUnit: id = {0x00000000}, file = "/tmp/main.c", language = "ISO C:1999"
|
||
|
|
Function: id = {0x0000004f}, name = "main", range = [0x0000000100000bc0-0x0000000100000dc9)
|
||
|
|
FuncType: id = {0x0000004f}, decl = main.c:9, clang_type = "int (int, const char **, const char **, const char **)"
|
||
|
|
Blocks: id = {0x0000004f}, range = [0x100000bc0-0x100000dc9)
|
||
|
|
id = {0x000000ae}, range = [0x100000bf2-0x100000dc4)
|
||
|
|
LineEntry: [0x0000000100000bf2-0x0000000100000bfa): /tmp/main.c:13:23
|
||
|
|
Symbol: id = {0x00000004}, range = [0x0000000100000bc0-0x0000000100000dc9), name="main"
|
||
|
|
Variable: id = {0x000000bf}, name = "path", type= "char [1024]", location = DW_OP_fbreg(-1072), decl = main.c:28
|
||
|
|
Variable: id = {0x00000072}, name = "argc", type= "int", <b>location = r13</b>, decl = main.c:8
|
||
|
|
Variable: id = {0x00000081}, name = "argv", type= "const char **", <b>location = r12</b>, decl = main.c:8
|
||
|
|
Variable: id = {0x00000090}, name = "envp", type= "const char **", <b>location = r15</b>, decl = main.c:8
|
||
|
|
Variable: id = {0x0000009f}, name = "aapl", type= "const char **", <b>location = rbx</b>, decl = main.c:8
|
||
|
|
</tt></pre></code>
|
||
|
|
<p>The interesting part is the variables that are listed. The variables are
|
||
|
|
the parameters and local variables that are in scope for the address that
|
||
|
|
was specified. These variable entries have locations which are shown in bold
|
||
|
|
above. Crash logs often have register information for the first frame in each
|
||
|
|
stack, and being able to reconstruct one or more local variables can often
|
||
|
|
help you decipher more information from a crash log than you normally would be
|
||
|
|
able to. Note that this is really only useful for the first frame, and only if
|
||
|
|
your crash logs have register information for your threads.
|
||
|
|
</div>
|
||
|
|
<div class="postfooter"></div>
|
||
|
|
</div>
|
||
|
|
<div class="post">
|
||
|
|
<h1 class="postheader">Using Python API to Symbolicate</h1>
|
||
|
|
<div class="postcontent">
|
||
|
|
<p>All of the commands above can be done through the python script bridge. The code below
|
||
|
|
will recreate the target and add the three shared libraries that we added in the darwin
|
||
|
|
crash log example above:
|
||
|
|
<code><pre><tt>triple = "x86_64-apple-macosx"
|
||
|
|
platform_name = None
|
||
|
|
add_dependents = False
|
||
|
|
target = lldb.debugger.CreateTarget("/tmp/a.out", triple, platform_name, add_dependents, lldb.SBError())
|
||
|
|
if target:
|
||
|
|
<font color=green># Get the executable module</font>
|
||
|
|
module = target.GetModuleAtIndex(0)
|
||
|
|
target.SetSectionLoadAddress(module.FindSection("__TEXT"), 0x100000000)
|
||
|
|
module = target.AddModule ("/usr/lib/system/libsystem_c.dylib", triple, None, "/build/server/a/libsystem_c.dylib.dSYM")
|
||
|
|
target.SetSectionLoadAddress(module.FindSection("__TEXT"), 0x7fff83f32000)
|
||
|
|
module = target.AddModule ("/usr/lib/system/libsystem_dnssd.dylib", triple, None, "/build/server/b/libsystem_dnssd.dylib.dSYM")
|
||
|
|
target.SetSectionLoadAddress(module.FindSection("__TEXT"), 0x7fff883db000)
|
||
|
|
module = target.AddModule ("/usr/lib/system/libsystem_kernel.dylib", triple, None, "/build/server/c/libsystem_kernel.dylib.dSYM")
|
||
|
|
target.SetSectionLoadAddress(module.FindSection("__TEXT"), 0x7fff8c0dc000)
|
||
|
|
|
||
|
|
load_addr = 0x00007fff8a1e6d46
|
||
|
|
<font color=green># so_addr is a section offset address, or a lldb.SBAddress object</font>
|
||
|
|
so_addr = target.ResolveLoadAddress (load_addr)
|
||
|
|
<font color=green># Get a symbol context for the section offset address which includes
|
||
|
|
# a module, compile unit, function, block, line entry, and symbol</font>
|
||
|
|
sym_ctx = so_addr.GetSymbolContext (lldb.eSymbolContextEverything)
|
||
|
|
print sym_ctx
|
||
|
|
|
||
|
|
</tt></pre></code>
|
||
|
|
</div>
|
||
|
|
<div class="postfooter"></div>
|
||
|
|
</div>
|
||
|
|
<div class="post">
|
||
|
|
<h1 class="postheader">Use Builtin Python module to Symbolicate</h1>
|
||
|
|
<div class="postcontent">
|
||
|
|
<p>LLDB includes a module in the "lldb" package named "lldb.utils.symbolication".
|
||
|
|
This module contains a lot of helper functions that simplify symbolication
|
||
|
|
by allowing you to create objects that represent symbolication class objects such as:
|
||
|
|
<ul>
|
||
|
|
<li>lldb.utils.symbolication.Address</li>
|
||
|
|
<li>lldb.utils.symbolication.Section</li>
|
||
|
|
<li>lldb.utils.symbolication.Image</li>
|
||
|
|
<li>lldb.utils.symbolication.Symbolicator</li>
|
||
|
|
</ul>
|
||
|
|
<p>Another package that is distributed on MacOSX builds on LLDB is the "lldb.macosx.crashlog"
|
||
|
|
module. This packages subclasses many of the above symbolication class objects. The module
|
||
|
|
installs a new "crashlog" command into the lldb command interpreter so that you can use
|
||
|
|
it to parse and symbolicate MacOSX crash logs:</p>
|
||
|
|
<code><pre><tt><b>(lldb)</b> script import lldb.macosx.crashlog
|
||
|
|
"crashlog" and "save_crashlog" command installed, use the "--help" option for detailed help
|
||
|
|
<b>(lldb)</b> crashlog --help
|
||
|
|
Usage: crashlog [options] <FILE> [FILE ...]
|
||
|
|
|
||
|
|
Symbolicate one or more darwin crash log files to provide source file and line
|
||
|
|
information, inlined stack frames back to the concrete functions, and
|
||
|
|
disassemble the location of the crash for the first frame of the crashed
|
||
|
|
thread. If this script is imported into the LLDB command interpreter, a
|
||
|
|
"crashlog" command will be added to the interpreter for use at the LLDB
|
||
|
|
command line. After a crash log has been parsed and symbolicated, a target
|
||
|
|
will have been created that has all of the shared libraries loaded at the load
|
||
|
|
addresses found in the crash log file. This allows you to explore the program
|
||
|
|
as if it were stopped at the locations described in the crash log and
|
||
|
|
functions can be disassembled and lookups can be performed using the
|
||
|
|
addresses found in the crash log.
|
||
|
|
|
||
|
|
Options:
|
||
|
|
-h, --help show this help message and exit
|
||
|
|
-v, --verbose display verbose debug info
|
||
|
|
-g, --debug display verbose debug logging
|
||
|
|
-a, --load-all load all executable images, not just the images found
|
||
|
|
in the crashed stack frames
|
||
|
|
--images show image list
|
||
|
|
--debug-delay=NSEC pause for NSEC seconds for debugger
|
||
|
|
-c, --crashed-only only symbolicate the crashed thread
|
||
|
|
-d DISASSEMBLE_DEPTH, --disasm-depth=DISASSEMBLE_DEPTH
|
||
|
|
set the depth in stack frames that should be
|
||
|
|
disassembled (default is 1)
|
||
|
|
-D, --disasm-all enabled disassembly of frames on all threads (not just
|
||
|
|
the crashed thread)
|
||
|
|
-B DISASSEMBLE_BEFORE, --disasm-before=DISASSEMBLE_BEFORE
|
||
|
|
the number of instructions to disassemble before the
|
||
|
|
frame PC
|
||
|
|
-A DISASSEMBLE_AFTER, --disasm-after=DISASSEMBLE_AFTER
|
||
|
|
the number of instructions to disassemble after the
|
||
|
|
frame PC
|
||
|
|
-C NLINES, --source-context=NLINES
|
||
|
|
show NLINES source lines of source context (default =
|
||
|
|
4)
|
||
|
|
--source-frames=NFRAMES
|
||
|
|
show source for NFRAMES (default = 4)
|
||
|
|
--source-all show source for all threads, not just the crashed
|
||
|
|
thread
|
||
|
|
-i, --interactive parse all crash logs and enter interactive mode
|
||
|
|
|
||
|
|
</tt></pre></code>
|
||
|
|
<p>The source for the "symbolication" and "crashlog" modules are available in SVN:
|
||
|
|
<br><a href="http://llvm.org/svn/llvm-project/lldb/trunk/examples/python/symbolication.py">symbolication.py</a>
|
||
|
|
<br><a href="http://llvm.org/svn/llvm-project/lldb/trunk/examples/python/crashlog.py">crashlog.py</a>
|
||
|
|
|
||
|
|
</div>
|
||
|
|
<div class="postfooter"></div>
|
||
|
|
</div>
|
||
|
|
|
||
|
|
</div>
|
||
|
|
</body>
|
||
|
|
</html>
|