181 lines
7.2 KiB
Plaintext
181 lines
7.2 KiB
Plaintext
|
|
Using physical DMA provided by OHCI-1394 FireWire controllers for debugging
|
|
---------------------------------------------------------------------------
|
|
|
|
Introduction
|
|
------------
|
|
|
|
Basically all FireWire controllers which are in use today are compliant
|
|
to the OHCI-1394 specification which defines the controller to be a PCI
|
|
bus master which uses DMA to offload data transfers from the CPU and has
|
|
a "Physical Response Unit" which executes specific requests by employing
|
|
PCI-Bus master DMA after applying filters defined by the OHCI-1394 driver.
|
|
|
|
Once properly configured, remote machines can send these requests to
|
|
ask the OHCI-1394 controller to perform read and write requests on
|
|
physical system memory and, for read requests, send the result of
|
|
the physical memory read back to the requester.
|
|
|
|
With that, it is possible to debug issues by reading interesting memory
|
|
locations such as buffers like the printk buffer or the process table.
|
|
|
|
Retrieving a full system memory dump is also possible over the FireWire,
|
|
using data transfer rates in the order of 10MB/s or more.
|
|
|
|
Memory access is currently limited to the low 4G of physical address
|
|
space which can be a problem on IA64 machines where memory is located
|
|
mostly above that limit, but it is rarely a problem on more common
|
|
hardware such as hardware based on x86, x86-64 and PowerPC.
|
|
|
|
Together with a early initialization of the OHCI-1394 controller for debugging,
|
|
this facility proved most useful for examining long debugs logs in the printk
|
|
buffer on to debug early boot problems in areas like ACPI where the system
|
|
fails to boot and other means for debugging (serial port) are either not
|
|
available (notebooks) or too slow for extensive debug information (like ACPI).
|
|
|
|
Drivers
|
|
-------
|
|
|
|
The ohci1394 driver in drivers/ieee1394 initializes the OHCI-1394 controllers
|
|
to a working state and enables physical DMA by default for all remote nodes.
|
|
This can be turned off by ohci1394's module parameter phys_dma=0.
|
|
|
|
The alternative firewire-ohci driver in drivers/firewire uses filtered physical
|
|
DMA, hence is not yet suitable for remote debugging.
|
|
|
|
Because ohci1394 depends on the PCI enumeration to be completed, an
|
|
initialization routine which runs pretty early (long before console_init()
|
|
which makes the printk buffer appear on the console can be called) was written.
|
|
|
|
To activate it, enable CONFIG_PROVIDE_OHCI1394_DMA_INIT (Kernel hacking menu:
|
|
Provide code for enabling DMA over FireWire early on boot) and pass the
|
|
parameter "ohci1394_dma=early" to the recompiled kernel on boot.
|
|
|
|
Tools
|
|
-----
|
|
|
|
firescope - Originally developed by Benjamin Herrenschmidt, Andi Kleen ported
|
|
it from PowerPC to x86 and x86_64 and added functionality, firescope can now
|
|
be used to view the printk buffer of a remote machine, even with live update.
|
|
|
|
Bernhard Kaindl enhanced firescope to support accessing 64-bit machines
|
|
from 32-bit firescope and vice versa:
|
|
- ftp://ftp.suse.de/private/bk/firewire/tools/firescope-0.2.2.tar.bz2
|
|
|
|
and he implemented fast system dump (alpha version - read README.txt):
|
|
- ftp://ftp.suse.de/private/bk/firewire/tools/firedump-0.1.tar.bz2
|
|
|
|
There is also a gdb proxy for firewire which allows to use gdb to access
|
|
data which can be referenced from symbols found by gdb in vmlinux:
|
|
- ftp://ftp.suse.de/private/bk/firewire/tools/fireproxy-0.33.tar.bz2
|
|
|
|
The latest version of this gdb proxy (fireproxy-0.34) can communicate (not
|
|
yet stable) with kgdb over an memory-based communication module (kgdbom).
|
|
|
|
Getting Started
|
|
---------------
|
|
|
|
The OHCI-1394 specification regulates that the OHCI-1394 controller must
|
|
disable all physical DMA on each bus reset.
|
|
|
|
This means that if you want to debug an issue in a system state where
|
|
interrupts are disabled and where no polling of the OHCI-1394 controller
|
|
for bus resets takes place, you have to establish any FireWire cable
|
|
connections and fully initialize all FireWire hardware __before__ the
|
|
system enters such state.
|
|
|
|
Step-by-step instructions for using firescope with early OHCI initialization:
|
|
|
|
1) Verify that your hardware is supported:
|
|
|
|
Load the ohci1394 or the fw-ohci module and check your kernel logs.
|
|
You should see a line similar to
|
|
|
|
ohci1394: fw-host0: OHCI-1394 1.1 (PCI): IRQ=[18] MMIO=[fe9ff800-fe9fffff]
|
|
... Max Packet=[2048] IR/IT contexts=[4/8]
|
|
|
|
when loading the driver. If you have no supported controller, many PCI,
|
|
CardBus and even some Express cards which are fully compliant to OHCI-1394
|
|
specification are available. If it requires no driver for Windows operating
|
|
systems, it most likely is. Only specialized shops have cards which are not
|
|
compliant, they are based on TI PCILynx chips and require drivers for Win-
|
|
dows operating systems.
|
|
|
|
2) Establish a working FireWire cable connection:
|
|
|
|
Any FireWire cable, as long at it provides electrically and mechanically
|
|
stable connection and has matching connectors (there are small 4-pin and
|
|
large 6-pin FireWire ports) will do.
|
|
|
|
If an driver is running on both machines you should see a line like
|
|
|
|
ieee1394: Node added: ID:BUS[0-01:1023] GUID[0090270001b84bba]
|
|
|
|
on both machines in the kernel log when the cable is plugged in
|
|
and connects the two machines.
|
|
|
|
3) Test physical DMA using firescope:
|
|
|
|
On the debug host,
|
|
- load the raw1394 module,
|
|
- make sure that /dev/raw1394 is accessible,
|
|
then start firescope:
|
|
|
|
$ firescope
|
|
Port 0 (ohci1394) opened, 2 nodes detected
|
|
|
|
FireScope
|
|
---------
|
|
Target : <unspecified>
|
|
Gen : 1
|
|
[Ctrl-T] choose target
|
|
[Ctrl-H] this menu
|
|
[Ctrl-Q] quit
|
|
|
|
------> Press Ctrl-T now, the output should be similar to:
|
|
|
|
2 nodes available, local node is: 0
|
|
0: ffc0, uuid: 00000000 00000000 [LOCAL]
|
|
1: ffc1, uuid: 00279000 ba4bb801
|
|
|
|
Besides the [LOCAL] node, it must show another node without error message.
|
|
|
|
4) Prepare for debugging with early OHCI-1394 initialization:
|
|
|
|
4.1) Kernel compilation and installation on debug target
|
|
|
|
Compile the kernel to be debugged with CONFIG_PROVIDE_OHCI1394_DMA_INIT
|
|
(Kernel hacking: Provide code for enabling DMA over FireWire early on boot)
|
|
enabled and install it on the machine to be debugged (debug target).
|
|
|
|
4.2) Transfer the System.map of the debugged kernel to the debug host
|
|
|
|
Copy the System.map of the kernel be debugged to the debug host (the host
|
|
which is connected to the debugged machine over the FireWire cable).
|
|
|
|
5) Retrieving the printk buffer contents:
|
|
|
|
With the FireWire cable connected, the OHCI-1394 driver on the debugging
|
|
host loaded, reboot the debugged machine, booting the kernel which has
|
|
CONFIG_PROVIDE_OHCI1394_DMA_INIT enabled, with the option ohci1394_dma=early.
|
|
|
|
Then, on the debugging host, run firescope, for example by using -A:
|
|
|
|
firescope -A System.map-of-debug-target-kernel
|
|
|
|
Note: -A automatically attaches to the first non-local node. It only works
|
|
reliably if only connected two machines are connected using FireWire.
|
|
|
|
After having attached to the debug target, press Ctrl-D to view the
|
|
complete printk buffer or Ctrl-U to enter auto update mode and get an
|
|
updated live view of recent kernel messages logged on the debug target.
|
|
|
|
Call "firescope -h" to get more information on firescope's options.
|
|
|
|
Notes
|
|
-----
|
|
Documentation and specifications: ftp://ftp.suse.de/private/bk/firewire/docs
|
|
|
|
FireWire is a trademark of Apple Inc. - for more information please refer to:
|
|
http://en.wikipedia.org/wiki/FireWire
|