Commit Graph

88 Commits

Author SHA1 Message Date
Paul Mackerras a12b51c478 perf tools: Fix sparse CPU numbering related bugs
At present, the perf subcommands that do system-wide monitoring
(perf stat, perf record and perf top) don't work properly unless
the online cpus are numbered 0, 1, ..., N-1.  These tools ask
for the number of online cpus with sysconf(_SC_NPROCESSORS_ONLN)
and then try to create events for cpus 0, 1, ..., N-1.

This creates problems for systems where the online cpus are
numbered sparsely.  For example, a POWER6 system in
single-threaded mode (i.e. only running 1 hardware thread per
core) will have only even-numbered cpus online.

This fixes the problem by reading the /sys/devices/system/cpu/online
file to find out which cpus are online.  The code that does that is in
tools/perf/util/cpumap.[ch], and consists of a read_cpu_map()
function that sets up a cpumap[] array and returns the number of
online cpus.  If /sys/devices/system/cpu/online can't be read or
can't be parsed successfully, it falls back to using sysconf to
ask how many cpus are online and sets up an identity map in cpumap[].

The perf record, perf stat and perf top code then calls
read_cpu_map() in the system-wide monitoring case (instead of
sysconf) and uses cpumap[] to get the cpu numbers to pass to
perf_event_open.

Signed-off-by: Paul Mackerras <paulus@samba.org>
Cc: Anton Blanchard <anton@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arnaldo Carvalho de Melo <acme@infradead.org>
LKML-Reference: <20100310093609.GA3959@brick.ozlabs.ibm.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-03-11 13:36:53 +01:00
Frederic Weisbecker 018cbffe68 Merge commit 'v2.6.33' into perf/core
Merge reason:
	__percpu annotations need the corresponding sparse address
space definition upstream.

Conflicts:
	tools/perf/util/probe-event.c (trivial)
2010-02-27 16:18:46 +01:00
Arnaldo Carvalho de Melo c7ad21af2c perf top: Use a macro instead of a constant variable
To overcome a silly gcc warning:

 cc1: warnings being treated as errors
 builtin-top.c: In function ‘lookup_sym_source’:
 builtin-top.c:291: warning: not protecting local variables:
 variable length buffer make: *** [builtin-top.o] Error 1
 make: *** Waiting for unfinished jobs....

That is emitted for this:

	const size_t pattern_len = BITS_PER_LONG / 4 + 2;
	char pattern[pattern_len + 1];

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1266866062-6287-1-git-send-email-acme@infradead.org>
[ -v2: macroify the naming style ]
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-02-25 12:26:16 +01:00
Kirill Smelkov 1a72cfa685 perf top: Fix help text alignment
Print this:

Mapped keys:
        [d]     display refresh delay.                  (2)
        [e]     display entries (lines).                (46)
        [f]     profile display filter (count).         (5)
        [F]     annotate display filter (percent).      (5%)
        [s]     annotate symbol.                        (NULL)
        [S]     stop annotation.
        [K]     hide kernel_symbols symbols.            (no)
        [U]     hide user symbols.                      (no)
        [z]     toggle sample zeroing.                  (0)
        [qQ]    quit.

instead of:

Mapped keys:
        [d]     display refresh delay.                  (2)
        [e]     display entries (lines).                (46)
        [f]     profile display filter (count).         (5)
        [F]     annotate display filter (percent).      (5%)
        [s]     annotate symbol.                        (NULL)
        [S]     stop annotation.
        [K]     hide kernel_symbols symbols.                    (no)
        [U]     hide user symbols.                      (no)
        [z]     toggle sample zeroing.                  (0)
        [qQ]    quit.

Signed-off-by: Kirill Smelkov <kirr@landau.phys.spbu.ru>
Acked-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
LKML-Reference: <20100212162059.GA30041@landau.phys.spbu.ru>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-02-14 09:46:02 +01:00
Arnaldo Carvalho de Melo 5f48536436 perf top: Use address pattern in lookup_sym_source
Because we may have aliases, like __GI___strcoll_l in
/lib64/libc-2.10.2.so that appears in objdump as:

$ objdump --start-address=0x0000003715a86420 \
           --stop-address=0x0000003715a872dc -dS /lib64/libc-2.10.2.so

0000003715a86420 <__strcoll_l>:
  3715a86420:	55                   	push   %rbp
  3715a86421:	48 89 e5             	mov    %rsp,%rbp
  3715a86424:	41 57                	push   %r15
[root@doppio linux-2.6-tip]#

So look for the address exactly at the start of the line instead
so that annotation can work for in these cases.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Kirill Smelkov <kirr@landau.phys.spbu.ru>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1265550376-12665-2-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-02-07 17:30:21 +01:00
Kirill Smelkov ee11b90b12 perf top: Fix annotate for userspace
First, for programs and prelinked libraries, annotate code was
fooled by objdump output IPs (src->eip in the code) being
wrongly converted to absolute IPs. In such case there were no
conversion needed, but in

   src->eip = strtoull(src->line, NULL, 16);
   src->eip = map->unmap_ip(map, src->eip); // = eip + map->start - map->pgoff

we were reading absolute address from objdump (e.g. 8048604) and
then almost doubling it, because eip & map->start are
approximately close for small programs.

Needless to say, that later, in record_precise_ip() there was no
matching with real runtime IPs.

And second, like with `perf annotate` the problem with
non-prelinked *.so was that we were doing rip -> objdump address
conversion wrong.

Also, because unlike `perf annotate`, `perf top` code does
annotation based on absolute IPs for performance reasons(*), new
helper for mapping objdump addresse to IP is introduced.

(*) we get samples info in absolute IPs, and since we do lots of
    hit-testing on absolute IPs at runtime in record_precise_ip(), it's
    better to convert objdump addresses to IPs once and do no conversion
    at runtime.

I also had to fix how objdump output is parsed (with hardcoded
8/16 characters format, which was inappropriate for ET_DYN dsos
with small addresses like '4ac')

Also note, that not all objdump output lines has associtated
IPs, e.g. look at source lines here:

    000004ac <my_strlen>:
    extern "C"
    int my_strlen(const char *s)
     4ac:   55                      push   %ebp
     4ad:   89 e5                   mov    %esp,%ebp
     4af:   83 ec 10                sub    $0x10,%esp
    {
        int len = 0;
     4b2:   c7 45 fc 00 00 00 00    movl   $0x0,-0x4(%ebp)
     4b9:   eb 08                   jmp    4c3 <my_strlen+0x17>

        while (*s) {
            ++len;
     4bb:   83 45 fc 01             addl   $0x1,-0x4(%ebp)
            ++s;
     4bf:   83 45 08 01             addl   $0x1,0x8(%ebp)

So we mark them with eip=0, and ignore such lines in annotate
lookup code.

Signed-off-by: Kirill Smelkov <kirr@landau.phys.spbu.ru>
[ Note: one hunk of this patch was applied by Mike in 57d8188 ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
LKML-Reference: <1265550376-12665-1-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-02-07 17:30:20 +01:00
Mike Galbraith 57d818895f perf annotate: Fix perf top module symbol annotation
Signed-off-by: Mike Galbraith <efault@gmx.de>
Cc: Kirill Smelkov <kirr@landau.phys.spbu.ru>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Arnaldo Carvalho de Melo <acme@infradead.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
LKML-Reference: <1265265106.6364.5.camel@marge.simson.net>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-02-04 09:33:28 +01:00
Kirill Smelkov 6cff0e8dba perf top: Teach it to autolocate vmlinux
By relying on logic in dso__load_kernel_sym(), we can
automatically load vmlinux.

The only thing which needs to be adjusted, is how --sym-annotate
option is handled - now we can't rely on vmlinux been loaded
until full successful pass of dso__load_vmlinux(), but that's
not the case if we'll do sym_filter_entry setup in
symbol_filter().

So move this step right after event__process_sample() where we
know the whole dso__load_kernel_sym() pass is done.

By the way, though conceptually similar `perf top` still can't
annotate userspace - see next patches with fixes.

Signed-off-by: Kirill Smelkov <kirr@landau.phys.spbu.ru>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
LKML-Reference: <1265223128-11786-9-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-02-04 09:33:28 +01:00
Arnaldo Carvalho de Melo 64abebf731 perf session: Create kernel maps in the constructor
Removing one extra step needed in the tools that need this,
fixing a bug in 'perf probe' where this was not being done.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Masami Hiramatsu <mhiramat@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1264633557-17597-4-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-01-29 09:20:58 +01:00
Arnaldo Carvalho de Melo 72b8fa1730 perf top: Exit if specified --vmlinux can't be used
As we do lazy loading of symtabs we only will know if the
specified vmlinux file is invalid when we actually have a hit in
kernel space and then try to load it. So if we get kernel hits
and there are _no_ symbols in the DSO backing the kernel map,
bail out.

Reported-by: Mike Galbraith <efault@gmx.de>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1264633557-17597-1-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-01-29 09:20:57 +01:00
Arnaldo Carvalho de Melo 0f35cd4cea perf top: Handle PERF_RECORD_{FORK,EXIT} events
As noticed by Mike, symbols in new tasks were not being
processed as we weren't processing these events.

Reported-by: Mike Galbraith <efault@gmx.de>
Tested-by: Mike Galbraith <efault@gmx.de>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1264086284-1431-2-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-01-27 12:46:48 +01:00
Arnaldo Carvalho de Melo 24bfef0f92 perf top: Fix sample counting
Broken since "5b2bb75 perf top: Support userspace symbols too".

Reported-by: Mike Galbraith <efault@gmx.de>
Tested-by: Mike Galbraith <efault@gmx.de>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1264086284-1431-1-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-01-27 12:46:48 +01:00
Kirill Smelkov 66aeb6d5cb perf top: Fix code typo in prompt_symbol()
sym_filter is what was (if ever) passed with -s option. What was
typed by user, and what we were looking for, is in buf.

Signed-off-by: Kirill Smelkov <kirr@landau.phys.spbu.ru>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
LKML-Reference: <1263396139-4798-3-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-01-13 17:39:44 +01:00
Arnaldo Carvalho de Melo b7cece7678 perf tools: Encode kernel module mappings in perf.data
We were always looking at the running machine /proc/modules,
even when processing a perf.data file, which only makes sense
when we're doing 'perf record' and 'perf report' on the same
machine, and in close sucession, or if we don't use modules at
all, right Peter? ;-)

Now, at 'perf record' time we read /proc/modules, find the long
path for modules, and put them as PERF_MMAP events, just like we
did to encode the reloc reference symbol for vmlinux. Talking
about that now it is encoded in .pgoff, so that we can use
.{start,len} to store the address boundaries for the kernel so
that when we reconstruct the kmaps tree we can do lookups right
away, without having to fixup the end of the kernel maps like we
did in the past (and now only in perf record).

One more step in the 'perf archive' direction when we'll finally
be able to collect data in one machine and analyse in another.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1263396139-4798-1-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-01-13 17:39:43 +01:00
Arnaldo Carvalho de Melo c410a33887 perf symbols: Move symbol filtering to event__preprocess_sample()
So that --dsos, --comm, --symbols can bem used in more tools,
like in perf diff:

$ perf record -f find / > /dev/null
$ perf record -f find / > /dev/null
$ perf diff --dsos /lib64/libc-2.10.1.so | head -5
   1        +22392124     /lib64/libc-2.10.1.so   _IO_vfprintf_internal
   2         +6410655     /lib64/libc-2.10.1.so   __GI_memmove
   3    +1   +9192692     /lib64/libc-2.10.1.so   _int_malloc
   4    -1  -15158605     /lib64/libc-2.10.1.so   _int_free
   5           +45669     /lib64/libc-2.10.1.so   _IO_new_file_xsputn
$

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1260914682-29652-3-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-12-16 08:53:49 +01:00
Arnaldo Carvalho de Melo 75be6cf487 perf symbols: Make symbol_conf global
This simplifies a lot of functions, less stuff to be done by
tool writers.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1260914682-29652-1-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-12-16 08:53:48 +01:00
Frederic Weisbecker c249a4ce79 perf tools: Make symbol_conf static
perf top, report and annotate all define their own symbol_conf,
it should be static.

Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1260843322-6602-1-git-send-regression-fweisbec@gmail.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-12-15 08:52:31 +01:00
Arnaldo Carvalho de Melo 4aa6563641 perf session: Move kmaps to perf_session
There is still some more work to do to disentangle map creation
from DSO loading, but this happens only for the kernel, and for
the early adopters of perf diff, where this disentanglement
matters most, we'll be testing different kernels, so no problem
here.

Further clarification: right now we create the kernel maps for
the various modules and discontiguous kernel text maps when
loading the DSO, we should do it as a two step process, first
creating the maps, for multiple mappings with the same DSO
store, then doing the dso load just once, for the first hit on
one of the maps sharing this DSO backing store.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1260741029-4430-6-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-12-14 16:57:17 +01:00
Arnaldo Carvalho de Melo b3165f4144 perf session: Move the global threads list to perf_session
So that we can process two perf.data files.

We still need to add a O_MMAP mode for perf_session so that we
can do all the mmap stuff in it.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1260741029-4430-5-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-12-14 16:57:16 +01:00
Arnaldo Carvalho de Melo d8f66248d6 perf session: Pass the perf_session to the event handling operations
They will need it to get the right threads list, etc.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1260741029-4430-1-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-12-14 16:57:13 +01:00
Arnaldo Carvalho de Melo 1ed091c45a perf tools: Consolidate symbol resolving across all tools
Now we have a very high level routine for simple tools to
process IP sample events:

	int event__preprocess_sample(const event_t *self,
				     struct addr_location *al,
				     symbol_filter_t filter)

It receives the event itself and will insert new threads in the
global threads list and resolve the map and symbol, filling all
this info into the new addr_location struct, so that tools like
annotate and report can further process the event by creating
hist_entries in their specific way (with or without callgraphs,
etc).

It in turn uses the new next layer function:

	void thread__find_addr_location(struct thread *self, u8 cpumode,
					enum map_type type, u64 addr,
					struct addr_location *al,
					symbol_filter_t filter)

This one will, given a thread (userspace or the kernel kthread
one), will find the given type (MAP__FUNCTION now, MAP__VARIABLE
too in the near future) at the given cpumode, taking vdsos into
account (userspace hit, but kernel symbol) and will fill all
these details in the addr_location given.

Tools that need a more compact API for plain function
resolution, like 'kmem', can use this other one:

	struct symbol *thread__find_function(struct thread *self, u64 addr,
					     symbol_filter_t filter)

So, to resolve a kernel symbol, that is all the 'kmem' tool
needs, its just a matter of calling:

	sym = thread__find_function(kthread, addr, NULL);

The 'filter' parameter is needed because we do lazy
parsing/loading of ELF symtabs or /proc/kallsyms.

With this we remove more code duplication all around, which is
always good, huh? :-)

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: John Kacur <jkacur@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1259346563-12568-12-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-11-27 20:22:02 +01:00
Arnaldo Carvalho de Melo 62daacb51a perf tools: Reorganize event processing routines, lotsa dups killed
While implementing event__preprocess_sample, that will do all of
the symbol lookup in one convenient function, I noticed that
util/process_event.[ch] were not being used at all, then started
looking if there were other functions that could be shared
and...

All those functions really don't need to receive offset + head,
the only thing they did was common to all of them, so do it at
one place instead.

Stats about number of each type of event processed now is done
in a central place.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: John Kacur <jkacur@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1259346563-12568-11-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-11-27 20:22:01 +01:00
Arnaldo Carvalho de Melo 95011c6007 perf symbols: Support multiple symtabs in struct thread
Making the routines that were so far specific to the kernel maps
useful for all threads.

This is done by making the kernel maps be contained in a kernel
"thread".

This gets the kernel specific routines closer to the userspace
counterparts, which will help in reducing the boilerplate for
resolving a symbol, as will be demonstrated in the next patches.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1259346563-12568-9-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-11-27 20:22:00 +01:00
Arnaldo Carvalho de Melo 6a4694a433 perf symbols: Better support for multiple symbol tables per dso
By using an array of rb_roots in struct dso we can, from a
struct map instance to get the right symbol rb_tree more easily.
This way we can have just one symbol lookup method for struct
map instances, map__find_symbol, instead of one per symtab type
(functions, variables).

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1259346563-12568-6-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-11-27 20:21:59 +01:00
Arnaldo Carvalho de Melo 3610583c29 perf symbols: Add a 'type' field to struct map
That way we will be able to check if the right symtab is loaded
in the underlying DSO.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1259346563-12568-5-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-11-27 20:21:59 +01:00
Arnaldo Carvalho de Melo fcf1203a91 perf symbols: Rename find_symbol routines to find_function
Paving the way for supporting variable in adition to function
symbols.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1259074912-5924-1-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-11-24 16:37:03 +01:00
Arnaldo Carvalho de Melo 364794845c perf tools: Introduce zalloc() for the common calloc(1, N) case
This way we type less characters and it looks more like the
kzalloc kernel counterpart.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1259071517-3242-3-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-11-24 16:37:02 +01:00
Arnaldo Carvalho de Melo b32d133aec perf symbols: Simplify symbol machinery setup
And also express its configuration toggles via a struct.

Now all one has to do is to call symbol__init(NULL) if the
defaults are OK, or pass a struct symbol_conf pointer with the
desired configuration.

If a tool uses kernel_maps__find_symbol() to look at the kernel
and modules mappings for a symbol but didn't call symbol__init()
first, that will generate a one time warning too, alerting the
subcommand developer that symbol__init() must be called.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1259071517-3242-2-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-11-24 16:37:02 +01:00
Arnaldo Carvalho de Melo 7cc017edb9 perf top: Always show the DSO column, even if its all the same
Ingo found it confusing, and I agree with that, for 'perf
report' its OK because it is static, but for a tool refreshing
it the eventual switch from column to summary at the top may
seem confusing.

Suggested-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1259071517-3242-1-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-11-24 16:37:02 +01:00
Arnaldo Carvalho de Melo 2890284bcf perf tools: Move graph_line and graph_dotted_line from top
So that they can be used in other tools.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1259005869-13487-1-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-11-23 21:55:20 +01:00
Arnaldo Carvalho de Melo cc612d8199 perf symbols: Look for vmlinux in more places
Now that we can check the buildid to see if it really matches,
this can be done safely:

  vmlinux
  /boot/vmlinux
  /boot/vmlinux-<uts.release>
  /lib/modules/<uts.release>/build/vmlinux
  /usr/lib/debug/lib/modules/%s/vmlinux

More can be added - if you know about distros that put the
vmlinux somewhere else please let us know.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1259001550-8194-1-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-11-23 19:51:48 +01:00
Arnaldo Carvalho de Melo c338aee853 perf symbols: Do lazy symtab loading for the kernel & modules too
Just like we do with the other DSOs. This also simplifies the
kernel_maps setup process, now all that the tools need to do is
to call kernel_maps__init and the maps for the modules and
kernel will be created, then, later, when
kernel_maps__find_symbol() is used, it will also call
maps__find_symbol that already checks if the symtab was loaded,
loading it if needed.

Now if one does 'perf top --hide_kernel_symbols' we won't pay
the price of loading the (many) symbols in /proc/kallsyms or
vmlinux.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1258757489-5978-4-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-11-21 14:11:33 +01:00
Arnaldo Carvalho de Melo 6671cb1674 perf symbols: Remove unrelated actions from dso__load_kernel_sym
It should just load kernel symbols, not load the list of
modules. There are more stuff to move to other routines, but
lets do it in several steps.

End goal is to be able to defer symbol table loading till we
find a hit for that map address range. So that the kernel &
modules are handled just like all the other DSOs in the system.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1258757489-5978-1-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-11-21 14:11:32 +01:00
Arnaldo Carvalho de Melo 2446042c93 perf symbols: Capture the running kernel buildid too
[root@doppio linux-2.6-tip]# perf record -a -f sleep 3s ; perf
buildid-list | grep vmlinux
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.171 MB perf.data (~7489
samples) ] 18e7cc53db62a7d35e9d6f6c9ddc23017d38ee9a vmlinux
[root@doppio linux-2.6-tip]#

Several refactorings were needed so that we can have symmetry
between dsos__load_modules() and dsos__load_kernel(), i.e. those
functions will respectively create and add to the dsos list the
loaded modules and kernel, with its buildids, but not load its
symbols. That is something the subcomands that need will have to
call dso__load_kernel_sym(), just like we do with modules with
dsos__load_module_sym()/dso__load_module_sym().

Next csets will actually use this info to stop producing bogus
results using mismatched vmlinux and .ko files.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Roland McGrath <roland@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1258582853-8579-4-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-11-19 08:28:13 +01:00
Arnaldo Carvalho de Melo b269876c8d perf top: Don't allocate the source parsing members upfront
Defer to parse_source() time allocating it.

Now we use about this much memory:

 1724 root      20   0 42104  10m  940 S  0.0  0.4   0:00.23 perf

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1258490282-1821-3-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-11-19 06:03:41 +01:00
Arnaldo Carvalho de Melo 5a8e5a3065 perf top: Allocate space only for the number of counters used
Reducing memory consumption on a typical desktop machine:

From:

32710 root      20   0  172m 142m 1056 S  0.0  4.7   0:00.37
perf

To:

  420 root      20   0 47528  16m 1056 R  0.3  0.5   0:00.24 perf

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1258490282-1821-2-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-11-19 06:03:40 +01:00
Arnaldo Carvalho de Melo 51a472decb perf top: Introduce helper function to access symbol from sym_entry
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1258490282-1821-1-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-11-19 06:03:38 +01:00
Arnaldo Carvalho de Melo 1a105f743d perf top: Suppress DSO column if only one is present
E.g.

[root@doppio ~]# perf top -U
---------------------------------------------------------------------------
   PerfTop:     482 irqs/sec  kernel:100.0% [1000Hz cycles],  (all, 2 CPUs)
---------------------------------------------------------------------------
DSO: vmlinux
             samples  pcnt function
             _______ _____ _________________________

              471.00 47.9% read_hpet
               57.00  5.8% acpi_os_read_port
               30.00  3.1% hpet_next_event
               30.00  3.1% find_busiest_group
               22.00  2.2% schedule
               18.00  1.8% sched_clock_local
               14.00  1.4% _spin_lock_irqsave
               14.00  1.4% native_read_tsc
               13.00  1.3% trace_hardirqs_off
                9.00  0.9% fget_light
                9.00  0.9% ioread8
                8.00  0.8% do_sys_poll

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1258479655-28662-3-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-11-19 06:03:37 +01:00
Arnaldo Carvalho de Melo 13cc5079f2 perf top: Auto adjust symbol and dso widths
We pre-calculate the symbol name length, then after we sort the
entries to print, calculate the biggest one and use that for the
symbol name width justification, then use the
dso->long_name->len to justificate the DSO name, deciding whether
using the short or long name depending on how much space we have
on the terminal.

IOW give as much info to the user as the terminal width allows.

Suggested-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1258479655-28662-2-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-11-19 06:03:36 +01:00
Arnaldo Carvalho de Melo 8ffcda1731 perf top: Introduce --hide_{user,kernel}_symbols
Default continues to be showing all symbols. 'K' and 'U' can be
used to toggle showing kernel and user symbols.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
LKML-Reference: <1258415125-15019-1-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-11-17 07:19:53 +01:00
Arnaldo Carvalho de Melo 3b6ed98895 perf top: Use all the lines in the screen
By querying the current number of rows, if the user specifies
the number of entries, use that instead. If the user uses the
'e' command to change the number of lines 0 will mean do it
automatically, any other number disables the auto resizing.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
LKML-Reference: <1258407027-384-2-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-11-17 07:19:53 +01:00
Ingo Molnar 39dc78b651 Merge commit 'v2.6.32-rc7' into perf/core
Merge reason: pick up perf fixlets

Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-11-15 09:50:41 +01:00
Pekka Enberg c10edee2e1 perf tools: Fix permission checks
The perf_event_open() system call returns EACCES if the user is
not root which results in a very confusing error message:

  $ perf record -A -a -f

    Error: perfcounter syscall returned with -1 (Permission denied)

    Fatal: No CONFIG_PERF_EVENTS=y kernel support configured?

It turns out that's because perf tools are checking only for
EPERM. Fix that up to get a much better error message:

  $ perf record -A -a -f
    Fatal: Permission error - are you root?

Signed-off-by: Pekka Enberg <penberg@cs.helsinki.fi>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
LKML-Reference: <1257696066-4046-1-git-send-email-penberg@cs.helsinki.fi>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-11-08 17:04:54 +01:00
Arnaldo Carvalho de Melo 00a192b395 perf tools: Simplify the symbol priv area mechanism
Before we were storing this in the DSO, but in fact this is a
property of the 'symbol' class, not something that will vary
among DSOs, so move it to a global variable and initialize it
using the existing symbol__init routine.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Mike Galbraith <efault@gmx.de>
LKML-Reference: <1256927305-4628-2-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-11-02 16:52:11 +01:00
Arnaldo Carvalho de Melo 66bd8424cc perf tools: Delay loading symtabs till we hit a map with it
So that we can have a quicker start on perf top and even
speedups in the other tools, as we can have maps with no hits,
so no need to load its symtabs.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Mike Galbraith <efault@gmx.de>
LKML-Reference: <1256773881-4191-1-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-10-29 08:23:40 +01:00
Arnaldo Carvalho de Melo 5b2bb75a0d perf top: Support userspace symbols too
Example:

Compiling the kernel with 'make -k 22 allyesconfig'

[root@emilia linux-2.6-tip]# perf top -r 90
------------------------------------------------------------------------------
   PerfTop:    3669 irqs/sec  kernel:59.9% [1000Hz cycles],  (all, 8 CPUs)
------------------------------------------------------------------------------

             samples  pcnt function                                 DSO
             _______ _____ ________________________________ ________________

             3062.00  6.5% clear_page_c                     [kernel]
             2233.00  4.8% _int_malloc                      /lib64/libc-2.5.so
             2100.00  4.5% yylex                            /home/acme/git/build/allyesconfig/scripts/genksyms/genksyms
             2029.00  4.3% memset                           /lib64/libc-2.5.so
             1224.00  2.6% page_fault                       [kernel]
             1075.00  2.3% __GI_strlen                      /lib64/libc-2.5.so
              863.00  1.8% sub_preempt_count                [kernel]
              822.00  1.8% __GI_memcpy                      /lib64/libc-2.5.so
              810.00  1.7% __GI_vfprintf                    /lib64/libc-2.5.so
              786.00  1.7% _int_free                        /lib64/libc-2.5.so
              775.00  1.7% __GI_strcmp                      /lib64/libc-2.5.so
              748.00  1.6% _spin_lock                       [kernel]
              699.00  1.5% main                             /home/acme/git/build/allyesconfig/scripts/basic/fixdep
              659.00  1.4% add_preempt_count                [kernel]
              649.00  1.4% yyparse                          /home/acme/git/build/allyesconfig/scripts/genksyms/genksyms
              645.00  1.4% preempt_trace                    [kernel]
              635.00  1.4% __GI___libc_free                 /lib64/libc-2.5.so
              597.00  1.3% trace_preempt_on                 [kernel]
              551.00  1.2% __GI___libc_malloc               /lib64/libc-2.5.so
              516.00  1.1% _spin_lock_irqsave               [kernel]
              481.00  1.0% copy_user_generic_string         [kernel]
              479.00  1.0% unmap_vmas                       [kernel]
              429.00  0.9% _IO_file_xsputn_internal         /lib64/libc-2.5.so
              425.00  0.9% __GI_strncpy                     /lib64/libc-2.5.so
              416.00  0.9% get_page_from_freelist           [kernel]
              414.00  0.9% malloc_consolidate               /lib64/libc-2.5.so
              406.00  0.9% get_parent_ip                    [kernel]
              362.00  0.8% __rmqueue                        [kernel]
              347.00  0.7% in_lock_functions                [kernel]
              316.00  0.7% __d_lookup                       [kernel]

[root@emilia linux-2.6-tip]#

More polishing is needed to print just DSO basename when not
--verbose, etc.

Supporting a 'comm' column requires some more reworking of 'perf
top' internals as we will need to use something like the hist
entries 'perf report' uses and will be done in another patch.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Mike Galbraith <efault@gmx.de>
LKML-Reference: <1256592199-9608-3-git-send-email-acme@redhat.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-10-27 13:51:54 +01:00
Arnaldo Carvalho de Melo 6beba7adbe perf tools: Unify debug messages mechanisms
We were using eprintf in some places, that looks at a global
'verbose' level, and at other places passing a 'v' parameter to
specify the verbosity level, unify it by introducing
pr_{err,warning,debug,etc}, just like in the kernel.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Mike Galbraith <efault@gmx.de>
LKML-Reference: <1256153646-10097-1-git-send-email-acme@redhat.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-10-23 08:22:47 +02:00
Arnaldo Carvalho de Melo c88e4bf60d perf top: Fix symbol annotation
We need to use map->unmap_ip() here too to match section
relative symbol address to the absolute address needed to match
objdump -dS addresses.

Reported-by: Mike Galbraith <efault@gmx.de>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1256061295-19835-1-git-send-email-acme@redhat.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-10-20 21:12:59 +02:00
Ingo Molnar c258449bc9 Merge branch 'perf/urgent' into perf/core
Merge reason: Queue up dependent patch.

Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-10-20 07:51:44 +02:00
Tim Blechmann dc79959aaf perf top: Fix --delay_secs 0 division by zero
Add delay_secs sanity check to handle_keypress,
this fixes a division by zero crash.

Signed-off-by: Tim Blechmann <tim@klingt.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
LKML-Reference: <4AD9EBFD.106@klingt.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-10-19 09:52:39 +02:00