Commit Graph

1342 Commits

Author SHA1 Message Date
Corey Ashford 4c635a4e04 perf tools: fix event parsing of comma-separated tracepoint events
There are number of issues that prevent the use of multiple tracepoint events
being specified in a -e/--event switch, separated by commas.

For example, perf stat -e irq:irq_handler_entry,irq:irq_handler_exit ...  fails
because the tracepoint event parsing code doesn't recognize the comma separator
properly.

This patch corrects those issues.

Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Julia Lawall <julia@diku.dk>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Reported-by: Michael Ellerman <michaele@au1.ibm.com>
LKML-Reference: <1291156021-17711-1-git-send-email-cjashfor@linux.vnet.ibm.com>
Signed-off-by: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-11-30 23:04:39 -02:00
Don Zickus 3e8e24f2fc perf packaging: add memcpy to perf MANIFEST
There seems to be a new dependency on arch/*/lib/memcpy*.S when compiling
the perf tool.  Make sure that file is included in the MANIFEST when
creating the tarball.

Cc: Ingo Molnar <mingo@elte.hu>
LKML-Reference: <1291155133-3499-2-git-send-email-dzickus@redhat.com>
Signed-off-by: Don Zickus <dzickus@redhat.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-11-30 23:00:10 -02:00
Arnaldo Carvalho de Melo 5b1c144475 perf debug: Simplify trace_event
No need to check that many times if debug_trace is on.

Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Stephane Eranian <eranian@google.com>
LKML-Reference: <new-submission>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-11-30 20:58:42 -02:00
Thomas Gleixner 5c891f3840 perf session: Allocate chunks of sample objects
The ordered sample code allocates singular reference objects struct
sample_queue which have 48byte size on 64bit and 20 bytes on 32bit. That's
silly. Allocate ~64k sized chunks and hand them out.

Performance gain: ~ 15%

Cc: Ingo Molnar <mingo@elte.hu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
LKML-Reference: <20101130163820.398713983@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-11-30 20:05:25 -02:00
Thomas Gleixner 020bb75a6d perf session: Cache sample objects
When the sample queue is flushed we free the sample reference objects. Though
we need to malloc new objects when we process further. Stop the malloc/free
orgy and cache the already allocated object for resuage. Only allocate when
the cache is empty.

Performance gain: ~ 10%

Cc: Ingo Molnar <mingo@elte.hu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
LKML-Reference: <20101130163820.338488630@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-11-30 20:04:18 -02:00
Thomas Gleixner fe17420784 perf session: Keep file mmaped instead of malloc/memcpy
Profiling perf with perf revealed that a large part of the processing time is
spent in malloc/memcpy/free in the sample ordering code. That code copies the
data from the mmap into malloc'ed memory. That's silly. We can keep the mmap
and just store the pointer in the queuing data structure. For 64 bit this is
not a problem as we map the whole file anyway. On 32bit we keep 8 maps around
and unmap the oldest before mmaping the next chunk of the file.

Performance gain: 2.95s -> 1.23s (Faktor 2.4)

Cc: Ingo Molnar <mingo@elte.hu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
LKML-Reference: <20101130163820.278787719@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-11-30 20:01:08 -02:00
Thomas Gleixner 55b44629f5 perf session: Use sensible mmap size
On 64bit we can map the whole file in one go, on 32bit we can at least map
32MB and not map/unmap tiny chunks of the file.

Base the progress bar on 1/16 of the data size.

Preparatory patch to get rid of the malloc/memcpy/free of trace data.

Cc: Ingo Molnar <mingo@elte.hu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
LKML-Reference: <20101130163820.213687773@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-11-30 19:59:34 -02:00
Thomas Gleixner d6513281c5 perf session: Simplify termination checks
No need to check twice.

Cc: Ingo Molnar <mingo@elte.hu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
LKML-Reference: <20101130163820.152886642@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-11-30 19:58:10 -02:00
Thomas Gleixner 85b99952cc perf session: Move ui_progress_update in __perf_session__process_events()
The progress bar is changed when the file offset changes. This happens only
when the next mmap is done. No need to call ui_progress_update() for every
event.

Cc: Ingo Molnar <mingo@elte.hu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
LKML-Reference: <20101130163820.094836523@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-11-30 19:57:13 -02:00
Thomas Gleixner 0331ee0cf4 perf session: Cleanup __perf_session__process_events()
Replace the pseudo C++ self argument with session and give the mmap related
variables a sensible name. shift is a complete misnomer - it took me several
rounds of cursing to figure out that it's not a shift value.

Cc: Ingo Molnar <mingo@elte.hu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
LKML-Reference: <20101130163820.029687218@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-11-30 19:57:01 -02:00
Thomas Gleixner 28990f75e6 perf session: Use appropriate pointer type instead of silly typecasting
There is no reason to use a struct sample_event pointer in struct sample_queue
and type cast it when flushing the queue.

Cc: Ingo Molnar <mingo@elte.hu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
LKML-Reference: <20101130163819.969462809@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-11-30 19:55:26 -02:00
Thomas Gleixner a1225decc4 perf session: Fix list sort algorithm
The homebrewn sort algorithm fails to sort in time order. One of the problem
spots is that it fails to deal with equal timestamps correctly.

My first gut reaction was to replace the fancy list with an rbtree, but the
performance is 3 times worse.

Rewrite it so it works.

Cc: Ingo Molnar <mingo@elte.hu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
LKML-Reference: <20101130163819.908482530@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-11-30 19:52:36 -02:00
Arnaldo Carvalho de Melo 068ffaa8bf perf tools: Fix lost and unknown events handling
Fix it by explaining what can be happening and giving the number of processed
and lost events.

Also holler if unknown events were found, that can be due to processing a
perf.data file collected using a newer tool where newer events got added on
reporting using an older perf tool, that or a bug, so ask for a report to be
made.

Works on both --tui and --stdio.

Suggested-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Stephane Eranian <eranian@google.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
LKML-Reference: <new-submission>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-11-27 02:41:01 -02:00
Shawn Bohrer 008f29d386 perf trace: Handle DT_UNKNOWN on filesystems that don't support d_type
Some filesystems like xfs and reiserfs will return DT_UNKNOWN for the
d_type.  Handle this case by calling stat() to determine the type.

Cc: Andreas Schwab <schwab@linux-m68k.org>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
LKML-Reference: <1290355779-3276-1-git-send-email-sbohrer@rgmadvisors.com>
Signed-off-by: Shawn Bohrer <sbohrer@rgmadvisors.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-11-27 01:33:04 -02:00
Ian Munsie 9d1faba5fe perf symbols: Correct final kernel map guesses
If a 32bit userspace perf is running on a 64bit kernel, the end of the final
map in the kernel would incorrectly be set to 2^32-1 rather than 2^64-1.

Cc: Ingo Molnar <mingo@elte.hu>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
LKML-Reference: <1290658375-10342-1-git-send-email-imunsie@au1.ibm.com>
Signed-off-by: Ian Munsie <imunsie@au1.ibm.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-11-27 01:32:53 -02:00
Arnaldo Carvalho de Melo 37982ba0a0 perf events: Default to using event__process_lost
Tool developers have to fill in a 'perf_event_ops' method table to
specify how to handle each event, so far the ones that were not
explicitely especified would get a stub that would just discard the
event.

Change that so that tool developers can get the lost event details and
the total number of such events at the end of 'perf report -D' output.

Suggested-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Stephane Eranian <eranian@google.com>
CC: Thomas Gleixner <tglx@linutronix.de>
Cc: Tom Zanussi <tzanussi@gmail.com>
LKML-Reference: <new-submission>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-11-26 19:39:47 -02:00
Arnaldo Carvalho de Melo baa2f6cedb perf record: Add option to disable collecting build-ids
Collecting build-ids for long running sessions may take a long time
because it needs to traverse the whole just collected perf.data stream
of events, marking the DSOs that had hits and then looking for the
.note.gnu.build-id ELF section.

For things like the 'trace' tool that records and right away consumes
the data on systems where its unlikely that the DSOs being monitored
will change while 'trace' runs, it is desirable to remove build id
collection, so add a -B/--no-buildid option to perf record to allow such
use case.

Longer term we'll avoid all this if we, at DSO load time, in the kernel,
take advantage of this slow code path to collect the build-id and stash
it somewhere, so that we can insert it in the PERF_RECORD_MMAP event.

Reported-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Stephane Eranian <eranian@google.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Tom Zanussi <tzanussi@gmail.com>
LKML-Reference: <new-submission>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-11-26 19:39:15 -02:00
Ingo Molnar 6c869e772c Merge branch 'perf/urgent' into perf/core
Conflicts:
	arch/x86/kernel/apic/hw_nmi.c

Merge reason: Resolve conflict, queue up dependent patch.

Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-11-26 15:07:02 +01:00
Hitoshi Mitake ea7872b9d6 perf bench: Add feature that measures the performance of the arch/x86/lib/memcpy_64.S memcpy routines via 'perf bench mem'
This patch ports arch/x86/lib/memcpy_64.S to perf bench mem
memcpy for benchmarking memcpy() in userland with tricky and
dirty way.

util/include/asm/cpufeature.h, util/include/asm/dwarf2.h, and
util/include/linux/linkage.h are mostly dummy files with small
wrappers, so that we are able to include memcpy_64.S
unmodified.

Signed-off-by: Hitoshi Mitake <mitake@dcl.info.waseda.ac.jp>
Cc: h.mitake@gmail.com
Cc: Miao Xie <miaox@cn.fujitsu.com>
Cc: Ma Ling <ling.ma@intel.com>
Cc: Zhao Yakui <yakui.zhao@intel.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Andi Kleen <andi@firstfloor.org>
LKML-Reference: <1290668693-27068-2-git-send-email-mitake@dcl.info.waseda.ac.jp>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-11-26 08:15:57 +01:00
Hitoshi Mitake 49ce8fc651 perf bench: Print both of prefaulted and no prefaulted results by default
After applying this patch, perf bench mem memcpy prints
both of prefualted and without prefaulted score of memcpy().

New options --no-prefault and --only-prefault are added
to print single result, mainly for scripting usage.

Usage example:

 | mitake@X201i:~/linux/.../tools/perf% ./perf bench mem memcpy -l 500MB
 | # Running mem/memcpy benchmark...
 | # Copying 500MB Bytes ...
 |
 |      634.969014 MB/Sec
 |        4.828062 GB/Sec (with prefault)
 | mitake@X201i:~/linux/.../tools/perf% ./perf bench mem memcpy -l 500MB --only-prefault
 | # Running mem/memcpy benchmark...
 | # Copying 500MB Bytes ...
 |
 |        4.705192 GB/Sec (with prefault)
 | mitake@X201i:~/linux/.../tools/perf% ./perf bench mem memcpy -l 500MB --no-prefault
 | # Running mem/memcpy benchmark...
 | # Copying 500MB Bytes ...
 |
 |      642.725568 MB/Sec

Signed-off-by: Hitoshi Mitake <mitake@dcl.info.waseda.ac.jp>
Cc: h.mitake@gmail.com
Cc: Miao Xie <miaox@cn.fujitsu.com>
Cc: Ma Ling <ling.ma@intel.com>
Cc: Zhao Yakui <yakui.zhao@intel.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Andi Kleen <andi@firstfloor.org>
LKML-Reference: <1290668693-27068-1-git-send-email-mitake@dcl.info.waseda.ac.jp>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-11-26 08:15:57 +01:00
Rabin Vincent 02a9d03772 perf symbols: Remove incorrect open-coded container_of()
At least on ARM, padding is inserted between rb_node and sym in struct
symbol_name_rb_node, causing "((void *)sym) - sizeof(struct rb_node)" to
point inside rb_node rather than to the symbol_name_rb_node.  Fix this
by converting the code to use container_of().

Cc: Ian Munsie <imunsie@au1.ibm.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Ming Lei <tom.leiming@gmail.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Tom Zanussi <tzanussi@gmail.com>
LKML-Reference: <20101123163106.GA25677@debian>
Signed-off-by: Rabin Vincent <rabin@rab.in>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-11-23 16:30:32 -02:00
Arnaldo Carvalho de Melo c1a3a4b90a perf record: Handle restrictive permissions in /proc/{kallsyms,modules}
The 59365d1 commit, even being reverted by 33e0d57, showed a non robust
behavior in 'perf record': it really should just warn the user that some
functionality will not be available.

The new behavior then becomes:

	[acme@felicio linux]$ ls -la /proc/{kallsyms,modules}
	-r-------- 1 root root 0 Nov 22 12:19 /proc/kallsyms
	-r-------- 1 root root 0 Nov 22 12:19 /proc/modules
	[acme@felicio linux]$ perf record ls -R > /dev/null
	Couldn't record kernel reference relocation symbol
	Symbol resolution may be skewed if relocation was used (e.g. kexec).
	Check /proc/kallsyms permission or run as root.
	[ perf record: Woken up 1 times to write data ]
	[ perf record: Captured and wrote 0.004 MB perf.data (~161 samples) ]
	[acme@felicio linux]$ perf report --stdio
	[kernel.kallsyms] with build id 77b05e00e64e4de1c9347d83879779b540d69f00 not found, continuing without symbols
	# Events: 98  cycles
	#
	# Overhead  Command    Shared Object                Symbol
	# ........  .......  ...............  ....................
	#
	    48.26%       ls  [kernel]         [k] ffffffff8102b92b
	    22.49%       ls  libc-2.12.90.so  [.] __strlen_sse2
	     8.35%       ls  libc-2.12.90.so  [.] __GI___strcoll_l
	     8.17%       ls  ls               [.]            11580
	     3.35%       ls  libc-2.12.90.so  [.] _IO_new_file_xsputn
	     3.33%       ls  libc-2.12.90.so  [.] _int_malloc
	     1.88%       ls  libc-2.12.90.so  [.] _int_free
	     0.84%       ls  libc-2.12.90.so  [.] malloc_consolidate
	     0.84%       ls  libc-2.12.90.so  [.] __readdir64
	     0.83%       ls  ls               [.] strlen@plt
	     0.83%       ls  libc-2.12.90.so  [.] __GI_fwrite_unlocked
	     0.83%       ls  libc-2.12.90.so  [.] __memcpy_sse2

	#
	# (For a higher level overview, try: perf report --sort comm,dso)
	#
[acme@felicio linux]$

It still has the build-ids for DSOs in the maps with hits:

[acme@felicio linux]$ perf buildid-list
77b05e00e64e4de1c9347d83879779b540d69f00 [kernel.kallsyms]
09c4a431a4a8b648fcfc2c2bdda70f56050ddff1 /bin/ls
af75ea9ad951d25e0f038901a11b3846dccb29a4 /lib64/libc-2.12.90.so
[acme@felicio linux]$

That can be used in another machine to resolve kernel symbols.

Cc: Eugene Teo <eugeneteo@kernel.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Jesper Juhl <jj@chaosbits.net>
Cc: Marcus Meissner <meissner@suse.de>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: Stephane Eranian <eranian@google.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Tom Zanussi <tzanussi@gmail.com>
LKML-Reference: <new-submission>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-11-22 14:01:55 -02:00
Corey Ashford d9cf837ef9 perf stat: Change and clean up sys_perf_event_open error handling
This patch makes several changes to "perf stat":

- "perf stat" will no longer go ahead and run the application when one or
more of the specified events could not be opened.
- Use error() and die() instead of pr_err() so that the output is more
consistent with "perf top" and "perf record".
- Handle permission errors in a more robust way, and in a similar way to
"perf record" and "perf top".

In addition, the sys_perf_event_open() error handling of "perf top" and "perf
record" is made more consistent and adds the following phrase when an event
doesn't open (with something ther than an access or permission error):

"/bin/dmesg may provide additional information."

This is added because kernel code doesn't have a good way of expressing
detailed errors to user space, so its only avenue is to use printk's.  However,
many users may not think of looking at dmesg to find out why an event is being
rejected.

Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Peter Zijlstra <fweisbec@gmail.com>
Cc: Ian Munsie <ianmunsi@au1.ibm.com>
Cc: Michael Ellerman <michaele@au1.ibm.com>
LKML-Reference: <1290217044-26293-1-git-send-email-cjashfor@linux.vnet.ibm.com>
Signed-off-by: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-11-20 13:04:15 -02:00
Robert Morell a71123977e perf tools: Remove hardcoded include paths for elfutils
This change removes the use of hardcoded absolute "/usr/include/elfutils" paths
from the perf build.  The problem with hardcoded paths is that it prevents them
from being overridden by $prefix or by -I in CFLAGS (e.g., for cross-compiling
purposes).

Instead, just include the "elfutils/" subdirectory as a relative path when
files are needed from that directory.

Tested by building perf:
- Cross-compiled for ARM on x86_64
- Built natively on x86_64
- Built on x86_64 with /usr/include/elfutils moved to another location
  and manually included in CFLAGS

Acked-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
LKML-Reference: <1289945793-31441-1-git-send-email-rmorell@nvidia.com>
Signed-off-by: Robert Morell <rmorell@nvidia.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-11-19 16:38:04 -02:00
Stephane Eranian f5b4a9c3ab perf stat: Add no-aggregation mode to -a
This patch adds a new -A option to perf stat. If specified then perf stat does
not aggregate counts across all monitored CPUs in system-wide mode, i.e., when
using -a. This option is not supported in per-thread mode.

Being able to get a per-cpu breakdown is useful to detect imbalances between
CPUs when running a uniform workload than spans all monitored CPUs.

The second version corrects the missing cpumap[] support, so that it works when
the -C option is used.

The third version fixes a missing cpumap[] in print_counter() and removes a
stray patch in builtin-trace.c.

Examples on a 4-way system:

# perf stat -a   -e cycles,instructions -- sleep 1
 Performance counter stats for 'sleep 1':
         9592808135  cycles
         3490380006  instructions             #      0.364 IPC
        1.001584632  seconds time elapsed

# perf stat -a -A -e cycles,instructions -- sleep 1
 Performance counter stats for 'sleep 1':
CPU0            2398163767  cycles
CPU1            2398180817  cycles
CPU2            2398217115  cycles
CPU3            2398247483  cycles
CPU0             872282046  instructions             #      0.364 IPC
CPU1             873481776  instructions             #      0.364 IPC
CPU2             872638127  instructions             #      0.364 IPC
CPU3             872437789  instructions             #      0.364 IPC
        1.001556052  seconds time elapsed

Cc: David S. Miller <davem@davemloft.net>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Robert Richter <robert.richter@amd.com>
LKML-Reference: <4ce257b5.1e07e30a.7b6b.3aa9@mx.google.com>
Signed-off-by: Stephane Eranian <eranian@google.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-11-19 16:16:53 -02:00
Tom Zanussi 7e55055e5b perf trace: update usage
Update usage to reflect the different perf trace variants.

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Acked-by: Arnaldo Carvalho de Melo <acme@ghostprotocols.net>
2010-11-10 08:20:45 -06:00
Tom Zanussi d3c4f798ac perf trace: update Documentation with new perf trace variants
Add documentation describing new 'perf trace' command changes
e.g. <command> handling and live-mode/top variants.

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Acked-by: Arnaldo Carvalho de Melo <acme@ghostprotocols.net>
2010-11-10 08:19:35 -06:00
Tom Zanussi b5b8731219 perf trace: live-mode command-line cleanup
This patch attempts to make the perf trace command-line for live-mode
commands more user-friendly and consistent with other perf commands.

The main change it makes is to allow <commands> to be run as part of
perf trace live-mode commands, as other perf commands do, instead of
the system-wide traces they're currently hard-coded to by the shell
scripts.

With this patch, the following live-mode trace now works as expected:

 $ perf trace rw-by-pid ls -al

The previous system-wide behavior for this command would still be
available by explicitly specifying -a:

 $ perf trace rw-by-pid -a ls -al

and if no <command> is specified, the output is also system-wide:

 $ perf trace rw-by-pid

Because live-mode requires both record and report steps to be invoked,
it isn't always possible to know which args to send to the report and
which to send to the record steps - mainly this is the case for report
scripts with optional args - in those cases it would be necessary to
use separate 'perf trace record' and 'perf trace report' steps.

For example:

 $ perf trace syscall-counts ls

Here we can't decide whether ls should be passed as a param to the
syscall-counts script or whether we should invoke ls as a <command>.
In these cases, we just say that we'll ignore optional script params
and always interpret the extra arguments as a <command>.

If the user instead wants the other interpretation, that can be
accomplished by using separate record and report commands explicitly:

 $ perf trace record syscall-counts
 $ perf trace report syscall-counts ls

So the rules that this patch implements, which seem to make the most
intuitive sense for live-mode commands:

- for commands with optional args and commands with no args, no args
  are sent to the report script, all are sent to the record step

- for 'top' commands i.e. that end with 'top', <commands> can't be
  used - all extra args are send to the report script as params

- for commands with required args, the n required args are taken to be
  the first n args after the script name and sent to the report
  script, and the rest are sent to the record step

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Acked-by: Arnaldo Carvalho de Melo <acme@ghostprotocols.net>
2010-11-10 08:16:51 -06:00
Tom Zanussi 34c86ea97e perf trace record: handle commands correctly
Because the perf-trace shell scripts hard-coded the use of the
perf-record system-wide param, a perf trace record session was always
system wide, even if it was given a command.

If given a command, perf trace record now only records the events for
the command, as users expect.

If no command is given, or if the '-a' option is used, the recorded
events are system-wide, as before.

root@tropicana:~# perf trace record syscall-counts ls -al
root@tropicana:~# perf trace
              ls-23152 [000] 39984.890387: sys_enter: NR 12 (0, 0, 0, 0, 0, 0)
              ls-23152 [000] 39984.890404: sys_enter: NR 9 (0, 0, 0, 0, 0, 0)

root@tropicana:~# perf trace record syscall-counts -a ls -al
root@tropicana:~# perf trace
    npviewer.bin-22297 [000] 39831.102709: sys_enter: NR 168 (0, 0, 0, 0, 0, 0)
              ls-23111 [000] 39831.107679: sys_enter: NR 59 (0, 0, 0, 0, 0, 0)

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Acked-by: Arnaldo Carvalho de Melo <acme@ghostprotocols.net>
2010-11-10 08:15:43 -06:00
Tom Zanussi bca647aac5 perf record: make the record options available outside perf record
Other perf commands that invoke perf record, such as perf trace, may
want to reuse the options used by perf record.

This makes them non-static and renames them to avoid clashes with
other 'options' variables.

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Acked-by: Arnaldo Carvalho de Melo <acme@ghostprotocols.net>
2010-11-10 08:11:30 -06:00
Tom Zanussi b0b6d914e2 perf trace scripting: remove system-wide param from shell scripts
Including -a unconditionally when recording doesn't allow for the
option of running scripts without it.  Future patches will add add it
back if needed at run-time.

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Acked-by: Arnaldo Carvalho de Melo <acme@ghostprotocols.net>
2010-11-10 08:08:20 -06:00
Tom Zanussi e8719adf30 perf trace scripting: fix some small memory leaks and missing error checks
Free the other two fields of script_desc which somehow got overlooked,
free malloc'ed args in case exec fails, and add missing checks for
failed mallocs.

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Acked-by: Arnaldo Carvalho de Melo <acme@ghostprotocols.net>
2010-11-10 07:52:32 -06:00
Corey Ashford 01797c5998 perf: Fix usages of profile_cpu in builtin-top.c to use cpu_list
profile_cpu was left over from an earlier implementation that
supported running perf top on a single CPU.  profile_cpu was no
longer set by any switch and usages of it resulted in dead code.

Instead, convert the code to use cpu_list, which is set by the
-C <cpu_list> option.

Also improved the printing of nr_cpus and cpu_list by correcting
the plurals.

Signed-off-by: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: a.p.zijlstra@chello.nl
Cc: acme@redhat.com
LKML-Reference: <1289269245-9388-1-git-send-email-cjashfor@linux.vnet.ibm.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-11-10 09:16:12 +01:00
Cyrill Gorcunov a3da8e4513 perf, ui: Eliminate stack-smashing protection compiler complaint
The gcc complains about small auto-var strings being allocated from stack space.
Make them const to avoid this:

 | CC util/ui/util.o
 | cc1: warnings being treated as errors
 | util/ui/util.c: In function ‘ui__dialog_yesno’:
 | util/ui/util.c:108: error: not protecting function: no buffer at least 8 bytes long
 | make: *** [util/ui/util.o] Error 1

The real bug is in the newtWinChoice() ABI - but that's an
externality we cannot fix here, so we use this workaround.

Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Acked-by: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
LKML-Reference: <20101106084724.GA5956@lenovo>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-11-10 09:04:32 +01:00
Ingo Molnar e25804a032 Merge branch 'perf/core' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux-2.6 into perf/urgent 2010-10-27 08:25:15 +02:00
Arnaldo Carvalho de Melo 00204c3396 perf python scripting: Add futex-contention script
The equivalent to this SystemTAP script:

http://sourceware.org/systemtap/wiki/WSFutexContention

[root@doppio ~]# perf trace futex-contention
Press control+C to stop and show the summary
^Cnpviewer.bin[15242] lock 7f0a8be19104 contended 29 times, 72806 avg ns
npviewer.bin[15242] lock 7f0a8be19130 contended 2 times, 1355 avg ns
synergyc[17245] lock f127f4 contended 1 times, 1830569 avg ns
firefox[15116] lock 7f2b7238af0c contended 168 times, 1230390 avg ns
synergyc[17245] lock f2fc20 contended 1 times, 33149 avg ns
npviewer.bin[15255] lock 7f0a8be19074 contended 155 times, 73047 avg ns
npviewer.bin[15255] lock 7f0a8be190a0 contended 127 times, 7088 avg ns
synergyc[17247] lock f12854 contended 1 times, 46741 avg ns
synergyc[17245] lock f12610 contended 1 times, 7358 avg ns
[root@doppio ~]#

Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Stephane Eranian <eranian@google.com>
Cc: Tom Zanussi <tzanussi@gmail.com>
LKML-Reference: <new-submission>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-10-26 17:07:33 -02:00
Arnaldo Carvalho de Melo 22d0594b31 perf python scripting: Fixup cut'n'paste error in sctop script
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Cc: Tom Zanussi <tzanussi@gmail.com>
LKML-Reference: <new-submission>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-10-26 15:21:15 -02:00
Arnaldo Carvalho de Melo b44308f540 perf scripting: Shut up 'perf record' final status
We want just the script output, not internal details about the record phase.

Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Cc: Tom Zanussi <tzanussi@gmail.com>
LKML-Reference: <new-submission>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-10-26 15:20:09 -02:00
Matt Fleming 0ab7368f8d perf record: Remove newline character from perror() argument
If we include a newline character in the string argument to perror()
then the output will be split across two lines like so,

    Unable to read perf file descriptor
    : No space left on device

Deleting the newline character prints a much more readable error,

    Unable to read perf file descriptor: No space left on device

Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Tom Zanussi <tzanussi@gmail.com>
LKML-Reference: <89e77b54659bc3798b23a5596c2debb7f6f4cf27.1283010281.git.matt@console-pimps.org>
Signed-off-by: Matt Fleming <matt@console-pimps.org>
Signed-off-by: Arnaldo Carvalho de Melo <acme@felicio.ghostprotocols.net>
2010-10-26 13:03:09 -02:00
Ingo Molnar 5df414c61e Merge branch 'perf/core' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux-2.6 into perf/urgent 2010-10-26 09:30:22 +02:00
Arnaldo Carvalho de Melo 7f6c1bd50d perf python scripting: Support fedora 11 (audit 1.7.17)
Where we don't have the audit.MACH_ARMEB constant.

Cc: David S. Miller <davem@davemloft.net>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Cc: Tom Zanussi <tzanussi@gmail.com>
LKML-Reference: <new-submission>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-10-25 22:12:01 -02:00
Arnaldo Carvalho de Melo a64fa198ba perf python scripting: Improve the syscalls-by-pid script
. Print message at script start telling how to get te summary
. Print the syscall names
. Accept both pid (if numeric) or COMM name

Now it looks like this:

[root@emilia tmp]# perf trace syscall-counts-by-pid
Press control+C to stop and show the summary
^C
syscall events by comm/pid:

comm [pid]/syscalls                            count
----------------------------------------  ----------

automount [1670]
  futex                                            2

sshd [2322]
  rt_sigprocmask                                   4
  select                                           2
  write                                            1
  read                                             1

perf [15178]
  read                                          2506
  open                                           794
  close                                          769
  write                                          240
  getdents                                       112
  lseek                                           16
  stat                                             9
  perf_counter_open                                5
  fcntl                                            5
  mmap                                             5
  statfs                                           2

perf [15179]
  read                                         56701
  open                                           499
  stat                                           176
  fstat                                          149
  close                                          109
  mmap                                            98
  brk                                             75
  rt_sigaction                                    66
  munmap                                          42
  mprotect                                        24
  lstat                                            7
  lseek                                            5
  getdents                                         4
  ioctl                                            3
  readlink                                         2
  futex                                            1
  statfs                                           1
  getegid                                          1
  geteuid                                          1
  getgid                                           1
  getuid                                           1
  getrlimit                                        1
  fcntl                                            1
  uname                                            1
  write                                            1
[root@emilia tmp]# fg
-bash: fg: current: no such job
[root@emilia tmp]# perf trace syscall-counts-by-pid 2322
Press control+C to stop and show the summary
^C
syscall events by comm/pid:

comm [pid]/syscalls                            count
----------------------------------------  ----------

sshd [2322]
  rt_sigprocmask                                   4
  select                                           2
  write                                            1
  read                                             1
[root@emilia tmp]# perf trace syscall-counts-by-pid sshd
Press control+C to stop and show the summary
^C
syscall events for sshd:

comm [pid]/syscalls                            count
----------------------------------------  ----------

sshd [2322]
  rt_sigprocmask                                   4
  select                                           2
  write                                            1
  read                                             1
[root@emilia tmp]#

Cc: David S. Miller <davem@davemloft.net>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Cc: Tom Zanussi <tzanussi@gmail.com>
LKML-Reference: <new-submission>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-10-25 18:48:15 -02:00
Arnaldo Carvalho de Melo 2e7d1e3fb8 perf python scripting: print the syscall name on sctop
[root@emilia tmp]# perf trace sctop 1
syscall events:

event                                          count
----------------------------------------  ----------
read                                          215400
futex                                           4029
write                                            376
brk                                               33
rt_sigprocmask                                    24
select                                            17
lseek                                              2
fsync                                              1
^C[root@emilia tmp]#

Cc: David S. Miller <davem@davemloft.net>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Cc: Tom Zanussi <tzanussi@gmail.com>
LKML-Reference: <new-submission>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-10-25 18:47:27 -02:00
Arnaldo Carvalho de Melo 6545aaa561 perf python scripting: Improve the syscalls-counts script
. Print message at script start telling how to get te summary
. Print the syscall name

Now it looks like this:

[root@emilia ~]# perf trace syscall-counts
Press control+C to stop and show the summary
^C
syscall events:

event                                          count
----------------------------------------  -----------
read                                          102752
open                                            1293
close                                            878
write                                            319
stat                                             185
fstat                                            149
getdents                                         116
mmap                                              98
brk                                               80
rt_sigaction                                      66
munmap                                            42
mprotect                                          24
lseek                                             21
lstat                                              7
rt_sigprocmask                                     4
futex                                              3
statfs                                             3
ioctl                                              3
readlink                                           2
select                                             2
getegid                                            1
geteuid                                            1
getgid                                             1
getuid                                             1
getrlimit                                          1
fcntl                                              1
uname                                              1
[root@emilia ~]#

Cc: David S. Miller <davem@davemloft.net>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Cc: Tom Zanussi <tzanussi@gmail.com>
LKML-Reference: <new-submission>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-10-25 18:47:11 -02:00
Arnaldo Carvalho de Melo 6cc7361440 perf python scripting: Improve the failed-syscalls-by-pid script
. Print message at script start telling how to get te summary
. Print the syscall name using the audit-lib-python package, if
  installed
. Print the errno string
. Accept both pid (if numeric) or COMM name

Now it looks like this:

[root@emilia ~]# perf trace failed-syscalls-by-pid
Press control+C to stop and show the summary
^C
syscall errors:

comm [pid]                           count
------------------------------  ----------

automount [1670]
  syscall: futex
    err = ETIMEDOUT                     39

irqbalance [1462]
  syscall: openat
    err = ENOENT                         4

perf [7888]
  syscall: lseek
    err = ESPIPE                         1
  syscall: open
    err = ENOENT                        24

perf [7889]
  syscall: ioctl
    err = EINVAL                         1
  syscall: readlink
    err = EINVAL                         2
  syscall: open
    err = ENOENT                       389
  syscall: stat
    err = ENOENT                       141
  syscall: lseek
    err = ESPIPE                         3
[root@emilia ~]#

[root@emilia ~]# perf trace failed-syscalls-by-pid 1670
Press control+C to stop and show the summary
^C
syscall errors:

comm [pid]                           count
------------------------------  ----------

automount [1670]
  syscall: futex
    err = ETIMEDOUT                      2
[root@emilia ~]#
[root@emilia ~]#
[root@emilia ~]#
[root@emilia ~]# perf trace failed-syscalls-by-pid automount
Press control+C to stop and show the summary
^C
syscall errors for automount:

comm [pid]                           count
------------------------------  ----------

automount [1669]
  syscall: futex
    err = ETIMEDOUT                      1

automount [1670]
  syscall: futex
    err = ETIMEDOUT                      5
[root@emilia ~]#

Cc: David S. Miller <davem@davemloft.net>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Cc: Tom Zanussi <tzanussi@gmail.com>
LKML-Reference: <new-submission>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-10-25 18:46:41 -02:00
Ingo Molnar b8ecad8b2f Merge branch 'perf/core' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux-2.6 into perf/urgent 2010-10-23 20:05:43 +02:00
Arnaldo Carvalho de Melo 66a301c380 perf probe: Fix format specified for Dwarf_Off parameter
Fixing the following error on 32-bit arches:

util/probe-finder.c: In function ‘line_range_search_cb’:
util/probe-finder.c:1734: error: format ‘%lx’ expects type ‘long
unsigned int’, but argument 3 has type ‘Dwarf_Off’

Reported-by: Ingo Molnar <mingo@elte.hu>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
LKML-Reference: <new-submission>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-10-23 15:31:20 -02:00
Ben Hutchings d1e95bb530 perf trace: Fix detection of script extension
The extension starts with the last dot in the name, not the first.

Cc: Ingo Molnar <mingo@elte.hu>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
LKML-Reference: <1286723462.2955.206.camel@localhost>
Signed-off-by: Ben Hutchings <ben@decadent.org.uk>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-10-23 15:31:20 -02:00
Ben Hutchings 44e668c6fa perf trace: Use $PERF_EXEC_PATH in canned report scripts
Set $PERF_EXEC_PATH before starting the record and report scripts, and
make them use it where necessary.

Cc: Ingo Molnar <mingo@elte.hu>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
LKML-Reference: <1286723403.2955.205.camel@localhost>
Signed-off-by: Ben Hutchings <ben@decadent.org.uk>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-10-23 15:31:20 -02:00
Sonny Rao ffec516976 perf tools: Document event modifiers
Existing documentation doesn't discuss event modifiers, so add a description of
what's currently possible to the documentation of perf-list.

Cc: Ingo Molnar <mingo@elte.hu>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Robert Richter <robert.richter@amd.com>
LKML-Reference: <1287107460-12112-1-git-send-email-sonnyrao@linux.vnet.ibm.com>
Signed-off-by: Sonny Rao <sonnyrao@linux.vnet.ibm.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-10-23 15:31:20 -02:00