That means that almost everything you can do with 'perf report'
can be done with 'perf diff', for instance:
$ perf record -f find / > /dev/null
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.062 MB perf.data (~2699
samples) ] $ perf record -f find / > /dev/null
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.062 MB perf.data (~2687
samples) ] perf diff | head -8
9.02% +1.00% find libc-2.10.1.so [.] _IO_vfprintf_internal
2.91% -1.00% find [kernel] [k] __kmalloc
2.85% -1.00% find [kernel] [k] ext4_htree_store_dirent
1.99% -1.00% find [kernel] [k] _atomic_dec_and_lock
2.44% find [kernel] [k] half_md4_transform
$
So if you want to zoom into libc:
$ perf diff --dsos libc-2.10.1.so | head -8
37.34% find [.] _IO_vfprintf_internal
10.34% find [.] __GI_memmove
8.25% +2.00% find [.] _int_malloc
5.07% -1.00% find [.] __GI_mempcpy
7.62% +2.00% find [.] _int_free
$
And if there were multiple commands using libc, it is also
possible to aggregate them all by using --sort symbol:
$ perf diff --dsos libc-2.10.1.so --sort symbol | head -8
37.34% [.] _IO_vfprintf_internal
10.34% [.] __GI_memmove
8.25% +2.00% [.] _int_malloc
5.07% -1.00% [.] __GI_mempcpy
7.62% +2.00% [.] _int_free
$
The displacement column now is off by default, to use it:
perf diff -m --dsos libc-2.10.1.so --sort symbol | head -8
37.34% [.] _IO_vfprintf_internal
10.34% [.] __GI_memmove
8.25% +2.00% [.] _int_malloc
5.07% -1.00% +2 [.] __GI_mempcpy
7.62% +2.00% -1 [.] _int_free
$
Using -t/--field-separator can be used for scripting:
$ perf diff -t, -m --dsos libc-2.10.1.so --sort symbol | head -8
37.34, , ,[.] _IO_vfprintf_internal
10.34, , ,[.] __GI_memmove
8.25,+2.00%, ,[.] _int_malloc
5.07,-1.00%, +2,[.] __GI_mempcpy
7.62,+2.00%, -1,[.] _int_free
6.99,+1.00%, -1,[.] _IO_new_file_xsputn
1.89,-2.00%, +4,[.] __readdir64
$
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1260978567-550-1-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Support event name syntax for --add option. This allows
users to specify event name for each new event.
The --add syntax is:
perf probe --add '[EVENT=]SRC:LINE ARGS'
or
perf probe --add '[EVENT=]FUNC[+OFFS|%return|:RLN][@SRC] ARGS'
e.g.
./perf probe --add myprobe1=schedule
Note: currently group name is not supported yet, because it
can cause name-space confliction with other tracepoint/
hw-breakpoint events.
Signed-off-by: Masami Hiramatsu <mhiramat@redhat.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Jim Keniston <jkenisto@us.ibm.com>
Cc: Ananth N Mavinakayanahalli <ananth@in.ibm.com>
Cc: Christoph Hellwig <hch@infradead.org>
Cc: Frank Ch. Eigler <fche@redhat.com>
Cc: Jason Baron <jbaron@redhat.com>
Cc: K.Prasad <prasad@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Cc: systemtap <systemtap@sources.redhat.com>
Cc: DLE <dle-develop@lists.sourceforge.net>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
LKML-Reference: <20091215153218.17436.84675.stgit@dhcp-100-2-132.bos.redhat.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Fix the short line displayed by 'perf' and also fix some other
details in the longer text.
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
LKML-Reference: <new-submission>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Update the perf-trace page with new and missing options and
remove some unused ones.
Signed-off-by: Tom Zanussi <tzanussi@gmail.com>
Cc: fweisbec@gmail.com
Cc: rostedt@goodmis.org
LKML-Reference: <1260867220-15699-7-git-send-email-tzanussi@gmail.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
As Ingo suggested, make "perf kmem" show help information.
"perf kmem stat [--caller] [--alloc] .." will show memory
statistics.
Signed-off-by: Li Zefan <lizf@cn.fujitsu.com>
Acked-by: Pekka Enberg <penberg@cs.helsinki.fi>
LKML-Reference: <4B20A195.8030106@cn.fujitsu.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Adds perf-trace-perl Documentation and a link to it from the
perf-trace page.
Signed-off-by: Tom Zanussi <tzanussi@gmail.com>
Cc: fweisbec@gmail.com
Cc: rostedt@goodmis.org
Cc: anton@samba.org
Cc: hch@infradead.org
LKML-Reference: <1259133352-23685-7-git-send-email-tzanussi@gmail.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Add the breakpoint events support with this new sysnopsis:
mem:addr[:access]
Where addr is a raw addr value in the kernel and access can be
either [r][w][x]
Example to profile tasklist_lock:
$ grep tasklist_lock /proc/kallsyms
ffffffff8189c000 D tasklist_lock
$ perf record -e mem:0xffffffff8189c000:rw -a -f -c 1
$ perf report
# Samples: 62
#
# Overhead Command Shared Object Symbol
# ........ ............... ............. ......
#
29.03% swapper [kernel] [k] _raw_read_trylock
29.03% swapper [kernel] [k] _raw_read_unlock
19.35% init [kernel] [k] _raw_read_trylock
19.35% init [kernel] [k] _raw_read_unlock
1.61% events/0 [kernel] [k] _raw_read_trylock
1.61% events/0 [kernel] [k] _raw_read_unlock
Coming soon:
- Support for symbols in the event definition.
- Default period to 1 for breakpoint events because these are
not high frequency events. The same thing is needed for trace
events.
Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Prasad <prasad@linux.vnet.ibm.com>
LKML-Reference: <1258987355-8751-4-git-send-email-fweisbec@gmail.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Prasad <prasad@linux.vnet.ibm.com>
Porcelain can ignore it, humans can make more sense of it.
Suggested-by: Frederic Weisbecker <fweisbec@gmail.com>
Suggested-by: Ingo Molnar <mingo@elte.hu>
Suggested-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
LKML-Reference: <1258415125-15019-2-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
With this we can list the buildids in a perf.data file so that
we can pipe them to other, distro specific tools that from the
buildids can figure out separate packages (foo-debuginfo) where
we can find the matching symtabs so that perf report can do its
job.
E.g:
[acme@doppio linux-2.6-tip]$ perf buildid-list | head -5
8e08b117e5458ad3f85da16d42d0fc5cd21c5869
520c2387a587cc5acfcf881e27dba1caaeab4b1f
ec8dd400904ddfcac8b1c343263a790f977159dc
7caedbca5a6d8ab39a7fe44bd28c07d3e14a3f3f
379bb828fd08859dbea73279f04abefabc95a6a3
[acme@doppio linux-2.6-tip]$ perf buildid-list -v | head -5
8e08b117e5458ad3f85da16d42d0fc5cd21c5869 /sbin/init
520c2387a587cc5acfcf881e27dba1caaeab4b1f /lib64/ld-2.10.1.so
ec8dd400904ddfcac8b1c343263a790f977159dc /lib64/libc-2.10.1.so
7caedbca5a6d8ab39a7fe44bd28c07d3e14a3f3f /sbin/udevd
379bb828fd08859dbea73279f04abefabc95a6a3 /lib64/libdl-2.10.1.so
[acme@doppio linux-2.6-tip]$
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
LKML-Reference: <1258396365-29217-5-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
This patch adds new document about perf-bench.
Man page and html will be provided for user.
Signed-off-by: Hitoshi Mitake <mitake@dcl.info.waseda.ac.jp>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1257853855-28934-3-git-send-email-mitake@dcl.info.waseda.ac.jp>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Conflicts:
tools/perf/Makefile
Merge reason:
- fix the conflict
- pick up the pr_*() infrastructure to queue up dependent patch
Signed-off-by: Ingo Molnar <mingo@elte.hu>
During the Kernel Summit demo of perf/ftrace/timechart, there
was a feature request to have a process filter for timechart so
that you can zoom into one or a few processes that you are
really interested in.
This patch adds basic support for this feature, the -p
(--process) option now can select a PID or a process name to be
shown. Multiple -p options are allowed, and the combined set
will be included in the output.
Signed-off-by: Arjan van de Ven <arjan@linux.intel.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
LKML-Reference: <20091020070939.7d0fb8a7@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
For doing work on the Linux power management components, I need to
make long (30+ seconds) traces. Currently, this then results in a
HUGE svg file, with mostly process data that isn't interesting.
This patch adds a --power-only mode to perf timechart that only
outputs the CPU power section of the SVG; this significantly
reduces the size of the SVG file, making even 30+ second traces
viewable with inkscape.
As a minor tweak for the same effect, the minimum text size is
decreased; current inkscape cannot zoom in deep enough to show text
this small, but it reduces inkscape compute time.
Signed-off-by: Arjan van de Ven <arjan@linux.intel.com>
Cc: peterz@infradead.org
LKML-Reference: <20090924154013.0675ab71@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
This patch adds a command line option for timechart that allows the
user to specify the width of the SVG file.
This patch also makes sure that each second of recording has at
least 200 units (pixels at 96 DPI) of width. This impacts
recordings longer than 5 seconds; recordings shorter than 5 second
will scale up to have a width of 1000 units for the whole recording
(as before).
Signed-off-by: Arjan van de Ven <arjan@linux.intel.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
LKML-Reference: <20090920181416.69570c5d@linux.intel.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
As suggested by Ingo, add a timechart man page help text, as well
as add it to the "perf help" overview.
Signed-off-by: Arjan van de Ven <arjan@linux.intel.com>
Cc: fweisbec@gmail.com
Cc: peterz@infradead.org
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <20090919133604.3767fa35@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Alias 'perf sched trace' to 'perf trace', for workflow completeness.
Add a bit of documentation for perf sched.
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
LKML-Reference: <new-submission>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
This turn-key tool allows scheduler measurements to be
conducted and the results be displayed numerically.
First baby step towards that goal: clone the new command off of
perf trace.
Fix a few other details along the way:
- add (minimal) perf trace documentation
- reorder a few places
- list perf trace in the mainporcelain list as well
as it's a very useful utility.
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
LKML-Reference: <new-submission>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
pushd tools/perf/Documentation
make html
popd
is failing for me...
ASCIIDOC perf-annotate.html
ERROR: unsafe: include file: /etc/asciidoc/./stylesheets/xhtml11.css
ERROR: unsafe: include file:
/etc/asciidoc/./stylesheets/xhtml11-manpage.css
ERROR: unsafe: include file:
/etc/asciidoc/./stylesheets/xhtml11-quirks.css
make: *** [perf-annotate.html] Error 1
Apparently asciidoc "unsafe" is the default mode of operation
in practice.
https://bugzilla.redhat.com/show_bug.cgi?id=506953
Works tidily now.
Signed-off-by: Kyle McMartin <kyle@redhat.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
LKML-Reference: <20090818164125.GM25206@bombadil.infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Rename it to examples.txt to avoid the perf-*.txt pattern in
the Makefile, otherwise 'make doc' fails because
perf-examples.txt is not formatted to be a man page:
ERROR: perf-examples.txt: line 1: manpage document title is mandatory
Signed-off-by: Carlos R. Mafra <crmafra@gmail.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <new-submission>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Improve and fix the handling of per-thread counter stats
recorded via perf record -s. Previously we only displayed
it in debug printouts (-D) and even that output was hard
to disambiguate.
I moved everything to utils/values.[ch] so that we may reuse
it in perf stat.
We get something like this now:
# PID TID cache-misses cache-references
4658 4659 495581 3238779
4658 4662 498246 3236823
4658 4663 499531 3243162
Then it'll be easy to add --pretty=raw to display a single line per thread/event.
By the way, -S was also used for --symbol... So I used -T/--thread here.
perf report: Add -T/--threads to display per-thread counter values
We get something like this now:
# PID TID cache-misses cache-references
4658 4659 495581 3238779
4658 4662 498246 3236823
4658 4663 499531 3243162
Per-thread arrays of counter values are managed in utils/values.[ch]
Signed-off-by: Brice Goglin <Brice.Goglin@inria.fr>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: paulus@samba.org
Signed-off-by: Ingo Molnar <mingo@elte.hu>
A few examples of how 'perf' can be used, from an e-mail by
Ingo Molnar http://lkml.org/lkml/2009/8/4/346.
Signed-off-by: Carlos R. Mafra <crmafra2@gmail.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Valdis.Kletnieks@vt.edu
LKML-Reference: <20090805185334.GA4535@Pilar.aei.mpg.de>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
We want to use a coherent flag for -S/--stat across all tools,
so free up -S in perf stat.
Signed-off-by: Brice Goglin <Brice.Goglin@inria.fr>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: paulus@samba.org
Signed-off-by: Ingo Molnar <mingo@elte.hu>
perf_counter tools: update perf top manual page to reflect
current implementation.
Signed-off-by: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Auto-adjust column width of perf report output to the
longest occuring string length.
Example:
[acme@doppio pahole]$ perf report --sort comm,dso,symbol | head -13
12.79% pahole /usr/lib64/libdw-0.141.so [.] __libdw_find_attr
8.90% pahole /lib64/libc-2.10.1.so [.] _int_malloc
8.68% pahole /usr/lib64/libdw-0.141.so [.] __libdw_form_val_len
8.15% pahole /lib64/libc-2.10.1.so [.] __GI_strcmp
6.80% pahole /lib64/libc-2.10.1.so [.] __tsearch
5.54% pahole ./build/libdwarves.so.1.0.0 [.] tag__recode_dwarf_type
[acme@doppio pahole]$
[acme@doppio pahole]$ perf report --sort comm,dso,symbol -d /lib64/libc-2.10.1.so | head -10
21.92% pahole /lib64/libc-2.10.1.so [.] _int_malloc
20.08% pahole /lib64/libc-2.10.1.so [.] __GI_strcmp
16.75% pahole /lib64/libc-2.10.1.so [.] __tsearch
[acme@doppio pahole]$
Also add these extra options to control the new behaviour:
-w, --field-width
Force each column width to the provided list, for large terminal
readability.
-t, --field-separator:
Use a special separator character and don't pad with spaces, replacing
all occurances of this separator in symbol names (and other output) with
a '.' character, that thus it's the only non valid separator.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <20090711014728.GH3452@ghostprotocols.net>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
So that we can filter by comm. Symbols in other comms won't be
accounted for.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1246399282-20934-3-git-send-email-acme@redhat.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
So that we can filter by dso. Symbols in other dsos won't be
accounted for.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1246399282-20934-2-git-send-email-acme@redhat.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
-l is not supported, it should be -S for scale.
Signed-off-by: Jaswinder Singh Rajput <jaswinderrajput@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
LKML-Reference: <1245703959.6167.16.camel@localhost.localdomain>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Several people have suggested that 'perf' has become a full-fledged
tool that should be moved out of Documentation/. Move it to the
(new) tools/ directory.
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
LKML-Reference: <new-submission>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Arnaldo Carvalho de Melo