adjust current v*pr_info() calls to fit an overview..detail scheme:
1- module level activity: add/remove, etc
2- command ingest, splitting, summary of effects.
per >control write
3- command parsing: op, flags, search terms
4- per-site change msg
can yield ~3k x 2 logs per echo "+p;-p" > command.
Summarize these 4 levels in MODULE_PARM_DESC, and update verbose=3 in Doc.
2- is new, to isolate a problem where a stress-test script (which
feeds ~4kb multi-command strings) would produce short writes,
truncating last command and causing parsing errors, which confused
test results. The script fix was to use syswrite, to deliver full
proper commands.
4- gets per-callsite "changed:" pr-infos, which are very noisy during
stress tests, and formerly obscured v1-3 messages, and overwhelmed the
static-key workload being tested.
The verbose parameter has previously seen adjustment:
commit 481c0e33f1 ("dyndbg: refine debug verbosity; 1 is basic, 2 more chatty")
The script driving these adjustments is:
!/usr/bin/perl -w
=for Doc
1st purpose was to benchmark the effect of wildcard queries on query
performance; if wildcards are risk free cheap enough, we can deploy
them in the (floating) format search. 1st finding: wildcards take 2x
as long to process.
2nd purpose was to benchmark real static-key changes VS simple flag
changes. Found ~100x decrease for the hard work.
The script maximizes workload per >control by packing it a ~4kb
string of "+p; -p;" commands; this uncovered some broken stuff.
The 85th query failed, and appears to be truncated, so is gramatically
incorrect. Its either an error here, or in the kernel. Its not
happening atm, retest.
Plot thickens: fail only happens doing +-p, not +-mf, likely load
dependent. Error remains consistent. Looks like a short write,
longer on writer than kernel-reader. Try syswrite on handle to
control this. That fixed short write.
=cut
use Getopt::Std;
getopts('vN:k:', \my %opts) or die <<EOH;
$0 options:
-v verbose
-k=n kernel dyndbg verbosity
-N=n number of loops.. tbrc
EOH
$opts{N} //= 10; # !undef, 0 tests too long.
my $ctrl = '/proc/dynamic_debug/control';
vx($opts{k}) if defined $opts{k}; # works on -k0
open(my $CTL, '>', $ctrl) or die "cant open $ctrl for writing: $!\n";
sub vx {
my $arg = shift;
my $cmd = "echo $arg > /sys/module/dynamic_debug/parameters/verbose";
system($cmd);
warn("vx problem: rc:$? err:$! qry: $cmd\n") if ($?);
}
sub qryOK {
my $qry = shift;
print "syntax test: <\n$qry>\n" if $opts{v};
my $bytes = syswrite $CTL, $qry;
printf "short read: $bytes / %d\n", length $qry if $bytes < length $qry;
if ($?) {
warn "rc:$? err:$! qry: $qry\n";
return 0;
}
return 1;
}
sub build_queries {
my ($cmd, $flags, $ct) = @_;
# build experiment and reference queries
my $cycle = " $cmd +$flags # on ; $cmd -$flags # off \n";
my $ref = " +$flags ; -$flags \n";
my $len = length $cycle;
my $max = int(4096 / $len); # break/fit to buffer size
$ct |= $max;
print "qry: ct:$max x << \n$cycle >>\n";
return unless qryOK($ref);
return unless qryOK($cycle);
my $wild = $cycle x $ct;
my $empty = $ref x $ct;
printf "len: %d, %d\n", length $wild, length $empty;
return { trial => $wild,
ref => $empty,
probe => $cycle,
zero => $ref,
count => $ct,
max => $max
};
}
my $query_set = build_queries(' file "*" module "*" func "*" ', "mf");
qryOK($query_set->{zero});
qryOK($query_set->{probe});
qryOK($query_set->{ref});
qryOK($query_set->{trial});
use Benchmark;
sub dobatch {
my ($cmd, $flags, $reps, $ct) = @_;
$reps ||= $opts{N};
my $qs = build_queries($cmd, $flags, $ct);
timethese($reps,
{
wildcards => sub {
syswrite $CTL, $qs->{trial};
},
no_search => sub {
syswrite $CTL, $qs->{ref};
}
}
);
}
sub bench_static_key_toggle {
vx 0;
dobatch(' file "*" module "*" func "*" ', "mf");
dobatch(' file "*" module "*" func "*" ', "p");
}
sub bench_verbose_levels {
for my $i (0..4) {
vx $i;
dobatch(' file "*" module "*" func "*" ', "mf");
}
}
bench_static_key_toggle();
__END__
Heres how the test-script runs:
:: verbose=3 parsing info
[ 48.401646] dyndbg: query 95: "file "*" module "*" func "*" -mf # off " mod:*
[ 48.402040] dyndbg: split into words: "file" "*" "module" "*" "func" "*" "-mf"
[ 48.402456] dyndbg: op='-'
[ 48.402615] dyndbg: flags=0x6
[ 48.402779] dyndbg: *flagsp=0x0 *maskp=0xfffffff9
[ 48.403033] dyndbg: parsed: func="*" file="*" module="*" format="" lineno=0-0
[ 48.403674] dyndbg: applied: func="*" file="*" module="*" format="" lineno=0-0
:: verbose=2 >control summary.
~300k site matches/changes per 4kb command
[ 48.404063] dyndbg: processed 96 queries, with 296160 matches, 0 errs
:: 2 queries against each other, no-search vs all-wildcard-search
qry: ct:48 x <<
file "*" module "*" func "*" +mf # on ; file "*" module "*" func "*" -mf # off
>>
len: 4080, 576
Benchmark: timing 10 iterations of no_search, wildcards...
no_search: 0 wallclock secs ( 0.00 usr + 0.03 sys = 0.03 CPU) @ 333.33/s (n=10)
(warning: too few iterations for a reliable count)
wildcards: 0 wallclock secs ( 0.00 usr + 0.09 sys = 0.09 CPU) @ 111.11/s (n=10)
(warning: too few iterations for a reliable count)
:: 2 queries, both doing real work / changing stati-key states.
qry: ct:49 x <<
file "*" module "*" func "*" +p # on ; file "*" module "*" func "*" -p # off
>>
len: 4067, 490
Benchmark: timing 10 iterations of no_search, wildcards...
no_search: 20 wallclock secs ( 0.00 usr + 20.36 sys = 20.36 CPU) @ 0.49/s (n=10)
wildcards: 21 wallclock secs ( 0.00 usr + 21.08 sys = 21.08 CPU) @ 0.47/s (n=10)
bash-5.1#
Thats 150k static-key-toggles / sec
~600x slower than simple flags
on qemu --smp 3 run
Signed-off-by: Jim Cromie <jim.cromie@gmail.com>
Link: https://lore.kernel.org/r/20211019210746.185307-1-jim.cromie@gmail.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Jim pointed out that using $module.dyndbg= is always a more flexible
choice for using dynamic debug on the command line. The $module.dyndbg
style is checked at boot and handles if $module is a builtin. If it is
actually a loadable module, it is handled again later when the module is
loaded.
If you just use dyndbg="module $module +p" dynamic debug is only enabled
when $module is a builtin.
It was recommended to illustrate wildcard usage as well.
Suggested-by: Jim Cromie <jim.cromie@gmail.com>
Signed-off-by: Andrew Halaney <ahalaney@redhat.com>
Signed-off-by: Jason Baron <jbaron@akamai.com>
Link: https://lore.kernel.org/r/1634139622-20667-4-git-send-email-jbaron@akamai.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
This param has been deprecated for a very long time now, let's rip it
out.
Signed-off-by: Andrew Halaney <ahalaney@redhat.com>
Signed-off-by: Jason Baron <jbaron@akamai.com>
Link: https://lore.kernel.org/r/1634139622-20667-3-git-send-email-jbaron@akamai.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
dynamic debug is "expecting pairs of match-spec <value>" so the example
for all files of which the paths include "usb" there is "file" missing.
Signed-off-by: Martin Kepplinger <martin.kepplinger@puri.sm>
Link: https://lore.kernel.org/r/20210303091646.773111-1-martin.kepplinger@puri.sm
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
This reverts commit 14775b0496 as there
were still some parsing problems with it, and the follow-on patch for
it.
Let's revisit it later, just drop it for now.
Cc: <jbaron@akamai.com>
Cc: Jim Cromie <jim.cromie@gmail.com>
Reported-by: Naresh Kamboju <naresh.kamboju@linaro.org>
Cc: Stephen Rothwell <sfr@canb.auug.org.au>
Fixes: 14775b0496 ("dyndbg: accept query terms like file=bar and module=foo")
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Current code expects "keyword" "arg" as 2 words, space separated.
Change to also accept "keyword=arg" form as well, and drop !(nwords%2)
requirement. Then in rest of function, use new keyword, arg variables
instead of word[i], word[i+1]
Acked-by: <jbaron@akamai.com>
Signed-off-by: Jim Cromie <jim.cromie@gmail.com>
Link: https://lore.kernel.org/r/20200719231058.1586423-15-jim.cromie@gmail.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Accept these additional query forms:
echo "file $filestr +_" > control
path/to/file.c:100 # as from control, column 1
path/to/file.c:1-100 # or any legal line-range
path/to/file.c:func_A # as from an editor/browser
path/to/file.c:drm_* # wildcards still work
path/to/file.c:*_foo # lead wildcard too
1st 2 examples are treated as line-ranges, 3-5 are treated as func's
Doc these changes, and sprinkle in a few extra wild-card examples and
trailing # explanation texts.
Acked-by: <jbaron@akamai.com>
Signed-off-by: Jim Cromie <jim.cromie@gmail.com>
Link: https://lore.kernel.org/r/20200719231058.1586423-14-jim.cromie@gmail.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
since cf964976484 in 2012, initialization is done with early_initcall,
update the Docs, which still say arch_initcall.
Acked-by: <jbaron@akamai.com>
Signed-off-by: Jim Cromie <jim.cromie@gmail.com>
Link: https://lore.kernel.org/r/20200719231058.1586423-3-jim.cromie@gmail.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Regarding:
commit 2b6783191d ("dynamic_debug: add trim_prefix() to provide source-root relative paths")
commit a73619a845 ("kbuild: use -fmacro-prefix-map to make __FILE__ a relative path")
2nd commit broke dynamic-debug's "file $fullpath" query form, but
nobody noticed because 1st commit had trimmed prefixes from
control-file output, so the click-copy-pasting of fullpaths into new
queries had ceased; that query form became unused.
Removing the function is cleanest, but it could be useful in
old-compiler corner cases, where __FILE__ still has /full/path,
and it safely does nothing otherwize.
So instead, quietly deprecate "file /full/path" query form, by
removing all /full/paths examples in the docs. I skipped adding a
back-compat note.
Acked-by: <jbaron@akamai.com>
Signed-off-by: Jim Cromie <jim.cromie@gmail.com>
Link: https://lore.kernel.org/r/20200719231058.1586423-2-jim.cromie@gmail.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Instead of enabling dynamic debug globally with CONFIG_DYNAMIC_DEBUG,
CONFIG_DYNAMIC_DEBUG_CORE will only enable core function of dynamic
debug. With the DYNAMIC_DEBUG_MODULE defined for any modules, dynamic
debug will be tied to them.
This is useful for people who only want to enable dynamic debug for
kernel modules without worrying about kernel image size and memory
consumption is increasing too much.
[orson.zhai@unisoc.com: v2]
Link: http://lkml.kernel.org/r/1587408228-10861-1-git-send-email-orson.unisoc@gmail.com
Signed-off-by: Orson Zhai <orson.zhai@unisoc.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Acked-by: Petr Mladek <pmladek@suse.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Jason Baron <jbaron@akamai.com>
Cc: Randy Dunlap <rdunlap@infradead.org>
Link: http://lkml.kernel.org/r/1586521984-5890-1-git-send-email-orson.unisoc@gmail.com
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
With the realization that having debugfs enabled on "production" systems
is generally not a good idea, debugfs is being disabled from more and
more platforms over time. However, the functionality of dynamic
debugging still is needed at times, and since it relies on debugfs for
its user api, having debugfs disabled also forces dynamic debug to be
disabled.
To get around this, also create the "control" file for dynamic_debug in
procfs. This allows people turn on debugging as needed at runtime for
individual driverfs and subsystems.
Reported-by: many different companies
Cc: Jason Baron <jbaron@akamai.com>
Acked-by: Will Deacon <will@kernel.org>
Link: https://lore.kernel.org/r/20200210211142.GB1373304@kroah.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Fix grammar about wildcards and insert a space between sentences.
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Cc: Jason Baron <jbaron@akamai.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: linux-doc@vger.kernel.org
Cc: Will Korteland <will@korte.land>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Fix a few "typos" in dynamic-debug-howto.rst.
s/dyndbg_query/ddebug_query/
s/sysfs/debugfs/
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Cc: Jason Baron <jbaron@akamai.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: linux-doc@vger.kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Fix minor typo.
Fix missing words in explaining parsing of last line number.
Link: http://lkml.kernel.org/r/ebb7ff42-4945-103f-d5b4-f07a6f3343a7@infradead.org
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Cc: Jason Baron <jbaron@akamai.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Two of the example command lines use an argument to echo of "-c" which
isn't valid in (most versions of) echo causing these examples to fail.
Correct the argument to "-n" which works correctly.
Signed-off-by: Steven Price <steven@ecrips.co.uk>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Place README, REPORTING-BUGS, SecurityBugs and kernel-parameters
on an user's manual book.
As we'll be numbering the user's manual, remove the manual
numbering from SecurityBugs.
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Jim Cromie