Enable Coccinelle SmPL patches to require a specific version of
Coccinelle. In the event that the version does not match we just
inform the user, if the user asked to go through all SmPL patches
we just inform them of the need for a new version of coccinelle for
the SmPL patch and continue on with the rest.
This uses the simple kernel scripts/ld-version.sh to create a weight
on the version provided by spatch. The -dirty attribute is ignored if
supplied, the benefit of scripts/ld-version.sh is it has a long history
and well tested.
While at it, document the // Options stuff as well.
v4: Document // Options and // Requires as well on
Documentation/coccinelle.txt.
Signed-off-by: Luis R. Rodriguez <mcgrof@kernel.org>
Acked-by: Nicolas Palix <nicolas.palix@imag.fr>
Signed-off-by: Michal Marek <mmarek@suse.com>
Coccinelle supports reading .cocciconfig, the order of precedence for
variables for .cocciconfig is as follows:
o Your current user's home directory is processed first
o Your directory from which spatch is called is processed next
o The directory provided with the --dir option is processed last, if used
Since coccicheck runs through make, it naturally runs from the kernel
proper dir, as such the second rule above would be implied for picking up a
.cocciconfig when using 'make coccicheck'.
'make coccicheck' also supports using M= targets.If you do not supply
any M= target, it is assumed you want to target the entire kernel.
The kernel coccicheck script has:
if [ "$KBUILD_EXTMOD" = "" ] ; then
OPTIONS="--dir $srctree $COCCIINCLUDE"
else
OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE"
fi
KBUILD_EXTMOD is set when an explicit target with M= is used. For both cases
the spatch --dir argument is used, as such third rule applies when
whether M= is used or not, and when M= is used the target directory can
have its own .cocciconfig file. When M= is not passed as an argument to
coccicheck the target directory is the same as the directory from where
spatch was called.
If not using the kernel's coccicheck target, keep the above precedence order
logic of .cocciconfig reading. If using the kernel's coccicheck target,
override any of the kernel's .coccicheck's settings using SPFLAGS.
We help Coccinelle when used against Linux with a set of sensible defaults
options for Linux with our own Linux .cocciconfig. This hints to coccinelle
git can be used for 'git grep' queries over coccigrep. A timeout of 200
seconds should suffice for now.
The options picked up by coccinelle when reading a .cocciconfig do not appear
as arguments to spatch processes running on your system, to confirm what
options will be used by Coccinelle run:
spatch --print-options-only
You can override with your own preferred index option by using SPFLAGS.
Coccinelle supports both glimpse and idutils. Glimpse had historically
provided the best performance, however recent benchmarks reveal idutils
is performing just as well. Due to some recent fixes however you however
will need at least coccinelle >= 1.0.6 if using idutils.
Coccinelle carries a script scripts/idutils_index.sh which creates the
idutils database with as follows:
mkid -i C --output .id-utils.index
If using just "--use-idutils" coccinelle expects your idutils database to be
on the top level of the kernel as a file named ".id-utils.index". If you do
not use this you can symlink your database file to it, or you can specify the
database file following the "--use-idutils" argument. Examples:
make SPFLAGS=--use-idutils coccicheck
This assumes you have $srctree/.id-utils.index, where $srctree is
the top level of the kernel.
make SPFLAGS="--use-idutils /full-path/to/ID" coccicheck
Here you specify the full path of the idutils ID database. Using
.cocciconfig is possible, however given the order of precedence followed
by Coccinelle, and since the kernel now carries its own .cocciconfig,
you will need to use SPFLAGS to use idutils if desired.
v4:
o Recommend upgrade for using idutils with coccinelle due to some
recent fixes.
o Refer to using --print-options-only for testing what options are
picked up by .cocciconfig reading.
o Expand commit log considerably explaining *why* .cocconfig from
two precedence rules are used when using coccicheck, and how to
properly override these if needed.
o Expand Documentation/coccinelle.txt
v3: Expand commit log a bit more
Signed-off-by: Luis R. Rodriguez <mcgrof@kernel.org>
Acked-by: Julia Lawall <julia.lawall@lip6.fr>
Signed-off-by: Michal Marek <mmarek@suse.com>
When debugging (using --profile or --show-trying) you want to
avoid supressing output, use --quiet instead. While at it, extend
documentation for SPFLAGS use.
For instance one can use:
$ export COCCI=scripts/coccinelle/misc/irqf_oneshot.cocci
$ make coccicheck DEBUG_FILE="poo.err" MODE=report SPFLAGS="--profile --show-trying" M=./drivers/mfd/arizona-irq.c
Expand Documentation/coccinelle.txt as well.
v4: expand Documentation/coccinelle.txt
v3: rebased, resolve conflicts, expand Documentation/coccinelle.txt
v2: use egrep instead of the *"=--option"* check, this doesn't work for
disjunctions.
Signed-off-by: Luis R. Rodriguez <mcgrof@kernel.org>
Acked-by: Julia Lawall <julia.lawall@lip6.fr>
Signed-off-by: Michal Marek <mmarek@suse.com>
Enable to capture stderr via a DEBUG_FILE variable passed to
coccicheck. You can now do:
$ rm -f cocci.err
$ export COCCI=scripts/coccinelle/free/kfree.cocci
$ make coccicheck MODE=report DEBUG_FILE=cocci.err
...
$ cat cocci.err
This will be come more useful once we add support to
use more things which would go into stderr, such as
profiling. That will be done separately in another
commit.
Expand Documentation/coccinelle.txt with details.
Signed-off-by: Luis R. Rodriguez <mcgrof@kernel.org>
Acked-by: Nicolas Palix <nicolas.palix@imag.fr>
Signed-off-by: Michal Marek <mmarek@suse.com>
Coccinelle has had parmap support since 1.0.2, this means
it supports --jobs, enabling built-in multithreaded functionality,
instead of needing one to script it out. Just look for --jobs
in the help output to determine if this is supported and use it
only if your number of processors detected is > 1.
If parmap is enabled also enable the load balancing to be dynamic, so
that if a thread finishes early we keep feeding it.
stderr is currently sent to /dev/null, addressing a way to capture
that will be addressed next.
If --jobs is not supported we fallback to the old mechanism.
We expect to deprecate the old mechanism as soon as we can get
confirmation all users are ready.
While at it propagate back into the shell script any coccinelle error
code. When used in serialized mode where all cocci files are run this
also stops processing if an error has occured. This lets us handle some
errors in coccinelle cocci files and if they bail out we should inspect
the errors. This will be more useful later to help annotate coccinelle
version dependency requirements. This will let you run only SmPL files
that your system supports.
Extend Documentation/coccinelle.txt as well.
As a small example, prior to this change, on an 8-core system:
Before:
$ export COCCI=scripts/coccinelle/free/kfree.cocci
$ time make coccicheck MODE=report
...
real 29m14.912s
user 103m1.796s
sys 0m4.464s
After:
real 16m22.435s
user 128m30.060s
sys 0m2.712s
v4:
o expand Documentation/coccinelle.txt to reflect parmap support info
o update commit log to reflect what we actually do now with stderr
o split out DEBUG_FILE use into another patch
o detect number of CPUs and if its 1 then skip parmap support,
note that if you still support parmap, but have 1 CPU you will
also go through the new branches, so the old complex multithreaded process
is skipped as well.
v3:
o move USE_JOBS to avoid being overriden
v2:
o redirect coccinelle stderr to /dev/null by default and
only if DEBUG_FILE is used do we pass it to a file
o fix typo of paramap/parmap
Signed-off-by: Luis R. Rodriguez <mcgrof@kernel.org>
Acked-by: Nicolas Palix <nicolas.palix@imag.fr>
Signed-off-by: Michal Marek <mmarek@suse.com>
SPFLAGS is set early, it means that any heuristics done on
coccicheck cannot be overridden currently. Move SPFLAGS
after OPTIONS and set this at the end. This lets you override
any heuristics as coccinelle treats conflicts by only listening
to the last option that makes sense.
v3: this patch was added in the v3 series
v4: Update Documentation/coccinelle.txt explaining how
SPFLAGS works as well.
Signed-off-by: Luis R. Rodriguez <mcgrof@kernel.org>
Acked-by: Nicolas Palix <nicolas.palix@imag.fr>
Signed-off-by: Michal Marek <mmarek@suse.com>
The naming convention of options has changed one year ago.
The options have been recently updated in the cocci file
and in scripts/coccicheck. This patch also adds this information
in the documentation.
Signed-off-by: Nicolas Palix <nicolas.palix@imag.fr>
Signed-off-by: Michal Marek <mmarek@suse.cz>
- The new default mode is 'report'.
- The available modes are detailed a bit more.
- Some information about the use of spatch options are
also given concerning the use of indexing tools.
Signed-off-by: Nicolas Palix <nicolas.palix@imag.fr>
Signed-off-by: Michal Marek <mmarek@suse.cz>
This adds parallelism by default to the "coccicheck" target using
spatch's "-max" and "-index" arguments.
Signed-off-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Nicolas Palix <nicolas.palix@imag.fr>
Signed-off-by: Michal Marek <mmarek@suse.cz>
The SPFLAGS variable allows to pass additional options
to spatch, e.g. -use_glimpse.
Signed-off-by: Nicolas Palix <nicolas.palix@imag.fr>
Signed-off-by: Michal Marek <mmarek@suse.cz>
Do not run with verbosity on/off depending on the ONLINE variable,
which gets set with C=1 or C=2, but allow the user to set the
verbosity using kernel default make V= paramemter.
Verbosity is off by default now.
Signed-off-by: Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>
CC: Julia Lawall <Julia.Lawall@lip6.fr>
Acked-by: Nicolas Palix <nicolas.palix@imag.fr>
Signed-off-by: Michal Marek <mmarek@suse.cz>
Adding documentation for the new M= option which limits Coccinelle
to a specific set of directories.
Signed-off-by: Greg Dietsche <Gregory.Dietsche@cuw.edu>
Acked-by: Julia Lawall <julia.lawall@lip6.fr>
Signed-off-by: Michal Marek <mmarek@suse.cz>
For doubleinit.cocci, Coccinelle 0.2.4 requires a comma after ... in a
field list. Coccinelle also now behaves gracefully when a definition is
provided for a virtual that doesn't exist, so there is no need for the
semantic patch code to check for this case.
Updated the documentation to reflect the fact that the best results will
now be obtained with Coccinelle version 0.2.4 or later.
Signed-off-by: Julia Lawall <julia@diku.dk>
A file used as example has been moved elsewhere.
Update the documentation accordingly
Signed-off-by: Nicolas Palix <npalix.work@gmail.com>
Reported-by: Julia Lawall <julia@diku.dk>
Signed-off-by: Michal Marek <mmarek@suse.cz>
- Add information about use of the C={1,2} make flag
- Add a description of the new chain mode mechanism
- Add a link to the wiki
Signed-off-by: Nicolas Palix <npalix.work@gmail.com>
Signed-off-by: Julia Lawall <julia@diku.dk>
Signed-off-by: Michal Marek <mmarek@suse.cz>
10.04 is Lucid, not Karmic.
Signed-off-by: Kulikov Vasiliy <segooon@gmail.com>
Acked-by: Nicolas Palix <npalix@diku.dk>
Signed-off-by: Michal Marek <mmarek@suse.cz>
The purpose of this file is to document how to use Coccinelle and its
spatch tool to check the Linux kernel.
It gives information on where and how to retrieve Coccinelle, and how
to use it with the Coccinelle scripts integrated in the Linux kernel.
Signed-off-by: Nicolas Palix <npalix@diku.dk>
Signed-off-by: Julia Lawall <julia@diku.dk>
Signed-off-by: Michal Marek <mmarek@suse.cz>
Luis R. Rodriguez