Sometimes a test kernel will crash or hang on reboot (this is even more
apparent when testing a config without CGROUPS on a box running
systemd). When this happens, on the next iteration of installing a
kernel, ktest will fail when it tries to install.
Have ktest do a check to see if the target can be connected to via ssh
before it tries to install. If it can't connect, then reboot again.
This time the reboot will fail because it can't connect and will force a
power cycle.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Commit fb16d891 "kconfig: replace 'oldnoconfig' with 'olddefconfig', and
keep the old name", changed ktest's default config update from
oldnoconfig to olddefconfig without adding oldnoconfig as a backup.
The make oldnoconfig works much better than its backup of:
yes '' | make oldconfig
But due to this change, and the fact that ktest is used to build lots of
older kernels (and for bisects), it forgoes the oldnoconfig completely.
Cc: Adam Lee <adam8157@gmail.com>
Cc: Michal Marek <mmarek@suse.cz>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
I installed Fedora 17 which no longer supports grub v1. I worked
with grub2 for a while, but there's so many issues with it and automated
rebooting, that I decided to switch to syslinux. Instead of using
the REBOOT_SCRIPT and add customized changes to get syslinux booted,
I thought it better to make ktest aware of syslinux and add options
to simplify the use of syslinux on a target test box.
Acked-by: H. Peter Anvin <hpa@zytor.com>
Cc: John 'Warthog9' Hawley <warthog9@kernel.org>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Before rebooting the target, run the sync command, as it seems that
either Grub2 or systemd gets screwed up if you update to reboot a kernel
once and do a reboot without doing a sync.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
As only grub or 'script' is supported for rebooting to a new kernel,
and Fedora 17 has dropped support for grub, I decided to add grub2
support as well (I also plan on adding syslinux/extlinux support too).
The options GRUB_FILE and GRUB_REBOOT were added to allow the user
to specify where to find the grub.cfg and what tool to use to reboot
into the next kernel respectively.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
In order to decide if ktest should bother installing modules on the
target box, it checks if the config file has CONFIG_MODULES=y. But it
also checks if the '=y' part exists. It only will install modules if the
config exists and is set with '=y'. But as the regex that was used
tests:
/^CONFIG_MODULES(=y)?/
this will also match:
CONFIG_MODULES_USE_ELF_RELA
as the '=y' part was optional and it did not test the rest of the line.
When this happens, ktest will stop checking the rest of the configs but
it will also think that no modules are needed to be installed. What it
should do is only jump out of the loop if it actually found a
CONFIG_MODULES that is set to true.
Otherwise, ktest wont install the necessary modules needed for proper
booting of the test target.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Pull kconfig changes from Michal Marek:
"kconfig in v3.7 is going to
- initialize ncurses only once in menuconfig
- be able to jump to a search result in menuconfig
- change the misnomer oldnoconfig to a more meaningful name
olddefconfig, keeping the old name as alias"
* 'kconfig' of git://git.kernel.org/pub/scm/linux/kernel/git/mmarek/kbuild:
kconfig: replace 'oldnoconfig' with 'olddefconfig', and keep the old name as an alias
menuconfig: Assign jump keys per-page instead of globally
menuconfig: Do not open code textbox scroll up/down
menuconfig: Add jump keys to search results
menuconfig: Extend dialog_textbox so that it can return to a scrolled position
menuconfig: Extend dialog_textbox so that it can exit on arbitrary keypresses
menuconfig: Remove superfluous conditionnal
kconfig: document oldnoconfig to what it really does in conf.c
kconfig/mconf.c: revision of curses initialization.
As 67d34a6a39 said, 'oldnoconfig' doesn't
set new symbols to 'n', but instead sets it to their default values.
So, this patch replaces 'oldnoconfig' with 'olddefconfig', stop making
people confused, and keep the old name 'oldnoconfig' as an alias,
because people already are dependent on its behavior with the
counter-intuitive name.
Signed-off-by: Adam Lee <adam8157@gmail.com>
Signed-off-by: Michal Marek <mmarek@suse.cz>
The ELSE IF statements do not work as expected if another ELSE statement
follows. This is because the $if_set is not set. If the ELSE IF
condition is true, the following ELSE should be ignored. But because the
$if_set is not set, the following ELSE will also be executed.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Add '=~' and '!~' to the list of allowed conditionals for DEFAULT and
TEST_START section if statements.
ie.
TEST_START IF TEST =~ .*test$
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
The option IGNORE_ERRORS is used to allow a test to succeed even if a
warning appears from the kernel. Sometimes kernels will produce warnings
that are not associated with a test, and the user wants to test
something else.
The IGNORE_ERRORS works for boot up, but was not preventing test runs to
succeed if the kernel produced a warning.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
The min configs are saved in a perl hash called force_configs, and this
hash is used to add configs to the .config file. But it was not being
reset between tests and a min config from a previous test would affect
the min config of the next test causing undesirable results.
Reset the force_config hash at the start of each test.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Usually the target is booted into a dependable kernel when a test
starts. The test will install the test kernel and reboot the box. But
there may be a time that the kernel is running an unreliable kernel and
the reboot may crash.
Have ktest detect crashes on a reboot and force a power-cycle instead.
This can usually happen if a test kernel was installed to run manual
tests, but the user forgot to reboot to the known good kernel.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
If the console is constantly outputting content, this can cause ktest
to get stuck waiting on the monitor to settle down.
The option MAX_MONITOR_WAIT is the maximum time (in seconds) for ktest
to wait for the console to flush.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
With a name like 'oldnoconfig' one may think that the config generated
would disable all configs that were not defined (selecting "no" for all
options). But this is not the case. It selects the default. If a config
has a 'default y', then it is added if not specified.
This broke the config bisect, because options not specified by a config
will just use the default, where it expected to turn off. This caused an
option to be enabled that disabled an option that would break the build.
The end result was that we never found the bad config at the end of the
test.
Instead of using 'make oldnoconfig', ktest now builds the options it
expects enabled and disabled. When it turns off an option, it will no
longer remove it, but actually set it to:
# CONFIG_FOO is not set.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
The config-bisect can take a bad config and bisect it down to find out
what config actually breaks the config. But as all tests will apply a
minconfig (defined by a user) to apply before booting, it is possible
that the minconfig could actually make the bad config work (minconfigs
can disable configs). The end result is that the config bisect test will
not find a config that breaks. This can be rather frustrating to the
user.
The CONFIG_BISECT_CHECK option, when set to 1, will make sure that the
bad config (with the minconfig applied) still fails before trying to
bisect.
And yes, I did get burned by this.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Add the PRE_INSTALL option that will allow a user to specify a shell
command to be executed before the install operation executes.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
In order to let the user add commands before and after ktest runs, the
PRE_KTEST and POST_KTEST options are defined. They hold shell commands
that will execute befor ktest runs its first test, as well as when it
completed its last test.
The PRE_TEST and POST_TEST will be run befor and after (respectively)
for a given test. They can either be global (done for all tests) or
defined by a single test.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
If the file that OUTPUT_MIN_CONFIG exists then ktest.pl will prompt the
user and ask them if the OUTPUT_MIN_CONFIG should be used as the
starting point for make_min_config instead of MIN_CONFIG.
This is usually the case, and to allow the user to do so, which is
helpful if the user is creating different min configs based on tests,
and they know one is a superset of another test, they can set
USE_OUTPUT_MIN_CONFIG to one, which will prevent kest.pl from prompting
to use the OUTPUT_MIN_CONFIG and it will just use it.
If USE_OUTPUT_MIN_CONIFG is set to zero, then ktest.pl will continue to
use MIN_CONFIG instead.
The default is that USE_OUTPUT_MIN_CONFIG is undefined.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Add a MIN_CONFIG_TYPE that can be set to 'test' or 'boot'. The default
is 'boot' which is what make_min_config has done previously: makes a
config file that is the minimum needed to boot the target.
But when MIN_CONFIG_TYPE is set to 'test', not only must the target
boot, but it must also successfully run the TEST. This allows the
creation of a config file that is the minimum to boot and also
perform ssh to the target, or anything else a developer wants.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
The PRE_BUILD and POST_BUILD options of ktest are added to allow the
user to add temporary patch to the system and remove it on builds. This
is sometimes use to take a change from another git branch and add it to
a series without the fix so that this series can be tested, when an
unrelated bug exists in the series.
The problem comes when a tagged commit is being used. For example, if
v3.2 is being tested, and we add a patch to it, the kernelrelease for
that commit will be 3.2.0+, but without the patch the version will be
3.2.0. This can cause problems when the kernelrelease is determined for
creating the /lib/modules directory. The kernel booting has the '+' but
the module directory will not, and the modules will be missing for that
boot, and may not allow the kernel to succeed.
The fix is to put the creation of the kernelrelease in the POST_BUILD
logic, before it applies the POST_BUILD operation. The POST_BUILD is
where the patch may be removed, removing the '+' from the kernelrelease.
The calculation of the kernelrelease will also stay in its current
location but will be ignored if it was already calculated previously.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
The change to let individual tests decide to reboot the machine on
success of the entire test also prevented errors from rebooting
when an error was detected.
The "no_reboot" variable was only cleared if the test had
reboot_on_success set. But the no_reboot variable also prevents the test
rebooting when an error was detected even when REBOOT_ON_ERROR was set.
Add a new "reboot_success" variable that is used to determine if the
test should reboot on success and not touch the no_reboot variable.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
When BISECT_REVERSE and BISECT_SKIP are used together with boot or test
testing, build failures are treated as boot or test failures and
'git bisect bad' is executed instead of 'git bisect skip'. This is because
the $ret value of -1 is treated as a build failure, but the $reverse_bisect
logic does not properly handle this.
Simple fix, only invert it if it is positive.
Link: http://lkml.kernel.org/r/1335235380-8509-1-git-send-email-Russ.Dill@ti.com
Signed-off-by: Russ Dill <Russ.Dill@ti.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
iQIcBAABAgAGBQJPafMcAAoJEIy3vGnGbaoALNUQAMZ8G+WDXD/RKtsO4V41rWOa
vCCUQ6BDTld53fLdTMwYBH9Uev9LLsQ/itOp8ne/apVn5DCghqm1rAfd5sIk9TaQ
zqnvF8QvZXnPCbUZe8mXH4ki2Shh+mvxkucAtsC9C4RBcu8QloIy7UGZPioEVxYj
TiadqKNicvSSIvEy3fM98qzbVu+GZZpZwfH0asiWV4rxLLovWNn2Ly7BGPLLHWa8
iamWVyG9QS51MzyEAxo4LBWGA4aK0k6+W5QxK2SmruA7g0YWzBf3DU5CfsXncj43
eOs/aVkBApPPIJ/VQDuhbdHAY+wIQAItojVAbSdebv0xC4SDtLbHHAgMVNtgLOIa
ysfLI+h8APbOh4nyqwNh5FuSnew4QvR1ZW47bZHmN7K7NjmcJ93geEvt//7eQnxz
X0iWJSCwAOyKORgsPU0lZgGM1lD50Qtmt1IZX5T87ksDSIhx5Lk/pAQo43nByj3g
e53swGxIQyaUqJwQ5ItJh7e8jQBZ6eyZDEWslro2S6uV9MFJnrH+cJJSk4LqO94y
dTu4uk/aCyXH3fbk1qfincfn5bzCvlrbSMsldbU6ODLFqyr8R1I8kcTZy/2k3i4q
eURDh94D6HEuu/QL1NvB5dKkuOfxmH3dr3S/5xv/IfIxruC+N+jGns0rmoFiE0KO
pnThls+OFAXhrw7YhV2d
=2NU4
-----END PGP SIGNATURE-----
Merge tag 'ktest-v3.4' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-ktest
Pull ktest changes from Steven Rostedt.
* tag 'ktest-v3.4' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-ktest:
ktest: Allow a test to override REBOOT_ON_SUCCESS
ktest: Fix SWITCH_TO_GOOD to also reboot the machine
ktest: Add SCP_TO_TARGET_INSTALL option
ktest: Add warning when bugs are ignored
ktest: Add INSTALL_MOD_STRIP=1 when installing modules
The option REBOOT_ON_SUCCESS is global, and will have the machine reboot
the the box if all tests are successful. But a test may not want the
machine to reboot, and perhaps have the kernel it loaded be used to
install the next kernel. Or the last test may set up a kernel that the
user may want to look at. In this case, the user could have the global
option REBOOT_ON_SUCCESS be true, but if a test is defined to run at the
end, that test can override the global option and keep the kernel it
installed for the user to log in with.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
When the option SWITCH_TO_GOOD is set, it will be called when the system
needs to reboot to the good server. But currently, this keeps the reboot
from happening. The SWITCH_TO_GOOD is just a way to get to a new kernel,
it may not mean to not reboot.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Currently the option used to scp both the modules to the target as well
as the kernel image are the same (SCP_TO_TARGET). But some embedded
boards may require them to be different. The modules may need to be put
directly on the board, but the kernel image may need to go to a
tftpserver.
Add the option SCP_TO_TARGET_INSTALL that will allow the user to change
the config so that they may have the modules and image got to different
machines.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
When IGNORE_ERRORS is set, ktest will not fail a test if a backtrace
is detected. But this can be an issue if the user added it in the
config but forgot to remove it. They may be left wondering why their
test did not fail, or even worse, why their bisect gave the wrong
commit.
Add a warning in the output if IGNORE_WARNINGS is set, and ktest detects
a kernel error.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Pull trivial tree from Jiri Kosina:
"It's indeed trivial -- mostly documentation updates and a bunch of
typo fixes from Masanari.
There are also several linux/version.h include removals from Jesper."
* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jikos/trivial: (101 commits)
kcore: fix spelling in read_kcore() comment
constify struct pci_dev * in obvious cases
Revert "char: Fix typo in viotape.c"
init: fix wording error in mm_init comment
usb: gadget: Kconfig: fix typo for 'different'
Revert "power, max8998: Include linux/module.h just once in drivers/power/max8998_charger.c"
writeback: fix fn name in writeback_inodes_sb_nr_if_idle() comment header
writeback: fix typo in the writeback_control comment
Documentation: Fix multiple typo in Documentation
tpm_tis: fix tis_lock with respect to RCU
Revert "media: Fix typo in mixer_drv.c and hdmi_drv.c"
Doc: Update numastat.txt
qla4xxx: Add missing spaces to error messages
compiler.h: Fix typo
security: struct security_operations kerneldoc fix
Documentation: broken URL in libata.tmpl
Documentation: broken URL in filesystems.tmpl
mtd: simplify return logic in do_map_probe()
mm: fix comment typo of truncate_inode_pages_range
power: bq27x00: Fix typos in comment
...
The make_min_config does not take into account when the build fails,
resulting in a invalid MIN_CONFIG .config file. When the build fails,
it is ignored and the boot test is executed, using the previous built
kernel. The configs that should be tested are not tested and they may
be added or removed depending on the result of the last kernel that
succeeded to be built.
If the build fails, mark the current config as a failure and the
configs that were disabled may still be needed.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
When testing a kernel that has warnings, ktest.pl will fail the test
when it sees the warning. If you need to test the the kernel and want
to ignore the errors that are produced, the option IGNORE_ERRORS has
been added. When IGNORE_ERRORS is set to something other than 0, it will
ignore call traces due to WARN_ON().
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
The REBOOT_TYPE may be either grub or script, if it is script
it is expected that a REBOOT_SCRIPT is defined.
With the SWITCH_TO_TEST which is the complement of SWITCH_TO_GOOD,
which does basically the same thing as REBOOT_SCRIPT and but for
both grub and script, the REBOOT_SCRIPT does not need to be mandatory
anymore.
Do not require the REBOOT_SCRIPT and always run the reboot code
for both grub and script.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
It becomes quite annoying when you go to run a test and then
realize that you typed an option name wrong, and the test starts
doing the default action and not what you expected it to do.
It is even more annoying when you wake up the next day after
running the test over night when you discover this.
By testing if all options specified in a config file are
used by either ktest or were used in one of the option's values
we can see if there are any dangling options that were not used.
In such a case, show the user the options that were not used
and ask them if they want to continue or not.
The option IGNORE_UNUSED was also added to allow the user to
override this feature.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Currently the patchcheck, bisect, and config_bisect variables
are only able to be set per test. You can not set a default
value for them.
By letting default values be set, it makes some config files
a bit easier, and also makes it easier to find typos in the
option names.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Initializing each default value by specifying the hash name is
ugly. This is one of the rare cases that the "perl way" is actually
much cleaner and easier to read.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
For machines that do no use grub, it may be needed to update an
external image (tftp) before doing a reboot into either the
test image or the known good image.
The option SWITCH_TO_GOOD is added, where if it is defined, the
command that is specified as its value will be executed before
doing a reboot into a known good image.
The option SWITCH_TO_TEST is added, where if it is defined, the
command that is specified as its value will be executed before
doing a reboot into the test image.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
When running the ktest git bisect test, if the BISECT_TYPE is "test",
the bisect is determined to be good or bad based off of the error
code of the test that is run. Currently, if the test returns 0,
it is considered a pass (good), a non-zero is considered a fail (bad).
But it has been requested to add more options, and also change
the meanings of the error codes of the test. For example, one may
want the test to detect if the commit is not good or bad,
(maybe the bisect came to a point where the code in question
does not exist). The test could report an error code that should tell
ktest to skip the commit.
Also, a test could detect that something is horribly wrong and the
biscet should just be aborted.
The new options:
BISECT_RET_GOOD
BISECT_RET_BAD
BISECT_RET_SKIP
BISECT_RET_ABORT
BISECT_RET_DEFAULT
have been added. The first 4 take an integer value that will
represent if the test should be considered a pass, fail, neither
good nor bad, or abort respectively.
The BISECT_RET_DEFAULT will bo whatever is not defined by the
above codes. If only BISECT_RET_DEFAULT is defined, then all tests
will do the default.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
All options can take variables "${var}". Before doing any processing
or decision making on the content of an option, evaluate it incase
there are variables that may change the outcome.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
The install process may also need to know what the kernel version
is, to add it to the name. Evaluate it for both install and
post install.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
If all the tests are only for build or install, do not ask
for options not needed to do the install, if the options do
not exist.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
When creating a new config, ask for the BUILD_OPTIONS variable
that lets users add things like -j20 to the make.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
When creating a ktest config or if te config only has build only
tests, some of the manditory config options are not needed.
Do not ask for them if all tests in the config file are just build
tests.
Suggested-by: Darren Hart <dvhart@linux.intel.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
When no argument is supplied to ktest, or the config applied does
not exist and a new config is being created, instead of just using
the default test type, give the user an option to pick the test type
of either 'build, install, or boot'. Other options may be added later
but then those would require more questions as they require more
fields. But that's for another release of ktest to add that feature.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
If a bisect is killed for some reason, have ktest detect that a bisect
is in progress and if so, allow the user to start the bisect where
it left off.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Typing in a full path when you know that the path exists within
the directory your are running is tedious and unnecessary.
Allow the user to use ${PWD} if they want a dynamic path name
which will be the path that ktest.pl is executed from
or use ${THIS_DIR} which is a variable assigned `pwd` and
the the variable will exist within the config, allowing the user
to change it and affect all other paths using this variable as well
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
When a user runs ktest without an argument, or the argument given
is not a config file that exists, ktest will ask the user a few
questions to create a simple ktest config file.
A few of the questions should have a default value set, that if anything
it will make it easier for the user to know what is suppose to
be in that value.
These new values are:
SSH_USER, BUILD_TARGET and TARGET_IMAGE
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Add a STORE_SUCCESSES option, to allow success logs to be stored, for
example to double-check or otherwise post-process the test logs.
Link: http://lkml.kernel.org/r/1321616131-21352-3-git-send-email-rabin@rab.in
Signed-off-by: Rabin Vincent <rabin@rab.in>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Steven Rostedt