725ca51695
For bootstrapping purposes, having rpm depend on Rust is painful, but directing people to unmaintained crypto code as an alternative is hair-raising. As a middle ground, let rpm be built without OpenPGP support at all, which at least gives you a functional rpm and rpm-build even if you can't sign or verify signatures. Achieving this is a moderately complex dance which can't meaningfully be split into multiple commits because everything is interconnected: Add a new WITH_SEQUOIA option to control use of Sequoia, on by default. When Sequoia is disabled, default to a newly added dummy PGP implementation instead which just returns error on everything. And finally, if the older WITH_INTERNAL_OPENPGP is enabled, use the old PGP implementation. As the intent is to cut out rpmpgp_legacy to a separate repository, sanity requires that we also split the openssl/libgcrypt code at the digest/signature fault line. It's not ideal, but the alternative of having unused crypto code on which an external component depends on is just not sustainable. This way, the signature side of things is quite neatly cut off with the PGP stuff. The diff looks big but there are no code/functional changes in the libgcrypt/openssl split. |
||
|---|---|---|
| .. | ||
| data | ||
| pinned | ||
| CMakeLists.txt | ||
| Dockerfile | ||
| Dockerfile.fedora | ||
| README.md | ||
| atlocal.in | ||
| local.at | ||
| mktree.common | ||
| mktree.oci | ||
| mktree.rootfs | ||
| package.m4.in | ||
| rpmbuild.at | ||
| rpmbuildid.at | ||
| rpmconfig.at | ||
| rpmconfig2.at | ||
| rpmconfig3.at | ||
| rpmconflict.at | ||
| rpmdb.at | ||
| rpmdepmatch.at | ||
| rpmdeps.at | ||
| rpmdevel.at | ||
| rpme.at | ||
| rpmgeneral.at | ||
| rpmi.at | ||
| rpmio.at | ||
| rpmmacro.at | ||
| rpmorder.at | ||
| rpmpgp.at | ||
| rpmpgpcheck.c | ||
| rpmpgppubkeyfingerprint.c | ||
| rpmpkgfmt.at | ||
| rpmpython.at | ||
| rpmquery.at | ||
| rpmreplace.at | ||
| rpmscript.at | ||
| rpmsigdig.at | ||
| rpmspec.at | ||
| rpmtests.sh | ||
| rpmvercmp.at | ||
| rpmverify.at | ||
| rpmvfylevel.at | ||
README.md
Tests
Introduction
This test-suite builds an OCI image matching the host OS with a fresh RPM
installation layered on top (with make install), ready to be tested in a
container, either automatically by a script (via make check) or in an
interactive shell (via make shell).
Prerequisites
To use this test-suite, you need either of:
[!IMPORTANT] CMake integration (native mode) is currently only available on Fedora Linux hosts with Podman installed, on other systems a fresh build of RPM will be done as part of a Fedora-based image (non-native mode).
Running all tests
To build the image (if not built yet) and perform automated testing, run:
make check
The number of tests performed depends on features enabled at configure time, at
least the CMake option ENABLE_PYTHON=<ON|OFF>. See also the
INSTALL file for more information.
If the test-suite is configured in native mode, it may sometimes be useful to verify that everything passes in non-native mode too as that's what our CI uses. To do that, run:
make ci
Selective testing
To run single tests, you can run the commands:
make check TESTOPTS="$NNN $MMM"
where NNN and MMM are the numbers of the tests to be run.
You can also select tests by keywords used in their description by using the command:
make check TESTOPTS="-k $KEYWORD"
Use multiple -k parameters to have tests with different keywords run. Use
-k $KEYWORD1,$KEYWORD2 to execute tests matching both KEYWORD1 and
KEYWORD2.
For all available options, see the output of the command:
./rpmtests --help
By default, tests are executed in parallel using all available cores, pass a
specific -jN value to limit.
Interactive testing
To drop into an interactive GNU Autotest shell, run:
make atshell
This is like a singular, empty RPMTEST_CHECK with a shell running in it and a
writable tree available at the path stored in $RPMTEST. From this shell, you
can run the same commands as a normal test would, such as runroot rpm. This
can be used to quickly prototype (or debug) a test.
You can also drop straight into the $RPMTEST container like so:
make shell
This is just a shorthand for make atshell followed by runroot_other bash.
To factory-reset the $RPMTEST tree, run:
make reset
To only build the OCI image, use:
make tree
You can also just use Podman or Docker directly to manage containers (the image
is named rpm when built). For example, to run a shell:
podman run -it rpm
Understanding the tests
Optimizations
Since the test-suite consists of several hundreds of tests and is meant to be run repeatedly during development, it's optimized for speed.
Instead of running each test in a separate OCI container, all the tests run in a single, immutable OCI container. Mutable, disposable snapshots of its root filesystem are then created on demand for tests that need write access to it (such as those installing or removing packages). These tests perform the desired operation through a lightweight, nested container with the snapshot set as its root directory.
This reduces the number of containers needed while still ensuring that each test operates in a pristine environment, without affecting any other test or the test logic itself. Snapshots are created with OverlayFS and nested containers with Bubblewrap which heavily reduces the startup time of each test when compared to a full-blown OCI runtime.
Since this is done from the immutable OCI container, that container needs to
run as --privileged in order to allow the creation of nested namespaces.
Layout
The OCI image is built from a Dockerfile.<osname> where <osname> is the
host OS name (e.g. fedora). Currently, we only have Dockerfile.fedora,
contributions for other distros are welcome. The Dockerfile symlink points
to the file that should be used in non-native mode (see above).
The test script that's executed in the OCI container is written in GNU
Autotest
and is compiled from the rpmtests.at file. The individual tests are split
into several files with the .at suffix, each covering a specific area within
RPM, such as package building, transactions or configuration, which are then
included by the top-level rpmtests.at script.
Writing tests
For the typical structure of a single test, consult GNU Autotest's documentation as well as the existing tests. Below are the specifics of RPM's test-suite:
- Use
RPMTEST_CHECKinstead ofAT_CHECK - Use
RPMTEST_CLEANUPinstead ofAT_CLEANUP - Use
RPMTEST_SETUPorRPMDB_INITto create a mutable snapshot (optional)- The absolute path to the snapshot's root is stored in the
$RPMTESTenvironment variable, modify the directory tree as you wish - To run RPM inside the snapshot, use the
runrootprefix - To run any other binary inside the snapshot, use
runroot_otherinstead
- The absolute path to the snapshot's root is stored in the
- If no snapshot was used, just call the RPM binaries normally
- Store any working files in the current directory (it's always writable)