Commit Graph

16450 Commits

Author SHA1 Message Date
Panu Matilainen 9c96c5d4ca Axe the experimental non-privileged chroot support from b4c832ca
While the chroot call itself works nicely for a regular user with this,
user/group mapping does not and cannot be made to work, because that
requires a fork and we cannot very well fork the process from deep down
here.

Better to put the irrepairable thing out of its misery, users are far
better of wrapping rpm with the unshare command instead.
2023-09-15 16:23:46 +03:00
Maxwell G 6867ef9f8b Add a Provides generator for rpm lua modules
Some packages in Fedora provide shared RPM lua code that's used by other
packages.
It would be nice to have automatic Provides for this code like we have
for rpm macros themselves.

Usecase:

Recently, forge.lua that's used by go-srpm-macros and fonts-srpm-macros
moved out of redhat-rpm-config.
I would like to be able to add `Requires: rpm_lua(fedora.srpm.forge)`
to those packages instead of having to rely on it being implicitly
pulled in by redhat-rpm-config or adding conditional dependencies on
forge-srpm-macros.
2023-09-15 12:32:52 +03:00
Panu Matilainen 1120c8cf7c Add a testcase for librpm development
Should've been in 6ec0e4069e but at that
time cmake in Fedora had a dependency on rpm making this hard or
at least ugly. With that dependency fixed, we can now run cmake in there.
2023-09-13 16:19:02 +03:00
Panu Matilainen 959b4e4750 Make the plugin API public
We've procrastinated on making this API public for about ten years now,
and in the meanwhile there has been exactly one disruptive change to
the API. As in, it might've just as well been public all along.

There will always be more things to improve wrt any API, but we're not
going to hold this hostage to one more thing or another anymore. Some
of them we'd like to do before this goes to a stable release (ie 4.20)
but doing this now to kinda enforce this actually happens this time
around, through come hell or high water.

Fixes: #1536
2023-09-13 14:51:25 +03:00
Panu Matilainen 1c98b67911 Drop top-level source for our global include directories
Having everything accessible to everything encourages fast and loose
includes from places one shouldn't be using, and makes it easy for
those cases to hide in plain sight as well. There were reasons for
the top-level include back in 2007 but our codebase is a rather
different beast these days. Limiting access through per-target
include directories on everything nicely highlights the exceptions
and makes the whole more controllable and manageable.

This change looks huge, but it's just due to stripping no longer valid
prefixes from all the gazillion internal includes. No rpm-side
functionality is affected, this is just source-level hygiene operation.
2023-09-13 13:34:17 +03:00
Panu Matilainen 86d99a1d20 Move all cli utility sources under tools/
Having stuff in the top-level directory is messy, and the top-level
cmake file is plenty big enough without all this. No code changes,
no functional changes.
2023-09-13 13:34:17 +03:00
Panu Matilainen 13eb82e9bf Move rpmhash.[CH] into misc/ for clarity
These are nothing like normal sources and are not compiled into librpm,
but having them in lib/ makes it appear they are. Which also confuses
the issue of where they are usable which is everywhere within the source
tree, not just places linking to librpm. No functional changes.
2023-09-13 13:34:17 +03:00
Panu Matilainen d4eb1f2fe6 Move top-level headers to misc/ and include that everywhere
Having stuff in the top-level directory is messy and prevents control
over what you can include from where. No functional changes.
2023-09-13 13:34:17 +03:00
Panu Matilainen 74783883cb Split our include_directories() per use-case, comment
No functional changes.
2023-09-13 13:34:17 +03:00
Panu Matilainen 34d983fa2a Fix compile regression from 071be753cf
A declaration immediately following a label is not valid C99, as crazy
as it is in a language that otherwise allows free placement of declarations.
It looks like something people forgot to update in the standard, first
in C99 and apparently C11 too. Recent gcc doesn't complain about it
but it doesn't make it right.

Suggested-by: Dmitry Mikushin <dmitry@kernelgen.org>
2023-09-13 11:04:33 +03:00
Panu Matilainen 08ae4ee23e Add multifile support to elf dependency generation
Multifile dependency generation is a huge performance boost when there
are multiple files of the same type. The Elf dependency generator is not
a particularly heavy in its startup so benefits less from this than
interpreted languages will. On my laptop coreutils requires generation
goes from circa 0.8s to 0.08s, something like Python/Perl interpreted
generators the difference would be much much bigger.

Of course, no reason not to improve performance when it comes this easy,
and serves as an example for other generators.
2023-09-05 14:31:17 +03:00
Panu Matilainen d3dbe17b2e Support optional multifile protocol in dependency generation
In this mode, extract all dependencies of a given type from all files
in a package in one go. Requires explicit support + enablement for each
generator separately:

Generators receive input as usual, but in this mode there are arbitrary
number of files. When outputting dependencies, the generator merely
needs to output `;<path>` to stdout before outputing dependencies of
that file, files with no dependencies may be omitted.
2023-09-05 14:31:17 +03:00
Panu Matilainen ae71dda9ad Low-level support for passing multiple files to dependency generators
Further preliminaries for supporting dependency generation for multiple
files in one go. Supposedly no functional changes here.
2023-09-05 14:31:17 +03:00
Panu Matilainen ed5571ac0b Refactor dependency generation out of rpmfcHelper()
Supposedly no functional changes, just preliminaries for the next steps.
2023-09-05 14:31:17 +03:00
Panu Matilainen 285d2d337c Split weeding out excludes to a separate loop
Preparations for the next steps: we need to have ready filelists to pass to
the actual generators to be able to do many at once.
2023-09-05 14:31:17 +03:00
Panu Matilainen f24c242efa Pass the entire attribute to applyAttr() for more flexibility
Will be needed in the next commits.
2023-09-05 14:31:17 +03:00
Panu Matilainen 35a460e509 Refactor file walk loop into rpmfcHelper()
Preliminaries for letting it handle multiple files at once
2023-09-05 14:31:17 +03:00
Panu Matilainen 2085295187 Drop no longer needed pull from updates-testing for rpm-sequoia
rpm-sequoia >= 1.4.0 has been in stable updates for a good while now.
2023-09-05 09:37:15 +03:00
Michal Domonkos 4a12e3fc21 Update translation submodule for new translations 2023-09-04 10:17:23 +02:00
Panu Matilainen 7d017eef51 Always run tests without --nouser and --nogroup, now that we can
With the new container based test-suite we no longer need these hacks.
Leaving the mknod conditional in place though as that may still be
restricted.
2023-09-01 15:41:02 +03:00
Panu Matilainen 4712c9c37b Run the test-suite with plugins enabled now that we can, yay 2023-09-01 11:21:44 +03:00
Panu Matilainen 5f23d76eb9 Emit a debug message instead of silence when DBus unavailable
Make systemd_inhibit plugin behave identically to dbus_announce in the
previous commit: when DBus service isn't available, emit a debug level
message rather than completely suppressing the message to keep things
troubleshootable.
2023-09-01 11:21:44 +03:00
Panu Matilainen 071be753cf Suppress inhibition lock warning message when DBus service is not available
Same as commit 708e61307b, this message
will just annoy and confuse users in situations where DBus service is
not running at all, such as single-user mode and minimal container type
environments. Like our own test-suite. Rather than entirely suppress
the error, issue a debug log though.
2023-09-01 11:21:44 +03:00
Panu Matilainen 6ec0e4069e Export our libraries as a cmake find_package() config
This is a fair amount of annoying voodoo boilerplate compared to what I
would've expected, but the ability to just `find_package(rpm)` is
very nice on the (API) user side.

Fixes: #2471
2023-09-01 10:43:33 +03:00
Panu Matilainen ffd55686ab Clean up hashtype definitions
Always #undef the "parameters" right after definition to make sure
they're always free for the next user. Which avoids annoying
double-undefs all over the place.
2023-08-31 09:58:18 +03:00
Panu Matilainen 5f98b1aa48 Document the main differences between rpm-sequoia and internal parser
Fixes: #2346
2023-08-29 13:19:15 +03:00
Panu Matilainen 3687458aa0 Add a build tunables section to the reference manual
The descriptions are terse and I've inevitably missed a whole bunch
of stuff here, but it's a start.

Fixes: #2466
2023-08-29 13:01:36 +03:00
Panu Matilainen 33c0478f96 Drop an extraneous Dependencies section from the spec manual 2023-08-29 13:01:36 +03:00
Panu Matilainen c10877b6bc Polish the build process docs a bit
Replace source links to documentation (we have it now!) and various
more cosmetic tweaks.
2023-08-29 13:01:36 +03:00
Panu Matilainen 21457de886 Fix misleading error message on some invalid filetrigger conditions (#2584)
We used to test the first argument in parseScript() and then again
for all arguments in parseRCPOT(), with different error messages in
each case as if they were separate issues. Only, the one in parseRCPOT()
didn't get a chance to execute because it was caught by checkDep() before
it (probably added after the file trigger check was initially added) and
so for invalid file trigger conditions (such as an unexpanded macro), you'd
get a very misleading error about dependency tokens where an absolute path
is expected, and worse, it was reporting an incorrect line and a line
number while at it.

Measure twice, cut once doesn't apply here. Check once and be consistent
about reporting it. Due to the way triggers are handled, we can't
report the line number correctly so don't even try, the wrong line
number is far worse than not having it at all. Add some tests to go.

Fixes: #2584
2023-08-23 15:01:53 +03:00
Panu Matilainen 3be76d7144 Default to C.UTF-8 locale, allow fallback to C
Mostly everything around us is UTF-8 these days, we need to get on with
the times. Especially now that glibc >= 2.35 finally supports it too.
Default to C.UTF-8 but allow disabling it via cmake option to fallback
to the traditional C locale instead.

Fixes: #2587
2023-08-23 15:01:28 +03:00
Panu Matilainen 59bbb932e5 Behave more consistently when target arch %optflags are not defined
You're about to fall into a deep dark hole, proceed at your own risk.

When building for a target architecture with no defined %optflags
(such as noarch), one would think that %optflags would be empty. Not so in
rpm, instead we get %optflags for the detected architecture, and there
are packages which rely on this behavior. And in this particular dark
corner, buildarchtranslate is not applied so one can get drastically
different %optflags than you'd get without an explicit --target, on the
same system. Which can break builds.

This behavior is just WRONG, %optflags should always reflect the target
architecture. But as long as there's no %_host_optflags, lets at least try
to be consistent about it. When we fall back to detected architecture
%optflags, at least use the ones after buildarchtranslate to return
consistent data within a host.

This supposedly fixes the case where our newly added x86_64
subarchitecture definitions haven't been overridden in vendor config and
somebody's noarch package uses cmake to install data, and falls over due
to nonsensical optflags from rpm.

Initial report: https://bugzilla.redhat.com/show_bug.cgi?id=2231727
2023-08-23 11:55:31 +03:00
Panu Matilainen 0e336e24c9 Move the %_sysusersdir definition to a more logical spot
Commit c7fd9fed99 introduced %_sysusersdir
into the group of global configure-related macros but that's not what
it's related to at all, it's only related to our sysusers integration
feature. Move it where it really belongs.
2023-08-22 14:15:29 +03:00
Panu Matilainen 24f974cff9 Revert recent %_root_prefix macro addition (RhBug:2233454)
Commit cececfb685 introduced %_root_prefix
macro but this clashes with pre-existing use in scl-utils:
https://bugzilla.redhat.com/show_bug.cgi?id=223345:

Just query the value from pkg-config if available and otherwise use
hardcoded value pointing to /usr instead. We don't need an intermediate
global macro for this.
2023-08-22 14:15:29 +03:00
Daan De Meyer aa0d6d8109 Add aliases for weak dependency queries to rpmspec.
These are all available on the rpm CLI so let's make them available
on the rpmspec CLI as well.
2023-08-21 12:10:13 +03:00
Michal Domonkos 5e9c71296e Avoid warning about double --chdir in "make check"
Newer versions of Bubblewrap emit a warning for some select options
(--chdir included) about only the last occurrence having effect, which
is what happens in "make check" due to the --chdir override.

Instead, just cd to the /srv directory in a shell and then run rpmtests
from there, sigh.  It's not the prettiest but it's better than always
printing the spurious warning.
2023-08-17 18:13:15 +02:00
Michal Domonkos 94ee5c6c23 Prevent build dir from becoming annoyingly sticky
With OverlayFS and unshare(1) being used in the test-suite, some of the
directories in the test tree, sandbox and rpmtests.dir/ end up missing
read/write or execute (for directories) permissions which means removing
them requires excessive force in the form of "unshare -r rm -rf".

CMake is clever enough to adjust the permissions behind the scenes on
"make clean" but manual removal should just work too, so fix that with
chmod(1) in the clean_up() trap.
2023-08-17 18:13:15 +02:00
Michal Domonkos 4fa0026430 Move exit trap to a function in mktree.fedora
This is more flexible as we can now add more commands to the trap, see
the next commit.

No functional changes.
2023-08-17 18:13:15 +02:00
Michal Domonkos 87f1b4545b Enable DNS resolution in "make shell"
Networking is currently enabled in test containers (we don't use
--unshare-net) but for domain names to be resolvable, we need an
/etc/resolv.conf, too, so add it from the host, similar to how e.g.
Podman does it.

We don't need network access in the test-suite but it can be handy for
interactive testing in "make shell", e.g. to allow gdb to fetch
debuginfo data through debuginfod automatically.
2023-08-17 18:13:15 +02:00
Michal Domonkos 3701e4893e Export the same DNF command-line in "make env"
When managing the test tree with DNF, it's actually desirable to use the
same options that were used during the tree construction (especially the
cachedir), so just do that.
2023-08-17 18:13:15 +02:00
Michal Domonkos c3ecf8ac44 Extend the MOTD in "make env" with a bit more info 2023-08-17 18:13:15 +02:00
Michal Domonkos 8e9ff82f55 Terminate stray bwrap processes on test failures 2023-08-17 18:13:15 +02:00
Michal Domonkos db35781bf1 Point RPM to correct database in Dockerfile
This just mirrors what mktree.fedora does already, and is only relevant
in "make shell" when using the mktree.podman backend, no other
functional change here.
2023-08-17 18:13:15 +02:00
Michal Domonkos 2585481662 Set standard, consistent HOME path in tests
Use /root instead of / (in runroot) or $PWD (in atlocal).  We already do
this in "make shell" so that the root user's dotfiles (e.g. ~/.bashrc)
are loaded from the base image, this commit extends that to "make env"
and fixes the following:

  - Calling "runroot rpm -qa" interactively now works as expected since
    the rpmdb from the base image built by mktree.fedora (which
    overrides %_dbpath in /root/.rpmmacros) is used

  - The user's native prompt is now used, instead of the shell's default

This commit has no effect on the tests themselves since we initialize an
empty rpmdb before use anyway.

There was no technical reason for HOME being set to /, really, other
than it being a historical artifact from the fakechroot era.
2023-08-17 18:13:15 +02:00
Michal Domonkos 1c3264ed88 Fix infinite recursion in "make env"
Using which(1) in a shell function that's supposed to shadow the
original command is silly, it just results in a recursive call.  What
was I thinking?

Instead, use "command" which is a better fit, see "help command".
2023-08-16 13:31:01 +02:00
Michal Domonkos 44c76481e7 Clean up dnf() function in "make env"
The "function" keyword is not needed (not POSIX either) and which(1) is
better than hardcoding the path.

No functional change.
2023-08-16 10:27:29 +03:00
Michal Domonkos ebd0905a3f Simplify BYPRODUCTS in "make tree"
The inner structure of mktree.output/ should be treated as an
implementation detail of the mktree backend in use.  All we care for
here is that it's purged on a "make clean".

No functional change since only mktree.fedora uses this directory
currently and it already recreates the other contents (inst/) on each
rebuild anyway.
2023-08-16 10:27:29 +03:00
Michal Domonkos e1ff74c4e5 Support common mktree.fedora image in build dirs
Using multiple out-of-source build directories (e.g. for topic branches
or experiments) currently means redownloading and recreating the base OS
image in each of them.  This is slow, ineffective and just not needed
since the image isn't specific to the build and can just be reused.

To fix it, if a mktree.output/ directory exists in the source directory,
use that, otherwise just use the build directory like we always did so
that all build artifacts remain in the build directory unless told
otherwise (which some may prefer).

This means the feature can be enabled by simply creating an empty
mktree.output/ directory in the source directory.  This is inspired by
mkosi(1), much like many other things in mktree.

We shall document this later, too.

If the source directory happens to be a git-worktree(1), go look into
the common worktree for mktree.output/.  This ensures that worktrees, if
one uses them, are not treated as separate source directories and will
reuse the same image, too.

If using a common mktree.output/, also use a common mktree.cache/ (DNF
metadata and package cache) as separating those two makes little sense
in such a case.
2023-08-16 10:27:29 +03:00
Panu Matilainen 095502dc09 Stop looking for apparently non-existent 7zip command
Since the initial commit 185596818f, we've
primarily looked for a command named "7zip" but defaulted to 7za when
not found. Looking closer it seems that there never was any command
called 7zip at all, at least in the OSS landscape. So don't default to
something that doesn't even exist, which also means we'll land on an
actually working value if/when 7z[a] doesn't happen to be present at
build-time (there's no other reason for it to be there).

Related to #2608
2023-08-16 10:18:20 +03:00
Panu Matilainen f9775b454e Look for alternative implementations of 7zip, like autoconf did
Fixes a regression from the cmake transition where we'd just assume 7zip
as the name of the executable when previously we looked through 7zip,
7za and 7a. Resume the former behavior.

Fixes: #2608
2023-08-15 14:38:21 +03:00