Some packages in Fedora provide shared RPM lua code that's used by other
packages.
It would be nice to have automatic Provides for this code like we have
for rpm macros themselves.
Usecase:
Recently, forge.lua that's used by go-srpm-macros and fonts-srpm-macros
moved out of redhat-rpm-config.
I would like to be able to add `Requires: rpm_lua(fedora.srpm.forge)`
to those packages instead of having to rely on it being implicitly
pulled in by redhat-rpm-config or adding conditional dependencies on
forge-srpm-macros.
Multifile dependency generation is a huge performance boost when there
are multiple files of the same type. The Elf dependency generator is not
a particularly heavy in its startup so benefits less from this than
interpreted languages will. On my laptop coreutils requires generation
goes from circa 0.8s to 0.08s, something like Python/Perl interpreted
generators the difference would be much much bigger.
Of course, no reason not to improve performance when it comes this easy,
and serves as an example for other generators.
Encode the actual sysusers.d line as base64 in the EVR string to be
passed to systemd-sysusers (or a compatibility script) by rpm during
installation. The source string is \0 padded as needed to avoid base64's
'=' padding in the output as this is not a welcome character in EVR
strings.
For example:
u dnsmasq - "Dnsmasq DHCP and DNS server" /var/lib/dnsmasq
user(dnsmasq) = dSBkbnNtYXNxIC0gIkRuc21hc3EgREhDUCBhbmQgRE5TIHNlcnZlciIgL3Zhci9saWIvZG5zbWFzcQAA
group(dnsmasq)
The EVR string is omitted from the group as that gets created as a
side-effect of the user creation and does not require calling
systemd-sysusers.
These often come from a separate package that is installed early on
and form the basis of system users and groups, augmented by sysusers
from other packages. We don't want to create such users, but we need to
be aware of where they come from for dependency resolution and ordering.
The kernel modules are technically ELF DSO's but we only care about
library level dependencies which the kernel modules will never have, so
processing them is just waste of time.
This "regressed" when we dropped the "exeonly" flag from ELF in
commit 8901a6be16.
There are some missing bits and pieces still to be done for cmake build,
but that is so much easier if you don't have to worry about keeping
compatibility with the system you're about to remove that it doesn't
make sense to drag this on any further. The sooner this is over, the
sooner it is over and we can start making use of cmake's advantages
instead of just trying to bend over backwards to maintain compatibility
with the autotools build.
This is an incomplete release-early version, NOT intended or
suitable for production use. It is intended to replace the autotools
based buildsystem in rpm 4.20, until then it'll be developed alongside
it. This causes some extra complications of course, but then we avoid
a huge flag-day, and that matters more.
To those wondering why cmake and not ${myfavorite}: the community around
us effectively made that choice for us. We've made a lot of noise about
bootstrap dependencies. When libsolv, dnf and all the related stack is
powered by cmake build, it'd be just foolish to go with anything else.
This way people working on the rpm stack have only one build system to
learn, there's peer support available nearby and bootstrap dependencies
are reduced, not increased. It also doesn't hurt that cmake is actually
and actively maintained.
OCaml cmxs files are static libraries, which can be loaded at runtime
via the Dynlink module. They apparently do not provide any useful
runtime dependency information to be stored into the packages
Provides/Requires list. Therefore just skip them.
Adjust attr, remove extension and ELF magic
Adjust ocamldeps, do nothing with cmxs files.
Fixes: a6fe37c39b
Signed-off-by: Olaf Hering <olaf@aepfle.de>
People mostly only care about removing any .la files present in their
packages, I've yet to see a single use of a desired/used libtool()
dependency during its 14 year existence in rpm. There are enough
provides in the world without creating unused ones just because.
ELF ET_DYN files can be shared libraries, executable files or both.
Requiring shared libraries that cannot actually be executed (or segfault
when you do so) is dumb, but rpm has traditionally required executable
bit to be set for requires to be generated.
Refine the definition of "executable" to mean files that actually *can*
be executed, ie those with PT_INTERP and whose actual behavior depends
on whether the executable bit is set or not.
In other words, pure shared libraries will have requires generated
regardless of their permission, but executable files can still disable
that by disabling the on-disk executable bit because unlike shared
libraries, they only require anything when being executed.
Also add rudimentary testcases for the scenarios involving a shared
library, traditional binary and a PIE binary. The test material needs
to be pre-built as we cannot predict what dependencies building on an
arbitrary compiler on arbitrary platform may have, but source is
included for manually regenerating as needed.
This was kinda ugly-but-necessary when added back in 2003 (commit
752cac72e2) but entirely redundant
since the "new" dependency generator in rpm >= 4.9.x with arbitrary
filtering support. The handful of packages using it can just as well
achieve the same (and more) without special hacks in rpm with:
%global __requires_exclude GLIBC_PRIVATE
%global __provides_exclude GLIBC_PRIVATE
The %__python_magic filter suddenly got actually working with file 5.39:
Before:
file.cpython-38.opt-1.pyc: data
After:
file.cpython-38.opt-1.pyc: python 3.8 byte-compiled
Hence, the filter started to pick all Python files regardless of their location.
Later, in the actual generator, paths like this were considered:
/opt/usr/lib/python3.X/...
And generated requirements on python(abi).
We don't actually need to filter the files by file magic,
so we drop it to get the previously accidentally working behavior.
We could choose if the path and magic filters are applied as OR or AND.
However, we don't want either.
We actually want to mach any files in Python directories regardless of their magic.
We *could* filter by file type (and executable bit) for provides,
but that would require us to split the attr files into two.
pythondeps.sh was written in shell and unlike the Python dist generators,
it uses no Python, it plainly determines the provide / requires from the path.
As the script was run for every Python file, we were potentially doing hundreds
of shelling outs to execute a script that calls grep and sed.
In Lua, this should be more efficient.
Fixes https://github.com/rpm-software-management/rpm/issues/1152
Dependencies between OCaml files containing native code are also tracked
by hashes, just like their bytecode counter parts.
OCaml native code files end with cmx, the relevent info is in cmx, cmxa
and cmxs files. Unlike all other OCaml files, cmxs files have ELF format.
OCaml has two variants of dependency tracking:
Interfaces, which are already tracked via "ocaml(MODULE) = HASH"
They are included in cmi, cma, cmo, cmx, cmxa and cmxs files.
Implementations, which are now tracked via "ocamlx(MODULE) = HASH"
They are included in cmx, cmxa and cmxs files.
Just like Interfaces can be excluded from the dependency list with the
option -i MODULE, Implementations are excluded with the option -x MODULE.
Previously the final rpm package also got an extra dependency to the
used ocaml runtime version. This is not strictly required because the
hashes are unique. Therefore this dependency is no longer created. The
option -c, which excluded this dependency, is still recognized.
Fixes#913
Signed-off-by: Olaf Hering <olaf@aepfle.de>
The current regular expression for matching on Python metadata files was
too greedy, leading to generating dependency information for Python modules
that were actually bundled in another module rather than actually being
installed in a way that is usable and accessible from any Python code
using the system interpreter.
This change tightens the regular expression so that we only match on
modules that are installed in a way that they'll register with
distutils/setuptools based tools (such as pip) and usable as
dependencies for other Python module packages.
Rather than having the pythondistdeps dependency generator run on
all Python code, this separate attr will activate it only if the
package includes Python dist metadata (such as egg-info or dist-info
for Python Eggs and Python Wheels, respectively).
This is much more efficient, too, since the generator is only run
in circumstances where it will provide useful information.
This is based on the dependency generator attr used in Mageia,
OpenMandriva, and other systems that use the generator.
Trying to include AM_CFLAGS through a configure generated rpm.am file
doesn't really work because at the time automake runs configure doesn't
exist yet to process rpm.am.in. Just define the AM_CFLAGS substitution
inside the Makefile.am files themselves.
Rename rpm.am.in back to rpm.am.
Signed-off-by: Mark Wielaard <mjw@redhat.com>
These provides are specifically for packages providing
AppStream files, which are either going to be *.appdata.xml
or *.metainfo.xml files in /usr/share/appdata or /usr/share/metainfo.
The upstream AppStream specification mandates *.metainfo.xml files
installed into /usr/share/metainfo, but there's still a large body
of legacy AppStream files installed in the legacy location.
For now, let's support both under the new metainfo() Provides.
This patch lets debuginfo packages provide build-id like follows:
debuginfo(build-id) = c63cb23876c5fa85f36beaff58f8557e1bf22517
Originally this patch was written by Jan Blunck <jblunck@suse.de>.
Signed-off-by: Igor Gnatenko <ignatenko@redhat.com>
Appdata files contain application information used by the AppStream
project. We generate two provides: appdata() to support enumeration
of all application packages and appdata(filenname) to make it easy
to link installed packages with appdata files.
Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
- Newer file is more eager to flag files as Perl module source,
causing false provides to be generated. Require both magic
and path to match for perllib classification to reduce the pain.
- libmagic sometimes adds trailing descriptions about encodings, line
terminators and whatnot, we dont want to care about those (RhBug:796218)
- not all python-related strings start with [pP]ython either, sometimes
libmagic says "a python script" or "a /usr/bin/python script" and
whatnot, so loose the start-of-line restriction as well
- file 5.10 has changed magics at least for perl and python scripts, samples:
5.09: a /usr/bin/perl -w script, ASCII text executable, with very long lines
5.10: Perl script, ASCII text executable, with very long lines
5.09: a /usr/bin/python script, ASCII text executable
5.10: Python script, ASCII text executable
Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
- Somewhere between file 5.05 and 5.07 it started adding encoding
to script descriptions, eg "<mumble> script text executable" became
"<mumble> script, <encoding> text executable" breaking what had
been working for 10+ years in the case of old find-requires.
- Permit either comma or space after "script", this works for both
old and new file.
Panu Matilainen