Avoids having to run find-lang.sh for each name and append the file lists
produced later. As this makes use of bash specific array functionality,
change the interpreter to bash explicitly.
Per-file changelogs do not belong to this world era. Merge the
authors into CREDITS (proyvind is already there) and remove the
hysterical embedded changelog.
Avoids having current directory in Python's search path and thus
arbitrary code execution when the macros are expanded.
Signed-off-by: Ville Skyttä <ville.skytta@iki.fi>
Thanks-to: Paul Wise <pabs@debian.org>
Thanks-to: Jakub Wilk <jwilk@jwilk.net>
Directory name doesn't matter, content does. In specification, both
appdata.xml and metainfo.xml should be in /usr/share/metainfo, but
tools should support /usr/share/appdata as for legacy location.
References: https://bugzilla.redhat.com/show_bug.cgi?id=1483644
Reported-by: Petr Pisar <ppisar@redhat.com>
Signed-off-by: Igor Gnatenko <i.gnatenko.brain@gmail.com>
The %_debugsource_template expects the debugsourcefiles.list file
to be in the (current) build dir. Make sure that is always the case
even if find-debuginfo.sh was invoked in a different (sub) directory
by prepending $BUILDDIR to the output file as written in description
"All file names in switches are relative to builddir (. if not given).".
Closes: https://github.com/rpm-software-management/rpm/issues/279
Based-on-patch-by: Mark Wielaard <mark@klomp.org>
Signed-off-by: Igor Gnatenko <i.gnatenko.brain@gmail.com>
In the minisymtab section (the .gnu_debugdata embedded ELF image) we
do not need unallocated sections (except for the SYMTAB and STRTAB
sections we are creating). We already remove PROGBITS and NOTES. Also
remove NOBITS sections. They should not really take up much (any) space
but they still add to the section tables. These sections might be created
with the new --keep-section support (which puts the actual section in
the main ELF binary, and a NOBITS variant in the .debug file).
Also binutils objcopy seems to sometimes add them anyway filled with
zeros instead of marking them NOBITS.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Use --keep-section SECTION or --remove-section SECTION to explicitly
keep a (non-allocated) section in the main executable or explicitly
remove it into the .debug file. SECTION is an extended wildcard pattern.
Both options can be given more than once.
https://bugzilla.redhat.com/show_bug.cgi?id=1465997
Signed-off-by: Mark Wielaard <mark@klomp.org>
commit 038bfe "Split directory traversal and debuginfo extraction"
put the core of a while loop into its own function 'do_file()'.
That means that instead of using 'continue' to break out early it now
needs to use 'return'. Otherwise the script will give errors like:
continue: only meaningful in a `for', `while', or `until' loop
https://bugzilla.redhat.com/show_bug.cgi?id=1465170
Signed-off-by: Mark Wielaard <mark@klomp.org>
There is no official way to mark an instruction range as being not
part of some actual source code, but as part of a compiler built-in
construct in DWARF. So different compilers have come up with fake
source file names like <built-in> or <__thread_local_inner macros>.
We already filtered out the strings "<internal>" and "<built-in>".
Just filter out all '(^|/)<[a-z _-]+>$'. They are fake files!
This is mainly to appease the rustc compiler which generates lots of
different variants to encode some instruction sequence is part of an
compiler generated macro expansion.
Signed-off-by: Mark Wielaard <mark@klomp.org>
This change allows the perl.prov script to detect new variants of Perl
package declarations that have been added in recent years.
In Perl 5.12 we got version numbers as part of the package declaration.
package Foo 1.0;
package Bar v1.0.0;
And in Perl 5.14 we got the package block syntax.
package Foo {...}
package Bar 1.0 {...}
package Baz v1.0.0 {...}
We noticed this problem recently because Dist::Zilla, a popular Perl
module, started using a version number as part of the package
declaration and perl(Dist::Zilla) could no longer be resolved.
dwz -m multi only works when there are multiple .debug input files.
With just one .debug file it doesn't really make sense to extract
the shared debug info into a separate file and dwz will complain:
dwz: Too few files for multifile optimization.
So only add -m multi if there is more than one .debug file.
Signed-off-by: Mark Wielaard <mark@klomp.org>
This adds some missing documentation for rpm macros and find-debuginfo.sh
options that were recently added (or renamed). -j N, --build-id-seed SEED,
--unique-debug-suffix SUFFIX and --unique-debug-src-base BASE.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Rename --ver-rel option to --build-id-seed, so that it reflects what
it does, not how it is used.
Remove implcit usage of the old --ver-rel option for --unique-debug-arch
and --unique-debug-src-base, instead already call find-debuginfo.sh with
the version included.
Rename --unique-debug-arch to --unique-debug-suffix because it now
also contains the package version.
It only makes sense to add a minisymtab for executables and shared
libraries. Other executable ELF files (like kernel modules) don't need it.
Since those don't have a dynsym section trying to add it will fail and
produce confusing errors from nm.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Some packages depend on the build-ids as generated during the build
and cannot handle rpmbuild recomputing them before generating the
package file list. Add -n, --no-recompute-build-id to debugedit and
add -n to find-debuginfo.sh set by defining the %_no_recompute_build_ids
macro for such packages. %_no_recompute_build_ids can not be used together
with %_unique_build_ids.
Signed-off-by: Mark Wielaard <mark@klomp.org>
pkgconf (alternative to freedesktop's pkgconfig implementation)
uses this flag to stop resolving dependencies after some level.
In our case, we are not interested in checking dependencies from
buildroot at all, we just generating top-level dependency list.
References: https://bugzilla.redhat.com/show_bug.cgi?id=1401463
Reported-by: Martin Sehnoutka <msehnout@redhat.com>
Signed-off-by: Igor Gnatenko <ignatenko@redhat.com>
In case that binary compiled from source generated in /tmp, a
/usr/src/debug/tmp directory will be created with the same mode as
/tmp, a.k.a 777, which should be avoided.
Fixes: rhbz#641022
Introduce a new macro _unique_debug_srcs that when set will pass
--unique-debug-src-base "%{name}" to find-debuginfo.sh which will
move sources into a unique "<name>-<ver>-<rel>.<arch>" directory
under /usr/src/debug/ and makes debugedit rewrite the source paths
in the debuginfo to use that unique directory name.
Traditionally the debug src dir was named after the builddir which
was defined through the %setup macro which used the -n name argument
to define the builddir name and source archive to use. The builddir
might not be unique though between package versions.
Now that debugedit doesn't have strict base and dest dir length
restrictions for rewriting the source dir paths this can now be made
more flexible.
The added testcases show the difference between the old and new way.
The hello2.spec file defines the name of the package as hello2, but
uses the %setup marcro with -n hello-1.0 to use the hello-1.0.tar.gz
archive. This would traditionally result in a hello-1.0 builddir
which would be moved under /usr/src/debug. Possibly conflicting
with any other package (version) that used the same builddir name.
When defining _unique_debug_srcs to 1 that builddir will be moved
to <name>-<ver>-<rel>.<arch> instead (hello2-1.0-1.<arch>).
The testcases check that both the actual package source filess under
/usr/debug/src/ and the source paths as found in the .debug files are
under the traditional or new unique directory names depending on whether
the new _unique_debug_srcs macro is defined.
Signed-off-by: Mark Wielaard <mark@klomp.org>
As Python wheels do not contain targetted Python version in the directory/file
name of their metadata like Python eggs do, and since the Python version is not
contained in the metadata either, it is necessary to get it from elsewhere.
Here it is parsed from the path the metadata resides at
(e.g. /usr/lib/pythonX.Y/site-packages/...)
There is no need for /usr/bin/python when byte compiling files in
/usr/lib/pythonX.Y (only /usr/bin/pythonX.Y). Moved check so we do not exit
prematurely.
Fixes: rhbz#1411588
Trying to include AM_CFLAGS through a configure generated rpm.am file
doesn't really work because at the time automake runs configure doesn't
exist yet to process rpm.am.in. Just define the AM_CFLAGS substitution
inside the Makefile.am files themselves.
Rename rpm.am.in back to rpm.am.
Signed-off-by: Mark Wielaard <mjw@redhat.com>
These provides are specifically for packages providing
AppStream files, which are either going to be *.appdata.xml
or *.metainfo.xml files in /usr/share/appdata or /usr/share/metainfo.
The upstream AppStream specification mandates *.metainfo.xml files
installed into /usr/share/metainfo, but there's still a large body
of legacy AppStream files installed in the legacy location.
For now, let's support both under the new metainfo() Provides.
add_minidebug uses nm to select the function symbols to include in the
mini-symtab table. But on arches that use function descriptors (like ppc64)
nm --format-posix doesn't make it clear which symbols are real functions
The symbols point to the (stripped away) function descriptor table).
Use --format=sysv style to match the ELF symbol type directly instead of
using the somewhat ambiguous symbol type char used in --format=posix style
in binutils nm.
https://bugzilla.redhat.com/show_bug.cgi?id=1052415
Signed-off-by: Mark Wielaard <mjw@redhat.com>
When creating the compressed mini-symtab section in find-debuginfo
add_minidebug we explicitly remove .gdb_index and .comment. But there
can be other non-empty sections in the debuginfo that shouldn't be
copied. For example rust binaries might have a .rustc section.
Explicitly remove any non-allocated PROGBITS or NOTE sections.
https://bugzilla.redhat.com/show_bug.cgi?id=1382394
Signed-off-by: Mark Wielaard <mjw@redhat.com>
Add a -j <n> option, which, when used, will spawn <n> processes to do the
debuginfo extraction in parallel. A pipe is used to dispatch the files among
the processes.
Signed-off-by: Michal Marek <mmarek@suse.com>
This siplifies the handling of hardlinks a bit and allows a later patch
to parallelize the debuginfo extraction.
Signed-off-by: Michal Marek <mmarek@suse.com>
rpm2cpio.sh was refactored to minimize the use of external tools.
* after refactoring the utility requires to work: dd, printf, and unarchivers;
* add check that file passed as argument is a rpm-file;
* fix signatures of compressed data.
Signed-off-by: Alexey Gladkov <gladkov.alexey@gmail.com>
Tomas Orsava from the Fedora Python SIG requested that
the dependency generator support only using pythonXdist(M)
format for both Provides and Requires, so now this capability
exists.
From setuptools's documentation:
These files are not eggs, strictly speaking. They simply provide a way
to reference an egg that is not physically installed in the desired
location. They exist primarily as a cross-platform alternative to
symbolic links, to support "installing" code that is being developed in
a different location than the desired installation location.
If we read .egg-link using pkg_resources.Distribution it will
never have version as it is just list of directories which should be
taken into account.
We could change into that directories and add eggs from those locations
for parsing, but RPM's dependency generator already passing all files
from built RPM so it just does not make any sense to traverse those
directories.
After all written above, let's just ignore .egg-link files.
Signed-off-by: Igor Gnatenko <ignatenko@redhat.com>
In 49197c930b we introduced skipping
metadata which has no version, but it's better to show some warning.
Signed-off-by: Igor Gnatenko <ignatenko@redhat.com>
For example, reading .egg-link using pkg_resources.Distribution returns
actual metadata, but it does not contain version. It returns traceback like:
File "/usr/lib/rpm/pythondistdeps.py", line 113, in <module>
pyver_major = dist.py_version.split('.')[0]
AttributeError: 'NoneType' object has no attribute 'split'
Traceback (most recent call last):
File "/usr/lib/rpm/pythondistdeps.py", line 113, in <module>
pyver_major = dist.py_version.split('.')[0]
AttributeError: 'NoneType' object has no attribute 'split'
Let's just skip such errors as we can't do much about that.
Reference: https://bugzilla.redhat.com/show_bug.cgi?id=1368673
Reported-and-tested-by: Igor Gnatenko <ignatenko@redhat.com>
This patch lets debuginfo packages provide build-id like follows:
debuginfo(build-id) = c63cb23876c5fa85f36beaff58f8557e1bf22517
Originally this patch was written by Jan Blunck <jblunck@suse.de>.
Signed-off-by: Igor Gnatenko <ignatenko@redhat.com>
Introduce a new macro _unique_debug_names that when set will pass
--unique-debug-arch "%{_arch}" to find-debuginfo.sh to create debuginfo
files which end in "-<ver>-<rel>.<arch>.debug" instead of simply ".debug".
Adds testcases for dwz and buildid with and without unique debug file names.
Signed-off-by: Mark Wielaard <mjw@redhat.com>
Introduces _include_gdb_index macro and -i flag to find-debuginfo.sh to
enable or disable adding a .gdb_index section to debug files. Adds tests
to make sure the .gdb_index is really added (or not) when requested.
Checks that gdb-add-index is actually installed instead of silently
failing if not. Similar for dwz.
Signed-off-by: Mark Wielaard <mjw@redhat.com>
Introduce a new macro _unique_build_ids that when set will pass the
version and release to find-debuginfo.sh and debugedit to recalculate
the build-id of ELF files.
Includes two new testcases to make sure the new setting works as expected
both when set and unset.
Signed-off-by: Mark Wielaard <mjw@redhat.com>
This patch moves the main ELF file build-id symlinks from the
debuginfo package into the main package. And uses different
base directories for the main ELF file build-id symlink.
For the main build-id use /usr/lib/.build-id and for the debug
build-id use /usr/lib/debug/.build-id.
There are two reasons for doing this. The main package and the
debuginfo package might get out of sync, or the debuginfo package
might not be installed at all. In which case finding the main ELF
file through the build-id symlink becomes impossible. Secondly by
moving the main ELF build-id symlink in its own directory the
/usr/lib/debug directory gets populated with only debuginfo files
which is convenient if the user might want to have that directory
populated through a network mountpoint.
To support the new logic the symlink code has been moved from
find-debuginfo.sh to build/files.c.
This also includes support for a new config %_build_id_links that
defaults to compat. The other settings are none, alldebug (the old
style) and separate. compat is like separate, but adds a compatibility
link under /usr/lib/debug/.build-id for the main build-id symlink.
There are several new testcases added to test the various settings
using the new keyword "buildid".
Signed-off-by: Mark Wielaard <mjw@redhat.com>
Some old tools might still use the .gnu_debuglink section to find
separate debuginfo files instead of build-id style lookups. When
dwz has compresses the .debug files the original CRC in the main
ELF file will no longer match. Make sure to run sepdebugcrcfix
after dwz to recalculate the CRC.
The original fix was created by Jan Kratochvil based on code
from GNU binutils BFD. https://bugzilla.redhat.com/show_bug.cgi?id=971119
I added a testcase to make sure the CRCs were all correctly
updated after dwz has run to compress a debuginfo package.
And a change (plus testcase) to make sure implicit suid binaries
didn't accidentially got their suid flag bit.
Signed-off-by: Mark Wielaard <mjw@redhat.com>
Support for minisymtab (a minimal function symbol table in a compressed
section in the main binary) has been in gdb and elfutils based tools
since some years. Fedora has had this as rpm-4.10.0-minidebuginfo.patch
since 2012.
The patch adjusts macros to pass -m to find-debuginfo.sh when
_include_minidebuginfo has been set. find-debuginfo.sh now takes -m
as argument to generate the .gnu_debugdata ELF section to be added
to the main executable.
To support the testcases a new macros.debug is added that is used to
generate debuginfo packages in the rpmbuild.at testsuite.
The original support was added to Fedora rpm by Alexander Larsson.
Lubos Kardos fixed a bug in it when strip -g was used. I added some
configuration macros and two testcases to check the basic support works
and for the strip -g bug.
Signed-off-by: Mark Wielaard <mjw@redhat.com>
The fact that pyc and pyo doesn't have same the content doens't mean
an error. It just means they can't be hard linked but that is not
an error. In normal circumstances the file brp-python-hardlink from
the rpm packages is often overrided by the file from redhat-rpm-config
that is why this problem wasn't revealed earlier.
Per the recommendation of Nick Coghlan and Toshio Kuratomi,
pythonXegg(M) is being renamed to pythonX.Ydist(M).
An option has also been added to add a pythonXdist(M)
Provides for distributions that may prefer to have it.
The option '--majorver-provides' is intended for use
if only one Python stack per major version will be
available at a given time, as unexpected results may
occur if there are multiple independent Python stacks
per major version available.
Consequently, it will not be on by default when using
the generator for generating Provides.
Additionally, .egg-info data is being replaced with
.dist-info data, so we need to handle that case, too.
See for more details:
https://lists.fedoraproject.org/archives/list/python-devel%40lists.fedoraproject.org/thread/SQBSAS4T25HK5YJBNBSFDD7KDQWDELL6/
Also, Thierry Vignaud brought up on rpm-maint that Mageia
currently uses "pythonegg(X)(M)" (e.g. "pythonegg(3)(rpm)"
for python3 rpm bindings package) in their Python packages
to pull in Python dependencies and requested a way to
not break Mageia.
After discussing with Florian Festi about it, Mageia's
pythonegg(X)(M) will be supported by adding '--legacy'
as a switch to generate legacy Provides/Requires to maintain
compatibility with Mageia's existing usage.
The '--legacy-provides' switch will enable pythonegg(X)(M)
Provides in addition to the new pythonX.Ydist(M) format to
allow for a easier transition.
rationale:
1) typelib deps are unrelated to python
2) typelib deps are computed by suse generator
3) suse generator is not present upstream
4) let's not reinvent internal deps generator here...
This commit 0d5929ba5e tries to ignore
"use" and "require" within multi-line print statements that start with
line like this "print <<EOF" but it incorrectly parses lines in
following format "print <<EOF unless $o" and every "use" or "require"
below this line to the end of file is ignored. That causes that some
requires which was previously found are not found now. This commit
fixes this problem (#1268021).
Using '/usr/bin/env python' could lead to unexpected results,
so switching to this guarantees that it will use the system-wide
Python environment, instead of any virtual environments or whatnot.
This change was brought over from the fix used by Mageia to speed
up the extraction of Python dependencies. As Mageia found out
(and verified to be true with this generator too), the generator
is now called on every file in the package. This means that the
pkg_resources import time is now a drag on the performance of
generating dependencies.
For more details, see the following link:
http://gitweb.mageia.org/software/rpm/rpm-setup/commit/pythoneggs.py?id=b77d47f90289413e20f295b93ae8c51012f53e3d
The egg dependency generator was slightly inconsistent here.
While it would generate python2egg() for Py2 Requires/Recommends and
pythonegg() for Py3 Requires/Recommends, it would generate python3egg()
for Py3 Provides and pythonegg() for Py2 Provides. Rather than make a
decision on which is the "proper" Python here, let's just handle them
the same way: pythonXegg(), with "X" being the Python major version.
This way, it's absolutely clear which Python it is for, and as
new major versions of Python become a reality, we should be able to
handle that quite easily and (of course) sanely. The approach is
inspired by the method used in Mageia's python dependency generator.
In case the generator is being used in an environment where
Python 2 is the default still, the appropriate import to ensure
that print() works has been added.
Additionally, the string formatting calls have been refactored
to use string.format() instead of old-style printf-esque calls.
This enhances the readability slightly by making it so that '%'
isn't being used for anything but RPM stuff in strings.
- Now script perl.req ignores "use" and "requires" on lines that are
part of printing or assigning multi-line string i. e. string that
hasn't starting and ending quote on the same line.
(RhBug:1024517)
- script perl.req was able to parse only multi-line print statements
with quoted tag e.g. 'print <<"tag"' or "print <<'tag'". Now it can
also parse "print <<tag".
- The php and Tcl (RhBug:1158580) have never been used.
- perldeps.pl was supposed to be a better perl generator but nobody has
bothered using it, maybe it just doesn't work?
- osgideps.pl has never been wired up with anything so any user
would have manually invoked it. Lets see if anybody hollers, at least
Fedora has entirely different, rpmfc-style generators for it
(RhBug:1158583)
- Replace platform specific find-{requires,provides} scripts with
trivial wrapper scripts that just call rpmdeps with suitable
arguments. This way the generated dependencies using the legacy
external dependency are at least roughly on par with the internal
depgen as, well, they're generated by the same thing.
- Changing from find-{requires,provides} scripts to rpmdeps could
be done by just changing __find_{requires,provides} macros, but
a lot of packages directly refer to the scripts instead so this
way we cover far more ground and in an backwards compatible way.
- The paths defined in macros.perl have never even existed in rpm,
would've always been suspect and nowadays mostly just plain useless
as they'd only be used by the deprecated external dependency
generator.
Appdata files contain application information used by the AppStream
project. We generate two provides: appdata() to support enumeration
of all application packages and appdata(filenname) to make it easy
to link installed packages with appdata files.
Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
Gnome software center needs to know what package to deinstall if it
needs to deinstall a desktop application. Looking up provides it much
cheaper than looking up which package owns a file.
We also add an empty application() provides to make it easy to
enumerate all packages containing desktop applications.
Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
Mark Wielaard