ELF ET_DYN files can be shared libraries, executable files or both.
Requiring shared libraries that cannot actually be executed (or segfault
when you do so) is dumb, but rpm has traditionally required executable
bit to be set for requires to be generated.
Refine the definition of "executable" to mean files that actually *can*
be executed, ie those with PT_INTERP and whose actual behavior depends
on whether the executable bit is set or not.
In other words, pure shared libraries will have requires generated
regardless of their permission, but executable files can still disable
that by disabling the on-disk executable bit because unlike shared
libraries, they only require anything when being executed.
Also add rudimentary testcases for the scenarios involving a shared
library, traditional binary and a PIE binary. The test material needs
to be pre-built as we cannot predict what dependencies building on an
arbitrary compiler on arbitrary platform may have, but source is
included for manually regenerating as needed.
No need to call edit_dwarf2() if debugedit is invoked just to extract
build-id (with `-i -n'). Otherwise, we will get `DWARF version 0
unhandled' warning for compressed .debug files:
/usr/lib/rpm/debugedit: ./usr/lib/debug/...e.ko.debug: DWARF version 0 unhandled
Context: We have kernel modules elfcompressed in ALT.
This was kinda ugly-but-necessary when added back in 2003 (commit
752cac72e2) but entirely redundant
since the "new" dependency generator in rpm >= 4.9.x with arbitrary
filtering support. The handful of packages using it can just as well
achieve the same (and more) without special hacks in rpm with:
%global __requires_exclude GLIBC_PRIVATE
%global __provides_exclude GLIBC_PRIVATE
It was possible to generate Requires, Recommends etc. using external dependency generators.
Adding ability to generate OrderWithRequires.
Example use case:
When a package contains a systemd unit, %systemd_* macros are usually used;
it is usefull to add "OrderWithRequires: systemd" in this case to ensure
that systemd is installed before that package.
It will help to avoid adding "%systemd_ordering" manually to all packages using systemd.
Having systemd preinstalled before packages with systemd scriptlets is really important,
otherwise those scriptlets fail silently, and the resulting ISO or chroot may be broken.
The same makes sense for e.g. systemd-sysusers, systemd-tmpfiles.
An RPM generator using this functionality was implemented: https://abf.io/import/order-rpm-generators
Rebuilding packages with systemd stuff in rosa2019.05 using this generator has already helped
to improve installation order, e.g. make e.g. openvpn be installed when systemd already exists
in a big transaction with ~3500 packages when building a big ISO image.
Before that openvpn was installed when systemd did not exist yet.
P.S. This patch adds %__find_orderwithrequires, maybe that is legacy code that should not be changed.
Signed-off-by: Mikhail Novosyolov <m.novosyolov@rosalinux.ru>
Not all architectures offer MAP_POPULATE. As MAP_POPULATE is only an
optimization to improve performance, it is safe to drop it when it is
unavailable. Originally fixed in a Debian patch.
Found by running the debugedit tests under valgrind.
If the old and new .debug_line offset isn't changed then we might
write out an uninitialized new_idx.
When compiling with -g3 gcc will generate a .debug_macro section
which has pointers to the .debug_str section. Since we might rewrite
the .debug_str section, we also need to update any .debug_macro
pointers.
Updated the debugedit.at testcase by building everything with -g
and add various checks to see the .debug_macro section looks OK
after running debugedit. Added a new rpmbuild.at testcase to check
handing of .debug_macro in the whole rpmbuild debuginfo pipeline
to double check the separate .debug file also contains the macros.
Original patch by Michael Schroeder <mls@suse.de>. Extended by
Mark Wielaard <mark@klomp.org> to deal with relocations and possible
multiple COMDAT .debug_macro sections.
This refactors the reading and writing of relocated values into seperate
helper functions (setup_relbuf and update_rela_data). It will be easier
to reuse this code in case we want to read/write relocated values in other
sections than DEBUG_INFO. The only functional change is that we explicitly
track whether the relocation data is updated, and only explicitly update
and write out the relocation data if so. In the case there were no strp
or stmt updates, there will also not be any relocation updates, even if
there is relocation data available.
All new debugedit testcases pass before and after this refactoring.
debugedit didn't have any direct tests, it was only tested indirectly
through some other debuginfo testcases. So add some testcases that
just test debugedit functionality directly.
The tests create different kinds of ELF files (object files, partially
linked files and executables) and run debugedit path replacements on
them, checking that the resulting .debug_str, .debug_info and .debug_line
sections look sane.
Testcases verified against various different gcc and binutils versions.
gcc 4.8 and gcc 6 generate slightly different .debuginfo
Also add debugedit --version. Which makes it easy to see we are
testing the correct version/install of debugedit in rpmtests.log.
Adjust autotest paths to include rpmlibexecdir, where debuginfo
is installed. Note that rpmlibexecdir != libexecdir (see rpm.am).
processFile() always returned errors for things like non-existent file
and non-elf file etc but we've ignored it all. Reflect such errors with
non-zero exit code as per common practise.
edit_attributes is run twice. Once for phase zero in which all strings are
collected. Then then for phase one in which the strings are rewritten. In
phase zero we also try to collect the comp_dir (either from the
DW_AT_comp_dir or the DW_AT_name of the compile unit). We were also
collecting the comp_dir is phase 1, which is unnecessary, and would not
actually work, since we would be using to old string table index for that,
which had already been rewritten.
Caught by the new string table index checks.
Signed-off-by: Mark Wielaard <mark@klomp.org>
debugedit would blindly use an .debug_str index from the .debug_info or
.debug_line sections assuming it would result in a valid string. Which
would crash and burn if the DWARF data was bogus when the string was
used. So check whenever converting an string index into a char pointer
so we can produce a more helpful error message.
https://bugzilla.redhat.com/show_bug.cgi?id=1543912
Signed-off-by: Mark Wielaard <mark@klomp.org>
edit_dwarf2 calculates the (new) offset in the line program by
taking the difference between the old and new idx, which are of type
size_t (unsigned), plus the size_diff of the header given as ssize_t
(signed), and adding that to the current r_offset, which is an Elf64_Addr
(unsigned). On 64bit architectures, where the size of Elf64_Addr and
ssize_t are the same this isn't a problem. But on 32bit architectures,
where the size of ssize_t is smaller than Elf64_Addr the smaller signed
result gets promoted to an unsigned long first causing issues if the
size_diff was negative.
This would have been caught by gcc -Wsign-conversion
warning: conversion to ‘long unsigned int’ from ‘ssize_t’ {aka ‘long int’}
may change the sign of the result
But enabling this by default gives a lot of false positives.
Found and fixed by Richard Biener <rguenther@suse.de>.
These are common in all cli operation, hiding them in "query flags" under
yet another (or actually two, QUERY_DIGEST / VERIFY_DIGEST etc) different
names and then translating back to internal vsflags everywhere doesn't
make anybody's life easier. No functional changes from cli point of view,
any theoretical API users will need changes of course.
This will send the output from rpmfcPrint() to stdout. This is an
alternative to using the --rpmfcdebug option, which will send the same
output to stderr. The two options have totally different use cases
though. While --alldeps is used when the output from rpmfcPrint() is
what is wanted, --rpmfcdebug can be used together with the other
output options, e.g., --requires, without affecting their output.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@gmail.com>
To count as a real directory prefix the string matched should either
be equal to the given prefix or start with the prefix plus '/'.
skip_dir_prefix is always used with base_dir or dest_dir which don't
end with a slash themselves.
This really only is an issue if a package would put a directory named
similar to the package source dir (which cargo on fedora does, by adding
a directory named cargo-vendor in the builddir itself).
Signed-off-by: Mark Wielaard <mark@klomp.org>
The filling of the buffer, buf, that was passed to rpmfcPrint() was
removed in d16bdb15 (in 2007). Since then, an uninitialized buffer was
used if the files to calculate the dependencies for were passed as
arguments on the command line.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@gmail.com>
The fix for rhbz#444310 (commit c1a5eb - Include empty CU current dirs)
was a little greedy. It would also include comp_dirs outside the build
root. Those are unnecessary and we don't have a good way to store them.
Such dirs (e.g. /tmp) would then show up at the root of /usr/src/debug.
Fix this by including only comp_dirs under base_dir. Also only output
all dirs once (during phase zero) and don't output empty dirs (which
was harmless but would produce a warning from cpio).
This still includes all empty dirs from the original rhbz#444310
nodir testcase and it is an alternative fix for rhbz#641022
(commit c707ab).
Both fixes are necessary in case of an unexpected mode for a directory
actually in the build root that we want to include in the source list.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Some packages depend on the build-ids as generated during the build
and cannot handle rpmbuild recomputing them before generating the
package file list. Add -n, --no-recompute-build-id to debugedit and
add -n to find-debuginfo.sh set by defining the %_no_recompute_build_ids
macro for such packages. %_no_recompute_build_ids can not be used together
with %_unique_build_ids.
Signed-off-by: Mark Wielaard <mark@klomp.org>
This problem was found by ALT rpm verify-elf brp script:
verify-elf: WARNING: ./usr/lib/rpm-plugins/systemd_inhibit.so: uses non-LFS functions: __lxstat
verify-elf: WARNING: ./usr/lib/rpm/sepdebugcrcfix: uses non-LFS functions: __xstat mmap open pread pwrite
Signed-off-by: Gleb Fotengauer-Malinovskiy <glebfm@altlinux.org>
We would put one too many slashes in between the new dest_dir and file name
part of the replacement of a DW_FORM_string in the .debug_info. If there
was file part then we would overwrite the first character of the name. If
there was no file part at all then this would overwrite the zero terminator
and cause a crash reading the rest of the data.
A crash did happen while building the docker package on fedora s390x.
https://bugzilla.redhat.com/show_bug.cgi?id=1434347
The reason neither issue would normally trigger is because if we do detect
that the dest_dir is larger than the base_dir we refuse to replace anything.
Signed-off-by: Mark Wielaard <mark@klomp.org>
We wouldn't replace the changed file names if replace_dirs was false,
but replace_files was true. This could overrun the new debug_line data
buffer if the original file name was larger than the replacement. It
wasn't found before because often when we need to replace files we
also would have to replace dirs.
This fixes the kubernetes build in fedora.
Signed-off-by: Mark Wielaard <mark@klomp.org>
debugedit doesn't read raw mmap data any longer. Which made the complex
way to read the build-id unnecessary (and it was broken for cross-endian).
Just use gelf_getnote to read the notes.
Also in some special cases when only the debug_info or build_id data
was updated, but no section changed size and we had to preserve the
allocated section headers we could hit a bug in elfutils that could
trash some section data in case there were gaps between non-dirty and
dirty sections. See https://sourceware.org/bugzilla/show_bug.cgi?id=21199
Add a workaround for that issue.
This fixes the kompose package build on fedora ppc64.
And makes it possible to replicate that issue on x86_64.
Signed-off-by: Mark Wielaard <mark@klomp.org>
debugedit --base to --dest rewriting of debug source file paths only
supported dest paths that were smaller or equal than the base path
(and the size should differ more than 1 character for correct debug lines).
All paths were changed "in place". Which could in theory mess up debug str
sharing.
This rewrite supports base and dest strings of any size (some limitations,
see below). This is done by reconstructing the debug_str and debug_line
tables and updating the references in the debug_info attributes pointing
to these tables. Plus, if necessary (only for ET_REL kernel modules),
updating any relocations for the debug_info and debug_line sections.
This has the nice benefit of merging any duplicate strings in the
debug_str table which might resulting on slightly smaller files.
kernel modules are ET_REL files that often contain a lot of duplicate
strings.
The rewrite uses elfutils (either libebl or libdw) to reconstruct the
debug_str table. Since we are changing some section sizes now we cannot
just use mmap and rawdata to poke the values, but need to read in and
write out the changed sections. This does take a bit more memory because
we now also need to keep track of all string/line references.
There are still some limitations (already in the original debugedit)
not fixed by this rewrite:
- DW_AT_comp_dir in .debug_info using DW_FORM_string can not be made
larger. We only warn about that now instead of failing. The only
producer of DW_FORM_string comp_dirs is binutils gas. It seems simpler
to fix gas than to try to support resizing the debug_info section.
- A DW_AT_name on a DW_TAG_compile_unit is only rewritten for DW_FORM_strp
not for DW_FORM_string. Probably no problem in practice since this
wasn't supported originally either.
- The debug_line program isn't scanned for DW_LNE_define_file which
could in theory define an absolute path that might need rewriting.
Again probably not a problem because this wasn't supported before
and there are no know producers for this construct.
To support the upcoming DWARFv5 in gcc 7 (not on by default), we will
need to add support for the new debug_line format and scan the new
debug_macro section that can have references to the debug_str table.
Signed-off-by: Mark Wielaard <mark@klomp.org>
I think the reliance on bfd.h was a mistake. The code was lifted from
bfd, but should be totally independent (it just calculates a CRC).
Fix the type to be a normal size_t and include sys/stat.h (which was
included through bfd.h) to get the definitions of stat and chmod.
Introduce a new macro _unique_build_ids that when set will pass the
version and release to find-debuginfo.sh and debugedit to recalculate
the build-id of ELF files.
Includes two new testcases to make sure the new setting works as expected
both when set and unset.
Signed-off-by: Mark Wielaard <mjw@redhat.com>
Some old tools might still use the .gnu_debuglink section to find
separate debuginfo files instead of build-id style lookups. When
dwz has compresses the .debug files the original CRC in the main
ELF file will no longer match. Make sure to run sepdebugcrcfix
after dwz to recalculate the CRC.
The original fix was created by Jan Kratochvil based on code
from GNU binutils BFD. https://bugzilla.redhat.com/show_bug.cgi?id=971119
I added a testcase to make sure the CRCs were all correctly
updated after dwz has run to compress a debuginfo package.
And a change (plus testcase) to make sure implicit suid binaries
didn't accidentially got their suid flag bit.
Signed-off-by: Mark Wielaard <mjw@redhat.com>
The RPM code contains setprogname()/getprogname() support implemented through compatiblity layer with very old GLIBC (internals supported back to '95 and earlier), before stabilization of the GNU C library. This compatiblity layer (__progname, __assert_progname, setprogname()) is supposed to support well archaic GLIBC, but on the other hand it pollutes the library namespace and introduces unpredicable compillation errors on BSD systems.
The functions setprogname() and getprogname() are natively supported in NetBSD and work the same way as __progname from the GNU C library (they are even implemented in the same way - but with a slightly changed logic). The support for very old (20 years and older) GNU C Library is obfuscating the code, because it uses defines over defines without a word of explaination why to do so.
It's important to note that the setprogname()/getprogname() was inconstiently implemented in the codebase, duplicating the code and/or functionality.
Add new generic functions getprogname() and setprogname() and bind it to:
- the current and for two decades stable GNU LIB C implementation,
- the current NetBSD implementation (introduces to NetBSD in 2002),
- fallback reimplementation functions of the setprogname() and getprogname() functionality for other systems.
Don't support anymore old GNU Lib C internals and don't support older NetBSD systems, as they aren't supported for many years.
Add to the codebase comments explaining the relevant codeparts.
- The original case of empty string ending up in a dependency is already
taken care of by commit 66a01c977e and
soname filtering. However if filtering is disabled, an an empty
(or all-whitespace) soname will produce gems like "()(64bit)" on
multilib arches, so we need to sanity check the soname itself in
elfdeps.
- The linker doesn't seem to care what kind of junk the soname contains,
we care just a little bit more as eg empty strings and whitespace
messes up other things.
AArch64 generates a relocation which must be handled similar to other
architectures. Adding this patch allows debugedit to run against the
kernel debuginfo.
Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
- The message is not about basedir and destdir, and printing them
here only makes the message even more confusing than it was, eg:
canonicalization unexpectedly shrank by one character \
('/home/pmatilai/rpmbuild/BUILD/' vs '/usr/src/debug/')
- This reverts commit 1eea433d99
- A dependency on the interpreter is likely to be redundant, but
presence of PT_INTERP is a useful indication of whether a file
is executable or not: all normal dynamically linked executables
have it whether PIE DSO or "plain" executable. Some libraries
also have it but they are supposed to be really executable:
for example glibc's libc.so and libQtCore.so have it and print out
config, version etc information when executed.
- For now, we dont actually use the PT_INTERP information for anything
apart from the optional require.
- Instead of vain heuristics on DT_SONAME presence, filter out
irregular sonames from all dependencies: linkable library names generally
must contain ".so" and start with "lib" for the linker to find it at all,
anything else is an exception of one kind or another (the prime exception
of ld.so variants we handle here). This weeds out provides for most
dlopen()'ed modules etc, and filtering both provides and requires
by the same rules means we wont generate requires for things that wont be
provided. Of course this also means we can omit things that are in
DT_NEEDED, but these should be rare exceptions which the new
--no-filter-soname switch is for.
Panu Matilainen