For kernel build, "${LISTFILE}.dirs.sed" is debugfiles.list.dirs.sed
and it contains 1782 lines of sed script.
It is applied to two files, both are about 4450 lines long.
This is slow (~30 seconds) because of ~16 million regex matches
performed by sed.
But we don't need or want regex match here
(and it's buggy, since dots in pattern match will be treated
as "any character", which is wrong here. For example,
/usr/lib/debug/lib/modules/5@4@0-0@rc7@0@fc31@test@x86_64/vdso/
would match
/usr/lib/debug/lib/modules/5.4.0-0.rc7.0.fc31.test.x86_64/vdso/
pattern, but it should not).
This change performs matching using shell string comparison ops.
For kernel build, this change results in run time of about one second.
Signed-off-by: Denys Vlasenko <dvlasenk@redhat.com>
There's code setting variable id in scripts/find-debuginfo.sh, but there's no
subsequent use. The only use of $id in the script is in do_file, where it is
a local variable.
The variable setting was introduced in commit 6b3b435fa "Add dwz debuginfo
compression support" in combination with a subsequent make_id_link using the
variable, but the make_id_link was removed in commit bbfe1f86b
"Add build-id links to rpm for all ELF files".
Remove the unused variable.
Prefer '[] && []' to '[ -a ]' and '[] || []' to '[ -o ]' in tests.
-a and -o to mean AND and OR in a [ .. ] test expression is not well
defined, and can cause incorrect results when arguments start with
dashes or contain !. Moreover binary -a and -o are inherently
ambiguous. test(1) man page recommends to use
'test EXPR1 && test EXPR2' or 'test EXPR1 || test EXPR2' instead.
It corrects warnings [SC2166] spotted by covscan.
Spotted by covscan:
warning: Prefer [ p ] && [ q ] as [ p -a q ] is not well defined. [SC2166]
Originally introduced in commit 9aae21d.
Thanks to Florian Festi for spotting this and proposing the solution.
With the introduction of the annobin gcc plugin to the build process,
built binary files have become larger. Sometimes significantly so.
This is a patch that adds a new step to the post-link process performed
by rpmbuild, to run the objcopy program with the --merge-notes option
specified. This will reduce the size of the annobin notes in binary
files, thus alleviating the size growth.
The find-debuginfo.sh flag -g had exactly this meaning. But from
version rpm-4.13.0-alpha flag -g changes its behavior. It affects
both libraries and executables.
For some packages the original behavior was preferred. That is why
the new find-debuginfo.sh flag --g-libs is created.
Options -g and --g-libs are mutually exclusive.
Since file 5.33, PIEs are identified by a new MIME type, meaning that,
currently, for such executables, the .gnu_debugdata section is not
added, even if -m is passed.
Signed-off-by: Ernestas Kulik <ekulik@redhat.com>
By standard, 'du' defaults to 512 byte units, but this differs between
implementations: GNU du defaults to 1024 unless POSIXLY_CORRECT is set
and also allows overriding via number of environment variables. It also
supports various other ways of controlling the block size, but the
only standard means to control the size is -k, and we really don't
want to break portability for the sake of FYI statistics.
Inspired by Marxim Liska's patch at in #554.
This is breaking suid bit handling in packaging unless an unreleased
bleeding edge version of elfutils is used. In other words it's not
an acceptable requirement and wont be for a long time. This needs
to be fixed in some other way.
This reverts commit 1c3bf3d649.
to make build results more reproducible
in spite of indeterministic filesystem readdir order.
For openSUSE, this helped to make squid, openssh, postfix and shadow
packages build reproducibly.
See https://reproducible-builds.org/ for why this is good.
This typo has always been there since the initial implementation in
Fedora-specific patches in 2012 and inherited upstream. Which suggests
something hasn't been working right all this time, or then this will
break something all of a sudden...
debugedit and dwz do not support DWARF compressed ELF sections, let's
just decompress those before extracting debuginfo.
Tested-by: Igor Gnatenko <i.gnatenko.brain@gmail.com>
A new version of file (5.33) changed the output for PIE executables.
Which are now represented as application/x-pie-executable; charset=binary.
The following change simply recognizes that binary format as one for
which -g applies. This fixes accidental stripping of the .symtab in
glibc (which use -g to keep that symbol table).
https://bugzilla.redhat.com/show_bug.cgi?id=1581224
Signed-off-by: Mark Wielaard <mark@klomp.org>
The %_debugsource_template expects the debugsourcefiles.list file
to be in the (current) build dir. Make sure that is always the case
even if find-debuginfo.sh was invoked in a different (sub) directory
by prepending $BUILDDIR to the output file as written in description
"All file names in switches are relative to builddir (. if not given).".
Closes: https://github.com/rpm-software-management/rpm/issues/279
Based-on-patch-by: Mark Wielaard <mark@klomp.org>
Signed-off-by: Igor Gnatenko <i.gnatenko.brain@gmail.com>
In the minisymtab section (the .gnu_debugdata embedded ELF image) we
do not need unallocated sections (except for the SYMTAB and STRTAB
sections we are creating). We already remove PROGBITS and NOTES. Also
remove NOBITS sections. They should not really take up much (any) space
but they still add to the section tables. These sections might be created
with the new --keep-section support (which puts the actual section in
the main ELF binary, and a NOBITS variant in the .debug file).
Also binutils objcopy seems to sometimes add them anyway filled with
zeros instead of marking them NOBITS.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Use --keep-section SECTION or --remove-section SECTION to explicitly
keep a (non-allocated) section in the main executable or explicitly
remove it into the .debug file. SECTION is an extended wildcard pattern.
Both options can be given more than once.
https://bugzilla.redhat.com/show_bug.cgi?id=1465997
Signed-off-by: Mark Wielaard <mark@klomp.org>
commit 038bfe "Split directory traversal and debuginfo extraction"
put the core of a while loop into its own function 'do_file()'.
That means that instead of using 'continue' to break out early it now
needs to use 'return'. Otherwise the script will give errors like:
continue: only meaningful in a `for', `while', or `until' loop
https://bugzilla.redhat.com/show_bug.cgi?id=1465170
Signed-off-by: Mark Wielaard <mark@klomp.org>
There is no official way to mark an instruction range as being not
part of some actual source code, but as part of a compiler built-in
construct in DWARF. So different compilers have come up with fake
source file names like <built-in> or <__thread_local_inner macros>.
We already filtered out the strings "<internal>" and "<built-in>".
Just filter out all '(^|/)<[a-z _-]+>$'. They are fake files!
This is mainly to appease the rustc compiler which generates lots of
different variants to encode some instruction sequence is part of an
compiler generated macro expansion.
Signed-off-by: Mark Wielaard <mark@klomp.org>
dwz -m multi only works when there are multiple .debug input files.
With just one .debug file it doesn't really make sense to extract
the shared debug info into a separate file and dwz will complain:
dwz: Too few files for multifile optimization.
So only add -m multi if there is more than one .debug file.
Signed-off-by: Mark Wielaard <mark@klomp.org>
This adds some missing documentation for rpm macros and find-debuginfo.sh
options that were recently added (or renamed). -j N, --build-id-seed SEED,
--unique-debug-suffix SUFFIX and --unique-debug-src-base BASE.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Rename --ver-rel option to --build-id-seed, so that it reflects what
it does, not how it is used.
Remove implcit usage of the old --ver-rel option for --unique-debug-arch
and --unique-debug-src-base, instead already call find-debuginfo.sh with
the version included.
Rename --unique-debug-arch to --unique-debug-suffix because it now
also contains the package version.
It only makes sense to add a minisymtab for executables and shared
libraries. Other executable ELF files (like kernel modules) don't need it.
Since those don't have a dynsym section trying to add it will fail and
produce confusing errors from nm.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Some packages depend on the build-ids as generated during the build
and cannot handle rpmbuild recomputing them before generating the
package file list. Add -n, --no-recompute-build-id to debugedit and
add -n to find-debuginfo.sh set by defining the %_no_recompute_build_ids
macro for such packages. %_no_recompute_build_ids can not be used together
with %_unique_build_ids.
Signed-off-by: Mark Wielaard <mark@klomp.org>
In case that binary compiled from source generated in /tmp, a
/usr/src/debug/tmp directory will be created with the same mode as
/tmp, a.k.a 777, which should be avoided.
Fixes: rhbz#641022
Introduce a new macro _unique_debug_srcs that when set will pass
--unique-debug-src-base "%{name}" to find-debuginfo.sh which will
move sources into a unique "<name>-<ver>-<rel>.<arch>" directory
under /usr/src/debug/ and makes debugedit rewrite the source paths
in the debuginfo to use that unique directory name.
Traditionally the debug src dir was named after the builddir which
was defined through the %setup macro which used the -n name argument
to define the builddir name and source archive to use. The builddir
might not be unique though between package versions.
Now that debugedit doesn't have strict base and dest dir length
restrictions for rewriting the source dir paths this can now be made
more flexible.
The added testcases show the difference between the old and new way.
The hello2.spec file defines the name of the package as hello2, but
uses the %setup marcro with -n hello-1.0 to use the hello-1.0.tar.gz
archive. This would traditionally result in a hello-1.0 builddir
which would be moved under /usr/src/debug. Possibly conflicting
with any other package (version) that used the same builddir name.
When defining _unique_debug_srcs to 1 that builddir will be moved
to <name>-<ver>-<rel>.<arch> instead (hello2-1.0-1.<arch>).
The testcases check that both the actual package source filess under
/usr/debug/src/ and the source paths as found in the .debug files are
under the traditional or new unique directory names depending on whether
the new _unique_debug_srcs macro is defined.
Signed-off-by: Mark Wielaard <mark@klomp.org>
add_minidebug uses nm to select the function symbols to include in the
mini-symtab table. But on arches that use function descriptors (like ppc64)
nm --format-posix doesn't make it clear which symbols are real functions
The symbols point to the (stripped away) function descriptor table).
Use --format=sysv style to match the ELF symbol type directly instead of
using the somewhat ambiguous symbol type char used in --format=posix style
in binutils nm.
https://bugzilla.redhat.com/show_bug.cgi?id=1052415
Signed-off-by: Mark Wielaard <mjw@redhat.com>
When creating the compressed mini-symtab section in find-debuginfo
add_minidebug we explicitly remove .gdb_index and .comment. But there
can be other non-empty sections in the debuginfo that shouldn't be
copied. For example rust binaries might have a .rustc section.
Explicitly remove any non-allocated PROGBITS or NOTE sections.
https://bugzilla.redhat.com/show_bug.cgi?id=1382394
Signed-off-by: Mark Wielaard <mjw@redhat.com>
Add a -j <n> option, which, when used, will spawn <n> processes to do the
debuginfo extraction in parallel. A pipe is used to dispatch the files among
the processes.
Signed-off-by: Michal Marek <mmarek@suse.com>
This siplifies the handling of hardlinks a bit and allows a later patch
to parallelize the debuginfo extraction.
Signed-off-by: Michal Marek <mmarek@suse.com>
Introduce a new macro _unique_debug_names that when set will pass
--unique-debug-arch "%{_arch}" to find-debuginfo.sh to create debuginfo
files which end in "-<ver>-<rel>.<arch>.debug" instead of simply ".debug".
Adds testcases for dwz and buildid with and without unique debug file names.
Signed-off-by: Mark Wielaard <mjw@redhat.com>
Introduces _include_gdb_index macro and -i flag to find-debuginfo.sh to
enable or disable adding a .gdb_index section to debug files. Adds tests
to make sure the .gdb_index is really added (or not) when requested.
Checks that gdb-add-index is actually installed instead of silently
failing if not. Similar for dwz.
Signed-off-by: Mark Wielaard <mjw@redhat.com>
Introduce a new macro _unique_build_ids that when set will pass the
version and release to find-debuginfo.sh and debugedit to recalculate
the build-id of ELF files.
Includes two new testcases to make sure the new setting works as expected
both when set and unset.
Signed-off-by: Mark Wielaard <mjw@redhat.com>
This patch moves the main ELF file build-id symlinks from the
debuginfo package into the main package. And uses different
base directories for the main ELF file build-id symlink.
For the main build-id use /usr/lib/.build-id and for the debug
build-id use /usr/lib/debug/.build-id.
There are two reasons for doing this. The main package and the
debuginfo package might get out of sync, or the debuginfo package
might not be installed at all. In which case finding the main ELF
file through the build-id symlink becomes impossible. Secondly by
moving the main ELF build-id symlink in its own directory the
/usr/lib/debug directory gets populated with only debuginfo files
which is convenient if the user might want to have that directory
populated through a network mountpoint.
To support the new logic the symlink code has been moved from
find-debuginfo.sh to build/files.c.
This also includes support for a new config %_build_id_links that
defaults to compat. The other settings are none, alldebug (the old
style) and separate. compat is like separate, but adds a compatibility
link under /usr/lib/debug/.build-id for the main build-id symlink.
There are several new testcases added to test the various settings
using the new keyword "buildid".
Signed-off-by: Mark Wielaard <mjw@redhat.com>
Some old tools might still use the .gnu_debuglink section to find
separate debuginfo files instead of build-id style lookups. When
dwz has compresses the .debug files the original CRC in the main
ELF file will no longer match. Make sure to run sepdebugcrcfix
after dwz to recalculate the CRC.
The original fix was created by Jan Kratochvil based on code
from GNU binutils BFD. https://bugzilla.redhat.com/show_bug.cgi?id=971119
I added a testcase to make sure the CRCs were all correctly
updated after dwz has run to compress a debuginfo package.
And a change (plus testcase) to make sure implicit suid binaries
didn't accidentially got their suid flag bit.
Signed-off-by: Mark Wielaard <mjw@redhat.com>
Support for minisymtab (a minimal function symbol table in a compressed
section in the main binary) has been in gdb and elfutils based tools
since some years. Fedora has had this as rpm-4.10.0-minidebuginfo.patch
since 2012.
The patch adjusts macros to pass -m to find-debuginfo.sh when
_include_minidebuginfo has been set. find-debuginfo.sh now takes -m
as argument to generate the .gnu_debugdata ELF section to be added
to the main executable.
To support the testcases a new macros.debug is added that is used to
generate debuginfo packages in the rpmbuild.at testsuite.
The original support was added to Fedora rpm by Alexander Larsson.
Lubos Kardos fixed a bug in it when strip -g was used. I added some
configuration macros and two testcases to check the basic support works
and for the strip -g bug.
Signed-off-by: Mark Wielaard <mjw@redhat.com>
Florian Festi