Commit Graph

14064 Commits

Author SHA1 Message Date
Panu Matilainen cfa106ee08 Add testcases for unpackaged files and directories detection
Unpackaged files we detect, directories we dont, never have.
2017-09-19 09:39:36 +03:00
Panu Matilainen 90802a894a Fix Ftell() past 2GB on 32bit architectures (RhBug:1492587)
Back in 2011 "somebody" forgot to apply brain when copying the return
type of "long" from ftell() to the Ftell() implementations within rpmio
(commit 61f5838aa8).

Fast-forward six years and suddenly TexLive in Fedora no longer builds
on 32bit architectures due to that thinko, appearing to be a regression
in commit 7d1a303c45. However that only
exposes the inner flaw of Ftell() as the code now relies on values
past the initial header range, for which the 2G of "long" has been more
than enough on 32bit architectures too.

Doh, dude...
2017-09-18 16:28:06 +03:00
Panu Matilainen e3cd2bc85e Avoid multiple strlen() calls on the constant prefix string when searching 2017-09-13 13:05:02 +03:00
Panu Matilainen e760730738 Fix file triggers failing to match on some packages (MgBug:18797)
Directory names as stored in RPMTAG_DIRNAMES are not sorted when
separated from basenames, so binary search has no chance of working.
While a linear search on the dirnames would be guaranteed to find *some*
matches when they exist, it could still miss some results as the
matches are not guaranteed to exist in a neat low-high range.

Construct the entire pathname for prefix comparisons to ensure sorted
paths and adjust the file trigger testcase to cover this too.

Thanks to Pascal Terjan for initial investigations and Thierry
Vignaud for providing a workable reproducer.
2017-09-13 13:01:33 +03:00
Pavlina Moravcova Varekova 4adc4f198c Add documentation for %load macro 2017-09-11 14:23:02 +03:00
Panu Matilainen 7481f0ff15 Always calculate disk space info, only don't report them if filtered
In the distant past when rpm just stat()'ed all mounted filesystems,
it was quite necessary to have the option of disabling the fs stats.
However now that we only stat() filesystems we actually use, the
situation is quite the reverse: if stat() eg hangs, we wouldn't be
able to complete the transaction anyway so it's better to find that
out before the transaction even starts, regardless of problem filters.

Also the filesystem info sets could be used to track other things besides
space, such as suid/sgid availability, selinux and file caps support etc.
2017-09-11 10:32:17 +03:00
Panu Matilainen 9e65d6ee9a Don't sync filesystems unless we actually did something in the transaction
Non-zero transaction members is not a meaningful test for this, as we can
can fail early for several reasons. Only sync if we actually executed
rpmtsProcess().
2017-09-11 10:32:17 +03:00
Panu Matilainen cf4e631283 Inode zero is just a regular inode
Inspired by a similar change in glibc:
https://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=686f2ea18374a541d203cfcc0e1dfba1666f49c2
2017-09-09 09:45:45 +03:00
Michael Schroeder 9809319df2 Allow rpmrichParse to be called with no callback
This is useful if you want to just verify the syntax or find
the end of a rich dependency.
2017-09-07 11:47:25 +02:00
Panu Matilainen eef82b0e81 Less naive version of sync-after-transaction (RhBug:1461765)
If syncfs() is available (ie on Linux), only sync modified filesystems.
In order to do this, keep the diskspace information around throughout
the transaction.

Skip the sync entirely on chroot installations for now, but this
too should be configurable (always/auto/never or so).

There's a bit of a chicken-egg problem with post-transaction plugins:
for example systemd_inhibit should only be released after syncing,
but OTOH some other plugins might be performing actions whose results
should be syncing. Leaving it alone for now.
2017-09-07 12:01:39 +03:00
Panu Matilainen 4bb954086a Fix PYTHONPATH in test-suite when prefix differs from system python (#265)
Prior to this, test-suite PYTHONPATH would be wrong for all but builds
using --prefix equal to system python location, and eg --prefix=/opt
would cause the testsuite to fall back to system rpm bindings instead
of the in-tree one. Ditto for dist-check.
2017-09-06 17:35:36 +03:00
Panu Matilainen 5763f40c21 Enable python build during dist-check
The test-suite fails if python is not enabled. An alternative solution
could be disabling python tests when not enabled, but the python
tests cover things that are not covered elsewhere so especially
for dist cutting the tests are quite important.
2017-09-06 17:33:30 +03:00
Panu Matilainen 8db7862638 Fix false negatives on signature/digest tests inside "make distcheck"
distcheck runs from a read-only source tree so the packages that we
copy around for fuzzing "fuzz" have read-only permisssions and
modifications fail, causing the tests to fail. So instead copy the
to-be-modified packages from inside testing/data where the permissions
are taken care of by commit ecb4182655.
2017-09-06 14:14:20 +03:00
Panu Matilainen b7a869f0f3 Sync disks at the end of transactions (RhBug:1461765)
Bit hysterical that we haven't done this...
On Linux we could call syncfs() for only those filesystems that we
actually touched but somehow I doubt it's worth the trouble.

Another option might be doing this at rpmtxnEnd() but maybe that's
excessive.
2017-09-06 09:14:49 +03:00
Michael Schroeder 37982efbac Add support for 'unless' rich dependencies
An (A unless B) dependency implements (A and not(B)). This kind is useful
for "or" type dependencies, e.g. "Conflicts" or "Supplements".

As "Conflicts: (A unless B)" is equivalent to "Requires: (B if A)", I
thought this type is not needed. But there is no such equivalence
for Supplements, thus the change in mind.

Like with "if" we also have a syntactic sugar "else" flavor:
(A unless B else C) is the same as ((A unless B) or (B and C))

This commit also makes the "else" handling code in depends.c much
easier to understand.
2017-09-06 08:32:45 +03:00
Panu Matilainen a8abf2cfd0 Eliminate the rest of idiotic assert()'s in rpmtd.c
Finish what was started in commit 9c979cffe9,
with greetings to self ten years ago...
2017-09-05 13:13:49 +03:00
Panu Matilainen 80b5c12eaa Use pkg-config for figuring python cflags and libs
Simplifies things a bit and also makes "PYTHON=python3 ./configure" work,
whereas it previously barfed on figuring the library names like
"libpython3.6m"
2017-09-05 12:00:30 +03:00
Panu Matilainen 6d610e9b9a Always execute scriptlet callbacks with owning header (RhBug:1485389)
Triggers and file triggers can and do execute scriptlets from installed
packages which are not part of the current transaction and thus have no
associated transaction element, making the scriptlet callbacks
inconsistent and cumbersome for API users.

Create a fake transaction element for the poor orphan scriptlets lacking
one to make it consistent (of course, creating rpmte's with all their
associated data just to pass a header pointer along is ridiculously
expensive but *shrug*).

Regular triggers used to execute in the context of the triggering
transaction element, make them run with the owning trigger header too.
2017-09-04 17:39:16 +03:00
Panu Matilainen 61aef0f835 Fix %transfiletriggerpostun diagnostic showing "unknown"
The file trigger diagnostics are ambiguous with regular triggers,
but "triggerpostun" is much closer to the mark and easier to
debug than  "unknown".
2017-09-04 11:13:36 +03:00
Panu Matilainen 9b4bcd15a3 Limit "<package> has N files" debug diagnostics to install/erase goals
This was only really meant for install/erase goals, and is not really
meaningful for scriptlet goals which came much later in the history.
2017-09-04 11:11:45 +03:00
Panu Matilainen d6be2e4312 Mark ndb and lmdb experimental in configure too
Nobody reads release notes, better carry the message here...
2017-09-01 14:03:39 +03:00
Panu Matilainen 44570e253d Remove leftover parentheses, no functional change.
Should've been in commit 206c12acc2.
2017-09-01 11:17:48 +03:00
Panu Matilainen 206c12acc2 Don't assume %{quote:...} is surrounded by whitespace
As implemented in commit cdbc7e1d8b,
%{quote:...} only works when its used to quote the entire argument
and produces garbage when used in middle of other text, such as
"x%{quote:y}z". Not sure there's actual need to do that, but producing
garbage is never good.

Only unquoted separators can split an argument, copy argument manually
to strip any quote characters and adjust the math to match. Oh and
add testcases too.

Thanks to Pavlina Moravcova Varekova for pointing this out.
2017-09-01 10:59:21 +03:00
Panu Matilainen 1a08ab7e30 Add documentation for all/most built-in macros
This hasn't seen much love in recent decades, ugh...
2017-08-31 15:15:29 +03:00
Panu Matilainen 58213365e9 Plug what's probably an ancient memleak in build code 2017-08-31 13:25:02 +03:00
Panu Matilainen 0034a5c6e5 Avoid macro expansion on every rpmlog() call + fix a related memleak
Move color config macro expand into a helper function that's only called
once per thread and use enums for the possible states, ints are nicer
to compare than strings. Also remember to free the expansion result to plug
a memleak as well.
2017-08-31 13:25:02 +03:00
Michael Schroeder c69cfdf901 Fix classification of ELF binaries with both setuid/setgid set
There is a ',' between multiple entries.
2017-08-31 12:01:24 +02:00
Panu Matilainen cdbc7e1d8b Support parametric macro argument quoting, take three (#222)
All the nice quote-characters are already spoken for, we need to
do something more special here. Add a special-purpose built-in %{quote:...}
macro which quotes its argument using ASCII unit separator character 0x1f
(so it really shouldn't get into anybodys way) and teach macro argument
splitting to support that.

So with %{quote:...} it's now possible to pass strings containing
whitespace and empty strings as arguments. It might not be pretty, but
it's at least POSSIBLE, and no existing user is bothered by this.
2017-08-31 12:36:32 +03:00
Panu Matilainen 08ed0b20c6 Add a few more parametric macro argument tests 2017-08-31 12:14:36 +03:00
Panu Matilainen 6adef6a3c7 Make queryformat array behavior consistent for all types
Always require matching number of elements regardless of data type
and number of elements, and error out otherwise. Behaving differently
for plain strings and binaries makes no sense, those checks are ancient
artifacts that have long since ceased to produce what might've been
originally intended behavior.

A remaining quirk is the case where a tag doesn't exist at all, this
produces (none)'s all the way down. Which doesn't seem terribly wrong,
but neither does it seem really consistent with the rest of it. Dunno.
2017-08-30 16:57:09 +03:00
Panu Matilainen 1517c1a721 Add more queryformat array testcases
There are some funny quirks here: query 2 case needs to succeed
because there's exactly one changelog entry in the package, but if
that were not the case it should fail (but doesn't, so the testcase
fails for now). So we test for that separately in query 3, and finally
test for %{=foo} (ie repeat first entry) which is required to sort out
different sized arrays in the general case.
2017-08-30 16:48:30 +03:00
Panu Matilainen 7eb620f595 Restore 4.13 behavior with single value tags in query format arrays
Rpm documentation actually states that you must use %{=TAG} for the case
commit 7f47cbbd7d was trying to fix, but
caused regressions in other situations. So the 4.12 behavior is little
more than accidental, rpm is actually supposed to error out here
(but doesn't, which is another bug).

This reverts commits 7f47cbbd7d and
ead9cdd587.
2017-08-30 15:35:01 +03:00
Panu Matilainen 9c979cffe9 Eliminate a bunch of idiotic assert()'s in rpmtd getters
In the cases where zero is a sane return for non-existent data,
return it instead of barfing up an assert. Many more unnecessary
asserts left there but these are rather obvious...
2017-08-30 14:26:19 +03:00
Pavlina Moravcova Varekova d78813fa94 When printMacro() is used, print newline after "(empty)" macro 2017-08-30 10:37:01 +03:00
Pavlina Moravcova Varekova ed782b3653 Bring manual for %{echo:...}, %{warn:...} , %{error:...} up to date 2017-08-30 10:35:37 +03:00
Pavlina Moravcova Varekova fa39cb730d %trace mb states at the beginning/end of expandMacro must be equal
Macro %trace switched on in a nested stage level writes the trace
until the level is leaved to a lower stage level. This did not
work e.g. for macros containing several condition macros.

For example:
%prep
%define Branch1() {%trace %global Leaf1 "1"}
%define Branch2() {%global Leaf2 "2"}
%define Main() {%{?test1: %Branch1} %{?test1: %Branch2}}
%Main

with the result:
  4>         %global^Leaf1 "1"
  2>     %{?test1: %Branch2}^
  3>       %Branch2^
  4>         %global^Leaf2 "2"

It was because macro expansion is context free. This patch fixes it.
2017-08-30 10:33:14 +03:00
Panu Matilainen b052d69474 Only fallback to MDB_RDONLY if readonly database is requested
Somewhat similarly to commit 70a1efa52b,
only allow read-only environment if read-only is requested, writers
are better off failing early.
2017-08-28 19:33:40 +03:00
Panu Matilainen 2822ccbcdf Fallback to DB_PRIVATE on readonly DB_VERSION_MISMATCH too (RhBug:1465809)
All these years BDB has been relying on undefined behavior by storing
POSIX thread objects in its persistent environment files (for the long
story, see RhBug:1394862). As a side-effect of fixing it in BDB,
all sorts of creatures from dark corners are getting dragged into
the daylight: rawhide glibc gets updated a lot and we're getting
DB_VERSION_MISMATCH hits from rpm queries from package scriptlets
(told you so...), other tools not closing their database handles
when they should etc etc. This lets those cases continue "working"
to the extent they always did (ie unreliably) for now.

We should log some diagnostic message in this case, but coming up
with an understandable and reasonably short message for this mess
isn't that easy :)
2017-08-28 19:17:24 +03:00
Panu Matilainen 70a1efa52b Limit automatic fallback to DB_PRIVATE to read-only operations
Only permit automatic fallback to the essentially lockless operation
of DB_PRIVATE when read-only database is requested. This isn't exactly
correct, as readers need locks for correct operation just like writers do,
but at least in the readonly case the database wont be damaged.

The exception to the rule is systems which don't support the shared mapping
at all so we don't have much choice. Explicitly configured
I-know-what-I'm-doing DB_PRIVATE is not affected either.
2017-08-28 19:17:08 +03:00
Igor Gnatenko 98d0a51e16 metainfo.prov: scan /usr/share/metainfo and /usr/share/appdata for both types
Directory name doesn't matter, content does. In specification, both
appdata.xml and metainfo.xml should be in /usr/share/metainfo, but
tools should support /usr/share/appdata as for legacy location.

References: https://bugzilla.redhat.com/show_bug.cgi?id=1483644
Reported-by: Petr Pisar <ppisar@redhat.com>
Signed-off-by: Igor Gnatenko <i.gnatenko.brain@gmail.com>
2017-08-25 12:51:28 +02:00
Panu Matilainen eadee821b3 Export rpmsqBlock() to python bindings as blockSignals() 2017-08-23 13:02:24 +03:00
Panu Matilainen ee33773218 Leave build scripts around in debug mode (#1442015)
We do this for install-time scripts too, so its only logical to do
so with build scripts as well.
Depends on commit 724830018a.
2017-08-23 11:52:50 +03:00
Panu Matilainen 724830018a Allow running rpmbuild with debug verbosity
Only force RPMLOG_INFO level if running with less verbosity (as is
the case by default), but leave RPMLOG_DEBUG alone. Quiet overrides
any number of -v's though.
2017-08-23 11:50:11 +03:00
Panu Matilainen 4e647cd979 Eliminate unnecessary "delete" argument to freeArgs()
This became unused and useless with the big scoping change in
commit 237f35f16f. No functional
changes here.
2017-08-22 10:25:56 +03:00
Panu Matilainen 609adaa758 Bump macro max recursion limit up
Fedora java packages are running into the old limit of sixteen nesting
levels (commit b3179e6de3), bump it up
to 64. The old limit lasted almost twenty years, lets see how far this
gets us.
2017-08-21 16:36:32 +03:00
Panu Matilainen 2986e7659d Remove redundant max_macro_depth initialization
Oldies but goodies: the max depth is already initialized in declaration
to _MAX_MACRO_DEPTH which has been the same 16 since late nineties. This
wasn't always the case of course...
2017-08-18 17:12:47 +03:00
Panu Matilainen 054de0f50f Use silent lookup for debuginfo packages (#1482144)
Noarch packages do not have debuginfo, this was causing harmless
but bogus error messages via lookupPackage(). Depends on commit
4afe09cbcf.
2017-08-18 12:46:59 +03:00
Panu Matilainen 4afe09cbcf Add a flag to allow quiet test for package existence with lookupPackage()
Turning "flag" into an actual bitfield requires testing for
PART_NAME/PART_SUBNAME differently, no actual changes here though.
2017-08-18 12:44:46 +03:00
Pavlina Moravcova Varekova 94e8cd6058 Make macro %{error:...} cause an actual failure 2017-08-17 16:16:23 +03:00
Pavlina Moravcova Varekova a3b153eb47 Unified messages printed by %{echo:...}, %{warn:...}, and %{error:...}
Send all through rpmlog(), unify formatting - previously echo and
warn were missing the trailing newline. The other change is that
echo now goes to stdout.
2017-08-17 16:15:39 +03:00