Commit Graph

15652 Commits

Author SHA1 Message Date
Panu Matilainen da3f141656 Fix rpmspec build issue introduced in 03b3df0ce0 2021-12-20 15:52:01 +02:00
Panu Matilainen 03b3df0ce0 Add an interactive macro shell mode to rpmspec
Handy for debugging and experimenting with macros, in and out of
spec context.

Placed in rpmspec because we don't want readline dependency on main rpm
executable, this is more of a packager tool anyway.
2021-12-20 13:26:23 +02:00
Panu Matilainen ec1a245937 Refactor looping over parsed spec into helper function
No functional changes, pre-requisites for the next step.
2021-12-20 12:43:21 +02:00
Panu Matilainen bc436bf8bb Further mimic standalone Lua interpreter behavior in rpmlua cli
The real Lua interpreter enters interactive mode if no arguments are
passed at all, iff execute was not specified, or explicitly requested.
We have no reason to behave any differently.
2021-12-20 12:31:20 +02:00
Panu Matilainen 298bb60a9f Add optional readline history support in rpmlua interactive mode
We don't want readline dependency in librpmio so need to do this the
hard way: add an optional callback through which rpmlua can supply
it's own readline-aware callback function.
2021-12-20 12:19:07 +02:00
Panu Matilainen 59ef11f849 Allow executing statement and script in the same rpmlua run
Apparently this is needed to support iLua, whatever that is. Anyway,
there's no reason not to allow this, just didn't think this would be
needed.
2021-12-20 10:08:13 +02:00
Panu Matilainen d5606c0b06 Fix rpmlua -i without arguments, sigh
A last-minute change broke -i when no other arguments are present.
The thing you can't test... sigh.
2021-12-17 15:23:01 +02:00
Panu Matilainen bf58cfd08f Add rpmlua wrapper command for running our Lua interpreter standalone
Being able to run stuff easily in rpm context helps developing and debugging
scriptlets and macros too. Supports running one-liner statements from
the cli, regular scripts and an interactive session.

This is placed into a separate executable for, well, separation and
simplicity, but it'll also give us means to link to readline without
dragging that to main rpm dependencies (but that's left for later).
2021-12-17 14:57:14 +02:00
Panu Matilainen 6728000344 Fix out of tree build regression wrt man page generation
Let make figure the actual paths, it seems to handle this just fine
once we stop meddling with it's affairs.

Fixes: #1851
2021-12-16 13:55:35 +02:00
Panu Matilainen b45fde60f9 Clarify behavior of conditionals on built-in macros in the docs 2021-12-16 12:50:03 +02:00
Panu Matilainen f993a6067b Only expose rpm.next_file() when there actually is one we can call
rpm.next_file() is only meaningful at all when there's a backing
function set for the scriptlet, so only expose it at those times.
We shouldn't be touching the rpm namespace at all here, but
it's required for backwards compatibility.
2021-12-15 16:57:58 +02:00
Panu Matilainen 12af9c0bf1 Allocate nextFileFunc dynamically when used
As the nextFileFunc is something that often is not there, this makes
testing for nextFileFunc presence a little more obvious and easy.
No actual functional change.
2021-12-15 16:57:58 +02:00
Panu Matilainen 523da38e4f Eliminate rpm.next_file() from the permanent Lua API
This screams for a closure, use one. This is not supposed to change
any functionality except, except only make the rpm.next_file() function
available in the context of scriptlets which is the only context it is
meaningful.
2021-12-15 16:57:58 +02:00
Panu Matilainen 81e855aeb1 Kick out outdated section on builtin macros
Builtins aren't special in >= 4.17 and the less the old special syntaxes
are documented the better.

While at it, point users to expressions and Lua for more complicated tests.

Fixes: #1864
2021-12-15 16:52:53 +02:00
Panu Matilainen 3d46bc95c5 Further improve build warning/error summary
Suppress summary output entirely on --quiet where the summary is wholly
redundant, and non-sensical (as witnessed by the test-case outputs)
at that because the summary title is omitted due to its NOTICE log level.

A build may have warnings without having any errors, lift the warnings
summary out of the rc-condition.

Finally, condense the output a bit, two empty lines at front seems a bit
excessive.
2021-12-15 12:40:53 +02:00
Michal Domonkos 7fd7a60ea2 Separate build warnings from error summary
Confusingly, the "RPM build errors" section also includes messages
logged as warnings.  That gives the false impression that they somehow
contributed to the actual build failure and therefore were turned into
errors.

This appears to be a historical artifact; when a message passes through
the logging system and is of the priority RPMLOG_WARNING or higher, we
save it in a global buffer (ctx->recs), which is then simply dumped with
rpmlogPrint() in the error summary.  This was probably good enough when
the summary was introduced (commit f2efc72, year 2000), as there were
almost no warnings generated by RPM at that time, however as they became
more abundant, the summary code was never revisited.

There are 3 ways to fix this discrepancy:

 1) Change the summary's title to "RPM build problems"
 2) Remove the summary altogether
 3) Don't show warnings in it

Options #1 and #2 would be too disruptive.  The error summary needs to
stay as is, for the following reasons:

 - While it usually just repeats the last error, not all errors
   terminate a build right away, so those can get drown in the output
   that follows after.  Examples: "File not found" in rpmInstall(), or
   macro expansion errors (note: we may have these terminate a build in
   the future).

 - It makes it immediately obvious that something went wrong when
   examining build logs, and the title "RPM build errors" undoubtedly
   has become the de-facto text string to search for.

That leaves us with option #3.  To further lessen the disruption, do
keep a summary of warnings, but put them under their own heading, and
only show it on build failures.  That way, we restrict all the extra
verbosity to error time (as it is now) and don't pollute the output of
otherwise good builds.  There may be packages with long-standing
warnings that are not feasible to fix for any reason, and having an
indented block of text resembling "RPM build errors" at the end of every
build would do no good.

Effectively, this commit is just cosmetic - it splits the error summary
into two, without any functional or API changes (apart from two function
additions).

Fixes: #793
2021-12-15 11:53:54 +02:00
Panu Matilainen 845364186d Limit maximum array size in the header to about one million
This is mainly to place *some* limit on number of files a package may
have. It's unlikely that rpm scales to million files in a package but
that much may actually work, whereas closer to int32 limits we run into
all manner of issues with internal types mismatching and whatnot.
2021-12-15 09:48:50 +02:00
Miro Hrončok 16f01c251c Fix invalid %autopatch example 2021-12-15 09:48:30 +02:00
Panu Matilainen 31e9daf823 Really fix IMA signature lengths assumed constant (#1833, RhBug:2018937)
Commit a79d7ae0f0 had two major flaws:

Firstly, it negletted to update the part that copies the data in place to
match the new layout where data lengths are not always equal, and the for
loop would overshoot its bounds on data shorter than maximum.

Secondly, rpmfilesFSignature() would now crash on packages with no IMA
signatures because fi->signaturelengths is not allocated. Take care not
to change API behavior wrt *len return value: set to zero if no
signatures are present.
2021-12-13 16:31:57 +02:00
Panu Matilainen 3955a445df Fix silly typo/thinko in commit 7b3a3f004f 2021-12-13 13:01:31 +02:00
Panu Matilainen cd6317ba61 Auto-print returned values from Lua macros
Avoids having to manually "print()" everything, at the cost of requiring
a "return" which is not really less to type but more natural for
programming, and other benefits likely to follow.
2021-12-13 10:56:14 +02:00
Florian Festi 7b3a3f004f Use root as default UID_0_USER and UID_0_GROUP
If /etc/passwd or /etc/group was not available during building rpm itself
these ended up empty. This affects builds done later on using rpmbuild.

Resolves: #1838
2021-12-13 10:53:44 +02:00
Tomohiro KATO d3ad33bacd Translated using Weblate (Japanese)
Currently translated at 94.2% (835 of 886 strings)

Translation: rpm/master
Translate-URL: https://translate.fedoraproject.org/projects/rpm/master/ja/
2021-12-13 10:48:47 +02:00
weidong 177479c3d8 Translated using Weblate (Chinese (Simplified) (zh_CN))
Currently translated at 83.5% (740 of 886 strings)

Translation: rpm/master
Translate-URL: https://translate.fedoraproject.org/projects/rpm/master/zh_CN/
2021-12-13 10:48:47 +02:00
Ettore Atalan 5115eddb22 Translated using Weblate (German)
Currently translated at 91.9% (815 of 886 strings)

Translation: rpm/master
Translate-URL: https://translate.fedoraproject.org/projects/rpm/master/de/
2021-12-13 10:48:47 +02:00
Bishop Clark d30e7dfaeb Documentation grammar fixes - apostrophes are not used for pluralization 2021-12-13 10:46:33 +02:00
Panu Matilainen a79d7ae0f0 Fix IMA signature lengths assumed constant (#1833, RhBug:2018937)
At least ECDSA and RSA signatures can vary in length, but the IMA code
assumes constant lengths and thus may either place invalid signatures on
disk from either truncating or overshooting, and segfault if the stars are
just so.

Luckily the signatures are stored as strings so we can calculate the
actual lengths at runtime and ignore the stored constant length info.
Extend hex2bin() to optionally calculate the lengths and maximum,
and use these for returning IMA data from the rpmfi(les) API.

Additionally update the signing code to store the largest IMA signature
length rather than what happened to be last to be on the safe side.
We can't rely on this value due to invalid packages being out there,
but then we need to calculate the lengths on rpmfiles populate so there's
not a lot to gain anyhow.

Fixes: #1833
2021-12-13 10:38:36 +02:00
Panu Matilainen 0c1ad364d6 Add Python bindings for rpmfilesFSignature() and rpmfilesVSignature()
Only, use more descriptive names than the C-side counterparts.
Python has nice facilities for dealing with binary data so return it
as such rather than converting to hex.
2021-12-02 10:39:38 +02:00
Panu Matilainen 0df9abd174 Fixup one missed adjustment to INITSTATE() callers
Guess this is rebase considered harmful: this should've been in
commit de9cc32b98 but in that fork,
rpmluaCallStringFunction() didn't yet exist so it couldn't be
adjusted there, but merging the PR broke the build.
2021-11-26 14:00:47 +02:00
Michael Schroeder b71495c967 Simplify opt,arg setup in lua macros
No need to do select() trickery.
2021-11-26 13:48:47 +02:00
Michael Schroeder 43e89bf74b Reinstantiate print buffer after poping it in lua interactive mode
We pop it to print its contents, but we forgot to push an empty
print buffer afterwards. This makes a difference as a newline
is not automatically added in print statements if a buffer is used.
2021-11-26 13:48:47 +02:00
Michael Schroeder 12c7fe7daa Fix indentation of rpmluaReadline and _rpmluaInteractive
No functual changes
2021-11-26 13:48:47 +02:00
Michael Schroeder de9cc32b98 Simplify INITSTATE macro
There's no need to keep the passed lua parameter as _lua.
2021-11-26 13:48:47 +02:00
Michael Schroeder fb70d0b7ec Drop lua parameter in addLuaSource function
The single caller uses the global lua state, so we might as
well get the state in the addLuaSource function.
2021-11-26 13:48:47 +02:00
Michael Schroeder 3698510144 Drop unused rpmluaRegister/rpmluaGetData/rpmluaSetData functions
They are no longer needed as of the lua binding consolidation
done in commit f1e721ccfc.
2021-11-26 13:48:47 +02:00
Thierry Vignaud ac107acdd5 Make --help clearer
Currently 'rpmspec --help' is confusing regarding --srpm &&
--builtrpms options.
This fixes that as well as making --help matches the man page.
2021-11-25 17:26:32 +01:00
Panu Matilainen 8338fe60eb Update INSTALL to reflect dropped Python 2 support
Should've been in commit 67f8f2b01d
2021-11-23 15:50:57 +02:00
Benno Rice 71456f2fc0 Allow an optional "override clock" for deterministic timestamps
When trying to achieve a fully reproducible build process for an
OS image timestamps are a major source of variance. The RPM
database contains two fields in package header records that
are relevant here:

- RPMTAG_INSTALLTIME which is an explicit timestamp indicating
  when the package was installed.
- RPMTAG_INSTALLTID which is an opaque value represending the
  transaction ID under which a package was installed but in terms
  of internal implementation is also a timestamp.

This change allows the presence of the SOURCE_DATE_EPOCH
environment variable, commonly used to override timestamps in
build systems, to institute an "override time" mode in which
values for RPMTAG_INSTALLTID and RPMTAG_INSTALLTIME become
predicatble.
2021-11-23 10:53:04 +02:00
Michael Schroeder ed9fccf7c0 Use lua_pushinteger in the hook conversion of type 'i' 2021-11-22 16:35:56 +02:00
Michael Schroeder ea2596b57d Fix off-by-one error in rpm_call
This seens to have been wrong since the first commit of this feature!
Add a testcase that tests the rpmhook functionality.
2021-11-22 16:35:56 +02:00
Michael Schroeder a34bf5bdf6 Switch the floating point type in rpmhook from float to double
There's no real reason why it should be float. Plus, the test if
the number is an integer does not work for big integers that
do not fit into a float.
2021-11-22 16:35:56 +02:00
Panu Matilainen b3d672a552 Fix spurious %transfiletriggerpostun execution (RhBug:2023311)
If a package has multiple %transfiletriggerpostun triggers, any one
of them matching would cause all of them to run, due to disconnect
in the intel gathering stage: we'd gather all the headers with matching
files into a lump, and then add any postun triggers found in them,
but this loses the triggering file information and causes all postuns
to run.

The triggers need to be added while looping over the file matches,
like runFileTriggers() does. Doing so actually simplifies the code.
These should really be unified to use the same code, but leaving
that exercise to another rainy day.
2021-11-22 13:05:39 +02:00
Panu Matilainen fff65e34f0 Restore scriptlet arguments passed as numbers to Lua scriptlets
Commit 717a3f7ecf changed Lua scriptlet
arguments from numbers to strings, and claimed that Lua will handle
this automatically so nobody will notice a thing. Not so, Lua only
converts automatically when doing so is unambiguous, such as doing
math. In comparison Lua cannot know which kind of comparison we want,
and rightly refuses to convert. Thus, the commit broke any Lua
scriptlets doing `if arg[2] > 1 then ... end` style comparisons.

As the whole rpmluav variable API is gone now, we can't just revert
the commit. Adding a preamble to the scriptlet to convert the args
to numbers may be hackish, but its a far lesser evil than bringing the
internal variable API back. Kudos to Michael Schroeder for the idea.

Fixes: #1790
2021-11-22 13:01:13 +02:00
Michael Schroeder d30b9a1611 Make rpmluaCallStringFunction more robust
Doing the function lookup with calls to lua_getfield is dangerous
because the lookup might trigger an __index function that may
contain an error. This results in the lua library calling abort()
as no exception handler is set.

So change the code to do the lookup via a pcall(). Also do the
string conversion with a pcall() because tostring is also not
safe.
2021-11-22 13:00:46 +02:00
Michael Schroeder bc126b41e8 Allow to call lua functions from the expresion parser
Lua functions start with a "lua:" prefix. The result of the
function call is always stringified.
2021-11-19 12:33:23 +02:00
Michael Schroeder 596e1ab81a Add rpmluaCallStringFunction() allowing to call a generic lua function
We reuse the rpmhook mechanism to pass the arguments to the
function. The function result is always stringified.
2021-11-19 12:33:23 +02:00
Michael Schroeder df4fbef531 Refactor arg handling in rpmluaHook
Split the individual arg setting/getting code into two
new function so that they can be reused.
2021-11-19 12:33:23 +02:00
Michael Schroeder b09a98c8c1 Add support for functions to the expression parser
We don't define any function yet but simply return an error.
2021-11-19 12:33:23 +02:00
Panu Matilainen d41143cb5f Simplify rpm_print(), fixing a Lua stack leak as a bonus
Rather than laborously call tostring() in Lua, use the C-side equivalent
of luaL_tostring(). This was new as of Lua 5.2, which explains why the
original version from 2004 did things the hard way.

Also fixes a stack leak from not popping "tostring" function after use.
2021-11-18 13:38:20 +02:00
Panu Matilainen 3aa1abc4ca Fix some Lua stack leaks in our initialization code 2021-11-18 11:53:17 +02:00