These are very different from the other built-ins as these do their
own parsing and return a continue address, so we need a separate
prototype for them. Having two pointers for each macro makes the table
a bit ugly but the alternative of tracking type by some other means
would lose type-safety. Other than all the boilerplate needed, this is
remarkably straightforward. No functional changes.
All built-in macros are now invoked via the lookup-table, which means
that adding new built-ins just got more straightforward and less ad hocey
pokey.
%trace differs from the others as it doesn't take any arguments,
it just silently eats any passed. Not that it really matters here.
No functional changes.
Annoyingly the amount of other boilerplate makes this slightly more code
than the original, but then saner code isn't always smaller, and it does
make expandMacros() smaller, which can only be a good thing.
No functional changes.
Handle log level determination inside doOutput() to make the interface
match with the others and as a bonus, make expandMacros() that little
bit smaller again. Empty argument needs to be handled differently
but shouldn't affect functionality.
No functional changes. chkexist and negate arguments are unused for
now, but they could perhaps be used for error condition handling later
(similar to %load)
Replace the "necessary but clunky" string comparisons in expandMacro()
with a lookup on the builtins. For starters, only the primitives
handled in doFoo() are converted, others will require varying amount
of further changes. chkexist argument is unused by doFoo() but will be
needed later for others, add now to avoid having to change more later.
No functional changes intended, anything like that would simply be a bug.
For current uses this shouldn't make any difference, but much of the
macro engine works with string lengths so to avoid having to create
\0-terminated versions of all strings we might look at...
There's an increasing number of placing wanting to know the number of
CPU's for parallel processing, and increasingly these things are running
in containers and such where the total host CPU count is not meaningful.
The algorithm that detects the end of a macro counts for all macro chars their level of nested. It iteratively moves from the first char of the macro further.
Before the patch the rpm function does not implement this algorithm correctly. After finishing with the last char of a macro line, it additionally skips one char of the next line. The first char in the line was omitted.
Probably affects every rpm version from this millenium.
It's much easier for rpm to point out the location of invalid macro
definitions than it is for humans to grep all the places a given rpm
version might look at. Log the macro file path once per file in case
of failures, and additionally return and error if all definitions fail.
Based on patch by Pavlina Moravcova Varekova.
Prior to this patch if a name of a newly created macro was the same as a built-in macro name, the macro looked correctly defined. But the newly defined macro could not be used.
It is obvious that a built-in macro can not be undefined.
Prior to this patch %{verbose:...} argument was always expanded, eg:
rpm -v --eval '%{verbose:%{echo:is verbose}}' \
--eval '%{!verbose:%{echo:is not verbose}}'
is verbose
is not verbose
Obviously both cannot be true.
Guard the whole mbAppendStr function with ifdef WITH_LUA to avoid a
warning when lua support isn't enabled because it is only used inside
the doLua function (which is empty unles WITH_LUA is defined).
Signed-off-by: Mark Wielaard <mark@klomp.org>
As implemented in commit cdbc7e1d8b,
%{quote:...} only works when its used to quote the entire argument
and produces garbage when used in middle of other text, such as
"x%{quote:y}z". Not sure there's actual need to do that, but producing
garbage is never good.
Only unquoted separators can split an argument, copy argument manually
to strip any quote characters and adjust the math to match. Oh and
add testcases too.
Thanks to Pavlina Moravcova Varekova for pointing this out.
All the nice quote-characters are already spoken for, we need to
do something more special here. Add a special-purpose built-in %{quote:...}
macro which quotes its argument using ASCII unit separator character 0x1f
(so it really shouldn't get into anybodys way) and teach macro argument
splitting to support that.
So with %{quote:...} it's now possible to pass strings containing
whitespace and empty strings as arguments. It might not be pretty, but
it's at least POSSIBLE, and no existing user is bothered by this.
Macro %trace switched on in a nested stage level writes the trace
until the level is leaved to a lower stage level. This did not
work e.g. for macros containing several condition macros.
For example:
%prep
%define Branch1() {%trace %global Leaf1 "1"}
%define Branch2() {%global Leaf2 "2"}
%define Main() {%{?test1: %Branch1} %{?test1: %Branch2}}
%Main
with the result:
4> %global^Leaf1 "1"
2> %{?test1: %Branch2}^
3> %Branch2^
4> %global^Leaf2 "2"
It was because macro expansion is context free. This patch fixes it.
Fedora java packages are running into the old limit of sixteen nesting
levels (commit b3179e6de3), bump it up
to 64. The old limit lasted almost twenty years, lets see how far this
gets us.
Oldies but goodies: the max depth is already initialized in declaration
to _MAX_MACRO_DEPTH which has been the same 16 since late nineties. This
wasn't always the case of course...
Send all through rpmlog(), unify formatting - previously echo and
warn were missing the trailing newline. The other change is that
echo now goes to stdout.
This simply breaks too many things - whole macro ecosystems exist based
on the assumption that quotes in arguments will pass to macros
untouched. Macro argument quoting support is necessary but it'll
need some entirely different approach that is either opt-in or
based on a different syntax altogether.
This reverts commit 47f0a899ef.
Some of scoping level values are defined in rpmmacros.h:
#define RMIL_DEFAULT -15
#define RMIL_MACROFILES -13
...
#define RMIL_GLOBAL 0
Before this patch RPM set these values decreased of 1 (for all macros).
Because of the (level - 1) in
pushMacro(dst, me->name, me->opts, me->body, (level - 1), me->flags);
in function copyMacros scoping level for some CMDLINE macros was decreased of 1.
Because of the (level - 1) in
pushMacro(mb->mc, n, o, b, (level - 1), ME_NONE);
in function doDefine scoping level values of all other macro types and
some CMDLINE macros was decreased of 1.
The patch shifts scoping level values back to the numbers defined
in rpmmacros.h.
This was spotted when investigating a regression introduced in
commit 237f35f16f where locally scoped
scoped macros did not get deleted on scope closing.
This also fixes the regression in question.
The first test in this patch tests automatic deleting of %defined macros.
The second test checks whether the behavior changed by
237f35f16f is still working correctly.
Thanks to Panu Matilainen for investigation and proposal of the main part
of the patch!
Single or double quotes can be used in macro arguments
as expected - they denote the begin and the end of the argument.
Example with 2-nd and 3-rd argument in double quotes:
./rpm --define "%foo() 1:%1 2:%2 3:%3" \
--eval '%foo Next_argument_will_be_empty "" "Last argument"'
1:Next_argument_will_be_empty 2: 3:Last argument
Example with all arguments in single quotes,
without spaces between arguments:
./rpm --define "%foo() 1:%1 2:%2 3:%3" \
--eval "%foo 'Next argument will be empty''''Last argument'"
1:Next argument will be empty 2: 3:Last argument
This essentially reverts commit 9ae7d1df31.
Leaving the testcase in place, just adjusting it to the expected
behavior where arguments that expand to empty are eaten away.
This actually matches what the shell does, so it follows the path of
least surprise in that sense - we emulate shell behavior with the
arguments etc already in many ways. What the shell has and rpm
has not is a way to pass those empty arguments by quoting. So
now we need a way to quote arguments - not just for passing empty
arguments but arguments containing whitespaces too.
There needs to be a way to pass escaped macros names (and other strings
containging %%) as parametric macro arguments. Add an internal mode
to preserve %% when expanding the arguments. Fixes a regression introduced
in commit 5adc56897b.
Thanks to Michael Schroeder for pointing this out!
This makes makes the behavior much more sensible eg in case an
argument expands to an empty strings, such as:
$ rpm --define '%foo() 1:%1 2:%2' --eval '%foo %nil bar'
After commit 5adc56897b rpm was
evaluating that to "1:bar 2:%2" which is hardly what you'd expect.
Thanks to Michael Schroeder for pointing this out! (#217)
This too is quite a fundamental change for macros: up to now, arguments
to parametric macros have not been expanded. It doesn't sound so bad
until you consider something like the case in RhBug:1397209:
%global rev 133
...
%setup %{?rev:-c}
%autosetup %{?rev:-c}
One would expect %setup and %autosetup behave the same when you replace
one with the other, but no. %setup gets "called" with -c, %autosetup
does not. Instead %autosetup receives a completely useless, literal
"%{?rev:-c}" as its argument. That's just brain-meltingly non-sensical.
This certainly has the potential to break advanced macro constructs,
but OTOH what breaks might well be only written that way in order to
work around the former behavior.
There are some funny subtleties involved as the argument expansion
must occur in the callers scope, ie before we create any of the
automatic macros. For example, Fedora's font packages break if only
this or the macro scope visibility enforcement is applied but start
working again once both are present.
When a parametric macro "calls" another parametric macro with fewer
arguments than it received, the inner macro would see the %<n>
macros of the outer call which is an obvious scoping violation
and quirky behavior, making macro writing harder than it needs be.
Similar scoping issues exist for manually defined macros but those
have further complications (because of %undefine semantics) that we
sheepishly avoid here by limiting the visibility enforcing to automatic
macros only (for now at least).
This changes the macro scoping rules quite fundamentally: macro
definitions are local to the parametric macro they were defined in,
and everything else is global. Among other things, this makes this
common spec idiom (RhBug:552944, RhBug:551971 etc) behave
deterministically because "foo" is placed into global scope:
%{?!foo: %define foo bar}
In theory it's certainly possible that this breaks somebodys carefully
crafted advanced macros but it seems quite unlikely, considering how
broken the alleged block-scoping has been. OTOH for macros defined
within parametric macros, nothing actually changes as that scoping has
always been enforced by rpm. The non-global define tracking is also
no longer needed for emitting warnings, because the case where it
was needed simply no longer exists.
Note that macro recursion depth is a different thing and still needs
to be tracked separately.
Since we actually setup all the same automatic macros whether there
are arguments or not, doing it centrally only makes sense. Shuffle
things around a bit in preparation for the next steps.
The missing space style-error has been recently coming common enough
somebody might think that IS the expected style, its not. Some of these
are actually very old, but fix across the board for consistency..
Strictly white-space only change.
The whole point of %trace is that some macro is being evaluated in a
surprising way, and you want to know what it is. This is often the
result of an unfortunately complex macro with difficult to see
intermediate results.
Shortening the output because it doesn't fit past 61 or so characters
does not help, and we have big screens these days. Don't do it.
Signed-off-by: Peter Jones <pjones@redhat.com>
In %trace mode, evaluating a macro which is undefined causes an invalid
read of 1 byte when searching for the end of the string:
trillian:~$ valgrind rpmspec --eval '%trace' --eval '%{?myUndefinedMacro}'
==21534== Memcheck, a memory error detector
==21534== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==21534== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright info
==21534== Command: rpmspec --trace --eval %{?myUndefinedMacro}
==21534==
1> %{?myUndefinedMacro}^==21534== Invalid read of size 1
==21534== at 0x55018D4: printMacro (macro.c:296)
==21534== by 0x5502DFC: expandMacro (macro.c:1077)
==21534== by 0x5503710: doExpandMacros (macro.c:1280)
==21534== by 0x5504AB6: rpmExpand (macro.c:1629)
==21534== by 0x508F59A: rpmcliAllArgCallback (poptALL.c:120)
==21534== by 0x6DAF71D: invokeCallbacksOPTION (popt.c:156)
==21534== by 0x6DAF75B: invokeCallbacksOPTION (popt.c:139)
==21534== by 0x6DB1428: poptGetNextOpt (popt.c:1515)
==21534== by 0x508F912: rpmcliInit (poptALL.c:302)
==21534== by 0x1095B2: main (rpmspec.c:63)
==21534== Address 0x8a010f3 is 0 bytes after a block of size 19 alloc'd
==21534== at 0x4C2DB9D: malloc (vg_replace_malloc.c:299)
==21534== by 0x5507C17: rmalloc (rpmmalloc.c:44)
==21534== by 0x5502788: expandMacro (macro.c:927)
==21534== by 0x5503710: doExpandMacros (macro.c:1280)
==21534== by 0x5504AB6: rpmExpand (macro.c:1629)
==21534== by 0x508F59A: rpmcliAllArgCallback (poptALL.c:120)
==21534== by 0x6DAF71D: invokeCallbacksOPTION (popt.c:156)
==21534== by 0x6DAF75B: invokeCallbacksOPTION (popt.c:139)
==21534== by 0x6DB1428: poptGetNextOpt (popt.c:1515)
==21534== by 0x508F912: rpmcliInit (poptALL.c:302)
==21534== by 0x1095B2: main (rpmspec.c:63)
==21534==
1> %{?_transaction_color}^
1> %{?_prefer_color}^
1> %{_netsharedpath}^
1> %{_install_langs}^
==21534==
==21534== HEAP SUMMARY:
==21534== in use at exit: 7,183 bytes in 71 blocks
==21534== total heap usage: 7,811 allocs, 7,740 frees, 3,500,361 bytes allocated
==21534==
==21534== LEAK SUMMARY:
==21534== definitely lost: 19 bytes in 1 blocks
==21534== indirectly lost: 0 bytes in 0 blocks
==21534== possibly lost: 0 bytes in 0 blocks
==21534== still reachable: 7,164 bytes in 70 blocks
==21534== suppressed: 0 bytes in 0 blocks
==21534== Rerun with --leak-check=full to see details of leaked memory
==21534==
==21534== For counts of detected and suppressed errors, rerun with: -v
==21534== ERROR SUMMARY: 5 errors from 1 contexts (suppressed: 0 from 0)
trillian:~$
This can easily be avoided by checking the first byte as well as the
second for our sentinal value (NUL).
Signed-off-by: Peter Jones <pjones@redhat.com>
When a %{lua:...} macro calls rpm.expand(), it re-enters the macro
environment at depth zero regardless of what the "calling" nesting
level was. This could lead to wrong macros getting mopped out on
complex, nested lua + native macro calls.
Store the depth in global macro context and pick up the initial value
from there to keep the nesting level increasing on when entering %{lua:...}
space.
These haven't been in any public release yet so there's time to
fiddle with stuff like this. Both always succeed currently so no
change in that respect but it's better to have room for expansion...
This WILL cause behavior changes in cases where there have been
illegal macro names etc which have previously been more or less
ignored with an logged error, but now they will actually abort.
Panu Matilainen