posix.redirect2null() is a decidedly rpm-specific, it should've really
always been rpm.redirect2null(). Move the code to rpmlua.c and make
it available in the rpm-namespace, but to preserve script compatibility
add it to the posix module too.
One step closer to unforking the posix extensions which is more or less
a pre-requisite for ticket #165.
Replace long since deprecated luaL_openlib() with modern counterparts
luaL_newlib() and luaL_setfuncs() and use luaL_requiref() for
loading our modules (these changes needed to go hand in hand, otherwise
it blows up every which way). Because these functions are only available
in Lua >= 5.2, this means we can drop the other compat checks as well,
just update the documenation for the new version requirement.
This also means that rpm now works on a modern Lua (tested with 5.3.5)
without requiring any compat modules and defines to be present in Lua. Whee!
Patch based on work done by @daurnimator in PR #169.
Resolves: #166
It was always an ugly hack and wrong to be in the posix module to
begin with, this makes the initialization and all much saner. And
is actually less code too.
Having the rpm-specific fork-tracking variable accessible only inside
the posix module forces us to put stuff in there that just doesn't
belong at all, such as os.exit() override. Moving it to librpmio
allows us to move such things to rpmlua.c where they belong.
Exporting integers like this is not exactly a good library design,
but this is supposed to be just a temporary crutch to get the other
pieces in place before changing the actual mechanism.
Eliminates the need for accessing what are supposed to be private
rpmlua fields from the outside. This does sacrifice the "perfect"
wrapping in rpmlua.h because it now needs to include lauxlib.h,
but defining our own struct just to avoid having to include a file
we'll be including in the code anyway doesn't seem worth it...
No functional changes.
There are any number of things that can go wrong and affect errno
inside rpmlog(), but having a function that by its declaration cannot
fail messing with errno is pretty dumb. Always save and restore
errno from rpmlog().
These are very different from the other built-ins as these do their
own parsing and return a continue address, so we need a separate
prototype for them. Having two pointers for each macro makes the table
a bit ugly but the alternative of tracking type by some other means
would lose type-safety. Other than all the boilerplate needed, this is
remarkably straightforward. No functional changes.
All built-in macros are now invoked via the lookup-table, which means
that adding new built-ins just got more straightforward and less ad hocey
pokey.
%trace differs from the others as it doesn't take any arguments,
it just silently eats any passed. Not that it really matters here.
No functional changes.
Annoyingly the amount of other boilerplate makes this slightly more code
than the original, but then saner code isn't always smaller, and it does
make expandMacros() smaller, which can only be a good thing.
No functional changes.
Handle log level determination inside doOutput() to make the interface
match with the others and as a bonus, make expandMacros() that little
bit smaller again. Empty argument needs to be handled differently
but shouldn't affect functionality.
No functional changes. chkexist and negate arguments are unused for
now, but they could perhaps be used for error condition handling later
(similar to %load)
Replace the "necessary but clunky" string comparisons in expandMacro()
with a lookup on the builtins. For starters, only the primitives
handled in doFoo() are converted, others will require varying amount
of further changes. chkexist argument is unused by doFoo() but will be
needed later for others, add now to avoid having to change more later.
No functional changes intended, anything like that would simply be a bug.
For current uses this shouldn't make any difference, but much of the
macro engine works with string lengths so to avoid having to create
\0-terminated versions of all strings we might look at...
There's an increasing number of placing wanting to know the number of
CPU's for parallel processing, and increasingly these things are running
in containers and such where the total host CPU count is not meaningful.
LUA_PATH global variable is not consulted when loading libraries in
Lua >= 5.1, package.path has replaced it. Rpm's Lua library path
was always supposed to be /usr/lib/rpm/lua/ but this has been broken
for the last ten years or so, oops. Make the directory a first-class
citizen: create it on install, add a macro for it, make it actually
work and ensure it stays that way by adding a test for it.
The shared string pool is in a very central role in several operations,
it's kinda embarrasing that we haven't had any thread protection
on it. Not that anybody has asked either, prior to coming up as part
of #226 (to enable threaded package creation).
Test-suite and couple of smoke-tests with the #226 pass, but only
lightly tested. Then again, it's relatively straightforward. As a general
rule, locks are taken on all exported interfaces on entry and released
on exit, internal callers never lock anything. In rpm usage at least,
performance hit seems negligible.
Centralize the log failure printing to a helper function, handle
the common conditions there. Remember the last error and only
print a new one if the error differs to avoid unnecessary flooding.
Finally, make the actual message more concise and mark it for i18n.
Oh and add a testcase as well.
This will still try to perror() in vain even if it's stderr that's
returning the errors in the first place, but maybe that's perror()'s
problem and not ours.
Commit 78b7a009cf caused reintroduced
some stream calls whose return value is not checked, causing the
failure to log message from getting output. In addition one of the
fputs() conditions was wrong (EPIPE instead of EOF).
Inspired by #453, adding configure-checks for unused digests algorithms
seems nonsensical, at no point in rpm history have these algorithms been
used for anything in rpm so there's not even backward compatibility to
care about. So the question becomes why do we appear to have (some)
support for those unused algorithms? So lets don't, problem solved...
At this point, if you want to avoid using shell you have only option
which is to use posix.fork() and posix.exec() which is too verbose and
not optimal (as Florian Weimer says, posix_spawn() can be implemented
more efficiently than usual fork() / execve() sequence).
Typical use-case is shown below.
-- Before
pid = posix.fork()
if pid == 0 then
assert(posix.exec("/foo/bar"))
elseif pid > 0 then
posix.wait(pid)
end
-- After
assert(rpm.execute("/foo/bar"))
Signed-off-by: Igor Gnatenko <i.gnatenko.brain@gmail.com>
In case /proc is not available to get the actual list of opened fds,
we fall back to iterating through the list of all possible fds.
It is possible that during the course of the program execution the limit
on number of open file descriptors might be lowered, so using the
current limit, as returned by sysconf(_SC_OPEN_MAX), might omit some
fds. Therefore, it is better to use rlim_max from the structure
filled in by gertlimit(RLIMIT_NOFILE) to make sure we're checking
all fds.
This slows down the function, but only in the case /proc is not
available, which should be rare in practice.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
In case maximum number of open files limit is set too high, both
luaext/Pexec() and lib/doScriptExec() spend way too much time
trying to set FD_CLOEXEC flag for all those file descriptors,
resulting in severe increase of time it takes to execute say
rpm or dnf.
This becomes increasingly noticeable when running with e.g. under
Docker, the reason being:
> $ docker run fedora ulimit -n
> 1048576
One obvious fix is to use procfs to get the actual list of opened fds
and iterate over it. My quick-n-dirty benchmark shows the /proc approach
is about 10x faster than iterating through a list of just 1024 fds,
so it's an improvement even for default ulimit values.
Note that the old method is still used in case /proc is not available.
While at it,
1. fix the function by making sure we modify (rather than set)
the existing flags. As the only known flag is FD_CLOEXEC,
this change is currently purely aesthetical, but in case
other flags will appear it will become a real bug fix.
2. get rid of magic number 3; use STDERR_FILENO
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Fixes#444
Commit 7a7c31f5 ("Set FD_CLOEXEC on opened files before exec from
lua script is called") copied the code that sets CLOEXEC flag on all
possible file descriptors from lib/rpmscript.c to luaext/lposix.c,
essentially creating two copies of the same code (modulo comments
and the unused assignment).
This commit moves the functionality into its own function, without
any code modifications, using the version from luaext/lposix.c.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
The algorithm that detects the end of a macro counts for all macro chars their level of nested. It iteratively moves from the first char of the macro further.
Before the patch the rpm function does not implement this algorithm correctly. After finishing with the last char of a macro line, it additionally skips one char of the next line. The first char in the line was omitted.
Probably affects every rpm version from this millenium.
These have claimed to be fread() / fwrite() "clones" but never *really*
were, and have been more or less consciously directed toward read()/write()
style semantics long ago, eg 3208913bdb
and 289ba88b30 talk about this. Changing
this long-standing behavior now would be little gain for some unnecessary
pain to callers, might just as well simply document the behavior and
be done with it.
It's much easier for rpm to point out the location of invalid macro
definitions than it is for humans to grep all the places a given rpm
version might look at. Log the macro file path once per file in case
of failures, and additionally return and error if all definitions fail.
Based on patch by Pavlina Moravcova Varekova.
Prior to this patch if a name of a newly created macro was the same as a built-in macro name, the macro looked correctly defined. But the newly defined macro could not be used.
It is obvious that a built-in macro can not be undefined.
Prior to this patch %{verbose:...} argument was always expanded, eg:
rpm -v --eval '%{verbose:%{echo:is verbose}}' \
--eval '%{!verbose:%{echo:is not verbose}}'
is verbose
is not verbose
Obviously both cannot be true.
Panu Matilainen