Upstreaming from Fedora (BZ#1791530)
That is, we add new provides that replace dots with a dash.
Package that used to provide python3dist(zope.component) and python3.8dist(zope.component)
now also provides python3dist(zope-component) and python3.8dist(zope-component).
Package that used to provide python3dist(a.-.-.-.a) now provides python3dist(a-a) as well.
This is consistent with pip behavior, `pip install zope-component` installs zope.component.
Historically, we have always used dist.key (safe_name) from setuptools,
but that is a non-standardized convention -- whether or not it replaces dots
with dashes is not even documented.
We say we use "canonical name" or "normalized name" everywhere, yet we didn't.
We really need to follow the standard (PEP 503):
https://www.python.org/dev/peps/pep-0503/#normalized-names
The proper function here would be packaging.utils.canonicalize_name
https://packaging.pypa.io/en/latest/utils/#packaging.utils.canonicalize_name
-- we reimplement it here to avoid an external dependency.
This is the first required step needed if we want to change our requirements later.
If we decide we don't, for whatever reason, this doesn't break anything.
size_t is unsigned, so returning -1 is not going to have the expected
behavior. Fix it to return ssize_t.
Signed-off-by: Jes Sorensen <jsorensen@fb.com>
Fixes regressions from commit 75ec16e660e784d7897b37cac1a2b9b135825f25:
the newly added provides of to-be-built packages were being used for
dependency resolution, such as spec satifying its own buildrequires,
and matched against conflicts in installed packages.
Source packages cannot obsolete anything or provide capabilities or files
to transactions, don't add them to rpmal at all. Explicitly skip checks
against source provides, similarly to what we already did with obsoletes.
Fixes: #1189
Binary packages come in different sizes and so their build time can vary
greatly. Dynamic scheduling, which we currently use for parallel
building, is a good strategy to combat such differences and load-balance
the available CPU cores.
That said, knowing that the build time of a package is proportional to
its size, we can reduce the overall time even further by cleverly
ordering the task queue.
As an example, consider a set of 5 packages, 4 of which take 1 unit of
time to build and one takes 4 units. If we were to build these on a
dual-core system, one possible unit distribution would look like this:
TIME --->
CPU 1 * * * * * * # package 1, 3 and 5
CPU 2 * * # package 2 and 4
Now, compare that to a different distribution where the largest package
5 gets built early on:
TIME --->
CPU 1 * * * * # package 5
CPU 2 * * * * # package 1, 2, 3 and 4
It's obvious that processing the largest packages first gives better
results when dealing with such a mix of small and large packages
(typically a regular package and its debuginfo counterpart,
respectively).
Now, with dynamic scheduling in OpenMP, we cannot directly control the
task queue; we can only generate the tasks and let the runtime system do
its work. What we can do, however, is to provide a hint to the runtime
system for the desired ordering, using the "priority" clause.
So, in this commit, we use the clause to assign a priority value to each
build task based on the respective package size (the bigger the size,
the higher the priority), to help achieve an optimal execution order.
Indeed, in my testing, the priorities were followed to the letter (but
remember, that's not guaranteed by the specification). Interestingly,
even without the use of priorities, simply generating the tasks in the
desired order resulted in the same execution order for me, but that's,
again, just an implementation detail.
Also note that OpenMP is allowed to stop the thread generating the tasks
at any time, and make it execute some of the tasks instead. If the
chosen task happens to be a long-duration one, we might hit a starvation
scenario where the other threads have exhausted the task queue and
there's nobody to generate new tasks. To counter that, this commit also
adds the "untied" clause which allows other threads to pick up where the
generating thread left off, and continue generating new tasks.
Resolves#1045.
Issue a warning if buildtree macros (%_sourcedir etc) contain undefined
macro(s) after expansion, such as things only defined during spec parse.
This always was a murky case that doesn't work in all scenarios, so
a warning seems appropriate. Actual behavior doesn't change here though.
The NSS library often changes in ways that somehow breaks rpm,
and these days upstream does not care about consumers of NSS other
than itself. This inflicts untold amounts of suffering on users
of rpm in distributions where rpm is linked to NSS.
Now that we have a couple of good, well-supported options, there is
no reason to keep supporting NSS as an option.
So now, we are deprecating it for later removal.
In todays' "look ma what crawled from under the bed" episode, we have
encounter a subpackage whose name is not derived from the main package
name, and a manually specified debuginfo package on that. This particular
combo manages to evade all our checks for duplicate package names, and
in the right phase of the moon actually creates corrupt packages due to
two threads end up writing to the same output file simultaneously. Which
is what happened in https://pagure.io/fedora-infrastructure/issue/8816
Catch the case and use the spec-defined variant (because getting rid
of it would be harder) but issue a warning because most likely this
is not what they really wanted.
This is normally unneeded because an index sync is already done
after each package is added to the database. But rebuilding an
empty database lead to no index sync being done and a mismatch
between the generation count. This in turn will trigger a
index regeneration.
Found by using ndb when running the test suite.
Somehow I was under the impression that mapped regions are unmapped when
the corresponding file descriptor is closed, but that's not the case
for POSIX systems.
Instead just map it read-write if one of the functions request
exclusive access. We keep track of the number of exclusive
locks and fall back to read-only mapping if the count reaches
zero again.
Pass the flags to rpmidxOpenXdb and use read only mode if the
user specified O_RDONLY. We already did that for rpmidxOpen in
the past but we always used read-write mode when using the Xdb.
We still open the Xdb itself in read-write mode so that we can
regenerate missing index databases.
We forgot to copy the mapped pointer if a slot was kept when
re-syncing after a generation mismatch. This led to the mapped
pointer being zero even if the map callback was set.
Also add an extra test for the map callback before setting the
protection flag to PROT_READ.
Found by testing the code with different mapping protection flags.
See the next commits.
pythondeps.sh was written in shell and unlike the Python dist generators,
it uses no Python, it plainly determines the provide / requires from the path.
As the script was run for every Python file, we were potentially doing hundreds
of shelling outs to execute a script that calls grep and sed.
In Lua, this should be more efficient.
Fixes https://github.com/rpm-software-management/rpm/issues/1152
Lots of cruft here - the build-aux move related changes from commit
cd6e4eb9e0, and all manner of historical
cruft that hasn't existed in over a decade. While at it, consolidate
it all to the toplevel .gitignore.
'fn' already contains full path to a file, so no need to prepend it once
more. This is actually breaking things.
Before:
D: Calling %{__pythonname_provides %{?__pythonname_provides_opts}}() on /home/brain/rpmbuild/BUILDROOT/hello-1-1.fc33.x86_64//home/brain/rpmbuild/BUILDROOT/hello-1-1.fc33.x86_64/usr/share/applications/org.gnome.Terminal.desktop
After:
D: Calling %{__pythonname_provides %{?__pythonname_provides_opts}}() on /home/brain/rpmbuild/BUILDROOT/hello-1-1.fc33.x86_64/usr/share/applications/org.gnome.Terminal.desktop
Fixes: https://github.com/rpm-software-management/rpm/issues/1162
Signed-off-by: Igor Raits <i.gnatenko.brain@gmail.com>
dirent.h and struct dirent are actually standard on this millenium, the
only thing that isn't is d_type member for which we have and retain
a specific test in configure.ac. Include <dirent.h> where needed,
relatively few places do which makes it even a bigger insult to have
this included from system.h.
Fixes the reproducable build test failing in Fedora rpm builds due to
%source_date_epoch_from_changelog being set on the outside, which
leaks the SOURCE_DATE_EPOCH environment into the test-suite and
changes the expectation.
Unlike that multiple statfs() variants, statvfs() is actually in
POSIX 1-2001 already and covers everything we need so there's little
point mucking with anything else. statvfs() is what Linux has been
using all along anyway this means no change on the primary platform.
If this actually regresses something, adding back OS-specific bits
is not a problem, but at least we'll get a clean start with that.
The message about telling your OS vendor about GNU utilities that
gets removed here dates back to 1997. The X syntax to chmod goes back
to at least 2004 specification of POSIX 1003.1, I think we can safely
assume that capability in 2020. And that OS vendors know about GNU :)
Tomas Orsava