- Handling both rule-types identically not only makes things more
consistent but also adds exclude_flags support practically for free:
"exeonly" and "magic_and_path" are usable for excludes too now, ditto
for any other flags that might be added in the future.
- having those expressions without argument is no more allowed
- white space between expression and argument is mandatory
- rpmbuild no more confuses %ifabcd macro with %if abcd
- fix criptic message in case of invalid %if condition
- Allow "magic_and_path" flag to configure that files must
match both regexpes to be sent to the dependency generator.
Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
- Lift the payload copy into a separate helper function to clean
things up a bit
- The code to handle read errors was unreachable before this,
causing any read error (rare as they might be) to be silently
ignored and package generation considered successful.
- This could/should perhaps use ufdCopy() instead but that ha
its own set of problems (not clean wrt large files, error reporting...)
- Simply always call addFileToFoo() and handle no scriptfile condition
as early success case there (and allocate sb only if actually needed).
- Consolidate error logging into addFileToTagAux(), removing the need
for umphteen nearly identical rpmlog error messages + related logic.
- Eliminate the now pointless addFileToFoo() wrapper functions
and just call the main thing directly.
- Centralize the work into addFileToTagAux(), turning the other
two helpers into dumb wrappers with the only difference being
append or no append. This simplifies and clarifies the exit
codes, stringbuf allocations etc a good deal.
- The payload macros are already expanded in writeRPM(), no point
re-expanding in cpio_doio() and handle fallback to default gzdio
in writeRPM() already in case the payload is not set in config
or is bogus. Also rpmExpand() cannot return NULL so there's no
point in checking for that.
- fopen() returns NULL on errors, never an opened stream with error
flag set. These are leftovers from past where rpmio fd was used
instead of FILE and probably the checks were bogus even back then too.
- use _default_source_url macro to specify default URL when it is
missing from the spec Source/Patch line
- this feature can be disabled by defining _disable_source_fetch to 1
- It's supposed to return the number of errors, instead of silently
ignoring errors from qva->qva_showPackage() to unused variable,
count them. That qva->qva_showPackage() currently never returns
errors is another story...
- An example of a package where uncompressed payload is useful is
out "installation-images" package. It's a meta package that is
not shipped to customers, but used for image creation purposes.
Compression just slows down the build process.
Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
- Turns out this change does more harm than good, people are actually
relying on this behavior to avoid zillion conditionals in the spec.
We'll want a saner way to conditionally disable/enable packages
but until we have that, revert to the former longstanding behavior.
- This effectively reverts commit d010ec1c90
- Fixes regression from commit a2d002a34b,
the librpmbuild "name cache" was used for unique string storage too
and not just user/group-info lookup cache, duh.
- At least for now, making this separation explicit in the code
by separating the "stash this string" operation from the actual
user/group lookups.
- The build version has getUidS() and getGidS() for which there is
no counterpart in the rpmug api but there's not much point to them:
they check whether the user/groupname exists but return our own
string back to us even if it doesn't.
- The build version also caches more than current rpmug, but has an ugly
hardwired limit causing (in theory) errors that can't be nicely
handled, and is the last piece relying on RPMLOG_CRIT actually
terminating the process. The librpm version doesn't fail, in the
worst case its just a bit slower. And that can be fixed anytime by
making it to use hash tables for caching.
- Add support for new "OrderWithRequires: foo" spec syntax
which has "if and only if foo is present in transaction, order
the transaction as if this package required foo" semantics.
While it looks, and in some ways is, a soft dependency, this
is not the same as recommends/suggests etc: those have unknown
depsolver policy dependent semantics attached to them, whereas
ordering hints have a clear definition and is only relevant for rpm
itself, depsolvers should not even look at the data.
- This allows packages to express correct ordering for optional
functionality, such as
%post
if [ -x %{_bindir}/register-component ]; then
%{_bindir}/register-component %{name}
fi
If the package containing %{_bindir}/register-component is included
in the same transaction, it makes sense to have it installed before the
package(s) that use it. But as it is fully optional, Requires would
not be appropriate. Using OrderWithRequires allows this to be expressed
without dragging in extraneous dependencies for optional functionality.
- This allows both excluding entire paths from dependency generation
and also excluding individual generated dependencies by regexes
settable from spec or other configuration.
- %__(provides|requires)_exclude regex controls is matched against
generated dependencies, %__(provides|requires)_exclude_from is
matched against the currently processed path, with buildroot
stripped out.
- We'll probably want some "higher level" macros to go with this,
but the mechanism is usable as-is already.
- Split the macro name generation and grabbing of argv-style output
into yet another helper to reduce the clutter in rpmfcHelper()
before adding more stuff in there.
- Every single package since rpm >= 3.0.4 or so has a versioned provide
through the automatically added N = VR provide, which hasn't gotten
tracked in all these years. Drop the useless dependency.
- Lots and lots of API changes, including what amounts to
rewriting of librpmbuild external API... The only exception is
librpmsign which is new and starts with 0
- Many of the scripts need to know the buildroot in order to figure
out correct resulting paths. This permits unifying the current
adhoc methods of passing the data to the scripts.
- New constructor rpmfcCreate() which takes buildroot as an argument,
and "flags" argument for future use. Calculate brlen at initialization
now that we can.
- Preserve rpmfcNew() as a compatibility wrapper as it's something
somebody could, in theory, have used legitimately.
- These were used, back in the day, to implement a hidden --specedit
switch, which in all its glory did: print a copy of a spec file,
with group, summary and description looked up from specspo.
This huge pile of junk was left behind from kicking out the
useless "feature" in commits 7b95061d3b and
0f991a1420.
- rpmParseState uses might actually qualify for a true enum here
but changing it would require changing half of librpmbuild and
many of them mix the parse states, other error codes and whatnot
in their types... just not worth the trouble, so use an int.
- The string from getStringBuf() is not modified, so use a const
variable for it, and a separate variable for the temporary
line splitting for triggers
- addChangelog() is a bad boy and modifies the string it grabs
from getStringBuf(), be clean about it and grab a copy through
argvJoin() instead
- addChangelog() would be better off walking and parsing over the
argv directly instead of a joined string of it all, but leaving
that exercise for some other rainy day
- Similar to commit 1e3db59b56,
no point using temporary string buffer when only end up splitting
it into an argv for processing. Incidently this would've been
a much less intrusive fix to RhBug:573339 than introducing
a whole new argvSplitString() function... oh well. Take care not
to introduce extra newlines in the process as argvSplitString() was
eating them before - need to use appendStringBuf() for spec->prep now.
- Callers know perfectly well which part they're passing to doScript()
so they can just as well pass in the name and the scriptlet to run,
avoiding silly stuff in the switch-case
- Similar to commit 1e3db59b56,
all these end up being passed to argvSplit() to process them line
by line in the end, collect them in the argv to start with saving
a whole lotta huffing and puffing in the process
- Collect the filenames directly into the ARGV_t files instead of
putting them to a StringBuf first and then argvSplit()'ing the
result, doh. Leftover stupidity from eliminating splitString()
usage in commit 94ff22b129
- Nobody in their right mind is likely to be calling what's really
an internal helper function in the build code. Simply missed in
the first round of librpmbuild API butchery.
- This eliminates the last remaining exposure of StringBuf in the API
- Many of these cases would be true enums from preambleList & similar,
except for the list terminating sentinel. Just switch all the
remaining rpmTag's to rpmTagVals to bring our enum-whacking to
a grande finale.
Spec files have a lot of metadata about a project. However one of the
most key components is the upstream version control system which was
notably lacking.
Resolve this by adding a "VCS" key. There is no specification
for contents of this key, given that the set of version control
systems (and features thereof) are not well-defined. However,
recommendations are:
* git: This URL should be in a form that can be passed to "git clone",
with the additional feature that an optional fragment identifier "#foo"
denotes a branch or tag.
- Handle source rpm query as a new query source type RPMQV_SPECSRPM,
rename the binary query to RPMQV_SPECRPMS, with RPMQV_SPECFILE
as a backwards compatibility alias to RPMQV_SPECRPMS.
It appears the access to the source header has started to get buried
into usability so add an getter so the implementation can change a
little without breaking callers.
- Iterators for both (these could easily done as self-iteration over
->next but to keep the api similar to other rpm iterators...)
- Minimal getters to satisfy python bindings needs
- These huge structs have no business being visible to the public.
Just brute-force split all remaining spec internal structures into
another internal header, including where needed.
- The only thing accessing these structs outside of librpmbuild is
the python spec bindings. Temporarily permit direct access to
rpmspec_internal.h until we have the necessary API in place.
- This is not really native rpm code but a copy of (glibc?) FTS
implementation, with slight modification to avoid symbol name
collisions. Stick it where the sun don't shine, we'd eventually
like to get rid of such bundled external code.
- Similar in spirit to commit 825691afb2,
these have fairly little in common really.
- Change the function names to force breakage on users (these are exported
in ABI but not in public headers so "users" are internal uses in
practise): the automatically generated bits in signature header are
not signatures, but digests (even size is a checksum of sorts) ...
and fix the couple of internal uses.
- Been 2.5 years now without seeing that trap trigger, should be safe
to remove now ... to silence a warning about comparing to non-existent
enumeration value.
- Enums are fine for defining the bitfield flags, but the bitfield
itself is not an enumeration. Add a separate typedef on "rpmFlags"
type (presenting a bitfield of flags) for all of these. Compilers
hardly care, but the typedefs give a nice visual clue for
us humans using these flags far away from ho^H^H definitions.
- Not needed by rpmbuild itself currently, but permits retrieving the
build dependency information in a format that's better suited for
further processing than an rpm problem set.
- Add new public rpmSpecCheckDeps() function which can be used for
checking build dependencies of a spec against a transaction set.
- Change checkSpec() in rpmbuild cli to use the new interface
- rpmbuild.c is now free of direct spec structure accesses
- Spec parse flags are specific to parsing a spec, not building it.
This could be relevant again if a higher level build API is added
to the library but not ATM.
- Targets are obviously relevant for building but not in the
context of rpmSpecBuild which expects the environment to be set up
already. If/when we add a higher level build interface to librpmbuild
this comes relevant again, until then bury inside the cli-tool.
- Simplify the target parsery by using argvSplit() and friends.
- The mode letters are only relevant for the rpmbuild cli tool. If/when
some of the related build logic is moved from the cli tool to the
library, we'll need a better way of expressing these combinations
than letters.
- Transaction set verification bits are not relevant to building,
bury inside the rpmbuild cli utility.
- Also the type was wrong, its rpmVerifyFlags, not rpmQueryFlags, duh
- Add another flag set to control package generation, passing around
as necessary. Use this to implement --nodirtokens functionality
in a cleaner manner.
- Call freeNames() at end of buildSpec() instead of requiring API
users to clean up after us. This does mean losing some of the
supposed caching advantage when building on more than one spec,
but hardly matters, one spec at a time is the typical build usage
anyway.
- For now just a wrapper around parseSpec() that returns *gasp*
a pointer to a parsed spec structure.
- Limit the amount of goo that needs passing - optimally we'd only need
the spec + parsing flags, but due to %{buildroot} macro abuse in specs
we'll need that for now too.
- This was broken for years without anybody complaining, should
be safe to conclude nobody will miss it later either. And if
somebody misses it, this is a job for rpmlint really.
- The whole rootDir seems dubious for build purposes, but in any case its
only relevant during the actual build process, not for spec parsing.
Set spec->rootDir from buildSpec() for the duration of the
build to avoid having to refactor doScript() and all right now.
- Cookies are not a property of spec but a given build, handle this
internally to buildSpec() except for --rebuild which passes the
srpm identifier cookie through buildArgs
- Accept the entire build arguments structure along with a spec, these
contain everything that we'll need
- Just a convenience wrapper for buildSpec() for now, more things
to follow...
- Update the sole existing caller accordingly
The %sepolicy section is used to describe SELinux policy to be included
in a package. It's syntax is similar to other sections (%files, %pre,
%post, etc.) in that you can provide a string and -n after the
declaration to specify policy should be added to a subpackage.
For example:
%sepolicy
# policy in this section will be added to the main package
%sepolicy foo
# policy in this section will be added to the '<mainpackage>-foo' subpackage
%sepolicy -n bar
# policy in this section will be added to the 'bar' subpackage
The %sepolicy section contains zero or more %semodule directives, with the
following format:
%semodule [OPTIONS] path/to/module.pp
The available options are:
-b, --base
The module is a base module
-n, --name=NAME
The name of the module. If not given, assumes the name is the basename of
the module file with file extensions removed.
-t, --types=TYPES
One or more comma-separated strings specifying which policy types the
module can work with. To explicitly state that a module can work with any
policy type, "default" can be specified as the value. If not specified,
assumes the module can work with any policy type, and assigns the types as
"default".
Below is an example of this new format:
%sepolicy
%semodule -n foo -t mls policy/foo.pp
%semodule -n bar -t strict,targeted,mls -b policy/bar.pp
This also adds new header tags to store the new information:
RPMTAG_POLICYNAMES (string array)
RPMTAG_POLICYTYPES (string array)
RPMTAG_POLICYTYPESINDEXES (uint32 array)
RPMTAG_POLICYFLAGS (uint32 array)
The index of NAMES and FLAGS maps directly to the index of RPMTAG_POLICIES.
However, because a single policy can have multiple types, the mapping for
TYPES is not direct. For this, the index maps to TYPESINDEXES, which
contains the index of the policy that the type maps to. This is similar to
how DIRINDEXES is used to map DIRNAMES and BASENAMES. As an example, the
previous %sepolicy section would have the following header tags:
RPMTAG_POLICIES:
0: <foo.pp data, base64 encoded>
1: <bar.pp data, base64 encoded>
RPMTAG_POLICYNAMES:
0: foo
1: bar
RPMTAG_POLICYFLAGS:
0: 0
1: 1 # assumes flag 1 == BASE
RPMTAG_POILCYTYPES: RPMTAG_POLICYTYPESINDEXES:
0: mls 0: 0
1: strict 1: 1
2: targeted 2: 1
3: mls 3: 1
- Fixes the long standing obscure behavior of missing %files
section causes build to complete without errors but without producing
any packages either. Packages without files in them are perfectly
valid (eg metapackages), requiring empty %files section for those
is just unnecessary spec noise.
- Remove %files section on one of the test-suite specs to create
a test-case for this as a side-effect.
- This of course introduces a slight incompatibility with older rpm's,
if compatibility is required then just add those empty %files in the
spec, they dont hurt anything but the readers eyes.
- Combining description /checking/ and arch/os/platform /adding/
into same for-loop (to save cycles or lines, dunno) makes no sense
whatsoever. No functional changes.
- All these are internal implementation details that no callers
should mess with. Bury inside rpmbuild_internal.h.
- spec, package and source structures are tougher nuts as python
bindings expect to expose some of the information, we'll need
at least some api to cover walking over package headers etc before
shutting the door completely.
- Yank everything qualifying as "internal helper function" into
internal-only headers, loosely grouped as follows:
1) Everything involving spec-manipulation goes into rpmbuild_internal.h
2) All other miscellaneous helpers go into rpmbuild_misc.h
- This leaves us a rather minimal, and nearly useful API into librpmbuild:
Users need to be able to parse a spec, query or build (parts of) it
and free up the spec structure when done. This is what we have now,
various still exposed structures not counting.
- The only thing that should ever call this is freePackages() which
was already made static. As newPackage() links the package structure
with the spec, its not safe for others to call it.
- This is a funky little implementation detail which no caller should
have to bother with. Move it to parseSpec.c which is the only place
needing it now and make static.
- processSourceFiles() is only ever called through buildSpec(),
and if you throw unparsed spec into that you're pretty much screwed
anyway. rpmbuild always called initSourceHeader() anyway, and now
its ensured for all successfully parsed specs.
- This is where it logically seems to belong to, after all parseSpec()
similarly populates all other metadata of the binaries etc too.
No point requiring callers to do this obscure extra call.
- processSourceFiles() is more about processing the physical files
ending up in the srpm payload, source + patch tags are mostly just
informational and can be added early
- readRPM() is not used or needed by anything, shred it
- make writeRPM() static inside the only place needing it: pack.c
- move struct cpioSourceArchive_s inside pack.c
- Signing is better done in an environment completely separated from
the process of actually building packages. This is how its generally
done anyway by distros, this change just forces the issue.
- While signing on build seems handy and saves some io-churn, this
is very insecure as your precious secret passphrase remains unencrypted
in memory for the entire duration of the build where it's accessible
to whatever happens to be building. This change doesn't
affect fix the issue of unencrypted passphrase residing in memory
and potentially getting swapped out, it only limits the exposure
to the actual signing process which isn't subject to unknown scripts
poking around.
- Minimally preserve rpmbuild's --sign functionality via popt --pipe hack
to call rpm --addsign when build completes
- Use a self-pipe to handle signal race on select(). pselect() would work
too but this is more portable and avoids other signal hassles.
- Use non-blocking IO for communicating with the child to avoid spin-happy
timeouts, just check all fd's properly before trying to use them
- Avoid leaking memory from readBuff on errors
This patch allows short-circuiting of building binary packages. To
prevent accidental use of short-circuited rpms, they are marked with
an unsatisfiable dependency "rpmlib(ShortCircuited)". A developer using
this feature for local development and testing will use "--nodeps".
Should such a rpm leak into the world, users will quickly notice
it does not install by default.
(v3: avoids API change, suggested by Panu Matilainen.
v2: added poisoning with unsatisfiable dependency.)
Michal
- Forgot how nasty hack parseBits() was... its abusing multilang
support and the multilang support passes "C" if no lang is specified,
which certainly isn't a valid qualifier. Make multilang vs tag with
optional qualifiers different types in PreambleRec, which allows
handling this more sanely
- Additionally build-dependencies dont take any qualifiers now
- A pre-transaction dependency is generally anything that must be
available at the start of the transaction, and cannot be resolved
by packages *in* the transaction. This lets %pretrans scriptlet
dependencies be expressed correctly, and could be also used for
other kinds of pre-conditions.
- rpmlib() dependencies are a special case of pre-trans dependencies
but leaving them handled separately as they cannot be provided by
anything in rpmdb either, whereas pretrans dependencies can.
- previously unknown qualifiers would go unnoticed if they happened
be last (including being the only qualifier), eg Requires(pre,junk)
or Requires(junk) would be cheerfully accepted but Requires(junk,pre) not.
- Rpm has never done anything useful with "BuildRequires(prep): foo"
style dependencies other than recorded them in src.rpms, nor is there much
point in properly supporting this in the future either
- Frees up four more bits from rpmsenseFlags for better uses
- pass the wanted dependency type to addReqProv() as tagN argument
(previously unused, eh...) instead of taking up three precious bits
for internal-only purposes from a constrained bitfield
- move most of the hardwired classification logic from rpmfc C-code
to macro-based configuration, supporting drop-in addition of arbitrary
new attributes + dependency extractors based on regex matching of
libmagic file types and paths
- just the initial rough conversion of our built-in dependency types,
various open questions + todo-items remain, plus likely fair amount
of more-or-less subtle breakage
- Add per-scriptlet type flag tags to control special behavior.
- Add rpmlib dependency on scriptlet expansion - if a package relies
on scriptlet expansion it cannot be correctly installed with
a version of rpm that doesn't support it.
- Expansion is always an opt-in behavior, enabled with -q and/or -e argument
in spec. We can't just blindly expand even macros as there's no telling
%{} constructs might mean in whatever language is used for the script.
- Queryformat expansion requires great care with strange and ugly escapes
when writing scriptlets, but OTOH it permits access arbitrary header
data at runtime, which has previously been completely impossible.
- The handling of these expansions needs unifying for all scriptlet types,
the trigger scriptlet handling is uuugly. Macro expansion could be
transparently done from rpmScriptRun(), but because of their similar
syntax, macro expansion needs to happen before query format expansion,
and we dont have the header available in rpmScriptrun()
- Previously we required a file to be classified as text and have
.desktop suffix, for now replace that with just a regex. We'll eventually
want/need some extra qualifying logic to handle cases like these,
eg "only apply path attributes if condition X is also true"
Panu Matilainen