Crossroad repo was finally moved to Freedesktop.
See: https://gitlab.freedesktop.org/freedesktop/freedesktop/-/issues/477
This should avoid the too many failures we have lately on TuxFamily
infrastructure (and also gives a proper bug tracker for Crossroad). I am very
thankful to TuxFamily for all these years! :-)
Let's not be said that I moved the repo because I don't like this friendly FLOSS
host anymore. I just needed something different for Crossroad project now.
The creation of the BMP welcome images for the Windows installer (part of
-Dwindows-installer=true build option) fails in the Windows job. After much
debugging, I could run GIMP, yet it was not enough. One of my hypothesis so far
is that the environment variables for DLLs won't work, since all the DLLs must
be in the same directory as the main binary (though with the WSL thing, I am
unsure, maybe it is still supposed to work), which only happens once GIMP is
installed. So GIMP runs successfully but not plug-ins.
Anyway I wasted too much time working on this and without a local Windows, it
just takes too long (mostly testing thanks to the CI) and is frustrating. Let's
just move to building both the localization files and the images on the main
Debian job (gimp-meson-debian), then use these as dependencies of the
win-installer-nightly job, i.e. when building the installer.
After discussion with Jernej, InnoSetup should now work better with rescaling
a big image properly to the window size, yet the ratio should still matter.
Apparently the welcome image is a hack and this is why it requires specific
ratio images. We don't use the big size yet, but since Jernej told me which
dimensions are expected, I already added the code for it to make it easier
later.
So anyway this code would allow us not to have to commit welcome images each
time, which are basically resized copy in BMP of the splash screen, slowly yet
surely filling up our repository with image duplicates.
After all, we develop a scriptable image editor! We should use it to edit images
and export in expected formats!
I only use this script for the devel installer for now, for testing and see how
it goes.
While Debian testing is the sensible choice for dependency requirements while we
are in full-dev mode, now we are getting closer to 3.0 release, and Debian 12
"bookworm" barely got out.
If we continue to develop at current pace, GIMP 3.0 should be released before
the next stable Debian, so we should not use dependencies unavailable on the
latest stable (otherwise GIMP 3.0 won't be distributable soon enough on Debian,
nor on Debian derivatives such as Ubuntu, Mint, etc.).
This is why we are now basing GIMP dependency requirement on Debian bookworm.
We'll likely get back to the next "testing" after GIMP 3.0 release.
This commit makes no real changes but style.
As discussed, this makes it much easier to compare commit diffs, rather
than an overlong line where you have to search which package may have
been added/removed/changed.
Switch to NASA-maintained cfitsio library for loading/exporting FITS images.
This allows us to import compressed FITS files (GZIP, HCOMP, PLIO, RICE) in
8/16/32 bit and float/double precision. It also simplifies export code using
the built-in cfitsio APIs.
The flatpak_ci_initiative.yml template we include got changed 2 days ago,
replacing the "only" by a "rules", which broke our CI file.
Cf. commit 193f63c20f87d38868c05beee6446b387b73e140 in repository citemplates.
The present commit should (hopefully) fix this error:
> Unable to create pipeline
> jobs:flatpak-nightly config key may not be used with `rules`: only
(even though the error is a bit cryptic, the problem is apparently that both
rules: and only: keys cannot be used together)
Python 3.10 introduced sysconfig.get_default_scheme() and somehow Debian decided
it was a good idea to move from the 'posix_prefix' to the 'posix_local' scheme.
See: https://lists.debian.org/debian-python/2022/03/msg00039.html
The main issue with this scheme is that it adds a local/ subdirectory to the set
--prefix. As a consequence, with --prefix=~/.local/, scripts are installed in
~/.local/local/bin/ (instead of proper ~/.local/bin/), data are now in
~/.local/local/share/ and so on.
As expected, this broke a lot of CIs for a lot of projects, as well as a lot of
custom scripts, usage of Python in virtualenv (this later case seems like it is
fixed by special-casing it in Python 3.11), flatpak, etc.
Setting DEB_PYTHON_INSTALL_LAYOUT environment variable to "deb" solves this by
changing the default scheme.
This URL is also useful to understand the issue:
https://askubuntu.com/questions/1406304/virtualenv-installs-envs-into-local-bin-instead-of-bin
As a side note, Debian is not the only one which made the mistake. Fedora also
did the same thing (and they have also their own different environment variable
to handle this): https://bugzilla.redhat.com/show_bug.cgi?id=2026979
In the last few days, our deps-win64 job started to fail with:
> $ update-alternatives --set x86_64-w64-mingw32-g++ /usr/bin/x86_64-w64-mingw32-g++-posix
> update-alternatives: error: no alternatives for x86_64-w64-mingw32-g++
The reason lies in this change in Debian testing 10 days ago:
----
gcc-mingw-w64 (25) unstable; urgency=medium
* Upgrade to GCC 12. Closes: #1023679. It is no longer possible to tweak
the installation directory for different thread models, so the -posix
and -win32 packages are no longer co-installable.
* Drop “Built-Using” from “Architecture: all” packages.
* Since the POSIX and Win32 packages are no longer co-installable,
drop support for alternatives and use symlinks to provide the full
set of command names.
* Standards-Version 4.6.1, no change required.
-- Stephen Kitt <skitt@debian.org> Mon, 12 Dec 2022 09:00:34 +0100
----
So from now on, we'll only install the posix variant of the cross-compiler.
Hopefully it will work for all packages we build.
I added this option in commit 38d6783299. It makes the script stateful, able to
remember all previously processed DLL, hence making the whole job much faster
and efficient.
- Use a relative path for GIMP_LOGO_PATH inside the gi-docgen generated
HTML, and not an absolute path taken from build dir (otherwise this
would break, for installed docs, but also for the tarball and the
developer website!).
- Also use either gimp-logo.png or gimp-devel-logo.png depending on
whether we are on a stable or unstable branch.
- Install these in images/ inside the GIMP docs folder, which
corresponds to the relative path given to GIMP_LOGO_PATH.
- The installed root dir will be $datadir/doc/gimp-2.99/, e.g.
/usr/share/doc/gimp-2.99/
- Inside this folder, the library references will be in libgimp-3.0/ and
libgimpui-3.0/ (instead of weird Gimp-3.0/ and GimpUi-3.0/). Note that
the root dir uses the application version (2.99) whereas the library
folder use the API versions. These are different in development phase.
- Archive the gi-docgen installed files, not taken from the build dir,
to avoid packaging temp files, such as the .toml files. Note that
`g-ir-docs` files are still taken from the build dir, as we don't
install them yet.
- Finally package all this in a directory before archiving in a tar.xz,
named the same as the directory (e.g. gimp-api-docs-2.99.13/ inside
gimp-api-docs-2.99.13.tar.xz).
If we want to encourage packagers to test the meson tarball (better to
do this now with development releases than later on stable releases!),
we should name it prominently. Therefore I will rename the autotools
tarball as gimp-*-autotools.tar.bz2 whereas the meson tarball will be
named gimp-*.tar.xz.
Also renaming `sources` CI job to `sources-autotools`.
The `ninja dist` step fails with:
> ERROR: Repository has uncommitted changes that will not be included in the dist tarball
> Use --allow-dirty to ignore the warning and proceed anyway
> FAILED: meson-dist
> /usr/bin/meson dist
Astonishingly if I run a `git diff` (or git status) in the CI, I saw no
changes and the dist suddenly succeeded. When I remove it, it fails
again. And of course, locally it's alright. No way to diagnose!
Anyway it's not a bad idea to leave some diagnostic logs in the CI so
here we go, properly outputting the diff and exiting the job with an
error when there is a diff. Because anyway, there should be none. It
would mean that there is likely some issue in versionned generated
files.
Maybe let's try to distribute the meson tarball next to the autotools
tarball for our next dev release, and announce that packagers are
invited to test the meson build from tarball and report back.
These were originally to distribute cross-built binaries. Nowadays, we
just use the native-made installer, which is also closer to what people
will really get for release versions.
So let's just remove these. I keep the crossroad builds as these are
still useful to detect Windows build bugs quickly, but we don't need
these distribution steps.
This also takes care of failures in the job, but since it's mostly a
useless job nowadays, rather than wasting my time investigating this, I
simplify the CI.
The gnome-runtime-images have been recently migrated to Quay. This is already reflected in the template.
Please note this MR has been created semi-automatically. If it doesn't make sense, feel free to close it.
intltool has long been dead upstream. Let's not poke the dead corpse,
please.
This commit is quite large, but that's mostly since trying to support a
hybrid of both gettext and intltool with both Meson and Autotools was
really hard, so I stopped trying.
Due to gettext relying on quite some things being at the exactly right
place in the autotools build (like `ABOUT-NLS` and `config.rpath`) we
really needed to cleanup the `autogen.sh` to only call `aclocal` and
`autoreconf`. No more strange magic; I tried to do it without changing
too much in the file, and things just broke. If people want to do
something more custom, they can just change the script directly. This
change also uncovered some problems in our `configure.ac`, like using
deprecated macros.
The following major changes happened:
* meson: Changed `custom_target()` to `i18n.merge_file()` for all
supported file types
* Added `.its` and `.loc` files for the GIMP-specific XML formats, so
that gettext understands them
* For the `.isl` (Window installer stuff) file, there's no easy way to
do this in gettext, so instead we start from an XML file (again with
its own ITS rules etc), translate that with gettext, and then use
`xsltproc` with a bit of magic to output the .isl file for each
language
* the `po*/Makefile.in.in` files are migrated to `Makevars` files,
which gettext natively understands.
Fixes: https://gitlab.gnome.org/GNOME/gimp/-/issues/8028
We were already avoiding re-processing a same DLL within the same run
(this can happen when 2 dependencies have themselves a common
dependency). But the dll_link.py script was stateless regarding previous
runs so we might be checking again the same DLLs multiple times (even
though we were not copying them again).
Let's make the script stateful with a new parameter to give a file where
all the previously processed DLL names are stored. I am hoping it would
improve the efficiency of the packaging-win32-native which is suddenly
extra slow (it always times out, even after raising the max job time;
now we time out after 2h30! The 64-bit packaging job just takes 1h,
which is too much already, but still much more reasonable).
Since this is probably much less tested than the default vector icon
case, any bug there might go unseen for some time. So let's add a new
pipeline to check this. I make so this pipeline is not actually run on
every commit (i.e. manually only), then I'll just add a scheduled
pipeline to check the non-vector icon case at regular intervals.
This pipeline also test the icon-list.mk files are not outdated.
Fixes#950
`.gitlab/search-common-ancestor.sh`'s original authors are
Philip Withnall and Frederic Martinsons.
(Jehan/reviewer's note: script further improved by Asalle)
gtk-doc has been slowly dying for the past few years; with gi-docgen we
have a nice successor.
This also makes sure the C documentation also uses the GIR file, which
in turn means faster build times (since all the C code doesn't have to
be parsed and recompiled again), and has a clear dependency graph.
See the [gi-docgen tutorial] for more info on how the system works.
[gi-docgen tutorial]: https://gnome.pages.gitlab.gnome.org/gi-docgen/tutorial.html
This workaround was placed to mitigate race condition.
It was useful for the time being, but not anymore.
Better solution will be introduced in following commit.
Reverting changes introduced in 9c6776fb.
Commit-triggered pipelines on `master` branch are apparently canceling
flatpak pipelines even though the flatpak jobs are not even in the
triggered ones.
In any case, these flatpak pipelines are not triggered often (only
through weekly schedule or manually triggered) so there is no need to
have such a flag.
See: https://gitlab.com/gitlab-org/gitlab/-/issues/340147
I completely forgot that since the installer is built on a Windows
runner, I cannot expect a standard Linux shell syntax.
As a consequence, commit de9a171a19 broke the "win-installer-nightly"
job with the following error:
> The token '&&' is not a valid statement separator in this version.
Rather than trying to find the equivalent command to run on powershell
or whatever, let's just compute the checksum at the end of our installer
creation script.
Somehow the test part of the flatpak job stopped working. The last
instance which worked was 3 weeks ago, but after a lot of debugging I
realized that it is not because of any code change on our side. The
exact same commit which worked 3 weeks ago won't anymore!
The standalone bundle is actually built and works fine when tested
independently. What fails is the `meson test` inside the flatpak
environment. Somehow when GIMP is rebuilt inside the test flatpak
environment, it doesn't build the plug-ins yet one of our tests
(save-and-export) requires plug-ins to open some file formats. Note that
I double-checked, the plug-ins were well built and loading any format
works fine in the standalone flatpak, just not in this specific step.
I am completely unsure what broke, yet it is apparently outside GIMP
code. So for now, I just copy-paste the whole flatpak job which we were
including from another repository and remove the `meson test` part.
Since GIMP looks for its icons at runtime and would output warnings if
it doesn't find them, it's better to install first. Not really sure it's
ideal though, but it will do for now. Maybe I should just g_printerr()
instead of g_warning().
… based on set labels.
If the label "5. Windows Installer" is set, the MR pipeline should
trigger the Windows installer pipeline as well.
If the label "5. Flatpak package" is set, it should generate the flatpak
(not publish it obviously, yet it can be downloaded and installed
manually).
This will allow us to easily test MRs and allow people to test our code
before we merge it to the main branches.
Currently we run the job on a weekly schedule. If we keep a retention
period of 2 days, it means that people will not have access to a nightly
Windows installer 5 days out of 7, which is not useful.
Gitlab has a retention policy not to delete artifacts for last jobs, but
it apparently actually bases this check on being the last pipeline, not
being actually the last job of a given name. Since we have pipelines
running all the time, this retention policy just won't apply to our
installer job which will get deleted.
Cf. https://gitlab.com/gitlab-org/gitlab/-/issues/332142
Without this, the flatpak build is just too long and Gitlab CI just
stops logging it. So we end up with a log saying the quite useless (at
the end):
> Job's log exceeded limit of 16777216 bytes.
> Job execution will continue but no more output will be collected.
We don't even reach babl/GEGL/GIMP builds, which are the most important.
With this little trick, I am redirecting output to a log file and simply
including said log into the artifacts, hence allowing to debug the full
build.
Argh! So it turns out that .publish_nightly template uses already an
only: key so we cannot use rules: (on the other hand I guess using only:
ourselves is alright and concatenate ours and the one in the template).
Fixes:
> jobs:flatpak-nightly config key may not be used with `rules`: only
This clashes with the usage of "rules:" and is even more useless as it
is about checking success of earlier stages (yet this job doesn't need
to wait for any other job).
Fixes CI error:
> jobs:flatpak config key may not be used with `rules`: when
To get a nightly flatpak, GNOME project offers a CI template[0] that can
be extended instead of having to define everything from scratch.
At the core is the "flatpak" job that tries to build the flatpak. If the
build finishes succesfully, the result should be available as a CI
artifact as a flatpak bundle. The app id is the same as on Flathub.
Another part is the "flatpak-nightly" job that publishes the flatpak if
the build succeeded. Without this job, the bundle stays only temporarily.
This job can and should only be run on the "master" branch. This jobs is
started only when the "flatpak" job finishes successfully.
Both jobs can be triggered only thorugh schedules[1]. To finish the
flatpak nightlies setup, a schedule needs to be set.
[0] https://gitlab.gnome.org/GNOME/Initiatives/-/wikis/DevOps-with-Flatpak
[1] https://docs.gitlab.com/ee/ci/pipelines/schedules.html
The logs are just too long for gitlab and ends with:
> Job's log exceeded limit of 4194304 bytes.
This doesn't prevent the job from actually finish successfully. Yet the
day when we will have a build issue, we won't have a way to debug if it
happens close to the end of the installer creation. So let's redirect
both stderr and stdout to a log file and include it in the artifacts.
Unlike what I said in my previous commit, it still just takes too long
to build the installer, whether I move some of the substeps around or
whether I increase the max duration. So rather than increasing max
duration a lot more, let's just add an intermediate stage.
The last stage (installer creation) takes just too much time, and it
exceeded the max execution time (80 min) in my last test build. Instead
of increasing this max execution time, let's move the run of
package-gimp-msys2.sh script in the same step as the build one. Maybe
adding an intermediate stage would have been better conceptually, but
every stage also takes some preparation and finalization time (setting
up the runner, loading, cleaning the uploading the cache, etc.) and our
installer pipeline is already long enough.
So let's just go like this for now.
As a side effect, the last job's log limit was exceeded too since I
added the Python option, which should hopefully also be fixed by moving
steps out of the installer job.
Argh! My previous commit logics was right, but should have been applied
to the Windows builds (where the issue happens). That's what happens
when I hack too late at night!
This is not a fix, only a workaround for the CI to at least not fail
randomly, for the time being.
Basically since meson is running parallel jobs, even when the build
fails because git-version.h doesn't exist yet, while a depending file is
being built, it will still be built immediately after, despite the
failure. As a consequence, re-running `ninja` immediately after (i.e.
running `ninja || ninja`) will make the second build work. If it doesn't
then it's another issue which has to be fixed. But at least we work
around this known race condition in the meson build for the time being.
It's very ugly, but better than current situation. :-/
By enabling this option on Linux, not only will the installer language
file be generated, but the `make check` will also now validate that the
lang list is consistent with existing gettext files (see commit
8a42c6ccc2). So it's useful information to know this for instance as
soon as some translators add a new localization.
Also oppositely remove the option on the MSYS2 native Windows 32-bit
build. For Windows, we only need this option once, as we use the
language files generated by the 64-bit build.
The goal is to make it easier to understand which pipeline is run when,
for future maintenance, and also to make it easy to trigger a specific
pipeline through Gitlab pipeline interface.
- Adding a patch sent to me by Sylvie Alexandre meant to help aalib
build on MSYS2 for Windows 32 and 64-bit.
- Additionally, add an additional patch from myself because it was still
not building properly.
- Also update the config.guess|sub files because the original ones (from
2001!) were just too old and not properly recognizing the host mingw
system (especially the 64-bit one apparently) in the MSYS2 CI jobs of
GIMP.
- Finally regenerate the whole aclocal/libtoolize/autoconf/automake
build system because these old files just don't play nice with recent
autotools though the source files still regenerate fine (despite with
some warnings, but nothing blocking).
- Add crt-git dependency because libws2_32 is in there.
We had an autotools build stopping at `make check` and a separate one
for `make distcheck`. This just seems very redundant hence a waste of
resources.
So let's drop the job gimp-autotools-debian then add a `make check` step
to gimp-distcheck-debian.
Also as a side change, I move the cppcheck to being a scheduled job.
It's not such resource intensive job, nor did it take much time, yet
it's not like we use this information constantly. Moreover it never
fails anyway (so it's not like it gives much information on a per-commit
basis, unless we explicitly look into the resulting files) and with the
ability to run custom jobs whenever we want, this is far enough if
sometimes we need to generate the cppcheck analysis more frequently.
The rest of the time, having 2 jobs of this a week (our current
schedule) is far enough.
Run InnoSetup in the Windows CI to build the installer from both the 32
and 64-bit builds.
Current limitations:
- No installer signature yet.
- Dependencies will have to be checked more thoroughly.
- Apart from babl and GEGL, we may want to make custom builds of any
package which has a patch in build/windows/patches/ (Windows-specific
patches) and build/patches/ (all platform patches).
- Plug-in interpreters (Python, Lua…) don't work. This will need to be
looked at in detail.
Globally this first automated installer build works fine though, as I
could install it in a Windows 10 VM and GIMP ran fine! So it's a first
step towards fully automated releases for Windows.
Having them at each commit is counter-productive, first because these
builds take so long and second because there seems to be quite few
Windows runners. So we end up constantly waiting for CI jobs from
previous commits (so we are just constantly waiting).
This is resource over-usage. So instead, I'll just set this in scheduled
jobs. For release preparation though, we'll have to set up a workflow
later to trigger these jobs off-schedule/on-demand (I can see there is a
Gitlab interface to do this), but this can wait for when the installer
is fully generated by the CI anyway.
Note: Vala API doesn't build well on the 32-bit build. Not sure why (the
meson logs for GObject Introspection build are just as empty as ever),
but it won't generate the VAPI. So I disabled the option on 32-bit.
This new job resulted in a package which allows to run GIMP on Windows
(as tested in a VM; at least it starts, I can create a new canvas and
paint). Of course I think this will need to be tweaked a little bit
more, as I'm sure we miss things here and there.
At the very least, even though I add the Python and Luajit binaries,
GIMP on Windows didn't find them. This will need to be investigated.
Also it looks like opening from a remote location may not work. Not sure
if this about a missing GIO module or maybe something which works
differently on Windows (I was not even able to drag'n drop from the
browser!). Anyway this needs to be looked at as well.
Note that gdk-pixbuf-query-loaders is apparently unneeded when GIMP is
built this way (unlike with our crossroad build).
All this to say that this is still an early attempt to full CI build for
Windows.
It doesn't invalidate the crossroad build, because cross-compilation
builds from Linux will always stay very important for Linux developers
to be able to easily fix Windows bugs too; yet the crossroad build has 2
major issues:
1. We haven't figured out yet how to run GObject Introspection tools for
cross-builds, so the crossroad builds are not full-featured (and this
is quite a major feature we are missing!).
2. Also I will want to run the installer in the CI at some point and the
one we use can only run on Windows itself AFAIK. We could try to run
it through Wine, but still anyway the point 1. is already quite a
blocker, let's do the simple thing.
Note that we will likely want to move to meson for this build, because
autotools is very slow on Windows. But as long as the few blocker meson
bugs are not fixed, let's stick to the slow yet good build.
These are interesting and may find very specific bugs from time to time,
but the usefulness is rare enough not to warrant to run at each commits.
This is just a waste of resources.
For scheduling finesse (in case we want to separate these in separate
scheduling), also rely on the existence of variables during scheduling.
Finally make sure that the non-scheduled builds are not run in schedule
pipelines (they are already run far enough).
The build rules were highly inspired by other projects on GNOME's
Gitlab. All of them used to build with ccache. It worked fine for the
main build, but completely broke GObject Introspection build on both
babl and GEGL. And the worse thing is that meson was absolutely not
displaying the error, just saying it failed (even in verbose mode). A
lot of time wasted trying to debug.
Therefore let's get rid of ccache, but only for babl and GEGL. Keep it
for GIMP itself as it works fine there.
Other minor changes:
* Build from the build dir, rather than source. The other way around
works too, but I actually find commands simpler this way.
* Adding artifacts.
Just an initial test to get a hang of the thing, mostly inspired from
GTK gitlab-ci rules adapted to our build.
All in one job (deps, babl, GEGL, GIMP itself) for now, for simplicity
of debugging. We'll see later to break this into CI sub-jobs.
- Only publish the bz2 tarball because that's what we currently provide
on download.gimp.org (let's see in the future for the xz tarball).
- Generate also a sha512 checksum. It's better to do it on the CI rather
than on the download server, otherwise it wouldn't protect against
transfer errors (from gitlab to download server).
- Rename the checksum files to ${filename}.SHA256SUMS (512 respectively)
for easy download without name clash with the global files listing all
the previous releases.
- Disable all meson jobs for tagged releases. They are currently not
reliable and may fail randomly (see issue #6257), even though without
code problems (this is also one of the reasons why autotools is still
our official build system and meson is still deemed experimental).
It's ok for regular builds, but not for tagged builds, which we need
as reliable as possible. The really important job is the distcheck one
for tags.
This reverts commit 32434d9fc3.
Argh sorry. I was planning on making a branch for this (not directly on
master!) and also the patch should obviously have been applied on a GTK
tree, not GIMP's. I'm probably tired! Reverting.
We were not building it by not installing libmng until now. But graphviz
must have pulled libmng. So disable explicitly the option in GIMP
configure step.
This reverts commit e869a11270.
Mitch could reproduce the issue and made each .actions file into
respective CLEANFILES in commit 0052803313. At least now the distcheck
job works, so let's just make its failure forbidden again.
I really don't like to flag the distcheck job as allowed to fail, but
the issue we have with it right now (#5790) is very annoying and I have
no idea where the weird uncleaned files come from. I can't reproduce
this locally and these files are seemingly never created here during a
distcheck.
Since it makes all our pipelines fail, this makes it harder to diagnose
and find real other bugs, so let's allow failure until we figure this
out.
For Python, Lua and Javascript, make the option boolean (with 'yes'
being the default). No need of a warning when not installing the
plug-ins as this would have been disabled explicitly anyway. When
installing the plug-ins, only make interpreter checks as precautionnary
verifications which don't actually change anything (except outputting
some warnings if interpreters are not found). Basically for these 3
bindings, the interpreters are only runtime dependencies anyway. So it
doesn't matter if they are not available at build time. In particular,
we get rid of the 'force' option.
Vala rules do not change as the vala compiler is indeed needed at build
time and current checks work correctly. I just add a "Vala plug-ins"
line in the summary message of the meson configuration, as it was
missing.
This should give a nice name to distribution archives so that they are
not all called `artifacts.zip`. Names will better describe their
contents (target OS or source and short commit hash, because for CI
builds, it's important to know which commit is being tested).
Also replace CI_COMMIT_REF_NAME in other artifact names by
CI_COMMIT_REF_SLUG. Otherwise if a branch has a slash (quite common in
branch names), only the part after the last slash is used for archive
naming.
Finally immediately exits from dependency build with error code (!= 0)
if `crossroad install` command failed.
… the CI.
There are 2 finale steps before finale binary distribution on Windows.
We must compile the GSettings XML schema files and register GdkPixbuf
loaders (for file format support in the GUI).
I used to provide a wrapper to be run inside Windows before first GIMP
run. Never did I realize that I can compile the distributed GSettings
schemas with the native `glib-compile-schemas` (works fine in my tests).
As for the GdkPixbuf loaders, we inspect DLL libraries, hence we do
require the target `gdk-pixbuf-query-loaders` which is unfortunately a
Windows executable. Yet it seems to work fine with Wine, so let's be
done with it in the CI instead of requiring manual steps from testers of
the CI builds. Then a few `sed` calls are enough to make the path in the
produced text file relative instead of absolute (which works fine, again
in my tests at least).
This means that I don't have to distribute the 2 binaries and the DLLs
they depend on anymore. Moreover let's remove the wrapper (but still
generate one which just calls GIMP so that we call it from the tree
root, where it's much less messy).
Note: I failed to install wine32 (32-bit Wine) on the Gitlab runner.
After following all instructions, I encountered weird errors. So
instead, I just make the win32-nightly job depend on win64-nightly and
copy `loaders.cache` from one to another, as it is a
platform-independent text file (as long as we provide the same GdkPixbuf
loaders on both of course, which we do).
The main purpose of these jobs is to only package the strict necessary
for a working GIMP under Windows, i.e. getting rid of all unnecessary
executables, and inspecting binary dependencies recursively to only
package used DLLs.
The dll_link.py script is taken from Siril codebase (see commit a86e82a8
on Siril repository, by FlorianBen). This was a very nice idea, and
makes for much smaller test archive (Siril is also GPLv3 so licensing is
ok for the reuse, also anyway it's just a small independent build
script).
Moreover having it as a separate job allows to have artifacts with only
the finale distribution (artifacts on the build job also have the build
directory and the whole prefix, which we want to keep in order to debug
when needed).
Hopefully I am not missing anything. Siril seems to package more, like
various gdk-pixbuf-*.exe, gspawn-*.exe and gdbus.exe. I am wondering if
these are actually necessary. I could run GIMP fine without these in
quick tests, but I guess I'll have to investigate a bit more to figure
this out. That's what nightly builds are for, after all, so hopefully
people will report if we miss some runtime dependencies.
I don't add this at the end of the distcheck job to make the interface
clearer, and also because the distcheck job will have full build
artifacts (allowing to debug a failing distcheck if necessary), whereas
the `sources` job will just publish tarballs and SHA256 sums generated
from these tarballs. Simple, clean.
The Linux CI job names are too long and are not recognizable on the web
GUI unless you hover the widgets with the mouse to read tooltips. Remove
the "/testing" part (if people want to know exactly which Debian we use
for our builds, they can always look at the script) and move left the
differenciating parts (i.e. autotools/meson/clang/distcheck) so that
these are visible in a glance, even when ellipsing long job names.
Problem with the CI is that the source is straight in our base directory
and therefore installed dependency files as well as cache or built files
are in subfolders.
Let's try to ignore as much as I can see. This should avoid a bunch of
warning and errors during report generation.
Seems that msys2 packages can sometimes be compressed as `.tar.zst`
instead of `.tar.xz`. I updated crossroad to handle this case. It now
requires the additional pypi `zstandard` package.
This is a new feature I implemented in the crossroad cross-compilation
tool. Msys2 repository has more packages and they are more up-to-date
compared to Fedora and Suse cross-built packages (the 2 other available
sources for pre-built Windows packages).
This allows to simplify a lot the dependency preparation for the Windows
CI, and speed things up.
`gegl` binary is being run for icon generations on gimp-2-10.
(cherry picked from commit 97549081fd)
Note: this commit is so far not needed on master as we don't call gegl
during build time unlike in gimp-2-10. But we could at some point.
Better to be thorough.
Also install missing xauth (required by xvfb-run).
`make check` was broken on CI, but got fixed by commits 707d3c6f64,
f6e9c6ee6f, 2c1efdedf0 and of course the present one.
Dependencies must be reinstalled on different jobs, even when related.
Fixes these CI errors on unit testing:
> GEGL-Message: 22:30:35.867: Module '/builds/GNOME/gimp/_install/lib/x86_64-linux-gnu/gegl-0.4/matting-levin.so' load error: libumfpack.so.5: cannot open shared object file: No such file or directory
> GEGL-MESSAGE: Module '/builds/GNOME/gimp/_install/lib/x86_64-linux-gnu/gegl-0.4/raw-load.so' load error: libraw.so.19: cannot open shared object file: No such file or directory
(cherry picked from commit a90d547cc0)
Not because they are working better than the ones in autotools, but
rather because it seems they are simply barely implemented! So of
course, the few tests currently there work.
The old custom ArchLinux got broken (apparently by some package
signature verification which fails, and obviously we don't want to
bypass these for security reasons).
I took the opportunity to port to Debian testing because this is GIMP's
base distribution for support (basically dependency versions must be in
Debian testing) so it makes sense that our CI is based off it as well.
Note though that I am not against additional CI tests so if someone
absolutely wants to get the Archlinux-based CI back and thinks it gets
us some additional worthy test, feel free to fix whatever was broken
then we may add it back (having both Debian testing and Archlinux CI).
Some issues may happen only for 32-bit builds. So it may be worth make a
Windows 32-bit build too.
Issue #2794 triggered this reasonning, though in this issue's case,
simply the build would not have discovered the bug since it was only a
build warning + run-time bug. Still long-term if we manage to get a
no-warning build, it could become relevant to discover more errors at
build time. So preparing the ground work here.
The GNU/Linux builds should start as soon as the Linux dependencies are
built. There is no need to wait for the Windows dependencies (and
reciprocally of course).
This should make for much faster CI total duration (with current
configuration).
Note: this "needs" keyword is quite a recent feature since gitlab 12.2,
3 months ago: https://docs.gitlab.com/ee/ci/yaml/#needs
A few commands need to be performed the first time for glib to work
properly, and gdk-pixbuf loaders to be found. I add them in a wrapper
script so that it's easy to ask people to test the dev builds (even
though it's not necessary to run these commands each time, but who
cares!).
Rather than having the whole Win32 cross-build into the 'gimp' stage,
break the dependencies and GIMP-only builds in 2 stages.
Since apparently we need to keep the same structure for the native and
cross build (otherwise we don't get parallel builds; in other words, I
didn't find the possibility to set separate pipelines up), I move babl
and GEGL into the same 'dependencies' stage.
Finally I remove the -base rules extended into actual jobs, except for
`.gimp-base` (this is the only which makes sense as it is actually
common to the meson and autotools build).
We don't need to pull 5 commits of history. Only the HEAD of the
selected branch is needed.
Also define it globally rather than re-defining it in every job to the
same value.
First replace the "when: on_failure" rule by a "when: always". We indeed
always want to get log artifact so that we can study a build if
necessary (neither only on failure nor on success; really on all cases,
since even an apparently successful build may have issues we might want
to diagnose).
Also expire all artifacts at 1 week (it seems the default on GNOME's
Gitlab is 4 weeks; we don't need to keep these so long. Even a few days
might be enough actually).
As for the artifacts contents, keep the build dirs rather than the
install dirs. Build dirs allow to check configuration logs and other
kind of logs which are the most useful when diagnosing a failed build.
Now install dirs are also interesting. Maybe we should provide them
again at some point. We'll see. For now I comment them out.
Still keep the install dir for dependencies though, since it seems this
is how data are passed from one job to another.
Note that ideally we would like to provide different artifacts depending
on failure or success but apparently this is currently not possible.
See: https://gitlab.com/gitlab-org/gitlab/issues/18744
Also not sure why for GIMP, the CI was only keeping the build app/tests/
directory. We should really keep the whole dir.
It was not necessary when I was only running the cross-build job. Not
sure why it is needed now. What do the parallel jobs share exactly in
this CI system? Anyway…
It looks like Arch does not have mingw64 cross-compilers in core package
repository. It does have some package in the user repository (AUR), but
I assume that such a repository cannot be deemed as safe.
Anyway I still tried, but apparently these AUR packages have to be built
and when I tried, I got this error:
> ERROR: Running makepkg as root is not allowed as it can cause
> permanent, catastrophic damage to your system.
Anyway it's all a big mess. Then I tried to move the cross-CI to Debian
testing, which is anyway our base compatibility system. Unfortunately I
encountered like what looked like some glibc++ macro problem on some
packages (most likely because the pre-built packages I use are Fedora
ones which likely uses a cross-compiler differently built from the
Debian one).
So in the end, for simplicity, I use a Fedora image, then I am sure to
get a good match between the system cross-compiler and the pre-built
dependencies.
This is the base system for our CI, and the former mypaint-brushes
package got bumped to v2, which broke the build. Instead of rolling
back, they added a mypaint-brushes1 package. Let's rely on it instead of
installing it ourselves now.
If I get it correctly, archlinux bumped the mypaint-brushes package to
v2, which broke our CI. The v1 and v2 brushes are not the same, and they
are not even compatible (GIMP only supports v1 brushes so far). These
should be different packages, hence the incremented major versionning.
For the archlinux change, see also:
https://git.archlinux.org/svntogit/packages.git/commit/trunk?h=packages/mypaint-brushes&id=dfd032b3aab40a46c751bd47713fb82bf11bd984
Let's just install the brushes ourselves from the correct branch.
With this kind of changes, as well as the weird patches they do when
they rename the pkg-config file of libmypaint, I think we should
consider not rely on this distribution for our CI.
Jehan