Python 3.10 introduced sysconfig.get_default_scheme() and somehow Debian decided
it was a good idea to move from the 'posix_prefix' to the 'posix_local' scheme.
See: https://lists.debian.org/debian-python/2022/03/msg00039.html
The main issue with this scheme is that it adds a local/ subdirectory to the set
--prefix. As a consequence, with --prefix=~/.local/, scripts are installed in
~/.local/local/bin/ (instead of proper ~/.local/bin/), data are now in
~/.local/local/share/ and so on.
As expected, this broke a lot of CIs for a lot of projects, as well as a lot of
custom scripts, usage of Python in virtualenv (this later case seems like it is
fixed by special-casing it in Python 3.11), flatpak, etc.
Setting DEB_PYTHON_INSTALL_LAYOUT environment variable to "deb" solves this by
changing the default scheme.
This URL is also useful to understand the issue:
https://askubuntu.com/questions/1406304/virtualenv-installs-envs-into-local-bin-instead-of-bin
As a side note, Debian is not the only one which made the mistake. Fedora also
did the same thing (and they have also their own different environment variable
to handle this): https://bugzilla.redhat.com/show_bug.cgi?id=2026979
In the last few days, our deps-win64 job started to fail with:
> $ update-alternatives --set x86_64-w64-mingw32-g++ /usr/bin/x86_64-w64-mingw32-g++-posix
> update-alternatives: error: no alternatives for x86_64-w64-mingw32-g++
The reason lies in this change in Debian testing 10 days ago:
----
gcc-mingw-w64 (25) unstable; urgency=medium
* Upgrade to GCC 12. Closes: #1023679. It is no longer possible to tweak
the installation directory for different thread models, so the -posix
and -win32 packages are no longer co-installable.
* Drop “Built-Using” from “Architecture: all” packages.
* Since the POSIX and Win32 packages are no longer co-installable,
drop support for alternatives and use symlinks to provide the full
set of command names.
* Standards-Version 4.6.1, no change required.
-- Stephen Kitt <skitt@debian.org> Mon, 12 Dec 2022 09:00:34 +0100
----
So from now on, we'll only install the posix variant of the cross-compiler.
Hopefully it will work for all packages we build.
I added this option in commit 38d6783299. It makes the script stateful, able to
remember all previously processed DLL, hence making the whole job much faster
and efficient.
- Use a relative path for GIMP_LOGO_PATH inside the gi-docgen generated
HTML, and not an absolute path taken from build dir (otherwise this
would break, for installed docs, but also for the tarball and the
developer website!).
- Also use either gimp-logo.png or gimp-devel-logo.png depending on
whether we are on a stable or unstable branch.
- Install these in images/ inside the GIMP docs folder, which
corresponds to the relative path given to GIMP_LOGO_PATH.
- The installed root dir will be $datadir/doc/gimp-2.99/, e.g.
/usr/share/doc/gimp-2.99/
- Inside this folder, the library references will be in libgimp-3.0/ and
libgimpui-3.0/ (instead of weird Gimp-3.0/ and GimpUi-3.0/). Note that
the root dir uses the application version (2.99) whereas the library
folder use the API versions. These are different in development phase.
- Archive the gi-docgen installed files, not taken from the build dir,
to avoid packaging temp files, such as the .toml files. Note that
`g-ir-docs` files are still taken from the build dir, as we don't
install them yet.
- Finally package all this in a directory before archiving in a tar.xz,
named the same as the directory (e.g. gimp-api-docs-2.99.13/ inside
gimp-api-docs-2.99.13.tar.xz).
If we want to encourage packagers to test the meson tarball (better to
do this now with development releases than later on stable releases!),
we should name it prominently. Therefore I will rename the autotools
tarball as gimp-*-autotools.tar.bz2 whereas the meson tarball will be
named gimp-*.tar.xz.
Also renaming `sources` CI job to `sources-autotools`.
The `ninja dist` step fails with:
> ERROR: Repository has uncommitted changes that will not be included in the dist tarball
> Use --allow-dirty to ignore the warning and proceed anyway
> FAILED: meson-dist
> /usr/bin/meson dist
Astonishingly if I run a `git diff` (or git status) in the CI, I saw no
changes and the dist suddenly succeeded. When I remove it, it fails
again. And of course, locally it's alright. No way to diagnose!
Anyway it's not a bad idea to leave some diagnostic logs in the CI so
here we go, properly outputting the diff and exiting the job with an
error when there is a diff. Because anyway, there should be none. It
would mean that there is likely some issue in versionned generated
files.
Maybe let's try to distribute the meson tarball next to the autotools
tarball for our next dev release, and announce that packagers are
invited to test the meson build from tarball and report back.
These were originally to distribute cross-built binaries. Nowadays, we
just use the native-made installer, which is also closer to what people
will really get for release versions.
So let's just remove these. I keep the crossroad builds as these are
still useful to detect Windows build bugs quickly, but we don't need
these distribution steps.
This also takes care of failures in the job, but since it's mostly a
useless job nowadays, rather than wasting my time investigating this, I
simplify the CI.
The gnome-runtime-images have been recently migrated to Quay. This is already reflected in the template.
Please note this MR has been created semi-automatically. If it doesn't make sense, feel free to close it.
intltool has long been dead upstream. Let's not poke the dead corpse,
please.
This commit is quite large, but that's mostly since trying to support a
hybrid of both gettext and intltool with both Meson and Autotools was
really hard, so I stopped trying.
Due to gettext relying on quite some things being at the exactly right
place in the autotools build (like `ABOUT-NLS` and `config.rpath`) we
really needed to cleanup the `autogen.sh` to only call `aclocal` and
`autoreconf`. No more strange magic; I tried to do it without changing
too much in the file, and things just broke. If people want to do
something more custom, they can just change the script directly. This
change also uncovered some problems in our `configure.ac`, like using
deprecated macros.
The following major changes happened:
* meson: Changed `custom_target()` to `i18n.merge_file()` for all
supported file types
* Added `.its` and `.loc` files for the GIMP-specific XML formats, so
that gettext understands them
* For the `.isl` (Window installer stuff) file, there's no easy way to
do this in gettext, so instead we start from an XML file (again with
its own ITS rules etc), translate that with gettext, and then use
`xsltproc` with a bit of magic to output the .isl file for each
language
* the `po*/Makefile.in.in` files are migrated to `Makevars` files,
which gettext natively understands.
Fixes: https://gitlab.gnome.org/GNOME/gimp/-/issues/8028
We were already avoiding re-processing a same DLL within the same run
(this can happen when 2 dependencies have themselves a common
dependency). But the dll_link.py script was stateless regarding previous
runs so we might be checking again the same DLLs multiple times (even
though we were not copying them again).
Let's make the script stateful with a new parameter to give a file where
all the previously processed DLL names are stored. I am hoping it would
improve the efficiency of the packaging-win32-native which is suddenly
extra slow (it always times out, even after raising the max job time;
now we time out after 2h30! The 64-bit packaging job just takes 1h,
which is too much already, but still much more reasonable).
Since this is probably much less tested than the default vector icon
case, any bug there might go unseen for some time. So let's add a new
pipeline to check this. I make so this pipeline is not actually run on
every commit (i.e. manually only), then I'll just add a scheduled
pipeline to check the non-vector icon case at regular intervals.
This pipeline also test the icon-list.mk files are not outdated.
Fixes#950
`.gitlab/search-common-ancestor.sh`'s original authors are
Philip Withnall and Frederic Martinsons.
(Jehan/reviewer's note: script further improved by Asalle)
gtk-doc has been slowly dying for the past few years; with gi-docgen we
have a nice successor.
This also makes sure the C documentation also uses the GIR file, which
in turn means faster build times (since all the C code doesn't have to
be parsed and recompiled again), and has a clear dependency graph.
See the [gi-docgen tutorial] for more info on how the system works.
[gi-docgen tutorial]: https://gnome.pages.gitlab.gnome.org/gi-docgen/tutorial.html
This workaround was placed to mitigate race condition.
It was useful for the time being, but not anymore.
Better solution will be introduced in following commit.
Reverting changes introduced in 9c6776fb.
Commit-triggered pipelines on `master` branch are apparently canceling
flatpak pipelines even though the flatpak jobs are not even in the
triggered ones.
In any case, these flatpak pipelines are not triggered often (only
through weekly schedule or manually triggered) so there is no need to
have such a flag.
See: https://gitlab.com/gitlab-org/gitlab/-/issues/340147
I completely forgot that since the installer is built on a Windows
runner, I cannot expect a standard Linux shell syntax.
As a consequence, commit de9a171a19 broke the "win-installer-nightly"
job with the following error:
> The token '&&' is not a valid statement separator in this version.
Rather than trying to find the equivalent command to run on powershell
or whatever, let's just compute the checksum at the end of our installer
creation script.
Somehow the test part of the flatpak job stopped working. The last
instance which worked was 3 weeks ago, but after a lot of debugging I
realized that it is not because of any code change on our side. The
exact same commit which worked 3 weeks ago won't anymore!
The standalone bundle is actually built and works fine when tested
independently. What fails is the `meson test` inside the flatpak
environment. Somehow when GIMP is rebuilt inside the test flatpak
environment, it doesn't build the plug-ins yet one of our tests
(save-and-export) requires plug-ins to open some file formats. Note that
I double-checked, the plug-ins were well built and loading any format
works fine in the standalone flatpak, just not in this specific step.
I am completely unsure what broke, yet it is apparently outside GIMP
code. So for now, I just copy-paste the whole flatpak job which we were
including from another repository and remove the `meson test` part.
Since GIMP looks for its icons at runtime and would output warnings if
it doesn't find them, it's better to install first. Not really sure it's
ideal though, but it will do for now. Maybe I should just g_printerr()
instead of g_warning().
… based on set labels.
If the label "5. Windows Installer" is set, the MR pipeline should
trigger the Windows installer pipeline as well.
If the label "5. Flatpak package" is set, it should generate the flatpak
(not publish it obviously, yet it can be downloaded and installed
manually).
This will allow us to easily test MRs and allow people to test our code
before we merge it to the main branches.
Currently we run the job on a weekly schedule. If we keep a retention
period of 2 days, it means that people will not have access to a nightly
Windows installer 5 days out of 7, which is not useful.
Gitlab has a retention policy not to delete artifacts for last jobs, but
it apparently actually bases this check on being the last pipeline, not
being actually the last job of a given name. Since we have pipelines
running all the time, this retention policy just won't apply to our
installer job which will get deleted.
Cf. https://gitlab.com/gitlab-org/gitlab/-/issues/332142
Without this, the flatpak build is just too long and Gitlab CI just
stops logging it. So we end up with a log saying the quite useless (at
the end):
> Job's log exceeded limit of 16777216 bytes.
> Job execution will continue but no more output will be collected.
We don't even reach babl/GEGL/GIMP builds, which are the most important.
With this little trick, I am redirecting output to a log file and simply
including said log into the artifacts, hence allowing to debug the full
build.
Argh! So it turns out that .publish_nightly template uses already an
only: key so we cannot use rules: (on the other hand I guess using only:
ourselves is alright and concatenate ours and the one in the template).
Fixes:
> jobs:flatpak-nightly config key may not be used with `rules`: only
This clashes with the usage of "rules:" and is even more useless as it
is about checking success of earlier stages (yet this job doesn't need
to wait for any other job).
Fixes CI error:
> jobs:flatpak config key may not be used with `rules`: when
To get a nightly flatpak, GNOME project offers a CI template[0] that can
be extended instead of having to define everything from scratch.
At the core is the "flatpak" job that tries to build the flatpak. If the
build finishes succesfully, the result should be available as a CI
artifact as a flatpak bundle. The app id is the same as on Flathub.
Another part is the "flatpak-nightly" job that publishes the flatpak if
the build succeeded. Without this job, the bundle stays only temporarily.
This job can and should only be run on the "master" branch. This jobs is
started only when the "flatpak" job finishes successfully.
Both jobs can be triggered only thorugh schedules[1]. To finish the
flatpak nightlies setup, a schedule needs to be set.
[0] https://gitlab.gnome.org/GNOME/Initiatives/-/wikis/DevOps-with-Flatpak
[1] https://docs.gitlab.com/ee/ci/pipelines/schedules.html
The logs are just too long for gitlab and ends with:
> Job's log exceeded limit of 4194304 bytes.
This doesn't prevent the job from actually finish successfully. Yet the
day when we will have a build issue, we won't have a way to debug if it
happens close to the end of the installer creation. So let's redirect
both stderr and stdout to a log file and include it in the artifacts.
Unlike what I said in my previous commit, it still just takes too long
to build the installer, whether I move some of the substeps around or
whether I increase the max duration. So rather than increasing max
duration a lot more, let's just add an intermediate stage.
Jehan