This commit uses definitions and references to reduce repetition of CI code.
Some people (like the GitLab devs?) prefer (almost imposes) this to be done in
multiple .yml's using 'worfklow:' but this makes the CI way more complicated
and very slow to read. So, let's just keep simple in one human readable file.
---
Other changes are:
- The custom native arch pipelines now they always generate a .zip package.
This helps to ensure that everything is in place and facilitate testing;
- The pacman update (-Syyuu) was removed since it is auto done by runners;
- The ccache config was moved to .yml. This does NOT affect local builds
since meson saves ccache by default for user in '%LocalAppData%/ccache'.
These commits introduced sensible drawbacks in our CI file organization.
Some of them I already commented in the MR but was "ignored" back then
probably due to the complexity of the MR. So, the let's revert:
- Unecessary repetition of libomp installation (4 times, now 1)
- Unecessary repetition of submodule initialization (6 times, now 4)
- Creation of new variables in the hope of more clarity. But,
they aren't consistent with the gimp-web-devel instructions
nor with other systems variables. This is terrible to undestand.
Please, let's not reinvent the wheel since this is a bad example to
packagers, since they tends to isolation and overly-engineering.
Also, made some little adjustments that I should have done in previous MR.
---
This MR was splitted. Even more simplification will be done in the future.
- With last commit, the Windows installer pipeline doesn't depend on
"gimp-debian-x64" job anymore since a native Windows build is now able to run
GIMP (or gimp-console) as a build-time tool as well. It makes the Windows
installer pipeline (and full custom native builds) self-sufficient.
- On the other hand, "gimp-win-x64-cross" and "gimp-win-x86-cross" now require
"gimp-debian-x64" since cross-compiling GIMP now requires a native GIMP in
order to generate some image data (such as the splash image, and probably soon
logo or icons, etc.). See gimp-data@5a03c71.
- Getting rid of "image-win-x64-cross" and "image-win-x86-cross" in favor of
"image-debian-x64" for all Debian as well as the cross-compilation jobs. They
are all based on the same Debian image (it was debian:bookworm for native
Linux jobs and debian:testing for cross-builds; now it will be debian:bookwork
for all) and it's just a few more packages (cross-compilation C and C++
toolchains) for the cross-builds. Moreover now the cross-builds also need the
native GIMP binary around, therefore native dependencies are needed as well.
It makes sense to factorize all 3 images into 1.
- Make sure we don't build bindings when cross-compiling since these won't work
in this case.
- Splash images will now be stored from gimp-data.
- The installer BMP image scripts also move in the same time.
- We don't need devel and non-devel variants of the BMP images in InnoSetup
scripts since the images are generated from the actual splash.
Meson subprojects just have too many problems and limitations and I can foresee
the maintenance headache and the future incoming false-positive bug reports if
we start using meson subprojects.
Comparing to the simplicity of git submodule which also has much better
notifications to help people understand when the submodule is not in sync and
how to remedy to it.
See commit gimp-data@c364adb explaining the main reasons in detail.
In our efforts to use Clang, now the nightly (in fact weekly) flatpak
is built with Clang too, not only GIMP but also the dependencies.
* However, not aalib. We welcome fixes regarding this cursed lib.
Additionally, updated some build options from some deps.
This commit unites two scripts (.sh + .bat) in one .ps1. PS was choosed, since:
1) We don't need MSYS2 to distribute with Inno at first. Now, who wants to
just distribute GIMP can do it easier natively without a separate environ.
2) PS is actually pretty solid and have a good number of cmdlets, unlike the
ancient CMD, which barely supported more than 9 parameters in .bat files.
Consequently, some redundant variable checks have been removed from the .ISS.
Also, changes the name of the job and script for better consistency with the
upcoming MS Store job.
The 'Docker' draft of GIMP was introduced in 09/04/2017 and then died ONE day
after, mostly because of "bad" timing (few years later we moved to GitLab CI).
Similar (in fact, even worse) than 'jhbuild' folder, update it to keep the
current development status is so hard that it is equal to do it from scratch.
So it is reasonable to delete it anyway.
These files are not deleted when the Unstable (GIMP 2.99) or Stable (GIMP 3)
version are installed because the root is different. So, they can be removed.
It was only used for the gimp-win-a64 job and was coming from MSYS2 repository
which already dropped it:
a98352b2ba
The first patch is still needed as the upstream fix is meant to appear in clang
18 according to bug report, yet our CI still uses clang 17.0.6. See:
https://github.com/llvm/llvm-project/issues/51286
Since !1171 we can use the CI scripts locally, but the initial implementation,
after some commits, revealed to be troublesome. So, now reworked to be simple.
As noted by Jehan, the way of defining the (many) GIMP versions can change.
So, let's take the GIMP version directly from the generated config.h file.
Also, changes the declaration of some variables to conform with "best pratices".
This list is referenced in the gimp-web-devel site, and my initial
implementation, in terms of ease to use, was a throwback.
Also, make the CI code more simple without changing the list file.
The rest of the time, we want to use HEAD of their main branches.
The last ctx update seem to have broken the CI build on Windows. While we'll
have to fix this, let's do this after we release GIMP.
Anyway we obviously don't want to use unstable code for actual point releases,
so it's a good thing that this made me think of adding such rules.
Here, the instructions was hard to find in the middle of maintainer stuff.
Also, be able to build GIMP without dev-oriented distros is something very
desirable, and the gimp-web-devel is the suited place fot it.
Aaaargh! I think this was the main issue which would explain why the problem was
mostly happening on the packaging job. In this job, we were still consistently
calling `pacman --noconfirm -Suy` (i.e.g "Synchronizing package databases")
which we had stopped to do for the CI builds as that was highly increasing
chances of locking pacman's process.
Fixing a case where the script continued after a failed `ninja`. After further
testing on how `set -e` works, it looks like a toplevel `false && true` command
will fail the script (which is what we want), but it won't when inside an `if`
block!
Breaking it down into 2 separate calls fail the script correctly though. So
let's do this.
MSYS2 pacman gets randomly stuck on Windows/Aarch64. The actual issue is still
being investigated by upstream projects, though anyway it's bad for us right
now, to the point that there are discussions to remove Aarch64 support from the
Windows installer (whereas it just got added recently!) in #10729.
This is an attempt to a workaround. Instead of getting stuck forever and waiting
until the whole job times out (per Gitlab CI settings), I time-out (after 3
minutes) the pacman command within our script and try again, up to 2 more times.
Hopefully one of the calls would succeed.
I also send a SIGKILL through the timeout (though I have no idea how signals
translate to Windows processes) and run again taskkill after this, which may
seem overkill. Interestingly I get output for both, which seems to indicate that
the kill succeeds in both cases (because of several processes?).
Anyway clearly it's a bit of random code not completely understood, but the
inability to test this all locally clearly doesn't help so it's good enough for
the time being.
See: https://github.com/msys2/MSYS2-packages/issues/4340
While this is the job of `set -e`, putting parts of the script in shell
functions hid error returns in the build-deps-msys2 script.
Cf. #10729.
For the build-gimp-crossroad script, let's just add `set -e`.
Cf. #10725.
The list contains, in addition to the GIMP deps, all babl and GEGL deps.
So, let's use it.
Also, nothing new under the sun. We already use a unique list on the
Windows build instruction page (for stable branch) and do the same to
Debian image job.
As suggested by Jehan, the manifest will be configured by meson.
Of course, for CI compliance, this pushes the job to a further stage:
'packaging', which is what Flatpak is about, after all.
The distribution job name has also been changed, because when Store job
is merged, this will be the new nomenclature of the distribution jobs,
which was not changed since !1171 by lack of time.
There is no point in keeping .patches in our source since we aren't applying
them and there are "backup" MRs in GTK: GNOME/gtk!3275 and GNOME/gtk!4432.
Now, the script reads the major.minor version automatically from the main
meson.build file, reducing maintenance work, which is already especially
high regarding flatpak because of the natural hashes updates etc.
Also, a small organization made to make the code more humanly readable.
As consensus on Windows contributors, it is more useful for most users
the generation of the desktop shortcut by default.
Now, users who don't want it should uncheck the option in custom install
or manually delete the shortcut after a full install.
The 'jhbuild' build of GIMP (Windows only) was added in 2013 and then maintained
by only one contributor (it even offered unnoficial nightly builds) until 2017,
when it never received a new jhbuild-specific commit again.
After so long, update it to keep up with current development status is quite hard,
so it is reasonable to remove this already dead version of GIMP.
* Fixes an ARM .patch that stop to working after !1171, causing job fail
* Changes the logic of system upgrade (pacman -Syyuu) to be less redundant
This also improves consistency with the Debian jobs
- Fix "no icons" errors generating loaders.cache with .cmd (CI-Cross).
! This is a sub-optimal fix, but it's better than a useless build.
- Fix "no interpreter" errors generating .interp with Meson (CI-Native)
and generating .interp and copying .typelib with .cmd (Local-Native).
! This is a sub-optimal fix to Local-Native, but plug-ins will work.
- Fix "no iso" error copying iso_639.xml with Meson (Local-Native).
The Inno installer scripts contents (only 3 files: files, gimp3264 and
32on64) and filenames have been organized, making them much easier to
read, and slightly less hardcoded so less prone to being misunderstood
and pervasively receiving packaging stuff.
Just to be clear, one more time: the Inno installer (or future MSIX)
scripts never should be the center of attention. This "installcentrism"
caused a domino effect of partially "abandoning" the packaging, build.sh
and the meson scripts, which explains the existence of this MR...
(Some things still hardcoded since wildcards in Inno are very limited.
Also, the rational ordering principles of this MR were not applied since
these scripts are heavily based on the x86 .zip package and changing the
order of things here, according to my tests, breaks things quite easily)
The CI crossbuild job now use the same main script (and the same
sub-scripts of linking and debug generation) of the native CI for
packaging. These unified scripts greatly facilitates maintenance.
The crossbuild deps script is now more consistent in relation to the
native one. As this is polished, the cross one is now polished too.
A crossroad gimp build script was created with a more clear code.
Also, finally make the script for packaging only, removing build step
stuff that shoudn't be here (glib-schemas generation) since this causes
disparity with the Local native build; and adding some packaging
decisions that shoudn't be in the installer scripts (eg. specific
folders of ghostscript, glib; no share/themes), which also facillitates
INNO (and future MSIX) maintenance.
The local builds now can use the same script of CI for sake of time.
The compatibility layer is very simple and makes the script more
used/tested, therefore more reliable.
The local builds now can use the same script of CI for sake of time.
The compatibility layer is very simple and makes the script more
used/tested, therefore more reliable.
Also, remove some redundacy on the code for better maintenance.
Now, we have only ONE list applicable for build-deps-crossroad.sh,
build-gimp-msys2.sh and package-gimp-msys2.sh, instead of hardcoding
3 times. This unified list greatly facilitates maintenance.
Debian changes:
- Since autotools has gone, we don't need to specify 'meson' in the
debian job and others.
- The "INSTALL_PREFIX" was renamed for the more usual "GIMP_PREFIX" and
the meson sintax of Debian jobs was also updated.
Windows changes:
- Then, clarify that the win64-nightly and win32-nightly jobs are, in
fact, in the 'packaging' step, since we don't really "distribute" GIMP
in .zip and the commands are almost the same of the packaging .SH
script, without scripted optimizations for Inno Installer (or future
.MSIX), crucial for distribution.
- We don't need to specify "native" sufix in any build since they are
the rule and cross builds are the exception.
General changes:
- The job names was changed to be more consistent and in accordance
with the folders present in the artifacts.
- The 'nightly' sufix was removed from the Inno Windows Installer job
and others, since this doesn't reflect the real build frequency.
- The scripts filenames are altered to stay "in order". This is not
essential but ultra convenient since it is easy to view and search.
(The -uni suffix is explained in a further commit)
- All artifacts names now have the commit to avoid apparently duplicate
files when downloading same step artifacts from different projects.
- Finally, rearrange the order of jobs rationally: first the OSes and
archs (from the most free and modern to the most closed and legacy),
then the stages (from 'prepare' to 'analysis'), ending with the
frequency of jobs (from the most frequent, called at each push, to
the least/weekly).
Overall, this changes, although difficult to review at the first
sight, will avoid in the future quite "dumb" issues like:
GNOME/gimp#10195
This is in part a port of commit 6f921b27bb from gimp-2-10 branch, except that I
could not easily cherry-pick because too much had changed already in the master
branch.
There are also a bunch of additional changes in some other glue scripts.
Also some rules which were in the 2.10 branch don't apply to the main one, such
as Python being copied from the x86 binaries, since clang-aarch64 has Python3
(unlike Python2).
See discussion in #9170 and !1091. It looks like this soon won't be a problem
with a recent llvm-windres (which is now patched). But until then, let's patch
upstream GIMP but only for this build.
Note: the additional `rm` call was done for this error:
> + mv _install-arm64 /home/SYSTEM
> mv: cannot move '_install-arm64' to '/home/SYSTEM/_install-arm64': Directory not empty
I guess the home directory is not properly wiped-out between runs, which is not
a huge problem as long as the runner is private to our project. Let's clean
things ourselves.
This localization was added recently but it was not in the installer scripts.
Discovered with unit testing: meson test gimp:build / windows-installer-langs
Organizing sub-dependencies into modules helps. Later if for instance we need to
remove or change OpenEXR, we know that they are related and that Imath can be
removed if we remove OpenEXR.
See: https://gitlab.gnome.org/GNOME/gimp/-/issues/9653#note_1780587
Looking at MSYS2 logs, it looks like they very recently "fixed" the search paths
for lua files, which in turn broke our workaround (searching in subdirectories
of bin/ instead of share/ and lib/).
This should work better (though untested) with the workaround removed now.
Commit on luajit package at MSYS2:
703c7bae2f
The creation of the BMP welcome images for the Windows installer (part of
-Dwindows-installer=true build option) fails in the Windows job. After much
debugging, I could run GIMP, yet it was not enough. One of my hypothesis so far
is that the environment variables for DLLs won't work, since all the DLLs must
be in the same directory as the main binary (though with the WSL thing, I am
unsure, maybe it is still supposed to work), which only happens once GIMP is
installed. So GIMP runs successfully but not plug-ins.
Anyway I wasted too much time working on this and without a local Windows, it
just takes too long (mostly testing thanks to the CI) and is frustrating. Let's
just move to building both the localization files and the images on the main
Debian job (gimp-meson-debian), then use these as dependencies of the
win-installer-nightly job, i.e. when building the installer.
The common order logic for list of directories in environment variables is that
left paths have precedence. This is at least the case for LD_LIBRARY_PATH (and
probably GI_TYPELIB_PATH too).
Make sure that our local libraries and introspected binaries (in the build
directory) are used and not any version installed on the system or by previous
`ninja install` calls.
This was a first attempt at fixing this error on the CI:
> Cannot open display:
Though it was not enough (see next commit calling plug-ins as non-interactive
when called without interface), it is still a useful change overall.
… being installed.
There is already most of the main code logic for this, though now plug-ins need
to be in their own subdirectories, which breaks for plug-ins/common/ and
plug-ins/python/, while I needed plug-ins in both these categories to generate
the Windows installer welcome images (file-png, and python-fu-eval in
particular).
Once again, meson was not very helpful, since all its functions still refuse to
output generated files in subdirectories, so I end up duplicating plug-in files
with a custom Python script.
This should fix the CI. It was working on my machine as GIMP was installed, but
such a build rule should work even without GIMP installed.
This will also be useful in the future when we'll want to run unit tests of
plug-ins through the finale GIMP binary itself.
After discussion with Jernej, InnoSetup should now work better with rescaling
a big image properly to the window size, yet the ratio should still matter.
Apparently the welcome image is a hack and this is why it requires specific
ratio images. We don't use the big size yet, but since Jernej told me which
dimensions are expected, I already added the code for it to make it easier
later.
So anyway this code would allow us not to have to commit welcome images each
time, which are basically resized copy in BMP of the splash screen, slowly yet
surely filling up our repository with image duplicates.
After all, we develop a scriptable image editor! We should use it to edit images
and export in expected formats!
I only use this script for the devel installer for now, for testing and see how
it goes.
While some packages may be needed only when building (and others only when
packaging), we should probably have a shared list of packages needed for both
steps so that we avoid discrepancies which lead to missing libraries in our
installer.
See: https://gitlab.gnome.org/GNOME/gimp/-/issues/9653#note_1777596
Don't set both a branch and a commit, otherwise flatpak-builder will
compare the HEAD of this branch (which may evolve with the commit).
Fixes:
> Failed to download sources: module qoi: Git commit for branch master is dfc056e813c98d307238d35f7f041a725d699dfc, but expected f6dffaf1e8170cdd79945a4fb60f6403e447e020
Because of this, the script was failing to get the version string, which
in turn was breaking InnoSetup.
This fixes the following InnoSetup bug:
> Error on line 116 in C:\_r\_builds\k3_3muaB\0\GNOME\gimp\build\windows\installer\gimp3264.iss: Value of [Setup] section directive "VersionInfoVersion" is invalid.
… for Windows.
Though it's useless for actually building the GIR files, we still need
this package now, for building script-fu with introspection abilities,
to generate GIMP and GEGL enums.
See the 2 previous commits for more information.
Switch to NASA-maintained cfitsio library for loading/exporting FITS images.
This allows us to import compressed FITS files (GZIP, HCOMP, PLIO, RICE) in
8/16/32 bit and float/double precision. It also simplifies export code using
the built-in cfitsio APIs.
It is actually available in the SDK but was removed from the runtime (relatively
recently, it would seem). As a more general rule, it seems that GNOME is phasing
it out slowly in favor of libappstream. So probably we should do the same
eventually.
Yet for now, to at least have a working nightly flatpak, let's add it to our
package.
I'm actually syncing with a branch which I can't test right now because flathub
seem to have some breakage. For the nightly, let's just directly push as anyway
we can't test in Gitlab MRs either because of the non-master jobs timeout.
See: https://github.com/flathub/org.gimp.GIMP/pull/202
We should always keep the diff between these files to a minimum.
- Fix the markdown styling.
- Add commands on how to build GIMP from the local repository instead of a brand
new clone (otherwise I don't see how one could develop with flatpak). I knew
it were possible, but until today, I never tried to do this so I had to test
first.
- The cron file was from the very early flatpak experiments before Flathub came
into the picture, as well as the GNOME Nightly repository. Back then, we
wanted to set up our own nightlies or release repository through a cron.
- It is still interesting to keep some instructions for local builds of the
flatpak as some people want to use this for development (but all the part
about exporting to a repository, signing, etc. is now unneeded for such use
case). So I'm updating the howto to more current recommendations.
- `flatpak-howto.txt` renamed to `README.md`.
Only the libwmf patches are still different. Apparently we may have fixed the
same bugs in different way on both branches. We should look later in details to
see if some patches are better than the other.
It's probably unneeded as the 2.99 installers are transient and anything
installed by GIMP 2.99 won't be in stable 3.0 anymore. Still, it's nice not to
have any weird warnings even in dev releases.
Now that we bumped our meson requirement, meson is complaining about
several features now deprecated even in the minimum required meson
version:
s/meson.source_root/meson.project_source_root/ to fix:
> WARNING: Project targets '>=0.56.0' but uses feature deprecated since '0.56.0': meson.source_root. use meson.project_source_root() or meson.global_source_root() instead.
s/meson.build_root/meson.project_build_root/ to fix:
> WARNING: Project targets '>=0.56.0' but uses feature deprecated since '0.56.0': meson.build_root. use meson.project_build_root() or meson.global_build_root() instead.
Fixing using path() on xdg_email and python ExternalProgram variables:
> WARNING: Project targets '>=0.56.0' but uses feature deprecated since '0.55.0': ExternalProgram.path. use ExternalProgram.full_path() instead
s/get_pkgconfig_variable *(\([^)]*\))/get_variable(pkgconfig: \1)/ to
fix:
> WARNING: Project targets '>=0.56.0' but uses feature deprecated since '0.56.0': dependency.get_pkgconfig_variable. use dependency.get_variable(pkgconfig : ...) instead
… Windows installer localization.
There are kind of 2 separate bugs here:
- Direct i18n.gettext() to the proper data directory where to find the
ITS file. Otherwise `meson compile gimp30-windows-installer-pot` and
`meson compile gimp30-windows-installer-update-po` complained about
not knowing XML and falling back to C, which is obviously a problem:
> /usr/bin/xgettext: warning: file 'build/windows/installer/lang/setup.isl.xml.in' extension 'xml' is unknown; will try C
- Set gt:escapeRule to "no" in the ITS file, otherwise the XML entity is
kept as-is in the po file (i.e. "&" stays "&" inside the po
files), but it's considered as raw text when merged back to XML, i.e.
that the '&' is properly converted to a XML entity, so we end up with
a double escape "&".
Now the po file will have a '&' which will still be converted to XML
entity at merge time. This is actually most likely better than asking
translators to handle XML entities themselves (with the possibility to
make typos and break the XML entity).
See https://savannah.gnu.org/bugs/?58643
The Hungarian language file for the Windows installer was recently moved
from unofficial to the officially supported languages. However, a new
release including Hungarian by default is not available yet. This causes
our CI to fail because it can't find Hungarian in unofficial.
We change our ci script to download Hungarian from the correct location
for official languages, and adapt gimp3264.iss to reflect the correct
location.
The language files provided by the InnoSetup project (either the main
ones or the "Unofficial" ones, i.e. less maintained ones) at least
provides the name of the language, possibly in English, ideally
self-localized in its own language.
Unfortunately Kabyle didn't have any language file so we were using the
Default one, which ended up showing the lang as a duplicate (and very
wrong) "English".
With this commit, I add code to provide our own very basic base language
file, which would at least contain the language name. There is also a
concept of language ID to be verified in Windows-provided list.
Unfortunately it doesn't have any (actually it was id-ed 0x1000 like
many other languages, which looked therefore to be the code for an
unsupported lang). InnoSetup docs tells us to leave 0 then. We can add
the ability to set a specific code later in the template if we add other
un-provided languages and if they have their own lang id.
With this base infrastructure, we should be able to better support more
languages.
Unfortunately the weird encoding of a string to bytes to get the UTF-8
BOM worked on my local machine, but not on the Windows CI. I'm not going
to fight it and fallback to a shell script.
I am guessing it should work fine on all platform since we use basically
the same sed call in build/windows/gitlab-ci/installer-gimp-msys2.sh
already.
Inno-Setup absolutely requires it to recognize UTF-8 translation files.
This should hopefully be the final fix to #8338.
Note that this fix is full of workaround for meson bugs or limitations.
While it was a one-liner in autotools, added to the existing rule, here
I have to add an additional (non-relevant) target rule, then uglily work
around 2 bugs:
https://github.com/mesonbuild/meson/issues/1564https://github.com/mesonbuild/meson/issues/7696
I can't say I'm so happy about the resulting change, even though it
seems to work. If anyone can propose a nicer build rule, it would be
welcome.
Whenever we have an element without translation, we try to use the value
without a `xml:lang` attribute. That selector was wrong though, which
leads to https://gitlab.gnome.org/GNOME/gimp/-/issues/8338, which should
now be fixed.
Why:
1) users can install .scm scripts to plug-ins dir
2) Crashing scripts do not crash extension-script-fu
Scripts (.scm files) have a shebang and are executable
and in a same-named subdir of plugin dir.
Interpreter/scripts create PDB procs of type PLUGIN unlike extension-script-fu
which creates PDB procs of type TEMPORARY, owned by extension-script-fu.
Unlike other interpreters, the interpreter is-a plugin outright,
not by virtue of the script subclassing GimpPlugin and using GI.
More details in /plug-ins/script-fu/interpreter/README
(1) On recent meson versions, it fixes this error:
> extensions/goat-exercises/meson.build:108:0: ERROR: i18n.merge_file keyword argument 'output' was of type array[str] but should have been str
As docs explains, 'output' only accepts one item in i18n.merge_file().
This bug also happens on older meson (but there the reported error is a
lot less useful as it doesn't mention local meson build code).
(2) `setup.isl.xml` is a temporary intermediary file used to create the
Windows installer. It must not be installed.
(3) `gimp30-windows-installer.mo` itself is only used to create
`setup.isl.xml`. It must not be installed as well.
(4) gimp-tips.(its|loc) files (same for gimp-tags ones) should not be
installed. They are only temporary data.
(5) Fix environment variable: s/GETTEXT_DATA_DIRS/GETTEXTDATADIRS/
Fixes:
> /usr/bin/msgfmt: cannot locate ITS rules for ../../../data/tips/gimp-tips.xml.in
(6) Fix various bugs in the *.setup.isl files creation in autotools
build (typo, wrong files used, order of options in `xsltproc`
apparently meaningful, and so on. I guess the autotools build was
not as well tested as the meson one :P).
(7) Fixing the unit test verifying language lists consistency.
(8) `setup.isl.xml.in` must be added to the distribution.
intltool has long been dead upstream. Let's not poke the dead corpse,
please.
This commit is quite large, but that's mostly since trying to support a
hybrid of both gettext and intltool with both Meson and Autotools was
really hard, so I stopped trying.
Due to gettext relying on quite some things being at the exactly right
place in the autotools build (like `ABOUT-NLS` and `config.rpath`) we
really needed to cleanup the `autogen.sh` to only call `aclocal` and
`autoreconf`. No more strange magic; I tried to do it without changing
too much in the file, and things just broke. If people want to do
something more custom, they can just change the script directly. This
change also uncovered some problems in our `configure.ac`, like using
deprecated macros.
The following major changes happened:
* meson: Changed `custom_target()` to `i18n.merge_file()` for all
supported file types
* Added `.its` and `.loc` files for the GIMP-specific XML formats, so
that gettext understands them
* For the `.isl` (Window installer stuff) file, there's no easy way to
do this in gettext, so instead we start from an XML file (again with
its own ITS rules etc), translate that with gettext, and then use
`xsltproc` with a bit of magic to output the .isl file for each
language
* the `po*/Makefile.in.in` files are migrated to `Makevars` files,
which gettext natively understands.
Fixes: https://gitlab.gnome.org/GNOME/gimp/-/issues/8028
This will allow to use the official Windows installer directly in the
Windows Store, as per the new proposed workflow by Microsoft.
Nevertheless our GIMP for Windows has a built-in update check which
would check if a new version exist and warn people (advising them to go
on the website and download the new installer to update). We obviously
don't want this on the Windows Store which has its own update channel.
It would be confusing.
Therefore I added a feature to disable the built-in update check (not
even showing in Preferences) by tweaking a single package variable. The
installer now comes with new option /DISABLECHECKUPDATE=true which will
add said variable.
JSON does not support comments and their support in Flatpak is possible
through use of json-glib[0]. This is problematic in fully utilizing
flatpak-external-data-checker because its JSON writer does not respect
existing comments. To solve this, make use of Fltpak's specific
behaviour where a "//" key is explicitely ignored while parsing[1].
[0] flatpak/flatpak-builder#363 (comment)
[1] 0e98b7ae19/src/builder-utils.c (L1250)
OpenEXR and mypaint-brushes offer next major versions. GIMP is probably
not ready for the changes in OpenEXR and mypaint-brushes are not yet
finalized. Put their versions under constraints so that automatic PR
creation for dependency bumps can be enabled.
It looks like the gi-docgen build is broken on Windows (though the CI
does show neither stdout nor stderr output, just a failure without
message). This should be fixed, but it's not necessary for the installer
at least.
Note: on autotools, the gi-docgen step works fine on Windows.
We didn't need to do this on the autotools build, simply because the
configure step is much more elaborated there, and was checking for the
header file as well as well as a working mng_create() API. But since
libmng was broken, the test failed, so we didn't need to disable it.
By the way, we should check when the `.pc` file was added, because if it
was after the required version, then the meson test is very wrong. It
should not have been different from the autotools file.
The meson build still has a bunch of issues and build bugs compared to
the autotools build, nevertheless the last blocker issue was dealt with
a few days ago (PDB source generation).
Moreover since the meson build on Windows especially makes such dramatic
difference, in terms of build speed, this is a big improvement for
Windows contributor's comfort, and as such is one less barrier of entry.
Anyway I believe that most Windows developers build GIMP with meson now
so sticking on autotools on this platform is just counter-productive.
This is why it was decided to now make meson the recommended build
system on Windows, as a further step toward a move to meson. It is still
not the recommended build system on the other platforms yet.
The --debug option so far would only output debug info. I want both the
run to actually occur and the debug to be printed, at least in some
cases. So I make this a choice option with 3 variants (no debug, debug
only and run + debug).
Similar to the change I pushed to gimp-help repository:
intltool-extract does not consider the `;` isl-syntax comments because
we tell it it is in the ini format. So let's have our source in actual
ini format (POTFILES does not like to look for translated strings in
constructed files, only from source files), then create the .isl files
from it in 3 steps (first, transform the comments in isl format; then
merge all language in a single .isl file; last generate one .isl per
language).
There were already a bunch of comments in the setup.isl.in file (though
not showing in the po files until now). I checked if they seemed
relevant, fixed some, completed others, duplicated the ones which match
several strings, etc.
(This improvement to allow comments for translators in the installer po
files was requested by Piotr Drąg on the gnome-i18n mailing list)
Some tools have been moved. `aclocal` (and likely other tools, but this
was the first one making an error in the deps-win*-native CI jobs) is
now in `automake-wrapper` package, which itself is a dependency of
`autotools`.
Cf. https://github.com/msys2/MINGW-packages/issues/11114
Instead of replacing various needed macros with random hardcoded values,
just make them mandatory and exit with explicit error when a needed
macro is missing.
Since 2afa019c70,
the Meson configure step will fail when building from tarball (missing `.git` and `gitversion.h` is present)
because `gitversion_h` variable will contain a `list[file]` but `custom_target` does not like that in its `depends` kwarg.
We were already avoiding re-processing a same DLL within the same run
(this can happen when 2 dependencies have themselves a common
dependency). But the dll_link.py script was stateless regarding previous
runs so we might be checking again the same DLLs multiple times (even
though we were not copying them again).
Let's make the script stateful with a new parameter to give a file where
all the previously processed DLL names are stored. I am hoping it would
improve the efficiency of the packaging-win32-native which is suddenly
extra slow (it always times out, even after raising the max job time;
now we time out after 2h30! The 64-bit packaging job just takes 1h,
which is too much already, but still much more reasonable).
This includes:
- "copy-icon" set
- Permission and cleanup rules updated
- Exiv2 bumped to 0.27.5
- Adding an x-checker-data for OpenEXR
- Poppler bumped to 22.01.00
- OpenBlas bumped to 0.3.19
- graphviz bumped to 2.50.0
Also improving a bit the download script by specifying the .isl or .islu
file extension. It's nicer than trying to download randomly, and also it
allows to better compare the list of downloaded files with the list in
gimp3264.iss script.
My previous command was also adding a linefeed just after the BOM. While
I'm not sure it would really break anything for processing these, it's
anyway much more correct to only add the 3 BOM bytes. So here is the
improved command.
Also some language files are supposed to be UTF-8 yet they are missing
the BOM markup (only method to recognize them for InnoSetup). This is
the case for Chinese Traditional. See issue #7676.
Make sure that this lang file has a BOM.
One is a patch for a GExiv2 version below the one we now require, the
second is a patch for GIMP which is now committed and pushed.
Cf. commit 7ed68556c7.
gtk-doc has been slowly dying for the past few years; with gi-docgen we
have a nice successor.
This also makes sure the C documentation also uses the GIR file, which
in turn means faster build times (since all the C code doesn't have to
be parsed and recompiled again), and has a clear dependency graph.
See the [gi-docgen tutorial] for more info on how the system works.
[gi-docgen tutorial]: https://gnome.pages.gitlab.gnome.org/gi-docgen/tutorial.html
Also when working on out-of-tree builds, it would not find the file
anyway and we get this output:
> chmod: cannot access 'test-installer-langs.sh': No such file or directory
Though this was not breaking the tests, it has clearly been useless
until we see this one today.
The patch we needed to test needs completion, so it's of no use to
continue building it until this happens.
Also for some reason, the x86_64 build of GTK3 takes forever and times
out (the same build for 32-bit x86 is done quickly as expected) on
repeated occasions. Since this is unneeded right now, rather than
wasting time on this, I just delete this dep build to use the pre-built
MSYS2 package.
Rewriting commit f8cdec1883 by Jernej Simončič for the development
code (which has a slightly different list of languages).
Also adding the meson version of this change.
Since now InnoSetup fully supports UTF-8 isl (as long as they have a
BOM), let's stop converting translations. Then we also avoid all
conversion errors and get a simpler/more robust build process.
I noticed in our build logs such output:
> Saving to: ‘Basque.isl.53’
Wget does not override same-named files and would append a number. The
thing is that we are not supposed to have other .isl files over there,
but I think current Windows runners on Gitlab are not properly wiped
out. That must be why we get remnant of old files.
Anyway this will make sure we override, hence use the last version of
translations (otherwise we are stuck to old versions as long as they are
not wiped out, since the downloaded file is not properly named).
This tool seems like a mess with various implementations (maybe
distributions patch it?). I'm trying to tweak the script so that it
works at least on the Debian testing/bookworm CI and on my Fedora 33
desktop (and hopefully on more, if not all distribs).
Note that I first thought to use the full option name (`--auto-display`)
but I just got the same error again. It seems xvfb-run from Debian
bookworm just doesn't have this option, and only the older option
--auto-servernum (which is said to be deprecated in favor of
--auto-display in Fedora's xvfb-run for instance, but apparently not on
Debian). At least this works on all distributions (or so it would seem).
Fixes:
> xvfb-run: invalid option -- 'd'
The [[ ]] is not POSIX, hence less portable.
I see our `ninja test` calls in Gitlab CI output a:
> /builds/GNOME/gimp/build/meson/run_test_env.sh: 7: [[: not found
(though it doesn't break the tests)
I forgot to do this so GIMP 2.99.8 official release is marked as
"unknown" instead of our official build. It's alright for this one
(especially for a dev release), just setting this straight for further
builds.
Anyway we disabled use of ccache in an earlier commit 2da70b3fb7 because
of a bug in MSYS2's CPython. So there is no need to call these commands
either. Also it seems to be breaking the 32-bit native Windows build
(from CI log, I am unsure this is because of ccache, but the break
happens just after running `ccache --zero-stats`).
This is untested on my side, because the bug only happens on native
builds with meson (our CI has cross-builds with meson and native builds
with autotools and I only do cross-builds locally) but I think/hope it
will work.
Basically we were using .full_path() because these rc files were also
used as input of some configure_file() calls which doesn't like custom
target objects as input (it wants strings or file objects). Yet a bug
in meson didn't like the colon used in native Windows full paths ('C:'
and such) when used in windows.compile_resources(). This has been fixed
by Luca Bacci in: https://github.com/mesonbuild/meson/pull/9368
Yet we just cannot depend on very early meson (or worse dev meson code).
On the other hand, if the input is a custom_tgt object, it uses the
object ID which we give as first parameter of custom_target() so we know
it's appropriately named without colons (such as 'gimp_plugins_rc').
Thus we should not bump into this issue again.
For the few usage in configure_file(), I just add a .full_path() only
when needed at call time.
Last but not least, I replace the bogus `meson --version` call by a
`python3 -c 'exit()'` as advised by Eli Schwartz:
2afa019c70 (note_1284951)
The reason is that it is apparently possible (or will be when some
reimplementation of meson will be done) that the `meson` executable
itself does not exist. On the other hand, `python3` should always be
there, as a mandatory dependency of the build tool.
In order to use an appropriate `python3`, I made the
pythonmod.find_installation() check required in our build (which should
not be a problem since it's a meson requirement as well), even when the
-Dpython option is false (this one depends on other requirements too
anyway, such as version and pygobject). This way I can call this meson
variable of discovered python in my bogus call, instead of calling a
(potentially different) python from PATH environment.
Bruno Lopes