Commit Graph

25 Commits

Author SHA1 Message Date
Masahiro Yamada 05e96e96a3 kbuild: use git-archive for source package creation
Commit 5c3d1d0abb ("kbuild: add a tool to list files ignored by git")
added a new tool, scripts/list-gitignored. My intention was to create
source packages without cleaning the source tree, without relying on git.

Linus strongly objected to it, and suggested using 'git archive' instead.
[1] [2] [3]

This commit goes in that direction - Remove scripts/list-gitignored.c
and rewrites Makefiles and scripts to use 'git archive' for building
Debian and RPM source packages. It also makes 'make perf-tar*-src-pkg'
use 'git archive' again.

Going forward, building source packages is only possible in a git-managed
tree. Building binary packages does not require git.

[1]: https://lore.kernel.org/lkml/CAHk-=wi49sMaC7vY1yMagk7eqLK=1jHeHQ=yZ_k45P=xBccnmA@mail.gmail.com/
[2]: https://lore.kernel.org/lkml/CAHk-=wh5AixGsLeT0qH2oZHKq0FLUTbyTw4qY921L=PwYgoGVw@mail.gmail.com/
[3]: https://lore.kernel.org/lkml/CAHk-=wgM-W6Fu==EoAVCabxyX8eYBz9kNC88-tm9ExRQwA79UQ@mail.gmail.com/

Fixes: 5c3d1d0abb ("kbuild: add a tool to list files ignored by git")
Fixes: e0ca16749a ("kbuild: make perf-tar*-src-pkg work without relying on git")
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2023-03-16 22:46:12 +09:00
Masahiro Yamada 81f59a26f3 kbuild: rpm-pkg: move source components to rpmbuild/SOURCES
Prepare to add more files to the source RPM.

Also, fix the build error when KCONFIG_CONFIG is set:
  error: Bad file: ./.config: No such file or directory

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2023-03-16 22:45:56 +09:00
Masahiro Yamada 1fc9095846 kbuild: tar-pkg: use tar rules in scripts/Makefile.package
Use %.tar, %.tar.gz, %.tar.bz2, %.tar.xz, %.tar.zst rules in
scripts/Makefile.package.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2023-02-26 15:23:30 +09:00
Masahiro Yamada e0ca16749a kbuild: make perf-tar*-src-pkg work without relying on git
Currently, perf-tar*-src-pkg only uses 'git archive', but it is better
to make it work without relying on git.

The file, HEAD, which saves the commit hash, will be included in the
tarball only when the source tree is managed by git. The git tree is
more precisely checked; it has been copied from scripts/setlocalversion.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2023-02-26 15:23:30 +09:00
Masahiro Yamada e785399559 kbuild: deb-pkg: switch over to source format 3.0 (quilt)
Change the source format from "1.0" to "3.0 (quilt)" because it works
more cleanly.

All files except .config and debian/ go into the orig tarball.
Add a single patch, debian/patches/config, and delete the ugly
extend-diff-ignore patterns.

The debian tarball will be compressed into *.debian.tar.xz by default.
If you like to use a different compression mode, you can pass the
command line option, DPKG_FLAGS=-Zgzip, for example.

The orig tarball only supports gzip for now. The combination of
gzip and xz is somewhat clumsy, but it is not a practical problem.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2023-02-26 15:22:37 +09:00
Masahiro Yamada b44aa8c96e kbuild: deb-pkg: make .orig tarball a hard link if possible
If '..' belongs to the same filesystem, create a hard link instead of
a copy. In most cases, you can save disk space.

I do not want to use 'mv' because keeping linux.tar.gz is useful to
avoid unneeded rebuilding of the tarball.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2023-02-26 15:20:38 +09:00
Masahiro Yamada 6eabebb1b6 kbuild: deb-pkg: hide KDEB_SOURCENAME from Makefile
scripts/Makefile.package does not need to know the value of
KDEB_SOURCENAME because the source name can be taken from
debian/changelog by using dpkg-parsechangelog.

Move the default of KDEB_SOURCENAME (i.e. linux-upstream) to
scripts/package/mkdebian.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2023-02-26 15:19:38 +09:00
Masahiro Yamada 6fc91752d7 kbuild: srcrpm-pkg: create source package without cleaning
If you run 'make (src)rpm-pkg', all objects are lost due to 'make clean',
which makes the incremental builds impossible.

Instead of cleaning, pass the exclude list to tar's --exclude-from
option.

Previously, the .config was contained in the source tarball.

With this commit, the source rpm consists of separate linux.tar.gz
and .config.

Remove stale comments. Now, 'make (src)rpm-pkg' works with O= option.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2023-02-26 15:18:47 +09:00
Masahiro Yamada 1ec9bb704f kbuild: rpm-pkg: build binary packages from source rpm
The build rules of rpm-pkg and srcrpm-pkg are almost the same.
Remove the code duplication.

Change rpm-pkg to build binary packages from the source package generated
by srcrpm-pkg.

This changes the output directory of the srpm generated by 'make rpm-pkg'
because srcrpm-pkg overrides _srcrpmdir.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2023-02-26 15:18:47 +09:00
Masahiro Yamada 7bf4582d7a kbuild: deb-pkg: create source package without cleaning
If you run 'make deb-pkg', all objects are lost due to 'make clean',
which makes the incremental builds impossible.

Instead of cleaning, pass the exclude list to tar's --exclude-from
option.

Previously, *.diff.gz contained some check-in files such as
.clang-format, .cocciconfig.

With this commit, *.diff.gz will only contain the .config and debian/.
The other source files will go into the .orig tarball.

linux.tar.gz is rebuilt only when the source files that would go into
the tarball are changed.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2023-02-26 15:18:07 +09:00
Masahiro Yamada 58e0e5c85e kbuild: deb-pkg: add --source-option=-sP
If the source package fails to build, ../linux.orig is left over.
In the next run of 'make deb-pkg', you will get the following error:

  dpkg-source: error: orig directory 'linux.orig' already exists, not overwriting, giving up; use -sA, -sK or -sP to override

You can manually remove ../linux.orig, but it is annoying.

Pass -sP down to dpkg-source.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2023-02-05 18:51:21 +09:00
Masahiro Yamada f6e09b07cc kbuild: do not put .scmversion into the source tarball
.scmversion is used by (src)rpm-pkg and deb-pkg to carry KERNELRELEASE.

In fact, deb-pkg does not rely on it any more because the generated
debian/rules specifies KERNELRELEASE from the command line.

Do likwise for (src)rpm-pkg, and remove this feature.
For the same reason, you do not need to save LOCALVERSION in the
spec file.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2023-01-30 13:00:30 +09:00
Jun ASAKA 9c9b55a594 kbuild: add a missing line for help message
The help message line for building the source RPM package was missing.
Added it.

Signed-off-by: Jun ASAKA <JunASAKA@zzy040330.moe>
Reviewed-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2022-12-30 17:22:14 +09:00
Linus Torvalds 6feb57c2fd Kbuild updates for v6.2
- Support zstd-compressed debug info
 
  - Allow W=1 builds to detect objects shared among multiple modules
 
  - Add srcrpm-pkg target to generate a source RPM package
 
  - Make the -s option detection work for future GNU Make versions
 
  - Add -Werror to KBUILD_CPPFLAGS when CONFIG_WERROR=y
 
  - Allow W=1 builds to detect -Wundef warnings in any preprocessed files
 
  - Raise the minimum supported version of binutils to 2.25
 
  - Use $(intcmp ...) to compare integers if GNU Make >= 4.4 is used
 
  - Use $(file ...) to read a file if GNU Make >= 4.2 is used
 
  - Print error if GNU Make older than 3.82 is used
 
  - Allow modpost to detect section mismatches with Clang LTO
 
  - Include vmlinuz.efi into kernel tarballs for arm64 CONFIG_EFI_ZBOOT=y
 -----BEGIN PGP SIGNATURE-----
 
 iQJJBAABCgAzFiEEbmPs18K1szRHjPqEPYsBB53g2wYFAmOeImsVHG1hc2FoaXJv
 eUBrZXJuZWwub3JnAAoJED2LAQed4NsG06IP/iVjuWFvnjDZT4X8X6zN8aKp1vtR
 EMkmoRtt5cD4CLb1MG4N7irYHgedQSx4rYceP45MyW1I3egl6Ct14RDyeQ1xSIZb
 XFTLDCZvfl/up3MdiqNAqKRS7x5lk9++7F0t+2SoQxKQyJvm735XreX+VhZ1FeLB
 qcHrmzJ5veky5Ry/3OkNUgKFBjKEAL+qKMc55uvkXqfTb3KoBa2r4VC1OaoYGRru
 R8oF9qQRnGVQAl/LbBVchmgSjxryxPrCvBGiKlK03VkXdzEMHMimEJh3BQ6e0PGo
 gajdk+4liy7z+jQnI7jFhvJjGKzkEP/Bc99M/uS92QX5MgpH6mqpHMoqqPiqW87K
 RmZH37FqRu1Vo8dpibmH6r2K6YD/HHRjaDHk1VuuCQYEn0dsNmokPXOqd/1v0I1i
 TXPjWOw1AID5vMJWllqxFhpeVvf0vx5BT/UNrh68MLqlJZzv2eMVJb4fNy6640ml
 U0NclMnOa3eOmf5z1T7/LqDRTa63Q0kpanRrBpcmVOaqW+ZpQ3SQjh4uBN1PyJHL
 cX3Skc341DyRlFiT54QhGKlm57MEb2gjhBZ3Z4J+b7sEFgvjXH/W8vcOGIKlppmA
 CfYMyres4OV+fJc89ONkWsvLiOP1OeUGPvytm33J5QMKXc8SzOLP0D/F8kjrDflm
 EROKuZ4EA5ej/rOy
 =Ig/Y
 -----END PGP SIGNATURE-----

Merge tag 'kbuild-v6.2' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild

Pull Kbuild updates from Masahiro Yamada:

 - Support zstd-compressed debug info

 - Allow W=1 builds to detect objects shared among multiple modules

 - Add srcrpm-pkg target to generate a source RPM package

 - Make the -s option detection work for future GNU Make versions

 - Add -Werror to KBUILD_CPPFLAGS when CONFIG_WERROR=y

 - Allow W=1 builds to detect -Wundef warnings in any preprocessed files

 - Raise the minimum supported version of binutils to 2.25

 - Use $(intcmp ...) to compare integers if GNU Make >= 4.4 is used

 - Use $(file ...) to read a file if GNU Make >= 4.2 is used

 - Print error if GNU Make older than 3.82 is used

 - Allow modpost to detect section mismatches with Clang LTO

 - Include vmlinuz.efi into kernel tarballs for arm64 CONFIG_EFI_ZBOOT=y

* tag 'kbuild-v6.2' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (29 commits)
  buildtar: fix tarballs with EFI_ZBOOT enabled
  modpost: Include '.text.*' in TEXT_SECTIONS
  padata: Mark padata_work_init() as __ref
  kbuild: ensure Make >= 3.82 is used
  kbuild: refactor the prerequisites of the modpost rule
  kbuild: change module.order to list *.o instead of *.ko
  kbuild: use .NOTINTERMEDIATE for future GNU Make versions
  kconfig: refactor Makefile to reduce process forks
  kbuild: add read-file macro
  kbuild: do not sort after reading modules.order
  kbuild: add test-{ge,gt,le,lt} macros
  Documentation: raise minimum supported version of binutils to 2.25
  kbuild: add -Wundef to KBUILD_CPPFLAGS for W=1 builds
  kbuild: move -Werror from KBUILD_CFLAGS to KBUILD_CPPFLAGS
  kbuild: Port silent mode detection to future gnu make.
  init/version.c: remove #include <generated/utsrelease.h>
  firmware_loader: remove #include <generated/utsrelease.h>
  modpost: Mark uuid_le type to be suitable only for MEI
  kbuild: add ability to make source rpm buildable using koji
  kbuild: warn objects shared among multiple modules
  ...
2022-12-19 12:33:32 -06:00
Paran Lee df1f1ea956 scripts: add rust in scripts/Makefile.package
Add rust argument at TAR_CONTENT in
scripts/Makefile.package script with alphabetical order.

Signed-off-by: Paran Lee <p4ranlee@gmail.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2022-11-24 09:27:57 +09:00
Ivan Vecera 8818039f95 kbuild: add ability to make source rpm buildable using koji
Changes:
- added new target 'srcrpm-pkg' to generate source rpm
- added required build tools to spec file
- removed locally compiled host tools to force their re-compile

Signed-off-by: Ivan Vecera <ivecera@redhat.com>
Signed-off-by: Jonathan Toppins <jtoppins@redhat.com>
Acked-by: Íñigo Huguet <ihuguet@redhat.com>
Tested-by: Ivan Vecera <ivecera@redhat.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2022-11-24 00:23:44 +09:00
Masahiro Yamada a3c4d4abaa kbuild: hard-code KBUILD_ALLDIRS in scripts/Makefile.package
My future plan is to list subdirectories in ./Kbuild. When it occurs,
$(vmlinux-alldirs) will not contain all subdirectories.

Let's hard-code the directory list until I get around to implementing
a more sophisticated way for generating a source tarball.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Tested-by: Nick Desaulniers <ndesaulniers@google.com>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2022-09-29 04:38:57 +09:00
Masahiro Yamada 6105e4f651 kbuild: rpm-pkg: pass 'linux' to --target option of rpmbuild
Presumably, _target_os is defined even if the --target flag does not
specify it, but it is better to make it explicit.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2022-07-27 21:18:00 +09:00
Paweł Jasiak 88f5e1e662 kbuild: Add make tarzst-pkg build option
Add tarzst-pkg and perf-tarzst-src-pkg targets to build zstd compressed
tarballs.

Signed-off-by: Paweł Jasiak <pawel@jasiak.dev>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2021-10-12 11:50:46 +09:00
Masahiro Yamada 82526ef433 kbuild: deb-pkg: change the source package name to linux-upstream
Change the source package name from 'linux-$(KERNELRELEASE)' to
'linux-upstream'.

Initially, I tried to use 'linux' to be aligned with the Debian
kernel package, but Ben suggested 'linux-upstream' so that it is
clearly distinguished from distribution packages. [1]

The filenames will be changed as follows:

[Before]
  linux-5.12.0-rc3+_5.12.0-rc3+-1.dsc
  linux-5.12.0-rc3+_5.12.0-rc3+.orig.tar.gz
  linux-5.12.0-rc3+_5.12.0-rc3+-1.diff.gz

[After]
  linux-upstream_5.12.0-rc3+-1.dsc
  linux-upstream_5.12.0-rc3+.orig.tar.gz
  linux-upstream_5.12.0-rc3+-1.diff.gz

Commit 3716001bcb ("deb-pkg: add source package") introduced
KDEB_SOURCENAME. If you are unhappy with the default name, you can
override it via KDEB_SOURCENAME.

[1]: https://lore.kernel.org/linux-kbuild/06ffa2a690d57f867b4bc1b42f0026917b1dd3cd.camel@decadent.org.uk/T/#m2c4afa0eca5ced5e57795b002f2dbcb05d7a4a44

Suggested-by: Ben Hutchings <ben@decadent.org.uk>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2021-04-25 05:30:45 +09:00
Denis Efremov e4a42c82e9 kbuild: fix broken builds because of GZIP,BZIP2,LZOP variables
Redefine GZIP, BZIP2, LZOP variables as KGZIP, KBZIP2, KLZOP resp.
GZIP, BZIP2, LZOP env variables are reserved by the tools. The original
attempt to redefine them internally doesn't work in makefiles/scripts
intercall scenarios, e.g., "make GZIP=gzip bindeb-pkg" and results in
broken builds. There can be other broken build commands because of this,
so the universal solution is to use non-reserved env variables for the
compression tools.

Fixes: 8dfb61dcba ("kbuild: add variables for compression tools")
Signed-off-by: Denis Efremov <efremov@linux.com>
Tested-by: Guenter Roeck <linux@roeck-us.net>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-06-11 20:14:41 +09:00
Denis Efremov 8dfb61dcba kbuild: add variables for compression tools
Allow user to use alternative implementations of compression tools,
such as pigz, pbzip2, pxz. For example, multi-threaded tools to
speed up the build:
$ make GZIP=pigz BZIP2=pbzip2

Variables _GZIP, _BZIP2, _LZOP are used internally because original env
vars are reserved by the tools. The use of GZIP in gzip tool is obsolete
since 2015. However, alternative implementations (e.g., pigz) still rely
on it. BZIP2, BZIP, LZOP vars are not obsolescent.

The credit goes to @grsecurity.

As a sidenote, for multi-threaded lzma, xz compression one can use:
$ export XZ_OPT="--threads=0"

Signed-off-by: Denis Efremov <efremov@linux.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-06-06 23:42:01 +09:00
Matteo Croce af7db99a1c kbuild: Add make dir-pkg build option
Add a 'dir-pkg' target which just creates the same directory structures
as in tar-pkg, but doesn't package anything.
Useful when the user wants to copy the kernel tree on a machine using
ssh, rsync or whatever.

Signed-off-by: Matteo Croce <mcroce@redhat.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-11-11 20:10:01 +09:00
Geert Uytterhoeven a64c0440dd kbuild: Wrap long "make help" text lines
Some "make help" text lines extend beyond 80 characters.
Wrap them before an opening parenthesis, or before 80 characters.

Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-11-11 20:10:01 +09:00
Masahiro Yamada 000ec95fbe kbuild: pkg: rename scripts/package/Makefile to scripts/Makefile.package
scripts/package/Makefile does not use $(obj) or $(src) at all.
It actually generates files and directories in the top of $(objtree).
I do not see much sense in descending into scripts/package/.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-08-25 07:42:00 +09:00