To build host programs, you need to add the program names to 'hostprogs'
to use the necessary build rule, but it is not enough to build them
because there is no dependency.
There are two types of host programs: built as the prerequisite of
another (e.g. gen_crc32table in lib/Makefile), or always built when
Kbuild visits the Makefile (e.g. genksyms in scripts/genksyms/Makefile).
The latter is typical in Makefiles under scripts/, which contains host
programs globally used during the kernel build. To build them, you need
to add them to both 'hostprogs' and 'always-y'.
This commit adds hostprogs-always-y as a shorthand.
The same applies to user programs. net/bpfilter/Makefile builds
bpfilter_umh on demand, hence always-y is unneeded. In contrast,
programs under samples/ are added to both 'userprogs' and 'always-y'
so they are always built when Kbuild visits the Makefiles.
userprogs-always-y works as a shorthand.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Acked-by: Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>
The host shared library rules are currently implemented in
scripts/Makefile.host, but actually GCC-plugin is the only user of
them. (The VDSO .so files are built for the target by different
build rules) Hence, they do not need to be treewide available.
Move all the relevant build rules to scripts/gcc-plugins/Makefile.
I also optimized the build steps so *.so is directly built from .c
because every upstream plugin is compiled from a single source file.
I am still keeping the multi-file plugin support, which Kees Cook
mentioned might be needed by out-of-tree plugins.
(https://lkml.org/lkml/2019/1/11/1107)
If the plugin, foo.so, is compiled from two files foo.c and foo2.c,
then you can do like follows:
foo-objs := foo.o foo2.o
Single-file plugins do not need the *-objs notation.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Acked-by: Kees Cook <keescook@chromium.org>
Kbuild supports the infrastructure to build host programs, but there
was no support to build userspace programs for the target architecture
(i.e. the same architecture as the kernel).
Sam Ravnborg worked on this in 2014 (https://lkml.org/lkml/2014/7/13/154),
but it was not merged. One problem at that time was, there was no good way
to know whether $(CC) can link standalone programs. In fact, pre-built
kernel.org toolchains [1] are often used for building the kernel, but they
do not provide libc.
Now, we can handle this cleanly because the compiler capability is
evaluated at the Kconfig time. If $(CC) cannot link standalone programs,
the relevant options are hidden by 'depends on CC_CAN_LINK'.
The implementation just mimics scripts/Makefile.host
The userspace programs are compiled with the same flags as the host
programs. In addition, it uses -m32 or -m64 if it is found in
$(KBUILD_CFLAGS).
This new syntax has two usecases.
- Sample programs
Several userspace programs under samples/ include UAPI headers
installed in usr/include. Most of them were previously built for
the host architecture just to use the 'hostprogs' syntax.
However, 'make headers' always works for the target architecture.
This caused the arch mismatch in cross-compiling. To fix this
distortion, sample code should be built for the target architecture.
- Bpfilter
net/bpfilter/Makefile compiles bpfilter_umh as the user mode helper,
and embeds it into the kernel. Currently, it overrides HOSTCC with
CC to use the 'hostprogs' syntax. This hack should go away.
[1]: https://mirrors.edge.kernel.org/pub/tools/crosstool/
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Acked-by: Sam Ravnborg <sam@ravnborg.org>
Nobody was opposed to raising minimum GCC version to 4.8 [1]
So, we will drop GCC <= 4.7 support sooner or later.
We always use C++ compiler for building plugins for GCC >= 4.8.
This commit drops the plugin support for GCC <= 4.7 a bit earlier,
which allows us to dump lots of code.
[1] https://lkml.org/lkml/2020/1/23/545
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Acked-by: Kees Cook <keescook@chromium.org>
In old days, the "host-progs" syntax was used for specifying host
programs. It was renamed to the current "hostprogs-y" in 2004.
It is typically useful in scripts/Makefile because it allows Kbuild to
selectively compile host programs based on the kernel configuration.
This commit renames like follows:
always -> always-y
hostprogs-y -> hostprogs
So, scripts/Makefile will look like this:
always-$(CONFIG_BUILD_BIN2C) += ...
always-$(CONFIG_KALLSYMS) += ...
...
hostprogs := $(always-y) $(always-m)
I think this makes more sense because a host program is always a host
program, irrespective of the kernel configuration. We want to specify
which ones to compile by CONFIG options, so always-y will be handier.
The "always", "hostprogs-y", "hostprogs-m" will be kept for backward
compatibility for a while.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
This '+' was added a long time ago:
| commit c23e6bf05f7802e92fd3da69a1ed35e56f9c85bb (HEAD)
| Author: Kai Germaschewski <kai@tp1.ruhr-uni-bochum.de>
| Date: Mon Oct 28 01:16:34 2002 -0600
|
| kbuild: Fix a "make -j<N>" warning
|
| diff --git a/scripts/Makefile.clean b/scripts/Makefile.clean
| index 2c843e0380bc..e7c392fd5788 100644
| --- a/scripts/Makefile.clean
| +++ b/scripts/Makefile.clean
| @@ -42,7 +42,7 @@ quiet_cmd_clean = CLEAN $(obj)
|
| __clean: $(subdir-ymn)
| ifneq ($(strip $(__clean-files) $(clean-rule)),)
| - $(call cmd,clean)
| + +$(call cmd,clean)
| else
| @:
| endif
At that time, cmd_clean contained $(clean-rule), which was able to
invoke sub-make. That was why cleaning with the -j option showed:
warning: jobserver unavailable: using -j1. Add '+' to parent make rule.
It is not the case any more; cmd_clean now just runs the 'rm' command.
The '+' marker is pointless.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
The only the difference between clean-files and clean-dirs is the -r
option passed to the 'rm' command.
You can always pass -r, and then remove the clean-dirs syntax.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
The host-progs has been kept as an alias of hostprogs-y for a long time
(at least since the beginning of Git era), with the clear prompt:
Usage of host-progs is deprecated. Please replace with hostprogs-y!
Enough time for the migration has passed.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Max Filippov <jcmvbkbc@gmail.com>
The comment is the same as in the top-level Makefile.
Also, the comments contain typos:
- the .PHONY variable -> the PHONY variable
- se we can ... -> so we can ...
Instead of fixing the typos, just remove the duplicated comments.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.
By default all files without license information are under the default
license of the kernel, which is GPL version 2.
Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier. The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.
This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.
How this work was done:
Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
- file had no licensing information it it.
- file was a */uapi/* one with no licensing information in it,
- file was a */uapi/* one with existing licensing information,
Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.
The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne. Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.
The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed. Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.
Criteria used to select files for SPDX license identifier tagging was:
- Files considered eligible had to be source code files.
- Make and config files were included as candidates if they contained >5
lines of source
- File already had some variant of a license header in it (even if <5
lines).
All documentation files were explicitly excluded.
The following heuristics were used to determine which SPDX license
identifiers to apply.
- when both scanners couldn't find any license traces, file was
considered to have no license information in it, and the top level
COPYING file license applied.
For non */uapi/* files that summary was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 11139
and resulted in the first patch in this series.
If that file was a */uapi/* path one, it was "GPL-2.0 WITH
Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 WITH Linux-syscall-note 930
and resulted in the second patch in this series.
- if a file had some form of licensing information in it, and was one
of the */uapi/* ones, it was denoted with the Linux-syscall-note if
any GPL family license was found in the file or had no licensing in
it (per prior point). Results summary:
SPDX license identifier # files
---------------------------------------------------|------
GPL-2.0 WITH Linux-syscall-note 270
GPL-2.0+ WITH Linux-syscall-note 169
((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21
((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17
LGPL-2.1+ WITH Linux-syscall-note 15
GPL-1.0+ WITH Linux-syscall-note 14
((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5
LGPL-2.0+ WITH Linux-syscall-note 4
LGPL-2.1 WITH Linux-syscall-note 3
((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3
((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1
and that resulted in the third patch in this series.
- when the two scanners agreed on the detected license(s), that became
the concluded license(s).
- when there was disagreement between the two scanners (one detected a
license but the other didn't, or they both detected different
licenses) a manual inspection of the file occurred.
- In most cases a manual inspection of the information in the file
resulted in a clear resolution of the license that should apply (and
which scanner probably needed to revisit its heuristics).
- When it was not immediately clear, the license identifier was
confirmed with lawyers working with the Linux Foundation.
- If there was any question as to the appropriate license identifier,
the file was flagged for further research and to be revisited later
in time.
In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases, confirmation
by lawyers working with the Linux Foundation.
Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights. The
Windriver scanner is based on an older version of FOSSology in part, so
they are related.
Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot checks
in about 15000 files.
In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect the
correct identifier.
Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial patch
version early this week with:
- a full scancode scan run, collecting the matched texts, detected
license ids and scores
- reviewing anything where there was a license detected (about 500+
files) to ensure that the applied SPDX license was correct
- reviewing anything where there was no detection but the patch license
was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
SPDX license was correct
This produced a worksheet with 20 files needing minor correction. This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.
These .csv files were then reviewed by Greg. Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected. This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.) Finally Greg ran the script using the .csv files to
generate the patches.
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Infrastructure for building independent shared library targets.
Based on work created by the PaX Team.
Signed-off-by: Emese Revfy <re.emese@gmail.com>
Acked-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Michal Marek <mmarek@suse.com>
Pull kbuild updates from Michal Marek:
- several cleanups in kbuild
- serialize multiple *config targets so that 'make defconfig kvmconfig'
works
- The cc-ifversion macro got support for an else-branch
* 'kbuild' of git://git.kernel.org/pub/scm/linux/kernel/git/mmarek/kbuild:
kbuild,gcov: simplify kernel/gcov/Makefile more
kbuild: allow cc-ifversion to have the argument for false condition
kbuild,gcov: simplify kernel/gcov/Makefile
kbuild,gcov: remove unnecessary workaround
kbuild: do not add $(call ...) to invoke cc-version or cc-fullversion
kbuild: fix cc-ifversion macro
kbuild: drop $(version_h) from MRPROPER_FILES
kbuild: use mixed-targets when two or more config targets are given
kbuild: remove redundant line from bounds.h/asm-offsets.h
kbuild: merge bounds.h and asm-offsets.h rules
kbuild: Drop support for clean-rule
scripts/Makefile.clean treats absolute path specially, but
$(objtree)/debian is no longer an absolute path since 7e1c0477 (kbuild:
Use relative path for $(objtree). Work around this by checking if the
path starts with $(objtree)/.
Reported-and-tested-by: Sedat Dilek <sedat.dilek@gmail.com>
Fixes: 7e1c0477 (kbuild: Use relative path for $(objtree)
Signed-off-by: Michal Marek <mmarek@suse.cz>
clean-rule has not been used since 94869f86 (kbuild: Accept absolute
paths in clean-files and introduce clean-dirs) ten years ago.
Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
Signed-off-by: Michal Marek <mmarek@suse.cz>
Makefile.clean includes Kbuild.include since commit 371fdc77
(kbuild: collect shorthands into scripts/Kbuild.include), so there is no
need for a local copy.
Signed-off-by: Michal Marek <mmarek@suse.cz>
The shorthand "clean" is defined in both the top Makefile and
scripts/Makefile.clean. Likewise, the "hdr-inst" is defined in
both the top Makefile and scripts/Makefile.headersinst.
To reduce code duplication, this commit collects them into
scripts/Kbuild.include like the "build" and "modbuiltin" shorthands.
It requires scripts/Makefile.clean to include scripts/Kbuild.include,
but its impact on the performance of "make clean" should be
negligible.
Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Signed-off-by: Michal Marek <mmarek@suse.cz>
$(if $(KBUILD_SRC),$(srctree)/) was a useful strategy
to omit a long absolute path for in-source-tree build
prior to commit 890676c65d
(kbuild: Use relative path when building in the source tree).
Now $(srctree) is "." when building in the source tree.
It would not be annoying to add "$(srctree)/" all the time.
Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Signed-off-by: Michal Marek <mmarek@suse.cz>
Kconfig never defines CONFIG_* as 'n'.
Now obj-n is only used in firmware/Makefile and it can be
replaced with obj-. No makefile uses lib-n.
Let's rip off obj-n and lib-n.
Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Acked-by: Peter Foley <pefoley2@pefoley.com>
Signed-off-by: Michal Marek <mmarek@suse.cz>
Commit 7d3cc8b tried to keep bounds.h and asm-offsets.h during make
clean by filtering these out of $(clean-files), but they are listed in
$(targets) and $(always) and thus removed automatically. Introduce a new
$(no-clean-files) variable to really skip such files in Makefile.clean.
Signed-off-by: Michal Marek <mmarek@suse.cz>
These checks has been present for several kernel releases (> 5).
So lets just get rid of them.
With this we no longer check for use of:
EXTRA_TARGETS, O_TARGET, L_TARGET, list-multi, export-objs
There were three remaining in-tree users of O_TARGET in some
unmaintained sh64 code - mail sent to the maintainer + list.
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
The kbuild system takes advantage of an incorrect behavior in GNU make.
Once this behavior is fixed, all files in the kernel rebuild every time,
even if nothing has changed. This patch ensures kbuild works with both
the incorrect and correct behaviors of GNU make.
For more details on the incorrect behavior, see:
http://lists.gnu.org/archive/html/bug-make/2006-03/msg00003.html
Changes in this patch:
- Keep all targets that are to be marked .PHONY in a variable, PHONY.
- Add .PHONY: $(PHONY) to mark them properly.
- Remove any $(PHONY) files from the $? list when determining whether
targets are up-to-date or not.
Signed-off-by: Paul Smith <psmith@gnu.org>
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
kbuild failed to locate Makefile for external modules.
This brought to my attention how the variables for directories
have different values in different usage scenarios.
Different kbuild usage scenarios:
make - plain make in same directory where kernel source lives
make O= - kbuild is told to store output files in another directory
make M= - building an external module
make O= M= - building an external module with kernel output seperate from src
Value assigned to the different variables:
|$(src) |$(obj) |$(srctree) |$(objtree)
make |reldir to k src |as src |abs path to k src |abs path to k src
make O= |reldir to k src |as src |abs path to k src |abs path to output dir
make M= |abs path to src |as src |abs path to k src |abs path to k src
make O= M= |abs path to src |as src |abs path to k src |abs path to k output
path to kbuild file:
make | $(srctree)/$(src), $(src)
make O= | $(srctree)/$(src)
make M= | $(src)
make O= M= | $(src)
From the table above it can be seen that the only good way to find the
home directory of the kbuild file is to locate the one of the two variants
that is an absolute path. If $(src) is an absolute path (starts with /)
then use it, otherwise prefix $(src) with $(srctree).
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Defining clean before including the kbuild file give us knowledge when
the kbuild file is included for cleaning. This is rarey usefull - but in
a corner case in klibc this proved necessary.
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
---
Initial git repository build. I'm not bothering with the full history,
even though we have it. We can create a separate "historical" git
archive of that later if we want to, and in the meantime it's about
3.2GB when imported into git - space that would just make the early
git days unnecessarily complicated, when we don't have a lot of good
infrastructure for it.
Let it rip!
Masahiro Yamada