Pull x86 fixes from Thomas Gleixner:
"Yet another pile of melted spectrum related changes:
- sanitize the array_index_nospec protection mechanism: Remove the
overengineered array_index_nospec_mask_check() magic and allow
const-qualified types as index to avoid temporary storage in a
non-const local variable.
- make the microcode loader more robust by properly propagating error
codes. Provide information about new feature bits after micro code
was updated so administrators can act upon.
- optimizations of the entry ASM code which reduce code footprint and
make the code simpler and faster.
- fix the {pmd,pud}_{set,clear}_flags() implementations to work
properly on paravirt kernels by removing the address translation
operations.
- revert the harmful vmexit_fill_RSB() optimization
- use IBRS around firmware calls
- teach objtool about retpolines and add annotations for indirect
jumps and calls.
- explicitly disable jumplabel patching in __init code and handle
patching failures properly instead of silently ignoring them.
- remove indirect paravirt calls for writing the speculation control
MSR as these calls are obviously proving the same attack vector
which is tried to be mitigated.
- a few small fixes which address build issues with recent compiler
and assembler versions"
* 'x86-pti-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (38 commits)
KVM/VMX: Optimize vmx_vcpu_run() and svm_vcpu_run() by marking the RDMSR path as unlikely()
KVM/x86: Remove indirect MSR op calls from SPEC_CTRL
objtool, retpolines: Integrate objtool with retpoline support more closely
x86/entry/64: Simplify ENCODE_FRAME_POINTER
extable: Make init_kernel_text() global
jump_label: Warn on failed jump_label patching attempt
jump_label: Explicitly disable jump labels in __init code
x86/entry/64: Open-code switch_to_thread_stack()
x86/entry/64: Move ASM_CLAC to interrupt_entry()
x86/entry/64: Remove 'interrupt' macro
x86/entry/64: Move the switch_to_thread_stack() call to interrupt_entry()
x86/entry/64: Move ENTER_IRQ_STACK from interrupt macro to interrupt_entry
x86/entry/64: Move PUSH_AND_CLEAR_REGS from interrupt macro to helper function
x86/speculation: Move firmware_restrict_branch_speculation_*() from C to CPP
objtool: Add module specific retpoline rules
objtool: Add retpoline validation
objtool: Use existing global variables for options
x86/mm/sme, objtool: Annotate indirect call in sme_encrypt_execute()
x86/boot, objtool: Annotate indirect jump in secondary_startup_64()
x86/paravirt, objtool: Annotate indirect calls
...
Disable retpoline validation in objtool if your compiler sucks, and otherwise
select the validation stuff for CONFIG_RETPOLINE=y (most builds would already
have it set due to ORC).
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Makefile changes:
- enable unused-variable warning that was wrongly disabled for clang
Kconfig changes:
- warn blank 'help' and fix existing instances
- fix 'choice' behavior to not write out invisible symbols
- fix misc weirdness
Coccinell changes:
- fix false positive of free after managed memory alloc detection
- improve performance of NULL dereference detection
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQIcBAABAgAGBQJafl86AAoJED2LAQed4NsGrBEP/23mRl+8dSYkTmlczP7stZHL
tGkMQIOj85usPV4JIxYHgge6XhU0CBFXNGnDnxGeNEtwnBr5QQNxuS2nPh7rHJIN
zX5rX/vwO9lzn2FFEdKrk8bito1IgnUHUvN/d0ikPWzY7GaMy6WrIWgyThllsLNE
W7hp3cpUQOhLL9PKXwglZ/oS4iTpEs0DwN93UXU7cp7zyRa0XtFfPf7/IJ2KY+Yl
a2TEsUuZ/slJoxLhacr6+TBAgqUyewWIs0nAGdjP2EVlSjxZJQYFEQq4KnLUO2gV
wLHH2snsZSBDfPDp0M9OOb737HE17NRmuLjWxUZZOMFz8tvfUT1454zhVAN2OtSQ
cP0RqVRrFiS721oxacZpAxKFrd7o4ugUHpftJMPQAq70T9JFFbapfCLvd+OblOb/
CWmDOOR37tvop5OCuaqaSMq7a+ZQt2cO5fogiEDdnjZkk2AH5GgsAHJIrl7hH4OT
P9UMcxaWSGbutdVkM4cMUmYMuAJjiFhx1fiD+hevB1KvemXRXrqhCb0wV+GRdcoU
MXGvOGVw5WyF/vFdjpjkY7KeCgpU3BTWH3pFC2a5vUCDqgD8yndwFghJMDfSjl6d
46DIqknyveq234GK/Yz5khlbY094yL8JrJU2duva/9fGV86tgOr29xgMK28Lpyh8
AYRGO9XgmehZrEHcAQ57
=jy5l
-----END PGP SIGNATURE-----
Merge tag 'kbuild-v4.16-2' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild
Pull more Kbuild updates from Masahiro Yamada:
"Makefile changes:
- enable unused-variable warning that was wrongly disabled for clang
Kconfig changes:
- warn about blank 'help' and fix existing instances
- fix 'choice' behavior to not write out invisible symbols
- fix misc weirdness
Coccinell changes:
- fix false positive of free after managed memory alloc detection
- improve performance of NULL dereference detection"
* tag 'kbuild-v4.16-2' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (21 commits)
kconfig: remove const qualifier from sym_expand_string_value()
kconfig: add xrealloc() helper
kconfig: send error messages to stderr
kconfig: echo stdin to stdout if either is redirected
kconfig: remove check_stdin()
kconfig: remove 'config*' pattern from .gitignnore
kconfig: show '?' prompt even if no help text is available
kconfig: do not write choice values when their dependency becomes n
coccinelle: deref_null: avoid useless computation
coccinelle: devm_free: reduce false positives
kbuild: clang: disable unused variable warnings only when constant
kconfig: Warn if help text is blank
nios2: kconfig: Remove blank help text
arm: vt8500: kconfig: Remove blank help text
MIPS: kconfig: Remove blank help text
MIPS: BCM63XX: kconfig: Remove blank help text
lib/Kconfig.debug: Remove blank help text
Staging: rtl8192e: kconfig: Remove blank help text
Staging: rtl8192u: kconfig: Remove blank help text
mmc: kconfig: Remove blank help text
...
Nearly all modern compilers support a stack-protector option, and nearly
all modern distributions enable the kernel stack-protector, so enabling
this by default in kernel builds would make sense. However, Kconfig does
not have knowledge of available compiler features, so it isn't safe to
force on, as this would unconditionally break builds for the compilers or
architectures that don't have support. Instead, this introduces a new
option, CONFIG_CC_STACKPROTECTOR_AUTO, which attempts to discover the best
possible stack-protector available, and will allow builds to proceed even
if the compiler doesn't support any stack-protector.
This option is made the default so that kernels built with modern
compilers will be protected-by-default against stack buffer overflows,
avoiding things like the recent BlueBorne attack. Selection of a specific
stack-protector option remains available, including disabling it.
Additionally, tiny.config is adjusted to use CC_STACKPROTECTOR_NONE, since
that's the option with the least code size (and it used to be the default,
so we have to explicitly choose it there now).
Link: http://lkml.kernel.org/r/1510076320-69931-4-git-send-email-keescook@chromium.org
Signed-off-by: Kees Cook <keescook@chromium.org>
Tested-by: Laura Abbott <labbott@redhat.com>
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Josh Triplett <josh@joshtriplett.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Various portions of the kernel, especially per-architecture pieces,
need to know if the compiler is building with the stack protector.
This was done in the arch/Kconfig with 'select', but this doesn't
allow a way to do auto-detected compiler support. In preparation for
creating an on-if-available default, move the logic for the definition of
CONFIG_CC_STACKPROTECTOR into the Makefile.
Link: http://lkml.kernel.org/r/1510076320-69931-3-git-send-email-keescook@chromium.org
Signed-off-by: Kees Cook <keescook@chromium.org>
Tested-by: Laura Abbott <labbott@redhat.com>
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Josh Triplett <josh@joshtriplett.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
In order to make stack-protector failures warn instead of unconditionally
breaking the build, this moves the compiler output sanity-check earlier,
and sets a flag for later testing. Future patches can choose to warn or
fail, depending on the flag value.
Link: http://lkml.kernel.org/r/1510076320-69931-2-git-send-email-keescook@chromium.org
Signed-off-by: Kees Cook <keescook@chromium.org>
Tested-by: Laura Abbott <labbott@redhat.com>
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Josh Triplett <josh@joshtriplett.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
With KASAN enabled the kernel has two different memset() functions, one
with KASAN checks (memset) and one without (__memset). KASAN uses some
macro tricks to use the proper version where required. For example
memset() calls in mm/slub.c are without KASAN checks, since they operate
on poisoned slab object metadata.
The issue is that clang emits memset() calls even when there is no
memset() in the source code. They get linked with improper memset()
implementation and the kernel fails to boot due to a huge amount of KASAN
reports during early boot stages.
The solution is to add -fno-builtin flag for files with KASAN_SANITIZE :=
n marker.
Link: http://lkml.kernel.org/r/8ffecfffe04088c52c42b92739c2bd8a0bcb3f5e.1516384594.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Acked-by: Nick Desaulniers <ndesaulniers@google.com>
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Michal Marek <michal.lkml@markovi.net>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Currently, GCC disables -Wunused-const-variable, but not
-Wunused-variable, so warns unused variables if they are
non-constant.
While, Clang does not warn unused variables at all regardless of
the const qualifier because -Wno-unused-const-variable is implied
by the stronger option -Wno-unused-variable.
Disable -Wunused-const-variable instead of -Wunused-variable so that
GCC and Clang work in the same way.
Signed-off-by: Prasad Sodagudi <psodagud@codeaurora.org>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
A pretty big batch of Kconfig updates. I have to mention the lexer
and parser of Kconfig are now built from real .l and .y sources.
So, flex and bison are the requirement for building the kernel.
Both of them (unlike gperf) have been stable for a long time. This
change has been tested several weeks in linux-next, and I did not
receive any problem report about this.
Summary:
- Add checks for mistakes, like the choice default is not in
choice, help is doubled
- Document data structure and complex code
- Fix various memory leaks
- Change Makefile to build lexer and parser instead of using
pre-generated C files
- Drop 'boolean' keyword, which is equivalent to 'bool'
- Use default 'yy' prefix and remove unneeded Make variables
- Fix gettext() check for xconfig
- Announce that oldnoconfig will be finally removed
- Make 'Selected by:' and 'Implied by' readable in help and
search result
- Hide silentoldconfig from 'make help' to stop confusing people
- Fix misc things and cleanups
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQIcBAABAgAGBQJac0O7AAoJED2LAQed4NsGxRQQAKiDkBmJUKzJykEr3BhEsCW1
1cKwsaCFSKQJZde+Edn14gqBQY3qvQVaWfnQpoBC69IOlMlVQyKqqdtjIUxmGTem
+age94JvgPT3oj7ELigUsL5bFL8CEZYpjwAVkO+Zd+7jPxlM4glNl9F3coTp9ZAl
kaFxnch8qPT3fb6xYSwTpkk28RFNfT8ixwImj0CH6wlwxI635o9wpqAmHCMD5Yid
wBlEB0I0aw9xVx/D2FAh1ZJ+fyOtPLotRTtUQ1kdlDyk98V6EJRfyh1XWF0xE/gH
WR7MAuBNXC6Uu1KqXLTJRWWE4NjBmW8e7OLTaoFQFXdnz/Rgd6gbPXuiiyKR6PUc
mA3h6F8uPDpoRpqOrn6pdu269n1ObuZuC1XZ3MVSXtW3OzLbUk/2GpHPVX8mrep5
SBGsfq4JYtQkgz9/wBrWpdAjl2QbtPJbM/izCbEWk7pod18dVnyHpo3i3YjsaSQf
piDK5JGKsHEisAzp6Onhc9EPeBLAjG236+LvBruaqjJNySXyT0S2kOwV0AMOD1yC
ave8ZQA0QxP7sV/s5DXGkFx5Nt/1LE8JWvcRN0+juavnkUklWfFfnk7epAnFfRPi
HCGfeQSzIQxQ+JV7/vlJ95FWvLZz+KzQ49X5aoYCVcEiFf2MV7RkDHSp5mPE4DnO
5JeHnTBpuvw4mnrZ6UNj
=rsi/
-----END PGP SIGNATURE-----
Merge tag 'kconfig-v4.16' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild
Pull Kconfig updates from Masahiro Yamada:
"A pretty big batch of Kconfig updates.
I have to mention the lexer and parser of Kconfig are now built from
real .l and .y sources. So, flex and bison are the requirement for
building the kernel. Both of them (unlike gperf) have been stable for
a long time. This change has been tested several weeks in linux-next,
and I did not receive any problem report about this.
Summary:
- add checks for mistakes, like the choice default is not in choice,
help is doubled
- document data structure and complex code
- fix various memory leaks
- change Makefile to build lexer and parser instead of using
pre-generated C files
- drop 'boolean' keyword, which is equivalent to 'bool'
- use default 'yy' prefix and remove unneeded Make variables
- fix gettext() check for xconfig
- announce that oldnoconfig will be finally removed
- make 'Selected by:' and 'Implied by' readable in help and search
result
- hide silentoldconfig from 'make help' to stop confusing people
- fix misc things and cleanups"
* tag 'kconfig-v4.16' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (37 commits)
kconfig: Remove silentoldconfig from help and docs; fix kconfig/conf's help
kconfig: make "Selected by:" and "Implied by:" readable
kconfig: announce removal of oldnoconfig if used
kconfig: fix make xconfig when gettext is missing
kconfig: Clarify menu and 'if' dependency propagation
kconfig: Document 'if' flattening logic
kconfig: Clarify choice dependency propagation
kconfig: Document SYMBOL_OPTIONAL logic
kbuild: remove unnecessary LEX_PREFIX and YACC_PREFIX
kconfig: use default 'yy' prefix for lexer and parser
kconfig: make conf_unsaved a local variable of conf_read()
kconfig: make xfgets() really static
kconfig: make input_mode static
kconfig: Warn if there is more than one help text
kconfig: drop 'boolean' keyword
kconfig: use bool instead of boolean for type definition attributes, again
kconfig: Remove menu_end_entry()
kconfig: Document important expression functions
kconfig: Document automatic submenu creation code
kconfig: Fix choice symbol expression leak
...
- fix cross-compilation for architectures that setup CROSS_COMPILE
in their arch Makefile
- fix Kconfig rational operators for bool / tristate
- drop a gperf-generated file from .gitignore
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQIcBAABAgAGBQJaWgIlAAoJED2LAQed4NsGniEQAKFEhwU3XErlJHhfz1o84oJT
ycfulcz1VDUrxpfYi7/uCiBNB7B5rajnH0fclkMwt4oV7wJgWbtDTzlLNIhX5vQI
e+zDc8GMv/9rBks8OBypsUwlW/3etP6PjL1uC2KgG8F/2gNoodJs9Y/5Jfb3esKF
Lh8A1LMqSKNG4B9pjRBQv1k2KX6K5R011UGKLg6qcek1lj8r9NpdgKo/jH5tjrTb
y5weRpIkofb4sqBjls+7H24DUWg2GVSunEIBBNyxqwn52UhcSNcC2s+jdKqwmS50
R2jP8ENXyiATCJfVdKguhiTQJ4xLbTbHrL1K9vGpimj+3PAf37VbRlhXPJ6FVyCm
vuxv6HHL9a7Pm7o/sQxWmHD6GQa6/DCD+j8LPR5ro3Imkh0zqTqvA8R3mnX3NnVz
lj2Bu+Ii+OaSgoN7B2lLgIkr8uc99CErEcqjI2fxKm5hVbuqGF9nciiGMLc0fXJW
9alfkdi911LR3SjmwvFngGtq0SXOTG830J2ERfoD4zCKVg5ZffrFyMPLPgdze3Uv
BKDynomNfCxciz1h4/MZunUOjrViUFfHXDwkSnkfXAmOGfCM1XoE7/aSOuTzvTnl
CT8Sk9RIa90AogVnkA92Zza93Itophpdqdnw8acJIDDrNaToNprzTLB4HNDcKWy9
3k7yoa63AXBvE1k+ofJK
=py+X
-----END PGP SIGNATURE-----
Merge tag 'kbuild-fixes-v4.15' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild
Pull Kbuild fixes from Masahiro Yamada:
- fix cross-compilation for architectures that setup CROSS_COMPILE in
their arch Makefile
- fix Kconfig rational operators for bool / tristate
- drop a gperf-generated file from .gitignore
* tag 'kbuild-fixes-v4.15' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild:
genksyms: drop *.hash.c from .gitignore
kconfig: fix relational operators for bool and tristate symbols
kbuild: move cc-option and cc-disable-warning after incl. arch Makefile
It appears that hardened gentoo enables "-fstack-check" by default for
gcc.
That doesn't work _at_all_ for the kernel, because the kernel stack
doesn't act like a user stack at all: it's much smaller, and it doesn't
auto-expand on use. So the extra "probe one page below the stack" code
generated by -fstack-check just breaks the kernel in horrible ways,
causing infinite double faults etc.
[ I have to say, that the particular code gcc generates looks very
stupid even for user space where it works, but that's a separate
issue. ]
Reported-and-tested-by: Alexander Tsoy <alexander@tsoy.me>
Reported-and-tested-by: Toralf Förster <toralf.foerster@gmx.de>
Cc: stable@kernel.org
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Jiri Kosina <jikos@kernel.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Allow users to use their favorite lexer / parser generators.
This is useful for me to test various flex and bison versions.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Geert reported commit ae6b289a37 ("kbuild: Set KBUILD_CFLAGS before
incl. arch Makefile") broke cross-compilation using a cross-compiler
that supports less compiler options than the host compiler.
For example,
cc1: error: unrecognized command line option "-Wno-unused-but-set-variable"
This problem happens on architectures that setup CROSS_COMPILE in their
arch/*/Makefile.
Move the cc-option and cc-disable-warning back to the original position,
but keep the Clang target options untouched.
Fixes: ae6b289a37 ("kbuild: Set KBUILD_CFLAGS before incl. arch Makefile")
Reported-by: Geert Uytterhoeven <geert@linux-m68k.org>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Tested-by: Geert Uytterhoeven <geert@linux-m68k.org>
The "rpm" has been kept for backward compatibility since pre-git era.
I am planning to remove it after the Linux 4.18 release. Annouce the
end of the support, prompting to use "rpm-pkg" instead.
If you use "rpm", it will work like "rpm-pkg", but warning messages
will be displayed as follows:
WARNING: "rpm" target will be removed after Linux 4.18
Please use "rpm-pkg" instead.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Set the clang KBUILD_CFLAGS up before including arch/ Makefiles,
so that ld-options (etc.) can work correctly.
This fixes errors with clang such as ld-options trying to CC
against your host architecture, but LD trying to link against
your target architecture.
Signed-off-by: Chris Fries <cfries@google.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
Reviewed-by: Matthias Kaehlcke <mka@chromium.org>
Tested-by: Matthias Kaehlcke <mka@chromium.org>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
"obj-y += foo/" syntax requires Kbuild to visit the "foo" subdirectory
and link built-in.o from that directory. This means foo/Makefile is
responsible for creating built-in.o even if there is no object to
link (in this case, built-in.o is an empty archive).
We have had several fixups like commit 4b024242e8 ("kbuild: Fix
linking error built-in.o no such file or directory"), then ended up
with a complex condition as follows:
ifneq ($(strip $(obj-y) $(obj-m) $(obj-) $(subdir-m) $(lib-target)),)
builtin-target := $(obj)/built-in.o
endif
We still have more cases not covered by the above, so we need to add
obj- := dummy.o
in several places just for creating empty built-in.o.
A key point is, the parent Makefile knows whether built-in.o is needed
or not. If a subdirectory needs to create built-in.o, its parent can
tell the fact when descending.
If non-empty $(need-builtin) flag is passed from the parent, built-in.o
should be created. $(obj-y) should be still checked to support the
single target "%/". All of ugly tricks will go away.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Sam Ravnborg <sam@ravnborg.org>
Most places use pwd and rely on $PATH lookup. Moving the remaining
absolute path /bin/pwd users over for consistency.
Also, a reason for doing /bin/pwd -> pwd instead of the other way around
is because I believe build systems should make little assumptions on
host filesystem layout. Case in point, we do this kind of patching
already in NixOS.
Ref. commit 028568d84d
("kbuild: revert $(realpath ...) to $(shell cd ... && /bin/pwd)").
Signed-off-by: Bjørn Forsman <bjorn.forsman@gmail.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
One of the most remarkable improvements in this cycle is, Kbuild is
now able to cache the result of shell commands. Some variables are
expensive to compute, for example, $(call cc-option,...) invokes the
compiler. It is not efficient to redo this computation every time,
even when we are not actually building anything. Kbuild creates a
hidden file ".cache.mk" that contains invoked shell commands and
their results. The speed-up should be noticeable.
Summary:
- Fix arch build issues (hexagon, sh)
- Clean up various Makefiles and scripts
- Fix wrong usage of {CFLAGS,LDFLAGS}_MODULE in arch Makefiles
- Cache variables that are expensive to compute
- Improve cc-ldopton and ld-option for Clang
- Optimize output directory creation
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQIcBAABAgAGBQJaDxaLAAoJED2LAQed4NsGIHQP/isMxxaIxIAWU56+ZcII74k7
639VgrKi9n5y25d1dBRTQg+vReHE6E2JbkCqpVOu11t7m0LT7yUK8v3WwyLf1qTN
GxnqZ/WMQU5/AYVqIWo8jN4FGHpivHJ6qbeiNJM9qN4RAkzG0sZUq746VaFZYmIR
Lu0Gf4m4qjifkkhXsQdWT5i7yNTidPqaL6GNb+FcFkEHlVre8jma0kJlgfHxru84
WmETpjQXvHAZ/R61vY6ekAWpqFhw3ecJY96A9npnx+SQVQdSNAdpaU0SK29jB0ON
/SAfpHg9oa/gD0LFOKV6zkjnAkd4TEjrJEiHHhz5gjT/SbS3T1llBIGZ1oV4X7Y0
Vlh9KWlm1FJJI4SIzc9qUaQMp6JtLfEfHKJCc45xVaN3fNrDnR8jl80x5+95ELga
dCkZgnq5u82MtTysCbHBESwDYQaVPyIrh7In+mduglaCqhqj9KoDjoLoiGfCg7SA
3tPflYVd629w5l5GrazJ40jWn1+ggMtgMOVooJNJ+dINCP+GxsUpH84Ww2Pdic+/
qLdud6TeqxrZDGzWXqKNLu8alM8NGgSr101l9gIf1oqSyy63duBpMrxGDoIJS3FU
rFDoFFUhlfkAXNbQHtVGNzKtcpCjURh992j9Fa1+NfMwSce5IHkMwTvPmNSRowi8
0llLjXhD/bxK6FpdvlV8
=zIdO
-----END PGP SIGNATURE-----
Merge tag 'kbuild-v4.15' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild
Pull Kbuild updates from Masahiro Yamada:
"One of the most remarkable improvements in this cycle is, Kbuild is
now able to cache the result of shell commands. Some variables are
expensive to compute, for example, $(call cc-option,...) invokes the
compiler. It is not efficient to redo this computation every time,
even when we are not actually building anything. Kbuild creates a
hidden file ".cache.mk" that contains invoked shell commands and their
results. The speed-up should be noticeable.
Summary:
- Fix arch build issues (hexagon, sh)
- Clean up various Makefiles and scripts
- Fix wrong usage of {CFLAGS,LDFLAGS}_MODULE in arch Makefiles
- Cache variables that are expensive to compute
- Improve cc-ldopton and ld-option for Clang
- Optimize output directory creation"
* tag 'kbuild-v4.15' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (30 commits)
kbuild: move coccicheck help from scripts/Makefile.help to top Makefile
sh: decompressor: add shipped files to .gitignore
frv: .gitignore: ignore vmlinux.lds
selinux: remove unnecessary assignment to subdir-
kbuild: specify FORCE in Makefile.headersinst as .PHONY target
kbuild: remove redundant mkdir from ./Kbuild
kbuild: optimize object directory creation for incremental build
kbuild: create object directories simpler and faster
kbuild: filter-out PHONY targets from "targets"
kbuild: remove redundant $(wildcard ...) for cmd_files calculation
kbuild: create directory for make cache only when necessary
sh: select KBUILD_DEFCONFIG depending on ARCH
kbuild: fix linker feature test macros when cross compiling with Clang
kbuild: shrink .cache.mk when it exceeds 1000 lines
kbuild: do not call cc-option before KBUILD_CFLAGS initialization
kbuild: Cache a few more calls to the compiler
kbuild: Add a cache for generated variables
kbuild: add forward declaration of default target to Makefile.asm-generic
kbuild: remove KBUILD_SUBDIR_ASFLAGS and KBUILD_SUBDIR_CCFLAGS
hexagon/kbuild: replace CFLAGS_MODULE with KBUILD_CFLAGS_MODULE
...
The flag enables Clang instrumentation of comparison operations
(currently not supported by GCC). This instrumentation is needed by the
new KCOV device to collect comparison operands.
Link: http://lkml.kernel.org/r/20171011095459.70721-2-glider@google.com
Signed-off-by: Victor Chibotaru <tchibo@google.com>
Signed-off-by: Alexander Potapenko <glider@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Andrey Konovalov <andreyknvl@google.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Alexander Popov <alex.popov@linux.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Quentin Casasnovas <quentin.casasnovas@oracle.com>
Cc: <syzkaller@googlegroups.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
In my view, it is not helpful to have a separate file just for
the coccicheck help message. Merge scripts/Makefile.help into
the top-level Makefile.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Julia Lawall <julia.lawall@lip6.fr>
I do not see any reason why $(wildcard ...) needs to be called twice
for computing cmd_files. Remove the first one.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
This tag contains the core RISC-V Linux port, which has been through
nine rounds of review on various mailing lists. The port is not
complete: there's some cleanup patches moving through the review
process, a whole bunch of drivers that need some work, and a lot of
feature additions that will be needed.
The patches contained in this tag have been through nine rounds of
review on the various mailing lists. I have some outstanding cleanup
patches, but since there's been so much review on these patches I
thought it would be best to submit them as-is and then submit explicit
cleanup patches so everyone can review them. This first patch set is
big enough that it's a bit of a pain to constantly rewrite, and it's
caused a few headaches with various contributors.
The port is definately a work in progress. While what's there builds
and boots with 4.14, it's a bit hard to actually see anything happen
because there are no device drivers yet. I maintain a staging branch
that contains all the device drivers and cleanup that actually works,
but those patches won't all be ready for a while. I'd like to get what
we currently have into your tree so everyone can start working from a
single base -- of particular importance is allowing the glibc
upstreaming process to proceed so we can sort out any possibly lingering
user-visible ABI problems we might have.
Copied below is the ChangeLog that contains the history of this patch
set:
(v9) As per suggestions on our v8 patch set, I've split the core architecture code
out from our drivers and would like to submit this patch set to be included
into linux-next, with the goal being to be merged in during the next merge
window. This patch set is based on 4.14-rc2, but if it's better to have it
based on something else then I can change it around.
This patch set contains just the core arch code for RISC-V, so while it builds
an nominally boots, you can't print or take an interrupt so it's not that
useful. If you're looking to actually boot a system it would probably be
better to use the full patch set listed below.
We've collected a handful of tags from reviewers, and the remainder of the
patch set only got minimal feedback last time. Here's what changed:
* We now use the device tree to initialize the timer driver so it's less
tighly coupled with the arch port.
* I cleaned up the defconfigs -- there's actually now just one, and it's
empty. For now I think we're OK with what the kernel sets as defaults, but
I anticipate we'll begin to expand this as people start to use the port
more.
* The VDSO symbols version is sane.
* We WFI while spinning in the boot loop.
* A handful of comments have been added.
While there are still a handful of FIXMEs in this patch set, we've started to
get enough interest from various users and contributors that maintaining an out
of tree patch set is starting to become a big burden. Hopefully the patches
are good enough to merge now, which will at least get everyone working in a
more reasonable manner as we clean up the remaining issues.
This patch set is also availiable on github
https://github.com/riscv/riscv-linux/tree/riscv-for-submission-v9-arch
as is the entire patch set necessary to get a more functional RISC-V system up
and running, including a handful of patches that aren't ready for upstream yet.
https://github.com/riscv/riscv-linux/tree/riscv-for-submission-v9
Hopefully I've managed to get everyone's feedback
Here's the change highlights from the whole patch set:
(v8) I know it may not be the ideal time to submit a patch set right now, as
it's the middle of the merge window, but things have calmed down quite a bit in
the last month so I thought it would be good to get everyone on the same page.
There's been a handful of changes since the last patch set, but most of them
are fairly minor:
* We changed PAGE_OFFSET to allowing mapping more physical memory on 64-bit
systems. This is user configurable, as it triggers a different code model
that generates slightly less efficient code.
* The device tree binding documentation is back, I'd managed to lose it at some
point.
* We now pass the atomic64 test suite. The SBI timer driver has been
* refactored.
(v7) It's been a while since my last patch set, but the changes han been fairly
minimal:
* The PCI cleanup patches have been dropped, we'll do them as a separate patch
set later.
* We've the Kconfig entries from CONFIG_ISA_* to CONFIG_RISCV_ISA_*, to make
grep easier.
* There have been a handful of memory model related tweaks in I/O land,
particularly relating the PCI and the upcoming platform specification.
There are significant comments in the relevant files. This is still a WIP,
but I think we're close to getting as good as we're going to get until we
end up with some more specifications.
(v6) As it's been only a day since the v5 patch set, the changes are pretty
minimal:
* The patch set is now based on linux-next/master, which I believe is a better
base now that we're getting closer to upstream.
* EARLY_PRINTK is no longer an option. Since the SBI console is reasonable,
there's no penalty to enabling it (and thus no benefit to disabling it).
* The mmap syscalls were refactored a bit.
(v5) Things have really started to calm down, so this is fairly similar to the
v4 patch set. The most interesting changes include:
* We've moved back to a single patch set.
* SMP support has been fixed, I was accidentally running on a non-SMP
configuration. There were various mistakes all over the tree as a result of
this.
* The cmpxchg syscalls have been removed, as they were deemed a bad idea. As
a result, RISC-V Linux systems mandate the A extension. The corresponding
Kconfig entry to enable builds on non-A systems has been removed.
* A few more atomic fixes: mostly fence changes, but those resulted in a
handful of additional macros that were no longer necessary.
* riscv_early_sie has been removed.
(v4) There have only been a few changes since the v3 patch set:
* The cmpxchg64 syscall is no longer enabled on 32-bit systems. It's not
possible to provide this on SMP systems, and it's not necessary as glibc
knows not to call it.
* We provide a ELF_HWCAP so users can determine the ISA of the machine the
kernel is running on.
* The multi-line comments are in a better form.
* There were a handful of headers that could be replaced with the asm-generic
versions, and a few unnecessary definitions.
* We no longer use printk, but instead use pr_*.
* A few Kconfig and defconfig entries have been cleaned up.
(v3) A highlight of the changes since the v2 patch set includes:
* We've split out all our drivers into separate patch sets, which I've already
sent out to the relevant maintainers. I haven't included those patches in
this patch set, but some of them are necessary to build our port. A git
tree that contains all our patch sets merged together lives at
<https://github.com/riscv/riscv-linux/tree/riscv-for-submission-v3>.
* The patch set is now split up differently: rather than being split per
directory it is split per topic. Hopefully this will make it easier to
review the port on the mailing list. The split is a bit rough, so you
probably still want to look at the patch set as a whole.
* atomic.h has been completely rewritten and is hopefully now correct. I've
attempted to sanitize the various other memory model related code as well,
and I think it should all be sane now aside from a handful of FIXMEs
commented in the code.
* We've changed the cmpexchg syscall to always exist and to not be
multiplexed. There is also a VDSO entry for compare and exchange, which
allows kernels with the A extension to execute user code without the A
extension reasonably fast.
* Our user-visible register state now contains enough space for the Q
extension for 128-bit floating point, as well as a few words to allow
extensibility to future ISA extensions like the eventual V extension for
vectors.
* A handful of driver cleanups, but these have been split into separate patch
sets now so I won't duplicate them here.
(v2) A highlight of the changes since the v1 patch set includes:
* We've split out our drivers into the right places, which means now there's
a lot more patches. I'll be submitting these patches to various subsystem
maintainers and including them in any future RISC-V patch sets until
they've been merged.
* The SBI console driver has been completely rewritten to use the HVC helpers
and is now significantly smaller.
* We've begun to use weaker barriers as opposed to just the big "fence".
There's still some work to do here, specifically:
- We need fences in the relaxed MMIO functions.
- The non-relaxed MMIO functions are missing R/W bits on their fences.
- Many AMOs need the aq and rl bits set.
* We now have thread_info in task_struct. As a result, sscratch now contains
TP instead of SP. This was necessary because thread_info is no longer on
the stack.
* A few shared routines have been added that we use instead of creating
another arch copy.
-----BEGIN PGP SIGNATURE-----
iQJHBAABCAAxFiEEAM520YNJYN/OiG3470yhUCzLq0EFAloLD8sTHHBhbG1lckBk
YWJiZWx0LmNvbQAKCRDvTKFQLMurQbCZEAC2IgWFOAhYDIv4s39jC/iuGcofuuwC
atTVgKSM8tUES5wBomoVxRH1yjDvmyb2jeq3gsp6gWPcchUpLMdfwf2MwW3NV3Mw
ESCZPwYiuFhORh1Jt5RSespjK+V9qMvCW0iU6cPE/9kAlPfMGGDv2vEttOFgOGEm
yVb1i0gHBcdzbw5H0xszBionUAQVXOFqkfO8AW8VPtFMdzZB6t9OBXRgHJLdWgmK
2Zr5pFN75uivNh4RI1KXHpUeD1kLRVICzG7Ak/aQCfKxWsJutFI1dnLFZmFOIoTf
2wgW4KsDsZakcA9rILtfo3SFH+mSD5PWzvv5G44yf9sEkGG9bSgxl29GeJYL7NzG
3Da9FVMvzjIhmxamPGHfFOFTxTud9+6GU6Lj0iBLpHzpcttjhNgE2NXzcY8r1uMD
BcSwkK3duybjeiZLpwnxOywZidCQDv6pZYyc50WBtV/oUG1fncj8DT2ZTIqGv1V8
L6D/MXSr1jt9oJeWzfDCxHlaGaHL6grrmyJ8L1tQKPjMp+DbBPFbMLfvbn/dlsat
mPqmfQZ4zydOVO53k6KiHozGQh6K+cuXMvNxrb9pCRy3etFV2wfTNxtbdeJSa7gj
xarC6vSia8KFVyXp5nydSks5woHGJFQ1kQYSLEORUWiL5zWILbtI6POzOZeYHgej
BvTzVq0AVIbxjA==
=xDIk
-----END PGP SIGNATURE-----
Merge tag 'riscv-for-linus-4.15-arch-v9-premerge' of git://git.kernel.org/pub/scm/linux/kernel/git/palmer/linux
Pull RISC-V architecture support from Palmer Dabbelt:
"This contains the core RISC-V Linux port, which has been through nine
rounds of review on various mailing lists. The port is not complete:
there's some cleanup patches moving through the review process, a
whole bunch of drivers that need some work, and a lot of feature
additions that will be needed.
The patches contained in this tag have been through nine rounds of
review on the various mailing lists. I have some outstanding cleanup
patches, but since there's been so much review on these patches I
thought it would be best to submit them as-is and then submit explicit
cleanup patches so everyone can review them. This first patch set is
big enough that it's a bit of a pain to constantly rewrite, and it's
caused a few headaches with various contributors.
The port is definately a work in progress. While what's there builds
and boots with 4.14, it's a bit hard to actually see anything happen
because there are no device drivers yet. I maintain a staging branch
that contains all the device drivers and cleanup that actually works,
but those patches won't all be ready for a while. I'd like to get what
we currently have into your tree so everyone can start working from a
single base -- of particular importance is allowing the glibc
upstreaming process to proceed so we can sort out any possibly
lingering user-visible ABI problems we might have.
Copied below is the ChangeLog that contains the history of this patch
set:
(v9) As per suggestions on our v8 patch set, I've split the core
architecture code out from our drivers and would like to submit
this patch set to be included into linux-next, with the goal
being to be merged in during the next merge window. This patch
set is based on 4.14-rc2, but if it's better to have it based on
something else then I can change it around.
This patch set contains just the core arch code for RISC-V, so
while it builds an nominally boots, you can't print or take an
interrupt so it's not that useful. If you're looking to actually
boot a system it would probably be better to use the full patch
set listed below.
We've collected a handful of tags from reviewers, and the
remainder of the patch set only got minimal feedback last time.
Here's what changed:
- We now use the device tree to initialize the timer driver so
it's less tighly coupled with the arch port.
- I cleaned up the defconfigs -- there's actually now just one,
and it's empty. For now I think we're OK with what the kernel
sets as defaults, but I anticipate we'll begin to expand this
as people start to use the port more.
- The VDSO symbols version is sane.
- We WFI while spinning in the boot loop.
- A handful of comments have been added.
While there are still a handful of FIXMEs in this patch set,
we've started to get enough interest from various users and
contributors that maintaining an out of tree patch set is
starting to become a big burden. Hopefully the patches are good
enough to merge now, which will at least get everyone working in
a more reasonable manner as we clean up the remaining issues.
(v8) I know it may not be the ideal time to submit a patch set right
now, as it's the middle of the merge window, but things have
calmed down quite a bit in the last month so I thought it would
be good to get everyone on the same page. There's been a handful
of changes since the last patch set, but most of them are fairly
minor:
- We changed PAGE_OFFSET to allowing mapping more physical
memory on 64-bit systems. This is user configurable, as it
triggers a different code model that generates slightly less
efficient code.
- The device tree binding documentation is back, I'd managed to
lose it at some point.
- We now pass the atomic64 test suite
- The SBI timer driver has been refactored.
(v7) It's been a while since my last patch set, but the changes han
been fairly minimal:
- The PCI cleanup patches have been dropped, we'll do them as a
separate patch set later.
- We've the Kconfig entries from CONFIG_ISA_* to
CONFIG_RISCV_ISA_*, to make grep easier.
- There have been a handful of memory model related tweaks in
I/O land, particularly relating the PCI and the upcoming
platform specification. There are significant comments in the
relevant files. This is still a WIP, but I think we're close
to getting as good as we're going to get until we end up with
some more specifications.
(v6) As it's been only a day since the v5 patch set, the changes are
pretty minimal:
- The patch set is now based on linux-next/master, which I
believe is a better base now that we're getting closer to
upstream.
- EARLY_PRINTK is no longer an option. Since the SBI console is
reasonable, there's no penalty to enabling it (and thus no
benefit to disabling it).
- The mmap syscalls were refactored a bit.
(v5) Things have really started to calm down, so this is fairly
similar to the v4 patch set. The most interesting changes
include:
- We've moved back to a single patch set.
- SMP support has been fixed, I was accidentally running on a
non-SMP configuration. There were various mistakes all over
the tree as a result of this.
- The cmpxchg syscalls have been removed, as they were deemed a
bad idea. As a result, RISC-V Linux systems mandate the A
extension. The corresponding Kconfig entry to enable builds
on non-A systems has been removed.
- A few more atomic fixes: mostly fence changes, but those
resulted in a handful of additional macros that were no
longer necessary.
- riscv_early_sie has been removed.
(v4) There have only been a few changes since the v3 patch set:
- The cmpxchg64 syscall is no longer enabled on 32-bit systems.
It's not possible to provide this on SMP systems, and it's
not necessary as glibc knows not to call it.
- We provide a ELF_HWCAP so users can determine the ISA of the
machine the kernel is running on.
- The multi-line comments are in a better form.
- There were a handful of headers that could be replaced with
the asm-generic versions, and a few unnecessary definitions.
- We no longer use printk, but instead use pr_*.
- A few Kconfig and defconfig entries have been cleaned up.
(v3) A highlight of the changes since the v2 patch set includes:
- We've split out all our drivers into separate patch sets,
which I've already sent out to the relevant maintainers. I
haven't included those patches in this patch set, but some of
them are necessary to build our port.
- The patch set is now split up differently: rather than being
split per directory it is split per topic. Hopefully this
will make it easier to review the port on the mailing list.
The split is a bit rough, so you probably still want to look
at the patch set as a whole.
- atomic.h has been completely rewritten and is hopefully now
correct. I've attempted to sanitize the various other memory
model related code as well, and I think it should all be sane
now aside from a handful of FIXMEs commented in the code.
- We've changed the cmpexchg syscall to always exist and to not
be multiplexed. There is also a VDSO entry for compare and
exchange, which allows kernels with the A extension to
execute user code without the A extension reasonably fast.
- Our user-visible register state now contains enough space for
the Q extension for 128-bit floating point, as well as a few
words to allow extensibility to future ISA extensions like
the eventual V extension for vectors.
- A handful of driver cleanups, but these have been split into
separate patch sets now so I won't duplicate them here.
(v2) A highlight of the changes since the v1 patch set includes:
- We've split out our drivers into the right places, which
means now there's a lot more patches. I'll be submitting
these patches to various subsystem maintainers and including
them in any future RISC-V patch sets until they've been
merged.
- The SBI console driver has been completely rewritten to use
the HVC helpers and is now significantly smaller.
- We've begun to use weaker barriers as opposed to just the big
"fence". There's still some work to do here, specifically:
- We need fences in the relaxed MMIO functions.
- The non-relaxed MMIO functions are missing R/W bits on their fences.
- Many AMOs need the aq and rl bits set.
- We now have thread_info in task_struct. As a result, sscratch
now contains TP instead of SP. This was necessary because
thread_info is no longer on the stack.
- A few shared routines have been added that we use instead of
creating another arch copy"
Reviewed-by: Arnd Bergmann <arnd@arndb.de>
* tag 'riscv-for-linus-4.15-arch-v9-premerge' of git://git.kernel.org/pub/scm/linux/kernel/git/palmer/linux:
RISC-V: Build Infrastructure
RISC-V: User-facing API
RISC-V: Paging and MMU
RISC-V: Device, timer, IRQs, and the SBI
RISC-V: Task implementation
RISC-V: ELF and module implementation
RISC-V: Generic library routines and assembly
RISC-V: Atomic and Locking Code
RISC-V: Init and Halt Code
dt-bindings: RISC-V CPU Bindings
lib: Add shared copies of some GCC library routines
MAINTAINERS: Add RISC-V
- kbuild cleanups and improvements for dtbs
- Code clean-up of overlay code and fixing for some long standing memory
leak and race condition in applying overlays
- Improvements to DT memory usage making sysfs/kobjects optional and
skipping unflattening of disabled nodes. This is part of kernel
tinification efforts.
- Final piece of removing storing the full path for every DT node. The
prerequisite conversion of printk's to use device_node format
specifier happened in 4.14.
- Sync with current upstream dtc. This brings additional checks to dtb
compiling.
- Binding doc tree wide removal of leading 0s from examples
- RTC binding documentation adding missing devices and some
consolidation of duplicated bindings
- Vendor prefix documentation for nutsboard, Silicon Storage Technology,
shimafuji, Tecon Microprocessor Technologies, DH electronics GmbH,
Opal Kelly, and Next Thing
-----BEGIN PGP SIGNATURE-----
iQItBAABCAAXBQJaCwaSEBxyb2JoQGtlcm5lbC5vcmcACgkQ+vtdtY28YcNzeA/8
C8uQhSsX2+UQZvFzcEA8KQAMGT3kYdrcf+gidRKwCEUWg1qscUEpTb3n3Rm5NUbU
RPD1s6GSlh6fJCMHDTQ6Tti/T59L7nZa2/AIGmUishGu4x4q1o18AobpFJmYP/EM
SJPwnmm5RV9WcZFao1y+sY3Xtn8DStxHO4cS+dyF5/EvPN9D8nbLJfu7bgTBAZww
HktIMB9kx+GTipRQZBvBwXoy5MJjthIZub4XwzesA4tGananj4cXlc0xaVxpdYy3
5bO6q5F7cbrZ2uyrF+oIChpCENK4VaXh80m0WHc8EzaG++shzEkR4he1vYkwnV+I
OYo4vsUg9dP8rBksUG1eYhS8fJKPvEBRNP7ETT5utVBy5I/tDEbo/crmQZRTIDIC
hZbhcdZlISZj0DzkMK2ZHQV9UYtRWzXrJbZHFIPP12GCyvXVxYJUIWb9iYnUYSon
KugygsFSpZHMWmfAhemw5/ctJZ19qhM5UIl2KZk5tMBHAf466ILmZjg0me6fYkOp
eADfwHJ1dLMdK79CVMHSfp+vArcZXp35B16c3sWpJB36Il97Mc/9siEufCL4GKX7
IBBnQBlbpSBKBejWVyI7Ip/Xp5u4qAQD+ZMJ9oLqBRqfWerHbDuOERlEOgwGqJYr
9v4HvP7V8eVUvAdqXka4EBfCyAgUzXDAxG2Dfmv9vGU=
=jgpN
-----END PGP SIGNATURE-----
Merge tag 'devicetree-for-4.15' of git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux
Pull DeviceTree updates from Rob Herring:
"A bigger diffstat than usual with the kbuild changes and a tree wide
fix in the binding documentation.
Summary:
- kbuild cleanups and improvements for dtbs
- Code clean-up of overlay code and fixing for some long standing
memory leak and race condition in applying overlays
- Improvements to DT memory usage making sysfs/kobjects optional and
skipping unflattening of disabled nodes. This is part of kernel
tinification efforts.
- Final piece of removing storing the full path for every DT node.
The prerequisite conversion of printk's to use device_node format
specifier happened in 4.14.
- Sync with current upstream dtc. This brings additional checks to
dtb compiling.
- Binding doc tree wide removal of leading 0s from examples
- RTC binding documentation adding missing devices and some
consolidation of duplicated bindings
- Vendor prefix documentation for nutsboard, Silicon Storage
Technology, shimafuji, Tecon Microprocessor Technologies, DH
electronics GmbH, Opal Kelly, and Next Thing"
* tag 'devicetree-for-4.15' of git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux: (55 commits)
dt-bindings: usb: add #phy-cells to usb-nop-xceiv
dt-bindings: Remove leading zeros from bindings notation
kbuild: handle dtb-y and CONFIG_OF_ALL_DTBS natively in Makefile.lib
MIPS: dts: remove bogus bcm96358nb4ser.dtb from dtb-y entry
kbuild: clean up *.dtb and *.dtb.S patterns from top-level Makefile
.gitignore: move *.dtb and *.dtb.S patterns to the top-level .gitignore
.gitignore: sort normal pattern rules alphabetically
dt-bindings: add vendor prefix for Next Thing Co.
scripts/dtc: Update to upstream version v1.4.5-6-gc1e55a5513e9
of: dynamic: fix memory leak related to properties of __of_node_dup
of: overlay: make pr_err() string unique
of: overlay: pr_err from return NOTIFY_OK to overlay apply/remove
of: overlay: remove unneeded check for NULL kbasename()
of: overlay: remove a dependency on device node full_name
of: overlay: simplify applying symbols from an overlay
of: overlay: avoid race condition between applying multiple overlays
of: overlay: loosen overly strict phandle clash check
of: overlay: expand check of whether overlay changeset can be removed
of: overlay: detect cases where device tree may become corrupt
of: overlay: minor restructuring
...
Pull x86 core updates from Ingo Molnar:
"Note that in this cycle most of the x86 topics interacted at a level
that caused them to be merged into tip:x86/asm - but this should be a
temporary phenomenon, hopefully we'll back to the usual patterns in
the next merge window.
The main changes in this cycle were:
Hardware enablement:
- Add support for the Intel UMIP (User Mode Instruction Prevention)
CPU feature. This is a security feature that disables certain
instructions such as SGDT, SLDT, SIDT, SMSW and STR. (Ricardo Neri)
[ Note that this is disabled by default for now, there are some
smaller enhancements in the pipeline that I'll follow up with in
the next 1-2 days, which allows this to be enabled by default.]
- Add support for the AMD SEV (Secure Encrypted Virtualization) CPU
feature, on top of SME (Secure Memory Encryption) support that was
added in v4.14. (Tom Lendacky, Brijesh Singh)
- Enable new SSE/AVX/AVX512 CPU features: AVX512_VBMI2, GFNI, VAES,
VPCLMULQDQ, AVX512_VNNI, AVX512_BITALG. (Gayatri Kammela)
Other changes:
- A big series of entry code simplifications and enhancements (Andy
Lutomirski)
- Make the ORC unwinder default on x86 and various objtool
enhancements. (Josh Poimboeuf)
- 5-level paging enhancements (Kirill A. Shutemov)
- Micro-optimize the entry code a bit (Borislav Petkov)
- Improve the handling of interdependent CPU features in the early
FPU init code (Andi Kleen)
- Build system enhancements (Changbin Du, Masahiro Yamada)
- ... plus misc enhancements, fixes and cleanups"
* 'x86-asm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (118 commits)
x86/build: Make the boot image generation less verbose
selftests/x86: Add tests for the STR and SLDT instructions
selftests/x86: Add tests for User-Mode Instruction Prevention
x86/traps: Fix up general protection faults caused by UMIP
x86/umip: Enable User-Mode Instruction Prevention at runtime
x86/umip: Force a page fault when unable to copy emulated result to user
x86/umip: Add emulation code for UMIP instructions
x86/cpufeature: Add User-Mode Instruction Prevention definitions
x86/insn-eval: Add support to resolve 16-bit address encodings
x86/insn-eval: Handle 32-bit address encodings in virtual-8086 mode
x86/insn-eval: Add wrapper function for 32 and 64-bit addresses
x86/insn-eval: Add support to resolve 32-bit address encodings
x86/insn-eval: Compute linear address in several utility functions
resource: Fix resource_size.cocci warnings
X86/KVM: Clear encryption attribute when SEV is active
X86/KVM: Decrypt shared per-cpu variables when SEV is active
percpu: Introduce DEFINE_PER_CPU_DECRYPTED
x86: Add support for changing memory encryption attribute in early boot
x86/io: Unroll string I/O when SEV is active
x86/boot: Add early boot support when running with SEV active
...
- The old driver statement has been added to the kernel docs.
- We have a couple of new helper scripts. find-unused-docs.sh from Sayli
Karnic will point out kerneldoc comments that are not actually used in
the documentation. Jani Nikula's documentation-file-ref-check finds
references to non-existing files.
- A new ftrace document from Steve Rostedt.
- Vinod Koul converted the dmaengine docs to RST
Beyond that, it's mostly simple fixes.
-----BEGIN PGP SIGNATURE-----
iQIcBAABAgAGBQJaCK37AAoJEI3ONVYwIuV63nwQALeqzVwGqqTwiyRyMqgEwMQM
je/6IurEwTHtyfwtW/mztCfNid1CLTiYZg7RET3/zlHjcUI/9VlV2dbBksGFgoQo
muHGqhwTJjXYREwjK3FkzrGckRsVZKJgdzmZYgukCCY6Ir7IffwJKYaLOCZN1S/l
4nBHQpt2nITo0WhdmZjaNRKOQxMA8nN5yGpOIl0neGE6ywIUMgauCCCHhxnOPVWg
ant1HliS8WR8Tizqt9wQgLCvs5lvklsBFibZPO9LBTPG2Zy3HIO9kb+npUAh2MTl
j0Wg39zzOFvVVErqErqUIwIuQ9IrfltHrEHYYoruTvDBXBiMKIcwApF+DS+H3WSp
TnDu3Qif4llM5SZsZGvcjawXNnbck+7SYOe9cyqpylV3SWMWrEX1tbUv6zVuVk+7
fencYBvEZgkJmWbjDeO/Z4S50STxRTzIxFwZgLft7g/RiHo9HvlubjjwQTqBFjxA
fVkolN7h69MGkrD8TF19eapyujqSXaNYH0pFYo87JNOjLgYmezUHyvHd8YeZJL31
Ll0h10HqSNVzJsjFolBMgrC3CcVjsEXdBufu0yVk45sAg9ZiMYOCpwa6Rtp+tfxa
uIBf1LKzfWSa0ocKx7+sMJt0B/CXwU3AMtsbYGyDhFhR2r3cp1NWBHf5nisz9etD
2Md9RDFAMLELZurewB9Q
=H6ud
-----END PGP SIGNATURE-----
Merge tag 'docs-4.15' of git://git.lwn.net/linux
Pull documentation updates from Jonathan Corbet:
"A relatively calm cycle for the docs tree again.
- The old driver statement has been added to the kernel docs.
- We have a couple of new helper scripts. find-unused-docs.sh from
Sayli Karnic will point out kerneldoc comments that are not actually
used in the documentation. Jani Nikula's
documentation-file-ref-check finds references to non-existing files.
- A new ftrace document from Steve Rostedt.
- Vinod Koul converted the dmaengine docs to RST
Beyond that, it's mostly simple fixes.
This set reaches outside of Documentation/ a bit more than most. In
all cases, the changes are to comment docs, mostly from Randy, in
places where there didn't seem to be anybody better to take them"
* tag 'docs-4.15' of git://git.lwn.net/linux: (52 commits)
documentation: fb: update list of available compiled-in fonts
MAINTAINERS: update DMAengine documentation location
dmaengine: doc: ReSTize pxa_dma doc
dmaengine: doc: ReSTize dmatest doc
dmaengine: doc: ReSTize client API doc
dmaengine: doc: ReSTize provider doc
dmaengine: doc: Add ReST style dmaengine document
ftrace/docs: Add documentation on how to use ftrace from within the kernel
bug-hunting.rst: Fix an example and a typo in a Sphinx tag
scripts: Add a script to find unused documentation
samples: Convert timers to use timer_setup()
documentation: kernel-api: add more info on bitmap functions
Documentation: fix selftests related file refs
Documentation: fix ref to power basic-pm-debugging
Documentation: fix ref to trace stm content
Documentation: fix ref to coccinelle content
Documentation: fix ref to workqueue content
Documentation: fix ref to sphinx/kerneldoc.py
Documentation: fix locking rt-mutex doc refs
docs: dev-tools: correct Coccinelle version number
...
Some $(call cc-option,...) are invoked very early, even before
KBUILD_CFLAGS, etc. are initialized.
The returned string from $(call cc-option,...) depends on
KBUILD_CPPFLAGS, KBUILD_CFLAGS, and GCC_PLUGINS_CFLAGS.
Since they are exported, they are not empty when the top Makefile
is recursively invoked.
The recursion occurs in several places. For example, the top
Makefile invokes itself for silentoldconfig. "make tinyconfig",
"make rpm-pkg" are the cases, too.
In those cases, the second call of cc-option from the same line
runs a different shell command due to non-pristine KBUILD_CFLAGS.
To get the same result all the time, KBUILD_* and GCC_PLUGINS_CFLAGS
must be initialized before any call of cc-option. This avoids
garbage data in the .cache.mk file.
Move all calls of cc-option below the config targets because target
compiler flags are unnecessary for Kconfig.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Douglas Anderson <dianders@chromium.org>
These are a few stragglers that I left out of the original patch to
cache calls to the C compiler ("kbuild: Add a cache for generated
variables") because they bleed out into the main Makefile and thus
uglify things a little bit. The idea is the same here, though.
Signed-off-by: Douglas Anderson <dianders@chromium.org>
Tested-by: Ingo Molnar <mingo@kernel.org>
Tested-by: Guenter Roeck <linux@roeck-us.net>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
While timing a "no-op" build of the kernel (incrementally building the
kernel even though nothing changed) in the Chrome OS build system I
found that it was much slower than I expected.
Digging into things a bit, I found that quite a bit of the time was
spent invoking the C compiler even though we weren't actually building
anything. Currently in the Chrome OS build system the C compiler is
called through a number of wrappers (one of which is written in
python!) and can take upwards of 100 ms to invoke even if we're not
doing anything difficult, so these invocations of the compiler were
taking a lot of time. Worse the invocations couldn't seem to take
advantage of the multiple cores on my system.
Certainly it seems like we could make the compiler invocations in the
Chrome OS build system faster, but only to a point. Inherently
invoking a program as big as a C compiler is a fairly heavy
operation. Thus even if we can speed the compiler calls it made sense
to track down what was happening.
It turned out that all the compiler invocations were coming from
usages like this in the kernel's Makefile:
KBUILD_CFLAGS += $(call cc-option,-fno-delete-null-pointer-checks,)
Due to the way cc-option and similar statements work the above
contains an implicit call to the C compiler. ...and due to the fact
that we're storing the result in KBUILD_CFLAGS, a simply expanded
variable, the call will happen every time the Makefile is parsed, even
if there are no users of KBUILD_CFLAGS.
Rather than redoing this computation every time, it makes a lot of
sense to cache the result of all of the Makefile's compiler calls just
like we do when we compile a ".c" file to a ".o" file. Conceptually
this is quite a simple idea. ...and since the calls to invoke the
compiler and similar tools are centrally located in the Kbuild.include
file this doesn't even need to be super invasive.
Implementing the cache in a simple-to-use and efficient way is not
quite as simple as it first sounds, though. To get maximum speed we
really want the cache in a format that make can natively understand
and make doesn't really have an ability to load/parse files. ...but
make _can_ import other Makefiles, so the solution is to store the
cache in Makefile format. This requires coming up with a valid/unique
Makefile variable name for each value to be cached, but that's
solvable with some cleverness.
After this change, we'll automatically create a ".cache.mk" file that
will contain our cached variables. We'll load this on each invocation
of make and will avoid recomputing anything that's already in our
cache. The cache is stored in a format that it shouldn't need any
invalidation since anything that might change should affect the "key"
and any old cached value won't be used.
Signed-off-by: Douglas Anderson <dianders@chromium.org>
Tested-by: Ingo Molnar <mingo@kernel.org>
Tested-by: Guenter Roeck <linux@roeck-us.net>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
We need to add "clean-files" in Makfiles to clean up DT blobs, but we
often miss to do so.
Since there are no source files that end with .dtb or .dtb.S, so we
can clean-up those files from the top-level Makefile.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Rob Herring <robh@kernel.org>
Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.
By default all files without license information are under the default
license of the kernel, which is GPL version 2.
Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier. The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.
This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.
How this work was done:
Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
- file had no licensing information it it.
- file was a */uapi/* one with no licensing information in it,
- file was a */uapi/* one with existing licensing information,
Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.
The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne. Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.
The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed. Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.
Criteria used to select files for SPDX license identifier tagging was:
- Files considered eligible had to be source code files.
- Make and config files were included as candidates if they contained >5
lines of source
- File already had some variant of a license header in it (even if <5
lines).
All documentation files were explicitly excluded.
The following heuristics were used to determine which SPDX license
identifiers to apply.
- when both scanners couldn't find any license traces, file was
considered to have no license information in it, and the top level
COPYING file license applied.
For non */uapi/* files that summary was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 11139
and resulted in the first patch in this series.
If that file was a */uapi/* path one, it was "GPL-2.0 WITH
Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 WITH Linux-syscall-note 930
and resulted in the second patch in this series.
- if a file had some form of licensing information in it, and was one
of the */uapi/* ones, it was denoted with the Linux-syscall-note if
any GPL family license was found in the file or had no licensing in
it (per prior point). Results summary:
SPDX license identifier # files
---------------------------------------------------|------
GPL-2.0 WITH Linux-syscall-note 270
GPL-2.0+ WITH Linux-syscall-note 169
((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21
((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17
LGPL-2.1+ WITH Linux-syscall-note 15
GPL-1.0+ WITH Linux-syscall-note 14
((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5
LGPL-2.0+ WITH Linux-syscall-note 4
LGPL-2.1 WITH Linux-syscall-note 3
((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3
((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1
and that resulted in the third patch in this series.
- when the two scanners agreed on the detected license(s), that became
the concluded license(s).
- when there was disagreement between the two scanners (one detected a
license but the other didn't, or they both detected different
licenses) a manual inspection of the file occurred.
- In most cases a manual inspection of the information in the file
resulted in a clear resolution of the license that should apply (and
which scanner probably needed to revisit its heuristics).
- When it was not immediately clear, the license identifier was
confirmed with lawyers working with the Linux Foundation.
- If there was any question as to the appropriate license identifier,
the file was flagged for further research and to be revisited later
in time.
In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases, confirmation
by lawyers working with the Linux Foundation.
Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights. The
Windriver scanner is based on an older version of FOSSology in part, so
they are related.
Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot checks
in about 15000 files.
In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect the
correct identifier.
Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial patch
version early this week with:
- a full scancode scan run, collecting the matched texts, detected
license ids and scores
- reviewing anything where there was a license detected (about 500+
files) to ensure that the applied SPDX license was correct
- reviewing anything where there was no detection but the patch license
was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
SPDX license was correct
This produced a worksheet with 20 files needing minor correction. This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.
These .csv files were then reviewed by Greg. Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected. This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.) Finally Greg ran the script using the .csv files to
generate the patches.
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
-----BEGIN PGP SIGNATURE-----
iG0EABECAC0WIQT0tgzFv3jCIUoxPcsxR9QN2y37KQUCWfswbQ8cZ3JlZ0Brcm9h
aC5jb20ACgkQMUfUDdst+ykvEwCfXU1MuYFQGgMdDmAZXEc+xFXZvqgAoKEcHDNA
6dVh26uchcEQLN/XqUDt
=x306
-----END PGP SIGNATURE-----
Merge tag 'spdx_identifiers-4.14-rc8' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/driver-core
Pull initial SPDX identifiers from Greg KH:
"License cleanup: add SPDX license identifiers to some files
Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.
By default all files without license information are under the default
license of the kernel, which is GPL version 2.
Update the files which contain no license information with the
'GPL-2.0' SPDX license identifier. The SPDX identifier is a legally
binding shorthand, which can be used instead of the full boiler plate
text.
This patch is based on work done by Thomas Gleixner and Kate Stewart
and Philippe Ombredanne.
How this work was done:
Patches were generated and checked against linux-4.14-rc6 for a subset
of the use cases:
- file had no licensing information it it.
- file was a */uapi/* one with no licensing information in it,
- file was a */uapi/* one with existing licensing information,
Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to
license had to be inferred by heuristics based on keywords.
The analysis to determine which SPDX License Identifier to be applied
to a file was done in a spreadsheet of side by side results from of
the output of two independent scanners (ScanCode & Windriver)
producing SPDX tag:value files created by Philippe Ombredanne.
Philippe prepared the base worksheet, and did an initial spot review
of a few 1000 files.
The 4.13 kernel was the starting point of the analysis with 60,537
files assessed. Kate Stewart did a file by file comparison of the
scanner results in the spreadsheet to determine which SPDX license
identifier(s) to be applied to the file. She confirmed any
determination that was not immediately clear with lawyers working with
the Linux Foundation.
Criteria used to select files for SPDX license identifier tagging was:
- Files considered eligible had to be source code files.
- Make and config files were included as candidates if they contained
>5 lines of source
- File already had some variant of a license header in it (even if <5
lines).
All documentation files were explicitly excluded.
The following heuristics were used to determine which SPDX license
identifiers to apply.
- when both scanners couldn't find any license traces, file was
considered to have no license information in it, and the top level
COPYING file license applied.
For non */uapi/* files that summary was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 11139
and resulted in the first patch in this series.
If that file was a */uapi/* path one, it was "GPL-2.0 WITH
Linux-syscall-note" otherwise it was "GPL-2.0". Results of that
was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 WITH Linux-syscall-note 930
and resulted in the second patch in this series.
- if a file had some form of licensing information in it, and was one
of the */uapi/* ones, it was denoted with the Linux-syscall-note if
any GPL family license was found in the file or had no licensing in
it (per prior point). Results summary:
SPDX license identifier # files
---------------------------------------------------|------
GPL-2.0 WITH Linux-syscall-note 270
GPL-2.0+ WITH Linux-syscall-note 169
((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21
((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17
LGPL-2.1+ WITH Linux-syscall-note 15
GPL-1.0+ WITH Linux-syscall-note 14
((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5
LGPL-2.0+ WITH Linux-syscall-note 4
LGPL-2.1 WITH Linux-syscall-note 3
((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3
((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1
and that resulted in the third patch in this series.
- when the two scanners agreed on the detected license(s), that
became the concluded license(s).
- when there was disagreement between the two scanners (one detected
a license but the other didn't, or they both detected different
licenses) a manual inspection of the file occurred.
- In most cases a manual inspection of the information in the file
resulted in a clear resolution of the license that should apply
(and which scanner probably needed to revisit its heuristics).
- When it was not immediately clear, the license identifier was
confirmed with lawyers working with the Linux Foundation.
- If there was any question as to the appropriate license identifier,
the file was flagged for further research and to be revisited later
in time.
In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases,
confirmation by lawyers working with the Linux Foundation.
Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights.
The Windriver scanner is based on an older version of FOSSology in
part, so they are related.
Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot
checks in about 15000 files.
In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect
the correct identifier.
Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial
patch version early this week with:
- a full scancode scan run, collecting the matched texts, detected
license ids and scores
- reviewing anything where there was a license detected (about 500+
files) to ensure that the applied SPDX license was correct
- reviewing anything where there was no detection but the patch
license was not GPL-2.0 WITH Linux-syscall-note to ensure that the
applied SPDX license was correct
This produced a worksheet with 20 files needing minor correction. This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.
These .csv files were then reviewed by Greg. Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected. This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.) Finally Greg ran the script using the .csv files to
generate the patches.
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>"
* tag 'spdx_identifiers-4.14-rc8' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/driver-core:
License cleanup: add SPDX license identifier to uapi header files with a license
License cleanup: add SPDX license identifier to uapi header files with no license
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.
By default all files without license information are under the default
license of the kernel, which is GPL version 2.
Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier. The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.
This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.
How this work was done:
Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
- file had no licensing information it it.
- file was a */uapi/* one with no licensing information in it,
- file was a */uapi/* one with existing licensing information,
Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.
The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne. Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.
The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed. Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.
Criteria used to select files for SPDX license identifier tagging was:
- Files considered eligible had to be source code files.
- Make and config files were included as candidates if they contained >5
lines of source
- File already had some variant of a license header in it (even if <5
lines).
All documentation files were explicitly excluded.
The following heuristics were used to determine which SPDX license
identifiers to apply.
- when both scanners couldn't find any license traces, file was
considered to have no license information in it, and the top level
COPYING file license applied.
For non */uapi/* files that summary was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 11139
and resulted in the first patch in this series.
If that file was a */uapi/* path one, it was "GPL-2.0 WITH
Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 WITH Linux-syscall-note 930
and resulted in the second patch in this series.
- if a file had some form of licensing information in it, and was one
of the */uapi/* ones, it was denoted with the Linux-syscall-note if
any GPL family license was found in the file or had no licensing in
it (per prior point). Results summary:
SPDX license identifier # files
---------------------------------------------------|------
GPL-2.0 WITH Linux-syscall-note 270
GPL-2.0+ WITH Linux-syscall-note 169
((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21
((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17
LGPL-2.1+ WITH Linux-syscall-note 15
GPL-1.0+ WITH Linux-syscall-note 14
((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5
LGPL-2.0+ WITH Linux-syscall-note 4
LGPL-2.1 WITH Linux-syscall-note 3
((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3
((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1
and that resulted in the third patch in this series.
- when the two scanners agreed on the detected license(s), that became
the concluded license(s).
- when there was disagreement between the two scanners (one detected a
license but the other didn't, or they both detected different
licenses) a manual inspection of the file occurred.
- In most cases a manual inspection of the information in the file
resulted in a clear resolution of the license that should apply (and
which scanner probably needed to revisit its heuristics).
- When it was not immediately clear, the license identifier was
confirmed with lawyers working with the Linux Foundation.
- If there was any question as to the appropriate license identifier,
the file was flagged for further research and to be revisited later
in time.
In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases, confirmation
by lawyers working with the Linux Foundation.
Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights. The
Windriver scanner is based on an older version of FOSSology in part, so
they are related.
Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot checks
in about 15000 files.
In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect the
correct identifier.
Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial patch
version early this week with:
- a full scancode scan run, collecting the matched texts, detected
license ids and scores
- reviewing anything where there was a license detected (about 500+
files) to ensure that the applied SPDX license was correct
- reviewing anything where there was no detection but the patch license
was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
SPDX license was correct
This produced a worksheet with 20 files needing minor correction. This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.
These .csv files were then reviewed by Greg. Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected. This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.) Finally Greg ran the script using the .csv files to
generate the patches.
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Pick up some of the MPX commits that modify the syscall entry code,
to have a common base and to reduce conflicts.
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Linus Torvalds