OpenCloudOS-Kernel/Documentation/process/changes.rst

509 lines
13 KiB
ReStructuredText
Raw Normal View History

.. _changes:
Minimal requirements to compile the Kernel
++++++++++++++++++++++++++++++++++++++++++
Intro
=====
This document is designed to provide a list of the minimum levels of
software necessary to run the 4.x kernels.
This document is originally based on my "Changes" file for 2.0.x kernels
and therefore owes credit to the same people as that file (Jared Mauch,
Axel Boldt, Alessandro Sigala, and countless other users all over the
'net).
Current Minimal Requirements
****************************
Upgrade to at **least** these software revisions before thinking you've
encountered a bug! If you're unsure what version you're currently
running, the suggested command should tell you.
Again, keep in mind that this list assumes you are already functionally
running a Linux kernel. Also, not all tools are necessary on all
systems; obviously, if you don't have any PC Card hardware, for example,
you probably needn't concern yourself with pcmciautils.
====================== =============== ========================================
Program Minimal version Command to check the version
====================== =============== ========================================
Documentation: raise minimum supported version of GCC to 5.1 commit fad7cd3310db ("nbd: add the check to prevent overflow in __nbd_ioctl()") raised an issue from the fallback helpers added in commit f0907827a8a9 ("compiler.h: enable builtin overflow checkers and add fallback code") Specifically, the helpers for checking whether the results of a multiplication overflowed (__unsigned_mul_overflow, __signed_add_overflow) use the division operator when !COMPILER_HAS_GENERIC_BUILTIN_OVERFLOW. This is problematic for 64b operands on 32b hosts. Also, because the macro is type agnostic, it is very difficult to write a similarly type generic macro that dispatches to one of: * div64_s64 * div64_u64 * div_s64 * div_u64 Raising the minimum supported versions allows us to remove all of the fallback helpers for !COMPILER_HAS_GENERIC_BUILTIN_OVERFLOW, instead dispatching the compiler builtins. arm64 has already raised the minimum supported GCC version to 5.1, do this for all targets now. See the link below for the previous discussion. Link: https://lore.kernel.org/all/20210909182525.372ee687@canb.auug.org.au/ Link: https://lore.kernel.org/lkml/CAK7LNASs6dvU6D3jL2GG3jW58fXfaj6VNOe55NJnTB8UPuk2pA@mail.gmail.com/ Link: https://github.com/ClangBuiltLinux/linux/issues/1438 Reported-by: Stephen Rothwell <sfr@canb.auug.org.au> Reported-by: Nathan Chancellor <nathan@kernel.org> Suggested-by: Rasmus Villemoes <linux@rasmusvillemoes.dk> Signed-off-by: Nick Desaulniers <ndesaulniers@google.com> Reviewed-by: Kees Cook <keescook@chromium.org> Reviewed-by: Nathan Chancellor <nathan@kernel.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2021-09-11 07:40:38 +08:00
GNU C 5.1 gcc --version
Documentation: Raise the minimum supported version of LLVM to 11.0.0 LLVM versions prior to 11.0.0 have a harder time with dead code elimination, which can cause issues with commonly used expressions such as BUILD_BUG_ON and the bitmask functions/macros in bitfield.h (see the first two issues links below). Whenever there is an issue within LLVM that has been resolved in a later release, the only course of action is to gate the problematic configuration or source code on the toolchain verson or raise the minimum supported version of LLVM for building the kernel, as LLVM has a limited support lifetime compared to GCC. GCC major releases will typically see a few point releases across a two year period on average whereas LLVM major releases are only supported until the next major release and will only see one or two point releases within that timeframe. For example, GCC 8.1 was released in May 2018 and GCC 8.5 was released in May 2021, whereas LLVM 12.0.0 was released in April 2021 and its only point release, 12.0.1, was released in July 2021, giving a minimal window for fixes to be backported. To resolve these build errors around improper dead code elimination, raise the minimum supported version of LLVM for building the kernel to 11.0.0. Doing so is a more proper solution than mucking around with core kernel macros that have always worked with GCC or disabling drivers for using these macros in a proper manner. This type of issue may continue to crop up and require patching, which creates more debt for bumping the minimum supported version in the future. This should have a minimal impact to distributions. Using a script to pull several different Docker images and check the output of 'clang --version': archlinux:latest: clang version 13.0.0 debian:oldoldstable-slim: clang version 3.8.1-24 (tags/RELEASE_381/final) debian:oldstable-slim: clang version 7.0.1-8+deb10u2 (tags/RELEASE_701/final) debian:stable-slim: Debian clang version 11.0.1-2 debian:testing-slim: Debian clang version 11.1.0-4 debian:unstable-slim: Debian clang version 11.1.0-4 fedora:34: clang version 12.0.1 (Fedora 12.0.1-1.fc34) fedora:latest: clang version 13.0.0 (Fedora 13.0.0-3.fc35) fedora:rawhide: clang version 13.0.0 (Fedora 13.0.0-5.fc36) opensuse/leap:15.2: clang version 9.0.1 opensuse/leap:latest: clang version 11.0.1 opensuse/tumbleweed:latest: clang version 13.0.0 ubuntu:bionic: clang version 6.0.0-1ubuntu2 (tags/RELEASE_600/final) ubuntu:latest: clang version 10.0.0-4ubuntu1 ubuntu:hirsute: Ubuntu clang version 12.0.0-3ubuntu1~21.04.2 ubuntu:rolling: Ubuntu clang version 13.0.0-2 ubuntu:devel: Ubuntu clang version 13.0.0-9 In every case, the distribution's version of clang is either older than the current minimum supported version of LLVM 10.0.1 or equal to or greater than the proposed 11.0.0 so nothing should change. Another benefit of this change is LLVM=1 works better with arm64 and x86_64 since commit f12b034afeb3 ("scripts/Makefile.clang: default to LLVM_IAS=1") enabled the integrated assembler by default, which only works well with clang 11+ (clang-10 required it to be disabled to successfully build a kernel). Link: https://github.com/ClangBuiltLinux/linux/issues/1293 Link: https://github.com/ClangBuiltLinux/linux/issues/1506 Link: https://github.com/ClangBuiltLinux/linux/issues/1511 Link: https://github.com/llvm/llvm-project/commit/fa496ce3c6774097080c8a9cb808da56f383b938 Link: https://groups.google.com/g/clang-built-linux/c/mPQb9_ZWW0s/m/W7o6S-QTBAAJ Link: https://github.com/ClangBuiltLinux/misc-scripts Signed-off-by: Nathan Chancellor <nathan@kernel.org> Reviewed-by: Miguel Ojeda <ojeda@kernel.org> Reviewed-by: Mark Brown <broonie@kernel.org> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com> Reviewed-by: Kees Cook <keescook@chromium.org> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2021-11-30 00:57:58 +08:00
Clang/LLVM (optional) 11.0.0 clang --version
GNU make 3.81 make --version
binutils 2.23 ld -v
kbuild: prepare to remove C files pre-generated by flex and bison In Linux build system convention, pre-generated files are version- controlled with a "_shipped" suffix. During the kernel building, they are simply shipped (copied) removing the suffix. This approach can reduce external tool dependency for the kernel build, but it is tedious to manually regenerate such artifacts from developers' point of view. (We need to do "make REGENERATE_PARSERS=1" every time we touch real source files such as *.l, *.y) Some months ago, I sent out RFC patches to run flex, bison, and gperf during the build. In the review and test, Linus noticed gperf-3.1 had changed the lookup function prototype. Then, the use of gperf in kernel was entirely removed by commit bb3290d91695 ("Remove gperf usage from toolchain"). This time, I tested several versions of flex and bison, and I was not hit by any compatibility issue except a flaw in flex-2.6.3; if you generate lexer for dtc and genksyms with flex-2.6.3, you will see "yywrap redefined" warning. This was not intentional, but a bug, fixed by flex-2.6.4. Otherwise, both flex and bison look fairly stable for a long time. This commit prepares some build rules to remove the _shipped files. Also, document minimal requirement for flex and bison. Rationale for the minimal version: The -Wmissing-prototypes option of GCC warns "no previous prototype" for lexers generated by flex-2.5.34 or older, so I chose 2.5.35 as the required version for flex. Flex-2.5.35 was released in 2008. Bison looks more stable. I did not see any problem with bison-2.0, released in 2004. I did not test bison-1.x, but bison-2.0 should be old enough. Tested flex versions: 2.5.35 2.5.36 2.5.37 2.5.39 2.6.0 2.6.1 2.6.2 2.6.3 (*) 2.6.4 (*) flex-2.6.3 causes "yywrap redefined" warning Tested bison versions: 2.0 2.1 2.2 2.3 2.4 2.4.1 2.5.1 2.6 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7 2.7.1 3.0 3.0.1 3.0.2 3.0.3 3.0.4 Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-12-10 00:02:29 +08:00
flex 2.5.35 flex --version
bison 2.0 bison --version
pahole 1.16 pahole --version
util-linux 2.10o fdformat --version
kmod 13 depmod -V
e2fsprogs 1.41.4 e2fsck -V
jfsutils 1.1.3 fsck.jfs -V
reiserfsprogs 3.6.3 reiserfsck -V
xfsprogs 2.6.0 xfs_db -V
squashfs-tools 4.0 mksquashfs -version
btrfs-progs 0.18 btrfsck
pcmciautils 004 pccardctl -V
quota-tools 3.09 quota -V
PPP 2.4.0 pppd --version
nfs-utils 1.0.5 showmount --version
procps 3.2.0 ps --version
udev 081 udevd --version
grub 0.93 grub --version || grub-install --version
mcelog 0.6 mcelog --version
iptables 1.4.2 iptables -V
openssl & libcrypto 1.0.0 openssl version
bc 1.06.95 bc --version
Sphinx\ [#f1]_ 1.7 sphinx-build --version
cpio any cpio --version
====================== =============== ========================================
.. [#f1] Sphinx is needed only to build the Kernel documentation
Kernel compilation
******************
GCC
---
The gcc version requirements may vary depending on the type of CPU in your
computer.
Clang/LLVM (optional)
---------------------
The latest formal release of clang and LLVM utils (according to
`releases.llvm.org <https://releases.llvm.org>`_) are supported for building
kernels. Older releases aren't guaranteed to work, and we may drop workarounds
from the kernel that were used to support older versions. Please see additional
docs on :ref:`Building Linux with Clang/LLVM <kbuild_llvm>`.
Make
----
You will need GNU make 3.81 or later to build the kernel.
Binutils
--------
Binutils 2.23 or newer is needed to build the kernel.
pkg-config
----------
The build system, as of 4.18, requires pkg-config to check for installed
kconfig tools and to determine flags settings for use in
'make {g,x}config'. Previously pkg-config was being used but not
verified or documented.
kbuild: prepare to remove C files pre-generated by flex and bison In Linux build system convention, pre-generated files are version- controlled with a "_shipped" suffix. During the kernel building, they are simply shipped (copied) removing the suffix. This approach can reduce external tool dependency for the kernel build, but it is tedious to manually regenerate such artifacts from developers' point of view. (We need to do "make REGENERATE_PARSERS=1" every time we touch real source files such as *.l, *.y) Some months ago, I sent out RFC patches to run flex, bison, and gperf during the build. In the review and test, Linus noticed gperf-3.1 had changed the lookup function prototype. Then, the use of gperf in kernel was entirely removed by commit bb3290d91695 ("Remove gperf usage from toolchain"). This time, I tested several versions of flex and bison, and I was not hit by any compatibility issue except a flaw in flex-2.6.3; if you generate lexer for dtc and genksyms with flex-2.6.3, you will see "yywrap redefined" warning. This was not intentional, but a bug, fixed by flex-2.6.4. Otherwise, both flex and bison look fairly stable for a long time. This commit prepares some build rules to remove the _shipped files. Also, document minimal requirement for flex and bison. Rationale for the minimal version: The -Wmissing-prototypes option of GCC warns "no previous prototype" for lexers generated by flex-2.5.34 or older, so I chose 2.5.35 as the required version for flex. Flex-2.5.35 was released in 2008. Bison looks more stable. I did not see any problem with bison-2.0, released in 2004. I did not test bison-1.x, but bison-2.0 should be old enough. Tested flex versions: 2.5.35 2.5.36 2.5.37 2.5.39 2.6.0 2.6.1 2.6.2 2.6.3 (*) 2.6.4 (*) flex-2.6.3 causes "yywrap redefined" warning Tested bison versions: 2.0 2.1 2.2 2.3 2.4 2.4.1 2.5.1 2.6 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7 2.7.1 3.0 3.0.1 3.0.2 3.0.3 3.0.4 Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-12-10 00:02:29 +08:00
Flex
----
Since Linux 4.16, the build system generates lexical analyzers
during build. This requires flex 2.5.35 or later.
Bison
-----
Since Linux 4.16, the build system generates parsers
during build. This requires bison 2.0 or later.
pahole:
-------
Since Linux 5.2, if CONFIG_DEBUG_INFO_BTF is selected, the build system
generates BTF (BPF Type Format) from DWARF in vmlinux, a bit later from kernel
modules as well. This requires pahole v1.16 or later.
It is found in the 'dwarves' or 'pahole' distro packages or from
https://fedorapeople.org/~acme/dwarves/.
Perl
----
You will need perl 5 and the following modules: ``Getopt::Long``,
``Getopt::Std``, ``File::Basename``, and ``File::Find`` to build the kernel.
BC
--
You will need bc to build kernels 3.10 and higher
OpenSSL
-------
Module signing and external certificate handling use the OpenSSL program and
crypto library to do key creation and signature generation.
You will need openssl to build kernels 3.7 and higher if module signing is
enabled. You will also need openssl development packages to build kernels 4.3
and higher.
System utilities
****************
Architectural changes
---------------------
DevFS has been obsoleted in favour of udev
(https://www.kernel.org/pub/linux/utils/kernel/hotplug/)
32-bit UID support is now in place. Have fun!
Linux documentation for functions is transitioning to inline
documentation via specially-formatted comments near their
definitions in the source. These comments can be combined with ReST
files the Documentation/ directory to make enriched documentation, which can
then be converted to PostScript, HTML, LaTex, ePUB and PDF files.
In order to convert from ReST format to a format of your choice, you'll need
Sphinx.
Util-linux
----------
New versions of util-linux provide ``fdisk`` support for larger disks,
support new options to mount, recognize more supported partition
types, have a fdformat which works with 2.4 kernels, and similar goodies.
You'll probably want to upgrade.
Ksymoops
--------
If the unthinkable happens and your kernel oopses, you may need the
ksymoops tool to decode it, but in most cases you don't.
It is generally preferred to build the kernel with ``CONFIG_KALLSYMS`` so
that it produces readable dumps that can be used as-is (this also
produces better output than ksymoops). If for some reason your kernel
is not build with ``CONFIG_KALLSYMS`` and you have no way to rebuild and
reproduce the Oops with that option, then you can still decode that Oops
with ksymoops.
Mkinitrd
--------
These changes to the ``/lib/modules`` file tree layout also require that
mkinitrd be upgraded.
E2fsprogs
---------
The latest version of ``e2fsprogs`` fixes several bugs in fsck and
debugfs. Obviously, it's a good idea to upgrade.
JFSutils
--------
The ``jfsutils`` package contains the utilities for the file system.
The following utilities are available:
- ``fsck.jfs`` - initiate replay of the transaction log, and check
and repair a JFS formatted partition.
- ``mkfs.jfs`` - create a JFS formatted partition.
- other file system utilities are also available in this package.
Reiserfsprogs
-------------
The reiserfsprogs package should be used for reiserfs-3.6.x
(Linux kernels 2.4.x). It is a combined package and contains working
versions of ``mkreiserfs``, ``resize_reiserfs``, ``debugreiserfs`` and
``reiserfsck``. These utils work on both i386 and alpha platforms.
Xfsprogs
--------
The latest version of ``xfsprogs`` contains ``mkfs.xfs``, ``xfs_db``, and the
``xfs_repair`` utilities, among others, for the XFS filesystem. It is
architecture independent and any version from 2.0.0 onward should
work correctly with this version of the XFS kernel code (2.6.0 or
later is recommended, due to some significant improvements).
PCMCIAutils
-----------
PCMCIAutils replaces ``pcmcia-cs``. It properly sets up
PCMCIA sockets at system startup and loads the appropriate modules
for 16-bit PCMCIA devices if the kernel is modularized and the hotplug
subsystem is used.
Quota-tools
-----------
Support for 32 bit uid's and gid's is required if you want to use
the newer version 2 quota format. Quota-tools version 3.07 and
newer has this support. Use the recommended version or newer
from the table above.
Intel IA32 microcode
--------------------
A driver has been added to allow updating of Intel IA32 microcode,
accessible as a normal (misc) character device. If you are not using
udev you may need to::
mkdir /dev/cpu
mknod /dev/cpu/microcode c 10 184
chmod 0644 /dev/cpu/microcode
as root before you can use this. You'll probably also want to
get the user-space microcode_ctl utility to use with this.
udev
----
``udev`` is a userspace application for populating ``/dev`` dynamically with
only entries for devices actually present. ``udev`` replaces the basic
functionality of devfs, while allowing persistent device naming for
devices.
FUSE
----
Needs libfuse 2.4.0 or later. Absolute minimum is 2.3.0 but mount
options ``direct_io`` and ``kernel_cache`` won't work.
Networking
**********
General changes
---------------
If you have advanced network configuration needs, you should probably
consider using the network tools from ip-route2.
Packet Filter / NAT
-------------------
The packet filtering and NAT code uses the same tools like the previous 2.4.x
kernel series (iptables). It still includes backwards-compatibility modules
for 2.2.x-style ipchains and 2.0.x-style ipfwadm.
PPP
---
The PPP driver has been restructured to support multilink and to
enable it to operate over diverse media layers. If you use PPP,
upgrade pppd to at least 2.4.0.
If you are not using udev, you must have the device file /dev/ppp
which can be made by::
mknod /dev/ppp c 108 0
as root.
NFS-utils
---------
In ancient (2.4 and earlier) kernels, the nfs server needed to know
about any client that expected to be able to access files via NFS. This
information would be given to the kernel by ``mountd`` when the client
mounted the filesystem, or by ``exportfs`` at system startup. exportfs
would take information about active clients from ``/var/lib/nfs/rmtab``.
This approach is quite fragile as it depends on rmtab being correct
which is not always easy, particularly when trying to implement
fail-over. Even when the system is working well, ``rmtab`` suffers from
getting lots of old entries that never get removed.
With modern kernels we have the option of having the kernel tell mountd
when it gets a request from an unknown host, and mountd can give
appropriate export information to the kernel. This removes the
dependency on ``rmtab`` and means that the kernel only needs to know about
currently active clients.
To enable this new functionality, you need to::
mount -t nfsd nfsd /proc/fs/nfsd
before running exportfs or mountd. It is recommended that all NFS
services be protected from the internet-at-large by a firewall where
that is possible.
mcelog
------
On x86 kernels the mcelog utility is needed to process and log machine check
events when ``CONFIG_X86_MCE`` is enabled. Machine check events are errors
reported by the CPU. Processing them is strongly encouraged.
Kernel documentation
********************
Sphinx
------
Please see :ref:`sphinx_install` in :ref:`Documentation/doc-guide/sphinx.rst <sphinxdoc>`
for details about Sphinx requirements.
Getting updated software
========================
Kernel compilation
******************
gcc
---
- <ftp://ftp.gnu.org/gnu/gcc/>
Clang/LLVM
----------
- :ref:`Getting LLVM <getting_llvm>`.
Make
----
- <ftp://ftp.gnu.org/gnu/make/>
Binutils
--------
- <https://www.kernel.org/pub/linux/devel/binutils/>
kbuild: prepare to remove C files pre-generated by flex and bison In Linux build system convention, pre-generated files are version- controlled with a "_shipped" suffix. During the kernel building, they are simply shipped (copied) removing the suffix. This approach can reduce external tool dependency for the kernel build, but it is tedious to manually regenerate such artifacts from developers' point of view. (We need to do "make REGENERATE_PARSERS=1" every time we touch real source files such as *.l, *.y) Some months ago, I sent out RFC patches to run flex, bison, and gperf during the build. In the review and test, Linus noticed gperf-3.1 had changed the lookup function prototype. Then, the use of gperf in kernel was entirely removed by commit bb3290d91695 ("Remove gperf usage from toolchain"). This time, I tested several versions of flex and bison, and I was not hit by any compatibility issue except a flaw in flex-2.6.3; if you generate lexer for dtc and genksyms with flex-2.6.3, you will see "yywrap redefined" warning. This was not intentional, but a bug, fixed by flex-2.6.4. Otherwise, both flex and bison look fairly stable for a long time. This commit prepares some build rules to remove the _shipped files. Also, document minimal requirement for flex and bison. Rationale for the minimal version: The -Wmissing-prototypes option of GCC warns "no previous prototype" for lexers generated by flex-2.5.34 or older, so I chose 2.5.35 as the required version for flex. Flex-2.5.35 was released in 2008. Bison looks more stable. I did not see any problem with bison-2.0, released in 2004. I did not test bison-1.x, but bison-2.0 should be old enough. Tested flex versions: 2.5.35 2.5.36 2.5.37 2.5.39 2.6.0 2.6.1 2.6.2 2.6.3 (*) 2.6.4 (*) flex-2.6.3 causes "yywrap redefined" warning Tested bison versions: 2.0 2.1 2.2 2.3 2.4 2.4.1 2.5.1 2.6 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7 2.7.1 3.0 3.0.1 3.0.2 3.0.3 3.0.4 Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-12-10 00:02:29 +08:00
Flex
----
- <https://github.com/westes/flex/releases>
Bison
-----
- <ftp://ftp.gnu.org/gnu/bison/>
OpenSSL
-------
- <https://www.openssl.org/>
System utilities
****************
Util-linux
----------
- <https://www.kernel.org/pub/linux/utils/util-linux/>
Kmod
----
- <https://www.kernel.org/pub/linux/utils/kernel/kmod/>
- <https://git.kernel.org/pub/scm/utils/kernel/kmod/kmod.git>
Ksymoops
--------
- <https://www.kernel.org/pub/linux/utils/kernel/ksymoops/v2.4/>
Mkinitrd
--------
- <https://code.launchpad.net/initrd-tools/main>
E2fsprogs
---------
- <https://www.kernel.org/pub/linux/kernel/people/tytso/e2fsprogs/>
- <https://git.kernel.org/pub/scm/fs/ext2/e2fsprogs.git/>
JFSutils
--------
- <http://jfs.sourceforge.net/>
Reiserfsprogs
-------------
- <https://git.kernel.org/pub/scm/linux/kernel/git/jeffm/reiserfsprogs.git/>
Xfsprogs
--------
- <https://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git>
- <https://www.kernel.org/pub/linux/utils/fs/xfs/xfsprogs/>
Pcmciautils
-----------
- <https://www.kernel.org/pub/linux/utils/kernel/pcmcia/>
Quota-tools
-----------
- <http://sourceforge.net/projects/linuxquota/>
Intel P6 microcode
------------------
- <https://downloadcenter.intel.com/>
udev
----
- <https://www.freedesktop.org/software/systemd/man/udev.html>
FUSE
----
- <https://github.com/libfuse/libfuse/releases>
mcelog
------
- <http://www.mcelog.org/>
cpio
----
- <https://www.gnu.org/software/cpio/>
Networking
**********
PPP
---
- <https://download.samba.org/pub/ppp/>
- <https://git.ozlabs.org/?p=ppp.git>
- <https://github.com/paulusmack/ppp/>
NFS-utils
---------
- <http://sourceforge.net/project/showfiles.php?group_id=14>
Iptables
--------
- <https://netfilter.org/projects/iptables/index.html>
Ip-route2
---------
- <https://www.kernel.org/pub/linux/utils/net/iproute2/>
OProfile
--------
- <http://oprofile.sf.net/download/>
NFS-Utils
---------
- <http://nfs.sourceforge.net/>
Kernel documentation
********************
Sphinx
------
- <https://www.sphinx-doc.org/>