2012-04-20 21:45:54 +08:00
|
|
|
/*
|
|
|
|
|
* ld script to make ARM Linux kernel
|
|
|
|
|
* taken from the i386 version by Russell King
|
|
|
|
|
* Written by Martin Mares <mj@atrey.karlin.mff.cuni.cz>
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
#include <asm-generic/vmlinux.lds.h>
|
2015-12-01 20:20:40 +08:00
|
|
|
#include <asm/cache.h>
|
2015-10-19 21:19:27 +08:00
|
|
|
#include <asm/kernel-pgtable.h>
|
2012-04-20 21:45:54 +08:00
|
|
|
#include <asm/thread_info.h>
|
|
|
|
|
#include <asm/memory.h>
|
|
|
|
|
#include <asm/page.h>
|
2015-01-22 09:36:06 +08:00
|
|
|
#include <asm/pgtable.h>
|
2012-04-20 21:45:54 +08:00
|
|
|
|
arm64: Update the Image header
Currently the kernel Image is stripped of everything past the initial
stack, and at runtime the memory is initialised and used by the kernel.
This makes the effective minimum memory footprint of the kernel larger
than the size of the loaded binary, though bootloaders have no mechanism
to identify how large this minimum memory footprint is. This makes it
difficult to choose safe locations to place both the kernel and other
binaries required at boot (DTB, initrd, etc), such that the kernel won't
clobber said binaries or other reserved memory during initialisation.
Additionally when big endian support was added the image load offset was
overlooked, and is currently of an arbitrary endianness, which makes it
difficult for bootloaders to make use of it. It seems that bootloaders
aren't respecting the image load offset at present anyway, and are
assuming that offset 0x80000 will always be correct.
This patch adds an effective image size to the kernel header which
describes the amount of memory from the start of the kernel Image binary
which the kernel expects to use before detecting memory and handling any
memory reservations. This can be used by bootloaders to choose suitable
locations to load the kernel and/or other binaries such that the kernel
will not clobber any memory unexpectedly. As before, memory reservations
are required to prevent the kernel from clobbering these locations
later.
Both the image load offset and the effective image size are forced to be
little-endian regardless of the native endianness of the kernel to
enable bootloaders to load a kernel of arbitrary endianness. Bootloaders
which wish to make use of the load offset can inspect the effective
image size field for a non-zero value to determine if the offset is of a
known endianness. To enable software to determine the endinanness of the
kernel as may be required for certain use-cases, a new flags field (also
little-endian) is added to the kernel header to export this information.
The documentation is updated to clarify these details. To discourage
future assumptions regarding the value of text_offset, the value at this
point in time is removed from the main flow of the documentation (though
kept as a compatibility note). Some minor formatting issues in the
documentation are also corrected.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Acked-by: Tom Rini <trini@ti.com>
Cc: Geoff Levand <geoff@infradead.org>
Cc: Kevin Hilman <kevin.hilman@linaro.org>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2014-06-24 23:51:36 +08:00
|
|
|
#include "image.h"
|
|
|
|
|
|
2014-11-25 23:26:13 +08:00
|
|
|
/* .exit.text needed in case of alternative patching */
|
|
|
|
|
#define ARM_EXIT_KEEP(x) x
|
|
|
|
|
#define ARM_EXIT_DISCARD(x)
|
2012-04-20 21:45:54 +08:00
|
|
|
|
|
|
|
|
OUTPUT_ARCH(aarch64)
|
2014-05-17 01:26:01 +08:00
|
|
|
ENTRY(_text)
|
2012-04-20 21:45:54 +08:00
|
|
|
|
|
|
|
|
jiffies = jiffies_64;
|
|
|
|
|
|
2012-12-08 02:40:43 +08:00
|
|
|
#define HYPERVISOR_TEXT \
|
|
|
|
|
/* \
|
ARM, arm64: kvm: get rid of the bounce page
The HYP init bounce page is a runtime construct that ensures that the
HYP init code does not cross a page boundary. However, this is something
we can do perfectly well at build time, by aligning the code appropriately.
For arm64, we just align to 4 KB, and enforce that the code size is less
than 4 KB, regardless of the chosen page size.
For ARM, the whole code is less than 256 bytes, so we tweak the linker
script to align at a power of 2 upper bound of the code size
Note that this also fixes a benign off-by-one error in the original bounce
page code, where a bounce page would be allocated unnecessarily if the code
was exactly 1 page in size.
On ARM, it also fixes an issue with very large kernels reported by Arnd
Bergmann, where stub sections with linker emitted veneers could erroneously
trigger the size/alignment ASSERT() in the linker script.
Tested-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-03-20 00:42:26 +08:00
|
|
|
* Align to 4 KB so that \
|
|
|
|
|
* a) the HYP vector table is at its minimum \
|
|
|
|
|
* alignment of 2048 bytes \
|
|
|
|
|
* b) the HYP init code will not cross a page \
|
|
|
|
|
* boundary if its size does not exceed \
|
|
|
|
|
* 4 KB (see related ASSERT() below) \
|
2012-12-08 02:40:43 +08:00
|
|
|
*/ \
|
ARM, arm64: kvm: get rid of the bounce page
The HYP init bounce page is a runtime construct that ensures that the
HYP init code does not cross a page boundary. However, this is something
we can do perfectly well at build time, by aligning the code appropriately.
For arm64, we just align to 4 KB, and enforce that the code size is less
than 4 KB, regardless of the chosen page size.
For ARM, the whole code is less than 256 bytes, so we tweak the linker
script to align at a power of 2 upper bound of the code size
Note that this also fixes a benign off-by-one error in the original bounce
page code, where a bounce page would be allocated unnecessarily if the code
was exactly 1 page in size.
On ARM, it also fixes an issue with very large kernels reported by Arnd
Bergmann, where stub sections with linker emitted veneers could erroneously
trigger the size/alignment ASSERT() in the linker script.
Tested-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-03-20 00:42:26 +08:00
|
|
|
. = ALIGN(SZ_4K); \
|
2012-12-08 02:40:43 +08:00
|
|
|
VMLINUX_SYMBOL(__hyp_idmap_text_start) = .; \
|
|
|
|
|
*(.hyp.idmap.text) \
|
|
|
|
|
VMLINUX_SYMBOL(__hyp_idmap_text_end) = .; \
|
|
|
|
|
VMLINUX_SYMBOL(__hyp_text_start) = .; \
|
|
|
|
|
*(.hyp.text) \
|
|
|
|
|
VMLINUX_SYMBOL(__hyp_text_end) = .;
|
|
|
|
|
|
2015-06-01 19:40:33 +08:00
|
|
|
#define IDMAP_TEXT \
|
|
|
|
|
. = ALIGN(SZ_4K); \
|
|
|
|
|
VMLINUX_SYMBOL(__idmap_text_start) = .; \
|
|
|
|
|
*(.idmap.text) \
|
|
|
|
|
VMLINUX_SYMBOL(__idmap_text_end) = .;
|
|
|
|
|
|
2016-04-28 00:47:12 +08:00
|
|
|
#ifdef CONFIG_HIBERNATION
|
|
|
|
|
#define HIBERNATE_TEXT \
|
|
|
|
|
. = ALIGN(SZ_4K); \
|
|
|
|
|
VMLINUX_SYMBOL(__hibernate_exit_text_start) = .;\
|
|
|
|
|
*(.hibernate_exit.text) \
|
|
|
|
|
VMLINUX_SYMBOL(__hibernate_exit_text_end) = .;
|
|
|
|
|
#else
|
|
|
|
|
#define HIBERNATE_TEXT
|
|
|
|
|
#endif
|
|
|
|
|
|
2014-10-11 00:42:55 +08:00
|
|
|
/*
|
|
|
|
|
* The size of the PE/COFF section that covers the kernel image, which
|
|
|
|
|
* runs from stext to _edata, must be a round multiple of the PE/COFF
|
|
|
|
|
* FileAlignment, which we set to its minimum value of 0x200. 'stext'
|
|
|
|
|
* itself is 4 KB aligned, so padding out _edata to a 0x200 aligned
|
|
|
|
|
* boundary should be sufficient.
|
|
|
|
|
*/
|
|
|
|
|
PECOFF_FILE_ALIGNMENT = 0x200;
|
|
|
|
|
|
|
|
|
|
#ifdef CONFIG_EFI
|
|
|
|
|
#define PECOFF_EDATA_PADDING \
|
|
|
|
|
.pecoff_edata_padding : { BYTE(0); . = ALIGN(PECOFF_FILE_ALIGNMENT); }
|
|
|
|
|
#else
|
|
|
|
|
#define PECOFF_EDATA_PADDING
|
|
|
|
|
#endif
|
|
|
|
|
|
2015-10-27 05:42:33 +08:00
|
|
|
#if defined(CONFIG_DEBUG_ALIGN_RODATA)
|
arm64: simplify kernel segment mapping granularity
The mapping of the kernel consist of four segments, each of which is mapped
with different permission attributes and/or lifetimes. To optimize the TLB
and translation table footprint, we define various opaque constants in the
linker script that resolve to different aligment values depending on the
page size and whether CONFIG_DEBUG_ALIGN_RODATA is set.
Considering that
- a 4 KB granule kernel benefits from a 64 KB segment alignment (due to
the fact that it allows the use of the contiguous bit),
- the minimum alignment of the .data segment is THREAD_SIZE already, not
PAGE_SIZE (i.e., we already have padding between _data and the start of
the .data payload in many cases),
- 2 MB is a suitable alignment value on all granule sizes, either for
mapping directly (level 2 on 4 KB), or via the contiguous bit (level 3 on
16 KB and 64 KB),
- anything beyond 2 MB exceeds the minimum alignment mandated by the boot
protocol, and can only be mapped efficiently if the physical alignment
happens to be the same,
we can simplify this by standardizing on 64 KB (or 2 MB) explicitly, i.e.,
regardless of granule size, all segments are aligned either to 64 KB, or to
2 MB if CONFIG_DEBUG_ALIGN_RODATA=y. This also means we can drop the Kconfig
dependency of CONFIG_DEBUG_ALIGN_RODATA on CONFIG_ARM64_4K_PAGES.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-03-30 23:43:09 +08:00
|
|
|
/*
|
|
|
|
|
* 4 KB granule: 1 level 2 entry
|
|
|
|
|
* 16 KB granule: 128 level 3 entries, with contiguous bit
|
|
|
|
|
* 64 KB granule: 32 level 3 entries, with contiguous bit
|
|
|
|
|
*/
|
|
|
|
|
#define SEGMENT_ALIGN SZ_2M
|
2015-01-22 09:36:06 +08:00
|
|
|
#else
|
arm64: simplify kernel segment mapping granularity
The mapping of the kernel consist of four segments, each of which is mapped
with different permission attributes and/or lifetimes. To optimize the TLB
and translation table footprint, we define various opaque constants in the
linker script that resolve to different aligment values depending on the
page size and whether CONFIG_DEBUG_ALIGN_RODATA is set.
Considering that
- a 4 KB granule kernel benefits from a 64 KB segment alignment (due to
the fact that it allows the use of the contiguous bit),
- the minimum alignment of the .data segment is THREAD_SIZE already, not
PAGE_SIZE (i.e., we already have padding between _data and the start of
the .data payload in many cases),
- 2 MB is a suitable alignment value on all granule sizes, either for
mapping directly (level 2 on 4 KB), or via the contiguous bit (level 3 on
16 KB and 64 KB),
- anything beyond 2 MB exceeds the minimum alignment mandated by the boot
protocol, and can only be mapped efficiently if the physical alignment
happens to be the same,
we can simplify this by standardizing on 64 KB (or 2 MB) explicitly, i.e.,
regardless of granule size, all segments are aligned either to 64 KB, or to
2 MB if CONFIG_DEBUG_ALIGN_RODATA=y. This also means we can drop the Kconfig
dependency of CONFIG_DEBUG_ALIGN_RODATA on CONFIG_ARM64_4K_PAGES.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-03-30 23:43:09 +08:00
|
|
|
/*
|
|
|
|
|
* 4 KB granule: 16 level 3 entries, with contiguous bit
|
|
|
|
|
* 16 KB granule: 4 level 3 entries, without contiguous bit
|
|
|
|
|
* 64 KB granule: 1 level 3 entry
|
|
|
|
|
*/
|
|
|
|
|
#define SEGMENT_ALIGN SZ_64K
|
2015-01-22 09:36:06 +08:00
|
|
|
#endif
|
|
|
|
|
|
2012-04-20 21:45:54 +08:00
|
|
|
SECTIONS
|
|
|
|
|
{
|
|
|
|
|
/*
|
|
|
|
|
* XXX: The linker does not define how output sections are
|
|
|
|
|
* assigned to input sections when there are multiple statements
|
|
|
|
|
* matching the same input section name. There is no documented
|
|
|
|
|
* order of matching.
|
|
|
|
|
*/
|
|
|
|
|
/DISCARD/ : {
|
|
|
|
|
ARM_EXIT_DISCARD(EXIT_TEXT)
|
|
|
|
|
ARM_EXIT_DISCARD(EXIT_DATA)
|
|
|
|
|
EXIT_CALL
|
|
|
|
|
*(.discard)
|
|
|
|
|
*(.discard.*)
|
2016-01-26 16:13:44 +08:00
|
|
|
*(.interp .dynamic)
|
arm64: relocatable: suppress R_AARCH64_ABS64 relocations in vmlinux
The linker routines that we rely on to produce a relocatable PIE binary
treat it as a shared ELF object in some ways, i.e., it emits symbol based
R_AARCH64_ABS64 relocations into the final binary since doing so would be
appropriate when linking a shared library that is subject to symbol
preemption. (This means that an executable can override certain symbols
that are exported by a shared library it is linked with, and that the
shared library *must* update all its internal references as well, and point
them to the version provided by the executable.)
Symbol preemption does not occur for OS hosted PIE executables, let alone
for vmlinux, and so we would prefer to get rid of these symbol based
relocations. This would allow us to simplify the relocation routines, and
to strip the .dynsym, .dynstr and .hash sections from the binary. (Note
that these are tiny, and are placed in the .init segment, but they clutter
up the vmlinux binary.)
Note that these R_AARCH64_ABS64 relocations are only emitted for absolute
references to symbols defined in the linker script, all other relocatable
quantities are covered by anonymous R_AARCH64_RELATIVE relocations that
simply list the offsets to all 64-bit values in the binary that need to be
fixed up based on the offset between the link time and run time addresses.
Fortunately, GNU ld has a -Bsymbolic option, which is intended for shared
libraries to allow them to ignore symbol preemption, and unconditionally
bind all internal symbol references to its own definitions. So set it for
our PIE binary as well, and get rid of the asoociated sections and the
relocation code that processes them.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
[will: fixed conflict with __dynsym_offset linker script entry]
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-07-24 20:00:13 +08:00
|
|
|
*(.dynsym .dynstr .hash)
|
2012-04-20 21:45:54 +08:00
|
|
|
}
|
|
|
|
|
|
2016-02-16 20:52:36 +08:00
|
|
|
. = KIMAGE_VADDR + TEXT_OFFSET;
|
2012-04-20 21:45:54 +08:00
|
|
|
|
|
|
|
|
.head.text : {
|
|
|
|
|
_text = .;
|
|
|
|
|
HEAD_TEXT
|
|
|
|
|
}
|
|
|
|
|
.text : { /* Real text segment */
|
|
|
|
|
_stext = .; /* Text and read-only data */
|
|
|
|
|
__exception_text_start = .;
|
|
|
|
|
*(.exception.text)
|
|
|
|
|
__exception_text_end = .;
|
|
|
|
|
IRQENTRY_TEXT
|
2016-03-26 05:22:05 +08:00
|
|
|
SOFTIRQENTRY_TEXT
|
2016-07-09 00:35:50 +08:00
|
|
|
ENTRY_TEXT
|
2012-04-20 21:45:54 +08:00
|
|
|
TEXT_TEXT
|
|
|
|
|
SCHED_TEXT
|
2016-10-08 08:02:55 +08:00
|
|
|
CPUIDLE_TEXT
|
2012-04-20 21:45:54 +08:00
|
|
|
LOCK_TEXT
|
arm64: Kprobes with single stepping support
Add support for basic kernel probes(kprobes) and jump probes
(jprobes) for ARM64.
Kprobes utilizes software breakpoint and single step debug
exceptions supported on ARM v8.
A software breakpoint is placed at the probe address to trap the
kernel execution into the kprobe handler.
ARM v8 supports enabling single stepping before the break exception
return (ERET), with next PC in exception return address (ELR_EL1). The
kprobe handler prepares an executable memory slot for out-of-line
execution with a copy of the original instruction being probed, and
enables single stepping. The PC is set to the out-of-line slot address
before the ERET. With this scheme, the instruction is executed with the
exact same register context except for the PC (and DAIF) registers.
Debug mask (PSTATE.D) is enabled only when single stepping a recursive
kprobe, e.g.: during kprobes reenter so that probed instruction can be
single stepped within the kprobe handler -exception- context.
The recursion depth of kprobe is always 2, i.e. upon probe re-entry,
any further re-entry is prevented by not calling handlers and the case
counted as a missed kprobe).
Single stepping from the x-o-l slot has a drawback for PC-relative accesses
like branching and symbolic literals access as the offset from the new PC
(slot address) may not be ensured to fit in the immediate value of
the opcode. Such instructions need simulation, so reject
probing them.
Instructions generating exceptions or cpu mode change are rejected
for probing.
Exclusive load/store instructions are rejected too. Additionally, the
code is checked to see if it is inside an exclusive load/store sequence
(code from Pratyush).
System instructions are mostly enabled for stepping, except MSR/MRS
accesses to "DAIF" flags in PSTATE, which are not safe for
probing.
This also changes arch/arm64/include/asm/ptrace.h to use
include/asm-generic/ptrace.h.
Thanks to Steve Capper and Pratyush Anand for several suggested
Changes.
Signed-off-by: Sandeepa Prabhu <sandeepa.s.prabhu@gmail.com>
Signed-off-by: David A. Long <dave.long@linaro.org>
Signed-off-by: Pratyush Anand <panand@redhat.com>
Acked-by: Masami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2016-07-09 00:35:48 +08:00
|
|
|
KPROBES_TEXT
|
2012-12-08 02:40:43 +08:00
|
|
|
HYPERVISOR_TEXT
|
2015-06-01 19:40:33 +08:00
|
|
|
IDMAP_TEXT
|
2016-04-28 00:47:12 +08:00
|
|
|
HIBERNATE_TEXT
|
2012-04-20 21:45:54 +08:00
|
|
|
*(.fixup)
|
|
|
|
|
*(.gnu.warning)
|
|
|
|
|
. = ALIGN(16);
|
|
|
|
|
*(.got) /* Global offset table */
|
|
|
|
|
}
|
|
|
|
|
|
arm64: simplify kernel segment mapping granularity
The mapping of the kernel consist of four segments, each of which is mapped
with different permission attributes and/or lifetimes. To optimize the TLB
and translation table footprint, we define various opaque constants in the
linker script that resolve to different aligment values depending on the
page size and whether CONFIG_DEBUG_ALIGN_RODATA is set.
Considering that
- a 4 KB granule kernel benefits from a 64 KB segment alignment (due to
the fact that it allows the use of the contiguous bit),
- the minimum alignment of the .data segment is THREAD_SIZE already, not
PAGE_SIZE (i.e., we already have padding between _data and the start of
the .data payload in many cases),
- 2 MB is a suitable alignment value on all granule sizes, either for
mapping directly (level 2 on 4 KB), or via the contiguous bit (level 3 on
16 KB and 64 KB),
- anything beyond 2 MB exceeds the minimum alignment mandated by the boot
protocol, and can only be mapped efficiently if the physical alignment
happens to be the same,
we can simplify this by standardizing on 64 KB (or 2 MB) explicitly, i.e.,
regardless of granule size, all segments are aligned either to 64 KB, or to
2 MB if CONFIG_DEBUG_ALIGN_RODATA=y. This also means we can drop the Kconfig
dependency of CONFIG_DEBUG_ALIGN_RODATA on CONFIG_ARM64_4K_PAGES.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-03-30 23:43:09 +08:00
|
|
|
. = ALIGN(SEGMENT_ALIGN);
|
arm64: mm: fix location of _etext
As Kees Cook notes in the ARM counterpart of this patch [0]:
The _etext position is defined to be the end of the kernel text code,
and should not include any part of the data segments. This interferes
with things that might check memory ranges and expect executable code
up to _etext.
In particular, Kees is referring to the HARDENED_USERCOPY patch set [1],
which rejects attempts to call copy_to_user() on kernel ranges containing
executable code, but does allow access to the .rodata segment. Regardless
of whether one may or may not agree with the distinction, it makes sense
for _etext to have the same meaning across architectures.
So let's put _etext where it belongs, between .text and .rodata, and fix
up existing references to use __init_begin instead, which unlike _end_rodata
includes the exception and notes sections as well.
The _etext references in kaslr.c are left untouched, since its references
to [_stext, _etext) are meant to capture potential jump instruction targets,
and so disregarding .rodata is actually an improvement here.
[0] http://article.gmane.org/gmane.linux.kernel/2245084
[1] http://thread.gmane.org/gmane.linux.kernel.hardened.devel/2502
Reported-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2016-06-23 21:53:17 +08:00
|
|
|
_etext = .; /* End of text section */
|
|
|
|
|
|
|
|
|
|
RO_DATA(PAGE_SIZE) /* everything from this point to */
|
|
|
|
|
EXCEPTION_TABLE(8) /* __init_begin will be marked RO NX */
|
2013-08-23 23:16:42 +08:00
|
|
|
NOTES
|
2012-04-20 21:45:54 +08:00
|
|
|
|
arm64: simplify kernel segment mapping granularity
The mapping of the kernel consist of four segments, each of which is mapped
with different permission attributes and/or lifetimes. To optimize the TLB
and translation table footprint, we define various opaque constants in the
linker script that resolve to different aligment values depending on the
page size and whether CONFIG_DEBUG_ALIGN_RODATA is set.
Considering that
- a 4 KB granule kernel benefits from a 64 KB segment alignment (due to
the fact that it allows the use of the contiguous bit),
- the minimum alignment of the .data segment is THREAD_SIZE already, not
PAGE_SIZE (i.e., we already have padding between _data and the start of
the .data payload in many cases),
- 2 MB is a suitable alignment value on all granule sizes, either for
mapping directly (level 2 on 4 KB), or via the contiguous bit (level 3 on
16 KB and 64 KB),
- anything beyond 2 MB exceeds the minimum alignment mandated by the boot
protocol, and can only be mapped efficiently if the physical alignment
happens to be the same,
we can simplify this by standardizing on 64 KB (or 2 MB) explicitly, i.e.,
regardless of granule size, all segments are aligned either to 64 KB, or to
2 MB if CONFIG_DEBUG_ALIGN_RODATA=y. This also means we can drop the Kconfig
dependency of CONFIG_DEBUG_ALIGN_RODATA on CONFIG_ARM64_4K_PAGES.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-03-30 23:43:09 +08:00
|
|
|
. = ALIGN(SEGMENT_ALIGN);
|
2012-04-20 21:45:54 +08:00
|
|
|
__init_begin = .;
|
|
|
|
|
|
|
|
|
|
INIT_TEXT_SECTION(8)
|
|
|
|
|
.exit.text : {
|
|
|
|
|
ARM_EXIT_KEEP(EXIT_TEXT)
|
|
|
|
|
}
|
2015-01-22 09:36:06 +08:00
|
|
|
|
2012-04-20 21:45:54 +08:00
|
|
|
.init.data : {
|
|
|
|
|
INIT_DATA
|
|
|
|
|
INIT_SETUP(16)
|
|
|
|
|
INIT_CALLS
|
|
|
|
|
CON_INITCALL
|
|
|
|
|
SECURITY_INITCALL
|
|
|
|
|
INIT_RAM_FS
|
2016-02-17 20:35:58 +08:00
|
|
|
*(.init.rodata.* .init.bss) /* from the EFI stub */
|
2012-04-20 21:45:54 +08:00
|
|
|
}
|
|
|
|
|
.exit.data : {
|
|
|
|
|
ARM_EXIT_KEEP(EXIT_DATA)
|
|
|
|
|
}
|
|
|
|
|
|
2015-12-01 20:20:40 +08:00
|
|
|
PERCPU_SECTION(L1_CACHE_BYTES)
|
2012-04-20 21:45:54 +08:00
|
|
|
|
2014-11-14 23:54:08 +08:00
|
|
|
. = ALIGN(4);
|
|
|
|
|
.altinstructions : {
|
|
|
|
|
__alt_instructions = .;
|
|
|
|
|
*(.altinstructions)
|
|
|
|
|
__alt_instructions_end = .;
|
|
|
|
|
}
|
|
|
|
|
.altinstr_replacement : {
|
|
|
|
|
*(.altinstr_replacement)
|
|
|
|
|
}
|
2016-01-26 16:13:44 +08:00
|
|
|
.rela : ALIGN(8) {
|
|
|
|
|
*(.rela .rela*)
|
|
|
|
|
}
|
2014-11-14 23:54:08 +08:00
|
|
|
|
2016-07-28 22:15:14 +08:00
|
|
|
__rela_offset = ABSOLUTE(ADDR(.rela) - KIMAGE_VADDR);
|
2016-04-18 23:09:43 +08:00
|
|
|
__rela_size = SIZEOF(.rela);
|
|
|
|
|
|
arm64: simplify kernel segment mapping granularity
The mapping of the kernel consist of four segments, each of which is mapped
with different permission attributes and/or lifetimes. To optimize the TLB
and translation table footprint, we define various opaque constants in the
linker script that resolve to different aligment values depending on the
page size and whether CONFIG_DEBUG_ALIGN_RODATA is set.
Considering that
- a 4 KB granule kernel benefits from a 64 KB segment alignment (due to
the fact that it allows the use of the contiguous bit),
- the minimum alignment of the .data segment is THREAD_SIZE already, not
PAGE_SIZE (i.e., we already have padding between _data and the start of
the .data payload in many cases),
- 2 MB is a suitable alignment value on all granule sizes, either for
mapping directly (level 2 on 4 KB), or via the contiguous bit (level 3 on
16 KB and 64 KB),
- anything beyond 2 MB exceeds the minimum alignment mandated by the boot
protocol, and can only be mapped efficiently if the physical alignment
happens to be the same,
we can simplify this by standardizing on 64 KB (or 2 MB) explicitly, i.e.,
regardless of granule size, all segments are aligned either to 64 KB, or to
2 MB if CONFIG_DEBUG_ALIGN_RODATA=y. This also means we can drop the Kconfig
dependency of CONFIG_DEBUG_ALIGN_RODATA on CONFIG_ARM64_4K_PAGES.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-03-30 23:43:09 +08:00
|
|
|
. = ALIGN(SEGMENT_ALIGN);
|
2015-12-09 20:44:38 +08:00
|
|
|
__init_end = .;
|
|
|
|
|
|
2013-11-05 00:38:47 +08:00
|
|
|
_data = .;
|
|
|
|
|
_sdata = .;
|
2015-12-01 20:20:40 +08:00
|
|
|
RW_DATA_SECTION(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE)
|
2016-08-25 01:27:29 +08:00
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Data written with the MMU off but read with the MMU on requires
|
|
|
|
|
* cache lines to be invalidated, discarding up to a Cache Writeback
|
|
|
|
|
* Granule (CWG) of data from the cache. Keep the section that
|
|
|
|
|
* requires this type of maintenance to be in its own Cache Writeback
|
|
|
|
|
* Granule (CWG) area so the cache maintenance operations don't
|
|
|
|
|
* interfere with adjacent data.
|
|
|
|
|
*/
|
|
|
|
|
.mmuoff.data.write : ALIGN(SZ_2K) {
|
|
|
|
|
__mmuoff_data_start = .;
|
|
|
|
|
*(.mmuoff.data.write)
|
|
|
|
|
}
|
|
|
|
|
. = ALIGN(SZ_2K);
|
|
|
|
|
.mmuoff.data.read : {
|
|
|
|
|
*(.mmuoff.data.read)
|
|
|
|
|
__mmuoff_data_end = .;
|
|
|
|
|
}
|
|
|
|
|
|
2014-10-11 00:42:55 +08:00
|
|
|
PECOFF_EDATA_PADDING
|
2013-11-05 00:38:47 +08:00
|
|
|
_edata = .;
|
2012-04-20 21:45:54 +08:00
|
|
|
|
|
|
|
|
BSS_SECTION(0, 0, 0)
|
2014-06-24 23:51:35 +08:00
|
|
|
|
|
|
|
|
. = ALIGN(PAGE_SIZE);
|
|
|
|
|
idmap_pg_dir = .;
|
|
|
|
|
. += IDMAP_DIR_SIZE;
|
|
|
|
|
swapper_pg_dir = .;
|
|
|
|
|
. += SWAPPER_DIR_SIZE;
|
|
|
|
|
|
2016-07-01 23:53:00 +08:00
|
|
|
#ifdef CONFIG_ARM64_SW_TTBR0_PAN
|
|
|
|
|
reserved_ttbr0 = .;
|
|
|
|
|
. += RESERVED_TTBR0_SIZE;
|
|
|
|
|
#endif
|
|
|
|
|
|
2012-04-20 21:45:54 +08:00
|
|
|
_end = .;
|
|
|
|
|
|
|
|
|
|
STABS_DEBUG
|
arm64: Update the Image header
Currently the kernel Image is stripped of everything past the initial
stack, and at runtime the memory is initialised and used by the kernel.
This makes the effective minimum memory footprint of the kernel larger
than the size of the loaded binary, though bootloaders have no mechanism
to identify how large this minimum memory footprint is. This makes it
difficult to choose safe locations to place both the kernel and other
binaries required at boot (DTB, initrd, etc), such that the kernel won't
clobber said binaries or other reserved memory during initialisation.
Additionally when big endian support was added the image load offset was
overlooked, and is currently of an arbitrary endianness, which makes it
difficult for bootloaders to make use of it. It seems that bootloaders
aren't respecting the image load offset at present anyway, and are
assuming that offset 0x80000 will always be correct.
This patch adds an effective image size to the kernel header which
describes the amount of memory from the start of the kernel Image binary
which the kernel expects to use before detecting memory and handling any
memory reservations. This can be used by bootloaders to choose suitable
locations to load the kernel and/or other binaries such that the kernel
will not clobber any memory unexpectedly. As before, memory reservations
are required to prevent the kernel from clobbering these locations
later.
Both the image load offset and the effective image size are forced to be
little-endian regardless of the native endianness of the kernel to
enable bootloaders to load a kernel of arbitrary endianness. Bootloaders
which wish to make use of the load offset can inspect the effective
image size field for a non-zero value to determine if the offset is of a
known endianness. To enable software to determine the endinanness of the
kernel as may be required for certain use-cases, a new flags field (also
little-endian) is added to the kernel header to export this information.
The documentation is updated to clarify these details. To discourage
future assumptions regarding the value of text_offset, the value at this
point in time is removed from the main flow of the documentation (though
kept as a compatibility note). Some minor formatting issues in the
documentation are also corrected.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Acked-by: Tom Rini <trini@ti.com>
Cc: Geoff Levand <geoff@infradead.org>
Cc: Kevin Hilman <kevin.hilman@linaro.org>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2014-06-24 23:51:36 +08:00
|
|
|
|
|
|
|
|
HEAD_SYMBOLS
|
2012-04-20 21:45:54 +08:00
|
|
|
}
|
2012-12-08 02:40:43 +08:00
|
|
|
|
|
|
|
|
/*
|
2015-06-01 19:40:33 +08:00
|
|
|
* The HYP init code and ID map text can't be longer than a page each,
|
ARM, arm64: kvm: get rid of the bounce page
The HYP init bounce page is a runtime construct that ensures that the
HYP init code does not cross a page boundary. However, this is something
we can do perfectly well at build time, by aligning the code appropriately.
For arm64, we just align to 4 KB, and enforce that the code size is less
than 4 KB, regardless of the chosen page size.
For ARM, the whole code is less than 256 bytes, so we tweak the linker
script to align at a power of 2 upper bound of the code size
Note that this also fixes a benign off-by-one error in the original bounce
page code, where a bounce page would be allocated unnecessarily if the code
was exactly 1 page in size.
On ARM, it also fixes an issue with very large kernels reported by Arnd
Bergmann, where stub sections with linker emitted veneers could erroneously
trigger the size/alignment ASSERT() in the linker script.
Tested-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-03-20 00:42:26 +08:00
|
|
|
* and should not cross a page boundary.
|
2012-12-08 02:40:43 +08:00
|
|
|
*/
|
ARM, arm64: kvm: get rid of the bounce page
The HYP init bounce page is a runtime construct that ensures that the
HYP init code does not cross a page boundary. However, this is something
we can do perfectly well at build time, by aligning the code appropriately.
For arm64, we just align to 4 KB, and enforce that the code size is less
than 4 KB, regardless of the chosen page size.
For ARM, the whole code is less than 256 bytes, so we tweak the linker
script to align at a power of 2 upper bound of the code size
Note that this also fixes a benign off-by-one error in the original bounce
page code, where a bounce page would be allocated unnecessarily if the code
was exactly 1 page in size.
On ARM, it also fixes an issue with very large kernels reported by Arnd
Bergmann, where stub sections with linker emitted veneers could erroneously
trigger the size/alignment ASSERT() in the linker script.
Tested-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-03-20 00:42:26 +08:00
|
|
|
ASSERT(__hyp_idmap_text_end - (__hyp_idmap_text_start & ~(SZ_4K - 1)) <= SZ_4K,
|
|
|
|
|
"HYP init code too big or misaligned")
|
2015-06-01 19:40:33 +08:00
|
|
|
ASSERT(__idmap_text_end - (__idmap_text_start & ~(SZ_4K - 1)) <= SZ_4K,
|
|
|
|
|
"ID map text too big or misaligned")
|
2016-04-28 00:47:12 +08:00
|
|
|
#ifdef CONFIG_HIBERNATION
|
|
|
|
|
ASSERT(__hibernate_exit_text_end - (__hibernate_exit_text_start & ~(SZ_4K - 1))
|
|
|
|
|
<= SZ_4K, "Hibernate exit text too big or misaligned")
|
|
|
|
|
#endif
|
2014-06-24 23:51:37 +08:00
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* If padding is applied before .head.text, virt<->phys conversions will fail.
|
|
|
|
|
*/
|
2016-02-16 20:52:36 +08:00
|
|
|
ASSERT(_text == (KIMAGE_VADDR + TEXT_OFFSET), "HEAD is misaligned")
|