riscv: Introduce alternative mechanism to apply errata solution
Introduce the "alternative" mechanism from ARM64 and x86 to apply the CPU
vendors' errata solution at runtime. The main purpose of this patch is
to provide a framework. Therefore, the implementation is quite basic for
now so that some scenarios could not use this schemei, such as patching
code to a module, relocating the patching code and heterogeneous CPU
topology.
Users could use the macro ALTERNATIVE to apply an errata to the existing
code flow. In the macro ALTERNATIVE, users need to specify the manufacturer
information(vendorid, archid, and impid) for this errata. Therefore, kernel
will know this errata is suitable for which CPU core. During the booting
procedure, kernel will select the errata required by the CPU core and then
patch it. It means that the kernel only applies the errata to the specified
CPU core. In this case, the vendor's errata does not affect each other at
runtime. The above patching procedure only occurs during the booting phase,
so we only take the overhead of the "alternative" mechanism once.
This "alternative" mechanism is enabled by default to ensure that all
required errata will be applied. However, users can disable this feature by
the Kconfig "CONFIG_RISCV_ERRATA_ALTERNATIVE".
Signed-off-by: Vincent Chen <vincent.chen@sifive.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-03-22 22:26:03 +08:00
|
|
|
// SPDX-License-Identifier: GPL-2.0-only
|
|
|
|
/*
|
|
|
|
* alternative runtime patching
|
|
|
|
* inspired by the ARM64 and x86 version
|
|
|
|
*
|
|
|
|
* Copyright (C) 2021 Sifive.
|
|
|
|
*/
|
|
|
|
|
|
|
|
#include <linux/init.h>
|
2022-05-12 03:29:12 +08:00
|
|
|
#include <linux/module.h>
|
riscv: Introduce alternative mechanism to apply errata solution
Introduce the "alternative" mechanism from ARM64 and x86 to apply the CPU
vendors' errata solution at runtime. The main purpose of this patch is
to provide a framework. Therefore, the implementation is quite basic for
now so that some scenarios could not use this schemei, such as patching
code to a module, relocating the patching code and heterogeneous CPU
topology.
Users could use the macro ALTERNATIVE to apply an errata to the existing
code flow. In the macro ALTERNATIVE, users need to specify the manufacturer
information(vendorid, archid, and impid) for this errata. Therefore, kernel
will know this errata is suitable for which CPU core. During the booting
procedure, kernel will select the errata required by the CPU core and then
patch it. It means that the kernel only applies the errata to the specified
CPU core. In this case, the vendor's errata does not affect each other at
runtime. The above patching procedure only occurs during the booting phase,
so we only take the overhead of the "alternative" mechanism once.
This "alternative" mechanism is enabled by default to ensure that all
required errata will be applied. However, users can disable this feature by
the Kconfig "CONFIG_RISCV_ERRATA_ALTERNATIVE".
Signed-off-by: Vincent Chen <vincent.chen@sifive.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-03-22 22:26:03 +08:00
|
|
|
#include <linux/cpu.h>
|
|
|
|
#include <linux/uaccess.h>
|
|
|
|
#include <asm/alternative.h>
|
|
|
|
#include <asm/sections.h>
|
|
|
|
#include <asm/vendorid_list.h>
|
|
|
|
#include <asm/sbi.h>
|
|
|
|
#include <asm/csr.h>
|
|
|
|
|
2022-05-12 03:29:20 +08:00
|
|
|
struct cpu_manufacturer_info_t {
|
riscv: Introduce alternative mechanism to apply errata solution
Introduce the "alternative" mechanism from ARM64 and x86 to apply the CPU
vendors' errata solution at runtime. The main purpose of this patch is
to provide a framework. Therefore, the implementation is quite basic for
now so that some scenarios could not use this schemei, such as patching
code to a module, relocating the patching code and heterogeneous CPU
topology.
Users could use the macro ALTERNATIVE to apply an errata to the existing
code flow. In the macro ALTERNATIVE, users need to specify the manufacturer
information(vendorid, archid, and impid) for this errata. Therefore, kernel
will know this errata is suitable for which CPU core. During the booting
procedure, kernel will select the errata required by the CPU core and then
patch it. It means that the kernel only applies the errata to the specified
CPU core. In this case, the vendor's errata does not affect each other at
runtime. The above patching procedure only occurs during the booting phase,
so we only take the overhead of the "alternative" mechanism once.
This "alternative" mechanism is enabled by default to ensure that all
required errata will be applied. However, users can disable this feature by
the Kconfig "CONFIG_RISCV_ERRATA_ALTERNATIVE".
Signed-off-by: Vincent Chen <vincent.chen@sifive.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-03-22 22:26:03 +08:00
|
|
|
unsigned long vendor_id;
|
|
|
|
unsigned long arch_id;
|
|
|
|
unsigned long imp_id;
|
2022-05-12 03:29:20 +08:00
|
|
|
void (*vendor_patch_func)(struct alt_entry *begin, struct alt_entry *end,
|
|
|
|
unsigned long archid, unsigned long impid,
|
|
|
|
unsigned int stage);
|
|
|
|
};
|
riscv: Introduce alternative mechanism to apply errata solution
Introduce the "alternative" mechanism from ARM64 and x86 to apply the CPU
vendors' errata solution at runtime. The main purpose of this patch is
to provide a framework. Therefore, the implementation is quite basic for
now so that some scenarios could not use this schemei, such as patching
code to a module, relocating the patching code and heterogeneous CPU
topology.
Users could use the macro ALTERNATIVE to apply an errata to the existing
code flow. In the macro ALTERNATIVE, users need to specify the manufacturer
information(vendorid, archid, and impid) for this errata. Therefore, kernel
will know this errata is suitable for which CPU core. During the booting
procedure, kernel will select the errata required by the CPU core and then
patch it. It means that the kernel only applies the errata to the specified
CPU core. In this case, the vendor's errata does not affect each other at
runtime. The above patching procedure only occurs during the booting phase,
so we only take the overhead of the "alternative" mechanism once.
This "alternative" mechanism is enabled by default to ensure that all
required errata will be applied. However, users can disable this feature by
the Kconfig "CONFIG_RISCV_ERRATA_ALTERNATIVE".
Signed-off-by: Vincent Chen <vincent.chen@sifive.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-03-22 22:26:03 +08:00
|
|
|
|
2022-05-12 03:29:20 +08:00
|
|
|
static void __init_or_module riscv_fill_cpu_mfr_info(struct cpu_manufacturer_info_t *cpu_mfr_info)
|
riscv: Introduce alternative mechanism to apply errata solution
Introduce the "alternative" mechanism from ARM64 and x86 to apply the CPU
vendors' errata solution at runtime. The main purpose of this patch is
to provide a framework. Therefore, the implementation is quite basic for
now so that some scenarios could not use this schemei, such as patching
code to a module, relocating the patching code and heterogeneous CPU
topology.
Users could use the macro ALTERNATIVE to apply an errata to the existing
code flow. In the macro ALTERNATIVE, users need to specify the manufacturer
information(vendorid, archid, and impid) for this errata. Therefore, kernel
will know this errata is suitable for which CPU core. During the booting
procedure, kernel will select the errata required by the CPU core and then
patch it. It means that the kernel only applies the errata to the specified
CPU core. In this case, the vendor's errata does not affect each other at
runtime. The above patching procedure only occurs during the booting phase,
so we only take the overhead of the "alternative" mechanism once.
This "alternative" mechanism is enabled by default to ensure that all
required errata will be applied. However, users can disable this feature by
the Kconfig "CONFIG_RISCV_ERRATA_ALTERNATIVE".
Signed-off-by: Vincent Chen <vincent.chen@sifive.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-03-22 22:26:03 +08:00
|
|
|
{
|
|
|
|
#ifdef CONFIG_RISCV_M_MODE
|
2022-05-12 03:29:20 +08:00
|
|
|
cpu_mfr_info->vendor_id = csr_read(CSR_MVENDORID);
|
|
|
|
cpu_mfr_info->arch_id = csr_read(CSR_MARCHID);
|
|
|
|
cpu_mfr_info->imp_id = csr_read(CSR_MIMPID);
|
riscv: Introduce alternative mechanism to apply errata solution
Introduce the "alternative" mechanism from ARM64 and x86 to apply the CPU
vendors' errata solution at runtime. The main purpose of this patch is
to provide a framework. Therefore, the implementation is quite basic for
now so that some scenarios could not use this schemei, such as patching
code to a module, relocating the patching code and heterogeneous CPU
topology.
Users could use the macro ALTERNATIVE to apply an errata to the existing
code flow. In the macro ALTERNATIVE, users need to specify the manufacturer
information(vendorid, archid, and impid) for this errata. Therefore, kernel
will know this errata is suitable for which CPU core. During the booting
procedure, kernel will select the errata required by the CPU core and then
patch it. It means that the kernel only applies the errata to the specified
CPU core. In this case, the vendor's errata does not affect each other at
runtime. The above patching procedure only occurs during the booting phase,
so we only take the overhead of the "alternative" mechanism once.
This "alternative" mechanism is enabled by default to ensure that all
required errata will be applied. However, users can disable this feature by
the Kconfig "CONFIG_RISCV_ERRATA_ALTERNATIVE".
Signed-off-by: Vincent Chen <vincent.chen@sifive.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-03-22 22:26:03 +08:00
|
|
|
#else
|
2022-05-12 03:29:20 +08:00
|
|
|
cpu_mfr_info->vendor_id = sbi_get_mvendorid();
|
|
|
|
cpu_mfr_info->arch_id = sbi_get_marchid();
|
|
|
|
cpu_mfr_info->imp_id = sbi_get_mimpid();
|
riscv: Introduce alternative mechanism to apply errata solution
Introduce the "alternative" mechanism from ARM64 and x86 to apply the CPU
vendors' errata solution at runtime. The main purpose of this patch is
to provide a framework. Therefore, the implementation is quite basic for
now so that some scenarios could not use this schemei, such as patching
code to a module, relocating the patching code and heterogeneous CPU
topology.
Users could use the macro ALTERNATIVE to apply an errata to the existing
code flow. In the macro ALTERNATIVE, users need to specify the manufacturer
information(vendorid, archid, and impid) for this errata. Therefore, kernel
will know this errata is suitable for which CPU core. During the booting
procedure, kernel will select the errata required by the CPU core and then
patch it. It means that the kernel only applies the errata to the specified
CPU core. In this case, the vendor's errata does not affect each other at
runtime. The above patching procedure only occurs during the booting phase,
so we only take the overhead of the "alternative" mechanism once.
This "alternative" mechanism is enabled by default to ensure that all
required errata will be applied. However, users can disable this feature by
the Kconfig "CONFIG_RISCV_ERRATA_ALTERNATIVE".
Signed-off-by: Vincent Chen <vincent.chen@sifive.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-03-22 22:26:03 +08:00
|
|
|
#endif
|
|
|
|
|
2022-05-12 03:29:20 +08:00
|
|
|
switch (cpu_mfr_info->vendor_id) {
|
2021-03-22 22:26:04 +08:00
|
|
|
#ifdef CONFIG_ERRATA_SIFIVE
|
|
|
|
case SIFIVE_VENDOR_ID:
|
2022-05-12 03:29:20 +08:00
|
|
|
cpu_mfr_info->vendor_patch_func = sifive_errata_patch_func;
|
2021-03-22 22:26:04 +08:00
|
|
|
break;
|
2022-05-12 03:29:21 +08:00
|
|
|
#endif
|
|
|
|
#ifdef CONFIG_ERRATA_THEAD
|
|
|
|
case THEAD_VENDOR_ID:
|
|
|
|
cpu_mfr_info->vendor_patch_func = thead_errata_patch_func;
|
|
|
|
break;
|
2021-03-22 22:26:04 +08:00
|
|
|
#endif
|
riscv: Introduce alternative mechanism to apply errata solution
Introduce the "alternative" mechanism from ARM64 and x86 to apply the CPU
vendors' errata solution at runtime. The main purpose of this patch is
to provide a framework. Therefore, the implementation is quite basic for
now so that some scenarios could not use this schemei, such as patching
code to a module, relocating the patching code and heterogeneous CPU
topology.
Users could use the macro ALTERNATIVE to apply an errata to the existing
code flow. In the macro ALTERNATIVE, users need to specify the manufacturer
information(vendorid, archid, and impid) for this errata. Therefore, kernel
will know this errata is suitable for which CPU core. During the booting
procedure, kernel will select the errata required by the CPU core and then
patch it. It means that the kernel only applies the errata to the specified
CPU core. In this case, the vendor's errata does not affect each other at
runtime. The above patching procedure only occurs during the booting phase,
so we only take the overhead of the "alternative" mechanism once.
This "alternative" mechanism is enabled by default to ensure that all
required errata will be applied. However, users can disable this feature by
the Kconfig "CONFIG_RISCV_ERRATA_ALTERNATIVE".
Signed-off-by: Vincent Chen <vincent.chen@sifive.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-03-22 22:26:03 +08:00
|
|
|
default:
|
2022-05-12 03:29:20 +08:00
|
|
|
cpu_mfr_info->vendor_patch_func = NULL;
|
riscv: Introduce alternative mechanism to apply errata solution
Introduce the "alternative" mechanism from ARM64 and x86 to apply the CPU
vendors' errata solution at runtime. The main purpose of this patch is
to provide a framework. Therefore, the implementation is quite basic for
now so that some scenarios could not use this schemei, such as patching
code to a module, relocating the patching code and heterogeneous CPU
topology.
Users could use the macro ALTERNATIVE to apply an errata to the existing
code flow. In the macro ALTERNATIVE, users need to specify the manufacturer
information(vendorid, archid, and impid) for this errata. Therefore, kernel
will know this errata is suitable for which CPU core. During the booting
procedure, kernel will select the errata required by the CPU core and then
patch it. It means that the kernel only applies the errata to the specified
CPU core. In this case, the vendor's errata does not affect each other at
runtime. The above patching procedure only occurs during the booting phase,
so we only take the overhead of the "alternative" mechanism once.
This "alternative" mechanism is enabled by default to ensure that all
required errata will be applied. However, users can disable this feature by
the Kconfig "CONFIG_RISCV_ERRATA_ALTERNATIVE".
Signed-off-by: Vincent Chen <vincent.chen@sifive.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-03-22 22:26:03 +08:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* This is called very early in the boot process (directly after we run
|
|
|
|
* a feature detect on the boot CPU). No need to worry about other CPUs
|
|
|
|
* here.
|
|
|
|
*/
|
2022-05-12 03:29:12 +08:00
|
|
|
static void __init_or_module _apply_alternatives(struct alt_entry *begin,
|
|
|
|
struct alt_entry *end,
|
|
|
|
unsigned int stage)
|
2022-05-12 03:29:11 +08:00
|
|
|
{
|
2022-05-12 03:29:20 +08:00
|
|
|
struct cpu_manufacturer_info_t cpu_mfr_info;
|
|
|
|
|
|
|
|
riscv_fill_cpu_mfr_info(&cpu_mfr_info);
|
|
|
|
|
2022-05-12 03:29:18 +08:00
|
|
|
riscv_cpufeature_patch_func(begin, end, stage);
|
|
|
|
|
2022-05-12 03:29:20 +08:00
|
|
|
if (!cpu_mfr_info.vendor_patch_func)
|
2022-05-12 03:29:11 +08:00
|
|
|
return;
|
|
|
|
|
2022-05-12 03:29:20 +08:00
|
|
|
cpu_mfr_info.vendor_patch_func(begin, end,
|
|
|
|
cpu_mfr_info.arch_id,
|
|
|
|
cpu_mfr_info.imp_id,
|
|
|
|
stage);
|
2022-05-12 03:29:11 +08:00
|
|
|
}
|
|
|
|
|
riscv: Introduce alternative mechanism to apply errata solution
Introduce the "alternative" mechanism from ARM64 and x86 to apply the CPU
vendors' errata solution at runtime. The main purpose of this patch is
to provide a framework. Therefore, the implementation is quite basic for
now so that some scenarios could not use this schemei, such as patching
code to a module, relocating the patching code and heterogeneous CPU
topology.
Users could use the macro ALTERNATIVE to apply an errata to the existing
code flow. In the macro ALTERNATIVE, users need to specify the manufacturer
information(vendorid, archid, and impid) for this errata. Therefore, kernel
will know this errata is suitable for which CPU core. During the booting
procedure, kernel will select the errata required by the CPU core and then
patch it. It means that the kernel only applies the errata to the specified
CPU core. In this case, the vendor's errata does not affect each other at
runtime. The above patching procedure only occurs during the booting phase,
so we only take the overhead of the "alternative" mechanism once.
This "alternative" mechanism is enabled by default to ensure that all
required errata will be applied. However, users can disable this feature by
the Kconfig "CONFIG_RISCV_ERRATA_ALTERNATIVE".
Signed-off-by: Vincent Chen <vincent.chen@sifive.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-03-22 22:26:03 +08:00
|
|
|
void __init apply_boot_alternatives(void)
|
|
|
|
{
|
|
|
|
/* If called on non-boot cpu things could go wrong */
|
|
|
|
WARN_ON(smp_processor_id() != 0);
|
|
|
|
|
2022-05-12 03:29:11 +08:00
|
|
|
_apply_alternatives((struct alt_entry *)__alt_start,
|
|
|
|
(struct alt_entry *)__alt_end,
|
|
|
|
RISCV_ALTERNATIVES_BOOT);
|
riscv: Introduce alternative mechanism to apply errata solution
Introduce the "alternative" mechanism from ARM64 and x86 to apply the CPU
vendors' errata solution at runtime. The main purpose of this patch is
to provide a framework. Therefore, the implementation is quite basic for
now so that some scenarios could not use this schemei, such as patching
code to a module, relocating the patching code and heterogeneous CPU
topology.
Users could use the macro ALTERNATIVE to apply an errata to the existing
code flow. In the macro ALTERNATIVE, users need to specify the manufacturer
information(vendorid, archid, and impid) for this errata. Therefore, kernel
will know this errata is suitable for which CPU core. During the booting
procedure, kernel will select the errata required by the CPU core and then
patch it. It means that the kernel only applies the errata to the specified
CPU core. In this case, the vendor's errata does not affect each other at
runtime. The above patching procedure only occurs during the booting phase,
so we only take the overhead of the "alternative" mechanism once.
This "alternative" mechanism is enabled by default to ensure that all
required errata will be applied. However, users can disable this feature by
the Kconfig "CONFIG_RISCV_ERRATA_ALTERNATIVE".
Signed-off-by: Vincent Chen <vincent.chen@sifive.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-03-22 22:26:03 +08:00
|
|
|
}
|
2022-05-12 03:29:12 +08:00
|
|
|
|
2022-05-12 03:29:21 +08:00
|
|
|
/*
|
|
|
|
* apply_early_boot_alternatives() is called from setup_vm() with MMU-off.
|
|
|
|
*
|
|
|
|
* Following requirements should be honoured for it to work correctly:
|
|
|
|
* 1) It should use PC-relative addressing for accessing kernel symbols.
|
|
|
|
* To achieve this we always use GCC cmodel=medany.
|
|
|
|
* 2) The compiler instrumentation for FTRACE will not work for setup_vm()
|
|
|
|
* so disable compiler instrumentation when FTRACE is enabled.
|
|
|
|
*
|
|
|
|
* Currently, the above requirements are honoured by using custom CFLAGS
|
|
|
|
* for alternative.o in kernel/Makefile.
|
|
|
|
*/
|
|
|
|
void __init apply_early_boot_alternatives(void)
|
|
|
|
{
|
|
|
|
#ifdef CONFIG_RISCV_ALTERNATIVE_EARLY
|
|
|
|
_apply_alternatives((struct alt_entry *)__alt_start,
|
|
|
|
(struct alt_entry *)__alt_end,
|
|
|
|
RISCV_ALTERNATIVES_EARLY_BOOT);
|
|
|
|
#endif
|
|
|
|
}
|
|
|
|
|
2022-05-12 03:29:12 +08:00
|
|
|
#ifdef CONFIG_MODULES
|
|
|
|
void apply_module_alternatives(void *start, size_t length)
|
|
|
|
{
|
|
|
|
_apply_alternatives((struct alt_entry *)start,
|
|
|
|
(struct alt_entry *)(start + length),
|
|
|
|
RISCV_ALTERNATIVES_MODULE);
|
|
|
|
}
|
|
|
|
#endif
|