UbiquitousOS
/
OpenCloudOS-Kernel
mirror of https://gitee.com/OpenCloudOS/OpenCloudOS-Kernel.git
11
0
Fork 0
Code Issues Projects Releases Wiki Activity
OpenCloudOS-Kernel/arch/riscv/kernel/patch.c

130 lines
2.7 KiB
C
Raw Normal View History

riscv: introduce interfaces to patch kernel code On strict kernel memory permission, we couldn't patch code without writable permission. Preserve two holes in fixmap area, so we can map the kernel code temporarily to fixmap area, then patch the instructions. We need two pages here because we support the compressed instruction, so the instruction might be align to 2 bytes. When patching the 32-bit length instruction which is 2 bytes alignment, it will across two pages. Introduce two interfaces to patch kernel code: riscv_patch_text_nosync: - patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed. riscv_patch_text: - patch code and always synchronize with stop_machine() Signed-off-by: Zong Li <zong.li@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-10 00:55:43 +08:00
// SPDX-License-Identifier: GPL-2.0-only
/*
* Copyright (C) 2020 SiFive
*/
#include <linux/spinlock.h>
#include <linux/mm.h>
riscv: Use text_mutex instead of patch_lock We don't need the additional lock protection when patching the text. There are two patching interfaces here: - patch_text: patch code and always synchronize with stop_machine() - patch_text_nosync: patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed. For the first one, stop_machine() is protected by its own mutex, and also the irq is already disabled here. For the second one, in risc-v real case now, it would be used to ftrace patching the mcount function, since it already running under kstop_machine(), no other thread will run, so we could use text_mutex on ftrace side. Signed-off-by: Zong Li <zong.li@sifive.com> Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-04-21 15:30:01 +08:00
#include <linux/memory.h>
riscv: introduce interfaces to patch kernel code On strict kernel memory permission, we couldn't patch code without writable permission. Preserve two holes in fixmap area, so we can map the kernel code temporarily to fixmap area, then patch the instructions. We need two pages here because we support the compressed instruction, so the instruction might be align to 2 bytes. When patching the 32-bit length instruction which is 2 bytes alignment, it will across two pages. Introduce two interfaces to patch kernel code: riscv_patch_text_nosync: - patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed. riscv_patch_text: - patch code and always synchronize with stop_machine() Signed-off-by: Zong Li <zong.li@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-10 00:55:43 +08:00
#include <linux/uaccess.h>
#include <linux/stop_machine.h>
#include <asm/kprobes.h>
#include <asm/cacheflush.h>
#include <asm/fixmap.h>
riscv: fix build warning of missing prototypes Add the missing header in file, it was lost in original implementation. The warning message as follows: - no previous prototype for 'patch_text_nosync' [-Wmissing-prototypes] - no previous prototype for 'patch_text' [-Wmissing-prototypes] Changed in v2: - Correct the typo of commit message. Signed-off-by: Zong Li <zong.li@sifive.com> Reported-by: kbuild test robot <lkp@intel.com> Reviewed-by: Atish Patra <atish.patra@wdc.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-06-01 15:10:58 +08:00
#include <asm/patch.h>
riscv: introduce interfaces to patch kernel code On strict kernel memory permission, we couldn't patch code without writable permission. Preserve two holes in fixmap area, so we can map the kernel code temporarily to fixmap area, then patch the instructions. We need two pages here because we support the compressed instruction, so the instruction might be align to 2 bytes. When patching the 32-bit length instruction which is 2 bytes alignment, it will across two pages. Introduce two interfaces to patch kernel code: riscv_patch_text_nosync: - patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed. riscv_patch_text: - patch code and always synchronize with stop_machine() Signed-off-by: Zong Li <zong.li@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-10 00:55:43 +08:00
riscv: Remove the 'riscv_' prefix of function name Refactor the function name by removing the 'riscv_' prefix, it would be better unless it could mix up with arch-independent functions. Signed-off-by: Zong Li <zong.li@sifive.com> Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-04-21 15:29:59 +08:00
struct patch_insn {
riscv: introduce interfaces to patch kernel code On strict kernel memory permission, we couldn't patch code without writable permission. Preserve two holes in fixmap area, so we can map the kernel code temporarily to fixmap area, then patch the instructions. We need two pages here because we support the compressed instruction, so the instruction might be align to 2 bytes. When patching the 32-bit length instruction which is 2 bytes alignment, it will across two pages. Introduce two interfaces to patch kernel code: riscv_patch_text_nosync: - patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed. riscv_patch_text: - patch code and always synchronize with stop_machine() Signed-off-by: Zong Li <zong.li@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-10 00:55:43 +08:00
void *addr;
u32 insn;
atomic_t cpu_count;
};
#ifdef CONFIG_MMU
riscv: Use NOKPROBE_SYMBOL() instead of __krpobes annotation The __kprobes annotation is old style, so change it to NOKPROBE_SYMBOL(). Signed-off-by: Zong Li <zong.li@sifive.com> Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-04-21 15:30:00 +08:00
static void *patch_map(void *addr, int fixmap)
riscv: introduce interfaces to patch kernel code On strict kernel memory permission, we couldn't patch code without writable permission. Preserve two holes in fixmap area, so we can map the kernel code temporarily to fixmap area, then patch the instructions. We need two pages here because we support the compressed instruction, so the instruction might be align to 2 bytes. When patching the 32-bit length instruction which is 2 bytes alignment, it will across two pages. Introduce two interfaces to patch kernel code: riscv_patch_text_nosync: - patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed. riscv_patch_text: - patch code and always synchronize with stop_machine() Signed-off-by: Zong Li <zong.li@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-10 00:55:43 +08:00
{
uintptr_t uintaddr = (uintptr_t) addr;
struct page *page;
if (core_kernel_text(uintaddr))
page = phys_to_page(__pa_symbol(addr));
else if (IS_ENABLED(CONFIG_STRICT_MODULE_RWX))
page = vmalloc_to_page(addr);
else
return addr;
BUG_ON(!page);
return (void *)set_fixmap_offset(fixmap, page_to_phys(page) +
(uintaddr & ~PAGE_MASK));
}
riscv: Use NOKPROBE_SYMBOL() instead of __krpobes annotation The __kprobes annotation is old style, so change it to NOKPROBE_SYMBOL(). Signed-off-by: Zong Li <zong.li@sifive.com> Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-04-21 15:30:00 +08:00
NOKPROBE_SYMBOL(patch_map);
riscv: introduce interfaces to patch kernel code On strict kernel memory permission, we couldn't patch code without writable permission. Preserve two holes in fixmap area, so we can map the kernel code temporarily to fixmap area, then patch the instructions. We need two pages here because we support the compressed instruction, so the instruction might be align to 2 bytes. When patching the 32-bit length instruction which is 2 bytes alignment, it will across two pages. Introduce two interfaces to patch kernel code: riscv_patch_text_nosync: - patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed. riscv_patch_text: - patch code and always synchronize with stop_machine() Signed-off-by: Zong Li <zong.li@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-10 00:55:43 +08:00
riscv: Use NOKPROBE_SYMBOL() instead of __krpobes annotation The __kprobes annotation is old style, so change it to NOKPROBE_SYMBOL(). Signed-off-by: Zong Li <zong.li@sifive.com> Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-04-21 15:30:00 +08:00
static void patch_unmap(int fixmap)
riscv: introduce interfaces to patch kernel code On strict kernel memory permission, we couldn't patch code without writable permission. Preserve two holes in fixmap area, so we can map the kernel code temporarily to fixmap area, then patch the instructions. We need two pages here because we support the compressed instruction, so the instruction might be align to 2 bytes. When patching the 32-bit length instruction which is 2 bytes alignment, it will across two pages. Introduce two interfaces to patch kernel code: riscv_patch_text_nosync: - patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed. riscv_patch_text: - patch code and always synchronize with stop_machine() Signed-off-by: Zong Li <zong.li@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-10 00:55:43 +08:00
{
clear_fixmap(fixmap);
}
riscv: Use NOKPROBE_SYMBOL() instead of __krpobes annotation The __kprobes annotation is old style, so change it to NOKPROBE_SYMBOL(). Signed-off-by: Zong Li <zong.li@sifive.com> Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-04-21 15:30:00 +08:00
NOKPROBE_SYMBOL(patch_unmap);
riscv: introduce interfaces to patch kernel code On strict kernel memory permission, we couldn't patch code without writable permission. Preserve two holes in fixmap area, so we can map the kernel code temporarily to fixmap area, then patch the instructions. We need two pages here because we support the compressed instruction, so the instruction might be align to 2 bytes. When patching the 32-bit length instruction which is 2 bytes alignment, it will across two pages. Introduce two interfaces to patch kernel code: riscv_patch_text_nosync: - patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed. riscv_patch_text: - patch code and always synchronize with stop_machine() Signed-off-by: Zong Li <zong.li@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-10 00:55:43 +08:00
riscv: Use NOKPROBE_SYMBOL() instead of __krpobes annotation The __kprobes annotation is old style, so change it to NOKPROBE_SYMBOL(). Signed-off-by: Zong Li <zong.li@sifive.com> Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-04-21 15:30:00 +08:00
static int patch_insn_write(void *addr, const void *insn, size_t len)
riscv: introduce interfaces to patch kernel code On strict kernel memory permission, we couldn't patch code without writable permission. Preserve two holes in fixmap area, so we can map the kernel code temporarily to fixmap area, then patch the instructions. We need two pages here because we support the compressed instruction, so the instruction might be align to 2 bytes. When patching the 32-bit length instruction which is 2 bytes alignment, it will across two pages. Introduce two interfaces to patch kernel code: riscv_patch_text_nosync: - patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed. riscv_patch_text: - patch code and always synchronize with stop_machine() Signed-off-by: Zong Li <zong.li@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-10 00:55:43 +08:00
{
void *waddr = addr;
bool across_pages = (((uintptr_t) addr & ~PAGE_MASK) + len) > PAGE_SIZE;
int ret;
riscv: Use text_mutex instead of patch_lock We don't need the additional lock protection when patching the text. There are two patching interfaces here: - patch_text: patch code and always synchronize with stop_machine() - patch_text_nosync: patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed. For the first one, stop_machine() is protected by its own mutex, and also the irq is already disabled here. For the second one, in risc-v real case now, it would be used to ftrace patching the mcount function, since it already running under kstop_machine(), no other thread will run, so we could use text_mutex on ftrace side. Signed-off-by: Zong Li <zong.li@sifive.com> Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-04-21 15:30:01 +08:00
/*
* Before reaching here, it was expected to lock the text_mutex
* already, so we don't need to give another lock here and could
* ensure that it was safe between each cores.
*/
lockdep_assert_held(&text_mutex);
riscv: introduce interfaces to patch kernel code On strict kernel memory permission, we couldn't patch code without writable permission. Preserve two holes in fixmap area, so we can map the kernel code temporarily to fixmap area, then patch the instructions. We need two pages here because we support the compressed instruction, so the instruction might be align to 2 bytes. When patching the 32-bit length instruction which is 2 bytes alignment, it will across two pages. Introduce two interfaces to patch kernel code: riscv_patch_text_nosync: - patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed. riscv_patch_text: - patch code and always synchronize with stop_machine() Signed-off-by: Zong Li <zong.li@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-10 00:55:43 +08:00
if (across_pages)
patch_map(addr + len, FIX_TEXT_POKE1);
waddr = patch_map(addr, FIX_TEXT_POKE0);
maccess: rename probe_kernel_{read,write} to copy_{from,to}_kernel_nofault Better describe what these functions do. Suggested-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2020-06-17 15:37:53 +08:00
ret = copy_to_kernel_nofault(waddr, insn, len);
riscv: introduce interfaces to patch kernel code On strict kernel memory permission, we couldn't patch code without writable permission. Preserve two holes in fixmap area, so we can map the kernel code temporarily to fixmap area, then patch the instructions. We need two pages here because we support the compressed instruction, so the instruction might be align to 2 bytes. When patching the 32-bit length instruction which is 2 bytes alignment, it will across two pages. Introduce two interfaces to patch kernel code: riscv_patch_text_nosync: - patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed. riscv_patch_text: - patch code and always synchronize with stop_machine() Signed-off-by: Zong Li <zong.li@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-10 00:55:43 +08:00
patch_unmap(FIX_TEXT_POKE0);
if (across_pages)
patch_unmap(FIX_TEXT_POKE1);
return ret;
}
riscv: Use NOKPROBE_SYMBOL() instead of __krpobes annotation The __kprobes annotation is old style, so change it to NOKPROBE_SYMBOL(). Signed-off-by: Zong Li <zong.li@sifive.com> Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-04-21 15:30:00 +08:00
NOKPROBE_SYMBOL(patch_insn_write);
riscv: introduce interfaces to patch kernel code On strict kernel memory permission, we couldn't patch code without writable permission. Preserve two holes in fixmap area, so we can map the kernel code temporarily to fixmap area, then patch the instructions. We need two pages here because we support the compressed instruction, so the instruction might be align to 2 bytes. When patching the 32-bit length instruction which is 2 bytes alignment, it will across two pages. Introduce two interfaces to patch kernel code: riscv_patch_text_nosync: - patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed. riscv_patch_text: - patch code and always synchronize with stop_machine() Signed-off-by: Zong Li <zong.li@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-10 00:55:43 +08:00
#else
riscv: Use NOKPROBE_SYMBOL() instead of __krpobes annotation The __kprobes annotation is old style, so change it to NOKPROBE_SYMBOL(). Signed-off-by: Zong Li <zong.li@sifive.com> Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-04-21 15:30:00 +08:00
static int patch_insn_write(void *addr, const void *insn, size_t len)
riscv: introduce interfaces to patch kernel code On strict kernel memory permission, we couldn't patch code without writable permission. Preserve two holes in fixmap area, so we can map the kernel code temporarily to fixmap area, then patch the instructions. We need two pages here because we support the compressed instruction, so the instruction might be align to 2 bytes. When patching the 32-bit length instruction which is 2 bytes alignment, it will across two pages. Introduce two interfaces to patch kernel code: riscv_patch_text_nosync: - patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed. riscv_patch_text: - patch code and always synchronize with stop_machine() Signed-off-by: Zong Li <zong.li@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-10 00:55:43 +08:00
{
maccess: rename probe_kernel_{read,write} to copy_{from,to}_kernel_nofault Better describe what these functions do. Suggested-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2020-06-17 15:37:53 +08:00
return copy_to_kernel_nofault(addr, insn, len);
riscv: introduce interfaces to patch kernel code On strict kernel memory permission, we couldn't patch code without writable permission. Preserve two holes in fixmap area, so we can map the kernel code temporarily to fixmap area, then patch the instructions. We need two pages here because we support the compressed instruction, so the instruction might be align to 2 bytes. When patching the 32-bit length instruction which is 2 bytes alignment, it will across two pages. Introduce two interfaces to patch kernel code: riscv_patch_text_nosync: - patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed. riscv_patch_text: - patch code and always synchronize with stop_machine() Signed-off-by: Zong Li <zong.li@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-10 00:55:43 +08:00
}
riscv: Use NOKPROBE_SYMBOL() instead of __krpobes annotation The __kprobes annotation is old style, so change it to NOKPROBE_SYMBOL(). Signed-off-by: Zong Li <zong.li@sifive.com> Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-04-21 15:30:00 +08:00
NOKPROBE_SYMBOL(patch_insn_write);
riscv: introduce interfaces to patch kernel code On strict kernel memory permission, we couldn't patch code without writable permission. Preserve two holes in fixmap area, so we can map the kernel code temporarily to fixmap area, then patch the instructions. We need two pages here because we support the compressed instruction, so the instruction might be align to 2 bytes. When patching the 32-bit length instruction which is 2 bytes alignment, it will across two pages. Introduce two interfaces to patch kernel code: riscv_patch_text_nosync: - patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed. riscv_patch_text: - patch code and always synchronize with stop_machine() Signed-off-by: Zong Li <zong.li@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-10 00:55:43 +08:00
#endif /* CONFIG_MMU */
riscv: Use NOKPROBE_SYMBOL() instead of __krpobes annotation The __kprobes annotation is old style, so change it to NOKPROBE_SYMBOL(). Signed-off-by: Zong Li <zong.li@sifive.com> Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-04-21 15:30:00 +08:00
int patch_text_nosync(void *addr, const void *insns, size_t len)
riscv: introduce interfaces to patch kernel code On strict kernel memory permission, we couldn't patch code without writable permission. Preserve two holes in fixmap area, so we can map the kernel code temporarily to fixmap area, then patch the instructions. We need two pages here because we support the compressed instruction, so the instruction might be align to 2 bytes. When patching the 32-bit length instruction which is 2 bytes alignment, it will across two pages. Introduce two interfaces to patch kernel code: riscv_patch_text_nosync: - patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed. riscv_patch_text: - patch code and always synchronize with stop_machine() Signed-off-by: Zong Li <zong.li@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-10 00:55:43 +08:00
{
u32 *tp = addr;
int ret;
riscv: Remove the 'riscv_' prefix of function name Refactor the function name by removing the 'riscv_' prefix, it would be better unless it could mix up with arch-independent functions. Signed-off-by: Zong Li <zong.li@sifive.com> Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-04-21 15:29:59 +08:00
ret = patch_insn_write(tp, insns, len);
riscv: introduce interfaces to patch kernel code On strict kernel memory permission, we couldn't patch code without writable permission. Preserve two holes in fixmap area, so we can map the kernel code temporarily to fixmap area, then patch the instructions. We need two pages here because we support the compressed instruction, so the instruction might be align to 2 bytes. When patching the 32-bit length instruction which is 2 bytes alignment, it will across two pages. Introduce two interfaces to patch kernel code: riscv_patch_text_nosync: - patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed. riscv_patch_text: - patch code and always synchronize with stop_machine() Signed-off-by: Zong Li <zong.li@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-10 00:55:43 +08:00
if (!ret)
flush_icache_range((uintptr_t) tp, (uintptr_t) tp + len);
return ret;
}
riscv: Use NOKPROBE_SYMBOL() instead of __krpobes annotation The __kprobes annotation is old style, so change it to NOKPROBE_SYMBOL(). Signed-off-by: Zong Li <zong.li@sifive.com> Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-04-21 15:30:00 +08:00
NOKPROBE_SYMBOL(patch_text_nosync);
riscv: introduce interfaces to patch kernel code On strict kernel memory permission, we couldn't patch code without writable permission. Preserve two holes in fixmap area, so we can map the kernel code temporarily to fixmap area, then patch the instructions. We need two pages here because we support the compressed instruction, so the instruction might be align to 2 bytes. When patching the 32-bit length instruction which is 2 bytes alignment, it will across two pages. Introduce two interfaces to patch kernel code: riscv_patch_text_nosync: - patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed. riscv_patch_text: - patch code and always synchronize with stop_machine() Signed-off-by: Zong Li <zong.li@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-10 00:55:43 +08:00
riscv: Use NOKPROBE_SYMBOL() instead of __krpobes annotation The __kprobes annotation is old style, so change it to NOKPROBE_SYMBOL(). Signed-off-by: Zong Li <zong.li@sifive.com> Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-04-21 15:30:00 +08:00
static int patch_text_cb(void *data)
riscv: introduce interfaces to patch kernel code On strict kernel memory permission, we couldn't patch code without writable permission. Preserve two holes in fixmap area, so we can map the kernel code temporarily to fixmap area, then patch the instructions. We need two pages here because we support the compressed instruction, so the instruction might be align to 2 bytes. When patching the 32-bit length instruction which is 2 bytes alignment, it will across two pages. Introduce two interfaces to patch kernel code: riscv_patch_text_nosync: - patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed. riscv_patch_text: - patch code and always synchronize with stop_machine() Signed-off-by: Zong Li <zong.li@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-10 00:55:43 +08:00
{
riscv: Remove the 'riscv_' prefix of function name Refactor the function name by removing the 'riscv_' prefix, it would be better unless it could mix up with arch-independent functions. Signed-off-by: Zong Li <zong.li@sifive.com> Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-04-21 15:29:59 +08:00
struct patch_insn *patch = data;
riscv: introduce interfaces to patch kernel code On strict kernel memory permission, we couldn't patch code without writable permission. Preserve two holes in fixmap area, so we can map the kernel code temporarily to fixmap area, then patch the instructions. We need two pages here because we support the compressed instruction, so the instruction might be align to 2 bytes. When patching the 32-bit length instruction which is 2 bytes alignment, it will across two pages. Introduce two interfaces to patch kernel code: riscv_patch_text_nosync: - patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed. riscv_patch_text: - patch code and always synchronize with stop_machine() Signed-off-by: Zong Li <zong.li@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-10 00:55:43 +08:00
int ret = 0;
if (atomic_inc_return(&patch->cpu_count) == 1) {
ret =
riscv: Remove the 'riscv_' prefix of function name Refactor the function name by removing the 'riscv_' prefix, it would be better unless it could mix up with arch-independent functions. Signed-off-by: Zong Li <zong.li@sifive.com> Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-04-21 15:29:59 +08:00
patch_text_nosync(patch->addr, &patch->insn,
riscv: introduce interfaces to patch kernel code On strict kernel memory permission, we couldn't patch code without writable permission. Preserve two holes in fixmap area, so we can map the kernel code temporarily to fixmap area, then patch the instructions. We need two pages here because we support the compressed instruction, so the instruction might be align to 2 bytes. When patching the 32-bit length instruction which is 2 bytes alignment, it will across two pages. Introduce two interfaces to patch kernel code: riscv_patch_text_nosync: - patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed. riscv_patch_text: - patch code and always synchronize with stop_machine() Signed-off-by: Zong Li <zong.li@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-10 00:55:43 +08:00
GET_INSN_LENGTH(patch->insn));
atomic_inc(&patch->cpu_count);
} else {
while (atomic_read(&patch->cpu_count) <= num_online_cpus())
cpu_relax();
smp_mb();
}
return ret;
}
riscv: Use NOKPROBE_SYMBOL() instead of __krpobes annotation The __kprobes annotation is old style, so change it to NOKPROBE_SYMBOL(). Signed-off-by: Zong Li <zong.li@sifive.com> Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-04-21 15:30:00 +08:00
NOKPROBE_SYMBOL(patch_text_cb);
riscv: introduce interfaces to patch kernel code On strict kernel memory permission, we couldn't patch code without writable permission. Preserve two holes in fixmap area, so we can map the kernel code temporarily to fixmap area, then patch the instructions. We need two pages here because we support the compressed instruction, so the instruction might be align to 2 bytes. When patching the 32-bit length instruction which is 2 bytes alignment, it will across two pages. Introduce two interfaces to patch kernel code: riscv_patch_text_nosync: - patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed. riscv_patch_text: - patch code and always synchronize with stop_machine() Signed-off-by: Zong Li <zong.li@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-10 00:55:43 +08:00
riscv: Use NOKPROBE_SYMBOL() instead of __krpobes annotation The __kprobes annotation is old style, so change it to NOKPROBE_SYMBOL(). Signed-off-by: Zong Li <zong.li@sifive.com> Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-04-21 15:30:00 +08:00
int patch_text(void *addr, u32 insn)
riscv: introduce interfaces to patch kernel code On strict kernel memory permission, we couldn't patch code without writable permission. Preserve two holes in fixmap area, so we can map the kernel code temporarily to fixmap area, then patch the instructions. We need two pages here because we support the compressed instruction, so the instruction might be align to 2 bytes. When patching the 32-bit length instruction which is 2 bytes alignment, it will across two pages. Introduce two interfaces to patch kernel code: riscv_patch_text_nosync: - patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed. riscv_patch_text: - patch code and always synchronize with stop_machine() Signed-off-by: Zong Li <zong.li@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-10 00:55:43 +08:00
{
riscv: Remove the 'riscv_' prefix of function name Refactor the function name by removing the 'riscv_' prefix, it would be better unless it could mix up with arch-independent functions. Signed-off-by: Zong Li <zong.li@sifive.com> Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-04-21 15:29:59 +08:00
struct patch_insn patch = {
riscv: introduce interfaces to patch kernel code On strict kernel memory permission, we couldn't patch code without writable permission. Preserve two holes in fixmap area, so we can map the kernel code temporarily to fixmap area, then patch the instructions. We need two pages here because we support the compressed instruction, so the instruction might be align to 2 bytes. When patching the 32-bit length instruction which is 2 bytes alignment, it will across two pages. Introduce two interfaces to patch kernel code: riscv_patch_text_nosync: - patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed. riscv_patch_text: - patch code and always synchronize with stop_machine() Signed-off-by: Zong Li <zong.li@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-10 00:55:43 +08:00
.addr = addr,
.insn = insn,
.cpu_count = ATOMIC_INIT(0),
};
riscv: Remove the 'riscv_' prefix of function name Refactor the function name by removing the 'riscv_' prefix, it would be better unless it could mix up with arch-independent functions. Signed-off-by: Zong Li <zong.li@sifive.com> Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-04-21 15:29:59 +08:00
return stop_machine_cpuslocked(patch_text_cb,
riscv: introduce interfaces to patch kernel code On strict kernel memory permission, we couldn't patch code without writable permission. Preserve two holes in fixmap area, so we can map the kernel code temporarily to fixmap area, then patch the instructions. We need two pages here because we support the compressed instruction, so the instruction might be align to 2 bytes. When patching the 32-bit length instruction which is 2 bytes alignment, it will across two pages. Introduce two interfaces to patch kernel code: riscv_patch_text_nosync: - patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed. riscv_patch_text: - patch code and always synchronize with stop_machine() Signed-off-by: Zong Li <zong.li@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-10 00:55:43 +08:00
&patch, cpu_online_mask);
}
riscv: Use NOKPROBE_SYMBOL() instead of __krpobes annotation The __kprobes annotation is old style, so change it to NOKPROBE_SYMBOL(). Signed-off-by: Zong Li <zong.li@sifive.com> Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-04-21 15:30:00 +08:00
NOKPROBE_SYMBOL(patch_text);