OpenCloudOS-Kernel/arch/s390/kernel/ftrace.h

27 lines
877 B
C
Raw Normal View History

s390/ftrace: implement hotpatching s390 allows hotpatching the mask of a conditional jump instruction. Make use of this feature in order to avoid the expensive stop_machine() call. The new trampolines are split in 3 stages: - A first stage is a 6-byte relative conditional long branch located at each function's entry point. Its offset always points to the second stage for the corresponding function, and its mask is either all 0s (ftrace off) or all 1s (ftrace on). The code for flipping the mask is borrowed from ftrace_{enable,disable}_ftrace_graph_caller. After flipping, ftrace_arch_code_modify_post_process() syncs with all the other CPUs by sending SIGPs. - Second stages for vmlinux are stored in a separate part of the .text section reserved by the linker script, and in dynamically allocated memory for modules. This prevents the icache pollution. The total size of second stages is about 1.5% of that of the kernel image. Putting second stages in the .bss section is possible and decreases the size of the non-compressed vmlinux, but splits the kernel 1:1 mapping, which is a bad tradeoff. Each second stage contains a call to the third stage, a pointer to the part of the intercepted function right after the first stage, and a pointer to an interceptor function (e.g. ftrace_caller). Second stages are 8-byte aligned for the future direct calls implementation. - There are only two copies of the third stage: in the .text section for vmlinux and in dynamically allocated memory for modules. It can be an expoline, which is relatively large, so inlining it into each second stage is prohibitively expensive. As a result of this organization, phoronix-test-suite with ftrace off does not show any performance degradation. Suggested-by: Sven Schnelle <svens@linux.ibm.com> Suggested-by: Vasily Gorbik <gor@linux.ibm.com> Co-developed-by: Heiko Carstens <hca@linux.ibm.com> Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com> Link: https://lore.kernel.org/r/20210728212546.128248-3-iii@linux.ibm.com Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2021-07-29 05:25:46 +08:00
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef _FTRACE_H
#define _FTRACE_H
#include <asm/types.h>
struct ftrace_hotpatch_trampoline {
u16 brasl_opc;
s32 brasl_disp;
s16: 16;
u64 rest_of_intercepted_function;
u64 interceptor;
} __packed;
extern struct ftrace_hotpatch_trampoline __ftrace_hotpatch_trampolines_start[];
extern struct ftrace_hotpatch_trampoline __ftrace_hotpatch_trampolines_end[];
extern const char ftrace_shared_hotpatch_trampoline_br[];
extern const char ftrace_shared_hotpatch_trampoline_br_end[];
extern const char ftrace_shared_hotpatch_trampoline_ex[];
extern const char ftrace_shared_hotpatch_trampoline_ex_end[];
extern const char ftrace_shared_hotpatch_trampoline_exrl[];
extern const char ftrace_shared_hotpatch_trampoline_exrl_end[];
extern const char ftrace_plt_template[];
extern const char ftrace_plt_template_end[];
#endif /* _FTRACE_H */