UbiquitousOS
/
OpenCloudOS-Kernel
mirror of https://gitee.com/OpenCloudOS/OpenCloudOS-Kernel.git
11
0
Fork 0
Code Issues Projects Releases Wiki Activity

asm-generic: introduce io_stop_wc() and add implementation for ARM64 

For memory accesses with write-combining attributes (e.g. those returned
by ioremap_wc()), the CPU may wait for prior accesses to be merged with
subsequent ones. But in some situation, such wait is bad for the
performance.

We introduce io_stop_wc() to prevent the merging of write-combining
memory accesses before this macro with those after it.

We add implementation for ARM64 using DGH instruction and provide NOP
implementation for other architectures.

Signed-off-by: Xiongfeng Wang <wangxiongfeng2@huawei.com>
Suggested-by: Will Deacon <will@kernel.org>
Suggested-by: Catalin Marinas <catalin.marinas@arm.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Link: https://lore.kernel.org/r/20211221035556.60346-1-wangxiongfeng2@huawei.com
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: hongrongxuan <hongrongxuan@huawei.com>

 Conflicts:
	arch/arm64/include/asm/barrier.h
	include/asm-generic/barrier.h
This commit is contained in:
Xiongfeng Wang 2021-12-21 11:55:56 +08:00 committed by Jianping Liu
parent f1759b64e4
commit 3370a82d1d
3 changed files with 28 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
Documentation/memory-barriers.txt
View File

@ -1937,6 +1937,14 @@ There are some more advanced barrier functions:
information on consistent memory. information on consistent memory.
(*) io_stop_wc();
For memory accesses with write-combining attributes (e.g. those returned
by ioremap_wc(), the CPU may wait for prior accesses to be merged with
subsequent ones. io_stop_wc() can be used to prevent the merging of
write-combining memory accesses before this macro with those after it when
such wait has performance implications.
=============================== ===============================
IMPLICIT KERNEL MEMORY BARRIERS IMPLICIT KERNEL MEMORY BARRIERS
=============================== ===============================

9
arch/arm64/include/asm/barrier.h
View File

@ -28,6 +28,13 @@
#define spec_bar() asm volatile(ALTERNATIVE("dsb nsh\nisb\n", \ #define spec_bar() asm volatile(ALTERNATIVE("dsb nsh\nisb\n", \
SB_BARRIER_INSN"nop\n", \ SB_BARRIER_INSN"nop\n", \
ARM64_HAS_SB)) ARM64_HAS_SB))
/*
* Data Gathering Hint:
* This instruction prevents merging memory accesses with Normal-NC or
* Device-GRE attributes before the hint instruction with any memory accesses
* appearing after the hint instruction.
*/
#define dgh() asm volatile("hint #6" : : : "memory")
#define mb() dsb(sy) #define mb() dsb(sy)
#define rmb() dsb(ld) #define rmb() dsb(ld)
@ -36,6 +43,8 @@
#define dma_rmb() dmb(oshld) #define dma_rmb() dmb(oshld)
#define dma_wmb() dmb(oshst) #define dma_wmb() dmb(oshst)
#define io_stop_wc() dgh()
/* /*
* Generate a mask for array_index__nospec() that is ~0UL when 0 <= idx < sz * Generate a mask for array_index__nospec() that is ~0UL when 0 <= idx < sz
* and 0 otherwise. * and 0 otherwise.

11
include/asm-generic/barrier.h
View File

@ -257,5 +257,16 @@ do { \
}) })
#endif #endif
/*
* ioremap_wc() maps I/O memory as memory with write-combining attributes. For
* this kind of memory accesses, the CPU may wait for prior accesses to be
* merged with subsequent ones. In some situation, such wait is bad for the
* performance. io_stop_wc() can be used to prevent the merging of
* write-combining memory accesses before this macro with those after it.
*/
#ifndef io_stop_wc
#define io_stop_wc do { } while (0)
#endif
#endif /* !__ASSEMBLY__ */ #endif /* !__ASSEMBLY__ */
#endif /* __ASM_GENERIC_BARRIER_H */ #endif /* __ASM_GENERIC_BARRIER_H */