The SiFive U74 (RV64GC; e.g. StarFive JH7110 / VisionFive 2) is a scalar,
in-order core with no RVV, so today it falls back to RISCV64_GENERIC whose
S/D GEMM uses the generic 2x2 C micro-kernel.
Per the U74 Core Complex Manual (Table 169) fmadd.d has a 7-cycle latency
at repeat rate 1 (fully pipelined). A 2x2 tile exposes only 4 independent
accumulator chains -- fewer than the FMA latency -- so the FP pipe stalls
on the accumulator dependency, and the 1:1 load:FMA ratio saturates the
single load/store pipe ("only one outstanding line fill", manual 8.2).
This adds a portable 4x4 GEMM micro-kernel and a dedicated U74 target:
- kernel/generic/gemmkernel_4x4.c: 16-accumulator 4x4 register tile. 16
independent chains exceed the 7-cycle latency, and the load:FMA ratio
drops to 1:2. 16 acc + 4 A + 4 B fit RV64G's 32 FP registers without
spilling. Full 4/2/1 edge handling in both M and N.
- U74 target wiring: getarch.c (FORCE_U74, 32 KiB/64 B L1D, 2 MiB L2),
param.h (S/D UNROLL 4/4; complex stays 2/2), kernel/riscv64/KERNEL.U74
(S/D GEMM -> gemmkernel_4x4 + gemm_[nt]copy_4; S/D TRMM -> existing
trmmkernel_4x4), Makefile.prebuild + Makefile.riscv64 (-mtune=sifive-u74),
TargetList.txt, cpuid_riscv64.c.
The 4x4 kernel was verified numerically against a naive reference GEMM,
driven through the real gemm_tcopy_4 / gemm_ncopy_4 packing routines,
across 27,436 M/N/K x alpha combinations covering every 4/2/1 tail case:
worst absolute error 0.
Build with: make TARGET=U74
The general-stride ROTM path already computes kx/ky with the BLAS negative-increment starting offset. Converting negative increments to positive strides and moving dx/dy again double-adjusted the address and could access the wrong elements. Keep the signed byte strides for RVV strided loads and stores.
ROTM has loop-carried dependencies when incx or incy is zero because the same element is updated repeatedly across iterations. The RVV strided load/store path would compute lanes from the same old value and write them back in parallel, producing results that do not match BLAS ROTM semantics. Use a scalar path for zero-stride cases.
Avoid loading C when beta is zero; ZA has already been initialized to zero.
For alpha == 0 or K == 0, skip A preprocessing and reuse the direct kernel with k = 0 to perform only the beta update.
The kernel_2x2 function uses 4 ZA tiles (0-3) each with svl slices.
Tiles 0/1 handle rows 0..svl-1 with slice indices 0..svl-1.
Tiles 2/3 handle rows svl..2*svl-1, so their slice indices
must start at 0, i.e. (i - svl) instead of i.
Fix all three tile 2/3 access sites:
- C load into ZA (svwrite_hor_za32_f32_m)
- C writeback for UPPER (svst1_hor_za32)
- C writeback for LOWER (svst1_hor_za32)
Fixes#5873
Corrected the register constraints for the PowerPC dcbt (Data Cache Block
Touch) instruction in Power10 kernel implementations. The dcbt instruction
has special behavior where if the first operand (RA) is r0, it uses the
value 0 instead of the register contents. Therefore, RA must use the "b"
constraint (any GPR except r0), while RB can use "r" (any GPR including r0).
Changes:
- Changed first operand constraint from "r" to "b" to exclude r0
- Changed second operand constraint from "b" to "r" for flexibility
This ensures correct prefetch behavior and compliance with PowerPC ISA
specifications, preventing potential issues where r0 might be incorrectly
used as the base address register.
Signed-off-by: Amrita H S <amritahs@linux.vnet.ibm.com>
Replace normal vector pair pointer dereferences with the optimized
__builtin_vsx_lxvp builtin across DGEMM, ZGEMM, and DGEMV kernels.
Also done some identation corrections in dgemm_kernel_power10.c.
This is done as part of POWER code cleanup and may not have any
performance impact.
Signed-off-by: Amrita H S <amritahs@linux.vnet.ibm.com>
benchmark/sgemm.goto before:
M= 200, N= 200, K= 200 : 9262.97 MFlops 0.001727 sec
after:
M= 200, N= 200, K= 200 : 30223.64 MFlops 0.000529 sec
Conveniently the registers are already allocated suitably for vector
operation, so the conversion from vfpv3 was rather straightforward.
Prefetching was left out because it doesn't help Cortex-A76,
only hurts it slightly.
According to ARM AAPCS (Procedure Call Standard) 5.1.2.1, only registers
s16-s31 must be preserved across subroutine calls; registers s0-s15
do not need to be preserved.
hmeiland