The general-stride ROTM path already computes kx/ky with the BLAS negative-increment starting offset. Converting negative increments to positive strides and moving dx/dy again double-adjusted the address and could access the wrong elements. Keep the signed byte strides for RVV strided loads and stores.
ROTM has loop-carried dependencies when incx or incy is zero because the same element is updated repeatedly across iterations. The RVV strided load/store path would compute lanes from the same old value and write them back in parallel, producing results that do not match BLAS ROTM semantics. Use a scalar path for zero-stride cases.
Avoid loading C when beta is zero; ZA has already been initialized to zero.
For alpha == 0 or K == 0, skip A preprocessing and reuse the direct kernel with k = 0 to perform only the beta update.
The kernel_2x2 function uses 4 ZA tiles (0-3) each with svl slices.
Tiles 0/1 handle rows 0..svl-1 with slice indices 0..svl-1.
Tiles 2/3 handle rows svl..2*svl-1, so their slice indices
must start at 0, i.e. (i - svl) instead of i.
Fix all three tile 2/3 access sites:
- C load into ZA (svwrite_hor_za32_f32_m)
- C writeback for UPPER (svst1_hor_za32)
- C writeback for LOWER (svst1_hor_za32)
Fixes#5873
Corrected the register constraints for the PowerPC dcbt (Data Cache Block
Touch) instruction in Power10 kernel implementations. The dcbt instruction
has special behavior where if the first operand (RA) is r0, it uses the
value 0 instead of the register contents. Therefore, RA must use the "b"
constraint (any GPR except r0), while RB can use "r" (any GPR including r0).
Changes:
- Changed first operand constraint from "r" to "b" to exclude r0
- Changed second operand constraint from "b" to "r" for flexibility
This ensures correct prefetch behavior and compliance with PowerPC ISA
specifications, preventing potential issues where r0 might be incorrectly
used as the base address register.
Signed-off-by: Amrita H S <amritahs@linux.vnet.ibm.com>
Replace normal vector pair pointer dereferences with the optimized
__builtin_vsx_lxvp builtin across DGEMM, ZGEMM, and DGEMV kernels.
Also done some identation corrections in dgemm_kernel_power10.c.
This is done as part of POWER code cleanup and may not have any
performance impact.
Signed-off-by: Amrita H S <amritahs@linux.vnet.ibm.com>
benchmark/sgemm.goto before:
M= 200, N= 200, K= 200 : 9262.97 MFlops 0.001727 sec
after:
M= 200, N= 200, K= 200 : 30223.64 MFlops 0.000529 sec
Conveniently the registers are already allocated suitably for vector
operation, so the conversion from vfpv3 was rather straightforward.
Prefetching was left out because it doesn't help Cortex-A76,
only hurts it slightly.
Martin Kroeker