Commit Graph

147 Commits

Author SHA1 Message Date
Martin Kroeker 75b93b4465
Fix missing declaration of info 2026-08-07 13:56:43 +02:00
Martin Kroeker 761ebe91ab
Fix return without code and buffer leak on other error path 2026-08-07 13:51:01 +02:00
Martin Kroeker ed2ba87203
Add error return on buffer assignment failure 2026-08-07 12:28:01 +02:00
Martin Kroeker b080247a21
Revert "Mark temporaries as volatile to avoid SIGBUS in multithreaded builds with gcc15+ on 32bit FreeBSD/x86" 2026-07-26 22:51:58 +02:00
Keno Fischer 7a07c8e1f4 Add an asynchronous cancellation API for in-flight operations
Long-running BLAS calls (a large gemm can run for minutes) cannot
currently be interrupted: callers embedding OpenBLAS (e.g. the Julia
runtime responding to a user's ^C) can only wait for completion or kill
the process. Add a minimal cooperative cancellation protocol:

Every thread owns a pointer-sized generation slot in thread-local
storage, whose stable address is returned by openblas_cancel_token().
Instrumented compute drivers advance the slot to a fresh even
generation at operation entry on the issuing thread (forwarding the
slot and generation to worker threads through blas_arg_t) and poll it
at block granularity. openblas_cancel(token, loaded_token) - callable
from any thread - sets the cancel bit (bit 0) iff the slot still holds
loaded_token, so a canceller that loaded the value while an operation
was in flight stops exactly that operation, while stale or racing
requests either miss or dirty an already-dead generation, both
harmless. There is no object lifecycle: nothing to allocate, bind,
reset, or free.

A cancelled operation returns quickly, leaving its output buffer in an
unspecified partially-updated state that the caller must discard; every
synchronization point in the threaded driver is still executed, so
sibling threads never stall and the library remains consistent for
subsequent calls. Coverage: the level-3 gemm/symm/hemm drivers
(level3.c and level3_thread.c).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012nCkyKUguncLLJrH9K5o7m
2026-07-19 04:20:28 +00:00
Martin Kroeker 15351019ea
Mark subbuffer as volatile to fix SIGBUS in multithreading with gcc15+ on 32bit FreeBSD/x86 2026-07-16 18:35:14 +02:00
Kumar Aditya a2261f9e52 fix thread safety of openmp backend 2026-07-02 15:39:09 +05:30
Kumar Aditya 298d53c061 fix data races in blas_server and level3_thread 2026-06-23 18:42:25 +05:30
Nathan Goldbaum 9363452742 Fix corruption due to lock sharding issues by centralizing locking 2026-06-14 15:32:51 -06:00
Martin Kroeker 54b939e4e3
Undefine GEMM_PREFERRED_SIZE before redefining to silence warning 2026-04-22 16:30:12 +02:00
Martin Kroeker c9185e91ad
Make GEMM_DIVIDE_RATE and GEMM_PREFERRED_SIZE available in DYNAMIC_ARCH builds 2026-03-20 15:34:04 +01:00
Martin Kroeker b7601ea92f
Retrieve cpu-specific GEMM_DIVIDE_LIMIT if DYNAMIC_ARCH 2026-03-19 08:29:15 +01:00
lujiaweics 1f3b81e562 Serialize accesses to parallelized syrk functions from multiple callers, like it was already done for GEMM in level3_thread.c and GEMM3M in level3_gemm3m_thread.c 2026-01-20 21:31:35 +08:00
Martin Kroeker b2b9abc20b
Revert "Enhancing Core Utilization in BLAS Calls: A Scalable Architecture" 2025-10-22 15:44:43 +02:00
Martin Kroeker 5b640b1cbc
add bgemm_thread_xx 2025-10-16 10:03:04 -07:00
Masato Nakagawa 7e29f11396 Multi-thread GEMM Performance Improvement on NeoverseV1 (DIVIDE_RATE=1) 2025-07-29 18:54:36 +09:00
youcai 41f9701ebc Fix cmake building with cblas_bgemm 2025-07-23 22:10:53 +08:00
Chris Sidebottom 48394384ef Use correct constants for per-target BGEMM/SBGEMM
This fixes the build and tests on `NEOVERSEV1` target, which was failing
with specific constants for `SBGEMM`

Co-authored-by: Ye Tao <ye.tao@arm.com>
2025-07-08 16:23:27 +01:00
Chris Sidebottom f95e7b0e32 Add infrastructure for BGEMM
Setting up all the infrastructure for BGEMM support in OpenBLAS, hopefully I found all the right places.

Derived mostly from the previous work done in https://github.com/OpenMathLib/OpenBLAS/pull/5287

Co-authored-by: Ye Tao <ye.tao@arm.com>
2025-07-08 16:22:41 +01:00
Chris Sidebottom 7a97c4ca97 Rename HALF -> BFLOAT16 in some more places 2025-07-07 10:13:39 +00:00
Masato Nakagawa 5253c8f165 Multi-thread Performance Improvement of GEMM with DIVIDE_RATE=1 for
A64FX.
2025-06-30 21:35:16 +09:00
Srangrang 9f13b2c6ac style: modify HALF to BFLOAT16 in benchmark folder 2025-06-15 20:57:05 +08:00
gkdddd 670ec6f757 Added shgemm_kernel_8x8 for RISCV64_ZVL128B and shgemm_kernel_16x8 for RISCV64_ZVL256B
Added HFLOAT16 support for RISCV64
Added shgemm_kernel_8x8 for RISCV64_ZVL128B and shgemm_kernel_16x8 for RISCV64_ZVL256B based on HFLOAT16
The instruction sets used are ZVFH and ZFH, which need to be supported by RVV1.0

Related to issue #5279
Co-authored-by Linjin Li <linjin_li@163.com>
2025-06-03 20:14:30 +08:00
Martin Kroeker 20f2ba0141
Move declaration of i for pre-C99 compilers 2025-05-21 23:44:17 +02:00
Masato Nakagawa 2351a98005 Update 2D thread-partitioned GEMM for M << N case. 2025-05-21 21:21:52 +09:00
Ruiyang Wu 02fd1df10b CMake: Pass `OpenMP` compiler and linker flags through CMake targets
Using `OpenMP::OpenMP_LANG` targets for CMake is less error-prone than
passing the compiler and linker flags manually. Furthermore, it allows
the user to customize those flags by setting `OpenMP_LANG_FLAGS`,
`OpenMP_LANG_LIB_NAMES`, and `OpenMP_omp_LIBRARY`.
2025-03-26 23:09:54 -04:00
Masato Nakagawa 80d3c2ad95 Add Improving Load Imbalance in Thread-Parallel GEMM 2025-03-11 20:18:20 +09:00
Martin Kroeker 77c638db67
Revert "Fix potential inaccuracy in multithreaded level3 related to SWITCH_RATIO" 2025-02-15 20:37:48 +01:00
John Hein 6cd9bbe531 fix signedness of pointer to integer type passed to blas_lock() 2025-02-01 17:22:57 -07:00
Martin Kroeker 8a1710dd0d
don't apply switch_ratio to tail of loop 2024-10-06 20:03:32 +02:00
shivammonaka 9e22d70957 Dynamic locking in Pthread Backend to allow multiple BLAS calls to be executed parallelly 2024-06-07 08:40:17 +05:30
Martin Kroeker db070a9223
add gemm_batch drivers 2024-05-31 18:29:27 +02:00
Martin Kroeker d0794f88dc
add gemm_batch driver 2024-05-29 15:49:20 +02:00
yamazaki-mitsufumi 51ab1903e7 Expanding the scop of 2D thread distribution 2024-04-18 18:20:25 +09:00
shivammonaka d49ebc54e1 Merge branch 'shivam-develop' into shivam-Locks 2024-02-29 11:58:14 +05:30
shivammonaka bc191015e3 Using OpenMP locks with NUM_PARALLEL 2024-02-29 11:47:05 +05:30
Martin Kroeker c4bd4a2e5d
fix improper function prototypes (empty parentheses) 2023-09-30 12:49:24 +02:00
Chris Sidebottom 32f2fafde7 Propagate SWITCH_RATIO to DYNAMIC_ARCH builds
Previously dynamic builds were either using the default SWITCH_RATIO
or one from the higher level architecture; this patch ensures the
dynamic builds can use this parameter as well.
2023-04-17 15:34:12 +01:00
Honglin Zhu 4989e039a5 Define SBGEMM_ALIGN_K for DYNAMIC_ARCH build 2022-10-27 14:10:26 +08:00
Honglin Zhu b00d5b9746 New sbgemm implementation for Neoverse N2
1. Use UZP instructions but not gather load and scatter store instructions to get lower latency.
    2. Padding k to a power of 4.
2022-10-26 15:09:41 +08:00
Wangyang Guo 3dc6052c7e initial support for Sapphire Rapids platform 2021-10-12 01:30:40 -07:00
Martin Kroeker 2f8220d757
Add sbgemm 2021-09-14 16:14:43 +02:00
Martin Kroeker 307c4c0786
Fix typo 2021-06-16 13:41:16 +02:00
Martin Kroeker e83df93975
Work around another recent macro name collision with winnt.h 2021-06-16 12:32:34 +02:00
Martin Kroeker a554712439
remove extra/intermediate size step for min_jj introduced in PR747 2020-12-08 21:01:36 +01:00
Martin Kroeker 5d26223f4a
remove extra/intermediate size step of min_jj from PR747 2020-12-08 20:59:56 +01:00
Martin Kroeker d3ff1f889f
Convert ifndefs to ifneq 2020-11-22 16:27:17 +01:00
Rajalakshmi Srinivasaraghavan b5d30b390d Fix build issues with bfloat16
This patch fixes compilation errors due to recent renaming from SH to SB
with BUILD_BFLOAT16.
2020-10-13 11:00:22 -05:00
Martin Kroeker 006c7f6671
Change "HALF" and "sh" to "BFLOAT16" and "sb" 2020-10-12 00:06:06 +02:00
Martin Kroeker 886a8e3190
Adapt for supporting only a subset of variable types 2020-10-11 14:57:32 +02:00