Skip to content

Use host USM for scalar oneMKL results - #604

Merged
michel2323 merged 1 commit into
mainfrom
mkl-scalar-host-usm
Aug 5, 2026
Merged

Use host USM for scalar oneMKL results#604
michel2323 merged 1 commit into
mainfrom
mkl-scalar-host-usm

Conversation

@michel2323

Copy link
Copy Markdown
Member

Closes #234.

The scalar-returning oneMKL BLAS wrappers hand the library a one-element device oneArray and copy the value back:

result = oneArray{$ret_type}([0])   # host vector -> device (H2D)
$fname(sycl_queue(queue), n, x, stride(x,1), result)
res = Array(result)                 # device -> host (D2H)
return res[1]

The two transfers dominate the call. Since we now support host-USM-backed arrays (oneArray{T,N,oneL0.HostBuffer} with host-side getindex/setindex!), the result can live in host USM instead: oneMKL writes straight into host-visible memory and we read it with a plain load. The C wrappers wait_and_throw() before returning, so the value is ready as soon as the ccall is.

New helper in lib/mkl/utils.jl:

function scalar_result(::Type{T}) where {T}
    res = oneArray{T, 1, oneL0.HostBuffer}(undef, 1)
    res[1] = zero(T)
    return res
end

Applied to all five sites in lib/mkl/wrappers_blas.jl: nrm2, dot/dotc/dotu, asum, iamax, iamin.

This also makes #234 moot as originally written — a dedicated ZeHostRef type isn't needed, since host-USM oneArrays cover the use case and oneL0.host_alloc handles the raw-pointer case.

Performance

nrm2 over a 4096-element Float32 vector, Arc A750:

µs/call
device buffer + D2H copy 3079.5
host USM 331.5

~9.5x. It's pure call overhead, so the win is independent of vector length.

Testing

Arc A750, NEO 26.18.38308, oneMKL 2025.3.0, Julia 1.12.6:

  • test/runtests.jl onemkl — 1112 pass, 48 broken, SUCCESS
  • test/runtests.jl gpuarrays/linalg array — 2492 pass, SUCCESS
  • All five wrappers spot-checked against LinearAlgebra for Float32/Float64/ComplexF32/ComplexF64

Notes

Two pre-existing things I left alone, happy to fold in if you'd rather:

  • iamin is typed x::StridedArray{$elty} where every sibling uses oneStridedArray (wrappers_blas.jl:838).
  • These wrappers ignore the int status the C functions return, so a oneMKL exception surfaces as a silent zero rather than an error. That's why scalar_result zero-initializes rather than leaving the buffer undef — it preserves the current behaviour on that path instead of returning garbage.

The scalar-returning BLAS wrappers (nrm2, dot/dotc/dotu, asum, iamax,
iamin) allocated their one-element result in device memory and copied it
back to the host, so every call paid for a device allocation, an H2D
transfer to zero-initialize it, and a D2H transfer to read it.

Allocate the result in host USM instead. oneMKL writes straight into
host-visible memory and we read the value with a plain load; the C
wrappers already wait_and_throw() before returning, so the result is
ready as soon as the ccall is.

nrm2 over a 4096-element Float32 vector on an Arc A750: 3079.5 -> 331.5
us/call. The win is call overhead, so it is independent of length.
@codecov

codecov Bot commented Aug 4, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.33%. Comparing base (8fd0eff) to head (6badafa).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #604      +/-   ##
==========================================
+ Coverage   80.24%   80.33%   +0.09%     
==========================================
  Files          50       50              
  Lines        3487     3488       +1     
==========================================
+ Hits         2798     2802       +4     
+ Misses        689      686       -3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@michel2323
michel2323 merged commit 95fa7d3 into main Aug 5, 2026
5 checks passed
@michel2323
michel2323 deleted the mkl-scalar-host-usm branch August 5, 2026 14:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Use host USM for scalar oneMKL results instead of a device round-trip

1 participant